Multi-UAV assisted mobile edge computing trajectory optimization method, device and equipment

Abstract

The present invention relates to a multi-UAV assisted mobile edge computing trajectory optimization method, device and equipment. First, multiple models are constructed, specifically: a user association model, which is used to define the corresponding relationship between UAVs and computing tasks; a UAV movement model, which is used to constrain the positions of UAVs and user positions and avoid UAV collisions; a task calculation model, which is used to calculate the task execution time; a UAV energy consumption model, which is used to constrain the energy consumption of UAVs. Then, a deep reinforcement learning model of the UAV is constructed based on the above models, and the meanings of parameters such as local observation, global observation, action, and reward are defined respectively. Among them, the action is defined according to the moving angle and distance of the UAV, and the reward function is defined according to the total task execution time and the task response failure rate, etc. The reward function is maximized to achieve the trajectory optimization of the UAV. This method can find the unloading decision and mobile trajectory with the minimum task completion time and task request failure rate.

Description

Technical Field

[0001] The present invention relates to the field of mobile edge computing technology, and in particular to a multi-UAV assisted mobile edge computing trajectory optimization method, device and equipment. Background Art

[0002] With the popularity of applications such as smart navigation and augmented reality, there are more and more computing-intensive and delay-sensitive tasks, and people expect to enjoy a more convenient life than ever before. Mobile edge computing (MEC) moves computing and storage resources to the edge of the network. It can enhance the ability of user devices to perform computing-intensive tasks by deploying devices such as base stations and access points, significantly reduce transmission and processing delays, save energy for mobile devices, and protect privacy and security. However, with the further integration of 5G and the core network, MEC nodes are further highly distributed. The existence of fixed base stations leads to poor response capabilities of MEC in response to disasters, and there are certain limitations in deployment. Drones are highly flexible and maneuverable, which can effectively solve the problems of limited flexibility and poor disaster response capabilities of MEC fixed base stations. Due to limited computing and energy resources, a drone can only achieve limited regional coverage and limited task offloading capabilities. Multiple drones can jointly provide stronger computing capabilities and a wider range of regional coverage. They can also flexibly move and adjust their positions according to needs to provide services in different areas. Therefore, mobile edge computing assisted by multiple drones will be more in line with the actual situation. In addition, since drones need to fly from different take-off points to specific areas to provide unloading assistance, different drone trajectories may result in different channel qualities, resulting in different communication delays and energy consumption. Therefore, it is necessary to consider the trajectory optimization problem of drones to obtain the minimum execution delay. In the case of limited onboard resources, the amount of computing tasks allocated to drones will also affect the computing delay. How to reasonably allocate resources so that tasks with higher urgency can be executed as quickly as possible is crucial. By jointly optimizing trajectories and user associations, the system can respond more flexibly to the computing requests of terminal devices and improve the adaptability and robustness of the system. Summary of the invention

[0003] Based on this, it is necessary to provide a multi-UAV assisted mobile edge computing trajectory optimization method, device and equipment to address the above technical problems, so that the task requests proposed by the user equipment can be completed in the shortest time and with the lowest response failure rate.

[0004] A multi-UAV assisted mobile edge computing trajectory optimization method, the method comprising:

[0005] Constructing a user association model; the user association model refers to the correspondence between the computing tasks to be offloaded by the drone and the user equipment; in the user association model, it is defined that in each time slot: each drone accepts a maximum of a preset number of computing tasks within its coverage area, each computing task can only be executed by the drone or the user equipment, and each user equipment generates at most one computing task request;

[0006] Construct a drone mobility model; the drone mobility model limits drones and user devices to a preset area and the minimum distance between any two drones;

[0007] Constructing a task calculation model; the task calculation model is used to calculate the task calculation time of each calculation task in each time slot;

[0008] Construct a UAV energy consumption model; the UAV energy consumption model is used to limit the total energy consumption of each UAV during the entire service life;

[0009] Based on the user association model, the UAV movement model, the task calculation model and the UAV energy consumption model, a deep reinforcement learning model of the UAV is constructed. Specifically, the UAV's own state, the information of the currently executed calculation task and the relative distance between the UAV and other UAVs within its coverage area are defined as local observations; the UAV's own state includes: the UAV's position, remaining energy and idle resources; the aggregated information of the local observations of all UAVs in the environment within the preset area is defined as a global observation; the flight angle and flight distance of the UAV in a unit time slot are defined as actions; the reward function is constructed based on the total task calculation time, the calculation task request response failure rate and the restrictions in the UAV movement model; the task request response failure means that the task calculation time exceeds the maximum tolerance time of the corresponding calculation task request;

[0010] With the goal of maximizing the reward function, the movement trajectory of the drone is optimized.

[0011] A multi-UAV assisted mobile edge computing trajectory optimization device, the device comprising:

[0012] A user association model building module is used to build a user association model; the user association model refers to the correspondence between the computing tasks to be offloaded by the drone and the user equipment; in the user association model, it is limited that in each time slot: each drone accepts a maximum of a preset number of computing tasks within its coverage area, each computing task can only be executed by a drone or a user equipment, and each user equipment generates at most one computing task request;

[0013] A drone mobility model building module is used to build a drone mobility model; the drone mobility model limits the drone and user equipment within a preset area and the minimum distance between any two drones;

[0014] A task calculation model building module is used to build a task calculation model; the task calculation model is used to calculate the task calculation time of each calculation task in each time slot;

[0015] The UAV energy consumption model building module is used to build the UAV energy consumption model; the UAV energy consumption model is used to limit the total energy consumption of each UAV during the entire service cycle;

[0016] The deep reinforcement learning model construction module is used to construct a deep reinforcement learning model of the drone based on the user association model, the drone movement model, the construction task calculation model and the drone energy consumption model. Specifically, the drone's own state, the information of the currently executed calculation task and the relative distance between it and other drones within its coverage area are defined as local observations; wherein the drone's own state includes: the drone's position, remaining energy and idle resources; the aggregated information of the local observations of all drones in the environment within the preset area is defined as a global observation; the flight angle and flight distance of the drone in a unit time slot are defined as actions; the reward function is constructed based on the total task calculation time, the calculation task request response failure rate and the restrictions in the drone movement model; the task request response failure means that the task calculation time exceeds the maximum tolerance time of the corresponding calculation task request;

[0017] The UAV trajectory optimization module is used to optimize the UAV trajectory with the goal of maximizing the reward function.

[0018] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0019] Constructing a user association model; the user association model refers to the correspondence between the computing tasks to be offloaded by the drone and the user equipment; in the user association model, it is defined that in each time slot: each drone accepts a maximum of a preset number of computing tasks within its coverage area, each computing task can only be executed by the drone or the user equipment, and each user equipment generates at most one computing task request;

[0020] Construct a drone mobility model; the drone mobility model limits the drone and user equipment to a preset area and the minimum distance between any two drones;

[0021] Constructing a task calculation model; the task calculation model is used to calculate the task calculation time of each calculation task in each time slot;

[0022] Construct a UAV energy consumption model; the UAV energy consumption model is used to limit the total energy consumption of each UAV during the entire service life;

[0023] Based on the user association model, the UAV movement model, the task calculation model and the UAV energy consumption model, a deep reinforcement learning model of the UAV is constructed. Specifically, the UAV's own state, the information of the currently executed calculation task and the relative distance between the UAV and other UAVs within its coverage area are defined as local observations; the UAV's own state includes: the UAV's position, remaining energy and idle resources; the aggregated information of the local observations of all UAVs in the environment within the preset area is defined as a global observation; the flight angle and flight distance of the UAV in a unit time slot are defined as actions; the reward function is constructed based on the total task calculation time, the calculation task request response failure rate and the restrictions in the UAV movement model; the task request response failure means that the task calculation time exceeds the maximum tolerance time of the corresponding calculation task request;

[0024] With the goal of maximizing the reward function, the movement trajectory of the drone is optimized.

[0025] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0026] Constructing a user association model; the user association model refers to the correspondence between the computing tasks to be offloaded by the drone and the user equipment; in the user association model, it is defined that in each time slot: each drone accepts a maximum of a preset number of computing tasks within its coverage area, each computing task can only be executed by the drone or the user equipment, and each user equipment generates at most one computing task request;

[0027] Construct a drone mobility model; the drone mobility model limits the drone and user equipment to a preset area and the minimum distance between any two drones;

[0028] Constructing a task calculation model; the task calculation model is used to calculate the task calculation time of each calculation task in each time slot;

[0029] Construct a UAV energy consumption model; the UAV energy consumption model is used to limit the total energy consumption of each UAV during the entire service life;

[0030] Based on the user association model, the UAV movement model, the task calculation model and the UAV energy consumption model, a deep reinforcement learning model of the UAV is constructed. Specifically, the UAV's own state, the information of the currently executed calculation task and the relative distance between the UAV and other UAVs within its coverage area are defined as local observations; the UAV's own state includes: the UAV's position, remaining energy and idle resources; the aggregated information of the local observations of all UAVs in the environment within the preset area is defined as a global observation; the flight angle and flight distance of the UAV in a unit time slot are defined as actions; the reward function is constructed based on the total task calculation time, the calculation task request response failure rate and the restrictions in the UAV movement model; the task request response failure means that the task calculation time exceeds the maximum tolerance time of the corresponding calculation task request;

[0031] With the goal of maximizing the reward function, the movement trajectory of the drone is optimized.

[0032] The above-mentioned multi-drone assisted mobile edge computing trajectory optimization method, device and equipment first construct multiple models, specifically: user association model, used to define the corresponding relationship between drones and computing tasks; drone movement model, used to constrain the location of drones and user positions and avoid drone collisions; task calculation model, used to calculate task execution time; drone energy consumption model, used to constrain drone energy consumption. Then, based on the above model, a deep reinforcement learning model of the drone is constructed, and the meanings of parameters such as local observation, global observation, action, and reward are defined respectively, wherein the action of the drone is defined according to the moving angle and distance of the drone, and the reward function is defined according to the total task execution time and task response failure rate, etc. Finally, the movement trajectory of the drone is optimized with the purpose of maximizing the reward function. The method can be used to jointly optimize the movement trajectory and user association of the drone without obtaining accurate global state information, so that the task request proposed by the user device can be completed in the shortest time and with the minimum response failure rate, that is, it can find the task offloading decision and movement trajectory with the minimum task completion time and task request failure rate, and the effect has a certain stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a schematic diagram of a multi-UAV assisted mobile edge computing system model;

[0034] Figure 2 A flowchart of a multi-UAV assisted mobile edge computing trajectory optimization method;

[0035] Figure 3 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0037] The movement area of ​​the UAV is defined as a three-dimensional rectangular area such as Figure 1 Consider a multi-UAV-assisted mobile edge computing scenario, which consists of M UAVs and N randomly assigned nodes with an edge length of , The height of the drone is fixed at H, and there are multiple user equipments randomly distributed in the horizontal two-dimensional coordinates. The UE set is denoted as , and the drone set is recorded as , define the set of time slots TS={1,2,…T}, and the maximum duration of each time slot is , assuming that the time slot is small enough and the position of the user equipment does not change.

[0038] The drone is equipped with computing resources and acts as a mobile edge server to provide computing services to ground users. Every time it moves to a new location, the drone decides whether to provide offloading services to ground users based on the urgency of the user's task request within its coverage area and the availability of its own onboard computing resources. For users, binary complete offloading is used to execute the offloading decision. The present invention aims to minimize the failure rate of task request responses of all users and the total time for task completion by optimizing the association between users and drones and the trajectory of drones while meeting the constraints of drone onboard resources. User association is defined here as the association between a task request issued by a user device and a drone. The failure of a task request response means that the task cannot be completed within the maximum tolerance time of the task. The main challenges in solving this problem are as follows:

[0039] (1) Dynamic task scheduling and decision-making: Each UAV must dynamically evaluate and decide whether to provide offload services to the user and execute the user's computing task request. This involves accurately predicting the urgency of the task, the computing resource requirements, and the resource availability at the current location of the UAV. Since the task requests submitted by the user device have a maximum tolerance time, an effective UAV scheduling algorithm must be designed to enable it to perform effective computing on the appropriate UAV.

[0040] (2) Multivariable combinatorial optimization problem: This problem is essentially a complex multivariable combinatorial optimization problem, which involves the movement trajectory of each drone, computing resource constraints, and the spatiotemporal dynamics of random arrival of user requests. In this case, it is difficult to find the global optimal solution or even impossible to accurately establish a solution model. Therefore, traditional optimization algorithms are difficult to handle scenarios of non-convex optimization and large-scale random and dynamic models.

[0041] (3) UAV energy management and mobility strategy: The energy consumption of a UAV is closely related to its movement path, execution calculation, etc. When optimizing the trajectory of a UAV and making unloading decisions, considering the UAV’s onboard energy, service range, resource constraints, etc., provides computing services to users, which increases the complexity of the problem and brings new challenges to the optimization goal.

[0042] In one embodiment, Figure 2 As shown, a multi-UAV assisted mobile edge computing trajectory optimization method is provided, comprising the following steps:

[0043] Step 202: construct a user association model.

[0044] The user association model refers to the correspondence between the computing tasks to be offloaded by the drone and the user device.

[0045] In the user association model, it is limited to each time slot:

[0046] 1) Each drone can accept a maximum of a preset number of computing tasks within its coverage area. During the offloading process, Orthogonal Frequency Division Multiplexing (OFDM) is used, which means that due to the limited number of subcarriers, each drone can only accept a maximum of If the distance between the drone and the user device does not exceed the preset value, it means that the user device is within the coverage of the drone. If the number of user devices associated with the drone is less than , then consider whether to establish an association with the user equipment and provide offloading services for it according to the urgency of the computing task.

[0047] 2) Each computing task can only be executed by the drone or the user device. That is, each computing task is either performed locally on the user device or offloaded to the drone for execution in a binary full offload mode.

[0048] 3) Each user device generates at most one computing task request in each time slot. The coordinates of , the task request information set of the user device is expressed as:

[0049] ;

[0050] in for The coordinates of Indicates the amount of data to be processed, Indicates the number of CPU cycles required to perform this task. Indicates the task type. Indicates the maximum delay that the task can tolerate, in units of time slots. Indicates the time when the task is issued. Indicates the completion time of the task, and the user's task request information set is . Define uninstall variables , , , Indicates time slot , Offload tasks to , It means Execute the computation locally.

[0051] Therefore, the following constraints exist:

[0052] ;

[0053] ;

[0054] .

[0055] In this method, the maximum service coverage of the drone is limited, so the offloading request needs to meet two conditions: 1. The user who submits the task request in the current time slot needs to be within the coverage of the drone; 2. The total number of users whose tasks the drone chooses to offload does not exceed . In the designed algorithm, every time the drone moves to a new location, it needs to sort the user requests within the coverage area according to the remaining delay, and give priority to executing tasks with less remaining delay, so as to minimize the number of tasks that are missed due to violating the prescribed delay. This is a sequential decision problem under the maximum energy constraint of the drone. The trajectory optimization of the drone is mainly to meet the computational delay of the task request issued by the user device and the limited resource requirements of the drone. The output strategy will meet the total computational constraint of each time slot.

[0056] Step 204: construct a drone movement model.

[0057] The drone mobility model restricts drones and user devices to a preset area and a minimum distance between any two drones.

[0058] The coordinates of UAV and UE are defined as three-dimensional coordinates. The height of UE is set to 0, and the height of UAV is set to a fixed value H. The starting point of each drone is randomly set within the specified area. The flight action of the drone is determined by the flight angle and the flight distance per unit time slot. The minimum distance between drones and user equipment in a square area and between two drones is limited. t time slots, The three-dimensional Cartesian coordinates of , all UAVs fly at a fixed altitude. and In time slot t The distance is expressed as: . and The distance is expressed as: . Specify the time slot t , any two drones should maintain a minimum distance , so we have:

[0059] .

[0060] The initial starting position of each drone is not fixed, and the flight action is determined by the horizontal flight angle. and within the unit time slot The flight distance is determined by , , assuming that the drone can fly at a constant speed in a time slot. Therefore, in time slot t, The coordinates are expressed as:

[0061] ;

[0062] .

[0063] In addition, assuming that each It is restricted to fly within the target area, and the constraint is expressed as:

[0064] ;

[0065] .

[0066] Step 206, constructing a task calculation model.

[0067] The task calculation model is used to calculate the task calculation time of each computing task in each time slot.

[0068] There are two ways to execute computing tasks on user devices: local computing or offloading to the drone mobile edge server for execution. The drone decides how to offload the tasks requested by the user based on the urgency of the user task request and the availability of computing resources within its coverage area.

[0069] Assume that in time slot t, exist within the coverage area, and The computational tasks are offloaded to However, in time slot t+1, No longer there Within the coverage area, if the computing task has not been completed at this time, The task will continue to execute until it is completed. After the task is completed, the calculation results can be found in Re-enter coverage, or Returned when requested.

[0070] Step 208: construct a drone energy consumption model.

[0071] The UAV energy consumption model is used to limit the total energy consumption of each UAV during its entire service life.

[0072] Step 210, building a deep reinforcement learning model of the drone based on the user association model, the drone movement model, the task calculation model and the drone energy consumption model.

[0073] Reinforcement learning modeling based on Markov decision process mainly includes three parts: state, action and reward. In each time slot, the UAV agent interacts with the environment and makes a decision based on the state. Take Action And get a reward , and then obtain a new state from the environment This method requires local observations to determine the next action, and global observations to understand the overall system state and other UAV agent behaviors, in order to balance the relationship between local optimization and global optimization. The local observations, global observations, reward functions for actions and feedback, and state transfer functions of the UAV agent are specifically defined as follows:

[0074] 1) The drone’s own state, the information of the computing task currently being performed, and the relative distance between it and other drones within its coverage area are defined as local observations.

[0075] Among them, the drone's own status includes: the drone's location, remaining energy, and idle resources. Since the drone has a radar, an embedded global positioning system (GPS) module, and a battery management system, it can continuously monitor the drone's real-time location and remaining battery power as well as the distance between it and other drones. It is also equipped with an internal monitoring system that can monitor and report the use of computing resources on the drone in real time, so the drone has full access to its own status information. The drone decides on the next move and user association decision based on this information.

[0076] 2) The aggregated information of local observations of all UAVs in the environment within the preset area is defined as global observation.

[0077] 3) The flight angle and flight distance of the drone in a unit time slot are defined as actions. Specifically, The value range of is [0,2π), The value range can be set to [0,20] meters.

[0078] 4) Construct a reward function based on the total task calculation time, the failure rate of calculating task request responses, and the limitations in the drone movement model.

[0079] The main optimization goal of this invention is to minimize the total time to complete the task and the failure rate of task request response. Therefore, the reward function is defined as

[0080] ,

[0081] ;

[0082] in, This is the penalty for a drone that exceeds the boundary or collides. If a drone flies out of the target area or two drones collide, the penalty item will be triggered. , Represents the parameter for adjusting the weight. The drone performs actions in each time slot t After arriving at the new location, the user can provide unloading services according to the task requests submitted by the users in the current location and get rewards. .

[0083] A task request response failure means that the task calculation time exceeds the maximum tolerance time of the corresponding calculation task request, that is, . Response failure is determined by whether the interval between the task's issuance time (in time slots) and completion time (in time slots) is greater than the maximum tolerance time. That is, if the task's completion time slot minus the task's issuance time slot is greater than the task's maximum tolerance time, the task response is considered to have failed. The drone is only responsible for completing the task calculation and returning the result, while whether the task is completed within the tolerance time is automatically determined by the system based on the time slot interval. If a task request fails to respond, it will no longer be recalculated and will be counted in the failed response task request, and finally used to calculate the failure rate of the task response.

[0084] Defining an invalid task set ,The failure rate of task request response is defined as the ratio of the number of tasks in the invalid task set to the total number of tasks, expressed as:

[0085] ;

[0086] 5) State transition: The state transition probability defines the system dynamics that depends only on the previous state and action, and represents the probability of the random variable S and these values ​​appearing in time slot t given the specific values ​​of the current state and action. Since both the state space and the action space are continuous and infinite, the space of state transition probabilities P is continuous and infinite and cannot be explicitly modeled. Therefore, in the absence of an explicit state transition probability model, this method chooses to use model-free RL techniques to enable the drone to learn to optimize joint trajectory design and user association through interaction with the environment.

[0087] Step 212, optimizing the movement trajectory of the drone with the goal of maximizing the reward function.

[0088] This method introduces deep reinforcement learning (DRL) technology, which enables drones to obtain reward feedback through interaction with the environment and adjust their execution strategies based on the feedback information to learn movement trajectories and user association strategies in a trial-and-error manner. Specifically, this method models the joint trajectory design and user association problem as a Markov decision process (MDP), where the Markov decision process optimizes a series of sequential decisions (i.e., each drone's movement decisions and user association decisions) to minimize the failure rate of task request responses and the total time to complete the task.

[0089] The above multi-UAV assisted mobile edge computing trajectory optimization method formulates the joint optimization of UAV trajectory and user association problem as a Markov decision process (MDP), in which the UAV makes decisions based on the state information at the beginning of each time slot t, including the remaining energy, available resources, and the current user device access volume, i.e., position movement and providing offloaded task requests. The sequential decision process is represented as the mapping relationship between the UAV state information, the task request information issued by the user device, and the current executable action decision. Specifically, at the beginning of each time slot t, each UAV moves to a new location, and each UAV decides whether to provide offload services to the user to perform the user's computing task based on the urgency of the user's task request within its coverage area and the availability of its own onboard computing resources. Under the RL framework, the UAV makes decisions to minimize the failure rate of task request response and the total time to complete the task in turn throughout the simulation time, while considering the safety issues within the simulation time (i.e., avoiding collisions between UAVs and flying out of the boundary).

[0090] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0091] In one embodiment, calculating the task computing time of each computing task in each time slot includes:

[0092] 1) If the computing task in the user device is offloaded to the drone in the current time slot, that is, offloaded computing is performed, the task computing time is calculated as follows:

[0093] Based on the free space channel model, obtain the computing task upload rate of the user device:

[0094] ;

[0095] in, Indicates UE in time slot Upload the data in its computing task to the The speed on the drone, represents the channel bandwidth, , , Indicates the power gain of the channel when the reference distance is 1m. represents the noise power, Indicates The height of the drone is a fixed value. Indicates The transmit power of each user equipment, Indicates User devices and Drone in time slot The distance Represents a service cycle.

[0096] Assume that all tasks should be served within the maximum duration of period TS Since the result of the calculation feedback is relatively small, the transmission delay of downloading the result back from the drone can be ignored.

[0097] Calculate the computing task offloading time of the user device based on the computing task upload rate of the user device:

[0098] ;

[0099] in, Indicates UE in time slot Offload the data in its computing tasks to the The time required to fly a drone, Indicates UE in time slot The amount of data for the computing task;

[0100] The drone's computing task execution time is calculated based on the number of CPU cycles required to execute the computing task on the user device:

[0101] ;

[0102] in, Indicates UE in time slot Offload the data in its computing tasks to the After the drone is launched, the time required for the drone to perform the computing task is Indicates UE in time slot The number of CPU cycles required to execute the computing task, Indicates Drone in time slot Can be assigned to The computing resources of each user device;

[0103] Obtaining the task computing time according to the sum of the computing task unloading time of the user equipment and the computing task execution time of the drone;

[0104] 2) If the computing task in the user device is executed by itself in the current time slot, that is, local computing is performed, the task computing time is calculated as follows:

[0105] ;

[0106] in, Indicates UE in time slot The time required to perform local calculations.

[0107] In summary, the total completion time of each computing task can be expressed as:

[0108] .

[0109] In one embodiment, assuming that the onboard energy of the drone has a maximum limit, the main energy consumption during the entire service cycle comes from flight energy consumption, hovering energy consumption and computing energy consumption; among which:

[0110] The flight energy consumption is calculated as follows:

[0111] ;

[0112] in, Indicates Drone in time slot The flight energy consumption, Indicates from Click to The distance of the point, Indicates The flight power per unit distance of a drone;

[0113] The hovering energy consumption is calculated as follows:

[0114] ;

[0115] in, Indicates Drone in time slot The hovering energy consumption, Indicates Drone in time slot The hover time is offloaded by the computing tasks of the user device. and the computing task execution time of the drone composition, Indicates The hovering power per unit time of a UAV, assuming The hovering power per unit time is a constant, which is related to the mass of the drone. , propeller radius l and number of propellers The hovering power of the UAV can be expressed as ;

[0116] in, is the Earth's gravitational acceleration, Indicates the air density;

[0117] The energy consumption is calculated as follows:

[0118] ;

[0119] in, Indicates Drone in time slot The computing energy consumption, represents the effective switched capacitance, Indicates Drone in time slot Execute The CPU power of the computing tasks offloaded from the user device, Indicates UE in time slot The number of CPU cycles required to execute the computing task;

[0120] Since the UAV has a total energy constraint, the total energy consumption during the entire service cycle satisfies the following inequality constraint:

[0121] ;

[0122] in, Indicates The total energy consumption of a drone during its entire service life, Represents the onboard energy limit of the UAV.

[0123] In one embodiment, the moving trajectory of the drone is optimized with the goal of maximizing the reward function, including:

[0124] In the training phase, training data is first sampled from the experience buffer; the training data includes: global observations of the current time slot, local observations of the drone in the current time slot, actions performed by the drone in the current time slot and the rewards obtained after performing the actions, global observations of the next time slot, and local observations of the drone in the next time slot;

[0125] The local observation of the drone in the current time slot is input into the Actor network, and the action of the drone in the next time slot, i.e., the flight angle and flight distance, is output to update the position of the drone;

[0126] The Critic network is used to evaluate the value of each drone state and calculate the advantage value. The Critic network has the same network architecture as the Actor network except for the input layer and output layer.

[0127] Use the advantage value to update the Actor network and Critic network, and finally obtain the trained Critic network and Actor network;

[0128] During the execution phase, each drone interacts with the environment based on the actions output by the trained Actor network.

[0129] Multi-agent reinforcement learning can provide a distributed perspective for intelligent task offloading of drone-mounted edge computing systems. This algorithm includes U agents and an edge computing environment, where each drone interacts with the environment as an agent and collects training data for training the policy model. Based on the framework of centralized training and decentralized execution, each agent has two stages, namely centralized training and decentralized execution. In the training stage, data is sampled directly from the experience buffer for policy learning. In the execution stage, the agent only needs to interact with the environment according to the actions output by the Actor network. Taking an agent as an example, it explains how to centralize the training of the MAPPO model (consisting of a Critic network and an Actor network) and how to execute the learned model in a decentralized manner. In the centralized training stage, the Critic acts as a central coordinator to calculate the centralized action value function based on the global state information. , which uses the behavior of all agents and local observation information for calculation. The Critic network is used to evaluate the actions of the agents from a global perspective and guide the Actor network to make better choices. Subsequently, the Q function is updated by minimizing the loss to achieve the update of the model. The loss function is as follows, where:

[0130] ;

[0131] ;

[0132] in , is the Critic network parameter, Update the state for the target network, Update parameters for the Critic network. On the other hand, the Actor network updates network parameters , each agent outputs an action based on the centralized Q function calculated by the Critic network and its own local observations. Specifically, the Actor network is Directly adjust the network parameter θ in the direction of the global state, and transform the partially observable MDP into a fully observable MDP by assuming a centralized value function, thereby achieving faster and simpler value learning. This is based on minimizing the actual return of the action value function and the current state action combination, that is, the input is the global state and the action combination of all agents, and the output is the action value function; the loss function in the pseudocode is based on minimizing the state value function, that is, the input is the state, and the output is the state value function; these two calculation methods are combined in the specific algorithm design.

[0133] The policy network (Actor network) is responsible for generating the probability of the drone's actions in each time slot and performing action sampling based on the probability. The value function network (Critic network) evaluates the value of each state (the drone's current position, onboard resources, urgency of surrounding tasks, etc.). The drone obtains current state information and rewards by interacting with the environment, and then calculates the gradient of the policy network (Actor network) to update the policy to maximize the reward function.

[0134] Specifically, we first set the basic parameters of the environment, such as the number of agents, the dimension of the observation space, the dimension of the action space, create the agent object, and initialize the replay buffer. During the training process, the strategy of each agent is determined by the Actor network, which takes the local observation data of the agent as input and then outputs the angle and distance of the next time slot to update the position of the drone. The Actor network is constructed using a multi-layer perceptron MLP. The Critic network structure is the same as the Actor network except for the input and output layers. It evaluates the value of each agent state and is used to calculate the advantage value. At the beginning of training, the agent interacts with the environment to collect experience, which is stored in the replay buffer. When the data in the buffer accumulates to a certain amount, the network is updated. First, the training data is obtained from the experience replay buffer, including data including observations, actions, rewards, next states, etc., which are used to calculate the policy advantage and update the value function. MAPPO uses GAE (Generalized Advantage Estimation) to calculate the advantage and use these advantage values ​​to update the Actor and Critic. The actor is updated by maximizing the PPO (Proximal Policy Optimization) objective function, which includes policy gradient loss and entropy regularization term to enhance the exploration of the policy. These updates are iterated multiple times on small batches of samples to improve the stability and effectiveness of the policy. The critic network is updated by minimizing the mean square error of the value function, with the goal of reducing the estimation error of the value function. This helps improve the accuracy and effectiveness of the Actor network's generated policy. Throughout the training process, some techniques are also applied to enhance the performance of the algorithm, such as entropy coefficient adjustment, gradient clipping, and learning rate decay. These techniques help stabilize the training process and improve the convergence and generalization ability of the algorithm. The data obtained from the playback buffer is used to calculate the advantage value and perform policy optimization. A variety of techniques are used to improve the training effect, such as advantage normalization, gradient clipping, learning rate decay, and policy clipping. These techniques help stabilize the training process, prevent overfitting, and improve the convergence speed and performance of the algorithm. In each training step, the policy loss of the Actor network and the value function loss of the Critic network are calculated based on the advantage value, and then the network parameters are updated through the Adam optimizer. In addition, the learning rate decay technique is added to gradually reduce the learning rate as the number of training steps increases to further optimize the convergence effect of the strategy. The specific training process is as follows:

[0135] MAPPO (Multi-Agent Proximal Policy Optimization for Mobile Edge Computing) algorithm

[0136] 1: Initialize the Actor network using orthogonal Parameters , parameters of the Critic network V , data buffer pool D={};

[0137] 2: Set the learning rate ;

[0138] 3:for episode=1 to episode_max;

[0139] 4: while t <T_max;

[0140] 5: ;

[0141] 6: do for all agents;

[0142] 7: Get the local observation of the current agent , global observation ;

[0143] 8: ;

[0144] 9: ;

[0145] 10: ;

[0146] 11: ;

[0147] 12:end for;

[0148] 13: Perform joint actions , get rewards , observation , ;

[0149] 14: do for all agents;

[0150] 15: Push to D;

[0151] 16:end for;

[0152] 17: end while;

[0153] 18: Sample data b from the experience buffer D;

[0154] 19: Use Calculate the advantage function of the data in b;

[0155] 20: Calculate the loss function of the Actor network:

[0156] renew ;

[0157] 21: Calculate the loss function of the Critic network renew ;

[0158] 22:end for;

[0159] The MAPPO algorithm uses a centralized training and decentralized execution framework to train the critic network with global states and input local states to output the action decisions of each drone. The dynamic balance between the local and global observations of the agent enables the drone to adapt to changes in the local environment while considering the overall system state and the behavior of other drones.

[0160] In order to solve the delay sensitivity problem of mobile edge computing user requests, improve the efficiency of mobile computing resources, and meet the shortcomings of conflict optimization algorithms in mixed integer nonlinear programming optimization problems with multiple variables and multiple constraints, the present invention proposes a multi-drone-assisted mobile edge computing scenario, and designs a distributed drone edge computing platform based on the MAPPO algorithm to meet the user's task requirements, adapt to the randomness of user task requests and the dynamic nature of onboard computing resources. By introducing deep reinforcement learning (DRL) technology, the drone maximizes the reward function by interacting with the environment. The proposed algorithm can handle the random arrival model of tasks in a dynamic environment while meeting the drone resource constraints. This depends on the strong generalization performance of the training model, which enables the drone to make intelligent decisions under any resource and task request state. Considering the trade-off between the urgency of the task and the response time, the algorithm needs to minimize the execution time of the task while ensuring the task completion rate, and maximize the total number of computing tasks completed under the limited onboard energy constraints of the drone.

[0161] The algorithm proposed in this paper can provide offloading services that meet the latency requirements for the latency-sensitive task requests randomly received by users in the edge computing environment. The optimal association combination is achieved by optimizing the movement trajectories of multiple drones during the execution of the offloading service to minimize the request failure rate. At the same time, the mobility security and regional security between multiple drones are considered. Specifically, the following goals can be achieved:

[0162] 1) Task allocation optimization: The MAPPO algorithm can dynamically decide on user task offloading based on the urgency of the user’s task request and the availability of the UAV’s onboard computing resources, and allocate tasks to appropriate UAVs, thereby optimizing the success rate and time of task completion.

[0163] 2) Minimizing task response time: The MAPPO algorithm optimizes the UAV’s movement trajectory and task allocation strategy, enabling the system to quickly respond to user task requests, minimize the task response failure rate, and improve the overall service quality.

[0164] 3) Dynamic environment adaptability: Since the MAPPO algorithm has the ability to learn and adapt to environmental changes, it can handle the dynamic changes of user task requests and drone resource status in the environment, thereby ensuring the efficient operation of the system in different scenarios. Since the MAPPO algorithm uses the characteristics of reinforcement learning, it can achieve long-term optimization in complex dynamic environments, and through interactive learning with the environment, find a balance between continuous exploration and utilization, and improve the system's adaptability and decision-making quality.

[0165] 4) The method proposed in this paper can jointly optimize the movement trajectory and user association of the UAV without obtaining precise system information, so that the task request proposed by the user device can be completed in the shortest time and with the lowest response failure rate.

[0166] In one embodiment, a multi-UAV assisted mobile edge computing trajectory optimization device is provided, including: a user association model construction module, a UAV movement model construction module, a task calculation model construction module, a UAV energy consumption model construction module, a deep reinforcement learning model construction module and a UAV movement trajectory optimization module, wherein:

[0167] A user association model building module is used to build a user association model; the user association model refers to the correspondence between the computing tasks to be offloaded by the drone and the user equipment; in the user association model, it is limited that in each time slot: each drone accepts a maximum of a preset number of computing tasks within its coverage area, each computing task can only be executed by a drone or a user equipment, and each user equipment generates at most one computing task request;

[0168] A drone mobility model building module is used to build a drone mobility model; the drone mobility model limits the drone and user equipment within a preset area and the minimum distance between any two drones;

[0169] A task calculation model building module is used to build a task calculation model; the task calculation model is used to calculate the task calculation time of each calculation task in each time slot;

[0170] The UAV energy consumption model building module is used to build the UAV energy consumption model; the UAV energy consumption model is used to limit the total energy consumption of each UAV during the entire service cycle;

[0171] The deep reinforcement learning model construction module is used to construct a deep reinforcement learning model of the drone based on the user association model, the drone movement model, the construction task calculation model and the drone energy consumption model. Specifically, the drone's own state, the information of the currently executed calculation task and the relative distance between it and other drones within its coverage area are defined as local observations; wherein the drone's own state includes: the drone's position, remaining energy and idle resources; the aggregated information of the local observations of all drones in the environment within the preset area is defined as a global observation; the flight angle and flight distance of the drone in a unit time slot are defined as actions; the reward function is constructed based on the total task calculation time, the calculation task request response failure rate and the restrictions in the drone movement model; the task request response failure means that the task calculation time exceeds the maximum tolerance time of the corresponding calculation task request;

[0172] The UAV trajectory optimization module is used to optimize the UAV trajectory with the goal of maximizing the reward function.

[0173] For the specific limitations of the multi-UAV assisted mobile edge computing trajectory optimization device, please refer to the limitations of the multi-UAV assisted mobile edge computing trajectory optimization method above, which will not be repeated here. Each module in the above-mentioned multi-UAV assisted mobile edge computing trajectory optimization device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0174] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as local observations of drones. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a multi-drone assisted mobile edge computing trajectory optimization method is implemented.

[0175] Those skilled in the art will understand that Figure 3The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0176] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method in the above embodiment when executing the computer program.

[0177] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in the above embodiment are implemented.

[0178] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0179] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0180] The above embodiments only express several implementation modes of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. A multi-UAV assisted mobile edge computing trajectory optimization method, characterized in that: The method comprises: Constructing a user association model; the user association model refers to the correspondence between the computing tasks to be offloaded by the drone and the user equipment; in the user association model, it is defined that in each time slot: each drone accepts a maximum of a preset number of computing tasks within its coverage area, each computing task can only be executed by the drone or the user equipment, and each user equipment generates at most one computing task request; Constructing a drone mobility model; the drone mobility model limits the drone and user equipment within a preset area and the minimum distance between any two drones; Constructing a task calculation model; the task calculation model is used to calculate the task calculation time of each calculation task in each time slot; Constructing a drone energy consumption model; the drone energy consumption model is used to limit the total energy consumption of each drone during the entire service cycle; Based on the user association model, the drone movement model, the construction task calculation model and the drone energy consumption model, a deep reinforcement learning model of the drone is constructed, specifically: the drone's own state, the information of the currently executed calculation task and the relative distance between the drone and other drones within its coverage range are defined as local observations; wherein the drone's own state includes: the drone's position, remaining energy and idle resources; the aggregated information of local observations of all drones in the environment within a preset area is defined as a global observation; the flight angle and flight distance of the drone in a unit time slot are defined as actions; a reward function is constructed according to the total task calculation time, the calculation task request response failure rate and the restrictions in the drone movement model; task request response failure means that the task calculation time exceeds the maximum tolerance time of the corresponding calculation task request; Optimizing the movement trajectory of the UAV with the goal of maximizing the reward function; In the training phase, training data is first sampled from the experience buffer; the training data includes: global observations of the current time slot, local observations of the drone in the current time slot, actions performed by the drone in the current time slot and rewards obtained after performing the actions, global observations of the next time slot, and local observations of the drone in the next time slot; The local observation of the drone in the current time slot is input into the Actor network, and the action of the drone in the next time slot, i.e., the flight angle and flight distance, is output to update the position of the drone; The Critic network is used to evaluate the value of each drone state and calculate the advantage value. The Critic network has the same network architecture as the Actor network except for the input layer and output layer. Use the advantage value to update the Actor network and Critic network, and finally obtain the trained Critic network and Actor network; In the execution phase, each drone interacts with the environment according to the actions output by the trained Actor network; The advantage value is calculated as follows: ； in, represents the advantage value at time slot t, is the timing differential error at time slot t, defined as: in is the immediate reward at time slot t, is the discount factor, ranging between [0,1], and It is a global observation and The value function of is a GAE parameter, and its range is also between [0,1]. is the attenuation factor, indicating that as time goes by, the impact of future temporal difference errors on the current advantage value will gradually decrease.

2. The method according to claim 1, characterized in that The calculation of the task calculation time of each computing task in each time slot includes: 1) If the computing task in the user device is offloaded to the drone in the current time slot, that is, offloaded computing is performed, the task computing time is calculated as follows: Get the computing task upload rate of the user's device: ； in, Indicates UE in time slot Upload the data in its computing task to the The speed on the drone, represents the channel bandwidth, = / , ≈2.2846, g0 represents the power gain of the channel when the reference distance is 1m, represents the noise power, Indicates The height of the drone is a fixed value. Indicates The transmit power of each user equipment, Indicates User devices and Drone in time slot The distance Represents a service cycle; Calculate the computing task offloading time of the user device based on the computing task upload rate of the user device: ； in, Indicates UE in time slot Offload the data in its computing tasks to the The time required to fly a drone, Indicates UE in time slot The amount of data for the computing task; The drone's computing task execution time is calculated based on the number of CPU cycles required to execute the computing task on the user device: ； in, Indicates UE in time slot Offload the data in its computing tasks to the After the drone is launched, the time required for the drone to perform the computing task is Indicates UE in time slot The number of CPU cycles required to execute the computing task, Indicates Drone in time slot Can be assigned to The computing resources of each user device; Obtaining the task computing time according to the sum of the computing task unloading time of the user equipment and the computing task execution time of the drone; 2) If the computing task in the user device is executed by itself in the current time slot, that is, local computing is performed, the task computing time is calculated as follows: ； in, Indicates UE in time slot The time required to perform local calculations.

3. The method according to claim 1, characterized in that The energy consumption of drones includes flight energy consumption, hovering energy consumption and computing energy consumption; among which: The flight energy consumption is calculated as follows: ； in, Indicates Drone in time slot The flight energy consumption, Indicates from Click to The distance of the point, Indicates The flight power per unit distance of a drone; The hovering energy consumption is calculated as follows: ； in, Indicates Drone in time slot The hovering energy consumption, Indicates Drone in time slot The hover time is offloaded by the computing tasks of the user device. and the computing task execution time of the drone composition, Indicates The hovering power per unit time of a UAV; The energy consumption is calculated as follows: ； in, Indicates Drone in time slot The computing energy consumption, represents the effective switched capacitance, Indicates Drone in time slot Execute The CPU power of the computing tasks offloaded from the user device, Indicates UE in time slot The number of CPU cycles required to execute the computing task; The above-mentioned limit on the total energy consumption of each drone during the entire service cycle is as follows: ； in, Indicates The total energy consumption of a drone during its entire service life, Represents the onboard energy limit of the UAV.

4. The method according to claim 1, characterized in that The reward function is constructed based on the total task calculation time, the failure rate of task request response, and the restrictions in the drone movement model: , ； ； ； in, Indicates time slot All rewards obtained by drones, Indicates Drone in time slot Rewards received, Indicates Penalty items are triggered when a drone collides with other drones or flies out of the preset area. represents the weight parameter of the reward, represents the weight parameter of the penalty term, Indicates time slot The failure rate of computing task request response, Indicates the total number of computing tasks, Indicates time slot The number of failed computing task requests. Indicates UE in time slot Offload the data in its computing tasks to the The first drone and The total time required for the UAV to perform the computing task.

5. The method according to claim 1, characterized in that The loss function of the Actor network is: ； in, Represents the parameters of the Actor network, represents the ratio of the new and old strategies, represents the advantage value at time slot t, represents the shear range of PPO, represents the weight of the entropy reward, represents the optimization strategy, Indicates in status Next action probability.

6. The method according to claim 1, characterized in that The loss function of the Critic network is: ； in, Represents the parameters of the Critic network, Indicates using the current parameters The critic network of the state The estimated value of ) represents the discounted value of the state value function at time t+1.

7. A multi-UAV assisted mobile edge computing trajectory optimization device, characterized in that: The device comprises: A user association model construction module, used to construct a user association model; the user association model refers to the correspondence between the computing tasks to be offloaded by the drone and the user equipment; in the user association model, it is defined that in each time slot: each drone accepts a maximum of a preset number of computing tasks within its coverage area, each computing task can only be executed by the drone or the user equipment, and each user equipment generates at most one computing task request; A drone mobility model building module, used to build a drone mobility model; the drone mobility model limits the drone and user equipment within a preset area and the minimum distance between any two drones; A task calculation model building module, used to build a task calculation model; the task calculation model is used to calculate the task calculation time of each calculation task in each time slot; A UAV energy consumption model building module is used to build a UAV energy consumption model; the UAV energy consumption model is used to limit the total energy consumption of each UAV during the entire service cycle; A deep reinforcement learning model construction module is used to construct a deep reinforcement learning model of the drone based on the user association model, the drone movement model, the construction task calculation model and the drone energy consumption model, specifically: the drone's own state, the information of the currently executed calculation task and the relative distance between the drone and other drones within its coverage range are defined as local observations; wherein the drone's own state includes: the drone's position, remaining energy and idle resources; the aggregated information of local observations of all drones in the environment within a preset area is defined as a global observation; the flight angle and flight distance of the drone in a unit time slot are defined as actions; a reward function is constructed based on the total task calculation time, the calculation task request response failure rate and the restrictions in the drone movement model; task request response failure means that the task calculation time exceeds the maximum tolerance time of the corresponding calculation task request; The UAV movement trajectory optimization module is used to optimize the UAV movement trajectory with the goal of maximizing the reward function. In the training stage, training data is first sampled from the experience buffer. The training data includes: global observations of the current time slot, local observations of the UAV in the current time slot, actions performed by the UAV in the current time slot and rewards obtained after performing the actions, global observations of the next time slot, and local observations of the UAV in the next time slot. The local observations of the UAV in the current time slot are input into the Actor network, and the actions of the UAV in the next time slot, i.e., flight angle and flight distance, are output to update the position of the UAV. The Critic network is used to evaluate the value of each UAV state for calculating the advantage value. The Critic network has the same network architecture as the Actor network except for the input layer and the output layer. The Actor network and the Critic network are updated using the advantage value, and finally the trained Critic network and the Actor network are obtained. In the execution phase, each drone interacts with the environment according to the actions output by the trained Actor network; the advantage value is calculated as follows: ； in, represents the advantage value at time slot t, is the timing differential error at time slot t, defined as: in is the immediate reward at time slot t, is the discount factor, ranging between [0,1], and It is a global observation and The value function of is a GAE parameter, and its range is also between [0,1]. is the attenuation factor, indicating that as time goes by, the impact of future temporal difference errors on the current advantage value will gradually decrease.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

Images