Mobile edge computing task unloading optimization method based on unmanned aerial vehicle and RIS assistance

Abstract

The invention relates to a mobile edge computing task unloading optimization method based on an unmanned aerial vehicle and RIS assistance. The method comprises the following steps: constructing a mobile edge computing system model based on unmanned aerial vehicle and RIS assistance and a communication channel model of the system; constructing a time delay model and an energy consumption model of a system processing task, and constructing an energy consumption model of the unmanned aerial vehicle; according to the time delay model and the energy consumption model of the mobile edge computing system processing task and the energy consumption model of the unmanned aerial vehicle, calculating the maximum time delay and the system energy consumption of the user equipment processing task; based on the unmanned aerial vehicle task scheduling constraint, the user equipment, the RIS and unmanned aerial vehicle movement range constraint, the task unloading proportion constraint, the RIS phase range constraint, the unmanned aerial vehicle resource constraint and the system minimum task constraint, a target optimization function P after task processing time delay and energy consumption weighting of the system are minimized; and solving the target optimization function P by using a deep reinforcement learning method to obtain an optimal joint optimization scheme. According to the invention, the task unloading performance of the mobile edge computing system can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of mobile edge computing, and particularly relates to an optimization method for mobile edge computing task offloading based on drones and RIS assistance. Background Art

[0002] With the development of 5G technology, new applications with characteristics such as computing-intensive and latency-sensitive are emerging continuously, such as online games, autonomous driving, virtual reality, and augmented reality. When dealing with resource-intensive applications, more advantageous mobile edge computing (MEC) assistance is required, which helps to reduce the computing resources and energy consumption of running resource-intensive applications. By deploying computing resources with data processing and service functions closer to user terminals, mobile edge computing MEC can provide users with faster service computing power, thereby reducing the energy consumption and computing time of user devices themselves. It reduces the need to offload a large amount of data centrally to the cloud, thereby alleviating the pressure on the core network and improving the quality of service of real-time response scenarios.

[0003] In recent years, with the continuous progress of information and technology, unmanned aerial vehicles (UAVs) have been experiencing explosive growth and have been widely used in civilian fields such as traffic monitoring, public safety, search and rescue, and disaster relief reconnaissance. UAVs not only have extensive geographical coverage capabilities but also have unique advantages of rapid deployment and easy programming. Various payloads can be installed on UAVs, such as Internet of Things sensors (such as cameras), miniaturized base stations, and embedded computing modules, to achieve various sensing, communication, and computing tasks. Reasonable deployment and operation of UAVs can provide reliable and cost-effective wireless communication solutions for many practical scenarios.

[0004] Reconfigurable intelligent surface (RIS) brings transformative capabilities to wireless communication by providing fine control over the propagation of electromagnetic waves. RIS is particularly beneficial in environments where the line of sight is blocked or the signal strength is weak. RIS can be used to optimize the signal path, bypass obstacles to reflect or refract signals, and ensure the robustness and reliability of the communication link. This ability is particularly valuable for UAVs operating in complex dynamic environments. For UAVs, the integration of RIS has significant advantages in terms of communication reliability, energy efficiency, and computing load. By improving the signal transmission efficiency, RIS reduces the need for on-board intensive processing, saves power, and extends the flight time. In addition, RIS can also optimize the signal path to ensure that UAVs maintain a robust communication link without consuming too much energy, directly contributing to extending the operating time and reducing the operating cost.

[0005] Most existing studies usually consider the auxiliary role of UAVs or RISs in mobile edge systems separately. To further unleash the potential of mobile edge computing technology, there are great advantages in jointly using UAVs and RISs to assist mobile edge computing for task offloading in the system. The high mobility of UAVs provides a higher degree of freedom for RIS deployment, establishing high-quality communication links at both the transceiver ends, thus alleviating the problem of communication link blockage and improving the comprehensive performance of mobile edge computing. However, there are still various problems: for example, in terms of energy management, the batteries of UAVs are small and the endurance time is relatively short, while the need to use UAVs for trajectory optimization to offload tasks in the entire service area and minimize the overall transmission and calculation delay may lead to problems of insufficient energy and extended task offloading time. Summary of the Invention

[0006] Aiming at the deficiencies of the existing technology, the present invention considers the scenario of long-distance communication between the cloud server and the mobile terminal. The cloud server is affected by ground signal blockage and shadow and cannot provide edge computing services to remote device areas. Therefore, the present invention proposes an optimization method for mobile edge computing task offloading based on UAV and RIS assistance.

[0007] An optimization method for mobile edge computing task offloading based on UAV and RIS assistance includes:

[0008] S1: Construct a mobile edge computing system model based on UAV and RIS assistance and the communication channel model of this system;

[0009] S2: According to the mobile edge computing system model, construct the delay model and energy consumption model for the system to process tasks, and construct the energy consumption model of the UAV.

[0010] S3: According to the delay model, energy consumption model for the mobile edge computing system to process tasks, and the energy consumption model of the UAV, calculate the maximum delay for the user equipment to process tasks and the system energy consumption;

[0011] S4: Based on the UAV task scheduling constraints, user equipment, RIS and UAV movement range constraints, task offloading ratio constraints, RIS phase range constraints, UAV resource constraints and system minimum task constraints in the mobile edge computing system, construct an objective optimization function P that minimizes the weighted delay and energy consumption of the system to process tasks;

[0012] S5: Use the deep reinforcement learning method to solve the objective optimization function P to obtain the optimal joint optimization scheme.

[0013] The beneficial effects of the present invention are:

[0014] In the mobile edge computing system, the present invention considers using an unmanned aerial vehicle (UAV) that performs task offloading as an aerial base station to cooperate with a ground base station (i.e., an edge cloud base station) to perform task offloading, supplemented by a UAV equipped with a reconfigurable intelligent surface (RIS) to provide a reflected link to enhance the communication link gain, overcoming the problem of long service response latency when the base station is far from the user equipment. The UAV that performs task offloading will partially calculate the user equipment tasks through the line-of-sight link and the RIS reflected link, and forward some of the computing tasks to the base station. The use of the RIS reflected link can enhance the gain of the user equipment. During the entire flight cycle of the UAV, on the premise of satisfying the battery capacity of the UAV and completing the computing tasks of the user equipment, by jointly optimizing the UAV (including the UAV equipped with RIS and the UAV that performs task offloading) task scheduling, trajectory, RIS phase, and cooperative offloading ratio, the weighted system processing task latency and system energy consumption are minimized, and an objective optimization function P is established. Finally, the TD3 algorithm is used to solve the objective optimization function P; the system performs real-time optimization on the UAV task scheduling, the UAV-edge cloud cooperative offloading ratio, and the trajectories of the RIS UAV and the task offloading UAV in the system model according to the optimal joint optimization scheme. The simulation results show that the UAV task scheduling, trajectory optimization, and task offloading framework constructed by the TD3 algorithm can obtain the minimum processing latency and system energy consumption, thereby improving the performance of the mobile edge computing system. Description of the Drawings

[0015] Figure 1 It is a flowchart of the steps of an embodiment of the present invention;

[0016] Figure 2 It is a mobile edge computing system model assisted by UAV and RIS in an embodiment of the present invention;

[0017] Figure 3 It is a schematic diagram of training the Actor network and the Critic network using the TD3 algorithm in an embodiment of the present invention;

[0018] Figure 4 It is a TD3 algorithm reward convergence graph in an embodiment of the present invention during simulation verification. Detailed Embodiment

[0019] The terms "first" and "second" etc. in the description and claims of this application are used to distinguish different objects, rather than to describe a specific order of the objects.

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] The embodiment of the present invention proposes a mobile edge computing task offloading optimization method based on drone and RIS assistance, referring to Figure 1 As shown, the method includes:

[0022] S1: Construct a UAV- and RIS-assisted mobile edge computing MEC system model and the communication channel model of the system.

[0023] Reference Figure 2 As shown, a mobile edge computing system assisted by drones and RIS is constructed, and the system includes: in the target area, an edge cloud base station for processing computing tasks, a drone equipped with RIS, a drone for performing task offloading (the drone is equipped with a processor to assist MEC task offloading) and multiple user equipments (UEs) distributed in different areas are set up; wherein the drone and the edge cloud base station provide computing services to the terminal (i.e., user equipment UEs) in a time-division manner, and the entire communication cycle It is divided into T time slots, that is, t∈{1,2,...,T}. In the MEC system, the set of user equipment UEs is represented by m={1,2,...,M}, where M represents the number of user equipments, and the user equipments are kept at a fixed height above the ground, which is represented by H UEs , the three-dimensional Cartesian coordinates of the user device are expressed as: w m (t) = [x m (t),y m (t)] T , w m (t) represents the user equipment UE served by the drone in time slot t m The two-dimensional coordinate information of (.) T Indicates transposition, x m (t) represents the ground x-axis coordinate of the user equipment corresponding to the tth time slot, y m (t) represents the ground y-axis coordinate of the user equipment at the tth time slot. The UAV equipped with RIS is kept at a fixed altitude, which is represented by H RIS ; Let the unloading drone maintain a fixed altitude, which is represented by H UAV The coordinates of the UAV equipped with RIS at the beginning of the time slot (i.e., the tth time slot) are expressed as Ω RIS =(qRIS (t), H RIS ), where q RIS (t) represents the two-dimensional coordinates of the UAV carrying the RIS projected on the ground at time slot t, and q RIS (t) = [x RIS (t), y RIS (t)] T ∈R 2×1 . The coordinates of the UAV performing task offloading at the start of the time slot (i.e., the t-th time slot) are respectively represented as Ω UAV = (q UAV (t), H UAV ), where q UAV (t) represents the two-dimensional coordinates of the UAV (UAV) projected on the ground at time slot t, and q UAV (t) = [x UAV (t), y UAV (t)] T ∈R 2×1 . At the end of this time slot (i.e., the (t + 1)-th time slot), the coordinates of the UAV (UAV) carrying the RIS are represented as Ω RIS = (q RIS (t + 1), H RIS ), and the coordinates of the UAV (UAV) performing task offloading are represented as Ω UAV = (q UAV (t + 1), H UAV ). Let the edge cloud base station be fixed on the ground, and an edge cloud server is set on the edge cloud base station, and its two-dimensional coordinates are fixedly represented as P s , P s = (x s , y s ).

[0024] In the mobile edge computing MEC system model, a channel model for system communication is constructed.

[0025] Users carry user equipment UEs and move randomly at a low speed within a limited area. In each time slot, the UAV flies to a fixed position and hovers, and then establishes communication with any one of the user equipment UEs. After the terminal (i.e., the user equipment) offloads the computing task to the server according to the allocation plan, the remaining computing tasks are executed locally. Assume that the transmitted computing tasks can partially use the computing resources of the UAV server (i.e., the UAV performing task offloading) to execute, and the UAV can transfer the remaining computing tasks to the edge cloud server (i.e., the edge cloud base station) for computing.

[0026] Assume that for the user equipment UE m and the UAV performing task offloading in the uplink communication, the channel gain of its line-of-sight link is expressed as:

[0027]

[0028] g m,u d(t)=α0d m,u (t) -2 (2)

[0029] where d m,u (t) represents the Euclidean distance between the user equipment UE and the drone performing task offloading at the t-th time slot, ||*|| represents the Euclidean norm, q m (t) represents the two-dimensional coordinates of the projection of the drone on the ground, w UAV (t) represents the two-dimensional coordinate information of the user equipment UE served by the drone at the t-th time slot, m H m represents the height maintained by the drone performing task offloading, and ɑ0 represents the channel gain at the reference distance d = 1m. UAV The RIS deployed on the drone redirects the signal between the drone and the user equipment. The RIS has R reflection elements forming a uniform linear array, and the passive phase shift of the RIS must be calculated to form the phase array of the rectangular array. To simulate the phase shift of the RIS, assume that θ

[0030] (t) represents the phase of the i-th reflection element at the t-th time slot, i Φ(t) represents the phase array of the R elements at time slot t, Φ(t) = diag{e ,e jθ1t ,...,e jθ2t ,...,e jθRt}, where j represents the imaginary unit and e represents the natural logarithm.

[0031] The channel gain g r,u (t) of the communication link between the drone performing task offloading and the RIS at the t-th time slot is specifically expressed as:

[0032]

[0033] where d r,u (t) represents the Euclidean distance between the drone performing task offloading and the drone carrying the RIS at the t-th time slot, q UAV (t + 1) represents the two-dimensional coordinates of the projection of the drone performing task offloading on the ground at time slot t + 1, q RIS (t + 1) represents the two-dimensional coordinates of the projection of the drone carrying the RIS on the ground at time slot t + 1, H UAV represents the height maintained by the drone performing task offloading, and H RIS represents the height maintained by the drone carrying the RIS, Denote the cosine of the angle of arrival (AoA) of the signal in the \(t\)-th time slot, \(d\) represents the antenna spacing, \(\lambda\) represents the carrier wavelength, \(j\) represents the imaginary unit, and \(e\) represents the natural logarithm.

[0034] User Equipment UE m Channel gain \(g(t)\) of the communication link between the User Equipment UE and the RIS at the \(t\)-th time slot, which is specifically expressed as: m,r (t), which is specifically expressed as:

[0035]

[0036] where \(d\) m,r (t) represents the Euclidean distance between the User Equipment UE and the RIS at the \(t\)-th time slot m and \(\cos\theta_{d}(t)\) represents the cosine of the angle of departure (AoD) of the signal in the \(t\)-th time slot. represents the cosine of the angle of departure (AoD) of the signal in the \(t\)-th time slot.

[0037] When the line-of-sight link between the drone performing task offloading and the user is blocked, the channel gain achieved with RIS assistance is specifically expressed as:

[0038]

[0039] where \(\alpha\) m (t) means that in the \(t\)-th time slot, the task scheduling of the drone only provides task offloading service for one User Equipment UE m and \(\alpha(t)=\{0, 1\}\). m (t) = {0, 1}.

[0040] To evaluate the probability that the line-of-sight link between the drone performing task offloading and the user is blocked, an air-to-ground channel model in an urban environment is considered, where the blocking probability \(p(t)\) between the drone performing task offloading and the User Equipment UE m at time slot \(t\) is specifically expressed as: m (t), which is specifically expressed as:

[0041]

[0042] where \(a\) and \(b\) are constant values depending on the environment.

[0043] Therefore, the achievable average channel gain between the drone performing task offloading and the RIS is specifically expressed as:

[0044]

[0045] The channel gain \(g(t)\) of the line-of-sight link in the downlink between the drone performing task offloading and the edge cloud server (i.e., the edge cloud base station) g，t is specifically expressed as

[0046]

[0047] g u,s g(t) = α0d u,s g(t) -2 (11)

[0048] where d u,s g(t) represents the Euclidean distance between the UAV performing task offloading and the edge cloud server at the t-th time slot.

[0049] Assume that the UAVs (including the UAV performing task offloading and the UAV carrying RIS) fly within the Ω region, and the communication ranges of the UAV carrying RIS and the UAV performing task offloading with the user equipment UE m are defined as:

[0050] ||q RIS / UAV (t + 1) - w m (t)|| 2 ≤ d max (12)

[0051] where d max represents the maximum communication range between the UAV carrying RIS and the UAV performing task offloading and the user equipment UE m and q RIS / UAV (t + 1) represents the two-dimensional coordinate information of the UAV carrying RIS or the UAV performing task offloading.

[0052] Considering the occlusion by obstacles, the uplink wireless transmission rate r m between the user equipment UE and the UAV performing task offloading, which is specifically expressed as: m,u (t), is specifically expressed as:

[0053]

[0054] where B m,u represents the communication bandwidth between the user equipment UE m and the UAV performing task offloading, P up is the transmission rate of the user equipment UEs uplink, σ 2 represents the noise power, and represents the achievable average channel gain between the UAV performing task offloading and RIS.

[0055] The downlink wireless transmission rate r u,s (t) between the UAV performing task offloading and the edge cloud server (i.e., the edge cloud base station) is specifically expressed as:

[0056]

[0057] Among them, B u,s represents the channel bandwidth between the drone performing task offloading and the edge cloud server, and P dw represents the transmission power of the downlink between the drone performing task offloading and the edge cloud server. g(t) represents

[0058] the channel gain of the downlink LOS link between the drone performing task offloading and the edge cloud server (i.e., the edge cloud base station).

[0059] By adding drones equipped with RIS, drones performing task offloading, and edge cloud base stations (equipped with edge cloud servers) to the mobile edge computing system model, using task processing devices to enhance the task offloading performance of the system, and adopting the high mobility of drones to enhance system compatibility.

[0060] S2: According to the mobile edge computing system MEC model, construct the delay model and energy consumption model for the system to process tasks, and construct the energy consumption model for the drone.

[0061] Specifically, the delay and energy consumption of the mobile edge computing system MEC for processing tasks include: the delay and energy consumption of the user equipment for local task execution, the delay and energy consumption of the user equipment for transmitting part of the computing tasks to the drone performing task offloading, the energy consumption of part of the task calculation of the drone performing task offloading, the transmission delay and energy consumption of the drone performing task offloading for offloading part of the tasks to the edge cloud server (i.e., the edge cloud base station), and the computing delay of the edge cloud server.

[0062] S201: Calculate the delay and energy consumption of the user equipment for local task execution.

[0063] In the mobile edge computing system MEC, for the tasks of user equipment UEs in each time slot, a partial offloading strategy is adopted. Let R m,u (t) represent the proportion of tasks offloaded to the drone server, and (1 - R m,u (t)) represent the proportion of the remaining tasks executed locally at the terminal (i.e., the user equipment UEs). That is, the delay l local,m (t) and energy consumption E local,m (t) of the user equipment for local task execution at time slot t are specifically expressed as:

[0064]

[0065] E local,m (t) = k m (f m ) 3 l local,m (t) (16)

[0066] Among them, l local,m(t) represents the user equipment UE at time slot t m The latency of local task execution, L m (t) represents the user equipment UE at the t-th time slot m The size of the computing task, c m Represents the user equipment UE m The CPU cycles required to process each unit byte, f m Represents the computing frequency of the user equipment UEs, E local,m (t) represents the user equipment UE at time slot t m The energy consumption of local task execution, k m Represents the CPU effective capacitance coefficient of the user equipment UEs.

[0067] S202: Calculate the latency, energy consumption of the user equipment transmitting the computing task to the drone for task offloading, and the computing energy consumption of the drone for task offloading.

[0068] The user transmits the remaining computing task to the drone for task offloading, and the transmission latency of the user equipment is expressed as:

[0069]

[0070] In the formula, l m,u (t) represents the user equipment UE m The latency generated by transmitting the remaining computing task to the drone for task offloading, R m,u (t) represents the user equipment UE m The task ratio of offloading part of the task to the drone server for task offloading, L m (t) represents the size of the computing task of the user equipment UEm at the t-th time slot, r m,u (t) represents the user equipment UE m The wireless transmission rate of the uplink between the user equipment and the drone for task offloading.

[0071] Let R u,s (t) represents the task ratio of the drone for task offloading to offload part of the task to the edge cloud server, (1 - R u,s (t)) represents the remaining task ratio executed by the drone for task offloading, and the computing latency generated on the MEC server of the drone for task offloading, which is specifically expressed as:

[0072]

[0073] Among them, l u (t) represents the computing latency required for the drone for task offloading to execute the task, c uDenote the CPU cycles required for the UAV server to process each unit byte as f u Denote the computing frequency of the CPU of the UAV server that executes task offloading.

[0074] In time slot t, the transmission energy consumption E m,u (t) consumed when offloading the computing task to the UAV server that executes task offloading is specifically expressed as:

[0075] E m,u (t) = P up l m,u (t) (19)

[0076] In the formula, E m,u (t) represents the transmission energy consumption when the user equipment UE m offloads the computing task to the UAV server that executes task offloading. P up represents the transmit power of the uplink of the user equipment UE m and l m,u (t) represents the delay generated when the user equipment offloads the remaining computing tasks to the UAV that executes task offloading.

[0077] The computing energy consumption E u (t) of the UAV MEC server that executes task offloading in the t-th time slot is specifically expressed as:

[0078] E u (t) = k u (f u ) 3 l u (t) (20)

[0079] In the formula, E u (t) represents the computing energy consumption of the UAV that executes task offloading in the t-th time slot, and k u represents the effective capacitance coefficient of the CPU of the UAV that executes task offloading.

[0080] S203: Calculate the transmission delay and energy consumption when the UAV that executes task offloading offloads part of the tasks to the edge cloud server, and the computing delay of the edge cloud server.

[0081] Assume that in time slot t, the UAV that executes task offloading further offloads to the edge cloud server (i.e., the edge cloud base station) for processing when handling part of the offloading tasks, and its transmission delay is expressed as:

[0082]

[0083] In the formula, l u,s(t) represents the transmission delay when the UAV performing task offloading offloads part of the offloading tasks to the edge cloud server, R m,u (t) represents the task ratio of the user equipment UEs offloading part of the tasks to the UAV server performing task offloading, L m (t) represents the user equipment UE at the t-th time slot m of the computing task size, R u,s (t) represents the task ratio of the UAV performing task offloading offloading part of the tasks to the edge cloud server, r u,s (t) represents the wireless transmission rate of the downlink between the UAV performing task offloading and the edge cloud server (i.e., the edge cloud base station).

[0084] The computing delay l generated by the computing tasks offloaded to the edge cloud server s (t) is expressed as:

[0085]

[0086] where c s represents the CPU cycles required for the edge cloud server to process each unit byte, f s represents the computing frequency of the edge cloud server CPU.

[0087] Assume that in time slot t, only the energy transmission energy consumption consumed by offloading the computing tasks from the UAV to the edge cloud server is considered, and the computing energy consumption of the edge cloud server is not considered. The transmission energy consumption E u,s (t), which is specifically expressed as:

[0088] E u,s (t) = P dw l u,s (t)(23)

[0089] In the formula, E u,s (t) represents the transmission energy consumption when the UAV performing task offloading offloads part of the tasks to the edge cloud server, P dw represents the transmission power of the downlink between the UAV performing task offloading and the edge cloud server (i.e., the edge cloud base station), l u,s (t) represents the transmission delay when the UAV performing task offloading offloads part of the tasks to the edge cloud server.

[0090] S204: Calculate the flight energy consumption and hovering energy consumption of the UAV.

[0091] In the t-th time slot, the UAV (including the UAV performing task offloading and the UAV carrying RIS) flies from the starting position to a new position and hovers, which is expressed as:

[0092] qRIS / UAV (t + 1) =

[0093] [x RIS / UAV (t) + v RIS / UAV (t)t fly cosβ RIS / UAV (t), y RIS / UAV (t) + v RIS / UAV (t)t fly sinβ RIS / UAV (t)] T

[0094] Where q RIS / UAV (t + 1) represents the hovering position of the UAV at time slot t + 1, x RIS / UAV (t) represents the ground x-axis coordinate corresponding to the UAV at the t-th time slot, y RIS / UAV (t) represents the ground y-axis coordinate corresponding to the UAV at the t-th time slot, v RIS / UAV (t) represents the flight speed of the UAV, v RIS / UAV (t) ∈ [0, v max , v max represents the maximum flight speed of the UAV, t fly represents the fixed flight time of the UAV, β RIS / UAV (t) represents the flight angle of the UAV, β RIS / UAV (t) ∈ [0, 2π].

[0095] The UAV performs flight motion with speed v RIS / UAV (t) and angle β RIS / UAV (t) parameters, and the energy consumed for each flight and the energy consumed for hovering are respectively expressed as:

[0096]

[0097] Among them, represents the energy consumption of each flight of the UAV at time slot t, φ = 0.5M u t fly , M u represents the mass of the UAV, t fly represents the fixed flight time of the UAV, represents the energy consumption of each hover of the UAV at time slot t, g0 is the earth's gravity, r u is the radius of the propeller of the rotor UAV, N U is the number of propellers, ρ A is the air density under standard atmospheric pressure, t hover represents the hovering time of the UAV.

[0098] In the MEC system, the size of the calculation result of the server is usually very small and can be ignored. Transmission delay and energy consumption are not considered in the downlink between the UAV and the user.

[0099] An embodiment of the present invention proposes a system model for mobile edge computing task offloading assisted by UAVs and RIS. Referring to Figure 2 as shown, when considering that the user equipment is far from the edge cloud base station, the UAV for executing task offloading, the UAV carrying RIS, and the edge cloud base station cooperate to complete the computing tasks of the user equipment, and the system performance is optimized by optimizing the moving trajectory of the UAV and the collaborative offloading ratio.

[0100] S3: According to the delay model, energy consumption model of the mobile edge computing system MEC, and the energy consumption model of the UAV, calculate the maximum delay D m (t) of the user equipment UE m for processing tasks and the system energy consumption E m (t).

[0101] To ensure the effective utilization of limited computing resources, this paper aims to minimize the maximum processing delay and system energy consumption of all UEs by jointly optimizing the UAV scheduling, UAV mobility, and computing task allocation in the system. m

[0102] At time slot t, the maximum delay D m (t) of the user equipment UE m for processing tasks and the system energy consumption E m (t) are respectively expressed as:

[0103] D m (t) = max{l local,m (t), l m,u (t) + max(l u (t), l u,s (t) + l s (t))}(26)

[0104] In the formula, max(.) represents maximizing, and l local,m (t) represents the delay of the user equipment UE m executing tasks locally at time slot t, l m,u (t) represents the delay generated when the user equipment offloads the remaining computing tasks to the UAV for executing task offloading, l u (t) represents the computing delay required for the UAV for executing task offloading to execute tasks, l u,s (t) represents the transmission delay when the UAV for executing task offloading offloads some offloading tasks to the edge cloud server, l s(t) represents the computing delay for the partial offloading task to be executed on the edge cloud server.

[0105]

[0106] In the formula, E local,m (t) represents the energy consumption of the user equipment UE for local task execution at time slot t, E m m,u (t) represents the transmission energy consumption when the user equipment offloads the computing task to the drone server for task offloading at time slot t, E u (t) represents the computing energy consumption of the drone for task offloading at the t-th time slot, E u,s (t) represents the transmission energy consumption when the drone for task offloading offloads part of the task to the edge cloud server, represents the energy consumption of the drone carrying the RIS for each flight at time slot t, represents the energy consumption of the drone for task offloading for each flight at time slot t, represents the energy consumption of the drone carrying the RIS for each hover at time slot t, represents the energy consumption of the drone for task offloading for each hover at time slot t.

[0107] In the embodiments of the present invention, by weighting the maximum processing delay and system energy consumption of the offloading task, the impact of processing delay and energy consumption on the system is balanced, and it is possible to complete the system computing task with the minimum maximum processing delay at a relatively minimum system energy consumption.

[0108] S4: Based on the drone task scheduling constraints, user equipment, RIS, and drone movement range constraints, task offloading ratio constraints, RIS phase range constraints, drone resource constraints, and system minimum task constraints in the mobile edge computing system MEC, construct an objective optimization function that minimizes the weighted delay and energy consumption of the system for processing tasks.

[0109] The objective optimization function P is expressed as:

[0110]

[0111] The first constraint condition C1:

[0112] The second constraint condition C2:

[0113] The third constraint condition C3:

[0114] In the formula, α m (t) indicates that the task scheduling of the drone at time slot t only provides task offloading services for one user equipment, q UAV ​(t + 1) represents the two-dimensional coordinates of the projection of the UAV performing task offloading on the ground at time slot t + 1, q RIS (t + 1) represents the two-dimensional coordinates of the projection of the UAV equipped with RIS on the ground at time slot t + 1, θ i (t) represents the phase array of R elements in RIS of the UAV equipped with RIS at time slot t, R u,s (t) represents the task ratio of the user equipment UEs offloading part of the tasks to the UAV server performing task offloading at time slot t, R u,s (t) represents the task ratio of the UAV performing task offloading offloading part of the tasks to the edge cloud server. T represents the number of time slots that divide the entire communication cycle of the system into multiple time slots, M represents the number of user equipment, and δ represents the first weight coefficient. represents the second weight coefficient. By adjusting δ, to adjust the weighted coefficients of the delay in processing tasks and the system energy consumption in the MEC system.

[0115] Solve the target optimization function P. By jointly optimizing the UAV task scheduling, trajectory, RIS phase, and cooperative offloading ratio, minimize the weighted sum of the delay in processing tasks and the system energy consumption in the system. Since the UAV position changes in real time with time slots, the target optimization function P is a coupled discrete non-convex problem. To solve this target optimization function more efficiently, the problem is converted into a Markov decision problem, and using the deep reinforcement learning method to solve it is a reliable method.

[0116] S5: Use the deep reinforcement learning method to solve the target optimization function to obtain the optimal joint optimization scheme.

[0117] Use the deep reinforcement learning method to solve the target function to obtain the optimal joint optimization scheme. The system performs real-time optimization of the UAV scheduling, the UAV-edge cloud cooperative offloading ratio, and the deflection angle and moving distance between the RIS UAV and the task offloading UAV in the system according to this scheme.

[0118] The specific steps of step S5 include:

[0119] S501: According to the target optimization function P, construct a Markov decision process, and define the state space, action space, and reward function of the intelligent agent.

[0120] Define the state space: The system state s at time slot t t ∈ S. The system state includes the energy and position of the UAV equipped with RIS, the energy and position of the UAV performing task offloading, the positions of the user equipment UEs, the remaining task volume of the system, and the task volume of the user equipment UEs. Therefore, the system state s at time slot t t is expressed as:

[0121]

[0122] Among them, represents the remaining energy of the UAV battery equipped with RIS at the \(t\)-th time slot, and \(q\) RIS (\(t\)) represents the position information of the UAV equipped with RIS. represents the remaining energy of the UAV battery performing task offloading at the \(t\)-th time slot, and \(q\) UAV (\(t\)) represents the position information of the UAV performing task offloading at time slot \(t\), and \(w\) m (\(t\)) represents the user equipment UE m served by the UAV, and \(L\) remain (\(t\)) represents the remaining task size that the system needs to complete within the entire time period, and \(L\) m (\(t\)) represents the user equipment UE m generates a random task size at the \(t\)-th time slot. When \(t = 1\), \(E\) battery (\(t\)) = \(E\) l and \(L\) remain (\(t\)) = \(L\).

[0123] Perform state normalization to preprocess the observed state, enabling the deep neural network to be trained more effectively.

[0124] Use the difference between the maximum and minimum values of each variable as the scaling factor to solve the problem of magnitude differences between input variables:

[0125] and \(L\) M (\(t\)) are in different magnitude ranges in the state set, which may lead to problems with difficult convergence during the training process. Normalize the input variables to enable the training to converge. Five scaling factors are used in the state normalization algorithm. Use the scaling factor \(\gamma\) l to scale the UAV battery capacity. Since RIS, UAV, and UEs have the same \(x\) and \(y\) coordinate ranges, \(\gamma\) x and \(\gamma\) y are respectively used to scale the \(x\) and \(y\) coordinates of RIS, UAV, and UEs. Use \(\gamma\) Dm to scale the remaining tasks within the entire time period, and use \(\gamma\) Due to scale the task size of each UEs in time slot \(t\);

[0126] Define the action space of the agent: The agent selects actions based on the current state of the system and the observed environment, including: the user equipment UEs that the UAV will serve at time slot \(t\) m , the deflection angle and moving distance of the UAV equipped with RIS, the deflection angle and moving distance of the UAV performing task offloading, the UAV task offloading ratio, and the further offloading to the edge cloud base station task offloading ratio. The action \(a\)t It can be expressed as:

[0127]

[0128] where m(t) represents the normalized user equipment served by the UAV in time slot t, and β RIS (t) represents the flight angle of the UAV carrying the RIS, and β UAV (t) represents the flight angle of the UAV performing task offloading, and v RIS (t) represents the flight speed of the UAV carrying the RIS in time slot t, and v UAV (t) represents the flight speed of the UAV performing task offloading in time slot t, and R m,u (t) represents the task ratio of the user equipment UEs to offload part of the tasks to the UAV server performing task offloading, and R u,s (t) represents the task ratio of the UAV performing task offloading to offload part of the tasks to the edge cloud server.

[0129] The Actor network of TD3 outputs continuous actions. For the user UE m (t) ∈ [1, M], the selected m(t) by the agent needs to be discretized. If m(t) = 1, then m' = M; if m(t) ≠ 1, then m' = [m(t) × M], where [·] is the rounding function. The flight angle, flight speed, and task offloading ratio of the UAV can be precisely optimized in the continuous action space, that is, the angle β(t) ∈ [0, 2π], the speed v(t) ∈ [0, v max , the offloading ratio R m,u (t) ∈ [0, 1] to the UAV and the offloading ratio R u,s (t) ∈ [0, 1] to the base station;

[0130] Define the reward function: The policy of the agent is reward-based, and choosing an appropriate reward function plays a crucial role in the performance of the TD3 framework. The goal is to maximize the reward by minimizing the weighted sum of the processing delay and system energy consumption, that is, the immediate reward r t is expressed as:

[0131]

[0132] where the processing delay of the system at time slot t is expressed as:

[0133]

[0134] The system energy consumption at time slot t is expressed as:

[0135]

[0136] And if m' = m, then α m (t) = 1, otherwise α m (t) = 0. δ represents the first weight coefficient, represents the second weight coefficient, which is used to adjust the weights of the system processing delay and the system energy consumption.

[0137] S502: According to the TD3 network structure, construct the corresponding Actor network and Critic network structures.

[0138] It should be noted that the TD3 network (Twin Delayed Deep Deterministic Policy Gradient) is a deep reinforcement learning algorithm for solving continuous control problems, and it is an improved version of the DDPG (Deep Deterministic Policy Gradient) algorithm.

[0139] The Actor network takes the state as the input layer, three fully connected layers, and the output layer, and uses relu and tanh as activation functions. Specifically, it includes:

[0140]

[0141] The output layer of the Actor network using the tanh activation function generates the corresponding action a t , and scales the output action within the range; the Critic network takes the state and the action respectively as the output of the fully connected layer of the input, and the result after splicing the two outputs is mapped to the Q value through the last fully connected layer and using the linear activation function in the output layer, which is used to evaluate the quality of the action generated by the Actor network.

[0142] S503: Based on the Actor network, the Critic network, and the Markov decision process, dynamically optimize the cooperative offloading ratio of the UAV-edge cloud, the UAV equipped with RIS, and the optimal trajectory of the UAV performing task offloading.

[0143] The overall framework of the TD3 algorithm is referred to Figure 3 as shown. The training processes of the Actor network and the Critic network specifically include:

[0144] S601: Initialize the network. Specifically, use the parameters θ of the Actor network μ , the parameters θ of the target Actor network μ' , the parameters of the Critic1 network the parameters of the Critic2 network the parameters of the target Critic1 network and the parameters of the target Critic2 network Initialize the Actor network μ(s t |θ μ ), the Actor target network μ'(s t |θ μ' ), the Critic1 network the Critic2 network the Critic1 target network and the Critic2 target network Initialize the experience replay buffer simultaneously;

[0145] S602: The Actor network generates an action a t according to the current state s t . The agent executes the action a t and observes the next time-slot state s t+1 and the immediate reward r t , and updates the state s t+1 . Then, store the experience tuple (s t , a t , r t , s t+1 ) into the experience replay buffer with a capacity of M;

[0146] S603: The agent randomly samples a batch of experience tuples (s t , a t , r t , s t+1 ) from the experience replay buffer, generates the next action through the target Actor network and calculates the target Q value expressed as:

[0147]

[0148] where θ μ' represents the parameters of the target Actor network, θ Q' represents the parameters of the target Critic network, ε is the noise added to the output of the Actor target network to avoid Actor overfitting, μ'(·) is the policy generated by the target Actor network, r t is the immediate reward, γ is the discount factor, s t+1 is the next time-slot state, and Q i '(·) represents the Q function of the target Q value network.

[0149] Update the target network using the Averaging method.

[0150] The TD error is expressed as:

[0151]

[0152] where yt represents the target Q value, s t is the system state at time slot t, Q(·) represents the Q value function under the current policy, a t is the task scheduling, flight action, and task offloading allocation action performed by the UAV, θ Q are the Critic network parameters.

[0153] Update the Critic network using the gradient descent method:

[0154]

[0155] Update the Actor network every d steps, and calculate the Actor policy gradient expressed as

[0156]

[0157] Update the Actor network using the gradient ascent:

[0158]

[0159] Update the target network using the Polyak Averaging method:

[0160]

[0161] θ μ' ← τθ μ' +(1 - τ)θ μ' (43)

[0162] Repeat these steps until the training is completed to optimize the flight path of the UAV, including the flight distance and angle, and finally obtain the optimal trajectory planning.

[0163] Simulation verification:

[0164] The simulation verification software: pycharm compilation software, specifically, the python3.10 version.

[0165] Figure 4 is the reward convergence graph of the TD3 algorithm in the embodiment of the present invention during simulation verification.

[0166] Refer to Figure 4 As shown, the present invention solves the UAV task scheduling, trajectory, RIS phase, and cooperative offloading ratio through the deep reinforcement learning method (specifically, the TD3 algorithm) to minimize the weighted system processing task delay and system energy consumption, verifying the effectiveness of the optimization method.

[0167] The above-described embodiments have further elaborated in detail the object, technical solution and advantages of the present invention. It should be understood that the above-described embodiments are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made to the present invention within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A mobile edge computing task offloading optimization method based on drones and RIS, characterized in that: include: Construct a mobile edge computing system model based on drones and RIS assistance and a communication channel model of the system; According to the mobile edge computing system model, the delay model and energy consumption model of the system processing tasks are constructed, and the energy consumption model of the drone is constructed. According to the delay model, energy consumption model of the mobile edge computing system processing tasks and the energy consumption model of the drone, the maximum delay and system energy consumption of the user equipment processing tasks are calculated; Based on the UAV task scheduling constraints in the mobile edge computing system, the user equipment, RIS and UAV movement range constraints, the task offloading ratio constraints, the RIS phase range constraints, the UAV resource constraints and the system minimum task constraints, the objective optimization function P that minimizes the weighted delay and energy consumption of the system processing tasks is constructed; The deep reinforcement learning method is used to solve the objective optimization function P and obtain the optimal joint optimization solution.

2. The method for optimizing mobile edge computing task offloading based on drone and RIS assistance according to claim 1 is characterized in that: The mobile edge computing system model based on drone and RIS assistance includes: an edge cloud base station for processing computing tasks, a drone equipped with RIS, a drone for performing task offloading, and M user devices. The edge cloud base station is provided with an edge cloud server.

3. The method for optimizing mobile edge computing task offloading based on drone and RIS assistance according to claim 1 is characterized in that: The latency and energy consumption of task processing in the mobile edge computing system include: the latency and energy consumption of local task execution by the user device, the latency and energy consumption of the user device transmitting part of the computing task to the drone that performs task offloading, the computing energy consumption of the drone that performs task offloading, the transmission latency and energy consumption of the drone that performs task offloading to offload part of the task to the edge cloud server, and the computing latency of the edge cloud server.

4. The method for optimizing mobile edge computing task offloading based on drone and RIS assistance according to claim 1 is characterized in that: The energy consumption of a drone includes its flight energy consumption and hovering energy consumption.

5. The method for optimizing mobile edge computing task offloading based on drone and RIS assistance according to claim 1 is characterized in that: The user equipment UE m The maximum delay D of processing tasks m The calculation formula of (t) is: D m (t)=max{l local,m (t),l m,u (t)+max(l u (t),l u,s (t)+l s (t))} In the formula, max(.) means to maximize, l local,m (t) represents the user equipment UE at time slot t m The latency of executing tasks locally, l m,u (t) represents the delay caused by the user equipment offloading the remaining computing tasks to the UAV that performs the task offloading, l u (t) represents the computational delay required for the unmanned aerial vehicle to perform the task, l u,s (t) represents the transmission delay of the UAV performing task offloading to offload part of the offloaded tasks to the edge cloud server, l s (t) represents the computational delay of executing some offloaded tasks on the edge cloud server; The system energy consumption E m The calculation formula of (t) is: In the formula, E local,m (t) represents the energy consumption of the local task executed by the user equipment at time slot t, E m,u (t) represents the transmission energy consumed by the user equipment to offload the computing task to the UAV server that performs the task offloading in time slot t, E u (t) represents the computational energy consumption of the UAV performing task offloading in the tth time slot, E u,s (t) represents the transmission energy consumption of the UAV performing task offloading to offload part of the task to the edge cloud server, represents the energy consumption of each flight of the UAV equipped with RIS in time slot t, represents the energy consumption of each flight of the UAV performing task offloading in time slot t, represents the energy consumption of each hovering of the UAV equipped with RIS in time slot t, It represents the energy consumption of each hovering of the UAV performing task offloading in time slot t.

6. The method for optimizing mobile edge computing task offloading based on drone and RIS assistance according to claim 1 is characterized in that: The objective optimization function P is expressed as: The first constraint C1: The second constraint The third constraint C3: In the formula, α m (t) indicates that the task scheduling of the UAV in time slot t only provides task offloading service to one user device, q UAV (t+1) represents the two-dimensional coordinates of the UAV that performs task unloading at time slot t+1 projected on the ground, q RIS (t+1) represents the two-dimensional coordinates of the UAV carrying RIS projected on the ground at time slot t+1, θ i (t) represents the time slot t, which represents the phase array of R elements in the RIS in the UAV equipped with RIS, R m,u (t) represents the proportion of tasks that the UEs offload to the UAV server that performs task offloading in time slot t, R u,s (t) represents the proportion of tasks that the drone performing task offloading offloads part of its tasks to the edge cloud server, T represents the number of time slots into which the entire communication cycle of the system is divided, M represents the number of user devices, δ represents the first weight coefficient, Represents the second weight coefficient.

7. The method for optimizing mobile edge computing task offloading based on drone and RIS assistance according to claim 1 is characterized in that: The optimization objective function P is solved by using a deep reinforcement learning method, and the specific process includes: According to the objective optimization function P, a Markov decision process is constructed to define the state space, action space and reward function of the intelligent agent; According to the TD3 network structure, construct the Actor network and Critic network structure; Based on the Actor network, the Critic network and the Markov decision process, the ratio of UAV-edge cloud server collaborative task offloading, the optimal trajectory of the UAV equipped with RIS and the optimal trajectory of the UAV performing task offloading are jointly and dynamically optimized.

8. The method for optimizing mobile edge computing task offloading based on drone and RIS assistance according to claim 7 is characterized in that: The system state in the state space is specifically expressed as: The fourth constraint C4: The fifth constraint C5: The sixth constraint C6: Seventh constraint Seventh constraint in, represents the remaining energy of the battery of the drone equipped with RIS at the tth time slot, q RIS (t) indicates the location information of the drone equipped with RIS, represents the remaining energy of the battery of the drone performing task offloading at the tth time slot, q UAV (t) represents the location information of the UAV performing task offloading at time slot t, w m (t) represents the user equipment UE served by the drone m Location information, L remain (t) represents the size of the remaining tasks that the system needs to complete in the entire time period, L m (t) represents user equipment UE m The task size is randomly generated in the tth time slot. When t=1, E battery (t) = E l , L remain (t) = L; The action a in the state space t Specifically expressed as: a t =(m(t),β RIS (t),β UAV (t),v RIS (t),v UAV (t),R m,u (t),R u,s (t)) Ninth constraint C9: Tenth constraint C10: Where m(t) represents the normalized served user equipment selected by the drone in time slot t, β RIS (t) represents the flight angle of the UAV equipped with RIS, β UAV (t) represents the flight angle of the UAV performing the task offloading, v RIS (t) represents the flight speed of the UAV equipped with RIS at time slot t, v UAV (t) represents the flight speed of the UAV performing task offloading in time slot t, R m,u (t) represents user equipment UE m The proportion of tasks that are partially offloaded to the drone server that performs task offloading, R u,s (t) represents the proportion of tasks that the drones performing task offloading have partially offloaded to the edge cloud server; The reward function is specifically expressed as: in, In the formula, represents the maximum processing delay of the system computing task in time slot t, ɑ m (t) indicates that the task scheduling of the drone in time slot t is only for one user equipment UE m Perform task offloading services. local,m (t) represents the time delay of the local execution of the task by the user equipment at time slot t, l m,u (t) represents the delay caused by the user equipment offloading the remaining computing tasks to the UAV that performs the task offloading, l u (t) represents the computational delay required for the unmanned aerial vehicle to perform the task, l u,s (t) represents the transmission delay of the UAV performing task offloading to offload part of the offloaded tasks to the edge cloud server, l s (t) represents the computational delay of executing some offloaded tasks on the edge cloud server, ε energy (t) represents the system energy consumption of the system computing task in time slot t, represents the energy consumption of each flight of the UAV equipped with RIS in time slot t, represents the energy consumption of each hovering of the UAV performing task offloading in time slot t, E local,m (t) represents the energy consumption of the local task executed by the user equipment at time slot t, E m,u (t) represents the transmission energy consumed by the user equipment to offload the computing task to the UAV server that performs the task offloading at time slot t, E u (t) represents the computational energy consumption of the UAV performing task offloading in the tth time slot, E u,s (t) represents the transmission energy consumption of the UAV performing task offloading to offload part of the task to the edge cloud server, M represents the number of user devices, δ represents the first weight coefficient, Represents the second weight coefficient.

Images