Method for enhancing safety communication and mobile edge calculation by carrying intelligent reflecting surface on unmanned aerial vehicle

Abstract

The invention discloses an IRS-UAV (unmanned aerial vehicle-mounted intelligent reflector) enhanced secure communication and mobile edge computing method, in which a relay UAV has limited data storage capability, and the IRS-UAV is equipped with an IRS to adaptively improve task uploading and task unloading. To minimize system costs, the trajectory of the UAV, the pairing of edge servers, the pairing of target users, the data offload ratio, and the reflective elements of the IRS are jointly optimized. In order to solve the problem of complex non-convex optimization, the invention provides an intelligent hybrid algorithm based on a deep dual-Q network (D3QN) algorithm and a deep deterministic policy gradient (DDPG) algorithm, and relates to an intelligent hybrid algorithm based on a deep dual-Q network (D3QN) algorithm and a deep deterministic policy gradient (DDPG) algorithm. Simulation results show that the intelligent scheme provided by the invention can effectively solve the security problem of complex unmanned aerial vehicle mobile communication, and good convergence is obtained in a short time. In addition, compared with a reference scheme, the scheme provided by the invention obviously reduces the system overhead.

Description

Technical Field

[0001] The present invention relates to a method for enhancing safety communication and mobile edge computing of an unmanned aerial vehicle equipped with an intelligent reflective surface, and belongs to the technical field of communications. Background Art

[0002] Due to the mobility and flexibility of unmanned aerial vehicles (UAVs), mobile edge computing (MEC) networks supporting UAVs have been widely used in constrained wireless communication environments. Specifically, edge servers have sufficient computing resources and can meet the computing needs of computationally intensive smart devices with only limited computing capabilities and energy power. In order to solve the communication between user equipment (UE) and edge servers, UAVs can provide flexible communication services and be responsible for part of the complex computing tasks. MEC (mobile edge computing, MEC) is considered to be an emerging paradigm that can strategically extend computing and storage resources beyond traditional cloud data centers and bring them closer to the edge network nodes of end users. By deploying computing tasks to edge devices, base stations, or routers that are physically closer to end users, MEC inherently promotes low-latency data processing and fast response times, thereby improving the service quality of mobile applications and Internet of Things (IoT) devices. In order to address the problem of overloaded or underloaded edge computing devices (ECDs) with disproportionate resource requests and to enhance the integrity of transmitted information during task offloading, a blockchain-enabled computation offloading method is proposed in Xu, X., Zhang, X., et al. Become: Blockchain-enabled computation offloading for IoT in mobile edge computing (IEEE Transactions on Industrial Informatics). A non-dominated sorting genetic algorithm III is used to generate resource allocation strategies. In Zhou, H., Jiang, K., et al. Deep reinforcement learning for energy-efficient computation offloading in mobile-edge computing (IEEE Internet of Things Journal), the authors studied the joint optimization of computation offloading and resource allocation in dynamic multi-user MEC systems, aiming to minimize the energy consumption of the entire MEC system. The optimization process simultaneously considers the latency constraints and uncertain resource requirements of heterogeneous computation tasks. However, the above works cannot meet the flexibility of communication tasks, especially when there are many obstacles and communication is restricted. Therefore, UAV-enabled MEC networks are considered due to the mobility of UAVs. Specifically, the drone is equipped with a signal transceiver to provide wireless communication services for target users and edge servers.Feng, W., Tang, J., et al. studied the UAV-aided non-orthogonal multiple access MEC system in the paper Hybrid beamforming design and resource allocation for UAV-aided wireless-powered mobile edge computing networks with NOMA (IEEE Journal on Selected Areas in Communications), where the MEC network can provide energy and computing services to the target. The total computing performance of the target is maximized while limiting energy consumption and coverage. The task priority factor is further considered in Zhou, W., Fan, L., et al. Nallanathan, A.: Priority-aware resource scheduling for UAV-mounted mobile edge computing networks (IEEE Transactions on Vehicular Technology). Computation utilization and priority impact are used to evaluate the performance of the system with bandwidth resources and computing power. In order to improve flexibility and coverage, multiple UAVs are considered in the MEC network in Yang, Z., Pan, C., et al. Energy efficient resource allocation in UAV-enabled mobile edge computing networks. (IEEE Transactions on Wireless Communications). The authors jointly optimize the transmit power, user association, computational capacity allocation, and trajectory to minimize the sum of the power. A low-complexity traditional algorithm that iteratively solves the three subproblems is proposed to solve this difficult optimization problem. UAVs act as computing nodes and relay nodes in Zhang, L., Ansari, N.: Latency-aware IoT service provisioning in UAV-aided mobile-edge computing networks. (IEEE Internet of Things Journal), where the authors adopt UAV-assisted MEC networks to significantly reduce the average latency of users. In order to minimize the average latency experienced by all user devices, the authors formulate the UAV-MEC problem and introduce a low-complexity approximation algorithm to address the optimization challenges posed by this formulation.However, given that UAV-assisted communication networks are often deployed in communication-constrained environments, it is imperative to explore how low-energy methods can further improve transmission efficiency, a factor not addressed in the previously mentioned studies. Summary of the invention

[0003] In view of the shortcomings of the prior art, the present invention studies a mobile intelligent reflecting surfaces (IRS) assisted UAV MEC network with multiple edge computing servers, wherein the relay UAV (Relay-UAV) has limited data storage capacity, and the intelligent reflecting surface UAV (IRS-UAV) is equipped with an IRS to adaptively improve task uploading and task offloading. In order to minimize the system cost, the trajectory of the UAV, the pairing of edge servers, the pairing of target users, the data offloading ratio and the reflective elements of the IRS are jointly optimized. For complex non-convex optimization problems, an intelligent hybrid algorithm based on the Deep Dueling Double Q-Network (D3QN) algorithm and the deep deterministic policy gradient (DDPG) algorithm is proposed.

[0004] The technical solution adopted by the present invention to solve its technical problem is: to establish a safe mobile edge computing method for a drone equipped with an intelligent reflective surface.

[0005] Step 1: Establish a smart reflective surface-assisted MEC channel model

[0006] There are Relay-UAV, IRS-UAV, multiple user devices and multiple edge servers in the EMC network. The relay UAV is equipped with a limited data storage of size L and provides data offloading service and task computing service. The IRS-UAV is equipped with an IRS to adaptively improve task uploading and task offloading, where the number of IRS reflective elements can be represented by N. The UAV hovers at a consistent altitude and can stop at a specific stop point, which can be represented by S = {1,...,s,...}. Therefore, the position of the relay UAV at the stop point s is represented by {x r,s ,y r,s ,h r}, the position of IRS-UAV at the stop point s is represented by {x i,s ,y i,s ,h i The set of edge computing servers can be represented by E = {1, ..., e, ..., E}, and the location of edge computing server e is represented by {x e ,y e,0}. The set of user devices can be represented by D = {1,...,d,...,D}. The position of user device d is represented by [x d ,y d ,0] is represented by the trajectories of the relay UAV and IRS-UAV, respectively. r = {q r,t} t∈T and Q i = {q i,t} i∈T Indicates that represents the time of each stop point selection, q i,t ∈S and q r,t ∈S. Therefore, the distance from the relay UAV and IRS-UAV at position t to the user device d can be recursively given by:

[0007]

[0008] The distances from the relay UAV and IRS-UAV to the computation server e at time t can be expressed as:

[0009]

[0010] The distance from the relay UAV to the IRS-UAV at time t can be expressed as:

[0011]

[0012] Similar to mmWave communications, the links from the UAV to the user device and computing server are considered line-of-sight communications. Therefore, the channel gain from the user device d to the relay UAV and IRS relay at time t can be expressed as

[0013]

[0014] Where μ represents the path loss exponent. The channel gain from IRS-UAV to relay UAV at time t is expressed as follows:

[0015]

[0016] Let φ i,t represents the diagonal matrix of IRS elements at time t, where

[0017]

[0018] where ε n,t and φ n,t denote the amplitude and phase shift of the IRS reflector element n, respectively. The upload data rate at the user device d when the UAV is at time t can be expressed as

[0019]

[0020] Where B represents bandwidth, P r represents the transmit power of the relay UAV, and represents the noise power at user device d at time t. The data offloading transmission rate from the relay UAV to the edge server e at time t can be expressed as

[0021]

[0022] Step 2: Build an EMC network delay model

[0023] The flight time consumption of the drone is expressed as

[0024]

[0025] Where V represents the flight speed of the UAV, and max(x,y) represents the maximum value between x and y. Let L d represents the size of the transmission data of user equipment d, and δ d represents the data offloading rate of user device d, L d ≤M,δ d Let β t,d indicates whether the relay UAV is addressing the data of user device d at time t. Therefore, the upload and offload delays can be expressed as:

[0026]

[0027] where ρ t Indicates whether the relay UAV performs the upload and unload tasks at time t. If the relay UAV uploads the task at time t, ρ t =1; otherwise, ρ t =0. t,e indicates whether the data in the buffer is offloaded to the edge computing server e at time t. If the relay UAV is paired with the computing server e to offload the data of the user device d, then σ t,e = l; otherwise, σ t,e =0.

[0028] The computation delay is expressed as follows:

[0029]

[0030] Among them, f r and f e represents the CPU cycles required for one bit at the drone side and the edge server, respectively, and F r and F eDenote the computing power of the drone and edge computing server respectively. The total delay includes flight delay, unloading and uploading delay, and computing delay, which can be expressed as follows:

[0031] T(Q r ,Q i )=T f +T l +T e .

[0032] Step 3: Build an energy consumption model

[0033] The flight energy cost of the drone is expressed as:

[0034] E f =T f P f

[0035] Where P f represents the flight power consumption of the relay UAV and IRS-UAV. The hovering energy cost of the relay UAV and IRS-UAV can be written as:

[0036] E h =P h (T l +T e ),

[0037] Where P h represents the hovering power consumption of relay UAV and IRS-UAV. For a mobile IRS-assisted IRS-enabled UAV, the total energy consumption of the MEC network can be expressed as

[0038] E=E f +E h . BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 The present invention proposes a MEC network with multiple users and edge servers supporting mobile IRS-assisted UAVs.

[0040] Figure 2 The present invention proposes an intelligent D3QN-DDPG based framework for mobile IRS-assisted UAVs to enable MEC networks.

[0041] Figure 3 The smart solution proposed for the present invention uses the convergence of the D3QN-DDPG algorithm.

[0042] Figure 4 The relationship between the time consumption of relay UAV access and different available bandwidths in the network proposed by the present invention.

[0043] Figure 5The relationship between the energy consumption of the network proposed in the present invention and the different available bandwidths accessed by the relay UAVs.

[0044] Figure 6 The relationship between the network system cost improvement proposed by the present invention and the different available bandwidths for relay UAV access. DETAILED DESCRIPTION

[0045] Intelligent Strategy of Mobile IRS-Assisted UAV Edge Computing Network Based on DRL

[0046] Step 1: Propose intelligent optimization schemes with real-time computing performance for IRS-UAV and Delayed UAV

[0047] Step 1-1: Solve the Markov decision process (MDP) problem.

[0048] Model-free reinforcement learning methods are used to address the decision-making challenges proposed in this study. The task of minimizing the system cost is conceptualized as a Markov decision process. The dynamics of the edge computing network for UAV-assisted mobility characterizes the environment, while the relay UAVs and IRS-UAV control center are considered as intelligent agents. When these agents operate in this environment, the MDP problem contains state space, action space, transition probability, and reward function. Further insights into the details of the MDP problem are elaborated below.

[0049] Calculate the action selected in the previous step, the time consumption obtained, and the energy consumption obtained. At time t, state S t ∈S can be expressed as

[0050] s t ={a t-1 ,T t ,E t},

[0051] where a t-1 represents the previously selected action at time t-1, T t represents the achievable transmission delay, and E t Indicates the achievable system consumption.

[0052] The actions of the agent are mapped to the action space, denoted by A. The agent actions at e include the trajectory of the IRS-UAV, the trajectory of the relay UAV, the pairing of the edge server, the pairing of the target user, and the data offloading ratio. t The action at ∈A can be expressed as:

[0053] a t {q r,t ,q i,t ,{φi,t},σ e,t ,β d,t ,δ t},

[0054] Among them, q r,t represents the repeater trajectory, q i,t represents the trajectory of IRS, σ t,e represents the pairing of edge servers, β t,d represents the pairing of target user n, and δ t represents the data offloading rate. The transition probability is given by L(s t+1 |s t ,a t ) represents the agent from state s t ∈S accesses the new state s t+1 ∈S, decision a t ∈A. The reward consists of the total system cost, time consumption penalty and energy consumption penalty, which can be expressed as

[0055] r=-C+v1p1+v e p e ,

[0056] where v1 and p e Represent the weights of time consumption penalty and energy consumption penalty respectively. l is the time consumption penalty, which can be expressed as

[0057] p t =max{0,T max -T},

[0058] Where T max Indicates the maximum tolerable delay. e is the penalty for energy consumption, expressed as follows:

[0059] p e =max{0,E max -E},

[0060] Where E max Represents the maximum tolerable energy cost.

[0061] The cumulative reward starting from time t is expressed as

[0062]

[0063] Where γ∈(0,1] represents the future impact factor, which is used to evaluate future rewards.

[0064] We can use the Bellman equation to evaluate the Q-value reward, expressed as

[0065]

[0066] The goal of this invention is to learn the optimal decision π * , where the optimal Bellman equation is expressed as

[0067]

[0068] Step 1-2: Propose an intelligent optimization solution based on D3QN-DDPG

[0069] The intelligent optimization scheme adopts D3QN-DDPG high-dimensional, large-scale reinforcement learning tasks. As a result, the D3QN algorithm skillfully identifies the best action strategy, including the trajectory of the UAV, the pairing of edge servers, and the pairing of target users within the context of the mobile IRS-assisted UAV-enabled MEC network. To train the network, the algorithm utilizes tuples {s t ,a t ,r t+1 ,s t+1}.

[0070] In D3QN, the Q value is calculated as follows:

[0071]

[0072] Where η represents the parameters associated with the connection layer, ξ corresponds to the parameters governing the value function, ω involves the parameters regulating the advantage function, and |A| represents the dimension. η can be updated to obtain the minimum loss function, where the loss function can be expressed as

[0073]

[0074] DDPG has several noteworthy advantages. First, it places a major emphasis on stability and convergence, employing deterministic policies that simplify the learning process and improve stability when compared to stochastic policies. This property helps accelerate convergence and promotes more predictable training dynamics. DDPG demonstrates excellent sample efficiency by leveraging an experience replay buffer (using previous experience for training). This mechanism improves sample efficiency by reducing the interdependencies between consecutive data points and lowering the risk of overfitting. DDPG excels in handling continuous action spaces and is particularly well suited for tasks such as robotic control and autonomous driving. It utilizes an actor-critic architecture to approximate deterministic policies (actors) and value functions (critics), effectively adapting to continuous action spaces. DDPG is known for its off-policy learning capability, which enables efficient learning from previous experience. This property ensures stability and data efficiency, allowing the agent to explore in an exploratory manner while still leveraging previously acquired knowledge. DDPG leverages the power of deep neural networks to skillfully approximate policy and value functions. Therefore, these exploits enable DDPG to adeptly handle high-dimensional state spaces, making it well suited for the mobile IRS-assisted UAV-enabled MEC network in this paper to optimize the offloading rate and reflection coefficient of IRS.

[0075] DDPG usually consists of two main parts. The actor network is responsible for approximating the deterministic policy. It takes the current state as input and outputs the corresponding action. This network usually includes multiple fully connected layers with activation function ReLU and a final layer with an activation function such as tanh or sigmoid to map the output to the action space. The critic network is used to estimate the value function, which provides feedback on the quality of the selected action. It takes the current state and action as input and outputs a Q value. Similar to the actor network, the critic network usually consists of fully connected layers with ReLU activation. The parameters ρ of the actuator are updated with the gradient, which can be expressed as

[0076]

[0077] ρ - represents the parameters of the target actor. The parameters of the critic are denoted by θ, and the parameters of the target critic are denoted by θ - Therefore, the review result can be expressed as

[0078]

[0079] The algorithm is effectively used to manage high-dimensional system states and action spaces within the context of a mobile IRS-assisted UAV-enabled MEC network. In addition, it has the ability to jointly optimize UAV trajectories, edge server pairings, target user pairings, data offloading rates, and IRS reflective element configurations. During the training phase, experiences of size L are randomly drawn from the experience buffer and a loss function is calculated to determine the loss value. The provided loss function is then used to calculate the inverse gradient, thereby facilitating the update of model parameters.

[0080] The effects of the present invention are further described in detail below in conjunction with simulation experiments, specifically including:

[0081] 1. Simulation system parameters

[0082] According to the configuration system parameters, the edge computing servers are located at (-150, 0, 0), (60, 20, 0), and (180, 100, 0), while the user locations are (100, 90, 0), (-120, -30, 0), (-125, -30, 0), (100, 80, 0), (115, 30, 0), (120, 30, 0), and (125, 30, 0), where the user data sizes are 20MB, 15MB, 10MB, 15MB, and 10MB, respectively. The drone maintains a fixed cruising altitude H0 = 40, and the mobile communication platform is precisely constructed along the x and y axis planes with an accuracy of 1 meter. Therefore, the drone's stopping points are established at intervals of three meters along the x and y axes. The deep network parameters of the intelligent drone planning scheme using D3QN-DDPG are consistent. The experience replay buffer size is configured as L=20000, and the discount factor for future reward evaluation is set to γ=0.99. For the convolutional networks in D3 QN, each network consists of four hidden layers with 512 neurons in each layer. Both the advantage network and the value network consist of four hidden layers with 512 neurons in each layer. The learning rate of the deep network in TD3 is 0.0003, and an Adam-based optimizer is used. In DDPG, both the actor network and the critic network consist of three hidden layers with 256 neurons in each layer. Using the Adam-based optimizer, the learning rates of the actor network and the critic network are set to 0.0002 and 0.0003, respectively. The details are as follows:

[0083]

[0084]

[0085] 2. Simulation content

[0086] Figure 3The convergence of the proposed intelligent solution using D3QN-DDPG is demonstrated in Figure 2. It is obvious that the proposed intelligent solution can reach convergence in a relatively short time. This is attributed to the effective hybrid space exploration capability of the hybrid algorithm based on D3QN-DDPG. The exploration capability of the algorithm enables it to efficiently explore the mobile IRS-assisted UAV-enabled EMC network and guide the intelligent agent to quickly solve complex problems. In addition, as the epochs increase, the proposed algorithm can improve the system cost and achieve a balance between system latency and energy consumption.

[0087] Two benchmark schemes are introduced for comparison purposes, Figure 4 The time cost is expressed as a function of the variation of available bandwidth accessed by the relay UAVs. The proposed algorithm consistently exhibits higher energy efficiency compared to other baseline schemes. This remarkable performance is attributed to the effectiveness of the proposed intelligent approach, which is based on a combination of D3QN (based on the Bellman equation) and a policy gradient (DDPG)-based algorithm. The proposed scheme excels in effectively optimizing various parameters, including jointly optimizing the trajectory of the UAVs, the pairing of edge servers, the pairing of target users, the data load reduction ratio, and the reflection elements of the IRS. It is also noteworthy that the proposed algorithm exhibits commendable time efficiency even when the available bandwidth is limited. This highlights the robustness and versatility of the proposed approach in achieving effective resource allocation and task optimization in relay UAV communication systems.

[0088] Figure 5 The energy consumption associated with varying available bandwidth is shown. Figure 5 The superior energy efficiency of the proposed smart scheme compared to other baseline schemes within the available bandwidth is clearly highlighted in the figure. This consistently superior performance emphasizes the effectiveness and versatility of the proposed algorithm. Moreover, it is noteworthy that the smart scheme exploits the synergy between D3QN and DDPG and shows clear advantages over the other baseline schemes. This further proves the credibility of the proposed scheme as an efficient solution for mobile IRS-assisted UAV-enabled EMC networks.

[0089] Figure 6 The improvement of system cost with respect to different available bandwidths for relay UAV access is shown when the base scheme is set to “random scheme”. In the random scheme, the parameters are randomly generated. Figure 6 It can be seen that the solution of using D3QN-SAC proposed in the present invention can support complex mobile IRS-assisted EMC networks, significantly improve network performance and reduce system costs.

Claims

1. A method for enhancing secure communication and mobile edge computing by using an intelligent reflective surface on a drone, characterized in that: The method comprises the following steps: Step 1: Establish a smart reflective surface-assisted MEC channel model There are delay UAVs, IRS-UAVs, multiple user devices, and multiple edge servers in the EMC network. The relay UAV is equipped with a limited data storage of size L and provides data offloading services and task computing services. The IRS-UAV is equipped with an IRS to adaptively improve task uploading and task offloading, where the number of IRS reflective elements can be represented by N. The UAV hovers at a consistent altitude and can stop at a specific stop point, which can be represented by S = {1,...,s,...}. Therefore, the position of the relay UAV at the stop point s is represented by {x r,s ,y r,s ,h r }, the position of IRS-UAV at the stop point s is represented by {x i,s ,y i,s ,h i The set of edge computing servers can be represented by E = {1, ..., e, ..., E}, and the position k of edge computing server e can be represented by {x e ,y e ,0}. The set of user devices can be represented by D = {1,...,d,...,D}. The position of user device d is represented by [x d ,y d ,0] is represented by the relay UAV and IRS-UAV trajectories, respectively. r = {q r,t } t∈T and Q i = {q i,t } i∈T Indicates, where Τ = {1, ..., t, ..., T} represents the time of each stop point, q i,t ∈S and q r,t ∈S. Therefore, the distance from the relay UAV and IRS-UAV to the user device d at time t can be recursively given by: The distances from the relay UAV and IRS-UAV to the computing server e at time t can be expressed as: The distance from the relay UAV to the IRS-UAV at time t can be expressed as: Similar to mmWave communications, the link from the UAV to the user device and computing server is considered to be line-of-sight communication. Therefore, the channel gain from the user device d to the relay UAV and IRS relay at time t can be expressed as Where μ represents the path loss exponent. The channel gain from IRS-UAV to relay UAV at time t is expressed as Let φ i,t The diagonal matrix representing the IRS elements at time t can be expressed as where ε n,t and φ n,t denote the amplitude and phase shift of the IRS reflector element n, respectively. The upload data rate at the user device d when the UAV is at time t can be expressed as Where B represents bandwidth, P r represents the transmit power of the relay UAV, and represents the noise power at user device d at time t. The data offloading transmission rate from the relay UAV to the edge server e at time t can be expressed as Step 2: Build an EMC network delay model The flight time consumption of the drone is expressed as Where V represents the flight speed of the UAV, and max(x,y) represents the maximum value between x and y. Let L d represents the size of the transmission data of user equipment d, and δ d represents the data offloading rate of user device d, L d ≤M,δ d Let β t,d indicates whether the relay UAV is addressing the data of user device d at time t. Therefore, the upload and offload delays can be expressed as: where ρ t Indicates whether the relay UAV performs the upload and unload tasks at time t. If the relay UAV uploads the task at time t, ρ t =1; otherwise, ρ t =0. t,e indicates whether the data in the buffer is offloaded to the edge computing server e at time t. If the relay UAV is paired with the computing server e to offload the data of the user device d, then σ t,e = l; otherwise, σ t,e =0. The computation delay is expressed as follows: Among them, f r and f e represents the CPU cycles required for one bit at the drone side and the edge server, respectively, and F r and F e Denote the computing power of the drone and edge computing server respectively. The total delay includes flight delay, unloading and uploading delay, and computing delay, which can be expressed as follows: T(Q r ,Q i )=T f +T l +T e 。 Step 3: Build an energy consumption model The flight energy cost of the drone is expressed as: E f =T f P f , Where P f represents the flight power consumption of the relay UAV and IRS-UAV. The hovering energy cost of the relay UAV and IRS-UAV can be written as: E h =P h (T l +T e ), Where P h represents the hovering power consumption of relay UAV and IRS-UAV. For a mobile IRS-assisted IRS-enabled UAV, the total energy consumption of the MEC network can be expressed as E=E f +E h 。 2. Based on a method for enhancing secure communication and mobile edge computing by using an intelligent reflective surface on a drone, step 1 includes: Propose a DRL-based mobile IRS-assisted UAV edge computing network intelligent strategy Step 1: Propose intelligent optimization schemes with real-time computing performance for IRS-UAV and Delayed UAV Step 1-1: Solve the Markov decision process (MDP) problem Model-free reinforcement learning methods are used to address the decision-making challenges proposed in this study. The task of minimizing the system cost is conceptualized as a Markov decision process. The dynamics of the edge computing network for UAV-assisted mobility characterizes the environment, while the relay UAVs and IRS-UAV control center are considered as intelligent agents. When these agents operate in this environment, the MDP problem contains state space, action space, transition probability, and reward function. Further insights into the details of the MDP problem are elaborated below. Calculate the previously selected action, the obtained time consumption and the obtained energy consumption. At time t, the state S t ∈S can be expressed as s t ={a t-1 ,T t ,E t }, where a t-1 represents the previously selected action at time t-1, T t represents the achievable transmission delay, and E t Indicates the achievable system consumption. The actions of the agent are mapped to the action space, denoted by A. The agent actions at e include the trajectory of the IRS-UAV, the trajectory of the relay UAV, the pairing of the edge server, the pairing of the target user, and the data offloading ratio. t The action at ∈A can be expressed as: a t {q} r,t ,q i,t ,{φ i,t },s e,t ,b d,t ,d t }, Among them, q r,t represents the repeater trajectory, q i,t represents the trajectory of IRS, σ t,e represents the pairing of edge servers, β t,d represents the pairing of target user n, and δ t represents the data offloading rate. The transition probability is given by L(s t+1 |s t ,a t ) represents the agent from state s t ∈S visits the new state s t+1 ∈S, decision a t ∈A. The reward consists of the total system cost, time consumption penalty and energy consumption penalty, which can be expressed as r=-C+v1p1+v e p e , where v1 and p e Represent the weights of time consumption penalty and energy consumption penalty respectively. l is the time consumption penalty, which can be expressed as p t =max{0,T max -T}, Where T max Indicates the maximum tolerable delay. e is the penalty for energy consumption, expressed as follows: p e =max{0,E max -AND}, Where E max Represents the maximum tolerable energy cost. The cumulative reward starting at time t is expressed as: Where γ∈(0,1] represents the future impact factor, which is used to evaluate future rewards. The Bellman equation can evaluate the Q-value reward, which is expressed as: The goal of this invention is to learn the optimal decision π * , where the optimal Bellman equation is expressed as 3. Based on a method for enhancing secure communication and mobile edge computing by using a drone equipped with an intelligent reflective surface, step 2 includes: Propose an intelligent optimization solution based on D3QN-DDPG The intelligent optimization scheme adopts D3QN-DDPG high-dimensional, large-scale reinforcement learning tasks. As a result, the D3QN algorithm skillfully identifies the best action strategy, including the trajectory of the UAV, the pairing of edge servers, and the pairing of target users within the context of the mobile IRS-assisted UAV-enabled MEC network. To train the network, the algorithm utilizes tuples {s t ,a t ,r t+1 ,s t+1 }. In D3QN, the Q value is calculated as follows: Where η represents the parameters associated with the connection layer, ξ corresponds to the parameters governing the value function, ω involves the parameters regulating the advantage function, and |A| represents the dimension. η can be updated to obtain the minimum loss function, where the loss function can be expressed as DDPG has several noteworthy advantages. First, it places a strong emphasis on stability and convergence, using deterministic policies that simplify the learning process and improve stability when compared to stochastic policies. This property helps accelerate convergence and promotes more predictable training dynamics. DDPG demonstrates excellent sample efficiency by leveraging an experience replay buffer (using previous experience for training). This mechanism improves sample efficiency by reducing the interdependence between consecutive data points and lowering the risk of overfitting. DDPG excels in handling continuous action spaces and is particularly well suited for tasks such as robotic control and autonomous driving. It leverages an actor-critic architecture to approximate deterministic policies (actors) and value functions (critics), effectively adapting to continuous action spaces. DDPG is known for its off-policy learning capabilities, enabling efficient learning from previous experience. This property ensures stability and data efficiency, allowing the agent to explore in an exploratory manner while still leveraging previously acquired knowledge. DDPG leverages the power of deep neural networks to skillfully approximate policy functions and value functions. As a result, these exploits enable DDPG to skillfully handle high-dimensional state spaces, making it well suited for the mobile IRS-assisted UAV-enabled MEC network in this paper to optimize the offloading rate and reflection coefficient of the IRS. DDPG usually consists of two main parts. The actor network is responsible for approximating the deterministic policy. It takes the current state as input and outputs the corresponding action. This network usually includes multiple fully connected layers with activation function ReLU and a final layer with an activation function such as tanh or sigmoid to map the output to the action space. The critic network is used to estimate the value function, which provides feedback on the quality of the selected action. It takes the current state and action as input and outputs a Q value. Similar to the actor network, the critic network usually consists of fully connected layers with ReLU activation. The parameters ρ of the actuator are updated with the gradient, which can be expressed as: ρ - represents the parameters of the target actor. The parameters of the critic are denoted by θ, and the parameters of the target critic are denoted by θ - Therefore, the review result can be expressed as: The algorithm is effectively used to manage high-dimensional system states and action spaces within the context of a mobile IRS-assisted UAV-enabled MEC network. In addition, it has the ability to jointly optimize UAV trajectories, edge server pairings, target user pairings, data offloading rates, and IRS reflective element configurations. During the training phase, experiences of size L are randomly drawn from the experience buffer and a loss function is calculated to determine the loss value. The provided loss function is then used to calculate the inverse gradient, thereby facilitating the update of model parameters.

Images