A resource allocation optimization method for multi-IRS UAV inter-sensing and computing integrated system

Abstract

The present invention discloses a resource configuration optimization method for a multi-IRS UAV inter-sensing and computing integrated system, which improves the system's safety rate by utilizing multi-IRS-assisted perception beams and communication beams and reducing the signal-to-noise ratio (SINR) of the eavesdropper through artificial noise. The present invention proposes a hybrid SAC intelligent optimization algorithm based on deep reinforcement learning. The algorithm uses a discrete SAC algorithm to jointly optimize discrete actions such as the deployment points, task offloading ratios, and artificial noise power of D-UAV and I-UAV, and uses a continuous SAC algorithm to optimize continuous actions such as IRS phase shift, with the goal of optimizing the system's safety rate. Simulation results show that the intelligent optimization scheme proposed in the present invention is superior to other comparison schemes under different transmission powers and different numbers of IRS arrays.

Description

Technical Field

[0001] The present invention belongs to the technical field of wireless communication and unmanned aerial vehicle network, and in particular relates to a resource configuration optimization method for a multi-IRS unmanned aerial vehicle inter-sensory computing integrated system. Background Art

[0002] In recent years, the development prospects of drone edge computing networks are broad, and they have become an important solution for data processing and communication tasks. With their excellent flexibility and rapid deployment capabilities, drones have gradually become a key component to enhance traditional wireless communication infrastructure. They have shown wide application potential in fields such as intelligent transportation, environmental monitoring, and disaster response. Drones have the advantages of flexible deployment and rapid response, which enable them to play an important role in a variety of IoT scenarios. For example, in environmental monitoring, drones can collect and analyze data in real time and provide efficient monitoring methods; in traffic monitoring, drones can realize dynamic observation and management of traffic flow; and in the field of aerial imaging, they can obtain high-resolution image data to provide support for urban planning and resource management. Especially in the air-to-ground / ground-to-air channels between drones and ground users, the dominant characteristics of line-of-sight (LoS) links greatly improve the efficiency of wireless communication. This direct line-of-sight propagation mode reduces signal attenuation and improves communication quality. In relevant research literature, drones are usually defined as air users or multiple air communication platforms, including mobile base stations (BS) and air relays, showing their flexible role in network architecture. Despite the many advantages of drones, their lack of onboard computing power and recorded energy prevents them from deploying large-scale intensive computing tasks. However, edge computing networks will promote computing applications and development by offloading computing tasks. In order to improve the security of drone edge computing systems, communication and perception integration (ISAC) technology has been introduced. ISAC is expected to achieve higher spectrum and energy efficiency, but most importantly, they will promote a new integration paradigm that will achieve mutual benefit from a co-design perspective, that is, perception and communication functions can assist each other. Thanks to these two advantages, the application of ISAC has expanded to many emerging fields, including smart manufacturing, indoor services such as human activity recognition, and environmental monitoring, vehicle-mounted networks, etc. ISAC provides drones with stronger anti-interference capabilities and higher data security by combining information transmission with environmental perception. However, the introduction of ISAC does not effectively solve the problem of line-of-sight occlusion, which may lead to signal attenuation and unstable communication. Summary of the invention

[0003] In order to solve the above problems, the present invention discloses a resource configuration optimization method for a multi-IRS UAV inter-sensing and computing integrated system. By utilizing multiple intelligent reflecting surfaces (IRS) to assist in perception and communication beams and combining artificial noise technology, the arrangement points, unloading ratio, artificial noise power and IRS phase shift of data UAVs (D-UAVs) and information UAVs (I-UAVs) are jointly optimized to maximize the system safety rate ( ). This solution can effectively improve the security and anti-interference ability of the drone network, and shows superior performance under different transmission powers and numbers of IRS arrays, and has better optimization effects than other solutions.

[0004] To achieve the above object, the technical solution of the present invention is as follows:

[0005] A resource configuration optimization method for a multi-IRS UAV intersensory computing integrated system includes the following steps:

[0006] Step 1: Establish a multi-intelligent reflective surface enhanced synaesthesia integrated drone edge computing system model;

[0007] Step 1-1: System model construction: The system includes two UAVs: I-UAV and D-UAV; D-UAV is responsible for collecting ground data, and I-UAV is equipped with IRS to assist D-UAV in data offloading and BS-MEC in communication and perception; define multiple decision points DPs of I-UAV and D-UAV, represented by vectors and The initial positions are and ;

[0008] Step 1-2: Establish flight delay model: define the time slots of I-UAV and D-UAV Flight delays in and ; Get the formula and ;in is the speed of the drone, To limit the flight time of the drone and ensure that it can reach the designated decision point on time; and Represent the decision points of I-UAV and D-UAV in the next time slot, and They represent the decision points of I-UAV and D-UAV in the current time slot respectively;

[0009] Step 1-3: Establish a waiting time model: define the waiting time as the flight time difference between the two drones ; By setting the flag variable , determine which drone reaches the decision point first ;

[0010] Step 1-4: Establish energy consumption model: The energy consumption model of I-UAV and D-UAV includes flight energy consumption and hovering energy consumption. , the energy constraint is , ;in, , They represent the flight power and hovering power of the I-UAV respectively; , They represent the flight power and hovering power of the D-UAV respectively; , They represent the maximum flight and hovering energy consumption of I-UAV and D-UAV, respectively;

[0011] Step 1-5: Establish signal model: Add artificial noise AN in the physical layer to enhance security; the signal model of BS-MEC is ;in, is the beamforming matrix of BS-MEC, is the transmission signal, It is artificial noise.

[0012] Step 2: Establish a channel model for the synaesthesia integrated UAV edge computing system enhanced by multiple intelligent reflective surfaces;

[0013] Step 2-1: System parameter initialization: define the coordinates of different nodes in the system, including the locations of BS-MEC, EAV, users and drones; the location of BS-MEC , the location of EAV , user location ; The positions of I-UAV and D-UAV in each time slot are and ,in and is the fixed drone flight altitude;

[0014] Step 2-2: Define the distance calculation model: The distance between BS-MEC and I-UAV is ;

[0015] ①I-UAV and users The distance is ;

[0016] ②The distance between BS-MEC and EAV is ;

[0017] ③The distance between D-UAV and I-UAV is ;

[0018] ④D-UAV and users The distance is ;

[0019] ⑤The distance between D-UAV and EAV is ;

[0020] in, , is the abscissa value and ordinate value of IRS; , are the horizontal and vertical coordinate values ​​of BS; , It is the user's horizontal coordinate value and vertical coordinate value; , is the horizontal and vertical coordinate values ​​of EAV; , is the horizontal coordinate value and vertical coordinate value of D_UAV;

[0021] Step 2-3: Define the channel gain model: Since line-of-sight LOS communication is used, the channel gain model follows the free space path loss model; the channel gain from BS-MEC to I-UAV is ;I-UAV and User The channel gain is ; The channel gain between D-UAV and I-UAV is ;D-UAV and User The channel gain is ; The channel gain between D-UAV and EAV is ;

[0022] Step 2-4: Define the IRS reflection model: Multiple IRSs in the system are used to assist communication and perception. The reflection matrix of each IRS is expressed as ;in, For the Reflection unit in time slot The reflection coefficient ,and is the reflection coefficient of the signal.

[0023] Step 3: Establish a partial unloading model and a computing model for the synaesthesia integrated UAV edge computing system enhanced by multiple intelligent reflective surfaces;

[0024] Step 3-1: System parameter initialization: Initialize the parameters related to offloading and calculation, including the amount of data collected by the D-UAV ; Partial offloading ratio of data ,in ; Channel bandwidth ; Transmission power of D-UAV ; Hovering power of I-UAV and D-UAV and ; Computational power of D-UAV ; Computing power of BS-MEC ; Computing power of D-UAV ; The number of CPU cycles required to execute each bit of data ; Noise power ; Channel gain matrix and ; Reflection matrix of IRS ;

[0025] Step 3-2: Establish the offload transmission rate calculation model: The transmission rate from D-UAV to BS-MEC is

[0026] ;in, and is the channel gain matrix from D-UAV to I-UAV and I-UAV to BS-MEC, through the reflection matrix of IRS To achieve data transmission;

[0027] Step 3-3: Establish an offloading delay calculation model: Based on the transmission rate, the delay of partial offloading is calculated as ; This formula means that The delay of partially unloading data in a time slot is determined by the amount of data unloaded. and transfer rate Determine; Among them, the calculation delay of BS-MEC is expressed as ;in, is the number of CPU cycles per bit of data, is the computing power of BS-MEC; the local computing latency of D-UAV is ;in, is the computing power of D-UAV, is the amount of data processed on the D-UAV;

[0028] Step 3-4: Establish energy consumption calculation model: The calculation energy consumption of D-UAV is ; , are the power of I-UAV and D-UAV respectively; the computing energy consumption of I-UAV is ; Here we assume that the I-UAV only hovers and does not perform computing tasks, so its energy consumption is only determined by the hovering time Decide.

[0029] Step 4: Establish the energy model of the synaesthesia integrated drone edge computing system enhanced by multiple intelligent reflective surfaces;

[0030] Step 4-1: Define relevant parameters: the size of data collected by the D-UAV ;The number of CPU cycles required to execute each bit of the task ; Ratio of uninstalled data ; Computing power of BS-MEC ; Computing power of D-UAV ; Power during D-UAV calculation ; Hovering power of D-UAV and I-UAV and ;

[0031] Step 4-2: Establish a time calculation model: In which, the partially unloaded data is calculated on the BS-MEC, and the BS-MEC calculation time is , the unloaded data is calculated on D-UAV, and the calculation time is ;

[0032] Step 4-3: Establish energy consumption model: The energy consumption of D-UAV when calculating the remaining data is , during the calculation process, the hovering energy consumption of I-UAV is .

[0033] Step 5: Establish an eavesdropping model of the synaesthesia-integrated UAV edge computing system enhanced by multiple intelligent reflective surfaces;

[0034] Step 5-1: D-UAV or BS-MEC to the user Data transfer rate :user Signal-to-noise ratio ;user Channel gain ; Eavesdroppers on users The achievable transfer rate ; The signal-to-noise ratio of the eavesdropper ; Matrix containing channels, phase adjustments, and other system parameters ;user Safe transmission rate ; Noise power ;

[0035] Step 5-2: Establish user data transmission rate model: Define the data rate from D-UAV or BS-MEC to user The rate at which data is transmitted; where is the bandwidth, is the user's signal-to-noise ratio SINR;

[0036] Step 5-3: Establish a user signal-to-noise ratio (SINR) calculation model: User SINR Expressed as ;in, Contains channel information of I-UAV and D-UAV, Represents the signal sent by the base station to the kth user;

[0037] Step 5-4: Establish the eavesdropper rate and signal-to-noise ratio calculation model: define the eavesdropper’s impact on the user The achievable transfer rate ;in, is the eavesdropper's signal-to-noise ratio SINR; Expressed as ;in, Contains channel, phase adjustment and other system matrices; , represent the beamforming matrix of BS and the transpose of the beamforming matrix of BS, respectively. and They represent the beamforming matrix transmitted by the BS to the i-th user and the transpose of the beamforming matrix transmitted by the BS to the i-th user, respectively. is the covariance matrix of artificial noise;

[0038] Step 5-5: Build a secure transmission rate model: define users The achievable safe transmission rate is ; This formula calculates the user The safe rate of Indicates taking a positive value, that is, when the difference is negative, it is taken as 0.

[0039] Step 6: Propose resource allocation and trajectory optimization problems and constraints to maximize the safety rate for all users;

[0040] Step 7: In this Markov decision process (MDP) framework, the multi-IRS-assisted secure ISAC-MEC system constitutes the dynamic environment, and the control center of the UAV acts as an intelligent agent. In addition to the environment and the intelligent agent, the MDP problem also includes various components such as state space, action space, state transition probability, and reward function. These elements are crucial to modeling and solving the MDP problem, and their specific definitions are as follows:

[0041] set up Represents the state space of the network: at time step When the network state includes multiple components , such as channel information, selected actions, and rewards obtained; therefore, the state is defined as ;in, and Respectively represent the time step The data collection channel status of I-UAV and D-UAV, Indicates that at time step Select the action, Indicates that at time step Rewards received;

[0042] set up Represents the action space of the network: at time step When Including the position of the drone, the reflection coefficient of the IRS; therefore, the action is defined as ;in, Indicates the deployment location of the I-UAV, Indicates the deployment location of the D-UAV, Represents the time step The reflection coefficient of IRS at ;

[0043] set up Represents the state transition probability of the environment, indicating that in state Next action When probability;

[0044] The reward function is a measure used by an intelligent system to evaluate the effectiveness of its behavior. The reward function provides a quantitative assessment of the performance of an intelligent agent by mapping its current state and action space to the reward function. The resulting reward value reflects the quality of the agent's action selection strategy and serves as a criterion for evaluating the agent's decision-making ability. ;in, is a given constant, is a penalty item, is a safe rate.

[0045] Step 8: Based on the reinforcement learning modeling in step 7, the present invention proposes an intelligent resource allocation algorithm based on Hybrid SAC to jointly optimize the deployment points, unloading ratio, artificial noise power and IRS phase shift of data UAV (D-UAV) and information UAV (I-UAV) to maximize the system safety rate (RSEC).

[0046] In the SAC algorithm, there are two action value functions Q (parameters are and ) and a policy function (Parameters are ) modeling; based on the idea of ​​Double DQN, SAC uses two Q networks, but each time a Q network is used, a network with a small Q value is selected to alleviate the problem of overestimation of the Q value; the loss function of any function Q is:

[0047] ;

[0048] Among them, R is the data collected by the strategy in the past, because SAC is an offline strategy algorithm; is the expected solution, is the reward obtained in the tth step, is a regularization coefficient; in order to make the training more stable, the target network Q is used here, and there are also two target Q networks , which corresponds one-to-one to the two Q networks; the update method of the target Q network in SAC is the same as that in DDPG;

[0049] Strategy The loss function of is obtained from the KL divergence and is simplified to

[0050] ,

[0051] in, Expressing hope, Representation strategy exist Taken under the state action, Represents its value network. The above process is understood as maximizing the function V, because ;

[0052] For the environment of continuous action space, the strategy of SAC algorithm outputs the mean and standard deviation of Gaussian distribution, but the process of sampling actions according to Gaussian distribution is not differentiable; therefore, reparameterization technique is needed; the reparameterization method is to start from a unit Gaussian distribution , then multiply the sampled value by the standard deviation and add the mean; this is considered to be sampling from the policy Gaussian distribution, and this is differentiable for the policy function; it is expressed as ,in is a noise random variable; taking into account the two functions Q, rewrite the loss function of the strategy:

[0053] ;

[0054] in, It is to seek expectations, It's a strategy. represents minimizing its value function, yes and A mapping relationship between is a noise random variable. For the SAC algorithm that processes discrete actions, the output of the policy network is modified to a softmax distribution on the discrete action space; the value network directly receives the state and the distribution of the discrete action space as input; the policy network outputs a discrete action distribution, so during the learning process of the value network, there is no need to calculate the next action. Instead of sampling, the value of the next state is obtained directly through probability calculation; similarly, the loss function of In the calculation, there is no need to sample the action;

[0055] The agent interacts with the system to gain training experience;

[0056] Step 8-1: The agent optimizes the deployment points, unloading ratio, artificial noise power and IRS phase shift of the data drone D-UAV and the information drone I-UAV through Hybrid SAC, and inputs them to enter the new system state, and obtains the current reward value through the reward function; in each iteration, the agent can store the new and old system states, action selections, and reward values ​​obtained by interacting with the environment into the experience pool to provide training samples for network training;

[0057] Step 8-2: Algorithm parameter update: When the agent interacts with the environment for the set number of times, the agent learns with the goal of minimizing the loss function by replaying the experience stored in the experience pool, and updates the network parameters through reverse gradient;

[0058] When the agent interacts with the environment for a set number of times, the agent randomly takes out a The ancestor , taking the minimum loss function as the goal, calculate the reverse gradient of the loss function and update the SAC network parameters;

[0059] If the number of training times does not reach the set number, return to the step of interaction between the agent and the system;

[0060] Step 8-3: After the algorithm converges, the network parameters are saved locally.

[0061] Step 9: Use the SAC-based intelligent resource allocation method to jointly optimize the deployment points of D-UAV and I-UAV, task offloading ratio, artificial noise power, and phase shift of multiple IRSs to maximize the system's safe rate ( ).

[0062] The beneficial effects of the present invention are:

[0063] The present invention proposes a Hybrid SAC algorithm, which jointly optimizes the deployment points, unloading ratio, artificial noise power and IRS phase shift of the data UAV (D-UAV) and information UAV (I-UAV) to maximize the system safety rate ( ). By applying the Hybrid SAC algorithm, the system proposed in the present invention exhibits a significantly higher probability of mission success, verifying its effectiveness in enhancing overall performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a system block diagram of the present invention.

[0065] Figure 2 This is a graph showing the convergence of the resource allocation algorithm in the present invention.

[0066] Figure 3 Trajectory map selected for the drone of the present invention.

[0067] Figure 4 It is the safe speed of the UAV of the present invention in the selected trajectory and random trajectory.

[0068] Figure 5 It is the safe speed of the UAV of the present invention under different numbers of IRS array elements. DETAILED DESCRIPTION

[0069] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0070] like Figure 1 As shown, the present invention provides a resource configuration optimization method for a multi-IRS UAV intersensory computing integrated system, the method comprising the following steps:

[0071] Step 1: Establish a multi-intelligent reflective surface enhanced synaesthesia integrated drone edge computing system model;

[0072] Step 1-1: System model construction: The system includes two UAVs: I-UAV and D-UAV; D-UAV is responsible for collecting ground data, and I-UAV is equipped with IRS to assist D-UAV in data offloading and BS-MEC in communication and perception; define multiple decision points DPs of I-UAV and D-UAV, represented by vectors and The initial positions are and ;

[0073] Step 1-2: Establish flight delay model: define the time slots of I-UAV and D-UAV Flight delays in and ; Get the formula and ;in is the speed of the drone, To limit the flight time of the drone and ensure that it can reach the designated decision point on time; and Represent the decision points of I-UAV and D-UAV in the next time slot, and They represent the decision points of I-UAV and D-UAV in the current time slot respectively;

[0074] Step 1-3: Establish a waiting time model: define the waiting time as the flight time difference between the two drones ; By setting the flag variable , determine which drone reaches the decision point first ;

[0075] Step 1-4: Establish energy consumption model: The energy consumption model of I-UAV and D-UAV includes flight energy consumption and hovering energy consumption. , the energy constraint is , ;in, , They represent the flight power and hovering power of the I-UAV respectively; , They represent the flight power and hovering power of the D-UAV respectively; , They represent the maximum flight and hovering energy consumption of I-UAV and D-UAV, respectively;

[0076] Step 1-5: Establish signal model: Add artificial noise AN in the physical layer to enhance security; the signal model of BS-MEC is ;in, is the beamforming matrix of BS-MEC, is the transmission signal, It is artificial noise.

[0077] Step 2: Establish a channel model for the synaesthesia integrated UAV edge computing system enhanced by multiple intelligent reflective surfaces;

[0078] Step 2-1: System parameter initialization: define the coordinates of different nodes in the system, including the locations of BS-MEC, EAV, users and drones; the location of BS-MEC , the location of EAV , user location ; The positions of I-UAV and D-UAV in each time slot are and ,in and is the fixed drone flight altitude;

[0079] Step 2-2: Define the distance calculation model: The distance between BS-MEC and I-UAV is ;

[0080] ①I-UAV and users The distance is ;

[0081] ②The distance between BS-MEC and EAV is ;

[0082] ③The distance between D-UAV and I-UAV is ;

[0083] ④D-UAV and users The distance is ;

[0084] ⑤The distance between D-UAV and EAV is ;

[0085] in, , is the abscissa value and ordinate value of IRS; , are the horizontal and vertical coordinate values ​​of BS; , It is the user's horizontal coordinate value and vertical coordinate value; , is the horizontal and vertical coordinate values ​​of EAV; , is the horizontal coordinate value and vertical coordinate value of D_UAV;

[0086] Step 2-3: Define the channel gain model: Since line-of-sight LOS communication is used, the channel gain model follows the free space path loss model; the channel gain from BS-MEC to I-UAV is ;I-UAV and User The channel gain is ; The channel gain between D-UAV and I-UAV is ;D-UAV and User The channel gain is ; The channel gain between D-UAV and EAV is ;

[0087] Step 2-4: Define the IRS reflection model: Multiple IRSs in the system are used to assist communication and perception. The reflection matrix of each IRS is expressed as ;in, For the Reflection unit in time slot The reflection coefficient ,and is the reflection coefficient of the signal.

[0088] Step 3: Establish a partial unloading model and a computing model for the synaesthesia integrated UAV edge computing system enhanced by multiple intelligent reflective surfaces;

[0089] Step 3-1: System parameter initialization: Initialize the parameters related to offloading and calculation, including the amount of data collected by the D-UAV ; Partial offloading ratio of data ,in ; Channel bandwidth ; Transmission power of D-UAV ; Hovering power of I-UAV and D-UAV and ; Computational power of D-UAV ; Computing power of BS-MEC ; Computing power of D-UAV ; The number of CPU cycles required to execute each bit of data ; Noise power ; Channel gain matrix and ; Reflection matrix of IRS ;

[0090] Step 3-2: Establish the offload transmission rate calculation model: The transmission rate from D-UAV to BS-MEC is

[0091] ;in, and is the channel gain matrix from D-UAV to I-UAV and I-UAV to BS-MEC, through the reflection matrix of IRS To achieve data transmission;

[0092] Step 3-3: Establish an offloading delay calculation model: Based on the transmission rate, the delay of partial offloading is calculated as ; This formula means that The delay of partially unloading data in a time slot is determined by the amount of data unloaded. and transfer rate Determine; Among them, the calculation delay of BS-MEC is expressed as ;in, is the number of CPU cycles per bit of data, is the computing power of BS-MEC; the local computing latency of D-UAV is ;in, is the computing power of D-UAV, is the amount of data processed on the D-UAV;

[0093] Step 3-4: Establish energy consumption calculation model: The calculation energy consumption of D-UAV is ; , are the power of I-UAV and D-UAV respectively; the computing energy consumption of I-UAV is ; Here we assume that the I-UAV only hovers and does not perform computing tasks, so its energy consumption is only determined by the hovering time Decide.

[0094] Step 4: Establish the energy model of the synaesthesia integrated drone edge computing system enhanced by multiple intelligent reflective surfaces;

[0095] Step 4-1: Define relevant parameters: the size of data collected by the D-UAV ;The number of CPU cycles required to execute each bit of the task ; Ratio of uninstalled data ; Computing power of BS-MEC ; Computing power of D-UAV ; Power during D-UAV calculation ; Hovering power of D-UAV and I-UAV and ;

[0096] Step 4-2: Establish a time calculation model: In which, the partially unloaded data is calculated on the BS-MEC, and the BS-MEC calculation time is , the unloaded data is calculated on D-UAV, and the calculation time is ;

[0097] Step 4-3: Establish energy consumption model: The energy consumption of D-UAV when calculating the remaining data is , during the calculation process, the hovering energy consumption of I-UAV is .

[0098] Step 5: Establish an eavesdropping model of the synaesthesia-integrated UAV edge computing system enhanced by multiple intelligent reflective surfaces;

[0099] Step 5-1: D-UAV or BS-MEC to the user Data transfer rate :user Signal-to-noise ratio ;user Channel gain ; Eavesdroppers on users The achievable transfer rate ; The signal-to-noise ratio of the eavesdropper ; Matrix containing channels, phase adjustments, and other system parameters ;user Safe transmission rate ; Noise power ;

[0100] Step 5-2: Establish user data transmission rate model: Define the data rate from D-UAV or BS-MEC to user The rate at which data is transmitted; where is the bandwidth, is the user's signal-to-noise ratio SINR;

[0101] Step 5-3: Establish a user signal-to-noise ratio (SINR) calculation model: User SINR Expressed as ;in, Contains channel information of I-UAV and D-UAV; Represents the signal sent by the base station to the kth user;

[0102] Step 5-4: Establish the eavesdropper rate and signal-to-noise ratio calculation model: define the eavesdropper’s impact on the user The achievable transfer rate ;in, is the eavesdropper's signal-to-noise ratio SINR; Expressed as ;in, Contains channel, phase adjustment and other system matrices; , represent the beamforming matrix of BS and the transpose of the beamforming matrix of BS, respectively. and They represent the beamforming matrix transmitted by the BS to the i-th user and the transpose of the beamforming matrix transmitted by the BS to the i-th user, respectively. is the covariance matrix of artificial noise;

[0103] Step 5-5: Build a secure transmission rate model: define users The achievable safe transmission rate is ; This formula calculates the user The safe rate of Indicates taking a positive value, that is, when the difference is negative, it is taken as 0.

[0104] Step 6: Propose resource allocation and trajectory optimization problems and constraints to maximize the safety rate for all users;

[0105] Step 7: In this Markov decision process (MDP) framework, the multi-IRS-assisted secure ISAC-MEC system constitutes the dynamic environment, and the control center of the UAV acts as an intelligent agent. In addition to the environment and the intelligent agent, the MDP problem also includes various components such as state space, action space, state transition probability, and reward function. These elements are crucial to modeling and solving the MDP problem, and their specific definitions are as follows:

[0106] set up Represents the state space of the network: at time step When the network state includes multiple components , such as channel information, selected actions, and rewards obtained; therefore, the state is defined as ;in, and Respectively represent the time step The data collection channel status of I-UAV and D-UAV, Indicates that at time step Select the action, Indicates that at time step Rewards received;

[0107] set up Represents the action space of the network: at time step When Including the position of the drone, the reflection coefficient of the IRS; therefore, the action is defined as ;in, Indicates the deployment location of the I-UAV, Indicates the deployment location of the D-UAV, Represents the time step The reflection coefficient of IRS at ;

[0108] set up Represents the state transition probability of the environment, indicating that in state Next action When probability;

[0109] The reward function is a measure used by an intelligent system to evaluate the effectiveness of its behavior. The reward function provides a quantitative assessment of the performance of an intelligent agent by mapping its current state and action space to the reward function. The resulting reward value reflects the quality of the agent's action selection strategy and serves as a criterion for evaluating the agent's decision-making ability. ;in, is a given constant, is a penalty item, is a safe rate.

[0110] Step 8: Based on the reinforcement learning modeling in step 7, the present invention proposes an intelligent resource allocation algorithm based on Hybrid SAC to jointly optimize the deployment points, unloading ratio, artificial noise power and IRS phase shift of data UAV (D-UAV) and information UAV (I-UAV) to maximize the system safety rate (RSEC).

[0111] In the SAC algorithm, there are two action value functions Q (parameters are and ) and a policy function (Parameters are ) modeling; based on the idea of ​​Double DQN, SAC uses two Q networks, but each time a Q network is used, a network with a small Q value is selected to alleviate the problem of overestimation of the Q value; the loss function of any function Q is:

[0112] ;

[0113] Among them, R is the data collected by the strategy in the past, because SAC is an offline strategy algorithm; is the expected solution, is the reward obtained in the tth step, is a regularization coefficient; in order to make the training more stable, the target network Q is used here, and there are also two target Q networks , which corresponds one-to-one to the two Q networks; the update method of the target Q network in SAC is the same as that in DDPG;

[0114] Strategy The loss function of is obtained from the KL divergence and is simplified to

[0115] ,

[0116] in, Expressing hope, Representation strategy exist Taken under the state action, Represents its value network. The above process is understood as maximizing the function V, because ;

[0117] For the environment of continuous action space, the strategy of SAC algorithm outputs the mean and standard deviation of Gaussian distribution, but the process of sampling actions according to Gaussian distribution is not differentiable; therefore, reparameterization technique is needed; the reparameterization method is to start from a unit Gaussian distribution , then multiply the sampled value by the standard deviation and add the mean; this is considered to be sampling from the policy Gaussian distribution, and this is differentiable for the policy function; it is expressed as ,in is a noise random variable; taking into account the two functions Q, rewrite the loss function of the strategy:

[0118] ;

[0119] in, It is to seek expectations, It's a strategy. represents minimizing its value function, yes and A mapping relationship between is a noise random variable. For the SAC algorithm that processes discrete actions, the output of the policy network is modified to a softmax distribution on the discrete action space; the value network directly receives the state and the distribution of the discrete action space as input; the policy network outputs a discrete action distribution, so during the learning process of the value network, there is no need to calculate the next action. Instead of sampling, the value of the next state is obtained directly through probability calculation; similarly, the loss function of In the calculation, there is no need to sample the action;

[0120] The agent interacts with the system to gain training experience;

[0121] Step 8-1: The agent optimizes the deployment points, unloading ratio, artificial noise power and IRS phase shift of the data drone D-UAV and the information drone I-UAV through Hybrid SAC, and inputs them to enter the new system state, and obtains the current reward value through the reward function; in each iteration, the agent can store the new and old system states, action selections, and reward values ​​obtained by interacting with the environment into the experience pool to provide training samples for network training;

[0122] Step 8-2: Algorithm parameter update: When the agent interacts with the environment for the set number of times, the agent learns with the goal of minimizing the loss function by replaying the experience stored in the experience pool, and updates the network parameters through reverse gradient;

[0123] When the agent interacts with the environment for a set number of times, the agent randomly takes out a The ancestor , taking the minimum loss function as the goal, calculate the reverse gradient of the loss function and update the SAC network parameters;

[0124] If the number of training times does not reach the set number, return to the step of interaction between the agent and the system;

[0125] Step 8-3: After the algorithm converges, the network parameters are saved locally.

[0126] Step 9: Use the SAC-based intelligent resource allocation method to jointly optimize the deployment points of D-UAV and I-UAV, task offloading ratio, artificial noise power, and phase shift of multiple IRSs to maximize the system's safe rate ( ).

[0127] The effects of the present invention are further described in detail below in conjunction with simulation experiments, specifically including:

[0128] Simulation system parameters: The simulation scenario includes The BS-MEC of the location is located in Specifically, the present invention sets 10 stop points for D-UAV to be selected by the intelligent agent, and also sets 10 stop points for I-UAV to be selected by the intelligent agent. The maximum transmission power of D-UAV is set to 20dBm, and the spectrum bandwidth is The noise variance of 90BS-MEC is KHz is 0.01. In this section, the performance of the proposed hybrid SAC intelligent optimization scheme is evaluated and compared with the baseline scheme in a multi-IRS-assisted secure ISAC-MEC system.

[0129] Among them, the first benchmark scheme randomly selects the stop point locations of I-UAV and D-UAV, and the other settings are consistent with the optimization scheme, which is called "random selection". The second benchmark scheme randomly sets the number of IRS array elements, while the other settings are consistent with the optimization scheme, which is called "IRS random". The third benchmark scheme is the case where IRS is not set in the scene, and the other settings are the same as the optimization scheme. This scheme is called "no IRS". At the same time, the fourth benchmark scheme sets the phase shift of all IRS sensors to the same fixed value, and the other settings are consistent with the optimization scheme, which is called "IRS fixed".

[0130] On this basis, the noise is set to -17.9dB. In addition, the channel fading from the drone to the IRS is modeled as LoS (line-of-sight) fading, and the channel from the IRS to the user and the eavesdropper is also modeled as LoS fading. Similarly, the channel from the drone to the user and the eavesdropper is also modeled as LoS fading. The fading coefficient is set to 3.75.

[0131] The present invention proposes a Hybrid SAC algorithm, which jointly optimizes the deployment points, unloading ratio, artificial noise power and IRS phase shift of the data UAV (D-UAV) and information UAV (I-UAV) to maximize the system safety rate (RSEC). By applying the Hybrid SAC algorithm, the system proposed by the present invention shows a significantly higher probability of mission success, verifying its effectiveness in enhancing overall performance.

[0132] Figure 2 The convergence of the proposed hybrid SAC intelligent optimization algorithm is demonstrated. As can be seen from the figure, at the beginning of training, the reward value obtained by the agent is low. As the training progresses, the reward value obtained by the agent rises rapidly and reaches a stable state after 4000 rounds of training, which fully reflects the effectiveness of the proposed hybrid SAC intelligent optimization algorithm. The proposed hybrid SAC intelligent optimization algorithm demonstrates its powerful ability in environmental exploration and its ability to adapt to high-dimensional action space. In addition, Figure 2 The stability of the algorithm is also shown. It converges quickly after 4000 rounds of training, and the agent obtains higher reward values ​​in the next 10000 rounds of training. After convergence, the agent does not experience a decrease in reward values ​​as training progresses, which fully reflects the stability and effectiveness of the proposed hybrid SAC intelligent optimization algorithm.

[0133] Figure 3 The deployment point selection of D-UAV and I-UAV is shown after the intelligent algorithm converges. It can be seen that the intelligent agent can accurately make a choice among the given random candidate deployment points, where D-UAV chooses a point closest to I-UAV because there is an occlusion between D-UAV and the user, and the LoS (line of sight) link needs to be rebuilt through IRS. I-UAV chooses a position close to the user, so that IRS can effectively reflect the signal and reduce the eavesdropping rate of eavesdroppers to improve the security rate. This shows that the proposed hybrid SAC intelligent optimization algorithm does not randomly select, but selects the best deployment point, which further proves the effectiveness of the algorithm.

[0134] Figure 4 The approximate safe rate ( ). With the increase of the number of IRS elements, the safety rates of the three schemes are significantly improved, which fully reflects the effectiveness of IRS. In particular, the proposed hybrid SAC intelligent optimization scheme outperforms the two comparison schemes of random selection and fixed phase shift under different numbers of IRS elements, which fully proves the effectiveness and superiority of the proposed optimization algorithm. According to the results, when the number of IRS elements is less than 48, the growth of the three schemes is small, but between 48 and 80, the growth of the three schemes is When the number of IRS exceeds 96, growth tended to stabilize.

[0135] Figure 5 The approximate safe rate ( ). With the increase of transmission power, the It shows an increasing trend. As can be seen from the figure, the proposed solution is better than the three solutions of IRS random, no IRS and random selection in all stages. In addition, with the increase of transmission power, the proposed hybrid SAC intelligent optimization algorithm has an The improvement effect is more significant, but when the transmission power reaches 1000mW, the growth tends to be stable, which further proves the superiority of the algorithm. The fluctuations of the four curves in the figure are due to the noise in the simulation setting and the randomness of the channel.

[0136] It should be noted that the above content only illustrates the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications all fall within the protection scope of the claims of the present invention.

Claims

1. A resource configuration optimization method for a multi-IRS UAV inter-sensory computing integrated system, characterized in that: The method comprises the following steps: Step 1: Establish a multi-intelligent reflective surface enhanced synaesthesia integrated drone edge computing system model; The specific process is: System model construction: The system includes two UAVs: I-UAV and D-UAV; D-UAV is responsible for collecting ground data, and I-UAV is equipped with IRS to assist D-UAV in data offloading and BS-MEC in communication and perception; define multiple decision points DPs of I-UAV and D-UAV, and use vectors to represent DP I =[q I1 ,q I2 ,...,q INI ] and DP D =[q D1 ,q D2 ,...,q DND ]; the initial positions are IP I and IP D ; Establish a flight delay model: define the flight delay of I-UAV and D-UAV in time slot i and Get the formula and Where V is the speed of the drone, δ is the flight time limit of the drone to ensure that it can reach the designated decision point on time; and q I [i+1] and q D [i+1] represents the decision points of I-UAV and D-UAV in the next time slot, q I [i] and q D [i] represents the decision points of I-UAV and D-UAV in the current time slot, respectively; Establish a waiting time model: define the waiting time as the flight time difference between the two drones By setting the flag variable flag wait [i], determine which drone reaches the decision point first Establish energy consumption model: The energy consumption model of I-UAV and D-UAV includes flight energy consumption and hovering energy consumption. The energy constraint is in, They represent the flight power and hovering power of the I-UAV respectively; They represent the flight power and hovering power of the D-UAV respectively; They represent the maximum flight and hovering energy consumption of I-UAV and D-UAV, respectively; Establish signal model: add artificial noise AN in the physical layer to enhance security; the signal model of BS-MEC is x B =Ws B +z; where W is the beamforming matrix of BS-MEC, s B is the transmission signal, z is the artificial noise; Step 2: Establish a channel model for the synaesthesia integrated UAV edge computing system enhanced by multiple intelligent reflective surfaces; The specific establishment process is as follows: System parameter initialization: define the coordinates of different nodes in the system, including the locations of BS-MEC, EAV, users and drones; the location of BS-MEC Q B =[x B ,y B ,0], the position of EAV Q E =[x E ,y E ,0], user location The positions of I-UAV and D-UAV in each time slot are Q I [i]=[x I [i],y I [i],h I ] and Q D [i]=[x D [i],y D [i],h D ], where h I and h D is the fixed drone flight altitude; Define the distance calculation model: where the distance between BS-MEC and I-UAV is ①I-UAV and user u m The distance is ②The distance between BS-MEC and EAV is ③The distance between D-UAV and I-UAV is ④D-UAV and user u m The distance is ⑤The distance between D-UAV and EAV is Among them, x I ,y I is the abscissa and ordinate value of IRS; x B ,y B are the horizontal and vertical coordinate values ​​of BS; is the user's horizontal and vertical coordinate values; x E ,y E is the horizontal and vertical coordinate values ​​of EAV; x D ,y D is the horizontal coordinate value and vertical coordinate value of D_UAV; Define the channel gain model: Since line-of-sight LOS communication is used, the channel gain model follows the free space path loss model; the channel gain from BS-MEC to I-UAV is I-UAV and user u m The channel gain is The channel gain between D-UAV and I-UAV is D-UAV and user u m The channel gain is The channel gain between D-UAV and EAV is Definition of IRS reflection model: Multiple IRSs in the system are used to assist communication and perception. The reflection matrix of each IRS is expressed as Among them, θ mn [i] is the reflection coefficient θ of the mth reflection unit in time slot i mn [i]∈[0,2π], and is the reflection coefficient of the signal; Step 3: Establish a partial unloading model and a computing model for the synaesthesia integrated UAV edge computing system enhanced by multiple intelligent reflective surfaces; The specific establishment process is as follows: System parameter initialization: Initialize the parameters related to offloading and calculation, including the amount of data collected by the D-UAV D; the partial offloading ratio of the data α, where 0≤α≤1; the channel bandwidth W; the transmission power P of the D-UAV r ; Hovering power of I-UAV and D-UAV and Computational power of D-UAV Computing Capacity of BS-MEC M ; Computing capability of D-UAV C D ; The number of CPU cycles S required to execute each bit of data; Noise power Channel gain matrix h D,I [i] and h B,I [i]; IRS reflection matrix Θ n [i]; Establish the offload transmission rate calculation model: The transmission rate from D-UAV to BS-MEC is Among them, h D,I [i] and h B,I [i] is the channel gain matrix from D-UAV to I-UAV and I-UAV to BS-MEC, through the reflection matrix Θ of IRS n [i] To achieve data transmission; Establish an offloading delay calculation model: Based on the transmission rate, the delay of partial offloading is calculated as The formula indicates that in the i-th time slot, the delay of partially unloading data is determined by the amount of unloaded data αD and the transmission rate R D,B [i] Determine; where the calculation delay of BS-MEC is expressed as Where S is the number of CPU cycles per bit of data, C M is the computing power of BS-MEC; the local computing latency of D-UAV is Among them, C D is the computing power of D-UAV, (1-α)D is the amount of data processed on D-UAV; Establish an energy consumption calculation model: Among them, the computing energy consumption of D-UAV is are the powers of I-UAV and D-UAV respectively; the computational energy consumption of I-UAV is Here we assume that the I-UAV only hovers and does not perform computing tasks, so its energy consumption is only determined by the hovering time. Decide; Step 4: Establish the energy model of the synaesthesia integrated drone edge computing system enhanced by multiple intelligent reflective surfaces; The establishment process is as follows: Define the relevant parameters: the size of data collected by D-UAV D; the number of CPU cycles required to perform each bit task S; the proportion of offloaded data α; the computing power C of BS-MEC M ; Computing capability of D-UAV C D ; Power during D-UAV calculation Hovering power of D-UAV and I-UAV and Establish a time calculation model: In which, the partially unloaded data is calculated on BS-MEC, and the BS-MEC calculation time is The computation time for computing the unoffloaded data on D-UAV is Establish energy consumption model: The energy consumption of D-UAV when calculating the remaining data is During the calculation process, the hovering energy consumption of I-UAV is Step 5: Establish an eavesdropping model of the synaesthesia-integrated UAV edge computing system enhanced by multiple intelligent reflective surfaces; The establishment process is as follows: D-UAV or BS-MEC to user u m Data transfer rate User m The signal-to-noise ratio Γ m [i]; user u m Channel gain The eavesdropper can m The achievable transfer rate Signal-to-noise ratio for eavesdroppers The matrix Λ[i] contains the channel, phase adjustment and other system parameters; user u m Safe transmission rate Noise Power Establish user data transmission rate model: define the transmission rate from D-UAV or BS-MEC to user u m The rate at which data is transmitted; where W is the bandwidth, Γ m [i] is the user's signal-to-noise ratio (SINR); Establish the user signal-to-noise ratio SINR calculation model: user u m SINRΓ m [i] represents in, Contains channel information of I-UAV and D-UAV; Bk Represents the signal sent by the base station to the kth user; Establish the eavesdropper rate and signal-to-noise ratio calculation model: define the eavesdropper's influence on user u m The achievable transfer rate in, is the eavesdropper's signal-to-noise ratio SINR; Expressed as where Λ[i] contains the channel, phase adjustment and other system matrices; w m , represents the beamforming matrix of BS and the transpose of the beamforming matrix of BS, respectively, and w i and represents the beamforming matrix transmitted by the BS to the i-th user and the transpose of the beamforming matrix transmitted by the BS to the i-th user, R z is the covariance matrix of artificial noise; Establish a secure transmission rate model: define user u m The achievable safe transmission rate is This formula calculates the user u m The safe rate of + Indicates taking a positive value, that is, when the difference is negative, it is taken as 0; Step 6: Propose resource allocation and trajectory optimization problems and constraints to maximize the safety rate for all users; Step 7: In the MDP framework, the multi-IRS-assisted secure ISAC-MEC system constitutes a dynamic environment, and the control center of the UAV acts as an intelligent agent; in addition to the environment and the intelligent agent, the MDP problem also includes the state space, action space, state transition probability, and reward function; set up Represents the state space of the network: At time step t, the network state includes multiple components Such as channel information, selected actions, and rewards obtained; therefore, the state is defined as Among them, {h D,I [i]} and {h B,I [i]} represent the data collection channel states of I-UAV and D-UAV at time step t, and Λ[i] represents the data collection channel states of I-UAV and D-UAV at time step a. t-1 The action selected, r t represents the reward obtained at time step t; set up Represents the action space of the network: At time step t, the action of the intelligent agent Including the position of the drone, the reflection coefficient of the IRS; therefore, the action is defined as a t = {q I [i],q D [i],Θ n [i],α,R z }; where q I [i] represents the deployment location of I-UAV, q D [i] represents the deployment location of D-UAV, Θ n [i] represents the reflection coefficient of IRS at time step t; set up Represents the state transition probability of the environment, indicating that in state Next action When The probability of The reward function is a measure used by an intelligent system to evaluate the effectiveness of its behavior. The reward function provides a quantitative assessment of the performance of an intelligent agent by mapping its current state and action space to the reward function. The resulting reward value reflects the quality of the agent's action selection strategy and serves as a criterion for evaluating the agent's decision-making ability. Among them, α E is a given constant, α pe is the penalty term, R SEC is the safe rate; Step 8: Based on the reinforcement learning modeling in step 7, an intelligent resource allocation algorithm based on HybridSAC is proposed to jointly optimize the deployment points, unloading ratio, artificial noise power and IRS phase shift of the data drone D-UAV and the information drone I-UAV to maximize the system safety rate RSEC; The HybridSAC algorithm converges the results through training, and the method comprises the following steps: In the SAC algorithm, two action value functions Q and one policy function π are modeled; based on the idea of ​​DoubleDQN, SAC uses two Q networks, but each time a Q network is used, a network with a small Q value is selected to alleviate the problem of overestimation of the Q value; the loss function of any function Q is: Among them, R is the data collected by the policy in the past, because SAC is an offline policy algorithm, is the expected solution, r t is the reward obtained in the tth step, and α is a regularization coefficient; in order to make the training more stable, the target network Q is used here, and there are also two target Q networks Q ω and Q w- , which corresponds one to one with the two Q networks; the update method of the target Q network in SAC is the same as that in DDPG; The loss function of strategy π is obtained from KL divergence and can be simplified to in, represents the expectation, π θ (a t ∣s t ) indicates that the strategy θ is in s t In the state, take a t Action, Q ω (s t ,a t ) represents its value network. The above process is understood as maximizing the function V, because For the environment of continuous action space, the strategy of SAC algorithm outputs the mean and standard deviation of Gaussian distribution, but the process of sampling actions according to Gaussian distribution is not differentiable; therefore, reparameterization technique is needed; the reparameterization method is to start from a unit Gaussian distribution Then multiply the sampled value by the standard deviation and add the mean; this is considered to be sampled from the policy Gaussian distribution, and this is differentiable for the policy function; it is expressed as a t =f θ (∈ t ;s t ), where ∈ t is a noise random variable; taking into account the two functions Q at the same time, rewrite the loss function of the strategy: in, is to find the expectation, π θ (f θ (∈ t ;s t )|s t ) is a strategy, Represents minimizing its value function, f θ (∈ t ;s t ) is t and a t A mapping relationship between ∈ t is a noise random variable, For the SAC algorithm that processes discrete actions, the output of the policy network is modified to a softmax distribution on the discrete action space; the value network directly receives the state and the distribution of the discrete action space as input; the policy network outputs a discrete action distribution, so during the learning process of the value network, there is no need to calculate the next action a t+1 Instead of sampling, the value of the next state is obtained directly through probability calculation; similarly, in the calculation of the loss function α, there is no need to sample the action; The agent interacts with the system to gain training experience; The agent optimizes the deployment points, unloading ratio, artificial noise power and IRS phase shift of the data drone D-UAV and the information drone I-UAV through HybridSAC, and inputs them to enter the new system state, and obtains the current reward value through the reward function; in each iteration, the agent can store the new and old system states, action selections, and reward values ​​obtained by interacting with the environment into the experience pool to provide training samples for network training; Algorithm parameter update: When the agent interacts with the environment for a set number of times, the agent learns to minimize the loss function by replaying the experience stored in the experience pool, and updates the network parameters through reverse gradient; When the agent interacts with the environment for a set number of times, the agent randomly takes out a tuple of size N from the experience pool (s i , a i , r i+1 ,s i+1 ) i∈N , taking the minimum loss function as the goal, calculate the reverse gradient of the loss function and update the SAC network parameters; If the number of training times does not reach the set number, return to the step of interaction between the agent and the system; When the algorithm converges, each network parameter is saved locally.

Images