A dual-mode secure communication method for unmanned aerial vehicles based on reconfigurable intelligent reflective surface and artificial noise

Abstract

The present invention relates to the technical field of unmanned aerial vehicle secure communication, and discloses a method for unmanned aerial vehicle assisted secure communication based on a reconfigurable intelligent reflective surface and artificial noise. The method integrates artificial noise and RIS technology to optimize the unmanned aerial vehicle communication system to enhance legal communication and interfere with illegal eavesdropping. A dual-mode joint optimization framework based on dual-delay deep deterministic policy gradient (TD3) is adopted to synchronously optimize the motion trajectory of the unmanned aerial vehicle, the phase shift configuration of RIS, and the interference power of artificial noise. In order to maximize the channel capacity difference between legitimate users and potential eavesdroppers, the security of physical layer communications is enhanced. Compared with the prior art, the present invention utilizes the synergistic advantages of artificial noise and RIS, improves the efficiency and reliability of communication, enhances the adaptability and flexibility of the system, and effectively copes with complex and changeable communication environments.

Description

Technical Field

[0001] The present invention relates to the technical field of secure wireless communications, and in particular to a drone-assisted secure communication method based on a reconfigurable intelligent reflective surface and artificial noise. Background Art

[0002] Wireless communication security is critical to protecting communication privacy and data integrity. Traditionally, encryption and authentication technologies have been widely adopted as passive security measures to prevent unauthorized access and data tampering. However, these basic security mechanisms have limitations in defending against active attack methods. Even if the communication data is encrypted, attackers may still obtain sensitive information by stealing wireless signals and attempting to decrypt them. In addition, in a man-in-the-middle attack, the attacker may forge an identity to bypass the authentication mechanism, thereby controlling or interfering with the communication link. Therefore, relying solely on traditional passive security measures is difficult to fully respond to various active attack methods, and more advanced and active security protection strategies are needed to ensure the absolute security of wireless communications.

[0003] In order to enhance the security of wireless communications, researchers have proposed active defense schemes including artificial jamming and secure communication technologies based on reconfigurable intelligent reflective surfaces. Artificial jamming introduces artificial noise signals into the communication link to disrupt potential eavesdroppers. Although it can block eavesdropping to a certain extent, it requires certain prior knowledge and may reduce the system spectrum and energy efficiency due to increased noise. On the other hand, RIS is composed of many programmable passive reflective units, which can actively reshape the wireless channel environment, create an ideal signal transmission path for legitimate users, maximize the concentration of wireless signal energy, and effectively suppress signal leakage to potential eavesdroppers.

[0004] Artificial jamming and RIS-based solutions have significant differences in principle and mechanism of action, but the two can complement each other and play a synergistic role. Artificial jamming can provide protection for the RIS modulation phase to prevent malicious attackers from interfering with RIS; while RIS can assist in the accurate generation and efficient propagation of interference signals, improving the effect of artificial jamming on eavesdroppers. In addition, RIS can also be used to actively enhance the signal reception quality of legitimate users, thereby offsetting the throughput loss that may be caused by artificial jamming. By optimizing the coordinated configuration of artificial jamming and RIS, both high security and high efficiency of wireless communications can be achieved, meeting the needs of more advanced security communications in the future.

[0005] Although RIS technology and artificial jamming provide new active defense means for secure communications in theory, existing research has some limitations. On the one hand, most of the work is still limited to idealized single-path or simplified channel models, ignoring the actual complex multipath propagation environment, such as urban scenarios. On the other hand, whether using artificial jamming or RIS-assisted defense solutions, researchers generally face the challenge of non-convex constrained optimization when establishing system models and solving optimization problems.

[0006] Traditional iterative optimization algorithms, such as the successive quadratic programming method and the dual decomposition method, require a good initial point and satisfy the convexity assumption, and are prone to suboptimal solutions for high-dimensional complex problems. Although intelligent optimization algorithms (such as the ant colony algorithm and the simulated annealing algorithm) have global search capabilities, they have slow convergence speed and low computational efficiency, especially in the iterative process of finding feasible solutions. In contrast, deep reinforcement learning (DRL) provides an end-to-end optimization strategy based on trial and error, which can autonomously learn and continuously adjust strategies to adapt to complex environments, breaking through many limitations of traditional algorithms. Introducing DRL into the field of wireless communication security optimization can not only get rid of the limitations of convexity assumptions and initial points, but also provide new ideas and tools for the autonomous optimization design of active defense strategies in complex scenarios, which is expected to significantly improve the security and robustness of the system and has broad application prospects. Summary of the invention

[0007] In view of the shortcomings of the prior art, the present invention provides a UAV assisted safety communication method based on a reconfigurable intelligent reflective surface and artificial noise, aiming to solve the problems existing in practical applications.

[0008] To achieve the above object, the present invention provides the following technical solution: a method for assisting safe communication of unmanned aerial vehicles based on a reconfigurable intelligent reflective surface and artificial noise, comprising the following steps:

[0009] S1: The drone obtains the channel state information of each transmission channel between the base station and the user and its own location information;

[0010] S2: The drone selects and switches to the best working mode based on the current channel status information;

[0011] S3: For each UAV working mode, convert the UAV and channel state information into a Markov decision process tuple;

[0012] S4: Use deep reinforcement learning to select the best jammer transmission power or phase shift configuration;

[0013] S5: When the channel state information between the base station and the user changes, return to step S1.

[0014] As a preferred technical solution of the present invention: it also includes a system for implementing the communication method, and the system is composed as follows:

[0015] P1: One equipped with N a A ground communication base station BS with transmitting antennas;

[0016] P2: K fixed terrestrial user equipment UEs equipped with a single antenna;

[0017] P3: A latent eavesdropper EAV whose detailed location is unknown;

[0018] P4: A multi-functional UAV carrying both c ×M r The uniform planar array composed of reflection units can be reconfigured into a smart reflection surface with a maximum transmission power of p max Artificial noise generating device.

[0019] As a preferred technical solution of the present invention: the position information of the drone itself in step S1 is: the mission duration of the drone is divided into a total of N t There are equal and non-overlapping time intervals of length δ, each of which can be expressed as n∈[1,...,N t ], the coordinates of the drone in the nth time interval are expressed as Q[n]=(q u [n],h u [n]), where q u [n]=(x u [n],y u [n]) is its horizontal coordinate, h u [n] is its vertical coordinate; in addition, the UAV dynamic model is modeled as:

[0020]

[0021]

[0022] h u [n+1]=h u [n]+vsinψ[n] (3)

[0023] Where v is the speed of the drone in this time interval, ψ and are the yaw angle and pitch angle of the UAV in this time interval respectively;

[0024] The coordinates of the user equipment on the ground are denoted as q k =(x k ,y k ), the precise coordinates of the latent eavesdropper are denoted as q ee =(x ee ,y ee), the estimated position of the eavesdropper is denoted as q e =(x e ,y e ), and the estimated errors in the horizontal direction are recorded as Δx e with Δy e , then the relationship between the actual coordinates and the estimated coordinates is expressed as:

[0025] x e =x ee +Δx e ,y e =y ee +Δy e (4)

[0026] At the same time, the coordinate estimation error is recorded as ε, and the error is required to satisfy The estimated location of the eavesdropper is thus confined to a circular area centered on its precise location.

[0027] As a preferred technical solution of the present invention: the optimal working mode described in step S2 has different communication channels, and the channel modeling for the two working modes is as follows:

[0028] T1: The mode of using RIS to enhance the communication between the ground base station and the ground legal user equipment is recorded as mode 1. In mode 1, there will mainly be the direct channel of BS-UE / EAV and the reflected channel of BS-RIS-UE / EAV. Considering the blocking of the direct channel by ground obstacles in complex environments, the Rice fading model is used to express the ground channel:

[0029]

[0030]

[0031] in, and is a circularly symmetric complex Gaussian variable with zero mean and unit variance, is an imaginary vector or matrix, N a is the number of base station antennas, d bk With d be are the Euclidean distances between BS and UE and EAV, respectively, κ1 is the path loss coefficient of the ground-to-ground channel;

[0032] T2: The reflection channel in mode 1 uses Rayleigh fading to model the small-scale fading of the signal, and the channel expression of the open space is:

[0033]

[0034] in M c With M rare the number of reflection units in the vertical and horizontal directions in the RIS array, ρ is the path loss at the reference distance of 1m, k2 is the path loss coefficient between space and ground, β r is the Rayleigh coefficient, and are respectively a determined direct channel and an indirect channel consisting of a circularly symmetric complex Gaussian variable with zero mean and unit variance, where the direct array response is It is expressed as:

[0035]

[0036] in, λ is the wavelength, d i is the spacing of the reflective units on the uniform planar array RIS, and are the cosine and sine of the horizontal arrival angle, is the sine value of the vertical arrival angle, and similarly, the channel of RIS-UE / EAV can be obtained and

[0037] T3: The reachability rates at UE and EAV in mode 1 are expressed as:

[0038]

[0039]

[0040] where p b is the constant transmission power of the base station, σ is the additive Gaussian white noise, is the reflection coefficient matrix of RIS, where is the phase shift of each reflector unit;

[0041] T4: In mode 2, the drone uses artificial noise transmitters to interfere with potential eavesdroppers. In this mode, the reachability at UE and EAV is expressed as:

[0042]

[0043]

[0044] Among them, p j is the transmission power of the artificial noise signal, g uk With g ue are the gains of interference signals to UE and EAV respectively;

[0045] T5: The worst confidentiality rates under mode 1 and mode 2 are:

[0046]

[0047]

[0048] T6: The safety rate optimization problem is defined as:

[0049]

[0050] Where Q is the UAV trajectory, θ is the RIS phase shift, P is the interference power, and N t is the total number of task time slots, K is the total number of users, and the constraints C1 and C2 limit the maximum speed of the UAV in horizontal and vertical directions respectively. and are the maximum speed of the drone in the horizontal and vertical directions, respectively, and δ is the time slot length; C3 and C4 limit the field of the drone's movement, q min With q max is the horizontal boundary of the site, h min With h max is the vertical boundary of the site; C5 and C6 limit the maximum yaw and pitch angles of the drone, ψ min With ψ max are the minimum and maximum yaw angles, and are the minimum and maximum pitch angles of the drone; C7 and C8 limit the RIS phase shift and artificial noise transmission power respectively, θ k [n] is the current RIS phase shift, p j [n] is the current interference transmission power, p max is the maximum interference transmission power of the UAV; c[n] in C9 is the working mode indicator variable, which limits the UAV to only one working mode at the same time.

[0051] As a preferred technical solution of the present invention: the steps of Markov decision process modeling in step S3 are as follows:

[0052] S3-1: Define state space s for mode 1 mode1 (n)={Q[n],h br [n],h rk [n],h bk [n]}, define the state space s for mode 2 mode2 (n)={Q[n],h bk [n]};

[0053] S3-2: Define the action space for mode 1 Defining the action space for mode 2

[0054] S3-3: The reward is designed as r(n) = R[n] - ξV[n], where is the confidentiality rate of the current time slot, V is the penalty for violating the constraint, and ξ is the penalty coefficient.

[0055] As a preferred technical solution of the present invention: the method based on deep reinforcement learning described in step S4 specifically uses a deep learning network architecture, which includes the following contents:

[0056] The original Actor network, the original Critic network, the target Actor network, and the target Critic network are all composed of three-layer fully connected networks. The first two layers of the Actor network have 256 and 128 neurons respectively, and the LeakyReLu function is used as the activation function;

[0057] The number of neurons in the output layer is equal to the dimension of the action space in the current working mode, and the Tanh function is used as the activation function. The first two layers of the critic network are configured with 128 and 64 neurons respectively, and the LeakyReLu function is used as the activation function;

[0058] The output layer dimension is 1, no activation function is used, and the experience replay pool is two independent pools B of size 1000000. mode1 With B mode2 , the agent’s interactive environment includes the main environment E and two main environment copies E1 and E2.

[0059] As a preferred technical solution of the present invention: the method based on deep reinforcement learning in step S4 specifically includes the following training steps:

[0060] S4-1: Initialize the main environment E and two copies of the main environment E1 and E2;

[0061] S4-2: Use random parameters to initialize the Critic network for mode 1 and mode 2 respectively Use random parameters to initialize the Actor network π for mode 1 and mode 2 respectively mode1 With π mode2 ;

[0062] S4-3: Use random parameters to initialize the target critic network for mode 1 and mode 2 respectively Use random parameters to initialize the target Actor network π for mode 1 and mode 2 respectively mode1 and π' mode2 ;

[0063] S4-4: Initialize the experience replay pool B for mode 1 and mode 2 respectively mode1 With B mode2 ;

[0064] S4-5: Synchronize the main environment E to E1 and E2;

[0065] S4-6: Observe each mode state s in the main environment E mode1 (n) and s mode2 (n);

[0066] S4-7: According to the status of each mode, Select action a for each mode mode1 (n) and a mode2 (n), where φ is the actor network parameter, is a Gaussian random variable with 0 mean and 0.1 variance, a min with a max These are the minimum and maximum values ​​of the action, respectively. The clip function limits them to the legal range.

[0067] S4-8: Execute action a of mode 1 in environment E1 mode1 (n) and get the reward r of mode 1 mode1 (n) and the sub-state s mode1 (n+1), execute action a of mode 2 in environment E2 mode2 (n) and get the reward r of mode 2 mode2 (n) and the sub-state s mode2 (n+1);

[0068] S4-9: Copy the environment with higher rewards to the main environment E;

[0069] S4-10: Combine the experience tuples of mode 1 and mode 2 (s mode1 (n),a mode1 (n),r mode1 (n),s mode1 (n+1)) and (s mode2 (n),a mode2 (n),r mode2 (n),s mode2 (n+1)) are stored in their respective experience playback B mode1 With B mode2 ;

[0070] S4-11: Passed Calculate the target Q value of each mode;

[0071] S4-12: Passed Update the Critic network of each mode, where N b is the sampling size, θ i is the parameter of the i-th critic network;

[0072] S4-13: If the current training round number is an even number, then pass Update the Actor network of each mode, where J is the expected return of the strategy;

[0073] S4-14: If the current training round number is an even number, then through θ′ i ←τθ i +(1-τ)θ′ i ,i=1,2 and φ′←τφ+(1-τ)φ′ respectively update the target Actor network and template Critic network parameters of each mode, where θ' and φ' are the parameters of the target Critic network and the target Actor network, respectively, and τ is the neural network update coefficient;

[0074] S4-15: If the preset number of training times has been reached, the training is completed, otherwise steps S4-5 to S4-14 are repeated.

[0075] Compared with the prior art, the present invention has the following beneficial effects:

[0076] The present invention proposes an innovative secure communication method that combines RIS with artificial noise, applies the two to different working modes, and adopts an intelligent optimization strategy based on deep reinforcement learning, showing excellent performance in many aspects such as security, efficiency, and robustness. Specifically, the RIS mode reshapes the wireless channel to create an ideal transmission path for legitimate users, maximizes the concentration of wireless signal energy, and effectively suppresses signal leakage to eavesdroppers, thereby significantly enhancing communication security; the artificial noise mode introduces interference signals to disrupt the eavesdropper's reception, forming a powerful supplement to traditional encryption technology and further improving anti-eavesdropping capabilities. The two modes can be flexibly switched and coordinated according to the scenario, and the optimization based on deep reinforcement learning can autonomously adjust the RIS phase and artificial noise parameters to maximize communication efficiency and quality while ensuring safety. This innovative method organically combines the advantages of RIS and artificial noise, and incorporates advanced intelligent optimization methods, showing excellent performance and broad application prospects that are superior to existing technologies in the field of secure communications. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 It is a schematic diagram of the process of the method proposed by the present invention;

[0078] Figure 2 This is a schematic diagram of the anti-eavesdropping secure communication model of a UAV based on a reconfigurable intelligent reflective surface according to the present invention;

[0079] Figure 3 Schematic diagram of the dual-mode joint optimization framework based on dual-delayed deep deterministic policy gradient;

[0080] Figure 4A comparison chart of 2D and 3D drone trajectories of TD3 of an embodiment of the present invention and the benchmark algorithms DDPG and SAC;

[0081] Figure 5 This is a comparison chart of the cumulative rewards of rounds of TD3 according to an embodiment of the present invention and the benchmark algorithms DDPG and SAC;

[0082] Figure 6 It is a cumulative distribution function diagram of the confidentiality rate of TD3 of the embodiment of the present invention and the benchmark algorithms DDPG and SAC;

[0083] Figure 7 It is a cumulative distribution function diagram of the confidentiality rate of TD3 under different maximum interference powers according to an embodiment of the present invention; DETAILED DESCRIPTION

[0084] Based on the attached schematic diagrams, several embodiments of the present invention are now described in detail to make the advantages and features of the present invention clearer and more specific, in order to provide those skilled in the art with a clear understanding of the protection scope of the claims of the present invention.

[0085] Embodiment: The UAV-assisted safe communication method based on reconfigurable intelligent reflective surface and artificial noise of the present invention comprises: a system equipped with N a There are K ground communication base stations with transmitting antennas, K fixed ground user equipments with single antennas, a latent eavesdropper EAV with unknown detailed location, and a multi-functional unmanned aerial vehicle (UAV) equipped with M c ×M r The uniform planar array composed of reflection units can be reconfigured into a smart reflection surface with a maximum transmission power of p max Artificial noise generating device.

[0086] like Figure 1-2 As shown, the drone-assisted safe communication method based on reconfigurable intelligent reflective surface and artificial noise of the present invention comprises the following steps: the mission duration T is divided into a total of N t There are equal and non-overlapping time intervals of length δ, each of which can be expressed as n∈[1,...,N t ], the device position coordinates involved in step S1 are defined as: the coordinates of the drone in the nth time interval are expressed as Q[n] = (q u [n],h u [n]), where q u [n]=(x u [n],y u [n]) is its horizontal coordinate, h u [n] is its vertical coordinate; in addition, the UAV dynamic model is modeled as:

[0087]

[0088]

[0089] h u [n+1]=h u [n]+vsinψ[n]

[0090] Where v is the speed of the drone in this time interval, ψ and are the yaw angle and pitch angle of the UAV in this time interval respectively.

[0091] The coordinates of the user equipment on the ground are denoted as q k =(x k ,y k ), the precise coordinates of the latent eavesdropper are denoted as q ee =(x ee ,y ee ), since the specific coordinates of the eavesdropper are usually not available in actual application scenarios, the estimated position of the eavesdropper is recorded as q e =(x e ,y e ), and the estimated errors in the horizontal direction are recorded as Δx e with Δy e , then the relationship between the actual coordinates and the estimated coordinates is expressed as:

[0092] x e =x ee +Δx e ,y e =y ee +Δy e

[0093] At the same time, the coordinate estimation error is recorded as ε, and the error is required to satisfy The estimated location of the eavesdropper is thus confined to a circular area centered on its precise location.

[0094] The channel modeling of the unmanned aerial vehicle assisted safety communication method based on reconfigurable intelligent reflective surface and artificial noise for two working modes is as follows:

[0095] The mode of using RIS to enhance the communication between the ground base station and the ground legal user equipment is recorded as mode 1. In mode 1, there will mainly be a direct channel of BS-UE / EAV and a reflected channel of BS-RIS-UE / EAV. Considering the blocking of the direct channel by ground obstacles in a complex environment, the Rice fading model is used to express the ground channel:

[0096]

[0097]

[0098] in, and is a circularly symmetric complex Gaussian variable with zero mean and unit variance, d bk With d be are the Euclidean distances between BS and UE and EAV respectively, and k1 is the path loss coefficient of the ground-to-ground channel.

[0099] The reflection channel in mode 1 uses Rayleigh fading to model the small-scale fading of the signal, and the channel expression of the open space is:

[0100]

[0101] in ρ is the path loss at the reference distance of 1m, k2 is the path loss coefficient between space and ground, β r is the Rayleigh coefficient, and are respectively a determined direct channel and an indirect channel consisting of a circularly symmetric complex Gaussian variable with zero mean and unit variance. It is expressed as:

[0102]

[0103] in, λ is the wavelength, d i is the spacing of the reflective units on the uniform planar array RIS, and are the cosine and sine of the horizontal arrival angle, Then is the sine value of the vertical arrival angle. Similarly, the channel of RIS-UE / EAV can be obtained and

[0104] The reachability rates at UE and EAV in mode 1 are expressed as:

[0105]

[0106]

[0107] where p b is the constant transmission power of the base station, σ is the additive Gaussian white noise, is the reflection coefficient matrix of RIS, where is the phase shift of each reflector unit;

[0108] In mode 2, the drone uses artificial noise transmitters to interfere with potential eavesdroppers. In this mode, the reachability at UE and EAV is expressed as:

[0109]

[0110]

[0111] Among them, p j is the transmission power of the artificial noise signal, g uk With g ue are the gains of interference signals to UE and EAV respectively;

[0112] The worst confidentiality rates under mode 1 and mode 2 are:

[0113]

[0114]

[0115] The safety rate optimization problem is defined as:

[0116]

[0117]

[0118]

[0119] C3:q min ≤q u [n]≤q max

[0120] C4:h min ≤h u [n]≤h max

[0121] C5:ψ min <ψ[n]<ψ max

[0122]

[0123] C7:0≤θ k [n]<2π

[0124] C8:0<p j [n]<p max

[0125] C9:c[n]∈{0,1}

[0126] Constraints C1 and C2 limit the maximum speed of the drone's horizontal and vertical movements respectively, C3 and C4 limit the size of the drone's movement area, C5 and C6 limit the drone's maximum yaw and pitch angles, C7 and C8 limit the RIS phase shift and artificial noise emission power respectively, and C9 limits the drone to only one working mode at the same time.

[0127] The steps to model the Markov decision process are as follows:

[0128] Define the state space s for mode 1 mode1 (n)={Q[n],h br [n],h rk [n],h bk [n]}, define the state space s for mode 2 mode2 (n)={Q[n],h bk [n]}; Define the action space for mode 1 Defining the action space for mode 2 The reward is designed as r(n) = R[n] - ξV[n], where V is the penalty for violating the constraint and ξ is the penalty coefficient;

[0129] like Figure 3 As shown, the present invention uses the neural network and other elements required by the dual-mode method of the double-delay deep deterministic policy gradient, and the framework includes the following components:

[0130] The original Actor network, the original Critic network, the target Actor network, and the target Critic network are all composed of three-layer fully connected networks. The first two layers of the Actor network have 256 and 128 neurons respectively, and the LeakyReLu function is used as the activation function; the number of neurons in the output layer is equal to the dimension of the action space in the current working mode, and the Tanh function is used as the activation function. The first two layers of the critic network are configured with 128 and 64 neurons respectively, and the LeakyReLu function is used as the activation function; the output layer dimension is 1, no activation function is used, and the experience replay pool is two independent pools of size 1000000. mode1 With B mode2 , the agent’s interactive environment includes the main environment E and two main environment copies E1 and E2.

[0131] The double-delayed deep deterministic policy gradient method used specifically includes the following training steps:

[0132] 1) Initialize the main environment E and two copies of the main environment E1 and E2;

[0133] 2) Use random parameters to initialize the Critic network for mode 1 and mode 2 respectively Use random parameters to initialize the Actor network π for mode 1 and mode 2 respectively mode1 With π mode2 ;

[0134] 3) Use random parameters to initialize the target Critic network for mode 1 and mode 2 respectively Q'mode1,2 , Q' mode2,1 , Q' mode2,2 , use random parameters to initialize the target Actor network π' for mode 1 and mode 2 respectively mode1 and π' mode2 ;

[0135] 4) Initialize the experience replay pool B for mode 1 and mode 2 respectively mode1 With B mode2 ;

[0136] 5) Synchronize the main environment E to E1 and E2;

[0137] 6) Observe each mode state s in the main environment E mode1 (n) and s mode2 (n);

[0138] 7) According to each mode state, through a(n)=πs(n),φ+òa min a max ) Select action a for each mode mode1 (n) and a mode2 (n);

[0139] 8) Execute action a of mode 1 in environment E1 mode1 (n) and get the reward r of mode 1 mode1 (n) and the sub-state s mode1 (n+1), execute action a of mode 2 in environment E2 mode2 (n) and get the reward r of mode 2 mode2 (n) and the sub-state s mode2 (n+1);

[0140] 9) Copy the environment with higher rewards to the main environment E;

[0141] 10) Combine the experience tuples (s) of mode 1 and mode 2 mode1 (n),a mode1 (n),r mode1 (n),s mode1 (n+1)) and (s mode2 (n),a mode2 (n),r mode2 (n),s mode2 (n+1)) are stored in their respective experience playback B mode1 With B mode2 ;

[0142] 11) Pass Calculate the target Q value of each mode;

[0143] 12) Pass Update the Critic network of each mode;

[0144] 13) If the current number of training rounds is an even number, Update the Actor network of each mode;

[0145] 14) If the current training round number is an even number, then through θ′ i ←τθ i +(1-τ)θ′ i ,i=1,2 and φ′←τφ+(1-τ)φ′ respectively update the target Actor network and template Critic network parameters of each mode;

[0146] 15) If the preset number of training times has been reached, the training is completed, otherwise repeat steps 5 to 14.

[0147] Experimental results: Figure 4 As shown in Figure 1, the trajectories generated by the TD3 algorithm and two baseline algorithms, DDPG and SAC, under two different device distributions. In scenario 1, the trajectories of TD3 and DDPG are similar. When faced with a potential eavesdropper at a relatively close distance, they choose to actively approach to enhance the interference effect of artificial noise, while the SAC algorithm tends to avoid the eavesdropper and use RIS to enhance legitimate communications. On the other hand, in scenario 2, the trajectory of SAC becomes more aggressive, approaching the possible location of the potential eavesdropper in the fastest path and circling. In contrast, TD3 adopts a strategy more similar to DDPG and balances the two operating modes.

[0148] like Figure 5 As shown in Figure 1, the figure illustrates the cumulative rewards of each round under the three algorithms. Before 500 cycles, the cumulative rewards obtained by TD3 and DDPG were similar. As the training progressed, the cumulative rewards obtained by TD3 increased significantly because the noise added to its behavior provided additional exploration capabilities. On the other hand, the SAC algorithm achieved the worst results among the three, which may be due to its greater sensitivity to Q-value estimation and its inferior exploration capabilities compared to other algorithms.

[0149] like Figure 6 As shown in the figure, the cumulative distribution function (CDF) of the confidentiality rate under different algorithm frameworks is shown. It is obvious that the security rate achieved by TD3 is higher than DDPG most of the time, and higher than SAC throughout the process. Its security rate is also much higher than other baseline algorithms.

[0150] like Figure 7 As shown in Figure 1, the impact of different maximum interference powers on the confidentiality rate under the TD3 algorithm is evaluated. As the maximum interference power increases, confidentiality is significantly improved. However, the maximum confidentiality rate that can be achieved is less affected by the interference power, which may be due to the fact that the interference acts on both legitimate users and eavesdroppers at the same time.

[0151] This embodiment shows through experimental comparison that the drone-assisted secure communication method based on reconfigurable intelligent reflective surface and artificial noise proposed in the present invention has obvious advantages in enhancing communication security and improving communication efficiency. Compared with baseline algorithms such as DDPG and SAC, the adopted TD3 algorithm can more effectively optimize RIS phase configuration and artificial noise generation, and significantly improve the confidentiality rate. It proves that this method has important application value and development potential in actual wireless communication systems.

[0152] The above examples are only specific embodiments of the present invention, and their descriptions are intended to help understanding rather than limit the scope of protection of the present invention. It should be pointed out that for those skilled in the art, without departing from the overall concept and design principle of the present invention, some equivalent transformations, modifications or extensions can be made to the present invention for specific application scenarios or actual needs, and these deformations and optimizations should fall within the scope of protection of the present invention. The scope of protection of the present invention shall be based on the records made in the claims.

Claims

1. A UAV-assisted secure communication method based on reconfigurable intelligent reflective surface and artificial noise, characterized in that: The specific steps include: S1: The drone obtains the channel state information of each transmission channel between the base station and the user and its own location information; S2: The drone selects and switches to the best working mode based on the current channel status information; S3: For each UAV working mode, convert the UAV and channel state information into a Markov decision process tuple; The steps of modeling Markov decision process are as follows: S3-1: Define state space s for mode 1 mode1 (n)={Q[n],h br [n],h rk [n],h bk [n]}, define the state space s for mode 2 mode2 (n)={Q[n],h bk [n]}, where Q[n] is the coordinate of the UAV in the nth time interval; S3-2: Define the action space for mode 1 Defining the action space for mode 2 S3-3: The reward is designed as r(n) = R[n] - ξV[n], where is the confidentiality rate of the current time slot, V is the penalty for violating the constraint, and ξ is the penalty coefficient; S4: Use deep reinforcement learning to select the best jammer transmission power or phase shift configuration; The deep reinforcement learning-based method specifically uses a deep learning network architecture that includes the following: The original Actor network, the original Critic network, the target Actor network, and the target Critic network are all composed of three-layer fully connected networks. The first two layers of the Actor network have 256 and 128 neurons respectively, and the LeakyReLu function is used as the activation function; The number of neurons in the output layer is equivalent to the dimension of the action space in the current working mode. In the critic network, the activation function is used to control the output characteristics of the neurons and introduce nonlinearity so that the model can learn more complex patterns. In the first two layers, the first layer is configured with 128 neurons and uses the Tanh activation function to adapt to the smooth conversion of the input; the second layer is configured with 64 neurons and uses the LeakyReLU activation function to learn complex feature patterns. In the output layer, since a continuous Q value needs to be output, the activation function is not used and the network calculation results are directly output; The output layer dimension is 1, no activation function is used, and the experience replay pool is two independent pools B of size 1000000. mode1 With B mode2 , the agent’s interactive environment includes the main environment E and two copies of the main environment E1 and E2; The method based on deep reinforcement learning specifically includes the following training steps: S4-1: Initialize the main environment E and two copies of the main environment E1 and E2; S4-2: Use random parameters to initialize the Critic network for mode 1 and mode 2 respectively Use random parameters to initialize the Actor network π for mode 1 and mode 2 respectively mode1 With π mode2 ; S4-3: Use random parameters to initialize the target critic network for mode 1 and mode 2 respectively Use random parameters to initialize the target Actor network π for mode 1 and mode 2 respectively mode1 and π' mode2 ; S4-4: Initialize the experience replay pool B for mode 1 and mode 2 respectively mode1 With B mode2 ; S4-5: Synchronize the main environment E to E1 and E2; S4-6: Observe each mode state s in the main environment E mode1 (n) and s mode2 (n); S4-7: According to each mode state, a(n)=clip(π(s(n),φ)+∈,a min ,a max ) Select action a for each mode mode1 (n) and a mode2 (n), where φ is the actor network parameter, ∈ is a Gaussian random variable with 0 mean and 0.1 variance, and a min with a max These are the minimum and maximum values ​​of the action, respectively. The clip function limits them to the legal range. S4-8: Execute action a of mode 1 in environment E1 mode1 (n) and get the reward r of mode 1 mode1 (n) and the sub-state s mode1 (n+1), execute action a of mode 2 in environment E2 mode2 (n) and get the reward r of mode 2 mode2 (n) and the sub-state s mode2 (n+1); S4-9: Copy the environment with higher rewards to the main environment E; S4-10: Combine the experience tuples of mode 1 and mode 2 (s mode1 (n),a mode1 (n),r mode1 (n),s mode1 (n+1)) and (s mode2 (n),a mode2 (n),r mode2 (n),s mode2 (n+1)) are stored in their respective experience replay pools B mode1 With B mode2 ; S4-11: Passed Calculate the target Q value of each mode; S4-12: Passed Update the Critic network of each mode, where N b is the sampling size, θ i is the parameter of the i-th critic network; S4-13: If the current training round number is an even number, then pass Calculate the policy gradient required to update the Actor network of each mode, where J is the expected return of the policy; S4-14: If the current number of training rounds is even, then through θ i ′←τθ i +(1-τ)θ i ′,i=1,2 and φ′←τφ+(1-τ)φ′ respectively update the target Actor network and template Critic network parameters of each mode, where θ' and φ' are the parameters of the target Critic network and the target Actor network, respectively, and τ is the neural network update coefficient; S4-15: If the preset number of training times has been reached, the training is completed, otherwise, steps S4-5 to S4-14 are repeated; S5: When the channel state information between the base station and the user changes, return to step S1.

2. According to claim 1, a method for drone-assisted safe communication based on reconfigurable intelligent reflective surface and artificial noise is characterized in that: The above communication method is applied to a system for implementing the communication method, and the system is constituted as follows: P1: One equipped with N a A ground communication base station BS having a transmitting antenna; P2: K fixed terrestrial user equipment UEs equipped with a single antenna; P3: A latent eavesdropper EAV whose detailed location is unknown; P4: A multi-functional UAV, equipped with M c ×M r The uniform planar array composed of reflection units can be reconfigured into a smart reflection surface with a maximum transmission power of p max Artificial noise generating device.

3. The method for unmanned aerial vehicle assisted safe communication based on reconfigurable intelligent reflective surface and artificial noise according to claim 2, characterized in that: The drone's own position information in step S1 is: Divide the drone's mission duration into a total of N t There are equal and non-overlapping time intervals of length δ, each of which can be expressed as n∈[1,...,N t ], the coordinates of the drone in the nth time interval are expressed as Q[n]=(q u [n],h u [n]), where q u [n]=(x u [n],y u [n]) is its horizontal coordinate, h u [n] is its vertical coordinate; in addition, the UAV dynamic model is modeled as: h u [n+1]=h u [n]+vsinψ[n] (3) Where v is the speed of the drone in this time interval, ψ and are the yaw angle and pitch angle of the UAV in this time interval respectively; The coordinates of the user equipment on the ground are denoted as q k =(x k ,y k ), the precise coordinates of the latent eavesdropper are denoted as q ee =(x ee ,y ee ), the estimated position of the eavesdropper is denoted as q e =(x e ,y e ), and the estimated errors in the horizontal direction are recorded as Δx e with Δy e , then the relationship between the actual coordinates and the estimated coordinates is expressed as: x e =x ee +Δx e ,y e =y ee +Δy e (4) At the same time, the coordinate estimation error is recorded as ε, and the error is required to satisfy The estimated location of the eavesdropper is thus confined to a circular area centered on its precise location.

4. The method for unmanned aerial vehicle assisted safe communication based on reconfigurable intelligent reflective surface and artificial noise according to claim 2, characterized in that: The optimal working mode described in step S2 has different communication channels, and the channel modeling for the two working modes is as follows: T1: The mode of using RIS to enhance the communication between the ground base station and the ground legal user equipment is recorded as mode 1. In mode 1, there will mainly be a direct channel of BS-UE / EAV and a reflected channel of BS-RIS-UE / EAV. Considering the blocking of the direct channel by ground obstacles in a complex environment, the Rice fading model is used to express the ground-to-ground channel: in, and is a circularly symmetric complex Gaussian variable with zero mean and unit variance, is an imaginary vector or matrix, N a is the number of base station antennas, d bk With d be are the Euclidean distances between BS and UE and EAV, respectively, κ1 is the path loss coefficient of the ground-to-ground channel; T2: The reflection channel in mode 1 uses Rayleigh fading to model the small-scale fading of the signal, and the channel expression of the open space is: in M c With M r are the number of reflection units in the vertical and horizontal directions in the RIS array, ρ is the path loss at the reference distance of 1m, k2 is the path loss coefficient between space and ground, β r is the Rayleigh coefficient, is an indirect channel consisting of circularly symmetric complex Gaussian variables with zero mean and unit variance, and the direct array response is It is expressed as: in, λ is the wavelength, d i is the spacing of the reflective units on the uniform planar array RIS, and are the cosine and sine of the horizontal arrival angle, is the sine value of the vertical arrival angle, and similarly, the channel of RIS-UE / EAV can be obtained and T3: The reachability rates at UE and EAV in mode 1 are expressed as: where p b is the constant transmission power of the base station, σ is the additive Gaussian white noise, is the reflection coefficient matrix of RIS, where is the phase shift of each reflector unit; T4: In mode 2, the drone uses artificial noise transmitters to interfere with potential eavesdroppers. In this mode, the reachability at UE and EAV is expressed as: Among them, p j is the transmission power of the artificial noise signal, g uk With g ue are the gains of interference signals to UE and EAV respectively; T5: The worst confidentiality rates under mode 1 and mode 2 are: T6: The safety rate optimization problem is defined as: Where Q is the UAV trajectory, θ is the RIS phase shift, P is the interference power, and N t is the total number of task time slots, K is the total number of users, and the constraints C1 and C2 limit the maximum speed of the UAV in horizontal and vertical directions respectively. and are the maximum speed of the drone in the horizontal and vertical directions, respectively, and δ is the time slot length; C3 and C4 limit the field of the drone's movement, q min With q max is the horizontal boundary of the site, h min With h max is the vertical boundary of the site; C5 and C6 limit the maximum yaw and pitch angles of the drone, ψ min With ψ max are the minimum and maximum yaw angles, and are the minimum and maximum pitch angles of the drone; C7 and C8 limit the RIS phase shift and artificial noise transmission power respectively, θ k [n] is the current RIS phase shift, p j [n] is the current interference transmission power, p max is the maximum interference transmission power of the UAV; c[n] in C9 is the working mode indicator variable, which limits the UAV to only one working mode at the same time.

Images