Intelligent covert communication method and device based on artificial interference and 3D beamforming

Abstract

The invention discloses an intelligent covert communication method and device based on artificial interference and 3D beamforming, and the method specifically comprises the steps: firstly building a multi-antenna covert communication system model based on channel state information and antenna gain network parameters; then, based on a radio frequency signal receiving principle, establishing a signal detection model of the monitoring machine, calculating an omission ratio and a false alarm rate of the monitoring machine, obtaining a minimum error detection probability of the monitoring machine, and establishing an optimization problem of hiding rate maximization; constructing a depth determination strategy gradient (DDPG) neural network; and finally, on the basis of the DDPG neural network, an intelligent covert communication algorithm based on artificial interference and 3D beam forming is adopted, and the covert rate of the covert communication system is optimized. According to the method, the transmission rate of the legal receiver can be maximized under the condition of meeting the concealment limitation, the communication performance and the concealment performance are enhanced, and the intelligent concealed communication requirements in various scenes can be met.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication security technology, and in particular to an intelligent covert communication method and device based on artificial interference and 3D beamforming. Background Art

[0002] With the rapid development of wireless communication technology, especially the promotion and application of 5G commercial technology and the upcoming 6G technology, human society has officially entered a new era driven by big data. In this era, massive amounts of information are transmitted in various wireless communication networks, including highly sensitive privacy information such as financial data, personal identity information, and medical health data. However, due to the inherent broadcast characteristics of wireless channels, these sensitive information are extremely vulnerable to security threats such as eavesdropping, tampering, and attacks during transmission, causing serious privacy leaks and data security issues. Therefore, in order to effectively respond to information security challenges in communication networks and ensure the transmission security of massive sensitive data, it has become an urgent need to develop more advanced information security strategies and protection methods.

[0003] Traditional cryptographic methods mainly use encryption / decryption technology at the upper layer of the network protocol stack to achieve secure data transmission. Encryption technology has always been the most important means of protecting data security, but there are problems with key distribution and key management, which greatly limits their application in decentralized wireless networks. At the same time, with the development of quantum computing, encryption technology based on high computational complexity faces the risk of being cracked. Physical layer security technology based on Shannon information theory uses the inherent randomness of background noise and the physical characteristics of wireless channels such as fading and reciprocity to provide information-theoretic security without relying on the computing power of eavesdroppers. Therefore, physical layer security technology is considered to be an important supplement to upper-layer cryptographic methods, which can provide stronger security for data transmission.

[0004] In recent years, a new secure communication paradigm - covert communication - has gradually become a hot topic of research in academia and industry. Unlike traditional cryptography and physical layer security technologies that focus on protecting information content from being cracked, the goal of covert communication is to hide the communication behavior itself through advanced signal processing technology, thereby preventing the communication process from being detected and identified. Covert communication technology disguises communication signals as background noise or other irrelevant signals, making it difficult for attackers to detect the existence of communication, thereby preventing them from conducting further attacks. It is the first barrier to ensure the security of wireless communications.

[0005] Traditional covert communication schemes usually reduce the transmission power of the transmitter so that the effective signal is hidden in the background noise or artificial interference signal to confuse the attacker and achieve the goal of low probability detection. However, these methods usually assume that the channel state information follows a known mathematical distribution, and based on this, a fixed power transmission scheme is designed to achieve the optimal covert rate. At the same time, this precise power control is often transformed into the solution of a non-convex optimization problem, which requires complex mathematical calculations. It is particularly noteworthy that since the attacker is usually in a passive receiving state and can autonomously adjust the monitoring strategy according to the changes in the electromagnetic environment, its key information such as location information, channel state information and decision threshold is difficult to obtain, resulting in traditional optimization methods being difficult to effectively deal with the problem of covert communication being attacked under non-ideal conditions, thus showing significant uncertainty and performance gap in actual deployment. To address the above challenges, model-driven deep learning methods can use unknown information as training parameters and interact with the network environment in real time and dynamically through reinforcement learning to obtain the optimal decision solution. This method can not only eliminate idealized assumptions, but also significantly reduce computing time. Therefore, deep learning is widely used in wireless communication networks to solve security problems encountered in various layers of the protocol stack. For example, in the physical layer, deep learning can be used for anti-interference, modulation recognition, and signal detection; in the data link layer, it can be used for link quality prediction, channel resource allocation, and traffic prediction; in the network layer, it can be used to find the optimal secure route; in the application layer, it can be used for data privacy protection, etc. However, the application of deep learning to the design of covert communication transmission schemes is still in its early stages, and further research is urgently needed to meet the needs of intelligent covert communication in various scenarios and promote its popularization and application in the field of wireless communication security. Summary of the invention

[0006] The purpose of the present invention is to provide an intelligent covert communication method and device based on artificial interference and 3D beamforming, which can maximize the transmission rate of the legitimate receiver under the condition of meeting the concealment restrictions, has strong communication performance and concealment performance, and can meet the intelligent covert communication needs in various scenarios.

[0007] The technical solution to achieve the purpose of the present invention is: an intelligent covert communication method based on artificial interference and 3D beamforming, comprising the following steps:

[0008] Step 1: Based on channel state information and antenna gain network parameters, a multi-antenna covert communication system model is established;

[0009] Step 2: Based on the principle of radio frequency signal reception, a signal detection model of the monitoring machine is established;

[0010] Step 3: Calculate the missed detection rate and false alarm rate of the monitoring machine, and obtain the minimum false detection probability of the monitoring machine, and establish the optimization problem of maximizing the concealment rate;

[0011] Step 4: Construct a deep deterministic policy gradient DDPG neural network;

[0012] Step 5: Based on the DDPG neural network, an intelligent covert communication algorithm based on artificial interference and 3D beamforming is used to optimize the covert rate of the covert communication system.

[0013] Furthermore, based on the channel state information and antenna gain network parameters described in step 1, a multi-antenna covert communication system model is established, as follows:

[0014] The multi-antenna covert communication system includes a multi-antenna transmitter, a single-antenna legal receiver, a cooperative node jammer and an illegal monitoring machine;

[0015] The goal of the transmitter is to covertly send data to the receiver through 3D beamforming technology, and use 3D beamforming technology to focus the main beam of the transmitter in the direction of the receiver in three-dimensional space; the jammer sends artificial interference signals as a cover for the hidden signal to deceive the monitoring machine, and adjusts the beamforming vector to improve the signal-to-interference-plus-noise ratio (SINR) of the receiver.

[0016] Assume that the transmitter is located at the origin of the coordinate system, and the positions of the jammer, receiver, and monitor in the three-dimensional coordinates are (x j ,y j ,z j ), (x b ,y b ,z b ) and (x w ,y w ,z w ); the complex Gaussian channel vectors of the transmitter-receiver, transmitter-monitor, jammer-receiver and jammer-monitor are respectively denoted as and Where 0 represents the zero matrix, N a and N j Represents the number of antennas of the transmitter and jammer, C ab , C aw , C jb , C jw The channel covariance matrices of transmitter-receiver, transmitter-monitor, jammer-receiver and jammer-monitor, respectively;

[0017] The channel adopts a time slot model, that is, the channel vector remains unchanged in each time slot and changes independently between each time slot; in each time slot, the signal vector of the transmitter and the interference signal vector of the jammer are expressed as and Where n represents the number of symbols in a time slot, the transmitter's transmitted signal is The transmission signal of the jammer is in is the beamforming vector used by the transmitter to transmit the message, is the beamforming vector used by the jammer to transmit the jamming signal, l = 1, 2, ..., n, and Where E{} is the expectation operator, and Respectively represent the dimension N in the complex field a ×1 and N j ×1 matrix, N a and N j Represent the number of antennas of the transmitter and jammer respectively;

[0018] For array antennas, the vertical and horizontal modes of the transmitting antenna have different antenna gains in dB, and the attenuation of the horizontal antenna is Among them m and θ 3dB are the maximum attenuation and horizontal 3D beamwidth of the transmitter antenna, θ0 represents the horizontal reference angle of the transmitter, and the horizontal angle of node m relative to the reference direction. x m and m They represent the horizontal and vertical coordinates of node m in the horizontal coordinate system respectively; the attenuation of the vertical antenna is in and are the vertical reference angle and vertical 3D beam width of the transmitter antenna; the vertical direction angle of node m relative to the transmitter Δz m is the height difference between the transmitter and node m, and the antenna gain of node m for:

[0019]

[0020] Where Ω max is the maximum antenna gain in the reference direction;

[0021] Given Then the linear antenna gain of node m is:

[0022]

[0023] in is the vertical reference angle of the transmitter antenna;

[0024] In summary, the lth received signal of node m in each time slot is expressed as:

[0025]

[0026] Among them, α, d am ,d jm 、(·) H denote the path loss index, the distance between the transmitter and the node m, the distance between the jammer and the node m, and the conjugate transpose operator, respectively; $ is the additive white noise received by node m; ψ0 means that the transmitter does not send hidden information to the receiver, and ψ1 means that the transmitter sends hidden information to the receiver; p a represents the transmission power of the transmitter, and They represent the channel gain coefficients between the jammer and node m and the transmitter and node m respectively;

[0027] The transmitter has known the channel state information of the receiver, so the transmitter can use the maximum ratio transmission technology, that is:

[0028] w a =h ab / |h ab |.

[0029] Furthermore, based on the principle of radio frequency signal reception as described in step 2, a signal detection model of a monitoring machine is established, wherein the monitoring machine is equipped with a radiometer for real-time monitoring of the received power of the signal, and by detecting whether the received signal strength exceeds a preset threshold, it is determined whether there is information being transmitted between the transmitter and the receiver. The specific steps are as follows:

[0030] Step 2.1: The monitor collects signal samples within the set time window and calculates the power average value Y of these samples. w ;

[0031] Step 2.2, based on the binary hypothesis test theory, the monitoring machine compares the calculated power average value with the preset detection threshold v; the mathematical expression used for the judgment test of the monitoring machine is:

[0032]

[0033] in Indicates the total power of the received signal of the monitoring machine. Indicates the judgment threshold of the monitoring machine. represents the tth symbol received by the monitor w;

[0034] Step 2.3: Based on the comparison result, the monitoring machine determines whether there is covert communication in the system. When the total received power exceeds the threshold, it is determined that information is transmitted from the transmitter to the receiver, that is, the ψ1 situation. On the contrary, when the total received power is lower than the threshold, it is determined that the transmitter does not transmit information to the receiver, that is, the ψ0 situation.

[0035] Step 2.4: In the real situation, the transmitter sends a hidden message to the receiver, but Y w ≤v and the monitoring machine makes ψ0, then missed detection occurs, and the missed detection probability is expressed as P MD ; When the transmitter does not transmit data to the receiver, but Y w >v and the monitoring machine makes a judgment of ψ1, then a false alarm occurs, and the false alarm probability is expressed as P FA The false alarm probability and missed detection probability constitute the detection error probability P of the monitoring machine. error , denoted as P error =P FA +P MD ; For any positive number ∈, when n→∞, P FA +P MD ≤1-∈, the communication between the transmitter and the receiver is considered to be hidden.

[0036] Furthermore, the missed detection rate and false alarm rate of the monitoring machine are calculated in step 3, and the minimum false detection probability of the monitoring machine is obtained, and the optimization problem of maximizing the concealment rate is established, which is specifically as follows:

[0037] Step 3.1: Calculate the false alarm probability of the monitoring machine and the probability of missed detection The formula is:

[0038]

[0039] in, is the judgment threshold of the monitoring machine, is the noise variance, is a random chi-square variable with 2n degrees of freedom, and γ is the interference power received by the monitoring station, expressed as:

[0040]

[0041] Among them, d jw is the distance from the jammer to the monitor, d aw is the distance from the transmitter to the jammer, is the antenna gain of the transmitter to the monitor, p a is the transmitting power of the transmitter, h ab / ||h ab || is the beamforming vector ω of the transmitter antenna a ;haw and h jw They represent the channel gain coefficients of transmitter-monitor and jammer-monitor, respectively. represents the beamforming vector of the jammer;

[0042] Step 3.2: According to Lebesgue's convergence theorem, when n→∞, replace The false alarm probability and missed detection probability are re-expressed as:

[0043]

[0044] make Then the probability density function f of γ is Γ (γ) is expressed as:

[0045]

[0046] Where γ is the interference power received by the monitor;

[0047] Step 3.3: According to the definition of false alarm probability and missed detection probability, P is obtained through integral calculation. FA and P MD The closed expression of is:

[0048]

[0049]

[0050] in is the judgment threshold of the monitoring machine, is the noise variance;

[0051] In order to satisfy P FA +P MD =1, the monitoring machine only considers situation;

[0052] Step 3.4, order The optimal detection threshold is:

[0053]

[0054] Step 3.5: For the multi-antenna system, the concealed transmission rate is maximized by optimizing the transmission power and 3D beamforming. The optimization problem of maximizing the concealed rate is established as follows:

[0055]

[0056] in, is the probability of data transmission in each time slot, Indicates the transmitter's transmit power limit. represents the interference power limit of the jammer, Tr(·) represents the trace operator, min(P FA +P MD )≥1-ε represents the system concealment constraint, ε represents the concealment constraint threshold, and Represent the maximum power limit of the transmitter and the jammer respectively, w j represents the beamforming vector of the jammer.

[0057] Furthermore, the construction of the deep deterministic policy gradient DDPG neural network described in step 4 is as follows:

[0058] A fully connected deep neural network is used to construct the Critic network and Actor network structures, which include an input layer, an output layer and two hidden layers respectively. The input dimension of the Critic network is equal to the cardinality of the state set and the action set, and the output value is the Q value function. The input and output dimensions of the Actor network are defined as the cardinality of the state and action respectively. The number of neurons in the hidden layer depends on the number of antennas of the transmitter and the jammer. The number of neurons in the hidden layer must be greater than the input and output dimensions. The action output by the Actor network will be input into the hidden layer 2 for calculation. Get the loss function and gradient, Δ q is the gradient calculation symbol, is the state-action value function, represents the weight and bias parameters of the neural network, s (t) represents the network status at time t, and α represents the action taken;

[0059] Before inputting into the Critic network and the Actor network, the state s is whitened to eliminate the correlation between the items in the state s; the hidden layer is batch normalized to overcome the change in the input distribution of each layer caused by the parameter change of the previous layer; the tanh function is used as the activation function to handle negative input;

[0060] The Adam algorithm is used to optimize the train-critic network and the train-actor network. The adaptive learning rate is determined by the following formula:

[0061]

[0062] Among them, λ c and λ a are the decay rates of the train-critic network and the train-actor network, and Represent the learning rate of the train-critic network at time t and t-1 respectively, and Represent the learning rate of the train-actor network at time t and t-1 respectively;

[0063] The action value output by the Actor network It is necessary to satisfy the constraints defined in the optimization problem of maximizing the concealment rate, so a normalization layer is added at the output of the Actor network, where Make sure the vertical angle of the transmitter antenna does not exceed 90°.

[0064] Furthermore, the DDPG neural network described in step 5 uses an intelligent covert communication algorithm based on artificial interference and 3D beamforming to optimize the covert rate of the covert communication system, as follows:

[0065] Step 5.1: Construct state, action, and immediate reward;

[0066] (1) State: The state s(t) at time step t is composed of the action a(t-1) obtained from the previous time step, the channel vector (h ab ,h jb ,h aw ,h jw ); Since the neural network can only accept real numbers but not complex numbers as input, if complex numbers are involved when constructing the state s, the real part and the imaginary part will be used as independent input units respectively; Since all inter-node channels obey the complex Gaussian distribution, the channel vector should be split into two terms, the real part Re and the imaginary part Im, that is, h ab =Re{h ab}+Im{h ab}, both parts are used as independent input items of the Critic network and the Actor network; since both the receiver and the monitor are single-antenna receivers, h ab With h aw The vector size is equal to the number of transmitter antennas M, and h jb With h jw The vector size is equal to the number of antennas N of the jammer; and in action, the beamforming vector w j The size of is also equal to the number of antennas N of the jammer, and is divided into real and imaginary parts. Therefore, the total number of items constructed from the actions in the t-1 time step is 2*K+2, where K is the maximum number of training steps; in summary, the dimension of the state space D s =2*(2*M+2*K)+2*K+2;

[0067] (2) Action: The action is the jammer transmitting the beamforming vector w j , Transmitter transmission power P a and the vertical angle of the transmitter antenna Composition, that is Similarly, to solve the real input problem, the jammer transmit beamforming vector w is j =Re{w j}+Im{w j} is split into two parts, the real part and the imaginary part, which are used as the input unit of the action. The dimension of the action space is D s =2*K+2;

[0068] (3) Reward: At the tth step of training, the reward is set to the reward for taking an action from the Actor network given the channel state information. The resulting system concealed rate capacity;

[0069] Step 5.2, Initialization: First initialize the experience replay buffer of size D Used to store training samples, ensure the diversity of empirical data and reduce the correlation between samples; then initialize the neural network parameters, including the train-actor network parameters target-actor network parameters Train-critic network parameters Target-critic network parameters The jammer is used to obtain the beamforming vector w of the jammer signal j Initialized as a complex vector of all 1s, the transmitter's transmission power P a Initialized to 0.1W, the vertical angle of the transmitter antenna Initialized to 0°;

[0070] Step 5.3, state construction: At time step t, the action and channel information (h ab ,h jb ,h aw ,h jw )Construct the current state s(t);

[0071] Step 5.4, action selection: The Actor network receives the state s(t) as input and generates a deterministic policy action a(t);

[0072] Step 5.5, reward calculation: According to the action output by the Actor network and the current channel information, calculate the immediate reward r(t), which is related to the covert communication rate. The goal is to maximize Bob's receiving rate while meeting the concealment constraint.

[0073] Step 5.6, network update: Use the Critic network to evaluate the actions of the actor network, calculate the corresponding Q value function, and use the stochastic gradient descent SGD method to update the parameters of the Critic network. The learning rate u in the update process cAt the same time, based on the feedback of the Critic network, by adjusting the parameters of the Actor network Maximize the Q-value function and use SGD to update the learning rate u a ;

[0074] Step 5.7, Experience Replay: At each time step, store the current state s(t), a(t), r(t+1) and the next state s(t+1) into the experience replay buffer And randomly extract samples of size W from the buffer for network training;

[0075] Step 5.8, target network update: update the parameters of the target critic network and the target actor network every set number of steps U.

[0076] An intelligent covert communication device based on artificial interference and 3D beamforming, the device is used to implement the intelligent covert communication method based on artificial interference and 3D beamforming, the device includes a covert communication system model building module, a signal detection model building module, an optimization problem building module, a neural network building module and an optimization module, wherein:

[0077] A covert communication system model building module builds a multi-antenna covert communication system model based on channel state information and antenna gain network parameters;

[0078] The signal detection model building module builds the signal detection model of the monitoring machine based on the principle of radio frequency signal reception;

[0079] The optimization problem building module is used to calculate the missed detection rate and false alarm rate of the monitoring machine, and obtain the minimum false detection probability of the monitoring machine, and establish the optimization problem of maximizing the concealment rate;

[0080] Neural network building module, used to build deep deterministic policy gradient DDPG neural network;

[0081] The optimization module, based on the DDPG neural network, adopts an intelligent covert communication algorithm based on artificial interference and 3D beamforming to optimize the covert rate of the covert communication system.

[0082] A mobile terminal comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, the intelligent covert communication method based on artificial interference and 3D beamforming is implemented.

[0083] Compared with the prior art, the present invention has the following significant advantages: (1) It adopts a covert communication scheme that coordinates artificial interference and 3D beamforming, and dynamically adjusts the interference transmission beam vector and power through interactive learning between neural networks and communication environments, so as to balance the relationship between concealment performance and transmission performance, and ensure that the transmission rate of the legitimate receiver is maximized under the condition of meeting the concealment restrictions, thereby achieving joint optimization of communication performance and concealment performance; (2) It adopts a multi-antenna transmitter to improve the beamforming capability of the transmitter, so that the beam can be better focused on a specific direction, and the signal-to-noise ratio of the receiver is improved, thereby improving the covert communication rate; (3) It solves the security problems encountered in each layer of the protocol stack, and improves the concealment and security of wireless communication. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Figure 1 It is a schematic diagram of the composition structure of a multi-antenna covert communication system in an embodiment of the present invention.

[0085] Figure 2 It is a flow chart of an intelligent covert communication method based on artificial interference and 3D beamforming of the present invention.

[0086] Figure 3 It is a working flow chart of the intelligent covert communication method in an embodiment of the present invention.

[0087] Figure 4 It is a schematic diagram of the structure of the DNN network of Actor and Critic in an embodiment of the present invention.

[0088] Figure 5 It is a flowchart of trial-and-error interaction between an intelligent agent and a communication environment in an embodiment of the present invention.

[0089] Figure 6 is a curve diagram of the relationship between the concealment rate and the training time step under different power limits in an embodiment of the present invention.

[0090] Figure 7 is a curve diagram showing the relationship between the concealment rate and the training time step under different numbers of antennas in an embodiment of the present invention.

[0091] Figure 8 is a curve diagram of the relationship between the concealment rate and the training time step under different concealment limits in an embodiment of the present invention.

[0092] Fig. 9 is a curve diagram of the relationship between the concealment rate and the training time step at different learning rates in an embodiment of the present invention.

[0093] Fig.10 is a curve diagram of the relationship between the concealment rate and the training time step at different attenuation rates in an embodiment of the present invention. DETAILED DESCRIPTION

[0094] The present invention considers a multi-antenna cooperative bunker concealed communication system such as Figure 1 As shown, one of the multi-antenna transmitters uses the artificial noise emitted by the jammer as a cover to confuse the signal detection of the monitoring machine, thereby realizing the hiding of the communication process. This model considers a more practical communication scenario, assuming that the channel state information from the legitimate node to the listener is unknown, resulting in the jammer being unable to accurately aim the main beam at the listener. In order to balance the relationship between concealment performance and transmission performance, the present invention designs a covert communication scheme that collaborates with artificial interference and 3D beamforming. In order to evaluate the communication performance and concealment performance of the scheme, the present invention derives closed expressions for the transmission rate and the probability of detection error, and on this basis studies the problem of maximizing the transmission rate under the constraints of concealment performance. Taking into account the complexity of the optimization problem, the present invention proposes a joint design scheme of artificial interference and 3D beamforming based on a deep deterministic policy gradient algorithm, which dynamically adjusts the interference transmission beam vector and power through interactive learning between a neural network and the communication environment to achieve an intelligent covert communication method.

[0095] The method designed by the present invention is as follows Figure 2 As shown in the figure, it mainly includes link initialization and signal model establishment, monitoring model establishment, calculation of the minimum detection error probability of the monitor and establishment of a covert rate optimization model, construction of a deep deterministic policy gradient (DDPG) neural network, and the proposal of an intelligent covert communication method based on artificial interference and 3D beamforming. Among them, link initialization and signal model establishment are mainly based on key parameters such as channel state information and antenna gain, and a monitoring machine receiving signal model is established; in the monitoring model, the monitoring receiver uses a radiometer to compare the received power with a preset threshold (i.e., a judgment test) as a decision mechanism to determine whether the system performs covert communication; the calculation of the minimum detection error probability of the monitor is mainly based on the fact that both the transmitted signal and the interference signal obey the Gaussian distribution, and the false alarm probability and the missed detection probability are derived respectively using the law of large numbers and Lebesgue's main theorem; the DDPG neural network constructed by the present invention includes a critic network and an actor network, wherein the deep neural network (Deep Neural Networks, DNN) structure adopted by the critic network and the actor network are all fully connected deep DNNs, including an input layer, an output layer and two hidden layers; finally, the optimization problem is converted into a Markov decision problem to construct the system state, dynamics and reward function, and based on the DDPG neural network, an intelligent covert transmission scheme of artificial interference and 3D beamforming is designed to obtain an optimized covert rate.

[0096] Combination Figure 3The present invention provides an intelligent covert communication method based on artificial interference and 3D beamforming, comprising the following steps:

[0097] Step 1: Based on channel state information and antenna gain network parameters, a multi-antenna covert communication system model is established;

[0098] Step 2: Based on the principle of radio frequency signal reception, a signal detection model of the monitoring machine is established;

[0099] Step 3: Calculate the missed detection rate and false alarm rate of the monitoring machine, and obtain the minimum false detection probability of the monitoring machine, and establish the optimization problem of maximizing the concealment rate;

[0100] Step 4: Construct a deep deterministic policy gradient DDPG (Deep Deterministic Policy Gradient) neural network;

[0101] Step 5: Based on the DDPG neural network, an intelligent covert communication algorithm based on artificial interference and 3D beamforming is used to optimize the covert rate of the covert communication system.

[0102] As a specific example, in step 1, a multi-antenna covert communication system model is established based on channel state information and antenna gain network parameters, as follows:

[0103] The solution designed by the present invention is specifically implemented as follows Figure 1 The multi-antenna covert communication system shown in FIG. 1 includes a multi-antenna transmitter Alice, a single-antenna legal receiver Bob, a cooperative node jammer Jammer, and an illegal eavesdropper Willie. The goal of the transmitter is to send data covertly to Bob through 3D beamforming technology. The 3D beamforming technology can make the main beam of the transmitter focus on the direction of the receiver in three-dimensional space. On the one hand, the jammer is responsible for sending artificial interference signals as a cover for the covert signal to deceive the eavesdropper, and on the other hand, it adjusts its beamforming vector to maximize the signal to interference plus noise ratio (SINR) of the receiver Bob.

[0104] Assume that the transmitter Alice is located at the origin of the coordinate system. The positions of the jammer Jammer, the receiver Bob, and the listener Willie are defined as (x j ,y j ,z j ), (x b ,y b ,z b ) and (x w ,y w ,z w). The complex Gaussian channel vectors of Alice-Bob, Alice-Willie, Jammer-Bob and Jammer-Willie are respectively denoted as and Where 0 represents the zero matrix, N a and N j Represents the number of antennas of the transmitter and jammer, C ab , C aw , C jb , C jw The channel covariance matrices of transmitter-receiver, transmitter-monitor, jammer-receiver and jammer-monitor are respectively.

[0105] The channel adopts a time slot model, that is, the channel vector remains unchanged in each time slot and changes independently between each time slot; in each time slot, the signal vector of the transmitter and the interference signal vector of the jammer are expressed as and Where n represents the number of symbols in a time slot, the transmitter's transmitted signal is The transmission signal of the jammer is in is the beamforming vector used by the transmitter to transmit the message, is the beamforming vector used by the jammer to transmit the jamming signal, l = 1, 2, ..., n, and Where E{} is the expectation operator, and Respectively represent the dimension N in the complex field a ×1 and N j ×1 matrix, N a and N j Represent the number of antennas of the transmitter and the jammer respectively.

[0106] For array antennas, the vertical and horizontal modes of the transmitting antenna have different antenna gains in dB, and the attenuation of the horizontal antenna is Among them m and θ 3dB are the maximum attenuation and horizontal 3D beamwidth of the transmitter antenna, θ0 represents the horizontal reference angle of the transmitter, and the horizontal angle of node m relative to the reference direction. x m and m They represent the horizontal and vertical coordinates of node m in the horizontal coordinate system respectively; the attenuation of the vertical antenna is in and are the vertical reference angle and vertical 3D beam width of the transmitter antenna; the vertical direction angle of node m relative to the transmitter Δz m is the height difference between the transmitter and node m, and the antenna gain of node m for:

[0107]

[0108] Where Ω max is the maximum antenna gain in the reference direction;

[0109] Given Then the linear antenna gain of node m is:

[0110]

[0111] in is the vertical reference angle of the transmitter antenna;

[0112] According to the above, the lth received signal of node m in each time slot is expressed as:

[0113]

[0114] Among them, α, d am ,d jm 、(·) H denote the path loss index, the distance between the transmitter and the node m, the distance between the jammer and the node m, and the conjugate transpose operator, respectively; $ is the additive white noise received by node m; ψ0 means that the transmitter does not send hidden information to the receiver, and ψ1 means that the transmitter sends hidden information to the receiver; p a represents the transmission power of the transmitter, and They represent the channel gain coefficients between the jammer and node m and the transmitter and node m respectively;

[0115] The transmitter has known the channel state information of the receiver, so the transmitter can use the maximum ratio transmission technology, that is: w a =h ab / |h ab |.

[0116] As a specific example, step 2 is implemented as follows:

[0117] In this system, the main task of the listener is to determine whether there is information being transmitted between the transmitter and the receiver by detecting the received signal strength. To achieve this goal, the listener is equipped with a radiometer to monitor the received power of the signal in real time. The core of signal detection is to determine whether the received signal exceeds a preset threshold. This judgment is achieved by comparing the total received power of the sampled signal with the threshold. Figure 3 As shown, the steps are as follows:

[0118] Step 2.1: When performing detection, the listener first collects signal samples within a certain time window and calculates the power average value Y of these samples. w ;

[0119] Step 2.2: Based on the binary hypothesis test theory, the listener compares the calculated power average with the preset detection threshold v. Based on the comparison result, the listener makes a judgment on whether there is covert communication in the system. Therefore, the listener's judgment test is expressed in mathematical expression as:

[0120]

[0121] in Indicates the total power of the received signal of the monitoring machine. Indicates the judgment threshold of the monitoring machine. represents the tth symbol received by the monitor w;

[0122] Step 2.3: Based on the comparison result, the monitoring machine determines whether there is covert communication in the system. When the total received power exceeds the threshold, it is determined that information is transmitted from the transmitter to the receiver, that is, the ψ1 situation. On the contrary, when the total received power is lower than the threshold, it is determined that the transmitter does not transmit information to the receiver, that is, the ψ0 situation.

[0123] Step 2.4: Based on the above assumptions, when the transmitter sends hidden information to the receiver in real situation, but Y w ≤v and the listener makes ψ0. At this time, Miss Detection (MD) occurs, and its probability can be expressed as P MD ; When the transmitter Alice does not transmit data to the receiver Bob, but Y w >v and the listener makes a judgment of ψ1. At this time, a false alarm (FA) occurs, and the false alarm probability is expressed as P FA In summary, the false alarm probability and missed detection probability constitute the detection error probability P of the monitoring machine. error , denoted as P error =P FA +P MD ; For any positive number ∈, when n→∞, P FA +PMD ≤1-∈, the communication between the transmitter and the receiver is considered to be hidden.

[0124] As a specific example, the missed detection rate and false alarm rate of the monitoring machine are calculated in step 3, and the minimum false detection probability of the monitoring machine is obtained, and the optimization problem of maximizing the concealment rate is established, which is specifically as follows:

[0125] Step 3.1: Calculate the false alarm probability of the monitoring machine and the probability of missed detection The formula is:

[0126]

[0127] in, is the judgment threshold of the monitoring machine, is the noise variance, is a random chi-square variable with 2n degrees of freedom, and γ is the interference power received by the monitoring station, expressed as:

[0128]

[0129] Among them, d jw is the distance from the jammer to the monitor, d aw is the distance from the transmitter to the jammer, is the antenna gain of the transmitter to the monitor, p a is the transmitting power of the transmitter, h ab / ||h ab || is the beamforming vector ω of the transmitter antenna a ;h aw and h jw They represent the channel gain coefficients of transmitter-monitor and jammer-monitor, respectively. represents the beamforming vector of the jammer.

[0130] Step 3.2: According to Lebesgue's convergence theorem, when n→∞, replace The false alarm probability and missed detection probability are re-expressed as:

[0131]

[0132]

[0133] make Then the probability density function f of γ is Γ (γ) is expressed as:

[0134]

[0135] Step 3.3: According to the definition of false alarm probability and missed detection probability, P is obtained through integral calculation.FA and P MD The closed expression of is:

[0136]

[0137] in is the judgment threshold of the monitoring machine, is the noise variance;

[0138] In order to satisfy P FA +P MD =1, the monitoring machine only considers situation;

[0139] Step 3.4, order The optimal detection threshold is:

[0140]

[0141] Step 3.5: For the multi-antenna system, the concealed transmission rate is maximized by optimizing the transmission power and 3D beamforming. The optimization problem of maximizing the concealed rate is established as follows:

[0142]

[0143] in, is the probability of data transmission in each time slot, Indicates the transmitter's transmit power limit. represents the interference power limit of the jammer, Tr(·) represents the trace operator, min(P FA +P MD )≥1-ε represents the system concealment constraint, ε represents the concealment constraint threshold, and Represent the maximum power limit of the transmitter and the jammer respectively, w j represents the beamforming vector of the jammer.

[0144] As a specific example, step 4 is implemented as follows:

[0145] Build as Figure 4 The Critic network and Actor network structures shown in the figure use a fully connected deep neural network, including an input layer, an output layer and two hidden layers. The input dimension of the Critic network is equal to the cardinality of the state set and the action set, and the output value is the Q value function. The input and output dimensions of the Actor network are defined as the cardinality of the state and action, respectively. The number of neurons in the hidden layer depends on the number of antennas of the transmitter Alice and the jammer Jammer. The number of neurons in the hidden layer must be greater than the input and output dimensions. The action output by the Actor network will be input into the hidden layer 2 for calculation Get the loss function and gradient, Δ q is the gradient calculation symbol, is the state-action value function, represents the weight and bias parameters of the neural network, s (t) Represents the network state at time t, and α represents the action taken. It should be noted that the correlation between the state s items will reduce the efficiency of the neural network approximation function. In order to solve this problem, the state s is whitened before being input into the Critic network and the Actor network to eliminate the correlation between the state s items. In order to overcome the changes in the input distribution of each layer due to the parameter changes of the previous layer, the hidden layer needs to be batch normalized. The batch normalization operation allows a higher learning rate and has low sensitivity to initialization. In the present invention, the tanh function is used as the activation function to handle negative inputs.

[0146] The Adam algorithm is used to optimize the train-critic network and the train-actor network. The adaptive learning rate is determined by the following formula:

[0147]

[0148] Among them, λ c and λ a are the decay rates of the train-critic network and the train-actor network, and Represent the learning rate of the train-critic network at time t and t-1 respectively, and Represent the learning rate of the train-actor network at time t and t-1 respectively.

[0149] The action value output by the Actor network It is necessary to satisfy the constraints defined in the optimization problem of maximizing the concealment rate, so a normalization layer is added at the output of the Actor network, where Make sure the vertical angle of the transmitter antenna does not exceed 90°.

[0150] As a specific example, the DDPG neural network described in step 5 uses an intelligent covert communication algorithm based on artificial interference and 3D beamforming to optimize the covert rate of the covert communication system, as follows:

[0151] This invention is based on the idea of ​​deep reinforcement learning. The intelligent agent continuously applies actions to the environment through trial-and-error interaction with the communication environment, observes the immediate rewards and state transitions of the environment, and gradually derives the best actions, such as Figure 5In order to better describe the intelligent covert transmission method of the present invention, the following states, actions, and instant rewards are constructed.

[0152] Step 5.1: Construct state, action, and immediate reward;

[0153] (1) State: The state s(t) at time step t is composed of the action a(t-1) obtained from the previous time step, the channel vector (h ab ,h jb ,h aw ,h jw ); Since the neural network can only accept real numbers but not complex numbers as input, if complex numbers are involved when constructing the state s, the real part and the imaginary part will be used as independent input units respectively; Since all inter-node channels obey the complex Gaussian distribution, the channel vector should be split into two terms, the real part Re and the imaginary part Im, that is, h ab =Re{h ab}+Im{h ab}, both parts are used as independent input items of the Critic network and the Actor network; since both the receiver and the monitor are single-antenna receivers, h ab With h aw The vector size is equal to the number of transmitter antennas M, and h jb With h jw The vector size is equal to the number of antennas N of the jammer; and in action, the beamforming vector w j The size of is also equal to the number of antennas N of the jammer, and is divided into real and imaginary parts. Therefore, the total number of items constructed from the actions in the t-1 time step is 2*K+2, where K is the maximum number of training steps; in summary, the dimension of the state space D s =2*(2*M+2*K)+2*K+2;

[0154] (2) Action: The action is the jammer transmitting the beamforming vector w j , Transmitter transmission power P a and the vertical angle of the transmitter antenna Composition, that is Similarly, to solve the real input problem, the jammer transmit beamforming vector w is j =Re{w j}+Im{w j} is split into two parts, the real part and the imaginary part, which are used as the input unit of the action. The dimension of the action space is D s =2*K+2;

[0155] (3) Reward: At the tth step of training, the reward is set to the reward for taking an action from the Actor network given the channel state information. The resulting system concealed rate capacity.

[0156] Step 5.2, Initialization: First initialize the experience replay buffer of size D Used to store training samples, ensure the diversity of empirical data and reduce the correlation between samples; then initialize the neural network parameters, including the train-actor network parameters target-actor network parameters Train-critic network parameters Target-critic network parameters The jammer is used to obtain the beamforming vector w of the jammer signal j Initialized as a complex vector of all 1s, the transmitter's transmission power P a Initialized to 0.1W, the vertical angle of the transmitter antenna Initialized to 0°;

[0157] Step 5.3, state construction: At time step t, the action and channel information (h ab ,h jb ,h aw ,h jw )Construct the current state s(t);

[0158] Step 5.4, action selection: The Actor network receives the state s(t) as input and generates a deterministic policy action a(t);

[0159] Step 5.5, reward calculation: According to the action output by the Actor network and the current channel information, calculate the immediate reward r(t), which is related to the covert communication rate. The goal is to maximize Bob's receiving rate while meeting the concealment constraint.

[0160] Step 5.6, network update: Use the Critic network to evaluate the actions of the actor network, calculate the corresponding Q value function, and use the stochastic gradient descent SGD method to update the parameters of the Critic network. The learning rate u in the update process c At the same time, based on the feedback of the Critic network, by adjusting the parameters of the Actor network Maximize the Q-value function and use SGD to update the learning rate u a ;

[0161] Step 5.7, Experience Replay: At each time step, store the current state s(t), a(t), r(t+1) and the next state s(t+1) into the experience replay buffer And randomly extract samples of size W from the buffer for network training;

[0162] Step 5.8, target network update: update the parameters of the target critic network and the target actor network every set number of steps U.

[0163] Through the above steps and training, the algorithm can adaptively learn the optimal covert communication strategy in a complex wireless communication environment, ensuring that the transmission rate of the legitimate receiver is maximized while meeting the concealment constraints. Therefore, the intelligent covert communication algorithm proposed in the present invention is summarized as Algorithm 1.

[0164] The process of the intelligent covert communication algorithm proposed by the present invention is as follows:

[0165] Table 1

[0166]

[0167] The following simulation experiments are conducted to analyze the concealment performance under various communication environments and network learning parameters to verify the effectiveness of the intelligent covert communication method proposed in the present invention.

[0168] Example

[0169] This embodiment uses simulation experiments to verify the performance of the intelligent covert communication method of the present invention. The simulation results are as follows: Figure 6 to Figure 10 In the simulation, all communication links are set as Rayleigh fading channels, and the channel coefficients obey a complex Gaussian distribution with a mean of 0 and a variance of 1. The communication system parameter settings involved are shown in Table 2:

[0170] Table 2

[0171]

[0172] Figure 6 The influence of power limitation on the average concealment rate, i.e., the average reward, is given. Four cases are shown, and the average hiding rate and the training time step are plotted. In data processing, the average reward is calculated using the following method to represent the average hiding rate:

[0173]

[0174] Where K is the maximum number of training steps.

[0175] from Figure 6 It can be seen that the average concealment rate will gradually converge with the increase of the time step. In the case of high jammer transmission power The average stealth rate will be lower than when the power is lower The convergence is faster in the case of , because the lower jammer transmit power makes the signal-to-noise ratio at the receiver higher. At a higher signal-to-noise ratio, the dynamic range of the instantaneous reward is larger, resulting in more fluctuations and poorer convergence. In the case of high transmitter power limit Compared to lower power The average concealment rate that can be obtained is higher. The results show that starting from the initial value, the proposed algorithm can learn from the environment and adjust the output action W j , p a and q a , so that the concealment rate value is close to the optimal solution.

[0176] Figure 7 The influence of the number of transmitter antennas and the number of jammer antennas on the average concealment rate is given. a =N j =4,N a =4,N j =8,N a =8,N j =4 and N a =N j = 8 four system settings, and plotted the relationship between the average concealment rate and the training time step. Figure 7 It can be seen from the figure that when the number of transmitter antennas is fixed (N a =4,8), when the number of jammer antennas is larger (N j =8) will lead to faster convergence and higher average concealment rate; when the number of jammer antennas is fixed (N j =4,8), when the number of transmitter antennas is larger (N a =8) leads to a higher average concealment rate. In general, the increase in the number of antennas has a greater effect on the improvement of the transmitter beamforming capability, which makes the beam better focused on a specific direction, improves the signal-to-noise ratio at the receiver, and thus improves the concealed communication rate.

[0177] Figure 8 The influence of the concealment limitation in the communication system on the communication rate is given. For the optimization problem of maximizing the concealment rate, according to its constraints, it can be seen that the parameter e ensures the concealment of the communication behavior between the transmitter and the receiver by stipulating the minimum detection error probability value of the monitor, and the smaller the e value, the higher the detection error rate of the monitor, that is, the stronger the communication concealment; otherwise, the lower. Figure 8It can be seen from the figure that for a fixed power and number of antennas, when the parameter e is small, the concealment of the system communication is higher, the signal-to-noise ratio at the receiver is reduced, and the concealment rate is reduced; when the parameter e is large, the concealment of the system communication is lower, it is easier to be eavesdropped by the monitoring machine, and the concealment rate will also be reduced. Therefore, in this simulation experiment, the concealment constraint is set to ε = 0.10.

[0178] Fig. 9 and Fig.10 The effects of different learning rates and decay rates on the performance and convergence speed of the DRL method are given. Fig. 9 The relationship between the average hidden rate and the time step under different learning rates (0.01, 0.001, 0.0001, 0.0005) is shown. It can be seen that different learning rates have a great impact on the performance of the DRL algorithm. Specifically, it takes a long time to converge when using a learning rate of 0.0005, but it achieves the best performance. The learning rates of 0.001 and 0.0001 converge faster, while too large a learning rate (such as 0.01) performs poorly. This is because too large a learning rate will increase oscillations, resulting in a sharp drop in performance. Therefore, the learning rate should choose an appropriate value to better improve the performance of the neural network. Fig.10 The relationship between the average hidden rate and the time step under different decay rates (0.001, 0.0001, 0.00001, 0.000001) is compared. The decay rate has similar conclusions to the learning rate, but has less impact on the performance and convergence speed of DRL. It can be seen that when the training decay rate is large, the three values ​​except the decay rate of 0.001 can almost achieve the best performance, and the gap between them is small, but the convergence curve of the decay rate of 0.00001 is smoother.

[0179] Finally, it should be pointed out that the performance of deep reinforcement learning-based algorithms is very sensitive to DNN initialization and other hyperparameters, such as sampling batch size. Hyperparameters need to be accurately defined under given system settings. Appropriate neural network hyperparameter settings will significantly improve the performance and convergence speed of deep reinforcement learning algorithms.

[0180] The above are only preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An intelligent covert communication method based on artificial interference and 3D beamforming, characterized in that: The following steps are involved: Step 1: Based on channel state information and antenna gain network parameters, a multi-antenna covert communication system model is established; Step 2: Based on the principle of radio frequency signal reception, a signal detection model of the monitoring machine is established; Step 3: Calculate the missed detection rate and false alarm rate of the monitoring machine, and obtain the minimum false detection probability of the monitoring machine, and establish the optimization problem of maximizing the concealment rate; Step 4: Construct a deep deterministic policy gradient DDPG neural network; Step 5: Based on the DDPG neural network, an intelligent covert communication algorithm based on artificial interference and 3D beamforming is used to optimize the covert rate of the covert communication system.

2. The intelligent covert communication method based on artificial interference and 3D beamforming according to claim 1 is characterized in that: Based on the channel state information and antenna gain network parameters described in step 1, a multi-antenna covert communication system model is established, as follows: The multi-antenna covert communication system includes a multi-antenna transmitter, a single-antenna legal receiver, a cooperative node jammer and an illegal monitoring machine; The goal of the transmitter is to covertly send data to the receiver through 3D beamforming technology, and use 3D beamforming technology to focus the main beam of the transmitter in the direction of the receiver in three-dimensional space; the jammer sends artificial interference signals as a cover for the hidden signal to deceive the monitoring machine, and adjusts the beamforming vector to improve the signal-to-interference-plus-noise ratio (SINR) of the receiver. Assume that the transmitter is located at the origin of the coordinate system, and the positions of the jammer, receiver, and monitor in the three-dimensional coordinates are (x j ,y j ,z j ), (x b ,y b ,z b ) and (x w ,y w ,z w ); the complex Gaussian channel vectors of the transmitter-receiver, transmitter-monitor, jammer-receiver and jammer-monitor are respectively denoted as and Where 0 represents the zero matrix, N a and N j Represents the number of antennas of the transmitter and jammer, C ab , C aw , C jb , C jw The channel covariance matrices of transmitter-receiver, transmitter-monitor, jammer-receiver and jammer-monitor, respectively; The channel adopts a time slot model, that is, the channel vector remains unchanged in each time slot and changes independently between each time slot; in each time slot, the signal vector of the transmitter and the interference signal vector of the jammer are expressed as and Where n represents the number of symbols in a time slot, the transmitter's transmitted signal is The transmission signal of the jammer is in is the beamforming vector used by the transmitter to transmit the message, is the beamforming vector used by the jammer to transmit the jamming signal, l = 1, 2, ..., n, and Where E{} is the expectation operator, and Respectively represent the dimension N in the complex field a ×1 and N j ×1 matrix, N a and N j Represent the number of antennas of the transmitter and jammer respectively; For array antennas, the vertical and horizontal modes of the transmitting antenna have different antenna gains in dB, and the attenuation of the horizontal antenna is Among them m and θ 3dB are the maximum attenuation and horizontal 3D beamwidth of the transmitter antenna, θ0 represents the horizontal reference angle of the transmitter, and the horizontal angle of node m relative to the reference direction. x m and m They represent the horizontal and vertical coordinates of node m in the horizontal coordinate system respectively; the attenuation of the vertical antenna is in and are the vertical reference angle and vertical 3D beam width of the transmitter antenna; the vertical direction angle of node m relative to the transmitter Δz m is the height difference between the transmitter and node m, and the antenna gain of node m for: Where Ω max is the maximum antenna gain in the reference direction; Given Then the linear antenna gain of node m is: in is the vertical reference angle of the transmitter antenna; In summary, the lth received signal of node m in each time slot is expressed as: Among them, α, d am d jm 、(·) H denote the path loss index, the distance between the transmitter and the node m, the distance between the jammer and the node m, and the conjugate transpose operator, respectively; $ is the additive white noise received by node m; ψ0 means that the transmitter does not send hidden information to the receiver, and ψ1 means that the transmitter sends hidden information to the receiver; p a represents the transmission power of the transmitter, and They represent the channel gain coefficients between the jammer and node m and the transmitter and node m respectively; The transmitter has known the channel state information of the receiver, so the transmitter can use the maximum ratio transmission technology, that is: w a =h ab / |h ab |。 3. The intelligent covert communication method based on artificial interference and 3D beamforming according to claim 2 is characterized in that: Based on the principle of RF signal reception as described in step 2, a signal detection model of the monitoring machine is established, wherein the monitoring machine is equipped with a radiometer for real-time monitoring of the received power of the signal. By detecting whether the received signal strength exceeds a preset threshold, it is determined whether there is information being transmitted between the transmitter and the receiver. The specific steps are as follows: Step 2.1: The monitor collects signal samples within the set time window and calculates the power average value Y of these samples. w ; Step 2.2, based on the binary hypothesis test theory, the monitoring machine compares the calculated power average value with the preset detection threshold v; the mathematical expression used for the judgment test of the monitoring machine is: in Indicates the total power of the received signal of the monitoring machine. Indicates the judgment threshold of the monitoring machine. represents the tth symbol received by the monitor w; Step 2.3: Based on the comparison result, the monitoring machine determines whether there is covert communication in the system. When the total received power exceeds the threshold, it is determined that information is transmitted from the transmitter to the receiver, that is, the ψ1 situation. On the contrary, when the total received power is lower than the threshold, it is determined that the transmitter does not transmit information to the receiver, that is, the ψ0 situation. Step 2.4: In the real situation, the transmitter sends a hidden message to the receiver, but Y w ≤v and the monitoring machine makes ψ0, then missed detection occurs, and the missed detection probability is expressed as P MD ; When the transmitter does not transmit data to the receiver, but Y w >v and the monitoring machine makes a judgment of ψ1, a false alarm occurs, and the false alarm probability is expressed as P FA The false alarm probability and missed detection probability constitute the detection error probability P of the monitoring machine. error , denoted as P error =P FA +P MD ; For any positive number ∈, when n→∞, P FA +P MD ≤1-∈, the communication between the transmitter and the receiver is considered to be hidden.

4. The intelligent covert communication method based on artificial interference and 3D beamforming according to claim 3 is characterized in that: Step 3 calculates the missed detection rate and false alarm rate of the monitoring machine, and obtains the minimum false detection probability of the monitoring machine, and establishes the optimization problem of maximizing the concealment rate, which is as follows: Step 3.1: Calculate the false alarm probability of the monitoring machine and the probability of missed detection The formula is: in, is the judgment threshold of the monitoring machine, is the noise variance, is a random chi-square variable with 2n degrees of freedom, and γ is the interference power received by the monitoring station, expressed as: Among them, d jw is the distance from the jammer to the monitor, d aw is the distance from the transmitter to the jammer, is the antenna gain of the transmitter to the monitor, p a is the transmitting power of the transmitter, h ab / ||h ab || is the beamforming vector ω of the transmitter antenna a ;h aw and h jw They represent the channel gain coefficients of transmitter-monitor and jammer-monitor, respectively. represents the beamforming vector of the jammer; Step 3.2: According to Lebesgue's convergence theorem, when n→∞, replace The false alarm probability and missed detection probability are re-expressed as: make Then the probability density function f of γ is Γ (γ) is expressed as: Where γ is the interference power received by the monitor; Step 3.3: According to the definition of false alarm probability and missed detection probability, P is obtained through integral calculation. FA and P MD The closed expression of is: in is the judgment threshold of the monitoring machine, is the noise variance; In order to satisfy P FA +P MD =1, the monitoring machine only considers situation; Step 3.4, order The optimal detection threshold is: Step 3.5: For the multi-antenna system, the concealed transmission rate is maximized by optimizing the transmission power and 3D beamforming. The optimization problem of maximizing the concealed rate is established as follows: in, is the probability of data transmission in each time slot, Indicates the transmitter's transmit power limit. represents the interference power limit of the jammer, Tr(·) represents the trace operator, min(P FA +P MD )≥1-ε represents the system concealment constraint, ε represents the concealment constraint threshold, and Represent the maximum power limit of the transmitter and the jammer respectively, w j represents the beamforming vector of the jammer.

5. The intelligent covert communication method based on artificial interference and 3D beamforming according to claim 4 is characterized in that: The construction of the deep deterministic policy gradient DDPG neural network described in step 4 is as follows: A fully connected deep neural network is used to construct the Critic network and Actor network structures, which include an input layer, an output layer and two hidden layers respectively. The input dimension of the Critic network is equal to the cardinality of the state set and the action set, and the output value is the Q value function. The input and output dimensions of the Actor network are defined as the cardinality of the state and action respectively. The number of neurons in the hidden layer depends on the number of antennas of the transmitter and the jammer. The number of neurons in the hidden layer must be greater than the input and output dimensions. The action output by the Actor network will be input into the hidden layer 2 for calculation. Get the loss function and gradient, Δ q is the gradient calculation symbol, is the state-action value function, represents the weight and bias parameters of the neural network, s (t) represents the network status at time t, and α represents the action taken; Before being input into the Critic network and the Actor network, the state s is whitened to eliminate the correlation between the items in the state s; Perform batch normalization on the hidden layers to overcome the changes in the input distribution of each layer caused by the parameter changes of the previous layer; use the tanh function as the activation function to handle negative inputs; The Adam algorithm is used to optimize the train-critic network and the train-actor network. The adaptive learning rate is determined by the following formula: Among them, λ c and λ a are the decay rates of the train-critic network and the train-actor network, and Represent the learning rate of the train-critic network at time t and t-1 respectively, and Represent the learning rate of the train-actor network at time t and t-1 respectively; The action value output by the Actor network It is necessary to satisfy the constraints defined in the optimization problem of maximizing the concealment rate, so a normalization layer is added at the output of the Actor network, where Make sure the vertical angle of the transmitter antenna does not exceed 90°.

6. The intelligent covert communication method based on artificial interference and 3D beamforming according to claim 5 is characterized in that: Based on the DDPG neural network described in step 5, an intelligent covert communication algorithm based on artificial interference and 3D beamforming is used to optimize the covert rate of the covert communication system, as follows: Step 5.1: Construct state, action, and immediate reward; (1) State: The state s(t) at time step t is composed of the action a(t-1) obtained from the previous time step, the channel vector (h ab ,h jb ,h aw ,h jw ); Since the neural network can only accept real numbers but not complex numbers as input, if complex numbers are involved when constructing the state s, the real part and the imaginary part will be used as independent input units respectively; Since all inter-node channels obey the complex Gaussian distribution, the channel vector should be split into two terms, the real part Re and the imaginary part Im, that is, h ab =Re{h ab }+Im{h ab }, both parts are used as independent input items of the Critic network and the Actor network; since both the receiver and the monitor are single-antenna receivers, h ab With h aw The vector size is equal to the number of transmitter antennas M, and h jb With h jw The vector size is equal to the number of antennas N of the jammer; and in action, the beamforming vector w j The size of is also equal to the number of antennas N of the jammer, and is divided into real and imaginary parts. Therefore, the total number of items constructed from the actions in the t-1 time step is 2*K+2, where K is the maximum number of training steps; in summary, the dimension of the state space D s =2*(2*M+2*K)+2*K+2; (2) Action: The action is the jammer transmitting the beamforming vector w j , Transmitter transmission power P a and the vertical angle of the transmitter antenna Composition, that is Similarly, to solve the real input problem, the jammer transmit beamforming vector w is j =Re{w j }+Im{w j } is split into two parts, the real part and the imaginary part, which are used as the input unit of the action. The dimension of the action space is D s =2*K+2; (3) Reward: At the tth step of training, the reward is set to the reward for taking an action from the Actor network given the channel state information. The resulting system concealed rate capacity; Step 5.2, Initialization: First initialize the experience replay buffer of size D Used to store training samples, ensure the diversity of empirical data and reduce the correlation between samples; then initialize the neural network parameters, including the train-actor network parameters target-actor network parameters Train-critic network parameters Target-critic network parameters The jammer is used to obtain the beamforming vector w of the jammer signal j Initialized as a complex vector of all 1s, the transmitter's transmission power P a Initialized to 0.1W, the vertical angle of the transmitter antenna Initialized to 0°; Step 5.3, state construction: At time step t, the action and channel information (h ab ,h jb ,h aw ,h jw )Construct the current state s(t); Step 5.4, action selection: The Actor network receives the state s(t) as input and generates a deterministic policy action a(t); Step 5.5, reward calculation: According to the action output by the Actor network and the current channel information, calculate the immediate reward r(t), which is related to the covert communication rate. The goal is to maximize Bob's receiving rate while meeting the concealment constraint. Step 5.6, network update: Use the Critic network to evaluate the actions of the actor network, calculate the corresponding Q value function, and use the stochastic gradient descent SGD method to update the parameters of the Critic network. The learning rate u in the update process c At the same time, based on the feedback of the Critic network, by adjusting the parameters of the Actor network Maximize the Q-value function and use SGD to update the learning rate u a ; Step 5.7, Experience Replay: At each time step, store the current state s(t), a(t), r(t+1) and the next state s(t+1) into the experience replay buffer And randomly extract samples of size W from the buffer for network training; Step 5.8, target network update: update the parameters of the target critic network and the target actor network every set number of steps U.

7. An intelligent covert communication device based on artificial jamming and 3D beamforming, characterized in that: The device is used to implement the intelligent covert communication method based on artificial interference and 3D beamforming as described in any one of claims 1 to 6, and the device includes a covert communication system model building module, a signal detection model building module, an optimization problem building module, a neural network building module and an optimization module, wherein: A covert communication system model building module builds a multi-antenna covert communication system model based on channel state information and antenna gain network parameters; The signal detection model building module builds the signal detection model of the monitoring machine based on the principle of radio frequency signal reception; The optimization problem building module is used to calculate the missed detection rate and false alarm rate of the monitoring machine, and obtain the minimum false detection probability of the monitoring machine, and establish the optimization problem of maximizing the concealment rate; Neural network building module, used to build deep deterministic policy gradient DDPG neural network; The optimization module, based on the DDPG neural network, adopts an intelligent covert communication algorithm based on artificial interference and 3D beamforming to optimize the covert rate of the covert communication system.

8. A mobile terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the intelligent covert communication method based on artificial interference and 3D beamforming as described in any one of claims 1 to 5 is implemented.

Images