Railway Internet of Things covert communication method and system based on deep reinforcement learning

Abstract

The invention discloses a railway Internet of Things covert communication method and system based on deep reinforcement learning, and belongs to the technical field of communication security, and the method comprises the steps: adding an intelligent metasurface in a railway Internet of Things communication system, and constructing a high-speed railway Internet of Things covert communication model based on RIS-MIMO; constructing a channel model of the railway Internet of Things covert communication system based on the time-varying characteristics of the channel; according to the channel model and the received signal model based on hypothesis testing, considering the priority of the railway Internet of Things communication service, and designing a hidden constraint condition; a joint optimization problem is formed by taking the maximum hidden throughput as a target and taking the designed hidden condition, the total power of the base station and the RIS phase shift as constraints. According to the method, after a low-complexity deep reinforcement learning framework is provided and joint optimization is implemented, hidden transmission of a railway Internet of Things communication system can be more reliable, meanwhile, power resources are saved, the cost is low, and deployment is easy.

Description

Technical Field

[0001] The present invention relates to the field of communication security technology, and in particular to a railway Internet of Things covert communication method and system based on deep reinforcement learning. Background Art

[0002] In the intelligent high-speed rail communication system enabled by the fifth generation of mobile communication technology (5G), the scale of IoT devices and the amount of data transmission will explode. However, in order to meet the needs of free movement of IoT terminals, the wireless networking method and the broadcast nature of wireless channels make 5G railway (5G-R) and the upcoming 6G-R vulnerable to interference and attacks by eavesdropping devices. The data generated by railway IoT devices includes confidential business data and user information. Once leaked, it will cause significant damage to the privacy of enterprises and users.

[0003] Although upper-layer encryption technology and physical layer security technology based on cryptography can protect data transmission to a certain extent, this means of fundamentally ensuring that information is not interpreted by illegal eavesdroppers will be in jeopardy with the advent of quantum computing and other supercomputing technologies. In fact, the secure transmission of private data not only includes the content of communication, but also the hiding of communication behavior itself is of great significance. Only undetected communication behavior can prevent it from being interfered with or interpreted. To this end, wireless covert communication technology, with its special advantage of being able to hide communication behavior, provides a solution to the concealment requirements of data communication in the railway Internet of Things. However, in the field of high-speed rail communication technology, eavesdroppers usually use high-precision energy meters for detection. In the prior art, a secure communication method that can not only ensure the quality of the received signal of the train in the 5G-R network, but also achieve reliable and covert transmission and effectively reduce the operating costs of the communication network has not yet been found.

[0004] Recently, Reconfigurable Intelligent Surface (RIS) has attracted wide attention due to its ability to independently reflect incident electromagnetic waves with adjustable phase shift, thereby enabling intelligent wireless environments to improve the quality of legal communications and protect the receiving end from detection by illegal devices. In addition, RIS-assisted wireless communication networks also have the characteristics of low cost, low power consumption, and simple structure, and have been widely used in the field of information security.

[0005] In existing research, researchers are more concerned about how to improve the overall concealed throughput of RIS-assisted high-speed rail communication systems, while ignoring the priority of services in the railway Internet of Things. It is definitely inappropriate to allocate the same power resources and concealment levels to railway services with different priorities, which will cause a waste of resources. For example: for high-priority services (railway emergency calls, emergency voice communications, etc.), a higher concealed transmission level should be allocated to ensure the safety of train operation. In addition, in high-speed mobile scenarios, channels change quickly and timeliness requirements are high. Existing technologies and methods are difficult to accurately obtain channel status information, resulting in serious loss of system performance. Summary of the invention

[0006] The purpose of the present invention is to provide a railway Internet of Things covert communication method and system based on deep reinforcement learning, which solves the problem of secure transmission of railway Internet of Things data under fast time-varying channels.

[0007] To achieve the above object, the present invention provides the following solutions:

[0008] A railway Internet of Things covert communication method based on deep reinforcement learning, comprising:

[0009] Add intelligent metasurface RIS to the railway IoT communication system and build a high-speed railway IoT covert communication model based on RIS-MIMO;

[0010] According to the RIS-MIMO-based high-speed rail IoT covert communication model, the channel model of the railway IoT covert communication system is constructed by considering the Doppler frequency shift compensation caused by the high-speed movement of the train and the channel uncertainty introduced by artificial noise;

[0011] According to the communication model of the railway Internet of Things covert communication system, a received signal model based on hypothesis testing is constructed;

[0012] Obtaining hidden constraint conditions according to the received signal model and railway communication service priority;

[0013] Combined with the concealment constraints, with the goal of maximizing the concealment throughput, a joint optimization problem of the beamforming at the transmitting and receiving ends and the phase shift of the smart metasurface RIS is constructed;

[0014] According to the time-varying nature of the high-speed rail channel, the joint optimization problem is transformed into a Markov decision process problem;

[0015] Based on the transformed Markov decision process problem, a deep reinforcement learning framework is used to deal with the non-convexity and coupling of the problem, obtain the optimal solution to the problem, and ensure reliable and low-cost covert communication in fast time-varying channel scenarios.

[0016] Optionally, the high-speed rail Internet of Things covert communication model based on RIS-MIMO includes:

[0017] A train moves from the center of the coverage area of ​​the base station BS to the edge of the coverage area, a trackside base station equipped with multiple antennas and a large uniform planar array UPA, the smart metasurface RIS deployed between the base station BS and the train, the eavesdropper Willie and the trackside buildings. It is assumed that a mobile relay MR is installed on the top of the train for communicating with the base station BS.

[0018] Optionally, considering the Doppler frequency shift compensation caused by the high-speed movement of the train and the channel uncertainty introduced by artificial noise, the channel model of the railway IoT covert communication system is constructed, including:

[0019] According to the RIS-MIMO-based high-speed rail IoT covert communication model, the base station BS, the intelligent metasurface RIS, and the eavesdropper Willie are fixed in position, and the mobile relay MR moves with the train. The channel matrix H of the BS-RIS link is BR , the channel matrix W of the BS-Willie link BW and the channel matrix W of the RIS-Willie link RW The channel matrix G of the BS-MR link is constructed as a quasi-static channel without Doppler shift. BM , the channel matrix H of the RIS-MR link RM And the channel matrix W of the MR-Willie link MW It is constructed as a time-varying channel containing Doppler frequency shift;

[0020] The construction of the time-varying channel is to represent the Doppler frequency shift through the Clarke-Jakes spectrum. For any time-varying channel H[t] at any time, let and H[t+Ts] represent the outdated channel state information CSI and the real-time channel state information CSI, respectively. The relationship between and H[t+Ts] is:

[0021]

[0022] In the formula, κ=J0(2πf D T S ) represents the time correlation coefficient, f D is the Doppler frequency shift, J0(·) is the first kind of zero-order Bessel function, ΔH is the error term, t is any time during the train operation, and Ts is the transmission delay;

[0023] The mobile relay MR generates interference signals with different powers, and the self-interference channel of the mobile relay MR is f MCN(0,φ), φ∈[0,1] is the self-interference cancellation coefficient, CN is the symbolic representation of the complex Gaussian distribution, assuming that the interference signal transmission power P of the mobile relay MR Mi Obey the interval [0, P Mi,max ], the probability density function is:

[0024]

[0025] Where P Mi,max is the maximum self-interference transmission power, x is [0, P Mi,max ] on different values.

[0026] Optionally, constructing a received signal model based on hypothesis testing includes: constructing a received signal model at the eavesdropper Willie;

[0027] The receiving signal model at the eavesdropper Willie is:

[0028]

[0029] Among them, Y W is the receiving signal model of the eavesdropper Willie, W RW The conjugate transpose of , Θ is the reflection coefficient matrix of RIS, s M is the self-interference signal from the mobile relay MR, Z W is the additive Gaussian white noise at the eavesdropper Willie, H0 indicates that no covert communication occurs, M is the receiving beamforming matrix at the mobile relay MR, W BW The conjugate transpose of H is the conjugate transpose of F, F is the transmit beamforming matrix at the base station BS, s B is a covert signal from the BS, and H1 indicates that covert communication occurs.

[0030] Optionally, obtaining the hidden constraint condition according to the received signal model and the railway communication service priority includes:

[0031] According to the received signal model at the eavesdropper Willie, a total detection error probability at the eavesdropper Willie is obtained;

[0032] According to the total detection error probability of the eavesdropper Willie, KL divergence and total variation distance are introduced, and Pinsker inequality is used to express the hidden constraint as follows:

[0033]

[0034] Where, T V (Lx , L y ) represents the variable L x With L y The total variation distance between x ||L y ) represents the value of the variable L x To L y KL divergence, (x,y) = (0,1) or (1,0);

[0035] Let 0≤ε≤1, and D(L x ||L y )≤2ε 2 Replace ξ≥1-ε as the concealment constraint, where ε is the prior value of the concealment constraint and ξ is the total detection error probability at the eavesdropper Willie;

[0036] Define the priority of high-speed rail communication services and quantify them with numbers. x ||L y )≤2ε 2 The values ​​of (x, y) and ε are related to the service priority.

[0037] Optionally, in combination with the concealment constraint condition, with the goal of maximizing the concealment throughput, constructing a joint optimization problem of the transceiver beamforming and the smart metasurface RIS phase shift includes:

[0038] Obtaining a concealed throughput at the mobile relay MR according to short packet communication with a limited block length;

[0039] Taking the maximization of concealed throughput as the goal and combining the concealed constraint condition to construct the joint optimization problem;

[0040] The joint optimization problem is:

[0041]

[0042]

[0043] Where F is the transmit beamforming matrix at the base station BS, M is the receive beamforming matrix at the mobile relay MR, θ is the reflection coefficient matrix of RIS, η is the concealed throughput at the mobile relay MR, D(L x ‖L y ) represents the value of the variable L x To L y KL divergence, ε is the prior value of the hidden constraint, P B is the maximum transmission power of the base station, N Rv is the total number of rows of RIS units, N Rhis the total number of columns of RIS units, m is the number of rows of RIS units, n is the number of columns of RIS units, C1, C2, and C3 are the concealment constraints that meet the covert communication conditions, the maximum transmission power constraints of the base station BS, and the unit modulus constraints of the RIS phase shift, δ is the decoding error probability at MR, K is the data packet length, and R is the information transmission rate from the base station BS to the mobile relay MR.

[0044] Optionally, converting the joint optimization problem into a Markov decision process problem includes:

[0045] The Markov decision process problem consists of a state set S, an action space A, and a state transition probability P. sa , reward function Re and γ discount factor;

[0046] The channel information of all nodes in the railway Internet of Things covert communication system, the transmission rate of the last time slot and the prior value of the covert constraint are used as the state set S;

[0047] Taking the transmit beamforming matrix at the base station BS, the receive beamforming matrix at the mobile relay MR and the RIS phase shift as the action space A;

[0048] The reward function Re is:

[0049]

[0050] Where Re is the reward function, M is the receive beamforming matrix at the mobile relay MR, H RM The conjugate transpose of , Θ is the reflection coefficient matrix of RIS, H BR is the channel matrix of the BS-RIS link, F H is the conjugate transpose of F, F is the transmit beamforming matrix at the base station BS, ω1 and ω2 are the weights of part 2 and part 3, ρ1 and ρ2 represent the satisfaction of the concealment constraint and the BS maximum transmit power constraint in each time slot t', part 1 represents the direct utility, that is, the effective received signal, part 2 and part 3 represent the cost function of each time slot t', corresponding to the situation where the concealment constraint requirements and the base station BS maximum transmit power requirements are not met, respectively.

[0051] Optionally, a deep reinforcement learning framework is used to handle the non-convexity and coupling of the problem. The optimal solution to the problem includes:

[0052] A deep reinforcement learning framework based on the dual actor-critic method DACM jointly optimizes the transceiver beamforming and the intelligent metasurface RIS phase shift, wherein the Actor network and the Critic network of the deep reinforcement learning framework are composed of two DNN networks.

[0053] Optional, dual actor-critic method DACM-based deep reinforcement learning framework to jointly optimize the transmit-receive beamforming and smart metasurface RIS phase shift includes:

[0054] S1. The central controller of the base station BS obtains status information s t’ , the state information s t’ Input the Actor network to perform action selection and obtain action a t’ 、Reward t’ and the next state s t’+1 ;

[0055] S2. The sequence (s t’ , a t’ , Re t’ ,s t’+1 ) is stored in the experience replay pool as a data set for training the network. According to the Banach fixed point theorem and the Bellman equation, the Critic network performs behavior evaluation based on the state value function and the action value function;

[0056] S3. Randomly sample some data from the experience replay pool to calculate the loss function and train the network parameters of the DNN network, return to S1, and repeatedly train until the network converges;

[0057] Wherein, the loss function is:

[0058]

[0059] in, is the loss function, τ is the size of the Mini-batch of the sampled data set, V * is the state value, s v is the state of the vth data set, is the network parameter of the DNN network, Q * is the action value, a v is the vth data set action.

[0060] The present invention also provides a railway Internet of Things covert communication system based on deep reinforcement learning, including: a communication model construction module, a channel model construction module, a received signal model construction module, a constraint condition design module, an optimization problem construction module, a deep reinforcement learning algorithm design module and an optimal solution calculation module;

[0061] The communication model building module is used to add the intelligent metasurface RIS to the railway Internet of Things communication system to build a high-speed railway Internet of Things hidden communication model based on RIS-MIMO;

[0062] The channel model building module is used to build a channel model of the railway Internet of Things covert communication system based on the RIS-MIMO-based high-speed railway Internet of Things covert communication model, taking into account the Doppler frequency shift compensation caused by the high-speed movement of the train and the channel uncertainty introduced by artificial noise;

[0063] The received signal model building module is used to build a received signal model based on hypothesis testing according to the communication model of the railway Internet of Things covert communication system;

[0064] The constraint condition design module is used to obtain hidden constraint conditions according to the received signal model and the railway communication service priority;

[0065] The optimization problem construction module is used to construct a joint optimization problem of the beamforming of the transmitting and receiving ends and the RIS phase shift of the intelligent metasurface in combination with the concealment constraint condition with the goal of maximizing the concealment throughput;

[0066] The deep reinforcement learning algorithm design module is used to handle the non-convexity and coupling of the problem using a deep reinforcement learning framework based on the transformed Markov decision process problem;

[0067] The optimal solution calculation module is used to adjust the model parameters of the deep reinforcement learning framework and train the network model until the model converges to obtain the optimal solution to the problem.

[0068] The beneficial effects of the present invention are as follows: the present invention solves the problem of secure transmission of railway Internet of Things data under fast time-varying channels, ensures the security and rationality of privacy data transmission in the railway Internet of Things, ensures driving safety and passenger privacy security while saving system resources, effectively reduces the security risks faced by the new generation of railway Internet of Things, and is low-cost and easy to deploy. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0070] Figure 1 This is a high-speed rail Internet of Things concealed communication model based on RIS-MIMO according to an embodiment of the present invention;

[0071] Figure 2 A deep reinforcement learning framework based on a dual-actor-critic method according to an embodiment of the present invention;

[0072] Figure 3A schematic diagram of a flow chart of a railway Internet of Things covert communication method based on deep reinforcement learning according to an embodiment of the present invention;

[0073] Figure 4 Schematic diagram of the impact of different hidden constraints on system performance in an embodiment of the present invention;

[0074] Figure 5 is the network parameter learning rate β of the deep reinforcement learning framework of the embodiment of the present invention π Schematic diagram of the impact on the performance of the DRL algorithm;

[0075] Figure 6 Schematic diagram of the training loss of a neural network in a deep reinforcement learning framework according to an embodiment of the present invention. DETAILED DESCRIPTION

[0076] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0077] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0078] Embodiment 1:

[0079] This embodiment provides a railway Internet of Things covert communication method based on deep reinforcement learning, including:

[0080] Add intelligent metasurface RIS to the railway IoT communication system and build a high-speed railway IoT covert communication model based on RIS-MIMO;

[0081] According to the RIS-MIMO-based high-speed rail IoT covert communication model, the channel model of the railway IoT covert communication system is constructed by considering the Doppler frequency shift compensation caused by the high-speed movement of the train and the channel uncertainty introduced by artificial noise.

[0082] According to the communication model of the railway Internet of Things covert communication system, a received signal model based on hypothesis testing is constructed;

[0083] Obtain hidden constraints based on the received signal model and railway communication service priorities;

[0084] Combined with the concealment constraints, the joint optimization problem of the transmit-receive beamforming and the smart metasurface RIS phase shift is constructed with the goal of maximizing the concealment throughput.

[0085] According to the time-varying nature of the high-speed rail channel, the joint optimization problem is transformed into a Markov decision process problem;

[0086] Based on the transformed Markov decision process problem, a deep reinforcement learning framework is used to deal with the non-convexity and coupling of the problem, obtain the optimal solution to the problem, and ensure reliable and low-cost covert communication in fast time-varying channel scenarios.

[0087] Furthermore, the high-speed rail IoT covert communication model based on RIS-MIMO includes:

[0088] A train moving from the center of the base station BS coverage area to the edge of the coverage area, a trackside base station equipped with multiple antennas and a large uniform planar array UPA, an intelligent metasurface RIS deployed between the base station BS and the train, the eavesdropper Willie and the trackside buildings. It is assumed that a mobile relay MR is installed on the top of the train to communicate with the base station BS.

[0089] Furthermore, considering the Doppler frequency shift compensation caused by the high-speed movement of the train and the channel uncertainty introduced by artificial noise, the channel model of the railway IoT covert communication system is constructed, including:

[0090] According to the RIS-MIMO-based high-speed rail IoT covert communication model, the base station BS, the intelligent metasurface RIS, and the eavesdropper Willie are fixed in position, and the mobile relay MR moves with the train. The channel matrix H of the BS-RIS link is BR , the channel matrix W of the BS-Willie link BW and the channel matrix W of the RIS-Willie link RW The channel matrix G of the BS-MR link is constructed as a quasi-static channel without Doppler shift. BM , H RM And the channel matrix W of the MR-Willie link MW It is constructed as a time-varying channel with Doppler shift.

[0091] Furthermore, constructing a received signal model based on hypothesis testing includes: a received signal model at the eavesdropper Willie.

[0092] Furthermore, according to the received signal model and the railway communication service priority, the hidden constraint conditions are obtained including:

[0093] According to the received signal model, the total detection error probability of the eavesdropper Willie is obtained; according to the total detection error probability of the eavesdropper Willie, the KL divergence and the total variation distance are introduced, and the hidden constraint is expressed by Pinsker inequality; the priority of the high-speed rail communication service is defined and quantified with numbers, and D(L x ||Ly )≤2ε 2 The values ​​of (x, y) and ε are related to the service priority.

[0094] Furthermore, combined with the concealment constraints, with the goal of maximizing the concealment throughput, the joint optimization problem of the transmit-receive beamforming and the smart metasurface RIS phase shift is constructed, including:

[0095] Obtain the concealed throughput at the mobile relay based on short packet communication with finite block length;

[0096] The goal is to maximize the concealment throughput and to construct a joint optimization problem in combination with concealment constraints.

[0097] Furthermore, the joint optimization problem is transformed into a Markov decision process problem including:

[0098] The Markov decision process problem consists of a state set S, an action space A, and a state transition probability P. sa , reward function Re and γ discount factor;

[0099] The channel information of all nodes in the railway IoT covert communication system, the transmission rate of the last time slot and the prior value of the covert communication constraint are taken as the state set S;

[0100] The transmit beamforming matrix at the base station BS, the receive beamforming matrix at the mobile relay MR and the RIS phase shift are taken as the action space A;

[0101] The reward function Re is:

[0102]

[0103] Where M is the receive beamforming matrix at the mobile relay MR, H RM The conjugate transpose of , Θ is the reflection coefficient matrix of RIS, H BR is the channel matrix of the BS-RIS link, F H is the conjugate transpose of F, F is the transmit beamforming matrix at the base station BS, ω1 and ω2 are the weights of part 2 and part 3, ρ1 and ρ2 represent the satisfaction of the concealment constraint and the BS maximum transmit power constraint in each time slot t', part 1 represents the direct utility, that is, the effective received signal, part 2 and part 3 represent the cost function of each time slot t', corresponding to the situation where the concealment constraint requirements and the base station BS maximum transmit power requirements are not met, respectively.

[0104] Furthermore, the Actor network and Critic network of the deep reinforcement learning framework are composed of two DNN networks.

[0105] Furthermore, the DACM-based deep reinforcement learning framework jointly optimizes the transmit-receive beamforming and the smart metasurface RIS phase shift including:

[0106] S1. The central controller of the base station obtains status information s t’ , the status information s t’ Input the Actor network to perform action selection and obtain action a t’ 、Reward t’ and the next state s t’+1 ;

[0107] S2. The sequence (s t’ , a t’ , Re t’ ,s t’+1 ) is stored in the experience replay pool as a data set for training the network. According to the Banach fixed point theorem and Bellman equation, the Critic network evaluates the behavior through the state value function and action value function.

[0108] S3. Randomly sample some data from the experience replay pool to calculate the loss function and train the network parameters of the DNN network. Return to S1 and train repeatedly until the network converges.

[0109] Among them, the loss function is:

[0110]

[0111] in, is the loss function, τ is the size of the Mini-batch of the sampled data set, V * is the status value, is the network parameter of the DNN network, Q * is the action value, a v is the vth data set action, s v is the state of the vth data set.

[0112] Combine the following Figure 1-Figure 6 The method of this embodiment is further described:

[0113] like Figure 3 As shown, the present invention provides a railway Internet of Things covert communication method based on deep reinforcement learning, comprising the following steps:

[0114] Add intelligent metasurfaces to the railway IoT communication system and build a high-speed railway IoT covert communication model based on RIS-MIMO;

[0115] According to the RIS-MIMO-based high-speed rail IoT covert communication model, the channel model of the railway IoT covert communication system is constructed by considering the Doppler frequency shift compensation caused by the high-speed movement of the train and the channel uncertainty introduced by artificial noise.

[0116] Based on the constructed railway IoT covert communication system channel model, a received signal model based on hypothesis testing is constructed;

[0117] Based on the received signal model based on hypothesis testing, the concealment constraints are derived;

[0118] Based on the security requirements and real-time performance of each sub-service of the high-speed rail, the priority of the high-speed rail communication service is defined and quantified with corresponding numbers. The design of hidden constraints will be related to the priority of the service.

[0119] Based on the concealment constraints and service priorities, the covert communication with limited block length in high-speed railway is considered, and the joint optimization problem of beamforming and RIS phase shift at the transceiver and transmitter is constructed with the goal of maximizing the covert throughput.

[0120] Based on the joint optimization problem, considering the time-varying nature of the high-speed rail channel, the optimization problem is transformed into a Markov Decision Process (MDP) problem.

[0121] Based on the transformed MDP problem, a low-complexity and intelligent deep reinforcement learning framework is proposed to deal with the non-convexity and coupling of the problem and obtain the optimal solution to the problem.

[0122] In this embodiment, if Figure 1 As shown in the figure, adding intelligent metasurface to the railway IoT communication system and constructing a high-speed railway IoT covert communication model based on RIS-MIMO include:

[0123] Passive intelligent metasurface RIS is deployed between adjacent base stations. It is assumed that there are illegal eavesdroppers with multiple antennas Willie at different locations and numbers, as well as trackside buildings of different numbers and heights. B A large uniform planar array (UPA) with N antennas W In the presence of Willie, an illegal eavesdropper, the root antenna secretly sends communication signals to the mobile relay (Mobile Raley, MR), which is equipped with N R The RIS is deployed between the BS and the MR. RThe function of the MR is to reduce network costs, assist the BS and trains in covert communication, avoid obstacles to legal communication, and alleviate the signal deviation caused by the Doppler effect. According to the above analysis, if the interference signal in the environment is not considered, the received signal at the MR is:

[0124]

[0125] The reflection coefficient matrix of RIS is defined as and in, βm , n、θ m,n They correspond to the reflection coefficient and phase shift of the mth row and nth column unit on the RIS surface, e j is the complex exponential function, Indicates the Nth R The phase shift of each RIS unit, q i represents the phase shift of the i-th RIS unit, the superscript T represents the transpose of the matrix and θ m,n ∈[0,2π),βm , n∈[0,1],Z M is the additive white Gaussian noise at the mobile relay MR.

[0126] In this embodiment, according to the high-speed rail Internet of Things covert communication model based on RIS-MIMO, considering the Doppler frequency shift compensation caused by the high-speed movement of the train and the channel uncertainty introduced by artificial noise, the channel model of the railway Internet of Things covert communication system is constructed, including:

[0127] In the RIS-MIMO-based high-speed rail IoT covert communication model, the positions of BS, RIS, and eavesdropper Willie are fixed, and MR moves at high speed with the train. BR , the channel matrix W of the BS-Willie link BW and the channel matrix W of the RIS-Willie link RW Modeled as a quasi-static channel without Doppler shift, G BM , H RM And the channel matrix W of the MR-Willie link MW Modeled as a time-varying channel with Doppler shift. Quasi-static channel H BR , W BW , W RW Can be expressed as:

[0128]

[0129] Where N B 、N R and N WThey represent the number of BS antennas, the number of RIS units and the number of Willie antennas respectively, and L BR , L BW and L RW Respectively represent channel H BR , W BW and W RW The total number of signal paths, ρ is the channel H BR The ρth signal path, ω is the channel W BW The ωth signal path, χ is the channel W RW The xth signal path, α ρ For channel H BR The complex channel gain, α ω is the channel W BW The complex channel gain, α χ is the channel W RW The complex channel gain, For channel H BR The normalized receive array response vector for the ρth signal path, For channel H BR The normalized transmit array response vector of the ρth signal path, is the channel W BW The normalized receive array response vector for the ωth signal path, is the channel W BW The normalized transmit array response vector of the ωth signal path, is the channel W RW The normalized receiver array response vector for the xth signal path, is the channel W RW The normalized transmit array response vector of the xth signal path. c H Represents vector a c The conjugate transpose of . and Respectively represent channel H BR The signal arrival azimuth and elevation angle of the ρth path is: and Respectively represent channel H BR The signal of the ρth path leaves the azimuth and elevation angles, and They represent the channel W BW The signal arrival azimuth and elevation angles of the ωth path are: and They represent the channel W BW The signal of the ωth path leaves the azimuth and elevation angles, and They represent the channel W RWThe signal arrival azimuth and elevation angle of the xth path, and They represent the channel W RW The signal of the xth path leaves the azimuth and elevation angles.

[0130] In high-speed mobile communication networks, the high time-varying channel caused by the high mobility of trains makes it difficult for base stations to obtain perfect CSI. If outdated CSI is used to design the beamforming and RIS phase shift at the transceiver end, it will cause very obvious performance loss. Therefore, this embodiment considers the Clarke-Jakes spectrum that is more suitable for the time-varying characteristics of the channel to represent the Doppler frequency shift. Specifically, for any time-varying channel H[t] at any time, let and H[t+Ts] represent outdated CSI and real-time CSI respectively, and the relationship between them is:

[0131]

[0132] In the formula, κ=J0(2πf D T S ) represents the time correlation coefficient, f D is the Doppler frequency shift, J0( · ) is the first kind of zero-order Bessel function, ΔH is the error term, t is any time during the train operation, and Ts is the transmission delay.

[0133] Therefore, the time-varying channel G BM , H RM and W MW All can be modeled in this way to avoid the impact of channel time variation caused by Doppler frequency shift on the system. BM , H RM and W MW They are modeled as:

[0134]

[0135] In the formula, and G BM [t+Ts] represents the outdated CSI and real-time CSI between BS and MR, respectively. and H RM [t+Ts] represents the outdated CSI and real-time CSI between RIS-MR, respectively. and W MW [t+Ts] represents the outdated CSI and real-time CSI between MR and Willie, t is any time during the train operation, and Ts is the transmission delay.

[0136] Assume that MR will generate interference signals of different powers to confuse Willie's detection of covert communication. MR's self-interference channel fM CN(0,φ), where φ∈[0,1] is the self-interference cancellation coefficient, which is determined by the efficiency of self-interference cancellation (SIC) performed at the MR. Mi Obey the interval [0, P Mi,max ], its probability density function is:

[0137]

[0138] Where P Mi,max is the maximum self-interference transmission power, x is [0, P Mi,max ] on different values.

[0139] In this embodiment, based on the constructed railway Internet of Things covert communication system channel model, constructing a received signal model based on hypothesis testing includes:

[0140] This embodiment considers Willie's binary detection model. The null hypothesis H0 indicates that no covert communication occurs, the BS sends an artificial noise signal in the form of chaotic noise, and the MR sends a signal that obeys [0, P Mi,max ]; Assume that H1 indicates that covert communication occurs, BS sends a covert signal in the form of chaotic noise, and MR sends a signal that obeys [0, P Mi,max ] is a self-interference signal uniformly distributed on the surface. Based on the above analysis, the MR receiving signal under the H0-H1 assumption can be obtained:

[0141]

[0142] The Willie received signal under the H0-H1 assumption can be expressed as:

[0143]

[0144] Where Y M is the received signal of the mobile relay MR, G BM is the channel matrix of the BS-MR link, H RM The conjugate transpose of Z M is the additive white Gaussian noise at the mobile relay MR, Y W is the receiving signal model of the eavesdropper Willie, P Mi is the interference signal transmission power of the mobile relay MR, W MW is the channel matrix of the MR-Willie link, W RW The conjugate transpose of , Θ is the reflection coefficient matrix of RIS, H RMis the channel matrix of the RIS-MR link, f M is the self-interference channel of mobile relay MR, s M is the self-interference signal from the mobile relay MR, Z W is the additive Gaussian white noise at the eavesdropper Willie, H0 indicates that no covert communication occurs, M is the receiving beamforming matrix at the mobile relay MR, W BW The conjugate transpose of BR is the channel matrix of the BS-RIS link, F H is the conjugate transpose of F, F is the transmit beamforming matrix at the base station BS, s B is a covert signal from the BS, and H1 indicates that covert communication occurs.

[0145] In this embodiment, the concealment constraint condition is derived according to the received signal model based on hypothesis testing;

[0146] In the covert communication process of this embodiment, the total detection error probability at the eavesdropper Willie is expressed as:

[0147] ξ=π0Pr{D1H0}+π1Pr{D0H1};

[0148] In the formula, D1 and D0 are binary decisions to infer whether the transmission of the BS occurs. D0 indicates that the base station BS has not communicated, and D1 indicates that the base station BS is communicating. π0 and π1=1-π0 represent the prior probabilities of the two hypotheses H0 and H1 respectively. H0 indicates that Willie judges that the BS has not communicated, and H1 indicates that Willie judges that the BS is communicating. Pr is the probability solution symbol. It should be emphasized that presetting the prior probability is conducive to improving the detection performance of the eavesdropper Willie. This embodiment assumes that π0=π1=0.5 (i.e., equal prior probability). From the perspective of the base station, the larger ξ, the better, so that even Willie cannot detect the transmission behavior of the BS using the best detector. Generally, the concealment constraint of the legitimate communication party can be modeled as ξ≥1-ε, where 0≤ε≤1 is the specified concealed communication constraint prior value to ensure that the transmitted information is sufficiently concealed.

[0149] This embodiment introduces KL divergence and total variation distance to provide a more stringent concealment constraint. Specifically, let:

[0150] ξ=1-T V (L x ,L y );

[0151] Where, T V (L x , L y ) represents the variable Lx With L y The total variation distance between (x,y) = (0,1) or (1,0). Usually, T V (L x , L y ) is not easy to solve. In order to solve this problem, we can use Pinsker inequality to get:

[0152]

[0153] In the formula, D(L x , L y ) represents the value of the variable L x To L y The KL divergence of .

[0154] From the above examples, it can be seen that D(L x ||L y )≤2ε 2 is a stricter constraint than ξ≥1-ε. Therefore, in order to better achieve reliable covert communication under a given ε, this embodiment adopts D(L x ||L y )≤2ε 2 Represents a hidden constraint.

[0155] In this embodiment, based on the security requirements and real-time performance of each sub-service of the high-speed rail, the priority of the high-speed rail communication service is defined and quantified with corresponding numbers. The design of the hidden constraint condition will be associated with the service priority and include:

[0156] The priority of high-speed rail communication services is defined as follows: services with high security requirements and immediate response (generally in milliseconds or even shorter) are defined as high priority, with a quantitative identification of 3; services with a certain time tolerance from service generation to transmission (the response time is significantly longer than high-priority services) and no strong security requirements are defined as medium priority, with a quantitative identification of 2; customers' ordinary short data communications / file transfers and low-security requirements services that do not require response measures are defined as low priority, with a quantitative identification of 1. Among them, high-priority services generally refer to railway emergency calls, emergency voice / video communications, driving-related dispatching communications and other services. Medium-priority services generally refer to non-driving-related dispatching communications, operation and maintenance voice / video communications and other services. Low-priority services generally refer to ordinary data communications / file transfers and other services for users in the car. In this definition, the priority of medium-priority and low-priority services can be increased as appropriate.

[0157] Based on the above priority definition, when constructing the optimization problem and designing the hidden constraint value, D(L x ||L y )≤2ε 2The values ​​of (x, y) and ε are associated with the priority of the service. Specifically, higher priority services should use (x, y) = (1, 0) and a smaller ε value to ensure the reliability of covert transmission.

[0158] In this embodiment, based on the concealment constraint conditions and service priorities, the concealed communication of high-speed rail with limited block length is considered, and the joint optimization problem of beamforming and RIS phase shift at the transmitting and receiving ends is constructed with the goal of maximizing the concealed throughput, including:

[0159] Considering short packet communication with finite block length, the concealed throughput at MR can be expressed as:

[0160] η=KR(1-δ);

[0161] Where δ is the decoding error probability at MR, K is the packet length, and R is the information transmission rate from BS to MR:

[0162]

[0163] In the formula, Q -1 (·) represents the inverse Q function, γ M represents the signal to interference plus noise ratio (SINR) at MR, which can be written as:

[0164]

[0165] Where M is the receive beamforming matrix at the mobile relay MR, G BM is the channel matrix of the BS-MR link, H RM The conjugate transpose of RM is the channel matrix of the RIS-MR link, Θ is the reflection coefficient matrix of RIS, H BR is the channel matrix of the BS-RIS link, F H is the conjugate transpose of F, F is the transmit beamforming matrix at the base station BS, P Mi is the interference signal transmission power of MR, φ is the self-interference cancellation coefficient, σ M is the additive white Gaussian noise power at the mobile relay.

[0166] When the packet length K and the decoding error probability δ at MR are determined, the RIS-MIMO-based high-speed rail IoT covert communication throughput maximization problem can be modeled as:

[0167]

[0168] Where N Rv ×N Rh=N R is the number of RIS units, N Rv is the total number of rows of RIS units, N Rh is the total number of columns of RIS units, m is the number of rows of RIS units, and n is the number of columns of RIS units. C1 is the concealment constraint that satisfies the covert communication conditions, C2 is the BS maximum transmit power constraint, C3 is the unit modulus constraint of RIS phase shift, ε is the prior value of the concealment constraint, and P B is the maximum transmit power of the base station.

[0169] In this embodiment, based on the joint optimization problem, taking into account the time-varying nature of the high-speed rail channel, the optimization problem is transformed into a Markov Decision Process (MDP) problem.

[0170] For the optimization problem constructed, this embodiment first converts it into a Markov decision process (MDP) problem. The MDP problem contains a large and complex state space and action space. Usually, an MDP consists of a five-tuple (S, A, P sa , Re, γ), where S represents the state set, A is the action space, P sa is the state transition probability, Re is the reward function, and γ is the discount factor. In each cycle, the agent observes the system state s t’ ∈S, choose to execute action a t’ ∈A, and then transfer to the new state s t’+1 , determine the benefit type by calculating the discount factor. Finally, obtain the timely reward Re(s) from the environmental feedback t’ ,a t’ ). The key elements of MDP are defined as follows:

[0171] State space: The state of a time slot t' includes the channel information of all nodes in the system, the transmission rate of the last time slot, and the prior value of the covert communication constraint, that is:

[0172]

[0173] Action space: Based on the observed system state s t’ , the agent takes the transmit beamforming matrix at the BS, the receive beamforming matrix at the MR, and the RIS phase shift as the output of the Actor network, namely:

[0174] a t' =[M (t') ,F (t') ,Θ (t') ];

[0175] State transition probability: P sa (s t’+1 |s t’ ,a t’ ) means to execute action a in the current state t’ Then transfer to the new state s t’+1 The probability, P sa Usually determined by the object model.

[0176] Reward function: When the agent performs an action in the current state, the reward is used as a signal to evaluate the quality of the covert beamforming strategy. The system performance can only be improved when the reward function of each learning step is related to the desired goal. Therefore, the goal of deep reinforcement learning is to maximize the signal-to-interference-to-noise ratio at the MR while satisfying the concealment constraint, the BS maximum transmit power constraint, and the unit mode constraint of the RIS phase shift. The reward function is expressed as:

[0177]

[0178] in,

[0179]

[0180] Where, part 1 represents the direct utility, i.e., the effective received signal, and part 2 and part 3 represent the cost function at a certain time t, corresponding to the situation where the concealment constraint requirement is not met and the BS maximum transmit power requirement is not met, respectively. The coefficients ω1 and ω2 are the weights of part 2 and part 3, respectively, which are used to balance the utility and cost. ρ1 and ρ2 represent the satisfaction imposed on the concealment constraint and the BS maximum transmit power constraint in each time slot t', respectively. If the relevant constraint requirements are met in the current time slot, ρ1=0, ρ2=0, indicating that the reward function is not penalized.

[0181] In this embodiment, based on the transformed MDP problem, a low-complexity and intelligent deep reinforcement learning framework is proposed to deal with the non-convexity and coupling of the problem, and obtain the optimal solution of the problem including:

[0182] After the optimization problem is transformed into an MDP problem, the state space of the MDP will be composed of a large amount of RIS-assisted high-speed rail environment information, and the variables in the action space are continuous. Therefore, a deep reinforcement learning framework based on the Double Actor-Critic Methods (DACM) is proposed to jointly optimize the beamforming and RIS phase shift at the transceiver end. The specific framework is shown in Figure 2Among them, the Actor network and the Critic network are composed of two DNNs, namely the training network for selection and the target network for evaluation. The experience replay pool is responsible for recycling reward information and state information and extracting samples to strengthen the learning and training of the network.

[0183] The deep reinforcement learning framework based on DACM mainly includes the following four steps to solve the problem of maximizing hidden throughput:

[0184] Step 1: The central controller of the BS is responsible for sensing and collecting state information in the environment t’ , and input it into the Actor network to perform action selection, and the Actor network outputs the corresponding action a t’ , the environment gets an immediate reward Re t’ , and then transfer to the next state s t’+1 .

[0185] Step 2: Sequence (s t’ , a t’ , Re t’ ,s t’+1 ) is stored in the experience replay pool as a data set for training the network. The Critic network evaluates behavior through state value function and action value function. According to Banach fixed point theorem and Bellman equation, the optimal state value function V*(s t’ ) and the optimal action value function Q*(s t’ , a t’ ) respectively satisfy the following equations:

[0186] V * (s t' )=E π [Re(s t'+1 |s,a)+γV * (s t'+1 )|s0=s];

[0187] Q * (s t' ,a t' )=E π [Re(s t'+1 |s,a)+γQ * (s t'+1 ,a t'+1 )|s0=s,a0=a];

[0188] Among them, s0 is the initial state, a0 is the initial action, E π represents the expectation of the variable under the policy π.

[0189] Step 3: In order to avoid the correlation of state information, some data is randomly sampled from the experience replay pool to calculate the loss function and train the network parameters of the DNN network. The loss function is given by the following formula:

[0190]

[0191] In the above formula, is the loss function, τ is the size of the Mini-batch of the sampled data set, V * is the state value, s v is the state of the vth data set, Q * is the action value, a v is the vth data set action, is the network parameter of the DNN network, which is given by the following formula:

[0192]

[0193] In the formula, β π is the learning rate, is the first-order partial derivative, is the loss function calculated at a certain time slot t'.

[0194] Strategy parameter π t’ The update rule of is given by:

[0195] π t'+1 = argmaxQ(s t' ,a t' );

[0196] In the formula, argmax represents the maximum value of the action value function Q corresponding to (s t’ ,a t’ ).

[0197] Step 4: The agent continues to obtain state information from the environment and repeatedly trains until the network converges, completing the training process and obtaining the optimal solution for the beamforming matrix and RIS phase shift at the transmit and receive ends.

[0198] In this embodiment, the execution steps for solving the concealed throughput maximization are shown in Table 1.

[0199] Table 1

[0200]

[0201] The specific conditions of this example are: the system center frequency is 28 GHz; the bandwidth is 100 MHz; the noise power is -134 dBm; the path loss model is: PL (d x )=PL(d0)+10ulog(d x / d0)+X σ, where PL(d0) represents the intercept, u ​​is the path loss index, d0 is the reference distance, and d x Indicates the distance between the transmitter and receiver related to the train position, X σ is shadow fading and follows a Gaussian random distribution with a mean of 0.

[0202] Due to the asymmetry of relative entropy, the results obtained by using D(L0||L1) and D(L1||L0) as hidden constraints are generally different. Figure 4 The figure shows a comparison of a set of experimental results performed by the present invention: under two KL divergences with the same initial condition, the variation of the system concealed throughput with the number of BS antennas is observed. Figure 4 As can be seen from the figure: As the number of BS antennas N B With the increase of , the throughput under the two hidden constraints increases, but D(L0||L1)≤2ε 2 The concealed throughput under the constraint is always greater than D(L1||L0)≤2ε 2 The hidden throughput under the constraint, which means that D(L1||L0)≤2ε 2 is the ratio D(L0||L1)≤2ε 2 More stringent concealment constraints. Therefore, when designing concealment constraints, the present invention considers using D(L1||L0)≤2ε 2 As the hidden constraint term of the concealed throughput maximization problem, the hidden constraint prior value ε is designed according to the priority of the service to achieve better system performance.

[0203] Figure 5 The figure shows the changing trend of system rewards with the number of training sets under different learning rates. It can be seen that different learning rates have different effects on the performance of deep reinforcement learning algorithms. Specifically, when β π = 0.1, the learning rate is too large, the behavior is oscillating, and its reward performance is much lower than β π =0.001. In addition, if the learning rate is set too small, the convergence time will be longer. As can be seen from the figure, the model built in this embodiment has a learning rate of β π = 0.001, it can learn the problem well and converge quickly. In addition, when setting β π =0.001, Figure 5 It can also be seen that when the number of training sets is about 100, the algorithm has converged quickly and remains stable as the number of training sets increases.

[0204] Figure 6 Shown is the relationship between the loss value of the neural network during training / validation and the number of training sets. Figure 6It can be seen that the training / loss value drops rapidly before the number of training sets reaches 100, and stabilizes after the number of training sets reaches 200. In addition, the validation loss is slightly higher than the training loss, which indicates that the DNN network parameters designed by the present invention have a good fitting ability in the mapping between input samples and output samples.

[0205] When solving the concealed throughput maximization problem constructed by the DACM framework proposed in this invention, the time complexity of the training phase is Where N D , T is the number of cycles shown in Table 1, τ is the size of Mini-batch, La and Lc represent the number of training layers of Actor network and Critic network respectively, and p e Indicates the number of neurons in the e-th layer of the corresponding DNN network, but the algorithm has a high complexity training phase at runtime and can be executed offline. In addition, once the network performance finally converges, the complexity of each step can be reduced to By comparison, it is found that the complexity of the algorithm proposed in the present invention is much lower than that of the prior art. Furthermore, the algorithm has the ability to continuously obtain information from the environment and optimize the network training model, which is very suitable for fast time-varying channel scenarios.

[0206] Embodiment 2:

[0207] A railway Internet of Things covert communication system based on deep reinforcement learning, comprising:

[0208] The communication model building module is used to add the intelligent metasurface RIS to the railway IoT communication system and build a high-speed railway IoT covert communication model based on RIS-MIMO;

[0209] The channel model building module is used to build the channel model of the railway Internet of Things covert communication system based on the RIS-MIMO high-speed railway Internet of Things covert communication model, taking into account the Doppler frequency shift compensation caused by the high-speed movement of the train and the channel uncertainty introduced by artificial noise;

[0210] A receiving signal model building module is used to build a receiving signal model based on hypothesis testing according to the communication model of the railway Internet of Things covert communication system;

[0211] A constraint design module is used to obtain hidden constraint conditions based on the received signal model and railway communication service priorities;

[0212] The optimization problem construction module is used to construct the joint optimization problem of the beamforming at the transmitting and receiving ends and the phase shift of the smart metasurface RIS in combination with the concealment constraints to maximize the concealment throughput;

[0213] A deep reinforcement learning algorithm design module, which is used to handle the non-convexity and coupling of problems based on the transformed Markov decision process using a deep reinforcement learning framework;

[0214] The optimal solution calculation module is used to adjust the model parameters of the deep reinforcement learning framework and train the network model until the model converges to obtain the optimal solution to the problem.

[0215] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A railway Internet of Things covert communication method based on deep reinforcement learning, characterized in that: include: Add intelligent metasurface RIS to the railway IoT communication system and build a high-speed railway IoT covert communication model based on RIS-MIMO; According to the RIS-MIMO-based high-speed rail IoT covert communication model, the channel model of the railway IoT covert communication system is constructed by considering the Doppler frequency shift compensation caused by the high-speed movement of the train and the channel uncertainty introduced by artificial noise; According to the communication model of the railway Internet of Things covert communication system, a received signal model based on hypothesis testing is constructed; Obtaining hidden constraint conditions according to the received signal model and railway communication service priority; Combined with the concealment constraints, with the goal of maximizing the concealment throughput, a joint optimization problem of the beamforming at the transmitting and receiving ends and the phase shift of the smart metasurface RIS is constructed; According to the time-varying nature of the high-speed rail channel, the joint optimization problem is transformed into a Markov decision process problem; Based on the transformed Markov decision process problem, a deep reinforcement learning framework is used to deal with the non-convexity and coupling of the problem, obtain the optimal solution to the problem, and ensure reliable and low-cost covert communication in fast time-varying channel scenarios.

2. According to the railway Internet of Things covert communication method based on deep reinforcement learning according to claim 1, it is characterized in that: The RIS-MIMO-based high-speed rail IoT covert communication model includes: A train moves from the center of the coverage area of ​​the base station BS to the edge of the coverage area, a trackside base station equipped with multiple antennas and a large uniform planar array UPA, the smart metasurface RIS deployed between the base station BS and the train, the eavesdropper Willie and the trackside buildings. It is assumed that a mobile relay MR is installed on the top of the train for communicating with the base station BS.

3. The railway Internet of Things covert communication method based on deep reinforcement learning according to claim 2 is characterized in that: Considering the Doppler frequency shift compensation caused by high-speed train movement and the channel uncertainty introduced by artificial noise, the channel model of the railway IoT covert communication system is constructed including: According to the RIS-MIMO-based high-speed rail IoT covert communication model, the base station BS, the intelligent metasurface RIS, and the eavesdropper Willie are fixed in position, and the mobile relay MR moves with the train. The channel matrix H of the BS-RIS link is BR , the channel matrix W of the BS-Willie link BW and the channel matrix W of the RIS-Willie link RW The channel matrix G of the BS-MR link is constructed as a quasi-static channel without Doppler shift. BM , the channel matrix H of the RIS-MR link RM And the channel matrix W of the MR-Willie link MW It is constructed as a time-varying channel containing Doppler frequency shift; The construction of the time-varying channel is to represent the Doppler frequency shift through the Clarke-Jakes spectrum. For any time-varying channel H[t] at any time, let and H[t+Ts] represent the outdated channel state information CSI and the real-time channel state information CSI, respectively. The relationship between and H[t+Ts] is: In the formula, κ=J0(2πf D T S ) represents the time correlation coefficient, f D is the Doppler frequency shift, J0(·) is the first kind of zero-order Bessel function, ΔH is the error term, t is any time during the train operation, and Ts is the transmission delay; The mobile relay MR generates interference signals with different powers, and the self-interference channel of the mobile relay MR is f M ~CN(0,φ), φ∈[0,1] is the self-interference cancellation coefficient, CN is the symbolic representation of the complex Gaussian distribution, assuming that the interference signal transmission power P of the mobile relay MR Mi Obey the interval [0, P Mi,max ], the probability density function is: Where P Mi,max is the maximum self-interference transmission power, x is [0, P Mi,max ] on different values.

4. The railway Internet of Things covert communication method based on deep reinforcement learning according to claim 3 is characterized in that: Constructing a received signal model based on hypothesis testing includes: constructing a received signal model at the eavesdropper Willie; The receiving signal model at the eavesdropper Willie is: Among them, Y W is the receiving signal model of the eavesdropper Willie, W RW The conjugate transpose of , Θ is the reflection coefficient matrix of RIS, s M is the self-interference signal from the mobile relay MR, Z W is the additive Gaussian white noise at the eavesdropper Willie, H0 indicates that no covert communication occurs, M is the receiving beamforming matrix at the mobile relay MR, W BW The conjugate transpose of H is the conjugate transpose of F, F is the transmit beamforming matrix at the base station BS, s B It is a covert signal from BS, and H1 indicates that covert communication occurs.

5. The railway Internet of Things covert communication method based on deep reinforcement learning according to claim 4 is characterized in that: According to the received signal model and the railway communication service priority, obtaining the hidden constraint condition includes: According to the received signal model at the eavesdropper Willie, a total detection error probability at the eavesdropper Willie is obtained; According to the total detection error probability of the eavesdropper Willie, KL divergence and total variation distance are introduced, and Pinsker inequality is used to express the hidden constraint as follows: Where, T V (L x , L y ) represents the variable L x With L y The total variation distance between x ||L y ) represents the value of the variable L x To L y KL divergence, (x,y) = (0,1) or (1,0); Let 0≤ε≤1, and D(L x ||L y )≤2ε 2 Replace ξ≥1-ε as the concealment constraint, where ε is the prior value of the concealment constraint and ξ is the total detection error probability at the eavesdropper Willie; Define the priority of high-speed rail communication services and quantify them with numbers. x ||L y )≤2ε 2 The values ​​of (x, y) and ε are related to the service priority.

6. The railway Internet of Things covert communication method based on deep reinforcement learning according to claim 3 is characterized in that: Combined with the concealment constraints, with the goal of maximizing the concealment throughput, the joint optimization problem of the transceiver beamforming and the smart metasurface RIS phase shift is constructed, including: Obtaining a concealed throughput at the mobile relay MR according to short packet communication with a limited block length; Taking the maximization of concealed throughput as the goal and combining the concealed constraint condition to construct the joint optimization problem; The joint optimization problem is: Where F is the transmit beamforming matrix at the base station BS, M is the receive beamforming matrix at the mobile relay MR, θ is the reflection coefficient matrix of RIS, η is the concealed throughput at the mobile relay MR, D(L x ‖L y ) represents the value of the variable L x To L y KL divergence, ε is the prior value of the hidden constraint, P B is the maximum transmission power of the base station, N Rv is the total number of rows of RIS units, N Rh is the total number of columns of RIS units, m is the number of rows of RIS units, n is the number of columns of RIS units, C1, C2, and C3 are the concealment constraints that meet the covert communication conditions, the maximum transmission power constraints of the base station BS, and the unit modulus constraints of the RIS phase shift, δ is the decoding error probability at MR, K is the packet length, and R is the information transmission rate from the base station BS to the mobile relay MR.

7. The railway Internet of Things covert communication method based on deep reinforcement learning according to claim 6 is characterized in that: Converting the joint optimization problem into a Markov decision process problem includes: The Markov decision process problem consists of a state set S, an action space A, and a state transition probability P. sa , reward function Re and γ discount factor; The channel information of all nodes in the railway Internet of Things covert communication system, the transmission rate of the last time slot and the prior value of the covert constraint are used as the state set S; Taking the transmit beamforming matrix at the base station BS, the receive beamforming matrix at the mobile relay MR and the RIS phase shift as the action space A; The reward function Re is: Where Re is the reward function, M is the receive beamforming matrix at the mobile relay MR, H RM The conjugate transpose of , Θ is the reflection coefficient matrix of RIS, H BR is the channel matrix of the BS-RIS link, F H is the conjugate transpose of F, F is the transmit beamforming matrix at the base station BS, ω1 and ω2 are the weights of part2 and part 3 respectively, ρ1 and ρ2 represent the satisfaction imposed on the concealment constraint and the BS maximum transmit power constraint in each time slot t' respectively, part 1 represents the direct utility, that is, the effective received signal, part 2 and part 3 represent the cost function of each time slot t', corresponding to the situation where the concealment constraint requirements and the base station BS maximum transmit power requirements are not met respectively.

8. The railway Internet of Things covert communication method based on deep reinforcement learning according to claim 7 is characterized in that: The deep reinforcement learning framework is used to deal with the non-convexity and coupling of the problem, and the optimal solution to the problem is obtained, including: A deep reinforcement learning framework based on the dual actor-critic method DACM jointly optimizes the transceiver beamforming and the intelligent metasurface RIS phase shift, wherein the Actor network and the Critic network of the deep reinforcement learning framework are composed of two DNN networks.

9. The railway Internet of Things covert communication method based on deep reinforcement learning according to claim 8 is characterized in that: The deep reinforcement learning framework based on the dual actor-critic method DACM jointly optimizes the transmit-receive beamforming and the smart metasurface RIS phase shift including: S1. The central controller of the base station BS obtains status information s t’ , the state information s t’ Input the Actor network to perform action selection and obtain action a t’ 、Reward t’ and the next state s t’+1 ; S2. The sequence (s t’ , a t’ , Re t’ ,s t’+1 ) is stored in the experience replay pool as a data set for training the network. According to the Banach fixed point theorem and the Bellman equation, the Critic network performs behavior evaluation based on the state value function and the action value function; S3. Randomly sample some data from the experience replay pool to calculate the loss function and train the network parameters of the DNN network, return to S1, and repeatedly train until the network converges; Wherein, the loss function is: Among them, L(θ) is the loss function, τ is the size of the Mini-batch of the sampled data set, V * is the state value, s v is the state of the vth data set, θ is the network parameter of the DNN network, Q * is the action value, a v is the vth data set action.

10. The system of the railway Internet of Things covert communication method based on deep reinforcement learning according to any one of claims 1 to 9, characterized in that: include: Communication model building module, channel model building module, received signal model building module, constraint condition design module, optimization problem building module, deep reinforcement learning algorithm design module and optimal solution calculation module; The communication model building module is used to add the intelligent metasurface RIS to the railway Internet of Things communication system to build a high-speed railway Internet of Things hidden communication model based on RIS-MIMO; The channel model building module is used to build a channel model of the railway Internet of Things covert communication system based on the RIS-MIMO-based high-speed railway Internet of Things covert communication model, taking into account the Doppler frequency shift compensation caused by the high-speed movement of the train and the channel uncertainty introduced by artificial noise; The received signal model building module is used to build a received signal model based on hypothesis testing according to the communication model of the railway Internet of Things covert communication system; The constraint condition design module is used to obtain hidden constraint conditions according to the received signal model and the railway communication service priority; The optimization problem construction module is used to construct a joint optimization problem of the beamforming of the transmitting and receiving ends and the RIS phase shift of the intelligent metasurface in combination with the concealment constraint condition with the goal of maximizing the concealment throughput; The deep reinforcement learning algorithm design module is used to handle the non-convexity and coupling of the problem using a deep reinforcement learning framework based on the transformed Markov decision process problem; The optimal solution calculation module is used to adjust the model parameters of the deep reinforcement learning framework and train the network model until the model converges to obtain the optimal solution to the problem.

Images