Safe and timely transmission method of RIS-assisted ISAC system based on short packet

Abstract

The invention discloses a safe and timely transmission method of an RIS-assisted ISAC system based on a short packet. According to the method, a meta-reinforcement learning method is adopted for the first time to uniformly allocate communication resources and sensing resources of the RIS-assisted ISAC system based on short packets, and meanwhile, communication security, sensing security and information freshness of users in the system are guaranteed. Through solving a non-convex multi-objective optimization problem, joint optimization of communication performance and perception performance is realized.

Description

Technical Field

[0001] The invention belongs to the field of wireless communications, and in particular relates to a secure and timely transmission method of a RIS-assisted ISAC system based on short packets. Background Art

[0002] In the upcoming 6G era, many emerging time-sensitive applications will require low-latency wireless communications and high-precision perception capabilities, such as autonomous driving and industrial IoT. Therefore, the integrated perception and communication (ISAC) technology is regarded as a promising approach and has received widespread attention from academia and industry. ISAC systems can use the same hardware, antennas, and spectrum resources to perform perception and communication tasks simultaneously, thereby reducing hardware and signaling costs and improving energy efficiency and spectrum utilization. In addition, ultra-reliable low-latency communications (URLLC) usually require high-reliability and low-latency signal transmission through short data packets to support time-sensitive applications. Reconfigurable smart surfaces (RIS), as planar arrays of passive and low-cost elements, can reflect signals in a desired manner, create a favorable propagation environment, and bring more degrees of freedom to ISAC systems, thereby improving system performance.

[0003] However, time-sensitive applications rely heavily on the secure and timely delivery of high-quality information. On the one hand, URLLC services transmit a large amount of confidential and critical control information in wireless channels, posing unprecedented challenges to transmission security. On the other hand, information freshness plays a vital role in time-sensitive applications, where outdated information can lead to erroneous operations in real-time control. Therefore, it is necessary to study the information freshness of the system. The Age of Information (AoI) metric is usually used to measure the information freshness of the system. However, the transmission methods in existing research cannot cope with the above-mentioned security and timeliness challenges. It is necessary to consider the security and information freshness issues of the RIS-assisted ISAC system based on short packets and design an effective transmission method to meet the strict requirements of time-sensitive applications. Summary of the invention

[0004] In order to study the physical layer security and information freshness issues in a RIS-assisted ISAC system based on short packets, the present invention discloses a secure and timely transmission method for a RIS-assisted ISAC system based on short packets.

[0005] The embodiments of the present invention provide the following technical methods:

[0006] A secure and timely transmission method for a RIS-assisted ISAC system based on short packets, the method comprising:

[0007] Step A: Establish a RIS-assisted ISAC system based on short packets, formulate a transmission protocol, and build a perception, communication, and eavesdropping model;

[0008] Step B, calculating a perception error and a perception eavesdropping error representing perceived security;

[0009] Step C, calculating the age of the security information and its interruption probability which characterize the communication security and information freshness;

[0010] Step D, establish a multi-objective optimization problem to simultaneously minimize the total SAOP and maximize the difference between the perception error and the perception eavesdropping error;

[0011] In step E, a meta-reinforcement learning algorithm based on M-A3C is designed to solve the multi-objective optimization problem according to the decomposition strategy to achieve safe and timely transmission.

[0012] Wherein, step A specifically includes:

[0013] A1. Establish a RIS-assisted ISAC system model based on short packets. The system includes an ISAC base station, a RIS, K single-antenna users and an eavesdropper. The base station has a uniform linear array with L≥1 transmitting antennas, and the RIS uses a uniform planar array with M reflection units. and Represents the set of reflection elements of the user set and RIS, and assumes that the eavesdropper is also a single-antenna user. In addition, assume that the K users and the eavesdropper are located in areas where direct links are unreachable, so the deployment of RIS effectively assists legitimate communication and perception, while the eavesdropper can also passively eavesdrop on communication and perception information through RIS;

[0014] A2, formulate the transmission protocol, consider a state update packet with a block length of N, which contains N p pilots, known by the receiver and transmitter, and NN p data symbols, the receiver uses these pilot symbols to estimate the channel state information, assuming N is the number of pilot symbols p The pilot coefficient N is defined as a multiple of , and the pilots are uniformly inserted between data symbols. i , so that N i -1 is the number of data symbols between two pilot signals, so N = N i N p ;

[0015] A3, the signal transmitted at the base station can be expressed as in, Indicates user The transmit beamforming vector, x k represents the communication symbol of user k, and Represents the perception signal, and defines the covariance of the perception signal as Then the transmit covariance matrix at the base station is

[0016] A4, constructing a perception model. Consider the static perception model in the ISAC system. In this system, the perception process is a channel parameter estimation process. Assuming that each user can use a perception receiver to perform channel estimation, the signal received by the perception receiver is expressed as Among them, the channel coefficient from the base station to the RIS is expressed as The channel coefficients from RIS to the sensing receiver are expressed as use represents the phase shift matrix of RIS, is the RIS phase shift of the mth reflector element, represents the additive white Gaussian noise received at the perceptual receiver, is the noise power. In (a), we define the vector In (b), define Here H s It is called a cascade channel;

[0017] A5, construct the communication model, assuming that perfect channel state information is difficult to obtain, and user k can only use the channel state information estimated by the perceptual receiver, then the signal received by user k is in, represents the additive Gaussian white noise received by user k, is the noise power, and the channel coefficient from RIS to user k is expressed as e s is the perceptual error of the channel estimation performed by the perceptual receiver. Therefore, the signal-to-interference-noise ratio of the signal received at user k is Then the achievable rate of user k is C k =log2(1+γ k );

[0018] A6, build the eavesdropping model, the signal received by the eavesdropper is in represents the additive white Gaussian noise received by the eavesdropper, is the noise power, and the channel coefficient from RIS to the eavesdropper is expressed as

[0019] Therefore, the signal-to-noise ratio of the eavesdropper to user k is Then the achievable rate at which the eavesdropper can eavesdrop on user k is C e,k =log2(1+γ e,k ), in addition, in order to theoretically quantify the worst-case communication performance when the eavesdropper has the maximum eavesdropping ability, in perfect zero-forcing detection, the upper limit of the eavesdropper's signal-to-noise ratio is its signal-to-noise ratio, and the upper limit of the signal-to-noise ratio of user k is also its signal-to-noise ratio, then there exists

[0020] Wherein, step B specifically includes:

[0021] B1, consider a linear filter at the perceptual receiver to obtain the received signal observation value z n Estimate channel state information. When estimating channel parameters, the perceptual receiver is used to estimate the cascade channel H s The transmitted signal is x i =1,i=N i k,0≤k≤N p -1, set is the estimated channel when used for the nth time A specific linear filter with zero elements in the communication data phase, the optimal filter With N p non-zero elements, that is Therefore, the estimated channel can be expressed as in, is the sampled received signal vector, and the channel error is defined as And the perceptual estimation mean square error is defined as That is, perceptual error;

[0022] B2, the linear minimum mean square error estimator can be written as in is the correlation matrix of z, For z and H s The cross-correlation matrix between , therefore, the perception error can be expressed as Among them, the cross-correlation matrix R zHs Calculated as Autocorrelation matrix R zz Calculated as Among them, R Hs [n] is the autocorrelation function, I is N p The identity matrix of order, R p represents the pilot autocorrelation matrix, which is a semidefinite Hermitian matrix and is defined as

[0023]

[0024] B3, and finally the perception error is where r n =[R Hs [n],R Hs [nN i ],...,R Hs [nN i (N p -1)]] T, using the pilot autocorrelation matrix R p The eigenvalue decomposition of , the perceptual error can be rewritten as in, For R p The eigenvalue of k is the corresponding eigenvector;

[0025] B4, assuming that the eavesdropper also has the ability to perceive channel parameters, the error of channel state information can be reduced by channel estimation. And assuming that the eavesdropper can know the pre-set transmission protocol between the base station and the user, that is, the pilot information, therefore, the eavesdropper also perceives the eavesdropping channel coefficient through the minimum mean square error estimator. Similarly, the perceived eavesdropping error can be obtained as Therefore, the signal-to-noise ratio of the eavesdropper to user k can be rewritten as

[0026] Wherein, step C specifically includes:

[0027] C1, assuming that each user has U state update cycles, in the uth, u=1,...,Uth state update cycle, the uth state update data packet is encoded into a codeword with a block length of N, and the encoded data packet is transmitted to user k, respectively. and represents the arrival time, service time, and departure time of the u-th finite block length status update packet of user k. The service time is defined as the time required for the base station to send the u-th finite block length status update packet to user k. Assume And apply the Bernoulli process to simulate the random arrival of each packet, defining is the interval between the i-th state update data packet and the j-th state update data packet, which can also be expressed as Define the cumulative service time from the i-th status update packet to the j-th status update packet as Considering a first-come, first-served queue, the arrival time of the u-th state update packet of user k is The total sojourn time of the u-th status update packet is

[0028] C2, defines the instantaneous AoI of the u-th status update packet of user k as In order to simultaneously measure the information freshness of the system and meet the requirements of communication security, the secure information age (SAoI) indicator is used. SAoI is defined as the instantaneous AoI of the eavesdropper is greater than the instantaneous AoI of user k, that is, the information age at the eavesdropper is older. In this system, the instantaneous AoI of the u-th state update data packet at user k and the eavesdropper is defined as Δ k (u) and Δ e(u), therefore, the instantaneous SAoI can be expressed as Δ S (u) = [Δ e (u)-Δ k (u)] + ,in[.] + represents max{Δ e (u)-Δ k (u),0}, therefore, the outage probability (SAOP) of instantaneous SAoI is defined as the instantaneous SAoI is less than or equal to the threshold The probability of Among them, η th is the SAoI threshold in channel symbols, is the threshold of target information lag between the eavesdropper and user k;

[0029] C3, since SAOP cannot be calculated directly, the upper bound expression of SAOP can be derived using the moment generating function in random network calculus. The mathematical expression of the moment generating function is defined as: for a random process X(s, t), if the e of the random process X(s, t) θx If the expectation exists, then the random process X(s,t) has a moment generating function And there is θ>0 is a free parameter;

[0030] C4, using the moment generating function, the closed expression of the upper bound of SAOP can be obtained as Where T represents the unit time of each channel symbol, ε k (N,γ k ) and ε e (N,γ e,k ) are the decoding error probabilities of the data packets transmitted by user k and the eavesdropper respectively, and the calculation formula is: V(γ k )=1-(1+γ k ) -2 represents the channel dispersion, is a Gaussian function, R k is the maximum achievable rate for user k.

[0031] Wherein, step D specifically comprises:

[0032] D1, in order to measure the communication security, perceived security and information freshness of the RIS-assisted ISAC system based on short packets, a multi-objective optimization problem is established. Specifically, by jointly optimizing the transmit beamforming vector w k , perception signal covariance S, RIS phase shift φ mThe total SAOP is minimized and the difference between the perception error and the perception eavesdropping error is maximized with the block length N, while satisfying the base station transmit power constraint, the RIS phase shift matrix unit modulus constraint, the maximum perception error constraint, and the minimum eavesdropping error constraint.

[0033] D2, the optimization problem can be expressed as

[0034]

[0035] e s ≤Ω,

[0036] e e ≥Λ,

[0037]

[0038] Among them, P max is the maximum transmission power, Ω is the perception error threshold, Λ is the perception eavesdropping error threshold,

[0039] is the unit modulus constraint of the RIS phase shift matrix;

[0040] D3. Since the maximum perception error constraint and the minimum perception eavesdropping error constraint in the optimization problem are both fractional constraints and there are coupled variables, the optimization problem is a non-convex multi-objective optimization problem.

[0041] Wherein, step E specifically includes:

[0042] E1, to solve the multi-objective optimization problem, the decomposition strategy is a simple and effective method to design a multi-objective optimization algorithm. By introducing J weight vectors satisfy Each weight vector corresponds to a specific sub-problem. Solving each sub-optimization problem yields a set of potential Pareto optimal solutions. When all J optimization problems are solved, the desired Pareto frontier can be obtained. According to the decomposition strategy, the multi-objective optimization problem can be decomposed into a set of sub-problems. Each sub-problem is regarded as a single-objective optimization problem. The sub-problem can be expressed as a Markov decision process.

[0043] E2, defines the state space, the state of the agent at time t is determined by the imperfect channel state information currently estimated The imperfect channel state information currently estimated by the eavesdropper and the transmission power P t Composition, of which The state can be expressed as The state of the agent is determined by the state at the previous moment and the action taken at the previous moment;

[0044] E3, defines the action space. The action of the agent consists of the transmit beamforming vector at time t, the covariance of the sensing signal, the RIS phase shift and the block length, which can be expressed as In each state, the agent takes different actions according to the established strategy to obtain the maximum reward;

[0045] E4, define the strategy space, the strategy of the agent is defined as given state s t Select an action t The probability of t |s t );

[0046] E5, define the strategy function, the strategy function is defined as is a parameter θ a A deep neural network is used to generate the agent's policy. This policy function takes the agent's state as input and outputs a policy.

[0047] E6, define the reward function, which is used to measure the benefits of the action selected by the agent. In order to minimize the total SAOP and maximize the difference between the perception error and the perception eavesdropping error, the reward function is defined as Different from the single-objective Markov decision process, in the multi-objective Markov decision process, in the state s t Next select action a t Then return a set of rewards [R1,...,R J ] T , using R(a t |s t ) to represent the reward vector;

[0048] E7, define the value function. The value function measures the value of each state, that is, the future return that can be achieved in each state. For an initial state s0, define its value function as where γ t is a discount factor;

[0049] E8, performs task training, trains on a set of training tasks, updates a set of sub-model parameters, and trains the sub-model parameters through asynchronous advantage action evaluation (A3C). A3C is a policy-based method that can be used to process continuous actions and state spaces. A3C achieves fast and stable training through multiple parallel agents. The state value function of A3C can be expressed as where θ v are the parameters of the deep neural network used to estimate the value function. A3C uses the v-step reward to update its parameters. The action value of the v-th step is defined as Introducing the advantage function To determine the advantage of the action taken in the current state, during the task training phase, first randomly sample from the given weight vector distribution p(λ) Weight vector For each weight vector λ j , the loss function of the policy network (Actor) is in, is the entropy of the strategy, ξ is the step size, and the cumulative gradient of the Actor is calculated as

[0050] where θ a ′ is the thread parameter of Actor, and the loss function of the value network (Critic) is The cumulative gradient of Critic is calculated as where θ v ' is the thread parameter of Critic;

[0051] E9, in A3C, each agent updates its parameter θ once after interacting with the environment a and θ v , and then pass the parameters to the global network. During the training process, each agent independently uses the accumulated gradients to update the global neural network model parameters. Each agent periodically synchronizes the latest parameters from the global network to update its local network parameters. This process is very similar to the principle of mean field approximation (MFA). The overall strategy controls the optimal strategy calculation of each participant, while the local strategy affects the overall optimization. Therefore, MFA can be used to handle the update of the global network, and the mean of the parameter gradient is used to update the parameters of the global network, which can be expressed as

[0052]

[0053] in are the strategy parameters and value parameters of the jth sub-model. The A3C algorithm based on mean field approximation (M-A3C) not only ensures the iterative update of the neural network in the optimal direction, but also reduces the possibility of oscillation during the convergence process of the neural network.

[0054] E10, in the meta-training stage, the strategy obtained in the task training stage is used to aggregate the sampled trajectories, and the parameters of the meta-model are adjusted by differentiation. The parameter update rule of the meta-model is

[0055]

[0056] Where ε is the update step size. After training is completed, the trained model will adapt to the new task according to the optimized parameters. Given any weight vector, it is only necessary to fine-tune the meta-model to obtain a satisfactory sub-model, thereby obtaining the corresponding Pareto frontier, solving the original multi-objective optimization problem, and achieving safe and timely transmission.

[0057] Compared with the prior art, the above technical method has the following advantages:

[0058] The present invention utilizes a short packet-based RIS-assisted ISAC system to provide secure and timely URLLC services to support time-sensitive applications, and uses the age of secure information and its interruption probability to simultaneously measure information freshness and communication security, and considers perceived security assurance. A meta-reinforcement learning algorithm based on M-A3C is designed to minimize the total SAOP and maximize the difference between the perceived error and the perceived eavesdropping error, solving the multi-objective optimization problem, achieving the dynamic allocation effect of perception and communication resources, and ensuring the security and timeliness of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present invention or the technical methods in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0060] Figure 1 A flowchart of a method for secure and timely transmission of a RIS-assisted ISAC system based on short packets provided by an embodiment of the present invention;

[0061] Figure 2 It is a RIS-assisted ISAC system model based on short packets. DETAILED DESCRIPTION

[0062] As described in the background technology section, how to improve the communication and perception security performance of the system while meeting the strict requirements of time-sensitive businesses such as industrial Internet of Things and autonomous driving on information freshness is an urgent problem to be solved by people in this field.

[0063] The core idea of ​​the present invention is to use the age of security information and its interruption probability as the metric of information freshness and communication security in the RIS-assisted ISAC system based on short packets, while considering the perceived security performance and using the meta-reinforcement learning algorithm to solve the multi-objective optimization problem. Compared with traditional transmission methods, this method can meet the strict requirements of time-sensitive applications for security and timeliness.

[0064] See also Figure 1The embodiment of the present invention provides a method for secure and timely transmission of a RIS-assisted ISAC system based on short packets, the method comprising:

[0065] Step A: Establish a RIS-assisted ISAC system based on short packets, formulate a transmission protocol, and build a perception, communication, and eavesdropping model;

[0066] Step B, calculating a perception error and a perception eavesdropping error representing perceived security;

[0067] Step C, calculating the age of the security information and its interruption probability which characterize the communication security and information freshness;

[0068] Step D, establish a multi-objective optimization problem to simultaneously minimize the total SAOP and maximize the difference between the perception error and the perception eavesdropping error;

[0069] In step E, a meta-reinforcement learning algorithm based on M-A3C is designed to solve the multi-objective optimization problem according to the decomposition strategy to achieve safe and timely transmission.

[0070] Wherein, step A specifically includes:

[0071] A1. Establish a RIS-assisted ISAC system model based on short packets. The system includes an ISAC base station, a RIS, K single-antenna users and an eavesdropper. The base station has a uniform linear array with L≥1 transmitting antennas, and the RIS uses a uniform planar array with M reflection units. and Represents the set of reflection elements of the user set and RIS, and assumes that the eavesdropper is also a single-antenna user. In addition, assume that the K users and the eavesdropper are located in areas where direct links are unreachable, so the deployment of RIS effectively assists legitimate communication and perception, while the eavesdropper can also passively eavesdrop on communication and perception information through RIS;

[0072] A2, formulate the transmission protocol, consider a state update packet with a block length of N, which contains N p pilots, known by the receiver and transmitter, and NN p data symbols, the receiver uses these pilot symbols to estimate the channel state information, assuming N is the number of pilot symbols p The pilot coefficient N is defined as a multiple of , and the pilots are uniformly inserted between data symbols. i , so that N i -1 is the number of data symbols between two pilot signals, so N = N i N p ;

[0073] A3, the signal transmitted at the base station can be expressed as in, Indicates user The transmit beamforming vector, x k represents the communication symbol of user k, and Represents the perception signal, and defines the covariance of the perception signal as Then the transmit covariance matrix at the base station is

[0074] A4, constructing a perception model. Consider the static perception model in the ISAC system. In this system, the perception process is a channel parameter estimation process. Assuming that each user can use a perception receiver to perform channel estimation, the signal received by the perception receiver is expressed as Among them, the channel coefficient from the base station to the RIS is expressed as The channel coefficients from RIS to the sensing receiver are expressed as use represents the phase shift matrix of RIS, is the RIS phase shift of the mth reflector element, represents the additive white Gaussian noise received at the perceptual receiver, is the noise power. In (a), we define the vector In (b), define Here H s It is called a cascade channel;

[0075] A5, construct the communication model, assuming that perfect channel state information is difficult to obtain, and user k can only use the channel state information estimated by the perceptual receiver, then the signal received by user k is in, represents the additive Gaussian white noise received by user k, is the noise power, and the channel coefficient from RIS to user k is expressed as e s is the perceptual error of the channel estimation performed by the perceptual receiver. Therefore, the signal-to-interference-noise ratio of the signal received at user k is Then the achievable rate of user k is C k =log2(1+γ k );

[0076] A6, build the eavesdropping model, the signal received by the eavesdropper is in represents the additive white Gaussian noise received by the eavesdropper, is the noise power, and the channel coefficient from RIS to the eavesdropper is expressed as

[0077] Therefore, the signal-to-noise ratio of the eavesdropper to user k is Then the achievable rate at which the eavesdropper can eavesdrop on user k is Ce,k =log2(1+γ e,k ), in addition, in order to theoretically quantify the worst-case communication performance when the eavesdropper has the maximum eavesdropping ability, in perfect zero-forcing detection, the upper limit of the eavesdropper's signal-to-noise ratio is its signal-to-noise ratio, and the upper limit of the signal-to-noise ratio of user k is also its signal-to-noise ratio, then there exists

[0078] Wherein, step B specifically includes:

[0079] B1, consider a linear filter at the perceptual receiver to obtain the received signal observation z n Estimate channel state information. When estimating channel parameters, the perceptual receiver is used to estimate the cascade channel H s The transmitted signal is x i =1,i=N i k,0≤k≤N p -1, set is the estimated channel when used for the nth time A specific linear filter with zero elements in the communication data phase, the optimal filter With N p non-zero elements, that is Therefore, the estimated channel can be expressed as in, is the sampled received signal vector, and the channel error is defined as And the mean square error of perceptual estimation is defined as That is, perceptual error;

[0080] B2, the linear minimum mean square error estimator can be written as in is the correlation matrix of z, For z and H s The cross-correlation matrix between , therefore, the perception error can be expressed as Among them, the cross-correlation matrix R zHs Calculated as Autocorrelation matrix R zz Calculated as Among them, R Hs [n] is the autocorrelation function, I is N p The identity matrix of order, R p represents the pilot autocorrelation matrix, which is a semidefinite Hermitian matrix and is defined as

[0081]

[0082] B3, and finally the perception error is in Using the pilot autocorrelation matrix Rp The eigenvalue decomposition of , the perceptual error can be rewritten as in, For R p The eigenvalue of k is the corresponding eigenvector;

[0083] B4, assuming that the eavesdropper also has the ability to perceive channel parameters, the error of channel state information can be reduced by channel estimation. And assuming that the eavesdropper can know the pre-set transmission protocol between the base station and the user, that is, the pilot information, therefore, the eavesdropper also perceives the eavesdropping channel coefficient through the minimum mean square error estimator. Similarly, the perceived eavesdropping error can be obtained as Therefore, the signal-to-noise ratio of the eavesdropper to user k can be rewritten as

[0084] Wherein, step C specifically includes:

[0085] C1, assuming that each user has U state update cycles, in the uth, u=1,...,Uth state update cycle, the uth state update data packet is encoded into a codeword with a block length of N, and the encoded data packet is transmitted to user k, respectively. and represents the arrival time, service time, and departure time of the u-th finite block length status update packet of user k. The service time is defined as the time required for the base station to send the u-th finite block length status update packet to user k. Assume And apply the Bernoulli process to simulate the random arrival of each packet, defining is the interval between the i-th state update data packet and the j-th state update data packet, which can also be expressed as Define the cumulative service time from the i-th status update packet to the j-th status update packet as Considering a first-come, first-served queue, the arrival time of the u-th state update packet of user k is The total sojourn time of the u-th status update packet is

[0086] C2, defines the instantaneous AoI of the u-th status update packet of user k as In order to simultaneously measure the information freshness of the system and meet the requirements of communication security, the secure information age (SAoI) indicator is used. SAoI is defined as the instantaneous AoI of the eavesdropper is greater than the instantaneous AoI of user k, that is, the information age at the eavesdropper is older. In this system, the instantaneous AoI of the u-th state update data packet at user k and the eavesdropper is defined as Δ k (u) and Δ e (u), therefore, the instantaneous SAoI can be expressed as Δ S(u) = [Δ e (u)-Δ k (u)] + ,in[.] + represents max{Δ e (u)-Δ k (u),0}, therefore, the outage probability (SAOP) of instantaneous SAoI is defined as the instantaneous SAoI is less than or equal to the threshold The probability of Among them, η th is the SAoI threshold in channel symbols, is the threshold of target information lag between the eavesdropper and user k;

[0087] C3, since SAOP cannot be calculated directly, the upper bound expression of SAOP can be derived using the moment generating function in random network calculus. The mathematical expression of the moment generating function is defined as: for a random process X(s, t), if the e of the random process X(s, t) θx If the expectation exists, then the random process X(s,t) has a moment generating function

[0088] And there is θ>0 is a free parameter;

[0089] C4, using the moment generating function, the closed expression of the upper bound of SAOP can be obtained as Where T represents the unit time of each channel symbol, ε k (N,γ k ) and ε e (N,γ e,k ) are the decoding error probabilities of the data packets transmitted by user k and the eavesdropper respectively, and the calculation formula is: V(γ k )=1-(1+γ k ) -2 represents the channel dispersion, is a Gaussian function, R k is the maximum achievable rate for user k.

[0090] Wherein, step D specifically comprises:

[0091] D1, in order to measure the communication security, perceived security and information freshness of the RIS-assisted ISAC system based on short packets, a multi-objective optimization problem is established. Specifically, by jointly optimizing the transmit beamforming vector w k , perception signal covariance S, RIS phase shift φ mThe total SAOP is minimized and the difference between the perception error and the perception eavesdropping error is maximized with the block length N, while satisfying the base station transmission power constraint, the unit modulus constraint of the RIS phase shift matrix, the maximum perception error constraint, and the minimum eavesdropping error constraint.

[0092] D2, the optimization problem can be expressed as

[0093]

[0094] e s ≤Ω,

[0095] e e ≥Λ,

[0096]

[0097] Among them, P max is the maximum transmission power, Ω is the perception error threshold, Λ is the perception eavesdropping error threshold,

[0098] is the unit modulus constraint of the RIS phase shift matrix;

[0099] D3. Since the maximum perception error constraint and the minimum perception eavesdropping error constraint in the optimization problem are both fractional constraints and there are coupled variables, the optimization problem is a non-convex multi-objective optimization problem.

[0100] Wherein, step E specifically includes:

[0101] E1, to solve the multi-objective optimization problem, the decomposition strategy is a simple and effective method to design a multi-objective optimization algorithm. By introducing J weight vectors satisfy Each weight vector corresponds to a specific sub-problem. Solving each sub-optimization problem yields a set of potential Pareto optimal solutions. When all J optimization problems are solved, the desired Pareto frontier can be obtained. According to the decomposition strategy, the multi-objective optimization problem can be decomposed into a set of sub-problems. Each sub-problem is regarded as a single-objective optimization problem. The sub-problem can be expressed as a Markov decision process.

[0102] E2, defines the state space, the state of the agent at time t is determined by the imperfect channel state information currently estimated The imperfect channel state information currently estimated by the eavesdropper and the transmission power P t Composition, of which and The state can be expressed as The state of the agent is determined by the state at the previous moment and the action taken at the previous moment;

[0103] E3, defines the action space. The action of the agent consists of the transmit beamforming vector at time t, the covariance of the sensing signal, the RIS phase shift and the block length, which can be expressed as In each state, the agent takes different actions according to the established strategy to obtain the maximum reward;

[0104] E4, define the strategy space, the strategy of the agent is defined as given state s t Select an action t The probability of t |s t );

[0105] E5, define the strategy function, the strategy function is defined as is a parameter θ a A deep neural network is used to generate the agent's policy. This policy function takes the agent's state as input and outputs a policy.

[0106] E6, define the reward function, which is used to measure the benefits of the action selected by the agent. In order to minimize the total SAOP and maximize the difference between the perception error and the perception eavesdropping error, the reward function is defined as Different from the single-objective Markov decision process, in the multi-objective Markov decision process, in the state s t Next select action a t Then return a set of rewards [R1,...,R J ] T , using R(a t |s t ) to represent the reward vector;

[0107] E7, define the value function. The value function measures the value of each state, that is, the future return that can be achieved in each state. For an initial state s0, define its value function as where γ t is a discount factor;

[0108] E8, performs task training, trains on a set of training tasks, updates a set of sub-model parameters, and trains the sub-model parameters through asynchronous advantage action evaluation (A3C). A3C is a policy-based method that can be used to process continuous actions and state spaces. A3C achieves fast and stable training through multiple parallel agents. The state value function of A3C can be expressed as where θ v are the parameters of the deep neural network used to estimate the value function. A3C uses the v-step reward to update its parameters. The action value of the v-th step is defined as Introducing the advantage function To determine the advantage of the action taken in the current state, during the task training phase, first randomly sample from the given weight vector distribution p(λ) Weight vector For each weight vector λ j , the loss function of the policy network (Actor) is in, is the entropy of the strategy, ξ is the step size, and the cumulative gradient of the Actor is calculated as

[0109] where θ a ′ is the thread parameter of Actor, and the loss function of the value network (Critic) is The cumulative gradient of Critic is calculated as where θ v ' is the thread parameter of Critic;

[0110] E9, in A3C, each agent updates its parameter θ once after interacting with the environment a and θ v , and then pass the parameters to the global network. During the training process, each agent independently uses the accumulated gradients to update the global neural network model parameters. Each agent periodically synchronizes the latest parameters from the global network to update its local network parameters. This process is very similar to the principle of mean field approximation (MFA). The overall strategy controls the optimal strategy calculation of each participant, while the local strategy affects the overall optimization. Therefore, MFA can be used to handle the update of the global network, and the mean of the parameter gradient is used to update the parameters of the global network, which can be expressed as

[0111]

[0112] in are the strategy parameters and value parameters of the jth sub-model. The A3C algorithm based on mean field approximation (M-A3C) not only ensures the iterative update of the neural network in the optimal direction, but also reduces the possibility of oscillation during the convergence process of the neural network.

[0113] E10, in the meta-training stage, the strategy obtained in the task training stage is used to aggregate the sampled trajectories, and the parameters of the meta-model are adjusted by differentiation. The parameter update rule of the meta-model is

[0114]

[0115] Where ε is the update step size. After training is completed, the trained model will adapt to the new task according to the optimized parameters. Given any weight vector, it is only necessary to fine-tune the meta-model to obtain a satisfactory sub-model, thereby obtaining the corresponding Pareto frontier, solving the original multi-objective optimization problem, and achieving safe and timely transmission.

[0116] Compared with the prior art, the above technical method has the following advantages:

[0117] The present invention utilizes a short packet-based RIS-assisted ISAC system to provide secure and timely URLLC services to support time-sensitive applications, and uses the age of secure information and its interruption probability to simultaneously measure information freshness and communication security, and considers perceived security assurance. A meta-reinforcement learning algorithm based on M-A3C is designed to minimize the total SAOP and maximize the difference between the perceived error and the perceived eavesdropping error, solving the multi-objective optimization problem, achieving the dynamic allocation effect of perception and communication resources, and ensuring the security and timeliness of the system.

[0118] The present invention discloses a secure and timely transmission method for a RIS-assisted ISAC system based on short packets. The method adopts a meta-reinforcement learning method for the first time to uniformly allocate communication resources and perception resources for a RIS-assisted ISAC system based on short packets, while ensuring the communication security, perception security and information freshness of users in the system. By solving a non-convex multi-objective optimization problem, the joint optimization of communication performance and perception performance is achieved.

[0119] The various parts in this manual are described in a progressive manner, and each part focuses on the differences from other parts. The same or similar parts between the various parts can be referenced to each other.

[0120] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A secure and timely transmission method for a RIS-assisted ISAC system based on short packets, characterized in that: The steps include: Step A: Establish a RIS-assisted ISAC system based on short packets, formulate a transmission protocol, and build a perception, communication, and eavesdropping model; Step B, calculating a perception error and a perception eavesdropping error representing perceived security; Step C, calculating the age of the security information and its interruption probability which characterize the communication security and information freshness; Step D, establish a multi-objective optimization problem to simultaneously minimize the total SAOP and maximize the difference between the perception error and the perception eavesdropping error; Step E, design a meta-reinforcement learning algorithm based on M-A3C to solve the multi-objective optimization problem according to the decomposition strategy to achieve safe and timely transmission; Wherein, step A specifically includes: A1. Establish a RIS-assisted ISAC system model based on short packets. The system includes an ISAC base station, a RIS, K single-antenna users and an eavesdropper. The base station has a uniform linear array with L≥1 transmitting antennas, and the RIS uses a uniform planar array with M reflection units. and Represents the set of reflection elements of the user set and RIS, and assumes that the eavesdropper is also a single-antenna user. In addition, assume that the K users and the eavesdropper are located in areas where direct links are unreachable, so the deployment of RIS effectively assists legitimate communication and perception, while the eavesdropper can also passively eavesdrop on communication and perception information through RIS; A2, formulate the transmission protocol, consider a state update packet with a block length of N, which contains N p pilots, known by the receiver and transmitter, and NN p data symbols, the receiver uses these pilot symbols to estimate the channel state information, assuming N is the number of pilot symbols p The pilot coefficient N is defined as a multiple of , and the pilots are uniformly inserted between data symbols. i , so that N i -1 is the number of data symbols between two pilot signals, so N = N i N p ; A3, the signal transmitted at the base station can be expressed as in, Indicates user The transmit beamforming vector, x k represents the communication symbol of user k, and Represents the perception signal, and defines the covariance of the perception signal as Then the transmit covariance matrix at the base station is A4, constructing a perception model. Consider the static perception model in the ISAC system. In this system, the perception process is a channel parameter estimation process. Assuming that each user can use a perception receiver to perform channel estimation, the signal received by the perception receiver is expressed as Among them, the channel coefficient from the base station to the RIS is expressed as The channel coefficients from RIS to the sensing receiver are expressed as use represents the phase shift matrix of RIS, is the RIS phase shift of the mth reflector element, represents the additive white Gaussian noise received at the perceptual receiver, is the noise power. In (a), we define the vector In (b), define Here H s It is called a cascade channel; A5, construct the communication model, assuming that perfect channel state information is difficult to obtain, and user k can only use the channel state information estimated by the perceptual receiver, then the signal received by user k is in, represents the additive Gaussian white noise received by user k, is the noise power, and the channel coefficient from RIS to user k is expressed as e s is the perceptual error of the channel estimation performed by the perceptual receiver. Therefore, the signal-to-interference-noise ratio of the signal received at user k is Then the achievable rate of user k is C k =log2(1+γ k ); A6, build the eavesdropping model, the signal received by the eavesdropper is in represents the additive white Gaussian noise received by the eavesdropper, is the noise power, and the channel coefficient from RIS to the eavesdropper is expressed as Therefore, the signal-to-noise ratio of the eavesdropper to user k is Then the achievable rate at which the eavesdropper can eavesdrop on user k is C e,k =log2(1+γ e,k ), in addition, in order to theoretically quantify the worst-case communication performance when the eavesdropper has the maximum eavesdropping ability, in perfect zero-forcing detection, the upper limit of the eavesdropper's signal-to-noise ratio is its signal-to-noise ratio, and the upper limit of the signal-to-noise ratio of user k is also its signal-to-noise ratio, then there exists Wherein, step B specifically includes: B1, consider a linear filter at the perceptual receiver to obtain the received signal observation value z n Estimate channel state information. When estimating channel parameters, the perceptual receiver is used to estimate the cascade channel H s The transmitted signal is x i =1,i=N i k,0≤k≤N p -1, set is the estimated channel when used for the nth time A specific linear filter with zero elements in the communication data phase, the optimal filter With N p non-zero elements, that is Therefore, the estimated channel can be expressed as in, is the sampled received signal vector, and the channel error is defined as And the mean square error of perceptual estimation is defined as That is, perceptual error; B2, the linear minimum mean square error estimator can be written as in is the correlation matrix of z, For z and H s The cross-correlation matrix between , therefore, the perception error can be expressed as Among them, the cross-correlation matrix R zHs Calculated as Autocorrelation matrix R zz Calculated as in, is the autocorrelation function, I is N p The identity matrix of order, R p represents the pilot autocorrelation matrix, which is a semidefinite Hermitian matrix and is defined as B3, and finally the perception error is in Using the pilot autocorrelation matrix R p The eigenvalue decomposition of , the perceptual error can be rewritten as in, For R p The characteristic value of k is the corresponding eigenvector; B4, assuming that the eavesdropper also has the ability to perceive channel parameters, the error of channel state information can be reduced by channel estimation. And assuming that the eavesdropper can know the pre-set transmission protocol between the base station and the user, that is, the pilot information, therefore, the eavesdropper also perceives the eavesdropping channel coefficient through the minimum mean square error estimator. Similarly, the perceived eavesdropping error can be obtained as Therefore, the signal-to-noise ratio of the eavesdropper to user k can be rewritten as Wherein, step C specifically includes: C1, assuming that each user has U state update cycles, in the uth, u=1,...,Uth state update cycle, the uth state update data packet is encoded into a codeword with a block length of N, and the encoded data packet is transmitted to user k, respectively. and represents the arrival time, service time, and departure time of the u-th finite block length status update packet of user k. The service time is defined as the time required for the base station to send the u-th finite block length status update packet to user k. Assume And apply the Bernoulli process to simulate the random arrival of each packet, defining is the interval between the i-th state update data packet and the j-th state update data packet, which can also be expressed as Define the cumulative service time from the i-th status update packet to the j-th status update packet as Considering a first-come, first-served queue, the arrival time of the u-th state update packet of user k is The total sojourn time of the u-th status update packet is C2, defines the instantaneous AoI of the u-th status update packet of user k as In order to simultaneously measure the information freshness of the system and meet the requirements of communication security, the secure information age (SAoI) indicator is used. SAoI is defined as the instantaneous AoI of the eavesdropper is greater than the instantaneous AoI of user k, that is, the information age at the eavesdropper is older. In this system, the instantaneous AoI of the u-th state update data packet at user k and the eavesdropper is defined as Δ k (u) and Δ e (u), therefore, the instantaneous SAoI can be expressed as Δ S (u) = [Δ e (u)-Δ k (u)] + ,in[.] + represents max{Δ e (u)-Δ k (u),0}, therefore, the outage probability (SAOP) of instantaneous SAoI is defined as the instantaneous SAoI is less than or equal to the threshold The probability of Among them, η th is the SAoI threshold in channel symbols, is the threshold of target information lag between the eavesdropper and user k; C3, since SAOP cannot be calculated directly, the upper bound expression of SAOP can be derived using the moment generating function in random network calculus. The mathematical expression of the moment generating function is defined as: for a random process X(s, t), if the e of the random process X(s, t) θx If the expectation exists, then the random process X(s,t) has a moment generating function And there is θ>0 is a free parameter; C4, using the moment generating function, the closed expression of the upper bound of SAOP can be obtained as Where T represents the unit time of each channel symbol, ε k (N,γ k ) and ε e (N,γ e,k ) are the decoding error probabilities of the data packets transmitted by user k and the eavesdropper respectively, and the calculation formula is: V(γ k )=1-(1+γ k ) -2 represents the channel dispersion, is a Gaussian function, R k is the maximum achievable rate for user k; Wherein, step D specifically comprises: D1, in order to measure the communication security, perceived security and information freshness of the RIS-assisted ISAC system based on short packets, a multi-objective optimization problem is established. Specifically, by jointly optimizing the transmit beamforming vector w k , perception signal covariance S, RIS phase shift φ m The total SAOP is minimized and the difference between the perception error and the perception eavesdropping error is maximized with the block length N, while satisfying the base station transmit power constraint, the RIS phase shift matrix unit modulus constraint, the maximum perception error constraint, and the minimum eavesdropping error constraint. D2, the optimization problem can be expressed as Among them, P max is the maximum transmission power, Ω is the perception error threshold, Λ is the perception eavesdropping error threshold, is the unit modulus constraint of the RIS phase shift matrix; D3, since the maximum perception error constraint and the minimum perception eavesdropping error constraint in the optimization problem are both fractional constraints and there are coupled variables, the optimization problem is a non-convex multi-objective optimization problem; Wherein, step E specifically includes: E1, to solve the multi-objective optimization problem, the decomposition strategy is a simple and effective method to design a multi-objective optimization algorithm. By introducing J weight vectors satisfy Each weight vector corresponds to a specific sub-problem. Solving each sub-optimization problem yields a set of potential Pareto optimal solutions. When all J optimization problems are solved, the desired Pareto frontier can be obtained. According to the decomposition strategy, the multi-objective optimization problem can be decomposed into a set of sub-problems. Each sub-problem is regarded as a single-objective optimization problem. The sub-problem can be expressed as a Markov decision process. E2, defines the state space, the state of the agent at time t is determined by the imperfect channel state information currently estimated The imperfect channel state information currently estimated by the eavesdropper and the transmission power P t Composition, of which The state can be expressed as The state of the agent is determined by the state at the previous moment and the action taken at the previous moment; E3, defines the action space. The action of the agent consists of the transmit beamforming vector at time t, the covariance of the sensing signal, the RIS phase shift and the block length, which can be expressed as In each state, the agent takes different actions according to the established strategy to obtain the maximum reward; E4, define the strategy space, the strategy of the agent is defined as given state s t Select an action t The probability of t |s t ); E5, define the strategy function, the strategy function is defined as is a parameter θ a A deep neural network is used to generate the agent's policy. This policy function takes the agent's state as input and outputs a policy. E6, define the reward function, which is used to measure the benefits of the action selected by the agent. In order to minimize the total SAOP and maximize the difference between the perception error and the perception eavesdropping error, the reward function is defined as Different from the single-objective Markov decision process, in the multi-objective Markov decision process, in the state s t Next select action a t Then return a set of rewards [R1,...,R J ] T , using R(a t |s t ) to represent the reward vector; E7, define the value function. The value function measures the value of each state, that is, the future return that can be achieved in each state. For an initial state s0, define its value function as where γ t is a discount factor; E8, performs task training, trains on a set of training tasks, updates a set of sub-model parameters, and trains the sub-model parameters through asynchronous advantage action evaluation (A3C). A3C is a policy-based method that can be used to process continuous actions and state spaces. A3C achieves fast and stable training through multiple parallel agents. The state value function of A3C can be expressed as where θ v are the parameters of the deep neural network used to estimate the value function. A3C uses the v-step reward to update its parameters. The action value of the v-th step is defined as Introducing the advantage function To determine the advantage of the action taken in the current state, during the task training phase, first randomly sample from the given weight vector distribution p(λ) Weight vector For each weight vector λ j , the loss function of the policy network (Actor) is in, is the entropy of the strategy, ξ is the step size, and the cumulative gradient of the Actor is calculated as where θ a ′ is the thread parameter of Actor, and the loss function of the value network (Critic) is The cumulative gradient of Critic is calculated as where θ v ' is the thread parameter of Critic; E9, in A3C, each agent updates its parameter θ once after interacting with the environment a and θ v , and then pass the parameters to the global network. During the training process, each agent independently uses the accumulated gradients to update the global neural network model parameters. Each agent periodically synchronizes the latest parameters from the global network to update its local network parameters. This process is very similar to the principle of mean field approximation (MFA). The overall strategy controls the optimal strategy calculation of each participant, while the local strategy affects the overall optimization. Therefore, MFA can be used to handle the update of the global network, and the mean of the parameter gradient is used to update the parameters of the global network, which can be expressed as in are the strategy parameters and value parameters of the jth sub-model. The A3C algorithm based on mean field approximation (M-A3C) not only ensures the iterative update of the neural network in the optimal direction, but also reduces the possibility of oscillation during the convergence process of the neural network. E10, in the meta-training stage, the strategy obtained in the task training stage is used to aggregate the sampled trajectories, and the parameters of the meta-model are adjusted by differentiation. The parameter update rule of the meta-model is Where ε is the update step size. After training is completed, the trained model will adapt to the new task according to the optimized parameters. Given any weight vector, it is only necessary to fine-tune the meta-model to obtain a satisfactory sub-model, thereby obtaining the corresponding Pareto frontier, solving the original multi-objective optimization problem, and achieving safe and timely transmission.

Images