A risk-sensitive reconfigurable intelligent surface scheduling method for mobile millimeter wave networks

Abstract

The application discloses a risk-sensitive reconfigurable intelligent surface scheduling method for a mobile millimeter wave network, comprising the following steps: constructing a millimeter wave communication system model assisted by multiple reconfigurable intelligent surfaces; simulating channel characteristics according to a channel model; defining an improved conditional value at risk A-CVaR as a low data rate risk measurement index; constructing a random constraint optimization model, the target of which is to jointly maximize the average value of user signal-to-noise ratio and data rate and A-CVaR; estimating A-CVaR based on a linear programming or heuristic search method; modeling a reinforcement learning problem; completing the interaction between a reinforcement learning intelligent agent and a network environment to obtain an adaptive risk-sensitive RIS scheduling strategy; and analyzing the achievable data rate and A-CVaR of the system based on the obtained RIS scheduling strategy. The application can effectively suppress the low-rate transmission risk while maximizing the system average transmission rate, significantly improve the reliability and link stability of scheduling, and is suitable for the actual deployment of multiple RIS-assisted millimeter wave communication in a complex dynamic environment.

Description

Technical Field

[0001] This invention belongs to the field of wireless communication technology, and relates to wireless resource management technology, specifically to a risk-sensitive reconfigurable intelligent surface scheduling method for mobile millimeter-wave networks. Background Technology

[0002] Millimeter-wave bands, with their abundant available bandwidth and underutilized high-frequency spectrum resources, offer significant development potential for wireless access networks. However, the inherent high path loss and weak penetration of this band severely limit link quality in harsh propagation environments, especially those with physical obstacles, making it difficult to fully leverage its advantages. Reconfigurable Intelligent Surfaces (RIS), composed of numerous tunable reflective elements, can actively control the propagation direction of electromagnetic waves, achieving directional reflection of millimeter-wave signals and effectively avoiding obstacles, thus considered a promising method for filling coverage gaps. However, in practical deployments, the signal propagation path after RIS redirection still faces the risk of being blocked by newly introduced obstacles, which may be located between the base station (BS) and the RIS, or between the RIS and the user equipment (UE). To enhance link robustness, an intuitive solution is to introduce multiple RISs, providing multiple alternative reflection paths to reduce the risk of single-point failure. However, how to efficiently coordinate the scheduling and resource allocation of multiple RISs remains a critical issue that urgently needs to be addressed.

[0003] Deploying multiple RISs typically creates multiple signal reflection paths to enable parallel transmission between the BS and UE. However, in scenarios where the UE moves at high speed, activating multiple RISs simultaneously can introduce adverse effects. As the UE moves in different directions and at different speeds, its relative position to each RIS continuously changes, making accurate tracking of dynamic channel conditions (including the alternation of line-of-sight (LoS) and non-line-of-sight (NLoS) paths) extremely difficult. Furthermore, since some RISs may not have a LoS link with the UE, evenly distributing or simultaneously allocating transmit power to multiple RISs often cannot guarantee efficient energy utilization. Therefore, a more reasonable strategy is to adaptively select the single RIS with the relatively best current transmission quality for signal reflection. However, this selection process must fully consider the real-time dynamic changes in channel conditions caused by user mobility and unknown obstacles, which places higher demands on the perception capability and decision-making timeliness of the scheduling algorithm.

[0004] In recent years, RIS-assisted millimeter-wave networks have attracted widespread attention. However, most existing work focuses on single-RIS-assisted scenarios, and research on multi-RIS networks is relatively limited, especially regarding the discussion of RIS selection strategies. Reference 1 (N. Mensi and DB Rawat, “Reconfigurable intelligent surface selection for wireless vehicular communications,” IEEE Wireless Commun. Lett., vol. 11, no.8, pp. 1743-1747, 2022.) proposes a general RIS selection method based on signal-to-noise ratio ranking and analyzes the ergodic capacity performance of the system under outdated channel state information. Reference 2 (Y. Fang, S. Atapattu, H. Inaltekin, and J. Evans, “Optimum reconfigurable intelligent surface selection for wireless networks,” IEEE Trans. Commun., vol. 70, no. 9, pp.6241-6258, 2022.) starts from location information and aims to maximize the end-to-end signal-to-noise ratio, studying RIS selection schemes corresponding to two path loss models: product scaling and summation scaling. Reference 3 (A. Bansal, N. Agrawal, K. Singh, C.-P. Li, and S. Mumtaz, “Ris selection scheme for uav-based multi-ris-aided multiuser downlink network with imperfect and outdated csi,” IEEE Trans. Commun., 2023.) derives the probability distribution of the signal-to-noise ratio under the assumption of non-ideal and outdated channel state information, and based on this, gives a probabilistic expression for selecting a specific RIS. The above studies provide valuable theoretical references for the RIS selection problem, but their focus is mainly on the formulation of general selection criteria or probabilistic analysis under specific channel conditions, and they have not yet been designed for real-time operation requirements.Reference 4 (B. Zhang, I. Filippini, Z. Lian, and Y. Su, “Adaptive obstacle-aware RIS switch for mobile mmwave access networks,” IEEE Commun. Lett., vol. 28, no. 5, pp. 1216-1220, 2024.) proposes an adaptive obstacle-aware RIS switch method to address dynamic changes in mobile environments. However, this method primarily focuses on the average transmission performance of the system, lacking sufficient consideration for instantaneous fluctuations in network conditions and extreme transmission conditions. In real-world dynamic and complex network environments, such factors often have a significant impact on overall communication quality.

[0005] In summary, existing methods suffer from two main shortcomings: First, they tend to focus on static selection criteria or probabilistic analysis under specific conditions, failing to adequately adapt to real-time dynamic operation scenarios. Second, their optimization objectives are generally performance-oriented, neglecting the randomness of user mobility and channel mutations caused by unknown obstacles, leading to increased service quality fluctuations and difficulty in effectively guaranteeing system transmission stability. Therefore, there is an urgent need to propose a risk-sensitive reconfigurable intelligent surface scheduling method for mobile millimeter-wave networks that balances average performance and risk control, thereby improving average user transmission rates while effectively suppressing performance fluctuations and ensuring robust system operation in complex dynamic environments. Summary of the Invention

[0006] Objective: This invention aims to provide a risk-sensitive, reconfigurable intelligent surface scheduling method for mobile millimeter-wave networks. Addressing channel uncertainty caused by user random movement and unknown obstacles, it introduces an improved Conditional Value at Risk (A-CVaR) to construct a stochastic constraint optimization model. By designing an A-CVaR estimation method and combining it with a reinforcement learning framework, an adaptive RIS scheduling strategy is trained to effectively control low-rate risks while improving the system's average transmission rate. Based on this, the boundary between the system's achievable data rate and the estimated and actual A-CVaR values ​​under the proposed strategy is analyzed, providing strong support for millimeter-wave communication systems in high-speed mobile environments.

[0007] Technical Solution: To achieve the above objectives, this invention provides a risk-sensitive reconfigurable smart surface scheduling method for mobile millimeter-wave networks, comprising the following steps:

[0008] S1: Construct a millimeter-wave communication system model assisted by multiple reconfigurable intelligent surfaces (RIS). The system model includes a base station (BS), multiple reconfigurable intelligent surfaces, mobile user equipment (UE), and randomly deployed unknown obstacles. The RIS is used to redirect millimeter-wave signals.

[0009] S2: Simulate channel characteristics based on channel model. The channel model includes line-of-sight (LoS) path loss model and non-line-of-sight (NLoS) obstacle occlusion attenuation model. The obstacle occlusion attenuation model quantifies the attenuation effect of three-dimensional obstacles on BS-RIS link and RIS-UE link based on blade diffraction model.

[0010] S3: Define the Adapted Conditional Value at Risk (A-CVaR) as a measure of low data rate risk, and set A-CVaR as the conditional expectation when a random variable is below the Adapted Value at Risk (A-VaR) threshold;

[0011] S4: Construct a stochastic constraint optimization model with the objective of jointly maximizing the average value of the user signal-to-noise ratio (SNR) and the data rate, and A-CVaR, with constraints imposed on RIS scheduling.

[0012] S5: Estimate A-CVaR based on linear programming or heuristic search methods;

[0013] S6: Modeling reinforcement learning problems, including state space, action space, and reward function. The state space of reinforcement learning is based on the user's normalized location information, the action space is based on the optional RIS set, and the reward function integrates signal-to-noise ratio factors, distance factors, and A-CVaR risk-sensitive factors.

[0014] S7: Complete the interaction between the reinforcement learning agent and the network environment, design the agent's learning process, train the reinforcement learning agent, and obtain an adaptive risk-sensitive RIS scheduling strategy.

[0015] S8: Based on the obtained RIS scheduling policy, analyze the system's achievable data rate and A-CVaR (including the estimated value and the boundary of the true value).

[0016] Furthermore, the millimeter-wave communication system model construction in step S1 includes:

[0017] A1: Define spatial geometric relationships: Let the coordinates of the base station antenna be... , User antenna coordinates at time 1 The set of RIS is , No. The center coordinates of the RIS are , its first The coordinates of each reflecting unit are The set of obstacles is , No. The coordinates of the obstacles are: Then the base station and the first The distance between the RIS is , No. RIS and The distance between users at any given time is Base station and the The RIS The distance between units is , No. The RIS Units and The distance between users at any given time is ;

[0018] A2: Define the user mobility model: including the time domain Discretized into equal-length units , User location at all times ,in The velocity vector has a magnitude and direction that follow a preset distribution, and the user's movement is constrained within the service area. ,in For moving boundaries.

[0019] Furthermore, the channel model in step S2 is expressed as follows:

[0020] LoS path loss model:

[0021] ,

[0022] in For millimeter wave signal wavelength, For the first The cell surface area of ​​a RIS For the first RIS gain, , These represent the antenna gains for the base station and the user, respectively. For the first RIS reflection gain, For normalized power radiation mode, The reflection coefficient, , From base station to the The RIS Unit 1, the 1st The RIS The distance from each unit to the user;

[0023] NLoS occlusion attenuation model:

[0024] The cross-section of the 3D obstacle is modeled as a rectangular screen, and the resulting occlusion attenuation is calculated based on the blade diffraction model. It is assumed that the rectangular screen can rotate around its center, so that the line-of-sight path always remains perpendicular to it. Each obstacle involved in occlusion generates additional attenuation based on the path loss model.

[0025] The operation of the NLoS occlusion attenuation model includes:

[0026] BS-RIS Link Attenuation Model: The first The obstacle is related to BS and the first obstacle. The attenuation caused by the path between RIS is

[0027] ,

[0028] RIS-UE Link Attenuation Model: The first The obstacle to the first RIS and UE at time The attenuation caused by the path between them is

[0029] ,

[0030] in , Let be the diffraction coefficients of the blade shape, and represent the diffraction coefficients of the first, second, and third strokes, respectively. A rectangular screen has a certain edge on BS to the first On the RIS path, and on the RIS path RIS and UE at time The blade diffraction effect along the path is specifically expressed as:

[0031] ,

[0032] ,

[0033] symbol Represents w_1|w_2|h_1|h_2, which are the four sides of a rectangular screen; , and These represent the screen edge and the BS and 3rd line, respectively. RIS and UE at time The projection distance; if we denote the two-dimensional vector of the screen as... and ,but The calculation method is as follows:

[0034] ,

[0035] ,

[0036] ,

[0037] ,

[0038] Calculated and .

[0039] Furthermore, in step S3, the random variable... (In this invention, signal-to-noise ratio or data rate is used), A-CVaR is a random variable. The conditional expectation below the improved Value at Risk (A-VaR) threshold, i.e. A-VaR is defined as , is a random variable satisfy Maximum threshold , Here, A-CVaR is the risk level parameter; it is estimated by the following formula, where For the set of sample times, for Observed values ​​of the random variable at time:

[0040] .

[0041] Furthermore, the objective of the stochastic constraint optimization problem in step S4 includes jointly maximizing the average value of SNR and A-CVaR;

[0042] User at any time The SNR is:

[0043] ,

[0044] in, and These are the BS transmit power and noise power, respectively. For binary decision variables (if the first... A RIS in If the event is scheduled, the value is 1; otherwise, it is 0. Note that... Randomness depends on scheduling decisions Time-varying UE location (This leads to RIS-UE link path loss) With obstacle attenuation ),lie in Unknown obstacles (which determine the BS-RIS-UE link obstacle attenuation) );

[0045] Therefore, the stochastic constraint optimization problem can be modeled as:

[0046] ,

[0047] ,

[0048] Among them, random variables Indicates the signal-to-noise ratio. For the moment The realized value, The risk weighting coefficient controls the importance of the expected tail SNR relative to the average SNR; the first term is the average SNR of the UE, and the second term is the risk level. A-CVaR of SNR; constraint condition is time. Only one RIS can be scheduled at most.

[0049] Furthermore, the estimation method for A-CVaR in step S5 includes:

[0050] Inspired by Brown's inequality, random variables are estimated by solving the following optimization problem. At risk level A-CVaR ,Right now :

[0051]

[0052] in, express exist The realized value at time 1. For inclusion A discrete-time set of sampling points;

[0053] By solving the above equation, based on Sample estimation A-CVaR in the data;

[0054] The solution methods include linear programming and heuristic search, as detailed below:

[0055] Linear programming method:

[0056] The estimation problem of A-CVaR is solved by linear programming; let , , The linear programming problem can then be restated as follows:

[0057]

[0058]

[0059] By constructing , , , The problem can be further simplified to:

[0060]

[0061] ,

[0062] in, and The dimensions are respectively and The identity matrix, It is a dimension of The zero vector. The above equation can be solved using a commercially optimized solver;

[0063] Heuristic search:

[0064] The optimal solution in the formula can also be approximated by heuristic search; based on random variables exist Implementation samples within the time period Through the following process Accuracy approximation calculation risk level A-CVaR below;

[0065] First, obtain The minimum and maximum values ​​are denoted as the lower boundary. and upper boundary Based on the interval segmentation ratio From the interval Select interior points and ;based on Formula to establish function ;calculate and ,if ,renew for Otherwise update for Update internal points accordingly , until the accuracy reaches Take the midpoint of the interval ,calculate As A-CVaR.

[0066] Furthermore, the reinforcement learning problem modeling in step S6 includes:

[0067] The reinforcement learning agent installed at the BS is used as the central controller. The agent learns: the channel state of different reflection paths composed of different RIS during UE movement; and unknown obstacles that cause different degrees of attenuation to millimeter wave signals.

[0068] Therefore, the state space, action space, and reward function are defined as follows:

[0069] state space ( ): at time The state is defined as the latest normalized UE location information: Action space ( The available RIS constitute the action space, i.e. Reward Function: When designing the reward function, consider the following three factors:

[0070] SNR Factor: To maximize the average SNR, the expected reward and the UE at time [time value missing] The instantaneous SNR, i.e. It is directly proportional to;

[0071] Distance factor: UE mobility can cause drastic changes in the spatial correlation between the BS or RIS and the UE, leading to significant fluctuations in SNR and ultimately making the model learning process difficult and unstable. To address this issue, [measures can be taken]. Time of the first The SNR corresponding to the block RIS introduces a distance factor. This can be improved. This factor is used to balance the varying path losses at different sparse user locations. Specifically, Based on BS and the The prior location information of the block RIS is calculated in advance; The solution can be approximated in advance by following these steps: divide the service area into a fine grid composed of regular cells; measure and record the distance between each reconfigurable smart surface and the center point of all grid cells; represent Time of the first The distance from the block RIS to the nearest cell center point to the current user equipment;

[0072] Risk-sensitive factors: To improve system reliability, a risk-sensitive factor based on A-CVaR is introduced into the reward function, while simultaneously increasing the expected total system throughput and data rate tail distribution. When introducing Average Conditional Value at Risk (A-CVaR), it's important to note that: A-CVaR is based on multiple actions within a time period; including these in a single-step reward could mislead the agent. Therefore, the A-CVaR difference over consecutive time periods is used to quantify the contribution of a single action to the A-CVaR change. A-CVaR calculation is based on a specific distribution, but the data rate distribution under different states changes with the reinforcement learning policy. Therefore, a grid cell aggregation of local states is used to lock the distribution for calculation, and A-CVaR is solved for each individual grid cell. A-CVaR based on the entire training cycle weakens the model's focus on real-time channel states; therefore, segmented durations are selected to estimate A-CVaR.

[0073] The reward function is defined as follows:

[0074] ,

[0075] ,

[0076] ,

[0077] In the formula, Normalized SNR is a fundamental component of rewards. This item mitigates spatial correlation changes caused by UE mobility; due to It is related to factors such as the size of the service area, the number of RIS, and their size. It can adapt to changes in the scene, therefore, it introduces Standardize rewards, and adjust their values ​​based on experience according to the scenario; utilize Function estimation Latest in the cell indivual A-CVaR, where For application The filtering function retains only the UE at time [time]. The most recent in the current cell indivual ; and The difference represents the contribution of the action to the change in A-CVaR; It is a coefficient for controlling the importance of risk items.

[0078] Furthermore, the process of training and obtaining the adaptive risk-sensitive RIS scheduling policy in step S7 includes:

[0079] The agent model consists of three parts: a preprocessing network, a policy network, and a value function network. Their connection weight vectors are denoted as follows: , and The preprocessing network receives the state input and extracts features, which are then fed in parallel into the policy network and the value function network; Preprocessing-Policy Network Used to approximate the policy function, which maps the system state to the probability of selecting each available action; preprocessing-value function network. Used to approximate the value function, thereby evaluating the performance of the current strategy;

[0080] The two networks are jointly trained to obtain the RIS scheduling strategy.

[0081] Specifically, the updates to the value function and policy function are based on the following principles:

[0082] value function Approximated by minimizing the value loss function in the following equation:

[0083] ,

[0084] ,

[0085] in, Value function and -Step-by-step return The difference between them is used to calculate the value loss.

[0086] To approximate the policy function, along the expected increase in return... ,Right now , direction update , . An unbiased estimate is To prevent the algorithm from converging prematurely to a suboptimal policy, policy entropy is included in the policy loss. ,Right now:

[0087] ,

[0088] ,

[0089] in, The importance of controlling entropy regularization.

[0090] make Iterative updates based on the following formula This minimizes the sum of value loss and strategy loss.

[0091] ,

[0092] in, To balance the coefficients of value loss and strategy loss, To update the step size.

[0093] The learning process includes Step 1. First, initialize. Set the gradient to zero Initialize the state. Perform the following operations in each loop. Update the model every step: based on the existing strategy Select the Each RIS acts as a relay; calculates respectively , and And obtain the new status based on the latest UE location information. Based on the collected data, initialize and calculate the return. Then calculate the gradient. And update Repeat the above operations iteratively until the model converges to a suitable RIS scheduling strategy.

[0094] Furthermore, the system achievable data rate analysis method in step S8 includes:

[0095] time Scheduling The achievable data rate of one RIS is ,in This represents the total path decay, including path loss and obstacle occlusion; based on this, the instantaneous expected reachable rate and the traversal reachable rate over the entire running cycle can be obtained.

[0096] Instantaneous expected achievable rate:

[0097] time The expected achievable rate is ,in This is for statistical expectation calculation; note that... Depends on strategy ,and The sampled rate is obtained from this strategy, and given that the trained strategy follows a classification distribution, the expected achievable rate can be derived accordingly; Recorded as and define For a moment Decision probability vector For a moment Given the data rate vector, the expected achievable rate is: ;

[0098] The achievable speed of traversing the entire runtime is:

[0099] To derive the full operating cycle The traversal speed within the range is denoted by: The time-scheduled RIS (i.e., the sampled action) is ,cycle The action sequence within is The set of all possible action sequences is Given a specific sequence The system's expected achievable rate is ; Traversal Period The traversal speed is achievable; given that Depend on The time-scheduling strategy is determined by both the time and the system state, and satisfies ,cycle Inner action sequence The probability of occurrence is The total cycle traversal rate is the probability-weighted sum of the expected achievable rates for all action sequences:

[0100]

[0101] ,

[0102] Introducing the rate matrix With the strategy matrix Then, the traversal rate can be simplified as follows:

[0103] ,

[0104] In the formula Represents the trace of the matrix.

[0105] Furthermore, the boundary analysis method for the system A-CVaR estimate and the true value in step S8 includes:

[0106] This invention derives and provides upper and lower bounds of the estimated and true values ​​of A-CVaR for each system state and the entire operating cycle, in order to measure the system reliability of the proposed scheduling method during training and policy execution.

[0107] A-CVaR estimates for each system state and the entire operating cycle:

[0108] For a single system state , For the running cycle The total number of states within a random variable The A-CVaR estimate is:

[0109]

[0110] In the formula, For state Random variables The This is the first implementation; For period Internal state Number of occurrences; complete cycle The internal A-CvaR estimate is expressed as:

[0111] ,

[0112] Upper and lower bounds of the true A-CVaR value for each system state and the entire operating cycle:

[0113] Although A-CVaR is analyzed for different tail distributions compared to CVaR, some properties of CVaR can be extrapolated to A-CVaR. Based on Brown's inequality, the upper and lower bounds of the true value of A-CVaR are derived; for any Individual system state The corresponding true value of A-CVaR The following boundary constraints must be satisfied:

[0114]

[0115]

[0116] ,

[0117] In the formula The support set representing a random variable;

[0118] True value of A-CVaR throughout the entire operating cycle The following boundary constraints must be satisfied:

[0119]

[0120]

[0121] .

[0122] Beneficial Effects: Compared with existing technologies, this invention proposes an adaptive risk-sensitive RIS scheduling method for multi-RIS-assisted millimeter-wave communication systems in dynamic scenarios involving user movement and unknown obstacles. This method effectively enhances operational reliability while improving the average transmission rate of the system. Specifically, this invention introduces an improved Conditional Value at Risk (A-CVaR) to construct a stochastic constraint optimization model and rationally transforms it into a reinforcement learning decision problem. Simultaneously, it provides a practical A-CVaR estimation method and designs an agent learning process to obtain a scheduling strategy with risk awareness capabilities through training. Furthermore, this invention analyzes the boundary between the system's achievable data rate and the estimated and actual A-CVaR values ​​for the proposed strategy, thus providing a theoretical basis and engineering implementation foundation for RIS scheduling in dynamic environments. Attached Figure Description

[0123] Figure 1 This is a flowchart of the method of the present invention;

[0124] Figure 2 A schematic diagram of a multi-RIS-assisted millimeter-wave network affected by mobile users and unknown obstacles;

[0125] Figure 3 This is a schematic diagram of the geometric relationships within the scene.

[0126] Figure 4 An improved conditional value-at-risk principle diagram;

[0127] Figure 5 This is a schematic diagram of the filtering function in the reward function;

[0128] Figure 6 Flowchart for training an intelligent agent model;

[0129] Figure 7 This is a comparison diagram between the method of the present invention and other methods. Detailed Implementation

[0130] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading this invention, any modifications of the invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0131] Example 1:

[0132] like Figure 1 As shown, this embodiment provides a risk-sensitive reconfigurable smart surface scheduling method for mobile millimeter-wave networks, including the following steps:

[0133] S1: Construct a millimeter-wave communication system model assisted by multiple reconfigurable intelligent surfaces (RIS). The system model includes a base station (BS), multiple reconfigurable intelligent surfaces, mobile user equipment (UE), and randomly deployed unknown obstacles. The RIS is used to redirect millimeter-wave signals.

[0134] S2: Simulate channel characteristics based on channel model. The channel model includes line-of-sight (LoS) path loss model and non-line-of-sight (NLoS) obstacle occlusion attenuation model. The obstacle occlusion attenuation model quantifies the attenuation effect of three-dimensional obstacles on BS-RIS link and RIS-UE link based on blade diffraction model.

[0135] S3: Define the Adapted Conditional Value at Risk (A-CVaR) as a measure of low data rate risk, and set A-CVaR as the conditional expectation when a random variable is below the Adapted Value at Risk (A-VaR) threshold;

[0136] S4: Construct a stochastic constraint optimization model with the objective of jointly maximizing the average value of the user signal-to-noise ratio (SNR) and the data rate, and A-CVaR, with constraints imposed on RIS scheduling.

[0137] S5: Estimate A-CVaR based on linear programming or heuristic search methods;

[0138] S6: Modeling reinforcement learning problems, including state space, action space, and reward function. The state space of reinforcement learning is based on the user's normalized location information, the action space is based on the optional RIS set, and the reward function integrates signal-to-noise ratio factors, distance factors, and A-CVaR risk-sensitive factors.

[0139] S7: Complete the interaction between the reinforcement learning agent and the network environment, design the agent's learning process, train the reinforcement learning agent, and obtain an adaptive risk-sensitive RIS scheduling strategy.

[0140] S8: Based on the obtained RIS scheduling policy, analyze the system's achievable data rate and A-CVaR (including the estimated value and the boundary of the true value).

[0141] The above method will be applied in a practical example, including the following steps:

[0142] Step 1: As Figure 2 and Figure 3 As shown, the system model is constructed as follows:

[0143] Define spatial geometric relationships: Let the coordinates of the base station antenna be... , User antenna coordinates at time 1 The set of RIS is , No. The center coordinates of the RIS are , its first The coordinates of each reflecting unit are The set of obstacles is , No. The coordinates of the obstacles are: Then the base station and the first The distance between the RIS is , No. RIS and The distance between users at any given time is Base station and the The RIS The distance between units is , No. The RIS Units and The distance between users at any given time is .

[0144] Define the user mobility model: in the time domain Discretized into equal-length units , User location at all times ,in The velocity vector has a magnitude and direction that follow a preset distribution, and the user's movement is constrained within the service area. ,in For moving boundaries.

[0145] Step 2: Simulate channel characteristics based on the LoS and NLoS channel model expressions:

[0146] LoS path loss model:

[0147] ,

[0148] in For millimeter wave signal wavelengths, For the first The cell surface area of ​​a RIS For the first RIS gain, , These represent the antenna gains for the base station and the user, respectively. For the first RIS reflection gain, For normalized power radiation mode, The reflection coefficient, , From base station to the The RIS Unit 1, the 1st The RIS The distance from each unit to the user.

[0149] NLoS occlusion attenuation model:

[0150] The cross-section of a 3D obstacle is modeled as a rectangular screen, and the resulting occlusion attenuation is calculated based on a blade diffraction model. It is assumed that the rectangular screen can rotate around its center, ensuring that the line-of-sight path remains perpendicular to it. Each obstacle involved in occlusion generates additional attenuation based on a path loss model.

[0151] BS-RIS link attenuation model: Part The obstacle is related to BS and the first obstacle. The attenuation caused by the path between RIS is

[0152] ,

[0153] RIS-UE Link Attenuation Model: The first The obstacle to the first RIS and UE at time The attenuation caused by the path between them is

[0154] ,

[0155] in , Let be the diffraction coefficients of the blade shape, and represent the diffraction coefficients of the first, second, A rectangular screen has a certain edge on BS to the first On the RIS path, and on the RIS path RIS and UE at time The blade diffraction effect along the path is specifically expressed as:

[0156] ,

[0157] ,

[0158] symbol This represents w_1|w_2|h_1|h_2, which are the four sides of a rectangular screen. , and These represent the screen edge and the BS and 3rd line, respectively. RIS and UE at time The projection distance. Let the two-dimensional vector of the screen be... and ,but The calculation method is as follows:

[0159] ,

[0160] ,

[0161] ,

[0162] ,

[0163] Approximately, and It can be calculated.

[0164] Step 3: As Figure 4 As shown, the definition of A-CVaR is: for random variables (Signal-to-noise ratio or data rate), A-CVaR is a random variable. The conditional expectation below the improved Value at Risk (A-VaR) threshold, i.e. A-VaR is defined as , is a random variable satisfy Maximum threshold , Here, A-CVaR is the risk level parameter. A-CVaR can be estimated using the following formula, where... For the set of sample times, for Observed values ​​of the random variable at time:

[0165] ,

[0166] Step 4: Model the stochastic constraint optimization problem as follows:

[0167] The target bit jointly maximizes the average value of SNR and A-CVaR.

[0168] User at any time The SNR is:

[0169] ,

[0170] in, and These are the BS transmit power and noise power, respectively. For binary decision variables (if the first... A RIS in If the time slot is scheduled, the value is 1; otherwise, it is 0. Note that... Randomness depends on scheduling decisions Time-varying UE location (This leads to RIS-UE link path loss) With obstacle attenuation ),lie in Unknown obstacles (which determine the BS-RIS-UE link obstacle attenuation) ).

[0171] Therefore, the stochastic constraint optimization problem can be modeled as:

[0172] ,

[0173] ,

[0174] Among them, random variables Indicates the signal-to-noise ratio. For the moment The realized value, This represents the risk weighting coefficient, controlling the importance of the expected tail SNR relative to the average SNR. The first term is the average SNR of the UE, and the second term is the risk level. A-CVaR under SNR. Constraints are at time [time]. Only one RIS can be scheduled at most.

[0175] Step 5: Estimate A-CVaR:

[0176] Inspired by Brown's inequality (DB Brown, "Large deviations bounds for estimating conditional value-at-risk," Oper. Res. Lett., vol. 35, no. 6, pp. 722-730, 2007.), we estimate random variables by solving the following optimization problem. At risk level A-CVaR ,Right now :

[0177] (1)

[0178] in, express exist The realized value at time 1. For inclusion The discrete-time set of sampling points. By solving the above equation, based on... The sample estimation A-CVaR in the model.

[0179] This embodiment provides two solution methods:

[0180] Method 1: Linear Programming

[0181] The estimation problem of A-CVaR can be solved using linear programming. Let... , , The linear programming problem can then be restated as follows:

[0182]

[0183]

[0184] By constructing , , , The problem can be further simplified to:

[0185]

[0186]

[0187] in, and The dimensions are respectively and The identity matrix, It is a dimension of The zero vector. The above equation can be solved using a commercial optimization solver.

[0188] Method 2: Heuristic Search

[0189] The optimal solution in formula (1) can also be approximated by heuristic search. Based on random variables exist Implementation samples within the time period Through the following process Accuracy approximation calculation risk level The following is an example of A-CVaR. First, obtain... The minimum and maximum values ​​are denoted as the lower boundary. and upper boundary Based on the interval segmentation ratio From the interval Select interior points and Establish a function based on formula (1) .calculate and ,if ,renew for Otherwise update for Update internal points accordingly , until the accuracy reaches Take the midpoint of the interval ,calculate As A-CVaR.

[0190] Step 6: Model the reinforcement learning problem as follows:

[0191] A reinforcement learning agent installed at the BS serves as the central controller. This agent learns: 1) the channel states of different reflection paths formed by different RIS during UE movement; and 2) unknown obstacles that cause varying degrees of attenuation to millimeter-wave signals. Therefore, the state space, action space, and reward function are defined as follows: State space ( ): at time The state is defined as the latest normalized UE location information: Action space ( The available RIS constitute the action space, i.e. Reward function:

[0192] When designing a reward function, consider the following three factors:

[0193] 1. SNR Factor: To maximize the average SNR, the expected reward and the UE at time [time value missing] are [factors missing]. The instantaneous SNR, i.e. It is directly proportional to .

[0194] 2. Distance Factor: UE mobility causes drastic changes in the spatial correlation between the BS or RIS and the UE, leading to significant fluctuations in SNR and ultimately making the model learning process difficult and unstable. To address this issue, [measures can be taken to address this problem]. Time of the first The SNR corresponding to the block RIS introduces a distance factor. This can be improved. This factor is used to balance the varying path losses at different sparse user locations. Specifically, Based on BS and the The prior location information of the block RIS is calculated in advance; The solution can be approximated in advance by following these steps: divide the service area into a fine grid composed of regular cells; measure and record the distance between each reconfigurable smart surface and the center point of all grid cells; represent Time of the first The distance from the block RIS to the nearest cell center point to the current user equipment.

[0195] 3. Risk Sensitive Factors: To improve system reliability, a risk sensitive factor based on A-CVaR is introduced into the reward function, while simultaneously increasing the expected total system throughput and data rate tail distribution. When introducing Average Conditional Value at Risk (A-CVaR), the following points should be noted: 1) A-CVaR is based on multiple actions within a time period. Including these in the single-step reward would mislead the agent. Therefore, the A-CVaR difference over consecutive time periods is used to quantify the contribution of a single action to the A-CVaR change; 2) A-CVaR calculation is based on a specific distribution, while the data rate distribution under different states changes with the reinforcement learning strategy. Therefore, the distribution is locked by aggregating local states' grid cells, and A-CVaR is solved for each individual grid cell; 3) A-CVaR based on the entire training cycle weakens the model's focus on real-time channel states. Therefore, segmented durations are selected to estimate A-CVaR.

[0196] In summary, the reward function defined in this invention is as follows:

[0197] ,

[0198] ,

[0199] ,

[0200] In the formula Normalized SNR is a fundamental component of rewards. This measure mitigates spatial correlation changes caused by UE mobility. Because... It is related to factors such as the size of the service area, the number of RIS, and their size. It can adapt to changes in the scene, therefore, it introduces Standardize rewards, and adjust their values ​​based on experience and the specific scenario. Utilize Function estimation Latest in the cell indivual A-CVaR, where For application The filtering function retains only the UE at time [time]. The nearest cell indivual ,like Figure 5 As shown. and The difference represents the contribution of the action to the change in A-CVaR; It is a coefficient for controlling the importance of risk items.

[0201] Step 7: Train an adaptive risk-sensitive RIS scheduling policy:

[0202] The agent model consists of three parts: a preprocessing network, a policy network, and a value function network. Their connection weight vectors are denoted as follows: , and The preprocessing network receives the state input and extracts features, which are then fed in parallel into the policy network and the value function network. Preprocessing-Policy Network Used to approximate the policy function, which maps the system state to the probability of selecting each available action; preprocessing-value function network. The value function is approximated to evaluate the performance of the current policy. The two networks are then jointly trained to obtain the RIS scheduling policy.

[0203] Specifically, the updates to the value function and policy function are based on the following principles:

[0204] value function Approximated by minimizing the value loss function in the following equation:

[0205] ,

[0206] ,

[0207] in, Value function and -Step-by-step return The difference between them is used to calculate the value loss.

[0208] To approximate the policy function, along the expected increase in return... ,Right now , direction update , . An unbiased estimate is To prevent the algorithm from converging prematurely to a suboptimal policy, policy entropy is included in the policy loss. ,Right now:

[0209] ,

[0210] ,

[0211] in, The importance of controlling entropy regularization.

[0212] make Iterative updates based on the following formula This minimizes the sum of value loss and strategy loss.

[0213] ,

[0214] in, To balance the coefficients of value loss and strategy loss, To update the step size.

[0215] like Figure 6 As shown, the learning process includes Step, each model Update once per step. First, initialize. Update the training sequence head pointer. Initialize the state and set the gradient to zero. The following operations are performed iteratively to generate model training data: based on the existing strategy. Select the Each RIS acts as a relay; calculates respectively , and And obtain the new status based on the latest UE location information. Based on the collected data, initialize and calculate the return. Calculate the gradient And update Repeat the above operations iteratively until the model converges to a suitable RIS scheduling strategy.

[0216] Step 8: Analyze the boundary between the system's achievable data rate and the estimated and actual A-CVaR values ​​for the proposed scheduling method:

[0217] time Scheduling The achievable data rate of one RIS is ,in This represents the total path decay, including path loss and obstacle occlusion. Based on this, the instantaneous expected reachable rate and the traversal reachable rate over the entire runtime can be obtained.

[0218] Instantaneous expected achievable rate:

[0219] time The expected achievable rate is ,in This is for calculating the expected value. Note that... Depends on strategy ,and Sampled from this strategy, and given that the trained strategy follows a classification distribution, the expected achievable rate can be derived accordingly. Recorded as and define For a moment Decision probability vector For a moment Given the data rate vector, the expected achievable rate is: .

[0220] The achievable speed of traversing the entire runtime is:

[0221] To derive the full operating cycle The traversal speed within the range is denoted as . The time-scheduled RIS (i.e., the sampled action) is ,cycle The action sequence within is The set of all possible action sequences is Given a specific sequence The system's expected achievable rate is Traversal Period The traversal speed is achievable. Given... Depend on The time-scheduling strategy is determined by both the time and the system state, and satisfies ,cycle Inner action sequence The probability of occurrence is The traversal rate over the entire runtime is the probability-weighted sum of the expected achievable rates for all action sequences:

[0222]

[0223]

[0224] Introducing the rate matrix With the strategy matrix Then, the traversal rate can be simplified as:

[0225] ,

[0226] In the formula Represents the trace of the matrix.

[0227] This invention derives and provides upper and lower bounds of the estimated and true values ​​of A-CVaR for each system state and the entire operating cycle, in order to measure the system reliability of the proposed scheduling method during training and policy execution.

[0228] A-CVaR estimates for each system state and the entire operating cycle:

[0229] For a single system state , For the running cycle The total number of states within a random variable The A-CVaR estimate is:

[0230]

[0231] In the formula, For state Random variables The This is the first implementation; For period Internal state The number of times it appears. Furthermore, the complete cycle. The internal A-CvaR estimate can be expressed as:

[0232] ,

[0233] Upper and lower bounds of the true A-CVaR value for each system state and the entire operating cycle:

[0234] Although A-CVaR is analyzed for different tail distributions compared to CVaR, some properties of CVaR can be extrapolated to A-CVaR. Based on Brown's inequality, the upper and lower bounds of the true value of A-CVaR can be derived. For any Individual system state The corresponding true value of A-CVaR The following boundary constraints must be satisfied:

[0235]

[0236]

[0237] ,

[0238] In the formula The support set represents the random variable.

[0239] True value of A-CVaR throughout the entire operating cycle The following boundary constraints must be satisfied:

[0240]

[0241]

[0242]

[0243] Example 2:

[0244] To verify the effectiveness and efficacy of the method of the present invention, this embodiment obtained data and conducted analysis through experiments:

[0245] (1) Communication scenario settings

[0246] Consider a 100 m × 100 m communication service area, where the Base Station (BS) is located at the leftmost midpoint of the area. Fifteen Reliable Arrays (RIS), each consisting of 100 × 100 units, are deployed at random locations and angles facing the base station. Ten obstacles are randomly positioned with radii of 2.5 m and heights of 6 m. In practice, the User Equipment (UE) moves in a horizontal two-dimensional plane; therefore, a random waypoint model is used to generate the UE's trajectory. Velocity vector. The modulus and orientation follow a uniform distribution, that is... , ,in m / s and m / s defines the UE speed range. The UE's movement time and pause time are respectively from... s and Uniform sampling is performed in the s. The heights of the BS, RIS, and UE are 10m, 22m, and 1.7m, respectively. The network operates at a frequency of 26GHz. The BS is equipped with one antenna panel with a transmit power of 34 dBm. The noise power on the UE side is... bandwidth For 200 MHz, noise figure It is 9 dB.

[0247] (2) Model settings

[0248] The preprocessing network contains one fully connected layer with 32 neurons. Both the policy network and the value network consist of five fully connected layers with 32 neurons each; their output layer activation functions are the Softmax function and the linear function, respectively. The complete training process involves... Step 1. The model was trained using an Intel(R) Core(TM) i5-12500 @3.00GHz and 24.0GB RAM device. , and The values ​​were set to 0.99, 0.01, and 0.5 respectively. The RMSProp optimizer was used to minimize the loss function, and the learning rate was set to 0.0007. Parameters during training... , , .

[0249] (3) Experimental results

[0250] Figure 7The performance of the proposed method was compared with that of existing benchmark methods. The RIS scheduling method proposed in this invention is referred to as "this invention". The benchmark methods include: (1) "Optimal": assuming that the base station has complete control over the channel information and can ideally schedule the best RIS every time. (2) "Terrain": theoretically, the traversal UE rate that can be achieved using the proposed method. (3) AOARS: the adaptive obstacle-aware RIS handover method in reference 4. (4) MPD: a real-time scheduling version adapted from reference 2, specifically using the RIS with the minimum BS-RIS and RIS-UE distance product. The achievable rate of this invention is only slightly different from that of "Terrain", which verifies the effectiveness of this invention. It also shows that the number of samples in the test phase is sufficient and the numerical simulation results are highly consistent with the theoretical analysis conclusions. The average rate of this invention is very close to "Optimal" and its performance is better than MPD, indirectly proving that it can learn autonomously and adapt to dynamic unknown network environments. The A-CVaR rate of this invention Close to "optimal", but The significantly lower performance compared to the "optimal" rate distribution explains, to some extent, the difference in rate distribution between the two strategies, leading to the performance gap. The algorithm proposed in this invention outperforms AOARS, demonstrating the importance of considering the tail distribution of the rate during training.

Claims

1. A risk-sensitive reconfigurable intelligent surface scheduling method for mobile millimeter-wave networks, characterized in that, Includes the following steps: S1: Construct a millimeter-wave communication system model assisted by multiple reconfigurable smart surfaces. The millimeter-wave communication system model includes a base station, multiple reconfigurable smart surfaces, mobile user equipment, and randomly deployed unknown obstacles. S2: Simulate channel characteristics based on channel model, which includes line-of-sight path loss model and non-line-of-sight obstacle occlusion attenuation model. The non-line-of-sight obstacle occlusion attenuation model quantifies the attenuation effect of three-dimensional obstacles on BS-RIS link and RIS-UE link based on blade diffraction model. S3: Define the improved conditional value of risk A-CVaR as a measure of low data rate risk, and set A-CVaR as the conditional expectation of a random variable falling below the improved value of risk A-VaR threshold; S4: Construct a stochastic constraint optimization model with the objective of jointly maximizing the average value of user signal-to-noise ratio and data rate, and A-CVaR, under constraints imposed on RIS scheduling. S5: Estimate A-CVaR based on linear programming or heuristic search methods; S6: Modeling reinforcement learning problems, including state space, action space, and reward function. The state space of the reinforcement learning is based on the user's normalized location information, the action space is based on the optional RIS set, and the reward function integrates signal-to-noise ratio factors, distance factors, and A-CVaR risk-sensitive factors. S7: Complete the interaction between the reinforcement learning agent and the network environment, design the agent's learning process, train the reinforcement learning agent, and obtain an adaptive risk-sensitive RIS scheduling strategy. S8: Based on the obtained RIS scheduling policy, analyze the system's achievable data rate and A-CVaR.

2. The risk-sensitive reconfigurable intelligent surface scheduling method for mobile millimeter-wave networks according to claim 1, characterized in that, The millimeter-wave communication system model construction in step S1 includes: A1: Define spatial geometric relationships: Let the coordinates of the base station antenna be... , User antenna coordinates at time 1 The set of RIS is , No. The center coordinates of the RIS are , its first The coordinates of each reflecting unit are The set of obstacles is , No. The coordinates of the obstacles are: Then the base station and the first The distance between the RIS is , No. RIS and The distance between users at any given time is Base station and the The RIS The distance between units is , No. The RIS Units and The distance between users at any given time is ; A2: Define the user mobility model: including the time domain Discretized into equal-length units , User location at all times ,in The velocity vector has a magnitude and direction that follow a preset distribution, and the user's movement is constrained within the service area. ,in For moving boundaries.

3. The risk-sensitive reconfigurable intelligent surface scheduling method for mobile millimeter-wave networks according to claim 2, characterized in that, The channel model in step S2 is expressed as follows: LoS path loss model: ，； in For millimeter wave signal wavelengths, For the first The cell surface area of ​​a RIS For the first RIS gain, , These represent the antenna gains for the base station and the user, respectively. For the first RIS reflection gain, For normalized power radiation mode, The reflection coefficient, , From base station to the The RIS Unit 1, the 1st The RIS The distance from each unit to the user; NLoS occlusion attenuation model: The cross-section of the 3D obstacle is modeled as a rectangular screen, and the resulting occlusion attenuation is calculated based on the blade diffraction model. It is assumed that the rectangular screen can rotate around its center, so that the line-of-sight path always remains perpendicular to it. Each obstacle involved in occlusion generates additional attenuation based on the path loss model; The operation of the NLoS occlusion attenuation model includes: BS-RIS link attenuation model: Part The obstacle is related to BS and the first obstacle. The attenuation caused by the path between RIS is ，； RIS-UE Link Attenuation Model: The first The obstacle to the first RIS and UE at time The attenuation caused by the path between them is ，； in , Let be the diffraction coefficients of the blade shape, and represent the diffraction coefficients of the first, second, A rectangular screen has a certain edge on BS to the first On the RIS path, and on the RIS path RIS and UE at time The blade diffraction effect along the path is specifically expressed as: ，； ，； symbol Represents w_1|w_2|h_1|h_2, which are the four sides of a rectangular screen; , and These represent the screen edge and the BS and 3rd line, respectively. RIS and UE at time The projection distance; if we denote the two-dimensional vector of the screen as... and ,but The calculation method is as follows: ,； ,； ,； ,； Approximate calculations were obtained. and .

4. The risk-sensitive reconfigurable intelligent surface scheduling method for mobile millimeter-wave networks according to claim 3, characterized in that, In step S3, the random variable is... A-CVaR is a random variable The conditional expectation below the improved Value at Risk (A-VaR) threshold, i.e. A-VaR is defined as , is a random variable satisfy Maximum threshold , Here, A-CVaR is the risk level parameter; it is estimated by the following formula, where For the set of sample times, for Observed values ​​of the random variable at time: 。 5. A risk-sensitive reconfigurable intelligent surface scheduling method for mobile millimeter-wave networks according to claim 4, characterized in that, The objective of the stochastic constraint optimization problem in step S4 includes jointly maximizing the average value of SNR and A-CVaR; User at any time The SNR is: ，； in, and These are the BS transmit power and noise power, respectively. For binary decision variables; Therefore, the stochastic constraint optimization problem can be modeled as: ,； ,； Among them, random variables Indicates the signal-to-noise ratio. For the moment The realized value, The risk weighting coefficient controls the importance of the expected tail SNR relative to the average SNR; the first term is the average SNR of the UE, and the second term is the risk level. A-CVaR of SNR; constraint condition is time. Only one RIS can be scheduled at most.

6. The risk-sensitive reconfigurable intelligent surface scheduling method for mobile millimeter-wave networks according to claim 1, characterized in that, The estimation method for A-CVaR in step S5 includes: Inspired by Brown's inequality, random variables are estimated by solving the following optimization problem. At risk level A-CVaR ,Right now : ； in, express exist The realized value at time 1. For inclusion A discrete-time set of sampling points; By solving the above equation, based on Sample estimation A-CVaR in the data; The solution methods include linear programming and heuristic search, as detailed below: Linear programming method: The estimation problem of A-CVaR is solved by linear programming; let , , The linear programming problem can then be restated as follows: ； ； By constructing , , , The problem can be further simplified to: ； ，； in, and The dimensions are respectively and The identity matrix, It is a dimension of The zero vector; the above equation can be solved using a commercial optimization solver; Heuristic search: The optimal solution in the formula can also be approximated by heuristic search; based on random variables exist Implementation samples within the time period Through the following process Accuracy approximation calculation risk level A-CVaR below; First, obtain The minimum and maximum values ​​are denoted as the lower boundary. and upper boundary Based on the interval segmentation ratio From the interval Select interior points and ;based on Formula to establish function ;calculate and ,if ,renew for Otherwise update for Update internal points accordingly , until the accuracy reaches Take the midpoint of the interval ,calculate As A-CVaR.

7. A risk-sensitive reconfigurable intelligent surface scheduling method for mobile millimeter-wave networks according to claim 6, characterized in that, The reinforcement learning problem modeling in step S6 includes: The reinforcement learning agent installed at the BS is used as the central controller. The agent learns: the channel state of different reflection paths composed of different RIS during UE movement; and unknown obstacles that cause different degrees of attenuation to millimeter wave signals. Therefore, the state space, action space, and reward function are defined as follows: State space: at time... The state is defined as the latest normalized UE location information: Action space: The available RIS constitute the action space, i.e. Reward Function: When designing the reward function, consider the following three factors: SNR Factor: To maximize the average SNR, the expected reward and the UE at time [time value missing] The instantaneous SNR, i.e. It is directly proportional to; Distance factor: The mobility of the UE can cause drastic changes in the spatial correlation between the BS or RIS and the UE, which in turn leads to large fluctuations in SNR, ultimately making the model learning process difficult and unstable. To address this issue, the following can be done: Time of the first The SNR corresponding to the block RIS introduces a distance factor. Improvements can be made; this factor is used to balance the varying path losses at different sparse user locations. How are the two distances obtained in the solution? Based on BS and the The prior location information of the block RIS is calculated in advance; The solution can be approximated in advance by following these steps: divide the service area into a fine grid composed of regular cells; measure and record the distance between each reconfigurable smart surface and the center point of all grid cells; represent Time of the first The distance from the block RIS to the nearest cell center point to the current user equipment; Risk-sensitive factors: To improve system reliability, a risk-sensitive factor based on A-CVaR is introduced into the reward function, while simultaneously increasing the expected total system throughput and data rate tail distribution. The reward function is defined as follows: ,； ,； ,； In the formula, Normalized SNR is a fundamental component of rewards. This item mitigates spatial correlation changes caused by UE mobility; introduces... Standardize rewards, and adjust their values ​​based on experience according to the scenario; utilize Function estimation Latest in the cell indivual A-CVaR, where For application The filtering function retains only the UE at time [time]. The nearest cell indivual ; and The difference represents the contribution of the action to the change in A-CVaR; It is a coefficient for controlling the importance of risk items.

8. A risk-sensitive reconfigurable intelligent surface scheduling method for mobile millimeter-wave networks according to claim 7, characterized in that, The process of training and obtaining the adaptive risk-sensitive RIS scheduling strategy in step S7 includes: The agent model consists of three parts: a preprocessing network, a policy network, and a value function network. Their connection weight vectors are denoted as follows: , and The preprocessing network receives the state input and extracts features, which are then fed in parallel into the policy network and the value function network; Preprocessing-Policy Network Used to approximate the policy function, which maps the system state to the probability of selecting each available action; preprocessing-value function network. Used to approximate the value function, thereby evaluating the performance of the current strategy; The two networks are jointly trained to obtain the RIS scheduling strategy.

9. A risk-sensitive reconfigurable intelligent surface scheduling method for mobile millimeter-wave networks according to claim 8, characterized in that, The system achievable data rate analysis method in step S8 includes: time Scheduling The achievable data rate of one RIS is ,in This represents the total path decay, including path loss and obstacle occlusion; based on this, the instantaneous expected reachable rate and the traversal reachable rate over the entire running cycle can be obtained. Instantaneous expected achievable rate: time The expected achievable rate is ,in For statistical expectation calculation; Depends on strategy ,and The sampled rate is obtained from this strategy, and given that the trained strategy follows a classification distribution, the expected achievable rate can be derived accordingly; Recorded as and define For a moment Decision probability vector For a moment Given the data rate vector, the expected achievable rate is: ; The achievable speed of traversing the entire runtime is: To derive the full operating cycle The traversal speed within the range is denoted as . The time-scheduled RIS is ,cycle The action sequence within is The set of all possible action sequences is Given a specific sequence The system's expected achievable rate is ; Traversal Period The traversal achievable speed; given that Depend on The time-scheduling strategy is determined by both the time and the system state, and satisfies ,cycle Inner action sequence The probability of occurrence is The total cycle traversal rate is the probability-weighted sum of the expected achievable rates for all action sequences: ； ； Introducing the rate matrix With the strategy matrix Then, the traversal rate can be simplified as follows: ,； In the formula Represents the trace of the matrix.

10. A risk-sensitive reconfigurable intelligent surface scheduling method for mobile millimeter-wave networks according to claim 9, characterized in that, The boundary analysis method for the system A-CVaR estimate and the true value in step S8 includes: A-CVaR estimates for each system state and the entire operating cycle: For a single system state , For the running cycle The total number of states within a random variable The A-CVaR estimate is: ； In the formula, For state Random variables The This is the first implementation; For period Internal state Number of occurrences; complete cycle The internal A-CvaR estimate is expressed as: ，； Upper and lower bounds of the true A-CVaR value for each system state and the entire operating cycle: Based on Brown's inequality, the upper and lower bounds of the true value of A-CVaR are derived; for any Individual system state The corresponding true value of A-CVaR The following boundary constraints must be satisfied: ； ； ,； In the formula The support set representing a random variable; True value of A-CVaR throughout the entire operating cycle The following boundary constraints must be satisfied: ； ； 。

Images