High-speed railway communication system optimization method based on adaptive federated reinforcement learning

Abstract

The application belongs to the technical field of high-speed railway communication and relates to a high-speed railway communication system optimization method based on adaptive federal reinforcement learning, which comprises the following steps: establishing a scene model, establishing a channel model, establishing a signal transmission model, establishing an energy consumption model, modeling an optimization problem, and constructing a double deep Q network model for decision-making. The method has practical significance in the actual application of the communication system optimization of high-speed railway trains in multiple scenes such as high-speed tunnel penetration, towns, and suburbs. Not only does the method eliminate the influence of channel fast time variation and scene frequent switching on federal aggregation performance, but also reduces the communication and calculation overhead caused by redundant synchronization. Moreover, the method can accurately realize joint optimization and decision-making of base station power and intelligent reflecting surface phase. The algorithm performs reinforcement learning decision-making in a discrete action space, and the adaptive aggregation trigger frequency is reduced. While achieving significant improvement in communication optimization performance, the algorithm also ensures real-time response capability in a high-speed time-varying environment.

Description

Technical Field

[0001] This invention belongs to the field of high-speed railway communication technology, specifically relating to an optimization method for high-speed railway communication systems based on adaptive federated reinforcement learning, and particularly to a joint optimization method for base station power and intelligent reflective surface phase of high-speed railway communication systems based on adaptive federated reinforcement learning. Background Technology

[0002] Currently, improving the operating speed of high-speed railways and reducing transportation time has become one of the important goals of high-speed rail development. As China's high-speed railways enter a new stage of operation at speeds of 400 km / h, the continuous breakthroughs in speed have also brought about various technical challenges. How to ensure efficient and convenient information transmission services in high-speed rail passenger vehicles has become a core technical problem that urgently needs to be solved.

[0003] Currently, the dominant high-speed rail communication system globally is the Railway Digital Mobile Communication System (GSM-R), due to its strong economic advantages and relatively stable operation. However, the GSM-R design specification typically supports a maximum operating speed of 350 km / h, which is insufficient to meet the communication needs of higher-speed trains. Therefore, the Railway Broadband Mobile Communication System (LTE-R) has gradually become a research focus as a replacement for GSM-R. LTE-R is theoretically capable of meeting the communication needs of high-speed trains with speeds up to 500 km / h and is widely considered to dominate the development of high-speed rail communication for a considerable period. However, promoting the complete replacement of GSM-R by LTE-R still faces many challenges, the most prominent of which is the high cost. Increasing data transmission rates often requires higher frequency bands and larger bandwidths, which not only reduces the coverage area of ​​individual base stations and increases the deployment density of base stations along the line but also leads to a significant increase in base station transmission power. Therefore, how to minimize the construction and maintenance costs of the entire high-speed rail communication system while ensuring the data transmission rate of high-speed rail passenger trains is a crucial issue that urgently needs to be addressed.

[0004] However, existing optimization methods for high-speed railway communication systems still rely on traditional centralized or distributed control methods. Centralized methods require a control center to collect and uniformly calculate the channel status of all base stations along the line in real time, resulting in enormous communication and computational overhead, and also posing a risk of privacy breaches. While distributed methods reduce data interaction between nodes, each agent relies solely on local observations for decision-making, lacking global collaboration, and thus offering very limited improvement in overall system energy efficiency. Existing federated learning methods, although balancing collaboration and privacy to some extent, generally employ fixed-period aggregation and equal-weight parameter averaging. In high-speed railway scenarios, this still requires manual intervention to adjust the aggregation strategy, failing to automatically adapt to rapid channel changes and easily leading to slow global model convergence or even performance degradation.

[0005] Existing federated reinforcement learning algorithms, such as classical aggregation methods like federated average aggregation, are unable to effectively address the rapidly changing time-varying channel states in high-speed rail scenarios. This is because the fixed-period parameter synchronization in these algorithms is essentially a blind operation unrelated to environmental changes. When trains travel at high speeds through different scenarios such as tunnels, towns, and suburbs, the channel distribution observed by each base station undergoes drastic changes, and existing methods cannot detect these changes and trigger aggregation alignment in a timely manner. Existing intelligent optimization methods can handle power control problems under stable channel conditions, but the significant Doppler shift and frequent scene switching caused by high-speed movement dominate the communication system, and existing algorithms cannot adaptively filter and respond in real time.

[0006] Therefore, in the field of high-speed railway communication, there is an urgent need to develop a new optimization method for high-speed railway communication systems based on adaptive federated reinforcement learning, so as to realize the joint optimization of base station power and intelligent reflective surface phase, in order to effectively solve the problems of maximizing system energy efficiency and real-time decision-making by multiple agents in the context of high-speed train movement and frequent scene switching. Summary of the Invention

[0007] The purpose of this invention is to provide an optimization method for high-speed railway communication systems based on adaptive federated reinforcement learning, so as to solve the problems of maximizing system energy efficiency and real-time decision-making through multi-agent collaboration under the background of high-speed train movement and frequent scene switching.

[0008] This invention is achieved through the following technical solution:

[0009] An optimization method for a high-speed railway communication system based on adaptive federated reinforcement learning includes the following steps:

[0010] S1. Establish a scenario model, divide the actual train route data into scenarios, set the layout of base stations, intelligent reflective surfaces in tunnels and data centers, and determine the projection position of mobile relays along the line, base station transmission power and intelligent reflective surface phase representation.

[0011] S2. Establish a channel model. Using the orthogonal time-delay Doppler diversity multiplexing modulation mechanism, establish a channel model for each scenario by adopting a combination of partitioned waveguide and Rayleigh small-scale fading modeling method, and determine the total equivalent channel representation of mobile relay.

[0012] S3. Establish a signal transmission model. When the smart reflective surface is activated, determine the mobile relay receiving signal. Based on the equivalent channel, adopt orthogonal time-delay Doppler diversity multiplexing to directly map the information to the time-delay-Doppler domain and transmit it as orthogonal time-delay-Doppler pulses.

[0013] S4. Using a fixed time slot framework and power as an indicator of energy consumption, calculate the total power of all base stations, the total power of multiple mobile relays, and the total power of multiple smart reflective surfaces to establish an energy consumption model.

[0014] S5. With the goal of maximizing system energy efficiency, a time-averaged expression is adopted, and a state space is constructed using basic statistical observations and scenario coding to establish an optimization problem model.

[0015] S6. Construct an adaptive federated learning framework that immediately triggers global aggregation when there is a significant performance divergence among clients or a sudden change in channel distribution.

[0016] S7. Construct a dual-depth Q-network model for a showdown, define the state space, action space, and reward function, select actions for the online network, and estimate the target network.

[0017] Further, in step S1, when dividing the actual route data into scenarios, the route data of the Chengdu-Chongqing Railway Central Line, which is 8 kilometers long, was captured on Google Maps and divided into three scenarios: tunnel, town, and suburban. Among them, the tunnel is 800 meters long, the town scenario is 1600 meters long, and the suburban scenario is 5600 meters long.

[0018] The train is 200 meters long, and each carriage is equipped with a mobile relay, totaling... Carriages; evenly distributed along the railway line One base station; deployed inside the tunnel and at the exit. Each smart reflective surface; a data center is set up inside the scene, which is wiredly connected to each base station and smart reflective surface to coordinate the control and information interaction between the smart reflective surface and the base station.

[0019] Furthermore, the projected position along the line of the m-th mobile relay is represented as:

[0020] , ;

[0021] in, This is the fixed offset of the carriage relative to the train head;

[0022] The system performs control at the time slot level, with a single step duration of [missing information]. The transmit power of each base station is taken from a discrete set. The agent selects a power action in each time slot; for each intelligent reflective surface, online quantization phase control is used, with the phase vector set as:

[0023] ;

[0024] in, The number of controllable reflective units deployed for each smart reflective surface; each Taken from discrete sets: ,in, The number of discrete numbering steps that can be taken for the phase of the intelligent reflective surface.

[0025] Further, step S2 specifically includes the following steps: using an orthogonal time-delay Doppler diversity multiplexing modulation mechanism, when deploying a smart reflector, the direct link and the secondary cascaded link of the smart reflector are superimposed to form an equivalent complex channel response; at discrete times... At that time, the first The complex channel coefficients of each mobile relay are:

[0026] (1)

[0027] in, This indicates large-scale loss. For small-scale losses, Indicates the integer grid delay index on the delay-Doppler plane. Indicates the integer grid Doppler index on the delay-Doppler plane;

[0028] The large-scale fading in the three scenarios is shown in Equation (2):

[0029] (2)

[0030] in, The loss index represents the loss rate in different scenarios. For reference distance The normalization constant, Indicates shadow decay; the speed of a high-speed train is denoted as... The carrier frequency is denoted as The speed of light is denoted as The angle of incidence is denoted as , Let be the multipath propagation distance of the m-th mobile relay under the time-delay-Doppler index. Then the Doppler frequency shift can be expressed as:

[0031] ;

[0032] The tunnel is a waveguide structure, and its channel consists of a few propagable modes and weak scattering caused by the tunnel walls. Its small-scale fading can be expressed as:

[0033] (3)

[0034] For the first The complex amplitude of a waveguide mode, It is the complex attenuation constant. For the first Doppler frequency shift along the path Indicates the total number of paths; Indicates the first The instantaneous propagation distance of each mobile relay and its corresponding modal equivalent propagation path; This represents the zero-mean weak scattering component;

[0035] In urban and suburban scenarios, Rayleigh fading is used to model its small-scale fading, and... Factors controlling the ratio of line-of-sight propagation to non-line-of-sight propagation:

[0036] (4)

[0037] in, This indicates the energy ratio of line-of-sight propagation to non-line-of-sight propagation. Includes Doppler frequency shift caused by motion, Indicates the initial phase; This represents the scattering component of non-line-of-sight propagation. This indicates that at time slot t, the first... Multipath propagation distance of mobile relays under delay-Doppler indexing;

[0038] When calculating the channels of a smart reflective surface, when deploying When the intelligent reflective surface of the first controllable reflective unit is used, the first The phase shift matrix of the block intelligent reflective surface can be expressed as:

[0039] ;

[0040] Definition of the first The base station to the The narrowband matrix of the intelligent reflective surface is Intelligent reflective surface to the first The narrowband vector of each mobile relay is The two sub-links each employ a fading model matched to the scene; due to the phase of the intelligent reflective surface... Controlled online by an intelligent agent, the first The cascaded equivalent channel of a smart reflective surface is:

[0041] (5)

[0042] The Doppler amplitude of the link between the base station and the smart reflective surface is 0, and its narrowband matrix is... It can be considered quasi-static and will not change with and change, Indicates transpose;

[0043] The equivalent complex channel response of a direct link can be denoted as: Then, for the first The total equivalent channel of a mobile relay can be expressed as:

[0044] (6)

[0045] For each mobile relay, when calculating the total equivalent channel, the cascaded equivalent channels from one base station and all smart reflective surfaces need to be considered.

[0046] Further, step S3 specifically includes the following steps:

[0047] Will Let be denoted as at discrete time The emission symbol, then the first The signals received by a mobile relay are represented as follows:

[0048] (7)

[0049] in, In the baseband model, it is a complex modulation symbol. The magnitude of this complex number does not directly represent the power; power refers to the square of the amplitude. Represent a complex normal distribution;

[0050] When the smart reflective surface is activated, the incident signal from the base station is reflected by the smart reflective surface to form a secondary link. The mobile relay then receives the following signal:

[0051] (8)

[0052] The intelligent reflective surface is a near-passive scatterer that does not introduce independent thermal noise terms, but simply changes the phase and amplitude of the original electromagnetic wave before reflecting it back.

[0053] Based on the equivalent channel of formula (6), orthogonal time-delay Doppler diversity multiplexing is adopted to directly map the information to the time-delay-Doppler domain and transmit it as orthogonal time-delay-Doppler pulses.

[0054] Furthermore, after time-delay-Doppler domain equalization, the residual symbol-level interference / carrier-level interference of the frequency components / time slot grids equivalent to the time-frequency grid within the same cell is ignored. Only the cross-cell interference term is retained and used for signal-to-interference-plus-noise ratio and rate modeling in formulas (9)-(11). Interference between cells It can be represented as:

[0055] (9)

[0056] in, To allocate transmit power to the delay-Doppler unit, It serves the first A mobile relay base station; based on total channel equivalent gain , No. The signal-to-interference-plus-noise ratio (SIR) of each mobile relay is:

[0057] (10)

[0058] in, This represents the noise power at the receiving end. Interference between different cells;

[0059] The total transmission rate of the system is expressed as:

[0060] (11)

[0061] in, The number of Doppler units, The number of delay units, To assign indicator variables, if in time slot No. The first community will be the first The first Doppler dimension and the second The time-delay-Doppler unit, composed of the time delay dimensions, is scheduled to the first... A mobile relay Otherwise, it is 0.

[0062] Further, step S4 specifically includes the following steps:

[0063] A fixed time slot framework is adopted, with a time slot duration of [missing information]. The duty cycles of base stations, mobile relays, and smart reflective surfaces remain constant in each time slot, and the duty cycles do not change with the policy; power is used as a metric for energy consumption, and the first... Each base station at discrete time The total transmit power of the mobile relay s serving it is Its power amplifier efficiency is If the system has a total of ; If there are 1 base station, then the total power of all base stations is:

[0064] (12)

[0065] in, This refers to the fixed circuit power of the base station;

[0066] Calculate Total power of mobile relays:

[0067] (13)

[0068] in, Its downlink forwarding power, For the power amplifier efficiency of mobile repeaters, For fixed circuit power;

[0069] Multiple smart reflective surfaces were deployed in urban and tunnel scenarios, each containing... There are several controllable reflection units, and the driving power of each reflection unit is given by... The fixed circuit power of each smart reflective surface is ,but The total power of the intelligent reflective surfaces is:

[0070] (14).

[0071] Further, step S5 specifically includes the following steps:

[0072] Energy efficiency of the system As an optimization target It is expressed as the quotient of the system's total downlink data transmission rate and total power;

[0073] (15)

[0074] In each time slot ,make Let be the power action vector for all base stations; Let be the set of phase matrices for all intelligent reflective surfaces; with the goal of maximizing energy efficiency and using a time-averaged representation, let the scenario of a high-speed train passing through require a total of _____. If there are multiple time slots, the optimization problem can be expressed as:

[0075] (16)

[0076] (16a)

[0077] (16b)

[0078] (16c)

[0079] Formula (16a) states that in any time slot, the transmit power of each base station must be taken from the discrete power level allowed by the hardware and must not exceed the maximum power generation; Formula (16b) stipulates that the phase angle of each smart reflective surface unit must be taken from the discrete phase set. Selected from; Formula (16c) indicates that in time slot Inside, base station The actual set of units used The sum of the allocated transmit power equals its total transmit power budget. That is, power is conserved;

[0080] Discrete power sets and For the action space, a state space is constructed using basic statistical observations and scene coding, and outputs in each time slot. and The reward function directly adopts the energy efficiency index, making the maximization of the agent's cumulative reward equivalent to the maximization of energy efficiency.

[0081] Further, step S6 specifically includes the following steps:

[0082] When the basic aggregation cycle is reached At that time, an aggregation is completed, that is... For the first Each round, for each client Get the most recent window The energy efficiency moving average of each time slot is used as the performance score. ,remember To estimate the dispersion across clients, exponential moving average statistics are used to estimate its steady-state level and fluctuation amplitude:

[0083] (17)

[0084] (18)

[0085] And set an online adaptive threshold:

[0086] (19)

[0087] when At that time, the global model is aggregated;

[0088] In the Each round, for each client In the recent window The empirical distribution is constructed from the basic statistical observations collected within the internal data. The control center maintains the reference distribution from the previous round. Define the distribution distance as the Jensen-Shannon divergence between each client and the reference distribution:

[0089] (20)

[0090] Pick the median of And use exponential moving average statistics to analyze Online estimation of steady-state level and fluctuation amplitude:

[0091] (twenty one)

[0092] (twenty two)

[0093] And set an online adaptive threshold:

[0094] (twenty three)

[0095] when At that time, the global model is aggregated;

[0096] When the global model is aggregated, three factors' performance scores are considered. Effective sample size and data staleness Set the number of clients to:

[0097] ;

[0098] For each client In the The effective sample size of the local training window before the second aggregation is:

[0099] ;

[0100] in, This refers to the actual number of update steps used for learning within the window, and is relevant to clients with a large number of samples and stable training. Larger; clients with fewer samples or greater volatility, Suppressed;

[0101] make For each client In the The local training window before the second aggregation has the following data staleness:

[0102] ;

[0103] in, Indicates client Last time participating in aggregation or completing local window training to the [number]th The number of rounds experienced between the first and second aggregations, for the first aggregation... The client in the first For each aggregation round, its parameter weights can be expressed as:

[0104] (twenty four)

[0105] This represents the normalized performance score. This represents the proportion of normalized effective samples. The global parameter, representing the normalized data staleness, after merging the parameters reported by each client, can be expressed as:

[0106] (25).

[0107] Further, step S7 specifically includes the following steps:

[0108] The first A client in a time slot Basic statistical observations and scene coding Defined as state space ;

[0109] (26)

[0110] in, One-hot encoding for tunnels, towns, and suburbs; Includes average effective signal-to-interference-plus-noise ratio Average transmission rate Average power Average interference intensity Scheduled delay - percentage of Doppler grid cells Normalized index of the gear selected by the action client , It can be represented as:

[0111] (27)

[0112] Action space Set as base station power action and intelligent reflective surface phase action;

[0113] (28)

[0114] Among them, for the first For a single base station, its power For the first A smart reflective surface For a control unit , Dimensions Action space size ;

[0115] If each mobile relay is configured to switch at most once per time slot, then the handover penalty can be expressed as:

[0116] (29)

[0117] in, The minimum number of switching operations during train operation will not be penalized for these switching operations; Indicates for the first For a mobile relay, the reward is 1 if a cell handover occurs, and 0 otherwise. The reward function can be expressed as:

[0118] (30)

[0119] Indicates the transmission rate. The Q-value of the dual-depth Q-network, representing the interference between cells, is expressed as:

[0120] (31)

[0121] in, For state value branches, For action advantage branches; each client Maintain local experience replay buffer When the buffer size reaches the preheating threshold After that, each step from Uniform random sampling;

[0122] A dual estimator employing a dual-depth Q-network is used in time slots. The objective of single-step timing difference is defined as follows:

[0123] (32)

[0124] in, To provide timely rewards, As a discount factor, A set of actions; For online Network, used in Select an action. For the goal The network is used to describe this action. Valuation is performed; the loss function can be expressed as:

[0125] (33).

[0126] Compared with the prior art, the beneficial effects of the present invention are:

[0127] This method has practical significance in optimizing communication systems in various scenarios, such as high-speed railway trains traversing tunnels, towns, and suburbs. It not only eliminates the impact of rapid channel time-varying and frequent scene switching on federated aggregation performance, but also reduces the communication and computational overhead caused by redundant synchronization. Furthermore, it can accurately achieve joint optimization decisions on base station power and intelligent reflector phase. This algorithm performs reinforcement learning decisions in a discrete action space. Simulation results show that, with a deployment scale of 5 base stations and 3 intelligent reflectors, compared to the baseline method, the system data transmission rate is increased by an average of approximately 15%, the total system energy consumption is reduced by an average of approximately 12%, and the number of adaptive aggregation triggers is reduced by approximately 30% compared to the fixed-period scheme. This method achieves a significant improvement in communication optimization performance while ensuring the algorithm's real-time response capability in high-speed, time-varying environments. Attached Figure Description

[0128] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0129] Figure 1 A flowchart illustrating the steps of an optimization method for a high-speed railway communication system based on adaptive federated reinforcement learning;

[0130] Figure 2 This is a flowchart for adaptive federated reinforcement learning. Detailed Implementation

[0131] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0132] Research has found that, in the actual context of high-speed railway communication scenarios, using traditional fixed-period aggregation and equal-weighted parameter averaging methods for federated model optimization inevitably triggers a large amount of unnecessary global synchronization during stable channel phases. This results in frequent redundant communication overhead, increasing the transmission burden between base stations and data centers, and also lowering the overall performance of the global model due to the equal weighting of inferior local models. Since the channel distribution changes of trains in different scenario sections are quantifiable, and the differences in local training performance among clients can be statistically analyzed in real time, the performance scores, effective sample sizes, and channel distribution drift of each client can be extracted and used as criteria for adaptive aggregation to eliminate the redundant synchronization and model drift problems caused by fixed-period aggregation. Based on this, joint optimization decisions regarding power actions and intelligent reflector phase are then made.

[0133] This invention first establishes a system model. In a high-speed railway communication scenario, when a train travels at high speed along a predetermined route, the 8-kilometer-long central line of the Chengdu-Chongqing Railway is divided into scenarios. Channel models are established for three typical scenarios—tunnels, towns, and suburbs—using path loss and small-scale fading models. At this point, base stations and intelligent reflective surfaces in each scenario constitute the main nodes of the communication system. By collecting channel state information over N time slots, the channel variation patterns under each scenario throughout the train's journey can be obtained. The number of time slots collected, N, must at least cover the duration of the train's complete traversal of one scenario segment. The channel state, base station transmit power, and intelligent reflective surface phase configuration under each scenario are jointly modeled to construct a joint optimization problem with maximizing system energy efficiency as the optimization objective. Discrete constraints are applied to transmit power and phase variables, resulting in a mixed-integer non-convex fractional programming problem, providing a unified optimization framework for the subsequent design of reinforcement learning decision-makers.

[0134] Secondly, an adaptive federated reinforcement learning approach is adopted. Each base station acts as a local client, using the current time slot's channel state and scene coding as state inputs. Power and phase control actions are output through a local adversarial dual deep Q network (D3QN) agent. After several rounds of local training, the local agents upload their model parameters to the data center. The data center adaptively determines whether to trigger aggregation and the aggregation weights of each client's parameters based on each client's performance score, effective sample size, and data staleness. When significant bifurcation occurs in client performance or abrupt changes in channel distribution, global aggregation is immediately triggered to align the local models. During stable channel phases, unnecessary synchronization is reduced to lower communication overhead. In high-speed rail scenarios, trains traverse different sections at high speeds, and channel states frequently switch. This algorithm can perceive these distribution changes in real time and respond adaptively, while redundant aggregation in stable sections is completely suppressed. The communication system does not incur additional computational and transmission loads for synchronization operations during stable channel phases.

[0135] Based on this, the present invention provides an optimization method for a high-speed railway communication system based on adaptive federated reinforcement learning, such as... Figures 1-2 As shown, it includes the following steps:

[0136] S1. Establish the scene model:

[0137] This invention extracts route data from Google Maps for the 8-kilometer-long central section of the Chengdu-Chongqing Railway, dividing it into three classic scenarios: tunnels, towns, and suburbs. The tunnels are 800 meters long, the town scenario is 1600 meters long, and the suburban scenario is 5600 meters long. A 200-meter-long high-speed train sequentially and at a constant speed passes through these three scenarios, with each carriage equipped with a mobile relay. Carriages. Evenly distributed along the railway line. One base station. To enhance coverage in tunnels and urban areas, base stations will be deployed inside tunnels and at exits. Each smart reflective surface is equipped with a data center within the scene, which is wiredly connected to each base station and smart reflective surface to coordinate control and information interaction between the smart reflective surfaces and base stations.

[0138] The projected position along the line of the m-th mobile relay can be represented as:

[0139] , ;

[0140] in, This is the fixed offset of the carriage relative to the train head.

[0141] The system performs control at the time slot level, with a single step duration of [missing information]. The transmit power of each base station is taken from a discrete set. The agent selects the power action in each time slot. For each intelligent reflective surface, online quantized phase control is used, with the phase vector set as follows:

[0142] ;

[0143] in, The number of controllable reflective units deployed for each smart reflective surface. Each Taken from discrete sets: ,in, The number of discrete numbering steps that can be taken for the phase of the intelligent reflective surface.

[0144] S2. Establish the channel model:

[0145] To accurately reflect the propagation characteristics of high-speed trains under conditions of high-speed movement, scene switching, and heterogeneous links, this invention employs a combined modeling approach of partitioned waveguides and Rayleigh small-scale fading for three typical environments: tunnels, urban areas, and suburban areas. The system utilizes Orthogonal Delay-Doppler Multiplexing (ODDM) modulation technology to overcome the severe biselective fading caused by the high-speed movement of high-speed trains. When intelligent reflective surfaces are deployed, the direct link and the secondary cascaded link of the intelligent reflective surface are superimposed to form an equivalent complex channel response. At discrete time intervals... At that time, the first The complex channel coefficients of each mobile relay are:

[0146] (1)

[0147] in, This indicates large-scale loss. For small-scale losses, Indicates the integer grid delay index on the delay-Doppler plane. Represents the integer grid Doppler index on the delay-Doppler plane.

[0148] Large-scale fading in wireless propagation modeling can essentially be uniformly represented by a logarithmic distance path loss model. Therefore, the large-scale fading in the three scenarios is shown in Equation (2):

[0149] (2)

[0150] in, The loss index represents the loss rate in different scenarios. For reference distance The normalization constant, This indicates the fading of the shadow.

[0151] The speed of high-speed trains is denoted as The carrier frequency is denoted as The speed of light is denoted as The angle of incidence is denoted as , Let be the multipath propagation distance of the m-th mobile relay under the time-delay-Doppler index. Then the Doppler frequency shift can be expressed as:

[0152] .

[0153] A tunnel can be approximated as a waveguide structure, and its channel consists of a few propagable modes and weak scattering caused by the tunnel walls. Its small-scale fading can be expressed as:

[0154] (3)

[0155] For the first The complex amplitude of a waveguide mode, It is the complex attenuation constant. For the first Doppler frequency shift along the path Indicates the total number of paths; Indicates the first The instantaneous propagation distance of each mobile relay and its corresponding modal equivalent propagation path; This represents the zero-mean weak scattering component, reflecting the random perturbation of wall roughness and local scattering.

[0156] In urban and suburban scenes, both line-of-sight propagation and non-line-of-sight propagation often exist. Therefore, Rayson fading is used to model its small-scale fading, and... The factor controls the ratio of line-of-sight propagation to non-line-of-sight propagation.

[0157] (4)

[0158] in, This indicates the energy ratio of line-of-sight propagation to non-line-of-sight propagation. Includes Doppler frequency shift caused by motion, Indicates the initial phase. This represents the scattering component of non-line-of-sight propagation. This indicates that at time slot t, the first... Mobile relay in delay-Doppler index in multipath propagation distance.

[0159] When calculating the channel of a smart reflector, since the smart reflector operates under narrowband conditions, the channel gain remains constant across the entire frequency band, ignoring the effects of frequency-selective fading. When deployed... When the intelligent reflective surface of the first controllable reflective unit is used, the first The phase shift matrix of the block intelligent reflective surface can be expressed as:

[0160] .

[0161] Definition of the first The base station to the The narrowband matrix of the intelligent reflective surface is (Narrowband refers to a technology that uses a narrow frequency range for data transmission in communication. It typically has a lower transmission rate and limited bandwidth, but features wide coverage, strong anti-interference capabilities, and low cost.) Intelligent reflective surfaces to the... The narrowband vector of each mobile relay is The two sub-links each employ a fading model matched to the scene. Due to the phase of the smart reflector... Controlled online by an intelligent agent, the first The cascaded equivalent channel of a smart reflective surface is:

[0162] (5)

[0163] Since the positions of the base station and the smart reflector are fixed, the Doppler amplitude of the link between the base station and the smart reflector is approximately zero, therefore its narrowband matrix... It can be considered quasi-static and will not change with and change, This indicates transpose.

[0164] The equivalent complex channel response of a direct link can be denoted as: Then, for the first The total equivalent channel of a mobile relay can be expressed as:

[0165] (6)

[0166] For each mobile relay, the received signal originates from the same base station, but the signal emitted by a single base station may be reflected by multiple smart reflective surfaces simultaneously. Therefore, when calculating the total equivalent channel, the cascaded equivalent channels from a single base station and all smart reflective surfaces need to be considered.

[0167] S3. Establish a signal transmission model:

[0168] Will Let be denoted as at discrete time The emission symbol, then the first The signal received by a mobile relay can be represented as:

[0169] (7)

[0170] in, In the baseband model, it is a complex modulation symbol. The magnitude of this complex number does not directly represent the power; power refers to the square of the amplitude. This represents a complex normal distribution.

[0171] When the smart reflective surface is activated, the incident signal from the base station is reflected by the smart reflective surface to form a secondary link. The mobile relay then receives the following signal:

[0172] (8)

[0173] Intelligent reflective surfaces are near-passive scatterers that do not introduce independent thermal noise terms, but simply change the phase and amplitude of the original electromagnetic waves before reflecting them back.

[0174] To suppress inter-carrier interference (ICI) and inter-symbol interference (ISI) caused by high-speed movement, this invention employs orthogonal delay-Doppler diversity multiplexing based on the equivalent channel in formula (6). Information is directly mapped to the delay-Doppler domain and transmitted as orthogonal delay-Doppler pulses. For ideal pulse shaping and delay / Doppler falling within an integer grid, ICI and ISI are significantly suppressed, allowing the rate-signal-to-noise ratio (SNR) relationship to be approximated as being evaluated independently for each DD-bin (delay-Doppler unit). Therefore, after equalization in the DD domain (delay-Doppler domain), this invention approximately ignores the residual ICI / ISI (residual symbol-level interference / carrier-level interference) of frequency components / time slot grids equivalent to the time-frequency grid within the same cell, retaining only the cross-cell interference term for SNR and rate modeling in formulas (9)-(11). Inter-cell interference It can be represented as:

[0175] (9)

[0176] in, To allocate transmit power to the delay-Doppler unit, It serves the first A mobile relay base station. Based on total channel equivalent gain. , No. The signal-to-interference-plus-noise ratio (SIR) of each mobile relay is:

[0177] (10)

[0178] in, This represents the noise power at the receiving end. To prevent interference between different communities.

[0179] Furthermore, the total transmission rate of the system is expressed as:

[0180] (11)

[0181] in, The number of Doppler units, The number of delay units, To assign indicator variables, if in time slot No. Each community will DD-bin That is, the first The first Doppler dimension and the second The time-delay-Doppler unit, composed of the time delay dimensions, is scheduled to the first... A mobile relay Otherwise, it is 0.

[0182] S4. Establish an energy consumption model:

[0183] Under the aforementioned scenarios and channel conditions, the energy consumption of high-speed rail communication systems mainly originates from base stations and mobile relays along the line. Furthermore, for future intelligent high-speed rail communication systems, the large-scale deployment of intelligent reflective surfaces will make their power consumption significant. This invention employs a fixed time-slot framework, with a time-slot duration of… The duty cycle of the base station, mobile relay, and smart reflective surface remains constant in each time slot, and does not change with the policy. Therefore, power is used as a metric for energy consumption. Let the... Each base station at discrete time The total transmit power of the mobile relay s serving it is Its power amplifier efficiency is If the system has a total of If there are 1 base station, then the total power of all base stations is:

[0184] (12)

[0185] in, This refers to the fixed circuit power of the base station.

[0186] Similarly, it can be calculated Total power of mobile relays:

[0187] (13)

[0188] in, Its downlink forwarding power, For the power amplifier efficiency of mobile repeaters, For fixed circuit power.

[0189] Multiple smart reflective surfaces were deployed in urban and tunnel scenarios, each containing... There are several controllable reflective units. Since the intelligent reflective surface operates in passive mode, its main energy consumption comes from the control unit and the PIN diode switch. Let the driving power of each reflective unit be... The fixed circuit power of each smart reflective surface is ,but The total power of the intelligent reflective surfaces is:

[0190] (14)

[0191] This assumes that the instantaneous overhead of phase switching is relatively small and is incorporated into... Therefore, the power of the intelligent reflective surface has no explicit functional relationship with the phase control. It should be noted that this invention only controls the transmission power; the fixed circuit power of each component remains constant. The fixed power is listed for reference in subsequent research; therefore, the fixed transmission power of each component in this invention is set to 0.

[0192] S5. Model the optimization problem:

[0193] In this invention, the optimization objective is not limited to increasing the data transmission rate, but also considers the system's energy consumption, which is a classic multi-objective optimization problem. Multi-objective optimization requires manually setting the weights of transmission rate and power, increasing the difficulty of the solution and introducing more subjective factors into the system. Therefore, this invention focuses on the system's energy efficiency. As an optimization target. It can be expressed as the quotient of the total downlink data transmission rate and the total power of the system.

[0194] (15)

[0195] In each time slot ,make Let be the power action vector for all base stations. This represents the set of phase matrices for all intelligent reflective surfaces. The goal is to maximize energy efficiency, and a time-averaged representation is used to reflect statistical steady-state performance in high-speed environments. Assume that a high-speed train needs to travel through the scenario studied in this invention for a total of [number missing] hours. If there are multiple time slots, the optimization problem can be expressed as:

[0196] (16)

[0197] (16a)

[0198] (16b)

[0199] (16c)

[0200] Formula (16a) states that in any time slot, the transmit power of each base station must be taken from the discrete power level allowed by the hardware and must not exceed the maximum power generation; Formula (16b) stipulates that the phase angle of each smart reflective surface unit must be taken from the discrete phase set. Selected from; Formula (16c) indicates that in time slot Inside, base station The actual set of units used The sum of the allocated transmit power equals its total transmit power budget. That is, power is conserved.

[0201] Formula (16) is essentially a joint optimization power phase control problem for a fractional target: the numerator is the transmission rate, and the denominator is the sum of the affine and constant terms of power consumption. Different base station powers enter the signal-to-interference-plus-noise ratio (SNR) of each link through interference coupling and affect the global rate, causing the target to... It exhibits non-convex coupling. Furthermore, The problem becomes a mixed-integer nonconvex fractional programming problem (MINLP) by using discrete sets. Introducing discrete variables for the phase of intelligent reflective surfaces further increases the combinatorial dimension, causing the search space to expand exponentially. Even with continuous relaxations such as successive convex approximation (SCA), iterative solutions to large-scale nonconvex subproblems are required in each time slot, which struggles to meet the real-time and robustness requirements of high-speed rail scenarios.

[0202] Given the NP-hard nature of the optimization problem and the rapid time-varying characteristics of high-speed rail channels, this invention employs a dual-depth Q-network to solve the problem. This is achieved using discrete power sets... and For the action space, a state space is constructed using basic statistical observations and scene coding, and outputs in each time slot. and The reward function directly adopts the energy efficiency index, making the maximization of the agent's cumulative reward equivalent to the maximization of energy efficiency. This paradigm has been proven to maintain scalability and stable convergence in high-speed time-varying environments and large-scale state-action spaces.

[0203] S6. Construct an adaptive federated learning framework:

[0204] When the basic aggregation cycle is reached At that time, an aggregation is completed, that is... When the value functions learned by clients in different scenarios on their respective local distributions have clearly diverged, it is necessary to immediately aggregate them to align perspectives, share knowledge of new scenarios, and prevent some clients from overfitting to the old distribution. For the [missing information]... Each round, for each client Get the most recent window The energy efficiency moving average of each time slot is used as the performance score. ,remember To determine the dispersion across clients, we use exponential moving average statistics to estimate its steady-state level and fluctuation amplitude:

[0205] (17)

[0206] (18)

[0207] And set an online adaptive threshold:

[0208] (19)

[0209] Therefore, when At that time, the global model is aggregated.

[0210] When the distribution of recent baseline statistical observations deviates significantly from the reference distribution, the global model should immediately align to share knowledge of the new scenario. However, in long-term stable periods, baseline statistical observations will exhibit a relatively stable state; therefore, the system should reduce unnecessary synchronization and lower communication load. Each round, for each client In the recent window The empirical distribution is constructed from the basic statistical observations collected within the internal data. The control center maintains the reference distribution from the previous round. Define the distribution distance as the Jensen-Shannon divergence between each client and the reference distribution:

[0211] (20)

[0212] Pick the median of And use exponential moving average statistics to analyze Online estimation of steady-state level and fluctuation amplitude:

[0213] (twenty one)

[0214] (twenty two)

[0215] And set an online adaptive threshold:

[0216] (twenty three)

[0217] Therefore, when At that time, the global model is aggregated.

[0218] When the global model is aggregated, three main factors are considered: performance scores. Effective sample size and data staleness The performance score directly characterizes the client's immediate contribution to the global optimization objective in the current segment, ensuring that aggregation proceeds in the right direction. The effective sample size reflects the sufficiency and variance level of the data upon which the update depends; the more abundant the sample, the more robust the estimate, thus suppressing small-sample noise. The number of clients is set as follows:

[0219] ;

[0220] For each client In the The effective sample size of the local training window before the second aggregation is:

[0221] ;

[0222] in, This refers to the actual number of update steps used for learning within the window. For clients with a large number of samples and stable training, its... Larger; clients with fewer samples or greater volatility, It is suppressed.

[0223] Since the number of clients in the scenario model of this invention is relatively small, a synchronous, fully participating set setting is adopted to simplify the calculation process. Therefore, the aggregation time of each client is approximately the same, hence let For the purpose of potential future research, this invention still provides... The general definition. For each client. In the The local training window before the second aggregation has the following data staleness:

[0224] ;

[0225] in, Indicates client Last time participating in aggregation or completing local window training to the [number]th The number of rounds that occur between each aggregation. The longer an update has not been participated in or the more delayed its postback, the more rounds it goes through. The larger. Therefore, the first The client in the first For each aggregation round, its parameter weights can be expressed as:

[0226] (twenty four)

[0227] This represents the normalized performance score. This represents the proportion of normalized effective samples. The global parameter, representing the normalized data staleness, after merging the parameters reported by each client, can be expressed as:

[0228] (25)

[0229] S7. Construct a dual-depth Q-network model for competition, namely the D3QN network model:

[0230] State space: This invention will use the first A client in a time slot Basic statistical observations and scene coding Defined as state space .

[0231] (26)

[0232] in, One-hot encoding for tunnels, towns, and suburbs. Includes average effective signal-to-interference-plus-noise ratio Average transmission rate Average power Average interference intensity Scheduled delay - Doppler unit percentage Normalized index of the gear selected by the action client .therefore, It can be represented as:

[0233] (27)

[0234] Action space: This invention utilizes action space Set as base station power action and smart reflective surface phase action.

[0235] (28)

[0236] Among them, for the first For a single base station, its power For the first A smart reflective surface For a control unit .therefore Dimensions Action space size .

[0237] Reward Function: Since the research scenario of this invention is based on multiple cells, it is necessary to consider the handover cost of the high-speed train between cells and the interference between cells. Theoretically, each mobile relay can always receive service from entering the scenario to leaving the scenario, and the number of handovers of the high-speed train within the scenario is... However, if the reward function only includes energy efficiency, the slight advantage in energy efficiency might lead to multiple handovers by high-speed trains within the overlapping area of ​​base stations. Each handover incurs control overhead and momentary interruptions. If each mobile relay is set to handover at most once per time slot, the handover penalty can be expressed as:

[0238] (29)

[0239] in, The minimum number of switching operations during train operation will not be penalized. Indicates for the first For a mobile relay, the reward value is 1 if a cell handover occurs, and 0 otherwise. Therefore, the reward function can be expressed as:

[0240] (30)

[0241] Indicates the transmission rate. The Q-value of the dual-depth Q-network, representing the interference between cells, is expressed as:

[0242] (31)

[0243] in, For state value branches, This is the action advantage branch. Each client... Maintain local experience replay buffer When the buffer size reaches the preheating threshold After that, each step from Uniform random sampling.

[0244] In order to suppress If the value is too high, this invention employs a dual estimator using a dual-depth Q-network. It operates within time slots. The objective of single-step timing difference is defined as follows:

[0245] (32)

[0246] in, To provide timely rewards, As a discount factor, For a set of actions. For online Network, used in Select an action. For the goal The network is used to describe this action. Perform a valuation. Note the parameters uploaded by the client-side model. For online The set of learnable parameters of the network, the target Network parameters do not need to be uploaded.

[0247] The loss function can be expressed as:

[0248] (33).

[0249] Example 1: Joint optimization of base station power and intelligent reflective surface phase in multiple scenarios of the Chengdu-Chongqing high-speed railway central line.

[0250] This implementation example uses a typical 8-kilometer section of the Chengdu-Chongqing Railway's central line as the application scenario, encompassing three scenarios: tunnels, towns, and suburbs. Five base stations and three intelligent reflective surfaces are evenly deployed along the line, with a high-speed train traveling at a constant speed of 400 km / h throughout the entire route. Each base station deploys a dual-depth Q-network agent, taking the current channel state and scenario coding as input, and outputting power and intelligent reflective surface phase control commands. The data center adaptively triggers global aggregation based on the performance scores of each client, the effective sample size, and the degree of channel distribution drift. Simulation results show that compared to the baseline scheme, the system's data transmission rate is increased by an average of approximately 15%, the total system energy consumption is reduced by an average of approximately 12%, and the number of adaptive aggregation triggers is reduced by approximately 30% compared to the fixed-period scheme, achieving a balance between green energy saving and efficient collaboration while ensuring communication quality.

[0251] Example 2: Enhanced communication coverage assisted by intelligent reflective surfaces in tunnel scenarios.

[0252] This implementation example addresses the coverage bottleneck problem in tunnel scenarios. An intelligent reflective surface with 64 controllable reflective units is deployed at the entrance, middle, and exit of an 800m long tunnel. When a train travels at high speed through the tunnel, the direct link is severely obstructed. The intelligent reflective surface uses adaptive phase modulation to reflect the incident signal from the base station, forming a secondary link that is superimposed on the direct link to compensate for the link loss caused by tunnel obstruction. Simulation results show that after deploying the intelligent reflective surface, the signal-to-interference-plus-noise ratio (SIR) of the train in the middle section of the tunnel is improved by an average of approximately 10dB compared to the solution without the intelligent reflective surface. The data transmission rate is significantly improved. Furthermore, when switching between tunnel entrance and exit scenarios, the adaptive aggregation mechanism can promptly detect sudden changes in channel distribution and trigger global parameter alignment, effectively avoiding overfitting of the model to older scenarios and ensuring the stability and real-time performance of the communication system in tunnel scenarios.

[0253] It will be understood by those skilled in the art that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. An optimization method for a high-speed railway communication system based on adaptive federated reinforcement learning, characterized in that, Includes the following steps: S1. Establish a scenario model, divide the actual train route data into scenarios, set the layout of base stations, intelligent reflective surfaces in tunnels and data centers, and determine the projection position of mobile relays along the line, base station transmission power and intelligent reflective surface phase representation. S2. Establish a channel model. Using the orthogonal time-delay Doppler diversity multiplexing modulation mechanism, establish a channel model for each scenario by adopting a combination of partitioned waveguide and Rayleigh small-scale fading modeling method, and determine the total equivalent channel representation of mobile relay. S3. Establish a signal transmission model. When the smart reflective surface is activated, determine the mobile relay receiving signal. Based on the equivalent channel, adopt orthogonal time-delay Doppler diversity multiplexing to directly map the information to the time-delay-Doppler domain and transmit it as orthogonal time-delay-Doppler pulses. S4. Using a fixed time slot framework and power as an indicator of energy consumption, calculate the total power of all base stations, the total power of multiple mobile relays, and the total power of multiple smart reflective surfaces to establish an energy consumption model. S5. With the goal of maximizing system energy efficiency, a time-averaged expression is adopted, and a state space is constructed using basic statistical observations and scenario coding to establish an optimization problem model. S6. Construct an adaptive federated learning framework that immediately triggers global aggregation when there is a significant performance divergence among clients or a sudden change in channel distribution. S7. Construct a dual-depth Q-network model for the competition, define the state space, action space, and reward function, select actions for the online network, and evaluate the target network. Step S6 specifically includes the following steps: When the basic aggregation cycle is reached, one aggregation is completed. For the first Each round, for each client Get the most recent window The energy efficiency moving average of each time slot is used as the performance score. ,remember To estimate the dispersion across clients, exponential moving average statistics are used to estimate its steady-state level and fluctuation amplitude: （17） （18） And set an online adaptive threshold: （19） when At that time, the global model is aggregated; In the Each round, for each client In the recent window The empirical distribution is constructed from the basic statistical observations collected within the internal data. The control center maintains the reference distribution from the previous round. Define the distribution distance as the Jensen-Shannon divergence between each client and the reference distribution: （20） Pick the median of And use exponential moving average statistics to... Online estimation of steady-state level and fluctuation amplitude: （21） （22） And set an online adaptive threshold: （23） when At that time, the global model is aggregated; When the global model is aggregated, three factors' performance scores are considered. Effective sample size and data staleness Set the number of clients to: ； For each client In the The effective sample size of the local training window before the second aggregation is: ； in, This refers to the actual number of update steps used for learning within the window, and is relevant to clients with a large number of samples and stable training. Larger; clients with fewer samples or greater volatility, Suppressed; make For each client In the The local training window before the second aggregation has the following data staleness: ； in, Indicates the client Last time participating in aggregation or completing local window training to the [number]th The number of rounds experienced between the first and second aggregations, for the first aggregation... The client in the first For each aggregation round, the parameter weights are expressed as follows: （24） This represents the normalized performance score. This represents the proportion of normalized effective samples. The global parameter, representing the normalized data staleness, after merging the parameters reported by each client, is as follows: （25）。 2. The optimization method for a high-speed railway communication system based on adaptive federated reinforcement learning according to claim 1, characterized in that: In step S1, when dividing the actual route data into scenarios, the route data of the Chengdu-Chongqing Railway Central Line, which is 8 kilometers long, was captured on Google Maps and divided into three scenarios: tunnel, town, and suburban. Among them, the tunnel is 800 meters long, the town scenario is 1600 meters long, and the suburban scenario is 5600 meters long. The train is 200 meters long, and each carriage is equipped with a mobile relay, totaling... Carriages; evenly distributed along the railway line One base station; deployed inside the tunnel and at the exit. Each smart reflective surface; a data center is set up inside the scene, which is wiredly connected to each base station and smart reflective surface to coordinate the control and information interaction between the smart reflective surface and the base station.

3. The optimization method for a high-speed railway communication system based on adaptive federated reinforcement learning according to claim 2, characterized in that, The projected position along the line of the m-th mobile relay is represented as: ， ； in, This is the fixed offset of the carriage relative to the train head; The system performs control at the time slot level, with a single step duration of [missing information]. The transmit power of each base station is taken from a discrete set. The agent selects a power action in each time slot; for each intelligent reflective surface, online quantization phase control is used, with the phase vector set as: ； in, The number of controllable reflective units deployed for each smart reflective surface; each Taken from discrete sets: ,in, The number of discrete numbering steps that can be taken for the phase of the intelligent reflective surface.

4. The optimization method for a high-speed railway communication system based on adaptive federated reinforcement learning according to claim 1, characterized in that: Step S2 specifically includes the following steps: using an orthogonal time-delay Doppler diversity multiplexing modulation mechanism, when a smart reflector is deployed, the direct link and the secondary cascaded link of the smart reflector are superimposed to form an equivalent complex channel response; at discrete times, the complex channel coefficient of the i-th mobile relay is: （1） in, This indicates large-scale loss. For small-scale losses, Indicates the integer grid delay index on the delay-Doppler plane. Indicates an integer grid Doppler index on the delay-Doppler plane; The large-scale fading in the three scenarios is shown in Equation (2): （2） in, The loss index represents the loss rate in different scenarios. For reference distance Normalization constant, Indicates shadow fading; the speed of the high-speed train is denoted as , and the carrier frequency is denoted as . The speed of light is denoted as The angle of incidence is denoted as , Let be the multipath propagation distance of the m-th mobile relay under the time-delay-Doppler index, then the Doppler frequency shift is expressed as: ; The tunnel is a waveguide structure, and its channel consists of a few propagable modes and weak scattering caused by the tunnel walls. Its small-scale fading is represented as follows: （3） For the first The complex amplitude of a waveguide mode, It is the complex attenuation constant. For the first Doppler frequency shift along the path Indicates the total number of paths; Indicates the first The instantaneous propagation distance of each mobile relay and its corresponding modal equivalent propagation path; This represents the zero-mean weak scattering component; In urban and suburban scenes, Rayleigh fading is used to model its small-scale fading, and a factor is used to control the ratio of line-of-sight propagation to non-line-of-sight propagation: （4） in, This indicates the energy ratio of line-of-sight propagation to non-line-of-sight propagation. Includes Doppler frequency shift caused by motion, Indicates the initial phase; This represents the scattering component of non-line-of-sight propagation. This indicates that at time slot t, the first... Multipath propagation distance of mobile relays under delay-Doppler indexing; When calculating the channels of a smart reflective surface, when deploying When the intelligent reflective surface has n controllable reflective units, the phase shift matrix of the i-th intelligent reflective surface is expressed as: ； Define the narrowband matrix from the b-th base station to the i-th smart reflective surface as follows: The narrowband vector from the smart reflective surface to the m-th mobile relay is The two sub-links each employ a fading model matched to the scene; due to the phase of the intelligent reflective surface... Controlled online by an intelligent agent, the cascaded equivalent channel of the i-th intelligent reflective surface is: （5） The Doppler amplitude of the link between the base station and the smart reflective surface is 0, and its narrowband matrix is... Considered quasi-static, it will not change with and Change indicates transposition; The equivalent complex channel response of a direct link is denoted as Then, the total equivalent channel for the i-th mobile relay is expressed as: （6） For each mobile relay, when calculating the total equivalent channel, the cascaded equivalent channels from one base station and all smart reflective surfaces need to be considered.

5. The optimization method for a high-speed railway communication system based on adaptive federated reinforcement learning according to claim 1, characterized in that: Step S3 specifically includes the following steps: Will Let be the transmitted symbol at discrete time t. Then the signal received by the m-th mobile relay is represented as: （7） in, In the baseband model, it is a complex modulation symbol. The magnitude of this complex number does not directly represent the power; power refers to the square of the amplitude. Represent a complex normal distribution; When the smart reflective surface is activated, the incident signal from the base station is reflected by the smart reflective surface to form a secondary link. The mobile relay then receives the following signal: （8） The intelligent reflective surface is a near-passive scatterer that does not introduce independent thermal noise terms, but simply changes the phase and amplitude of the original electromagnetic wave before reflecting it back. Based on the equivalent channel of formula (6), orthogonal time-delay Doppler diversity multiplexing is adopted to directly map the information to the time-delay-Doppler domain and transmit it as orthogonal time-delay-Doppler pulses.

6. The optimization method for a high-speed railway communication system based on adaptive federated reinforcement learning according to claim 5, characterized in that: After time-delay-Doppler domain equalization, the residual symbol-level interference / carrier-level interference of the frequency components / time slot grids equivalent to the time-frequency grid within the same cell is ignored. Only the cross-cell interference term is retained and used for signal-to-interference-plus-noise ratio and rate modeling in formulas (9)-(11). Interference between cells Represented as: （9） in, To allocate transmit power to the delay-Doppler unit, It is a base station serving the m-th mobile relay; based on the total channel equivalent gain. The signal-to-interference-plus-noise ratio (SIR) of the m-th mobile relay is: （10） in, This represents the noise power at the receiving end. Interference between different cells; The total transmission rate of the system is expressed as: （11） in, The number of Doppler units, The number of delay units, To assign indicator variables, if in time slot t, the b-th cell will... The first Doppler dimension and the second If a delay-Doppler unit consisting of delay dimensions is assigned to the m-th mobile relay, then... Otherwise, it is 0.

7. The optimization method for a high-speed railway communication system based on adaptive federated reinforcement learning according to claim 1, characterized in that: Step S4 specifically includes the following steps: A fixed time slot framework is adopted, with a time slot duration of [missing information]. The duty cycles of base stations, mobile relays, and smart reflective surfaces remain constant in each time slot, and the duty cycles do not change with the policy; power is used as a metric for energy consumption, assuming... The total transmission power of the i-th base station to its served mobile relay s at discrete time t is: Its power amplifier efficiency is If the system has a total of ; If there are 1 base station, then the total power of all base stations is: （12） in, This refers to the fixed circuit power of the base station; Calculate Total power of mobile relays: （13） in, Its downlink forwarding power, For the power amplifier efficiency of mobile repeaters, For fixed circuit power; Multiple smart reflective surfaces were deployed in urban and tunnel scenarios, each containing... There are several controllable reflection units, and the driving power of each reflection unit is given by... The fixed circuit power of each smart reflective surface is ,but The total power of the intelligent reflective surfaces is: （14）。 8. The optimization method for a high-speed railway communication system based on adaptive federated reinforcement learning according to claim 1, characterized in that: Step S5 specifically includes the following steps: Energy efficiency of the system As an optimization target It is expressed as the quotient of the system's total downlink data transmission rate and total power; （15） In each time slot t, let Let be the power action vector for all base stations; Let be the set of phase matrices for all intelligent reflective surfaces; with the goal of maximizing energy efficiency and using a time-averaged representation, let the scenario of a high-speed train passing through require a total of _____. If there are 1 time slot, the optimization problem can be expressed as: （16） （16a） （16b） （16c） Formula (16a) states that in any time slot, the transmit power of each base station must be taken from the discrete power level allowed by the hardware and must not exceed the maximum power generation; Formula (16b) stipulates that the phase angle of each smart reflective surface unit must be taken from the discrete phase set. Selected from the options; Formula (16c) represents the set of units actually used by the base station within time slot t. The sum of the allocated transmit power equals its total transmit power budget. That is, power is conserved; Discrete power sets and For the action space, a state space is constructed using basic statistical observations and scene coding, and outputs in each time slot. and The reward function directly adopts the energy efficiency index, making the maximization of the agent's cumulative reward equivalent to the maximization of energy efficiency.

9. The optimization method for a high-speed railway communication system based on adaptive federated reinforcement learning according to claim 1, characterized in that: Step S7 specifically includes the following steps: The first Statistical observations of one client in time slot t and scene coding Defined as state space ; （26） in, One-hot encoding for tunnels, towns, and suburbs; Includes average effective signal-to-interference-plus-noise ratio Average transmission rate Average power Average interference intensity Scheduled delay - percentage of Doppler grid cells Normalized index of the gear selected by the action client , Represented as: （27） Action space Set as base station power action and intelligent reflective surface phase action; （28） Among them, for the first For a single base station, its power For the i-th smart reflective surface For a single control unit , Dimensions Action space ; If each mobile relay is configured to switch at most once per time slot, then the handover penalty is expressed as: （29） in, The minimum number of switching operations during train operation will not be penalized for these switching operations; This means that for the m-th mobile relay, the reward is 1 if a cell handover occurs, and 0 otherwise. The reward function is expressed as: （30） Indicates the transmission rate. The Q-value of the dual-depth Q-network, representing the interference between cells, is expressed as: （31） in, For state value branch, For action advantage branches; each client Maintain local experience replay buffer When the buffer size reaches the preheating threshold After that, each step from Uniform random sampling; The dual estimator employing a dual deep Q-network is defined as follows for single-step time-series difference objective in time slots: （32） in, To provide timely rewards, Let A be the discount factor, and A be the set of actions. For online Network, used for selection action, For the goal The network is used to evaluate this action; the loss function is expressed as: （33）。

Images