Urban rail transit power supply control system and method

Abstract

The invention discloses an urban rail transit power supply control system and method, and the method comprises the following steps: S1, collecting the operation data of a rail transit power supply system, and carrying out the preprocessing; s2, constructing an improved MuZero reinforcement learning model, and optimizing a rail transit power supply control strategy through Monte Carlo tree search; s3, modeling a rail transit power supply resource allocation process by adopting a game theory, and calculating optimal power supply resource allocation by solving Nash equilibrium; s4, adjusting the power supply control scheduling of the rail transit power supply system based on the rail transit power supply control strategy and the optimal power supply resource allocation; and S5, updating a rail transit power supply control strategy by using the improved MuZero reinforcement learning model, and optimizing power supply resource allocation. The improved MuZero reinforcement learning and game theory method is adopted, rail transit power supply control scheduling is optimized, and the advantages of balanced power supply distribution, efficient energy recovery and intelligent self-adaptive adjustment of the system are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of rail transit power supply control, and particularly to an urban rail transit power supply control system and method. Background Art

[0002] In an urban rail transit system, power supply control is a key link to ensure the stable operation of trains. The rail transit system mainly relies on substations to provide electric energy to supply power to the running trains. At the same time, the trains will generate feedback energy during the braking process. How to reasonably allocate the power supply and optimize the energy recovery is an important factor affecting the operation efficiency of the rail transit system. Traditional rail transit power supply control mainly adopts a rule-based scheduling strategy, relying on fixed control logic and empirical formulas. By real-time monitoring the rail transit load, grid status, substation power supply capacity, and train energy recovery situation, the power supply is allocated. However, this fixed-rule control method has many limitations and is difficult to adapt to the complex and changeable rail transit operation environment, resulting in low utilization rate of power supply resources, insufficient power supply stability, and poor energy recovery efficiency.

[0003] With the rapid development of rail transit, the train operation density is increasing continuously, the grid load volatility is enhanced, and the power supply system faces higher scheduling requirements. Existing power supply control methods usually adopt traditional optimization algorithms or control methods based on linear prediction for power distribution. However, these methods are often based on idealized models and lack in-depth modeling of the complexity of the rail transit system, making it difficult to handle the real-time changes in power supply demand. In addition, there is a competitive relationship in the train power supply in the rail transit system. The traction demands of different trains are different, and the substation power supply capacity is limited. During peak hours, the traction power of some trains may be insufficient, affecting the overall operation efficiency. Existing methods usually adopt a static weight allocation method and lack the ability of dynamic adjustment, making it difficult to achieve reasonable allocation of power supply resources among trains, resulting in some trains may obtain more power supply than needed, while some other trains cannot obtain enough traction power, affecting the overall balance of the system.

[0004] In addition, in the rail transit system, braking energy recovery is one of the important ways to improve energy efficiency. The train will convert kinetic energy into electric energy during deceleration or braking and feed it back to the power grid or power supply system. However, there are still major problems in the energy recovery allocation of existing power supply control methods. The current energy recovery mechanism usually adopts a fixed allocation method and fails to fully consider the train operation status, grid load conditions, and real-time energy demands of other trains, resulting in the recovered energy being difficult to be effectively utilized, and some energy may even be wasted due to grid overload. Due to the lack of an intelligent allocation mechanism, the energy recovery utilization rate of the rail transit system fluctuates greatly under different operating conditions, restricting the improvement of the overall energy efficiency.

[0005] Regarding the optimization of rail transit power supply dispatching, in recent years, some studies have introduced machine learning and reinforcement learning methods, attempting to optimize the power supply control strategy in a data-driven manner. However, most existing reinforcement learning methods rely on a preset environment model and are difficult to adapt to the complex and changeable operating conditions of the rail transit system, resulting in a large error in the power supply dispatching strategy obtained through training in actual applications. In addition, when optimizing the power supply strategy, reinforcement learning methods usually only consider the operating requirements of individual trains and fail to conduct global optimization by combining the competition relationship between trains, lacking the ability of game analysis, resulting in deficiencies in the global optimality of the power supply allocation scheme. Summary of the Invention

[0006] An object of the present invention is to propose an urban rail transit power supply control system and method. The present invention uses an improved MuZero reinforcement learning and game theory method, combines Monte Carlo tree search to optimize rail transit power supply dispatching, constructs a non-cooperative game model to solve the Nash equilibrium, realizes the dynamic equilibrium allocation of power supply resources, and intelligently adjusts the traction power and energy recovery strategy under the condition of limited power supply capacity, improves the stability, energy efficiency utilization rate and adaptive dispatching ability of the rail transit power supply system, avoids the limitations of traditional fixed rule dispatching, and improves the accuracy of power supply control dispatching and the flexibility of system response.

[0007] According to an urban rail transit power supply control method of an embodiment of the present invention, the method includes the following steps:

[0008] S1. Collect the operation data of the rail transit power supply system, preprocess the operation data, and generate a training sample set;

[0009] S2. Construct an improved MuZero reinforcement learning model, construct a state space, an action space and a reward function based on the training sample set, and optimize the rail transit power supply control strategy through Monte Carlo tree search;

[0010] S3. Use game theory to model the rail transit power supply resource allocation process, establish the competition relationship between trains based on the non-cooperative game model, define the revenue function of the trains, and calculate the optimal power supply resource allocation by solving the Nash equilibrium;

[0011] S4. Based on the rail transit power supply control strategy and the optimal power supply resource allocation, adjust the power supply control dispatching of the rail transit power supply system;

[0012] S5. Based on the historical operation data of the rail transit power supply system, use the improved MuZero reinforcement learning model to update the rail transit power supply control strategy, calculate the adjustment range of the rail transit power supply parameters, and optimize the power supply resource allocation.

[0013] Optionally, the operation data includes rail transit load, grid status, substation power supply parameters, train energy recovery situation, and historical energy consumption data, and the preprocessing includes data cleaning, abnormal data removal, feature extraction, feature dimensionality reduction, normalization, and standardization.

[0014] Optionally, the S2 specifically includes:

[0015] S21. Construct an improved MuZero reinforcement learning model based on the training sample set. The improved MuZero reinforcement learning model includes a representation network, a dynamics model, and a prediction model. The improved MuZero reinforcement learning model introduces a non-linear term into the value network in the prediction model to optimize the state value estimation of the rail transit power supply control system, and further optimize the power supply control scheduling;

[0016] S22. Construct a state space, which is used to characterize the operation state data of the rail transit power supply system. The operation state data includes rail transit load status, grid status, substation power supply parameters, train energy recovery situation, and historical energy consumption data, and define a state vector:

[0017] ;

[0018] Wherein, represents the state vector of the rail transit power supply system at time , represents the rail transit load status, represents the grid status, represents the substation power supply parameters, represents the train energy recovery situation, represents the historical energy consumption data;

[0019] Define a state update equation:

[0020] ;

[0021] Wherein, represents the state vector of the rail transit power supply system at time , represents the state transition function, which describes the change of the power supply system state after taking an action, represents the environmental noise term, which describes the uncertain factors in the system operation process, represents the training parameters of the representation network of the MuZero reinforcement learning model, represents the action vector;

[0022] S23. Construct an action space, which includes traction power supply power adjustment, energy recovery distribution adjustment, substation power supply strategy adjustment, and train power supply strategy adjustment, and define an action vector:

[0023] ;

[0024] Among them, represents the action vector, describing the control strategy of the rail transit power supply system at the time moment; represents the adjustment of traction power supply; represents the adjustment of braking energy recovery; represents the adjustment of substation power supply strategy; represents the adjustment of train power supply strategy;

[0025] S24. Construct a reward function , and the reward function is the weighted sum of power supply stability reward, energy recovery efficiency reward and power supply balance reward, which is used to optimize the rail transit power supply control strategy;

[0026] S25. Construct a representation network, and use the representation network to extract potential features from the original state of the rail transit power supply system , and the representation network adopts the combination of convolutional neural network and gated recurrent unit to extract time series features and spatial features:

[0027] ;

[0028] Among them, represents the internal state after encoding, represents the transformation function of the representation network, which is composed of the combination of convolutional neural network and gated recurrent unit, represents the gated recurrent unit, represents the convolutional neural network, represents the training parameters of the representation network;

[0029] S26. Construct a dynamic model, and use the dynamic model to predict the state change of the power supply control system after executing the action. The dynamic model is composed of a recurrent neural network and a feedforward neural network:

[0030] ;

[0031] Among them, represents the next moment state after state prediction, represents the internal state after encoding, represents the reward function, describing the reward value based on state prediction, represents the dynamic model, represents the recurrent neural network;

[0032] S27. Build a prediction model, and use the prediction model to calculate the action probability distribution and state value of the rail transit power supply control strategy. The prediction model adopts a dual-network structure of a policy network and a value network:

[0033] ;

[0034] ;

[0035] Among them, represents the action probability distribution function, represents the natural exponential function, represents the state-action value function of the rail transit power supply optimization system, and is normalized using the temperature parameter for normalization, represents the temperature parameter, represents the current state Under this, all optional actions taken by the rail transit power supply control system, represents the state value function, and represent the parameters of the state value function, represents the number of neurons in the hidden layer, represents the weight coefficient for learning, represents the non-linear activation function, and represent the weight matrix and bias of the hidden layer;

[0036] S28. Calculate the optimal power supply control strategy based on Monte Carlo tree search, and construct the probability distribution of the search tree:

[0037] ;

[0038] Among them, represents the action score of, represents the state-action value function of the rail transit power supply optimization system, represents the exploration factor, represents the state number of visits to, represents the action selected in the state number of visits to, represents the logarithmic function with base e;

[0039] After the search is completed, select the action with the most visits as the optimal power supply control strategy:

[0040] ;

[0041] Among them, ​Represents the optimal power supply control strategy, Represents the variable value when the function value is the largest, Represents in the state Select an action The number of accesses;

[0042] S29. Obtain the true state value of the rail transit power supply system by Monte Carlo sampling, and cumulatively calculate the future reward to obtain the true state value:

[0043] ;

[0044] Among them, Represents the true state value obtained by Monte Carlo sampling, Represents the maximum number of backtracking steps, Represents the discount factor, which is used to reduce the impact of long-term rewards, Represents at time The immediate reward at this time, Represents time The state value estimate at this time;

[0045] Based on the optimal power supply control strategy, construct the loss function of the rail transit power supply optimization system, use gradient descent to optimize the representation network, dynamic model and prediction model, adjust the power supply parameters of the rail transit power supply system, update the state space, and repeat training the MuZero reinforcement learning model to optimize the rail transit power supply control strategy:

[0046] ;

[0047] Among them, Represents the loss function, Represents the true state value obtained by Monte Carlo sampling, Represents the optimal power supply control strategy calculated by Monte Carlo tree search, Represents the norm.

[0048] Optionally, the specific content of S3 includes:

[0049] S31. Establish a non-cooperative game model for rail transit power supply resource allocation, and set the train set in the rail transit system as the game participants:

[0050] ;

[0051] Among them, Represents the train set, Represents the total number of trains in the rail transit system. Each train is a participant in the game, and the decision variable is the traction power supply;

[0052] S32. Calculate the revenue function for each train based on the optimized power supply control strategy, and define the non - cooperative game revenue function for the optimization of rail transit power supply control:

[0053] ;

[0054] Among them, represents the revenue function of train , represents the weight of traction power satisfaction, which measures the degree of satisfaction of the train's traction demand, represents train 's traction power demand under the current track position, represents the fairness adjustment factor to prevent some trains from over - occupying power supply resources, represents the weight of energy recovery utilization rate, represents train 's generated braking energy recovery power, represents train 's maximum braking energy recovery power, represents the power supply of the substation in the rail transit system, represents the traction power supply selected by train ;

[0055] S33. Establish the competitive relationship between trains based on the non - cooperative game model, and define the power supply resource allocation strategy of the rail transit power supply system to make the traction power supply of all trains satisfy the Nash equilibrium. The Nash equilibrium means that under the optimal traction power supply selected by all trains , any single train cannot obtain higher revenue by adjusting its own traction power supply , that is, the power supply strategies of each train reach a stable state in the game process:

[0056] ;

[0057] Among them, represents the optimal revenue of train , represents the optimal traction power supply of train at the Nash equilibrium point, represents the set of traction power strategies of train , represents the total number of trains in the rail transit system, represents "any";

[0058] S34. Based on the solution result of the Nash equilibrium, adjust the power supply resource allocation strategy of the rail transit system to make the substation power supply satisfy:

[0059] ;

[0060] Wherein, represents the optimized power supply of the rail transit substation, represents the optimal traction power supply at the Nash equilibrium point of the train of the optimal traction power supply.

[0061] A urban rail transit power supply control system according to an embodiment of the present invention includes:

[0062] A data acquisition and preprocessing module, configured to acquire the operation data of the rail transit power supply system, and perform preprocessing to generate a training sample set;

[0063] A reinforcement learning decision module, configured to construct an improved MuZero reinforcement learning model, and construct a state space, an action space, and a reward function to optimize the power supply control strategy;

[0064] A power supply resource allocation module, configured to model the rail transit power supply resource allocation process by using game theory, solve the Nash equilibrium based on non-cooperative game, and calculate the optimal power supply resource allocation scheme;

[0065] A power supply scheduling module, configured to adjust the power supply power distribution of the rail transit power supply system based on the power supply control strategy and the optimal power supply resource allocation scheme, so that the substation power supply power meets the load demand of the rail transit system;

[0066] A power supply strategy optimization module, configured to optimize the power supply control strategy based on the historical operation data of the rail transit power supply system, and dynamically adjust the power supply parameters.

[0067] The beneficial effects of the present invention are:

[0068] First of all, the present invention combines an improved MuZero reinforcement learning model with game theory methods to optimize the scheduling and resource allocation of the rail transit power supply system, realizing intelligent, adaptive optimization and efficient energy utilization of power supply control. By using data acquisition and preprocessing technologies, the real-time operation status of the rail transit system is accurately extracted, and the power supply control strategy is trained by combining reinforcement learning methods, enabling the power supply scheduling to dynamically adapt to the changes in the rail transit load. Compared with the traditional power supply management method that relies on fixed rules, the present invention can adjust the traction power supply and energy recovery strategies under different operation states, improve the rationality of power supply resource allocation, and avoid problems such as energy waste and insufficient train traction power caused by rigid power supply scheduling.

[0069] Secondly, the present invention optimizes the allocation of rail transit power supply resources by using game theory. A competitive relationship between trains is established through a non-cooperative game model, and the optimal traction power is solved based on Nash equilibrium, enabling each train to obtain fair and reasonable power support under limited power supply capacity, improving the balance of the power supply system and the stability of train operation. For the problem of braking energy recovery, the present invention optimizes the energy recovery power allocation through reinforcement learning, comprehensively considering the train operation state, grid load, and power supply system scheduling, enabling the recovered energy to be efficiently utilized, reducing power waste, and improving the overall energy efficiency of the system.

[0070] Finally, compared with the prior art, the present invention breaks through the limitation of fixed-rule scheduling, uses reinforcement learning to achieve adaptive optimization of the power supply control strategy, combines game theory methods to improve the fairness and efficiency of power supply allocation, enables the rail transit system to still operate stably under limited power supply resources. At the same time, Monte Carlo tree search optimizes the power supply decision-making, improves the dynamic response ability of the power supply system, enables it to adjust the power supply scheduling according to the actual operation requirements of the train, and enhances the flexibility of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention, and do not constitute a limitation to the present invention. In the accompanying drawings:

[0072] Figure 1 is a flowchart of a method for controlling the power supply of urban rail transit proposed by the present invention;

[0073] Figure 2 is a schematic structural diagram of an improved MuZero reinforcement learning model for a method for controlling the power supply of urban rail transit proposed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0074] Now, the present invention will be further described in detail with reference to the accompanying drawings. These drawings are all simplified schematic diagrams, only illustrating the basic structure of the present invention in a schematic manner, so they only show the components related to the present invention.

[0075] Refer to Figure 1 and Figure 2 , a method for controlling the power supply of urban rail transit, includes the following steps:

[0076] S1. Collect the operation data of the rail transit power supply system, preprocess the operation data, and generate a training sample set;

[0077] S2. Construct an improved MuZero reinforcement learning model, construct a state space, an action space, and a reward function based on the training sample set, and optimize the rail transit power supply control strategy through Monte Carlo tree search;

[0078] S3. Model the rail transit power supply resource allocation process using game theory, establish the competition relationship between trains based on the non - cooperative game model, define the revenue function of the trains, and calculate the optimal power supply resource allocation by solving the Nash equilibrium;

[0079] S4. Based on the rail transit power supply control strategy and the optimal power supply resource allocation, adjust the power supply control and dispatching of the rail transit power supply system;

[0080] S5. Based on the historical operation data of the rail transit power supply system, use the improved MuZero reinforcement learning model to update the rail transit power supply control strategy, calculate the adjustment range of the rail transit power supply parameters, and optimize the power supply resource allocation.

[0081] In this embodiment, the operation data includes rail transit load, grid status, substation power supply parameters, train energy recovery situation, and historical energy consumption data. The pre - processing includes data cleaning, abnormal data elimination, feature extraction, feature dimensionality reduction, normalization, and standardization.

[0082] In this embodiment, the specific content of S2 is as follows:

[0083] S21. Build an improved MuZero reinforcement learning model based on the training sample set. The improved MuZero reinforcement learning model includes a representation network, a dynamics model, and a prediction model. The improved MuZero reinforcement learning model introduces a non - linear term into the value network of the prediction model to optimize the state value estimation of the rail transit power supply control system, and further optimize the power supply control and dispatching;

[0084] S22. Build a state space, which is used to characterize the operation state data of the rail transit power supply system. The operation state data includes rail transit load status, grid status, substation power supply parameters, train energy recovery situation, and historical energy consumption data, and define the state vector:

[0085] ;

[0086] Among them, represents the state vector of the rail transit power supply system at time , represents the rail transit load status, represents the grid status, represents the substation power supply parameters, represents the train energy recovery situation, represents the historical energy consumption data;

[0087] Define the state update equation:

[0088] ;

[0089] Among them, represents the state vector of the rail transit power supply system at time moment, represents the state transition function, which describes the change of the power supply system state after taking an action, represents the environmental noise term, which describes the uncertain factors during the system operation, represents the training parameters of the representation network of the MuZero reinforcement learning model, represents the action vector;

[0090] S23. Construct an action space, where the action space includes traction power supply power adjustment, energy recovery distribution adjustment, substation power supply strategy adjustment, and train power supply strategy adjustment, and define the action vector:

[0091] ;

[0092] Among them, represents the action vector, which describes the control strategy of the rail transit power supply system at time moment, represents the adjustment of the traction power supply power, represents the adjustment of the braking energy recovery, represents the adjustment of the substation power supply strategy, represents the adjustment of the train power supply strategy;

[0093] S24. Construct a reward function , and the reward function is the weighted sum of the power supply stability reward, energy recovery efficiency reward, and power supply balance reward, which is used to optimize the rail transit power supply control strategy;

[0094] S25. Construct a representation network, and use the representation network to extract potential features from the original state of the rail transit power supply system. The representation network adopts a combination of a convolutional neural network and a gated recurrent unit to extract temporal features and spatial features:

[0095] ;

[0096] Among them, represents the internal state after encoding, represents the transformation function of the representation network, which is composed of a combination of a convolutional neural network and a gated recurrent unit, represents the gated recurrent unit, represents the convolutional neural network, represents the training parameters of the representation network;

[0097] S26. Construct a dynamic model and use the dynamic model to predict the state change of the power supply control system after performing an action. The dynamic model consists of a recurrent neural network and a feedforward neural network:

[0098] ;

[0099] Among them, represents the next moment state after state prediction, represents the internal state after encoding, represents the reward function, describing the reward value based on state prediction, represents the dynamic model, represents the recurrent neural network;

[0100] S27. Construct a prediction model and use the prediction model to calculate the action probability distribution and state value of the rail transit power supply control strategy. The prediction model adopts a dual-network structure of a policy network and a value network:

[0101] ;

[0102] ;

[0103] Among them, represents the action probability distribution function, represents the natural exponential function, represents the state-action value function of the rail transit power supply optimization system, which is normalized using the temperature parameter for normalization, represents the temperature parameter, represents the current state Under this, all optional actions taken by the rail transit power supply control system, represents the state value function, and represent the parameters of the state value function, represents the number of neurons in the hidden layer, represents the learning weight coefficient, represents the non-linear activation function, and represent the weight matrix and bias of the hidden layer;

[0104] S28. Calculate the optimal power supply control strategy based on Monte Carlo tree search and construct the probability distribution of the search tree:

[0105] ;

[0106] Among them, represents the action score of, Represents the state-action value function of the rail transit power supply optimization system, Represents the exploration factor, Represents the state The number of visits, Represents at the state Select the action The number of visits, Represents the natural logarithm function with base e;

[0107] After the search is completed, select the action with the most visits as the optimal power supply control strategy:

[0108] ;

[0109] Among them, Represents the optimal power supply control strategy, Represents the variable value when taking the maximum function value, Represents at the state Select the action The number of visits;

[0110] S29. Obtain the true state value of the rail transit power supply system by Monte Carlo sampling, and cumulatively calculate the future rewards to obtain the true state value:

[0111] ;

[0112] Among them, Represents the true state value obtained by Monte Carlo sampling, Represents the maximum number of backtracking steps, Represents the discount factor, used to reduce the impact of long-term rewards, Represents at time The immediate reward, Represents time The state value estimate at;

[0113] Based on the optimal power supply control strategy, construct the loss function of the rail transit power supply optimization system, use gradient descent to optimize the representation network, dynamic model and prediction model, adjust the power supply parameters of the rail transit power supply system, update the state space, and repeat training the MuZero reinforcement learning model to optimize the rail transit power supply control strategy:

[0114] ;

[0115] Among them, Represents the loss function, Represents the true state value obtained by Monte Carlo sampling, Represents the optimal power supply control strategy calculated by Monte Carlo tree search, Represents the norm.

[0116] In this embodiment, step S3 specifically includes:

[0117] S31. Establish a non - cooperative game model for the allocation of rail transit power supply resources, and set the set of trains in the rail transit system as game participants:

[0118] ;

[0119] Among them, represents the set of trains, represents the total number of trains in the rail transit system. Each train is a participant in the game, and the decision variable is the traction power supply;

[0120] S32. Based on the optimized power supply control strategy, calculate the revenue function of each train, and define the non - cooperative game revenue function for optimizing the rail transit power supply control:

[0121] ;

[0122] Among them, represents the revenue function of train , represents the weight of traction power satisfaction, which measures the degree of satisfaction of the train's traction demand, represents train 's traction power supply demand at the current track position, represents the fairness adjustment factor to prevent some trains from over - occupying power supply resources, represents the weight of the energy recovery utilization rate, represents train 's generated braking energy recovery power, represents train 's maximum braking energy recovery power, represents the power supply of the substation in the rail transit system, represents train 's selected traction power supply;

[0123] S33. Based on the non - cooperative game model, establish the competitive relationship between trains, and define the power supply resource allocation strategy of the rail transit power supply system, so that the traction power supply of all trains satisfies the Nash equilibrium. The Nash equilibrium means that under the optimal traction power supply selected by all trains , any single train cannot obtain higher revenue by adjusting its own traction power supply , that is, the power supply strategies of each train reach a stable state during the game process:

[0124] ;

[0125] Among them, represents the optimal revenue of the train , is expressed as the optimal traction power supply of the train at the Nash equilibrium point , represents the train 's traction power strategy set, represents the total number of trains in the rail transit system, represents "any";

[0126] S34. Based on the solution result of the Nash equilibrium, adjust the power supply resource allocation strategy of the rail transit system so that the substation power supply satisfies:

[0127] ;

[0128] Among them, represents the optimized power supply of the rail transit substation, is expressed as the optimal traction power supply of the train at the Nash equilibrium point .

[0129] An urban rail transit power supply control system includes:

[0130] A data collection and preprocessing module, which is used to collect the operation data of the rail transit power supply system, preprocess it, and generate a training sample set;

[0131] A reinforcement learning decision-making module, which is used to build an improved MuZero reinforcement learning model, build a state space, an action space, and a reward function, and optimize the power supply control strategy;

[0132] A power supply resource allocation module, which is used to model the rail transit power supply resource allocation process by game theory, solve the Nash equilibrium based on non-cooperative game, and calculate the optimal power supply resource allocation plan;

[0133] A power supply scheduling module, which is used to adjust the power supply power distribution of the rail transit power supply system based on the power supply control strategy and the optimal power supply resource allocation plan, so that the substation power supply meets the load demand of the rail transit system;

[0134] A power supply strategy optimization module, which is used to optimize the power supply control strategy based on the historical operation data of the rail transit power supply system and dynamically adjust the power supply parameters.

[0135] Embodiment 1:

[0136] To verify the feasibility of the present invention in implementation, the present invention is applied to the power supply control system of an electrified railway. The railway line is 100 kilometers long. The railway system adopts a sectional power supply method, and each power supply section is supported by multiple substations. At the same time, braking energy recovery occurs during the train operation, and the power supply load fluctuates with the change of the train operation state. Due to the complex railway operation environment, there are significant differences in the power supply demands in different sections, resulting in unbalanced distribution of power supply resources, insufficient traction power during peak hours, and low energy recovery utilization rate during low valley hours, affecting the overall operation efficiency of the power supply system.

[0137] In this railway system, the traditional power supply control mainly relies on a fixed rule scheduling strategy, which adjusts according to the real-time power supply capacity of the substation and the train traction power demand. However, due to the highly dynamic nature of train operation and the frequent change of power supply demand, the fixed rule power supply scheduling method is difficult to meet the optimization requirements in a complex environment. Especially during peak hours, due to the increase in the number of trains and the sharp increase in traction power demand, the load of some substations exceeds their maximum power supply capacity, resulting in insufficient traction power for some trains and the need to reduce the speed, affecting the train punctuality rate.

[0138] This embodiment adopts the railway power supply control method based on improved MuZero reinforcement learning and game theory of the present invention, and deploys an intelligent power supply optimization system in this railway power supply system. The data acquisition and preprocessing module monitors the power supply state of the railway system in real time, including track load, grid state, substation power supply parameters, train energy recovery situation, and historical energy consumption data. The MuZero reinforcement learning model is used to optimize the power supply control strategy, and the power supply resource allocation is optimized through game theory.

[0139] Table 1 Comparison Table of Experimental Data

[0140] Experimental Index Traditional Method Method of the Present Invention Improvement Effect Improvement in Traction Power Distribution Efficiency (%) - 21% Improved by 21% Improvement in Power Supply Balance (%) - 19% Improved by 19% Improvement in Energy Recovery Utilization Rate (%) 40% 78% Improved by 95% Reduction in Overall Energy Consumption (%) - 15% Reduced by 15% Improvement in Power Supply Scheduling Response Speed (%) 480 ms 140 ms Improved by 71% Reduction in Load Sudden Change Adjustment Time (%) 5000 ms 1800 ms Reduced by 64% Reduction in Energy Recovery Strategy Adjustment Time (%) 1200 ms 350 ms Reduced by 70%

[0141] In terms of the traction power distribution efficiency, the present invention adopts MuZero reinforcement learning and game theory to optimize the power supply strategy, which improves the traction power distribution efficiency by 21%. Compared with the traditional method, it can more accurately meet the train traction demand, avoid the problems of power supply resource waste or insufficient power supply. At the same time, in terms of power supply balance, the optimal traction power is solved through the Nash equilibrium, making the power supply resource allocation among trains more reasonable, and the power supply balance is improved by 19%, reducing the situation that some trains reduce their operation efficiency due to insufficient power supply.

[0142] In terms of the energy recovery utilization rate, the present invention optimizes the braking energy recovery strategy, increasing the energy recovery utilization rate from 40% of the traditional method to 78%, a 95% increase, significantly reducing energy waste. At the same time, the overall energy consumption of the system is reduced by 15%, indicating that the method of the present invention can effectively reduce unnecessary energy consumption and improve the operating efficiency of the power supply system.

[0143] The power supply scheduling response time of the method of the present invention is 140 ms. Compared with 480 ms of the traditional method, it is increased by 71%, shortening the power supply adjustment time, enabling the system to adapt to changes in rail transit loads faster. In terms of load mutation adjustment, the method of the present invention can adjust the power supply parameters within 1800 ms, while the traditional method requires 5000 ms, and the adjustment time is shortened by 64%, ensuring that the power supply system can still operate stably in case of emergencies. In addition, in terms of the adjustment of the energy recovery strategy, the adjustment time of the method of the present invention is shortened by 70%, from 1200 ms of the traditional method to 350 ms, further enhancing the dynamic adaptability of the system.

[0144] As described above, it is only the preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, making equivalent substitutions or changes, should be covered within the protection scope of the present invention.

Claims

1. A power supply control method for urban rail transit, characterized in that It includes the following steps: S1. Collect the operation data of the rail transit power supply system, preprocess the operation data, and generate a training sample set; S2. Construct an improved MuZero reinforcement learning model, construct a state space, an action space, and a reward function based on the training sample set, and optimize the rail transit power supply control strategy through Monte Carlo tree search; S3. Use game theory to model the rail transit power supply resource allocation process, establish the competition relationship between trains based on the non-cooperative game model, define the revenue function of the trains, and calculate the optimal power supply resource allocation by solving the Nash equilibrium; S4. Adjust the power supply control and scheduling of the rail transit power supply system based on the rail transit power supply control strategy and the optimal power supply resource allocation; S5. Based on the historical operation data of the rail transit power supply system, use the improved MuZero reinforcement learning model to update the rail transit power supply control strategy, calculate the adjustment range of the rail transit power supply parameters, and optimize the power supply resource allocation.

2. The urban rail transit power supply control method according to claim 1, characterized in that The operation data includes rail transit load, grid status, substation power supply parameters, train energy recovery situation, and historical energy consumption data. The preprocessing includes data cleaning, abnormal data elimination, feature extraction, feature dimensionality reduction, normalization, and standardization.

3. A power supply control method for urban rail transit according to claim 1, characterized in that, The specific content of S2 includes: S21. Construct an improved MuZero reinforcement learning model based on the training sample set. The improved MuZero reinforcement learning model includes a representation network, a dynamics model, and a prediction model. The improved MuZero reinforcement learning model introduces a non-linear term into the value network in the prediction model to optimize the state value estimation of the rail transit power supply control system, and further optimize the power supply control and scheduling; S22. Construct a state space, which is used to characterize the operation state data of the rail transit power supply system. The operation state data includes rail transit load status, grid status, substation power supply parameters, train energy recovery situation, and historical energy consumption data, and define a state vector: ； Among them, represents time the state vector of the rail transit power supply system at a certain moment, represents the rail transit load status, represents the power grid status, represents the substation power supply parameters, represents the train energy recovery situation, represents the historical energy consumption data; Define a state update equation: ； Among them, represents time the state vector of the rail transit power supply system at a moment, represents the state transition function, describing the change of the power supply system state after taking actions, represents the environmental noise term, describing the uncertain factors during the system operation, represents the training parameters of the representation network of the MuZero reinforcement learning model, represents the action vector; S23. Construct an action space, which includes traction power supply power adjustment, energy recovery allocation adjustment, substation power supply strategy adjustment, and train power supply strategy adjustment, and define an action vector: ； Among them, represents the action vector, describing the control strategy of the rail transit power supply system at a certain moment, represents the adjustment of traction power supply, represents the adjustment of braking energy recovery, represents the adjustment of substation power supply strategy; S24. Construct a reward function , where the reward function is a weighted sum of power supply stability reward, energy recovery efficiency reward, and power supply balance reward, and is used to optimize the rail transit power supply control strategy; S25. Construct a representation network, and use the representation network to extract potential features from the original state of the rail transit power supply system The representation network adopts a combination of a convolutional neural network and a gated recurrent unit to extract temporal features and spatial features: ； Among them, represents the encoded internal state, represents the transformation function of the representation network, which is composed of a convolutional neural network and a gated recurrent unit, represents the gated recurrent unit, represents the convolutional neural network, represents the training parameters of the representation network; S26. Construct a dynamics model, and use the dynamics model to predict the state change of the power supply control system after executing an action. The dynamics model consists of a recurrent neural network and a feedforward neural network: ； Among them, represents the state at the next moment after state prediction, represents the internal state after encoding, represents the reward function, describing the reward value based on state prediction, represents the dynamic model, represents the recurrent neural network; S27. Construct a prediction model, and use the prediction model to calculate the action probability distribution and state value of the rail transit power supply control strategy. The prediction model adopts a dual-network structure of a policy network and a value network: ； ； Among them, represents the action probability distribution function, represents the natural exponential function, represents the state-action value function of the rail transit power supply optimization system, and uses the temperature parameter for normalization, represents the temperature parameter, represents the current state Under this, all optional actions taken by the rail transit power supply control system, represents the state value function, and represent the parameters of the state value function, represents the number of neurons in the hidden layer, represents the weight coefficient of learning, represents the non-linear activation function, and represent the weight matrix and bias of the hidden layer; S28. Calculate the optimal power supply control strategy based on Monte Carlo tree search, and construct the probability distribution of the search tree: ； Among them, represents the score of the action , represents the state-action value function of the rail transit power supply optimization system represents the exploration factor represents the state of the number of visits represents the number of visits when selecting the action in the state ; represents the natural logarithm function with base e; After the search is completed, select the action with the most visit times as the optimal power supply control strategy: ； Among them, represents the optimal power supply control strategy, represents the variable value when the function value is the largest, represents at state select action the number of accesses; S29. Use Monte Carlo sampling to obtain the true state value of the rail transit power supply system, and cumulatively calculate the future reward to obtain the true state value: ； Among them, represents the true state value obtained by Monte Carlo sampling, represents the maximum number of backtracking steps, represents the discount factor, which is used to reduce the impact of future rewards, represents at time the immediate reward, represents the state value estimate at time ; Based on the optimal power supply control strategy, construct the loss function of the rail transit power supply optimization system, use gradient descent to optimize the representation network, dynamic model and prediction model, adjust the power supply parameters of the rail transit power supply system, update the state space, and repeat training the MuZero reinforcement learning model to optimize the rail transit power supply control strategy: ； Among them, represents the loss function, represents the true state value obtained by Monte Carlo sampling, represents the optimal power supply control strategy calculated by Monte Carlo tree search, represents the norm.

4. A power supply control method for urban rail transit according to claim 1, characterized in that, The specific steps of S3 are as follows: S31. Establish a non-cooperative game model for the allocation of rail transit power supply resources, and set the set of trains in the rail transit system as game participants: ； Among them, represents the set of trains, represents the total number of trains in the rail transit system. Each train is a participant in the game, and the decision variable is the traction power supply; S32. Based on the optimized power supply control strategy, calculate the revenue function of each train, and define the non-cooperative game revenue function for the optimization of rail transit power supply control: ； Among them, represents the revenue function of the train . represents the weight of traction power satisfaction, which measures the degree of satisfaction of the train's traction demand. represents the train 's traction power supply demand at the current track position. represents the fairness adjustment factor to prevent some trains from over-occupying power supply resources. represents the weight of the energy recovery utilization rate. represents the train 's generated braking energy recovery power. represents the train 's maximum braking energy recovery power. represents the power supply of the substation of the rail transit system. represents the train 's selected traction power supply. S33. Establish the competition relationship between trains based on the non - cooperative game model, define the power supply resource allocation strategy of the rail transit power supply system, so that the traction power supply of all trains satisfies the Nash equilibrium, and the Nash equilibrium means that among all trains the optimal traction power supply selected under which, any single train cannot obtain higher benefits by adjusting its own traction power supply That is, the power supply strategies of each train reach a stable state during the game process: ； Among them, represents the optimal revenue of the train and is expressed as the optimal traction power supply of the train at the Nash equilibrium point and represents the set of traction power strategies of the train and represents the total number of trains in the rail transit system represents "any". S34. Based on the Nash equilibrium solution result, adjust the power supply resource allocation strategy of the rail transit system to make the substation power supply power meet: ； Among them, represents the optimized power supply of the rail transit substation, which represents the optimal traction power supply of the train at the Nash equilibrium point.

5. An urban rail transit power supply control system, which executes the urban rail transit power supply control method according to any one of claims 1 to 4, characterized in that, It includes: The data collection and preprocessing module is used to collect the operation data of the rail transit power supply system and perform preprocessing to generate a training sample set; The reinforcement learning decision-making module is used to construct an improved MuZero reinforcement learning model, and construct a state space, action space and reward function to optimize the power supply control strategy; The power supply resource allocation module is used to model the rail transit power supply resource allocation process using game theory, solve the Nash equilibrium based on non-cooperative games, and calculate the optimal power supply resource allocation plan; The power supply scheduling module is used to adjust the power supply power distribution of the rail transit power supply system based on the power supply control strategy and the optimal power supply resource allocation plan to make the substation power supply power meet the load demand of the rail transit system; The power supply strategy optimization module is used to optimize the power supply control strategy based on the historical operation data of the rail transit power supply system and dynamically adjust the power supply parameters.

Images