Control system and control method based on reinforcement learning under train flywheel energy storage fault

Abstract

The invention provides a reinforcement learning-based control system under a train flywheel energy storage fault. The reinforcement learning-based control system is characterized by comprising a train control center, a direct current traction station and a flywheel energy storage system, the invention further discloses a control method for the control system, and the control method is characterized in that when a certain flywheel energy storage device is diagnosed to be faulty by the bearing fault diagnosis module, the flywheel array control module cuts out the faulty flywheel energy storage device from the flywheel array, and then the charging and discharging voltage threshold value of each normal flywheel device is adjusted by adopting a DDPG algorithm; a multi-source information fusion model is utilized to process train operation data, passenger data and line loss data to obtain total energy of a new flywheel energy storage array, and then an A2C algorithm is adopted to distribute the total energy and then absorb or release the total energy. By adopting the control system and the control method, the stability of a traction network and the normal operation of a train can be ensured when the flywheel energy storage array fails, and the maintenance and use cost of flywheel energy storage equipment can be reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of transportation, and in particular to a control system and a control method based on reinforcement learning under a train flywheel energy storage failure. Background Art

[0002] Urban rail transit trains start and brake frequently, and the energy that can be recovered during train braking accounts for about 40% of the total traction energy consumption. Therefore, the recovery of train braking energy is very important for achieving energy-saving operation of trains. Flywheel energy storage technology is one of the effective means to solve the above problems. It is an energy storage technology that stores electrical energy in the form of mechanical energy in a high-speed rotating flywheel rotor. The braking energy of the train is as high as MW level or above. In the existing technology, multiple flywheel energy storage devices of specific power levels are usually connected in parallel to build a large-capacity flywheel array energy storage system to absorb the above braking energy, and release the stored energy to power the train traction when the voltage on the DC bus is low.

[0003] The flywheel energy storage device in the flywheel energy storage array is generally composed of a power conversion circuit and its controller, a motor, a flywheel rotor, a support bearing, a protective shell, etc. Among them, the rolling bearing as a support bearing is an important component in the flywheel energy storage device. Since it is in a complex and changeable working environment such as high temperature, variable speed, and variable load for a long time, as the running time of the rotating machinery increases, the rolling bearing is very likely to fail, which will cause the whole machine to be paralyzed. In the prior art, the processing method for the above-mentioned fault is: when only one flywheel energy storage device in the flywheel energy storage array stops working due to a fault, the entire flywheel energy storage array stops working. The above-mentioned fault handling control method not only causes waste of other non-fault flywheel energy storage devices in the flywheel energy storage array, but also affects the normal recovery and utilization of the braking energy of the train running line, which not only causes energy waste and increases operating costs, but also even affects the normal operation of the train. Moreover, for tourist sightseeing trains in some remote mountainous areas and high-altitude areas, once a flywheel energy storage array fails and stops working, it takes a long time for maintenance personnel to reach the traction substation for repair. The equipment recovery cycle is long and the maintenance cost is high, which will cause great economic losses both in terms of energy recovery and normal train operation. Summary of the invention

[0004] In response to the problems in the background technology, the present invention provides a control system based on reinforcement learning under a train flywheel energy storage failure, and a control method for the above-mentioned control system, so as to solve the problem in the prior art that the flywheel energy storage array is completely shut down and paralyzed due to bearing failure, affecting the normal operation and energy storage of the train, and thus increasing the maintenance and operating costs.

[0005] To achieve the purpose of the present invention, the present invention provides a control system based on reinforcement learning under a train flywheel energy storage fault, and its innovation lies in: comprising a train control center, a DC traction station and a flywheel energy storage system; the flywheel energy storage system is arranged in the DC traction station; the train running section powered by the DC traction station is recorded as the traction section, and the DC traction station supplies power to the train running in the traction section through the DC bus; the train control center can obtain the train running data, passenger data and line loss data on the traction section;

[0006] The flywheel energy storage system includes a flywheel array control module and a flywheel energy storage array, wherein the flywheel energy storage array includes a plurality of flywheel energy storage devices; the flywheel array control module includes a communication module, an energy prediction module, an energy distribution module, and a charge and discharge threshold prediction module; the energy prediction module is connected to the energy distribution module; the communication module is wirelessly connected to the train control center;

[0007] The energy prediction module includes a multi-source information fusion model, which includes a first base learner, a second base learner and a third base learner respectively constructed by using three models of GRU, Informer and AE, and also includes a secondary learner constructed by using an OLS model; the energy prediction module can process train operation data, passenger data and line loss data to obtain the total energy that the flywheel energy storage array needs to store or release;

[0008] The energy distribution module includes an energy distribution weight model constructed by the A2C algorithm. The energy distribution module can generate energy distribution weights of each flywheel energy storage device through the energy distribution weight model, and distribute the total energy according to the energy distribution weight to obtain the energy that each flywheel energy storage device needs to store or release;

[0009] The charge and discharge threshold prediction module can process the train operation data, the charge state value of the flywheel energy storage array and the output energy and power data of the DC traction station to obtain the charge and discharge voltage thresholds of each flywheel energy storage device;

[0010] The flywheel energy storage device includes a flywheel energy storage unit, a grid-connected static switch and a bearing fault diagnosis module; one end of the grid-connected static switch is connected in parallel with the DC bus, and the other end of the grid-connected static switch is connected to the flywheel energy storage unit; the flywheel energy storage unit is connected to the flywheel array control module; the bearing fault diagnosis module is connected to the flywheel array control module, the bearing fault diagnosis module can collect the bearing working data of the flywheel energy storage unit, the bearing fault diagnosis module can also process the collected bearing working data to obtain the bearing fault diagnosis result information, and the bearing fault diagnosis module can also transmit the fault diagnosis result information to the flywheel array control module;

[0011] The flywheel array control module can control the closing of the grid-connected static switch according to the bearing fault diagnosis result information of the flywheel energy storage device; the flywheel array control module can control the corresponding flywheel energy storage device to adjust the charging and discharging voltage threshold according to the charging and discharging voltage threshold; the flywheel array control module can distribute the total energy to each flywheel energy storage device to realize the storage or release of the total energy.

[0012] The present invention also relates to a control method for the above control system, the innovation of which is:

[0013] The energy prediction module trains the multi-source information fusion model according to the method to obtain an available multi-source information fusion model;

[0014] Each bearing fault diagnosis module periodically collects the bearing working data of the corresponding flywheel energy storage unit, and then each bearing fault diagnosis module uses the ViT algorithm to process the acquired bearing working data to obtain the bearing fault diagnosis results of each corresponding flywheel energy storage unit; the bearing fault diagnosis results include: faulty and normal; the flywheel energy storage device with a bearing fault diagnosis result of faulty is recorded as a faulty flywheel device, and the flywheel energy storage device with a bearing fault diagnosis result of normal is recorded as a normal flywheel device;

[0015] When a faulty flywheel occurs,

[0016] 1) A bearing fault diagnosis module under the faulty flywheel device generates bearing fault information, and then the bearing fault diagnosis module transmits the bearing fault information to the flywheel array control module;

[0017] 2) The flywheel array control module controls the grid-connected static switch under the faulty flywheel device to be disconnected; assuming that there are N normal flywheel devices in total, the flywheel energy storage array composed of the N normal flywheel devices is recorded as a new flywheel energy storage array;

[0018] 3) The flywheel array control module obtains train operation data, passenger data and line loss data from the train control center through the communication module; the flywheel array control module obtains the state of charge value SOC of the new flywheel energy storage array 阵列 , the output energy E of the DC traction station sub and the output power P of the DC traction station sub ;

[0019] 4) the charge and discharge threshold prediction module obtains the charge and discharge voltage threshold set according to method 2, and then the flywheel array control module controls each corresponding normal flywheel device to adjust its own charge and discharge voltage threshold according to the multiple charge and discharge voltage thresholds governed by the charge and discharge voltage threshold set;

[0020] The energy prediction module obtains the total energy that the new flywheel energy storage array needs to absorb or release according to method three, and then the energy prediction module transmits the total energy information to the energy distribution module; then the energy distribution module obtains the energy set to be distributed according to method four; then the flywheel array control module controls each corresponding normal flywheel device to absorb or release the distributed energy according to the energy set to be distributed; the value of the total energy includes positive and negative values, when the total energy is positive, the new flywheel energy storage array needs to absorb the total energy on the DC bus to store energy; when the total energy is negative, the new flywheel energy storage array needs to release the total energy to the DC bus to supply energy;

[0021] The method 1 comprises:

[0022] 1) Collecting the relationship data between the train operation data, passenger data and line loss data and the total energy absorbed or released by the flywheel energy storage array from the existing data and generating a data set, recording the data set generated from the relationship data between the train operation data and the total energy absorbed or released by the flywheel energy storage array as the first data set, recording the data set generated from the relationship data between the passenger data and the total energy absorbed or released by the flywheel energy storage array as the second data set, and recording the data set generated from the relationship data between the line loss data and the total energy absorbed or released by the flywheel energy storage array as the third data set;

[0023] 2) using the first data set, the second data set and the third data set to train the first base learner, the second base learner and the third base learner respectively; the predicted values ​​of the three base learners and the data features of the three data sets are made into a secondary data set;

[0024] 3) Using the secondary data set to train the secondary learner; the multi-source information fusion model obtained after the training is recorded as the available multi-source information fusion model;

[0025] The second method comprises:

[0026] Taking the charge and discharge threshold prediction module as the intelligent agent, the power supply system of the traction section, the flywheel energy storage system and the running train of the traction section as the environment of the intelligent agent, the DDPG algorithm is used to obtain the charging voltage threshold of each normal flywheel device, and the DDPG algorithm is used to obtain the discharging voltage threshold of each normal flywheel device;

[0027] The state s of DDPG is defined according to Formula 1; the charging voltage threshold of each normal flywheel device is obtained and the action u of DDPG is defined according to Formula 2. ch , obtain the discharge voltage threshold of each normal flywheel device and define the action u of DDPG according to formula 3 ds ; Define the DDPG reward r according to formula 4;

[0028] The formula 1 is:

[0029] s=[(d1,p1),…,(d i , p i ),…,(d k , p k ), SOC 阵列 , E sub , P sub ]

[0030] Where k is the number of trains running on the traction section, d i is the position of the i-th train, i is an integer from 1 to k, p i is the power of the i-th train;

[0031] The formula 2 is:

[0032] u ch =[u ch1 ,…,u chj ,…,u chN ]

[0033] Among them, u chj is the charging voltage threshold of the jth normal flywheel device, where j is an integer from 1 to N;

[0034] The formula three is:

[0035] u ds =[u ds1 ,…,u dsj ,…,u dsN ]

[0036] Among them, u dsj is the discharge voltage threshold of the jth normal flywheel device;

[0037] The formula 4 is:

[0038] r=-0.5Δv%-0.5Δe%

[0039] Among them, Δv% is the voltage stabilization rate of the agent within the time step ΔT, Δe% is the energy saving rate of the agent within the time step ΔT, and ΔT is the set value;

[0040] After obtaining the charging voltage threshold and discharging voltage threshold of each normal flywheel device according to the above method, (u chj ,u dsj ) is recorded as the charge and discharge voltage threshold of the jth normal flywheel device, and [(u ch1 ,u ds1 ),…,(u chj ,u dsj ),…,(u chN ,u dsN )] as the charge and discharge voltage threshold set;

[0041] The third method comprises:

[0042] A) The train operation data, passenger data and line loss data are respectively used as the input data of the first base learner, the second base learner and the third base learner in the available multi-source information fusion model, and the three preliminary prediction values ​​of the total energy are obtained after being processed by the three learners;

[0043] B) using the three preliminary predicted values ​​of the total energy as input data of the secondary learner, and obtaining the total energy that the new flywheel energy storage array needs to absorb or release through fusion processing of the secondary learner;

[0044] The fourth method comprises:

[0045] a) The energy distribution module uses the A2C algorithm to obtain the energy distribution weight of each normal flywheel device through the energy distribution weight model, wherein the working parameters, real-time state of charge value, standard charge and discharge voltage threshold, network voltage difference and charge and discharge operation of each normal flywheel device are taken as the state, the energy distribution weight of each normal flywheel device is taken as the action, and the minimum difference between the maximum and minimum state of charge values ​​of each normal flywheel device after the total energy distribution is taken as the reward target; the sum of the energy distribution weights of all normal flywheel devices is 1;

[0046] b) The energy distribution module distributes the total energy according to the energy distribution weight of each normal flywheel device to obtain the to-be-distributed energy of each normal flywheel device. The to-be-distributed energy of all normal flywheel devices constitutes an to-be-distributed energy set.

[0047] Furthermore, in step 2), the QPSO algorithm is used to adjust the hyperparameters of the three base learners.

[0048] The principle of the present invention is as follows:

[0049] When the flywheel energy storage array is shut down due to a rolling bearing failure in one of the energy storage units, it not only affects the energy recovery and utilization on the DC bus of the traction network, but also causes the DC bus voltage to fluctuate, affecting the normal operation of the train. The present invention proposes a new concept: when the bearing of one of the flywheel energy storage devices (units) fails, the failed flywheel energy storage device is cut off from the flywheel energy storage array, and the remaining flywheel energy storage devices are allowed to form a new flywheel energy storage array to continue working, so that the stability of the track power supply system can be guaranteed to the greatest extent, and the recovery and utilization of braking energy can continue to be realized. However, in the prior art, the above scheme still has the following problems to be solved: First, after the failed flywheel energy storage device is cut out, the structure of the entire flywheel energy storage array changes, and its processing capacity for energy recovery and utilization changes. In particular, if the charge and discharge voltage thresholds of each flywheel energy storage device are not adjusted and optimized, and energy storage control is still performed according to the original configuration, it will cause voltage fluctuations in the DC bus of the traction network, affecting the normal power supply to the train. Secondly, due to the decrease in the energy processing capacity of the new flywheel energy storage array, the spare space for energy processing is compressed. In the process of energy recycling, in order to optimize and give full play to the energy processing capacity of the remaining flywheel energy storage device, it is necessary to accurately predict the total energy on the traction network. This total energy is related to train operation data, passenger data and line loss data, and the influencing factors are very complex. For example, under the same traction station, there are train departure intervals, vehicle density, vehicle formation, passenger flow, passenger luggage weight, traction network voltage, traction, braking current, train to traction station and other factors that are changing at any time, and the existing energy prediction method is only for a single train, and does not consider the changes brought about by the traction and braking interaction between trains on the same traction section. That is, the task of the flywheel energy storage system in the traction station under regenerative braking and traction will be reduced; at the same time, the existing energy prediction method does not consider the actual problem of network line energy loss when the current is transmitted in the traction network during train traction and braking. Therefore, it is difficult for the existing technology to achieve dynamic matching of the charging and discharging voltage thresholds of each flywheel energy storage device in the flywheel energy storage array with the real-time operation conditions of the line. The existing technology has prediction difficulties under multi-vehicle and dynamic operation conditions.

[0050] In view of the above difficulties, the present invention adopts the following solutions to solve them:

[0051] This application uses the DDPG deep deterministic policy gradient algorithm to solve the charge and discharge voltage thresholds of each flywheel energy storage device. In this process, the influence of factors such as the position and power of each running train in the traction section, the charge state of the flywheel energy storage array, the output energy and output power of the traction station on the charge and discharge voltage thresholds of the flywheel energy storage device is fully considered. With voltage stabilization and energy saving as the reward goals, through the intelligent agent search output, the predicted values ​​of the charge and discharge voltage thresholds of each flywheel energy storage device that are more matched with the current line operation conditions and more accurate can be obtained.

[0052] For the total energy absorbed or released by the traction section, this application creatively introduces the concept of the Stacking algorithm. From a data-driven perspective, it uses three machine learning models, GRU, Informer and AE, as base learners for energy prediction for multi-train operation data, in-vehicle and station passenger data, and traction line loss data. Among them, the Informer algorithm is born after innovative improvements to the Transformer. While inheriting the core advantages of the Transformer, it has made improvements in three aspects, including the introduction of sparse self-attention, self-attention distillation, and one-step forward prediction mechanisms, which improves the computational efficiency of the model and reduces GPU resource consumption. The predicted values ​​obtained by the above three models are then integrated through the OLS model as a secondary learner combined with the advantages of multiple different base learners, so that the accuracy of the total energy that needs to be absorbed or released by the new flywheel energy storage array obtained after the final fusion to improve the overall prediction model algorithm is greatly improved.

[0053] Moreover, the present invention introduces models constructed by intelligent algorithms in both the bearing fault diagnosis stage and the total energy distribution stage, so that both the accuracy of bearing fault diagnosis and the rationality of total energy distribution are greatly optimized compared with traditional methods. Therefore, the present invention adopts artificial intelligence methods from the diagnosis of bearing faults of each flywheel energy storage device to the acquisition of new charge and discharge voltage thresholds, and then to the prediction of the total energy of the flywheel energy storage array and the distribution of total energy. Through the above-mentioned "four-in-one" technical solution, the flywheel energy storage array automatically maintains the remaining non-fault flywheel energy storage devices to continue working after some flywheel energy storage devices fail, so as to meet the reasonable absorption and utilization of braking energy in the traction section, stabilize the traction network pressure, and ensure the normal operation of the train; at the same time, since the number of maintenance times of the flywheel energy storage array is reduced, the overall utilization rate of the equipment is improved, and the cost of equipment maintenance and use is greatly reduced.

[0054] It can be seen that the present invention has the following beneficial effects: the control system and control method described in the present invention can effectively solve the problem of partial flywheel energy storage device failure due to bearing reasons, and automatically maintain the continued operation of the flywheel energy storage array, which not only ensures the stability of the traction power supply system and the normal operation of the train, but also ensures the recovery and utilization of the braking recovery energy of the traction network, which not only reduces the operating cost of the traction power supply, but also greatly reduces the maintenance and use cost of the flywheel energy storage equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings of the present invention are as follows:

[0056] Attached Figure 1 The figure is a schematic diagram of the connection of the hardware involved in the present invention.

[0057] In the figure: 1. Flywheel array control module; 2. Flywheel energy storage unit; 3. Grid-connected static switch; 4. Bearing fault diagnosis module; 5. DC bus. DETAILED DESCRIPTION

[0058] The present invention will be further described below in conjunction with the embodiments.

[0059] The control system based on reinforcement learning under the train flywheel energy storage fault described in the present invention includes a train control center, a DC traction station and a flywheel energy storage system; the flywheel energy storage system is arranged in the DC traction station; the train running section powered by the DC traction station is recorded as the traction section, and the DC traction station supplies power to the train running in the traction section through the DC bus 5; the train control center can obtain the train operation data, passenger data and line loss data on the traction section; the train operation data includes train model and formation data, train speed and power information, line slope, vehicle density, train position, and train power; the passenger data includes passenger flow, real-time number of people in the car, luggage weight, and load in the car; the line loss data includes the distance from the train to the traction station, the train traction (braking) current, traction network resistance information, weather environment information (rain and snow conditions), and wind resistance intensity.

[0060] The flywheel energy storage system includes a flywheel array control module 1 and a flywheel energy storage array, wherein the flywheel energy storage array includes a plurality of flywheel energy storage devices; the flywheel array control module 1 includes a communication module, an energy prediction module, an energy distribution module, and a charge and discharge threshold prediction module; the energy prediction module is connected to the energy distribution module; the communication module is wirelessly connected to the train control center;

[0061] The energy prediction module includes a multi-source information fusion model, which includes a first base learner, a second base learner and a third base learner respectively constructed by three models: GRU (Gate Recurrent Unit), Informer (Long Sequence Time Series Prediction Model) and AE (Auto-Encoder), and also includes a secondary learner constructed by an OLS (ordinary least squares) model; the energy prediction module can process train operation data, passenger data and line loss data to obtain the total energy that the flywheel energy storage array needs to store or release;

[0062] The energy distribution module includes an energy distribution weight model constructed by an A2C (Advantage Actor-Critic) algorithm. The energy distribution module can generate energy distribution weights of each flywheel energy storage device through the energy distribution weight model, and distribute the total energy according to the energy distribution weight to obtain the energy that each flywheel energy storage device needs to store or release;

[0063] The charge and discharge threshold prediction module can process the train operation data, the charge state value of the flywheel energy storage array and the output energy and power data of the DC traction station to obtain the charge and discharge voltage thresholds of each flywheel energy storage device. The charge and discharge voltage thresholds described in the present application include: the voltage value at which the flywheel energy storage device starts discharging and the voltage value at which charging starts, and the voltage at which discharging starts is lower than the voltage at which charging starts.

[0064] The flywheel energy storage device includes a flywheel energy storage unit 2, a grid-connected static switch 3 and a bearing fault diagnosis module 4; the flywheel energy storage unit 2 adopts the conventional configuration of the prior art, including a flywheel, a motor, a bearing, a power electronic device, a DC / AC bidirectional converter, a unit controller, etc.; one end of the grid-connected static switch 3 is connected in parallel with the DC bus 5, and the other end of the grid-connected static switch 3 is connected to the flywheel energy storage unit 2; the flywheel energy storage unit 2 is connected to the flywheel array control module 1; the bearing fault diagnosis module 4 is connected to the flywheel array control module 1, the bearing fault diagnosis module 4 can collect the bearing working data of the flywheel energy storage unit 2, the bearing fault diagnosis module 4 can also process the collected bearing working data to obtain the bearing fault diagnosis result information, and the bearing fault diagnosis module 4 can also transmit the fault diagnosis result information to the flywheel array control module 1; the bearing working data mainly includes a vibration signal that can reflect the bearing fault information.

[0065] The flywheel array control module 1 can control the closing of the grid-connected static switch 3 according to the bearing fault diagnosis result information of the flywheel energy storage device; the flywheel array control module 1 can control the corresponding flywheel energy storage device to adjust the charging and discharging voltage threshold according to the charging and discharging voltage threshold; the flywheel array control module 1 can distribute the total energy to each flywheel energy storage device to realize the storage or release of the total energy.

[0066] The control method for the above control system includes:

[0067] The energy prediction module trains the multi-source information fusion model according to the method to obtain an available multi-source information fusion model;

[0068] Each bearing fault diagnosis module 4 periodically collects the bearing working data of the corresponding flywheel energy storage unit 2, performs preprocessing, and finally converts it into a time-frequency representation diagram. Among them, the preprocessing method can adopt technical means such as discrete wavelet transform, continuous wavelet transform and fast Fourier transform. For example, a preprocessing process is: first decompose the collected vibration signal into sub-signals of different frequency bands by discrete wavelet transform, and then convert these sub-signals into a time-frequency representation diagram by continuous wavelet transform.

[0069] Then the preprocessed data is input into the visual diagnosis module, which can process and analyze the input data and map it into a classification output result of whether there is a fault or the type of fault. In the present invention, each bearing fault diagnosis module 4 uses the ViT (Vision Transformer) algorithm to process the acquired bearing working data to obtain the bearing fault diagnosis results of each corresponding flywheel energy storage unit 2; ViT is an image classification model based on the Transformer architecture, which divides the input image into a series of image blocks, and converts these image blocks into vectors of fixed size through a linear projection layer, and retains the position information of the image through encoding vectors, and then these encoded image block sequences are input into the Transformer encoder. The Transformer encoder is mainly composed of a multi-head attention layer and a multi-layer perceptron (MLP) layer. The multi-head attention layer divides the input into multiple heads, so that each head can learn different levels of self-attention. Finally, the outputs of all heads are connected, and the output category of the network is obtained by adding the MLP head to the Transformer encoder. This Transformer-based ViT model has lower computational overhead and memory consumption than traditional convolutional neural networks, as well as fewer training parameters and training sample requirements. The above-mentioned ViT model is one of the implementation methods of the visual diagnosis module. Other methods such as convolutional neural network (CNN), long short-term memory network (LSTM), autoencoder, Transformer, generative adversarial network (GAN), gated recurrent unit (GRU), etc. can all achieve similar functions. Further optimization can be made on the basis of the above method. For example, multiple independent diagnosis modules can be set in the visual diagnosis module, and each module can perform independent fault diagnosis on the input data. Finally, the preliminary diagnosis results of multiple independent modules are fused using a soft voting method (weighted voting according to the independent fault output probability or confidence), and then the final fault diagnosis result is obtained. Such a method makes the module more generalizable and robust, and can further improve the accuracy of model diagnosis. The bearing fault diagnosis results include: faulty and normal; the flywheel energy storage device with a bearing fault diagnosis result of faulty is recorded as a faulty flywheel device, and the flywheel energy storage device with a bearing fault diagnosis result of normal is recorded as a normal flywheel device; when the fault diagnosis result is normal, no action is required. When a faulty flywheel device occurs,

[0070] 1) The bearing fault diagnosis module 4 under the faulty flywheel device generates bearing fault information, and then the bearing fault diagnosis module 4 transmits the bearing fault information to the flywheel array control module 1;

[0071] 2) the flywheel array control module 1 controls the grid-connected static switch 3 under the control of the faulty flywheel device to be disconnected, so as to cut the faulty flywheel device out of the flywheel energy storage array; assuming that there are N normal flywheel devices in total, the flywheel energy storage array composed of the N normal flywheel devices is recorded as a new flywheel energy storage array;

[0072] 3) The flywheel array control module 1 obtains train operation data, passenger data and line loss data from the train control center through the communication module; the flywheel array control module 1 obtains the state of charge value SOC of the new flywheel energy storage array 阵列 , the output energy E of the DC traction station sub and the output power P of the DC traction station sub ;

[0073] 4) the charge and discharge threshold prediction module obtains the charge and discharge voltage threshold set according to method 2, and then the flywheel array control module 1 controls each corresponding normal flywheel device to adjust its own charge and discharge voltage threshold according to the multiple charge and discharge voltage thresholds governed by the charge and discharge voltage threshold set;

[0074] The energy prediction module obtains the total energy that the new flywheel energy storage array needs to absorb or release according to method three, and then the energy prediction module transmits the total energy information to the energy distribution module; then the energy distribution module obtains the energy set to be distributed according to method four; then the flywheel array control module 1 controls each corresponding normal flywheel device to absorb or release the distributed energy according to the energy set to be distributed; the value of the total energy includes positive and negative values, when the total energy is positive, the new flywheel energy storage array needs to absorb the total energy on the DC bus 5 to store energy, and when the total energy is negative, the new flywheel energy storage array needs to release the total energy to the DC bus 5 to supply energy;

[0075] The method 1 comprises:

[0076] 1) Collecting the relationship data between the train operation data, passenger data and line loss data and the total energy absorbed or released by the flywheel energy storage array from the existing data and producing a data set, wherein the above data sets are all pre-processed by data cleaning, position point discretization sampling, normalization and other data according to the existing technical methods, and the data set produced by the relationship data between the train operation data and the total energy absorbed or released by the flywheel energy storage array is recorded as the first data set, the data set produced by the relationship data between the passenger data and the total energy absorbed or released by the flywheel energy storage array is recorded as the second data set, and the data set produced by the relationship data between the line loss data and the total energy absorbed or released by the flywheel energy storage array is recorded as the third data set;

[0077] 2) Using the first data set, the second data set and the third data set to train the first base learner, the second base learner and the third base learner respectively; the predicted values ​​of the three base learners and the data features of the three data sets are made into secondary data sets; the hyperparameters of the three base learners can be adjusted by using the QPSO (Quantum-behaved Particle Swarm Optimization) algorithm.

[0078] 3) Using the secondary data set to train the secondary learner; the multi-source information fusion model obtained after the training is recorded as the available multi-source information fusion model;

[0079] The second method comprises:

[0080] Taking the charge and discharge threshold prediction module as the intelligent agent, the power supply system of the traction section, the flywheel energy storage system and the running train of the traction section as the environment of the intelligent agent, the DDPG (Deep Deterministic Policy Gradient) algorithm is used to obtain the charging voltage threshold of each normal flywheel device, and the DDPG algorithm is used to obtain the discharge voltage threshold of each normal flywheel device;

[0081] The state s of DDPG is defined according to Formula 1; the charging voltage threshold of each normal flywheel device is obtained and the action u of DDPG is defined according to Formula 2. ch , obtain the discharge voltage threshold of each normal flywheel device and define the action u of DDPG according to formula 3 ds ; Define the DDPG reward r according to formula 4;

[0082] The formula 1 is:

[0083] s=[(d1,p1),…,(d i , p i ),…,(d k , p k ), SOC 阵列 , E sub , P sub ]

[0084] Where k is the number of trains running on the traction section, d i is the position of the i-th train, i is an integer from 1 to k, p i is the power of the i-th train;

[0085] The formula 2 is:

[0086] u ch =[u ch1 ,…,u chj ,…,u chN ]

[0087] Among them, u chj is the charging voltage threshold of the jth normal flywheel device, where j is an integer from 1 to N;

[0088] The formula three is:

[0089] u ds =[u ds1 ,…,u dsj ,…,u dsN ]

[0090] Among them, u dsj is the discharge voltage threshold of the jth normal flywheel device;

[0091] The formula 4 is:

[0092] r=-0.5Δv%-0.5Δe%

[0093] Among them, Δv% is the voltage stabilization rate of the agent within the time step ΔT, Δe% is the energy saving rate of the agent within the time step ΔT, and ΔT is the set value;

[0094] After obtaining the charging voltage threshold and discharging voltage threshold of each normal flywheel device according to the above method, (u chj ,u dsj ) is recorded as the charge and discharge voltage threshold of the jth normal flywheel device, and [(u ch1 ,u ds1 ),…,(u chj ,u dsj ),…,(u chN ,u dsN )] as the charge and discharge voltage threshold set;

[0095] The third method comprises:

[0096] A) The train operation data, passenger data and line loss data are respectively used as the input data of the first base learner, the second base learner and the third base learner in the available multi-source information fusion model, and the three preliminary prediction values ​​of the total energy are obtained after being processed by the three learners;

[0097] B) using the three preliminary predicted values ​​of the total energy as input data of the secondary learner, and obtaining the total energy that the new flywheel energy storage array needs to absorb or release through fusion processing of the secondary learner;

[0098] The fourth method comprises:

[0099] a) The energy distribution module uses the A2C algorithm to obtain the energy distribution weight of each normal flywheel device through the energy distribution weight model, wherein the working parameters, real-time state of charge value, standard charge and discharge voltage threshold, network voltage difference and charge and discharge operation of each normal flywheel device are taken as the state, the energy distribution weight of each normal flywheel device is taken as the action, and the minimum difference between the maximum and minimum state of charge values ​​of each normal flywheel device after the total energy distribution is taken as the reward target; the sum of the energy distribution weights of all normal flywheel devices is 1;

[0100] b) The energy distribution module distributes the total energy according to the energy distribution weight of each normal flywheel device (the energy distribution weight of each normal flywheel device is multiplied by the total energy) to obtain the energy to be distributed of each normal flywheel device. The energy to be distributed of all normal flywheel devices constitutes an energy set to be distributed.

[0101] The GRU algorithm, Informer algorithm, AE algorithm, OLS algorithm, A2C algorithm, ViT algorithm, DDPG algorithm, QPSO algorithm, Stacking algorithm, discrete wavelet transform, continuous wavelet transform and fast Fourier transform used in the present invention are all very common processing means or calculation methods in the prior art. Those skilled in the art can obtain relevant content from the literature related to the prior art.

Claims

1. A control system based on reinforcement learning under train flywheel energy storage failure, characterized in that: It includes a train control center, a DC traction station and a flywheel energy storage system; the flywheel energy storage system is arranged in the DC traction station; the train running section powered by the DC traction station is recorded as the traction section, and the DC traction station supplies power to the train running in the traction section through the DC bus; the train control center can obtain the train running data, passenger data and line loss data on the traction section; The flywheel energy storage system includes a flywheel array control module and a flywheel energy storage array, wherein the flywheel energy storage array includes a plurality of flywheel energy storage devices; the flywheel array control module includes a communication module, an energy prediction module, an energy distribution module, and a charge and discharge threshold prediction module; the energy prediction module is connected to the energy distribution module; the communication module is wirelessly connected to the train control center; The energy prediction module includes a multi-source information fusion model, which includes a first base learner, a second base learner and a third base learner respectively constructed by using three models of GRU, Informer and AE, and also includes a secondary learner constructed by using an OLS model; the energy prediction module can process train operation data, passenger data and line loss data to obtain the total energy that the flywheel energy storage array needs to store or release; The energy distribution module includes an energy distribution weight model constructed by the A2C algorithm. The energy distribution module can generate energy distribution weights of each flywheel energy storage device through the energy distribution weight model, and distribute the total energy according to the energy distribution weight to obtain the energy that each flywheel energy storage device needs to store or release; The charge and discharge threshold prediction module can process the train operation data, the charge state value of the flywheel energy storage array and the output energy and power data of the DC traction station to obtain the charge and discharge voltage thresholds of each flywheel energy storage device; The flywheel energy storage device includes a flywheel energy storage unit, a grid-connected static switch and a bearing fault diagnosis module; one end of the grid-connected static switch is connected in parallel with the DC bus, and the other end of the grid-connected static switch is connected to the flywheel energy storage unit; the flywheel energy storage unit is connected to the flywheel array control module; the bearing fault diagnosis module is connected to the flywheel array control module, the bearing fault diagnosis module can collect the bearing working data of the flywheel energy storage unit, the bearing fault diagnosis module can also process the collected bearing working data to obtain the bearing fault diagnosis result information, and the bearing fault diagnosis module can also transmit the fault diagnosis result information to the flywheel array control module; The flywheel array control module can control the closing of the grid-connected static switch according to the bearing fault diagnosis result information of the flywheel energy storage device; the flywheel array control module can control the corresponding flywheel energy storage device to adjust the charging and discharging voltage threshold according to the charging and discharging voltage threshold; the flywheel array control module can distribute the total energy to each flywheel energy storage device to realize the storage or release of the total energy.

2. A control method based on reinforcement learning for train flywheel energy storage failure, characterized in that: The hardware involved includes: a train control center, a DC traction station and a flywheel energy storage system; the flywheel energy storage system is arranged in the DC traction station; the train running section powered by the DC traction station is recorded as the traction section, and the DC traction station supplies power to the train running in the traction section through the DC bus; the train control center can obtain the train running data, passenger data and line loss data on the traction section; The flywheel energy storage system includes a flywheel array control module and a flywheel energy storage array, wherein the flywheel energy storage array includes a plurality of flywheel energy storage devices; the flywheel array control module includes a communication module, an energy prediction module, an energy distribution module, and a charge and discharge threshold prediction module; the energy prediction module is connected to the energy distribution module; the communication module is wirelessly connected to the train control center; The energy prediction module includes a multi-source information fusion model, which includes a first base learner, a second base learner and a third base learner respectively constructed by using three models of GRU, Informer and AE, and also includes a secondary learner constructed by using an OLS model; the energy prediction module can process train operation data, passenger data and line loss data to obtain the total energy that the flywheel energy storage array needs to store or release; The energy distribution module includes an energy distribution weight model constructed by the A2C algorithm. The energy distribution module can generate energy distribution weights of each flywheel energy storage device through the energy distribution weight model, and distribute the total energy according to the energy distribution weight to obtain the energy that each flywheel energy storage device needs to store or release; The charge and discharge threshold prediction module can process the train operation data, the charge state value of the flywheel energy storage array and the output energy and power data of the DC traction station to obtain the charge and discharge voltage thresholds of each flywheel energy storage device; The flywheel energy storage device includes a flywheel energy storage unit, a grid-connected static switch and a bearing fault diagnosis module; one end of the grid-connected static switch is connected in parallel with the DC bus, and the other end of the grid-connected static switch is connected to the flywheel energy storage unit; the flywheel energy storage unit is connected to the flywheel array control module; the bearing fault diagnosis module is connected to the flywheel array control module, the bearing fault diagnosis module can collect the bearing working data of the flywheel energy storage unit, the bearing fault diagnosis module can also process the collected bearing working data to obtain the bearing fault diagnosis result information, and the bearing fault diagnosis module can also transmit the fault diagnosis result information to the flywheel array control module; The flywheel array control module can control the closing of the grid-connected static switch according to the bearing fault diagnosis result information of the flywheel energy storage device; the flywheel array control module can control the corresponding flywheel energy storage device to adjust the charge and discharge voltage threshold according to the charge and discharge voltage threshold; the flywheel array control module can distribute the total energy to each flywheel energy storage device to realize the storage or release of the total energy; The control method comprises: The energy prediction module trains the multi-source information fusion model according to the method to obtain an available multi-source information fusion model; Each bearing fault diagnosis module periodically collects the bearing working data of the corresponding flywheel energy storage unit, and then each bearing fault diagnosis module uses the ViT algorithm to process the acquired bearing working data to obtain the bearing fault diagnosis results of each corresponding flywheel energy storage unit; the bearing fault diagnosis results include: faulty and normal; the flywheel energy storage device with a bearing fault diagnosis result of faulty is recorded as a faulty flywheel device, and the flywheel energy storage device with a bearing fault diagnosis result of normal is recorded as a normal flywheel device; When a faulty flywheel occurs, 1) A bearing fault diagnosis module under the faulty flywheel device generates bearing fault information, and then the bearing fault diagnosis module transmits the bearing fault information to the flywheel array control module; 2) The flywheel array control module controls the grid-connected static switch under the faulty flywheel device to be disconnected; assuming that there are N normal flywheel devices in total, the flywheel energy storage array composed of the N normal flywheel devices is recorded as a new flywheel energy storage array; 3) The flywheel array control module obtains train operation data, passenger data and line loss data from the train control center through the communication module; the flywheel array control module obtains the state of charge value SOC of the new flywheel energy storage array 阵列 , the output energy E of the DC traction station sub and the output power P of the DC traction station sub ; 4) the charge and discharge threshold prediction module obtains the charge and discharge voltage threshold set according to method 2, and then the flywheel array control module controls each corresponding normal flywheel device to adjust its own charge and discharge voltage threshold according to the multiple charge and discharge voltage thresholds governed by the charge and discharge voltage threshold set; The energy prediction module obtains the total energy that the new flywheel energy storage array needs to absorb or release according to method three, and then the energy prediction module transmits the total energy information to the energy distribution module; then the energy distribution module obtains the energy set to be distributed according to method four; then the flywheel array control module controls each corresponding normal flywheel device to absorb or release the distributed energy according to the energy set to be distributed; the value of the total energy includes positive and negative values, when the total energy is positive, the new flywheel energy storage array needs to absorb the total energy on the DC bus to store energy; when the total energy is negative, the new flywheel energy storage array needs to release the total energy to the DC bus to supply energy; The method 1 comprises: 1) Collecting the relationship data between the train operation data, passenger data and line loss data and the total energy absorbed or released by the flywheel energy storage array from the existing data and generating a data set, recording the data set generated from the relationship data between the train operation data and the total energy absorbed or released by the flywheel energy storage array as the first data set, recording the data set generated from the relationship data between the passenger data and the total energy absorbed or released by the flywheel energy storage array as the second data set, and recording the data set generated from the relationship data between the line loss data and the total energy absorbed or released by the flywheel energy storage array as the third data set; 2) using the first data set, the second data set and the third data set to train the first base learner, the second base learner and the third base learner respectively; the predicted values ​​of the three base learners and the data features of the three data sets are made into a secondary data set; 3) Using the secondary data set to train the secondary learner; the multi-source information fusion model obtained after the training is recorded as the available multi-source information fusion model; The second method comprises: Taking the charge and discharge threshold prediction module as the intelligent agent, the power supply system of the traction section, the flywheel energy storage system and the running train of the traction section as the environment of the intelligent agent, the DDPG algorithm is used to obtain the charging voltage threshold of each normal flywheel device, and the DDPG algorithm is used to obtain the discharging voltage threshold of each normal flywheel device; The state s of DDPG is defined according to Formula 1; the charging voltage threshold of each normal flywheel device is obtained and the action u of DDPG is defined according to Formula 2. ch , obtain the discharge voltage threshold of each normal flywheel device and define the action u of DDPG according to formula 3 ds ; Define the DDPG reward r according to formula 4; The formula 1 is: s=[(d1,p1),…,(d i ,p i ),…,(d k ,p k ),SOC 阵列 ,E sub ,P sub ] Where k is the number of trains running on the traction section, d i is the position of the i-th train, i is an integer from 1 to k, p i is the power of the i-th train; The formula 2 is: in ch =[in ch1 ,…,in chj ,…,in chN ] Among them, u chj is the charging voltage threshold of the jth normal flywheel device, where j is an integer from 1 to N; The formula three is: in ds =[in ds1 ,…,in dsj ,…,in dsN ] Among them, u dsj is the discharge voltage threshold of the jth normal flywheel device; The formula 4 is: r=--0.5Δv%-0.5Δe% Among them, Δv% is the voltage stabilization rate of the agent within the time step ΔT, Δe% is the energy saving rate of the agent within the time step ΔT, and ΔT is the set value; After obtaining the charging voltage threshold and discharging voltage threshold of each normal flywheel device according to the above method, (u chj ,u dsj ) is recorded as the charge and discharge voltage threshold of the jth normal flywheel device, and [(u ch1 ,u ds1 ),…,(u chj ,u dsj ),…,(u chN ,u dsN )] as the charge and discharge voltage threshold set; The third method comprises: A) The train operation data, passenger data and line loss data are respectively used as the input data of the first base learner, the second base learner and the third base learner in the available multi-source information fusion model, and the three preliminary prediction values ​​of the total energy are obtained after being processed by the three learners; B) using the three preliminary predicted values ​​of the total energy as input data of the secondary learner, and obtaining the total energy that the new flywheel energy storage array needs to absorb or release through fusion processing of the secondary learner; The fourth method comprises: a) The energy distribution module uses the A2C algorithm to obtain the energy distribution weight of each normal flywheel device through the energy distribution weight model, wherein the working parameters, real-time state of charge value, standard charge and discharge voltage threshold, network voltage difference and charge and discharge operation of each normal flywheel device are taken as the state, the energy distribution weight of each normal flywheel device is taken as the action, and the minimum difference between the maximum and minimum state of charge values ​​of each normal flywheel device after the total energy distribution is taken as the reward target; the sum of the energy distribution weights of all normal flywheel devices is 1; b) The energy distribution module distributes the total energy according to the energy distribution weight of each normal flywheel device to obtain the to-be-distributed energy of each normal flywheel device. The to-be-distributed energy of all normal flywheel devices constitutes an to-be-distributed energy set.

3. The control method based on reinforcement learning under train flywheel energy storage failure as claimed in claim 2, characterized in that: In the step 2), the QPSO algorithm is used to adjust the hyperparameters of the three base learners.

Images