Method for controlling rail traffic of a plurality of railway vehicles, computing unit and computer program

Abstract

The invention relates to a method (100) for controlling a rail traffic of a plurality of rail vehicles (215), comprising: - receiving (101) status data of the rail traffic; - receiving (103) a control target (KPI1, KPI2, KPIN) for controlling the rail traffic of the plurality of rail vehicles (215); - Selecting (105) at least one action selection rule (Π1, Π2, ΠN) from a plurality of action selection rules (Π1, Π2, ΠN) based on the control objective (KPI1, KPI2, KPIN), wherein the action selection rules (Π1, Π2, ΠN) are set up to determine control actions of the rail traffic based on state data of the rail traffic, wherein by executing the control actions by the rail vehicles (215) the rail traffic can be transformed into a state optimized with respect to a control objective (KPI1, KPI2, KPIN);- Executing (107) the at least one selected action selection rule (Π1, Π2, ΠN) on the received state data and determining control actions; and - providing (109) the control actions for controlling the majority of rail vehicles (215).;

Description

[0001] The invention relates to a method for controlling rail traffic of a plurality of rail vehicles.

[0002] The control of rail traffic involving a plurality of rail vehicles, such as in a subway or tram network, can be optimized for various control objectives, such as advantageous energy consumption or the shortest possible delays. For this purpose, optimized timetables can be generated according to which the rail traffic, and in particular the majority of rail vehicles, can be controlled. These optimized timetables are usually determined or calculated in an offline state, i.e., not during rail operation. However, if situations arise during rail operation in which deviations from the predefined and optimized timetable are necessary, the problem arises as to how the most optimal control of rail traffic can be achieved in the online state, i.e., during operation.

[0003] WO 2020 / 043397 A1 discloses energy optimization in the operation of a rail vehicle fleet. Furthermore, US Pat. No. 6,459,964 B1 describes a method for revising a train timetable, and EP 2 619 067 B1 describes a method for automatically controlling a plurality of rail-bound vehicles.

[0004] The invention is based on the object of providing an improved method for controlling rail traffic of a plurality of rail vehicles.

[0005] This object is achieved by a method for controlling rail traffic of a plurality of rail vehicles according to independent claim 1. Advantageous embodiments are the subject of the subordinate claims.

[0006] According to one aspect of the invention, a method for controlling rail traffic of a plurality of rail vehicles is provided, the method comprising: Receiving status data of the plurality of rail vehicles of the rail transport system, wherein the status data describe control states of the plurality of rail vehicles and comprise at least position data of the rail vehicles within a rail transport network; Receiving a control target for controlling the rail traffic of the plurality of rail vehicles;Selecting at least one action selection rule from a plurality of action selection rules based on the control objective, wherein the action selection rules are configured to determine control actions of the rail traffic based on status data of the rail traffic, wherein, by executing the control actions by the rail vehicles, the rail traffic can be transferred to a state optimized with respect to a control objective, wherein each action selection rule is configured for optimization with respect to an individual control objective, and wherein the at least one selected action selection rule is configured for optimization with respect to the received control objective; executing the at least one selected action selection rule on the received status data and determining control actions; and providing the control actions for controlling the plurality of rail vehicles.

[0007] This makes it possible to achieve the technical advantage of providing an improved method for controlling rail traffic involving a plurality of rail vehicles, in which method a change to a control objective according to which the rail traffic is to be controlled or optimized can be made during the operation of the rail traffic and the movement of the rail vehicles within a rail transport network. For this purpose, a plurality of different action selection rules are available, by means of which control actions can be determined based on status data of the rail traffic, which control actions, when executed by the plurality of rail vehicles of the rail traffic, enable an optimization of the rail traffic with regard to a predetermined control objective. Upon receipt of a control objective to be changed, an action selection rule can be selected which is optimized for the respective control objective.is configured to determine corresponding control actions that, when executed by the respective rail vehicles, are configured to achieve the corresponding control objective. This makes it possible to change the control objective at any time during rail traffic operation, and to easily determine control actions for optimizing rail traffic with respect to the selected control objective by executing the respective action selection rule. This eliminates the need for complex adjustments to the rail traffic control system.

[0008] An action selection rule, in the sense of the application, is a policy known from the field of reinforcement learning, which is designed to determine, for any given state of rail traffic, appropriate control actions that are suitable for transferring rail traffic to a state optimized with regard to a predetermined control objective.

[0009] A control objective, as defined in the application, is a target according to which rail traffic for a majority of rail vehicles is to be optimized or improved. A control objective or a corresponding optimization criterion for rail traffic can include, for example, energy consumption, an overall delay of rail vehicles, or similar factors relevant to the operation of a majority of rail vehicles.

[0010] For the purposes of the application, rail transport is a collection of a plurality of rail vehicles within a rail network and is characterized by states in which control states of the individual rail vehicles include at least position information of the rail vehicles within the rail network and time information, for example, delay times of the plurality of rail vehicles relative to a predetermined timetable, energy consumption, or other control parameters of the plurality of rail vehicles as a whole. A change in rail transport during the control of the rail vehicles within the rail network is described by state changes in rail transport, in which the control states of the plurality of rail vehicles as well as the control parameters, such as energy consumption or total delay, undergo a temporal change.

[0011] Control actions, as defined in the application, are actions or control specifications according to which the majority of rail vehicles are to be controlled. Control actions, as defined in the application, can include, in addition to explicit control commands to the individual rail vehicles, time specifications that describe, for example, arrival times, departure times, or idle times of the individual rail vehicles at stops within the rail network. The rail vehicles must be controlled in such a way that they meet or adhere to the corresponding time specifications of the specifications.

[0012] For the purposes of the application, optimizing rail traffic with respect to a selected or predetermined control objective involves improving the current state of rail traffic with respect to the respective control objective. Thus, for the purposes of the application, optimizing rail traffic does not necessarily have to lead to an optimal state of rail traffic with respect to the respective control objective, but can merely be limited by an improvement with respect to the control objective and, in comparison, to previous states of rail traffic.

[0013] The method according to the invention for controlling rail traffic can thus be carried out during rail traffic operation, during which the individual rail vehicles are moved within the rail network. The optimization of rail traffic can be carried out with respect to a predetermined timetable according to which the majority of rail vehicles within the rail network are to be controlled.

[0014] According to one embodiment, the received control target is a combined control target and comprises a plurality of control targets, wherein the control targets are included in the combined control target with individual weightings; comprising: Selecting a plurality of action selection rules, wherein each action selection rule is configured to optimize with respect to one of the plurality of control objectives of the combined control objective; and executing the plurality of selected action selection rules according to an execution ratio, wherein the execution ratio determines a time proportion of a total execution duration of the plurality of selected action selection rules for each selected selection rule, and wherein an execution of the plurality of action selection rules according to the execution ratio satisfies the combined control objective.

[0015] This can achieve the technical advantage of further improving rail traffic control by allowing for the consideration of combined control objectives and the corresponding optimization of rail traffic with respect to such control objectives. A combined control objective comprises a plurality of predetermined control objectives, each of which is incorporated into the combined control objective with a corresponding weighting. This enables rail traffic control with respect to a plurality of control objectives.

[0016] A combined control objective, as defined in the application, is defined by a plurality of at least two predetermined control objectives, which are incorporated into the combined control objective with a corresponding weighting. Thus, for example, a rail transport system can be controlled simultaneously with respect to the control objectives of energy consumption and delay reduction, for example, in a ratio of 60 to 40. For this purpose, a plurality of corresponding action selection rules are selected, each optimized for the individual predetermined control objectives of the combined control objective, and executed according to an execution ratio. The execution ratio is designed such that the execution of the selected action selection rules according to the execution ratio leads to the fulfillment of the combined control objective.

[0017] For the purposes of the application, executing a plurality of action selection rules according to an execution ratio corresponds to executing the control actions determined by the individual action selection rules according to the respective execution ratio. For the purposes of the application, an execution ratio describes a temporal portion of the respective execution of the action selection rule within a predetermined execution period for each action selection rule to be considered. For the purposes of the application, executing the individual control actions of the various action selection rules according to the execution ratio means that, according to the respective execution ratio, the rail traffic is controlled according to the respective action selection rules.

[0018] According to one embodiment, the execution ratio for each action selection rule determines an execution duration and / or a temporal sequence of the plurality of action selection rules.

[0019] This can achieve the technical advantage of enabling a precise and easy-to-determine execution ratio by determining an execution duration within the respective execution period for each action selection rule and / or an execution sequence for the majority of action selection rules involved.

[0020] According to one embodiment, the execution ratio determines an execution probability for each selected action selection rule, according to which the action selection rule is to be executed in an execution period.

[0021] This provides the technical advantage of providing a precise and easily determined execution ratio. For this purpose, an execution probability of the respective action selection rule being executed within a corresponding execution period is determined for each action selection rule involved. The execution probability describes the probability that the rail traffic will be controlled according to the control actions determined by the respective action selection rule during a predetermined period.

[0022] According to one embodiment, the selected action selection rules are executed cyclically, wherein the rail traffic is controlled cyclically according to the control actions determined by the action selection rules.

[0023] This can achieve the technical advantage of enabling precise control of rail traffic.

[0024] According to one embodiment, the selection and / or execution of the action selection rules is performed by a time-sharing function, wherein the time-sharing function is trained by machine learning to determine appropriate execution ratios for combined control objectives.

[0025] This provides the technical advantage of enabling precise selection of the respective control actions to be executed and precise determination of the respective execution ratios. The time-sharing function, which is trained using machine learning techniques to determine appropriate execution ratios of the respective action selection rules for any combined control objectives, enables precise and reliable control of a highly complex rail traffic system comprising a large number of rail vehicles and optimization of such a system with regard to combined control objectives. The use of machine learning techniques enables more precise determination of the corresponding execution ratios and thus more effective and efficient control of rail traffic.

[0026] According to the invention, the action selection rules are trained through reinforcement learning.

[0027] According to one embodiment, the time-sharing function is trained by reinforcement learning.

[0028] This provides the technical advantage of providing precise and efficient training of the time allocation function or the action selection rules.

[0029] According to one embodiment, the training of the action selection rules and / or the time division function is carried out based on simulation data, wherein the simulation data is based on a simulation of a rail traffic of a plurality of rail vehicles.

[0030] This offers the technical advantage of enabling effective and efficient training of the action selection rules and the time allocation function. Furthermore, generating corresponding training data from actual journeys of the majority of rail vehicles is eliminated.

[0031] According to the invention, the training of the action selection rules comprises maximizing reward functions and the reward functions define control objectives and / or combined control objectives.

[0032] According to one embodiment, training the time-sharing function comprises maximizing reward functions, wherein the reward functions define control objectives and / or combined control objectives.

[0033] This can achieve the technical advantage of effective and efficient training of the action selection rules or the time allocation function.

[0034] According to one embodiment, the action selection rules and the time-division functions are trained simultaneously. This can achieve the technical advantage of enabling improved control of rail traffic. The simultaneous training of the time-division function and the plurality of action selection rules with respect to combined control objectives makes it possible to adapt the individual control actions of the various action selection rules to their interaction with the other control selection rules. This adaptation can achieve better performance of the action selection rules than would be possible with individual training of the individual action selection rules for individual control objectives and merely a combination of the individually trained action selection rules using the time-division function to fulfill a combined control objective.

[0035] According to one embodiment, the action selection rules and / or the time allocation function are implemented as neural networks.

[0036] This can achieve the technical advantage of providing a powerful and robust design of the action selection rules or time allocation functions.

[0037] According to one embodiment, control objectives include: minimizing energy consumption of the plurality of rail vehicles and / or minimizing energy consumption of individual rail vehicles and / or minimizing a total delay of the plurality of rail vehicles and / or minimizing a maximum delay of individual rail vehicles and / or minimizing a quadratic or absolute deviation sum of the delays with respect to a mean or median delay value of the plurality of rail vehicles.

[0038] This can achieve the technical advantage of enabling precise control of rail traffic, whereby rail traffic can be optimized with regard to relevant control objectives.

[0039] According to one embodiment, the control actions include arrival times and / or departure times and / or idle times of the plurality of rail vehicles at stops within the rail transport network.

[0040] This can achieve the technical advantage of enabling efficient control of rail traffic while taking combined control objectives into account. By taking into account the departure and / or arrival times and / or idle times of rail vehicles at stops within the rail network in the respective control actions, precise control of rail traffic can be achieved. The corresponding arrival and / or departure times and / or idle times can be related to a predetermined, optimized timetable. The control of the individual rail vehicles of the rail traffic can be designed in such a way that the departure and / or arrival times and / or idle times defined in the control actions are adhered to by the respective rail vehicles.

[0041] According to a second aspect, a computing unit is provided, wherein the computing unit is designed to carry out the method for controlling rail traffic of a plurality of rail vehicles according to one of the preceding embodiments.

[0042] According to a third aspect of the invention, a computer program product comprising instructions is provided which, when the program is executed by a data processing unit, cause the data processing unit to carry out the method for controlling rail traffic of a plurality of rail vehicles according to one of the preceding embodiments.

[0043] The above-described features and advantages of this invention, as well as the manner in which they are achieved, will become clearer and more readily understood through the explanations of the following highly simplified schematic representations of preferred embodiments. In each case: FIG 1 shows a schematic representation of a system for controlling rail traffic according to an embodiment; FIG 2 shows a flowchart of a method for controlling rail traffic according to an embodiment; FIG 3 shows a diagram of an optimization of rail traffic with respect to two control objectives; FIG 4 shows a schematic representation of a temporal execution of two action selection rules according to an execution ratio; FIG 5 shows a schematic representation of an architecture for a time-division function according to an embodiment; FIG 6 shows a further schematic representation of an architecture for a time-division function according to a further embodiment; FIG 7 shows a diagram of a hypervolume of an optimization state of rail traffic; and FIG 8 shows a schematic representation of a computer program product. FIG 1 shows a schematic representation of a system 200 for optimizing rail traffic according to an embodiment.

[0044] According to the embodiment shown, a system 200 for optimizing rail traffic comprises a plurality of modules that can be executed on a computing unit 201. The system 200 can further be divided into an offline subsystem 202 and an online subsystem 204, wherein the offline subsystem 202 is executed offline, i.e., independently of the operation of the rail vehicles 215, while the online subsystem 204 is executed during the operation of the rail vehicles 215.

[0045] The central component of the offline subsystem 202 is a timetable optimization module 211. The timetable optimization module 211 is used to create an optimized timetable for a rail traffic operation of a plurality of rail vehicles in a rail network for a predetermined period of time. The timetable optimization module 211 can transmit the optimized timetable created in the offline subsystem 202 to the online subsystem 204 via a first interface so that the timetable can be executed to control the rail traffic.

[0046] In actual rail traffic operations, the timetable, now called the online timetable, is managed by a timetable management module 205. For this purpose, position data of individual rail vehicles can be transmitted via an automatic rail vehicle tracking module 207 via a second interface S2 to the timetable management module 205 so that the latter can perform a comparison between the target movements of the timetable and the actual movements actually performed by the rail vehicles.

[0047] The timetable management module 205 can transmit commands for selecting the appropriate rail routes to be used to an automatic route selection module 209 via a third interface S3. Thus, appropriate rail routes can be reserved in a timely manner to ensure the operation of the rail vehicles with minimal delays.

[0048] The timetable management module 205 further has a bidirectional interface S2, S4 with an automatic rail vehicle control module 203. In the event that the online timetable needs to be changed, either due to a new target timetable from the timetable optimization module 211 or due to changes by a train dispatcher, the automatic rail vehicle control module 203 can be informed accordingly to effect a change to the online timetable. For this purpose, the automatic rail vehicle control module 203 also requires current position data of the rail vehicles in order to determine current delays of the rail vehicles and to initiate appropriate control measures in the event of major deviations.

[0049] These regulations can include corresponding control actions to be executed by the rail vehicles, with which the desired departure times and / or arrival times and stopping times of the rail vehicles at stops on the railway lines of the rail transport network are to be achieved or maintained in order to cancel the delay and change the timetable. The corresponding regulations and / or control actions can be transmitted to the timetable management module 205 via a fifth interface S5.

[0050] Finally, in order to adapt the timetable, the corresponding regulations and control actions and the corresponding optimized arrival times / departure times / stopping times are transmitted from the automatic timetable management module 205 via a sixth interface S6 to automatic rail vehicle control modules 213 of the individual rail vehicles 315 of the rail transport so that they can carry out the desired changes or control actions in order to achieve or maintain the optimized arrival times / departure times / stopping times.

[0051] In the case of automatically driving rail vehicles, the automatic rail vehicle control module 213 can determine energy-optimized travel trajectories of the rail vehicle 203 that best match the current online timetable. In the case of manual driving, the driver is advised but is free to implement the regulations.

[0052] In the embodiment shown, the automatic rail vehicle control module 203 is configured to execute the method according to the invention for controlling rail traffic of a plurality of rail vehicles 215 based on the status data of a rail traffic of a plurality of rail vehicles to be optimized.

[0053] FIG 2 shows a flowchart of a method 100 for controlling rail traffic according to an embodiment.

[0054] The method 100 according to the invention for controlling rail traffic of a plurality of rail vehicles 215 can be implemented by a system 200 according to the embodiment in Figure 1 and in particular by an automatic rail vehicle control module 203.

[0055] To control rail traffic, according to the method 100 according to the invention, status data of the rail traffic of the plurality of rail vehicles 215 are first received in a method step 101. The status data describe states of the rail traffic and include at least position data of the plurality of rail vehicles, according to which positioning of the rail vehicles within the rail network is possible. The status data can also include control criteria of the rail traffic, for example, energy consumption of the rail traffic or of the individual rail vehicles or delays of the rail vehicles both as a whole and individually, as well as other relevant criteria for controlling rail traffic.

[0056] The method can be executed during rail traffic operations, in which the rail vehicles are moved within the transport network according to a timetable. The status data describes the status of the rail traffic and describes at least the positions within the transport network where vehicles are located at a specific time. The status data can also include time information, for example, delay times of individual rail vehicles relative to an optimized timetable.

[0057] In a further method step 103, a control target KPI 1 , KPI 2 , KPI N is received. The control target can be given, for example, by an energy consumption of the entire rail vehicles or an overall delay of the entire rail vehicles or by individual delays of the individual rail vehicles.

[0058] The control target KPI 1 , KPI 2 , KPI N can be received during rail traffic operation and thus during the movement of the rail vehicles within the transport network. This enables a change to a control target KPI 1 , KPI 2 , KPI N during online rail traffic operation. The control target specifies a criterion according to which the rail traffic can be controlled. For example, rail traffic can be controlled in such a way that the total energy consumption of the rail vehicles is minimal. Alternatively, rail traffic can be controlled to avoid or minimize delays to the timetable.

[0059] In a method step 105, at least one action selection rule π 1 , π 2 , π N is subsequently selected from a plurality of pre-stored action selection rules π 1 , π 2 , π N. The action selection rules π 1 , π 2 , π N are set up to determine control actions based on status data of the rail traffic which, when executed by the rail vehicles of the rail traffic, lead to an optimization of the rail traffic with regard to a predetermined control objective KPI 1 , KPI 2 , KPI N. The individual action selection rules π 1 , π 2 , π N are optimized with regard to an individual control objective KPI 1 , KPI 2 , KPI N. For example, an action selection rule π 1 , π 2 , π N is optimized for an optimization with respect to a total energy consumption of the rail traffic, while another action selection rule is optimized with respect to a minimization of a total delay of the rail vehicles of the rail traffic.An optimization of an action selection rule π 1 , π 2 , π N with respect to a control target KPI 1 , KPI 2 , KPI N is to be understood in such a way that the respective action selection rule π 1 , π 2 , π N is set up to determine corresponding control actions which, when executed by the rail vehicles, lead to the optimization of rail traffic with respect to the respective control target KPI 1 , KPI 2 , KPI N. In method step 105, the action selection rule π 1 , π 2 , π N which is optimized with respect to the received control target KPI 1 , KPI 2 , KPI N is accordingly selected from the plurality of pre-stored action selection rules π 1 , π 2 , π N.

[0060] The control actions determined by the action selection rules π 1 , π 2 , π N based on the status data of the rail traffic can, in addition to or alternatively to actual control commands to the rail vehicles, include time specifications in the form of arrival, departure and / or idle times of the rail vehicles at stops of the transport network.An action selection rule π 1 , π 2 , π N , which is optimized for energy consumption of the majority of rail vehicles, for example, is thus set up to determine arrival, departure and / or standstill times of the rail vehicles for stops to be controlled in the future based on the status data of the rail traffic, in which at least the positions of the rail vehicles are described at a specific point in time, arrival, departure and / or standstill times are designed in such a way that when the rail vehicles are controlled in such a way that compliance with the arrival, departure and / or standstill times by the rail vehicles is achieved, energy consumption of the rail vehicles is minimized.

[0061] The correspondingly determined arrival, departure, and / or idle times can vary for different control objectives. For example, to reduce energy consumption, it may be advantageous to have a vehicle stay at a stop for a longer period in order to synchronize the vehicle's departure with the acceleration and / or braking of other vehicles. However, longer idle times may not be conducive to reducing delays of rail vehicles, as they would actually generate further delays. The correspondingly determined arrival, departure, and / or idle times also depend on the status data of the rail traffic and, in particular, on the positioning of the individual vehicles.

[0062] In a method step 107, the at least one selected action selection rule π 1 , π 2 , π N is applied to the received rail traffic status data, and corresponding control actions are determined. The determined control actions, for example, the corresponding arrival, departure, and / or idle times, are configured such that, with appropriate control of the vehicles, the rail traffic can be placed in a status that is optimized with respect to the respective control objective or at least improved compared to earlier statuses. For example, in a status generated in this way, a delay to the predetermined timetable can be reduced or energy consumption can be lowered.

[0063] In a method step 109, the control actions according to the embodiment are Figure 1to the various modules of system 200, so that control of rail traffic is possible. The control here comprises the rail vehicles being controlled in such a way that the arrival, departure and / or idle times determined in the method step are adhered to by the vehicles. The corresponding arrival, departure and / or idle times can take into account, in addition to a stop immediately ahead for a respective vehicle on a route of the transport network, a plurality of stops that can be reached by the vehicle on the route within a predetermined period of time when controlled accordingly. The corresponding control actions can thus comprise a set of arrival, departure and / or idle times for a plurality of stops for a rail vehicle.

[0064] According to one embodiment, a combined control target KPI is received in method step 103. The combined control target KPI comprises a plurality of predetermined control targets KPI 1 , KPI 2 , KPI N , each of which is included in the combined control target KPI with individual weightings W 1 , W 2 , WN. A combined control target KPI can thus, for example, comprise a simultaneous optimization with regard to energy consumption and with regard to a minimization of an overall delay of rail traffic, wherein the optimization with regard to energy consumption and the optimization with regard to the overall delay can be included in the combined control target KPI in any ratio, for example 60 to 40. Control can thus be carried out in such a way that, for example, a 60% reduction in energy consumption and a 40% reduction in delay are achieved.The control objectives described and the ratios and percentages given are merely exemplary and do not represent a limitation of the invention.

[0065] In method step 105, a plurality of action selection rules π 1 , π 2 , π N of the prestored action selection rules π 1 , π 2 , π N are subsequently selected. The selected action selection rules π 1 , π 2 , π N are optimized for the corresponding control objectives KPI 1 , KPI 2 , KPI N of the combined control objective KPI.

[0066] In method step 107, the action selection rules π 1 , π 2 , π N of the selected plurality are subsequently executed according to an execution ratio. The execution ratio describes, for each of the selected action selection rules π 1 , π 2 , π N, a temporal proportion of a total execution duration of the plurality of executed action selection rules π 1 , π 2 , π N and enables the combined control objective KPI to be met by executing the selected action selection rules π 1 , π 2 , π N according to the corresponding execution ratio. To control rail traffic, the rail vehicles are controlled according to the control actions determined by the selected action selection rules π 1 , π 2 , π N in the respective execution ratio.The execution ratio can correspond to the weighting of the individual control objectives KPI 1 , KPI 2 , KPI N of the combined control objective KPI, so that, for example, according to the example described above, rail traffic is controlled for 60% of a control period according to an optimization with regard to energy consumption and for 40% of the execution period according to an optimization with regard to an overall delay. The rail vehicles are controlled according to the control actions determined by the action selection rules π 1 , π 2 , π N. However, the execution ratio can also deviate from the respective weighting W 1 , W 2 , WN of the individual control objectives KPI 1 , KPI 2 , KPI N of the combined control objective KPI.

[0067] According to one embodiment, the execution ratio can describe an execution duration t 1 , t 2 , t N for each of the individual action selection rules π 1 , π 2 , π N and / or determine a temporal sequence of the plurality of action selection rules π 1 , π 2 , π N. Thus, following the example described above, the rail traffic can be optimized with regard to energy consumption for a correspondingly determined execution duration t 1 , t 2 and optimized with regard to an overall delay for a corresponding execution duration t 1 , t 2 . The execution durations t 1 , t 2 can be divided into corresponding time windows so that, for example, the rail traffic can be optimized alternately with regard to one control objective and subsequently with regard to the other.

[0068] The control of the rail traffic according to the execution ratio and the correspondingly determined execution durations t 1 , t 2 , t N for the various action selection rules π 1 , π 2 , π N means that the rail vehicles are controlled for one execution duration in such a way that they reach or maintain arrival, departure and / or standstill times that are suitable for achieving a first control objective, and are controlled for a further execution duration in order to reach or maintain other arrival, departure and / or standstill times that are set up to achieve a further control objective of the combined control objective.

[0069] According to a further embodiment, the execution ratio can determine an execution probability P for each selected action selection rule π 1 , π 2 , π N, according to which execution probability P the action selection rule π 1 , π 2 , π N is to be executed in an execution period. According to the execution ratio, it is thus stochastically determined at which probability P each action selection rule π 1 , π 2 , π N is to be executed in a corresponding execution period. This means that the rail traffic can be optimized according to the corresponding execution probability P during an execution period with regard to energy consumption or the overall delay. An execution time can, for example, be given by a cycle time in a cyclical control of the rail traffic, so that for each cycle time of a cycle of the rail traffic control to be executed, the rail traffic is optimized with regard tothe respective control objectives KPI 1 , KPI 2 , KPI N can be optimized.

[0070] The selection in method step 105 and / or the execution in method step 107 of the plurality of action selection rules π 1 , π 2 , π N can be carried out by a time-division function 300. The time-division function 300 can be trained using machine learning techniques to determine appropriate execution ratios while taking into account combined control objectives (KPIs). The time-division function 300 can be trained, for example, using reinforcement learning. In particular, the time-division function 300 can be implemented by a correspondingly trained neural network 303.

[0071] Accordingly, the action selection rules π 1 , π 2 , π N are trained through reinforcement learning to optimize with respect to the respective control objectives KPI 1 , KPI 2 , KPI N. The action selection rules π 1 , π 2 , π N can also be implemented as appropriately trained neural networks.

[0072] The training of the time allocation function 300 or the action selection rules π 1 , π 2 , π N can be carried out based on simulation data from a rail transport simulation. The simulation data can be provided by a corresponding simulation, for example, by the simulation program Falko.

[0073] The training of the action selection rules π 1 , π 2 , π N comprises maximizing a reward function and / or the training of the time allocation function 300 can comprise maximizing a reward function which accordingly comprises the respective control objectives KPI 1 , KPI 2 , KPI N or combined control objectives KPI.

[0074] The training of the action selection rules π 1 , π 2 , π N can be carried out individually, so that each action selection rule π 1 , π 2 , π N is trained on an individual control objective KPI 1 , KPI 2 , KPI N. Based on the pre-trained action selection rules π 1 , π 2 , π N, the time allocation function 300 can then be trained on various combined control objectives KPI, each of which includes the control objectives KPI 1 , KPI 2 , KPI N of the individual action selection rules π 1 , π 2 , π N, taking into account different weightings W 1 , W 2 , WN, to determine corresponding execution ratios.

[0075] Alternatively, a simultaneous training of the action selection rules π 1 , π 2 , π N and the time division function 300 can be performed. During the simultaneous training of the action selection rules π 1 , π 2 , π N and the time division function 300, the pre-trained action selection rules π 1 , π 2 , π N optimized for the individual control objectives KPI 1 , KPI 2 , KPI N can be trained to fulfill the combined control objective KPI, while the time division function 300 is trained to determine a corresponding execution ratio.

[0076] The method according to the invention is applicable to a plurality of arbitrary control objectives KPI 1 , KPI 2 , KPI N and corresponding combined control objectives KPI and to a correspondingly arbitrary number of appropriately trained action selection rules π 1 , π 2 , π N.

[0077] FIG 3shows a diagram of an optimization of a rail transport with respect to two control objectives (KPI 1 , KPI 2 , KPI N ).

[0078] Figure 3 shows a diagram of a Pareto front of a rail transport that is controlled or optimized with respect to two control objectives KPI 1 and KPI 2. The diagram in Figure 3shows a percentage optimization O 1 with respect to the first control objective KPI 1 and a percentage optimization O 2 with respect to the second control objective KPI 2 . Several optimized states Opt, Opt1, Opt2 are shown. The points of the Pareto front shown show optimizations of rail traffic with respect to the two optimization objectives KPI 1, KPI 2 for different weightings W 2 of the second control objective KPI 2 . The point Opt1 shows an optimized state of rail traffic for a weighting W 2 equal to 0 of the second control objective KPI 2 , so that rail traffic in the optimization state Opt1 shown is 100% optimized with respect to the first control objective KPI 1. The optimized state Opt2, on the other hand, shows an optimization of the rail traffic for a weighting W 2 equal to 100 of the second control objective KPI 2 , so that in this optimized state the rail traffic is exclusively optimized with regard toof the second control objective KPI 2 is optimized. The further optimized states show different ratios of the two control objectives KPI 1 , KPI 2 , which are summarized in a combined control objective KPI according to the respective weightings W 1 , W 2 .

[0079] The diagram shown shows an optimization of rail traffic with respect to a reference state Ref, so that the percentage optimizations O 1 , O 2 represent improvements in rail traffic with respect to the first and second control objectives KPI 1 , KPI 2 with respect to the reference state Ref.

[0080] The numerical values ​​shown are merely examples and do not represent an actual optimization of a rail service. The optimization process of the rail service shown can, for example, describe a situation in which the rail service has deviated from a predetermined optimized timetable due to the operation of the rail vehicles, and thus by carrying out the method according to the invention and by correspondingly optimizing the rail service with regard to the respective control objectives KPI 1, KPI 2, KPI N an improvement or return of the rail service to the predetermined optimized timetable taking into account the respective control objectives is to be brought about.

[0081] FIG 4 shows a schematic representation of a temporal execution of two action selection rules (π 1 , π 2 ) according to an execution ratio.

[0082] Figure 4shows a temporal progression of an execution of two action selection rules π 1 , π 2 , over an execution period, which in the embodiment shown is given by a cycle time tc. In the embodiment shown, the first action selection rule π 1 is executed for a first execution period t 1 , while the second action selection rule π 2 is subsequently executed for a second execution period t 2 . In the embodiment shown, the execution period of the cycle time tc is each executed in individual time periods ts. The action selection rules π 1 , π 2 are each executed in the time periods ts and thus cause transitions of the rail traffic into different states S t , S t+1 , ..., S t+9 , according to the respective control actions. The representation in Figure 4 shows an example of an execution relationship in which each action selection rule π 1 , π 2 is assigned a corresponding execution duration t 1 , t 2 .

[0083] Alternatively, corresponding execution probabilities P can be assigned to the shown first and second action selection rules π 1 , π 2 , so that the action selection rules π 1 , π 2 are executed in the individual time periods ts according to the respectively assigned execution probabilities P.

[0084] FIG 5 shows a schematic representation of an architecture for a time-sharing function 300 according to one embodiment.

[0085] Figure 5shows an architecture of a time-sharing function 300 for executing and controlling a rail traffic. In the execution function shown, the time-sharing function 300 for controlling the rail traffic is applied to a combined control objective KPI, which comprises control objectives KPI 1 , KPI 2 , KPI N provided with the corresponding weightings W 1 , W 2 , WN. The time-sharing function 300 is configured to determine an execution ratio according to the Figure 4 shown embodiment, which determines a corresponding execution time for each action selection rule π 1 , π 2 , π N .

[0086] In the embodiment shown, the time division function 300 comprises a time division module 301 and a selection module 302. The time division module 301 is configured to determine corresponding execution ratios for combined control objectives KPI. The selection module 302 is further configured to select the respective action selection rules π 1 , π 2 , π N based on the determined execution ratios. In the embodiment shown, the time division module 301 is embodied as a neural network 303. In the embodiment shown, the selection module 302 further comprises an Argmax function configured to select the respective action selection rules π 1 , π 2 , π N.

[0087] In addition to the control objectives KPI 1 , KPI 2 , KPI N provided with the weights W 1 , W 2 , WN , a modulation operator mod is also included as an activation function in the neural network 303 of the time division module 301. The modulation operator mod allows for the division of the cycle time tc as the execution period into the Figure 4 Deviating from the embodiment shown, the modulation operator mod can also be arranged as an activation function in deeper layers of the neural network. Overall, the time periods ts shown in the Figure 5 , 6 The structures of the neural networks shown are merely examples and do not describe the structures of actually executable neural networks.

[0088] In the architecture shown, the time division function 300 is trained to determine a corresponding execution ratio for any combined control objectives KPI, according to which action selection rules π 1 , π 2 , π N optimized for the individual control objectives KPI 1 , KPI 2 , KPI N are to be executed.

[0089] To execute the time-division function 300 and the action selection rules π 1 , π 2 , π N for controlling rail traffic according to the inventive method 100, status data of the rail vehicles is received during the operation of the rail traffic and during the movement of the plurality of rail vehicles. The status data can indicate, for example, that a current status of the rail traffic deviates from a predetermined timetable.

[0090] Rail traffic control can thus aim to bring rail traffic back into line with the optimized timetable and reduce delays accordingly. This control can be aimed at various control objectives (KPI 1, KPI 2, KPI N). For example, rail traffic control can be aimed at being as energy-efficient as possible. Alternatively, the goal can be to reduce delays as quickly as possible.

[0091] In the embodiment shown, the control of rail traffic is sought taking into account a combined control objective KPI, which includes a plurality of control objectives KPI 1 , KPI 2 , KPI N with a specific weighting. For example, the control of rail traffic can be sought such that the restoration of the optimal timetable takes place while taking into account minimized energy consumption and the shortest possible time until compliance with the optimized timetable is achieved.

[0092] To control rail traffic, a corresponding combined control target KPI is received, which includes the respective desired control targets KPI 1 , KPI 2 , KPI N with corresponding weightings, whereby the weightings W 1 , W 2 , WN for each KPI 1 , KPI 2 , KPI N describe a prioritization within the control of rail traffic.

[0093] For control purposes, the time division function 300 is applied to the received combined control target KPI, and a corresponding execution ratio is determined. The execution ratio specifies a ratio according to which the action selection rules π 1 , π 2 , π N associated with or optimized for the respective control targets KPI 1 , KPI 2 , KPI N are to be executed, and according to which the rail traffic is to be controlled with respect to the respective control actions of the action selection rules π 1 , π 2 , π N.

[0094] According to the determined execution ratio, the selection module 302 selects the corresponding action selection rules π 1 , π 2 , π N and applies the selected action selection rules π 1 , π 2 , π N to the received rail traffic status data. Applying the action selection rules π 1 , π 2 , π N to the status data includes, as described above, determining corresponding control actions and, if applicable, corresponding arrival, departure, and idle times that are optimized for the respective control objectives. Each action selection rule π 1 , π 2 , π N and the control actions that can be determined accordingly are optimized for one of the control objectives.

[0095] By executing an action selection rule π 1 , π 2 , π N on the state data of the rail traffic, the action selection rule π 1 , π 2 , π N is able to determine control actions which, when executed by the rail vehicles, are designed to transfer the rail traffic from a current state to a state which is optimized with regard to the respective control objective or at least improved compared to the previous state.

[0096] In the embodiment shown, the execution ratio describes a temporal ratio that assigns an execution duration to each participating action selection rule π 1 , π 2 , π N , which duration specifies the time period during which the respective action selection rule π 1 , π 2 , π N is to be executed. The temporal ratio is set up such that the respective combined control objective KPI is achieved.

[0097] The control of the rail traffic can be carried out cyclically, so that according to predetermined control cycles, state data and control targets are received, the time division function 300 is applied to the control targets and corresponding execution ratios are determined, the corresponding action selection rules π 1 , π 2 , π N are executed on the received state data according to the respective execution ratios, corresponding control actions are determined and the rail vehicles are controlled according to the control actions and the rail traffic is thus transferred to a state that is optimized or at least improved with regard to the respective control target.

[0098] The depicted representation of the time-division function 300 is merely an example. The depicted neural network 301 does not represent an actual neural network.

[0099] FIG 6shows a further schematic representation of a training architecture for a time-splitting function 300 according to another embodiment.

[0100] Figure 6 shows a further embodiment of an architecture for a time-sharing function 300. In the embodiment shown, the time-sharing function 300 is trained to determine an execution ratio that determines corresponding execution probabilities P for each action selection rule π 1 , π 2 , π N . In contrast to the embodiment in Figure 5 the selection module 300 comprises a softmax function 305 which is configured to select the corresponding action selection rules π 1 , π 2 , π N based on the execution probabilities P determined by the time division module 301.

[0101] The execution of the architecture is analogous to the execution of the architecture in Fig. 6 Instead of the execution period described there, the execution period in Fig. 7 However, in the architecture shown, the various action selection rules are executed for predetermined time periods according to the execution probabilities P determined by the time division function 300.

[0102] FIG 7 shows a diagram of a hypervolume HV of an optimization state of a rail transport.

[0103] Figure 7 shows a graphical representation of a hypervolume HV for various optimized states of a rail traffic system to be optimized with respect to two control objectives KPI 1 , KPI 2 . The hypervolume can serve as an objective parameter for determining the quality of a training process or the quality of the optimization capabilities of the individual action selection rules π 1 , π 2 , π N or the time allocation function 300.

[0104] FIG 8shows a schematic representation of a computer program product 40, comprising instructions which, when the program is executed by a computing unit, cause the computing unit to carry out the method 100 according to one of the above-mentioned embodiments.

[0105] In the embodiment shown, the computer program product 400 is stored on a storage medium 401. The storage medium 401 can be any storage medium known from the prior art.

[0106] Although the invention has been illustrated and described in detail by the preferred embodiment, the invention is not limited to the disclosed examples and other variations may be derived therefrom by those skilled in the art without departing from the scope of the invention.

Claims

1. Method (100) for controlling rail traffic of a plurality of railway vehicles (215), comprising: - receiving (101) status data of the rail traffic, wherein the status data describes control statuses of the rail traffic of the plurality of railway vehicles (215) and comprises at least position data of the plurality of railway vehicles; - receiving (103) a control target (KPI1, KPI2, KPIN) for controlling the rail traffic of the plurality of railway vehicles (215); - selecting (105) at least one action selection rule (π1, π2, πN) from a plurality of action selection rules (π1, π2, πN) based on the control target (KPI1, KPI2, KPIN), wherein the action selection rules (π1, π2, πN) are designed to determine control actions of the rail traffic on the basis of status data of the rail traffic, wherein when the control actions are executed by means of the railway vehicles (215), the rail traffic can be transferred into a status which is optimised with respect to a control target (KPI1, KPI2, KPIN), wherein each action selection rule (π1, π2, πN) is designed for the optimisation with respect to an individual control target (KPI1, KPI2, KPIN), and wherein the at least one selected action selection rule (π1, π2, πN) is designed for an optimisation with respect to the received control target (KPI1, KPI2, KPIN); - executing (107) the at least one selected action selection rule (π1, π2, πN) on the received status data and determining control actions; and - providing (109) the control actions for actuating the plurality of railway vehicles (215), wherein the action selection rules (π1, π2, πN) are trained by means of reinforcement learning, the training of the action selection rules (π1, π2, πN) comprises a maximisation of reward functions and the reward functions define control targets (KPI1, KPI2, KPIN) and / or combined control targets (KPI).

2. Method (100) according to claim 1, wherein the received control target (KP11, KP12, KP1N) is a combined control target (KPI) and comprises a plurality of control targets (KPI1, KPI2, KPIN) and wherein the control targets (KPI1, KPI2, KPIN) with individual weightings (W1, W2, WN) go into the combined control target (KPI); comprising: - selecting (105) a plurality of action selection rules (π1, π2, πN), wherein each action selection rule (π1, π2, πN) is designed in each case for optimisation with respect to one of the plurality of control targets (KPI1, KPI2, KPIN) of the combined control target (KPI); and - executing (107) the plurality of selected action selection rules (π1, π2, πN) according to an execution condition, wherein the execution condition for each selected action selection rule (π1, π2, πN) determines a temporal portion of an overall execution duration of the plurality of selected action selection rules (π1, π2, πN), and wherein an execution of the plurality of action selection rules according to the execution condition fulfils the combined control target (KPI).

3. Method (100) according to claim 2, wherein an execution duration (t1, t2) and / or a temporal sequence of the plurality of action selection rules (π1, π2, πN) is determined by the execution condition for each action selection rule (π1, π2, πN).

4. Method (100) according to claim 2, wherein an execution probability (P) is determined by the execution condition for each selected action selection rule (π1, π2, πN), according to which the action selection rule (π1, π2, πN) is to be executed in an execution time period.

5. Method (100) according to one of the preceding claims 2 to 4, wherein the selected action selection rules (π1, π2, πN) are executed cyclically, and wherein the rail traffic is controlled cyclically according to the control actions determined by the action selection rules (π1, π2, πN).

6. Method (100) according to one of the preceding claims, wherein the selection and / or the execution of the action selection rules (π1, π2, πN) is executed by a time division function (300), and wherein the time division function (200) is trained, by means of machine learning, to determine corresponding execution conditions for combined control targets (KPI).

7. Method (100) according to one of the preceding claims, wherein the time division function (200) is trained by means of reinforcement learning.

8. Method (100) according to claim 7, wherein the training of the action selection rules (π1, π2, πN) and / or the time division function (300) is executed on the basis of simulation data, and wherein the simulation data is based on a simulation of rail traffic of a plurality of railway vehicles (215).

9. Method (100) according to claim 7 or 8, wherein the training of the time division function (300) comprises a maximisation of reward functions, and wherein the reward functions define control targets (KP11, KPI2, KPIN) and / or combined control targets (KPI).

10. Method (100) according to claim 7, 8 or 9, wherein the action selection rules (π1, π2, πN) and the time division functions (300) are trained simultaneously.

11. Method (100) according to one of the preceding claims, wherein the action selection rules (π1, π2, πN) and / or the time division function (300) are embodied as neural networks (303).

12. Method (100) according to one of the preceding claims, wherein control targets (KP11, KPI2, KPIN) comprise: minimisation of an energy consumption of the plurality of railway vehicles (215) and / or minimisation of energy consumptions of individual railway vehicles (215) and / or minimisation of a total delay of the plurality of railway vehicles (215) and / or minimisation of a maximum delay of individual railway vehicles (215) and / or minimisation of a square or absolute deviation of the delays with respect to an average or median delay value of the plurality of railway vehicles (215).

13. Method (100) according to one of the preceding claims, wherein the control actions comprise arrival times and / or departure times and / or standstill times of the plurality of railway vehicles (215) at stations within the rail traffic network.

14. Computing unit (201), wherein the computing unit (201) is embodied to execute the method (100) for controlling rail traffic of a plurality of railway vehicles according to one of the preceding claims 1 to 13.

15. Computer program product (400) comprising commands which, on execution of the program by a data processing unit, cause this to execute the method (100) for controlling rail traffic of a plurality of railway vehicles according to one of the preceding claims 1 to 13.

Images