Railway bridge hole arrangement method and system based on reinforcement learning

Abstract

The present application relates to a kind of railway bridge hole arrangement method and system based on reinforcement learning.The artificial manual bridge hole arrangement design has the problems of heavy workload, and low automation level.This method obtains plane topographic map and three-dimensional ground model;Extract the control factor of plane topographic map and three-dimensional ground model;Input line design data;According to line design data, the relationship between line and control factor is calculated;According to line design data, the projection distance of line to ground is calculated, and the bridge arrangement range is determined;The influence factors of hole span arrangement are discretized;Discretized influence factors are used as input conditions, Q-learning algorithm is started, and bridge hole span is calculated.The present application uses Q-learning reinforcement learning algorithm, based on plane topographic map, three-dimensional ground model, combined with line data, bridge component template, automatically calculates pier position, realizes bridge hole span automatic arrangement, and greatly improves the efficiency of bridge arrangement.

Description

Technical Field

[0001] This invention relates to the field of railway bridge design technology, specifically to a method and system for railway bridge hole layout based on reinforcement learning. Background Technology

[0002] Bridge span layout is a crucial aspect of railway bridge design. After selecting a suitable bridge structure, the span arrangement must be determined. Besides meeting design flow and water level requirements, the river channel should generally not be compressed. For riverbanks with flood control, emergency response, and traffic requirements, pedestrian and vehicle passageways must be provided. For meandering rivers, the span layout should allow for flexibility, and necessary guiding works should be incorporated based on the river conditions to ensure bridge safety and safe flood discharge.

[0003] Currently, the design of spans for railway bridges generally relies on the engineering experience of designers, and calculations, verifications, and layouts are carried out manually in accordance with design standards. This results in a large workload and a low level of automation, which seriously affects the optimization of bridge design schemes and the improvement of bridge span design efficiency. Furthermore, it is prone to design errors, which can affect operational safety.

[0004] Therefore, it is necessary to propose new methods to overcome the above-mentioned shortcomings. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for designing bridge spans based on reinforcement learning, in order to solve the problems of large workload and low level of automation in manual bridge span design.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A reinforcement learning-based method for locating holes in railway bridges, the method comprising:

[0008] Obtain planar topographic maps and 3D terrain models;

[0009] Extracting the controlling factors from planar topographic maps and three-dimensional terrain models;

[0010] Input the circuit design data;

[0011] Calculate the relationship between the route and control factors based on the route design data;

[0012] Based on the route design data, calculate the projected distance from the route to the ground and determine the bridge layout range;

[0013] Discretize the influencing factors of the hole span arrangement;

[0014] Using the discretized influencing factors as input conditions, the Q-Learning algorithm is started to calculate the bridge span.

[0015] Furthermore, the control factors include ground elevation, topographic feature coordinates, control boundary coordinates, and control boundary orientation.

[0016] Furthermore, the route design data includes route mileage, horizontal coordinates, and vertical elevation.

[0017] Furthermore, based on the route design data, the relationship between the route and control factors is calculated, including:

[0018] Extract the planar coordinates of the route from the route design data;

[0019] Obtain the coordinates of terrain features and control boundaries from the control factors;

[0020] Based on the plane coordinates of the route, the coordinates of the terrain features, and the coordinates of the control boundary, calculate the coordinates of the intersection points and the oblique angles between the route and the terrain features, which serve as the relationship between the route and the control factors.

[0021] Furthermore, based on the route design data, the projected distance from the route to the ground is calculated to determine the bridge layout range, including:

[0022] Extract the longitudinal profile elevation of the route from the route design data;

[0023] Calculate the projected distance from the line to the ground based on the longitudinal profile elevation of the line;

[0024] The scope of bridge layout is determined based on the projected distance of the railway line to the ground.

[0025] Furthermore, the factors influencing the arrangement of spans include route data, plane coordinates, longitudinal profile elevation, structures that the bridge needs to cross, and the standard span of the bridge.

[0026] Further, the calculation of the bridge span includes:

[0027] Standard beam types and conventional continuous beam spans are selected as the set of selectable actions in the action space;

[0028] Define a reward function to give positive rewards to suitable solutions and negative rewards to unsuitable solutions, forming a reward matrix R;

[0029] Set the initial values ​​and learning efficiency of the Q table;

[0030] Set the current state as the initial state, select an action from all possible actions in the current state, and use the Q function to calculate the new state value:

[0031]

[0032] in:

[0033] s is the state space;

[0034] 'a' represents learning efficiency;

[0035] γ is the discount factor;

[0036] Update the value of the corresponding state in the Q table;

[0037] Assuming the current state is the next state corresponding to the action, update repeatedly;

[0038] Determine if the current state is the final state. If it is true, return to the initial state; otherwise, repeat the training.

[0039] Output the bridge span arrangement.

[0040] On the other hand, a reinforcement learning-based railway bridge span layout system is provided, the system being used to implement the method, including:

[0041] The acquisition module is used to acquire planar topographic maps and 3D terrain models;

[0042] The extraction module is used to extract control factors from planar topographic maps and three-dimensional terrain models;

[0043] The input module is used to input circuit design data;

[0044] The calculation module is used to calculate the relationship between the line and control factors based on the line design data;

[0045] The scope determination module is used to calculate the projected distance from the line to the ground based on the line design data, and to determine the bridge layout scope.

[0046] The discretization module is used to discretize the influencing factors of the hole span arrangement;

[0047] The aperture module is used to take the discretized influencing factors as input conditions to start the Q-Learning algorithm and calculate the bridge span.

[0048] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0049] This invention provides a method and system for railway bridge span layout based on reinforcement learning. It adopts the Q-Learning reinforcement learning algorithm, based on planar topographic maps and three-dimensional ground models, combined with line data and bridge component templates, to automatically calculate the pier and abutment positions, realize the automatic layout of bridge spans, and significantly improve the efficiency of bridge layout. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained from these drawings without creative effort.

[0051] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0052] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0053] It should be noted that the terms “comprising” and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product or device.

[0054] It should also be noted that although the order of steps is mentioned in the method description, in some cases, steps may be performed in a different order than that described here, and this should not be interpreted as a restriction on the order of steps.

[0055] This invention provides a reinforcement learning-based method for railway bridge span layout. It employs the Q-Learning reinforcement learning algorithm to automatically calculate pier locations, enabling automatic span arrangement. The Q-Learning algorithm can find the optimal strategy through trial and error, repeated exploration, and learning, even without prior environmental information. Based on the idea of ​​dynamic programming, it uses a Q-function to evaluate the long-term reward (i.e., state-action value) of each state-action pair and provides the optimal strategy.

[0056] like Figure 1 The method includes:

[0057] S1: Obtain the planar topographic map and the 3D terrain model. The planar topographic map is in dwg format, and the 3D terrain model is in dgn format.

[0058] S2: Extract control factors from the planar topographic map and the 3D terrain model. Control factors include ground elevation, topographic feature coordinates, control boundary coordinates, and control boundary orientation.

[0059] S3: Input route design data. Route design data includes route mileage, horizontal coordinates, and vertical profile elevation.

[0060] S4: Based on the route design data, calculate the relationship between the route and control factors, including:

[0061] S401: Extract the planar coordinates of the route from the route design data;

[0062] S402: Obtain the coordinates of terrain features and control boundaries from the control factors;

[0063] S403: Based on the plane coordinates of the route, the coordinates of the terrain features, and the coordinates of the control boundary, calculate the coordinates of the intersection points and the oblique angles between the route and the terrain features, as the relationship between the route and the control factors. These relationships will be used as external parameters input into the algorithm.

[0064] S5: Based on the route design data, calculate the projected distance from the route to the ground and determine the bridge layout range, including:

[0065] S501: Extract the longitudinal profile elevation of the route from the route design data;

[0066] S502: Calculate the projected distance from the line to the ground based on the longitudinal profile elevation of the line;

[0067] S503: Determine the bridge layout range based on the projected distance from the railway line to the ground. This range controls the initial mileage of the bridge, and the algorithm can automatically fine-tune this range.

[0068] S6: Discretize the influencing factors of the hole span arrangement.

[0069] Factors influencing the arrangement of spans include route data, plane coordinates, longitudinal profile elevation, structures the bridge needs to cross, and the standard span of the bridge.

[0070] S7: Using the discretized influencing factors as input conditions, start the Q-Learning algorithm to calculate the bridge span, including:

[0071] S701: Select standard beam type and conventional continuous beam span as the set of selectable actions in the action space;

[0072] S702: Define a reward function that provides positive rewards for suitable solutions and negative rewards for unsuitable solutions, forming a reward matrix R;

[0073] S703: Set the initial value of the Q table and the learning efficiency;

[0074] S704: Set the current state to the initial state (the initial state is a state without any hole spans), select one action from all possible actions in the current state, and calculate the new state value:

[0075]

[0076] in:

[0077] s is the state space;

[0078] 'a' represents learning efficiency;

[0079] γ is the discount factor;

[0080] S705: Update the value of the corresponding state in the Q table;

[0081] S706: Assume the current state is the next state corresponding to the action, and update repeatedly;

[0082] S707: Determine if the current state is the final state. If it is true, return to the initial state; otherwise, repeat the training.

[0083] S708: Based on the above steps, iteratively solve the problem and output the bridge span arrangement.

[0084] On the other hand, a reinforcement learning-based railway bridge span layout system is provided for implementing the above methods, including:

[0085] The acquisition module is used to acquire planar topographic maps and three-dimensional terrain models, corresponding to S1 of the above method;

[0086] The extraction module is used to extract the control factors of the planar topographic map and the three-dimensional terrain model, corresponding to S2 of the above method;

[0087] The input module is used to input line design data, corresponding to S3 of the above method;

[0088] The calculation module is used to calculate the relationship between the line and control factors based on the line design data, corresponding to S4 of the above method;

[0089] The range determination module is used to calculate the projected distance from the line to the ground based on the line design data, and determine the bridge layout range, corresponding to S5 of the above method;

[0090] The discretization module is used to discretize the influencing factors of the hole span arrangement, corresponding to S6 of the above method;

[0091] The aperture module is used to take the discretized influencing factors as input conditions to start the Q-Learning algorithm and calculate the bridge span, corresponding to S7 of the above method.

[0092] Those skilled in the art will understand that all or part of the functions of the embodiments of the present invention can be implemented by hardware or by computer program. When all or part of the functions in the above embodiments are implemented by computer program, the program can be stored in a computer-readable storage medium, which may include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to achieve the above functions. For example, the program can be stored in the memory of a device, and when the program in the memory is executed by the processor, all or part of the above functions can be achieved. In addition, when all or part of the functions in the above embodiments are implemented by computer program, the program can also be stored in a storage medium such as a server, another computer, disk, optical disk, flash drive, or portable hard drive, and can be downloaded or copied to the memory of a local device, or the system of the local device can be updated. When the program in the memory is executed by the processor, all or part of the functions in the above embodiments can be achieved.

[0093] The above examples illustrate the present invention only to aid in understanding it and are not intended to limit the scope of the invention. Those skilled in the art can make various simple deductions, modifications, or substitutions based on the principles of this invention.

Claims

1. A reinforcement learning-based method for arranging spans in railway bridges, characterized by: The method includes: Obtain planar topographic maps and 3D terrain models; Extracting the controlling factors from planar topographic maps and three-dimensional terrain models; Input the circuit design data; Calculate the relationship between the route and control factors based on the route design data; Based on the route design data, calculate the projected distance from the route to the ground and determine the bridge layout range; Discretize the influencing factors of the hole span arrangement; Using the discretized influencing factors as input conditions, the Q-Learning algorithm is started to calculate the bridge span. in: Based on the route design data, calculate the relationship between the route and control factors, including: Extract the planar coordinates of the route from the route design data; Obtain the coordinates of terrain features and control boundaries from the control factors; Based on the plane coordinates of the route, the coordinates of the terrain features, and the coordinates of the control boundary, calculate the coordinates of the intersection points and the oblique angles between the route and the terrain features, as a representation of the relationship between the route and the control factors; Based on the route design data, calculate the projected distance from the route to the ground and determine the bridge layout range, including: Extract the longitudinal profile elevation of the route from the route design data; Calculate the projected distance from the line to the ground based on the longitudinal profile elevation of the line; The bridge layout area is determined based on the projected distance of the line to the ground. Calculating the span of a bridge includes: Standard beam types and conventional continuous beam spans are selected as the set of selectable actions in the action space; Define a reward function to give positive rewards to suitable solutions and negative rewards to unsuitable solutions, forming a reward matrix R; Set the initial values ​​and learning efficiency of the Q table; Set the current state as the initial state, select an action from all possible actions in the current state, and use the Q function to calculate the new state value: in: s is the state space; 'a' represents learning efficiency; γ is the discount factor; Update the value of the corresponding state in the Q table; Assuming the current state is the next state corresponding to the action, update repeatedly; Determine if the current state is the final state. If it is true, return to the initial state; otherwise, repeat the training. Output the bridge span arrangement.

2. The reinforcement learning-based method for railway bridge span layout according to claim 1, characterized in that: Controlling factors include ground elevation, topographic and feature coordinates, control boundary coordinates, and control boundary orientation.

3. The reinforcement learning-based method for railway bridge span layout according to claim 2, characterized in that: The route design data includes route mileage, horizontal coordinates, and vertical profile elevation.

4. The reinforcement learning-based method for railway bridge span layout according to claim 3, characterized in that: Factors influencing the arrangement of spans include route data, plane coordinates, longitudinal elevation, structures the bridge needs to cross, and the standard span of the bridge.

5. A railway bridge span layout system based on reinforcement learning, characterized in that: The system is used to implement the method according to any one of claims 1-4, comprising: The acquisition module is used to acquire planar topographic maps and 3D terrain models; The extraction module is used to extract control factors from planar topographic maps and three-dimensional terrain models; The input module is used to input circuit design data; The calculation module is used to calculate the relationship between the line and control factors based on the line design data; The scope determination module is used to calculate the projected distance from the line to the ground based on the line design data, and to determine the bridge layout scope. The discretization module is used to discretize the influencing factors of the hole span arrangement; The aperture module is used to take the discretized influencing factors as input conditions to start the Q-Learning algorithm and calculate the bridge span. in: Based on the route design data, calculate the relationship between the route and control factors, including: Extract the planar coordinates of the route from the route design data; Obtain the coordinates of terrain features and control boundaries from the control factors; Based on the plane coordinates of the route, the coordinates of the terrain features, and the coordinates of the control boundary, calculate the coordinates of the intersection points and the oblique angles between the route and the terrain features, as a representation of the relationship between the route and the control factors; Based on the route design data, calculate the projected distance from the route to the ground and determine the bridge layout range, including: Extract the longitudinal profile elevation of the route from the route design data; Calculate the projected distance from the line to the ground based on the longitudinal profile elevation of the line; The bridge layout area is determined based on the projected distance of the line to the ground. Calculating the span of a bridge includes: Standard beam types and conventional continuous beam spans are selected as the set of selectable actions in the action space; Define a reward function to give positive rewards to suitable solutions and negative rewards to unsuitable solutions, forming a reward matrix R; Set the initial values ​​and learning efficiency of the Q table; Set the current state as the initial state, select an action from all possible actions in the current state, and use the Q function to calculate the new state value: in: s is the state space; 'a' represents learning efficiency; γ is the discount factor; Update the value of the corresponding state in the Q table; Assuming the current state is the next state corresponding to the action, update repeatedly; Determine if the current state is the final state. If it is true, return to the initial state; otherwise, repeat the training. Output the bridge span arrangement.

Images