Underground expressway reinforcement learning intelligent line selection method based on GIS and InSAR and computer system

Abstract

The invention discloses an underground expressway reinforcement learning intelligent line selection method based on a GIS and an InSAR, and the method comprises the steps: providing urban environment multi-source data based on the GIS and the InSAR, building a rasterized observation space, building an initial linear continuous line space, and jointly building a line selection space of a reinforcement learning model; the strategy network of the reinforcement learning model receives parameter information of the line selection space and outputs linear parameters; the linetype builder corrects linetype parameters and updates the line selection space; the multi-index evaluation model outputs reward and punishment values according to the parameter information of the new line selection space, a value network of the reinforcement learning model receives the reward and punishment values, calculates a gradient for updating, and transmits a gradient updating strategy model; circulating until a round of line selection is completed, and calculating an accumulated reward value; and when the accumulated average reward value does not reach the optimization target, circulating until the optimization target is reached, and finding the optimal design. The method has the advantage of being beneficial to improving the efficiency and accuracy of underground road design.

Description

Technical Field

[0001] The present invention relates to an application technology of artificial intelligence in road route selection design, and in particular to an intelligent route selection method for underground expressways based on GIS and InSAR. Background Art

[0002] With the continuous integration of regions and the superposition of urban functions in my country, the contradictions of urban population, space, resources, etc. have become increasingly prominent, and the development and utilization of underground space resources has become a trend and necessity. In major cities, as an extension and supplement to the ground road system, underground expressways have developed rapidly. However, the selection of new underground roads requires many indicators to be considered, and quantitative evaluation is difficult. In addition to the linear parameters, obstacle avoidance, and traffic function improvement required for conventional road selection, it is also affected by geological conditions and other factors. Improper construction will cause deformation of the surrounding environment and cracking of houses. Therefore, how to comprehensively consider various factors and optimize the route is an important challenge.

[0003] Traditional intelligent route selection methods for road line design are mostly based on exploring in simplified profile groups or grid spaces based on known line intersection sets. A large number of possible route solutions cannot be detected, which has great limitations. Current reinforcement learning methods are mostly aimed at pathfinding problems in grid spaces, and the generated routes are mostly grid or polyline routes, which do not have the linear elements required by the specifications and cannot be applied to underground expressway route selection.

[0004] The current problem is:

[0005] 1) In the line selection stage of urban underground expressways, considering various influencing factors, the current planning and design methods have problems such as insufficient acquisition and quantitative application of multi-source data. Most of them are based on the surface feature information in GIS or CAD drawings, and use experience or hierarchical analysis to determine parameters such as plane line type, vertical line type, and horizontal and vertical combination. There is a lack of comprehensive consideration of multi-source data such as ground, underground, and settlement in the line selection area, and the quantitative level is low;

[0006] 2) The current heuristic optimization algorithm for road alignment design has great limitations, and the reinforcement learning model based on the pathfinding problem cannot meet the basic requirements of underground road alignment design in both action space design and line selection space design. This is where this application needs to focus on improvement. Summary of the invention

[0007] The technical problem to be solved by the present invention is to provide an intelligent line selection method for underground expressways based on GIS and InSAR, which is beneficial to improving the efficiency and accuracy of underground road design and reducing the interference of human factors.

[0008] In order to solve the above technical problems, the present invention provides an intelligent line selection method for underground expressway based on GIS and InSAR, comprising the following steps:

[0009] Step S1, based on the urban environment multi-source data provided by the Geographic Information System (GIS) and the Interferometric Synthetic Aperture Radar (InSAR), a rasterized observation space is established, and the initial linear continuous line space is established according to the standardized parameterized underground rapid plane line shape, and the two together construct the line selection space of the reinforcement learning model;

[0010] Furthermore, the line selection space of the reinforcement learning model is constructed, including the following steps:

[0011] Step S11, establish a GIS raster data set based on the remote sensing information, geological information, and hydrological information provided by GIS, extract the settlement data of previous years calculated by InSAR, and calculate the average rate, average value, and extreme value of past settlement by statistical methods to form an InSAR settlement raster data set. The two are unified into raster data with the same accuracy to form an observation space, providing state information for the subsequent reinforcement learning model;

[0012] Step S12, parameterizing the three types of basic plane line types required in the road specification, the line types are divided into main line types and secondary line types according to the combination characteristics required by the design specification, wherein a default set of unit line types is composed of a main line type and a secondary line type, and a set of unit line types after parameterization is represented by three parameters: radius, turning angle, and clothoid parameter. Finally, the underground expressway design line composed of multiple sets of line types is represented by a unique sequence formed by the parameters of each set and forms a line space, providing accurate line information of the final optimized line;

[0013] Further, step S12 includes: constructing mathematical equations of straight lines, transition curves, and circular curves by mathematical methods, wherein the straight lines and circular curves are represented as main line types by radius R and rotation angle θ, the transition curves are represented as secondary line types by clothoid parameter A, a set of unit line types is composed of secondary line types spliced ​​with a main line type, and a set of parameters (R i ,θ i ,A i ) is determined, and the line finally composed of multiple groups of unit line types is determined by parameter sequence

[0014] [(R0,θ0,A0),(R1,θ1,A1),(R2,θ2,A2),…,(R n ,θ n ,A n )]express.

[0015] Step S2, the policy network of the reinforcement learning model receives parameter information of the line selection space and outputs line type parameters;

[0016] Furthermore, the reinforcement learning model includes: the reinforcement learning network includes a value network, a target value network, a strategy network and a target strategy network, wherein the strategy network and the target strategy network output line type parameters (R i ,θ i ,A i ).

[0017] Step S3, the line type builder modifies the line type parameters according to the specification requirements and updates the line selection space;

[0018] Furthermore, the line type builder generates a line type parameter (R i ,θ i ,A i ) Update line space;

[0019] The line type builder is constructed based on the line type parameters (R i ,θ i ,A i ) Determine the types of main and secondary line types, calculate and generate the line type parameter equation based on the coordinates and direction of the line end, and modify the line type parameters according to the specification requirements. The specific parameter equation is as follows:

[0020] straight line: Circular curve: Cloak line:

[0021] Where: α represents the direction of the end of the line, (x0, y0) is the position coordinate of the end of the line, s, s' are the parameters of the clothoid, where s = A 2 / R i ,s'=A 2 / R i-1 ;

[0022] The sign of the clothoid depends on the line type parameters (R i ,θ i ,A i ) and (R i-1 ,θ i-1 ,A i-1 )Sure.

[0023] Step S4, the multi-index evaluation model outputs reward and punishment values ​​according to the parameter information of the new line selection space, the value network of the reinforcement learning model receives the reward and punishment value calculation gradient for update, and transmits the gradient to update the strategy model;

[0024] Furthermore, the multi-index evaluation model and reinforcement model training include:

[0025] Step S41, the multi-index evaluation model comprehensively evaluates the currently generated route from multiple factors according to the updated route selection space. The multi-index evaluation model refers to factors including the design parameters of the current route, the ground feature classification, obstacle information, target information of GIS remote sensing information, and the settlement information provided by InSAR; the multi-index evaluation model needs to consider the evaluation aspects including the reachability, obstacle avoidance, economy, and safety of the target; the specific reward and punishment value calculation is as follows:

[0026] Reward=(w1r1+w2r2+w3r3+w4r4+w5r5)×β (4);

[0027] Reward for approaching the target: r = (d i -d i-1 ) / D(5);

[0028] Rewards for course correction:

[0029]

[0030] Penalty for pointing to an obstacle: r3 = sense_r / obstacle ≤20° (7);

[0031] Line length penalty: r4 = (1-(d i -d i-1 ) / l) (8);

[0032] Penalty for subsidence risk: r5 = (0.01n i -m i ) / l (9);

[0033] Where, d i is the distance between the route generated in step i and the target; sense_r is the range of state information acquisition; obstacle ≤20° is the distance to the nearest obstacle within 20° of the line direction; l is the length of the currently generated set of unit lines; n i The total number of blocks that the current set of unit lines passes through; m i The high-risk blocks that the current set of line types passes through; reward_shaping is the correction of the reward value, which increases as the line approaches the end point. When the line reaches the end point, a fixed reward value of 100 is obtained, and a fixed penalty value of -100 is obtained when it hits an obstacle;

[0034] Step S42: The reinforcement learning value network and the target value network calculate the gradient according to the obtained reward value to update the value network, and transfer the gradient to the value policy network to update it at the same time. The target network partially copies the parameters of the policy network and the value network for soft update. The gradient is calculated as follows:

[0035] GradQ=Q+Reward-Q' (10);

[0036] In the formula, GradQ represents the updated gradient of the value network, Q represents the future maximum reward value estimated by the value network, Reward represents the reward and punishment value returned by the evaluation model, and Q' represents the future maximum reward value estimated by the target value network after the update.

[0037] Step S5, looping steps S2-S4 until a round of line selection is completed and the accumulated reward value is calculated;

[0038] Furthermore, completing a round of line selection includes: looping steps S2-S4 until a round of line selection is completed, wherein the conditions for completing a round of line selection are reaching the end point, hitting an obstacle, and exceeding a limited number of cycles; after a round of line selection is completed, counting the total value of the entire reward value.

[0039] Step S6, when the accumulated average reward value does not reach the optimization target, loop steps S1-S5 until the optimization target is reached and the optimal design is found;

[0040] Furthermore, achieving the optimization goal includes: looping steps S1-S5 until the total reward value reaches the optimization goal, and the condition for achieving the optimization goal is that the total reward value continues to rise and reaches a stable state, at which time the converged strategy network generates a route that is the optimal route.

[0041] The beneficial effects of the present invention are:

[0042] 1) Designing the line from the perspective of linear parameters can achieve standardized underground express line design in one step;

[0043] Traditional heuristic algorithms often need to use pathfinding algorithms or traversal algorithms to initially obtain a gridded line selection corridor, and then use heuristic algorithms to optimize the line; however, this application designs lines from the perspective of line type parameters, which can effectively decompose the specifications into each step of line type generation, ensuring the standardization of the line while achieving line generation at one time;

[0044] 2) The line selection space is set as a combination of linear continuous space and raster data space, which can help the model fully explore the environment, realize accurate line design, and explore the global optimal solution;

[0045] Traditional heuristic algorithms are mostly optimized based on the intersection set of an initial line. Since the number of initial intersection sets is fixed and the position can only be moved on the section or grid node, in actual design, the number and distribution of intersection sets are not constrained, which leads to the neglect of a large number of possible line combinations when designing lines based on the distribution of intersection sets. There are huge limitations and it is often impossible to obtain the global optimal solution. However, the lines of this application are drawn in a continuous space, which can ensure the accurate design of the lines, and the lines can be freely drawn within the specification range.

[0046] 3) The evaluation model combines GIS, InSAR and line parameter information to consider multiple requirements including economy and safety, making it more comprehensive;

[0047] 4) Traditional path-finding problems often only consider factors such as target arrival, obstacle avoidance, and earth excavation. The combined data is often limited to elevation maps, and the factors considered are relatively narrow. However, this application considers the remote sensing information, geological information, and hydrological information provided by GIS to establish a GIS raster data set, and the historical settlement data calculated by InSAR, as well as line design information. Data can be obtained from the line selection space to achieve rapid evaluation and achieve optimization goals in all aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The drawings constituting a part of the present application are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0049] Figure 1 is a flow chart of a specific embodiment of the present invention;

[0050] Figure 2 is a reinforcement learning flow chart of a specific embodiment of the present invention;

[0051] Figure 3 It is a schematic diagram of the process of constructing a line type by a standardized line type builder according to an embodiment of the present invention;

[0052] Figure 4 It is a schematic diagram of a process of multi-factor evaluation for underground road line selection according to an embodiment of the present invention;

[0053] Figure 5 It is a schematic diagram of a final generated line according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0055] like Figure 1 As shown, the present invention provides an intelligent line selection method for underground expressway based on GIS and InSAR, comprising the following steps:

[0056] Step S1, establishing a rasterized observation space based on multi-source data of the urban environment provided by the Geographic Information System (GIS) and the Interferometric Synthetic Aperture Radar (InSAR), and establishing a linear continuous line space based on the standardized parameterized underground rapid plane line shape, the two together constitute the line selection space of the reinforcement learning model;

[0057] Specifically, S1 includes the following steps S11 to S12:

[0058] Step S11, obtain relevant data in the line selection area through GIS, including hydrological information, surface remote sensing images, elevation, slope, slope aspect, and land feature classification data, and calculate the settlement rate, settlement extreme value, and mean settlement data in the area based on InSAR statistics. Align the above data to grids of the same size to complete the establishment of the observation space, and provide state information for the subsequent reinforcement learning model;

[0059] Step S12: The line parameter space is a linear continuous space. A set of initial line endpoints, directions and line type parameters are required. The endpoint positions can be determined according to the coordinates of the starting point in the observation space. The initial line type parameters are in the form of (R0, θ0, A0), which are set according to the line type before the starting point of the line. The line space is then formed according to the line formation sequence generated by the reinforcement learning model, which is in the form of [(R0, θ0, A0), (R1, θ1, A1), (R2, θ2, A2), ..., (R n ,θ n ,A n )], providing accurate route information for the final optimized route.

[0060] Step S2, the strategy network of the strong chemical model receives parameter information of the line selection space and outputs line type parameters;

[0061] See also Figure 2 The reinforcement learning model includes a value network, a target value network, a policy network and a target policy network. The target network has the same network structure as the corresponding network. The value network receives the state information and line type parameters (R i ,θ i ,A i ) predicts the maximum reward value to be obtained in the future, and the policy network receives the state information and outputs the linear parameter (R i ,θ i ,A i );

[0062] Specifically, depending on the different state information, the value network and the strategy network use a multi-layer linear connection network or a convolutional network, and the output value is kept within the required range through the activation function. In addition, in order to allow the model to explore more linear types, an exploration rate that conforms to the normal distribution is initially added to the output linear type parameter (R i ,θ i ,A i ), the exploration rate decreases slowly as the model training strategy gradually stabilizes to prevent interference with the convergence of the model.

[0063] S3, the line type builder modifies the line type parameters according to the specification requirements and updates the line selection space;

[0064] See also Figure 3 Specifically, the normalized line type builder divides line types into primary line types and secondary line types, where primary line types include straight lines and circular curves, and secondary line types are transition curves or straight lines. The normalized line type builder reads line type parameters (R i ,θ i ,A i ), according to the radius R i The size of determines whether the main line type is a straight line or a circular curve. The threshold is set to 10000m according to the specification, that is, lines exceeding 10000m are set as straight lines and drawn according to formula (1), and the rest are circular curves and drawn according to formula (3). Then, combined with the previous line type parameters (R i-1 ,θ i-1 ,A i-1 ) determines the type of transition curve. The transition curve can be a clothoid curve, and in some cases a straight line can also be used. Here, the type is determined according to the length of different transition curves. When the clothoid curve is shorter, the line is drawn according to formula (3). The specification correction is to correct the size of the generated line according to the specification requirements. When the radius of the generated line is less than the minimum available radius of 200m, or the clothoid curve parameter A does not meet the specification requirements, the builder corrects the input line type parameters according to the specification to ensure that the line type combination is consistent with the specification. Finally, the line selection space is updated according to the corrected line type parameters.

[0065] S4, the multi-index evaluation model outputs reward and punishment values ​​according to the parameter information of the new line selection space, the value network of the reinforcement learning model receives the reward and punishment value calculation gradient for update, and transmits the gradient to update the strategy model;

[0066] See also Figure 4The line selection space is composed of linear continuous space and grid space. The evaluation model obtains the design parameters of the line from the linear continuous space, and projects the generated line type into the grid space for unified evaluation through the formulas (1)-(3) provided in the line type builder. The data group in the grid space should include the classification of land objects, obstacle classification, target information, etc. The multi-factor comprehensive evaluation can calculate the reward and punishment values ​​based on formulas (4)-(9) combined with the grid data.

[0067] Specifically, the reinforcement learning model training includes: obtaining the future maximum reward value predicted by the value network through four networks respectively, and then calculating the gradient according to formula (10), and then the value model is updated by back propagation based on the gradient, and the policy network is updated based on the gradient of the transmitted linear parameters. In order to ensure the stability of the model during training, the soft update method is adopted when updating the target network, and the corresponding network parameters are copied at a lower rate for update. In addition, an experience pool is constructed in the model, and the state information, linear parameters, and reward and punishment values ​​in the same cycle are stored as a group to improve the training efficiency of the data.

[0068] S5, loop S2-S4 until a round of line selection is completed and the accumulated reward value is calculated;

[0069] Specifically, the completion of a round of line selection can be judged by a multi-factor evaluation model. The evaluation model needs to determine whether the line currently intersects with environmental obstacles, whether it has reached the end point, whether it has exceeded the number of cycles, and other basic information. When the conditions for completing a round of line selection are met, the current cycle ends, and the environment is reset for the next round of line selection. If the end condition is not met, the cycle continues until the end condition is met.

[0070] S6, when the accumulated average reward value does not reach the optimization target, loop S1-S5 until the optimization target is reached and the optimal design is found.

[0071] See also Figure 5 Specifically, the model can calculate the total reward value of each round of training after completing it. When the model is training, the total reward value of each round will gradually increase and eventually reach a stable convergence value. At this point, the optimization of the target is completed and the final optimization route is generated.

[0072] Through continuous iteration and optimization, the reinforcement learning model can gradually learn the optimal line design solution.

[0073] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A reinforcement learning intelligent line selection method for underground expressways based on GIS and InSAR, comprising the following steps: Step S1, based on the urban environment multi-source data provided by the Geographic Information System (GIS) and the Interferometric Synthetic Aperture Radar (InSAR), a rasterized observation space is established, and the initial linear continuous line space is established according to the standardized parameterized underground rapid plane line shape, and the two together construct the line selection space of the reinforcement learning model; Step S2, the policy network of the reinforcement learning model receives parameter information of the line selection space and outputs line type parameters; Step S3, the line type builder modifies the line type parameters according to the specification requirements and updates the line selection space; Step S4, the multi-index evaluation model outputs reward and punishment values ​​according to the parameter information of the new line selection space, the value network of the reinforcement learning model receives the reward and punishment value calculation gradient for update, and transmits the gradient to update the strategy model; Step S5, looping steps S2-S4 until a round of line selection is completed and the accumulated reward value is calculated; Step S6: When the accumulated average reward value does not reach the optimization target, steps S1-S5 are looped until the optimization target is reached and the optimal design is found.

2. The method for intelligent line selection of underground expressways based on GIS and InSAR according to claim 1, characterized in that: The step S1 comprises the following steps: Step S11, establish a GIS raster data set based on the remote sensing information, geological information, and hydrological information provided by GIS, extract the settlement data of previous years calculated by InSAR, and calculate the average rate, average value, and extreme value of past settlement by statistical methods to form an InSAR settlement raster data set. The two are unified into raster data with the same accuracy to form an observation space, providing state information for the subsequent reinforcement learning model; Step S12, parameterize the three types of basic plane line types required by the road specifications, and divide the line types into main line types and secondary line types according to the combination characteristics required by the design specifications, wherein a default set of unit line types is composed of a main line type and a secondary line type, and a set of unit line types after parameters are represented by three parameters: radius, turning angle, and spiral line parameters. Finally, the underground expressway design line composed of multiple sets of line types is represented by a unique sequence formed by the parameters of each group and forms a line space, providing accurate line information of the final optimized line.

3. The method for intelligent line selection of underground expressways based on GIS and InSAR according to claim 2 is characterized in that: The step S12 comprises: Mathematical equations of straight lines, transition curves, and circular curves are constructed by mathematical methods. Straight lines and circular curves are represented by radius R and rotation angle θ as the main line types, and transition curves are represented by clothoid parameters A as the secondary line types. A set of unit line types is composed of secondary line types spliced ​​with a main line type, which is composed of a set of parameters (R i ,θ i , A i ) is determined, and the line finally composed of multiple groups of unit line types is determined by the parameter sequence [(R0, θ0, A0), (R1, θ1, A1), (R2, θ2, A2), …, (R n ,θ n , A n )]express.

4. The method for intelligent line selection of underground expressways based on GIS and InSAR according to claim 1, characterized in that: The step S2 comprises: The reinforcement learning network includes a value network, a target value network, a policy network, and a target policy network. The policy network and the target policy network output line type parameters (R i ,θ i , A i ).

5. The method for intelligent line selection of underground expressways based on GIS and InSAR according to claim 1, characterized in that: The step S3 comprises: the line type builder generates a line type parameter (R i ,θ i ,A i ) Update line space; The line type builder is constructed based on the line type parameters (R i ,θ i ,A i ) Determine the types of main and secondary line types, calculate and generate the line type parameter equation based on the coordinates and direction of the line end, and modify the line type parameters according to the specification requirements. The specific parameter equation is as follows: straight line: Circular curve: Cloak line: Where: α represents the direction of the end of the line, (x0, y0) is the position coordinate of the end of the line, s, s' are the parameters of the clothoid, where s = A 2 / R i ,s'=A 2 / R i-1 ; The sign of the clothoid depends on the line type parameters (R i ,θ i ,A i ) and (R i-1 ,θ i-1 ,A i-1 )Sure.

6. The method for intelligent line selection of underground expressways based on GIS and InSAR according to claim 1, characterized in that: The step S4 comprises: Step S41, the multi-index evaluation model comprehensively evaluates the currently generated route from multiple factors according to the updated route selection space. The multi-index evaluation model refers to factors including the design parameters of the current route, the ground feature classification, obstacle information, target information of GIS remote sensing information, and the settlement information provided by InSAR; the multi-index evaluation model needs to consider the evaluation aspects including the reachability, obstacle avoidance, economy, and safety of the target; the specific reward and punishment value calculation is as follows: Reward=(w1r1+w2r2+w3r3+w4r4+w5r5)×β (4); Reward for approaching the target: r = (d i -d i-1 ) / D (5); Rewards for course correction: Penalty for pointing to an obstacle: r3 = sense_r / obstacle ≤20° (7); Line length penalty: r4 = (1-(d i -d i-1 ) / l) (8); Penalty for subsidence risk: r5 = (0.01n i -m i ) / l (9); Where, d i is the distance between the route generated in step i and the target; sense_r is the acquisition range of the status information; obstacle ≤20° is the distance to the nearest obstacle within 20° of the line direction; l is the length of the currently generated set of unit lines; n i The total number of blocks that the current set of unit lines passes through; m i The high-risk blocks that the current set of line types passes through; reward_shaping is the correction of the reward value, which increases as the line approaches the end point. When the line reaches the end point, a fixed reward value of 100 is obtained, and when it hits an obstacle, a fixed penalty value of -100 is obtained; Step S42: The reinforcement learning value network and the target value network calculate the gradient according to the obtained reward value to update the value network, and transfer the gradient to the value policy network to update it at the same time. The target network partially copies the parameters of the policy network and the value network for soft update. The gradient is calculated as follows: GradQ=Q+Reward-Q' (10); In the formula, GradQ represents the updated gradient of the value network, Q represents the future maximum reward value estimated by the value network, Reward represents the reward and punishment value returned by the evaluation model, and Q' represents the future maximum reward value estimated by the target value network after the update.

7. The method for intelligent line selection of underground expressways based on GIS and InSAR according to claim 1, characterized in that: The step S5 comprises: Steps S2-S4 are repeated until a round of line selection is completed, wherein the conditions for completing a round of line selection are reaching the end point, hitting an obstacle, and exceeding the limited number of cycles; after a round of line selection is completed, the total value of the entire reward value is counted.

8. The method for intelligent line selection of underground expressways based on GIS and InSAR according to claim 1, characterized in that: The step S6 comprises: Steps S1-S5 are repeated until the total reward value reaches the optimization target. The condition for achieving the optimization target is that the total reward value continues to rise and reaches a stable state. At this time, the route generated by the converged strategy network is the optimal route.

9. A computer system comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the underground expressway reinforcement learning intelligent line selection method based on GIS and InSAR as described in any one of claims 1-8.

Images