Tunnel arch wall reverse construction parameter optimization method and system based on reinforcement learning

Abstract

The application discloses a tunnel arch wall reverse construction parameter optimization method and system based on reinforcement learning, comprising the following steps: acquiring data of engineering basic parameters; defining state space and action space according to arch wall concrete reverse construction parameters; constructing a comprehensive risk function according to the data of engineering basic parameters and construction parameters, and setting a reward function according to the comprehensive risk function; the comprehensive risk includes concrete cold joint risk, formwork side pressure out-of-control risk and concrete layer thickness deviation risk; constructing a reinforcement learning intelligent agent, and outputting a construction parameter combination with minimum comprehensive risk. The application couples key construction parameters with three types of risks, namely, cold joint risk, formwork side pressure out-of-control risk and layer thickness deviation risk, establishes corresponding comprehensive risk functions and reward functions for learning, forms a closed-loop control system of 'risk quantification-comprehensive evaluation-intelligent optimization-site application', greatly reduces the dependence on construction experience, improves the efficiency of construction parameter determination, and helps to reduce construction risks.

Description

Technical Field

[0001] This invention relates to the field of tunnel construction technology, specifically to a method and system for optimizing reverse construction parameters of tunnel arch walls based on reinforcement learning. Background Technology

[0002] The reverse casting technology for tunnel arch walls can improve traditional casting problems such as difficult vibration compaction, easy formation of cold joints at the arch foot, and uneven stress on the formwork. However, it also brings greater sensitivity to construction rhythm. Existing reverse casting construction methods mainly rely on: construction experience to control pumping speed; engineers adjusting layer thickness based on experience; roughly determining the interlayer interval time based on the initial setting time of concrete; lack of unified quantitative risk assessment indicators; and lack of systematic optimization methods, with most parameter combinations being empirically optimal rather than globally optimal.

[0003] Existing methods have the following shortcomings: 1) Lack of quantitative risk perception: There are no continuous, differentiable cold joint risk models or template lateral pressure risk models, making it difficult to use them for optimization algorithms; 2) Parameter coupling cannot be intuitively judged manually: Pumping speed v, layer thickness h, and interlayer interval time of pouring layers. The effects of cold joints, lateral pressure, and density are opposite, making it difficult to obtain the optimal equilibrium solution manually; 3) Lack of intelligent optimization framework: Existing methods do not adopt intelligent optimization strategies such as reinforcement learning, but rely solely on manual adjustment; 4) Traditional lateral pressure models are not suitable for reverse casting; 5) The peak lateral pressure in the traditional model is located at the arch waist, while the peak lateral pressure in reverse casting is located in the middle and lower section of the sidewall.

[0004] In summary, the reverse casting process is subject to the coupled control of multiple continuous variables such as pumping speed, layer thickness, and interlayer interval time. The resulting risks, including cold joint risks, excessive formwork side pressure risks, and uneven compaction, have long lacked scientific quantification and global optimization methods. Therefore, this invention necessitates establishing a new risk model applicable to reverse casting and employing intelligent optimization methods to achieve parameter optimization. Summary of the Invention

[0005] This invention addresses the common quality risks and uncertainties in the construction schedule of reverse casting of tunnel lining arch walls by proposing a method and system for optimizing tunnel arch wall reverse construction parameters based on reinforcement learning.

[0006] This invention aims to solve the following core technical problems:

[0007] (1) The relationship between the parameters of the reverse pouring construction rhythm is complex and the optimal combination cannot be obtained through manual experience;

[0008] (2) There is a lack of a quantifiable, continuous, and optimizable risk assessment system for reverse casting;

[0009] (3) Existing construction methods cannot avoid the coupled risks of cold joints, excessive pressure on formwork, and uncontrolled thickness;

[0010] (4) There is a lack of intelligent algorithm framework that can automatically optimize the construction rhythm and achieve low-risk and high-efficiency pouring control.

[0011] To achieve the above objectives, the present invention adopts the following technical solution:

[0012] In a first aspect, the present invention provides a method for optimizing reverse construction parameters of tunnel arch walls based on reinforcement learning, comprising the following steps:

[0013] Step 1: Obtain data on basic engineering parameters, including arch height, concrete density, initial setting time of concrete, ambient temperature during construction, and allowable lateral pressure on the formwork.

[0014] Step 2: Define the state space and action space based on the reverse construction parameters of the arch wall concrete; the construction parameters include the layer thickness of concrete pouring, the interlayer interval time, and the pumping speed.

[0015] Step 3: Based on the basic engineering parameters and the reverse construction parameters of the arch wall concrete, construct a comprehensive risk function, and set a reward function based on the comprehensive risk function; the comprehensive risks include the risk of cold joints in concrete, the risk of uncontrolled side pressure on the formwork, and the risk of deviation in concrete layer thickness;

[0016] Step 4: Construct a reinforcement learning agent to output the combination of construction parameters that minimizes overall risk.

[0017] Furthermore, in step two, the state space and the action space are in one-to-one correspondence, with each state corresponding to a unique action; wherein, the action space is a discrete action space, which is constructed as follows: the value range of the construction parameters is determined based on engineering experience, several discrete points are selected from the value range, and multiple combinations of construction parameters are constructed according to the orthogonality principle as the discrete action space.

[0018] Furthermore, the range of values ​​for the construction parameters is determined according to formula (2.1):

[0019] (2.1);

[0020] In the formula: These are the lower and upper limits of the value range, respectively; These are the theoretical lower limit and theoretical upper limit of the construction parameters, respectively. For safety redundancy of construction parameters;

[0021] The discrete points of the construction parameters are selected according to the following method:

[0022] For the pumping speed of concrete pouring, the number of discrete points should be no less than 5, preferably 5-9;

[0023] For the thickness of each layer in concrete pouring, the number of discrete points should be no less than 4, preferably 4-7;

[0024] For the inter-layer interval time of concrete pouring, the number of discrete points should be no less than 7, preferably 7-12.

[0025] Furthermore, in step three, the reward function is determined according to formula (3.1):

[0026] (3.1);

[0027] (3.2);

[0028] In the formula: As a reward; For comprehensive risks; These are the risks of cold joints in concrete, uncontrolled side pressure on formwork, and deviations in concrete layer thickness. These are the weighting coefficients for the risks of cold joints in concrete, runaway formwork lateral pressure, and deviations in concrete layer thickness, respectively, satisfying... It is recommended to take This reflects the engineering weighting principle of "cold joint risk as the main concern, lateral pressure as the key factor, and thickness deviation as a secondary but not negligible factor".

[0029] Furthermore, in step three, the risk of cold joints in concrete is determined according to formula (3.3):

[0030] (3.3);

[0031] In the formula: To mitigate the risks of cold seams, These are risks associated with excessively short intervals, risks of cold seams exceeding the recommended aging period, and risks related to temperature sensitivity.

[0032] in, Determine according to formula (3.4):

[0033] (3.4);

[0034] In the formula: The initial setting time of concrete (min) represents the time from the completion of pouring to the beginning of the loss of plasticity. The interlayer interval (min) between two adjacent concrete layers; This is the short-interval risk coefficient, a dimensionless empirical coefficient used to adjust the sensitivity of "excessively short intervals" to risk. It is recommended to use [value missing]. ;

[0035] in, Determine according to formula (3.5):

[0036] (3.5);

[0037] In the formula: The risk factor for cold seams exceeding the aging period is dimensionless and reflects the risk level exceeding 0.8. "The slope of the risk increasing over time is recommended to be..." =1.0~3.0;

[0038] in, Determine according to formula (3.6):

[0039] (3.6);

[0040] In the formula: The ambient temperature during construction (°C); For reference temperature, 20℃ is used; The temperature sensitivity coefficient is dimensionless; it is recommended to take [value missing]. .

[0041] Furthermore, in step three, the risk of uncontrolled pressure runaway on the template side is determined according to formula (3.7):

[0042] (3.7);

[0043] In the formula: The allowable lateral pressure on the template (kPa) is determined by structural calculations or specifications. The equivalent lateral pressure (kPa) acting on the formwork during the reverse casting process;

[0044] in, Determine according to formula (3.8):

[0045] (3.8);

[0046] In the formula: The unit weight of concrete (kg / m³) 3 v is the pumping speed, in m³ / min; the equivalent pouring height is taken as... ; The current layer thickness (m); The height of the arch wall; Here, is an empirical coefficient for lateral pressure, and is a dimensionless coefficient. Considering the influence of pouring method, formwork stiffness, etc., on lateral pressure, it is recommended to take . .

[0047] Furthermore, in step three, the risk of concrete layer thickness deviation is determined according to formula (3.9):

[0048] (3.9);

[0049] In the formula: The current layer thickness (m); The optimal layer thickness (m) is given by engineering experience, such as 0.35m.

[0050] Furthermore, in step four, the Q-learning algorithm is used to construct the reinforcement learning agent, and the update formula for the Q-value function is as shown in formula (4.1):

[0051] (4.1);

[0052] In the formula: In the state Select action Value estimation at that time; The actual action chosen by the agent at time t; For the next state Given any action from the set of all possible actions, use it to find the maximum Q value; The learning rate, typically ranging from 0.05 to 0.1, is used to control the magnitude of each update. This is a discount factor, typically with a value of 0.95, indicating the importance placed on future rewards; The immediate reward for the current step, i.e., the negative overall risk;

[0053] in, Determine according to formula (4.2):

[0054] (4.2).

[0055] Furthermore, in step four, the ε-greedy strategy is used to explore the combination of construction parameters that minimizes overall risk.

[0056] Secondly, the present invention provides a tunnel arch wall reverse construction parameter optimization system based on reinforcement learning, comprising:

[0057] Data acquisition module: Acquires data on basic engineering parameters, including arch height, concrete unit weight, concrete initial setting time, construction ambient temperature, and allowable lateral pressure of formwork;

[0058] State and Action Space Construction Module: Based on the reverse construction parameters of the arch wall concrete, a state space and an action space are constructed; the construction parameters include the layer thickness of concrete pouring, the interlayer interval time, and the pumping speed.

[0059] Reward function construction module: Based on the basic engineering parameters and the reverse construction parameters of the arch wall concrete, a comprehensive risk function is constructed, and a reward function is set according to the comprehensive risk function; the comprehensive risks include concrete cold joint risk, formwork lateral pressure runaway risk, and concrete layer thickness deviation risk.

[0060] Reinforcement Learning Exploration Module: Constructs a reinforcement learning agent to output the combination of construction parameters that minimizes overall risk.

[0061] Compared with the prior art, the present invention has the following beneficial technical effects:

[0062] (1) Through the above technical solution, the present invention tightly couples the key construction parameters in the reverse casting of tunnel arch walls with three types of risks: cold joints, template side pressure and layer thickness deviation, forming a complete closed-loop control system of "risk quantification-comprehensive evaluation-intelligent optimization-on-site application", which effectively solves the problems of parameter setting relying on experience, difficulty in quantifying risks and inability to achieve global optimization in the existing technology;

[0063] (2) Based on the inter-layer interval time, pumping speed and layer thickness of concrete pouring, the present invention establishes the corresponding risks of cold joints in concrete, the risk of uncontrolled side pressure of formwork and the risk of deviation in concrete layer thickness, and then establishes a comprehensive risk function, which is used as the reward function for reinforcement learning, so that the final combination of construction parameters has the lowest comprehensive risk, greatly reducing the dependence on construction experience, which can improve the efficiency of determining construction parameters and help reduce construction risks. Attached Figure Description

[0064] Figure 1 This is a flowchart of the construction parameter optimization method based on Q-learning reinforcement learning according to the present invention.

[0065] Figure 2 For discrete action space in v– – Distribution map in space.

[0066] Figure 3 Risk of cold seams With interval time The change graph.

[0067] Figure 4 Side pressure of template under different layer thicknesses Graph showing the variation with pump speed v.

[0068] Figure 5 Risk of layer thickness deviation Follow The change graph.

[0069] Figure 6The convergence curve for Q-learning training.

[0070] Figure 7 This is a comparison chart of the results of random search and Q-learning optimization.

[0071] Figure 8 This is a comparison chart of the predicted and measured values ​​of the risk of runaway pressure on the template side.

[0072] Figure 9 This is a comparison chart of recommended values ​​and measured average values ​​for layer thickness risk.

[0073] Figure 10 This is the vt comprehensive risk contour map when h=0.35m is fixed. Detailed Implementation

[0074] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0075] This invention proposes a method and system for optimizing reverse construction parameters of tunnel arch walls based on reinforcement learning. The method focuses on the risks of cold joints, excessive lateral pressure on formwork, and layer thickness deviation during the concrete pouring process of the arch wall. It constructs a continuously differentiable risk quantification model. Based on this model, it discretizes three key construction parameters—pumping speed, layer thickness, and interlayer interval time—and uses reinforcement learning algorithms such as Q-learning to search for the globally minimum risk combination of construction parameters (actions), thereby achieving intelligent optimization of the reverse pouring rhythm.

[0076] Implementation method one;

[0077] Please see Figure 1-3 This embodiment provides a method for optimizing reverse construction parameters of tunnel arch walls based on reinforcement learning. Figure 1 The flowchart of the construction parameter optimization method based on Q-learning reinforcement learning of the present invention includes the following steps.

[0078] Step 1: Obtain data on the basic parameters of the concrete project, including the arch height, concrete unit weight, initial setting time of concrete, ambient temperature during construction, and allowable lateral pressure of the formwork.

[0079] Among them, the initial setting time of concrete Construction ambient temperature Used to determine the risk of concrete cold joints Template allows lateral pressure Concrete density Arch height Used to determine the risk of runaway pressure on the template side. .

[0080] Step 2: Define the state space and action space based on the reverse construction parameters of the arch wall concrete; the construction parameters include the layer thickness of concrete pouring, the interlayer interval time, and the pumping speed.

[0081] The state space and action space are in one-to-one correspondence, with each state corresponding to a unique action; wherein, the action space is a discrete action space, which is constructed as follows: the range of values ​​for construction parameters is determined based on engineering experience, several discrete points are selected from the range of values, and multiple combinations of construction parameters are constructed according to the orthogonality principle as the discrete action space.

[0082] For example, the set of state spaces is Each state corresponds to a unique action, which in turn corresponds to a unique combination of construction parameters. , where n represents the state number. The largest number, These are discrete point numbers representing the pumping speed, layer thickness, and interlayer interval time of concrete pouring, respectively.

[0083] The following section provides a detailed explanation of the design principles for the number of discrete points, interval endpoints, and spacing between discrete points.

[0084] (1) Number of discrete points.

[0085] To balance model accuracy and computational feasibility, the recommended number of discrete points for the three types of parameters is shown in Table 1 below:

[0086] .

[0087] The principle of minimum discrete points: It should be able to cover the "risk trend inflection points" of parameter changes (such as the upper limit of short intervals in cold seams, the optimal interval zone, and the starting point of aging risk).

[0088] (2) Determination of discrete intervals.

[0089] The endpoints of the interval must cover all reasonable ranges that may occur during the concrete pouring process. The range of construction parameters is determined by the following formula:

[0090] ;

[0091] In the formula: These are the lower and upper limits of the value range, respectively; These are the theoretical lower limit and theoretical upper limit of the construction parameters, respectively. This provides safety redundancy for construction parameters.

[0092] The following are methods for determining the value ranges of three types of construction parameters.

[0093] For pumping speed *v*, the pumping speed is related to the concrete supply capacity, pump head, and equipment model. Based on the above principles, the lower limit of the value range is taken as the minimum stable output of the equipment, typically 0.8 times the rated minimum pumping capacity, and the upper limit of the value range is taken as the maximum allowable conveying speed of the equipment, typically 1.1 times the rated maximum pumping capacity (including a safety margin). For example, for a common 0.5-1 m³ / min pump truck, then... m 3 / min, m 3 / min, yielding a range of approximately 0.20-1.10m. 3 / min.

[0094] For the layer thickness h, it is determined by referring to the effective thickness of concrete vibration and the range of action of the vibrator. Based on the above principles, the lower limit of the value range is the minimum thickness that ensures compaction, typically 0.12-0.15m, and the upper limit is the maximum thickness that avoids honeycomb-like pitting and insufficient vibration, typically 0.40-0.45m. Therefore, a generally acceptable range is: .

[0095] For interval time Due to the initial setting time of concrete Decision. Short-interval risk. Optimal range Risks of exceeding the aging period Therefore, the recommended interval is... .For example ,but To support extreme operating conditions, the time can be extended to 60 minutes.

[0096] (3) Determination of the distance between discrete points.

[0097] Key principle: Encrypt in areas where risks change rapidly, and decrypt in areas where changes are slow.

[0098] The methods for determining the spacing between discrete points for the three types of construction parameters are as follows:

[0099] 1) For the discrete point spacing of pumping speed v, the pressure on the side affected by pumping speed increases linearly, therefore the actual risk changes smoothly. Equal spacing is sufficient, for example, a discrete set (0.20, 0.35, 0.50, 0.70, 0.85, 1.00, 1.10). If the pump truck model has different pressure effects, the spacing can be appropriately increased in certain high-pressure sections.

[0100] 2) The discrete point spacing of the layer thickness h shows that the layer thickness deviation risk function is parabolic, indicating that the greater the deviation of h, the faster the rise, and the optimal point... The nearby risks are low. Therefore, the following approach is adopted: The sampling is sparse and dispersed in the vicinity, and denser sampling is performed in areas that are too large or too small, such as discrete sets. ;

[0101] 3) Regarding the inter-layer interval time The discrete point spacing is most sensitive to cold seam risk, and it is treated in three stages: A. Short intermittent stage →Rapid changes→Encryption, e.g., 5-minute intervals; B. Optimal phase → Slow change → Slightly sparse, for example, an interval of 10 minutes; C. Long interval phase →Risk escalates rapidly→Re-encrypt, for example, with a 5-minute interval; For example, discrete sets can be set. .

[0102] This implementation provides an example of constructing a discrete action space:

[0103] 1) Discretization of pumping speed v The unit is m 3 / min, 7 nodes cover the commonly used range from low speed and high quality to high speed and high efficiency;

[0104] 2) Discretization of layer thickness h, The unit is m, where m is... The remaining nodes represent different working conditions, such as thinner or thicker.

[0105] 3) Inter-layer interval time Discretization, The unit is min, covering the full range from continuous pouring to fully intermittent pouring, with a focus on the empirically optimal range of 15-40 min.

[0106] Therefore, an action is defined as The total number of possible actions is Please see Figure 2 These actions together constitute the action space in reinforcement learning. The action space corresponds one-to-one with the state space. That is, each action corresponds to a combination of construction parameters and a set of feasible reverse pouring rhythm schemes.

[0107] Step 3: Based on the basic engineering parameters and the reverse construction parameters of the arch wall concrete, construct a comprehensive risk function, and set a reward function based on the comprehensive risk function; the comprehensive risks include the risk of cold joints in concrete, the risk of uncontrolled lateral pressure on the formwork, and the risk of deviation in the thickness of concrete layers.

[0108] First, we will explain the methods for constructing the functions or models that constitute the three types of risks in the overall risk framework.

[0109] (1) Cold seam risk model .

[0110] To characterize the interlayer interval time Regarding the impact on interface strength, this invention decomposes the risk of cold joints into three parts: "risk of excessively short intermittent periods," "risk of cold joints exceeding the recommended aging period," and "temperature-sensitive term," constructing the following dimensionless risk function:

[0111] ;

[0112] In the formula: To mitigate the risks of cold seams, These are risks associated with excessively short intervals, risks of cold seams exceeding the recommended aging period, and risks related to temperature sensitivity.

[0113] When the next layer of concrete is poured before the previous layer has fully stabilized, problems such as insufficient vibration and excessive interface disturbance can easily occur, increasing the risk of insufficient intervals between pours. Represented as:

[0114] ;

[0115] In the formula: The initial setting time of concrete (min) represents the time from the completion of pouring to the beginning of the loss of plasticity. The interlayer interval (min) between two adjacent concrete layers; This is the short-interval risk coefficient, a dimensionless empirical coefficient used to adjust the sensitivity of "excessively short intervals" to risk. It is recommended to use [value missing]. ; Indicates only when A penalty is incurred when the interval is not less than [time period]. When that time, the value of this item is 0.

[0116] This implementation method adopts The reason for using 0.3 as the penalty threshold for short-interval risks is that concrete remains in a highly plastic stage for approximately 30% of its initial setting time after pouring, with extremely low interfacial shear strength. This makes it prone to interfacial disturbance, segregation, and cold joint hazards during continued pouring and vibration. Extensive engineering experience and standards have shown that when the interfacial interval is less than one-third of the initial setting time, the probability of quality accidents increases significantly. Therefore, using 0.3 as the penalty threshold has sufficient engineering and theoretical basis.

[0117] when If the curing time is too long, the surface of the lower concrete layer will begin to harden and lose water, the interfacial bond strength will decrease sharply, and the risk of cold joints will increase significantly, thus increasing the risk of cold joints beyond the curing period. Represented as:

[0118] ;

[0119] In the formula: The risk factor for cold seams exceeding the aging period is dimensionless and reflects the "risk factor for cold seams exceeding the aging period". "The slope of risk increasing over time; based on engineering experience," .

[0120] This implementation method adopts The time threshold for assessing the risk of cold joints exceeding the set time is chosen because extensive engineering experience, standards, and interfacial bond strength tests all indicate that when the interval exceeds approximately 80% of the initial setting time, the surface of the underlying concrete enters a significant water loss and hardening stage, resulting in a sharp decline in interfacial bond capacity and an inability to form an effective bond with the newly poured concrete. Therefore, the risk function should exhibit a rapid upward trend after this point, and dimensionless parameters should be used. Adjust the slope of risk ascent. Only when When this value is greater than 0, it reflects the engineering experience of "entering the interface failure stage ahead of time".

[0121] The lower the ambient temperature, the slower the hydration reaction and the more fragile the interface. When considering the effects of temperature, temperature-sensitive risks should be taken into account. Represented as:

[0122] ;

[0123] In the formula: The ambient temperature during construction (°C); For reference temperature, 20℃ is used; For the temperature sensitivity coefficient, which is dimensionless, and considering concrete maturity theory and common engineering experience, it is recommended to take [value missing]. .

[0124] when At that time, the item This can be considered as 0, indicating that high temperature no longer amplifies the risk of cold seams.

[0125] thus, exist Both excessively short and excessively long values ​​increase linearly or quasi-linearly with the degree of deviation, reaching 0.3. -0.8 The interval being close to 0 reflects the inter-layer interval time. The optimal range.

[0126] (2) Risk model of runaway pressure on template side .

[0127] To describe the relationship between pumping speed and template lateral pressure, this invention proposes a new equivalent lateral pressure expression based on the stress characteristics of reverse casting:

[0128] ;

[0129] In the formula: The equivalent lateral pressure (kPa) acting on the formwork during the reverse casting process; The unit weight of concrete (kg / m³) 3 v is the pumping speed, in m³ / min; the equivalent pouring height is taken as... ; The current layer thickness (m); The height of the arch wall; Here, is an empirical coefficient for lateral pressure, and is a dimensionless coefficient. Considering the influence of pouring method, formwork stiffness, etc., on lateral pressure, and combining common standard lateral pressure formulas with practical experience in tunnel and lining engineering, it is recommended to take . " / 200" is an empirical scaling factor used to convert theoretical values ​​to the kPa level measured in typical engineering projects.

[0130] To convert lateral pressure into a dimensionless risk indicator, this invention employs a secondary enhancement approach to describe the risk of template lateral pressure runaway. :

[0131] ;

[0132] In the formula: The allowable lateral pressure (kPa) for the template is determined by structural calculations or specifications.

[0133] when hour This indicates that the risk is controllable; when Approaching or exceeding hour, The sharp increase reflects the "non-linear amplification" characteristic of the template instability risk.

[0134] (3) Layer thickness deviation risk model .

[0135] To characterize the impact of layer thickness deviation from the optimal thickness on compaction and construction efficiency, this invention constructs a system based on the "optimal layer thickness". "A risk function that is centrally symmetric:"

[0136] ;

[0137] In the formula: The current layer thickness (m); The optimal layer thickness (m) is given by engineering experience, such as 0.35m.

[0138] when hour ;when At times, insufficient vibration depth can easily lead to voids and honeycomb structures, increasing the risk; when As the number of layers and interfaces increases, the accumulated risk of cold joints and construction costs rise, which also manifests as increased risk. The square form ensures that the risk increases twice with the deviation, which is consistent with the actual law that "the greater the deviation, the faster the probability of quality problems occurs."

[0139] Based on the risks of cold joints in concrete, uncontrolled lateral pressure in formwork, and deviations in concrete layer thickness, a comprehensive risk function can be constructed. This invention weights and superimposes the three dimensionsless risks mentioned above to construct a comprehensive risk function suitable for optimizing the rhythm of reverse pouring construction:

[0140] ;

[0141] In the formula: For comprehensive risks; These are the risks of cold joints in concrete, uncontrolled side pressure on formwork, and deviations in concrete layer thickness. These are the weighting coefficients for the risks of cold joints in concrete, runaway formwork lateral pressure, and deviations in concrete layer thickness, respectively, satisfying... It is recommended to take This reflects the engineering weighting principle of "cold joint risk as the main concern, lateral pressure as the key factor, and thickness deviation as a secondary but not negligible factor".

[0142] Comprehensive Risks Functions have the following characteristics: The three variables are continuously differentiable and have no singularities, which facilitates the use of reinforcement learning and other optimization algorithms; by adjusting the weights, different risk focuses can be emphasized for different projects; multiple heterogeneous risks are uniformly normalized into a scalar index, which facilitates comparison and decision-making.

[0143] Based on the comprehensive risk function, the reward function is set as follows:

[0144] ;

[0145] In the formula: As a reward.

[0146] Comprehensive Risks The smaller, the more rewards The larger the value, the more the goal of "minimizing risk" is transformed into the reinforcement learning problem of "maximizing long-term cumulative reward".

[0147] Step 4: Construct a reinforcement learning agent to output the combination of construction parameters that minimizes overall risk.

[0148] This embodiment employs the Q-learning algorithm to perform reinforcement learning search on a finite, discretized construction parameter space. However, this invention is not limited to a specific algorithm form; any reinforcement learning method that can achieve optimal parameter combination search based on a reward function within a discrete or continuous action space can be used as an equivalent substitute, including but not limited to TD temporal difference, deep Q-networks, policy gradient, Monte Carlo tree search, etc. These alternative algorithms, based on the same optimization objective function and risk model, can achieve the same or better construction parameter optimization results as Q-learning and are equivalent technical solutions of this invention.

[0149] This implementation uses the Q-learning algorithm as an example to automatically search for the construction parameter combination with the lowest overall risk among the 315 alternative options (actions). Please refer to [link / reference]. Figure 1 , Figure 1 This is a flowchart of a construction parameter optimization method based on Q-learning reinforcement learning.

[0150] The update formula for the Q-value function is:

[0151] ;

[0152] In the formula: In the state Select action Value estimation at that time; The actual action chosen by the agent at time t; For the next state Any action from the set of all possible actions is used to find the maximum Q value, i.e. ; The learning rate, typically ranging from 0.05 to 0.1, is used to control the magnitude of each update. This is a discount factor, typically with a value of 0.95, indicating the importance placed on future rewards; The immediate reward for the current step, i.e., the negative overall risk;

[0153] Since this problem involves static parameter optimization and does not involve the physical evolution of the time series, the state transition can be simply defined as:

[0154] ;

[0155] That is, "the next state is the selected scheme number".

[0156] This invention employs an ε-greedy exploration strategy as its behavioral strategy. Besides ε-greedy, other methods such as Softmax exploration, UCB confidence bound algorithm, noise perturbation exploration, parameterized noise exploration, and entropy-based exploration can also be used. Essentially, these methods aim to achieve a balance between exploration and utilization when "finding the optimal combination of construction parameters," and are equivalent alternatives to this invention. ε-greedy is chosen to achieve efficient traversal of the finite discrete action space. Using Softmax, UCB, or noise exploration can achieve the same effect. This invention is not limited to a specific exploration algorithm; any strategy capable of completing the search within the discrete action space is an equivalent implementation of the technical solution of this invention.

[0157] To strike a balance between "exploring new solutions" and "utilizing current best practices," this implementation method employs an ε-greedy strategy to obtain the combination of construction parameters that minimizes overall risk. The specific method of the ε-greedy strategy is as follows:

[0158] Randomly select an action with probability ε to explore;

[0159] The action with the highest current Q value is selected with probability 1-ε and utilized.

[0160] ε decays linearly with the training process, such as gradually decreasing from 0.9 to 0.05, allowing the algorithm to fully search the parameter space in the early stages and then focus on a fine search around the optimal solution in the later stages.

[0161] After multiple iterations, the Q-value matrix gradually converges. At this point, selecting the action with the largest Q-value in any initial state yields a near-globally optimal combination of construction parameters for arch wall concrete pouring. Specific implementation examples:

[0163] Please see Figure 2-10 This embodiment is based on the construction of the right-line arch wall of a tunnel in Guangdong Province. The surrounding rock of this section is grade IV to V moderately weathered granite. The net height of the arch wall is about 5.5m, and the length of a single section is 12.0m. A reverse layered pouring process using a trolley is adopted. The following are the main risks in on-site construction: the risk of cold joints at the interface due to excessively short or long intervals between layers; the risk of excessive side pressure on the formwork due to excessive pumping speed; and the risk of vibration difficulties or low efficiency due to deviations from the recommended layer thickness.

[0164] Therefore, in this embodiment, three monitoring sections (denoted as 1#, 2#, and 3#) were set up in a typical section of the arch wall to collect data on the maximum lateral pressure of the template and the measured layer thickness. The accuracy of the method of the present invention was verified based on the on-site monitoring data.

[0165] First, this embodiment provides the empirically recommended ranges and on-site values ​​for pumping speed, layer thickness, and interlayer interval time, which are based on construction experience.

[0166] (1) Regarding the pumping speed, please provide an empirically recommended range and the actual value used on site. Generally, it is recommended to control it within a certain range. The lower limit is approximately If the pouring time is too long, the risk of cold joints will increase significantly, and the efficiency will be too low; the upper limit should not exceed If the height is further increased, the lateral pressure and impact effect on the arch wall formwork will significantly increase, easily exceeding the bearing capacity of conventional steel formwork systems. Considering the Class IV-V surrounding rock, the clear height of the arch wall of approximately 5.5m, the performance of the truck-mounted pump, and the rigidity of the formwork system, the on-site construction personnel will control the pumping speed within a medium-to-high range. Before the introduction of the method of this invention, on-site construction personnel often took a "compromise value" of the interval, that is... They believed that this approach could ensure a certain level of efficiency without significantly impacting the template.

[0167] (2) Regarding the layer thickness, given that the tunnel arch concrete is a vertical curved surface component, the conventional layer thickness is mostly controlled within... Approximately 0.35m is considered the "experienced optimal value," balancing ease of concrete vibration, air bubble rise path, and pouring efficiency. If the concrete is too thin (e.g., <0.25m), more layers are needed, resulting in lower efficiency; if it's too thick (e.g., >0.45m), vibration cannot penetrate to the bottom, easily causing honeycomb and pitted surfaces. Design and on-site construction typically adopt... Before adopting the method of this invention, on-site construction workers generally controlled the spacing at a uniform 0.35m. Slight adjustments were made in individual locations due to dense reinforcement or construction interference, but overall the spacing remained consistent. It fluctuates within a certain range.

[0168] (3) Regarding the interlayer interval, considering the initial setting time of C30 ordinary Portland cement concrete at around 20℃, it is generally believed that too short an interval (close to continuous pouring) will lead to the risk of "pouring the upper layer before the lower layer has initially set," which can easily cause formwork vibration and segregation. Too long an interval will result in obvious cold joints. The usually given optimal interval is... The concrete mix design used in this project had an initial setting time of approximately [time missing] measured inside the tunnel. Construction workers generally rely on experience to select and control the area. It takes about 30 minutes, which is considered to be beneficial for the lower layer of concrete to slightly "slurry up and finish" without forming obvious cold joints.

[0169] Secondly, this embodiment provides the basic engineering parameters, action space, reward function, and solution process.

[0170] Step 1, the basic engineering parameters are:

[0171] The arch wall is approximately 5.5m high, the concrete density is approximately 2450kg / m³, and the initial setting time of the concrete is approximately... The ambient temperature during construction is close to 20℃, and the allowable lateral pressure on the formwork is... .

[0172] Step two, the motion space is:

[0173] Pumping speed within the range of values Seven representative points were selected within the range, namely: .

[0174] Layer thickness within the range of values Five representative points were selected within the range at intervals of 0.05m, namely... ,in As an ideal thickness for experience.

[0175] Interlayer interval time within the range of values Nine discrete points were selected within the range at intervals of 5-10 minutes, namely... .

[0176] The three parameters can be combined to create 7 × 5 × 9 = 315 possible "construction actions" or combinations of construction parameters, such as... Figure 2 Discrete action space in The spatial distribution shows that each point corresponds to a set. The solution provides a limited action space for reinforcement learning algorithms.

[0177] Step 3: Obtain the comprehensive risk function and reward function.

[0178] Figure 3-5 The variation patterns of the corresponding functions for concrete cold joint risk, formwork lateral pressure runaway risk, and concrete layer thickness deviation risk are shown respectively. The comprehensive risk function can be obtained by weighting the three types of risks.

[0179] Please see Figure 3 , Figure 3 Cold seam risk Rc varies with interval time The changes illustrate the overall shape of the function: the risk is approximately zero between 15 and 35 minutes, increases significantly after the interval exceeds 40 minutes, and reaches approximately 0.43 at 60 minutes. In this embodiment, the tunnel construction environment temperature is close to 20°C, therefore the risk of the temperature-sensitive term can be ignored.

[0180] Please see Figure 4 , Figure 4 The variation of template side pressure p with pump speed v under different layer thicknesses indicates that: at the three layer thicknesses At this rate, the side pressure increases approximately linearly with the pump speed, approaching the allowable value at 1.0 m³ / min and slightly exceeding the limit at 1.2 m³ / min. Therefore, subsequent optimizations will favor selecting a pump speed of around 0.8~1.0 m³ / min.

[0181] Please see Figure 5 , Figure 5 The variation of layer thickness deviation risk Rh with h is shown.

[0182] Step four: Construct a reinforcement learning agent to output the combination of construction parameters that minimizes overall risk.

[0183] Among them, learning rate Discount factor The ε-greedy exploration strategy linearly decayed from 0.9 to 0.05. The number of training epochs was set to 2000.

[0184] Please see Figure 6 , Figure 6 In the Q-learning training convergence curve, the blue curve represents the risk per round after moving average, and the orange dashed line represents the historical minimum risk. It can be seen that the algorithm rapidly reduces the overall risk in the first 200 rounds, then slowly oscillates and gradually stabilizes at around 0.021, indicating that Q-learning can find a stable optimal or near-optimal solution within a limited action space.

[0185] To verify the advantages of intelligent optimization over traditional experience or random search, 3000 random samples were used as a benchmark. Figure 7 A comparison of the results of random search and Q-learning optimization is presented, showing the total risk and individual components of the optimal solution obtained by the two methods:

[0186] Random search for the optimal solution: , , , ;

[0187] Optimal Q-learning solution: , , , .

[0188] It can be seen that the overall risk levels of the two are similar, but the parameters given by Q-learning are closer to the ideal layer thickness in engineering experience, and the construction rules of "controlling pump speed and moderate interval time" are automatically learned during the training process, which has stronger interpretability and scalability.

[0189] The final recommended combination of construction parameters in this embodiment is as follows: , , The on-site construction workers, based on their experience and the selected construction parameters, combined... , , Clearly, the combination of construction parameters obtained by the method of this invention is more accurate and closer to the actual project than the combination of construction parameters obtained by construction experience. It can improve the construction efficiency of arch wall concrete pouring while ensuring the lowest overall risk.

[0190] To verify the engineering reliability of the risk model and optimization results, resistance-type lateral pressure gauges and layer thickness gauges were installed on three monitoring sections of the arch wall (sections 1, 2, and 3). Under construction conditions close to the recommended parameters (pump speed approximately 0.8~0.9 m³ / min, layer thickness 0.33~0.37 m, and interval 18~25 min), the maximum template lateral pressure and the actual layer thickness were recorded.

[0191] Please see Figure 8 According to the model calculation, the maximum lateral pressure at the three cross-sections is approximately 15.5, 16.7, and 15.0 kPa, respectively; the corresponding measured values ​​are 15.3, 17.0, and 15.2 kPa. The average deviation is approximately 0.2 kPa, and the relative error does not exceed 3%, indicating that the template lateral pressure risk function (or model) can well reflect the influence of pump speed and layer thickness on molding, meeting the risk assessment requirements.

[0192] Please see Figure 9 The model recommends an optimal layer thickness of 0.35m. On-site sampling and statistics were conducted in 10 consecutive warehouse sections, and the measured average thickness was 0.346m with a standard deviation of approximately 0.015m. Figure 9 The results show a high degree of consistency between the two, indicating that the recommended thickness not only meets the operating habits of construction workers, but also ensures a balance between compaction and efficiency.

[0193] Using the recommended thickness Under the premise of pumping speed and interval time Perform a two-dimensional scan to calculate the overall risk under all combinations. , to obtain Figure 10 fixed of Comprehensive risk contour map. The contour map shows that the comprehensive risk is lowest (dark purple to light blue area) in the region with a pump speed of 0.6~0.9 m³ / min and an interval of 15~30 min, corresponding to the optimal construction range learned by Q-learning. When the pump speed exceeds 1.0 m³ / min or the interval is greater than 40 min, the risk increases sharply, mainly driven by the risks of formwork overpressure and cold joints. Although low pump speeds and longer intervals do not lead to overpressure, they significantly increase the risks of cold joints and project duration, and therefore are not recommended. Based on this, the construction parameters can be divided into: safe zone, control zone, and prohibited zone, providing a basis for on-site technical briefings and dynamic adjustments.

[0194] This embodiment takes the concrete pouring of the tunnel arch wall as the engineering background and constructs a comprehensive risk assessment function or model integrating "cold joint risk – lateral pressure risk – thickness deviation risk". A finite action space is obtained by discretizing the construction parameters, and the optimal combination of construction parameters is automatically searched using the Q-learning reinforcement learning algorithm. The model is validated using measured lateral pressure and layer thickness data. The results show that the quantitative description of the three types of risks can well reflect the experience judgment of on-site engineers (see...). Figures 3-5 Q-learning converges to a stable, low-risk solution within approximately 2000 training rounds. Figure 6 ), with performance better than or at least not worse than large-scale random search ( Figure 7 The measured values ​​of template lateral pressure and delamination thickness are highly consistent with the model predictions. Figure 8 , Figure 9 The model's engineering applicability was verified; the comprehensive risk contour map ( Figure 10 The invention intuitively presents a "safe and efficient pouring interval," facilitating its application in similar tunnel projects. Therefore, the "reinforcement learning-based method for optimizing parameters and controlling risks in reverse pouring of tunnel arch walls" proposed in this invention can automatically provide the optimal or near-optimal combination of pump speed, layer thickness, and interlayer interval time while meeting safety requirements, demonstrating significant engineering application value.

[0195] Field application and dynamic correction: The optimal combination is applied to field construction, and the model parameters can be adjusted appropriately based on real-time monitoring data such as lateral pressure and concrete temperature. The optimization program is then run again to achieve dynamic correction.

[0196] In layered concrete pouring, industry experience typically recommends a layer thickness of approximately 0.35m. However, traditional experience values ​​are based solely on long-term manual practice and lack quantifiable evaluation, scalability, and optimizability, making them difficult to adapt to varying working conditions, materials, and formwork systems. This invention addresses the risk of layer thickness deviation. The technical significance of introducing a comprehensive risk function system lies not in providing a new thickness value, but in achieving substantial technical improvements in the following three aspects.

[0197] (1) Model and mathematically represent empirical values ​​so that they can be quantified and integrated into the system risk assessment system;

[0198] Although engineering experience recommends a thickness of 0.35m, this experience cannot explain: how much deviation from this value would significantly increase risk; whether the impact of "too large" and "too small" thicknesses on construction quality and cost is symmetrical; or how to obtain the optimal thickness under multiple coupled risks. This invention, based on the optimal layer thickness given by engineering experience, uses the square of the normalized value of the current layer thickness as a deviation function and incorporates it as part of a comprehensive risk function. It no longer relies entirely on engineering experience but elevates experience into a quantifiable, calculable, and optimizable technical tool.

[0199] (2) Verify the rationality of empirical values ​​and provide generalization ability across projects;

[0200] In this embodiment, the optimal layer thickness obtained through model calculation and optimization algorithm under specific engineering conditions is still approximately 0.35m. This result demonstrates that the risk model of this invention is highly consistent with actual engineering practices; the recommended value is indeed the optimal decision under the current working conditions. More importantly, the model of this invention can automatically provide different optimal values ​​when working conditions change. For example, when the formwork stiffness increases and the lateral pressure risk decreases, the model may recommend a larger layer thickness; in high-temperature environments and with accelerated initial setting of concrete, the model will actively converge to a smaller thickness; when vibration conditions are limited or the pouring speed increases, the model can generate different optimization results. Therefore, the significance of the model lies in its adaptive capability, rather than a fixed numerical value.

[0201] (3) Ensure that the intelligent optimization algorithm converges within the engineering scope to avoid unreasonable thickness decisions;

[0202] Without a layer thickness deviation risk model, reinforcement learning algorithms (such as Q-learning) may produce extreme solutions that are unacceptable in engineering (such as 0.15m or 0.45m). Subsequently: the optimization search space is limited to a range consistent with construction principles; the risk minimization objective guides the algorithm to automatically regress to the engineering feasible range; thus, the results of intelligent optimization are engineering-executable. Therefore, the thickness deviation risk model of this invention plays a crucial constraining and corrective role in the entire risk system and is an indispensable component.

[0203] In summary, the significance of the model lies not in changing empirical values, but in endowing them with scientific rigor and optimizability. In general, this invention elevates "0.35m" from an empirical constant to: a verifiable engineering parameter; a calculable optimization variable; a quantifiable risk control indicator; and an intelligent decision-making factor that can automatically adjust to changes in working conditions. Therefore, the value of this model lies not in changing empirical values, but in: constructing, for the first time in the industry, a quantitative risk description method with layered thickness, transforming traditional experience into a mathematical model that can be used for computer optimization and intelligent control, thereby significantly improving construction risk management capabilities and the level of intelligence.

[0204] Implementation method two;

[0205] Please see Figure 1 This embodiment provides a reinforcement learning-based reverse construction parameter optimization system for tunnel arch walls, used to implement the aforementioned reinforcement learning-based reverse construction parameter optimization method for tunnel arch walls, specifically including:

[0206] Data acquisition module: Acquires data on basic engineering parameters, including arch height, concrete unit weight, concrete initial setting time, construction ambient temperature, and allowable lateral pressure of formwork.

[0207] State and Action Space Construction Module: Based on the reverse construction parameters of the arch wall concrete, a state space and an action space are constructed; the construction parameters include the layer thickness of concrete pouring, the interlayer interval time, and the pumping speed.

[0208] Reward function construction module: Based on the basic engineering parameters and the reverse construction parameters of the arch wall concrete, a comprehensive risk function is constructed, and a reward function is set according to the comprehensive risk function; the comprehensive risks include concrete cold joint risk, formwork lateral pressure runaway risk, and concrete layer thickness deviation risk.

[0209] Reinforcement Learning Exploration Module: Construct a reinforcement learning agent to explore and output the combination of construction parameters that minimizes overall risk.

[0210] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0211] The method of this invention constitutes a transferable and universal "risk-optimization integration" technical system, which can be adapted to different forms of vertical or layered concrete pouring processes. For "other vertical or layered concrete pouring processes" and "construction scenarios with formwork bearing constraints", the technical framework of this invention remains consistent, with adjustments only made to some model input parameters or local expressions according to the actual engineering situation; the essential method does not need to be changed.

[0212] This type of working condition shares the same risk mechanism as the reverse casting of the arch wall in this invention. Therefore, only the geometric dimensions of the component (such as the casting height H), material parameters, and allowable lateral pressure of the formwork need to be replaced with the corresponding engineering parameters. The risk model form and optimization method remain completely unchanged. The reason for the unchanged technology is that: the cold joint formation mechanism is independent of the component shape and depends only on the interval time and initial setting characteristics; the growth law of the formwork lateral pressure is related to "casting height – pump speed – concrete rheology", and is not unique to arch walls; the risk of layer thickness deviation exists in all vertical casting scenarios; reinforcement learning is sensitive to "action space + reward function" but not to geometric boundary form. Therefore, this invention does not require modification of the technical principle; it can be extended to all vertical casting or layered casting processes simply by replacing the engineering parameters.

[0213] This invention's method is applicable to construction conditions with formwork load-bearing constraints, such as: vertical shaft and vertical trench wall pouring, layered pouring of columns and shear walls, layer-by-layer pouring of large-volume concrete, and pouring of arch frames / side walls with adjustable formwork systems. Its core control factors remain: the relationship between concrete lateral pressure, pump speed, and pouring height; the limitation of formwork structural strength; the quality of interlayer bonding surfaces (cold joint risk); and the impact of layer thickness on compaction and construction efficiency. The three risk models of this invention correspond to the above control factors and can therefore be directly applied. Potential changes are limited to parameter substitution. Therefore, the technical framework remains unchanged, requiring only parameter conversion to adapt to different construction scenarios.

[0214] For other vertical structures, the equivalent height needs to be changed to This involves replacing engineering parameters, not changing the method steps of this invention; the parameter discrete range needs to be adjusted according to specific working conditions, but the reward function remains applicable. The intelligent optimization framework of this invention is universal and requires no changes; only parameter adjustments are needed to adapt to different working conditions. Only a few engineering parameters need to be replaced under different working conditions, such as component height, allowable lateral pressure, and recommended thickness. This invention constructs a portable and scalable technical platform whose functionality is not limited to reverse casting of arch walls but can adapt to all vertical or layered casting processes.

[0215] The above embodiments are only for illustrating the technical concept and features of the present invention. Their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be used to limit the scope of protection of the present invention. All equivalent changes or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for optimizing reverse construction parameters of tunnel arch walls based on reinforcement learning, characterized in that: Includes the following steps: Step 1: Obtain data on basic engineering parameters, including arch height, concrete density, initial setting time of concrete, ambient temperature during construction, and allowable lateral pressure of formwork. Step 2: Define the state space and action space based on the reverse construction parameters of the arch wall concrete; the construction parameters include the layer thickness of concrete pouring, the interlayer interval time, and the pumping speed. Step 3: Based on the basic engineering parameters and the reverse construction parameters of the arch wall concrete, construct a comprehensive risk function, and set a reward function based on the comprehensive risk function; the comprehensive risks include the risk of cold joints in concrete, the risk of uncontrolled side pressure on the formwork, and the risk of deviation in concrete layer thickness; The reward function is determined according to formula (3.1): （3.1）； （3.2）； In the formula: As a reward; For comprehensive risks; , , These are the risks of cold joints in concrete, uncontrolled side pressure on formwork, and deviations in concrete layer thickness. , , These are the weighting coefficients for the risks of cold joints in concrete, runaway formwork lateral pressure, and deviations in concrete layer thickness, respectively, satisfying... ; Step 4: Construct a reinforcement learning agent to output the combination of construction parameters that minimizes overall risk.

2. The method for optimizing reverse construction parameters of tunnel arch walls based on reinforcement learning according to claim 1, characterized in that: In step two, the state space and the action space are in one-to-one correspondence, and each state corresponds to a unique action; wherein, the action space is a discrete action space, which is constructed as follows: the value range of the construction parameters is determined based on engineering experience, several discrete points are selected from the value range, and multiple combinations of construction parameters are constructed according to the orthogonal principle as the discrete action space.

3. The method for optimizing reverse construction parameters of tunnel arch walls based on reinforcement learning according to claim 2, characterized in that: The range of values ​​for the construction parameters is determined according to formula (2.1): （2.1）； In the formula: These are the lower and upper limits of the value range, respectively; These are the theoretical lower limit and theoretical upper limit of the construction parameters, respectively. For safety redundancy of construction parameters; The discrete points of the construction parameters are selected according to the following method: For the pumping speed of concrete pouring, the number of discrete points shall not be less than 5; For the thickness of each layer in concrete pouring, the number of discrete points shall not be less than 4; For the inter-layer interval time of concrete pouring, the number of discrete points shall not be less than 7.

4. The method for optimizing reverse construction parameters of tunnel arch walls based on reinforcement learning according to claim 1, characterized in that: In step three, the risk of cold joints in concrete is determined according to formula (3.3): （3.3）； In the formula: To mitigate the risks of cold seams, , , These are risks associated with excessively short intervals, risks of cold seams exceeding the recommended aging period, and risks related to temperature sensitivity. in, Determine according to formula (3.4): （3.4）； In the formula: This refers to the initial setting time of the concrete. This refers to the interlayer interval between two adjacent concrete layers; For short-interval risk coefficients; in, Determine according to formula (3.5): （3.5）； In the formula: Risk factor for cold joints exceeding the aging period; in, Determine according to formula (3.6): （3.6）； In the formula: The ambient temperature during construction; For reference temperature; This is the temperature sensitivity coefficient.

5. The method for optimizing reverse construction parameters of tunnel arch walls based on reinforcement learning according to claim 1, characterized in that: In step three, the risk of uncontrolled pressure runaway on the template side is determined according to formula (3.7): （3.7）； In the formula: Allowable lateral pressure for the template; This refers to the equivalent lateral pressure acting on the formwork during the reverse casting process; in, Determine according to formula (3.8): （3.8）； In the formula: This refers to the unit weight of concrete. For pumping speed; equivalent pouring height, take ; This represents the current layer thickness. The height of the arch wall; This is the empirical coefficient for lateral pressure.

6. The method for optimizing reverse construction parameters of tunnel arch walls based on reinforcement learning according to claim 1, characterized in that: In step three, the risk of concrete layer thickness deviation is determined according to formula (3.9): （3.9）； In the formula: This represents the current layer thickness. The optimal layer thickness is given by engineering experience.

7. The method for optimizing reverse construction parameters of tunnel arch walls based on reinforcement learning according to claim 1, characterized in that: In step four, the Q-learning algorithm is used to construct the reinforcement learning agent, and the update formula for the Q-value function is shown in formula (4.1): （4.1）； In the formula: In the state Select action Value estimation at that time; The actual action chosen by the agent at time t; For the next state Any action from the set of all possible actions; The learning rate; Discount factor; The immediate reward for the current step; in, Determine according to formula (4.2): （4.2）。 8. The method for optimizing reverse construction parameters of tunnel arch walls based on reinforcement learning according to claim 1, characterized in that: In step four, the ε-greedy strategy is used to explore the combination of construction parameters that minimizes overall risk.

9. A tunnel arch wall reverse construction parameter optimization system based on reinforcement learning, characterized in that: include: Data acquisition module: Acquires data on basic engineering parameters, including arch height, concrete unit weight, concrete initial setting time, construction ambient temperature, and allowable lateral pressure of formwork. State and Action Space Construction Module: Based on the reverse construction parameters of the arch wall concrete, a state space and an action space are constructed; the construction parameters include the layer thickness of concrete pouring, the interlayer interval time, and the pumping speed. Reward function construction module: Based on the basic engineering parameters and the reverse construction parameters of the arch wall concrete, a comprehensive risk function is constructed, and a reward function is set according to the comprehensive risk function; the comprehensive risks include concrete cold joint risk, formwork lateral pressure runaway risk, and concrete layer thickness deviation risk. Reinforcement Learning Exploration Module: Constructs a reinforcement learning agent to output the combination of construction parameters that minimizes overall risk; To achieve the method for optimizing reverse construction parameters of tunnel arch walls based on reinforcement learning as described in any one of claims 1-8.

Images