An Optimization Method for Construction Process of Cable-stayed Tension Structures Based on Reinforcement Learning

Abstract

This invention discloses a reinforcement learning-based optimization method for the construction process of cable-stayed tension structures, comprising the following steps: S1, randomly sorting the forming process sequence of the cable-stayed structure multiple times to generate a sample space matrix; S2, constructing a combined optimization model for the forming process of the cable-stayed structure based on reinforcement learning, and inputting the sample space matrix into the combined optimization model; S3, determining the range of various hyperparameters in the training of the combined optimization model, and finding the optimal training hyperparameters using the controlled variable method; S4, solving the sample space matrix using the model with the optimal training hyperparameters, and outputting the optimal forming process sequence. This invention, based on the Q-learning algorithm, explores the trade-off between the steel brace installation sequence and key mechanical performance characteristics during the forming process of the cable-stayed structure. The optimized steel brace installation sequence can reduce the maximum steel brace stress and the maximum cable lifting force, providing convenience for the selection of tooling in the actual forming process.

Description

Technical Field

[0001] This invention relates to the field of intelligent construction of cable-stayed tension structures, and in particular to an optimization method for the construction process of cable-stayed tension structures based on reinforcement learning. Background Technology

[0002] As the steel bracing is installed, the maximum internal force of the cable-stayed structure continuously increases during the forming process. Excessive internal force can pose significant challenges to on-site construction, thus necessitating optimization of the forming process. The forming process of a spatial cable-stayed structure involves several dynamically changing performance parameters: maximum cable unit internal force, maximum steel bracing stress, and maximum cable clamp node displacement. These parameters are closely related to the tonnage of the tooling and the range of the tensioning equipment, the cross-sectional area of ​​the steel bracing, and the lifting height of the cable clamp node, significantly impacting the overall forming scheme's complexity and cost control. The core independent variable in the forming process is the sequence of working conditions, while the relevant performance parameters, as dependent variables, change with the sequence of forming stages.

[0003] In a general sense, multi-objective combinatorial optimization problems involve making optimal decisions when weighing two or more conflicting objectives. Therefore, the optimization problem of the forming process of spatial cable-stayed structures is essentially a sorting problem within multi-objective combinatorial optimization, seeking the optimal balance between structural mechanical performance and tensioning cost by searching for the optimal sequence of forming stages. However, current traditional combinatorial optimization methods, such as scalarized functions and multi-strategy algorithms, involve a large number of iterations and parameter evolutions, requiring powerful computing power, specially tuned algorithms, and considerable time costs. Furthermore, it is difficult to determine and reasonably allocate the contribution ratio of each performance index parameter to the forming process, especially since the uncertainty of each index parameter is closely related to the highly nonlinear characteristics of the cable-stayed structure itself. In the actual design of the forming process of spatial cable-stayed structures, improvements are often made based on the experience of mature tensioning schemes, lacking a systematic and reliable standard to determine whether it is optimal. Summary of the Invention

[0004] Purpose of the invention: The purpose of this invention is to provide a reinforcement learning-based optimization method for the construction process of cable tension structures, which can improve the mechanical properties of the original structure while reducing the difficulty and cost of actual tensioning construction.

[0005] Technical solution: A reinforcement learning-based optimization method for the construction process of cable-stayed tension structures, comprising the following steps:

[0006] S1, the sequence of the cable-stayed structure forming process is randomly sorted multiple times to generate a sample space matrix;

[0007] S2, a combined optimization model for the forming process of cable tension structure is constructed based on reinforcement learning, and the sample space matrix is ​​input into the combined optimization model;

[0008] S3, determine the range of various hyperparameters in the training of the combinatorial optimization model, and find the optimal training hyperparameters by controlling the variable method;

[0009] S4 uses a combined optimization model with optimal training hyperparameters to solve the sample space matrix and outputs the optimal forming process sequence.

[0010] Furthermore, in step S2, the steps for constructing the combined optimization model of the cable-stayed structure forming process are as follows:

[0011] S21, Based on the vector finite element method, a vector finite element model of the cable-stayed structure is constructed, and the encapsulation environment Env is constructed by combining the vector finite element model with the physical variables required in the forming process of the cable-stayed structure.

[0012] S22, select key mechanical performance characteristic parameters in the cable-stayed structure forming process in the state space S, and use them as objective function variables of the combined optimization model; determine the expression of the reward function R by combining multiple characteristic parameters;

[0013] S23, in the action space A, determine the main dependent variables acting on the encapsulation environment Env, and make a judgment based on the research object of the sequence combination optimization of the cable structure forming process to determine the most critical forming process sequence;

[0014] S24. An agent is constructed based on the Q-learning algorithm. The agent interacts with the encapsulated environment Env and dynamically updates its internal value function according to the reward function value, outputting the optimal formation process sequence.

[0015] Furthermore, in step S21, the physical variables required during the cable-stayed structure forming process include: material density ρ, damping coefficient ξ, elastic modulus E, mass coordinates (x, y, z), external load L, constraint conditions C, and initial prestress P.

[0016] Furthermore, in step S22, the key mechanical performance characteristic parameters during the cable-stayed structure forming process include: the maximum stress σ of the steel structure, the maximum displacement d of the mass point, and the maximum internal force F of the cable; the combined weight coefficients of the three characteristic parameters in the calculation formula of the reward function value R under the initial state are set, and the contribution and influence of each characteristic parameter are judged by the training results, and the combined weight coefficients are redistributed; the calculation formula of the reward function value R is as follows:

[0017] R = w1σ + w2d + w3F

[0018] w1 + w2 + w3 = 1

[0019] In the formula, w1, w2, and w3 represent the combined weighting coefficients of the maximum stress σ of the steel structure, the maximum displacement d of the mass point, and the maximum internal force F of the cable, respectively.

[0020] Furthermore, in step S23, the installation sequence data of the steel bracing is collected and preprocessed. Based on the preprocessed data, an action space A is constructed and sent to the agent as a training set for model optimization training.

[0021] Furthermore, a portion of the steel bracing installation sequence data was randomly selected as the test set, and the rest of the data was used as the training set. A K-fold cross-validation strategy was adopted, in which a portion of the training set was used as the validation set for each training session, and the remainder was used as the training set for that session.

[0022] Furthermore, in step S24, the steps for constructing the agent based on the Q-learning algorithm are as follows:

[0023] S241, determine the values ​​of the learning rate α, discount factor γ, and exploration factor ε in the Q-learning algorithm, as well as the number of iterations N and the cycle time step t;

[0024] S242, Initialize the learning reward function R and state space S of the Q-learning algorithm;

[0025] S243, Based on the installation sequence data of the steel bracing in the action space A, calculate the maximum stress σ of the steel structure, the maximum displacement d of the mass point and the maximum internal force F of the cable for the installation sequence of the steel bracing based on the ε-greedy strategy, observe the reward function value and generate a new state space S';

[0026] S244, based on the new state space S', iterate the Q-learning value function according to the Bellman equation and update the state space S, and select the next action from the updated state space S' based on ε-greedy.

[0027] S245, Repeat steps S242-S244 until the reward function value meets the requirements or the maximum cycle period is reached.

[0028] Furthermore, in step S3, the steps for finding the optimal training hyperparameters using the controlled variable method are as follows:

[0029] S31, determine several hyperparameters that affect the training results of the combined optimization model for the cable-stayed structure forming process;

[0030] S32, Select a certain hyperparameter, keep other hyperparameters fixed, and gradually try different values ​​of hyperparameters to conduct model optimization experiments for the same action space A;

[0031] S33: Record the model performance under different values ​​of the same hyperparameter and find the optimal value, fix the optimal value of the hyperparameter; select other hyperparameters and repeat step S32 until all hyperparameters are optimized.

[0032] Compared with the prior art, the significant advantages of this invention are as follows:

[0033] 1. This invention is based on the Q-learning algorithm, which utilizes the interaction between an agent with a clear goal and an uncertain environment and the great potential of handling complex nonlinear and multi-objective optimization problems to explore the trade-off between the installation sequence of steel supports and key mechanical performance parameters in the forming process of cable-stayed structures. While maintaining or even improving the mechanical performance of the original structure, it reduces the difficulty and cost of actual tensioning construction, and provides methodological guidance for optimizing tensioning and forming schemes for similar spatial cable-stayed structures.

[0034] 2. This invention constructs a vector finite element model of a cable-stayed structure based on the vector finite element method. Static analysis is performed on the model using VFIFE, and the static equilibrium result after iterative convergence is used as the initial analysis step for the forming process. This simplifies the complex nonlinear finite element solution to a problem of solving the motion equations of a discrete mass model, significantly reducing convergence time and improving the efficiency and stability of the analysis. The method of this invention maintains consistency with the actual operation sequence and is applicable to the forming process analysis of typical cable-stayed dome structures.

[0035] 3. When training the combined optimization model of cable-stayed structure forming process with steel support installation sequence data, the present invention adopts a K-fold cross-validation strategy. Through multiple cross-validations on the training set, the possibility of overfitting can be reduced, and the model can be prevented from being overfitted on the training set. This can effectively improve the generalization ability of the model, help select the optimal parameter combination, and enable the model to achieve the best performance on the validation set. Attached Figure Description

[0036] Figure 1 This is the overall flowchart of the present invention;

[0037] Figure 2 This is a schematic diagram of the original steel bracing installation sequence of the large-span spatial steel structure roof in this embodiment;

[0038] Figure 3 This is a schematic diagram of the vector finite element model in this embodiment;

[0039] Figure 4 This is a graph showing how the reward function changes over time in this embodiment;

[0040] Figure 5 This is a graph showing the change of the reward function as the cycle progresses under different group combination coefficients in this embodiment. Detailed Implementation

[0041] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0042] This invention proposes an optimization method for the construction process of cable-stayed tension structures based on reinforcement learning, such as... Figure 1 As shown, the specific steps are as follows:

[0043] Step 1: Randomly sort the sequence of the cable-stayed structure forming process multiple times to generate a sample space matrix.

[0044] This embodiment selects the original steel bracing installation sequence of the complex, large-span spatial steel structure roof of a city stadium, such as... Figure 2 As shown, the randperm() function in Matlab is used to randomly sort the order multiple times, and all possible results are treated as a whole sample space [Seq]. n Input the action space A in the reinforcement learning model to prepare the initial data for model training.

[0045] Step 2: Construct a combined optimization model for the cable-stayed structure forming process based on reinforcement learning, and input the sample space matrix into the combined optimization model.

[0046] The steps for constructing a combined optimization model for the forming process of cable-stayed structures are as follows:

[0047] Step 21: Construct a vector finite element model of the cable-stayed structure based on the Vector Finite Element Method (VFIFE). Leveraging the advantages of the VFIFE method in dynamic analysis, a computational framework for the cable-stayed tension structure forming process is established using Matlab. First, the model information is initialized and the forming objective is defined. Static analysis is performed using VFIFE, and the static equilibrium result is used as the starting step for the forming process after iterative convergence. Specific requirements for each working condition are then set sequentially, and the nodal configurations and element internal force vectors for each forming stage are solved, ultimately obtaining all steps of the forward-sequence forming process analysis. For example... Figure 3 As shown, this embodiment uses VFIFE modeling in MATLAB 2022b. The preprocessing module inputs physical quantities such as node coordinates, element type and cross-sectional area, connection sequence, load parameters, prestress, and constraints to build a physical model of the cable-stayed structure for VFIFE analysis in the MATLAB environment. An encapsulated environment (Env) is constructed by combining the vector finite element model with the physical variables required during the cable-stayed structure forming process. In this embodiment, the physical variables required during the cable-stayed structure forming process include: material density ρ, damping coefficient ξ, elastic modulus E, mass coordinates (x, y, z), external load L, constraint condition C, and initial prestress P.

[0048] Step 22: Select key mechanical performance characteristic parameters in the state space S during the cable-stayed structure forming process. These key mechanical performance characteristic parameters include: maximum stress σ of the steel structure, maximum displacement d of the mass point, and maximum internal force F of the cable. These parameters are used as the objective function variables of the combined optimization model. The expression of the reward function R is determined by combining multiple characteristic parameters. The process of determining the reward function R is as follows: Set the combined weight coefficients of the three characteristic parameters in the formula of the reward function R under the initial state. Determine the contribution and influence of each characteristic parameter based on the training results, and redistribute the combined weight coefficients. The formula of the reward function R is as follows:

[0049] R=w1σ+w2d+w3F (1)

[0050] w1+w2+w3=1 (2)

[0051] In the formula, w1, w2, and w3 represent the combined weighting coefficients of the maximum stress σ of the steel structure, the maximum displacement d of the mass point, and the maximum internal force F of the cable, respectively.

[0052] Step 23: In the action space A, determine the main dependent variables acting on the encapsulation environment Env, and make a judgment based on the research object of the sequence combination optimization of the cable-stayed structure forming process to determine the most critical forming process sequence.

[0053] In this embodiment, the installation sequence of the steel supports is selected as the primary forming sequence. From the overall sample space [Seq] of action space A. n A number of samples are selected and sent to the agent as the training set for model optimization training. A portion of the samples are randomly selected as the test set, and the remaining data is used as the training set. A K-fold cross-validation strategy is adopted, in which a portion of the training set is used as the validation set for each training iteration, and the remainder is used as the training set for that iteration.

[0054] Step 24: Construct an agent based on the Q-learning algorithm. The agent interacts with the encapsulation environment (Env) and dynamically updates its internal value function according to the value of the reward function, outputting the optimal formation process sequence.

[0055] The steps for constructing an agent based on the Q-learning algorithm are as follows:

[0056] Step 241: Determine the values ​​of the learning rate α, discount factor γ, and exploration factor ε in the Q-learning algorithm, as well as the number of iterations N and the cycle time step t.

[0057] Step 242: Initialize the learning reward function R and state space S of the Q-learning algorithm.

[0058] Step 243: Based on the steel bracing installation sequence data sample in action space A, calculate the maximum stress σ of the steel structure, the maximum displacement d of the mass point, and the maximum internal force F of the cable for the steel bracing installation sequence using the ε-greedy strategy, observe the reward function value, and generate a new state space S'.

[0059] Step 244: Based on the new state space S', iterate the Q-learning value function according to the Bellman equation and update the state space S'. Select the next action from the updated state space S' based on the ε-greedy policy.

[0060] Step 245: Repeat steps 242-244 until the reward function value meets the requirements or the maximum cycle period is reached.

[0061] Step 3: Determine the range of various hyperparameters in the training of the combinatorial optimization model, and find the optimal training hyperparameters using the controlled variable method. The steps for finding the optimal training hyperparameters using the controlled variable method are as follows:

[0062] Step 31: Determine several hyperparameters that affect the training results of the combined optimization model for the cable-stayed structure forming process.

[0063] Step 32: Select a certain hyperparameter, keep the other hyperparameters fixed, and gradually try different values ​​of hyperparameters to conduct model optimization experiments for the same action space A.

[0064] Step 33: Record the model performance under different values ​​of the same hyperparameter and find the optimal value, fix the optimal value of the hyperparameter; select other hyperparameters and repeat the above steps until all hyperparameters are optimized.

[0065] The process of optimizing the hyperparameters for model training in this embodiment is as follows: To avoid randomness in training, the combined optimization model of the cable-stayed structure forming process is trained three times at different numbers of cycles. The mean of the reward function values ​​is taken as the ordinate, and the number of cycles is taken as the abscissa. The change of the reward function with the progression of cycles during the training process is as follows: Figure 4 As shown. When the number of periods N = 10 or 20, the reward function value fluctuates significantly with the progress of the periods, indicating that the trial-and-error search effect is not ideal and a good delayed reward is not obtained. When the number of periods N = 50, from Figure 4 It can be seen that after approximately 35 cycles, the reward function value gradually stabilizes and converges to a maximum value, so there is no need to increase the number of cycles to ensure maximizing the expected reward. Therefore, the experiment determined that the number of cycles N for subsequent training should be 50, ensuring that the reward function converges to a maximum value while also improving model training efficiency.

[0066] This embodiment optimizes the combination of three characteristic parameters in the state space S by rationally allocating their combination coefficients to reflect the proportion of each characteristic parameter's contribution to the forming process. It identifies the most critical characteristic parameters in the actual tensioning process and the key difficulties they represent in the construction process. According to the definition of multi-objective combination optimization, the sum of the combination coefficients of each characteristic parameter is 1. That is, the sum of the combination coefficient k1 representing the maximum steel bracing stress, the combination coefficient k2 representing the maximum cable clamp node displacement, and the combination coefficient k3 representing the maximum cable lifting force is 1. This embodiment designates combination coefficients k1, k2, and k3 as the main contributing factors, and the rest as secondary contributing factors. The proposed allocation of combination coefficients is shown in Table 1.

[0067] Table 1. Determined combination coefficients of characteristic parameters in state space S

[0068] definition Proposed combination 1 Proposed combination 2 Proposed combination 3 <![CDATA[Combination coefficient κ1 of maximum steel diagonal brace stress]]> 0.6 0.2 0.2 <![CDATA[Combination coefficient κ2 of the maximum displacement of the cable clamp node]]> 0.2 0.6 0.2 <![CDATA[Combination coefficient κ3 of maximum cable hoisting force]]> 0.2 0.2 0.6

[0069] Based on the three sets of proposed combination coefficients in Table 1, and taking the number of cycles N = 50, these coefficients are substituted into the reinforcement learning model for training. The change of the reward function values ​​of different groups with the progression of cycles is as follows: Figure 5 As shown. By Figure 5 It can be seen that the reward function value output by pseudo-combination 2 as the state space S is greater than that of pseudo-combination 1 and pseudo-combination 3. That is, the hyperparameter combination with the combination coefficient of the maximum cable clamp node displacement as the main contributing factor has the ability to maximize the reward function value under the same conditions. In the process of cable tension structure forming, it is more important and effective to control the maximum cable clamp node displacement than to control the maximum steel brace stress and the maximum cable lifting force.

[0070] In this embodiment, the number of model training cycles N, which balances efficiency and accuracy, and the optimal combination coefficients k1, k2, and k3 of the feature parameters in the state space are determined sequentially using the controlled variable method. These, along with the default hyperparameters, form the optimal training hyperparameter combination. Using this optimal training hyperparameter combination, the dynamic feedback of the reward function value, state space, and action space can better guide the agent's interaction with the environment. The optimal hyperparameter values ​​for model training in this embodiment are shown in Table 2 below.

[0071] Table 2 Optimal Training Hyperparameter Combinations

[0072] definition symbol Value Learning rate α 0.1 Discount factor γ 0.9 Exploration Factors ε 0.1 Action space dimension n 100 cycle N 40 Combination coefficient 1 - Maximum steel brace stress <![CDATA[κ1]]> 0.2 Combination coefficient 2 - Maximum cable clamp node displacement <![CDATA[κ2]]> 0.6 Combination coefficient 3 - maximum cable lifting force <![CDATA[κ3]]> 0.2

[0073] Step 4: Use the combined optimization model of the optimal training hyperparameters to solve the sample space matrix and output the optimal forming process sequence.

[0074] This embodiment incorporates the original steel bracing installation sequence of the complex, large-span spatial steel structure roof of a city stadium into the model for verification. The original steel bracing installation sequence is as follows:

[0075] [Seq] design =[27,35,34,28,29,33,32,30,31,7,15,14,8,13,12,10,11]

[0076] By fully interacting with the environment using the agent stored under the optimal training hyperparameter combination, the optimal steel bracing installation sequence can be output [Seq]. prediction for:

[0077] [Seq] prediction =[7,8,10,11,13,12,14,27,15,28,29,30,31,33,32,35,34]

[0078] According to the optimal steel bracing installation sequence, the maximum steel bracing stress, the maximum ring cable clamp node position coordinates, and the maximum cable internal force of each component unit on each axis during the forming process were obtained after reinforcement learning optimization. These results were compared with the simulation analysis results of the vector finite element model of the cable-stayed structure through VFIFE. The specific error results are detailed in Table 3.

[0079] Table 3. Differences between VFIFE analysis results and reinforcement learning optimization results

[0080]

[0081]

[0082] As shown in Table 3, after optimization through reinforcement learning, the maximum steel brace stress at each axial position is 1.3% to 6.9% lower than the analysis results of VFIFE. Similarly, the internal force of each cable is 1% to 5.2% lower than the calculation results of VFIFE. This indicates that the optimized installation sequence of the steel braces effectively achieves the optimization requirements of the two secondary objectives in the multi-objective combined optimization, reduces the maximum steel brace stress and the maximum cable lifting force, and provides convenience for the tooling selection in the actual forming process.

[0083] As an optimization result of reinforcement learning, the absolute values ​​of the maximum differences in displacement changes of the ring cable clamp nodes in the X, Y, and Z directions are all smaller than the displacement changes calculated by VFIFE, indicating that reinforcement learning is effective in this type of nonlinear combinatorial optimization problem. The reinforcement learning-based optimization method for the forming process of cable tension structures proposed in this embodiment can be applied to the optimization of a certain forming stage in actual engineering, and achieves the expected results in the optimization of secondary objectives in multi-objective combinatorial optimization, with relatively uniform result distribution.

[0084] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for optimizing the construction process of cable-stayed tension structures based on reinforcement learning, characterized in that, Includes the following steps: S1, the sequence of the cable-stayed structure forming process is randomly sorted multiple times to generate a sample space matrix; S2, a combined optimization model for the forming process of cable tension structure is constructed based on reinforcement learning, and the sample space matrix is ​​input into the combined optimization model; S3, determine the range of various hyperparameters in the training of the combinatorial optimization model, and find the optimal training hyperparameters by controlling the variable method; S4 uses a combined optimization model with optimal training hyperparameters to solve the sample space matrix and outputs the optimal forming process sequence; In step S2, the steps for constructing the combined optimization model of the cable-stayed structure forming process are as follows: S21, Based on the vector finite element method, a vector finite element model of the cable-stayed structure is constructed. An encapsulation environment is then built by combining the vector finite element model with the physical variables required during the cable-stayed structure forming process. ; S22, select key mechanical performance characteristic parameters in the cable-stayed structure forming process in the state space S, and use them as objective function variables of the combined optimization model; determine the expression of the reward function R by combining multiple characteristic parameters; S23, determine the action to be performed on the encapsulation environment in action space A. The main dependent variables are determined based on the research object of the sequence combination optimization of the cable-stayed structure forming process, and the most critical forming process sequence is determined. S24, Construct an agent based on the Q-learning algorithm, and then connect the agent with the encapsulation environment. It interacts with the system and dynamically updates its internal value function based on the reward function value, outputting the optimal formation process sequence.

2. The reinforcement learning-based method for optimizing the construction process of cable-stayed tension structures according to claim 1, characterized in that, In step S21, the physical variables required during the forming process of the cable-stayed structure include: material density. Damping coefficient ξ, elastic modulus E, mass coordinates (x, y, z), external load L, constraint condition C, and initial prestress P.

3. The reinforcement learning-based method for optimizing the construction process of cable-stayed tension structures according to claim 1, characterized in that, In step S22, the key mechanical performance characteristics during the cable-stayed structure forming process include: the maximum stress of the steel structure. The maximum displacement d of the mass and the maximum internal force F of the cable are determined. The combined weight coefficients of the three feature parameters in the reward function value R are set in the initial state. The contribution and influence of each feature parameter are judged based on the training results, and the combined weight coefficients are reallocated. The formula for calculating the reward function value R is as follows: In the formula, , , These represent the maximum stress of the steel structure. The combined weighting coefficient of the maximum displacement d of the mass and the maximum internal force F of the cable.

4. The reinforcement learning-based method for optimizing the construction process of cable-stayed tension structures according to claim 1, characterized in that, In step S23, the installation sequence data of the steel bracing is collected and preprocessed. Based on the preprocessed data, an action space A is constructed and sent to the agent as a training set for model optimization training.

5. The reinforcement learning-based method for optimizing the construction process of cable-stayed tension structures according to claim 4, characterized in that, A portion of the steel bracing installation sequence data is randomly selected as the test set, and the rest is used as the training set. A K-fold cross-validation strategy is adopted, in which a portion of the training set is used as the validation set for each training iteration, and the remainder is used as the training set for that iteration.

6. The reinforcement learning-based method for optimizing the construction process of cable-stayed tension structures according to claim 1, characterized in that, In step S24, the steps for constructing the agent based on the Q-learning algorithm are as follows: S241, Determine the learning rate in the Q-learning algorithm. Discount Factor , Exploration factors in strategy The values ​​of N and the cycle time step t; S242, Initialize the learning reward function R and state space S of the Q-learning algorithm; S243, based on the installation sequence data of the steel diagonal braces in motion space A, based on Strategy calculation of the maximum stress of the steel structure based on the installation sequence of the steel bracing Observe the maximum displacement d of the mass and the maximum internal force F of the cable, and generate a new state space S'. S244, based on the new state space S', iterates the Q-learning value function according to the Bellman equation and updates the state space S. Choose the next action from the updated state space S'; S245, Repeat steps S242-S244 until the reward function value meets the requirements or the maximum cycle period is reached.

7. The reinforcement learning-based construction process optimization method for cable-stayed tension structures according to claim 1, characterized in that, In step S3, the steps for finding the optimal training hyperparameters using the controlled variable method are as follows: S31, determine several hyperparameters that affect the training results of the combined optimization model for the cable-stayed structure forming process; S32, Select a certain hyperparameter, keep other hyperparameters fixed, and gradually try different values ​​of hyperparameters to conduct model optimization experiments for the same action space A; S33: Record the model performance under different values ​​of the same hyperparameter and find the optimal value, fix the optimal value of the hyperparameter; select other hyperparameters and repeat step S32 until all hyperparameters are optimized.

Images