Reinforcement learning-based cable-strut tension structure construction process optimization method

Abstract

The invention discloses a reinforcement learning-based cable-strut tension structure construction process optimization method, which comprises the following steps of S1, performing multiple random sorting on a cable-strut structure forming process sequence to generate a sample space matrix; s2, constructing a cable-strut structure forming process combinatorial optimization model based on reinforcement learning, and inputting the sample space matrix into the combinatorial optimization model; s3, determining the range of various hyper-parameters in combined optimization model training, and searching an optimal training hyper-parameter through a control variable method; and S4, solving the sample space matrix by using the optimal training hyper-parameter model, and outputting an optimal forming process sequence. Based on the Q-learning algorithm, the tradeoff between the mounting sequence of the steel braces and key mechanical property characteristic parameters in the forming process of the cable-strut structure is explored, the optimized mounting sequence of the steel braces can reduce the maximum stress of the steel braces and the maximum lifting force of the inhaul cable, and convenience is provided for tool selection in the actual forming process.

Description

Technical Field

[0001] The present invention relates to the field of intelligent construction of cable-strut tension structures, and in particular to a method for optimizing the construction process of a cable-strut tension structure based on reinforcement learning. Background Art

[0002] With the installation of steel diagonal braces, the maximum internal force of the cable-rod tension structure is increasing during the forming process. Excessive internal force will bring considerable difficulties to the actual construction on site, so there is a need to optimize the forming process. During the forming process of the spatial cable-rod structure, there are several dynamically changing performance index parameters: maximum cable unit internal force, maximum steel brace stress, maximum cable clamp node displacement, etc. The above parameters are closely related to the tooling tonnage and tensioning equipment range, steel brace cross-sectional area, cable clamp node lifting height, etc., which have an important impact on the difficulty and cost control of the overall forming solution. The core independent variable of the forming process is the working condition sequence. The relevant performance index parameters, as dependent variables, change with the change of the forming stage sequence.

[0003] In a broad sense, the multi-objective combinatorial optimization problem is to make the best decision when weighing between two or more conflicting objectives. Therefore, the optimization problem of the forming process of the spatial cable-bar structure is essentially a sorting problem in the multi-objective combinatorial optimization problem. By searching for the optimal forming stage sequence, the optimal balance point between the structural mechanical properties and the tension forming cost is found. However, the current traditional combinatorial optimization methods, such as scalarization functions and multi-strategy algorithms, involve a large number of iterations and parameter evolutions, which require powerful computing power, specially tuned algorithms, and considerable time costs; and it is difficult to judge and reasonably allocate the contribution ratio of each performance index parameter to the forming process, especially the uncertainty of each index parameter is closely related to the highly nonlinear characteristics of the cable-bar structure itself; in the actual design process of the spatial cable-bar structure forming process combination, it is often improved based on the experience of mature tensioning schemes, and there is a lack of a systematic and reliable standard to determine whether it is optimal. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a method for optimizing the construction process of a cable-strut tension structure based on reinforcement learning, so as to improve the mechanical properties of the original structure while reducing the difficulty and cost of actual tensioning construction.

[0005] Technical solution: A reinforcement learning-based cable-strut tension structure construction process optimization method, comprising the following steps:

[0006] S1, randomly sort the order of the cable-strut structure forming process multiple times to generate a sample space matrix;

[0007] S2, constructing a combined optimization model for the forming process of the cable-strut tension structure based on reinforcement learning, and inputting the sample space matrix into the combined optimization model;

[0008] S3, determining the range of various hyperparameters in the training of the combinatorial optimization model, and finding the optimal training hyperparameters by the control variable method;

[0009] S4, using the combined optimization model of the optimal training hyperparameters to solve the sample space matrix and output the optimal forming process sequence.

[0010] Furthermore, in step S2, the steps of constructing a combined optimization model of the cable-rod structure forming process are as follows:

[0011] S21, construct a vector finite element model of the cable-bar structure based on the vector finite element method, and construct an encapsulation environment Env by combining the vector finite element model with the physical variables required in the forming process of the cable-bar structure;

[0012] S22, selecting key mechanical performance characteristic parameters in the cable-rod structure forming process in the state space S and using them as objective function variables of the combined optimization model; determining the expression of the reward function R by combining multiple characteristic parameters;

[0013] S23, determine the main dependent variables acting on the encapsulation environment Env in the action space A, make a judgment based on the research object of the sequential combinatorial optimization of the cable-rod structure forming process, and determine the most critical forming process sequence;

[0014] S24, builds an intelligent agent based on the Q-learning algorithm, interacts with the encapsulation environment Env through the intelligent agent, and dynamically updates its own internal value function according to the reward function value, and outputs the optimal forming process sequence.

[0015] Further, in step S21, the physical variables required in the cable-strut structure forming process include: material density ρ, damping coefficient ξ, elastic modulus E, particle coordinates (x, y, z), external load L, constraint condition C and initial prestress P.

[0016] Further, in step S22, the key mechanical performance characteristic parameters in the cable-rod structure forming process include: maximum stress σ of the steel structure, maximum displacement d of the mass point, and maximum internal force F of the cable; the combined weight coefficients of the three characteristic parameters in the reward function value R calculation formula under the initial state are set, and the contribution influence of each characteristic parameter is judged by the training results, and the combined weight coefficients are reallocated; the calculation formula of the reward function value R is as follows:

[0017] R=w1σ+w2d+w3F

[0018] w1+w2+w3=1

[0019] Where w1, w2, and w3 represent the combined weight coefficients of the maximum stress σ of the steel structure, the maximum displacement d of the mass point, and the maximum internal force F of the cable, respectively.

[0020] Further, in step S23, the steel brace installation sequence data is collected and preprocessed, and the action space A is constructed according to the preprocessed data, and sent to the intelligent agent Agent as a training set for model optimization training.

[0021] Furthermore, a part of the steel brace installation sequence data is randomly selected as the test set, and the other data is used as the training set; the K-fold cross-validation strategy is adopted, and a part of the training set is taken as the validation set for each training, and the rest is used as the training set for that training.

[0022] Further, in step S24, the steps of constructing an intelligent agent based on the Q-learning algorithm are as follows:

[0023] S241, determining the learning rate α, the discount factor γ, the exploration factor ε in the ε-greedy strategy, the number of cycles N, and the cycle time step t in the Q-learning algorithm;

[0024] S242, initializing the learning reward function R and state space S of the Q-learning algorithm;

[0025] S243, according to the steel brace installation sequence data in the action space A, the maximum stress σ of the steel structure, the maximum displacement d of the mass point and the maximum internal force F of the cable of the steel brace installation sequence are calculated based on the ε-greedy strategy, the reward function value is observed and a new state space S' is generated;

[0026] S244, based on the new state space S', iterate the Q-learning value function according to the Bellman equation and update the state space S, and select the next action from the updated state space S' based on ε-greedy;

[0027] S245, repeat steps S242-S244 until the reward function value meets the requirements or reaches the maximum cycle period.

[0028] Further, in step S3, the steps of finding the optimal training hyperparameters by the control variable method are as follows:

[0029] S31, determining several hyperparameters that affect the training results of the combined optimization model of the cable-strut structure forming process;

[0030] S32, select a certain hyperparameter, keep other hyperparameters fixed, and gradually try hyperparameters with different values ​​to perform model optimization experiments on the same action space A;

[0031] S33, record the model performance under different values ​​of the same hyperparameter and find the optimal value, fix the optimal value of the hyperparameter; select other hyperparameters and repeat step S32 until all hyperparameters are optimized.

[0032] Compared with the prior art, the present invention has the following significant effects:

[0033] 1. Based on the Q-learning algorithm, the present invention utilizes its great potential for interaction between intelligent agents with clear goal guidance and uncertain environments and for processing complex nonlinear and multi-objective optimization problems, and explores the trade-off between the installation sequence of steel supports and key mechanical performance characteristic parameters in the cable-strut structure forming process; while maintaining or even improving the mechanical properties of the original structure, the actual tensioning construction difficulty and cost are reduced, and methodological guidance is provided for the optimization of tensioning forming schemes for similar spatial cable-strut structures;

[0034] 2. The present invention constructs a vector finite element model of the cable-rod structure based on the vector finite element method, performs static analysis on the model through VFIFE, and uses the static equilibrium result as the starting analysis step of the forming process after iterative convergence; the complex nonlinear finite element solution is simplified to the motion equation solution problem of the discrete particle model, which greatly reduces the convergence time and improves the efficiency and stability of the analysis. The method of the present invention is consistent with the actual operation sequence and is suitable for the forming process analysis of a typical cable dome structure;

[0035] 3. The present invention adopts a K-fold cross-validation strategy when inputting steel support installation sequence data into the combined optimization model of the cable-rod structure forming process for training; multiple cross-validations of the training set can reduce the possibility of overfitting and avoid the situation where the model is overfitted on the training set; it can effectively improve the generalization ability of the model, help select the optimal parameter combination, and enable the model to achieve the best performance on the validation set. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is the overall flow chart of the present invention;

[0037] Figure 2 Schematic diagram of the original steel diagonal brace installation sequence of the large-span space steel structure roof in this embodiment;

[0038] Figure 3 is a schematic diagram of a vector finite element model in this embodiment;

[0039] Figure 4 is a graph showing how the reward function in this embodiment changes as the cycle progresses;

[0040] Figure 5 Graph showing how the reward function under different sets of quasi-combination coefficients in this embodiment changes as the cycle progresses. DETAILED DESCRIPTION

[0041] The present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0042] The present invention proposes a method for optimizing the construction process of a cable-strut tension structure based on reinforcement learning. Figure 1 As shown, the specific steps are as follows:

[0043] Step 1: Randomly sort the forming process sequence of the cable-strut structure multiple times to generate a sample space matrix.

[0044] This embodiment selects the original steel diagonal brace installation sequence of the complex large-span spatial steel structure roof of a certain city stadium, such as Figure 2 As shown, the order is randomly sorted multiple times using the randperm() function in Matlab, and all the possible results above are taken as a whole sample space [Seq] n , input the action space A in the reinforcement learning model to prepare the initialization data for model training.

[0045] Step 2: construct a combinatorial optimization model for the cable-strut structure forming process based on reinforcement learning, and input the sample space matrix into the combinatorial optimization model.

[0046] The steps to construct the combined optimization model of the cable-strut structure forming process are as follows:

[0047] Step 21, construct a vector finite element model of the cable-strut structure based on the vector finite element method (VFIFE), use the vector finite element method to show the advantages of dynamic analysis, and establish a calculation framework for the cable-strut tension structure forming process based on Matlab. First, initialize the model information and clarify the forming target. Perform static analysis through VFIFE, and use the static equilibrium result as the starting analysis step of the forming process after iterative convergence. Set the specific requirements of each working condition in turn, solve the node configuration and unit internal force vector of each forming stage, and finally obtain all the steps of the forward forming process analysis. Figure 3 As shown, this embodiment is modeled in MATLAB2022b through VFIFE, and physical quantities such as node coordinates, unit type and cross-sectional area, connection sequence, load parameters, prestress and constraints are input through the preprocessing module to build a physical model of the cable-bar structure for VFIFE analysis in the MATLAB environment. The encapsulation environment Env is constructed by combining the vector finite element model with the physical variables required in the forming process of the cable-bar structure. In this embodiment, the physical variables required in the forming process of the cable-bar structure include: material density ρ, damping coefficient ξ, elastic modulus E, particle coordinates (x, y, z), external load L, constraint condition C, and initial prestress P.

[0048] Step 22, select the key mechanical performance characteristic parameters in the cable-strut structure forming process in the state space S. The key mechanical performance characteristic parameters in the cable-strut structure forming process include: maximum stress σ of the steel structure, maximum displacement d of the mass point and maximum internal force F of the cable, and use them as the objective function variables of the combined optimization model. Determine the expression of the reward function R by combining multiple characteristic parameters. The determination process of the reward function R: set the combined weight coefficients of the three characteristic parameters in the reward function R formula in the initial state, judge the contribution and influence of each characteristic parameter through the training results, and redistribute the combined weight coefficients. The formula of the reward function R is as follows:

[0049] R=w1σ+w2d+w3F (1)

[0050] w1+w2+w3=1 (2)

[0051] Where w1, w2, and w3 represent the combined weight coefficients of the maximum stress σ of the steel structure, the maximum displacement d of the mass point, and the maximum internal force F of the cable, respectively.

[0052] Step 23, determine the main dependent variables acting on the packaging environment Env in the action space A, make a judgment based on the research object of the sequential combinatorial optimization of the cable-rod structure forming process, and determine the most critical forming process sequence.

[0053] In this embodiment, the installation order of the steel support is selected as the most important forming order. From the overall sample space of the action space A [Seq] n Select several samples from the dataset and send them to the agent as the training set for model optimization training. Randomly select a part of the samples as the test set, and the rest as the training set. Use the K-fold cross-validation strategy, and take a part of the training set as the validation set for each training, and the rest as the training set for this training.

[0054] Step 24, construct an intelligent agent based on the Q-learning algorithm, interact with the encapsulation environment Env through the intelligent agent, and dynamically update its own internal value function according to the value of the reward function, and output the optimal forming process sequence.

[0055] The steps to build an agent based on the Q-learning algorithm are as follows:

[0056] Step 241, determine the learning rate α in the Q-learning algorithm, the discount factor γ, the exploration factor ε in the ε-greedy strategy, the number of cycles N, and the cycle time step t.

[0057] Step 242, initialize the learning reward function R and state space S of the Q-learning algorithm.

[0058] Step 243, according to the data sample of the steel brace installation sequence in the action space A, the maximum stress σ of the steel structure, the maximum displacement d of the mass point and the maximum internal force F of the cable of the steel brace installation sequence are calculated based on the ε-greedy strategy, the reward function value is observed and a new state space S' is generated.

[0059] Step 244, based on the new state space S', iterate the Q-learning value function according to the Bellman equation and update the state space S', and select the next action from the updated state space S' based on the ε-greedy strategy.

[0060] Step 245, repeat steps 242 to 244 until the reward function value meets the requirement or reaches the maximum cycle period.

[0061] Step 3: Determine the range of various hyperparameters in the combined optimization model training and find the optimal training hyperparameters by the control variable method. The steps to find the optimal training hyperparameters by the control variable method are as follows:

[0062] Step 31, determining several hyperparameters that have an impact on the training results of the combined optimization model of the cable-strut structure forming process.

[0063] Step 32, select a certain hyperparameter, fix other hyperparameters, and gradually try hyperparameters with different values ​​to perform model optimization experiments on the same action space A.

[0064] Step 33, record the model performance under different values ​​of the same hyperparameter and find the optimal value, fix the optimal value of the hyperparameter; select other hyperparameters and repeat the above steps until all hyperparameters are optimized.

[0065] The process of optimizing the model training hyperparameters in this embodiment is as follows: In order to avoid the randomness of training, the cable-rod structure forming process combined optimization model is repeatedly trained three times at different cycle numbers, and the mean of the reward function value is taken as the ordinate and the cycle number as the abscissa. The change of the reward function with the progress of the cycle during the training process is shown in the figure. Figure 4 As shown in the figure. When the number of cycles N = 10 or 20, the reward function value fluctuates greatly with the progress of the cycle, proving that the trial and error search effect is not ideal and a good delayed benefit is not obtained. When the number of cycles N = 50, Figure 4 It can be seen that after about 35 cycles, the reward function value gradually stabilizes and converges to a maximum value, so there is no need to increase the number of cycles to ensure the maximum return expectation. From this experiment, the number of cycles N for subsequent training is determined to be 50, which ensures that the reward function converges to a maximum value and also improves the model training efficiency.

[0066] This embodiment reflects the contribution ratio of each characteristic parameter to the forming process by reasonably allocating the combination coefficients of the three, determines the characteristic parameters that need to be paid most attention to in the actual tension forming process and the key and difficult points of the construction process represented by them, and optimizes the combination of the three characteristic parameters in the state space S. According to the definition of multi-objective combination optimization, the sum of the combination coefficients of each characteristic parameter is 1, that is, the sum of the combination coefficient k1 representing the maximum steel brace stress, the combination coefficient k2 representing the maximum cable clamp node displacement, and the combination coefficient k3 representing the maximum cable lifting force is 1. This embodiment specifies the combination coefficients k1, k2 and k3 as the main contribution influencing factors, and the rest are secondary contribution influencing factors. The proposed allocation of combination coefficients is shown in Table 1:

[0067] Table 1 Proposed combination coefficients of characteristic parameters in state space S

[0068] definition Proposed combination 1 Proposed combination 2 Proposed combination 3 <![CDATA[Combination coefficient κ1 of maximum steel diagonal brace stress]]> 0.6 0.2 0.2 <![CDATA[Combination coefficient κ2 of the maximum displacement of the cable clamp node]]> 0.2 0.6 0.2 <![CDATA[Combination coefficient κ3 of maximum cable hoisting force]]> 0.2 0.2 0.6

[0069] According to the three groups of quasi-combination coefficients in Table 1, we take the number of cycles N = 50 and substitute them into the reinforcement learning model for training. The reward function values ​​of different groups change as the cycle progresses. Figure 5 As shown. Figure 5 It can be seen that the reward function value with quasi-combination 2 as the output of state space S is greater than that of quasi-combination 1 and quasi-combination 3, that is, the hyperparameter combination with the combination coefficient of the maximum cable clamp node displacement as the main contribution influencing factor has the ability to maximize the reward function value under the same conditions. In the forming process of the cable-strut tension structure, it is more important and effective to determine the control of the maximum cable clamp node displacement than to control the maximum steel brace stress and the maximum cable lifting force.

[0070] This embodiment uses the control variable method to determine the number of model training cycles N that take into account both efficiency and accuracy, as well as the optimal combination coefficients k1, k2 and k3 of the characteristic parameters in the state space, and together with the default value hyperparameters, they form the optimal training hyperparameter combination. Using the optimal training hyperparameter combination, the dynamic feedback of the reward function value, state space, and action space can better guide the agent to interact with the environment. The optimal hyperparameter values ​​for model training in this embodiment are shown in Table 2 below.

[0071] Table 2 Optimal training hyperparameter combinations

[0072] definition symbol Value Learning Rate α 0.1 Discount Factor γ 0.9 Exploration Factor ε 0.1 Action space dimensions n 100 cycle N 40 Combination coefficient 1-maximum steel brace stress <![CDATA[κ1]]> 0.2 Combination coefficient 2-maximum cable clamp node displacement <![CDATA[κ2]]> 0.6 Combination factor 3 - Maximum cable lifting force <![CDATA[κ3]]> 0.2

[0073] Step 4: Use the combined optimization model of the optimal training hyperparameters to solve the sample space matrix and output the optimal forming process sequence.

[0074] This example brings the original steel brace installation sequence of the complex large-span spatial steel structure roof of a certain city stadium into the model verification. The original steel brace installation sequence is:

[0075] [Seq] design =[27,35,34,28,29,33,32,30,31,7,15,14,8,13,12,10,11]

[0076] The intelligent agent saved under the optimal training hyperparameter combination interacts fully with the environment to output the optimal steel brace installation sequence [Seq] prediction for:

[0077] [Seq] prediction =[7,8,10,11,13,12,14,27,15,28,29,30,31,33,32,35,34]

[0078] According to the optimal steel brace installation sequence, the maximum steel brace stress, the maximum ring cable clamp node position coordinates and the maximum cable internal force of the component unit on each axis during the forming process after reinforcement learning optimization are obtained, and compared with the simulation analysis results of the vector finite element model of the cable-strut structure through VFIFE. The specific error results are shown in Table 3.

[0079] Table 3 Difference between VFIFE analysis results and reinforcement learning optimization results

[0080]

[0081]

[0082] It can be seen from Table 3 that after the steel brace stress is optimized by reinforcement learning, the maximum steel brace stress at each axis position is between 1.3% and 6.9% smaller than the analysis result of VFIFE. Similarly, the internal force of each cable is between 1% and 5.2% smaller than the calculation result of VFIFE, indicating that the optimized steel brace installation sequence has better achieved the optimization requirements of the two secondary objectives in the multi-objective combined optimization, reduced the maximum steel brace stress and the maximum cable lifting force, and provided convenience for the selection of tooling in the actual forming process.

[0083] As the optimization result of reinforcement learning, the absolute value of the maximum difference of the displacement change of the ring cable clamp node in the three directions of X, Y and Z is smaller than the displacement change calculated by VFIFE, indicating that reinforcement learning is effective in this type of nonlinear combination optimization problem. The reinforcement learning-based cable-strut tension structure forming process optimization method proposed in this embodiment can be applied to the optimization of a certain forming stage of an actual project, and achieves the expected effect in the optimization of the secondary objectives of the multi-objective combination optimization, and the result distribution is relatively uniform.

[0084] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for optimizing the construction process of a cable-strut tension structure based on reinforcement learning, characterized in that: The following steps are involved: S1, randomly sort the order of the cable-strut structure forming process multiple times to generate a sample space matrix; S2, constructing a combined optimization model for the forming process of the cable-strut tension structure based on reinforcement learning, and inputting the sample space matrix into the combined optimization model; S3, determining the range of various hyperparameters in the training of the combinatorial optimization model, and finding the optimal training hyperparameters by the control variable method; S4, using the combined optimization model of the optimal training hyperparameters to solve the sample space matrix and output the optimal forming process sequence.

2. The method for optimizing the construction process of a cable-strut tension structure based on reinforcement learning according to claim 1, characterized in that: In step S2, the steps of constructing the combined optimization model of the cable-strut structure forming process are as follows: S21, construct a vector finite element model of the cable-bar structure based on the vector finite element method, and construct an encapsulation environment Env by combining the vector finite element model with the physical variables required in the forming process of the cable-bar structure; S22, selecting key mechanical performance characteristic parameters in the cable-rod structure forming process in the state space S and using them as objective function variables of the combined optimization model; determining the expression of the reward function R by combining multiple characteristic parameters; S23, determine the main dependent variables acting on the encapsulation environment Env in the action space A, make a judgment based on the research object of the sequential combinatorial optimization of the cable-rod structure forming process, and determine the most critical forming process sequence; S24, builds an intelligent agent based on the Q-learning algorithm, interacts with the encapsulation environment Env through the intelligent agent, and dynamically updates its own internal value function according to the reward function value, and outputs the optimal forming process sequence.

3. The method for optimizing the construction process of a cable-strut tension structure based on reinforcement learning according to claim 2, characterized in that: In step S21, the physical variables required in the cable-strut structure forming process include: material density ρ, damping coefficient ξ, elastic modulus E, particle coordinates (x, y, z), external load L, constraint condition C and initial prestress P.

4. The method for optimizing the construction process of a cable-strut tension structure based on reinforcement learning according to claim 2, characterized in that: In step S22, the key mechanical performance characteristic parameters in the cable-strut structure forming process include: the maximum stress σ of the steel structure, the maximum displacement d of the mass point and the maximum internal force F of the cable; the combined weight coefficients of the three characteristic parameters in the reward function value R calculation formula in the initial state are set, and the contribution influence of each characteristic parameter is judged by the training results, and the combined weight coefficients are reallocated; the calculation formula of the reward function value R is as follows: R=w1σ+w2d+w3F w1+w2+w3=1 Where w1, w2, and w3 represent the combined weight coefficients of the maximum stress σ of the steel structure, the maximum displacement d of the mass point, and the maximum internal force F of the cable, respectively.

5. The method for optimizing the construction process of a cable-strut tension structure based on reinforcement learning according to claim 2, characterized in that: In step S23, the steel brace installation sequence data is collected and preprocessed, and the action space A is constructed according to the preprocessed data, and sent to the intelligent agent Agent as a training set for model optimization training.

6. The method for optimizing the construction process of a cable-strut tension structure based on reinforcement learning according to claim 5, characterized in that: A portion of the steel brace installation sequence data is randomly selected as a test set, and the other data is used as a training set; a K-fold cross-validation strategy is adopted, and a portion of the training set is taken as a validation set for each training, and the rest is taken as a training set for that training.

7. The method for optimizing the construction process of a cable-strut tension structure based on reinforcement learning according to claim 2, characterized in that: In step S24, the steps of constructing an intelligent agent based on the Q-learning algorithm are as follows: S241, determining the learning rate α, the discount factor γ, the exploration factor ε in the ε-greedy strategy, the number of cycles N, and the cycle time step t in the Q-learning algorithm; S242, initializing the learning reward function R and state space S of the Q-learning algorithm; S243, according to the steel brace installation sequence data in the action space A, the maximum stress σ of the steel structure, the maximum displacement d of the mass point and the maximum internal force F of the cable of the steel brace installation sequence are calculated based on the ε-greedy strategy, the reward function value is observed and a new state space S' is generated; S244, based on the new state space S', iterate the Q-learning value function according to the Bellman equation and update the state space S, and select the next action from the updated state space S' based on ε-greedy; S245, repeat steps S242-S244 until the reward function value meets the requirements or reaches the maximum cycle period.

8. The method for optimizing the construction process of a cable-strut tension structure based on reinforcement learning according to claim 1, characterized in that: In step S3, the steps of finding the optimal training hyperparameters by the control variable method are as follows: S31, determining several hyperparameters that affect the training results of the combined optimization model of the cable-strut structure forming process; S32, select a certain hyperparameter, keep other hyperparameters fixed, and gradually try hyperparameters with different values ​​to perform model optimization experiments on the same action space A; S33, record the model performance under different values ​​of the same hyperparameter and find the optimal value, fix the optimal value of the hyperparameter; select other hyperparameters and repeat step S32 until all hyperparameters are optimized.

Images