Steel-concrete composite beam component section optimization method based on deep reinforcement learning

Through deep reinforcement learning, the design of steel-concrete composite beams is optimized, and the reinforcement learning environment and PPO algorithm are used to train the agents, which solves the problems of complexity and low efficiency of steel-concrete composite beams, and achieves a fast and efficient design process, which is significantly Improves design efficiency and quality.

CN118917204APending Publication Date: 2024-11-08HANGZHOU TONGMSK DESIGN CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411091602.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The design process of steel-concrete composite beams is complicated, and existing software is difficult to achieve customized and refined design, and requires manual iteration, which is time-consuming and labor-consuming.

Method used

Using a method based on deep reinforcement learning, a reinforcement learning environment for steel-concrete composite beam components is built, state space, action space and reward functions are defined, and the agent is trained using the proximal strategy optimization algorithm (PPO algorithm) to optimize the cross-sectional design of steel beams.

Benefits of technology

The superior steel beam cross-section design that meets the requirements of the national standard specifications within 0.1 seconds is obtained, which significantly improves design efficiency, saves labor costs, and reduces material use and carbon emissions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118917204A_ABST
    Figure CN118917204A_ABST
Patent Text Reader

Abstract

The invention discloses a steel-concrete composite beam component section optimization method based on deep reinforcement learning. The method comprises the following steps: establishing a reinforcement learning environment containing national standard specification constraint conditions, defining a state space, an action space and a reward function of the environment, training a PPO agent capable of autonomously designing a steel-concrete composite beam component by adopting a near-end strategy optimization algorithm PPO, selecting the agent which shows the optimal performance in the training process, and carrying out the design of the steel-concrete composite beam component. And deploying the optimal intelligent agent to a production environment, and performing optimization design on the section of the steel-concrete composite beam component by using the intelligent agent. According to the method, modeling is conducted on the design problem of the steel-concrete composite beam component through the reinforcement learning normal form, the intelligent agent is trained through the PPO algorithm, the optimal steel beam section can be autonomously designed in the given section parameter range, the structural design efficiency can be improved, the material cost of a steel structure building is saved, and carbon emission is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of structural design, and in particular relates to a cross-section optimization method for steel-concrete composite beam components based on deep reinforcement learning. Background Art

[0002] In steel structures, steel-concrete composite beams are a type of composite bending-resistant member formed by concrete slabs, shear connectors, and steel beams. During the normal use of a building, the concrete slabs mainly bear pressure, and the steel beams mainly bear tension. The two work together to reduce the cracking of the concrete slabs and improve the stability of the steel beams, greatly improving the bending bearing capacity of the composite member. Therefore, steel-concrete composite beams are widely used in actual engineering projects.

[0003] Regarding the design of steel-concrete composite beams, my country's standard "Standard for Design of Steel Structures" GB50017-2017 provides relevant design methods and regulations. By meeting the requirements for the cross-sectional grade of the components of plastic design, the steel-concrete composite beam can be plastically designed. During the construction phase, it is necessary to verify the bending, shear, deflection and other indicators of the steel beam components under the construction constant and live loads; during the use phase, the steel and concrete beams form a whole, and at this time, the number of shear connectors needs to be verified to ensure that they meet the requirements for transmitting shear force; the steel-concrete composite beams need to be verified under the constant and live loads during the use phase. The standard "Standard for Design of Steel Structures" GB50017-2017 of my country ensures the safety and reliability of the design of steel-concrete composite beams from the aspects of structure, bearing capacity, deflection, etc.

[0004] However, steel-concrete composite beams usually need to be verified in two stages: the construction stage and the use stage. The loads and cross-sectional forms of the two stages are different, so the design process is more complicated and there are more verification items. To obtain the ideal design results, it is usually necessary to manually iterate the design repeatedly. At present, commercial structural design software is difficult to achieve the level of customized and refined design. Summary of the invention

[0005] In order to solve the problems existing in the background technology, the present invention provides a steel-concrete composite beam component section optimization method based on deep reinforcement learning. Through the component section optimization method in the present invention, a better steel beam section that meets the requirements of the national standard "Steel Structure Design Standard" GB50017-2017 under specified design conditions can be quickly obtained, and the design quality is high.

[0006] The technical solution of the present invention is as follows, comprising the following steps:

[0007] Step 1) Build a reinforcement learning environment for steel-concrete composite beam components and define the state space, action space and reward function of the reinforcement learning environment;

[0008] Step 2), using a proximal strategy optimization algorithm and a reinforcement learning environment to train an intelligent agent, wherein the intelligent agent is used to optimize the cross-section design of a steel-concrete composite beam component;

[0009] Specifically, an agent with a multi-layer policy neural network and a value neural network is initialized, and a reinforcement learning environment containing the national standard "Steel Structure Design Standard" GB50017-2017 component specification constraints is interacted to obtain training data samples. The proximal policy optimization algorithm (PPO algorithm for short) is used, and the discount factor γ, learning rate α and other parameters are set. The parameters of the neural network are updated using the collected training data samples. After multiple rounds of interaction with the environment and neural network parameter learning, the design level of the agent is gradually improved, and the training process is stopped after the preset number of training steps is finally reached.

[0010] Step 3) Select the agent with the best performance in the training process, and use the best agent to optimize the cross-section of the steel-concrete composite beam component, so as to obtain the optimal combination of design parameters of the steel beam component.

[0011] In step 1), the state space of the reinforcement learning environment includes the three states of design conditions, component design parameters and constraint conditions; the design conditions include the steel beam span L, the steel beam spacing D0, the concrete floor thickness h c , concrete strength grade, steel strength grade, additional dead load p k and live load q k The component design parameters include the steel beam height h, the steel beam web thickness t w , steel beam flange thickness t f , steel beam flange width b f and shear connector spacing s d The constraints include the following ten steel beam component constraints c i :

[0012] The width-to-thickness ratio constraint of the steel beam component is obtained according to the following formula:

[0013]

[0014] Where: c1 represents the first width-to-thickness ratio constraint of the steel beam component; c2 represents the second width-to-thickness ratio constraint of the steel beam component; b f Indicates the width of the steel beam flange; t w Indicates the web thickness of the steel beam; t f represents the thickness of the steel beam flange; f y Indicates the yield strength of steel; h w Indicates the height of the web of the steel beam, h = hw +2t f ;

[0015] The expression of the bearing capacity constraint condition that the steel beam components need to meet during the construction phase is as follows:

[0016]

[0017] Where: c3 represents the first bearing capacity constraint condition of the steel beam component in the construction stage; c4 represents the second bearing capacity constraint condition of the steel beam component in the construction stage; c5 represents the third bearing capacity constraint condition of the steel beam component in the construction stage; M d1 represents the design bending moment of the steel beam at the mid-span during the construction phase; γ x W represents the plastic development coefficient of the steel beam; s represents the section modulus of the steel beam; f represents the design value of steel strength; It represents the overall stability coefficient of steel beam under bending; Q d1 represents the design shear force of the steel beam support during the construction stage; f v Indicates the design value of steel shear strength;

[0018] The expression of the deflection constraint condition that the steel beam components need to meet during the construction phase is as follows:

[0019]

[0020] Where: c6 represents the deflection constraint condition of the steel beam component during the construction phase; v s Indicates the mid-span deflection of the steel beam during the construction stage; [v] indicates the standard limit value of the mid-span deflection of the steel beam specified in the "Steel Structure Design Standard" GB50017-2017;

[0021] The constraint condition on the number of shear connectors that the steel-concrete composite beam components need to meet during the service stage is obtained by the following formula:

[0022]

[0023] Where: c7 represents the number constraint of shear connectors of steel-concrete composite beam components; n r Indicates the actual number of steel beam shear connectors; n0 indicates the number of shear connectors required to meet the full shear connection requirements calculated according to the "Steel Structure Design Standard" GB50017-2017;

[0024] The bearing capacity constraint conditions that the steel beam components need to meet during the service phase are obtained according to the following formula:

[0025]

[0026] Where: c8 represents the first bearing capacity constraint condition of the steel beam component in the service stage; c9 represents the second bearing capacity constraint condition of the steel beam component in the service stage; M d2 M represents the design bending moment at the mid-span of the steel beam during the service period; r Indicates the design value of the mid-span bending bearing capacity of the steel beam during the service period; Q d2 It represents the design shear force of the steel beam support during the service stage;

[0027] The expression of the deflection constraint condition that the steel beam component needs to meet during the service stage is as follows:

[0028]

[0029] Where: c 10 It represents the deflection constraint condition of the steel beam component during the service stage; v represents the mid-span deflection of the steel beam during the service stage.

[0030] In step 1), the action space of the reinforcement learning environment is used to update the component design parameters. The action space adopts the following action vector a t Characterization:

[0031] a t =[d1 A1 d2A2 d3A3 d4A4 d5A5]

[0032] A1, A2, A3, A4, A5∈{1,0,1}

[0033] Among them, a t represents the action vector a at step t t ; d1~d5 represent the preset design parameter variation coefficient;

[0034] The method of updating the component design parameters through the action space is as follows: first, the component design parameters are constructed into a design parameter vector, and the design parameter vector g of the t+1 step is updated according to the following formula t+1 To update:

[0035] g t+1 =g t +a t

[0036] g t =[ht w b f t f s d ]

[0037] Among them, g t represents the design parameter vector at step t.

[0038] In step 1), the expression of the reward function in the reinforcement learning environment is as follows:

[0039] r t =F(s t )-F(s t-1 )

[0040]

[0041] Where: r t represents the reward value of the tth step; s t represents the environmental state at step t; s t-1 represents the environmental state at step t-1; c i,t represents c corresponding to the i-th constraint condition in step t i Value; w i represents the calculation weight of the preset i-th constraint condition; a and k represent the preset value for controlling the piecewise function f(c i,t ) constant parameters of the shape; f(c i,t ) represents a piecewise function; F(s t ) represents the potential energy function;

[0042] Termination reward r in reinforcement learning environment term is defined as follows:

[0043] If the action vector a output by the agent t The value of the component design parameter in the design parameter vector exceeds the preset design parameter range, so the termination reward r term =-50, and the agent terminates the action and does not execute the next step;

[0044] If the agent reaches the pre-designed step, and the constraints c in the reinforcement learning environment are i The values ​​are all greater than 0, so the termination reward r term =1;

[0045] If the agent reaches the pre-designed step, and the constraints c in the reinforcement learning environment are i At least one of the values ​​is not greater than 0, so the termination reward r term =-1.

[0046] The specific steps of step 2) are:

[0047] Step 2.1), first initialize the agent with a multi-layer policy neural network and a value neural network; define the initial value of the state in the reinforcement learning environment;

[0048] Step 2.2), then the current state of the reinforcement learning environment (design conditions, component design parameters and constraints) and the reward value of the reward function are input into the agent, the agent outputs an action vector based on the input of the reinforcement learning environment and the proximal strategy optimization algorithm, and the action vector is input back to the reinforcement learning environment, and the component design parameters are updated based on the current action vector in the action space;

[0049] Step 2.3) and step 2.2) constitute a complete training data collection process. For a reinforcement learning environment, each step of the training data collection process obtains a set of training data samples, and the training data samples include design conditions, component design parameters, constraint conditions, action vectors and reward values. Repeat several steps of the training data collection process until N sets of training data samples are obtained;

[0050] Step 2.4), every time N sets of training data samples are obtained, the neural network parameters in the agent are updated and trained using the current N sets of training data samples. When the number of updates of the agent reaches the preset number of updates M, the update training of the agent is stopped.

[0051] The specific steps of step 3) are:

[0052] Step 3.1) Select the agent with the highest cumulative reward value in the update training as the optimal agent. The cumulative reward value R is obtained according to the following formula:

[0053]

[0054] Step 3.2) Input the design conditions under the actual working conditions into the optimal intelligent agent, use the optimal intelligent agent to optimize the design parameters of the steel beam section in the steel-concrete composite beam component, output the optimal combination of steel beam component design parameters, and set the steel beam in the steel-concrete composite beam according to the optimal combination of steel beam component design parameters.

[0055] The steel-concrete composite beam member is a composite bending-resistant member mainly formed by an I-shaped steel beam, a concrete floor slab, and a shear-resistant connector.

[0056] The technical principles of the present invention are as follows:

[0057] 1. Establish a reinforcement learning environment that includes the constraints of the national standard "Steel Structure Design Standard" GB50017-2017 component specification, and define the state space, action space and reward function of the environment. The design process is regarded as a continuous decision-making process, namely the Markov decision process. Within the given design step range, the agent performs preset actions according to the observed environmental state and obtains corresponding rewards. The ultimate task is to obtain the maximum cumulative reward.

[0058] 2. Use the Proximal Policy Optimization (PPO) algorithm to train a PPO agent that can autonomously design steel-concrete composite beam components. During the training process, the agent learns through data from interactions with the environment and improves its design capabilities toward the goal of obtaining the maximum cumulative reward. Finally, the trained PPO agent is deployed in the production environment.

[0059] The beneficial effects of the present invention are:

[0060] 1. The present invention applies the idea of ​​reinforcement learning to the design of steel-concrete composite beam components, and regards the design process as a continuous decision-making process, namely, the Markov decision process; at the same time, a reinforcement learning environment including the component specification constraints of the national standard "Steel Structure Design Standard" GB50017-2017 is designed, and the state space, action space and reward function for the structural design task are proposed, which is conducive to the optimization design of the cross-section of the steel-concrete composite beam component.

[0061] 2. When designing steel-concrete composite beams, the present invention requires designers to repeatedly iterate the design due to many constraints and considerations, which is time-consuming and labor-intensive. After using this intelligent design method, designers can obtain a more optimal design section in just 0.1 seconds, greatly improving design efficiency.

[0062] 3. The present invention can be widely applied to various buildings using steel-concrete composite beam components, which is conducive to promoting the application of such components in buildings.

[0063] 4. The present invention models the design problem of steel-concrete composite beam components by adopting the reinforcement learning paradigm and trains the intelligent agent with the PPO algorithm, so that it can autonomously design a better steel beam section within a given section parameter range, which is beneficial to improving the efficiency of structural design, saving the material cost of steel structure buildings, and reducing carbon emissions. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a schematic diagram of the process of the present invention;

[0065] Figure 2 This is a schematic diagram of a steel-concrete composite beam structure applicable to the present invention;

[0066] Figure 3 This is a diagram of the training process of the PPO agent in the present invention.

[0067] Figure 4 This is a data graph of the cumulative reward value during the agent training process.

[0068] Figure 5 Schematic diagram of the process of optimizing steel beam cross-section using the optimal agent. DETAILED DESCRIPTION

[0069] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0070] The present invention is further described in detail below in conjunction with embodiments.

[0071] Embodiment 1:

[0072] The embodiment and specific steps of the intelligent design method of steel-concrete composite beam components based on deep reinforcement learning of the present invention are as follows. Figure 1 and Figure 3 As shown. Figure 2 As shown, the steel-concrete composite beam member is a composite bending-resistant member formed by an I-shaped steel beam, a concrete floor slab, and a shear connector.

[0073] Step 1: Establish a reinforcement learning environment that includes the component specification constraints of the national standard "Steel Structure Design Standard" GB50017-2017, and define the state space, action space and reward function of the reinforcement learning environment; the state space of the reinforcement learning environment includes the three states of design conditions, component design parameters and constraints; the design conditions include the steel beam span L, steel beam spacing D0, concrete floor thickness h c , concrete strength grade, steel strength grade, additional dead load p k and live load q k ; Component design parameters include steel beam height h, steel beam web thickness t w , steel beam flange thickness t f , steel beam flange width b f and shear connector spacing s d ; The constraints include the following ten steel beam component constraints c i :

[0074] The width-to-thickness ratio constraint of the steel beam component is obtained according to the following formula:

[0075]

[0076] Where: c1 represents the first width-to-thickness ratio constraint of the steel beam component; c2 represents the second width-to-thickness ratio constraint of the steel beam component; b f Indicates the width of the steel beam flange; t w Indicates the web thickness of the steel beam; t f represents the thickness of the steel beam flange; f y Indicates the yield strength of steel; h w Indicates the height of the web of the steel beam, h = h w +2tf ;

[0077] The expression of the bearing capacity constraint condition that the steel beam components need to meet during the construction phase is as follows:

[0078]

[0079] Where: c3 represents the first bearing capacity constraint condition of the steel beam component in the construction stage; c4 represents the second bearing capacity constraint condition of the steel beam component in the construction stage; c5 represents the third bearing capacity constraint condition of the steel beam component in the construction stage; M d1 represents the design bending moment of the steel beam at the mid-span during the construction phase; γ x W represents the plastic development coefficient of the steel beam; s represents the section modulus of the steel beam; f represents the design value of steel strength; It represents the overall stability coefficient of steel beam under bending; Q d1 represents the design shear force of the steel beam support during the construction stage; f v Indicates the design value of steel shear strength;

[0080] The expression of the deflection constraint condition that the steel beam components need to meet during the construction phase is as follows:

[0081]

[0082] Where: c6 represents the deflection constraint condition of the steel beam component during the construction phase; v s Indicates the mid-span deflection of the steel beam during the construction stage; [v] indicates the standard limit value of the mid-span deflection of the steel beam specified in the "Steel Structure Design Standard" GB50017-2017;

[0083] The constraint condition on the number of shear connectors that the steel-concrete composite beam components need to meet during the service stage is obtained by the following formula:

[0084]

[0085] Where: c7 represents the number constraint of shear connectors of steel-concrete composite beam components; n r Indicates the actual number of shear connectors for steel beams; n0 indicates the number of shear connectors required to meet the requirements of full shear connection according to the "Standard for Design of Steel Structures" GB50017-2017;

[0086] The bearing capacity constraint conditions that the steel beam components need to meet during the service phase are obtained according to the following formula:

[0087]

[0088] Where: c8 represents the first bearing capacity constraint condition of the steel beam component in the service stage; c9 represents the second bearing capacity constraint condition of the steel beam component in the service stage; Md2 M represents the design bending moment at the mid-span of the steel beam during the service period; r Indicates the design value of the mid-span bending bearing capacity of the steel beam during the service period; Q d2 It represents the design shear force of the steel beam support during the service stage;

[0089] The expression of the deflection constraint condition that the steel beam component needs to meet during the service stage is as follows:

[0090]

[0091] Where: c 10 It represents the deflection constraint condition of the steel beam component during the service stage; v represents the mid-span deflection of the steel beam during the service stage.

[0092] The action space of the reinforcement learning environment is used to update the component design parameters. The action space uses the following action vector a t Characterization:

[0093] a t =[d1 A1 d2A2 d3A3 d4A4 d5A5]

[0094] A1, A2, A3, A4, A5∈{1,0,1}

[0095] Among them, a t represents the action vector a at step t t ; d1~d5 represent the preset design parameter variation coefficient;

[0096] The method of updating the component design parameters through the action space is as follows: first, the component design parameters are constructed into a design parameter vector, and the design parameter vector g of the t+1 step is updated according to the following formula t+1 To update:

[0097] g t+1 =g t +a t

[0098] g t =[ht w b f t f s d ]

[0099] Among them, g t represents the design parameter vector at step t.

[0100] The expression of the reward function in the reinforcement learning environment is as follows:

[0101] r t =F(s t )-F(s t-1 )

[0102]

[0103]

[0104] Where: r t represents the reward value of the tth step; s t represents the environmental state at step t; s t-1 represents the environmental state at step t-1; c i,t represents c corresponding to the i-th constraint condition in step t i Value; w i represents the calculation weight of the preset i-th constraint condition; a and k represent the preset value for controlling the piecewise function f(c i,t ) constant parameters of the shape; f(c i,t ) represents a piecewise function; F(s t ) represents the potential energy function;

[0105] Termination reward r in reinforcement learning environment term is defined as follows:

[0106] If the action vector a output by the agent t The value of the component design parameter in the design parameter vector exceeds the preset design parameter range, so the termination reward r term =-50, and the agent terminates the action and does not execute the next step;

[0107] If the agent reaches the pre-designed step, and the constraints c in the reinforcement learning environment are i The values ​​are all greater than 0, so the termination reward r term =1;

[0108] If the agent reaches the pre-designed step, and the constraints c in the reinforcement learning environment are i At least one of the values ​​is not greater than 0, so the termination reward r term =-1.

[0109] In the specific implementation, the design conditions of the given environment state space are shown in Table 1:

[0110] Table 1

[0111] Design conditions Numerical range Steel beam span L(m) 4~20, interval 1 <![CDATA[Spacing D0 (m) of steel beams]]> 2~8, interval 0.5 <![CDATA[Concrete floor slab thickness h c (mm)]]> 120,130,150,180,200 Concrete strength grade C30,C35,40 Steel strength grade Q355 <![CDATA[Additional dead load p k (kN / m 2 )]]> 0.5~3.0, interval 0.5 <![CDATA[Live load q k (kN / m 2 )]]> 2,2.5,3,3.5,4

[0112] The design parameters of the given environment state space are shown in Table 2:

[0113] Table 2

[0114] Design parameters Numerical range Steel beam height h(mm) 300~1200, interval 50 <![CDATA[Flange width b of steel beam f (mm)]]> 100~600, 50 intervals <![CDATA[Thickness t of steel beam flange f (mm)]]> 6,8,12,14,16,20,25,30,35,40 <![CDATA[Web thickness t of steel beam w (mm)]]> 6,8,12,14,16,20,25,30,35,40 <![CDATA[Shearing connector spacing s d (mm)]]> 250,200,150,100

[0115] The verification results of the standard constraints in the environmental state space c1~c 10The range is a real number, and the specific value is calculated according to the formula during the training process.

[0116] Define the action space of the environment as the action vector, the action vector a t Each element in the i and design parameter variation coefficient d i The basic element values ​​are -1, 0, 1, where 1 represents an increase, 0 represents unchanged, and -1 represents a decrease. For example, the action vector [0, -1, 1, 0, 0] represents the design parameter vector [ht w b f t f s d ], do the following: increase b by one level f , keep h,t f ,s d No change, reduce t by one level w Among them, the series variation of the design parameters is determined by the design parameter variation coefficient d i Control, for example, d1 to d5 correspond to [ht w b f t f s d ], d1, d3, and d5 are all taken as 50, and d2 and d4 can be taken according to the current web thickness and flange thickness of the steel beam, as shown in Table 2, and are generally taken as 2, 4, or 5.

[0117] Take a = 1.0, k = 0.1, and define w i The weight matrix W is composed of:

[0118] W=[0.5,0.5,2,2,3,1,0.5,4,3,2];

[0119] Among them, w i represents the i-th element in the weight matrix W.

[0120] The process reward function of the environment is:

[0121] r t =F(s t )-F(s t-1 )

[0122]

[0123] Where: r t represents the reward value of the tth step; s t represents the environmental state at step t; s t-1 represents the environmental state at step t-1; c i,t represents the c of step t i value;

[0124] The termination reward function of the environment is defined as: When the agent reaches the final design step, it determines whether the design meets the specification requirements (that is, all the constraints c at this time i The values ​​are all greater than 0); if the specification requirements are met, a positive reward r will be obtained term =1, otherwise, a negative reward r is obtained term = -1. In addition, when the action of the agent exceeds the preset range and causes the design parameters to exceed the preset range, the action will be considered illegal, the design process will be terminated, and the agent will be given a reward of -50.

[0125] Step 2. Initialize an intelligent agent with a three-layer policy neural network and a value neural network with 256 parameters in each layer, and interact with a reinforcement learning environment that contains the national standard "Steel Structure Design Standard" GB50017-2017 component specification constraints to obtain training data samples. Use the proximal policy optimization algorithm (PPO algorithm for short), set the discount factor γ = 0.9, the policy neural network learning rate α1 = 0.0001, and the value neural network learning rate α2 = 0.0005, and use the collected training data samples to update the parameters of the neural network. The preset number of updates M = 10000, each update will collect N = 1024 training data samples. The agent training process, the next Figure 4 shown.

[0126] like Figure 3 As shown in the figure, during the training process of the intelligent agent, multiple reinforcement learning environments can be used to conduct joint training with an intelligent agent. k represents the kth reinforcement learning environment; s n k represents the state of the k-th reinforcement learning environment at step n; a n k represents the action of the kth reinforcement learning environment at step n; R n k represents the cumulative reward value at the nth step of the kth reinforcement learning environment; v n k Represents the output value of the value neural network at step n;

[0127] If multiple reinforcement learning environments are selected, when selecting the optimal agent, the agent with the largest sum of the reward values ​​corresponding to all training data samples and the termination reward during a single update training process should be selected as the optimal agent.

[0128] Step 3: Evaluate the performance of the agents during training and select the agents with the highest cumulative rewards as the agents for final application. Given a design condition, the design conditions are: steel beam span L = 7m, steel beam spacing D0 = 3m, concrete floor thickness h c =150mm, concrete strength C35, steel strength Q355, additional dead load p k =2.5kN / m 2 ,q k =2.5kN / m 2 , then the design process of the intelligent agent is as follows Figure 5 After 0.065s, the intelligent agent gives the design parameters of this working condition as 300×6×150×20mm and the distance between the connectors is 150mm.

[0129] In summary, the method of the present invention is an intelligent design method for steel-concrete composite beam components based on deep reinforcement learning, and can take into account the design-related provisions of the national standard "Steel Structure Design Standard" GB50017-2017. The component design process is fast and efficient, and it only takes about 0.1 seconds to obtain a relatively satisfactory steel beam design cross-sectional size and shear connector spacing. If manual optimization and adjustment is used, it usually takes about 30 minutes, so the design efficiency can be significantly improved.

[0130] The contents described in the embodiments of this specification are merely an enumeration of the implementation forms of the inventive concept. The protection scope of the present invention should not be regarded as limited to the specific forms described in the embodiments. The protection scope of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A cross-section optimization method for steel-concrete composite beam components based on deep reinforcement learning. It is characterized in that The following steps are involved: Step 1) Build a reinforcement learning environment for steel-concrete composite beam components and define the state space, action space and reward function of the reinforcement learning environment; Step 2), using a proximal strategy optimization algorithm and a reinforcement learning environment to train an intelligent agent, wherein the intelligent agent is used to optimize the cross-section design of a steel-concrete composite beam component; Step 3) Select the agent with the best performance in the training process, and use the best agent to optimize the cross-section of the steel-concrete composite beam component, so as to obtain the optimal combination of design parameters of the steel beam component.

2. According to claim 1, a method for optimizing the cross section of a steel-concrete composite beam member based on deep reinforcement learning, Features: In the step 1), the state space of the reinforcement learning environment includes design conditions, component design parameters and constraint conditions; the design conditions include the steel beam span L, the steel beam spacing D 0 , concrete floor thickness h c , concrete strength grade, steel strength grade, additional dead load p k and live load q k The component design parameters include the steel beam height h, the steel beam web thickness t w , steel beam flange thickness t f , steel beam flange width b f and shear connector spacing s d The constraints include the following ten steel beam component constraints c i : The width-to-thickness ratio constraint of the steel beam component is obtained according to the following formula: Where: c 1 represents the first width-to-thickness ratio constraint of the steel beam component; c 2 represents the second width-to-thickness ratio constraint of the steel beam component; b f Indicates the width of the steel beam flange; t w Indicates the web thickness of the steel beam; t f represents the thickness of the steel beam flange; f y Indicates the yield strength of steel; h w Indicates the web height of the steel beam; The expression of the bearing capacity constraint condition that the steel beam components need to meet during the construction phase is as follows: Where: c 3 represents the first bearing capacity constraint condition of the steel beam component during the construction phase; c 4 represents the second bearing capacity constraint condition of the steel beam component during the construction phase; c 5 represents the third bearing capacity constraint condition of the steel beam component during the construction phase; M d1 represents the design bending moment of the steel beam at the mid-span during the construction phase; γ x W represents the plastic development coefficient of the steel beam; s represents the section modulus of the steel beam; f represents the design value of steel strength; It represents the overall stability coefficient of steel beam under bending; Q d1 represents the design shear force of the steel beam support during the construction stage; f v Indicates the design value of steel shear strength; The expression of the deflection constraint condition that the steel beam components need to meet during the construction phase is as follows: Where: c 6 represents the deflection constraint condition of the steel beam component during the construction phase; v s represents the mid-span deflection of the steel beam during the construction stage; [v] represents the standard limit value of the mid-span deflection of the steel beam; The constraint condition on the number of shear connectors that the steel-concrete composite beam components need to meet during the service stage is obtained by the following formula: Where: c 7 represents the number constraint of shear connectors of steel-concrete composite beam components; n r Indicates the actual number of steel beam shear connectors; n 0 It indicates the number of shear connectors required to meet the full shear connection requirements according to the code calculation; The bearing capacity constraint conditions that the steel beam components need to meet during the service phase are obtained according to the following formula: Where: c 8 represents the first bearing capacity constraint condition of the steel beam component during the service stage; c 9 represents the second bearing capacity constraint condition of the steel beam component during the service stage; M d2 M represents the design bending moment at the mid-span of the steel beam during the service period; r Indicates the design value of the mid-span bending bearing capacity of the steel beam during the service period; Q d2 It represents the design shear force of the steel beam support during the service stage; The expression of the deflection constraint condition that the steel beam component needs to meet during the service stage is as follows: Where: c 10 It represents the deflection constraint condition of the steel beam component during the service stage; v represents the mid-span deflection of the steel beam during the service stage.

3. According to claim 2, a method for optimizing the cross section of a steel-concrete composite beam member based on deep reinforcement learning, Features: In step 1), the action space of the reinforcement learning environment is used to update the component design parameters. The action space adopts the following action vector a t Characterization: a t =[d 1 A 1 d 2 A 2 d 3 A 3 d 4 A 4 d 5 A 5 ] A 1 ,A 2 ,A 3 ,A 4 ,A 5 ∈{1,0,1} Among them, a t represents the action vector a at step t t ;d 1 ~d 5 Indicates the preset design parameter variation coefficient; The method of updating the component design parameters through the action space is as follows: first, the component design parameters are constructed into a design parameter vector, and the design parameter vector g of the t+1 step is updated according to the following formula t+1 To update: g t+1 =g t +a t g t =[ht w b f t f s d ] Among them, g t represents the design parameter vector at step t.

4. According to claim 2, a method for optimizing the cross section of a steel-concrete composite beam member based on deep reinforcement learning, Features: In step 1), the expression of the reward function in the reinforcement learning environment is as follows: r t =F(s t )-F(s t-1 ) Where: r t represents the reward value of the tth step; s t represents the environmental state at step t; s t-1 represents the environmental state at step t-1; c i,t represents c corresponding to the i-th constraint condition in step t i Value; w i represents the calculation weight of the i-th constraint; a and k represent the weights used to control the piecewise function f(c i,t ) constant parameters of the shape; f(c i,t ) represents a piecewise function; F(s t ) represents the potential energy function; Termination reward r in reinforcement learning environment term is defined as follows: If the action vector a output by the agent t The value of the component design parameter in the design parameter vector exceeds the preset design parameter range, so the termination reward r term =-50, and the agent terminates the action and does not execute the next step; If the agent reaches the pre-designed step, and the constraints c in the reinforcement learning environment are i The values ​​are all greater than 0, so the termination reward r term =1; If the agent reaches the pre-designed step, and the constraints c in the reinforcement learning environment are i At least one of the values ​​is not greater than 0, so the termination reward r term =-1.

5. According to claim 2, a method for optimizing the cross section of a steel-concrete composite beam member based on deep reinforcement learning, Features: The specific steps of step 2) are: Step 2.1), first initialize the agent with a multi-layer policy neural network and a value neural network; define the initial value of the state in the reinforcement learning environment; Step 2.2), then input the current state of the reinforcement learning environment and the reward value of the reward function into the intelligent agent, the intelligent agent outputs an action vector based on the input of the reinforcement learning environment and the proximal strategy optimization algorithm, and inputs the action vector back into the reinforcement learning environment, and the component design parameters are updated based on the current action vector in the action space; Step 2.3) and step 2.2) constitute a complete training data collection process, wherein each training data collection process obtains a set of training data samples, and the training data collection process is repeated several times until N sets of training data samples are obtained; Step 2.4), every time N sets of training data samples are obtained, the intelligent agent is updated and trained using the current N sets of training data samples. When the number of updates of the intelligent agent reaches the preset number of updates M, the update training of the intelligent agent is stopped.

6. According to claim 2, a method for optimizing the cross section of a steel-concrete composite beam member based on deep reinforcement learning, Features: The specific steps of step 3) are: Step 3.1) Select the agent with the highest cumulative reward value in the update training as the optimal agent. The cumulative reward value R is obtained according to the following formula: Step 3.2) Input the design conditions under the actual working conditions into the optimal intelligent agent, output the optimal combination of design parameters of the steel beam component, and set the steel beam in the steel-concrete composite beam according to the optimal combination of design parameters of the steel beam component.

7. According to claim 1, a method for optimizing the cross section of a steel-concrete composite beam member based on deep reinforcement learning, Features: The steel-concrete composite beam member is a composite bending-resistant member mainly formed by an I-shaped steel beam, a concrete floor slab, and a shear-resistant connector.

Citation Information

Cited By

  • Aircraft wing beam section multi-objective optimization design method based on reinforcement learning

    CN119180097A

  • A multi-objective optimization design method for aircraft wing beam cross section based on reinforcement learning

    CN119180097B