A multi-objective optimization design method for aircraft wing beam cross section based on reinforcement learning

Abstract

The invention discloses a multi-objective optimization design method for an aircraft wing beam cross section based on reinforcement learning, including space definition, beam structure virtual environment construction and intelligent optimization design, specifically including step S1 determining state space, step S2 determining action space, step S3 setting design evaluation value function, step S4 building beam structure virtual environment, step S5 establishing beam structure intelligent design body, step S6 training beam structure intelligent design body, step S7 using beam structure intelligent design body for optimization design and step S8 checking whether the design scheme is reasonable and feasible; by adopting the multi-objective optimization design method for an aircraft wing beam structure based on reinforcement learning, multi-objective optimization can be achieved, and design experience can be stored, and design space can be further broadened on this basis in the future to improve design efficiency. In addition, the invention can be used for the design of other types of beam structures through simple improvements, and has a wide range of application value.

Description

Technical Field

[0001] The present invention belongs to the field of aircraft structures, and in particular, relates to a multi-objective optimization design method for an aircraft wing beam section based on reinforcement learning. Background Art

[0002] Aircraft structural design is one of the core technologies in the aviation field, and the beam is the main load-bearing structure of the aircraft. Among them, the cantilever beam structure is widely used in aircraft wings, and its design quality is related to many aspects such as aircraft performance, safety and economic benefits. In the design of beam structures, the design of beam cross-section is very critical, which is related to the mechanical properties and weight of the beam.

[0003] Traditional aircraft structural design methods are mainly based on experience and trial and error. This method is time-consuming and labor-intensive, with low design efficiency and difficulty in ensuring the optimal design results. Traditional optimization algorithms also have the disadvantages of poor transferability and the need for re-iterative optimization for different design requirements. Aircraft design is a process that requires the integration of multiple requirements from various disciplines and repeated iterative optimization, which will make the time required for traditional optimization reach a level that is unacceptable to the project. Therefore, a rapid structural design method is urgently needed.

[0004] In recent years, reinforcement learning, as a new intelligent algorithm, has been widely used in the field of machine learning because it can autonomously learn and optimize in an uncertain environment. Reinforcement learning can effectively improve design efficiency and quality by continuously improving the design through trial and error learning and feedback mechanism. Summary of the invention

[0005] In view of the above defects, the present invention provides a multi-objective optimization design method for an aircraft wing beam section based on reinforcement learning, including space definition, beam structure virtual environment construction and intelligent optimization design, which specifically includes the following steps:

[0006] S1. Construct a state vector based on the parameters of the beam section to complete the definition of the state space;

[0007] S2, define specific actions according to the constituent parameters in the state space to complete the definition of the action space;

[0008] S3, setting a design evaluation value function according to the set beam section evaluation index;

[0009] S4. Based on the definition of the state space and the definition of the action space, a virtual environment is built to simulate the behavior of the beam structure. The virtual environment can update the state according to the action of the intelligent design body of the beam structure and calculate the design evaluation value;

[0010] S5. Create an intelligent design agent for beam structures that understands the relationship between states and actions for learning and optimization in a virtual environment.

[0011] S6. Training the intelligent design body of the beam structure to predict different states and action results;

[0012] S7, using the trained beam structure intelligent design body to optimize the design in a virtual environment and generate a new design solution;

[0013] S8. Evaluate the design solution generated in step S7 according to the actual situation to check whether it is feasible.

[0014] Furthermore, the parameters in step S1 include but are not limited to the width, height, flange thickness, web thickness, shape and length of the beam section.

[0015] Furthermore, the design evaluation value function in step S3 is as follows:

[0016] ;

[0017] In the above formula, r To design the evaluation function, σ is the maximum stress of the structure, σ s is the material yield strength, C is a fixed penalty value, A is the cross-sectional area, k is a fixed coefficient greater than 0. When the design value exceeds the design boundary value, the design will not be evaluated and a fixed penalty value C will be directly given.

[0018] Further, the evaluation values ​​before and after the design change are calculated according to the design evaluation value function in step S3. r and r_ , and calculate the reward value of the current action reward :

[0019] ;

[0020] The updated status s_ and reward value reward Feedback to the beam structure intelligent design body.

[0021] Furthermore, the step S5 specifically includes the following steps:

[0022] S5.1. Establish all states s Each design action a Design action evaluation table ;

[0023] S5.2. Establish action selection function:

[0024] Select the design action with the highest evaluation value or a random design action by random method;

[0025] S5.3. Establish the current design status s , Design Action a , New Design Status s_ and reward value reward Method for updating the design action evaluation value table:

[0026] ;

[0027] In the above formula, is the reduction factor, lr is the learning rate.

[0028] Furthermore, the step S6 specifically includes the following steps:

[0029] S6.1. Set the maximum width, maximum height, maximum flange thickness, maximum web thickness, beam length, material yield strength, and loading conditions of the beam section in step S1, and set the number of training times and the number of steps for a single training session;

[0030] S6.2. At the beginning of each training, the initial design state is randomly generated;

[0031] S6.3. Use the beam structure intelligent design body to generate design actions in the current design state;

[0032] S6.4, transferring the current design state and the design action performed to the beam virtual environment constructed in step S4, and obtaining an updated new design state and an evaluation value of the new design state;

[0033] S6.5, updating the design action evaluation value table according to the design action evaluation value table updating method in step S5.3;

[0034] S6.6, setting the updated design state as the current design state;

[0035] S6.7, repeat steps S6.3 to S6.6 until the set number of single training steps is reached;

[0036] S6.8, repeat steps S6.2 to S6.7 to perform training until the set number of training times is reached;

[0037] S6.9. Save the design action evaluation value table.

[0038] Furthermore, the step S7 specifically includes the following steps:

[0039] S7.1, loading the design action evaluation value table after training in step S6.9, setting the number of design attempts and the number of design steps in each design attempt;

[0040] S7.2, randomly generate initial design states;

[0041] S7.3, use the beam structure intelligent design body to generate design change actions;

[0042] S7.4, transmitting the current design state and the performed design action to the beam virtual environment, obtaining the updated design state and the evaluation value of the new design state;

[0043] S7.5, setting the updated state as the current state;

[0044] S7.6, repeat steps S7.3 to S7.5 until the single design step number is reached;

[0045] S7.7, repeat steps S7.2 to S7.6 to perform training until the set number of design attempts is reached;

[0046] S7.8. Output the design with the highest evaluation value.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] By adopting the multi-objective optimization design method of aircraft wing beam structure based on reinforcement learning, multi-objective optimization can be achieved, and design experience can be stored, on which basis the design space can be further expanded and design efficiency can be improved. In addition, the present invention can be used for the design of other types of beam structures through simple improvements, and has a wide range of application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a flowchart of the present invention;

[0050] Figure 2 It is a schematic diagram of the inner beam structure of an aircraft wing in the present invention;

[0051] Figure 3 It is a schematic diagram of the cross section of an I-beam in the present invention;

[0052] Figure 4 It is a schematic diagram of the beam section design method based on reinforcement learning in the present invention;

[0053] Figure 5 The curve showing the maximum stress of the beam changing with the number of training times during the training process of the present invention;

[0054] Figure 6 The curve showing the change of the cross-sectional area of ​​the beam with the number of training times during the training process of the present invention;

[0055] Figure 7 The curve showing the change of the beam design evaluation value with the number of training times during the training process of the present invention;

[0056] Figure 8 The curve showing the maximum stress of the beam changing with the number of training times during the test process of the present invention;

[0057] Fig. 9 The curve showing the variation of the cross-sectional area of ​​the beam with the number of training times during the test process of the present invention;

[0058] Fig.10 The curve showing the change of the beam design evaluation value with the number of training times during the test process of the present invention;

[0059] Fig.11 This is a schematic diagram of the I-beam interface design with the highest evaluation value in the embodiment. DETAILED DESCRIPTION

[0060] In order to facilitate the understanding of the present invention, the device of the present invention will be described more fully below with reference to the relevant drawings. Embodiments of the device are given in the drawings. However, the device can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0061] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "set" should be understood in a broad sense, for example, they can be fixedly connected or set, or detachably connected or set, or integrally connected or set. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0062] Example

[0063] like Figure 1 As shown, this embodiment provides a multi-objective optimization design method for an aircraft wing beam cross section based on reinforcement learning, which specifically includes the following steps:

[0064] S1. The definition of state space needs to fully consider the characteristics of the specific design and the indicators that need to be optimized. In this embodiment, the I-shaped aircraft wing beam (such as Figure 2 The cross-sectional width ( b ),high( h )、Edge thickness( t1 )、Web thickness( t2 ) These four variables (such as Figure 3 As shown), it is formed into a state vector, that is, the state space is determined;

[0065] S2. According to the state space and specific design requirements, determine the action space, that is, the set of directions and quantities of single-step changes and adjustments of each parameter. In this embodiment, 8 design actions are set, namely, section width b +10mm, section width b-10mm, section height h +10mm, section height h -10mm, edge thickness t1 +1mm, edge thickness t1 -1mm, web thickness t2 +1mm and web thickness t2 -1mm;

[0066] S3, such as Figure 4 As shown, the design evaluation value function is set according to the set beam section evaluation index. In this embodiment, weight and strength are set as evaluation indexes. Since weight and cross-sectional area ( A ) is linearly related, and the smaller it is, the higher the evaluation value is. -k A , A is the cross-sectional area, k is a fixed coefficient greater than 0 and is taken as 1. When the maximum stress of the structure ( σ ) is greater than the material yield strength ( σ s ), the evaluation value is:

[0067] ;

[0068] In the above formula, r is the design evaluation value function, C is a fixed penalty value, which is 1000. If the design value exceeds the design boundary value, the design will not be evaluated, but a fixed penalty value will be directly given, which is set to 1E-9;

[0069] S4. Based on the definition of the state space and the definition of the action space, a virtual environment is built to simulate the behavior of the beam structure. The virtual environment can update the state according to the action of the intelligent design body of the beam structure and calculate the design evaluation value, specifically:

[0070] In this embodiment, the updated state is calculated ( s_ ), that is, the section width ( b_ ),high( h_ )、Edge thickness( t1_ ) and web thickness ( t2_ ), and calculate the maximum stress ( σ ) and cross-sectional area ( A ),Right now:

[0071] ;

[0072] According to the set design evaluation value function, the evaluation values ​​before and after the design change are calculated respectively ( r , r_ ), calculate the reward value of the current action ( reward ):

[0073] ;

[0074] Finally, the updated state ( s_ ) and reward value ( reward ) is fed back to the beam structure intelligent design body;

[0075] S5. Create an intelligent design entity for beam structures that understands the relationship between states and actions for learning and optimization in a virtual environment. Specifically:

[0076] S5.1. Establish all states ( s ) for each design action ( a ) design action evaluation value table ;

[0077] S5.2. Establish an action selection function. Use a random method to select the action with the highest evaluation value using a greedy algorithm under a certain probability, and randomly select an action under other probabilities. In this embodiment, the random function generates a real number between 0 and 1. When the value is less than or equal to 0.9, the action with the highest evaluation value is selected, and when it is greater than 0.9, a random action is selected;

[0078] S5.3. Establish the current design status ( s )、Design Action( a )、New Design Status( s_ ) and reward value ( reward ) is designed to update the action evaluation value table. The action evaluation value table update method is:

[0079] ;

[0080] In the above formula, is the reduction factor, lr is the learning rate;

[0081] S6, such as Figure 5-Figure 7 As shown in the figure, the intelligent design body of the beam structure is trained to predict different states and action results, as follows:

[0082] S6.1. Set the maximum width, maximum height, maximum flange thickness, maximum web thickness, beam length, material yield strength, and loading conditions of the beam section, and set the number of training times and the number of steps for a single training. In this embodiment, the maximum width is 100 mm, the maximum height is 100 mm, the maximum flange thickness is 10 mm, the maximum web thickness is 10 mm, the beam length is 10 m, the material selected is the 7075 series aluminum alloy, which is a common load-bearing structural material in aviation, with a yield strength of 503 MPa, the number of training times is 10,000 times, and the number of design steps in a single training is 80 steps;

[0083] S6.2. At the beginning of each training, the initial design state is randomly generated;

[0084] S6.3, use the beam structure intelligent design body to generate design actions;

[0085] S6.4, transferring the current design state and the design action performed to the beam virtual environment constructed in step S4, and obtaining an updated new design state and an evaluation value of the new design state;

[0086] S6.5, updating the design action evaluation value table according to the design action evaluation value table updating method in step S5.3;

[0087] S6.6, setting the updated design state as the current design state;

[0088] S6.7, repeat steps S6.3 to S6.6 until the set number of single training steps is reached;

[0089] S6.8, repeat steps S6.2 to S6.7 to perform training until the set number of training times is reached;

[0090] S6.9, save the design action evaluation value table. Note that what is saved is the evaluation value of a design action under a specific state calculated according to step S5.3, rather than the evaluation value of a design state calculated in S3;

[0091] S7. Use the trained beam structure intelligent design body to optimize the design in the virtual environment and generate a new design scheme, as follows:

[0092] S7.1, load the design action evaluation value table after training in step S6.9, set the number of design attempts and the number of design steps in each design attempt. In this embodiment, the number of design attempts is 50 times, and the number of design steps in each design attempt is 40 steps;

[0093] S7.2, randomly generate initial design states;

[0094] S7.3, use the beam structure intelligent design body to generate design change actions;

[0095] S7.4, transmitting the current design state and the performed design action to the beam virtual environment, obtaining the updated design state and the evaluation value of the new design state;

[0096] S7.5, setting the updated state as the current state;

[0097] S7.6, repeat steps S7.3 to S7.5 until the single design step number is reached;

[0098] S7.7. Repeat steps S7.2 to S7.6 for training until the set number of design attempts is reached, where the first 10 rows and the last 10 rows of design solutions are shown in the following table:

[0099] n b h t2 σ A r 1 40 90 1 448.5 636 -636 2 30 100 1 468.3 564 -564 3 30 100 6 479.8 840 -840 4 90 90 4 495 704 -704 5 60 90 5 484.7 780 -780 6 40 80 3 487.7 758 -758 7 80 50 8 495.9 1264 -1264 8 60 90 1 481.7 562 -562 9 40 90 1 448.5 636 -636 10 70 100 1 472.8 514 -514 … … … … … … … 41 60 90 1 481.7 562 -562 42 40 100 1 447 568 -568 43 40 90 1 448.5 636 -636 44 30 100 1 468.3 564 -564 45 90 50 8 499.8 1220 -1220 46 30 100 6 479.8 840 -840 47 100 60 9 495.1 1086 -1086 48 40 70 7 492.7 1018 -1018 49 50 90 1 475.3 580 -580 50 90 100 2 494.1 552 -552

[0100] S7.8, so the design with the highest evaluation value can be obtained (such as Fig.11 shown), i.e. the section width b =70mm, height h =100mm, edge strip thickness t1 =3mm and web thickness t2 =1mm, its stress value σ =472.8MPa, cross-sectional area A =514mm 2 , evaluation value r =-514;

[0101] S8, such as Figure 8-Figure 10 As shown, the design solution generated in step S7 is tested and evaluated according to the actual situation to check whether it is feasible. =503MPa, which is greater than the maximum stress of the beam in the design scheme, so the beam will not be damaged and the design scheme is feasible.

[0102] It should be noted that the structure described in the present invention can be implemented in a variety of different forms and is not limited to the described embodiments. Any equivalent transformations made by ordinary technicians in this field using the contents of the present invention description and drawings, or directly or indirectly applied to other related technical fields, such as the loading and unloading of other items, are included in the protection scope of the present invention.

Claims

1. A multi-objective optimization design method for aircraft wing beam cross section based on reinforcement learning, characterized by: Including space definition, beam structure virtual environment construction and intelligent optimization design, specifically including the following steps: S1. Construct a state vector based on the parameters of the beam section to complete the definition of the state space; S2, define specific actions according to the constituent parameters in the state space to complete the definition of the action space; S3, setting a design evaluation value function according to the set beam section evaluation index; The evaluation value function r for: ; In the above formula, r To design the evaluation function, σ is the maximum stress of the structure, σ s is the material yield strength, C is a fixed penalty value, A is the cross-sectional area, k It is a fixed coefficient greater than 0. When the design value exceeds the design boundary value, the design will not be evaluated and a fixed penalty value will be directly given. C That's it; According to the value function r Calculate the evaluation values ​​before and after the design change r and r_ , and calculate the reward value of the current action reward : ； The updated status s_ and reward value reward Feedback to the beam structure intelligent design body; S4. Based on the definition of the state space and the definition of the action space, a virtual environment is built to simulate the behavior of the beam structure. The virtual environment can update the state according to the action of the intelligent design body of the beam structure and calculate the design evaluation value; S5. Create an intelligent design agent for beam structures that understands the relationship between states and actions for learning and optimization in a virtual environment. S6. Training the intelligent design body of the beam structure to predict different states and action results; S7, using the trained beam structure intelligent design body to optimize the design in a virtual environment and generate a new design solution; S8. Evaluate the design solution generated in step S7 according to the actual situation to check whether it is feasible.

2. The multi-objective optimization design method for an aircraft wing beam cross section based on reinforcement learning as claimed in claim 1, characterized in that: The parameters in step S1 include but are not limited to the width, height, flange thickness, web thickness, shape and length of the beam section.

3. The multi-objective optimization design method for an aircraft wing beam cross section based on reinforcement learning as claimed in claim 1, characterized in that: The step S5 specifically comprises the following steps: S5.

1. Establish all states s Each design action a Design action evaluation table ; S5.

2. Establish action selection function: Select the design action with the highest evaluation value or a random design action by random method; S5.

3. Establish the current design status s , Design Action a , New Design Status s_ and reward value reward Method for updating the design action evaluation value table: ； In the above formula, is the reduction factor, lr is the learning rate.

4. The multi-objective optimization design method for an aircraft wing beam cross section based on reinforcement learning as claimed in claim 3, characterized in that: The step S6 specifically comprises the following steps: S6.

1. Set the maximum width, maximum height, maximum flange thickness, maximum web thickness, beam length, material yield strength, and loading conditions of the beam section in step S1, and set the number of training times and the number of steps for a single training; S6.

2. At the beginning of each training, the initial design state is randomly generated; S6.

3. Use the beam structure intelligent design body to generate design actions in the current design state; S6.4, transferring the current design state and the design action performed to the beam virtual environment constructed in step S4, and obtaining an updated new design state and an evaluation value of the new design state; S6.5, updating the design action evaluation value table according to the design action evaluation value table updating method in step S5.3; S6.6, setting the updated design state as the current design state; S6.7, repeat steps S6.3 to S6.6 until the set number of single training steps is reached; S6.8, repeat steps S6.2 to S6.7 to perform training until the set number of training times is reached; S6.

9. Save the design action evaluation value table.

5. The multi-objective optimization design method for an aircraft wing beam cross section based on reinforcement learning as claimed in claim 1, characterized in that: The step S7 specifically includes the following steps: S7.1, loading the design action evaluation value table after training in step S6.9, setting the number of design attempts and the number of design steps in each design attempt; S7.2, randomly generate initial design states; S7.3, use the beam structure intelligent design body to generate design change actions; S7.4, transmitting the current design state and the performed design action to the beam virtual environment, obtaining the updated design state and the evaluation value of the new design state; S7.5, setting the updated state as the current state; S7.6, repeat steps S7.3 to S7.5 until the single design step number is reached; S7.7, repeat steps S7.2 to S7.6 to perform training until the set number of design attempts is reached; S7.

8. Output the design with the highest evaluation value.

Images