A general intelligent design method for composite laminates based on reinforcement learning

Abstract

The application discloses a composite laminate general intelligent design method based on reinforcement learning, which comprises the following steps: firstly, a composite laminate design environment is established, a layer angle combination is defined as a state space, adding, deleting and modifying a layer are defined as an action space, and mechanical properties are calculated and rewards are evaluated in combination with a classic laminate theory and a Tsai-Hill failure criterion; secondly, an intelligent design body is created, a design action scoring table is established, and a state value is updated in combination with a greedy algorithm; subsequently, the design body is trained through traversal of a load factor; finally, the trained intelligent body is used for general design, and an optimal layer arrangement scheme is output; the application overcomes the defect that a traditional optimization algorithm is highly dependent on single load fitting, the trained intelligent body can output a layer scheme meeting strength requirements and being lightest in weight at one time when facing a brand-new load, and the design efficiency and generality of the composite material are greatly improved.

Description

Technical Field

[0001] This invention belongs to the field of composite material design technology, and more specifically, relates to a general intelligent design method for composite laminates based on reinforcement learning. Background Technology

[0002] Composite materials have many advantages, such as high specific strength and specific stiffness, and good fatigue resistance. They account for an increasingly large proportion of aircraft materials, with the proportion exceeding 50% in aircraft such as the Airbus A350 and Boeing 787. In aircraft, composite laminates are the most common structural form and are widely used in aircraft skin, beams, walls and other structures.

[0003] Traditional aircraft composite laminate design methods are primarily based on experience and trial and error, resulting in low design efficiency and difficulty in achieving optimal results. While conventional optimization algorithms (such as data-driven methods based on static database fitting) have some application, these methods often require collecting data from scratch and iteratively optimizing for different structures and load requirements, lacking generalization ability. Aircraft design is a process that integrates multiple disciplines and iterative optimization, and the number of laminates to be designed is enormous.

[0004] The manufacturing characteristic of composite materials, which consist of multiple overlapping layers, dictates that their design parameters (such as the number of layers and the layup angle) are highly discrete. This discreteness makes them naturally suitable for "action-oriented" reinforcement learning methods (such as Q-Learning) rather than numerical searches of continuous functions. By training the agent to perform actions such as "adding, deleting, and rotating," allowing it to accumulate design experience, the problem of poor generality in traditional static data fitting can be fundamentally solved. Summary of the Invention

[0005] To address the above shortcomings, this invention provides a general intelligent design method for composite laminates based on reinforcement learning, comprising the following steps:

[0006] S1. Establish the design environment for composite laminates: This includes establishing a state space characterizing the layup state of the laminate, establishing an action space matching the discrete manufacturing characteristics of composite materials, and calculating the mechanical properties and failure index of the composite laminate under different states based on the set material mechanical properties and external loads. And construct a reward function for multi-objective optimization;

[0007] S2. Create an intelligent design entity for composite laminates: Establish and initialize a design action scoring table for each design action in all states. A motion selection function based on a combination of greedy algorithm and random exploration is defined, and a motion scoring table is designed. Update method;

[0008] S3. Perform strategy learning and training on the intelligent design agent: Define load factors, and use the traversal of load factors as environmental input to enable the intelligent design agent to continuously interact with the design environment under different loads and materials, and update the design action scoring table according to the reward function. Train the agent's action logic rules;

[0009] S4. Perform general design using the trained intelligent design entity: Load the training completed design action scoring table. Given any specific external load and design requirements, the intelligent design entity will use a greedy strategy based on the current state to output the action sequence with the highest evaluation value, thus generating the final composite laminate layup design scheme.

[0010] Further, in step S1, the state space is composed of the combination of the angles of each ply on one side of the symmetrical ply, and the formula for calculating the total laminate ply state value s is:

[0011]

[0012] Where n is the current ply number and m is the ply angle number. From the outside to the inside Layer state values;

[0013] The maximum value of s The following formula is used to derive:

[0014]

[0015] Furthermore, in step S1, the action space includes three design actions: adding a ply to the innermost layer, rotating the ply angle of the current innermost ply by a specific angle, and deleting the innermost ply.

[0016] Based on the action, determine whether the current number of layers n is out of bounds, and update the state value to obtain the new design state s_.

[0017] Further, in step S1, the calculation of the mechanical properties of the composite laminate includes: calculating the stiffness matrix of each ply under plane stress, and introducing the thickness direction coordinate z to calculate the transformation matrix [Q]. ij The tensile stiffness was calculated based on the constitutive equation of the anisotropic laminate. Coupling stiffness and bending stiffness Then, the stress and strain of each layer are calculated.

[0018] Furthermore, the failure index in step S1 Calculations were performed using the Tsai-Hill failure criterion:

[0019]

[0020]

[0021]

[0022] Among them, when The material was considered to have failed at that time; and The stresses in directions 1 and 2 of the material are S represents the shear stress; X and Y represent the strengths of the material in directions 1 and 2, respectively; and S represents the shear strength. and These represent the tensile strength and compressive strength in direction 1 of the material, respectively. and These represent the tensile strength and compressive strength of the material in direction 2, respectively.

[0023] Furthermore, in step S1, the reward function comprehensively considers the mechanical performance failure index and the weight of the laminate, and its current state design evaluation value Defined as:

[0024]

[0025] Where K1 is the composite material failure penalty factor, K2 is the composite material lightweighting reward factor, n is the number of layers, and f is the failure index;

[0026] If the number of plies n exceeds the maximum number of plies If n=0, a fixed penalty value is applied.

[0027] Furthermore, in step S2, the greedy ratio of the action selection function... The probability increases with the number of training steps; when randomly selecting actions, a non-uniform probability distribution is used, so that the probability of triggering actions that increase the number of layers is higher than that of actions that delete the number of layers.

[0028] Further, in step S2, the design action scoring table The update formula is:

[0029]

[0030]

[0031]

[0032] in, This represents the current design state, where 'a' represents the design action. For the new design status, Here, lr is the reduction factor, lr is the learning rate, and reward is the reward value for the designed action. To design an action scoring sheet, Score the current prediction. Rate the expected outcome.

[0033] Compared with the prior art, the present invention has the following advantages:

[0034] 1. Completely different from data-driven fitting: The design process is transformed into strategy learning of discrete actions (addition, deletion, modification), perfectly matching the actual manufacturing process characteristics of composite materials.

[0035] 2. Excellent versatility (one-time training, universal design): The intelligent agent learns universal mechanical balance and lightweight laws, and does not need to be retrained when facing new working conditions, which greatly shortens the development cycle of aircraft composite material parts.

[0036] 3. Multi-objective intelligent optimization: A self-built reward system that includes failure and weight penalties is used to achieve maximum weight reduction while ensuring that the structure is not damaged. Attached Figure Description

[0037] Figure 1 This is a flowchart of the present invention;

[0038] Figure 2 This is a schematic diagram of the reinforcement learning-based composite laminate design method in this invention;

[0039] Figure 3 This is a schematic diagram of the composite material laminate layup in this invention;

[0040] Figure 4 This is a schematic diagram of the forces acting on the composite laminate in this invention;

[0041] Figure 5 This is the curve showing the change of the maximum stress of the composite laminate during the training process in this invention with the number of training sessions;

[0042] Figure 6 This is a curve showing the change in the number of composite laminate layers as a function of the number of training sessions during the training process in this invention.

[0043] Figure 7 This is a curve showing the change of the composite laminate design evaluation value with the number of training sessions during the training process in this invention;

[0044] Figure 8 This is the curve showing the change of the maximum stress of the composite laminate as a function of the number of training sessions during the testing process in this invention;

[0045] Figure 9 This is a curve showing the change in cross-sectional area of ​​the composite laminate as a function of the number of training sessions during the testing process in this invention;

[0046] Figure 10This is a curve showing the change of the composite laminate design evaluation value with the number of training sessions during the testing process in this invention;

[0047] Figure 11 This is a schematic diagram of the composite laminate design with the highest evaluation value in the examples. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] Example

[0050] like Figure 1-2 As shown, this embodiment provides a general intelligent design method for composite laminates based on reinforcement learning, including the following steps:

[0051] S1. Establish a composite laminate design environment to complete state transition, mechanical property calculation, failure assessment, and reward value feedback, specifically:

[0052] S1.1 Establish the state space and generate the corresponding layering scheme based on the input state:

[0053] Because symmetrically laid composite laminates are the most commonly used in practical applications and avoid problems such as warping, such as Figure 3 As shown, all laminates in this invention are symmetrically plyed (unless otherwise specified, the ply descriptions generally represent only half of the actual ply). In this embodiment, the combination of ply angles is taken as the state value, and the maximum number of ply layers is set. =15, ply angle number m=3 (i.e., 0°, 45°, 90° three ply angles), a total of 15 layers, each layer has 3 ply states (s i ), as shown in Table 1:

[0054] Table 1. Layer Status Corresponding to Ply Angles

[0055] 1 0° 2 45° 3 90°

[0056] Since the mechanical properties of symmetrical plywood are balanced, the formula for calculating the overall plywood state value s is as follows:

[0057]

[0058] Where n is the current number of plies, and m is the ply angle. In this embodiment, m=3, therefore the formula is:

[0059]

[0060] The maximum value of the current design state value s The following formula is used to derive:

[0061]

[0062] In this embodiment, by substituting m=3 into the above formula, =15, calculated as follows This determines the size of the state space.

[0063] For the input s, the state values ​​s of each layer from the outside to the inside are first decomposed by the modulo operation. i Then, according to Table 1, convert them to the angles corresponding to each ply.

[0064] S1.2 Establish the action space and calculate the state changes of the laminate based on the input design actions:

[0065] Three design actions are defined, as shown in Table 2 below. For each input action, the system first determines whether adding or deleting a ply will cause the number of ply counts to exceed the maximum limit or drop to 0. Then, the specific action is executed, and the current number of ply counts n and the ply status s are updated. n Then update the state value to get the new design state s_;

[0066] Table 2 shows the specific actions corresponding to the action values.

[0067] 0 <![CDATA[n=n+1,s n =1]]> Add a 0° ply to the innermost layer. 1 <![CDATA[s n =s n %4+1]]> The ply angle of the current ply is increased by 45° (it will automatically reset to 0° when it reaches 90°). 2 <![CDATA[Dels n ,n=n-1]]> Remove the innermost layer

[0068] S1.3 Calculate the mechanical properties of composite laminates under different conditions:

[0069] For a layup under plane stress, the stress-strain relationship is:

[0070]

[0071] Among them, C11, C12, C21, C22 and C66 are stiffness coefficients;

[0072]

[0073]

[0074]

[0075] Where E1 and E2 are the elastic moduli in the fiber direction (direction 1) and perpendicular to the fiber direction (direction 2) within the layup, respectively, and G 12 Shear modulus and Given two different Poisson ratios, their relationship is expressed by the following formula:

[0076]

[0077] Considering the ply angle The transformation relationship of the elastic constants of layer i along the x-axis and y-axis can be written as:

[0078]

[0079] In the formula , These are the normal stresses in the x and y directions, respectively. For the shear stress in the xy plane, , The normal strains in the x and y directions are respectively. Let x be the shear strain in the xy plane, and z be the coordinate in the thickness direction. , The curvature of the deformation about the x and y axes, respectively, [Q ij The transformation matrix is ​​as follows:

[0080]

[0081]

[0082]

[0083]

[0084]

[0085]

[0086] The constitutive equation for anisotropic laminates can be written as:

[0087]

[0088]

[0089]

[0090]

[0091] In the formula, in the formula It is tensile stiffness. It is the coupling stiffness. It is the bending stiffness; for symmetrical laminates, the coupling stiffness is... , It is the height of the bottom surface of the kth layer;

[0092] S1.4, such as Figure 4 As shown, based on the input material mechanical properties and external load, the overall strain and curvature are calculated using the following formula, and then the stress and strain of each layer are obtained:

[0093]

[0094]

[0095]

[0096]

[0097] In the formula, For laminate strain, Deformation curvature (referring to the curvature of the laminate when it bends) For axial force, For bending moment, and These are the tensile stiffness matrix, the coupling stiffness matrix, and the bending stiffness matrix, respectively.

[0098] In this embodiment, the composite material unidirectional tape is T700, and its mechanical properties are shown in Table 3:

[0099] Table 3 Mechanical Performance Parameters of T700 Unidirectional Belt

[0100] 1500 135 10 5 5 3.8 2550 1470 69 345 97

[0101] External load is:

[0102]

[0103] S1.5. Determine whether the ply has failed based on the selected failure criterion:

[0104] In this embodiment, the failure index and the number of ply layers are set as evaluation value functions, and the change in the evaluation value caused by the current design action is used as the reward value. The failure index measures whether the current ply design meets the strength requirements under a specified load, i.e., whether the laminate will fail, while the number of ply layers is directly related to the weight. Failure criteria for calculating the failure index include maximum stress, maximum strain, Tsai-Wu, Tsai-Hill, Hashin, and Chang-Chang. This embodiment uses the Tsai-Hill failure criterion, namely:

[0105]

[0106]

[0107]

[0108] in The failure index is when The material was considered to have failed at that time; and These are the stresses in directions 1 and 2 of the material (usually the fiber direction and the in-plane direction perpendicular to the fibers). S represents the shear stress; X and Y represent the strengths of the material in directions 1 and 2, respectively; and S represents the shear strength. and These represent the tensile strength and compressive strength in direction 1 of the material, respectively. and These represent the tensile strength and compressive strength of the material in direction 2, respectively.

[0109] S1.6, (Based on the actual design requirements of composite laminates) Set the reward function for the design action:

[0110] Since this embodiment does not consider material changes, the composite material density and unidirectional tape thickness are constants, meaning the weight and number of plies are linearly related. Therefore, with a failure index not greater than 1, a lower number of plies results in a higher design evaluation value.

[0111]

[0112] Where K1 is the composite material failure penalty factor, K2 is the composite material lightweighting reward factor, n is the number of layers, and f is the failure index. In this embodiment, , ,Right now:

[0113]

[0114] Additionally, if the number of layers n exceeds the limit (i.e. When this occurs, a fixed penalty value is directly applied, set to [value]. ,Right now:

[0115]

[0116] The reward function is defined as the difference between the evaluation values ​​before and after performing an action:

[0117]

[0118] Where value is the ply design evaluation value in the current state, and value_ is the ply design evaluation value after the design action changes.

[0119] S2. Create an intelligent design entity for composite laminates that can autonomously design and learn within the environment:

[0120] S2.1 Establish a design action scoring table for each design action (a) under all states (s). And initialize;

[0121] S2.2 Establish an action selection function. Using a random method, a greedy algorithm is employed to select the action with the highest evaluation value under certain probabilities. Under other probabilities, an action is randomly selected, as detailed below:

[0122] Using the greedy ratio The method of increasing the threshold as training progresses is to first set an initial threshold (i.e., the greedy ratio). ) After every 1000 steps of training, until Each time an action is selected, a real number between 0 and 1 is generated by a random function. When this value is less than... Choose the action with the highest evaluation value, and select the action with a higher evaluation value. An action is randomly selected. Since actions involving adding layers are more frequent in actual design, the random selection involves generating an integer p between 0 and 9 and assigning different actions unequal probabilities.

[0123] If p < 6, then perform the add layer (action 0);

[0124] If p≥8, then execute action 2, which is to delete the layer;

[0125] If the result is an integer between 6 and 7, then action 1 is performed, which is to rotate the ply direction;

[0126] S2.3 Establish a method for updating the design action scoring table:

[0127] Design Action Scoring Table based on current design state s, design action a, new design state s_, and design action reward value. renew:

[0128]

[0129]

[0130]

[0131] In the formula, Here, lr is the reduction factor, lr is the learning rate, and reward is the reward value for the designed action. To design an action scoring sheet, Score the current prediction. Rate the expected outcome;

[0132] S3. Training the intelligent design agent:

[0133] The composite material properties are set and the load factor is defined. Training parameters are also set. In this embodiment, the composite unidirectional strip is set to T700 unidirectional strip, and its material mechanical properties are shown in Table 3. The training times are 10,000 times, and the number of design steps in a single training session is 50 steps. Each training session starts with a single layer at 0° layup. The intelligent design agent generates a design action a based on the current design state s, and then transmits the current design state s and the generated design action a to the composite laminate design environment built in step S1 to obtain the updated new design state s_ and the reward value reward for the design action. Then, the design action scoring table in step S2.3 is used to determine the reward value reward. Update method update design action scoring table And assign the new design state s_ to the current design state s, repeating until the set number of training steps and training iterations are reached (e.g., ...). Figure 5-7 (As shown), finally save the design action scoring sheet. ;

[0134] This allows the intelligent design entity to learn the general rules of layering, rather than being limited to fitting a single load;

[0135] S4. Use intelligent design bodies for general design:

[0136] Load the designed action scoring sheet after training in step S3. The design attempt is 1 time, with 20 design steps, starting with a single 0° layup. Then, the composite laminate intelligent design body generates design change actions. It's important to note that at this point, a greedy method is used to directly select the action with the highest current state evaluation value, instead of a random method. The current design state s and the generated design action a are then passed to the composite laminate design environment to obtain the updated design state s_ and the evaluation value value_ for the new design state. The updated state is then set as the current state, and this process is repeated until the required number of design steps (e.g., ...) is reached. Figure 8-10 As shown), the design with the highest evaluation value is finally obtained (e.g. Figure 11 As shown), there are a total of 24 ply layers, with ply angles of [45 / -45 / 0 / 0 / 0 / 45 / -45 / 0 / 0 / 0 / 45 / -45]. s The maximum failure index is 0.9, and the design evaluation value is 60.

[0137] S5. Inspect whether the composite material laminate in the design scheme will be damaged:

[0138] In this embodiment, the design scheme generated in step S4 was evaluated using finite element simulation. The results show that the composite laminate will not fail under the corresponding load, which not only verifies the feasibility of the design scheme, but also proves that the intelligent design entity can directly output an excellent and highly versatile design scheme without retraining.

[0139] It should be noted that the structure described in this invention can be implemented in many different forms and is not limited to the embodiments described. Any equivalent transformations made by those skilled in the art based on the description and drawings of this invention, or direct or indirect applications in other related technical fields, are included within the protection scope of this invention.

Claims

1. A general intelligent design method for composite laminates based on reinforcement learning, characterized in that, Includes the following steps: S1. Establish the design environment for composite laminates: This includes establishing a state space characterizing the layup state of the laminate, establishing an action space matching the discrete manufacturing characteristics of composite materials, and calculating the mechanical properties and failure index of the composite laminate under different states based on the set material mechanical properties and external loads. And construct a reward function for multi-objective optimization; S2. Create an intelligent design entity for composite laminates: Establish and initialize a design action scoring table for each design action in all states. A motion selection function based on a combination of greedy algorithm and random exploration is defined, and a motion scoring table is designed. Update method; S3. Perform strategy learning and training on the intelligent design agent: Define load factors, and use the traversal of load factors as environmental input to enable the intelligent design agent to continuously interact with the design environment under different loads and materials, and update the design action scoring table according to the reward function. Train the agent's action logic rules; S4. Perform general design using the trained intelligent design entity: Load the training completed design action scoring table. Given any specific external load and design requirements, the intelligent design entity will use a greedy strategy based on the current state to output the action sequence with the highest evaluation value, thus generating the final composite laminate layup design scheme.

2. The general intelligent design method for composite laminates based on reinforcement learning as described in claim 1, characterized in that: In step S1, the state space is composed of the combination of the angles of each ply on one side of the symmetrical ply, and the formula for calculating the current design state s is: Where n is the current ply number and m is the ply angle number. From the outside to the inside Layer state values; The maximum value of s The following formula is used to derive:

3. The general intelligent design method for composite laminates based on reinforcement learning as described in claim 1, characterized in that: In step S1, the action space includes three design actions: adding a ply to the innermost layer, rotating the ply angle of the current innermost ply by a specific angle, and deleting the innermost ply. Based on the action, determine whether the current number of layers n is out of bounds, and update the state value to obtain the new design state s_.

4. The general intelligent design method for composite laminates based on reinforcement learning as described in claim 1, characterized in that: In step S1, calculating the mechanical properties of the composite laminate includes: calculating the stiffness matrix of each ply under plane stress, and using the thickness direction coordinate z to calculate the transformation matrix [Q]. ij The tensile stiffness was calculated based on the constitutive equation of the anisotropic laminate. Coupling stiffness and bending stiffness Then, the stress and strain of each layer are calculated.

5. The general intelligent design method for composite laminates based on reinforcement learning as described in claim 1, characterized in that: The failure index in step S1 Calculations were performed using the Tsai-Hill failure criterion: Among them, when The material was considered to have failed at that time; and The stresses in directions 1 and 2 of the material are S represents the shear stress; X and Y represent the strengths of the material in directions 1 and 2, respectively; and S represents the shear strength. and These represent the tensile strength and compressive strength in direction 1 of the material, respectively. and These represent the tensile strength and compressive strength of the material in direction 2, respectively.

6. The general intelligent design method for composite laminates based on reinforcement learning as described in claim 5, characterized in that: In step S1, the reward function comprehensively considers the mechanical performance failure index and the weight of the laminate, and its current state design evaluation value is... Defined as: Where K1 is the composite material failure penalty factor, K2 is the composite material lightweighting reward factor, n is the number of layers, and f is the failure index; If the number of plies n exceeds the maximum number of plies If n=0, a fixed penalty value is applied.

7. The general intelligent design method for composite laminates based on reinforcement learning as described in claim 1, characterized in that: In step S2, the greedy ratio of the action selection function The probability increases with the number of training steps; when randomly selecting actions, a non-uniform probability distribution is used, so that the probability of triggering actions that increase the number of layers is higher than that of actions that delete the number of layers.

8. The general intelligent design method for composite laminates based on reinforcement learning as described in claim 1, characterized in that: In step S2, the design action scoring table The update formula is: in, This represents the current design state, where 'a' represents the design action. For the new design status, Here, lr is the reduction factor, lr is the learning rate, and reward is the reward value for the designed action. Score the current prediction. Rate the expected outcome.

Images