A wing box lightweight size optimization method and system based on proximal policy optimization

Abstract

The application belongs to the field of finite element structure optimization, and discloses a wing box lightweight size optimization method and system based on a proximal policy optimization, which constructs a wing box size optimization problem as a reinforcement learning sequential decision process, takes size parameters of structural components such as a wing box skin, a spar, a rib, a wall plate, a reinforcing rib or a stringer as design variables, takes minimization of structural mass as an optimization target, and takes strength constraints, displacement constraints and manufacturing constraints as design constraints. The method solves the problems of high finite element evaluation cost, insufficient training stability of a PPO algorithm with fixed hyperparameters, high artificial parameter adjustment cost, difficulty in stably maintaining constraint feasibility and insufficient adaptability of a training strategy in existing high-fidelity finite element structure optimization, can improve the training stability of the PPO algorithm in high-fidelity wing box size optimization, reduce the artificial parameter adjustment cost, and obtain a lightweight design scheme under the premise of meeting the structural strength and displacement constraints.

Description

Technical Field

[0001] This invention belongs to the field of finite element structure optimization, specifically relating to a method and system for lightweight wing box size optimization based on near-end strategy optimization. Background Technology

[0002] The wing box is the main load-bearing structure in an aircraft wing, typically composed of components such as skin, spars, ribs, panels, stiffeners, or stringers. Its structural dimensions directly affect the wing's structural mass, strength margin, stiffness characteristics, and deformation response. In aerospace structural design, lightweight wing box dimensional optimization usually requires minimizing structural mass while meeting strength, stiffness, displacement, and manufacturing constraints, thereby improving the aircraft's fuel economy and overall performance.

[0003] In current engineering practice, wing box size optimization typically requires repeated calls to finite element analysis programs to calculate structural mass, stress response, displacement response, and failure constraints under different structural size schemes. For high-fidelity finite element models containing multiple load-bearing components and numerous local stress constraints, the cost of a single analysis is high, and the structural response is significantly affected by load conditions, material parameters, boundary conditions, and component coupling relationships. Therefore, how to obtain a lightweight design that meets structural safety constraints within a limited simulation evaluation budget is a key technical problem in wing box structural optimization.

[0004] Traditional wing box structure optimization methods mainly include gradient-based optimization methods and gradient-free optimization methods. Gradient-based optimization methods are highly efficient when the objective and constraint functions are relatively smooth and sensitivity information is readily available. However, for structural optimization problems involving high-fidelity finite element models, complex load conditions, and nonlinear failure constraints, sensitivity calculation and constraint handling are quite challenging. Gradient-free optimization methods, such as genetic algorithms and particle swarm optimization, do not rely on gradient information and have a certain global search capability. However, they typically require repeated finite element analyses of a large number of candidate designs, resulting in high computational costs and making it difficult to meet the rapid iteration requirements of engineering design.

[0005] To reduce the number of real evaluations in expensive simulation optimization, existing technologies have proposed surrogate-assisted optimization and Bayesian optimization methods. These methods typically utilize existing simulation samples to construct surrogate models and select new candidate evaluation points through acquisition functions or local search strategies, thereby reducing the number of real simulation calls. While these methods have certain advantages in expensive black-box optimization, they primarily focus on the candidate point selection process and often struggle to directly formulate reusable continuous size adjustment strategies. Furthermore, they do not fully integrate the structural mass, stress response, displacement response, and failure constraint feedback from the finite element analysis of the wing box to construct a closed-loop optimization strategy.

[0006] In recent years, deep reinforcement learning has provided a new technical approach for structural size optimization. It can transform the structural optimization problem into a sequential decision-making process between an agent and its environment. The agent outputs size adjustment actions based on the current structural state and learns optimization strategies step by step through environmental feedback. Proximal Policy Optimization (PPO) algorithms, due to their good policy update stability and applicability to continuous action spaces, can be used for continuous adjustment of the thickness or dimensional parameters of components such as wing box skin, spars, and ribs. However, in high-fidelity wing box size optimization tasks, each agent interaction may involve calling the finite element solver to calculate structural mass, stress response, displacement response, and failure constraint indicators. Therefore, the cost of acquiring training samples is high, placing higher demands on training stability and sample utilization efficiency.

[0007] Existing fixed-hyperparameter PPO algorithms still have shortcomings when applied to high-fidelity dimensional optimization of wing boxes. The training performance of PPO algorithms is typically highly sensitive to hyperparameters such as the learning rate, discount factor, and clipping coefficient. Improper hyperparameter settings can easily lead to problems such as excessively large policy update amplitudes, training oscillations, slow convergence, or unstable final optimization results. Relying on repeated parameter tuning based on human experience requires multiple rounds of training and numerous finite element method calls, further increasing computational costs. Therefore, fixed-hyperparameter PPO algorithms struggle to adapt to the dynamic requirements of exploration capability, convergence stability, and constraint satisfaction at different training stages in wing box structure optimization.

[0008] To address the hyperparameter tuning problem, existing technologies include group-based online hyperparameter optimization methods. These methods maintain multiple parallel training individuals, allowing poorly performing individuals to inherit the model parameters of better-performing individuals and update the hyperparameters, thus achieving joint optimization of model weights and hyperparameters during training. However, general online hyperparameter optimization methods typically use ordinary training rewards or validation performance as the basis for individual evaluation, without specifically designing for the finite element feedback characteristics in wing box structure optimization. For wing box lightweight size optimization tasks, evaluating the merits of a training strategy should consider not only round rewards but also engineering indicators such as the degree of structural mass reduction, failure constraint indicators, displacement response, constraint margin, and feasible solution ratio. If the dynamic parameter tuning process is not coupled with the aforementioned structural constraint responses, problems may arise such as increased rewards but structural infeasibility, or decreased mass but violation of strength constraints.

[0009] Furthermore, general reinforcement learning training frameworks are typically not specifically designed for engineering constraint feedback in wing box structure optimization. Lightweight dimensional optimization of wing boxes requires not only reducing structural mass but also satisfying strength, displacement, and failure constraints. Due to the large number of local stress constraints in the finite element model of a wing box, directly using all local constraints as reinforcement learning feedback increases the complexity of state and reward design. In engineering structural optimization, the Kreisselmeier-Steinhauser function (KS-function) is often used to aggregate multiple local failure constraints, transforming them into a single continuous constraint index for constraint evaluation and optimization feedback. Therefore, how to reasonably incorporate finite element feedback information such as structural mass, displacement response, KS failure index, and constraint margin into the reinforcement learning state, reward, and training performance evaluation is a key issue in improving strategy feasibility and engineering applicability.

[0010] Meanwhile, the wing box structure dimensions have clear engineering upper and lower bounds and manufacturing constraints. The continuous actions output by the agent need to be stably mapped to actual physical size changes, ensuring that the updated dimensions of components such as the skin, spars, and ribs remain within a reasonable range. Improper handling of action boundaries can easily lead to invalid actions or infeasible structural dimensions, reducing training efficiency and affecting policy stability. Therefore, it is necessary to construct a state space, action space, and reward function optimized for wing box dimensions, enabling the agent to learn lightweight size adjustment strategies that satisfy engineering constraints under finite element feedback.

[0011] In summary, while existing technologies offer methods such as finite element structural analysis, traditional structural optimization, surrogate-assisted optimization, the PPO algorithm, and online hyperparameter tuning, a comprehensive engineering closed-loop solution for lightweight wing box dimensional optimization is still lacking. This solution should unify reinforcement learning agents, finite element solution environments, structural mass feedback, KS failure constraint aggregation, continuous dimension action mapping, and dynamic hyperparameter adaptive training mechanisms. In particular, existing methods have not adequately addressed the challenge of simultaneously improving PPO algorithm training stability, reducing manual hyperparameter tuning costs, maintaining structural constraint feasibility, and outputting reusable dimensional optimization strategies under the condition of high-fidelity finite element evaluation costs. Therefore, it is necessary to provide a method and system capable of automatically optimizing wing box dimensions under finite element feedback, dynamically adjusting key hyperparameters of the PPO algorithm during training, and reducing wing box mass while satisfying structural strength constraints. Summary of the Invention

[0012] The purpose of this invention is to provide a method and system for lightweight wing box size optimization based on near-end strategy optimization, in order to solve the problems existing in high-fidelity finite element structural optimization, such as high finite element evaluation cost, insufficient training stability of PPO algorithm with fixed hyperparameters, high cost of manual parameter tuning, difficulty in maintaining the feasibility of constraints, and insufficient adaptability of training strategies under different initial structural or load conditions. This invention can improve the training stability of PPO algorithm in high-fidelity wing box size optimization, reduce the cost of manual parameter tuning, and obtain a lightweight design scheme while meeting structural strength and displacement constraints.

[0013] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0014] Firstly, a method for optimizing the lightweight size of wing boxes based on near-end strategy optimization is provided, including:

[0015] Using the dimensional parameters of the load-bearing components of the wing box structure as design variables, a set of design variables including each design variable and its upper and lower bounds is established.

[0016] The current state vector is obtained and input into multiple parallel-trained proximal policy optimization agents to obtain normalized continuous actions. The state vector contains finite element analysis results and design variables. Each dimension of the normalized continuous action corresponds to the adjustment direction and adjustment magnitude of each design variable.

[0017] Based on the set of design variables, the actual size adjustment of each design variable is determined according to the normalized continuous action, and the wing box structure size is updated based on the actual size adjustment.

[0018] Finite element analysis is performed on the updated wing box structure dimensions, and the finite element analysis results including structural mass and KS failure index are output. The next state vector is generated, and the reward is calculated and the network parameters of the near-end policy optimization agent are updated based on the finite element analysis results.

[0019] According to the perturbation period, the PB2 dynamic hyperparameter adaptive mechanism is introduced to maintain multiple proximal policy optimization agents trained in parallel; until the training is completed, the proximal policy optimization agent with the best performance is output as the policy model.

[0020] The strategy model is used to continuously adjust the wing box structure dimensions of the aircraft to be optimized in a finite number of steps to obtain lightweight size design results.

[0021] Several alternative methods are provided below, but they are not intended as additional limitations on the overall solution above. They are merely further additions or optimizations. Provided there are no technical or logical contradictions, each alternative method can be combined individually with respect to the overall solution above, or multiple alternative methods can be combined with each other.

[0022] Preferably, the state vector includes finite element analysis results and design variables, as follows:

[0023] The state vector contains only the structural mass, KS failure index, and design variables after scaling using the scaling coefficient;

[0024] Alternatively, the state vector may include structural mass after scaling using the scaling coefficient, KS failure index, and design variables, and may also include at least one of the following: maximum displacement response, maximum local stress, load factor, normalized continuous action of the previous environmental interaction step, reward of the previous environmental interaction step, current environmental interaction step number, and finite element analysis convergence flag.

[0025] Preferably, the step of determining the actual size adjustment amount of each design variable based on the normalized continuous action, in conjunction with the set of design variables, includes:

[0026] Based on the minimum distance between the current design variable and its upper or lower bound, determine the action scaling factor for each design variable to form an action scaling factor vector; the action scaling factor increases as the minimum distance increases and decreases as the minimum distance decreases.

[0027] The step size decay coefficient is determined based on the current number of interaction steps in the environment, and the step size decay coefficient decreases as the number of interaction steps in the environment increases.

[0028] The normalized continuous motion, motion scaling factor vector, and step size decay factor are multiplied element-wise. The result of the element-wise multiplication is then applied to the maximum physical size adjustment to obtain the actual size adjustment of each design variable.

[0029] Preferably, updating the wing box structure dimensions based on the actual size adjustment includes:

[0030] The actual size adjustments of each design variable are superimposed on the current wing box structure size, and boundary constraints are executed based on the upper and lower bounds of the design variables to obtain the updated wing box structure size, which is the updated design variable.

[0031] Preferably, the calculation of the reward based on the finite element analysis results includes calculating a quality reward item, a strength constraint penalty item, and a displacement constraint penalty item;

[0032] The quality reward item is used to characterize the degree of structural quality reduction, including normalizing the initial structural quality and calculating the relative reduction in structural quality before and after the update of the wing box structural dimensions in the current iteration;

[0033] The strength constraint penalty term is used to characterize the degree to which the KS failure index exceeds the failure constraint threshold;

[0034] The displacement constraint penalty term is used to characterize the extent to which the maximum displacement response exceeds the displacement constraint threshold.

[0035] Preferably, the KS failure index is obtained by aggregating all local stress constraints or failure constraints through the KS function.

[0036] Preferably, the PB2 dynamic hyperparameter adaptive mechanism includes:

[0037] Calculate the performance metrics of each proximal policy optimization agent during the perturbation period;

[0038] Based on the performance metrics, all proximal policy optimization agents are ranked and divided into a high-performance agent set and a low-performance agent set.

[0039] For low-performance individuals, the policy network parameters, value network parameters, policy network optimizer state, and value network optimizer state are inherited from high-performance individuals.

[0040] Update the hyperparameters of low-performing individuals within a preset search range, and train the low-performing individuals based on the inherited network parameters and optimizer state, as well as the updated hyperparameters.

[0041] Preferably, the performance metric is the average round reward of the near-end policy optimization agent during the perturbation period.

[0042] Preferably, the hyperparameters include the learning rate, discount factor, and pruning factor.

[0043] Secondly, a lightweight wing box size optimization system based on near-end strategy optimization is provided, including:

[0044] The task initialization module is used to establish a set of design variables, including each design variable and its upper and lower bounds, using the dimensional parameters of the load-bearing components of the wing box structure as design variables.

[0045] The reinforcement learning environment interaction module is used to obtain the current state vector, which is then input into multiple parallel-trained proximal policy optimization agents to obtain normalized continuous actions. The state vector contains finite element analysis results and design variables. Each dimension of the normalized continuous action corresponds to the adjustment direction and adjustment magnitude of each design variable.

[0046] The size update module is used to combine the set of design variables, determine the actual size adjustment amount of each design variable according to the normalized continuous action, and update the wing box structure size based on the actual size adjustment amount;

[0047] The finite element solution and state update module is used to perform finite element analysis on the updated wing box structure dimensions, output finite element analysis results including structural mass and KS failure index, and generate the next state vector;

[0048] The reward calculation module is used to calculate the reward based on the finite element analysis results and update the network parameters of the near-end policy optimization agent.

[0049] The PPO policy training module is used to maintain multiple parallel-trained proximal policy optimization agents according to the perturbation period by introducing the PB2 dynamic hyperparameter adaptive mechanism; until the training is completed, the proximal policy optimization agent with the best performance is output as the policy model.

[0050] The strategy output and application module is used to continuously adjust the wing box structure dimensions to be optimized in a finite number of steps using the strategy model, so as to obtain the lightweight size design result.

[0051] Compared with the prior art, the present invention has at least the following beneficial effects:

[0052] First, this invention establishes a boundary-sensitive size update process between the reinforcement learning agent and the finite element method (FEM). It determines the action scaling factor based on the distance of each design variable to its corresponding upper and lower boundaries, and combines this with a preset maximum physical size adjustment and the step size decay factor of the current environmental interaction step to convert normalized continuous actions into actual structural size adjustments. This method automatically reduces the size adjustment amplitude when design variables approach their boundaries, minimizing out-of-bounds actions and repeated boundary truncation. The FEM further calculates the updated structure's mass and KS failure index, and combines the scaled structural response with the current design variable parameters to form the next state, thereby achieving continuous closed-loop adjustment of the wing box structure dimensions.

[0053] Second, this invention introduces a dynamic hyperparameter adaptation mechanism based on a population-based game-theoretic optimization algorithm (Population-Based Bandits, PB2) during the high-fidelity finite element feedback training of the wing box. This maintains multiple parallel-trained PPO individuals, and uses the average round reward calculated based on structural quality changes and KS failure constraints as the performance evaluation index for each individual. After a preset perturbation period, the lower-performing individual loads the policy network parameters, value network parameters, policy network optimizer state, and value network optimizer state of the higher-performing individual, and continues training after updating the learning rate, discount factor, and pruning coefficient. This approach preserves the policy parameters and optimizer history states already obtained by the superior individual through finite element interaction, reducing the workload of finite element analysis calls and manual parameter tuning caused by repeated training from randomly initialized states under different hyperparameter configurations.

[0054] Third, this invention constructs a reinforcement learning state using structural mass, KS failure index, and current structural component thickness, and performs scaling transformations on different physical quantities to reduce the magnitude difference between the structural response and dimensional parameters. In reward calculation, a quality reward is set based on the change in structural mass relative to the initial structural mass, and a penalty constraint is applied based on the degree to which the KS failure index exceeds the failure constraint threshold. When the KS failure index exceeds the allowable threshold, an out-of-bounds penalty is applied, and the current round is terminated. Therefore, it is possible to suppress the agent from entering the strength-infeasible region during the continuous reduction of structural mass, ensuring that the average round reward formed based on this reward simultaneously reflects both the lightweighting degree of the wing box and the satisfaction of the KS strength constraint.

[0055] Fourth, this invention employs a continuous action mapping mechanism jointly controlled by boundary distance and optimization process. First, the continuous actions output by the agent are clipped to a preset normalized range. Then, a nonlinear boundary scaling factor is determined based on the minimum distance from the current design variable to its upper and lower boundaries. This is combined with a preset maximum physical size adjustment and a step size decay factor that decreases with the number of interaction steps to obtain the actual size adjustment. Finally, boundary constraints are applied to the updated structural dimensions. This mechanism automatically reduces the action amplitude when the structural dimensions approach the boundary of any design variable and gradually refines the size adjustments in the later stages of optimization, thereby reducing the probability of frequent boundary crossings, repeated boundary truncation, and large oscillations near the boundaries.

[0056] Fifth, the strategy model trained by this invention does not output a fixed wing box thickness scheme all at once. Instead, it repeatedly executes state construction, action output, boundary-sensitive dimension updates, and finite element analysis during the application phase. For the same wing box configuration, when the finite element load conditions change, the finite element solution module recalculates the current structural mass and KS failure index, and feeds back the new state consisting of the structural mass, KS failure index, and current structural component thickness to the strategy model. This allows the strategy model to continue outputting corresponding dimensional adjustment actions based on the differences in structural response caused by different load conditions. Thus, the same trained strategy model can perform finite-step closed-loop dimensional adjustments on the same type of wing box under different load levels without retraining, and obtain a lightweight dimensional scheme that is within the preset thickness boundary range and satisfies the KS strength constraint. Attached Figure Description

[0057] Figure 1 This is a flowchart of a method for lightweighting and optimizing the size of an airfoil box based on near-end strategy optimization according to the present invention.

[0058] Figure 2 This is a flowchart of the PPO-finite element feedback closed-loop optimization process of the present invention;

[0059] Figure 3 This is a flowchart of the PB2 dynamic hyperparameter adaptive training process of the present invention;

[0060] Figure 4 This is a comparison chart of the structural mass, maximum displacement, and maximum local failure index of the wing box before and after optimization of the present invention. Detailed Implementation

[0061] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0063] This embodiment provides a method for lightweight wing box size optimization based on near-end strategy optimization. The method constructs the wing box size optimization problem as a reinforcement learning sequential decision process, using the size parameters of structural components such as wing box skin, spars, ribs, panels, stiffeners or stringers as design variables, minimizing structural mass as the optimization objective, and strength constraints, displacement constraints and manufacturing constraints as design constraints.

[0064] like Figure 1 As shown, this embodiment provides a method for lightweighting and optimizing the size of an airfoil box based on near-end strategy optimization, specifically including the following:

[0065] Step 1: Using the dimensional parameters of the load-bearing components of the wing box structure as design variables, establish a set of design variables that includes each design variable and its upper and lower bounds.

[0066] This embodiment uses the dimensional parameters of the main load-bearing components of the aircraft wing box structure as design variables. The wing box structure includes load-bearing components such as upper skin, lower skin, front wing spars, rear wing spars, ribs, panels, stiffeners, or stringers. The wing box structure can be discretized using shell elements, plate-shell elements, beam elements, or combinations thereof using finite element methods. In this embodiment, skin thickness, spars thickness, and rib thickness are used as dimensional design variables. In other embodiments, the wing box can be divided into multiple design regions along the spanwise, chordwise, or partitioned manner, and the skin thickness, spars thickness, rib thickness, stiffener dimensions, or stringer dimensions of different regions can be used as higher-dimensional design variables. This embodiment does not impose any limitations on this approach.

[0067] Furthermore, based on the actual size constraints of the wing box, a set of design variables is established, including each design variable and its upper and lower bounds. In this embodiment, the design variable vector is taken as:

[0068]

[0069] in, This represents the vector of design variables for the wing box structure dimensions. To design variable dimensions, For the first One structural dimension design variable, superscript This indicates the matrix transpose.

[0070] Design variables satisfy upper and lower bound constraints:

[0071]

[0072] in, and These are the lower and upper bound vectors of the design variables, respectively. and The dimensions are the same as the design variable vector, so as to set upper and lower bounds for each design variable. The lower bound can be determined according to the manufacturing process, stability requirements, or minimum plate thickness requirements, and the upper bound can be determined according to the structural load-bearing capacity, weight restrictions, or engineering design scope.

[0073] In this embodiment, a mathematical model for lightweight dimensional optimization of the wing box is further established. The optimization objective is to reduce the structural mass of the wing box while satisfying structural strength, displacement, and manufacturing constraints. The optimization problem can be expressed as:

[0074]

[0075]

[0076] in, For the wing box structure quality. These are failure constraint indices obtained through finite element analysis and KS function aggregation. This is the failure constraint threshold. This represents the maximum displacement response of the structure, specifically the maximum absolute value of the displacement components corresponding to each node or each degree of freedom. This is the displacement constraint threshold. For the first The displacement response corresponding to each node or degree of freedom. This represents the total number of displacement response components. This indicates the absolute value operation.

[0077] In this embodiment, a finite element model of the wing box structure is obtained, and the thickness or dimensional parameters of the corresponding structural components are updated according to the current design variables. The finite element analysis model calculates the wing box structure response under given loads, boundary conditions, and material parameters, and the output includes at least one or more of the following: structural mass, stress response, displacement response, and failure constraint indicators.

[0078] The finite element response set can be represented as:

[0079]

[0080] in, For the finite element response set, This represents the finite element analysis process. For stress response set, This is the set of displacement responses.

[0081] Since the wing box finite element model contains a large number of local stresses or failure constraints, directly using all local constraints as reinforcement learning feedback would increase the complexity of the state dimension and reward function. Therefore, this embodiment does not simply treat engineering constraints as ordinary state components or reward items, but uses the KS function to aggregate multiple local failure constraints in the wing box finite element model into a single continuous failure index, and uses the KS failure index simultaneously for state construction, constraint penalties, boundary violation penalties, and round termination judgment. By setting a constraint threshold below the allowable threshold, the agent is continuously penalized before entering the scalar infeasibility region, thereby reducing the consumption of finite element evaluation resources on obviously infeasible size solutions.

[0082] Let the first The local failure constraints are: The KS failure index can then be expressed as:

[0083]

[0084]

[0085] in, To design variable vectors The corresponding KS failure index, Design variable vector The corresponding maximum local failure constraint, For the number of local constraints, For KS aggregation parameters, The KS function can approximate the maximum local constraint value while maintaining continuity, making it suitable as part of reinforcement learning states, rewards, and evaluations of trained individuals.

[0086] In one implementation, local constraints can be constructed using the ratio of element stress to allowable material stress:

[0087]

[0088] in, For the first Equivalent stress of a unit or component Let be the allowable stress of the material. When At that time, it was assumed that the structure met the strength constraint requirements.

[0089] The finite element analysis model in this embodiment can use the TACS (Toolkit for the Analysis of Composite Structures) finite element solver, or other finite element solvers or equivalent structural analysis programs that can output structural mass, stress response, displacement response and constraint aggregation index. This invention does not limit this.

[0090] Step 2: Obtain the current state vector and input it into multiple parallel-trained proximal policy optimization agents to obtain normalized continuous actions.

[0091] This embodiment constructs a PPO-finite element feedback closed-loop reinforcement learning environment: the current wing box structure state is input into the proximal policy optimization agent, the state vector contains finite element analysis results and design variables, and the proximal policy optimization agent outputs normalized continuous actions corresponding to the size variables of each structural component according to the current wing box structure state. Each dimension of the normalized continuous action corresponds to the adjustment direction and adjustment range of the size of each structural component of the wing box.

[0092] like Figure 2 As shown, this embodiment constructs the wing box size optimization problem as a Markov decision process. The reinforcement learning agent receives the current structural state at each environmental interaction step and outputs continuous actions. The environment updates the wing box structural dimensions based on the actions, calls the finite element analysis model to calculate the structural response, constructs a reward and the next state based on the structural response, and feeds the next state and reward back to the agent, forming a closed-loop interactive process of state-action-size update-finite element analysis-reward feedback.

[0093] The state space is defined in this embodiment as follows:

[0094] The state vector is composed of finite element feedback information and current structural dimension information. Typically, the state vector includes the wing box structural mass, the KS failure index, and the current structural component dimensional parameters. Scaling transformations are applied to the structural mass, KS failure index, and current structural component dimensional parameters to bring the state components of different dimensions and orders of magnitude into numerical ranges suitable for the strategy network's processing. The scaled structural mass, KS failure index, and current structural component dimensional parameters are then combined to form the state vector.

[0095] In other implementations, the state vector may also include one or more of the following: local stress maximum value, load factor, constraint margin, historical actions, historical rewards, current environmental interaction steps, finite element solution convergence flag, or multi-load response information.

[0096] In this embodiment, the state vector is defined as:

[0097]

[0098] in, For the first Environment interaction step state vector, The KS failure index corresponding to the current design variable. The structural mass corresponding to the current design variable. This represents the maximum displacement response corresponding to the current design variable. For the current number The design variable vector for the environmental interaction step. For loading factor, , , and These are the scaling transformation coefficients, with superscripts. This is a matrix transpose operation. For the current environment interaction step index within a single training round, and ; This represents the total number of environment interaction steps in a single training round, and is only used during the policy training phase.

[0099] This embodiment defines the continuous action space as follows:

[0100] In this embodiment, the PPO agent outputs normalized continuous actions:

[0101]

[0102] in, Indicates the first Environmental interaction step motion vectors, Indicates the first The first step in the environmental interaction action vector Each structural dimension design variable corresponds to an action, and each dimension of the action vector corresponds to the adjustment direction and magnitude of that structural dimension design variable. Continuous actions are normalized vectors in the action space. After action clipping, maximum step size scaling, boundary distance scaling, and step size decay, continuous actions are mapped to actual size adjustments. Actions represent continuous adjustments to skin thickness, sparsity thickness, and rib thickness.

[0103] Step 3: Combine the set of design variables, determine the actual size adjustment of each design variable according to the normalized continuous action, and update the wing box structure size based on the actual size adjustment.

[0104] This embodiment calculates the distances from the current values ​​of each dimensional variable to the lower and upper boundaries of the design variables, and determines the motion scaling factor based on the smaller of the two distances. This ensures that the closer a dimensional variable is to the boundary of any design variable, the smaller its actual adjustment range. Furthermore, it combines normalized continuous motion with a preset maximum physical dimension adjustment and a step-size decay factor that varies with the current environmental interaction step to obtain the actual thickness adjustment of the skin, spars, and ribs. Boundary constraints are then applied to the updated thickness variables, thereby reducing out-of-bounds actions, repeated boundary truncation, and infeasible dimension sampling.

[0105]

[0106] in, For the first The step size decay coefficient corresponding to each environmental interaction step. The initial step size decay coefficient, This represents the attenuation rate. With... As the step size increases, the decay coefficient gradually decreases, resulting in a gradual reduction in the adjustment range of structural dimensions in the later stages of optimization. In this embodiment, , The parameters can be adjusted according to the range of design variables and the required optimization accuracy.

[0107] It should be noted that the exponential decay function is an existing parameter decay method. In other embodiments, the step size decay coefficient is not limited to using only the exponential decay function, provided that the step size decay coefficient gradually decreases as the environmental interaction steps increase. Furthermore, another key point of this invention is that the step size decay coefficient and the action scaling coefficient determined based on the distance from the design variable to the upper and lower boundaries are applied together to the normalized continuous action, so that the actual size adjustment is simultaneously adjusted by the current optimization process and the boundary position of the design variable.

[0108] Unlike action mapping methods that directly convert normalized actions into geometric parameters or physical dimensions according to a fixed ratio, the actual size adjustment amount in this invention is simultaneously adjusted by the current design variable boundary distance and the current environmental interaction steps. The boundary scaling factor is used to characterize the position of the current structural dimension within the allowable design range, and the step size decay factor is used to characterize the current optimization process, so that the same normalized action corresponds to different actual size adjustments at different boundary positions and different environmental interaction steps.

[0109] Specifically, this embodiment establishes a boundary-sensitive action mapping mechanism: to prevent the agent's output actions from causing design variables to frequently exceed engineering boundaries, normalized continuous actions are mapped to actual physical size adjustment amounts.

[0110]

[0111] Adjust according to actual size Update the wing box structure dimensions and apply boundary constraints to the updated wing box structure dimensions so that the updated design variables are between the corresponding lower and upper boundaries of the design variables.

[0112]

[0113] in, For the first The actual size adjustment amount for the environmental interaction step is in vector form, consistent with the dimensions of the design variables. Adjust the step size to achieve the maximum dimension. This represents element-wise multiplication. For the first The environmental interaction step boundary scaling factor is a vector of action scaling coefficients that are consistent with the dimensions of the design variables, with each component corresponding to a design variable. For the first The environmental interaction step size decay coefficient is a scalar that applies the same decay ratio to all design variables. This represents the clipping function. For the first Design variables for environmental interaction steps For the first Design variables for environmental interaction steps.

[0114] In one implementation, the first Boundary scaling factor for each design variable It can be represented as:

[0115]

[0116] in, For the first Environmental interaction step one The current value of each design variable, and Variables The lower and upper bounds. When a design variable approaches its boundary, The size is reduced, thereby decreasing the dimensional adjustment range in the corresponding direction and lowering the probability of invalid trimming actions and infeasible dimension sampling. The boundary scaling factor is determined based on the distance from the current dimension to the upper and lower bounds of the design variable, and the boundary scaling factor increases as the minimum distance increases and decreases as the minimum distance decreases. It is used to reduce the corresponding dimensional adjustment range when the design variable is close to the upper or lower bound.

[0117] It should be noted that the specific expressions for the boundary scaling factor and step size decay coefficient described above are only one preferred embodiment. Those skilled in the art can make equivalent substitutions or reasonable adjustments to the mapping function according to the actual wing box structural dimensions, design variable range, and training stability requirements.

[0118] Step 4: Perform finite element analysis on the updated wing box structure dimensions, output finite element analysis results including structural mass and KS failure index, generate the next state vector, and calculate the reward based on the finite element analysis results and update the network parameters of the near-end policy optimization agent.

[0119] This embodiment calls the finite element analysis model to update the design variable vector. Structural analysis was performed on the corresponding wing box structure to obtain the first Structural quality of environmental interaction steps and KS failure index ; respectively on structural quality KS Failure Index and the updated design variable vector A scaling transformation is performed, and the scaled structural mass, KS failure index, and design variables constitute the state vector for the next environmental interaction step. Rewards are calculated based on structural quality and KS failure index, forming a closed-loop interaction between intelligent agent, boundary-sensitive dimension update, finite element analysis, and state reward feedback.

[0120] The reward function is defined in this embodiment as follows:

[0121] The reward function comprehensively considers both the reduction in structural mass and the satisfaction of constraints. It includes a mass reward term, a strength constraint penalty term, and a displacement constraint penalty term, expressed as follows:

[0122]

[0123] The quality reward term characterizes the degree of structural quality reduction. It includes normalizing the initial structural quality and calculating the relative reduction in structural quality before and after the wing box structural size update in the current iteration. The strength constraint penalty term includes applying a positive truncation operator to the difference between the KS failure index obtained after the wing box structural size update and the failure constraint threshold to characterize the degree to which the KS failure index exceeds the failure constraint threshold. The displacement constraint penalty term includes applying a positive truncation operator to the difference between the maximum displacement response obtained after the wing box structural size update and the displacement constraint threshold to characterize the degree to which the maximum displacement response exceeds the displacement constraint threshold.

[0124] in, For the first Rewards for each step of environmental interaction For the initial wing box mass, and The structural mass before and after the size update are shown respectively. The updated KS failure index, This is the updated maximum displacement response. and These are the failure constraint threshold and the displacement constraint threshold, respectively. , and These are the weighting coefficients for quality reward, strength constraint penalty, and displacement constraint penalty, respectively. The quality reward is positive when the structural quantity decreases; the penalty increases with the degree of constraint violation when the KS failure index or maximum displacement response exceeds the corresponding constraint threshold. The current training round can be terminated when the structural response exceeds the allowable range.

[0125] This embodiment uses a proximal policy optimization algorithm to train the agent, and maintains multiple parallel training individuals during the training process. Each training individual has corresponding policy network parameters, value network parameters, optimizer state, and a learning rate. Discount Factor and cutting factor The PPO hyperparameters, including those for the agent, are as follows:

[0126] (1) Construct an actor-critic network.

[0127] In this embodiment, the Actor-Critic network to be trained is used to form the PPO agent, and the PPO agent adopts the Actor-Critic architecture. The Actor network is based on the current state vector. Output the action distribution and sample continuous action vectors from it. The Critic network estimates the state value based on the current state, which is used to calculate the advantage function and guide policy updates.

[0128] The policy network can be represented as:

[0129]

[0130] A value network can be represented as:

[0131]

[0132] in, For policy network parameters, These are the parameters of the value network.

[0133] (2) Construct the objective function of PPO pruning agent.

[0134] PPO improves training stability by limiting the update magnitude between the new and old policies. The probability ratio between the new and old policies is defined as:

[0135]

[0136] PPO trimming proxy objective function for:

[0137]

[0138] in, For the first The probability ratio of the new and old strategies in the environmental interaction steps. The network parameters were set before the update. This is the cutting factor. For the first Environmental interaction step advantage function estimate, It expresses expectation.

[0139] The advantage function is calculated based on the reward sequence formed by finite element feedback, the discounted return, and the value network output. The advantage function can be estimated from the discounted return and the value function:

[0140]

[0141]

[0142] in, As a discount factor, In exchange for a discount, For the first Rewards for interactive steps in the environment.

[0143] (3) Interact with the finite element environment and update network parameters.

[0144] In each training round, the PPO agent adjusts its actions based on the current state output size. The environment updates the design variables based on the actions and calls the finite element analysis model to obtain the new structural response, reward, and next state. The agent collects the sequence of states, actions, rewards, and next states, updates the policy network using the PPO pruning objective function, and updates the value network using the value loss function.

[0145] Value network loss function Represented as:

[0146]

[0147] Through the above process, the agent gradually learns strategies from finite element feedback states to continuous size adjustment actions.

[0148] Step 5: According to the perturbation period, introduce the PB2 dynamic hyperparameter adaptive mechanism to maintain multiple parallel-trained proximal policy optimization agents; until the training is completed, output the proximal policy optimization agent with the best performance as the policy model.

[0149] like Figure 3 As shown, this embodiment introduces the PB2 dynamic hyperparameter adaptive mechanism during PPO training to adjust key PPO hyperparameters online, reducing manual parameter tuning costs and improving training stability. Training performance is evaluated based on the average round reward of each training individual according to a preset perturbation period. The average round reward is calculated based on the structural quality and KS failure index fed back from the finite element analysis model. Lower-performing training individuals inherit the policy network parameters, value network parameters, policy network optimizer state, and value network optimizer state of higher-performing training individuals. The PB2 mechanism updates the learning rate, discount factor, and pruning coefficient of the lower-performing training individuals, allowing them to continue training based on the inherited network parameters and optimizer state.

[0150] During training, each PPO training individual interacts with its corresponding finite element feedback environment. The finite element analysis model calculates the structural mass and KS failure index for the wing box structure after each size adjustment, calculates the reward based on the change in structural mass and KS failure constraints, and determines the average round reward of the training individual based on the rewards of each interaction step. The PB2 dynamic hyperparameter adaptive mechanism sorts the training individuals according to a preset perturbation period, using the average round reward as the performance evaluation index. Training individuals with lower performance load the policy network parameters, value network parameters, policy network optimizer state, and value network optimizer state of higher-performance training individuals, and after the PB2 mechanism updates the learning rate, discount factor, and pruning coefficient, continue PPO training from the inherited training state.

[0151] This invention does not make any algorithmic changes to the PB2 mechanism itself. The PB2 mechanism is configured in the PPO-finite element closed loop for wing box size optimization. The performance evaluation of the trained individuals comes from the finite element feedback reward formed by structural quality and KS failure index, and together with the boundary sensitive dimension motion mapping and finite element analysis process, it constitutes a complete training process.

[0152] In one implementation, the PB2 dynamic hyperparameter adaptive mechanism establishes a performance prediction model based on historical hyperparameter configurations and corresponding training performance, and selects a new hyperparameter configuration from the hyperparameter search space according to the acquisition function. The hyperparameter search space includes the learning rate range, the discount factor range, and the pruning coefficient range. This mechanism reduces the number of finite element evaluations required for manual hyperparameter tuning and repeated full training.

[0153] The PB2 program calls the PPO agent and feedforward neural network policy. Both the policy network and value network employ a two-layer fully connected hidden layer structure, with each layer containing 64 neurons and using the ReLU activation function. The policy network inputs a 5-dimensional wingbox structure state and outputs a 3-dimensional continuous action; the output layer uses the Tanh activation function. The value network inputs a 5-dimensional state and outputs a 1-dimensional state value; the output layer uses a linear function. The policy network and value network use the Adam optimizer for parameter updates. The continuous action output by the policy network is sampled using a diagonal Gaussian distribution with a standard deviation of 0.2. Each training iteration of PPO collects 100 environmental interaction steps, performs a maximum of 10 size adjustment steps per round, and updates network parameters 10 times. The PB2 dynamic hyperparameter adaptive mechanism sets up 5 parallel training individuals, each training individual trains for 120 iterations, and performs hyperparameter adjustment every 15 iterations. The search ranges for the learning rate, discount factor, and PPO pruning coefficient are

[10] . -5 5×10 -3 [0.9, 0.999] and [0.10, 0.30], the above ranges are only preferred embodiments and do not constitute a limitation on the scope of protection of the present invention.

[0154] Step 5.1: Initialize the training population.

[0155] Let the set of individuals (agents) trained in parallel be:

[0156]

[0157] in, To determine the number of training individuals. individual trainees Includes model parameter set and hyperparameter set :

[0158]

[0159]

[0160] in, For the first The policy network parameters of each training individual, For the first The value network parameters of each trained individual and The first The policy network optimizer state and the value network optimizer state of each training individual. For the first The learning rate corresponding to each training individual For the first The discount factor corresponding to each training individual For the first The PPO clipping coefficient corresponding to each training individual.

[0161] Step 5.2: Evaluation of individual training performance.

[0162] Each training individual is trained for one perturbation cycle in an independent finite element feedback environment. For the first... For each training individual, the cumulative reward for each round within the disturbance period is calculated, and the average round reward is used as the performance index for that individual. The reward for each step consists of the reward for changes in wing box structural mass, the penalty for exceeding the failure constraint threshold in the KS failure index, and the penalty for exceeding the displacement constraint threshold in the maximum displacement response. Therefore, the average round reward comprehensively reflects the training individual's performance in reducing wing box structural mass and maintaining KS strength constraints.

[0163] The performance index is the average round reward of the training individual in the current disturbance period. The average round reward is obtained by accumulating or statistically analyzing the rewards of each environmental interaction step. The rewards of each environmental interaction step are determined based on the changes in the wing box structure mass and the KS failure index, thus establishing a correlation between the PB2 hyperparameter update and the wing box lightweighting target and strength constraint evaluation.

[0164] In one implementation, the first The performance metrics of a training individual can be expressed as:

[0165]

[0166] in, For the first Performance metrics for each training individual For the first Average round reward for each training individual The number of rounds within the perturbation period. For the first time in the disturbance period Cumulative rewards for each round.

[0167] This performance metric constrains the PB2 dynamic parameter tuning process to the feasibility of the finite element structure, avoiding the problem of structural infeasibility caused by updating hyperparameters solely based on ordinary training rewards.

[0168] Step 5.3: Train individual sorting and checkpoint inheritance.

[0169] After each disturbance cycle, based on performance indicators All training individuals are sorted, and a set of high-performance individuals and a set of low-performance individuals are selected. For low-performance individuals, they inherit the policy network parameters, value network parameters, and optimizer states of high-performance individuals.

[0170] The inheritance process can be represented as:

[0171]

[0172] in, This represents the set of model parameters for low-performing training individuals. This represents the set of model parameters for high-performing training individuals. Through inheritance, low-performing individuals do not need to be retrained from random initialization, but can continue training from a better policy state, thus reusing the training experience gained from finite element samples.

[0173] Checkpoints must at least save the policy network parameters, value network parameters, policy network optimizer state, value network optimizer state, current hyperparameter configuration, and historical training records. When a lower-performing training individual loads a checkpoint from a selected (e.g., uniformly random sampled) high-performing training individual, it does not reinitialize the policy and value networks. Instead, it continues training based on the inherited network parameters and optimizer states, allowing the inherited training individual to continue utilizing the consumed finite element evaluation samples without reinitialization.

[0174] Step 5.4, PB2 hyperparameter update.

[0175] After completing the checkpoint inheritance, the PB2 mechanism establishes a performance prediction model based on historical hyperparameter configurations and corresponding training performance, and generates new hyperparameter configurations for low-performance individuals within a preset hyperparameter search space.

[0176] Historical observation data set Represented as:

[0177]

[0178] in, For the first Subhistorical hyperparameter set To correspond to the training time, To correspond to the training performance metrics, Configure the total number of iterations for historical hyperparameters. PB2 dynamically updates PPO hyperparameters, including the learning rate. Discount Factor and cutting factor And update online within the preset range respectively.

[0179] In one implementation, PB2 uses a Gaussian process surrogate model to describe the relationship between hyperparameters and training performance:

[0180]

[0181] in, Let be the objective function, representing the set of historical hyperparameters input. Training time The output shows the prediction performance of the model under this configuration and training phase. For Gaussian process surrogate model, It is a mean function. Let be the covariance function.

[0182] The predicted mean and standard deviation of candidate hyperparameters are obtained based on the surrogate model, and a new hyperparameter configuration is selected through a confidence upper bound acquisition function, so that network parameter inheritance and hyperparameter update are completed in the same training process:

[0183]

[0184] in, The selected optimal candidate hyperparameter, For the hyperparameter search space, To explore weights, This represents the predicted standard deviation.

[0185] Thus, after inheriting the model parameters and optimizer state of the high-performance individual, the low-performance individual continues to train using the new hyperparameter configuration, achieving collaborative optimization of the policy network weights and training hyperparameters.

[0186] Step 5.5: Repeat the training until the termination condition is met.

[0187] Within each training cycle, sequential action mapping, finite element analysis, KS constraint aggregation, reward calculation, PPO network parameter update, and PB2 hyperparameter update are executed sequentially, ensuring that size adjustment, constraint evaluation, and hyperparameter tuning are completed within the same closed-loop training process. That is, steps 5.2 to 5.4 are repeated until one or more termination conditions are met, including the maximum number of training epochs, the maximum number of finite element evaluations, the average epoch reward convergence condition, the structural quality convergence condition, or the feasible solution ratio. After training, the training individual with the best performance index is selected from the training population, and its policy model is output.

[0188]

[0189] in, This is the optimal policy model after training.

[0190] Step 6: Use the strategy model to continuously adjust the wing box structure dimensions of the aircraft to be optimized in a finite number of steps to obtain the lightweight size design results.

[0191] In the strategy application phase, the initial design of the wing box structure to be optimized is input into the finite element analysis model to obtain the initial structural response and construct the initial state. The trained strategy model outputs continuous actions based on the current state. The dimension update module maps these actions to actual dimension adjustments, and the finite element analysis model analyzes the updated structure and feeds back the new state. This process is repeated for a finite number of steps until the maximum number of closed-loop dimension adjustment steps is reached or the constraint conditions are met. After a finite-step closed-loop iteration, a lightweight wing box dimension design scheme that satisfies the design constraints is output. The strategy model can be used for dimension optimization of wing box structures with different initial thicknesses, different load factors, different constraint thresholds, or of the same type.

[0192] The strategy application process can be represented as:

[0193]

[0194]

[0195]

[0196]

[0197] in, This is the mapping function from action to physical size. This is a state constructor.

[0198] The final output is a lightweight wing box size design that meets the constraints:

[0199]

[0200] in, For the initial design variable vector, This refers to the maximum number of closed-loop size adjustment steps allowed for a single wing box size optimization task during the strategy application phase. and These are used in the training and application phases respectively, and their meanings are independent of each other. For the final optimized design, This represents the closed-loop optimization mapping in the strategy application phase, which involves using the trained strategy model to output size adjustment actions based on the current state, and sequentially executing size updates, finite element analysis, and state feedback, repeating the iteration until the maximum number of closed-loop size adjustment steps is reached or the preset termination condition is met.

[0201] Strategy reuse under multiple load conditions: In another embodiment, the trained strategy model can be used for dimensional optimization of the same wing box structure under different load conditions. In multi-load applications, load factors, maximum displacement, KS failure index, or other finite element response information are input as state components into the strategy model, enabling the strategy to adaptively adjust dimensional parameters according to the structural response under different load levels.

[0202] The trained policy model is used for wing box size optimization under different initial thicknesses, load factors, or constraint thresholds. In multi-load applications, the aforementioned state vector construction method is still used, where the load factor... It serves both as the load input for finite element analysis and as a component of the state vector. For different load factors, the finite element analysis model recalculates the KS failure index and maximum displacement response based on the current design variables, and the updated structural response, structural mass, current design variables, and load factors together constitute a new state vector.

[0203] Therefore, the multi-load state can be represented as:

[0204]

[0205] in, For the first The state vector of the environmental interaction step. For the first The design variable vector for the environmental interaction step. The KS failure index is calculated using a finite element model under the load factor. This represents the maximum displacement response under the corresponding load conditions. For the current wing box structure quality, , , and These are the scaling factors for the KS failure index, structural mass, maximum displacement response, and design variables, respectively, with superscripts indicating their approximate values. This represents the matrix transpose operation. The structural mass is determined solely by the current structural dimensions and material properties, and does not directly depend on the load factor. The load factor is not used as an independent variable. For different load conditions, the finite element analysis model recalculates the KS failure index and maximum displacement response based on the corresponding load factor, and feeds the updated structural response into a new state to the strategy model. The strategy model continuously outputs dimensional adjustment actions based on the new state, thereby obtaining a lightweight design that meets the strength and displacement constraints under the corresponding load conditions.

[0206] It should be noted that strategy reuse under multiple load conditions does not require the strategy model to output a fixed thickness distribution. Instead, it dynamically generates continuous dimensional adjustments based on the finite element feedback state. Therefore, this strategy model can be applied to dimensional optimization tasks with different initial thicknesses, different load factors, different constraint thresholds, or similar wing box structures.

[0207] Example verification and application effect description.

[0208] To illustrate the feasibility of the method in this embodiment, a lightweight dimensional optimization was performed using a finite element model of a half-wing reference wing box of a transport aircraft. This wing box model is a typical beam-ribbed wing box structure, mainly including upper and lower skins, a front spar, a rear spar, and load-bearing components such as ribs, and is discretized using plate and shell finite element elements. In the finite element analysis, the structural mass, stress response, displacement response, and KS failure index were calculated under given load and boundary conditions. The design variables selected were skin thickness, spar thickness, and rib thickness. In the initial design, the skin thickness was 8.00 mm, the spar thickness was 4.00 mm, the rib thickness was 3.00 mm, and the initial wing box structure mass was 1227.22 kg.

[0209] Under nominal load conditions, the adaptive near-end optimization strategy model trained in this embodiment was used to perform finite-step dimensional adjustments on the wing box structure. After optimization, the skin thickness, sparsity thickness, and rib thickness were 6.75 mm, 3.09 mm, and 2.01 mm, respectively, and the wing box structure mass was reduced to 1007.32 kg, a reduction of approximately 17.92% compared to the initial design mass. The optimized KS failure index remained below the allowable threshold, indicating that the obtained lightweight design met the strength constraints.

[0210] like Figure 4 As shown, in terms of structural response, the maximum displacement of the wing box increased compared to the initial design, from 1.10m to 1.32m. However, the displacement field distribution remained continuous and smooth, with no abnormal local deformation. Meanwhile, the maximum local failure index increased from 0.80 to 0.96, but remained within the allowable range, indicating that the optimized structure made fuller use of the load-bearing capacity without compromising strength safety constraints.

[0211] Furthermore, the trained strategy model was applied to the same type of wing box structure under different load conditions. Under a 1.5g load, the optimized wing box mass was reduced to 781.97kg, a reduction of approximately 36.28%, with a KS failure index of 0.7773, satisfying the strength constraint. Under a 2.0g load, the optimized wing box mass was reduced to 901.51kg, a reduction of approximately 26.54%, with a KS failure index of 0.8853, also satisfying the strength constraint. These results demonstrate that the strategy model does not output a fixed-size solution, but rather can adaptively adjust the structural dimensions based on finite element feedback and load conditions.

[0212] The specific models, load conditions, dimensional parameters, and optimization results described above are only used to illustrate the technical effects of this embodiment and do not constitute a limitation on the scope of protection of this invention.

[0213] Therefore, the method in this embodiment can achieve automated lightweight optimization of the wing box structure dimensions under finite element feedback, and reduce the structural mass while meeting strength constraints. Simultaneously, the PB2 dynamic hyperparameter adaptive mechanism improves the training stability of the near-end optimization strategy model, reduces the need for repeated manual parameter tuning, and enables the trained strategy model to have a certain degree of engineering reusability.

[0214] For those skilled in the art, without departing from the concept of the present invention, equivalent substitutions or reasonable adjustments can be made to the design variable type, finite element solver type, state construction method, action mapping method, reward function form, PB2 performance evaluation index, hyperparameter search range, and load conditions in the above embodiments, and all such substitutions and adjustments should fall within the protection scope of the present invention.

[0215] In another embodiment, the present invention also provides a lightweight wing box size optimization system based on near-end policy optimization, including a task initialization module, a reinforcement learning environment interaction module, a size update module, a finite element solution and state update module (specifically divided into a finite element solution module and a state construction module), a reward calculation module, a PPO policy training module, and a policy output and application module.

[0216] The task initialization module is used to determine the wing box structure design variables, upper and lower bounds of the design variables, material parameters, load conditions, finite element model, and optimization termination conditions.

[0217] The reinforcement learning environment module is used to build a closed-loop interactive environment between the agent and the finite element solution module, and to manage state, action and reward signals.

[0218] The size update module is used to map the continuous actions output by the PPO agent to the wing box structure size adjustment amount, and update the structure size according to the design variable boundary constraints.

[0219] The finite element solution module is used to perform finite element analysis on the updated wing box structure and output the structural mass, stress response, displacement response and KS failure index.

[0220] The state construction module is used to construct reinforcement learning states based on finite element feedback results, current size parameters, and load information.

[0221] The reward calculation module is used to calculate the reward signal based on the degree of structural quality reduction, KS failure index, displacement response, and constraint margin.

[0222] The PPO policy training module is used to train policy networks and value networks based on state, action, reward, and finite element feedback. It introduces the PB2 dynamic hyperparameter adaptive mechanism to maintain multiple parallel training individuals, performing checkpoint inheritance and hyperparameter updates based on training performance, and dynamically adjusting the learning rate, discount factor, and pruning coefficient.

[0223] The strategy output and application module is used to output the trained strategy model and apply the strategy model to the lightweight size optimization of the target wing box structure.

[0224] For specific limitations regarding the wing box lightweight size optimization system based on near-end strategy optimization, please refer to the description of the wing box lightweight size optimization method based on near-end strategy optimization, which will not be repeated in this embodiment.

[0225] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this invention.

[0226] The above embodiments merely illustrate several implementation methods of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the appended claims.

Claims

1. A method for lightweighting and optimizing the size of an airfoil box based on near-end strategy optimization, characterized in that, include: Using the dimensional parameters of the load-bearing components of the wing box structure as design variables, a set of design variables including each design variable and its upper and lower bounds is established. Obtain the current state vector and input it into multiple parallel-trained proximal policy optimization agents to obtain normalized continuous actions; The state vector contains the finite element analysis results and design variables; Each dimension of the normalized continuous motion corresponds to the adjustment direction and adjustment magnitude of each design variable; Based on the set of design variables, the actual size adjustment of each design variable is determined according to the normalized continuous action, and the wing box structure size is updated based on the actual size adjustment. Finite element analysis is performed on the updated wing box structure dimensions, and the finite element analysis results including structural mass and KS failure index are output. The next state vector is generated, and the reward is calculated and the network parameters of the near-end policy optimization agent are updated based on the finite element analysis results. Based on the perturbation period, a PB2 dynamic hyperparameter adaptive mechanism is introduced to maintain multiple parallel-trained proximal policy optimization agents; The best-performing proximal policy optimization agent is output as the policy model after training is completed. The strategy model is used to continuously adjust the wing box structure dimensions of the aircraft to be optimized in a finite number of steps to obtain lightweight size design results.

2. The wing box lightweight size optimization method based on near-end strategy optimization according to claim 1, characterized in that, The state vector includes finite element analysis results and design variables, as detailed below: The state vector contains only the structural mass, KS failure index, and design variables after scaling using the scaling coefficient; Alternatively, the state vector may include structural mass after scaling using the scaling coefficient, KS failure index, and design variables, and may also include at least one of the following: maximum displacement response, maximum local stress, load factor, normalized continuous action of the previous environmental interaction step, reward of the previous environmental interaction step, current environmental interaction step number, and finite element analysis convergence flag.

3. The wing box lightweight size optimization method based on near-end strategy optimization according to claim 1, characterized in that, The determination of the actual dimensional adjustment amount of each design variable based on the normalized continuous action, by combining the set of design variables, includes: Based on the minimum distance between the current design variable and its upper or lower bound, determine the action scaling factor for each design variable to form an action scaling factor vector; the action scaling factor increases as the minimum distance increases and decreases as the minimum distance decreases. The step size decay coefficient is determined based on the current number of interaction steps in the environment, and the step size decay coefficient decreases as the number of interaction steps in the environment increases. The normalized continuous motion, motion scaling factor vector, and step size decay factor are multiplied element-wise. The result of the element-wise multiplication is then applied to the maximum physical size adjustment to obtain the actual size adjustment of each design variable.

4. The wing box lightweight size optimization method based on near-end strategy optimization according to claim 1, characterized in that, The updating of the wing box structure dimensions based on actual size adjustments includes: The actual size adjustments of each design variable are superimposed on the current wing box structure size, and boundary constraints are applied based on the upper and lower bounds of the design variables to obtain the updated wing box structure size, which is the updated design variable.

5. The wing box lightweight size optimization method based on near-end strategy optimization according to claim 1, characterized in that, The calculation of rewards based on finite element analysis results includes calculating quality reward items, strength constraint penalty items, and displacement constraint penalty items; The quality reward item is used to characterize the degree of structural quality reduction, including normalizing the initial structural quality and calculating the relative reduction in structural quality before and after the update of the wing box structural dimensions in the current iteration; The strength constraint penalty term is used to characterize the degree to which the KS failure index exceeds the failure constraint threshold; The displacement constraint penalty term is used to characterize the extent to which the maximum displacement response exceeds the displacement constraint threshold.

6. The wing box lightweight size optimization method based on near-end strategy optimization according to claim 1, characterized in that, The KS failure index is obtained by aggregating all local stress constraints or failure constraints using the KS function.

7. The wing box lightweight size optimization method based on near-end strategy optimization according to claim 1, characterized in that, The PB2 dynamic hyperparameter adaptive mechanism includes: Calculate the performance metrics of each proximal policy optimization agent during the perturbation period; Based on the performance metrics, all proximal policy optimization agents are ranked and divided into a high-performance agent set and a low-performance agent set. For low-performance individuals, the policy network parameters, value network parameters, policy network optimizer state, and value network optimizer state are inherited from high-performance individuals. Update the hyperparameters of low-performing individuals within a preset search range, and train the low-performing individuals based on the inherited network parameters and optimizer state, as well as the updated hyperparameters.

8. The wing box lightweight size optimization method based on near-end strategy optimization according to claim 7, characterized in that, The performance metric is the average round reward of the near-end policy optimization agent during the perturbation period.

9. The wing box lightweight size optimization method based on near-end strategy optimization according to claim 7, characterized in that, The hyperparameters include the learning rate, discount factor, and pruning factor.

10. A lightweight wing box size optimization system based on near-end strategy optimization, characterized in that, include: The task initialization module is used to establish a set of design variables, including each design variable and its upper and lower bounds, using the dimensional parameters of the load-bearing components of the wing box structure as design variables. The reinforcement learning environment interaction module is used to obtain the current state vector, which is then input into multiple parallel-trained proximal policy optimization agents to obtain normalized continuous actions. The state vector contains the finite element analysis results and design variables; Each dimension of the normalized continuous motion corresponds to the adjustment direction and adjustment magnitude of each design variable; The size update module is used to combine the set of design variables, determine the actual size adjustment amount of each design variable according to the normalized continuous action, and update the wing box structure size based on the actual size adjustment amount; The finite element solution and state update module is used to perform finite element analysis on the updated wing box structure dimensions, output finite element analysis results including structural mass and KS failure index, and generate the next state vector; The reward calculation module is used to calculate the reward based on the finite element analysis results and update the network parameters of the near-end policy optimization agent. The PPO policy training module is used to maintain multiple parallel-trained proximal policy optimization agents by introducing the PB2 dynamic hyperparameter adaptive mechanism according to the perturbation period. The best-performing proximal policy optimization agent is output as the policy model after training is completed. The strategy output and application module is used to continuously adjust the wing box structure dimensions to be optimized in a finite number of steps using the strategy model, so as to obtain the lightweight size design result.

Images