Method and system for wing flutter active suppression based on imitative learning elastic approximate dynamic programming

Abstract

The application discloses an elastic approximate dynamic programming wing flutter active suppression method and system based on imitation learning. The application establishes a wing nonlinear aeroelastic state space model, and generates an expert control trajectory dataset offline based on nonlinear model predictive control (NMPC); a double Actor-Critic network architecture including a value network, a control strategy network and a disturbance strategy network is constructed, the control strategy network is trained to minimize cumulative cost, and the disturbance strategy network is trained to maximize cumulative cost; a dynamic attenuation NMPC imitation learning regularization term is introduced in the middle stage of training to guide network updating; after training convergence, only the control strategy network is deployed online, and real-time anti-disturbance control is realized through single forward propagation. The application has the effects of high-precision, strong real-time performance, high smoothness and low energy consumption on flutter suppression, and can effectively reduce the wear of an actuator.

Description

Technical Field

[0001] This invention belongs to the field of aeroelastic control technology for aerospace vehicles, and in particular, it is a method and system for active suppression of wing flutter based on imitation learning and approximate dynamic programming. Background Technology

[0002] Aeroelasticity is an interdisciplinary field that studies the interaction of aerodynamic forces, elastic forces, and inertial forces. Its research is crucial for aircraft design and flight safety. In the field of aeroelasticity, flutter is a self-excited oscillatory instability phenomenon that originates from the non-conservative coupling between aerodynamics and structural dynamics. Once the flight speed exceeds the critical flutter velocity, flutter can easily trigger catastrophic structural damage within seconds, seriously threatening aircraft safety. In recent years, as modern aircraft have continued to develop towards lightweight, high flexibility, and large aspect ratios, the increased flexibility of wing structures has led to a gradual narrowing of the flutter safety margin, forcing academia and industry to invest heavily in research on active flutter suppression (AFS).

[0003] To address the aforementioned challenges, several control strategies have been proposed. The first category is based on sliding mode control (SMC). For example, Luo et al. (Luo M., Gao M., Cai G. Delayed full-state feedback control of airfoil flutter using sliding mode control method. Journal of Fluids and Structures, 2016, 61:262–273.) transformed the time-delayed state equation into a standard state equation with implicit time delay through a special integral transformation, and then used SMC to suppress time-delayed feedback flutter in the two-dimensional airfoil. Liu et al. (Liu Q., Xu Y., Xu C., Kurths J. The sliding mode control for an airfoil system driven by harmonic and colored Gaussian noise excitations. Applied Mathematical Modelling, 2018, 64:249–264.) addressed the problem of catastrophic structural failures and even aircraft disintegration caused by random jumping phenomena induced by colored noise. They established an airfoil model with colored Gaussian noise and implemented SMC (Simple Mode Control) on the airfoil system under harmonic forces and colored noise excitation. Although SMC can handle nonlinearity, its control input suffers from high-frequency chattering, which is extremely detrimental to flight control servos and other actuators, easily leading to mechanical wear and fatigue failure. While the PID controller commonly used in industrial scenarios has a simple structure, its parameters heavily rely on engineering experience, and its ability to handle complex nonlinearities is very limited.

[0004] The second category is based on Model Predictive Control (MPC) methods. MPC has been widely used in the field of active flutter suppression of airfoils due to its ability to effectively handle input and state constraints. Linear MPC (LMPC) simplifies online optimization into a convex quadratic programming (QP) problem that can be solved efficiently by linearizing the aeroelastic system at the equilibrium point, resulting in good real-time performance (Darabseh, T., Tarabulsi, A., Mourad, A.-HIDiscrete-Time Model Predictive Controller Using Laguerre Functions for ActiveFlutter Suppression of a 2D Wing with a Flap. Aerospace, 2022, 9(9): 475.). However, once the system operating point deviates from the linearized region (e.g., encountering large wind speed disturbances), the accuracy of the approximate model of LMPC drops sharply, failing to capture the complex coupling between nonlinear aerodynamic forces and structural stiffness, leading to a severe deterioration in control performance. To address this issue, Nonlinear Multiprocessing Control (NMPC) embeds complete nonlinear dynamics into the optimization process, maintaining extremely high control accuracy across all operating conditions (Artola, M., Goizueta, N., Wynn, A., Palacios, R. Aeroelastic Control and Estimation with a Minimal Nonlinear Modal Description. AIAA Journal, 2021, 59(7): 2697–2713.). However, because NMPC requires solving nonconvex nonlinear programming (NLP) at each control step, its computational load is extremely large, making it difficult to meet the millisecond-level real-time requirements of flight control. Therefore, traditional MPC falls into the fundamental dilemma of LMPC sacrificing accuracy for real-time performance, while NMPC sacrifices real-time performance for accuracy. More importantly, under external disturbances, the control inputs of both LMPC and NMPC will also fluctuate drastically, which not only greatly increases control energy consumption but also seriously threatens the lifespan of the actuators.

[0005] The third category is the Approximate Dynamic Programming (ADP) method based on neural networks. ADP approximates the numerical solution of the Hamilton-Jacobi-Bellman equations by training a neural network offline. During online inference, only one forward propagation of the network is needed to output the control quantity, which has a real-time advantage far exceeding that of NMPC. For example, Jia et al. (Jia S., Tang Y., Sun J., Ding Q. Data-driven active flutter control of airfoil with input constraints based on adaptive dynamic programming method. Journal of Vibration and Control, 2022, 28(13–14):1804–1817.) used data-driven modeling and the ADP method to suppress the limiting cycle oscillation caused by airfoil pitch-boom coupling without relying on an accurate nonlinear aeroelastic model. However, traditional ADP lacks an adversarial training mechanism for external extreme disturbances. Therefore, elastic approximate dynamic programming (RADP) considers introducing a disturbance network into the training process of traditional ADP to simulate external disturbances. However, the training process of RADP converges slowly, and the control accuracy of the controller trained by RADP is difficult to reach the level of MPC.

[0006] Based on the above existing technologies, it can be found that the current mainstream wing flutter suppression methods generally have the following common technical defects: (1) Conflict between efficiency and accuracy: LMPC is not accurate enough, and NMPC cannot meet the millisecond-level real-time requirements; (2) Lack of anti-disturbance smoothness in control: Whether it is SMC or MPC, when facing random disturbances such as external gusts, the control quantity will fluctuate significantly at high frequency, resulting in actuator wear, which not only increases maintenance costs but also increases control energy consumption; (3) Contradiction between training and deployment: Although the traditional RADP method has the advantage of real-time inference, it blindly explores in the initial training stage and is difficult to converge quickly, which hinders its practical application process.

[0007] Therefore, how to construct an active wing flutter suppression framework that can accelerate network convergence speed and improve control smoothness and low energy consumption under external disturbances while ensuring high accuracy and high real-time performance is a key technical problem that urgently needs to be solved in this field. Summary of the Invention

[0008] The purpose of this invention is to address the technical shortcomings of existing active flutter suppression methods for airfoils, such as insufficient accuracy of LMPC, poor real-time performance of NMPC, non-smooth SMC control, and slow convergence speed of traditional RADP networks. This invention proposes an active flutter suppression method for airfoils based on NMPC-guided elastic approximate dynamic programming (NGRADP). By introducing a dual Actor-Critic adversarial architecture, this invention achieves game-theoretic training between the control strategy and external disturbances. Simultaneously, it incorporates a regularization term based on the imitation learning of NMPC expert control trajectories to guide network parameter updates and improve control accuracy. The aim is to achieve flutter suppression with high accuracy, strong real-time performance, high smoothness, and low energy consumption.

[0009] The technical solution to achieve the objective of this invention is as follows: On the one hand, a method for active suppression of wing flutter based on imitation learning and elastic approximate dynamic programming is provided, the method comprising the following steps:

[0010] Step 1: Construct a state-space dynamics model of the wing aeroelastic system and define a cost function and an objective function that include state deviation, control energy, and control smoothness.

[0011] Step 2: Construct a nonlinear model predictive controller based on the state-space dynamics model, and use the nonlinear model predictive controller to solve offline to generate an expert control trajectory dataset;

[0012] Step 3: Construct a dual Actor-Critic adversarial neural network framework that includes a value network, a control policy network, and a perturbation policy network;

[0013] Step 4: Train the control policy network with the goal of minimizing the cumulative cost, train the perturbation policy network with the goal of maximizing the cumulative cost, and evaluate the cumulative cost through the value network, wherein an imitation learning regularization term based on the expert control trajectory dataset is introduced during the training of the control policy network.

[0014] Step 5: Alternately optimize and update the value network, control policy network, and perturbation policy network until the network converges;

[0015] Step 6: After training convergence, remove the value network and the perturbation policy network, solidify the control policy network, and use the control policy network to output control commands based on the sampled real-time wing status during the online control phase, which are then applied to the wing control channel to implement active flutter suppression.

[0016] Further, step 1 involves constructing a state-space dynamics model of the wing's aeroelastic system and defining a cost function and objective function that include state deviation, control energy, and control smoothness. Specifically, this includes:

[0017] Step 1-1: Construct a state vector using the wing's pitch angle, pitch rate, heave displacement, and heave velocity;

[0018] Steps 1-2 involve unifying linear structural dynamics, linear aerodynamic coupling relationships, nonlinear stiffness terms, and control inputs into the state equations to construct a state-space dynamic model of the wing's aeroelastic system.

[0019]

[0020] In the formula, For state vectors, The pitch angle, The pitch angular velocity, For buoyancy displacement, Floating / sinking speed; superscript Indicates transpose; Let R be the system matrix containing linear structural dynamics and linear aerodynamic coupling terms, and let R denote a real matrix. The nonlinear vector field is caused by pitch cubic stiffness. These are the corresponding dimensionless nonlinear coefficients; To control the input vector, The equivalent control torque acting on the pitch channel, This is the equivalent control force acting on the buoyancy channel;

[0021] Steps 1-3 define the performance index function to be optimized, which includes state deviation penalty, control energy penalty, and control smoothing increment penalty. :

[0022]

[0023] In the formula, For the terminal time, For the cost of operation, For the cost of the terminal, Let be the state vector at time t. These are the control input vectors at the current time t and the previous time, respectively. for The state vector at any given time;

[0024] The operating cost is defined as follows:

[0025]

[0026] In the formula, Let represent the error state at time t. For the target equilibrium state, The state weight matrix is... To control the energy weight matrix, To control the incremental smoothing weight matrix;

[0027] Terminal cost is defined as:

[0028]

[0029] In the formula, This is the terminal state weight matrix, used to improve the convergence accuracy of the predicted state at the end of the time domain;

[0030] Steps 1-4 thus construct a finite-time control problem that satisfies both dynamic constraints and actuator input constraints, i.e., the objective function is:

[0031]

[0032] In the formula, and These are the lower and upper bound vectors controlling the input, respectively. Indicates by adjusting Make The value is the smallest.

[0033] Further, step 2 involves constructing a nonlinear model predictive controller based on the state-space dynamics model, and using the nonlinear model predictive controller to generate an expert control trajectory dataset offline, specifically including:

[0034] Step 2-1: Based on the state-space dynamics model, construct a nonlinear model predictive controller, namely the NMPC controller, using the CasaADi and IPOPT solvers.

[0035] Step 2-2: Discretize the state-space dynamics model to construct a discrete finite-time domain optimization objective function that includes the prediction time domain. :

[0036]

[0037] In the formula, This represents the number of forward prediction steps; Number the prediction step; To be at the current sampling time Known error state The first prediction The error state at time t is... Predicted state at time ; for Predictive control input at specific times; To predict control increments; for Predictive control input at specific times; To predict the end state in the time domain;

[0038] Steps 2-3: Under the condition of satisfying the upper and lower limits of the control input, use the nonlinear model predictive controller to solve the nonlinear programming problem and obtain the optimal control sequence.

[0039]

[0040] In the formula, The continuous nonlinear aeroelastic model is based on the sampling period. Discretized state transition mapping; For time step The optimal predictive control sequence obtained at the specified location; asterisk This represents the optimal solution to the optimization problem;

[0041] In solving the nonlinear programming problem, a hot-start strategy is adopted, which shifts the optimal control sequence obtained at the previous sampling time forward and uses it as the initial guess for the current optimization; for the end of the sequence, the last term of the previous optimal sequence is copied as the initial guess.

[0042] Steps 2-4: Repeat steps 2-1 to 2-2 under multiple initial states and disturbance conditions to collect the states and corresponding optimal control inputs, forming an expert control trajectory dataset.

[0043] Furthermore, the dual Actor-Critic adversarial neural network framework described in step 3 specifically includes:

[0044] The network input error state is constructed by the deviation between the target state and the current state. ;

[0045] The value network is used to fit the cumulative discount cost under the current error state, and the output layer uses the ReLU activation function to output a non-negative scalar value;

[0046] The control strategy network is used to generate control commands, and the output layer uses the Tanh activation function to output normalized control and disturbance quantities.

[0047] The perturbation policy network is used to generate adversarial external perturbations during the training phase, and the output layer uses the Tanh activation function to output normalized control and perturbation quantities.

[0048] The value network, control policy network, and disturbance policy network all adopt a deep neural network structure containing at least two hidden layers and with the activation function ELU.

[0049] Furthermore, the single-step cost function under the combined action of the control policy network and the disturbance policy network is defined as follows: :

[0050]

[0051] in, for Error state at time, The target equilibrium state; The control input is the output of the control strategy network; Adversarial perturbations generated for the perturbation policy network; The perturbation penalty weight matrix; Indicates the previous moment, i.e. Time-based control input; ,here and It has no practical meaning and is only used as an indicator variable;

[0052] The disturbance term generated by the disturbance policy network is introduced into the single-step cost function in a negative form, so that the disturbance policy network finds the most unfavorable disturbance by increasing the cumulative cost, and the control policy network learns disturbance rejection and smooth control actions by decreasing the cumulative cost.

[0053] Furthermore, in step 4, an imitation learning regularization term based on the expert control trajectory dataset is introduced during the training process of the control policy network. Specifically, this involves introducing an imitation learning regularization term based on the expert control trajectory dataset into the loss function of the control policy network. The dynamic guidance coefficient of the imitation learning regularization term employs a dynamic decay mechanism during training, first setting it to zero, then enabling it, and gradually decaying and annealing it to zero. Specifically, this includes:

[0054] Define imitation learning items :

[0055]

[0056] In the formula, For the current moment The control policy network output at the location, For the corresponding state in the expert control trajectory dataset The output is controlled by experts at the location. Indicates the calculation of Euclidean distance;

[0057] Construct the total loss function of the control policy network. :

[0058]

[0059] In the formula, For the k-th step to the k-th step The cumulative cost of the discounts on each step for Error state at time, Discount factor and This is used to reduce the weight of long-term costs in the current loss function; Let be the dynamic guidance coefficient for the j-th training round; This represents the number of forward prediction steps; Indicates the value network to the first Step end error state The cumulative cost estimate;

[0060] In the initial training phase, the dynamic guidance coefficient is set to 0 to disable the imitation learning term. In the middle training phase, the imitation learning term is enabled, and the dynamic guidance coefficient gradually decays with each training round. In the later training phase, the dynamic guidance coefficient is set to zero again to ensure that the final minimax game objective remains unchanged, as shown below:

[0061]

[0062]

[0063]

[0064] In the formula, To enable rounds for imitation learning items, To guide the end of the round, For the first +1 round of training dynamic guidance coefficient.

[0065] Furthermore, the alternating optimization update described in step 5 specifically includes:

[0066] Construct the Actor-Critic 1 module: fix the network parameters of the perturbation strategy. The value network parameters are updated based on the N-step temporal difference error. Subsequently, based on the control strategy network total loss function Update the control strategy network parameters ;

[0067] Construct the Actor-Critic 2 module: Fix the updated network parameters of the control policy. The value network parameters are updated again based on the same N-step temporal difference error. Subsequently, based on the perturbation loss function Update the network parameters of the perturbation strategy ;

[0068] The Actor-Critic 1 and Actor-Critic 2 modules are executed alternately until the losses of the value network, control policy network, and perturbation policy network reach the convergence criterion and form a game saddle point.

[0069] Furthermore, the convergence of the alternating optimization update is guaranteed by the compression mapping property of the N-step Bellman operator. When the optimal perturbation strategy is fixed, the control strategy minimizes the value function; when the optimal control strategy is fixed, the perturbation strategy maximizes the value function, thus converging to the optimal saddle point value.

[0070]

[0071] In the formula, The optimal value function at the saddle point; The optimal control strategy; The optimal perturbation strategy; and These are candidate control strategies and candidate perturbation strategies, respectively. This means that while the control strategy minimizes the cumulative cost, the perturbation strategy maximizes the cumulative cost. To predict the end-time value discount term; for Error status at any given time; This indicates the optimal perturbation strategy. When the value function is fixed, the control strategy that minimizes the value function is the one that minimizes the value function. Indicating the optimal control strategy When fixed, the perturbation strategy that maximizes the value function; and This represents the strategy itself that achieves the minimum or maximum value. It has no practical meaning and is only used as an indicator variable.

[0072] Furthermore, in step 6, during the online control phase, the control strategy network is used to output control commands based on the sampled real-time wing status, specifically including:

[0073] Step 6-1: Remove the value network, the disturbance policy network, and the solution algorithm of the nonlinear model predictive controller from the online control program, and retain only the control policy network with fixed parameters;

[0074] Step 6-2: During the real-time control sampling period, the wing state is collected in real time and an error vector is constructed as the input to the control strategy network;

[0075] Step 6-3: The control strategy network calculates and outputs the original control signal through a single forward propagation.

[0076] Step 6-4: Perform physical amplitude saturation limiting processing on the original control signal:

[0077]

[0078] In the formula, It is a channel-by-channel saturation function; These are the network parameters for the optimal control strategy that are fixed after training. This is the control input at time k;

[0079] Step 6-5: Apply the limited control commands to the pitch control channel and buoyancy control channel of the wing respectively.

[0080] On the other hand, a learning-based elastic approximate dynamic programming-based active wing flutter suppression system is provided for implementing the learning-based elastic approximate dynamic programming-based active wing flutter suppression method. The system includes:

[0081] The first module is used to: construct a state-space dynamics model of the wing aeroelastic system, and define a cost function and an objective function that include state deviation, control energy, and control smoothness;

[0082] The second module is used to: construct a nonlinear model predictive controller based on the state-space dynamics model, and use the nonlinear model predictive controller to solve offline to generate an expert control trajectory dataset;

[0083] The third module is used to implement: constructing a dual Actor-Critic adversarial neural network framework that includes a value network, a control policy network, and a perturbation policy network;

[0084] The fourth module is used to: train the control policy network with the goal of minimizing the cumulative cost, train the perturbation policy network with the goal of maximizing the cumulative cost, and evaluate the cumulative cost through the value network, wherein an imitation learning regularization term based on the expert control trajectory dataset is introduced during the training process of the control policy network.

[0085] The fifth module is used to alternately optimize and update the value network, control policy network, and perturbation policy network until the network converges.

[0086] The sixth module is used to: remove the value network and the perturbation policy network after training convergence, solidify the control policy network, and use the control policy network to output control commands based on the sampled real-time wing status during the online control phase, which are then applied to the wing control channel to implement active flutter suppression.

[0087] On the other hand, a computer-readable storage medium is provided on which a computer program is stored, which, when executed by a processor, implements the aforementioned active wing flutter suppression method based on imitation learning and elastic approximate dynamic programming.

[0088] Compared with the prior art, the significant advantages of this invention are:

[0089] (1) By using the “offline training-online inference” deployment architecture, the heavy optimization calculation is transferred to the offline training stage. The online control only needs to perform a single forward propagation of the control policy network to obtain the control quantity. The solution speed is about 3.4 times that of LMPC and 78 times that of NMPC. Moreover, the inference time is basically unaffected by the increase of the prediction time domain. It can stably meet the millisecond-level real-time requirements of flight flutter control and provide a reliable guarantee for the real-time active suppression of highly flexible wings.

[0090] (2) In the offline training stage, NMPC high-precision expert control trajectory is introduced to carry out imitation learning guidance, guiding the control strategy network to approach the nonlinear optimal control solution; compared with the traditional elastic approximate dynamic programming (RADP), the tracking accuracy of pitch angle (root mean square error (RMSE)) is improved by about 21.4%, and the suppression accuracy of floating displacement (0.3492) is also better than LMPC (0.4013) and NMPC (0.3638), and the overall state convergence accuracy reaches a high level.

[0091] (3) To address the problem of high-frequency fluctuations in control quantity during the process of model predictive control (MPC) dealing with external disturbances, this invention constructs a minimax game between the control strategy and the external disturbance based on a dual Actor-Critic architecture, thereby optimizing the control strategy under adverse disturbance conditions. Under external Gaussian noise disturbances, the root mean square value of the control increment of the method of this invention is only 1 / 5 to 1 / 4 of the corresponding results of traditional linear model predictive control (LMPC) and nonlinear model predictive control (NMPC), which can significantly reduce high-frequency fluctuations in control quantity while maintaining state suppression accuracy, effectively reducing servo wear and reducing control energy consumption.

[0092] (4) To address the problem of low training search efficiency in traditional RADP methods, this invention adopts a dynamic annealing imitation learning mechanism: the imitation term is not enabled in the initial stage of random initialization. When the training progresses to the intermediate stage where the NMPC expert trajectory can be stably obtained, the imitation learning term is enabled in the control policy network loss to guide the network to update along the expert control path. In the later stage of training, the guiding coefficient is gradually annealed to zero. This mechanism can improve the training loss reduction speed and training stability without affecting the final minimax game objective.

[0093] (5) Based on the constant translation lemma and the compression mapping bound of the N-step Bellman operator (firstly, the compression bound of the N-step Bellman operator is established by the constant translation lemma, so that the value function sequence converges to the optimal value function; then, through the intermediate value function chain, it is proved that the value of the control strategy does not increase monotonically after the imitation term is annealed to zero, and the value of the perturbation strategy does not decrease monotonically, and finally the game saddle point is formed), the mathematical reliability of the convergence of the value function and the strategy sequence to the game saddle point under the alternating optimization strategy is derived and proved, which provides a mathematical basis for the ideal operator iteration of the NGRAP algorithm, but does not replace the finite sample neural network training error analysis, the closed-loop stability verification of specific aircraft, and the engineering safety assessment.

[0094] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0095] Figure 1 This is a flowchart illustrating the overall implementation of the NMPC-guided Elastic Approximate Dynamic Programming (NGRADP) active wing flutter suppression method of this invention.

[0096] Figure 2 This is a diagram showing the limit cycle oscillation state response of a two-degree-of-freedom airfoil system without applied control in one embodiment.

[0097] Figure 3 This is a schematic diagram illustrating the overall architecture of the NGRAP algorithm in one embodiment.

[0098] Figure 4 This is a flowchart of the dual Actor-Critic alternating iterative optimization process in the NGRAP algorithm in one embodiment.

[0099] Figure 5 This is a graph showing the evolution of the loss function during the training process of three neural networks (value network, control policy network, and perturbation policy network) in one embodiment. Figure 5 (a) in the figure shows the change in value network loss when the control strategy is fixed. Figure 5 (b) shows the change in value network loss under the fixed perturbation strategy. Figure 5 (c) in the figure is the loss function graph of the two policy networks.

[0100] Figure 6 This is a comparison chart of how the loss functions of NGRAP and RADP change with the number of iterations in one embodiment. Figure 6 (a) in the figure is a comparison chart showing how the value network loss changes with the number of iterations. Figure 6 (b) in the figure is a comparison of the network loss of the control strategy as a function of the number of iterations.

[0101] Figure 7This is a comparison diagram of the state changes of NGRAP and RADP during flutter suppression in one embodiment.

[0102] Figure 8 This is a comparison chart of control changes between NGRAP, LMPC, and NMPC in one embodiment. Figure 8 (a) in the graph is a curve showing the change of LMPC control input over time. Figure 8 (b) in the figure is a graph showing the NMPC control input changing over time. Figure 8 (c) in the figure is a graph showing the change of NGRAP control input over time.

[0103] Figure 9 This is a quantitative comparison chart of NGRAP, LMPC, and NMPC in terms of control accuracy, control amplitude, and control smoothness in one embodiment. Figure 9 Figure (a) shows a comparison of the state control accuracy (root mean square error) of the three methods in terms of pitch angle and heave displacement. Figure 9 (b) in the figure is a comparison of the three methods in terms of the root mean square value of the control input (control amplitude) and the root mean square value of the control increment (control smoothness).

[0104] Figure 10 This is a comparison of the average single-step control solution time of the present invention with LMPC and NMPC in different prediction time domains in one embodiment. Detailed Implementation

[0105] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0106] It should be noted that if the embodiments of the present invention involve descriptions such as "first" and "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include at least one of those features. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0107] In one embodiment, an active flutter suppression method for wing based on imitation learning and approximate dynamic programming is provided, applicable to the nonlinear aeroelastic instability control of highly flexible, high aspect ratio wings. The method includes the following steps:

[0108] Step 1: Construct a state-space dynamics model of the wing aeroelastic system and define a cost function and an objective function that include state deviation, control energy, and control smoothness.

[0109] Step 2: Construct a nonlinear model predictive controller based on the state-space dynamics model, and use the nonlinear model predictive controller to solve offline to generate an expert control trajectory dataset;

[0110] Step 3: Construct a dual Actor-Critic adversarial neural network framework that includes a value network, a control policy network, and a perturbation policy network;

[0111] Step 4: Train the control policy network with the goal of minimizing the cumulative cost, train the perturbation policy network with the goal of maximizing the cumulative cost, and evaluate the cumulative cost through the value network, wherein an imitation learning regularization term based on the expert control trajectory dataset is introduced during the training of the control policy network.

[0112] Step 5: Alternately optimize and update the value network, control policy network, and perturbation policy network until the network converges;

[0113] Step 6: After training convergence, remove the value network and the perturbation policy network, solidify the control policy network, and use the control policy network to output control commands based on the sampled real-time wing status during the online control phase, which are then applied to the wing control channel to implement active flutter suppression.

[0114] Furthermore, in one embodiment, step 1, which involves constructing a state-space dynamics model of the wing aeroelastic system and defining a cost function and an objective function that include state deviation, control energy, and control smoothness, specifically includes:

[0115] Step 1-1: Construct a state vector using the wing's pitch angle, pitch rate, heave displacement, and heave velocity;

[0116] Steps 1-2 involve unifying linear structural dynamics, linear aerodynamic coupling relationships, nonlinear stiffness terms, and control inputs into the state equations to construct a state-space dynamic model of the wing's aeroelastic system.

[0117]

[0118] In the formula, For state vectors, The pitch angle, The pitch angular velocity, For buoyancy displacement, Floating / sinking speed; superscript Indicates transpose; Let R be the system matrix containing linear structural dynamics and linear aerodynamic coupling terms, and let R denote a real matrix. The nonlinear vector field is caused by pitch cubic stiffness. These are the corresponding dimensionless nonlinear coefficients; To control the input vector, The equivalent control torque acting on the pitch channel, This is the equivalent control force acting on the buoyancy channel;

[0119] Steps 1-3 define the performance index function to be optimized, which includes state deviation penalty, control energy penalty, and control smoothing increment penalty. :

[0120]

[0121] In the formula, For the terminal time, For the cost of operation, For the cost of the terminal, Let be the state vector at time t. These are the control input vectors at the current time t and the previous time, respectively. for The state vector at any given time;

[0122] The operating cost is defined as follows:

[0123]

[0124] In the formula, Let represent the error state at time t. For the target equilibrium state, The state weight matrix is... To control the energy weight matrix, To control the incremental smoothing weight matrix;

[0125] Terminal cost is defined as:

[0126]

[0127] In the formula, This is the terminal state weight matrix, used to improve the convergence accuracy of the predicted state at the end of the time domain;

[0128] Steps 1-4 thus construct a finite-time control problem that satisfies both dynamic constraints and actuator input constraints, i.e., the objective function is:

[0129]

[0130] In the formula, and These are the lower and upper bound vectors controlling the input, respectively. Indicates by adjusting Make The value is the smallest.

[0131] This step is a foundational preliminary step for the entire method. It is primarily used to generate the zero-equilibrium target state offline, verify the system's limit cycle oscillations, and provide the mathematical basis for multi-objective trade-offs in subsequent sub-steps. The online control phase employs direct inference of the control strategy network from step 6, eliminating the need for real-time solving of the aforementioned integral optimization problem.

[0132] Furthermore, in one embodiment, step 2 involves constructing a nonlinear model predictive controller based on the state-space dynamics model, and using the nonlinear model predictive controller to generate an expert control trajectory dataset offline, specifically including:

[0133] Step 2-1: Based on the state-space dynamics model, construct a nonlinear model predictive controller, namely the NMPC controller, using the CasaADi and IPOPT solvers.

[0134] Step 2-2: Discretize the state-space dynamics model to construct a discrete finite-time domain optimization objective function that includes the prediction time domain. :

[0135]

[0136] In the formula, This represents the number of forward prediction steps; Number the prediction step; To be at the current sampling time Known error state The first prediction The error state at time t is... Predicted state at time ; for Predictive control input at specific times; To predict control increments; for Predictive control input at specific times; To predict the end state in the time domain;

[0137] Steps 2-3: Under the condition of satisfying the upper and lower limits of the control input, use the nonlinear model predictive controller to solve the nonlinear programming problem and obtain the optimal control sequence.

[0138]

[0139] In the formula, The continuous nonlinear aeroelastic model is based on the sampling period. Discretized state transition mapping; For time step The optimal predictive control sequence obtained at the specified location; asterisk This represents the optimal solution to the optimization problem;

[0140] In solving the nonlinear programming problem, a hot-start strategy is adopted, which shifts the optimal control sequence obtained at the previous sampling time forward and uses it as the initial guess for the current optimization; for the end of the sequence, the last term of the previous optimal sequence is copied as the initial guess.

[0141] Steps 2-4: Repeat steps 2-1 to 2-3 under multiple initial states and disturbance conditions to collect the states and corresponding optimal control inputs, forming an expert control trajectory dataset.

[0142] In this process, based on the smooth continuity of the optimal control trajectory, the present invention adopts a hot start strategy to shift the optimal control sequence of the previous moment as the initial guess of the current optimization variable, thereby improving the optimization convergence speed of the solution vector.

[0143] This step is performed after the modeling in step 1 is completed. NMPC, acting as a high-precision expert controller, generates the bootstrap dataset. Furthermore, this step provides the data source for imitation learning in step 4.

[0144] Furthermore, in one embodiment, the dual Actor-Critic adversarial neural network framework described in step 3 specifically includes:

[0145] The network input error state is constructed by the deviation between the target state and the current state. This serves as the common input for the three networks:

[0146]

[0147] In the formula, It includes the pitch angle, pitch rate, buoyancy displacement, and buoyancy velocity deviations relative to the zero equilibrium point. The state dimension is 4, consistent with the state-space model in step 1.

[0148] The value network is used to fit the cumulative discount cost under the current error state, and the output layer uses the ReLU activation function to output a non-negative scalar value;

[0149] The control policy network is used to generate control commands, and the output layer uses the Tanh activation function to output normalized control quantities.

[0150] The perturbation policy network is used to generate adversarial external perturbations during the training phase, and the output layer uses the Tanh activation function to output normalized perturbation quantities.

[0151] The value network, control policy network, and disturbance policy network all adopt a deep neural network structure containing at least two hidden layers and with the activation function ELU.

[0152] Preferably, in some embodiments, the single-step cost function under the combined action of the control policy network and the disturbance policy network is defined as :

[0153]

[0154] in, for Error state at time, The target equilibrium state; The control input is the output of the control strategy network; Adversarial perturbations generated for the perturbation policy network; The perturbation penalty weight matrix; Indicates the previous moment, i.e. Time-based control input; ,here and It has no practical meaning and is only used as an indicator variable;

[0155] The disturbance term generated by the disturbance policy network is introduced into the single-step cost function in a negative form, so that the disturbance policy network finds the most unfavorable disturbance by increasing the cumulative cost, and the control policy network learns disturbance rejection and smooth control actions by decreasing the cumulative cost.

[0156] The condition for executing this step is to complete the construction of the nonlinear dynamic model and objective function in step 1. Its purpose is to transform the wing flutter suppression problem into a network fitting and game training problem, providing a network architecture and utility evaluation criteria for subsequent offline training.

[0157] Further, in one embodiment, step 4 introduces an imitation learning regularization term based on the expert control trajectory dataset during the training process of the control policy network. Specifically, this involves introducing an imitation learning regularization term based on the expert control trajectory dataset into the loss function of the control policy network. The dynamic guidance coefficient of the imitation learning regularization term employs a dynamic decay mechanism during training, first setting it to zero, then enabling it, and gradually decaying and annealing it to zero. Specifically, this includes:

[0158] Define imitation learning items :

[0159]

[0160] In the formula, For the current moment The control policy network output at the location, For the corresponding state in the expert control trajectory dataset The output is controlled by experts at the location. Indicates the calculation of Euclidean distance;

[0161] Construct the total loss function of the control policy network. :

[0162]

[0163] In the formula, For the k-th step to the k-th step The cumulative cost of the discounts on each step for Error state at time, Discount factor and This is used to reduce the weight of long-term costs in the current loss function; Let be the dynamic guidance coefficient for the j-th training round; This represents the number of forward prediction steps; Indicates the value network to the first Step end error state The cumulative cost estimate.

[0164] In the initial training phase, the dynamic guidance coefficient is set to 0 to disable the imitation learning term. In the middle training phase, the imitation learning term is enabled, and the dynamic guidance coefficient gradually decays with each training round. In the later training phase, the dynamic guidance coefficient is set to zero again to ensure the final minimax game objective remains unchanged, represented as:

[0165]

[0166]

[0167]

[0168] In the formula, To enable rounds for imitation learning items, To guide the end of the round, For the first +1 round of training dynamic guidance coefficient.

[0169] Here, in the initial training phase, due to the random initialization of states and network parameters and the significant burden of solving nonlinear programming problems, let... Imitation learning is not enabled. Once training reaches an intermediate stage where it can stably generate NMPC expert trajectories, then... Furthermore, it continuously decays, and the control strategy is guided by expert control paths. By gradually reducing the value until it reaches zero, the final game objective is not altered by the imitation term.

[0170] This step is performed only after the expert controller from step 2 has been established. Its purpose is to accelerate the offline training of the network and improve the accuracy of the control network's output actions through imitation learning, without increasing the computational cost during online inference.

[0171] Furthermore, in one embodiment, the alternating optimization update in step 5 specifically includes:

[0172] (1) Construct the Actor-Critic 1 module: fix the network parameters of the perturbation strategy. The value network parameters are updated based on the N-step temporal difference error. Subsequently, based on the control strategy network total loss function Update the control strategy network parameters To reduce the loss of control strategy;

[0173] Here, the value network parameters are updated using the following formula:

[0174]

[0175] In the formula, For the first Value network parameters before the round update; The updated intermediate value network parameters in the Actor-Critic 1 module; The learning rate of the value network; For based on The value network loss function constructed using the step TD error; This represents the gradient of the value network loss function with respect to the current value network parameters. Alternating iteration rounds;

[0176] Here, the control policy network is updated using the following formula:

[0177]

[0178] In the formula, and The first Control strategy network parameters before and after the update; To control the learning rate of the strategy network; For inclusion The loss function for the step-by-step cumulative cost, end-value estimation, and NMPC imitation error control strategy; This update represents the gradient of the control policy loss function with respect to the control policy network parameters. It is performed under the condition that the perturbation policy is fixed, and is used to reduce accumulated cost and improve control smoothness.

[0179] (2) Construct the Actor-Critic 2 module: fix the updated network parameters of the control strategy. The value network parameters are updated again based on the same N-step temporal difference error. Subsequently, based on the perturbation loss function Update the network parameters of the perturbation strategy To reduce the loss of control strategy;

[0180] Here, the updated formula is:

[0181]

[0182]

[0183] In the formula, and The first Perturbation policy network parameters before and after the round update; The perturbation strategy network learning rate; Let the perturbation strategy loss function be used. This represents the gradient of the perturbation policy loss function with respect to the perturbation policy network parameters. Since... Containing a negative sign, minimize using gradient descent. This is equivalent to performing gradient ascent on the cumulative cost, thereby enabling the perturbation strategy to seek more unfavorable external perturbations.

[0184] (3) The Actor-Critic 1 module and the Actor-Critic 2 module are executed alternately until the losses of the value network, control strategy network and perturbation strategy network reach the convergence criterion and form a game saddle point.

[0185] Furthermore, the convergence of the alternating optimization update is guaranteed by the compressibility mapping property of the N-step Bellman operator. When the optimal perturbation strategy is fixed, the control strategy minimizes the value function; when the optimal control strategy is fixed, the perturbation strategy maximizes the value function, thus converging to the optimal saddle point value.

[0186]

[0187]

[0188]

[0189] In the formula, The optimal value function at the saddle point; The optimal control strategy; The optimal perturbation strategy; and These are candidate control strategies and candidate perturbation strategies, respectively. This means that while the control strategy minimizes the cumulative cost, the perturbation strategy maximizes the cumulative cost. To predict the end-time value discount term; for Error status at any given time; This indicates the optimal perturbation strategy. When the value function is fixed, the control strategy that minimizes the value function is the one that minimizes the value function. Indicating the optimal control strategy When fixed, the perturbation strategy that maximizes the value function; and This represents the strategy itself that achieves the minimum or maximum value. It has no practical meaning and is only used as an indicator variable.

[0190] The execution condition for this step is to complete the loss function setting in step 4. Its purpose is to decompose the minimax game into two alternating single-layer policy evaluation and policy improvement processes, and to provide a mathematical basis for the convergence of the algorithm under the assumed conditions.

[0191] Furthermore, in one embodiment, step 6, during the online control phase, utilizes the control strategy network to output control commands based on the sampled real-time wing status, specifically including:

[0192] Step 6-1: Remove the value network, the disturbance policy network, and the solution algorithm of the nonlinear model predictive controller from the online control program, and retain only the control policy network with fixed parameters;

[0193] Step 6-2: Within the real-time control sampling period, collect the wing state in real time and construct the network input error state. As input to the control policy network;

[0194] Step 6-3: The control strategy network calculates and outputs the original control signal, i.e., the control input, through a single forward propagation.

[0195] Step 6-4, perform physical amplitude saturation on the original control signal ( Amplitude limiting:

[0196]

[0197] In the formula, It is a channel-by-channel saturation function; These are the network parameters for the optimal control strategy that are fixed after training. This is the control input at time k;

[0198] Step 6-5: Apply the limited control commands to the pitch control channel and buoyancy control channel of the wing respectively.

[0199] This step is performed when the network parameters have been frozen and embedded in the onboard computing platform. Its purpose is to apply the offline-trained "intelligent strategy" to actual flight conditions. A single-step calculation takes only microseconds to milliseconds, ensuring millisecond-level real-time performance, actuator-friendly operation, and energy conservation while meeting high-precision flutter suppression requirements.

[0200] By way of example, in one embodiment, the method of the present invention will be described in detail with reference to the accompanying drawings.

[0201] Combination Figure 1 This invention provides an active flutter suppression method for wing based on nonlinear model predictive control (NMPC)-guided elastic approximate dynamic programming. This method places the computationally intensive nonlinear model predictive control and adversarial network training in the offline phase, while the converged control strategy network is placed in the online phase. The offline phase sequentially completes nonlinear aeroelastic system modeling, NMPC expert control trajectory generation, dual Actor-Critic network construction, NMPC imitation learning guidance, and alternating game training of control strategy and disturbance strategy. The online phase retains only the control strategy network, obtaining control commands through a single forward propagation based on the real-time wing state. The method includes the following steps.

[0202] Step 1: Establish a nonlinear aeroelastic state-space model for the two-degree-of-freedom airfoil and construct performance indices that simultaneously consider state suppression accuracy, control energy, and control smoothness. Specifically:

[0203] Step 1-1: Select the wing segment pitch angle, pitch rate, heave displacement, and heave velocity as state variables. The state vector is defined as follows:

[0204]

[0205] In the formula, Indicates pitch angle, Indicates pitch angular velocity, Indicates buoyancy displacement. Indicates buoyancy, superscript This represents the matrix transpose. In this embodiment, the wing segment equilibrium state is set to zero, so that the flutter suppression task is transformed into an adjustment problem of the state converging towards the zero equilibrium point.

[0206] Steps 1-2 involve uniformly incorporating structural inertial forces, damping forces, linear aeroelastic coupling terms, cubic nonlinear stiffness terms, and control inputs into the state equations, resulting in:

[0207]

[0208] In the formula, The system matrix characterizing the coupling relationship between linear structural dynamics and aerodynamics. For nonlinear vector fields, This is the equivalent control input applied to the pitch and buoyancy channels.

[0209] Here, the cubic stiffness nonlinearity and the control input are expressed as follows:

[0210]

[0211] In the formula, and The dimensionless coefficients are the result of mapping the pitch cubic stiffness term to the two motion channels. For the pitch channel equivalent control torque, This represents the equivalent control force for the buoyancy channel. The control input acts only on the second and fourth state equations in the state vector.

[0212] Steps 1-3, in this embodiment, take the dimensionless mass ratio. centroid offset Square of gyration The uncoupled natural frequencies of buoyancy and pitch are taken as 28.1 Hz and 62.8 Hz, respectively. The elastic axis is located at the quarter-chord length position, and the dimensionless inflow velocity is taken as... Substituting the above parameters into the dimensionless model, we obtain the explicit dynamic model used for controller training and comparative simulation:

[0213]

[0214] The above equation gives the continuous-time evolution relationship of the four state components in this embodiment, where and This describes the effect of pitch cubic stiffness nonlinearity in two state channels. The initial state is taken when no control is applied. A 30-second simulation can yield the following results. Figure 2 The continuous limit cycle oscillation shown indicates the nonlinear flutter condition that needs to be actively suppressed.

[0215] Steps 1-4 define a continuous-time performance index function to address the three objectives: rapid state convergence, constrained control energy, and smooth control changes between adjacent time steps.

[0216]

[0217] In the formula, For the terminal time, For the cost of operation, For the cost of the terminal, for The state vector at time t, Each is the current The control input vector at time step and the previous time step. for The state vector at time t.

[0218] Here, the operating cost is specifically defined as:

[0219]

[0220] In the formula, Let represent the error state at time t. Let Q be the state of equilibrium, and let Q be the state weight matrix. To control the energy weight matrix, The control increment smoothing weight matrix is ​​used. The first term is used to penalize deviations from the equilibrium point in pitch and buoyancy states. The second term is used to limit the amplitude of the control input and control energy consumption. The third term is used to suppress abrupt changes in adjacent control commands, thereby reducing the degree of frequent reverse or impact actions of the actuator.

[0221] Here, the terminal cost is defined as:

[0222]

[0223] In the formula, This is the terminal state weight matrix, used to improve the convergence accuracy of the predicted state at the end of the time domain.

[0224] This constitutes a finite-time control problem that satisfies both dynamic constraints and actuator input constraints:

[0225]

[0226] In the formula, and These are the lower and upper bound vectors controlling the input, respectively. Indicates by adjusting Make The value is the smallest.

[0227] The aforementioned objective function not only serves as the optimization objective for the NMPC expert controller, but also provides a consistent state-energy-smoothness evaluation benchmark for the subsequent cost function design of the NGRAP network.

[0228] Step 2: Based on the nonlinear aeroelastic state-space model from Step 1, construct an NMPC expert controller and generate the expert control trajectory required for imitation learning offline. Specifically:

[0229] Step 2-1, let the control sampling time interval be... The continuous-time dynamics are written using first-order Euler discretization as follows:

[0230]

[0231] In the formula, For discrete sampling times, , The first The state and control input at each sampling time, For the first The state at each sampling time point. Other numerical integration methods that meet the same sampling period and accuracy requirements can also be used to obtain the discrete mapping. This embodiment uses the above formula to clarify the state rolling prediction relationship.

[0232] Step 2-2, let the prediction time domain length be... Control increment The discrete finite-time performance index is:

[0233]

[0234] In the formula, This represents the number of forward prediction steps; Number the prediction step; To be at the current sampling time Known error state The first prediction The error state at time t is... Predicted state at time ; for Predictive control input at specific times; To predict control increments; for Predictive control input at specific times; To predict the end state in the time domain.

[0235] This embodiment directly embeds the complete nonlinear dynamics into the prediction process, without replacing it with an equilibrium point linearization model under large deviation conditions. Therefore, it can retain the influence of cubic stiffness and aeroelastic coupling on the predicted state.

[0236] Steps 2-3 are used to predict control sequences. To optimize the variables, solve:

[0237]

[0238] In the formula, The continuous nonlinear aeroelastic model is based on the sampling period. Discretized state transition mapping; For time step The optimal predictive control sequence obtained at the specified location; asterisk This represents the optimal solution to the optimization problem.

[0239] This embodiment uses CasADi to construct the nonlinear programming problem and IPOPT to solve it. It should be noted that the software and solver described are merely preferred tools for implementing this nonlinear optimization, and there is no restriction on using other implementation methods capable of solving nonlinear programming problems with the same constraints.

[0240] To reduce the number of iterations required to solve the nonlinear programming problem repeatedly at adjacent sampling times, the optimal control sequence obtained at the previous sampling time is shifted forward to construct the initial guess for the current optimization.

[0241]

[0242] In the formula, For the current number The first time step The initial guess for hot start of a predictive control variable, superscript This represents the initial iteration value of the nonlinear programming solver; The control quantity corresponding to the current prediction step after shifting forward the optimal control sequence obtained at the previous sampling time; superscript This represents the optimal solution of the previous round of nonlinear programming; Number the prediction step; This indicates that the initial guess at the end of the current prediction time domain is obtained by copying the last term of the previous optimal sequence. This process utilizes the temporal continuity of the optimal control trajectory to reduce the solver's search range without changing the optimization objective and constraints.

[0243] Steps 2-4: Repeat steps 2-1 to 2-3 under multiple initial states and disturbance conditions, record the states and NMPC optimal control inputs, and form an expert trajectory dataset. :

[0244]

[0245] In the formula, This is a dataset of expert control trajectories obtained offline by NMPC. The number of expert trajectory entries; Number the expert's trajectory; For the first The number of valid sampling points contained in each expert trajectory; Number the sampling points within this trajectory; For the first The trajectory in the first The wing state vector at each sampling point; For NMPC in state The expert-controlled input is obtained from the following.

[0246] This dataset is only used to guide the control policy network during the offline training phase; the NMPC solver is not called during the online suppression phase.

[0247] Step 3: Construct a dual Actor-Critic adversarial architecture comprising a value network, a control policy network, and a perturbation policy network. Specifically:

[0248] Step 3-1, set the target state Construct the network input error state:

[0249]

[0250] In the formula, Includes pitch angle, pitch rate, heave displacement, and heave velocity relative to the zero equilibrium point. The deviation is as follows. The state dimension is 4, consistent with the state-space model in step 1.

[0251] Step 3-2, respectively using parameters , and Let represent the trainable parameters of the value network, control policy network, and perturbation policy network. The input-output relationship of the three networks is as follows:

[0252]

[0253] In the formula, This is a cumulative cost estimate starting from the current error state. To control the network output of the policy, This is the output of the perturbation strategy network.

[0254] In this embodiment, all three networks contain two hidden layers, each with 256 neurons and employing the ELU activation function. The value network output layer uses the ReLU activation function to output a non-negative scalar, while the control policy network and perturbation policy network output layers use the Tanh activation function to output normalized vectors.

[0255] Step 3-3: Establish the single-step cost function under the combined effect of the control strategy and the disturbance strategy:

[0256]

[0257] In the formula, , This is the perturbation penalty weight matrix. The control input is the output of the control strategy network; Adversarial perturbations generated for the perturbation policy network; The perturbation penalty weight matrix; Indicates the previous moment, i.e. The control input at each moment; the state error term, control energy term, and control increment term all enter the cost with a positive sign. The disturbance term enters the cost with a negative sign, causing the disturbance strategy to seek unfavorable disturbances by increasing the cumulative cost in the future, while the control strategy learns disturbance-resistant and smooth control actions by decreasing the cumulative cost. It is a known quantity in the current policy update.

[0258] Step 4, combined Figure 3 Construct an N-step discounted game that minimizes the control policy and maximizes the perturbation policy, and define the training signals for the three networks. Specifically:

[0259] Step 4-1, Set the discount factor The game objective within the finite prediction time domain is:

[0260]

[0261] In the formula, the control strategy Located in the minimization direction, perturbation strategy Located in the direction of maximization, The single-step cost function for each prediction step.

[0262] pass This minimum-maximum relationship means that during the training phase, the control strategy continuously faces perturbation strategies that can amplify state deviations or control costs, thus avoiding learning control laws only for nominally undisturbed operating conditions.

[0263] Step 4-2, define the infinite time-domain discount value function as:

[0264]

[0265] In the formula, In order to implement a given control strategy and perturbation strategies Under the current error state The cumulative discount cost in the infinite time domain at the start; For the system from the first Starting at each sampling time, in the strategy and Scrolling under action Error status after step; This is used to reduce the weight of future costs in current value. The value function satisfies a first-order Bellman self-consistency relation:

[0266]

[0267] In the formula, The error state at the next sampling moment is obtained after executing the current control input and disturbance input; This represents the current single-step cost; This is the discount term in the next-state value function. The formula indicates that the current value equals the sum of the current single-step cost and the discounted value of the next state.

[0268] To improve the efficiency of information utilization during value learning, this embodiment adopts an N-step approach:

[0269]

[0270] In the formula, For from the first Step to the first Step Step-by-step cumulative discount cost; For scrolling End-point error status after step; for Estimating the discounted value corresponding to the end state of each step. The actual cost of the sampling step and the first sampling step The value estimation of each step is combined to enable the value network to utilize a length of [length missing] in a single update. State-control-perturbation sequence;

[0271] Step 4-3, define the cumulative discount cost over N steps:

[0272]

[0273] An N-step temporal difference error is constructed based on the difference between the two sides of the Bellman self-consistent equation, and the value network employs squared loss. :

[0274]

[0275] In the formula, the value within square brackets represents the N-step TD error. Minimizing this loss allows the value neural network to progressively approximate the true cumulative discount cost under a given control policy and disturbance policy.

[0276] Step 4-4: The perturbation policy network aims to increase the cumulative cost and terminal value over N steps. Its loss function is defined as:

[0277]

[0278] The negative sign in the above expression makes using the gradient descent optimizer equivalent to performing gradient ascent on the value within the parentheses. This perturbation strategy is used to generate unfavorable perturbations during the training phase, rather than limiting the actual engineering perturbation to a fixed upper bound or a fixed waveform.

[0279] Furthermore, an NMPC expert trajectory imitation learning term is introduced into the control policy network loss: this term is not enabled in the initial training phase, is enabled in the intermediate phase when expert trajectories can be stably generated, and the guiding weights are gradually annealed to zero in the later stages of training. Specifically:

[0280] (1) Select the NMPC control input corresponding to the current training state from the expert dataset in step 2, and define the imitation error:

[0281]

[0282] In the formula, For the current time step The control policy network output at the location, In the state corresponding to the expert control trajectory set The output is controlled by experts at the location. This indicates the calculation of Euclidean distance. Also used as an indicator variable, it has no practical meaning. This term directly measures the Euclidean distance between the network control action and the high-precision nonlinear optimization control action.

[0283] (2) The N-step cost of the control strategy and the imitation error are jointly written into the control network loss:

[0284]

[0285] In the formula, For the first The dynamic guidance coefficients for round-based training. In the initial stage of random initialization, the imitation term is not enabled; after entering the intermediate stage where NMPC expert trajectories can be stably generated, the imitation term guides the control strategy to the vicinity of the high-performance control trajectory; in the later stages of training, expert constraints are gradually removed, allowing the control and perturbation strategies to reach the saddle point according to the original minimum-maximum objective. The guidance coefficients satisfy:

[0286]

[0287] In the formula, Enable rounds for the imitation item. This refers to the round at which the guidance ends. The guidance coefficient is zero in the initial stage, takes a continuously decaying positive value in the intermediate stage, and then... The initial value is then set to zero again. This later zeroing is to ensure the algorithm can independently discover the optimal solution. In this invention, the guiding coefficient specifically employs exponential decay.

[0288] Step 5, combined Figure 4 The strategy evaluation, control strategy improvement, and disturbance strategy improvement are completed by alternating dual Actor-Critic iterations.

[0289] Step 5-1: Initialize three network parameters and state samples. When executing the Actor-Critic 1 module, fix the perturbation policy network parameters. The error state is determined by the current control strategy and the disturbance strategy. After rolling N steps, the sequence required for training is obtained; then the value network parameters are updated using the following formula:

[0290]

[0291] In the formula, For the first Value network parameters before the round update; The updated intermediate value network parameters in the Actor-Critic 1 module; The learning rate of the value network; This represents the gradient of the value network loss function with respect to the current value network parameters. For alternating iteration rounds.

[0292] Step 5-2: During the policy enhancement phase of Actor-Critic 1, keep the perturbation policy fixed and update the control policy network using the following formula:

[0293]

[0294] In the formula, and The first Control policy network parameters before and after the update. The gradient of the control policy loss function with respect to the control policy network parameters; The learning rate of the policy network is controlled by the update direction, which is simultaneously affected by the state control cost, the control smoothing cost, and the NMPC imitation error during training.

[0295] Step 5-3: When executing the Actor-Critic 2 module, fix the already updated control policy network, re-evaluate the value network with a similar N-step TD error, and then update the perturbation policy network.

[0296]

[0297]

[0298] In the formula, and The first The perturbation policy network parameters before and after the round update. This represents the gradient of the perturbation policy loss function with respect to the perturbation policy network parameters; This is the learning rate of the perturbation policy network. Because... Since the negative sign is already included, gradient descent in the above formula is actually equivalent to increasing the cumulative cost corresponding to the perturbation strategy, thereby enabling the perturbation strategy to seek more unfavorable external perturbations.

[0299] Step 5-4: Repeat steps 5-1 to 5-3 in the order of "fixing the perturbation policy and updating the value / control network — fixing the control policy and updating the value / perturbation network". Stop offline training when the losses of the three networks remain stable over consecutive training rounds, and changing any single policy alone cannot further improve its corresponding objective. Optimal value satisfies:

[0300]

[0301] In the formula, The optimal value function at the saddle point; The optimal control strategy; The optimal perturbation strategy; and These are candidate control strategies and candidate perturbation strategies, respectively. This means that while the control strategy minimizes the cumulative cost, the perturbation strategy maximizes the cumulative cost. To predict the end-time value discount term

[0302] Corresponding optimal control strategy and optimal perturbation strategy They respectively satisfy:

[0303]

[0304]

[0305] In the formula, For a given control strategy And adopt the optimal perturbation strategy At that time, from the current error state The optimal value obtained from the start. To adopt the optimal control strategy And given a perturbation strategy At that time, from the current error state The optimal value obtained upon departure; and These represent the strategies corresponding to achieving the minimum or maximum value, respectively. This indicates the optimal perturbation strategy. When the value function is fixed, the control strategy that minimizes the value function is the one that minimizes the value function. Indicating the optimal control strategy When fixed, the perturbation strategy that maximizes the value function;

[0306] Step 6: Delete the offline training auxiliary network and deploy the online control policy network.

[0307] Step 6-1: After training, solidify the optimal control strategy network parameters. The value network, perturbation policy network, NMPC optimizer, and expert dataset are removed from the online control program. The online phase retains four stages: state acquisition or state estimation, error calculation, forward propagation of the control policy network, and control limiting.

[0308] Step 6-2, acquire at each sampling time , , and The error state is obtained by subtracting the equilibrium state from the error state and then input into the control strategy network. Finally, magnitude constraints are applied to the network output.

[0309]

[0310] In the formula, This is a channel-by-channel saturation function. When the network output is between the upper and lower limits, it retains its original value; when it exceeds the upper limit, the upper limit is used; when it falls below the lower limit, the lower limit is used. (After limiting) and These are applied to the pitch and buoyancy equivalent control channels, respectively. The state is reacquired at the next sampling time, and the above process is repeated until the pitch and buoyancy responses enter the neighborhood of the equilibrium point.

[0311] Since the online phase only performs one forward propagation of the control policy network, it does not require solving the nonlinear programming problem required by NMPC, nor does it require updating any network parameters. Therefore, its single-step computational cost is mainly determined by the matrix multiplication and addition operations of the fully connected network. This offline training-online inference pruning method preserves the high-precision control information of NMPC in the policy parameters, while avoiding the computational latency caused by online iterative solutions.

[0312] In one embodiment, an active flutter suppression system for wing based on imitation learning and elastic approximate dynamic programming is provided for implementing the active flutter suppression method for wing based on imitation learning and elastic approximate dynamic programming. The system includes:

[0313] The first module is used to: construct a state-space dynamics model of the wing aeroelastic system, and define a cost function and an objective function that include state deviation, control energy, and control smoothness;

[0314] The second module is used to: construct a nonlinear model predictive controller based on the state-space dynamics model, and use the nonlinear model predictive controller to solve offline to generate an expert control trajectory dataset;

[0315] The third module is used to implement: constructing a dual Actor-Critic adversarial neural network framework that includes a value network, a control policy network, and a perturbation policy network;

[0316] The fourth module is used to: train the control policy network with the goal of minimizing the cumulative cost, train the perturbation policy network with the goal of maximizing the cumulative cost, and evaluate the cumulative cost through the value network, wherein an imitation learning regularization term based on the expert control trajectory dataset is introduced during the training process of the control policy network.

[0317] The fifth module is used to alternately optimize and update the value network, control policy network, and perturbation policy network until the network converges.

[0318] The sixth module is used to: remove the value network and the perturbation policy network after training convergence, solidify the control policy network, and use the control policy network to output control commands based on the sampled real-time wing status during the online control phase, which are then applied to the wing control channel to implement active flutter suppression.

[0319] Specific limitations regarding the imitation learning-based elastic approximate dynamic programming-based active flutter suppression system for wings can be found in the limitations described above for the imitation learning-based elastic approximate dynamic programming-based active flutter suppression method for wings, and will not be repeated here. Each module in the aforementioned imitation learning-based elastic approximate dynamic programming-based active flutter suppression system for wings can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independent of the processor in a computer device, or stored in software in the memory of a computer device, so that the processor can call and execute the corresponding operations of each module.

[0320] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements:

[0321] Step 1: Construct a state-space dynamics model of the wing aeroelastic system and define a cost function and an objective function that include state deviation, control energy, and control smoothness.

[0322] Step 2: Construct a nonlinear model predictive controller based on the state-space dynamics model, and use the nonlinear model predictive controller to solve offline to generate an expert control trajectory dataset;

[0323] Step 3: Construct a dual Actor-Critic adversarial neural network framework that includes a value network, a control policy network, and a perturbation policy network;

[0324] Step 4: Train the control policy network with the goal of minimizing the cumulative cost, train the perturbation policy network with the goal of maximizing the cumulative cost, and evaluate the cumulative cost through the value network, wherein an imitation learning regularization term based on the expert control trajectory dataset is introduced during the training of the control policy network.

[0325] Step 5: Alternately optimize and update the value network, control policy network, and perturbation policy network until the network converges;

[0326] Step 6: After training convergence, remove the value network and the perturbation policy network, solidify the control policy network, and use the control policy network to output control commands based on the sampled real-time wing status during the online control phase, which are then applied to the wing control channel to implement active flutter suppression.

[0327] For specific constraints on each step, please refer to the constraints on the active flutter suppression method for elastic approximate dynamic programming based on imitation learning mentioned above, which will not be repeated here.

[0328] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program being implemented when executed by a processor:

[0329] Step 1: Construct a state-space dynamics model of the wing aeroelastic system and define a cost function and an objective function that include state deviation, control energy, and control smoothness.

[0330] Step 2: Construct a nonlinear model predictive controller based on the state-space dynamics model, and use the nonlinear model predictive controller to solve offline to generate an expert control trajectory dataset;

[0331] Step 3: Construct a dual Actor-Critic adversarial neural network framework that includes a value network, a control policy network, and a perturbation policy network;

[0332] Step 4: Train the control policy network with the goal of minimizing the cumulative cost, train the perturbation policy network with the goal of maximizing the cumulative cost, and evaluate the cumulative cost through the value network, wherein an imitation learning regularization term based on the expert control trajectory dataset is introduced during the training of the control policy network.

[0333] Step 5: Alternately optimize and update the value network, control policy network, and perturbation policy network until the network converges;

[0334] Step 6: After training convergence, remove the value network and the perturbation policy network, solidify the control policy network, and use the control policy network to output control commands based on the sampled real-time wing status during the online control phase, which are then applied to the wing control channel to implement active flutter suppression.

[0335] For specific constraints on each step, please refer to the constraints on the active flutter suppression method for elastic approximate dynamic programming based on imitation learning mentioned above, which will not be repeated here.

[0336] To verify the above embodiment, this embodiment uses the same two-degree-of-freedom nonlinear airfoil model as in step 1. The training batch size is set to 64, the optimizer is Adam, and the sampling time is controlled. s, state dimension is 4, control input dimension is 2, N forward time domain steps are taken Discount factor The learning rates for the control policy network, value network, and perturbation policy network are respectively set to... , and All three networks employ two 256-neuron ELU hidden layers; the control and perturbation output layers use Tanh, and the value output layer uses ReLU.

[0337] To cover different oscillation amplitudes, the training states are randomly initialized using the following formula:

[0338]

[0339] In the formula, The mean is variance is The Gaussian distribution.

[0340] In this embodiment, the state weight, control energy weight, and control smoothness weight are taken as follows:

[0341]

[0342] The above values ​​are used for comparison and verification in this embodiment and do not limit the retuning of each weight according to the actuator's capabilities and state adjustment requirements under other operating conditions.

[0343] The total number of training epochs was 3000, and the loss changes of the first three networks were recorded. Combined with... Figure 5 The losses of all three networks eventually approached zero, indicating that the value assessment, control strategy, and perturbation strategy tended to stabilize in this embodiment. Combined with... Figure 6Under the same random seed, same initialization and same number of training rounds, NGRAP’s control network and value network loss decreases faster than RADP without NMPC imitation terms, indicating that NMPC expert guidance can shorten the policy search process and accelerate network training.

[0344] When verifying control performance, both the upper and lower limits of the control input are taken as... and The initial state is taken A Gaussian perturbation with a mean of 0 and a standard deviation of 0.001 was superimposed, and the simulation duration was 5 seconds. The root mean square error of the pitch angle and heave displacement was used to evaluate the state suppression accuracy.

[0345]

[0346] In the formula, This represents the number of sampling points during online inference. and For reference pitch angle and reference heave displacement, both are set to zero in this embodiment; and The smaller the value, the higher the precision of state control. Figure 7 In the comparison shown, RADP's and The values ​​are 0.10550 and 0.34960 respectively, while NGRAP's are 0.08293 and 0.34919 respectively; NGRAP's root mean square error of pitch angle is about 21.4% lower than RADP.

[0347] To evaluate the control energy, the first... Root mean square value of control input for each control channel:

[0348]

[0349] In the formula, Corresponding pitch control channel, This corresponds to the buoyancy control channel. To evaluate the smoothness of changes in adjacent control commands, the root mean square value of the control increment is defined as follows:

[0350]

[0351] In the formula, The smaller the value, the smoother the control changes between adjacent sampling times. Combined with... Figure 8 and Figure 9Under the same initial conditions, external disturbances, and control limiting, the root mean square errors of pitch angle for LMPC, NMPC, and NGRAP are 0.0802, 0.0764, and 0.0829, respectively, and the root mean square errors of heave and buoyancy displacement are 0.4013, 0.3638, and 0.3492, respectively. The root mean square values ​​of the two control inputs for NGRAP are 0.3737 and 0.6795, and the root mean square values ​​of the control increment are 0.0150 and 0.0255, respectively. Its control increment is approximately one-quarter to one-fifth of the corresponding results for LMPC and NMPC, indicating that it reduces high-frequency fluctuations in control commands while maintaining state suppression accuracy.

[0352] To evaluate online real-time performance, the solution time required from the input of the current state to obtaining the control command was recorded at each sampling moment, and the average single-step solution time was calculated:

[0353]

[0354] In the formula, For the first Control calculation time at each sampling moment The average single-step solution time is calculated. The simulation was repeated 10 times on a 2.4 GHz CPU and the average value was taken. Combined with... Figure 10 When predicting the time domain At that time, the single-step solution speed of NGRAP is approximately 3.4 times that of LMPC and approximately 78 times that of NMPC. As the prediction time domain increases, the solution time of LMPC and NMPC increases, while NGRAP does not perform prediction time domain optimization during the online phase, and its forward propagation time remains essentially unchanged. change.

[0355] As can be seen from the above embodiments, the present invention provides offline guidance to the control policy network through NMPC expert control trajectories, improves control adaptability under disturbance conditions through alternating game between control and disturbance policies, and obtains smoother, lower-energy-consumption control commands that can meet real-time computing requirements through control increment penalties and online policy network pruning. The above parameters and comparison results illustrate a specific reproducible implementation of the present invention. Without changing the main technical principles of nonlinear model predictive control expert guidance, dual Actor-Critic adversarial training, and online deployment of only the control policy network, the sampling time, network size, weight matrix, and number of training rounds can be adjusted according to different wing segment parameters, control input boundaries, and onboard computing resources.

[0356] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention without departing from its spirit and scope should be included within the protection scope of the present invention.

Claims

1. A method for active suppression of wing flutter based on imitation learning and elastic approximate dynamic programming, characterized in that, The method includes the following steps: Step 1: Construct a state-space dynamics model of the wing aeroelastic system and define a cost function and an objective function that include state deviation, control energy, and control smoothness. Step 2: Construct a nonlinear model predictive controller based on the state-space dynamics model, and use the nonlinear model predictive controller to solve offline to generate an expert control trajectory dataset; Step 3: Construct a dual Actor-Critic adversarial neural network framework that includes a value network, a control policy network, and a perturbation policy network; Step 4: Train the control policy network with the goal of minimizing the cumulative cost, train the perturbation policy network with the goal of maximizing the cumulative cost, and evaluate the cumulative cost through the value network, wherein an imitation learning regularization term based on the expert control trajectory dataset is introduced during the training of the control policy network. Step 5: Alternately optimize and update the value network, control policy network, and perturbation policy network until the network converges; Step 6: After training convergence, remove the value network and the perturbation policy network, solidify the control policy network, and use the control policy network to output control commands based on the sampled real-time wing status during the online control phase, which are then applied to the wing control channel to implement active flutter suppression.

2. The active flutter suppression method for elastic approximate dynamic programming based on imitation learning for wing flutter according to claim 1, characterized in that, Step 1 involves constructing a state-space dynamics model of the wing's aeroelastic system and defining a cost function and objective function that include state deviation, control energy, and control smoothness. Specifically, this includes: Step 1-1: Construct a state vector using the wing's pitch angle, pitch rate, heave displacement, and heave velocity; Steps 1-2 involve unifying linear structural dynamics, linear aerodynamic coupling relationships, nonlinear stiffness terms, and control inputs into the state equations to construct a state-space dynamic model of the wing's aeroelastic system. In the formula, For state vectors, The pitch angle, The pitch angular velocity, For buoyancy displacement, Floating / sinking speed; superscript Indicates transpose; Let R be the system matrix containing linear structural dynamics and linear aerodynamic coupling terms, and let R denote a real matrix. The nonlinear vector field is caused by pitch cubic stiffness. These are the corresponding dimensionless nonlinear coefficients; To control the input vector, The equivalent control torque acting on the pitch channel, This is the equivalent control force acting on the buoyancy channel; Steps 1-3 define the performance index function to be optimized, which includes state deviation penalty, control energy penalty, and control smoothing increment penalty. : In the formula, For the terminal time, For the cost of operation, For the cost of the terminal, Let be the state vector at time t. These are the control input vectors at the current time t and the previous time, respectively. for The state vector at any given time; The operating cost is defined as follows: In the formula, Let represent the error state at time t. For the target equilibrium state, The state weight matrix is... To control the energy weight matrix, To control the incremental smoothing weight matrix; Terminal cost is defined as: In the formula, This is the terminal state weight matrix, used to improve the convergence accuracy of the predicted state at the end of the time domain; Steps 1-4 thus construct a finite-time control problem that satisfies both dynamic constraints and actuator input constraints, i.e., the objective function is: In the formula, and These are the lower and upper bound vectors controlling the input, respectively. Indicates by adjusting Make The value is the smallest.

3. The active suppression method for wing flutter based on imitation learning and approximate dynamic programming according to claim 2, characterized in that, Step 2 involves constructing a nonlinear model predictive controller based on the state-space dynamics model, and then using the nonlinear model predictive controller to generate an expert control trajectory dataset offline. Specifically, this includes: Step 2-1: Based on the state-space dynamics model, construct a nonlinear model predictive controller, namely the NMPC controller, using the CasaADi and IPOPT solvers. Step 2-2: Discretize the state-space dynamics model to construct a discrete finite-time domain optimization objective function that includes the prediction time domain. : In the formula, This represents the number of forward prediction steps; Number the prediction step; To be at the current sampling time Known error state The first prediction The error state at time t is... Predicted state at time ; for Predictive control input at specific times; To predict control increments; for Predictive control input at specific times; To predict the end state in the time domain; Steps 2-3: Under the condition of satisfying the upper and lower limits of the control input, use the nonlinear model predictive controller to solve the nonlinear programming problem and obtain the optimal control sequence. In the formula, The continuous nonlinear aeroelastic model is based on the sampling period. Discretized state transition mapping; For time steps The optimal predictive control sequence obtained at the specified location; asterisk This represents the optimal solution to the optimization problem; In solving the nonlinear programming problem, a hot-start strategy is adopted, which shifts the optimal control sequence obtained at the previous sampling time forward and uses it as the initial guess for the current optimization; for the end of the sequence, the last term of the previous optimal sequence is copied as the initial guess. Steps 2-4: Repeat steps 2-1 to 2-2 under multiple initial states and disturbance conditions to collect the states and corresponding optimal control inputs, forming an expert control trajectory dataset.

4. The active flutter suppression method for elastic approximate dynamic programming based on imitation learning for airfoils according to claim 1, characterized in that, The dual Actor-Critic adversarial neural network framework described in step 3 specifically includes: The network input error state is constructed by the deviation between the target state and the current state. ; The value network is used to fit the cumulative discount cost under the current error state, and the output layer uses the ReLU activation function to output a non-negative scalar value; The control strategy network is used to generate control commands, and the output layer uses the Tanh activation function to output normalized control and disturbance quantities. The perturbation policy network is used to generate adversarial external perturbations during the training phase, and the output layer uses the Tanh activation function to output normalized control and perturbation quantities. The value network, control strategy network, and disturbance strategy network all adopt a deep neural network structure containing at least two hidden layers and with the activation function ELU. The single-step cost function under the combined action of the control policy network and the disturbance policy network is defined as follows: : in, for Error state at time, The target equilibrium state; The control input is the output of the control strategy network; Adversarial perturbations generated for the perturbation policy network; The perturbation penalty weight matrix; Indicates the previous moment, i.e. Time-based control input; ,here and It has no practical meaning and is only used as an indicator variable; The disturbance term generated by the disturbance policy network is introduced into the single-step cost function in a negative form, so that the disturbance policy network finds the most unfavorable disturbance by increasing the cumulative cost, and the control policy network learns disturbance rejection and smooth control actions by decreasing the cumulative cost.

5. The active suppression method for wing flutter based on imitation learning and approximate dynamic programming according to claim 4, characterized in that, In step 4, an imitation learning regularization term based on the expert control trajectory dataset is introduced during the training process of the control policy network. Specifically, this involves introducing an imitation learning regularization term based on the expert control trajectory dataset into the loss function of the control policy network. Furthermore, the dynamic guidance coefficient of the imitation learning regularization term employs a dynamic decay mechanism during training, first setting it to zero, then enabling it, and gradually decaying and annealing it to zero. This includes: Define imitation learning items : In the formula, For the current moment The control policy network output at the location, For the corresponding state in the expert control trajectory dataset, The output is controlled by experts at the location. Indicates the calculation of Euclidean distance; Construct the total loss function of the control policy network. : In the formula, For the k-th step to the k-th step The cumulative cost of the discounts on each step for Error state at time, Discount factor and This is used to reduce the weight of long-term costs in the current loss function; Let be the dynamic guidance coefficient for the j-th training round; This represents the number of forward prediction steps; Indicates the value network to the first Step end error state The cumulative cost estimate; In the initial training phase, the dynamic guidance coefficient is set to 0 to disable the imitation learning term. In the middle training phase, the imitation learning term is enabled, and the dynamic guidance coefficient gradually decays with each training round. In the later training phase, the dynamic guidance coefficient is set to zero again to ensure that the final minimax game objective remains unchanged, as shown below: In the formula, To enable rounds for imitation learning items, To guide the end of the round, For the first +1 round of training dynamic guidance coefficient.

6. The active flutter suppression method for elastic approximate dynamic programming based on imitation learning for wing flutter according to claim 5, characterized in that, The alternating optimization update described in step 5 specifically includes: Construct the Actor-Critic 1 module: fix the network parameters of the perturbation strategy. The value network parameters are updated based on the N-step temporal difference error. Subsequently, based on the control strategy network total loss function Update the control strategy network parameters ; Construct the Actor-Critic 2 module: Fix the updated network parameters of the control policy. The value network parameters are updated again based on the same N-step temporal difference error. Subsequently, based on the perturbation loss function Update the network parameters of the perturbation strategy ; The Actor-Critic 1 and Actor-Critic 2 modules are executed alternately until the losses of the value network, control policy network, and perturbation policy network reach the convergence criterion and form a game saddle point.

7. The active flutter suppression method for elastic approximate dynamic programming based on imitation learning for wing flutter according to claim 6, characterized in that, The convergence of the alternating optimization update is guaranteed by the compressibility mapping property of the N-step Bellman operator. When the optimal perturbation strategy is fixed, the control strategy minimizes the value function; when the optimal control strategy is fixed, the perturbation strategy maximizes the value function, thus converging to the optimal saddle point value. In the formula, The optimal value function at the saddle point; The optimal control strategy; The optimal perturbation strategy; and These are candidate control strategies and candidate perturbation strategies, respectively. This means that while the control strategy minimizes the cumulative cost, the perturbation strategy maximizes the cumulative cost. To predict the end-time value discount term; for Error status at any given time; This indicates the optimal perturbation strategy. When the value function is fixed, the control strategy that minimizes the value function is the one that minimizes the value function. Indicating the optimal control strategy When fixed, the perturbation strategy that maximizes the value function; and This represents the strategy itself that achieves the minimum or maximum value. It has no practical meaning and is only used as an indicator variable.

8. The active flutter suppression method for elastic approximate dynamic programming based on imitation learning for wings according to claim 7, characterized in that, In step 6, during the online control phase, the control strategy network is used to output control commands based on the sampled real-time wing status, specifically including: Step 6-1: Remove the value network, the disturbance policy network, and the solution algorithm of the nonlinear model predictive controller from the online control program, and retain only the control policy network with fixed parameters; Step 6-2: During the real-time control sampling period, the wing state is collected in real time and an error vector is constructed as the input to the control strategy network; Step 6-3: The control strategy network calculates and outputs the original control signal through a single forward propagation. Step 6-4: Perform physical amplitude saturation limiting processing on the original control signal: In the formula, It is a channel-by-channel saturation function; These are the network parameters for the optimal control strategy that are fixed after training. This is the control input at time k; Step 6-5: Apply the limited control commands to the pitch control channel and buoyancy control channel of the wing respectively.

9. A wing flutter active suppression system based on imitation learning and elastic approximate dynamic programming, used to implement the method as described in any one of claims 1 to 8, characterized in that, The system includes: The first module is used to: construct a state-space dynamics model of the wing aeroelastic system, and define a cost function and an objective function that include state deviation, control energy, and control smoothness; The second module is used to: construct a nonlinear model predictive controller based on the state-space dynamics model, and use the nonlinear model predictive controller to solve offline to generate an expert control trajectory dataset; The third module is used to implement: constructing a dual Actor-Critic adversarial neural network framework that includes a value network, a control policy network, and a perturbation policy network; The fourth module is used to: train the control policy network with the goal of minimizing the cumulative cost, train the perturbation policy network with the goal of maximizing the cumulative cost, and evaluate the cumulative cost through the value network, wherein an imitation learning regularization term based on the expert control trajectory dataset is introduced during the training process of the control policy network. The fifth module is used to: alternately optimize and update the value network, control policy network, and perturbation policy network until the network converges; The sixth module is used to: remove the value network and the perturbation policy network after training convergence, solidify the control policy network, and use the control policy network to output control commands based on the sampled real-time wing status during the online control phase, which are then applied to the wing control channel to implement active flutter suppression.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the active suppression method for wing flutter based on imitation learning, as described in any one of claims 1 to 8.

Images