Adaptive dynamic programming control method for flexible flapping wing system under time-varying constraints

Abstract

The application discloses a kind of self-adapting dynamic programming control methods of flexible flapping wing system under time-varying constraint, it is related to flapping wing intelligent control technical field.Method includes: based on defined virtual variable, define error variable;Based on defined time-varying constraint, design barrier lyapunov function;Based on error variable and barrier lyapunov function, design new lyapunov function;Based on new lyapunov function, construct execution neural network;Radial basis function neural network approximation and estimate long-term cost function are used;Based on long-term cost function, design evaluation neural network;Define external reinforcement signal and internal enhancement signal, design reference neural network;Based on the above three neural networks, construct flapping wing control system based on self-adapting dynamic programming;Flapping wing unmanned aerial vehicle can be stably tracked to desired trajectory in complex scene based on flapping wing control system of self-adapting dynamic programming.The tracking effect to desired trajectory can be improved by using the application.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of flapping intelligent control, in particular to a self-adaptive dynamic programming control method and device for a flexible flapping system under time-varying constraints. BACKGROUND

[0002] As a bionic flexible flapping unmanned aerial vehicle, the flapping unmanned aerial vehicle imitates the flight mode of birds and insects in nature. In recent years, the related technology has attracted widespread attention. Considerable progress has been made in this research field. By utilizing the periodic flapping motion of the flexible wing, the flapping unmanned aerial vehicle can maintain a stable flight attitude and perform gliding and hovering maneuvering actions. Compared with traditional rotary-wing unmanned aerial vehicles and fixed-wing unmanned aerial vehicles, the bionic flapping unmanned aerial vehicle has several unique advantages. Low noise, low energy consumption, lightweight structure and high maneuverability are the most notable features. These characteristics enable the flapping unmanned aerial vehicle to fly for a long time with extremely low energy consumption. The flapping unmanned aerial vehicle is particularly suitable for tasks that require flexible navigation in narrow spaces or operation under energy-limited conditions. However, the working environment of the flapping unmanned aerial vehicle in real life is often very complex, such as unknown disturbances from the external environment or people and environmental constraints, and it is an extremely arduous challenge for the flapping unmanned aerial vehicle to achieve anti-interference and strictly follow the limited conditions in these scenarios.

[0003] As a key research topic of the control theory of flexible flapping wing systems, reinforcement learning control is a typical control method, but there are many problems, such as limited action space, low sample efficiency, slow and unstable convergence, and difficulty in handling continuous control. The existing research proposes an Actor-Critic control architecture method on the basis of the original. The method realizes the organic combination of policy optimization and value evaluation by introducing an actor neural network based on policy and a critic neural network based on value function. Compared with single policy or single value method, the Actor-Critic architecture can balance policy stability and policy improvement speed in theory, and is more suitable for handling high-dimensional continuous control tasks, so it is widely used in the control problems of flexible flapping wing, mechanical arm and unmanned aerial vehicle complex systems. However, the Actor-Critic control method still has several key problems in practical application. The current research often relies too much on the accuracy of the evaluation neural network, but in complex nonlinear systems, the evaluation neural network is prone to inaccurate evaluation due to estimation error, sampling bias or insufficient training. Since the policy update direction of the execution neural network depends on the gradient feedback of the evaluation neural network, once the evaluation neural network learns incorrectly, it will directly mislead the policy evolution direction of the execution neural network, causing system performance degradation or even divergence. In addition, since the execution neural network and the evaluation neural network usually use an alternating update mechanism, their training process is prone to mutual interference. That is, the execution neural network has updated the policy according to the current value function before the evaluation neural network has fully converged, and vice versa. Such cross-influence may cause oscillation, instability or local optimal solution in the training process of the two. And it also depends on human prior knowledge, which often makes it difficult to adapt quickly in the absence of such knowledge, affecting the learning efficiency and control accuracy of the entire system. SUMMARY

[0004] In order to solve the technical problems in the prior art that the flapping wing unmanned aerial vehicle system control method highly depends on external environmental information, has limited generalization ability and cannot meet the time-varying constraints of specific outputs, an embodiment of the present application provides a self-adaptive dynamic programming control method and device for a flexible flapping wing system under time-varying constraints. The technical solution is as follows:

[0005] On the one hand, a self-adaptive dynamic programming control method for a flexible flapping wing system under time-varying constraints is provided, which is realized by a self-adaptive dynamic programming control device for a flexible flapping wing system under time-varying constraints. The method comprises:

[0006] S1, define time-varying constraints and virtual variables; define error variables based on virtual variables; design a barrier Lyapunov function based on time-varying constraints; design a new Lyapunov function based on error variables and barrier Lyapunov function; construct an execution neural network based on the new Lyapunov function;

[0007] S2, approximating and estimating the long-term cost function by using a radial basis function neural network; and designing an evaluation neural network based on the long-term cost function;

[0008] S3, defining an external reinforcement signal and an internal reinforcement signal; and designing a reference neural network based on the external reinforcement signal and the internal reinforcement signal;

[0009] S4, constructing an adaptive dynamic programming based flapping control system based on the execution neural network, the evaluation neural network and the reference neural network; and the adaptive dynamic programming based flapping control system enables the flapping wing unmanned aerial vehicle to stably track the desired trajectory in a complex scene.

[0010] In another aspect, an adaptive dynamic programming control device for a flexible flapping wing system under time-varying constraints is provided, the device being applied to an adaptive dynamic programming control method for the flexible flapping wing system under time-varying constraints, and the device comprising:

[0011] A first construction unit is configured to define time-varying constraints and virtual variables; define error variables based on the virtual variables; design barrier Lyapunov functions based on the time-varying constraints; design new Lyapunov functions based on the error variables and the barrier Lyapunov functions; and construct an execution neural network based on the new Lyapunov functions;

[0012] A first design unit is configured to approximate and estimate a long-term cost function by using a radial basis function neural network; and design an evaluation neural network based on the long-term cost function;

[0013] A second design unit is configured to define an external reinforcement signal and an internal reinforcement signal; and design a reference neural network based on the external reinforcement signal and the internal reinforcement signal;

[0014] A second construction unit is configured to construct an adaptive dynamic programming based flapping control system based on the execution neural network, the evaluation neural network and the reference neural network; and the adaptive dynamic programming based flapping control system enables the flapping wing unmanned aerial vehicle to stably track the desired trajectory in a complex scene.

[0015] In another aspect, an adaptive dynamic programming control device for a flexible flapping wing system under time-varying constraints is provided, the device comprising: a processor; and a memory having computer readable instructions stored thereon, the computer readable instructions, when executed by the processor, implementing any one of the adaptive dynamic programming control methods for the flexible flapping wing system under time-varying constraints.

[0016] In another aspect, a computer readable storage medium is provided, the storage medium having stored therein at least one instruction, the at least one instruction being loadable by a processor and executable to implement any one of the adaptive dynamic programming control methods for a flexible flapping wing system under time-varying constraints.

[0017] The technical scheme provided by the embodiments of the present application has at least the following beneficial effects:

[0018] The embodiments of the present application take into account the influence of unknown interference from the external environment or people and system uncertainty in the working environment of the flapping wing unmanned aerial vehicle, introduce a reference neural network with nonlinear approximation capability in the control law, and perform real-time learning and online compensation on the external interference and system uncertainty of the flapping wing system, thereby significantly increasing the anti-interference ability and vibration suppression ability of the system.

[0019] The embodiments of the present application redesign the execution neural network through the barrier Lyapunov function, and achieve the effect that the control output strictly follows the time-varying constraint. In this way, not only the tracking effect of the expected trajectory is improved, but also the performance of the flapping wing system under the time-varying constraint is further enhanced.

[0020] Based on the traditional Actor-Critic control method, the embodiments of the present application introduce an internal reinforcement signal, connect the reference neural network and the evaluation neural network together, and ensure that the two can exchange information more closely, thereby reducing the dependence on external environmental information and improving the adaptive ability of the controller.

[0021] The embodiment of the present application designs an adaptive control based on a double evaluation neural network for time-varying constraints. A reference neural network is added to the traditional Actor-Critic control method. The reference neural network generates an internal reinforcement signal, which is continuous and dynamically adjustable, ranging from [-1, 1], and is used as the input of the evaluation neural network. The internal reinforcement signal can be adaptively modified with the evolution of the system state. The evaluation neural network and the reference neural network are integrated through the internal reinforcement signal to form a double evaluation architecture. The evaluation neural network can evaluate the current system performance by comparing it with the reference signal, thereby identifying the deviation between the actual result and the expected target. This feedback informs the system of its efficiency in achieving its target. In addition, considering that flexible flight systems often face non-constant output constraints in actual operation, the present application focuses on the more complex and realistic control requirement of time-varying constraints, further improving the constraint processing capability of the controller. To this end, a time-varying barrier Lyapunov function mechanism is introduced into the execution neural network in the Actor-Critic control architecture, and the control law is redesigned. This mechanism can apply nonlinear reinforcement constraints when the control quantity approaches the boundary, ensuring that the system state is strictly maintained within the predefined time-varying constraint range. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0023] Figure 1 is a flow chart of an adaptive dynamic programming control method for a flexible flapping wing system under time-varying constraints provided by the embodiment of the present application;

[0024] Figure 2 is a relationship diagram of the output and constraints of a flapping wing system provided by the embodiment of the present application;

[0025] Figure 3 is a block diagram of a flapping wing control system based on adaptive dynamic programming provided by the embodiment of the present application;

[0026] Figure 4 is a block diagram of an adaptive dynamic programming control device for a flexible flapping wing system under time-varying constraints provided by the embodiment of the present application;

[0027] Figure 5 is a structural schematic diagram of an adaptive dynamic programming control device for a flexible flapping wing system under time-varying constraints provided by the embodiment of the present application. DETAILED DESCRIPTION

[0028] The technical solutions in the present application will be described below with reference to the drawings.

[0029] In the embodiments of the present application, the words such as "example", "for example" are used to represent an example, illustration or description. Any embodiment or design scheme described as "example" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. In fact, the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present application, the meaning expressed by "and / or" can be both, or can be one of the two.

[0030] In the embodiments of the present application, "image" and "picture" can be used interchangeably at times, and it should be pointed out that the meanings expressed are consistent when the distinction is not emphasized. "Of", "corresponding" and "corresponding" can be used interchangeably at times, and it should be pointed out that the meanings expressed are consistent when the distinction is not emphasized.

[0031] In the embodiments of the present application, sometimes the subscript such as W1 can be written in the form of non-subscript such as W1, and the meanings expressed are consistent when the distinction is not emphasized.

[0032] In order to make the technical problems, technical solutions and advantages of the present application more clear, the following will be described in detail with reference to the drawings and specific embodiments.

[0033] The embodiment of the present application provides a self-adaptive dynamic programming control method of a flexible flapping wing system under time-varying constraints, which can be realized by a self-adaptive dynamic programming control device of the flexible flapping wing system under time-varying constraints. The self-adaptive dynamic programming control device of the flexible flapping wing system under time-varying constraints can be a terminal or a server. As shown in the flow chart of the self-adaptive dynamic programming control method of the flexible flapping wing system under time-varying constraints, the processing flow of the method can include the following steps: Figure 1

[0034] S1, define time-varying constraints and virtual variables; define error variables based on the virtual variables; design a barrier Lyapunov function based on the time-varying constraints; design a new Lyapunov function based on the error variables and the barrier Lyapunov function; and construct an execution neural network based on the new Lyapunov function.

[0035] Wherein, the long-term cost function is represented by the following formula (1):

[0036]

[0037] Wherein, K represents a discount factor for weighing future costs; the tracking error is defined as ​γ represents a state vector, γ d is represented as a desired trajectory; wherein υ(n) represents an instantaneous cost function, represented by the following formula (2):

[0038]

[0039] wherein R represents a first positive definite matrix; D represents a second positive definite matrix; F represents a control input of the flapping wing system.

[0040] wherein, as Figure 2 shown in the figure is a relationship between the output and the constraint of the flapping wing system provided by the embodiment of the application; wherein the dashed line represents the constraint, and the solid line represents the system output; wherein, in order to ensure that the output of the flapping wing system can accurately follow the desired trajectory and strictly follow the time-varying constraint, the embodiment of the application utilizes the barrier Lyapunov function to design this part, wherein the error variable is defined as represents the error variable; η represents a virtual variable; represents the derivative of the state variable.

[0041] wherein, since the constant constraint can be regarded as a special time-varying constraint, the more representative time-varying constraint q c (t) is adopted; therefore, the time-varying boundary of the state error is set as q a (t) = γ d (t) - q c (t), wherein γ d (t) represents the desired trajectory; represents the lower bound of the output constraint; q b (t) represents the upper bound of the tracking error.

[0042] Optionally, the barrier Lyapunov function is represented by the following formula (3):

[0043]

[0044] wherein g(i) represents a switching function that adjusts the barrier Lyapunov function V1 according to the tracking error . represents the lower bound of the tracking error . represents the upper bound of the tracking error.

[0045] wherein, when approaching the upper bound of the constraint g(i) = 1 when approaching the lower bound of the constraint g(i) = 0, the transformation of the error coordinate is defined as λ i = g(i) λbi +[1-g(i)]λ ai , where λ ai Indicates the current error The relative proportion between the upper and lower boundaries; λ bi Indicates the current error The relative proportion between the upper boundary and the lower boundary; both are variables used for coordinate transformation, and V1 can be rewritten as the following formula (4):

[0046]

[0047] Where, λ i The sign of the error is represented by λ. ai and λ bi The merged vector between them is used to uniformly describe the constraints; when λ i When V<1, V1 is positive definite, and further differentiation can be performed, which can be expressed by the following formula (5):

[0048]

[0049] The dummy variable η is represented by the following formula (6):

[0050]

[0051] in, This represents a dynamic control gain that automatically adapts to the rate of change of time-varying boundaries. It ensures that system errors can be rapidly adjusted during boundary changes, thereby always satisfying time-varying output constraints and guaranteeing system stability. To ensure that when... When it is zero, Bounded, Defined as a small positive constant, K1 = diag[k 11 ,k 12 ,…,k 1n+1 Since ] is a gain matrix, further derivation of formula (5) yields the following formulas (6)-(7):

[0052]

[0053] In this embodiment of the invention, in order to process Coupling terms in Design a new Lyapunov function.

[0054] Alternatively, the new Lyapunov function can be expressed by the following formula (8):

[0055]

[0056] Where V2 represents the new Lyapunov function; express transpose; This represents the error variable defined when designing and executing the neural network.

[0057] Optionally, S1 constructs an execution neural network based on a new Lyapunov function, including:

[0058] Differentiating the new Lyapunov function yields the control inputs of the flapping-wing system; the system's control inputs include dynamic deterministic states and external disturbances.

[0059] The control input of the flapping wing system is expressed by the following formula (9):

[0060]

[0061] Where F represents the control input of the flapping wing system; Indicates tracking error; K1 represents the error variable defined when designing and executing the neural network; K2 represents the gain matrix; f dis Indicates external interference; denoted by η; denoted by S(γ); ​​denoted by M(γ); denoted by M(γ); Indicates the derivative of a dummy variable;

[0062] Based on the control input of the flapping wing system, a radial basis function neural network is adopted. By defining the first estimation error, the update rule of the execution neural network is designed, and the execution neural network is constructed based on the update rule of the execution neural network.

[0063] Among them, the dynamic deterministic state and external disturbances are approximated by a radial basis function neural network, which is expressed by the following formula (10):

[0064]

[0065] Where Φ=Φ * +Φ, where Φ represents the estimated weight, Φ represents the error of the weight, and Φ * S represents the ideal network weights; a (W a () represents the vector of basis functions in the neural network;

[0066] Among them, for continuous functions The specific representation of the radial basis function neural network is expressed by the following formula (11):

[0067] g(W)=Φ T S(W),S(W)={s1(W),s1(W),…}(11)

[0068] Among them, s i(W) is defined based on a Gaussian kernel function, and is expressed by the following equation (12):

[0069]

[0070] where μ i = [μ i1 , μ i2 , …, μ ik ] represents a center point, η i is a parameter, W represents an input vector of the radial basis function neural network, and W defines an arbitrary continuous function on a compact set , which can be expressed by the radial basis function neural network, and is expressed by the following equation (13):

[0071] g(W) = Φ *T S(W) + ε (13)

[0072] where ε is an approximation error.

[0073] where, since the true model is unknown, the accurate value of in the control input is unknown, and the system uncertainty and actuator fault are estimated online by the following equation (14):

[0074]

[0075] where W represents an input vector of the execution neural network; and ε a represents an approximation error of the execution neural network when approximating the unknown nonlinear function. Based on W a , the estimation error is defined. a Based on the estimation error, the total error term e of the execution neural network is defined, and is expressed by the following equation (15):

[0076] e a = τ a + Y(J(t) - J d (t)) (15)

[0077] where Y > 0 is a constant, J d (t) is the expected value function, and E a is a core objective function defined in the learning of the execution neural network, which is a numerical index for measuring the goodness of the strategy, and is expressed by the following equation (16):

[0078]

[0079] Alternatively, the update rule of the execution neural network is expressed by the following equation (17):

[0080]

[0081] where, ξ a > 0 represents the learning rate used by the neural network; τ a represents the defined estimation error; Y represents a constant greater than zero; represents the estimated value of the cost function; S a represents the basis function vector in the execution neural network; W a represents the input vector of the execution neural network; represents the estimated execution neural network weight.

[0082] S2, using a radial basis function neural network to approximate and estimate the long-term cost function; based on the long-term cost function, designing an evaluation neural network.

[0083] Optionally, S2, using a radial basis function neural network to approximate and estimate the long-term cost function; based on the long-term cost function, designing an evaluation neural network, comprising:

[0084] Using a radial basis function neural network to approximate and estimate the long-term cost function, deriving the long-term cost function by introducing an internal reinforcement signal, defining a second estimation error, designing an update law of the evaluation neural network, and constructing the evaluation neural network based on the update law of the evaluation neural network.

[0085] In a feasible implementation, the long-term cost function is represented by the following formula (18):

[0086]

[0087] where, represents the input vector of the evaluation neural network, and ε c represents the approximation error of the evaluation neural network when approximating the unknown nonlinear function; represents the estimated value of the cost function; R(t) is an internal reinforcement signal, and deriving the long-term cost function can obtain the following formula (19):

[0088]

[0089] where, K represents a discount factor, used to define the degree of future cost decay in the long-term performance index, and when K→∞, transforming formula (19) can obtain the following formula (20):

[0090]

[0091] where, define E crepresents the training target of the evaluation neural network, minimizing Ec can make the evaluation neural network more accurately approximate the target function J(t), thereby more effectively guiding the evaluation neural network to optimize the control strategy, based on the training target E c , the update law of the evaluation neural network is designed.

[0092] wherein the initial update law of the evaluation neural network is represented by the following formula (21):

[0093]

[0094] In a feasible implementation, e c is substituted into formula (21) for transformation, and the final update law of the evaluation neural network is obtained.

[0095] wherein the update law of the evaluation neural network is represented by the following formula (22):

[0096]

[0097] wherein ξ c > 0 represents the learning rate of the evaluation neural network, represents a combination term representing the gradient direction in the weight update of the evaluation neural network; K represents a discount factor; S c represents the basis function output vector of the evaluation neural network; represents the derivative of the evaluation neural network with respect to time; υ(t) represents the instantaneous cost function; represents the estimated evaluation neural network weight; E c represents the predefined first error; represents the transpose of the estimated evaluation neural network weight.

[0098] S3, define external reinforcement signals and internal reinforcement signals; based on the external reinforcement signals and the internal reinforcement signals, design a reference neural network.

[0099] wherein the embodiments of the present application adopt two types of different reinforcement signals: one type is an external reinforcement signal, which is derived from environmental feedback outside the system; the other type is an internal reinforcement signal, which is generated by a reference module to guide the system behavior. Among them, the external reinforcement signal r(t) is based on the state error defined as: when r(t) = -1, and when r(t) = 0. Wherein ||·|| represents the Euclidean 2-norm, and c is a constant threshold. When the signal value is 0, it means that the tracking performance meets the requirements, and when the value is -1, it means that the performance does not meet the requirements.

[0100] where the internal reinforcement signal R(t) is generated by the reference neural network, whose value range is [-1, 1] and changes adaptively with the system state: in the feedforward stage, the signal serves as an input of the critic neural network to assist the evaluation of the control policy; in the feedback stage, the critic neural network updates its parameters based on the prediction error of the internal reinforcement signal. This signal, as a key intermediary between the reference neural network and the critic neural network, realizes the collaborative integration and adaptive learning of the control framework.

[0101] Optionally, the external reinforcement signal and the internal reinforcement signal are defined in S3; based on the external reinforcement signal and the internal reinforcement signal, a reference neural network is designed, comprising:

[0102] The external reinforcement signal and the internal reinforcement signal are defined; based on the external reinforcement signal and the internal reinforcement signal and a defined error, an update rule of the reference neural network is designed, and the reference neural network is constructed based on the update rule of the reference neural network;

[0103] wherein the internal reinforcement signal can be represented as wherein ε r represents an approximation error, which is the residual error when the reference neural network approximates the target function, and is an inevitable small error; represents the transpose of the weight vector of the ideal reference neural network.

[0104] wherein the internal reinforcement signal is defined as represents the input vector of the reference neural network; wherein the defined error is represented by the following formulas (23)-(24):

[0105] e r = Ω (J (t) - J d (t) ) + r (t) (23)

[0106]

[0107] wherein e r represents the error function of the reference neural network; r(t) represents the external reinforcement signal; E r represents a predefined second error; Ω represents a normal number; J d (t) represents the expected cost function value, which is the target value that the critic neural network hopes to approximate; represents the estimated value of the cost function.

[0108] wherein the update rule of the reference neural network is represented by the following formula (25):

[0109]

[0110] wherein represents the estimated reference neural network weight; E rrepresents a predefined second error; ξ r represents a learning rate of the reference neural network; e r represents an error function of the reference neural network; J(t) represents a cost function; R(t) represents an internal reinforcement signal; e r (t) represents an error function of the reference neural network; S r represents a vector of basis function outputs of the reference neural network; W r represents an input vector of the reference neural network.

[0111] In the embodiment, the stability of the evaluation network is verified by defining a Lyapunov function.

[0112] In an embodiment, the specific verification process includes:

[0113] (1) defining a Lyapunov function, wherein the Lyapunov function is represented by the following formula (26):

[0114]

[0115] wherein, represents the transpose of ; and represents an evaluation neural network weight estimation error, which is a key variable for measuring the gap between the current learning effect of the evaluation neural network and an ideal target.

[0116] (2) deriving the formula (27) of the Lyapunov function after derivation by deriving the above formula (26):

[0117]

[0118] a basis function output vector of the network; ξ c represents a learning rate of the evaluation neural network, when e c → 0, the instantaneous cost function is represented as κ is a small constant; based on the formula (27) and the instantaneous cost function, the instantaneous cost function can be transformed to obtain the following formula (28):

[0119]

[0120] wherein, and ||δ c ||≤ε c,max , δ c represents a total approximation error term generated by the evaluation neural network when estimating the cost function, including a function approximation error and an error term caused by gradient propagation; ε c,max represents δ cupper bound of J(t) for theoretical stability analysis; the derivation of Lyapunov function is further represented by formula (29):

[0121]

[0122] (3) the inequality further obtain the inequality about the derivative of Lyapunov function, which is represented by formula (30) as follows:

[0123]

[0124] In a feasible implementation, the embodiment of the present application designs the Lyapunov function to verify the stability of the reference neural network, and the specific implementation process includes:

[0125] (1) define the Lyapunov function, which is represented by formula (31) as follows:

[0126]

[0127] wherein, represents the weight estimation error of the reference neural network, which is a key variable to measure the gap between the current learning effect of the reference neural network and the ideal target; represents the transpose of .

[0128] (2) based on transform formula (31), the weight update law of the reference neural network can be obtained, which is represented by formula (32) as follows:

[0129]

[0130] (4) based on formula (32), the Lyapunov function is derived, and the derivation process is represented by formula (33) as follows:

[0131]

[0132] wherein, ξ r represents the learning law of the reference neural network; S r represents the basis function output vector of the reference neural network; E r represents the second error defined in advance, which can measure the deviation of J(t) from the target.

[0133] In a feasible implementation, the embodiment of the present application designs the Lyapunov function to verify the stability of the system as a whole, and the specific implementation process includes:

[0134] (1) define the Lyapunov function, which is represented by formula (34) as follows:

[0135]

[0136] wherein, denotes the transpose of ; denotes the evaluation neural network estimation error, which is a key variable used to measure the gap between the current learning effect of the evaluation neural network and the ideal target; denotes the transpose of ; denotes the execution neural network estimation error, which is a key variable used to measure the gap between the current learning effect of the execution neural network and the ideal target.

[0137] (2) Derivation of the Lyapunov function, wherein the derivation process is represented by the following formula (35):

[0138]

[0139] (3) wherein, wherein τ a denotes the defined estimation error, and wherein ε a denotes the approximation error of the execution neural network in approximating the unknown nonlinear function; therefore, formula (35) can be further represented by the following formula (36):

[0140]

[0141] (4) Transform formula (36) according to the inequality (37) related to the estimation value of the following cost function to obtain formula (38):

[0142]

[0143] wherein I is the unit matrix; wherein a is a positive number, representing the lower limit of the rate of Lyapunov function decline, which is related to the control gain and the minimum eigenvalue of the matrix, represented by the following formula (38), and b is a positive number, representing the maximum upper bound of the total disturbance or error term in the system, which is derived from the approximation error, represented by the following formula (39):

[0144]

[0145] wherein α s denotes the lower bound constraint of S a ; α r denotes the lower bound constraint of S r ; denotes the lower bound constraint of S ιlower bound constraint is set to ensure that the system learning process is always effective, avoiding network stopping updating or updating slowly, leading to convergence failure, and the lower bound can extract the explicit minimum convergence rate a in Lyapunov analysis, thereby providing strict mathematical support for the stability conclusion of the flapping wing control system based on adaptive dynamic programming.

[0146]

[0147] wherein ε a represents the approximation error when the neural network is executed to approximate the unknown nonlinear function.

[0148] min(2k 1i )>0,2λ min (K2-I)>0, α s ≤||S a ||,α r ≤||S r ||,α ι ≤||ι||

[0149] S4, based on the execution neural network, the evaluation neural network and the reference neural network, a flapping wing control system based on adaptive dynamic programming is constructed; the flapping wing control system based on adaptive dynamic programming enables the flapping wing unmanned aerial vehicle to stably track the expected trajectory in a complex scene.

[0150] In a feasible implementation, for the dynamic characteristics of the biomimetic flexible flapping wing unmanned aerial vehicle system, under the control law and the bounded initial condition, the system realizes semi-global uniform ultimate bounded stability. The system output can asymptotically track the expected trajectory, that is, when t→∞, γ(t)→γ d (t). The time-varying output constraint is not violated throughout the process, that is Error variables Φ c , Φ a and Φ r are uniformly bounded, and each variable is maintained in a compact set, denoted as and respectively. Both represent the change of system error between adjacent sampling time points, which are used to analyze and design the stability conditions of the system in discrete implementation. The above set is defined as follows:

[0151]

[0152] wherein, represents the maximum allowed value of the error variable in the lower bound direction; represents the maximum allowed value of the error variable the maximum allowed value in the upper bound direction;

[0153]

[0154] where D = 2(V(0) + b / a), a and b are two constants and The following formula (45) is proved:

[0155]

[0156] Integrating the formula (45), the following formula (46) is obtained:

[0157]

[0158] Based on the formula (46), according to and The relationship of the above three Lyapunov functions can be obtained by the above three Lyapunov functions, which is expressed by the following formula (47):

[0159] That is, the following formula (48) is true:

[0160]

[0161] From the formula (48), it can be further obtained that And since g(i) = 1 when , g(i) = 0 when , It can be further known that the following formula (49) is true:

[0162]

[0163] From , it can be obtained that It can be further known that the following formula (50) is true:

[0164]

[0165] From , it can be obtained that Therefore, the following formulas (51)-(53) are true:

[0166] ||Φ c || 2 ≤ 2(V(0) + b / a) (51)

[0167] ||Φ a || 2 ≤ 2(V(0) + b / a) (52)

[0168] ||Φ r || 2 ≤2(V(0)+b / a) (53)

[0169] Wherein, the closed loop system has semi-global uniform ultimate boundedness;Wherein output error Can ensure convergence to compact set

[0170] Wherein, as Figure 3 The figure shows a kind of flapping wing control system based on adaptive dynamic programming provided by the embodiment of the application;Wherein, the flapping wing control system based on adaptive dynamic programming includes: execution neural network, evaluation neural network, reference neural network and flapping wing system.

[0171] The embodiment of the application considers that flapping wing unmanned aerial vehicle working environment can be influenced by unknown disturbance from external environment or person and system uncertainty, introduces reference neural network with nonlinear approximation ability in control law, carries out real-time learning and online compensation to external disturbance and system uncertainty of flapping wing system, significantly increases anti-interference ability and vibration suppression ability of system.

[0172] The embodiment of the application redesigns execution neural network by barrier lyapunov function, realizes the effect that control output strictly follows time-varying constraint. By this way, not only the tracking effect to expected trajectory is improved, but also the performance of flapping wing system under time-varying constraint is further enhanced.

[0173] The embodiment of the application introduces internal reinforcement signal based on traditional Actor-Critic control method, connects reference neural network and evaluation neural network together, guarantees that the two can carry out closer information exchange, thereby reduces the degree of dependence on external environment information, improves the adaptive ability of controller.

[0174] The embodiment of the application designs an adaptive control based on a double evaluation neural network for time-varying constraints. A reference neural network is added to the traditional Actor-Critic control method. The reference neural network generates an internal reinforcement signal, which is continuous and dynamically adjustable, ranging from [-1, 1], and is used as the input of the evaluation neural network. The internal reinforcement signal can be adaptively modified with the evolution of the system state. The evaluation neural network and the reference neural network are integrated through the internal reinforcement signal to form a double evaluation architecture. The evaluation neural network can evaluate the current system performance by comparing it with the reference signal, thereby identifying the deviation between the actual result and the expected target. This feedback informs the system about its efficiency in achieving its target. In addition, considering that flexible flight systems often face non-constant output constraints in actual operation, the application focuses on the more complex and realistic control requirement of time-varying constraints, further improving the constraint processing capability of the controller. Therefore, a time-varying barrier Lyapunov function mechanism is introduced into the execution neural network in the Actor-Critic control architecture, and the control law is redesigned. This mechanism can apply nonlinear reinforcement constraints when the control quantity approaches the boundary, ensuring that the system state is strictly maintained within the predefined time-varying constraint range.

[0175] Figure 4 is a device block diagram of an adaptive dynamic programming control for a flexible flapping wing system under time-varying constraints provided by the embodiment of the application. The device is used for an adaptive dynamic programming control method for a flexible flapping wing system under time-varying constraints. Referring to Figure 4 , the device includes a first construction unit 410, a first design unit 420, a second design unit 430, and a second construction unit 440. Among them:

[0176] The first construction unit 410 is used to define time-varying constraints and virtual variables; define error variables based on virtual variables; design barrier Lyapunov functions based on time-varying constraints; design new Lyapunov functions based on error variables and barrier Lyapunov functions; and construct an execution neural network based on the new Lyapunov functions;

[0177] The first design unit 420 is used to approximate and estimate a long-term cost function using a radial basis function neural network; and design an evaluation neural network based on the long-term cost function;

[0178] The second design unit 430 is used to define an external reinforcement signal and an internal reinforcement signal; and design a reference neural network based on the external reinforcement signal and the internal reinforcement signal;

[0179] The second construction unit 440 is configured to construct an adaptive dynamic programming based flapping control system based on executing the neural network, evaluating the neural network and referring to the neural network, so that the flapping unmanned aerial vehicle can stably track the expected trajectory in a complex scene.

[0180] Optionally, the barrier Lyapunov function is represented by the following formula (1):

[0181]

[0182] wherein g(i) represents a switching function according to a tracking error adjust the barrier Lyapunov function V1; represent a lower bound of the tracking error ; and represent an upper bound of the tracking error.

[0183] Optionally, the new Lyapunov function is represented by the following formula (2):

[0184]

[0185] wherein V2 represents the new Lyapunov function; represent a transpose of ; and represent an error variable defined when the execution neural network is designed.

[0186] Optionally, the construction of the execution neural network based on the new Lyapunov function comprises:

[0187] deriving the new Lyapunov function to obtain a control input of the flapping system; wherein the control input of the system comprises a dynamic deterministic state and an external disturbance;

[0188] wherein the control input of the system is represented by the following formula (3):

[0189]

[0190] wherein F represents the control input of the flapping system; represent a tracking error; represent an error variable defined when the execution neural network is designed; K2 represents a gain matrix; f dis represent an external disturbance; represent ; η represents a virtual variable; S(γ) represents a stiffness matrix; M(γ) represents a mass matrix; represent a derivative of the virtual variable.

[0191] Based on the control input of the flapping wing system, a radial basis function neural network is adopted, a first estimation error is defined, an update rule of the execution neural network is designed, and an execution neural network is constructed based on the update rule of the execution neural network.

[0192] Optionally, the update rule of the execution neural network is represented by the following formula (4):

[0193]

[0194] Wherein, ξ a > 0 represents a learning rate used by the execution neural network; τ a represents the defined estimation error; Y represents a constant greater than zero; represents an estimated value of the cost function; S a represents a basis function vector in the execution neural network; W a represents an input vector of the execution neural network; represents an estimated execution neural network weight.

[0195] Optionally, the first design unit 420 is configured to:

[0196] A radial basis function neural network is adopted to approximate and estimate the long-term cost function, the long-term cost function is derived by introducing an internal reinforcement signal, a second estimation error is defined, an update law of the evaluation neural network is designed, and an evaluation neural network is constructed based on the update law of the evaluation neural network.

[0197] Wherein, the update law of the evaluation neural network is represented by the following formula (5):

[0198]

[0199] Wherein, ξ c > 0 represents a learning rate of the evaluation neural network, represents a combination term representing a gradient direction in the evaluation neural network weight update; K represents a discount factor; S c represents a basis function output vector of the evaluation neural network; represents a derivative of the evaluation neural network with respect to time; υ(t) represents an instantaneous cost function; represents an estimated evaluation neural network weight; E c represents a predefined first error; represents a transpose of the estimated evaluation neural network weight.

[0200] Optionally, the second design unit 430 is configured to:

[0201] Define the external reinforcement signal and the internal reinforcement signal; based on the external reinforcement signal, the internal reinforcement signal, and the defined error, design the update rule of the reference neural network; and construct the reference neural network based on the update rule of the reference neural network.

[0202] The update rule of the reference neural network is expressed by the following formula (6):

[0203]

[0204] in, E represents the estimated reference neural network weights; r ξ represents a predefined second error; r >0 indicates the learning rate of the reference neural network; e r J(t) represents the error function of the reference neural network; J(t) represents the cost function; R(t) represents the internal enhancement signal; e r (t) represents the error function of the reference neural network; S r W represents the vector output by the basis functions of the reference neural network. r This represents the input vector of the reference neural network.

[0205] The embodiments of the present invention take into account the unknown interference and system uncertainty that the flapping-wing UAV may be subject to from the external environment or human factors in the working environment. A reference neural network with nonlinear approximation capability is introduced into the control law to learn and compensate for the external interference and system uncertainty of the flapping-wing system in real time, which significantly increases the system's anti-interference capability and vibration suppression capability.

[0206] This invention redesigns the execution neural network using a barrier Lyapunov function, achieving strict adherence to time-varying constraints in the control output. This approach not only improves the tracking of the desired trajectory but also further enhances the performance of the flapping-wing system under time-varying constraints.

[0207] The embodiments of the present invention are based on the traditional Actor-Critic control method, and introduce an internal enhancement signal to connect the reference neural network and the evaluation neural network together, so as to ensure that the two can communicate more closely, thereby reducing the dependence on external environmental information and improving the controller's adaptive capability.

[0208] The embodiment of the application designs a self-adaptive control based on a double evaluation neural network for time-varying constraints. A reference neural network is added to the traditional Actor-Critic control method. The reference neural network generates an internal reinforcement signal, which is continuous and dynamically adjustable, ranging from [-1, 1], and is used as the input of the evaluation neural network. The internal reinforcement signal can be adaptively modified with the evolution of the system state. The evaluation neural network and the reference neural network are integrated through the internal reinforcement signal to form a double evaluation architecture. The evaluation neural network can evaluate the current system performance by comparing it with the reference signal, thereby identifying the deviation between the actual result and the expected target. This feedback informs the system about its efficiency in achieving its target. In addition, considering that flexible flight systems often face non-constant output constraints in actual operation, the application focuses on the more complex and realistic control requirement of time-varying constraints, further improving the constraint processing capability of the controller. Therefore, a time-varying barrier Lyapunov function mechanism is introduced into the execution neural network in the Actor-Critic control architecture, and the control law is redesigned. This mechanism can apply nonlinear reinforcement constraints when the control quantity approaches the boundary, ensuring that the system state is strictly maintained within the predefined time-varying constraint range.

[0209] Figure 5 is a structural schematic diagram of a self-adaptive dynamic programming control device for a flexible flapping wing system under time-varying constraints provided by the embodiment of the application, as Figure 5 shown, the self-adaptive dynamic programming control device for a flexible flapping wing system under time-varying constraints can include the self-adaptive dynamic programming control apparatus for a flexible flapping wing system under time-varying constraints shown in Figure 4 Optionally, the self-adaptive dynamic programming control device 510 for a flexible flapping wing system under time-varying constraints can include a first processor 2001.

[0210] Optionally, the self-adaptive dynamic programming control device 510 for a flexible flapping wing system under time-varying constraints can further include a memory 2002 and a transceiver 2003.

[0211] Among them, the first processor 2001 and the memory 2002 and the transceiver 2003, such as can be connected through the communication bus.

[0212] The various constituent components of the self-adaptive dynamic programming control device 510 for a flexible flapping wing system under time-varying constraints will be specifically introduced below: Figure 5

[0213] ​The first processor 2001 is a control center of the adaptive dynamic programming control device 510 for the flexible flapping wing system under time-varying constraints, which can be one processor or a collective name of multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), which can also be application specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present application, such as one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs).

[0214] Optionally, the first processor 2001 can execute various functions of the adaptive dynamic programming control device 510 for the flexible flapping wing system under time-varying constraints by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0215] In a specific implementation, as an embodiment, the first processor 2001 can include one or more CPUs, such as the CPU0 and the CPU1 shown in FIG. 8. Figure 5

[0216] In a specific implementation, as an embodiment, the adaptive dynamic programming control device 510 for the flexible flapping wing system under time-varying constraints can also include multiple processors, such as the first processor 2001 and the second processor 2004 shown in FIG. 8. Each of these processors can be a single-CPU or a multi-CPU. The processor here can refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions). Figure 5

[0217] The memory 2002 is used to store software programs for implementing the schemes of the present application, and is controlled by the first processor 2001 for execution. The specific implementation manner can refer to the above method embodiments, which will not be described here.

[0218] ​​Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or exist independently, and may be connected via the interface circuit of the adaptive dynamic programming control device 510 for the time-varying constrained flexible flapping wing system. Figure 5 (Not shown in the figure) is coupled to the first processor 2001, and the embodiments of the present invention do not specifically limit this.

[0219] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0220] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 5 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the sending function.

[0221] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and can be controlled via the interface circuit of the adaptive dynamic programming control device 510 for the flexible flapping wing system under time-varying constraints. Figure 5 (Not shown in the figure) is coupled to the first processor 2001, and the embodiments of the present invention do not specifically limit this.

[0222] It should be noted that, Figure 5 The structure of the adaptive dynamic programming control device 510 for the flexible flapping wing system under time-varying constraints shown in the figure does not constitute a limitation on the router. The actual adaptive dynamic programming control device for the flexible flapping wing system under time-varying constraints may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0223] In addition, the technical effects of the adaptive dynamic programming control device 510 of the flexible flapping wing system under time-varying constraints can refer to the technical effects of the adaptive dynamic programming control method of the flexible flapping wing system under time-varying constraints described in the above method embodiments, which will not be repeated here.

[0224] It should be understood that the first processor 2001 in the embodiments of the present application can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processor can also be any conventional processor.

[0225] It should also be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example but not limitation, many forms of random access memory (RAM) are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DRAM).

[0226] The above-described embodiments can be implemented in whole or in part by software, hardware (e.g., circuitry), firmware, or any combination thereof. When implemented in software, the above-described embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are wholly or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center through wired (e.g., infrared, wireless, microwave, etc.) or wireless means. The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. containing one or more available medium collections. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium can be a solid state disk.

[0227] It should be understood that the term "and / or" herein merely describes an association relationship of associated objects, which means that there can be three relationships, for example, A and / or B can represent three cases of A alone, A and B together, and B alone, where A and B can be singular or plural. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects, but can also represent an "and / or" relationship, which can be understood in the context before and after.

[0228] In the present application, "at least one" means one or more, and "multiple" means two or more. "At least one of the following" or the like means any combination of the items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0229] It should be understood that in various embodiments of the present application, the size of the sequence number of the above-described processes does not mean the order of execution, and the execution order of the processes should be determined by their functions and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0230] Those skilled in the art can clearly understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0231] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the devices, apparatuses and units described above can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0232] In several embodiments provided by the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0233] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0234] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically independently, or two or more units can be integrated into one unit.

[0235] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the present application that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0236] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An adaptive dynamic programming control method for a flexible flapping-wing system under time-varying constraints, characterized in that, The method includes: S1. Define time-varying constraints and dummy variables; define error variables based on dummy variables; design barrier Lyapunov functions based on time-varying constraints; design new Lyapunov functions based on error variables and barrier Lyapunov functions; construct execution neural networks based on new Lyapunov functions. The update rule of the execution neural network is expressed by the following formula (1): （1） in, This indicates the learning rate used to execute the neural network; This represents the defined estimation error; It represents a constant greater than zero; This represents an estimate of the cost function; This represents the vector of basis functions in the neural network. This represents the input vector for executing the neural network; This represents the estimated weights of the execution neural network; S2. Use a radial basis function neural network to approximate and estimate the long-run cost function; design an evaluation neural network based on the long-run cost function. The update law for evaluating neural networks is expressed by the following formula (2): （2） in, The learning rate is used to evaluate the neural network. This represents the combination term indicating the gradient direction in the evaluation of neural network weight updates; K represents the discount factor. This represents the output vector of the basis functions used to evaluate the neural network. This represents the time derivative of the evaluation neural network; Represents the instantaneous cost function; This represents the estimated evaluation neural network weights; This represents a predefined first error; This represents the transpose of the estimated evaluation neural network weights; S3. Define the external enhancement signal and the internal enhancement signal; design a reference neural network based on the external enhancement signal and the internal enhancement signal; The update rule of the reference neural network is expressed by the following formula (3): （3） in, This represents the estimated reference neural network weights; This represents a predefined second error; This represents the learning rate of the reference neural network; This represents the error function of the reference neural network; Represents the cost function; Indicates an internal enhancement signal; This represents the error function of the reference neural network; This represents the vector output by the basis functions of the reference neural network. This represents the input vector of the reference neural network; S4. Based on the execution neural network, evaluation neural network and reference neural network, an adaptive dynamic programming-based flapping wing control system is constructed; the adaptive dynamic programming-based flapping wing control system enables flapping wing UAVs to stably track the desired trajectory in complex scenarios.

2. The adaptive dynamic programming control method for a flexible flapping-wing system under time-varying constraints according to claim 1, characterized in that, The barrier Lyapunov function is expressed by the following formula (4): （4） in, The switching function is based on the tracking error. Adjusting the barrier Lyapunov function ; Indicates tracking error The lower bound; This represents the upper bound of the tracking error.

3. The adaptive dynamic programming control method for a flexible flapping-wing system under time-varying constraints according to claim 1, characterized in that, The new Lyapunov function is expressed by the following formula (5): （5） in, Represent the new Lyapunov function; express transpose; This represents the error variable defined when designing and executing the neural network.

4. The adaptive dynamic programming control method for a flexible flapping-wing system under time-varying constraints according to claim 1, characterized in that, The S1, based on the new Lyapunov function, constructs an execution neural network, including: Differentiating the new Lyapunov function yields the control inputs of the flapping-wing system; the system's control inputs include dynamic deterministic states and external disturbances. The control input of the flapping wing system is expressed by the following formula (6): （6） Where F represents the control input of the flapping wing system; Indicates tracking error; This represents the error variable defined when designing and executing the neural network; Represents the gain matrix; Indicates external interference; express; Represents a dummy variable; Represents the stiffness matrix; Represents the mass matrix; Indicates the derivative of a dummy variable; Based on the control input of the flapping wing system, a radial basis function neural network is adopted. By defining the first estimation error, the update rule of the execution neural network is designed, and the execution neural network is constructed based on the update rule of the execution neural network.

5. The adaptive dynamic programming control method for a flexible flapping-wing system under time-varying constraints according to claim 1, characterized in that, The S2 uses a radial basis function neural network to approximate and estimate the long-run cost function; based on the long-run cost function, an evaluation neural network is designed, including: A radial basis function neural network is used to approximate and estimate the long-term cost function. The long-term cost function is differentiated by introducing an internal reinforcement signal. By defining a second estimation error, an update law for the evaluation neural network is designed. Based on the update law of the evaluation neural network, an evaluation neural network is constructed.

6. The adaptive dynamic programming control method for a flexible flapping-wing system under time-varying constraints according to claim 1, characterized in that, S3 is defined as external enhancement signal and internal enhancement signal; A reference neural network is designed based on external and internal enhancement signals, including: Define the external reinforcement signal and the internal reinforcement signal; based on the external reinforcement signal, the internal reinforcement signal, and the defined error, design the update rule of the reference neural network; and construct the reference neural network based on the update rule of the reference neural network.

7. An adaptive dynamic programming control device for a flexible flapping-wing system under time-varying constraints, wherein the adaptive dynamic programming control device for a flexible flapping-wing system under time-varying constraints is used to implement the adaptive dynamic programming control method for a flexible flapping-wing system under time-varying constraints as described in any one of claims 1-6, characterized in that, The device includes: The first building unit is used to define time-varying constraints and dummy variables; based on the dummy variables, error variables are defined; based on the time-varying constraints, a barrier Lyapunov function is designed; based on the error variables and the barrier Lyapunov function, a new Lyapunov function is designed; based on the new Lyapunov function, an execution neural network is constructed. The first design unit is used to approximate and estimate the long-run cost function using a radial basis function neural network; and to design an evaluation neural network based on the long-run cost function. The second design unit is used to define the external enhancement signal and the internal enhancement signal; and to design a reference neural network based on the external enhancement signal and the internal enhancement signal. The second building unit is used to construct an adaptive dynamic programming-based flapping wing control system based on an execution neural network, an evaluation neural network, and a reference neural network. The adaptive dynamic programming-based flapping wing control system enables flapping wing UAVs to stably track the desired trajectory in complex scenarios.

8. An adaptive dynamic programming control device for a flexible flapping wing system under time-varying constraints, characterized in that, The adaptive dynamic programming control device for the flexible flapping wing system under time-varying constraints includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 6.

Images