A helicopter control method, system and terminal based on incremental dual heuristic programming

Abstract

This invention provides a helicopter control method, system, and terminal based on incremental dual heuristic programming, relating to the field of helicopter control technology. It constructs a learning framework including a helicopter flight dynamics model, a reward function, an execution neural network, an evaluation neural network, a target-evaluation neural network, and an incremental nonlinear dynamic inverse model. Based on learning rules, the weights of the execution neural network, evaluation neural network, and target-evaluation neural network, as well as the parameters of the incremental nonlinear dynamic inverse model, are updated. The helicopter control law is determined based on the trained execution neural network, evaluation neural network, target-evaluation neural network, and incremental nonlinear dynamic inverse model, achieving intelligent helicopter control. This invention applies incremental dual heuristic programming to helicopter control, effectively solving the helicopter control problems caused by model nonlinearity and system coupling, and improving the adaptability and robustness of helicopter control.

Description

Technical Field

[0001] This invention relates to the field of helicopter control technology, and in particular to a helicopter control method, system and terminal based on incremental dual heuristic programming. Background Technology

[0002] The outstanding characteristics of helicopters are their ability to perform low-altitude (a few meters above the ground), low-speed (starting from hovering), and constant-direction maneuvering, especially their ability to take off and land vertically in small areas. These characteristics give them a wide range of applications and development prospects. In military applications, they are widely used for ground attack, amphibious assault, weapons transport, logistical support, battlefield medical care, reconnaissance patrols, command and control, communications, anti-submarine warfare, mine clearance, and electronic warfare. In civilian applications, they are used for short-haul transportation, medical evacuation, disaster relief and rescue, emergency rescue, equipment hoisting, geological exploration, forest fire fighting, and aerial photography. The transport of personnel and supplies between offshore oil wells and bases is an important aspect of their civilian applications.

[0003] However, due to the complexity of helicopter dynamics and its inherent static instability, helicopter control is a nonlinear, coupled control system with multiple inputs and multiple outputs, making it a challenging problem.

[0004] In existing technologies, helicopter control methods are mainly divided into three types: first, classical control methods, which are mainly based on gain scheduling, PID feedback technology, etc.; second, modern control methods, which are mainly based on fuzzy control, incremental nonlinear dynamic inverse control, etc.; and third, control methods based on deep reinforcement learning, which construct control laws through intelligent agent trial and error methods.

[0005] Classical control methods suffer from high development difficulty, high modeling accuracy requirements, poor adaptability, and insufficient fault tolerance. Modern control methods are overly sensitive to modeling errors, require rapid and accurate acceleration measurements, and exhibit weak controller robustness. Deep reinforcement learning-based control methods suffer from low sampling efficiency and require large amounts of training data. Summary of the Invention

[0006] This invention provides a helicopter control method based on incremental dual heuristic programming. This invention solves the problems of being too sensitive to modeling errors, requiring fast and accurate acceleration measurement, and having weak controller robustness.

[0007] Helicopter control methods include:

[0008] S1, Construct a learning framework, which includes a helicopter flight dynamics model, a reward function, an execution neural network, an evaluation neural network, a target-evaluation neural network, and an incremental nonlinear dynamic inverse model;

[0009] S2, based on the learning rules, updates the weights and incremental nonlinear dynamic inverse model parameters of the execution neural network, evaluation neural network, and target-evaluation neural network;

[0010] S3 determines the helicopter control law based on the trained execution neural network, evaluation neural network, target-evaluation neural network, and incremental nonlinear dynamic inverse model, thereby realizing intelligent helicopter control.

[0011] It should be further noted that the helicopter flight dynamics model in S1 provides an interactive environment, which specifically includes:

[0012] The state-space variable s is defined as follows:

[0013]

[0014] Action space variable 'a' is defined as follows:

[0015] a=[δ col ,δ lon ,δ lat ,δ ped ]

[0016] Where, v, w is the linear velocity in the body coordinate system;

[0017] p, q, r are the angular velocities in the body coordinate system;

[0018] φ,θ, The Euler angles are the body coordinate system relative to the inertial coordinate system.

[0019] x, y, z are the position vectors of the body coordinate system relative to the inertial coordinate system;

[0020] λ 0mr Main rotor inflow ratio;

[0021] λ 0tr The tail rotor inflow ratio;

[0022] δ col Input the total distance;

[0023] δ lon Longitudinal input of the control stick;

[0024] δ lat Lateral input via control stick;

[0025] δ ped Foot pedal rudder input.

[0026] It should be further noted that the reward function in S1 is used to evaluate the system state, and it specifically includes:

[0027] reward function The definition is as follows:

[0028]

[0029] Where P is a p×n dimensional state selection matrix;

[0030] Q is a p×p state weight matrix;

[0031] s t+1 These are the state-space variables at time t+1;

[0032] Let t be the reference state-space variable;

[0033] derivative of reward function The definition is as follows:

[0034]

[0035] It should be further noted that the neural network executed in S1 is used to learn a deterministic policy, which specifically includes:

[0036] The loss function L of the neural network A as follows:

[0037]

[0038] in, Let t be the state value function at time t; Let t+1 be the state value function; Here, γ is the reward function; γ is the reward discount factor.

[0039] Execute the gradient of the loss function of the neural network The definition is as follows:

[0040]

[0041] in, The derivative of the reward function; The evaluation value of the target-evaluation neural network output; G t+1 The predictive control matrix is ​​the output of the incremental nonlinear dynamic inverse model; a t The output value of the neural network is ω. a To execute neural network weights; Calculations are performed by executing a neural network backpropagation algorithm;

[0042] The weights ω of the neural network a The updated announcement is as follows:

[0043]

[0044] Where, ηa To implement the learning rate for the neural network.

[0045] It should be further noted that the evaluation neural network in S1 is used to calculate the partial derivative of the state value function with respect to the state. Specifically, it includes:

[0046] Evaluate the loss function L of the neural network λ The definition is as follows:

[0047]

[0048] in, The definition is as follows:

[0049]

[0050] state-space variable s t+1 For s t The derivative is defined as follows:

[0051]

[0052] Among them, F t+1 G is the predicted state matrix output by the incremental nonlinear dynamic inverse model; t+1 This is the predictive control matrix output by the incremental nonlinear dynamic inverse model; Calculations are performed by executing a neural network backpropagation algorithm;

[0053] Evaluate the gradient of the loss function of a neural network The definition is as follows:

[0054]

[0055] Evaluating the weights ω of a neural network c The updated announcement is as follows:

[0056]

[0057] Where, η c To evaluate the learning rate of a neural network.

[0058] It should be further noted that the target-evaluation neural network in S1 is used to calculate the partial derivative of the state value function with respect to the state. Updating the weights of the objective-evaluation neural network using a soft policy function with weights The calculation formula is as follows:

[0059]

[0060] Where τ is the mixing factor.

[0061] It should be further noted that the incremental nonlinear dynamic inverse model in S1 is used to solve the predicted state matrix F. t and predictive control matrix G t The iterative calculation formula is as follows:

[0062]

[0063] in, Δs t+1 = t-1Δst +G t-1 Δa t ,

[0064] κ is the forgetting coefficient.

[0065] The present invention also provides a helicopter control system based on incremental dual heuristic programming, the system comprising: a main rotor flapping module, a main rotor aerodynamic module, a main rotor module, an inflow ratio solving module, a tail rotor aerodynamic module, a tail rotor module, a fuselage module, a horizontal tail module, a vertical tail module, and a motion equation solving module.

[0066] The main rotor flapping module is used to solve for the taper angle a0, longitudinal flapping angle a1, and lateral flapping angle b1 in the cp coordinate system with a constant main rotor pitch. The equations are as follows:

[0067]

[0068]

[0069]

[0070] Where, γ h It is a dimensionless Locke number; Normalized flapping frequency; θ0 is the total rotor pitch angle; θ tw For linear blade torque angle; θ 1s Longitudinal periodic pitch; θ 1c Lateral periodic pitch; μ x Rotor advance ratio; λ is rotor inflow ratio; The dimensionless roll rate of the organism; The dimensionless pitch rate of the body;

[0071] The main rotor aerodynamic module is used to solve for the force F in the dp coordinate system at the rotor tip plane. mr,dp and torque Q mr,dp Force F mr,dp The calculation formula is as follows:

[0072]

[0073] in, The calculation formula is as follows:

[0074]

[0075] F mr,cp The calculation formula is as follows:

[0076]

[0077] Where ρ is the air density; A is the rotor area; C H C is the backforce coefficient; S C is the lateral force coefficient. T C is the lift coefficient; H C S and C T The calculation formula is as follows:

[0078]

[0079]

[0080]

[0081] Where σ is the rotor realism; C L C is the rotor lift coefficient; D This is the rotor drag coefficient;

[0082] Torque Q mr,dp The calculation formula is as follows:

[0083] Q mr,dp =ρA(ΩR) 2 RC Q

[0084] Among them, C Q The torque coefficient is calculated using the following formula:

[0085]

[0086] in, The inflow ratio is dp relative to the blade tip plane, where V is the airspeed, Ω is the rotor speed, R is the rotor radius, and α is the rotor radius. dp λ is the incoming flow angle relative to the blade tip plane dp; λ0 is the incoming flow ratio;

[0087] The main rotor module is used to apply the force F under the dp coordinate system of the main rotor tip plane. mr,dp and torque M mr,dp Force F converted to body coordinates mr and torque M mr Force F mr The calculation formula is as follows:

[0088]

[0089] Among them, a 1R =a1-θ 1s b 1R =b1+θ 1c γ is the forward tilt angle of the propeller hub shaft;

[0090] Torque M mr The calculation formula is as follows:

[0091]

[0092] Where, x mr y mr , z mr These are the distances from the main rotor to the helicopter's center of mass in the airframe coordinate system;

[0093] The inflow ratio solution module is used to calculate the inflow ratio λ0. The differential equation for the inflow ratio λ0 is as follows:

[0094]

[0095] in, C is a time constant. T Defined before; The calculation formula is as follows:

[0096]

[0097] Where μ = V / ΩR;

[0098] The tail rotor aerodynamic module is used to solve for the tail rotor thrust T in the local coordinate system. tr The calculation formula is as follows:

[0099]

[0100] Among them, A tr Ω represents the area of ​​the tail rotor disk. tr Tail rotor speed; R tr Tail rotor radius; Tail rotor thrust coefficient; The calculation formula is as follows:

[0101]

[0102]

[0103]

[0104] Where u, v, w are the airspeed of the aircraft; p, q, r are the angular velocities of the aircraft; x tr , ztr These are the distances between the tail rotor and the helicopter's center of mass, respectively; K tr The influence factor of the main rotor downwash on the tail rotor; λ 0tr The tail rotor inflow ratio;

[0105] The tail rotor module is used to calculate the tail rotor thrust F in the helicopter's center-of-mass coordinate system. tr and torque M tr F re The calculation formula is as follows:

[0106]

[0107] in, S vt This refers to the vertical tail area;

[0108] M tr The calculation formula is as follows:

[0109]

[0110] Where, x tr and z tr This is the distance from the tail rotor to the helicopter's center of mass.

[0111] The fuselage module is used to calculate the forces F on the fuselage in the helicopter's center-of-mass coordinate system. fus and torque M fus F fus The calculation formula is as follows:

[0112]

[0113] Among them, R y R z Let α be the rotation matrix; fus For fuselage angle of attack; β fus For fuselage sideslip angle; - fus For fuselage parasitic drag; R fus The calculation formula is as follows:

[0114]

[0115] Where F0 is the area of ​​parasitic drag on the fuselage;

[0116] M fus The calculation formula is as follows:

[0117]

[0118] in, and The calculation formula is as follows:

[0119]

[0120]

[0121] Among them, K fus For correction factor, The equivalent volume of a circular cross-section in a horizontal plane; α fus The fuselage elevation angle; The equivalent volume of the lateral circular cross-section; β fus The sideslip angle of the aircraft;

[0122] The horizontal tail module is used to calculate the force F of the horizontal tail in the helicopter's center-of-mass coordinate system. ht and torque M ht F ht The calculation formula is as follows:

[0123]

[0124] Among them, L ht The calculation formula is as follows:

[0125]

[0126] in, S ht The area is the wing surface area. α is the slope of the wing lift curve; ht The horizontal tail fin elevation angle;

[0127] M fus The calculation formula is as follows:

[0128]

[0129] Where, x ht This is the distance from the horizontal stabilizer to the helicopter's center of mass.

[0130] The vertical tail module is used to calculate the force F of the vertical tail in the helicopter's center-of-mass coordinate system. vt and torque M vt F vt The calculation formula is as follows:

[0131]

[0132] Among them, L vt The calculation formula is as follows:

[0133]

[0134] in, S vt The vertical tail fin area; β is the slope of the lift curve of the vertical tail wing;vt The sideslip angle of the vertical tail fin;

[0135] vt The calculation formula is as follows:

[0136]

[0137] Where, x vt and z vt This is the distance from the horizontal stabilizer to the helicopter's center of mass.

[0138] The equations of motion solution module is used to solve for the velocity, displacement, angular velocity, and Euler angles of the helicopter's center of mass.

[0139] The present invention also provides a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of a helicopter control method based on incremental dual heuristic programming.

[0140] As can be seen from the above technical solutions, the present invention has the following advantages:

[0141] The helicopter control method based on incremental dual heuristic programming provided by this invention combines the advantages of incremental nonlinear dynamic inverse control and deep reinforcement learning, and has the characteristics of strong adaptability and high training sampling efficiency. It can solve the problems of nonlinear and system coupling control of helicopters in the prior art, and improve the adaptability and robustness of helicopter control. Attached Figure Description

[0142] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0143] Figure 1 A schematic diagram of the learning framework for intelligent helicopter control methods;

[0144] Figure 2 A structural diagram of the helicopter flight dynamics model for intelligent helicopter control methods;

[0145] Figure 3 The structure diagram of the evaluation neural network and the execution neural network for the longitudinal control channel of a helicopter;

[0146] Figure 4 The structure diagram of the evaluation neural network and the execution neural network for the helicopter altitude control channel;

[0147] Figure 5To track the weight update curves from the input layer to the hidden layer of the neural network executing the longitudinal control channel of the helicopter during the experiment;

[0148] Figure 6 To track the weight update curves from the hidden layer to the output layer of the neural network executing the longitudinal control channel of the helicopter during the experiment;

[0149] Figure 7 To track the altitude and pitch angle curves during the experiment. Detailed Implementation

[0150] The helicopter control method based on incremental dual heuristic programming provided by this invention can acquire and process related data using artificial intelligence technology. Specifically, the embodiments of this invention can acquire and process related data using artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application devices that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0151] The method also has machine learning capabilities, wherein the machine learning and deep learning in the method of this invention typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and formulaic learning.

[0152] The helicopter control method based on incremental dual heuristic programming involved in this invention applies incremental dual heuristic programming to helicopter control through three stages: building a learning framework, online training and learning, and testing and application. It establishes a helicopter system model through online identification and continuously learns through a trial-and-error exploration mechanism to establish the helicopter control law. This provides a method to solve the helicopter control problem caused by model nonlinearity and system coupling, and improves the adaptability and robustness of helicopter control.

[0153] The helicopter control method based on incremental dual heuristic programming is applied to one or more terminals. The terminal is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0154] The terminal can be any electronic product that can interact with the user, such as personal computers, tablets, smartphones, personal digital assistants (PDAs), interactive network television (IPTV), etc.

[0155] The terminal may also include network devices and / or user equipment. The network devices include, but are not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0156] The network where the terminal is located includes, but is not limited to, the Internet, wide area network, metropolitan area network, local area network, and virtual private network (VPN).

[0157] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0158] Please see Figure 1 The diagram shown is a schematic representation of a helicopter control method based on incremental dual heuristic programming in a specific embodiment. The method includes:

[0159] S1, Construct a learning framework, which includes a helicopter flight dynamics model, a reward function, an execution neural network, an evaluation neural network, a target-evaluation neural network, and an incremental nonlinear dynamic inverse model;

[0160] In one exemplary embodiment, the helicopter flight dynamics model provides an interactive environment, which specifically includes:

[0161] The state-space variable s is defined as follows:

[0162]

[0163] Action space variable 'a' is defined as follows:

[0164] a=[δ col ,δ lon ,δ lat ,δ ped ]

[0165] Where, v, w is the linear velocity in the body coordinate system;

[0166] p, q, r are the angular velocities in the body coordinate system;

[0167] φ,θ, The Euler angles are the body coordinate system relative to the inertial coordinate system.

[0168] x, y, z are the position vectors of the body coordinate system relative to the inertial coordinate system;

[0169] λ 0mr Main rotor inflow ratio;

[0170] λ 0tr The tail rotor inflow ratio;

[0171] δ col Input the total distance;

[0172] δ lon Longitudinal input of the control stick;

[0173] δ lat Lateral input via control stick;

[0174] δ ped Foot pedal rudder input.

[0175] The reward function of this invention is used to evaluate the system state, and specifically includes:

[0176] reward function The definition is as follows:

[0177]

[0178] Where P is a p×n dimensional state selection matrix;

[0179] Q is a p×p state weight matrix;

[0180] s t+1 These are the state-space variables at time t+1;

[0181] Let t be the reference state-space variable;

[0182] derivative of reward function The definition is as follows:

[0183]

[0184] In embodiments of the present invention, the execution neural network is used to learn a deterministic policy, specifically including:

[0185] The loss function L of the neural network A as follows:

[0186]

[0187] in, Let t be the state value function at time t; Let t+1 be the state value function; Here, γ is the reward function; γ is the reward discount factor.

[0188] Execute the gradient of the loss function of the neural network The definition is as follows:

[0189]

[0190] in, The derivative of the reward function; The evaluation value of the target-evaluation neural network output; G t+1 The predictive control matrix is ​​the output of the incremental nonlinear dynamic inverse model; a t The output value of the neural network is ω. a To execute neural network weights; Calculations are performed by executing a neural network backpropagation algorithm;

[0191] The weights ω of the neural network a The updated announcement is as follows:

[0192]

[0193] Where, η a To implement the learning rate for the neural network.

[0194] In one exemplary embodiment, the evaluation neural network is used to compute the partial derivative of the state value function with respect to the state. Specifically, it includes:

[0195] Evaluate the loss function L of the neural network λ The definition is as follows:

[0196]

[0197] in, The definition is as follows:

[0198]

[0199] state-space variable s t+1 For s t The derivative is defined as follows:

[0200]

[0201] Among them, F t+1 G is the predicted state matrix output by the incremental nonlinear dynamic inverse model; t+1This is the predictive control matrix output by the incremental nonlinear dynamic inverse model; Calculations are performed by executing a neural network backpropagation algorithm;

[0202] Evaluate the gradient of the loss function of a neural network The definition is as follows:

[0203]

[0204] Evaluating the weights ω of a neural network c The updated announcement is as follows:

[0205]

[0206] Where, η c To evaluate the learning rate of a neural network.

[0207] The target-evaluation neural network of this invention is used to calculate the partial derivative of the state value function with respect to the state. Updating the weights of the objective-evaluation neural network using a soft policy function with weights The calculation formula is as follows:

[0208]

[0209] Where τ is the mixing factor.

[0210] In one exemplary embodiment, an incremental nonlinear dynamic inverse model is used to solve for the predicted state matrix F. t and predictive control matrix G t The iterative calculation formula is as follows:

[0211]

[0212] in,

[0213] κ is the forgetting coefficient.

[0214] S2, based on the learning rules, updates the weights and incremental nonlinear dynamic inverse model parameters of the execution neural network, evaluation neural network, and target-evaluation neural network;

[0215] S3 determines the helicopter control law based on the trained execution neural network, evaluation neural network, target-evaluation neural network, and incremental nonlinear dynamic inverse model, thereby realizing intelligent helicopter control.

[0216] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0217] The helicopter control method based on incremental dual heuristic programming described above can effectively solve the helicopter control problems caused by model nonlinearity and system coupling, and improve the adaptability and robustness of helicopter control. Furthermore, this invention can also improve helicopter control accuracy based on incremental dual heuristic programming, thereby achieving timely and scientific monitoring, management, and control of the entire helicopter control process.

[0218] The following are embodiments of a helicopter control system based on incremental dual heuristic programming provided in this disclosure. This system and the helicopter control methods based on incremental dual heuristic programming in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the helicopter control system based on incremental dual heuristic programming, please refer to the embodiments of the helicopter control methods based on incremental dual heuristic programming.

[0219] like Figure 2 As shown, the system includes: main rotor flapping module, main rotor aerodynamic module, main rotor module, inflow ratio solving module, tail rotor aerodynamic module, tail rotor module, fuselage module, horizontal tail module, vertical tail module, and motion equation solving module;

[0220] To facilitate rotor modeling, three coordinate systems are first constructed: the hub plane sp coordinate system, where the hub plane sp is perpendicular to the rotation axis, the y-axis is along the rotation axis, the y-axis is parallel to the fuselage longitudinal axis, and the y-axis satisfies the right-hand rule; the pitch-invariant plane cp coordinate system, where the pitch-invariant plane cp is a constructed plane that does not consider blade flapping motion, and the x-axis is deflected from the x-axis of the hub plane sp coordinate system by a longitudinal periodic pitch θ. 1s The y-axis deflects laterally and periodically at a periodic pitch θ relative to the hub plane sp coordinate system. 1c The z-axis satisfies the right-hand rule; the blade tip plane dp coordinate system, the blade tip plane dp is the rotor blade tip trajectory plane, the x-axis is deflected by the longitudinal flapping angle a1 in the x-axis of the constant pitch plane cp coordinate system, the y-axis is deflected by the lateral flapping angle b1 in the y-axis of the constant pitch plane cp coordinate system, and the z-axis satisfies the right-hand rule.

[0221] The main rotor flapping module is used to solve for the taper angle a0, longitudinal flapping angle a1, and lateral flapping angle b1 in the cp coordinate system with a constant main rotor pitch. The equations are as follows:

[0222]

[0223]

[0224]

[0225] Where, γ h It is a dimensionless Locke number; Normalized flapping frequency; θ0 is the total rotor pitch angle; θ tw For linear blade torque angle; θ 1s Longitudinal periodic pitch; θ 1c Lateral periodic pitch; μ x Rotor advance ratio; λ is rotor inflow ratio; The dimensionless roll rate of the organism; The dimensionless pitch rate of the body;

[0226] The main rotor aerodynamic module is used to solve for the force F in the dp coordinate system at the rotor tip plane. mr, and torque Q mr, Force F mr, The calculation formula is as follows:

[0227]

[0228] in, The calculation formula is as follows:

[0229]

[0230] F mr, The calculation formula is as follows:

[0231]

[0232] Where ρ is the air density; A is the rotor area; C H C is the backforce coefficient; S C is the lateral force coefficient. T C is the lift coefficient; H C S and C T The calculation formula is as follows:

[0233]

[0234]

[0235]

[0236] Where σ is the rotor realism; C L C is the rotor lift coefficient; D This is the rotor drag coefficient;

[0237] Torque Q mr,dp The calculation formula is as follows:

[0238] Q mr,dp =ρA(ΩR) 2 RC Q

[0239] Among them, C Q The torque coefficient is calculated using the following formula:

[0240]

[0241] in, The inflow ratio is dp relative to the blade tip plane, where V is the airspeed, Ω is the rotor speed, R is the rotor radius, and α is the rotor radius. dp λ is the incoming flow angle relative to the blade tip plane dp; λ0 is the incoming flow ratio;

[0242] The main rotor module is used to apply the force F under the dp coordinate system of the main rotor tip plane. mr,dp and torque M mr,dp Force F converted to body coordinates mr and torque M mr Force F mr The calculation formula is as follows:

[0243]

[0244] Among them, a 1R =a1-θ 1s b 1R =b1+θ 1c γ is the forward tilt angle of the propeller hub shaft;

[0245] Torque M mr The calculation formula is as follows:

[0246]

[0247] Where, x mr y mr , z mr These are the distances from the main rotor to the helicopter's center of mass in the airframe coordinate system;

[0248] The inflow ratio solution module is used to calculate the inflow ratio λ0. The differential equation for the inflow ratio λ0 is as follows:

[0249]

[0250] in, C is a time constant. T Defined before; The calculation formula is as follows:

[0251]

[0252] Where μ = V / ΩR;

[0253] The tail rotor aerodynamic module is used to solve for the tail rotor thrust T in the local coordinate system. tr The calculation formula is as follows:

[0254]

[0255] Among them, A tr Ω represents the area of ​​the tail rotor disk. tr Tail rotor speed; R tr Tail rotor radius; Tail rotor thrust coefficient; The calculation formula is as follows:

[0256]

[0257]

[0258]

[0259] Where u, v, w are the airspeed of the aircraft; p, q, r are the angular velocities of the aircraft; x tr , z tr These are the distances between the tail rotor and the helicopter's center of mass, respectively; K tr The influence factor of the main rotor downwash on the tail rotor; λ 0tr The tail rotor inflow ratio;

[0260] The tail rotor module is used to calculate the tail rotor thrust F in the helicopter's center-of-mass coordinate system. tr and torque M tr F re The calculation formula is as follows:

[0261]

[0262] in, S vt This refers to the vertical tail area;

[0263] M tr The calculation formula is as follows:

[0264]

[0265] Where, x tr and z tr This is the distance from the tail rotor to the helicopter's center of mass.

[0266] The fuselage module is used to calculate the forces F on the fuselage in the helicopter's center-of-mass coordinate system. fus and torque M fus F fus The calculation formula is as follows:

[0267]

[0268] Among them, R y R z Let α be the rotation matrix; fus For fuselage angle of attack; β fus For fuselage sideslip angle; - fus For fuselage parasitic drag; R fus The calculation formula is as follows:

[0269]

[0270] Where F0 is the area of ​​parasitic drag on the fuselage;

[0271] M fus The calculation formula is as follows:

[0272]

[0273] in, and The calculation formula is as follows:

[0274]

[0275]

[0276] Among them, K fus For correction factor, The equivalent volume of a circular cross-section in a horizontal plane; α fus The fuselage elevation angle; The equivalent volume of the lateral circular cross-section; β fus The sideslip angle of the aircraft;

[0277] The horizontal tail module is used to calculate the force F of the horizontal tail in the helicopter's center-of-mass coordinate system. ht and torque M ht F ht The calculation formula is as follows:

[0278]

[0279] Among them, L ht The calculation formula is as follows:

[0280]

[0281] in, S ht The area is the wing surface area. α is the slope of the wing lift curve; ht The horizontal tail fin elevation angle;

[0282] M fus The calculation formula is as follows:

[0283]

[0284] Where, x ht This is the distance from the horizontal stabilizer to the helicopter's center of mass.

[0285] The vertical tail module is used to calculate the force F of the vertical tail in the helicopter's center-of-mass coordinate system. vt and torque M vt F vt The calculation formula is as follows:

[0286]

[0287] Among them, L vt The calculation formula is as follows:

[0288]

[0289] in, S vt The vertical tail fin area; β is the slope of the lift curve of the vertical tail wing; vt The sideslip angle of the vertical tail fin;

[0290] vt The calculation formula is as follows:

[0291]

[0292] Where, x vt and z vt This is the distance from the horizontal stabilizer to the helicopter's center of mass.

[0293] The motion equation solving module is used to solve for the velocity, displacement, angular velocity, and Euler angles of the helicopter's center of mass. The method of this invention is verified below using simulation results.

[0294] To verify the effectiveness and real-time performance of the helicopter intelligent control method based on incremental dual heuristic programming of this invention, altitude and pitch angle tracking flight experiments were conducted. The altitude and longitudinal control channels adopted the helicopter intelligent control method based on incremental dual heuristic programming, while the helicopter lateral control channel adopted PID control.

[0295] The evaluation neural network and execution neural network structure of the helicopter longitudinal control channel based on incremental dual heuristic programming are as follows: Figure 3 As shown. The evaluation neural network structure is a three-layer network structure: the input layer neural network has two neurons, q and θ-θ. R The hidden layer has 10 neurons, all using a two-zone tangent activation function; the output layer has 2 neurons, which are the derivative neurons of the state value function with respect to the auxiliary state. and the derivative of the state value function with respect to the tracking state of the neuron The neural network structure is a three-layer network: the input layer has two neurons, q and θ-θ. R The hidden layer has 10 neurons, all using a two-zone tangent activation function; the output layer has 1 neuron, using an execution-restricted two-zone tangent activation function, outputting the action δ. lon .

[0296] The evaluation neural network and execution neural network structure of the helicopter altitude control channel based on incremental dual heuristic programming are as follows: Figure 4 As shown. The evaluation neural network structure is a three-layer network structure: the input layer neural network has two neurons, w and hh. R The hidden layer has 10 neurons, all using a two-zone tangent activation function; the output layer has 2 neurons, which are the derivative neurons of the state value function with respect to the auxiliary state. and the derivative of the state value function with respect to the tracking state of the neuron The neural network structure is a three-layer network: the input layer has two neurons, w and hh. R The hidden layer has 10 neurons, all using a two-zone tangent activation function; the output layer has 1 neuron, using an execution-restricted two-zone tangent activation function, outputting the action δ. con .

[0297] The experimental parameters were set as follows: reward discount factor γ = 0.8; initial variance σ of the neural network weights. ω =0.8; Target-evaluation neural network weight mixing factor τ = 0.01; Vertical control channel executes neural network learning rate Vertical control channel evaluates the learning rate of the neural network The high-level control channel executes the neural network learning rate. High-level control channels evaluate the learning rate of neural networks The forgetting coefficient κ of the incremental nonlinear dynamic inverse model is 0.999; the initial matrix of the incremental nonlinear dynamic inverse model is F0 = , G0 = 0, P0 = ×10 8 Reference lateral displacement y ref =0; Reference yaw angle ψ ref =0; Reference sideslip angle φ ref = trim , where φ trim The initial trim value is the sideslip angle; the collective pitch control input δ col The value range is [2, 18] degrees; the longitudinal input δ of the control stick lon The value range is [10, -5.5] degrees; the lateral input of the control stick is δ. lat The value range is [-6, 4] degrees; the foot pedal rudder input δped The value range is [18, -6] degrees.

[0298] Reference height h ref for:

[0299]

[0300] Reference pitch angle θ ref for:

[0301]

[0302] The attitude tracking control experiment is as follows Figures 5 to 7 As shown; Figure 5 The weight update curve from the input layer to the hidden layer of the neural network executing the longitudinal control channel of the helicopter; Figure 6 The weight update curve from the hidden layer to the output layer of the execution neural network for the helicopter's longitudinal control channel; Figure 7 A comparison chart of tracking altitude h and pitch angle θ, through Figure 7 It can be seen that the altitude control error is kept within ±0.05m and the pitch angle control error is kept within ±0.2°. As can be seen from the simulation effect diagram, the simulation results meet the expected results, which verifies the rationality of the control method designed in this invention.

[0303] The units and algorithm steps of the various examples described in the embodiments disclosed in the helicopter control method can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0304] In the helicopter control method of the present invention, computer program code for performing the operations of the present disclosure can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, or partially on the user's computer and partially on a remote computer.

[0305] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A helicopter control method based on incremental dual heuristic programming, characterized in that the method... include: S1, Construct a learning framework, which includes a helicopter flight dynamics model, a reward function, an execution neural network, an evaluation neural network, a target-evaluation neural network, and an incremental nonlinear dynamic inverse model; The incremental nonlinear dynamic inverse model in S1 is used to solve the predicted state matrix. and predictive control matrix The iterative calculation formula is as follows: in, , , , , Forgetting factor; S2, based on the learning rules, updates the weights and incremental nonlinear dynamic inverse model parameters of the execution neural network, evaluation neural network, and target-evaluation neural network; S3 determines the helicopter control law based on the trained execution neural network, evaluation neural network, target-evaluation neural network, and incremental nonlinear dynamic inverse model, thereby realizing intelligent helicopter control.

2. The helicopter control method based on incremental dual heuristic programming according to claim 1, characterized in that, The helicopter flight dynamics model in S1 provides an interactive environment, which specifically includes: State space variables The definition is as follows: Action space variables The definition is as follows: in, The velocity at the bottom of the body coordinate system; The angular velocity in the body coordinate system; The Euler angles are the body coordinate system relative to the inertial coordinate system. This is the position vector of the body coordinate system relative to the inertial coordinate system; Main rotor inflow ratio; The tail rotor inflow ratio; Input the total distance; Longitudinal input of the control stick; Lateral input via control stick; Foot pedal rudder input.

3. The helicopter control method based on incremental dual heuristic programming according to claim 1, characterized in that, The reward function in S1 is used to evaluate the system state, and it specifically includes: reward function The definition is as follows: in, for dimensional state selection matrix; for State weight matrix; for Time-space variables; for Always refer to the state-space variables; derivative of reward function The definition is as follows: 。 4. The helicopter control method based on incremental dual heuristic programming according to claim 1, characterized in that, The neural network executed in S1 is used to learn a deterministic policy, specifically including: Execute the loss function of the neural network as follows: in, for Time-state value function; for Time-state value function; For the reward function; As a reward discount factor; Execute the gradient of the loss function of the neural network The definition is as follows: in, The derivative of the reward function; The goal is to evaluate the output of the neural network. This is the predictive control matrix output by the incremental nonlinear dynamic inverse model; The output value is executed to reflect the output of the neural network. To execute neural network weights; Calculations are performed by executing a neural network backpropagation algorithm; Execute the weights of the neural network The updated announcement is as follows: in, To implement the learning rate for the neural network.

5. The helicopter control method based on incremental dual heuristic programming according to claim 1, characterized in that, The evaluation neural network in S1 is used to calculate the partial derivative of the state value function with respect to the state. Specifically, it includes: Evaluating the loss function of a neural network The definition is as follows: in, ; The definition is as follows: State space variables right The derivative is defined as follows: in, This is the predicted state matrix output by the incremental nonlinear dynamic inverse model; This is the predictive control matrix output by the incremental nonlinear dynamic inverse model; Calculations are performed by executing a neural network backpropagation algorithm; Evaluate the gradient of the loss function of a neural network The definition is as follows: Evaluating the weights of a neural network The updated announcement is as follows: in, To evaluate the learning rate of a neural network.

6. The helicopter control method based on incremental dual heuristic programming according to claim 1, characterized in that, In S1, the target-evaluation neural network is used to calculate the partial derivative of the state value function with respect to the state. The weights of the objective-evaluation neural network are updated using a soft policy function. The calculation formula is as follows: in, It is a mixing factor.

7. A helicopter control system based on incremental dual heuristic programming, characterized in that, The system employs the helicopter control method based on incremental dual heuristic programming as described in any one of claims 1 to 6; The system includes: main rotor flapping module, main rotor aerodynamic module, main rotor module, inflow ratio solving module, tail rotor aerodynamic module, tail rotor module, fuselage module, horizontal tail module, vertical tail module, and kinematic equation solving module; The main rotor flapping module is used to solve the plane with constant main rotor pitch. taper angle in coordinate system Vertical swing angle and lateral swing angle Solve the equation as follows: in, It is a dimensionless Locke number; To normalize the swing frequency; This refers to the total rotor pitch angle; For linear blade torque angle; Longitudinal periodic pitch; Lateral periodic pitch; Rotor advance ratio; The rotor airflow ratio; The dimensionless roll rate of the organism; The dimensionless pitch rate of the body; The main rotor aerodynamic module is used to solve for the aerodynamics at the rotor tip plane. Force in coordinate system and torque ,force The calculation formula is as follows: in, The calculation formula is as follows: The calculation formula is as follows: in, air density; The rotor area; This is the backforce coefficient; This is the lateral force coefficient; The lift coefficient; , and The calculation formula is as follows: in, For rotor realism; This is the rotor lift coefficient; This is the rotor drag coefficient; Torque The calculation formula is as follows: in, The torque coefficient is calculated using the following formula: in, relative to the tip plane The inflow ratio, of which Airspeed, This refers to the rotor speed; The rotor radius; relative to the tip plane The angle of the incoming flow; Incoming flow ratio; The main rotor module is used to mount the main rotor tip plane. Force in coordinate system and torque Force converted to body coordinates and torque ,force The calculation formula is as follows: in, ; ; The forward tilt angle of the propeller hub shaft; torque The calculation formula is as follows: in, , , These are the distances from the main rotor to the helicopter's center of mass in the airframe coordinate system; The inflow ratio calculation module is used to calculate the inflow ratio. Inflow ratio The differential equation is as follows: in, It is a time constant; Defined before; The calculation formula is as follows: in, ; The tail rotor aerodynamic module is used to solve for the tail rotor thrust in the local coordinate system. The calculation formula is as follows: in, The area of ​​the tail rotor disk; Tail rotor speed; Tail rotor radius; Tail rotor thrust coefficient; The calculation formula is as follows: in, Airspeed; The angular velocity of the machine body; These are the distances between the tail rotor and the helicopter's center of gravity, respectively. The factors influencing the main rotor downwash flow on the tail rotor; The tail rotor inflow ratio; The tail rotor module is used to calculate the tail rotor thrust in the helicopter's center-of-mass coordinate system. and torque , The calculation formula is as follows: in, , This refers to the vertical tail area; The calculation formula is as follows: in, and This is the distance from the tail rotor to the helicopter's center of mass. The fuselage module is used to calculate the forces on the fuselage in the helicopter's center-of-mass coordinate system. and torque , The calculation formula is as follows: in, , It is a rotation matrix; Angle of attack of the fuselage; For the fuselage sideslip angle; Parasitic drag on the fuselage; The calculation formula is as follows: in, This refers to the area of ​​the fuselage that generates parasitic drag. The calculation formula is as follows: in, and The calculation formula is as follows: in, For correction factor, The equivalent volume of a circular cross-section in a horizontal plane; The fuselage elevation angle; The equivalent volume of the lateral circular cross-section; The sideslip angle of the aircraft; The horizontal tail module is used to calculate the forces on the horizontal tail in the helicopter's center-of-mass coordinate system. and torque , The calculation formula is as follows: in, The calculation formula is as follows: in, , The area is the wing surface area. The slope of the wing lift curve; The horizontal tail fin elevation angle; The calculation formula is as follows: in, This is the distance from the horizontal stabilizer to the helicopter's center of mass. The vertical tail module is used to calculate the forces on the vertical tail in the helicopter's center-of-mass coordinate system. and torque , The calculation formula is as follows: in, The calculation formula is as follows: in, , The vertical tail fin area; The slope of the lift curve of the vertical tail wing; The sideslip angle of the vertical tail fin; The calculation formula is as follows: in, and This is the distance from the horizontal stabilizer to the helicopter's center of mass. The equations of motion solution module is used to solve for the velocity, displacement, angular velocity, and Euler angles of the helicopter's center of mass.

8. A terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the helicopter control method based on incremental dual heuristic programming as described in any one of claims 1 to 6.

Images