Intelligent flight control method for four-tilt-rotor aircraft
By constructing a high-precision nonlinear flight dynamics model and an end-to-end intelligent controller, and combining reinforcement learning training, the control stability problem of a four-tilt rotor aircraft during mode switching was solved, achieving stable and robust control across the entire flight envelope.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-11
- Publication Date
- 2026-03-10
AI Technical Summary
Traditional control methods struggle to maintain consistent control performance across the entire flight envelope of a quad tiltrotor aircraft, especially during mode switching, where they are prone to introducing transient attitude disturbances. Furthermore, existing end-to-end control methods neglect multi-objective requirements, resulting in insufficient robustness and stability.
A high-precision nonlinear flight dynamics model was constructed, an end-to-end feedforward neural network intelligent controller was designed, and through reinforcement learning training, a near-end policy optimization algorithm was adopted, combined with a reward function of multiple performance indicators, to achieve continuous control of attitude, speed and heading.
It achieves stable control of a four-tilt rotor aircraft across the entire flight envelope, reduces the impact of modeling inaccuracies, improves the stability and generalization ability of the strategy, possesses unified control capability across modes, and avoids transient disturbances of traditional methods.
Smart Images
Figure CN121635403A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of intelligent flight control of aircraft, in particular to an intelligent flight control method for a four-tilt-rotor aircraft. BACKGROUND
[0002] The four-tilt-rotor aircraft belongs to a hybrid configuration with the characteristics of both helicopters and fixed-wing aircraft. The four nacelles can be independently tilted, so that the aircraft can realize vertical take-off and hovering in helicopter mode, and realize high-speed forward flight in fixed-wing mode, and continuously switch between the two modes through the tilting process. Due to the significant aerodynamic interference between the rotor, wing and fuselage, the dynamic characteristics of this type of aircraft are strongly nonlinear, and present completely different dominant dynamic characteristics in different flight modes. This cross-mode discontinuity makes it difficult for traditional linear control methods to maintain consistent control performance throughout the flight envelope.
[0003] In the prior art, the method of combining stability augmentation controller with mode switching logic is often used, that is, independent control laws are designed for helicopter mode, transition mode and fixed-wing mode, and the control law switching is triggered according to the tilt angle or aerodynamic parameters during flight. However, this method not only requires a large number of manual parameter tuning, but also the switching between controllers is often difficult to achieve smoothness, which can easily introduce instantaneous attitude disturbance during tilting. In addition, the uncertainty of the model, the difficulty in accurately modeling the aerodynamic coupling and external wind disturbance make the traditional control method have limited robustness in complex task environment.
[0004] In recent years, with the development of deep learning and reinforcement learning, end-to-end control methods have begun to be applied to the field of aircraft control. Deep reinforcement learning can learn control strategies directly from nonlinear dynamics through a large number of interactions in a simulation environment, avoiding the complex process of manually constructing control laws. However, the research on this type of method for multi-modal configuration aircraft is still limited. At the same time, the dynamic model relied on by deep reinforcement learning must have high accuracy, especially for the strong coupling system of tilt-rotor aircraft. If the model cannot fully reflect the rotor interference, inertia characteristic changes and body aerodynamic force distribution, the stability and generalization ability of the trained strategy cannot be guaranteed.
[0005] In addition, existing end-to-end flight control works mostly use unified performance indicators for training, but ignore the multi-objective requirements of the aircraft in terms of speed tracking, attitude stability, control energy consumption and steady-state error, resulting in deviations in robustness, response speed or control smoothness of the strategy. For tilt-rotor aircrafts with complex structure and diverse tasks, there is a lack of a method that can simultaneously consider multiple performance indicators and realize cross-mode unified control.
[0006] Therefore, it is urgent to provide an intelligent flight control method for a four-tilt-rotor aircraft to solve the above problems. SUMMARY
[0007] The present application aims to solve the technical problems in the background art, and provides an intelligent flight control method for a four-tilt-rotor aircraft.
[0008] The above object of the present application is achieved by the following technical solutions. The intelligent flight control method for the four-tilt-rotor aircraft comprises the following steps: S1: Constructing a high-precision nonlinear flight dynamics model of the four-tilt-rotor aircraft, which is based on the aerodynamic modeling of the rotors, wings, vertical tails and fuselage components, and combines the rotor-rotor aerodynamic interference and rotor-wing coupling factors to form a set of dynamic equations describing the six-degree-of-freedom motion of the aircraft; S2: Designing an end-to-end feedforward neural network intelligent controller to map the current attitude, angular velocity, speed deviation and yaw error of the aircraft and other state quantities into nineteen continuous control commands; S3: Based on the dynamic model, a simulation environment is constructed, and a proximal policy optimization algorithm is used to train the intelligent controller through reinforcement learning, and a reward function is used to guide the policy convergence, so as to realize stable control of the four-tilt-rotor aircraft within the full flight envelope.
[0009] As a preferred technical solution of the present application, the dynamic model constructed in S1 comprises: The component-level modeling method is used to establish the aerodynamic force models of the rotors, wings, vertical tails and fuselage respectively; The rotor-rotor aerodynamic interference model and the rotor-wing coupling model are established; The six-degree-of-freedom nonlinear flight dynamics equation set is established, which includes the resultant force equation, the resultant moment equation, the angular velocity differential equation, the attitude angle differential equation and the position differential equation; A unified mass property calculation method is constructed in combination with the different inertia characteristics of the aircraft in the rotor mode, the fixed-wing mode and the tilt transition mode.
[0010] As a preferred technical solution of the present application, the nineteen control input quantities of the aircraft include: The tilt angle of the four rotors, the total pitch angle of the four rotors, the longitudinal cyclic pitch angle of the four rotors, the lateral cyclic pitch angle of the four rotors, the front wing aileron differential angle, the rear wing aileron linkage angle and the rudder deflection angle.
[0011] As a preferred technical solution of the present application, the neural network controller in S2 adopts a feedforward fully connected network structure, which includes an input layer, two hidden layers and an output layer, and specifically comprises: The input layer comprises 13 neurons, the first hidden layer comprises 64 neurons, the second hidden layer comprises 64 neurons, and the output layer comprises 19 neurons; The hidden layer adopts a tanh activation function, and the output layer adopts a linear activation function.
[0012] As a preferred technical solution of the application, the 13-dimensional state vector of the input layer comprises attitude angles, angular velocities, speed tracking errors, yaw errors and integral terms corresponding to the above errors of the aircraft; wherein the error is the deviation between the current flight state and the expected state.
[0013] As a preferred technical solution of the application, the error integral term is calculated by exponentially decaying weighting of the error sequence, and the weighted sequence is smoothed to suppress systematic deviation and avoid numerical divergence caused by error accumulation.
[0014] As a preferred technical solution of the application, S3 adopts a proximal policy optimization algorithm (PPO) to train the intelligent controller, comprising: The state set is defined as a 13-dimensional input state, and the action set is defined as a 19-dimensional continuous control variable; An Actor-Critic framework based on a policy network and a value network is constructed; A truncated objective function, multi-step advantage estimation and adaptive KL penalty term are used for policy update; In the dynamic simulation environment, the sampling and updating are repeated until the policy converges.
[0015] As a preferred technical solution of the application, the reward function comprises the following parts: Survival reward for encouraging the aircraft to maintain stable flight; Speed tracking error cost for punishing the deviation between the current speed and the expected speed; Control action cost for punishing the sum of squares of the nineteen control instructions to suppress control energy consumption and encourage action smoothing; Flight stability cost for suppressing attitude jitter caused by excessive angular velocity change; Error integral term cost for suppressing steady-state error; Yaw error cost for punishing the deviation between the current heading and the target heading; wherein adjustable weights are set for each reward item to adapt to different flight tasks.
[0016] As a preferred technical solution of the application, the controller trained by reinforcement learning can realize continuous control of attitude, speed and heading in rotor mode, fixed-wing mode and tilt transition mode, has adaptive compensation ability for aerodynamic interference and nonlinear coupling, and can maintain flight stability within the full flight envelope.
[0017] Compared with the prior art, the present invention has the following beneficial effects: This invention constructs a high-precision nonlinear dynamic model of a four-tilt rotor aircraft, systematically considering the independent aerodynamic characteristics and coupled interference effects of components such as the rotor, wings, vertical tail, and fuselage. It also fully establishes the six-degree-of-freedom flight dynamics equations, attitude and position differential equations, and mass characteristic expressions, enabling the model to accurately reflect the dynamic differences of the aircraft in helicopter, transitional, and fixed-wing modes. This model provides a reliable dynamic environment for training control strategies, significantly reducing training bias caused by inaccurate modeling and improving the stability and generalization of strategy learning.
[0018] The end-to-end intelligent controller based on a fully connected feedforward network designed in this invention employs a mapping structure corresponding to 13-dimensional state inputs and 19-dimensional continuous manipulator outputs. Through a carefully constructed state input design, the introduction of integral terms, and a reward function composed of multiple performance indicators, it achieves comprehensive constraints on attitude error, velocity deviation, control energy consumption, and attitude stability. The network structure uses a 64-unit tanh activation function in the hidden layer, enabling the controller to maintain good nonlinear approximation capability across a wide state space. This achieves unified control across modes and high robustness, avoiding transient disturbances caused by traditional multi-control law switching.
[0019] This invention employs the Proximal Policy Optimization (PPO) algorithm for reinforcement learning training of the intelligent controller. Through mechanisms such as time-decaying sliding integral, policy pruning, and multi-step sequence sampling, it effectively suppresses gradient divergence during policy updates, improving training stability. The controller trained in the constructed simulation environment can simultaneously meet multiple flight performance requirements, including velocity tracking accuracy, attitude stability, energy consumption smoothness, and yaw maintenance. Furthermore, it can achieve continuous and stable control output between helicopter mode, transition mode, and fixed-wing mode, significantly improving the overall adaptability and control performance of tiltrotor aircraft in complex mission flights. Attached Figure Description
[0020] Figure 1 This is a diagram of the intelligent controller architecture. Detailed Implementation
[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the following description is provided in conjunction with embodiments and appendices. Figure 1 The present invention will be further described in detail below. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0022] The present invention will be further described in detail below with reference to specific embodiments, but the present invention is not limited to these embodiments. Equivalent modifications made by those skilled in the art without departing from the principles of the present invention should fall within the protection scope of the present invention.
[0023] The present invention provides an intelligent flight control method for a four-tilt rotor aircraft, which is applied to a multi-mode aircraft with four tiltable nacelles and a compound wing layout. This type of aircraft can achieve vertical take-off and landing and hovering in helicopter mode, and high-speed cruise in fixed-wing mode, and can continuously switch between the two modes through the tilting process.
[0024] To address the challenges of complex control law design, insufficient robustness, and difficulties in manual parameter tuning associated with traditional multi-flight mode switching, this invention constructs a unified control scheme based on "high-precision dynamic modeling + end-to-end intelligent controller + reinforcement learning training + online flight control".
[0025] Step 1: High-precision four-tilt rotor dynamics modeling: First, a nonlinear six-degree-of-freedom flight dynamics model of a four-tilt rotor aircraft in all flight modes is constructed as the unified physical basis for intelligent controller training and online control. In this step, a multibody dynamics approach of "component-level modeling + disturbance coupling modeling" is adopted, establishing aerodynamic models for each component: rotor, wing, vertical tail, and fuselage. By superimposing the forces and moments generated by each component within the aircraft system, the resultant force and resultant moment of the entire aircraft can be obtained.
[0026] First, a nonlinear six-degree-of-freedom flight dynamics model of a four-tilt rotor aircraft is constructed within its entire flight envelope. Employing the multibody dynamics component modeling approach, aerodynamic models are performed separately for the rotors, wings, vertical tail, and fuselage, introducing rotor-rotor aerodynamic disturbances and rotor-wing coupling models. In the body-axis coordinate system, the aerodynamic forces and moments of the four rotors, wings, and fuselage are superimposed into the resultant forces and moments of the entire aircraft, which can be expressed in matrix form:
[0027] In the formula, These are the three components of the resultant force under the body axis system; The three components of the resultant moment under the body axis are represented by the subscript Rotor, which indicates the aerodynamic and aerodynamic moment contributions of the four tiltrotors and their nacelles; Winq represents the main wing; fuselaqe represents the fuselage; and VTail represents the vertical tail. The forces and moments of each component are given by the sub-model, where the rotor section further includes the average aerodynamic model of the rotor disk, the influence of the rotor disk tilt angle and flow field disturbances, and the wing and tail sections include aerodynamic components such as lift, solar force, and pitching moment.
[0028] Considering the impact of the tilting process on inertia, the formula for calculating the mass characteristics under the full-mode is established:
[0029] in, These are the reference moment of inertia and product of inertia for the aircraft in helicopter mode; These parameters, which are integrated with the tilt angles of the four nacelles, reflect the overall tilt state. These are constant coefficients obtained through experiments or finite element calculations. This linear correction allows for the dynamic reflection of the tilting process's influence on the inertia matrix without altering the overall mass, ensuring continuity of the dynamic model across helicopter, tilting, and fixed-wing modes.
[0030] Regarding the rigid body dynamics equations, this embodiment divides the body motion equations into two parts: dynamic equations and kinematic equations. The dynamic equations include angular velocity differential equations and linear velocity differential equations. The angular velocity differential equations are derived from the net torque acting on the four tiltrotor aircraft, using the angular momentum theorem, to obtain the angular velocities in three directions. Differential equation:
[0031] In the formula, These are the angular velocities around the x, y, and z axes of the aircraft, corresponding to roll, pitch, and yaw, respectively. Let L be the angular acceleration; L, M, and N be the resultant torques of the body axis system; and I_x, I_y, I_z, and I_{xy} constitute the body's inertia matrix. This equation fully considers the effects of inertia difference, inertia product, and angular velocity coupling terms on motion.
[0032] The linear velocity differential equation is established in the body axis system based on Newton's second law. Combining the contributions of gravity, Coriolis terms, and resultant forces, we obtain: , , The differential relationships of the velocity components in the three directions are shown in the matrix form of the velocity differential equation:
[0033] In the formula, , , The linear velocity in the body axis system. , , Its time derivative; , , Angular velocity; , For roll angle and pitch angle; It is the acceleration due to gravity; , , The resultant force is the force within the body axis. The kinematic equations include the attitude angle differential equations and the position differential equations.
[0034] The attitude angle differential equation describes the angular velocity and Euler angles. , , The relationship between (roll, pitch, yaw) is shown in the attitude angle differential equation matrix:
[0035] in, , , The time derivative of the attitude angle is given, and the trigonometric function terms in the matrix reflect the attitude change constraints under Euler angle parameterization.
[0036] The position differential equation is established in the ground coordinate system based on the relationship between position and velocity, as shown in your document:
[0037] In the formula, , , This represents the position components of the aircraft in the ground coordinate system; The direction cosine matrix from the machine system to the ground coordinate system, whose elements are composed of attitude angles. , , Decide.
[0038] Combining the above resultant force matrix, angular velocity differential equation, linear velocity differential equation, and attitude and position kinematic equations, the nonlinear flight dynamics equations of the four tiltrotor aircraft in all flight modes can be obtained, abstractly written as:
[0039] In the formula, For time; For the state vector of a four-tilt rotor aircraft; The control input vectors include rotor control, flap and aileron control, rudder control, and nacelle tilt angle control, and include:
[0040] in, All are four-dimensional vectors, representing the tilt angle, collective pitch angle, longitudinal periodic pitch angle, and lateral periodic pitch angle of the four rotors. These are the canard aileron differential angle (controls roll), aft aileron linkage angle (controls pitch), and rudder deflection angle (controls yaw), totaling 19 control inputs.
[0041] Through the above modeling steps, this embodiment obtains a high-precision nonlinear dynamic model that can cover helicopter mode, fixed-wing mode and tilt transition mode, providing a unified simulation environment for intelligent controller training.
[0042] After obtaining a high-precision dynamic model, this embodiment designs an end-to-end intelligent controller structure to achieve direct mapping from flight state vectors to 19-dimensional continuous control variables.
[0043] Step Two: Intelligent Controller Design; The controller employs a fully connected feedforward neural network architecture, comprising an input layer, two hidden layers, and an output layer. (Based on your document's description and...) Figure 1 As shown, the network topology is: 13 neurons in the input layer. The first hidden layer has 64 neurons. The second hidden layer contains 64 neurons. The output layer has 19 neurons.
[0044] Both hidden layers use the tanh activation function, with the output range centered at zero. This helps to alleviate gradient vanishing and meet the small output requirements in the initial stage of control; the output layer uses a linear activation function, which allows the 19-dimensional manipulation to change continuously within the allowable range.
[0045] Controller input state vector Carefully designed to characterize the aircraft's attitude, motion state, and deviation from the target trajectory, including:
[0046] in, For roll angle and pitch angle; It is the angular velocity of the three axes; The tracking errors are for forward, lateral, and vertical speeds, as well as yaw angle. This is the integral term corresponding to the error.
[0047] error Defined as the deviation between the current value and the target value:
[0048] In the formula, the "target value" is given by the upper-level task planning or trajectory planning module, and the "current value" is provided by the sensor or state estimation module.
[0049] To enhance the controller's memory capability and steady-state error elimination capability, this embodiment introduces an error integral term. Its continuous-time form is defined as follows: With smooth kernel Convolution:
[0050] in, This is a smoothing kernel function used to perform a weighted average of historical errors.
[0051] In discrete time, the corresponding implementation is as follows:
[0052] In the formula, For the current discrete time, For the first Error value per control cycle; It is a constant decay coefficient, used to gradually decay the influence of earlier time-time errors on the current integral value, avoiding numerical explosion caused by long time intervals.
[0053] Through this design, the neural network can utilize both current and historical error information, thereby maintaining good steady-state tracking performance even in the presence of modeling errors and external disturbances.
[0054] The 19 neurons in the output layer correspond to the aforementioned 19 manipulated inputs, and their outputs, after appropriate normalization and limiting, serve as... , , , , , , The control commands enable coordinated control of the four rotors, front and rear wings, and rudder.
[0055] Step 3: Training the controller based on reinforcement learning; In this step, a high-fidelity simulation environment is constructed using the aforementioned dynamic model, the flight control problem is modeled as a Markov decision process, and the intelligent controller is trained using a proximal policy optimization (PPO) algorithm based on the Actor-Critic structure.
[0056] The Markov decision process is represented as :in Represents a set of states. Represents a set of actions. It is a transfer function, representing the state in the current state. and actions Next, next state The probability distribution; It is the scalar reward after the action is performed; It is the initial state. It is a discount factor. This is the maximum time range. Strategy Used to model the current state Select action The probability. The goal is to find a strategy. parameters To maximize the expected reward within a finite time frame:
[0057] in, It is all trajectories The probability distribution, This is the initial state distribution. In this study, the states... For the 13-dimensional vector defined in step two, the action... Contains 19 values, strategy It is a neural network controller based on the PPO algorithm.
[0058] The reward function is the core of guiding reinforcement learning behavior. The reward function design in this embodiment follows the principles of "precise tracking + smooth control + attitude stabilization + elimination of steady-state error + maintaining the correct heading," as shown in the formula in your document: As a survival reward, a constant is given for each control cycle to encourage the controller to maintain stable flight for as long as possible; For speed tracking error cost, The penalty is the deviation from the target speed; To control the cost of actions, The sum of squares of all 19-dimensional control commands is penalized to minimize control energy consumption and encourage smooth control output. For flight stability costs, It penalizes high rates of attitude change in order to suppress airframe vibration and improve flight stability; Cost is the integral term of the error. This directly penalizes the magnitude of the integral term, forcing the neural network controller to learn to actively eliminate steady-state errors by utilizing the integral state in its strategy. As a cost for yaw error, the deviation of the yaw angle from the target value is penalized, encouraging the aircraft to maintain the correct heading.
[0059] , , , , These are the weight values for each of the above items, which can be adjusted according to the adaptability of the flight mission.
[0060] During training, a large number of flight trajectory samples are generated using parallel simulation. At each time step, the environment is adjusted according to the current state. and the control actions output by the neural network strategy The next state is calculated using the aforementioned nonlinear dynamic equations. Simultaneously, the immediate reward is calculated based on the reward function. The PPO algorithm uses these samples to estimate the advantage function and adjusts the policy network parameters. The parameters of the value network are updated. In the early stage of training, a larger exploration noise and a smaller KL penalty coefficient are used to enhance the policy exploration ability. As the training iterations proceed, the penalty coefficient is adaptively adjusted to gradually reduce the policy update step size, ensuring stability during the convergence phase.
[0061] The intelligent controller is considered to have completed training when the cumulative reward converges after multiple iterations and the trajectory maintains acceptable attitude and velocity parameters within the entire flight envelope. The trained and validated controller demonstrates good eye-tracking performance and robustness in helicopter mode, fixed-wing mode, and tilt-transition mode. It can automatically adapt to aerodynamic disturbances and model uncertainties, achieving unified control of the four tiltrotor aircraft throughout the entire process from vertical takeoff and landing, hovering to high-speed forward flight.
[0062] This implementation method does not require designing control laws separately for different modes. Instead, it obtains the end-to-end control strategy in one go through a data-driven approach, which significantly reduces the complexity of controller design and facilitates engineering applications and subsequent expansion.
[0063] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for intelligent flight control of a quad tiltrotor aircraft, the method comprising: Comprising the following steps: S1: Constructing a high-precision nonlinear flight dynamics model of the four tilting rotor aircraft, which is based on the modeling of the aerodynamic forces of the rotor, wing, vertical tail and fuselage components, and combines rotor-rotor aerodynamic interference and rotor-wing coupling factors to form a set of dynamic equations describing the six-degree-of-freedom motion of the aircraft; S2: Designing an end-to-end feedforward neural network intelligent controller that maps the current attitude, angular velocity, speed deviation and yaw error of the aircraft and other state quantities to nineteen continuous control commands; S3: Based on the dynamics model, a simulation environment is constructed, and a near-end policy optimization algorithm is used to train the intelligent controller, and the policy is guided to converge through a reward function, achieving stable control of the four tilting rotor aircraft within the full flight envelope.
2. The intelligent flight control method of a quad tiltrotor aircraft according to claim 1, wherein: The dynamics model constructed in S1 includes: The component-level modeling method is used to establish the aerodynamic force models of the rotor, wing, vertical tail and fuselage respectively; A rotor-rotor aerodynamic interference model and a rotor-wing coupling model are established; A set of six-degree-of-freedom nonlinear flight dynamics equations is established, including the equations of resultant force, resultant moment, angular velocity differential equation, attitude angle differential equation and position differential equation; A unified mass property calculation method is constructed in combination with the different inertia characteristics of the aircraft in rotor mode, fixed-wing mode and tilting transition mode.
3. The intelligent flight control method of a quad tiltrotor aircraft according to claim 1, wherein: The nineteen control input quantities of the aircraft include: The tilt angle of the four rotors, the total pitch angle of the four rotors, the longitudinal cyclic pitch angle of the four rotors, the lateral cyclic pitch angle of the four rotors, the front wing aileron differential angle, the rear wing aileron linkage angle, and the rudder deflection angle.
4. The intelligent flight control method of a quad tiltrotor aircraft according to claim 1, wherein: The neural network controller in S2 uses a feedforward fully connected network structure, including an input layer, two hidden layers and an output layer, as follows: The input layer contains 13 neurons, the first hidden layer has 64 neurons, the second hidden layer has 64 neurons, and the output layer has 19 neurons; The hidden layers use tanh activation function, and the output layer uses linear activation function.
5. The intelligent flight control method of a quad tiltrotor aircraft according to claim 1, wherein: The 13-dimensional state vector of the input layer includes the attitude angle, angular velocity, speed tracking error, yaw error and integral terms corresponding to the above errors of the aircraft; wherein the error is the deviation between the current flight state and the desired state.
6. The intelligent flight control method of a quad tiltrotor aircraft according to claim 5, wherein: The error integral term is calculated by exponentially decaying weighting of the error sequence, and the weighted sequence is smoothed to suppress systematic bias and avoid numerical divergence caused by error accumulation.
7. The intelligent flight control method of a quad tiltrotor aircraft according to claim 1, wherein: S3 uses the near-end policy optimization algorithm to train the intelligent controller, including: Defining the state set as a 13-dimensional input state and the action set as a 19-dimensional continuous control quantity; Constructing an Actor-Critic framework based on policy network and value network; Using a truncated objective function, multi-step advantage estimation and adaptive KL penalty term for policy update; Cycle sampling and updating in the dynamics simulation environment until the policy converges.
8. The intelligent flight control method of a quad tiltrotor aircraft according to claim 1, wherein: The reward function includes the following parts: Survival reward for encouraging the aircraft to maintain stable flight; Speed tracking error cost for punishing the deviation between the current speed and the desired speed; Control action cost, used to penalize the sum of squares of the nineteen control commands, to suppress control energy consumption and encourage action smoothness; Flight stability cost, used to suppress attitude jitter caused by too large angular velocity changes; Error integral term cost, used to suppress steady-state error; Yaw error cost, used to penalize the deviation of the current heading from the target heading; wherein adjustable weights are set for each of the above reward terms to adapt to different flight tasks.
9. The intelligent flight control method of a quad tilt-rotor aircraft according to claim 1, wherein the controller trained through reinforcement learning can realize continuous control of attitude, velocity and heading in rotor mode, fixed-wing mode and tilt transition mode, has self-adaptive compensation capability for aerodynamic interference and nonlinear coupling, and can maintain flight stability within the entire flight envelope.