Hypersonic variable wingspan aircraft attitude control method and system based on reinforcement learning

Abstract

The invention discloses a hypersonic variable wingspan aircraft attitude control method based on reinforcement learning, aiming at the characteristics of complex flight environment, strong nonlinearity of control input and aerodynamic characteristics, strong coupling and the like of a hypersonic variable configuration aircraft. The method comprises the following steps: acquiring discrete relationships between variables such as wingspan length, rudder deflection angle, flight attitude and flight speed and aerodynamic parameters through a computational fluid mechanics method, and then fitting the discrete relationships by utilizing a neural network method to realize aerodynamic parameter identification under different flight working conditions and appearance conditions; secondly, establishing a dynamic model of the hypersonic variable-wingspan aircraft by using an aerodynamic parameter identification network; and finally, realizing attitude control of the hypersonic variable-wingspan aircraft based on a DQN reinforcement learning method. The method can effectively solve the problems that a traditional control method is complex in controller design and strict in model precision requirement, and simulation experiment results prove that the method can effectively achieve attitude angle tracking.

Description

Technical Field

[0001] The present invention relates to the field of aircraft control technology, and in particular to a hypersonic variable-span aircraft attitude control method and system based on reinforcement learning. Background Art

[0002] Hypersonic aircraft have the advantages of high speed, long range, and wide range of maneuvers, making it one of the current research hotspots. However, the existing fixed-shape aerodynamic layout makes it impossible for aircraft flying across large airspaces and full-speed ranges to achieve optimal performance throughout the entire flight phase. By introducing deformation technology in hypersonic aircraft, the performance limitations of existing gliding aircraft can be effectively improved, so that the potential of the aircraft can be better utilized. In recent years, due to the complex flight environment and flight missions, the demand for aircraft with mission flexibility and versatility, as well as the progress of new materials, new structures and other technologies, hypersonic deformable aircraft that can adaptively change their shape according to the flight environment, mission requirements, etc. and always have the best flight performance have received great attention from major countries.

[0003] However, compared with traditional aircraft, hypersonic aircraft have significant differences in flight envelope, aerodynamic shape and kinematic characteristics, and there are strong coupling, strong nonlinearity and strong uncertainty between their aerodynamics, motion, structure and control. This characteristic is further exacerbated after considering the variable configuration, which makes the attitude control problem of hypersonic variable-wingspan aircraft extremely challenging.

[0004] On the other hand, while the deformable capability brings performance improvement to hypersonic aircraft, it also puts forward new requirements for its attitude control: hypersonic variant aircraft needs to have autonomous deformation capability in complex battlefield environments; the deformation process makes the aircraft system present multi-modal, strong nonlinear and strong coupling characteristics; the aircraft is extremely susceptible to various internal and external disturbances during the deformation flight process, and the flight stability is difficult to guarantee; and due to the increase in control mechanisms, it is necessary to solve complex control allocation problems. The traditional controller design method for fixed-shape aircraft has been difficult to meet the needs of hypersonic variant aircraft. The powerful learning ability and nonlinear expression ability of reinforcement learning provide new ideas for the design of variant aircraft systems. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art. The present invention proposes a hypersonic variable-span aircraft attitude control method based on reinforcement learning. The control method has good attitude tracking capability and is used to solve the shortcomings of traditional control methods, such as complex controller design and strict requirements on model accuracy.

[0006] The present invention provides a hypersonic variable-span aircraft attitude control method based on reinforcement learning, comprising the following steps:

[0007] Step 1: Solve the Kenavi-Stokes equations through CFD to calculate the aerodynamic loads of the hypersonic vehicle under different flight conditions and wingspan conditions;

[0008] Step 2: Establish a radial basis function neural network (RBF), and fit the aerodynamic loads obtained in step 1 through the RBF network to achieve aerodynamic parameter identification of the hypersonic variable-span aircraft;

[0009] Step 3: Based on the trained RBF network, a longitudinal dynamic model of the hypersonic variable-span aircraft is established;

[0010] Step 4: Construct a DQN reinforcement learning network based on the longitudinal dynamics model, and use the DQN reinforcement learning network to perform attitude control of the hypersonic variable-span aircraft.

[0011] Furthermore, the specific operation steps of step 2 are:

[0012] Step 2.1: Based on the obtained aerodynamic load data, construct the RBF neural network structure;

[0013] Step 2.2: According to the established RBF network, the aerodynamic load data of the hypersonic variable-span aircraft calculated in step 1 is used as a training sample, and the discrete samples are fitted through the network to obtain the continuous relationship between variables such as flight conditions, wingspan length, and rudder deflection and aerodynamic parameters.

[0014] Furthermore, the specific steps of step 2.1 include:

[0015] Step 2.1.1: Take the aircraft wingspan, rudder angle, flight speed, and flight angle of attack as the input of the radial basis function neural network RBF input layer, and take the lift coefficient C L , drag coefficient C D , pitch moment coefficient C M and thrust coefficient C T As the output of the output layer, several nodes are set in the hidden layer, and each node is the Gaussian activation function of the RBF network:

[0016]

[0017] Among them, x p is the pth input variable of the network input layer, c i is the mean of the Gaussian basis function of the i-th neuron node in the hidden layer of the network, is the variance of the Gaussian function of the i-th neuron node in the hidden layer of the network;

[0018] And the standard deviation σ of the Gaussian function i for:

[0019]

[0020] Among them, c max is the maximum distance between the centroids of h Gaussian functions;

[0021] Step 2.1.2: Express the output of the RBF network output layer as:

[0022]

[0023] Among them, w ij represents the connection weight between the i-th neuron in the hidden layer and the j-th neuron in the output layer;

[0024] The calculation formula of the connection weight is:

[0025]

[0026] Furthermore, the longitudinal dynamic model established in step 3 is:

[0027]

[0028] Wherein, V, h, γ, α, q represent the speed, altitude, track angle, angle of attack and pitch velocity of the aircraft respectively, m represents the mass of the aircraft, Iyy represents the moment of inertia, g represents the acceleration of gravity, L, D, T, M represent the lift, drag, thrust and pitch moment respectively;

[0029] In the formula,

[0030]

[0031] in, S, z T , is the flight pressure, reference area, thrust arm, and mean aerodynamic chord length; C L , C D , C M , C T They represent lift coefficient, drag coefficient, pitch moment coefficient and thrust coefficient respectively;

[0032] The fitting expressions of lift coefficient, drag coefficient, pitch moment coefficient and thrust coefficient in the above formula are:

[0033] C force(i) =Net force(i) (α,l,δ e )

[0034] Among them, C force(i) is the aerodynamic coefficient C L , C D , C M, C T ;Net force(i) is the RBF network generated corresponding to each aerodynamic coefficient.

[0035] Furthermore, the steps of step 4 include:

[0036] Step 4.1: Set up a reinforcement learning environment, describe the state of the aircraft at each moment through the longitudinal dynamics model, set the pitch angle and pitch angle tracking error as state observations, set the wingspan length, rudder angle and throttle opening as actions, and determine the reward function and training truncation based on the attitude angle controllable threshold and attitude angle tracking error;

[0037] Step 4.2: Build the DQN deep learning framework by setting up the agent to construct the DQN deep learning network;

[0038] Step 4.3: Use a deep learning algorithm to train the DQN deep learning network, and use the trained DQN deep learning network to interact with the reinforcement learning environment to perform attitude control of a hypersonic variable-span aircraft.

[0039] Furthermore, the DQN deep learning framework constructed in step 4.2 includes:

[0040] Current value network, which is used to approximate the Q-value function using a convolutional neural network or a fully connected neural network;

[0041] The experience replay memory area is used to store the agent's experience in the environment. Whenever the agent interacts with the environment, the experience tuple is stored in the experience replay buffer;

[0042] The target value network has the same structure as the current value network and is used to copy the parameters of the current value network every N time steps to calculate the target Q value;

[0043] The DQN error function measures the difference between the Q value predicted by the Q network and the target Q value.

[0044] A hypersonic variable-wingspan aircraft attitude control system based on reinforcement learning is implemented based on the hypersonic variable-wingspan aircraft attitude control method, and includes the following modules:

[0045] Aircraft aerodynamic load calculation module, used to solve the Kenavi-Stokes equations through CFD to calculate the aerodynamic loads of hypersonic aircraft under different flight conditions and wingspan conditions;

[0046] The aerodynamic parameter identification module is used to establish a radial basis function neural network (RBF), fit the aerodynamic load through the RBF network, and realize the aerodynamic parameter identification of the hypersonic variable-span aircraft;

[0047] The longitudinal dynamics model building module is used to build the longitudinal dynamics model of the hypersonic variable-span aircraft based on the trained RBF network;

[0048] The aircraft attitude control module is used to build a DQN reinforcement learning network based on the longitudinal dynamics model, and use the DQN reinforcement learning network to perform attitude control of a hypersonic variable-span aircraft.

[0049] Therefore, the present invention adopts the above-mentioned hypersonic variable-span aircraft attitude control method based on reinforcement learning, which has the following beneficial effects:

[0050] First, the present invention overcomes the shortcomings of insufficient calculation accuracy and limited application scope of aerodynamic engineering calculation methods by utilizing computational fluid technology, and avoids the problems of high calculation complexity and low calculation accuracy caused by multi-rigid body modeling through whole machine aerodynamic calculation;

[0051] Second, the present invention uses a neural network method to identify the aerodynamic coefficients of hypersonic variant aircraft, targeting the strong coupling and strong nonlinear characteristics between the aerodynamic shape, flight attitude and aerodynamic parameters of the hypersonic variant aircraft, and uses the powerful nonlinear fitting ability of the RBF neural network to effectively overcome the problems of weak nonlinear expression ability, poor description ability of parameter coupling relationship and low fitting accuracy of the traditional polynomial fitting method;

[0052] Third, the present invention uses a reinforcement learning method based on the DQN algorithm to achieve attitude control of the aircraft based on the reinforcement learning method, targeting the characteristics of hypersonic variant aircraft control, aerodynamics, strong coupling and strong nonlinearity, and effectively overcomes the problem of complex design of hypersonic aircraft attitude controller. Simulation experiments show the effectiveness of the method proposed in the present invention.

[0053] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A flow chart of the method proposed by the present invention;

[0055] Figure 2 It is the aerodynamic configuration of a hypersonic variable-span aircraft.

[0056] Figure 3 (a)-(d) are the aerodynamic coefficients of the aircraft at different wingspans.

[0057] Figure 4 (a)-(d) are the regression results of RBF network fitting sample data.

[0058] Figure 5 Schematic diagram of the reinforcement learning environment in the present invention.

[0059] Figure 6 Schematic diagram of the reinforcement learning framework in the present invention.

[0060] Figure 7 This is the attitude angle tracking process obtained by the present invention. DETAILED DESCRIPTION

[0061] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art make various changes or modifications to the present invention, and these equivalent forms also fall within the scope limited by the claims attached to the application.

[0062] 1. Attitude control method of hypersonic variable-wingspan aircraft based on reinforcement learning

[0063] like Figure 1 As shown, the method comprises the following steps:

[0064] Step 1: Use CFD software to solve the Reynolds-averaged (RANS) Navier-Stokes equations to obtain the aerodynamic loads on the aircraft surface. The equation can be expressed as the following integral form:

[0065]

[0066] Among them, dΩ is the volume element, dS is the area element, n is the normal vector of dS, Q is the conservative variable vector, F(Q,V grid ) and F V (Q) are the inviscid flux vector and the viscous flux vector respectively; V grid To control the volume speed.

[0067] The most commonly used method in computational fluid dynamics (CFD) is the finite volume (FV) method. FV is similar to the finite element method. First, the spatial region is discretized to form a three-dimensional grid or grid point, and then a suitable algorithm is applied to solve the equation of motion (Euler equation for inviscid fluids and Navier-Stokes equation for viscous fluids). Finally, the pressure and fluid velocity at each point in the space are obtained.

[0068] Specifically, the modeling software Catia V5R21 was used to create a three-dimensional model of the hypersonic variable-span aircraft. Then, Pointwise V17.1R3 was used to divide the flight surface and flow field into computational grids to discretize the computational space. Finally, the CFD software Ansys Fluent 15.0 was used to solve the lift, drag and pitch moment of the hypersonic aircraft during re-flight under different flight conditions and wingspan conditions.

[0069] Step 2: Use RBF neural network to fit the aerodynamic loads obtained in step 1 to realize the aerodynamic parameter identification of hypersonic variable-span aircraft;

[0070] RBF neural network belongs to the type of forward neural network, which is a three-layer forward network. The first layer is the input layer; the second layer is the hidden layer. The transformation function of the neurons in the hidden layer, namely the radial basis function, is a non-negative nonlinear function that is radially symmetric and decays to the center point. This function is a local response function, while the previous forward network transformation functions are all global response functions; the third layer is the output layer. The basic idea of ​​RBF network is: use RBF as the "basis" of the hidden unit to form the hidden layer space, and the hidden layer transforms the input vector, transforming the low-dimensional pattern input data into the high-dimensional space, so that the linearly inseparable problem in the low-dimensional space can be linearly separable in the commercial dimensional space.

[0071] The specific steps of step 2 include:

[0072] Step 2.1: Based on the obtained aerodynamic load data, construct the RBF neural network structure;

[0073] Specifically, first, the aircraft wingspan, rudder angle, flight speed, and flight angle of attack are used as the input of the network input layer, and the lift coefficient C L , drag coefficient C D , pitch moment coefficient C M and thrust coefficient C T As the output of the network output layer, the hidden layer sets h nodes, each node is a Gaussian activation function of the radial basis neural network (RBF), and the Gaussian activation function is:

[0074]

[0075] Among them, x p is the pth input variable of the network input layer, c i is the mean of the Gaussian basis function of the i-th neuron node in the hidden layer of the network, is the variance of the Gaussian function of the i-th neuron node in the hidden layer of the network;

[0076] Using the self-organizing center selection learning method, the variance of the Gaussian function is expressed as:

[0077]

[0078] Among them, c max is the maximum distance between the centroids of h Gaussian functions;

[0079] Secondly, according to the radial basis function network structure, the output of the RBF network is expressed as:

[0080]

[0081] Among them, w ij represents the connection weight between the i-th neuron in the hidden layer and the j-th neuron in the output layer. This value can be directly calculated using the least squares method, that is, solving the loss function with respect to w ij The partial derivative of , making it equal to 0, can be simplified to obtain the calculation formula of the connection weight:

[0082]

[0083] Step 2.2: According to the established RBF neural network, the aerodynamic characteristics data of the hypersonic variable-wingspan aircraft calculated by CFD are used as training samples. The discrete samples are fitted by training the RBF neural network to obtain the continuous relationship between the aircraft wingspan, rudder angle, flight speed, flight angle of attack and the lift coefficient, drag coefficient, pitch moment coefficient and thrust coefficient.

[0084] Step 3: Based on the RBF neural network trained in step 2.2, a longitudinal dynamic model of the hypersonic variable-span aircraft is established:

[0085]

[0086] Among them, V, h, γ, α, q represent the speed, altitude, track angle, angle of attack and pitch velocity of the aircraft respectively, m is the mass of the aircraft, Iyy is the moment of inertia (assuming that the change in wingspan does not affect the size of Iyy), g is the acceleration of gravity, L, D, T, M are aerodynamic forces, which represent lift, drag, thrust, pitch moment respectively, and the aerodynamic force expression is:

[0087]

[0088] in, S, z T , is the flight pressure, reference area, thrust arm, and mean aerodynamic chord length; C L , C D , C M , C T They represent lift coefficient, drag coefficient, pitch moment coefficient and thrust coefficient respectively;

[0089] Using the RBF neural network obtained by discrete sample training in step 2.2, the aerodynamic coefficient in equation (7) is expressed as:

[0090] C force(i) =Net force(i) (α,l,δ e )(8)

[0091] In the formula, Cforce(i) is the aerodynamic coefficient C L , C D , C M , C T , Net force(i) is the RBF neural network Net corresponding to each aerodynamic coefficient obtained in step 2.2 L 、Net D 、Net M and Net T .

[0092] Step 4: Perform attitude control of hypersonic variable-span aircraft based on DQN reinforcement learning method;

[0093] Set up the reinforcement learning environment, such as Figure 5 As shown in the figure, the hypersonic variable-wingspan aircraft is regarded as a reinforcement learning agent, and the DQN reinforcement learning framework is built through the agent setting, as shown in the figure. Figure 6 As shown; a DQN network model suitable for attitude control of hypersonic variable-span aircraft is trained through this reinforcement learning framework.

[0094] The specific steps of step 4 are:

[0095] Step 4.1: Set up the environment and incorporate the longitudinal dynamics model;

[0096] First, a reinforcement learning environment is set up, and then the longitudinal dynamics model established in step 3 is used in the reinforcement learning environment to describe the state of the aircraft at each moment;

[0097] Specifically, setting up the reinforcement learning environment includes: setting the state observation (Observation) to the pitch angle and pitch angle tracking error, and the action (Action) to the aircraft control input, including wingspan length, rudder angle and throttle opening; designing the reward function (Reward) and training truncation according to the attitude angle controllable threshold and attitude angle tracking error.

[0098] Step 4.2: Set up the agent;

[0099] The DQN deep learning framework is built by setting up the intelligent agent. The framework mainly includes the following modules:

[0100] ① Current value network (Q network), using convolutional neural network (CNN) or fully connected neural network to approximate the Q value function;

[0101] ② Experience replay memory area, which is used to store the agent's experience in the environment. Whenever the agent interacts with the environment, the experience tuple is stored in the experience replay buffer;

[0102] ③ Target value network: The target value network has the same structure as the current value network (Q network). It copies parameters from the current value network every N time steps and updates the target value network to calculate the target Q value.

[0103] ④DQN error function: The error function is used to measure the difference between the Q value predicted by the Q network and the target Q value.

[0104] Step 4.3: Initialize the current value network and the target value network, randomly generate weights θ, and let θ′=θ.

[0105] Step 4.4: Use the deep learning algorithm in the prior art (i.e., Algorithm 1) to train the target value network. The reinforcement learning process nests two loops, i.e., a total of M rounds of training are required, and each round of training requires T steps of iteration. The specific training iteration steps are:

[0106]

[0107] Step 4.5: Use the trained DQN model to interact with the environment to achieve attitude control of the hypersonic variable-span aircraft.

[0108] As a method for obtaining aerodynamic parameters, computational fluid dynamics (CFD) is used in step one to solve the aerodynamic parameters of a hypersonic variable configuration vehicle, which can effectively overcome the problem of low calculation accuracy of traditional engineering calculation methods, and can solve the aerodynamic force of the entire aircraft, avoiding the large calculation errors and high calculation complexity caused by rigid body modeling.

[0109] As a data fitting method, in step 2, based on the CFD calculation data in step 1, the RBF neural network is used to fit the aerodynamic coefficients of the hypersonic variable-wingspan aircraft. The RBF neural network has a strong nonlinear fitting ability. Using the RBF neural network to fit the aerodynamic parameters of the hypersonic variable-wingspan aircraft can effectively overcome the problems of weak nonlinear expression ability, poor description ability of parameter coupling relationship and low fitting accuracy in the traditional polynomial fitting method.

[0110] As an intelligent attitude control method, step four is based on the hypersonic variable-span aircraft dynamics model established in step three. In view of the strong coupling and strong nonlinearity of control, aerodynamics, and motion of hypersonic variable-span aircraft, the attitude control of the aircraft is realized based on the DQN reinforcement learning algorithm, which effectively overcomes the complex design problem of hypersonic aircraft attitude controller.

[0111] Example

[0112] In order to verify the effect of the method proposed in the present invention, a simulation experiment is carried out according to the following steps.

[0113] First, CFD technology is used to solve the Navier-Stokes equations based on Reynolds average (RANS) to obtain Figure 2 The aerodynamic loads on the surface of the hypersonic variable-span aircraft are shown in the calculation results. Figure 3 shown.

[0114] Secondly, the RBF neural network technology is used to fit the above calculation results to realize the aerodynamic parameter identification of hypersonic variable-span aircraft.

[0115] Specifically, the four variables of aircraft wingspan, rudder angle, flight speed, and flight angle of attack are used as the network input layer, and the lift coefficient C L , drag coefficient C D , pitch moment coefficient C M The three aerodynamic parameters are used as the network output layer, and a total of 20 hidden layer nodes are set. The regression of the fitted data of the aerodynamic parameters fitting through the RBF network is shown in the figure. Figure 4 shown.

[0116] Secondly, based on the aerodynamic parameter identification network, the longitudinal dynamic model of the hypersonic variable-span aircraft is established, as shown in formula (6).

[0117] Finally, based on the DQN reinforcement learning method (such as Figure 6 ), to achieve pitch attitude angle tracking of hypersonic variable-span aircraft,

[0118] Figure 7 The attitude angle tracking process is obtained. It can be seen from the figure that when the initial pitch angle of the aircraft is 0° and the target pitch angle is 3°, the aircraft is controlled by the method proposed in this patent, and the attitude angle tracking can be achieved in a relatively short time (7s), indicating that the method proposed in the present invention can achieve the expected attitude control effect.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A hypersonic variable-span aircraft attitude control method based on reinforcement learning, characterized in that: The following steps are involved: Step 1: Solve the Kenavi-Stokes equations through CFD to calculate the aerodynamic loads of the hypersonic vehicle under different flight conditions and wingspan conditions; Step 2: Establish a radial basis function neural network (RBF), and fit the aerodynamic loads obtained in step 1 through the RBF network to achieve aerodynamic parameter identification of the hypersonic variable-span aircraft; Step 3: Based on the trained RBF network, a longitudinal dynamic model of the hypersonic variable-span aircraft is established; Step 4: Construct a DQN reinforcement learning network based on the longitudinal dynamics model, and use the DQN reinforcement learning network to perform attitude control of the hypersonic variable-span aircraft.

2. A hypersonic variable-span aircraft attitude control method based on reinforcement learning as claimed in claim 1, characterized in that: The specific steps for step 2 are: Step 2.1: Based on the obtained aerodynamic load data, construct the RBF neural network structure; Step 2.2: According to the established RBF network, the aerodynamic load data of the hypersonic variable-span aircraft calculated in step 1 is used as a training sample, and the discrete samples are fitted through the network to obtain the continuous relationship between variables such as flight conditions, wingspan length, and rudder deflection and aerodynamic parameters.

3. A hypersonic variable-span aircraft attitude control method based on reinforcement learning as claimed in claim 2, characterized in that: The specific steps of step 2.1 include: Step 2.1.1: Take the aircraft wingspan, rudder angle, flight speed, and flight angle of attack as the input of the radial basis function neural network RBF input layer, and take the lift coefficient C L , drag coefficient C D , pitch moment coefficient C M and thrust coefficient C T As the output of the output layer, several nodes are set in the hidden layer, and each node is the Gaussian activation function of the RBF network: Among them, x p is the pth input variable of the network input layer, c i is the mean of the Gaussian basis function of the i-th neuron node in the hidden layer of the network, σ i 2 is the variance of the Gaussian function of the i-th neuron node in the hidden layer of the network; And the standard deviation σ of the Gaussian function i for: Among them, c max is the maximum distance between the centroids of h Gaussian functions; Step 2.1.2: Express the output of the RBF network output layer as: Among them, w ij represents the connection weight between the i-th neuron in the hidden layer and the j-th neuron in the output layer; The calculation formula of the connection weight is:

4. A hypersonic variable-span aircraft attitude control method based on reinforcement learning as claimed in claim 3, characterized in that: The longitudinal dynamic model established in step 3 is: Wherein, V, h, γ, α, q represent the speed, altitude, track angle, angle of attack and pitch velocity of the aircraft respectively, m represents the mass of the aircraft, Iyy represents the moment of inertia, g represents the acceleration of gravity, L, D, T, M represent the lift, drag, thrust and pitch moment respectively; In the formula, in, S, z T , is the flight pressure, reference area, thrust arm, and mean aerodynamic chord length; C L , C D , C M , C T They represent lift coefficient, drag coefficient, pitch moment coefficient and thrust coefficient respectively; The fitting expressions of lift coefficient, drag coefficient, pitch moment coefficient and thrust coefficient in the above formula are: C force(i) =Net force(i) (a,l,d) e ) Among them, C force(i) is the aerodynamic coefficient C L , C D , C M , C T ;Net force(i) is the RBF network generated corresponding to each aerodynamic coefficient.

5. A hypersonic variable-span aircraft attitude control method based on reinforcement learning as claimed in claim 4, characterized in that: The steps in Step 4 include: Step 4.1: Set up the reinforcement learning environment, set the pitch angle and pitch angle tracking error as state observations, set the wingspan length, rudder angle and throttle opening as actions, and then determine the reward function and training truncation according to the attitude angle controllable threshold and attitude angle tracking error, and describe the state of the aircraft at each moment through the longitudinal dynamics model; Step 4.2: Build the DQN deep learning framework by setting up the agent to construct the DQN deep learning network; Step 4.3: Use a deep learning algorithm to train the DQN deep learning network, and use the trained DQN deep learning network to interact with the reinforcement learning environment to perform attitude control of a hypersonic variable-span aircraft.

6. A hypersonic variable-span aircraft attitude control method based on reinforcement learning as claimed in claim 4, characterized in that: The DQN deep learning framework constructed in step 4.2 includes: Current value network, which is used to approximate the Q-value function using a convolutional neural network or a fully connected neural network; The experience replay memory area is used to store the agent's experience in the environment. Whenever the agent interacts with the environment, the experience tuple is stored in the experience replay buffer; The target value network has the same structure as the current value network and is used to copy the parameters of the current value network every N time steps to calculate the target Q value; The DQN error function measures the difference between the Q value predicted by the Q network and the target Q value.

7. A hypersonic variable-span aircraft attitude control system based on reinforcement learning, implemented based on the method described in any one of claims 1 to 6, characterized in that: Includes the following modules: Aircraft aerodynamic load calculation module, used to solve the Kenavi-Stokes equations through CFD to calculate the aerodynamic loads of hypersonic aircraft under different flight conditions and wingspan conditions; The aerodynamic parameter identification module is used to establish a radial basis function neural network (RBF), fit the aerodynamic load through the RBF network, and realize the aerodynamic parameter identification of the hypersonic variable-span aircraft; The longitudinal dynamics model building module is used to build the longitudinal dynamics model of the hypersonic variable-span aircraft based on the trained RBF network; The aircraft attitude control module is used to build a DQN reinforcement learning network based on the longitudinal dynamics model, and use the DQN reinforcement learning network to perform attitude control of a hypersonic variable-span aircraft.

Images