A Plasma Virtual Control Surface and Control Mechanism Allocation Method for High-Speed ​​Vehicle Reentry Flight

Abstract

This invention provides a plasma virtual control surface and control mechanism allocation method for high-speed aircraft reentry flight. First, it determines whether the aircraft is entering the reentry phase. Second, it calculates the required torque based on the attitude controller. Then, it designs control allocation models for the plasma active flow control system, aerodynamic control surface mechanism, and RCS thrust reverse control system based on a deep deterministic policy gradient reinforcement learning algorithm. The high-speed aircraft completes reentry flight based on the control allocation parameters obtained from the control allocation model. This invention utilizes plasma jets for auxiliary control, providing a new approach to precise aircraft control. This method can rationally allocate control measures according to the early, middle, and late stages of reentry flight, and clarify the relevant parameters of each control system. By using reinforcement learning algorithms to handle the nonlinear characteristics of the plasma actuator input power and control output, as well as the nonlinearity and uncertainty of the aerodynamic characteristics in the attitude control model, it can improve the stability of the control system and has good application prospects.

Description

Technical Field

[0001] This invention relates to the field of high-speed aircraft reentry flight control allocation, and specifically to a plasma virtual control surface and control mechanism allocation method for high-speed aircraft reentry flight. Background Technology

[0002] Research on high-speed vehicle reentry is an important means to advance aerospace technology and expand our understanding of nature. However, high-speed vehicle reentry also faces many challenges, such as the extremely high temperatures posing a severe challenge to thermal protection technology, the complex aerodynamic environment increasing the difficulty of navigation and control, and the high research and development costs also restricting its development.

[0003] Plasma jets can provide auxiliary control for high-speed aircraft. By adjusting the jet parameters, the flow field around the aircraft can be altered, enabling attitude adjustments. During flight, this enhances the aircraft's stability and maneuverability, allowing it to cope with complex flight environments and providing a new approach to precise control of high-speed aircraft, thereby improving its performance. However, the introduction of auxiliary control inevitably brings control allocation problems. In the early, middle, and late stages of reentry flight, the required control methods need to be implemented through different operating modes, but the transition process between modes is difficult to control, and the timing of the activation of various control methods is hard to accurately control. In addition, the excitation power and control output of plasma exhibit significant nonlinear characteristics, and the attitude control model is also a simplified uncertain model, which itself has strong nonlinearity with respect to aerodynamic characteristics. Summary of the Invention

[0004] The technical problem to be solved:

[0005] The technical problem this invention aims to solve is how to rationally allocate control measures based on the nonlinear characteristics of plasma excitation power and control output, as well as the nonlinearity and uncertainty of aerodynamic characteristics, and effectively address the control challenges during the transition process between modes.

[0006] To address the aforementioned technical problems, this invention proposes a plasma virtual control surface and control mechanism allocation method for high-speed aircraft reentry flight. By using reinforcement learning algorithms to handle the nonlinear relationships and uncertainties, the method achieves attitude control of the high-speed aircraft during the reentry phase and improves the stability of the control system.

[0007] The technical solution of this invention is as follows:

[0008] A method for controlling and allocating plasma virtual control surfaces and control mechanisms for high-speed aircraft reentry flight includes the following steps:

[0009] Step 1: Determine the reentry flight phase that the aircraft will enter based on the flight status parameters; the reentry flight phase is divided into three stages: early reentry flight, mid reentry flight, and late reentry flight.

[0010] Step 2: Calculate the desired torque M based on the aircraft's attitude angular velocity;

[0011] Step 3: Design control allocation models for the plasma active flow control system, aerodynamic control surface mechanism, and RCS reverse thrust control system based on deep deterministic policy gradient reinforcement learning algorithm;

[0012] Step 4: The aircraft completes reentry flight based on the control allocation parameters obtained from the control allocation model.

[0013] Furthermore, during the initial stage of reentry flight, the flight control actuators consist solely of the RCS thrust reverser system;

[0014] During the mid-stage of reentry flight, the flight control actuator consists of a hybrid structure of three different types of structures: aerodynamic control surfaces, RCS reverse thrust control system, and plasma active flow control system.

[0015] During the later stages of reentry flight, the flight control actuators consist of aerodynamic control surfaces and a plasma active flow control system.

[0016] Furthermore, in step 2, according to the formula

[0017]

[0018] Calculate the desired torque M, where ω = [p, q, r] T It is the vector representation of the aircraft's attitude angular rate, where I is the inertial tensor matrix.

[0019] Furthermore, the process of establishing the control allocation model in step 3 is as follows:

[0020] Set the agent structure: The agent is a policy model based on deep deterministic policy gradient;

[0021] Setting environmental conditions: The environmental conditions include the aircraft's pitch angle, yaw angle, roll angle, as well as flight speed, angle of attack, and flight altitude;

[0022] Set the agent action: The agent action is the control parameter of the plasma active flow control system, the aerodynamic control surface mechanism and the RCS reverse thrust control system.

[0023] The state transition process is set up: the state transition process is based on the 6-DOF model of the aircraft;

[0024] The reward mechanism is set up as follows: The reward mechanism is calculated according to different flight stages. The absolute value of the difference between the control torque generated by the output action and the desired torque is taken as the first reward value. The absolute value of the difference between the control parameters of the plasma active flow control system, the aerodynamic control surface mechanism and the RCS reverse thrust control system and the physical constraints is taken as the remaining reward values.

[0025] Set up an experience pool and use the time difference as the model objective function.

[0026] Furthermore, the agent is composed of a multi-layer fully connected network, with the input and output dimensions determined by the environmental state and the agent's actions, respectively, and the intermediate hidden layer is set as a multi-layer fully connected neural network.

[0027] Furthermore, the actions of the intelligent agent are the control parameters of the plasma active flow control system, the aerodynamic control surface mechanism, and the RCS reverse thrust control system; wherein the control parameters of the plasma active flow control system include the input power w = [wp1, ... wp] of each channel plasma actuator. n ] T wp1 is the input power of the first-channel plasma actuator in the plasma active flow control system; the control parameters of the aerodynamic control surface mechanism include the deflection angle δ = [δ1,…,δ] of each aerodynamic control surface. m ] T δ1 is the deflection angle of the first aerodynamic control surface in the aerodynamic control mechanism; the control parameters of the RCS thrust reverser control system include the proportional coefficients P = [p1,…,p] of each nozzle. l ] T p1 is the proportional coefficient of the first nozzle in the thrust reverser control system.

[0028] Furthermore, the reward mechanism calculation formula is as follows:

[0029] Reward=Reward1+Reward2+Reward3+Reward4

[0030] Reward1=-‖MM action *mask i ||, i = {1, 2, 3}

[0031] mask1 = {1 1*n ,0 1*m ,0 1*l}

[0032] mask2 = {1 1*n ,1 1*m ,1 1*l}

[0033] mask3 = {0 1*n,1 1*m ,1 1*l}

[0034] Reward2=-‖δ-D A ||

[0035] Reward3=-‖PD R ||

[0036] Reward4=-‖wD pl ||

[0037] Where Reward represents the reward value, Reward1 is the first reward value, Reward2 is the reward value corresponding to the aerodynamic control surface mechanism, Reward3 is the reward value corresponding to the RCS reverse thrust control system, Reward4 is the reward value corresponding to the plasma active flow control system, mask1 is the control state vector in the early stage of reentry flight, mask2 is the control state vector in the middle stage of reentry flight, and mask3 is the control state vector in the late stage of reentry flight. A D R and D pl These are the physical constraints on the aerodynamic control surface deflection angle, the RCS thrust reverser control nozzle proportional coefficient, and the plasma actuator input power, respectively. action =B a U T B a U represents the control allocation matrix for the plasma active flow control system, the aerodynamic control surface mechanism, and the RCS reverse thrust control system, where U represents the agent's action.

[0038] Furthermore, the present invention also proposes an electronic device including a processor and a memory storing a program, the program including instructions that, when executed by the processor, cause the processor to perform the above-described method. A computer-readable storage medium storing a program including instructions that, when executed by the processor of an electronic device, cause the electronic device to perform the above-described method.

[0039] Beneficial effects

[0040] This invention proposes a plasma virtual control surface and control mechanism allocation method for high-speed aircraft reentry flight. Utilizing plasma jets for auxiliary control, it provides a new approach to precise aircraft control. This method can rationally allocate control measures according to the early, middle, and late stages of reentry flight, and clearly define the relevant parameters of each control system. By employing reinforcement learning algorithms to handle the nonlinear characteristics of the plasma actuator input power and control output, as well as the nonlinearity and uncertainty of the aerodynamic characteristics in the attitude control model, the stability of the control system can be improved, demonstrating promising application prospects.

[0041] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0042] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0043] Figure 1 This is a design flowchart of the present invention.

[0044] Figure 2 This is the training framework based on DDPG reinforcement learning involved in this invention.

[0045] Figure 3 This is a block diagram of the control system for the reentry flight phase of a high-speed aircraft equipped with a DDPG strategy network, which is involved in this invention. Detailed Implementation

[0046] The embodiments of the present invention are described in detail below. These embodiments are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0047] This embodiment mainly addresses the control allocation problem of the plasma active flow control system (i.e., plasma virtual control surface), aerodynamic control surface mechanism, and RCS thrust reverser control system during the reentry flight of a high-speed aircraft. A control allocation method for the plasma virtual control surface and control mechanism in high-speed aircraft reentry flight is proposed. First, the reentry flight phase of the aircraft is determined. Second, the required torque is calculated based on the attitude controller. Then, a control allocation model for the plasma active flow control system, aerodynamic control surface mechanism, and RCS thrust reverser control system is designed based on the Deep Deterministic Policy Gradient (DDPG) reinforcement learning algorithm. Finally, the high-speed aircraft completes reentry flight based on the control allocation parameters obtained from the control allocation model. Figure 1 As shown, the specific steps include:

[0048] Step 1: Determine the reentry flight phase that the aircraft will enter based on the flight status parameters; the reentry flight phase is divided into three stages: early reentry flight, mid reentry flight, and late reentry flight.

[0049] During the initial stage of reentry, the thin air results in low dynamic pressure and low control efficiency of the aerodynamic control surface mechanism. In order to ensure stable control of the high-speed aerospace vehicle in the early stage of reentry, aerodynamic control surfaces are not used, and the flight control actuator consists only of the RCS reverse thrust system.

[0050] During the mid-flight phase, the increased dynamic pressure leads to a continuous increase in the control efficiency of the aerodynamic operating mechanism. Therefore, a hybrid heterogeneous actuator structure is adopted for composite control of the aircraft during this stage. At this time, the flight control actuator consists of three heterogeneous hybrid structures: aerodynamic control surfaces, RCS reverse thrust control system, and plasma active flow control system.

[0051] During the later stages of reentry, the gradually increasing dynamic pressure enables the aerodynamic control surface mechanism to control the high-speed aerospace vehicle. Therefore, there is no need to control the RCS reverse thrust control system. Thus, the flight control actuator at this stage consists of aerodynamic control surfaces and a plasma active flow control system.

[0052] Step 2: Calculate the required torque based on the dynamic equations of the angular rate loop in the attitude controller of the high-speed aerospace vehicle.

[0053] Using the attitude angular velocity of a high-speed aircraft, according to the formula

[0054]

[0055] The required torque M, i.e. the desired torque, is calculated, where ω = [p, q, r]. T It is the vector representation of the aircraft's attitude angular rate, where I is the inertial tensor matrix.

[0056] Step 3: Design control allocation models for plasma active flow control system, aerodynamic control surface mechanism and RCS reverse thrust control system based on deep deterministic policy gradient (DDPG) reinforcement learning algorithm.

[0057] Reinforcement learning training environments can be built based on well-known simulators, referring to... Figure 2 The training framework based on DDPG reinforcement learning involved in this invention comprises four parts: environment, policy network, evaluation network, and experience pool. The environment provides the basic conditions for the high-speed aircraft to operate. The policy network outputs policies to guide the high-speed aircraft's flight. The evaluation network scores the actions of the policy. The policy network stores the current state, action, transition state, reward, and discount obtained at each step in the experience pool. At regular intervals, the policy network and the target policy network, as well as the action network and the target action network, perform soft updates. Then, the policy network and the action-value network copy weight parameters using the well-known AC reinforcement learning framework. The relevant model composition is designed as follows:

[0058] (1) Set up the agent structure

[0059] The agent is a policy model of DDPG, which consists of a multi-layer fully connected network. The input and output dimensions are determined by the environment state and the agent's actions, respectively. The intermediate hidden layer is set as a two-layer fully connected neural network with 128*128 nodes in each layer.

[0060] (2) Set the environment status

[0061] The environmental state s includes the aircraft's flight attitude data, including pitch angle, yaw angle, roll angle, as well as flight speed, angle of attack, and flight altitude.

[0062] (3) Set agent actions

[0063] The agent's actions are the control parameters of the plasma active flow control system, the aerodynamic control surface mechanism, and the RCS reverse thrust control system. The control parameters of the plasma active flow control system include the input power w = [wp1, ... wp] of each channel's plasma actuator. n ] T wp1 is the input power of the first-channel plasma actuator in the plasma active flow control system; the control parameters of the aerodynamic control surface mechanism include the deflection angle δ = [δ1,…,δ] of each aerodynamic control surface. m ] T δ1 is the deflection angle of the first aerodynamic control surface in the aerodynamic control mechanism; the control parameters of the RCS thrust reverser control system include the proportional coefficients P = [p1,…,p] of each nozzle. l ] T p1 is the proportional coefficient of the first nozzle in the thrust reverser control system; the combined actions of the above intelligent agent are as follows:

[0064] U = [w; δ; P]

[0065] In this embodiment, taking the mid-reentry flight phase of the aircraft as an example, this stage employs aerodynamic control surfaces (8 in number) + active plasma flow control (2 in number) + RCS (4 in number). The actions of the intelligent agent are as follows:

[0066] U=[wp1,wp2;δ1,…,δ8;p1,…,p4]

[0067] Each motion component is directly associated with the corresponding mechanism input, facilitating online learning.

[0068] (4) Set up the state transition process

[0069] State transition is a continuous transformation process of an intelligent agent moving in an environment. The state transition in this invention is obtained through a 6-DOF model of a high-speed aircraft. This model typically involves the translational dynamics of the aircraft's center of mass and the rotational dynamics around the center of mass. In general, lateral forces can be ignored compared to lift and drag. Therefore, the following nonlinear motion equations are used in the guidance method design:

[0070]

[0071] Where R is the distance from the Earth's center to the spacecraft, V is the spacecraft's velocity, σ is the spacecraft's roll angle, θ and φ are longitude and latitude respectively, γ and ψ are the spacecraft's track roll angle and heading angle respectively, g is the acceleration due to gravity, L is the lift force on the spacecraft, D is the drag force on the spacecraft, and ω... E ω is the angular velocity of rotation in the Earth coordinate system (the aircraft is flying at a high altitude, so the rotation of the Earth coordinate system needs to be considered), and m is the mass of the aircraft.

[0072] The calculation is performed by following the agent's output policy using a 6-DOF model. Under each policy, a corresponding state component can be output, thus providing the environmental state for each iteration.

[0073] (5) Set up a reward mechanism

[0074] The reward mechanism scores the aircraft's environmental conditions and the agent's output actions to evaluate the effectiveness of the strategy. The actions of this invention involve three parts: a plasma active flow control system, an aerodynamic control surface mechanism, and an RCS reverse thrust control system. Therefore, the reward is obtained by weighting these three parts together.

[0075] After the attitude controller calculates the required torque M, it needs to determine the actual actuator composition based on the flight mode to determine the actual flight torque. The entire flight process is divided into three stages: pre-reentry flight, mid-reentry flight, and post-reentry flight. The actuator compositions for these stages are pure RCS thrust reverse control, aerodynamic control surfaces + plasma active flow control + RCS thrust reverse control, and aerodynamic control surfaces + plasma active flow control, respectively. Therefore, this invention designs a mask array for torque calculation based on priority, dividing it into three groups corresponding to the three flight stages. After the strategy model outputs the action, different masks are used to calculate the control torque according to the flight stage. The absolute value of the difference between the calculated torque and the desired torque is inverted and used as the first reward value. The absolute values ​​of the differences between the control parameters of the plasma active flow control system, the aerodynamic control surface mechanism, and the RCS thrust reverse control system and their physical constraints are inverted and used as the remaining reward values. Specifically, as follows:

[0076] Reward=Reward1+Reward2+Reward3+Reward4

[0077] Reward1=-‖MM action *mask i ||, i = {1, 2, 3}

[0078] mask1 = {1 1*n ,0 1*m ,0 1*l}

[0079] mask2 = {1 1*n ,11*m ,1 1*l}

[0080] mask3 = {0 1*n ,1 1*m ,1 1*l}

[0081] Reward2=-‖δ-D A ||

[0082] Reward3=-‖PD R ||

[0083] Reward4=-‖wD pl ||

[0084] Among them, D A D R and D pl These are the physical constraints for the aerodynamic control surface deflection angle, the RCS thrust reverser nozzle proportional coefficient, and the plasma actuator input power. action =B a U T B a The control allocation matrix for the three types of heterogeneous actuators, that is, the matrix that maps the relationship between actual output and torque, is determined by the composition and internal principle of the mechanism. U is the action of the intelligent agent, that is, the actual output of the actuator.

[0085] (6) Set up an experience pool

[0086] In reinforcement learning, the experience pool plays a crucial role. It stores data generated from the interaction between the agent and the environment, and then uses it through random sampling. This breaks down data correlations and improves data utilization. The experience pool storage tuple of this invention contains... <s,a,s ′ ,r,τ>, where s is the environmental state, s ′ Let 'a' be the new state obtained through the state transition equation, 'r' be the output value of the policy model, 'r' be the immediate reward of the corresponding policy model, and 'τ' be the discounted reward. After batching the data from the experience pool, the time difference (TD-error) is used as the model objective function.

[0087] TD=r+ρV(s ′ )-V(s)

[0088] Where V(.) represents the state value, and ρ is a set coefficient. Then, the neuron weights in the network are trained using gradient descent backpropagation, and the weights are stored once every H iterations.

[0089] Regarding the extraction of experience pool samples, this invention employs a well-known priority experience pool algorithm. The experience pool samples are sorted according to their TD difference, and then 64 or 128 samples are selected from the top-ranked samples for extraction. This approach effectively improves convergence speed and reward value, significantly enhancing the agent's learning ability.

[0090] The final control allocation model is obtained by training the control allocation model constructed in the above process online.

[0091] Step 4: The high-speed aircraft completes reentry flight based on the control allocation parameters obtained from the control allocation model.

[0092] refer to Figure 3 First, the attitude controller calculates the required torque. Then, through the policy network of the trained DDPG agent, it outputs the corresponding policy control of the aerodynamic control surface deflection angle, the plasma jet exciter input power, and the on / off state of the RCS reverse thrust control system nozzle, thereby controlling the attitude stability of the high-speed aircraft and achieving full control of the early, middle, and late stages of reentry flight.

[0093] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention without departing from the principles and spirit of the present invention.

Claims

1. A method for control allocation of plasma virtual control surfaces and actuation mechanisms for high-speed vehicle reentry flight, characterized in that: Includes the following steps: Step 1: Determine the reentry flight phase that the aircraft will enter based on the flight status parameters; the reentry flight phase is divided into three stages: early reentry flight, mid reentry flight, and late reentry flight. During the initial stage of reentry flight, the flight control actuators consist only of the RCS reverse thrust system; During the mid-stage of reentry flight, the flight control actuator consists of a hybrid structure of three different types of structures: aerodynamic control surfaces, RCS reverse thrust control system, and plasma active flow control system. During the later stages of reentry flight, the flight control actuators consist of aerodynamic control surfaces and a plasma active flow control system. Step 2: Calculate the desired torque based on the aircraft's attitude angular velocity. ; Step 3: Design control allocation models for the plasma active flow control system, aerodynamic control surface mechanism, and RCS reverse thrust control system based on the deep deterministic policy gradient reinforcement learning algorithm; the process of establishing the control allocation model is as follows: Set the agent structure: The agent is a policy model based on deep deterministic policy gradient; Setting environmental conditions: The environmental conditions include the aircraft's pitch angle, yaw angle, roll angle, as well as flight speed, angle of attack, and flight altitude; Set the agent action: The agent action is the control parameter of the plasma active flow control system, the aerodynamic control surface mechanism and the RCS reverse thrust control system. The state transition process is set up: the state transition process is based on the 6-DOF model of the aircraft; The reward mechanism is set up as follows: The reward mechanism is calculated according to different flight stages. The absolute value of the difference between the control torque generated by the output action and the desired torque is taken as the first reward value. The absolute value of the difference between the control parameters of the plasma active flow control system, the aerodynamic control surface mechanism and the RCS reverse thrust control system and the physical constraints is taken as the remaining reward values. Set up an experience pool; and use the time difference as the model objective function; Step 4: The aircraft completes reentry flight based on the control allocation parameters obtained from the control allocation model.

2. The plasma virtual control surface and control mechanism allocation method for high-speed aircraft reentry flight according to claim 1, characterized in that: In step 2, according to the formula Calculate the desired torque ,in It is the vector representation of the aircraft's attitude angular rate. It is the inertia tensor matrix.

3. The plasma virtual control surface and control mechanism allocation method for high-speed aircraft reentry flight according to claim 1, characterized in that: The agent is composed of a multi-layer fully connected network. The input and output dimensions are determined by the environmental state and the agent's actions, respectively. The intermediate hidden layers are set as multi-layer fully connected neural networks.

4. The plasma virtual control surface and control mechanism allocation method for high-speed aircraft reentry flight according to claim 2, characterized in that: The actions of the intelligent agent are the control parameters of the plasma active flow control system, the aerodynamic control surface mechanism, and the RCS reverse thrust control system; wherein the control parameters of the plasma active flow control system include the input power of each channel plasma actuator. , The input power is the plasma actuator of the first channel in the plasma active flow control system; the control parameters of the aerodynamic control surface mechanism include the deflection angle of each aerodynamic control surface. , The deflection angle of the first aerodynamic control surface in the aerodynamic control mechanism; the control parameters of the RCS thrust reverser control system include the proportional coefficients of each nozzle. , This is the proportional coefficient of the first nozzle in the thrust reverser control system.

5. The plasma virtual control surface and control mechanism allocation method for high-speed aircraft reentry flight according to claim 4, characterized in that: The formula for calculating the reward mechanism is as follows: in As a reward value, The first reward value, This represents the reward value corresponding to the pneumatic control surface mechanism. This is the reward value corresponding to the RCS reverse thrust control system. This is the reward value corresponding to the plasma active flow control system. This is the control state vector during the initial stage of reentry flight. This is the control state vector during the mid-reentry flight. This is the control state vector during the later stages of reentry flight. , and These are the physical constraints on the aerodynamic control surface deflection angle, the RCS thrust reverser control nozzle proportional coefficient, and the plasma actuator input power. ,in The control allocation matrix is ​​used for the plasma active flow control system, the aerodynamic control surface mechanism, and the RCS reverse thrust control system. Actions of the intelligent agent.

6. An electronic device comprising a processor and a memory storing a program, the program comprising instructions, characterized in that: When executed by the processor, the instructions cause the processor to perform the method of any one of claims 1-5.

7. A computer-readable storage medium storing a program, the program comprising instructions, characterized in that: When the instructions are executed by the processor of the electronic device, the electronic device causes the electronic device to perform the method of any one of claims 1-5.

Images