Aircraft control methods, devices, equipment, and media based on deep reinforcement learning

Abstract

This invention relates to a method, apparatus, device, and medium for aircraft control based on deep reinforcement learning. The method includes: arranging jet exciters on the aircraft to form a three-axis attitude control system; acquiring the actual angular acceleration of the three axes in real time; designating any one of the three-axis attitude channels as the master control channel and determining the desired angular acceleration of the master control channel; constructing an accuracy constraint term using the desired angular acceleration and actual angular acceleration of the master control channel; constructing a decoupling constraint term by suppressing the actual angular acceleration of non-master control channels to zero; constructing an energy consumption term using the energy consumption corresponding to the driving voltage of each jet exciter; constructing a control objective loss function; training the aircraft control model using a deep reinforcement learning algorithm based on the control objective loss function to obtain a trained aircraft control model; and outputting voltage commands through the trained aircraft control model to control the operation of each jet exciter. This invention can achieve decoupled control of each attitude channel while simultaneously considering control accuracy and energy consumption.

Description

Technical Field

[0001] This invention relates to the field of aircraft control technology, and in particular to an aircraft control method, apparatus, device, and medium based on deep reinforcement learning. Background Technology

[0002] Attitude control of an aircraft is a core element ensuring stable flight and precise mission completion, directly impacting flight safety, maneuverability, and mission efficiency. Traditional aircraft attitude control schemes primarily rely on mechanical control surfaces to achieve pitch, roll, and yaw adjustments. The physical deflection of these mechanical surfaces alters the aerodynamic shape of the aircraft surface, thereby adjusting the flow field distribution and generating the required control torque. However, this type of mechanical control surface control has several drawbacks. First, the mechanical deflection of the control surfaces requires a certain response time, making it difficult to meet the rapid control demands of high-dynamic flight scenarios. Second, long-term operation of the mechanical structure is prone to wear, jamming, and other malfunctions, reducing system reliability. Furthermore, the deflection of the control surfaces themselves increases the aircraft's aerodynamic drag, adversely affecting flight energy efficiency.

[0003] To overcome the limitations of traditional mechanical control surfaces, jet-based flight control technology in the field of active flow control has gradually become an important development direction for aircraft attitude control. Currently, jet technologies used for aircraft attitude control mainly include steady jets, plasma jets, synthetic jets, and synthetic dual-jet technologies. Among them, synthetic dual-jet technology simplifies mechanical system design significantly because it does not rely on an additional air source device; it achieves fluid intake and exhaust solely through the periodic movement of internal cavities and diaphragms. Simultaneously, its unique configuration effectively improves energy utilization and enhances the control effect on local flow fields, demonstrating excellent application potential in aircraft attitude control.

[0004] However, current jet-based aircraft attitude control suffers from several drawbacks. The inherent interrelationships between the pitch, roll, and yaw control channels in terms of aerodynamic layout and flow field interaction mean that jet control actions in a single channel can easily cause unintended aerodynamic disturbances to other channels, leading to additional torques in non-target channels—a phenomenon known as multi-axis torque coupling. This coupling effect significantly reduces attitude control accuracy, increases flight maneuverability, and may even trigger attitude fluctuations in highly dynamic flight scenarios, limiting the engineering application and performance improvement of jet flight control technology across various aircraft. Summary of the Invention

[0005] Therefore, it is necessary to provide a deep reinforcement learning-based aircraft control method, device, equipment, and medium that can achieve decoupled control of each attitude channel while taking into account control accuracy and energy consumption, in order to address the above-mentioned technical problems.

[0006] A deep reinforcement learning-based aircraft control method, the method comprising:

[0007] A jet exciter is installed on the aircraft to form a three-axis attitude control; the actual angular acceleration of the three axes is acquired in real time through an airborne inertial measurement unit.

[0008] Assume any one of the three-axis attitude channels is the main control channel, and determine the desired angular acceleration of the main control channel; construct an accuracy constraint term using the desired angular acceleration and the actual angular acceleration of the main control channel; construct a decoupling constraint term by suppressing the actual angular acceleration of the non-main control channels to zero; construct an energy consumption term using the energy consumption corresponding to the driving voltage of each jet exciter; construct a control target loss function based on the accuracy constraint term, the decoupling constraint term, and the energy consumption term.

[0009] A deep reinforcement learning algorithm is used to minimize the loss function of the control target as the training objective. The parameters of the aircraft control model are iteratively updated until the convergence condition is met, and a well-trained aircraft control model is obtained.

[0010] The trained aircraft control model outputs voltage commands to control the operation of each jet exciter.

[0011] On the other hand, a deep reinforcement learning-based aircraft control device is also provided, comprising:

[0012] The exciter placement and data acquisition module is used to place jet exciters on the aircraft to form three-axis attitude control; the actual angular acceleration of the three axes is acquired in real time through the airborne inertial measurement unit.

[0013] The loss function construction module is used to set any one of the three-axis attitude channels as the main control channel, determine the desired angular acceleration of the main control channel; construct an accuracy constraint term using the desired angular acceleration and the actual angular acceleration of the main control channel; construct a decoupling constraint term by suppressing the actual angular acceleration of the non-main control channels to zero; construct an energy consumption term using the energy consumption corresponding to the driving voltage of each jet exciter; and construct a control target loss function based on the accuracy constraint term, the decoupling constraint term, and the energy consumption term.

[0014] The training module is used to employ a deep reinforcement learning algorithm to iteratively update the parameters of the aircraft control model with the goal of minimizing the control target loss function, until the convergence condition is met, thereby obtaining a trained aircraft control model.

[0015] The aircraft control module is used to control the operation of each jet exciter by outputting voltage commands through a trained aircraft control model.

[0016] On the other hand, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-mentioned deep reinforcement learning-based aircraft control method.

[0017] Furthermore, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the aforementioned deep reinforcement learning-based aircraft control method.

[0018] Compared with existing technologies, the deep reinforcement learning-based aircraft control method, apparatus, device, and medium provided by this invention have the following beneficial effects:

[0019] 1. By setting the main control channel, a precision constraint term is constructed between the desired angular acceleration and the actual angular acceleration of the main control channel, and a decoupling constraint term is constructed to suppress the angular acceleration of the non-main channel to zero. This enables the main channel angular acceleration to accurately track the desired angular acceleration, while the non-main channel angular acceleration remains zero, achieving decoupled control of multi-axis attitude. This avoids weakening of the main channel output, effectively eliminates attitude interference caused by multi-axis torque coupling in traditional jet control, significantly improves the attitude control accuracy of the aircraft, avoids attitude fluctuations caused by coupling effects, and achieves precise decoupled control of the aircraft's three-axis attitude.

[0020] 2. The loss function also incorporates the energy consumption corresponding to the driving voltage of each jet exciter to construct an energy consumption term. During reinforcement learning, it can automatically optimize the voltage allocation strategy of each exciter, minimizing the total energy consumption of each synthetic dual jet exciter while meeting the requirements of decoupling and accuracy, thus significantly improving energy utilization efficiency.

[0021] 3. By deploying jet exciters on the aircraft, the flow field is established rapidly, and dynamic compensation can be completed within a short time scale. Combined with the onboard inertial measurement unit to obtain the actual three-axis angular acceleration in real time, feedback data can be processed in a timely manner. Combined with the trained aircraft control model, it can meet the rapid control requirements in highly dynamic flight scenarios and improve the attitude control response speed and system reliability. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention, and those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating the deep reinforcement learning-based aircraft control method in Example 1.

[0024] Figure 2This is a schematic diagram of the upper working state of the pitch and dual-film three-cavity synthetic dual-jet actuator in Example 1;

[0025] Figure 3 This is a schematic diagram of the lower working state of the pitch and dual-film three-cavity synthetic dual-jet actuator in Example 1;

[0026] Figure 4 This is a schematic diagram of the single-film dual-cavity reverse jet exciter in Example 1.

[0027] Figure 5 This is a framework diagram of the deep reinforcement learning-based aircraft control method in Example 1;

[0028] Figure 6 This is a structural block diagram of the deep reinforcement learning-based aircraft control device in Example 2;

[0029] Figure 7 This is a diagram of the internal structure of the computer device in Example 3.

[0030] Explanation of reference numerals in the attached drawings: 1. First jet exciter; 2. Second jet exciter; 4. First jet; 5. Second jet; 6. Trailing edge of outer wing section; 7. First cavity; 8. Second cavity; 9. Piezoelectric diaphragm; 10. Coanda surface.

[0031] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0033] It is understood that the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0034] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0035] Example 1

[0036] like Figure 1 As shown, a deep reinforcement learning-based aircraft control method is provided, including the following steps:

[0037] Step 201: Deploy jet exciters on the aircraft to form three-axis attitude control; obtain the actual angular acceleration of the three axes in real time through the airborne inertial measurement unit.

[0038] Step 202: Assume any one of the three-axis attitude channels is the main control channel, and determine the desired angular acceleration of the main control channel; construct an accuracy constraint term using the desired angular acceleration and actual angular acceleration of the main control channel; construct a decoupling constraint term by suppressing the actual angular acceleration of the non-main control channels to zero; construct an energy consumption term using the energy consumption corresponding to the driving voltage of each jet exciter; construct the control target loss function based on the accuracy constraint term, the decoupling constraint term, and the energy consumption term.

[0039] Step 203: Using a deep reinforcement learning algorithm, with the goal of minimizing the control target loss function as the training objective, the parameters of the aircraft control model are iteratively updated until the convergence condition is met, thus obtaining the trained aircraft control model.

[0040] Step 204: The trained aircraft control model outputs voltage commands to control the operation of each jet exciter.

[0041] In the specific implementation of step 201, firstly, according to the functional division requirements of the aircraft's pitch, roll, and yaw three-axis attitude control, a double-membrane three-cavity synthetic double-jet exciter is arranged on the trailing edge of the aircraft's wing body to generate pitch control torque, denoted as the third jet exciter; a double-membrane three-cavity synthetic double-jet exciter is arranged on the trailing edge 6 of the outer wing section of the aircraft to generate roll control torque, denoted as the second jet exciter 2; and a single-membrane two-cavity reverse jet exciter is arranged in the middle of the aircraft's fuselage to generate yaw control torque, denoted as the first jet exciter 1, forming the actuator basis for three-axis attitude control. Among them, the dual-membrane three-cavity synthetic dual-jet exciter and the single-membrane dual-cavity reverse jet exciter are both driven by the piezoelectric diaphragm 9 to generate periodic jets. The excitation frequency range is 500-2000Hz, and the jet energy is linearly proportional to the exciter driving voltage. The dual-membrane three-cavity configuration changes the flow field circulation through the synergistic effect of the jets in the first cavity 7 and the second cavity 8. The single-membrane dual-cavity reverse jet configuration generates a virtual drag effect by simultaneously generating reverse jets on the upper and lower wing surfaces, respectively adapting to the torque requirements of different attitudes.

[0042] Specifically, such as Figure 2 As shown, the upper working state of the pitch and dual-diaphragm three-cavity synthetic dual-jet actuator is illustrated. Driven by the piezoelectric diaphragm 9, the first cavity 7 and the second cavity 8 generate the first jet 4 and the second jet 5, respectively. Due to the presence of the trailing edge Coanda surface 10, the first jet 4 becomes a wall-attached flow, effectively altering the circulation distribution. Furthermore, the presence of the second jet 5 provides assistance to the downstream jet, further influencing the local flow field and generating rolling torque and nose-down torque. Additionally... Figure 3The pitch and dual-film three-cavity synthetic dual-jet actuator's lower working states are demonstrated, along with its basic working principle. Figure 2 The same applies, so I won't go into details here. Figure 4 A schematic diagram of the single-film dual-cavity reverse jet exciter shows that, driven by the piezoelectric diaphragm 9, the upper and lower wing surfaces simultaneously generate the first jet 4 and the second jet 5 in opposite directions, thereby producing the effect of a virtual drag rudder, which is used to generate yaw control torque.

[0043] Then, the angular velocity signal of the aircraft is acquired in real time through the airborne inertial measurement unit (IMU). The acquired angular velocity signal is processed by differentiation to output the actual angular acceleration of the aircraft's pitch, roll and yaw axes, which serves as the feedback input data for subsequent attitude control. During the differentiation process, smoothing methods can be used to reduce data fluctuations and ensure the accuracy of the angular acceleration data.

[0044] Understandably, this step provides a functionally adapted actuator configuration and precise real-time data acquisition. On the one hand, it achieves a clear division of labor among the three-axis attitude control actuators, laying the hardware foundation for subsequent multi-channel decoupled control. On the other hand, it acquires and processes angular acceleration data in real time through the IMU, ensuring the timeliness and accuracy of control feedback and avoiding control deviations caused by data lag or errors.

[0045] In the specific implementation of step 202, according to the current flight mission requirements, any one of the pitch, roll and yaw attitude channels is set as the main control channel, and the desired angular acceleration of the main control channel is determined according to the flight performance indicators.

[0046] By using the deviation between the desired angular acceleration and the actual angular acceleration as the core constraint for control optimization, and introducing energy consumption factors, a deep reinforcement learning control allocation model is established. Considering real-world requirements, this invention uses angular acceleration as a direct constraint: ensuring that the angular acceleration of the main channel accurately tracks the desired value, while suppressing the angular acceleration of non-main channels to zero, thereby achieving decoupling of torque channels in different directions; simultaneously, under the above constraints, the energy consumption of the jet actuator is further minimized, achieving a control effect that prioritizes the main channel and optimizes energy efficiency.

[0047] Let the pitch, roll, and yaw angular accelerations be respectively... Let the main control channel index be... main Corresponding to pitch angle acceleration or roll acceleration or yaw acceleration ; The desired angular acceleration of the main channel, The actual angular acceleration of the main channel.

[0048] The actual angular acceleration of the main control channel Desired angular acceleration of the main control channel Construct a precision constraint term to minimize the error between the angular acceleration of the main channel and the desired angular acceleration, ensuring that the angular acceleration of the main channel accurately tracks the desired value.

[0049] The actual angular acceleration of the non-master control channel The decoupling constraint term is constructed by suppressing it to zero. The angular acceleration of the non-master channel is minimized by the decoupling constraint term and made as close to zero as possible to eliminate the coupling effect.

[0050] Total energy consumption corresponding to the driving voltage of each jet exciter An energy consumption term is constructed, and high energy consumption is penalized through this term to minimize the energy consumption of the jet actuator and ensure optimal energy efficiency in the control process.

[0051] Then, based on the accuracy constraint, decoupling constraint, and energy consumption term, the control objective loss function is constructed, with the following expression:

[0052] ;

[0053] In the formula, , , This represents the weighting factor used to balance accuracy and energy consumption, where, Used to ensure the control accuracy of the main channel. Used to eliminate multiaxial torque coupling Used to reduce control energy consumption. It's worth noting that this is typically... The value of is much greater than The value of is determined to ensure that the main channel takes priority.

[0054] It is understandable that this step, by constructing a control objective loss function that integrates accuracy, decoupling, and energy consumption, clarifies the control accuracy requirements of the main channel, specifically suppresses coupling interference from non-main channels, and also takes into account energy consumption optimization, providing a clear and comprehensive optimization objective for the subsequent training of deep reinforcement learning networks.

[0055] In the specific implementation of step 203, an aircraft control model is constructed, and the network input and output parameters are defined first. Among them, the state space is defined as the actual angular acceleration of the aircraft's pitch, roll, and yaw axes obtained by the inertial measurement unit, as well as the desired angular acceleration of the main control channel, to ensure that the network can perceive the current attitude state and control target in real time.

[0056] The aircraft control actions are continuous. The action space is defined as the continuous values ​​of the drive voltages of each jet exciter, adapting to the linear relationship between jet energy and voltage, to achieve continuous control of the exciter output torque. Each controller (such as pitch, roll, yaw) and its associated jet exciter voltage are set as actions, and reinforcement learning algorithms adjust these voltage values ​​through training.

[0057] Deep reinforcement learning algorithms employ the proximal policy optimization (PPO) algorithm. The PPO algorithm evaluates the current policy's performance in the environment and updates it based on three-axis angular acceleration feedback. For example... Figure 5 As shown, during the training process, the aircraft control model outputs voltage commands for each exciter based on state-space data. With the goal of minimizing the control target loss function, the parameters of the aircraft control model are adjusted through backpropagation and gradient optimization until the convergence condition is met. This allows the trained aircraft control model to effectively solve the problems of control accuracy and energy efficiency.

[0058] It is understandable that this step, through the stability of the PPO algorithm and the fitting ability of deep neural networks, enables the aircraft control model to automatically optimize the voltage distribution scheme during iterative training, while simultaneously meeting the multi-objective requirements of achieving the main channel accuracy target, decoupling non-main channels, and minimizing energy consumption. Furthermore, the network has a certain degree of adaptability, which can adapt to the attitude control requirements under different flight conditions.

[0059] In the specific implementation of step 204, during the actual flight of the aircraft, the actual pitch, roll, and yaw angular accelerations collected and processed in real time by the airborne inertial measurement unit are continuously input into the trained aircraft control model.

[0060] The trained aircraft control model outputs the target drive voltage command for each jet exciter in real time based on the input real-time three-axis actual angular acceleration and the learned optimization strategy.

[0061] The piezoelectric diaphragm 9 of the corresponding jet exciter is controlled according to the voltage command to generate the required vibration frequency and amplitude, enabling the exciter to produce a jet that meets the torque requirements. Through the coordinated work of each jet exciter, the angular acceleration of the main control channel is accurately tracked to the desired angular acceleration, while the angular acceleration of the non-main control channels remains at zero. At the same time, the total energy consumption of each exciter is kept to a minimum, completing the real-time decoupled control of the aircraft's attitude and improving the reliability of the attitude control system.

[0062] It should be understood that, although this embodiment Figure 1 The steps are shown sequentially as indicated by the arrows, but they are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order in which these steps are performed; they can be executed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0063] Example 2

[0064] Based on the deep reinforcement learning-based aircraft control method in Embodiment 1, this embodiment discloses a deep reinforcement learning-based aircraft control device, such as... Figure 6 As shown, the deep reinforcement learning-based aircraft control device includes: an exciter placement and data acquisition module 401, a loss function construction module 402, a training module 403, and an aircraft control module 404, wherein:

[0065] The exciter placement and data acquisition module 401 is used to place jet exciters on the aircraft to form three-axis attitude control; and to acquire the actual angular acceleration of the three axes in real time through the airborne inertial measurement unit.

[0066] The loss function construction module 402 is used to set any one of the three-axis attitude channels as the main control channel, determine the desired angular acceleration of the main control channel; construct an accuracy constraint term with the desired angular acceleration and actual angular acceleration of the main control channel; construct a decoupling constraint term with the actual angular acceleration of the non-main control channels suppressed to zero; construct an energy consumption term with the energy consumption corresponding to the driving voltage of each jet exciter; and construct the control target loss function based on the accuracy constraint term, the decoupling constraint term, and the energy consumption term.

[0067] The training module 403 is used to employ a deep reinforcement learning algorithm to iteratively update the parameters of the aircraft control model with the goal of minimizing the control target loss function, until the convergence condition is met, thereby obtaining a trained aircraft control model.

[0068] The aircraft control module 404 is used to output voltage commands through the trained aircraft control model to control the operation of each jet exciter.

[0069] In this embodiment, the specific working process and working principle of the exciter placement and data acquisition module 401, the loss function construction module 402, the training module 403, and the aircraft control module 404 are the same as those in Embodiment 1, and therefore will not be described again in this embodiment. Each unit module can be implemented entirely or partially through software, hardware, or a combination thereof. Each unit module can be embedded in or independent of the processor in a computer device in hardware form, or it can be stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to the above unit modules.

[0070] Example 3

[0071] like Figure 7 The diagram illustrates a terminal device disclosed in this embodiment, comprising a transmitter, a receiver, a memory, and a processor. The transmitter transmits instructions and data, the receiver receives instructions and data, the memory stores computer-executed instructions, and the processor executes the computer-executed instructions stored in the memory to implement the method described in Embodiment 1 above.

[0072] It is important to note that the aforementioned memory can be either standalone or integrated with the processor. When the memory is set up independently, the terminal device also includes a bus for connecting the memory and the processor.

[0073] Example 4

[0074] This embodiment discloses a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the method in Embodiment 1 above.

[0075] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0076] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0077] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A method for aircraft control based on deep reinforcement learning, characterized in that, The method comprises: arranging a fluidic exciter on an aircraft to form three-axis attitude control; acquiring three-axis actual angular acceleration in real time through an on-board inertial measurement unit; setting any one of the three-axis attitude channels as a main control channel, determining the expected angular acceleration of the main control channel; constructing a precision constraint term with the expected angular acceleration and the actual angular acceleration of the main control channel; constructing a decoupling constraint term with the actual angular acceleration of the non-main control channel being zero, and constructing an energy consumption term with the energy consumption corresponding to the driving voltage of each fluidic exciter; constructing a control target loss function according to the precision constraint term, the decoupling constraint term and the energy consumption term; using a deep reinforcement learning algorithm, taking minimizing the control target loss function as the training target, iteratively updating the parameters of the aircraft control model until the convergence condition is met, and obtaining the trained aircraft control model; outputting the voltage command through the trained aircraft control model to control the operation of each fluidic exciter; arranging a fluidic exciter on an aircraft to form three-axis attitude control, comprising: arranging a double-membrane three-cavity synthetic double fluidic exciter on the wing-body tail edge of the aircraft to generate a pitch control moment; arranging a double-membrane three-cavity synthetic double fluidic exciter on the outer wing segment tail edge of the aircraft to generate a roll control moment; arranging a single-membrane double-cavity reverse fluidic exciter in the middle of the aircraft body to generate a yaw control moment.

2. The deep-reinforcement-learning-based aircraft control method according to claim 1, characterized in that, acquiring three-axis actual angular acceleration in real time through an on-board inertial measurement unit, comprising: acquiring the angular velocity signal of the aircraft in real time through the on-board inertial measurement unit, performing differential operation processing on the angular velocity signal, and outputting the actual angular acceleration of the three-axis pitch, roll and yaw of the aircraft.

3. The deep-reinforcement-learning-based aircraft control method according to claim 1 or 2, characterized in that The control target loss function expression is: ； wherein , , represents a weight factor for balancing accuracy and energy consumption; main represents a main control channel index, corresponding to pitch angular acceleration or roll angular acceleration or yaw angular acceleration ; represents an actual angular acceleration of the main control channel; represents a desired angular acceleration of the main control channel; represents an actual angular acceleration of the non-main control channel; represents the total energy consumption corresponding to the driving voltage of each fluidic actuator.

4. The deep-reinforcement-learning-based aircraft control method according to claim 3, characterized in that, When training the aircraft control model, the state space of the aircraft control model is the three-axis actual angular acceleration and the expected angular acceleration of the main control channel; the action space is the continuous value of the driving voltage of each fluidic exciter.

5. The deep-reinforcement-learning-based aircraft control method according to claim 4, characterized in that, The deep reinforcement learning algorithm is a proximal policy optimization algorithm, which adjusts the policy through back propagation and gradient optimization to obtain the trained aircraft control model.

6. An aircraft control device based on deep reinforcement learning, characterized in that, The device comprises: an exciter arrangement and data acquisition module for arranging a fluidic exciter on an aircraft to form three-axis attitude control; acquiring three-axis actual angular acceleration in real time through an on-board inertial measurement unit; a loss function construction module for setting any one of the three-axis attitude channels as a main control channel, determining the expected angular acceleration of the main control channel; constructing a precision constraint term with the expected angular acceleration and the actual angular acceleration of the main control channel; constructing a decoupling constraint term with the actual angular acceleration of the non-main control channel being zero, and constructing an energy consumption term with the energy consumption corresponding to the driving voltage of each fluidic exciter; constructing a control target loss function according to the precision constraint term, the decoupling constraint term and the energy consumption term; a training module for using a deep reinforcement learning algorithm, taking minimizing the control target loss function as the training target, iteratively updating the parameters of the aircraft control model until the convergence condition is met, and obtaining the trained aircraft control model; An aircraft control module is configured to output a voltage instruction by a trained aircraft control model to control the operation of each jet exciter. In the exciter arrangement and the data acquisition module, a double-film three-cavity synthetic double jet exciter is arranged at the wing-body-tail edge of the aircraft to generate a pitch control moment; a double-film three-cavity synthetic double jet exciter is arranged at the outer wing section tail edge of the aircraft to generate a roll control moment; and a single-film double-cavity reverse jet exciter is arranged in the middle of the aircraft body to generate a yaw control moment. 7.A computer device, comprising a memory and a processor, and characterized in that, The memory stores a computer program, and the processor executes the computer program to implement the steps of the aircraft control method based on deep reinforcement learning in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, A computer program is stored thereon, and the computer program is executed by a processor to implement the steps of the aircraft control method based on deep reinforcement learning in any one of claims 1 to 5.

Images