A TBCC engine safety intelligent mode conversion control method

Abstract

This invention discloses a safe intelligent mode transition control method for a TBCC engine. It includes: acquiring control parameters for different mode transitions of the TBCC engine; constructing a deep deterministic policy gradient model based on the control parameters; training the deep deterministic policy gradient model using a deep deterministic policy gradient network; and using the trained deep deterministic policy gradient model to perform multivariate control during mode transitions. This invention overcomes the drawback of traditional techniques requiring the establishment of mathematical models and does not involve constructing an additional mathematical model for the TBCC engine. During the training of the agent in each mode transition process, Gaussian noise is added to the action network to improve the diversity of the experience pool samples. This invention utilizes random mode transition initial points, and all training and simulation verification processes incorporate environmental disturbances and the random influence of engine health parameters, verifying the adaptability and robustness of the intelligent controller.

Description

Technical Field

[0001] This invention relates to the field of engine control technology, and in particular to a safe intelligent mode switching control method for TBCC engines. Background Technology

[0002] Given the long-range, high-altitude flight requirements of hypersonic vehicles, propulsion systems need to maintain continuous thrust output across a wide Mach number range. TBCC engines, with their low technical risk and ground-based takeoff and landing capabilities, offer significant advantages in operability and economy, attracting close attention from relevant fields both domestically and internationally. However, compared to single-engine systems, TBCC engines, due to their unique structure and aerodynamic design, introduce additional coupling and nonlinearity into their operation. Furthermore, their large operating range and high uncertainty raise safety concerns that require further investigation.

[0003] Existing control methods for the mode transition process of TBCC engines mainly include switching control based on sequential quadratic programming and H-based control. ∞ The system employs various control methods, including loop shaping multivariable control, safety boundary-based multimode switching control, and hybrid sensitivity multivariable closed-loop control. Under most flight conditions, the TBCC engine operates in a single mode, which is a SISO system. In the TBCC simulation model, a PID controller is used for thrust closed-loop control, with the input being the engine's fuel flow rate in each single mode. During mode transitions, it is generally necessary to maintain a constant total output thrust of the TBCC engine, while keeping altitude and Mach number as stable as possible to achieve the transition from thrust provided by low-speed flow channels to thrust provided by high-speed flow channels. Unlike when individual sub-engines operate independently, where the combined inlet splitter and the common nozzle stub cone positions are fixed, the geometry of the combined inlet and common nozzle changes during mode transitions. This leads to changes in the inlet conditions of each flow channel, resulting in increased nonlinearity in the engine system. Furthermore, the mode transition process encounters multiple disturbances, with strong disturbances potentially causing engine instability.

[0004] Existing mode transition control (TBCC) methods heavily rely on precise mathematical models and lack support from TBCC prototypes and high-fidelity component-level models. Furthermore, they are constrained by the limitations of various control theories, resulting in a trade-off between different approaches. These methods fail to meet the actual control requirements of TBCC, exhibiting insufficient practicality, difficulty in guaranteeing robustness and safety, and a lack of optimization and adaptability. For example, existing safety PID control methods, which use a PID controller to control the fuel flow in the high-speed flow channel during mode transitions and determine the final action based on state variable observations, suffer from poor control performance, difficulty in simultaneously ensuring safety and control performance, and poor anti-interference capabilities. Another example is the H-based... ∞The loop-shaped multivariable control of the TBCC engine obtains the state-space equations of each mode transition process of the TBCC engine based on system identification, and solves the robust stabilization problem. The drawback is that it requires a high degree of accuracy in the mathematical model of the TBCC engine, but given the strong nonlinearity and high coupling of the TBCC engine, it is difficult to obtain a highly accurate mathematical model, resulting in poor practicality. Summary of the Invention

[0005] In view of this, the purpose of this invention is to propose a safe intelligent mode transition control method for TBCC engines. This method addresses the challenges of multi-component coordinated control, safety threats, and strong interference faced by hybrid TBCC engines in the process of tracking thrust during mode transitions, and overcomes the technical problems of insufficient practicality, difficulty in ensuring robustness and safety, and lack of optimization and adaptability of existing technologies.

[0006] According to one aspect of the present invention, a safe intelligent mode transition control method for a TBCC engine is provided, comprising: acquiring control parameters for different mode transitions of the TBCC engine; constructing a deep deterministic policy gradient model based on the control parameters; training the deep deterministic policy gradient model using a deep deterministic policy gradient network; and using the trained deep deterministic policy gradient model to perform multivariate control during mode transitions.

[0007] The above technical solution proposes a safety intelligent control method for TBCC engines based on a deep reinforcement learning model. This overcomes the technical problem that traditional control methods rely on precise mathematical models. For TBCC engines, which integrate high coupling, strong nonlinearity, and MIMO, traditional mathematical modeling methods struggle to achieve sufficiently high accuracy, thus limiting the control performance of traditional algorithms.

[0008] In some embodiments, the deep deterministic strategy gradient model includes a turbojet-ejector mode transition control model, an ejector-sub-burning mode transition control model, and a sub-burning-super-burning mode transition control model.

[0009] In some embodiments, the construction of a deep deterministic policy gradient model based on the control parameters includes: defining the state space, action space, and reward function for the three types of mode transitions based on the turbulent jet-ejector mode transition control parameters, the ejector-sub-combustion mode transition control parameters, and the sub-combustion-supercombustion mode transition control parameters, respectively.

[0010] In the above technical solution, for the design of the DDPG reinforcement learning controller, since the initial weights ω and biases b of the neurons in the Actor network and Critic network are randomly generated, they are not suitable for the control of the engine during the mode transition process. Therefore, the neural networks of three agents are trained for the three mode transition processes of the TBCC engine, and the trained network parameters are saved.

[0011] In some embodiments, the state space of the turbojet-ejector mode conversion control model is selected as the error between the thrust generated by the TBCC engine and the target thrust, the thrust error integral, and the total thrust generated; the action space is selected as the angle of the turbojet splitter in the combined air intake, the angle of the ejector channel splitter, the fuel quantity of the turbojet engine and the fuel supply of the ejector rocket, and the stroke of the corresponding common tailpipe plug cone.

[0012] In some embodiments, the state space of the ejector-substantive combustion mode conversion control model is selected as the error between the thrust generated by the TBCC engine and the target thrust, the thrust error integral, and the total thrust generated; the action space is selected as the fuel supply of the ejector rocket and the fuel supply of the substantive combustion chamber.

[0013] In some embodiments, the action space of the subsonic combustion-supersonic combustion mode conversion control model is selected as the angle of the combined intake duct ejector channel splitter, the fuel supply of the subsonic combustion chamber, the fuel supply of the supersonic combustion chamber, and the stroke of the corresponding shared tailpipe plug cone.

[0014] In the above technical solution, the selection of parameters is related to thrust tracking and safety boundaries. Model control requires ensuring that only one subloop remains online at any given time. When the incoming flow conditions reach the mode switching point of the TBCC engine, the switching control system can promptly perform the switching operation, ensuring that the controller in the corresponding mode quickly switches into the system. Seamless system switching is ensured during controller switching between modes. Since the aforementioned parameters directly participate in mode switching, selecting these indicators ensures more precise model control.

[0015] In some embodiments, the reward function includes an error reward function and a safety reward function.

[0016] In the above technical solution, the design of the reward function is crucial when using reinforcement learning for controller design. The reward value directly affects the fitting of the evaluation network to the value function, and thus indirectly affects the parameter updates of the action network. The quality of its design has a decisive impact on the final control effect and even the convergence of the algorithm. For the engine mode transition process, according to the control objective, the thrust tracking performance and engine safety are directly reflected in the reward function. Therefore, two sub-reward functions, an error reward function and a safety reward function, were designed for this purpose.

[0017] In some embodiments, training the deep deterministic policy gradient model using a deep deterministic policy gradient network includes: initializing the parameters of the deep deterministic policy gradient network and randomly setting mode transition points to determine engine parameters and thrust targets; determining the engine's state space from the deep deterministic policy gradient network's experience pool containing Gaussian noise based on the network parameters and thrust targets, and using the state space as the training input of the deep deterministic policy gradient network; and training the constructed deep deterministic policy gradient model using the deep deterministic policy gradient network with the state space as the training input.

[0018] In the aforementioned technical solutions, traditional controller design often relies on precise mathematical models. However, for the TBCC engine, which integrates high coupling, strong nonlinearity, and MIMO, traditional mathematical modeling methods struggle to achieve sufficiently high accuracy, thus limiting the control performance of traditional algorithms. For the DDPG-based reinforcement learning controller design, the initial weights ω and biases b within the neurons of its Actor and Critic networks are randomly generated, making them unsuitable for engine control during mode transitions. Therefore, this invention designs a randomized initial point for mode transitions in the training strategy. Furthermore, since DDPG employs a deterministic policy gradient and inherently lacks exploratory capabilities, Gaussian noise is added to the action output in the early stages of training to increase the diversity of the experience pool, ensuring the model's adaptability and robustness.

[0019] According to another aspect of the present invention, a TBCC engine safety intelligent mode switching control device is provided, comprising: at least one processor, and a memory communicatively connected to the processor, wherein the memory stores instructions executable by at least one processor, and when the instructions are executed by at least one processor, the above-described TBCC engine safety intelligent mode switching control method is performed.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, storing a computer program that, when executed by a processor, implements the TBCC engine safety intelligent mode switching control method as described above.

[0021] It can be observed that the above scheme, based on a deep reinforcement learning model, overcomes the shortcomings of traditional technologies that require the establishment of mathematical models. It does not require the construction of an additional mathematical model for the TBCC engine, demonstrating the advantages of this invention: model independence and strong practicality. During the training process of the agent in each mode transition, the DDPG algorithm adds Gaussian noise to the action network during training, enriching the diversity of the experience pool samples. Furthermore, this invention designs a random mode transition initial point in the training strategy, and incorporates environmental interference and the random influence of engine health parameters in all training and simulation verification processes, verifying the adaptability and robustness of the intelligent controller. This invention, through the design of error sub-reward functions and safety sub-reward functions, achieves simultaneous assurance of thrust tracking and safety performance, solving the drawback of traditional controllers that "sacrifice one for the other." Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating an embodiment of the TBCC engine safety intelligent mode conversion control method of the present invention;

[0024] Figure 2 This is a schematic diagram of the switching control system structure of an embodiment of the TBCC engine safety intelligent mode switching control method of the present invention;

[0025] Figure 3 This is a basic control-update framework diagram of an embodiment of the TBCC engine safety intelligent mode conversion control method of the present invention;

[0026] Figure 4 This is a real-time training overall framework diagram of an embodiment of the TBCC engine safety intelligent mode conversion control method of the present invention;

[0027] Figure 5 This is a flowchart of the real-time control algorithm for a TBCC engine—control thread—based on an embodiment of the TBCC engine safety intelligent mode conversion control method of the present invention.

[0028] Figure 6 This is a flowchart of the real-time control algorithm for a TBCC engine—the update thread—based on an embodiment of the TBCC engine safety intelligent mode conversion control method of the present invention.

[0029] Figure 7This is a detailed structural diagram of the interaction between the model and the intelligent agent in an embodiment of the TBCC engine safety intelligent mode conversion control method of the present invention;

[0030] Figure 8 This is a 20-second mode conversion process diagram of a TBCC engine thrust curve, representing an embodiment of the TBCC engine safety intelligent mode conversion control method of the present invention.

[0031] Figure 9 This is a diagram showing the actuation law of each component in the turbojet-ejector mode conversion of an embodiment of the intelligent mode conversion control method for TBCC engine of the present invention—a diagram showing the change in turbojet fuel quantity;

[0032] Figure 10 This is a diagram showing the actuation law of each component in the turbojet-ejector mode conversion—rocket fuel quantity variation—of an embodiment of the TBCC engine safety intelligent mode conversion control method of the present invention.

[0033] Figure 11 This is a diagram showing the actuation law of each component in the turbojet-ejector mode conversion of an embodiment of the intelligent mode conversion control method for TBCC engine of the present invention—the angle change of the turbojet channel splitter plate;

[0034] Figure 12 This is a diagram showing the actuation law of each component in the turbojet-ejector mode conversion of an embodiment of the intelligent mode conversion control method for TBCC engine of the present invention—the change in the stroke of the turbojet channel plug cone;

[0035] Figure 13 This is a diagram showing the actuation law of each component in the turbojet-ejector mode conversion of an embodiment of the intelligent mode conversion control method for TBCC engines of the present invention—specifically, the angle change diagram of the ejector channel splitter plate.

[0036] Figure 14 This is a diagram showing the actuation law of each component in the turbojet-ejector mode conversion of an embodiment of the intelligent mode conversion control method for TBCC engines of the present invention—the stroke variation diagram of the ejector channel plug cone;

[0037] Figure 15 This is a graph showing the change in compressor surge margin as an example of the turbojet-ejector mode conversion safety protection effect of an embodiment of the TBCC engine safety intelligent mode conversion control method of the present invention.

[0038] Figure 16 This is a diagram showing the safety protection effect of turbojet-ejector mode conversion in a turbojet-ejector configuration according to an embodiment of the intelligent mode conversion control method for TBCC engines of the present invention—a graph illustrating the temperature change in the rocket combustion chamber.

[0039] Figure 17 This is a thrust tracking curve of the intelligent controller for ejector-subsonic combustion mode conversion, an embodiment of the intelligent mode conversion control method for TBCC engine safety according to the present invention.

[0040] Figure 18 This is a diagram showing the actuation law of each component in the ejector-sub-fuel mode conversion of an embodiment of the TBCC engine safety intelligent mode conversion control method of the present invention—the change in fuel quantity in the sub-fuel combustion chamber;

[0041] Figure 19 This is a diagram showing the actuation law of each component during the ejector-sub-fuel mode conversion—a variation diagram of rocket fuel quantity—based on an embodiment of the TBCC engine safety intelligent mode conversion control method of the present invention.

[0042] Figure 20 This is a diagram showing the safety protection effect of ejector-submerged combustion mode conversion in an embodiment of the TBCC engine safety intelligent mode conversion control method of the present invention—the change in the non-starting margin of the submerged combustion intake.

[0043] Figure 21 This is a diagram showing the safety protection effect of ejector-sub-fuel mode conversion in an embodiment of the TBCC engine safety intelligent mode conversion control method of the present invention—sub-fuel combustion chamber temperature change.

[0044] Figure 22 This is a thrust tracking curve of the intelligent controller for subsonic combustion-supercombustion mode conversion in an embodiment of the TBCC engine safety intelligent mode conversion control method of the present invention;

[0045] Figure 23 This is a diagram showing the actuation law of each component in the sub-combustion-supercombustion mode conversion of an embodiment of the TBCC engine safety intelligent mode conversion control method of the present invention—a diagram showing the change in fuel quantity in the supercombustion combustion chamber;

[0046] Figure 24 This is a diagram showing the actuation law of each component in the sub-combustion-supercombustion mode conversion of an embodiment of the TBCC engine safety intelligent mode conversion control method of the present invention—a diagram showing the change in fuel quantity in the sub-combustion afterburner.

[0047] Figure 25 This is a diagram showing the actuation law of each component in the subsonic combustion-supersonic combustion mode conversion of an embodiment of the TBCC engine safety intelligent mode conversion control method of the present invention—specifically, the angle change diagram of the ejector channel splitter plate.

[0048] Figure 26 This is a diagram showing the actuation law of each component in the sub-combustion-supercombustion mode conversion of an embodiment of the TBCC engine safety intelligent mode conversion control method of the present invention—the stroke variation diagram of the ejector channel plug cone;

[0049] Figure 27 This is a diagram showing the safety protection effect of subsonic combustion-supersonic combustion mode switching in an embodiment of the TBCC engine safety intelligent mode switching control method of the present invention—the change in the non-starting margin of the supersonic combustion intake.

[0050] Figure 28This is a diagram showing the safety protection effect of the sub-combustion-supercombustion mode transition in an embodiment of the TBCC engine safety intelligent mode transition control method of the present invention—a graph of supercombustion chamber temperature change. Detailed Implementation

[0051] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be particularly noted that the following embodiments are for illustrative purposes only and do not limit the scope of the invention. Similarly, the following embodiments are only some, not all, embodiments of the present invention, and all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] This invention provides a safe and intelligent mode transition control method for TBCC engines. It addresses the challenges of multi-component coordinated control, safety threats, and strong interference faced by hybrid TBCC engines during the mode transition and thrust tracking process, and overcomes the technical problems of insufficient practicality, difficulty in ensuring robustness and safety, and lack of optimization and adaptability of existing technologies.

[0053] Please see Figure 1 , Figure 1 This is a flowchart illustrating the intelligent mode transition control method for TBCC engine safety according to the present invention. It should be noted that if substantially the same result is obtained, the method of the present invention is not necessarily identical. Figure 1 The sequence of processes shown is limited.

[0054] like Figure 1 As shown, the method includes the following steps:

[0055] S101: Obtain control parameters for different mode transitions of the TBCC engine.

[0056] In this embodiment, the control parameters include turbojet-ejector mode conversion control parameters, ejector-subsonic combustion mode conversion control parameters, and subsonic combustion-supercombustion mode conversion control parameters.

[0057] In this embodiment, the control parameters are as follows.

[0058] (1) Turbojet-ejector mode conversion

[0059] The turbojet-ejector mode transition is the transition from turbojet mode to ejector mode. Mach number 2 is selected as the mode transition point. The control inputs and actuation trends of each component during the mode transition process are shown in Table 1.

[0060] Table 1 Control inputs and variation patterns of turbojet-ejector mode conversion

[0061]

[0062] The safety boundaries involved in the turbojet-ejector mode transition include the turbojet engine compressor surge margin SM. HPC With rocket combustion chamber temperature T i The restrictions are as follows:

[0063]

[0064] (2) Ejector-sub-burning mode conversion

[0065] The ejector-sub-burning mode transition is the transition from ejector mode to sub-burning mode, which does not involve the switching of the flow channel. In this embodiment, Mach number 3 is selected as the mode transition point. The control inputs and actuation trends of each component in the mode transition process are shown in Table 2.

[0066] Table 2. Control inputs and variation patterns for ejector-sub-burn mode transition.

[0067]

[0068] The safety boundaries involved in the ejector-submerged combustion mode transition include the inlet ignition inert boundary and the submerged combustion chamber temperature T. s Limitations, for characterizing the problem of the intake manifold not starting, are achieved by using the intake manifold inlet and outlet back pressure ratio π. ram As an indicator, the specific formula is as follows:

[0069]

[0070] (3) Sub-combustion-supercombustion mode transition

[0071] The sub-combustion-supercombustion mode transition is the transition from sub-combustion mode to supercombustion mode. In this embodiment, Mach number 4.5 is selected as the mode transition point. The control inputs and actuation trends of each component in the mode transition process are shown in Table 3.

[0072] Table 3 Control inputs and variation patterns for subsonic to supersonic mode transition

[0073]

[0074] The safety boundaries involved in the transition from subsonic to supersonic combustion modes include the inlet non-starting boundary of the scramjet engine and the combustion chamber temperature T of the scramjet engine. s Limitations, for characterizing the problem of scram air intake failure, are achieved by using the inlet and outlet back pressure ratio π of the intake. scr As an indicator, the specific formula is as follows:

[0075]

[0076] S102: Construct a deep deterministic policy gradient model based on the control parameters.

[0077] In this embodiment, the deep deterministic strategy gradient model includes a turbojet-ejector mode conversion control model, an ejector-sub-burning mode conversion control model, and a sub-burning-superburning mode conversion control model.

[0078] In this embodiment, S102 further includes defining the state space, action space, and reward function for the three types of mode transitions based on the turbojet-ejector mode transition control parameters, the ejector-sub-burning mode transition control parameters, and the sub-burning-superburning mode transition control parameters.

[0079] In this embodiment, for the design of the DDPG reinforcement learning controller, since the initial weights ω and biases b of the neurons in the Actor network and Critic network are randomly generated and are not suitable for engine control during the mode transition process, the neural networks of three agents are trained for the three mode transition processes of the TBCC engine respectively, and the trained network parameters are saved.

[0080] In this embodiment, the state space of the turbojet-ejector mode conversion control model is selected as the error between the thrust generated by the TBCC engine and the target thrust, the thrust error integral, and the total thrust generated; the action space is selected as the combined inlet turbojet splitter angle, the ejector channel splitter angle, the turbojet engine fuel quantity, the ejector rocket fuel supply quantity, and the corresponding common tailpipe stub cone stroke. The state space of the ejector-substantive combustion mode conversion control model is selected as the error between the thrust generated by the TBCC engine and the target thrust, the thrust error integral, and the total thrust generated; the action space is selected as the ejector rocket fuel supply quantity and the substantive combustion chamber fuel supply quantity. The action space of the substantive combustion-superstantive combustion mode conversion control model is selected as the combined inlet ejector channel splitter angle, the substantive combustion chamber fuel supply quantity, the superstantive combustion chamber fuel supply quantity, and the corresponding common tailpipe stub cone stroke.

[0081] The above selections are related to thrust tracking and safety boundaries. Model control ensures that only one subloop remains online at any given time. When the incoming flow conditions reach the mode switching point, the switching control system can promptly perform the switching operation, ensuring that the controller in the corresponding mode quickly switches into the system. Seamless system switching is ensured during controller switching between modes. Since the above parameters directly participate in mode switching, these indicators are selected to ensure more precise model control.

[0082] In this embodiment, the reward function includes an error reward function and a safety reward function. When designing a controller using reinforcement learning, the design of the reward function is crucial. The reward value directly affects the fitting of the evaluation network to the value function, and thus indirectly affects the parameter updates of the action network. The quality of its design has a decisive impact on the final control effect and even the convergence of the algorithm. For the engine mode transition process, based on the control objective, the thrust tracking performance and engine safety are directly reflected in the reward function. Therefore, two sub-reward functions, an error reward function and a safety reward function, are designed for these purposes.

[0083] In this embodiment, the specifics are as follows:

[0084] The mode transition process of a TBCC engine includes: turbojet-ejector mode transition, ejector-subsonic combustion mode transition, and subsonic combustion-supersonic combustion mode transition. For the multi-mode control problem of the TBCC engine, this embodiment designs a switching control system. Its main task is to enable the control system to quickly switch to the corresponding mode controller as the TBCC engine's operating state changes, and the switching operation should not affect the system's stability.

[0085] Due to the multi-objective and multi-tasking nature of the switching system, the switching controller design must meet the following requirements:

[0086] 1. At any given time, only one sub-loop remains online.

[0087] 2. When the incoming flow conditions reach the mode switching point, the switching control system of the TBCC engine can make a timely switching operation to ensure that the controller in the corresponding mode quickly switches into the system.

[0088] 3. Ensure seamless system switching during controller switching between different modes.

[0089] To ensure the above requirements, this embodiment adopts a two-layer control structure, including a sub-controller that ensures good tracking performance of each thrust closed-loop control sub-loop and an upper-layer switching controller that ensures seamless and rapid switching. A reasonable mode transition rule is designed based on the operating characteristics of the TBCC engine. A schematic diagram of the switching control system is shown below. Figure 2 As shown.

[0090] This embodiment addresses three modal transition processes, designing and training three agents based on the DDPG algorithm, and performing overall control system framework design, neural network structure design, reward function design, and controller parameter training strategy design.

[0091] The TBCC engine intelligent controller proposed in this invention adopts a unified basic framework, and the control-update basic framework is as follows: Figure 3As shown, it mainly consists of a TBCC engine simulation model and an intelligent controller. The main components of the intelligent controller include actuators, evaluators, and an experience playback pool R.

[0092] The basic framework mainly includes an actuator and an evaluator. The controlled object is a simulation model of the specific TBCC engine mode transition process. The Actor network in the actuator acts as the controller, and its inputs are the state, i.e., the controlled variable, engine safety parameters, and other information. After passing through the controller, it outputs action commands, which act on the TBCC engine and generate the state and reward value for the next time step, feeding them back to the controller. The array packet for each round is stored in the experience replay pool. The parameters of the Actor network in the actuator are used by optimizer 1 to derive the policy.

[0093] Gradients are updated. Simultaneously, the parameters of the target Actor network are updated using the soft update criterion. In the evaluator, the Critic network evaluates the action commands generated by the Actor network, outputting Q-values ​​used for Actor network updates. The Critic network's parameters are updated by optimizer 2 using a loss function minimization method. The parameters of the target Critic network are also updated using the soft update criterion.

[0094] The intelligent controller network design for the TBCC engine mode conversion includes the following three parts:

[0095] (1) Intelligent controller network design for turbojet-ejector mode conversion

[0096] Based on the knowledge of control objectives and traditional controller design, state s is selected as the error e between the thrust generated by the TBCC engine and the target thrust. Thrust Thrust error integral es Thrust And the total thrust generated, Thrust, is output as the angle Sp of the combined inlet turbojet splitter. Trb1 With Sp Trb2 Ejector channel splitter angle Sp Ejc Turbojet engine fuel consumption Wf Trb1 With Wf Trb2 and the fuel supply Wf of the ejector rocket Roc And the corresponding common tailpipe plug cone stroke Cone Trb1 Cone Trb2 Cone Ejc It is important to note that the state needs to be normalized before being input into the neural network.

[0097] Exploration is crucial for agents. Since DDPG uses deterministic policy gradients, it inherently lacks exploration capabilities. Therefore, Gaussian noise is added to the action output in the early stages of training to increase the diversity of the experience pool.

[0098] The update direction of the parameter θ of the online action network is related to Q(s) t ,a t The value function increases in the same direction, and the network update is similar to that of general supervised learning algorithms, updating the network weights using backpropagation of the error based on the network loss function. For the Actor network, the chain rule is applied to the network update, and the policy gradient obtained by differentiating the expected future reward function J with respect to the network parameters θ can be expressed as follows:

[0099]

[0100] Function J is used to measure the performance of the executor, and is defined based on N training samples sampled in a batch as follows:

[0101]

[0102] For the vortex-ejector mode conversion problem, which involves 6-dimensional input and has coupling effects, a simple 3-layer neural network is difficult to complete the search task. In this embodiment, an input layer, three intermediate layers and an output layer are finally determined. The input layer is only used to receive state input. Each intermediate layer has 30 neurons. The range of each output parameter is determined based on the prior knowledge of the traditional controller. The tanh activation function is used to map the output layer results to [-1,1] to limit the output range.

[0103] The target Actor network samples actions through an experience pool and provides action input to the target Critic network, whose input is the next state s of the TBCC engine. t The network outputs the next action 'a'. Since it is copied from the Actor network, it has the same structure. The target action network uses a "soft" update mode, as shown in the following equation:

[0104] θ'←ξθ+(1-ξ)θ' (6)

[0105] To ensure the stability of the learning process, ξ << 1 is required. Although this sacrifices the speed of parameter updates and reduces the learning speed, it enhances the robustness of the algorithm.

[0106] The online evaluation network takes state s and action a as input and aims to fit an action-value function Q(s,a) to provide direction for the parameter updates of the action network. Its policy gradient is derived from minimizing the loss function of the online evaluation function. Each time, N empirical samples (s,a) are randomly sampled in mini-batches from the empirical replay pool R. i ,a i ,r i ,s i+1 To achieve the target value y i The fitting loss function can be expressed as:

[0107]

[0108] in:

[0109] y i =r i +γQ t (s′ i ,π(s′ i |θ t )|φ t (8)

[0110] The evaluation network consists of two paths. The state path uses two intermediate layers, each containing 30 neurons for feature extraction, with ReLU activation to achieve sparsity. For the action path, given its higher dimensionality and coupling between components, the network structure uses three intermediate layers, each with 30 neurons, again employing ReLU activation. The output layer outputs the current action value as a scalar with a neuron count of 1.

[0111] The input to the target evaluation network is the state s′ at the next time step. t and the output a′ of the target action network t The output is the value of the action at the next moment, Q'(s′). t ,a′ t The purpose is to optimize the update and iteration direction of the online action network and improve its convergence speed. It has the same structure as the online evaluation network, and the parameter φ′ is also updated using a "soft" update method.

[0112] φ'←ξφ+(1-ξ)φ' (9)

[0113] The TBCC engine structure selected in this embodiment adopts a four-channel structure with symmetrical axis distribution, and has two axisymmetric turbojet channels. Therefore, the control quantities of the turbojet channels are handled consistently in this embodiment, that is, the final output action set is:

[0114] C={Sp Trb Sp Ejc ,Wf Trb ,Wf Roc Cone Trb Cone Ejc} (10) The hyperparameter design of the intelligent controller for the turbojet-ejector mode conversion process is shown in Table 4.

[0115] Table 4 Hyperparameters of the intelligent controller for turbojet-ejector mode conversion

[0116]

[0117] (2) Intelligent controller network design for ejector-sub-burn mode switching

[0118] Compared to the turbojet-ejector mode transition process, the ejector-substantive combustion mode transition process only involves the switching control of the fuel supply system, with a relatively lower action dimension and the state variable s remaining unchanged. The output then becomes the fuel supply quantity Wf of the ejector rocket. Roc Fuel supply Wf to the sub-fuel combustion chamber Ram Furthermore, the complexity of the action network and the evaluation network is adjusted accordingly, specifically by reducing the number of hidden layer neurons in the action network and reducing the depth of the action paths in the evaluator network.

[0119] The hyperparameter design of the intelligent controller for the ejector-sub-burning mode transition process is shown in Table 5.

[0120] Table 5 Hyperparameters of the intelligent controller for ejector-sub-burning mode conversion

[0121]

[0122] (3) Intelligent controller network design for sub-combustion-supercombustion mode switching

[0123] In this embodiment, the scramjet passage of the TBCC engine is in a normally open state; therefore, the scramjet passage does not include geometric adjustments. The output of the Actor network includes the combined inlet ejector channel splitter angle Sp. Ejc Submersible combustion chamber fuel supply Wf Ram With the supersonic combustion chamber fuel supply Wf Scr And the corresponding common tailpipe plug cone stroke Cone Ejc .

[0124] The hyperparameter design of the intelligent controller for the sub-combustion-supercombustion mode transition process is shown in Table 6.

[0125] Table 6 Hyperparameters of the Intelligent Controller for Sub-combustion-Supercombustion Mode Conversion

[0126]

[0127]

[0128] When designing controllers using reinforcement learning, the design of the reward function is crucial. The reward value directly affects the fitting of the evaluation network to the value function, and indirectly affects the parameter updates of the action network. The quality of its design has a decisive impact on the final control effect and even the convergence of the algorithm. For the engine mode transition process, the thrust tracking performance and engine safety are directly reflected in the reward function. This embodiment designs sub-reward functions r for these aspects. err With r uns The final reward function r is composed of these two parts:

[0129] r = r err +r uns (11)

[0130] For thrust tracking, i.e., maintaining the actual thrust F generated by the TBCC engine during mode transition. total With thrust command value F dmd They are equal, therefore the thrust error e is included in the reward function. Thrust And it is segmented according to the accuracy parameters in the thrust target:

[0131]

[0132] in:

[0133] k1>k2>k3>0 (13)

[0134] In this embodiment, the error reward function in the same form as Equation (12) is used for the three mode conversion processes. Only the specific proportional coefficient is adjusted. The specific values ​​are shown in Table 7.

[0135] Table 7. Proportional coefficients of the reward function for each mode conversion error of the TBCC engine.

[0136]

[0137] For a numerical description of the safety of TBCC engines, this embodiment introduces a unified safety margin ζ expression as follows:

[0138]

[0139] Where x is the various physical quantities subject to safety restrictions in this embodiment, x max This represents the maximum value that the physical quantity can reach before entering a dangerous operating condition. Regarding safety constraints, we define them in the reward function r. uns Introducing a control barrier function B γ (x) is used to ensure that the safety margin ζ of each state in the control system is positive definite, and the safety set is defined as follows:

[0140] C:{x∈R n :ζ(x)≥0} (15)

[0141] Control barrier function B γ The significance of (x) lies in imposing a corresponding penalty when x approaches the safety boundary, thereby adjusting the control strategy. Marvi et al. proved that the control barrier function can guarantee the safety of the system. γ (x) has the following properties:

[0142] (1)

[0143] (2) When the system state is at the safety boundary, the penalty value is very large, i.e., ζ→0,B γ (x)→-∞;

[0144] (3) When the system state is within the safe set, B γ (x) decreases monotonically toward the safety boundary.

[0145] The control barrier function used in this embodiment is in the following form:

[0146]

[0147] It meets the above requirements for the control barrier function, and by setting different parameters γ according to different safety boundary conditions, a trade-off between safety and controller performance can be made. A corresponding proportional coefficient ω is set according to the penalty intensity. i The final reward function for the safe component is:

[0148]

[0149] Based on the safety boundary conditions for each mode transition process, the specific reward function for the safety component of each mode transition process is as follows:

[0150] Turbojet-ejector mode transition:

[0151]

[0152] Ejector-sub-burn mode transition:

[0153]

[0154] Sub-combustion to supercombustion mode transition:

[0155]

[0156] The specific parameter settings are shown in Table 8.

[0157] Table 8. Parameter settings for the safety reward function during each mode transition process.

[0158]

[0159] S103: The deep deterministic policy gradient model is trained using a deep deterministic policy gradient network.

[0160] In this embodiment, S103 includes: initializing the parameters of the deep deterministic policy gradient network and randomly setting mode transition points to determine engine parameters and thrust targets; determining the engine's state space from the deep deterministic policy gradient network experience pool containing Gaussian noise based on the network parameters and thrust targets, and using the state space as the training input of the deep deterministic policy gradient network; and training the constructed deep deterministic policy gradient model using the deep deterministic policy gradient network with the state space as the training input.

[0161] Traditional controller design often relies on precise mathematical models. However, for TBCC engines, which integrate high coupling, strong nonlinearity, and MIMO, traditional mathematical modeling methods struggle to achieve sufficiently high accuracy, thus limiting the control performance of traditional algorithms. For DDPG-based reinforcement learning controller design, the initial weights ω and biases b within the neurons of its Actor and Critic networks are randomly generated, making them unsuitable for engine control during mode transitions. Therefore, this invention designs a randomized initial point for mode transitions in the training strategy. Furthermore, since DDPG uses deterministic policy gradients and inherently lacks exploratory capabilities, Gaussian noise is added to the action outputs during the initial training phase to increase the diversity of the experience pool, ensuring the model's adaptability and robustness.

[0162] In this embodiment, the training method is as follows:

[0163] Traditional controller design often relies on precise mathematical models. However, for the TBCC engine, which integrates high coupling, strong nonlinearity, and MIMO, traditional mathematical modeling methods struggle to achieve sufficiently high accuracy, thus limiting the control performance of traditional algorithms. For DDPG-based reinforcement learning controller design, the initial weights ω and biases b within the neurons of the Actor and Critic networks are randomly generated and unsuitable for engine control during mode transitions. Therefore, three neural networks are trained for the three mode transition processes of the TBCC engine, and the trained network parameters are saved. This embodiment uses a hybrid TBCC engine simulation model built on the Matlab / Simulink platform and employs the Reinforce Learning Toolbox in Matlab to construct the agents. The state data generated by the model is directly input into the agents to improve training data accuracy and avoids the cross-platform data interaction problems that arise when using third-party platforms like Python to build the agents.

[0164] The real-time training environment consists of two parts: the model (Environment) and the agent (Agent). The overall framework of real-time training is as follows: Figure 4 As shown.

[0165] The model and agent interact via data to achieve the single-step update of agent network parameters and training optimization mentioned earlier. The specific interaction process is as follows: First, the agent's network parameters are initialized. Then, the initial setting function ResetFcn is called to randomly select mode transition points within the range of each mode transition process, determine the engine environmental conditions such as altitude and Mach number, and use these to determine the thrust target Thrust. dmd The state input s of the TBCC engine is obtained by combining the acquired instructions with the initial network parameters. t , will s t Inputting into an Actor network yields the output action a. t The simulation program is applied to the engine to execute a step-size simulation program to obtain the next state s′. t And obtain the corresponding reward value r through the set reward function. t The data set s generated in this round t ,a t ,s′ t ,r t The real-time training process of an agent, stored in an experience pool, can be divided into a control thread and an update thread, such as... Figure 5 , Figure 6 As shown.

[0166] The training method described above is implemented using the Reinforcement Learning Toolbox. The Reinforcement Learning Toolbox (hereinafter referred to as the RL Toolbox) is a simplified configuration tool provided by Matlab for training agents using reinforcement learning algorithms. It provides applications, function names, and Simulink modules for numerous reinforcement learning algorithms such as DQN, PPO, SAC, and DDPG. The toolbox supports representing policy value functions using deep neural networks or table lookups and enables training through interaction with the modeling environment in Matlab or Simulink. It also supports custom hyperparameters, hardware scheduling, and multithreading.

[0167] In addition to the basic state observations and rewards, an isdone input is added to terminate the round early. Based on simulation results, in the early stages of agent training, the policy network may generate unreasonable action inputs, causing partial solver crashes. To prevent this, a high-level isdone input is generated to terminate the round early when the engine's thrust output deviates significantly from the command value. Furthermore, a severe penalty term is added to the reward function.

[0168]

[0169] The specific structure of the interaction between the model and the agent in the Simulink environment is as follows: Figure 7 As shown.

[0170] For each mode transition process, a corresponding training environment was created in Matlab. The training environment needed to include the dimensions of the state variables and the upper and lower bounds of each state variable during each mode transition process, with specific values ​​mapped to the range [-1, 1]. The model training step size was uniformly set to 0.01 seconds. The training process is as follows: Figure 5 , Figure 6 The agent network parameters for each mode transition are as described above and will not be repeated here. The training conditions for each mode transition process are as follows:

[0171] Table 9. Training Conditions and Parameters for Turbojet-Ejector Mode Conversion

[0172]

[0173]

[0174] Table 10 Training Conditions and Parameters for Ejector-Sub-combustion Mode Conversion

[0175]

[0176] Table 11 Training Conditions and Parameters for Sub-Gas-Super-Gas Mode Conversion

[0177]

[0178] S104: Employs a trained deep deterministic policy gradient model for multivariate control during mode transitions.

[0179] In this embodiment, a trained model is used to perform intelligent control and simulation testing of the mode transitions of the TBCC engine. Specifically:

[0180] (1) Simulation experiment of vortex-ejector mode conversion

[0181] The environmental conditions were selected as follows: Mach number of 2.3 and altitude of 11726.57m at the mode transition point. Atmospheric gusts were incorporated into the simulation experiment. The thrust maintenance command during the mode transition phase was 8728N. Mode transition was initiated at 2s. The thrust curve of the TBCC engine during the mode transition process within 20s is shown below. Figure 8 As shown.

[0182] The actuation laws of each component in the turbojet-ejector mode conversion are as follows: Figure 9-14As shown, the fuel supply of the TBCC engine gradually stops, and the rocket fuel quantity increases accordingly. At the same time, under the action of the air intake splitter plate and tail nozzle plug cone in their respective channels, the turbojet channel slowly closes while the ejector channel gradually opens, realizing the transition from turbojet mode to ejector mode.

[0183] The safety limiting effect of the vortex-ejector mode is as follows: Figure 15-16 As shown, the intelligent controller for turbojet-ejector mode conversion effectively ensures that the TBCC engine does not enter the dangerous state of compressor surge or rocket combustion chamber overheating during the turbojet-ejector mode conversion process.

[0184] (2) Simulation experiment of ejector-sub-burn mode transition

[0185] The environmental conditions were selected as follows: Mach number of 3.0 at the mode transition point and altitude of 15106.80m. Atmospheric gusts were included in the simulation experiment. The thrust maintenance command during the mode transition phase was 10137N. Mode transition was initiated at 2s. The thrust curve of the TBCC engine during the mode transition process within 20s is shown below. Figure 17 As shown.

[0186] The actuation law of each component in the ejector-sub-burning mode transition is as follows: Figure 18 , 19 As shown, the fuel supply to the TBCC engine gradually stopped, and the fuel quantity in the subsonic combustion chamber increased accordingly, thus realizing the transition from ejection mode to subsonic combustion mode.

[0187] Ejector-sub-burn mode safety limiting effect such as Figure 20 , 21 As shown, the intelligent controller for ejector-subsonic combustion mode conversion effectively ensures that the TBCC engine will not enter the subsonic combustion intake and fail to start, or that the subsonic combustion chamber will overheat during the ejector-subsonic combustion mode conversion process.

[0188] (3) Simulation experiment of sub-combustion-supercombustion mode transition

[0189] The environmental conditions selected for the mode transition point were an incoming Mach number of 4.5 and an altitude of 20272.55m. Atmospheric gusts were incorporated into the simulation experiment. The thrust holding command during the mode transition phase was 5549N. Mode transition was initiated at 2s, and the mode transition process within 20s was as follows: Figure 22 As shown.

[0190] The actuation laws of each component during the sub-combustion-supercombustion mode transition are as follows: Figure 23-26 As shown, the fuel supply to the subsonic combustion chamber of the TBCC engine gradually stops, while the amount of supersonic fuel increases accordingly. At the same time, the ejector channel is slowly closed by the action of the intake manifold and the tailpipe plug cone, thus realizing the transition from subsonic combustion mode to supersonic combustion mode.

[0191] The safety limiting effect of subsonic-supersonic modes is as follows: Figure 27 , 28 As shown, the intelligent controller for subsonic combustion-supersonic combustion mode conversion effectively ensures that the TBCC engine will not enter the supersonic intake manifold and fail to start, or that the supersonic combustion chamber will not overheat during the subsonic combustion-supersonic combustion mode conversion process.

[0192] It can be observed that in this embodiment, the interaction of this method is based on the newly released RL Toolbox from Matlab, enabling direct data interaction between the agent and the TBCC engine model on a single platform without constructing an additional mathematical model of the TBCC engine. This demonstrates the advantages of the invention: model independence and strong practicality. During the training process of the agent in each mode transition, the DDPG algorithm adds random Gaussian noise to the action network during training, enriching the diversity of the experience pool samples. In addition, the invention designs a random mode transition initial point in the training strategy, and incorporates environmental interference and random influences on engine health parameters in all training and simulation verification processes, verifying the adaptability and robustness of the intelligent controller. The invention achieves simultaneous guarantee of thrust tracking and safety performance through the design of error sub-reward function and safety sub-reward function, solving the drawback of "at the expense of one" in traditional controllers.

[0193] The present invention also provides a TBCC engine safety intelligent mode switching control device, comprising: at least one processor, and a memory communicatively connected to the processor, wherein the memory stores instructions executable by at least one processor, and when the instructions are executed by at least one processor, the above-described TBCC engine safety intelligent mode switching control method is performed.

[0194] In this embodiment, the memory and processor are connected via a bus, which can include any number of interconnected buses and bridges, connecting various circuits of one or more processors and the memory together. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be further described in this embodiment. A bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0195] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. The memory 52 can be used to store data used by the processor during operation.

[0196] The present invention further provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the above-described method embodiments.

[0197] The above description is only a part of the embodiments of the present invention and does not limit the scope of protection of the present invention. Any equivalent device or equivalent process transformation made based on the content of the present invention specification and drawings, or direct or indirect application in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A safe intelligent mode conversion control method for a TBCC engine, characterized in that, include: Obtain control parameters for different mode transitions of the TBCC engine; Construct a deep deterministic policy gradient model based on the control parameters; The deep deterministic policy gradient model is trained using a deep deterministic policy gradient network. A trained deep deterministic policy gradient model is used to perform multivariate control during mode transitions; The deep deterministic strategy gradient model includes a turbojet-ejector mode conversion control model, an ejector-subsonic combustion mode conversion control model, and a subsonic combustion-supersonic combustion mode conversion control model; The construction of the deep deterministic policy gradient model based on the control parameters includes: Based on the control parameters for turret-ejector mode transition, ejector-subsonic combustion mode transition, and subsonic combustion-supersonic combustion mode transition, the state space, action space, and reward function for the three types of mode transitions are defined respectively. The state space of the turbojet-ejector mode conversion control model is selected as the error between the thrust generated by the TBCC engine and the target thrust, the thrust error integral, and the total thrust generated. The action space is selected as the angle of the combined air intake turbojet splitter, the angle of the ejector channel splitter, the fuel quantity of the turbojet engine and the fuel supply of the ejector rocket, and the stroke of the corresponding common tail nozzle plug cone. The state space of the ejector-substantive combustion mode conversion control model is selected as the error between the thrust generated by the TBCC engine and the target thrust, the thrust error integral, and the total thrust generated; the action space is selected as the fuel supply of the ejector rocket and the fuel supply of the substantive combustion chamber. The action space of the subsonic-supersonic mode conversion control model is selected as the angle of the combined intake ejector channel splitter, the fuel supply of the subsonic combustion chamber, the fuel supply of the supersonic combustion chamber, and the stroke of the corresponding common tailpipe plug cone.

2. The intelligent mode switching control method for TBCC engine safety as described in claim 1, characterized in that, The reward function includes an error reward function and a safety reward function.

3. The intelligent mode switching control method for TBCC engine safety as described in claim 1, characterized in that, The step of training the deep deterministic policy gradient model using a deep deterministic policy gradient network includes: Initialize the parameters of the deep deterministic policy gradient network and randomly set the mode transition point to determine the engine parameters and thrust target; The engine's state space is determined from the experience pool of a deep deterministic policy gradient network containing Gaussian noise based on network parameters and thrust target, and the state space is used as the training input for the deep deterministic policy gradient network. The constructed deep deterministic policy gradient model is trained using a deep deterministic policy gradient network that uses the state space as training input.

4. A safe intelligent mode conversion control device for TBCC engines, characterized in that, include: At least one processor, and a memory communicatively connected to the processor, wherein, The memory stores instructions that can be executed by at least one processor, which, when executed by at least one of the processors, perform a TBCC engine safety intelligent mode switching control method as described in any one of claims 1 to 3.

5. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the TBCC engine safety intelligent mode switching control method according to any one of claims 1 to 3.

Images