High-through-flow double-variable cycle engine fusion model construction method based on reinforcement learning

Abstract

The invention discloses a method for constructing a fusion model of a high-flow double-variable cycle engine based on reinforcement learning, and belongs to the field of aero-engine modeling. The method comprises the following steps: based on an aerodynamic thermodynamic mechanism of working of the high-through-flow double-variable-cycle engine, selecting a common working equation to realize complete machine performance matching; designing a fusion model architecture based on a Newton-Rafson algorithm and a double-delay depth deterministic strategy gradient algorithm TD3; after an input and output structure and a reward system are designed according to the working mechanism of the fusion model, system hyper-parameters are set according to reward and common working equation residual errors. The high-through-flow double-variable-cycle engine fusion model established by the method is high in convergence speed, high in precision and good in real-time performance, can break through the limitation that a traditional model can only carry out simulation at a specific point, reduces the number of iterations of the model under abrupt-change input parameters, and guarantees the convergence within a full envelope range.

Description

Technical Field

[0001] The present invention relates to the technical field of aero-engine modeling, and in particular to a method for constructing a high-throughput dual-variable cycle engine fusion model based on reinforcement learning. Background Art

[0002] As aircraft engines develop towards high thrust-to-weight ratio, high performance and high economy, it is crucial to improve the reliability of control systems and fault diagnosis systems. As the cornerstone of the above research, aircraft engine models are currently a hot topic in the aviation field, ensuring their real-time and accuracy. However, in order to meet the power requirements of the next generation of high-altitude and high-speed aircraft, the engine structure is becoming more and more complex, which poses a huge challenge to the accuracy and real-time performance of the engine dynamic performance model. Therefore, it is becoming increasingly important to develop a dynamic real-time modeling method for aircraft engines.

[0003] The widely used engine models include component-level models, piecewise linear models, linear variable parameter models and data-driven models. These methods have their own advantages and disadvantages in practical applications. The component-level model based on the principle of aerodynamic thermodynamics can ensure the accuracy requirements of the entire flight envelope, but the complex engine structure will undoubtedly increase the dimension of the common working equations, which greatly increases the computational burden of the iterative algorithm. The use of piecewise linear models for aeroengine modeling can be traced back to 1977, when Teren studied the shortest acceleration time of turbofan engines based on this. The piecewise linear model has a simple structure and powerful real-time performance simulation capabilities, but studies have shown that its accuracy cannot be guaranteed when facing transition processes with obvious mutation input parameters. In order to solve this problem, the linear variable parameter model is applied to the field of aeroengine modeling. Based on the state space equation, the linear variable parameter model can effectively link the engine input and output, with small computational complexity and good dynamic performance. However, due to the individual differences of aeroengines caused by manufacturing tolerances and performance degradation, the two methods of piecewise linear model and linear variable parameter model cannot cover all flight conditions. Different from the above modeling methods, data-driven models based on large amounts of data and a reasonable network architecture can cover the entire flight envelope and operating conditions, but how to ensure its accuracy remains a challenge.

[0004] In summary, considering the modeling accuracy of the entire flight envelope, the engine component-level model is undoubtedly the best choice. In order to ensure the real-time performance of the engine component-level model under complex configurations, domestic and foreign scholars have proposed many improvement methods, which are mainly divided into three aspects: iterative algorithms, local linearization, and fusion models. However, iterative algorithms and local linearization methods have more or less the problem of accuracy loss. Therefore, the study of fusion models combined with data-driven algorithms has become a hot topic. However, existing fusion modeling methods usually require engine data covering the entire envelope and all operating conditions to ensure accuracy, and convergence is difficult to guarantee. Summary of the invention

[0005] In order to solve the limitations of traditional models that simulation can only be carried out at specific points and convergence is difficult to ensure under the full envelope, the high-throughput dual-variable cycle engine fusion model construction method based on reinforcement learning proposed in the present invention constructs a high-throughput dual-variable cycle engine fusion model, aiming to design the fusion model architecture based on the selection of the common working equation, combining the Newton-Raphson algorithm and the double-delay deep deterministic policy gradient algorithm TD3, and designing its input and output structure and reward system, adjusting the system hyperparameters, and finally establishing an engine dynamic performance model with fast convergence, high precision and good real-time performance.

[0006] The embodiment of the present invention provides a method for constructing a high-throughput dual-variable cycle engine fusion model based on reinforcement learning, comprising the following steps:

[0007] Step 1: Based on the aerodynamic thermodynamic mechanism of the high-throughput dual-variable cycle engine, select the common working equation to achieve the performance matching of the whole machine;

[0008] Step 2: combining the common working equation, designing a high-throughput dual-variable cycle engine fusion model architecture based on the Newton-Raphson algorithm and the double-delay deep deterministic policy gradient algorithm TD3;

[0009] Step 3: Design the system input and output structure and reward system based on the working mechanism of the high-throughput dual-variable cycle engine fusion model;

[0010] Step 4: According to the residual of the joint working equation and the designed reward system, the hyperparameters of the high-throughput dual-variable cycle engine fusion model are tuned.

[0011] Optionally, in one embodiment of the present invention, step 1 specifically includes:

[0012] Step 1.1: Analyze the aerodynamic thermodynamic working mechanism of each component of the high-throughflow dual-variable cycle engine, and establish a mathematical model of typical components of the high-throughflow dual-variable cycle engine in combination with the working mechanism of each component of the existing variable cycle engine;

[0013] Step 1.2: Analyze the typical components of the high-throughput dual-cycle engine: including the front fan FFan, the rear fan RFan, the core drive fan CDFS, the high-pressure compressor Comp, the high-pressure turbine Hturb, the intermediate-pressure turbine Mturb and the low-pressure turbine Lturb, and the working mechanism of the seven rotating parts and three rotating shafts, and determine the number of common working equations;

[0014] Step 1.3: Combining the working principles of engine flow continuity, static pressure balance and power conservation, the common working equation is selected as:

[0015]

[0016] in, Indicates flow rate, P s represents static pressure, η represents mechanical efficiency of rotor shaft, N represents power; digital subscript represents engine section number, 131 represents outlet of third duct, 142 represents outlet of second duct, 16 represents outlet of rear duct, cool242 represents cooling air of low-pressure turbine guide vane, cool261 represents cooling air of intermediate-pressure turbine guide vane, 29 represents outlet of first duct, 311 represents inlet of hot end of heat exchanger, 4 represents outlet of combustion chamber, 41 represents outlet of high-pressure turbine guide vane, 43 represents outlet of high-pressure turbine, 44 represents outlet of high-pressure turbine guide vane, 46 represents outlet of intermediate-pressure turbine, 48 represents outlet of dual-variable combustion chamber, 49 represents outlet of low-pressure turbine guide vane, 6 represents outlet of low-pressure turbine, 7 represents outlet of afterburner, 9 represents outlet of tail nozzle; subscript H represents high-pressure shaft, M represents intermediate-pressure shaft, L represents low-pressure shaft, EX represents engine accessories.

[0017] Optionally, in one embodiment of the present invention, step 2 specifically includes:

[0018] Step 2.1: Compare and analyze the mechanism of deep reinforcement learning algorithms, and select the algorithm required to establish the high-throughput dual-variable cycle engine fusion model;

[0019] Step 2.2: Combine the double-delay deep deterministic policy gradient algorithm TD3 and the Newton-Raphson algorithm to design the high-throughput dual-variable cycle engine fusion model architecture.

[0020] Optionally, in one embodiment of the present invention, the high-throughput dual-variable cycle engine fusion model includes two sets of iterative systems, inner and outer loops. When the initial iteration parameters are inaccurate and the residual e of the engine common working equation is greater than a specified constant ε, the outer loop system based on the TD3 agent is used for iteration; when the residual e is less than the specified constant ε, the Newton-Raphson algorithm is used for inner loop iteration;

[0021] During the training process, since the initial iteration parameters are randomly given, the residual e of the common working equation group is greater than the specified constant ε. The high-throughput dual-variable cycle engine fusion model is connected to the Actor of the outer loop system for training. The action A output by the Actor t and the state S obtained by the engine flow calculation module t , S t+1 and reward r t Stored together in the playback buffer for the training of Critic, Critic based on the expected return Q t The error is updated to further optimize the Actor's update strategy;

[0022] During the use phase, when the residual e is greater than the specified constant ε, the trained Actor is called to output the initial iteration parameters until the residual e is less than the specified constant ε. At this time, the high-throughput dual-variable cycle engine fusion model is connected to the inner loop and iterated by the Newton-Raphson algorithm until the high-throughput dual-variable cycle engine fusion model converges.

[0023] Optionally, in one embodiment of the present invention, step 3 specifically includes:

[0024] Step 3.1: Analyze the working mechanism of the high-throughput dual-variable cycle engine fusion model. The TD3-based intelligent agent includes two networks, Actor and Critic. In the high-throughput dual-variable cycle engine fusion model, Actor is used to generate initial iteration parameters, and Critic is used to judge the quality of the current initial iteration parameters and guide the Actor network update;

[0025] Step 3.2: Design the input and output structures of the Actor network and Critic network;

[0026] Step 3.3: The setting of the agent reward system is to guide the agent to generate accurate iteration parameters faster, so it is necessary to comprehensively consider the convergence and convergence speed of the high-throughput dual-variable cycle engine fusion model. The convergence of the training stage is mainly determined by the sum of the absolute values ​​of the residuals of the common working equations. The convergence rate is expressed as the number of required inner loop Newton-Raphson algorithm iterations K t Determine the flag bit that the inner loop can converge As the boundary, it can converge Reward t The settings are:

[0027]

[0028] The subscript t represents the current time.

[0029] Optionally, in one embodiment of the present invention, the quality of the initial iteration parameters of the engine depends on the working environment E of the engine. T and the current state,

[0030] The input parameters that determine the high-throughput dual-cycle engine environment include altitude H, Mach number Ma, and main combustion chamber fuel flow W. f , Dual variable combustion chamber fuel flow W fD , afterburner fuel flow W fA , tail nozzle throat area A8, tail nozzle outlet area A9, intermediate pressure turbine guide vane angle A MT , Low-pressure turbine guide vane angle A LT , so the simulation input parameters of the high-throughput dual-variable cycle engine fusion model are as follows:

[0031] E T =[H t Ma t W f,t W fD,t W fA,t A 8,t A 9,t A MT,t A LT,t ]

[0032] Among them, the subscript t represents the current moment;

[0033] The current state of the high-throughput dual-variable cycle engine is represented by the residual e of the common working equation, so the input of the Actor network is:

[0034] S t =[E T e t ]

[0035] Since the output of the Actor network is the initial iteration parameter I, the state S-action A pair of the input and output of the Actor network is:

[0036] S t =[E T e t ] → A t =I t

[0037] The input of the Critic network is the above state S-action A pair, and the output is the expected reward Q t , so the input and output structure of the Critic network is as follows:

[0038] S t ~A t →Q t .

[0039] Optionally, in one embodiment of the present invention, step 4 is specifically:

[0040] Step 4.1: Select the Actor network and Critic network structure based on experience;

[0041] Step 4.2: Preliminary setting of agent hyperparameters;

[0042] Step 4.3: Tune system hyperparameters according to the residual of the joint working equation and the reward size;

[0043] Step 4.4: Obtain the high-throughput dual-variable cycle engine fusion model.

[0044] The method for constructing a high-throughput dual-variable cycle engine fusion model based on reinforcement learning in an embodiment of the present invention has the following features:

[0045] Beneficial effects:

[0046] 1. The common working equation of the high-throughflow dual-variable cycle engine selected by the present invention can achieve the performance matching of the whole machine.

[0047] 2. The fusion model architecture established by the present invention does not require pre-provided training data, which solves the problem of difficulty in obtaining full-envelope and full-operating-condition engine data in traditional methods.

[0048] 3. The high-throughput dual-variable cycle engine fusion model based on deep reinforcement learning proposed in the present invention has fast convergence speed, high accuracy and good real-time performance. It can break through the limitation that traditional models can only perform simulations at specific points, reduce the number of model iterations under conditions of sudden input parameters, and ensure convergence within the full envelope. It plays a positive role in promoting the application of engine onboard models and improving the performance reliability and safety of the entire engine.

[0049] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0051] Figure 1 A flowchart of a method for constructing a high-throughput dual-variable cycle engine fusion model based on reinforcement learning according to an embodiment of the present invention;

[0052] Figure 2 A schematic diagram of the structure of a high-throughflow dual-variable cycle engine according to an embodiment of the present invention;

[0053] Figure 3 It is a schematic diagram of the high-throughput dual-variable cycle engine fusion model architecture of an embodiment of the present invention;

[0054] Figure 4 This is a test diagram of the applicable scope of the high-throughflow dual-variable cycle engine fusion model according to an embodiment of the present invention;

[0055] Figure 5 It is a relative error diagram of the high-throughflow dual-variable cycle engine fusion model under various working conditions according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0057] Figure 1 The present invention provides a flowchart of a method for constructing a high-throughput dual-variable cycle engine fusion model based on reinforcement learning according to an embodiment of the present invention.

[0058] like Figure 1 As shown, the method for constructing a high-throughput dual-variable cycle engine fusion model based on reinforcement learning includes the following steps:

[0059] Step 1: Based on the aerodynamic thermodynamic mechanism of the high-throughput dual-variable cycle engine, select the common working equation to achieve overall machine performance matching.

[0060] In step 1, the aerodynamic thermodynamic working mechanism of each component of the high-throughflow dual-variable cycle engine is first analyzed, and the model of each component of the high-throughflow dual-variable cycle engine is established in combination with the modeling mechanism of each component of the existing variable cycle engine;

[0061] Firstly, based on the high-throughput dual-cycle engine including seven rotating parts and three rotating shafts, namely, the front fan FFan, the rear fan RFan, the core drive fan CDFS, the high-pressure compressor Comp, the high-pressure turbine Hturb, the intermediate-pressure turbine Mturb and the low-pressure turbine Lturb, the number of common working equations is determined.

[0062] Next, combining the working principles of engine flow continuity, static pressure balance and power conservation, the common working equation is selected as follows:

[0063]

[0064] in, Indicates flow rate, P srepresents static pressure, η represents turbine component efficiency, N represents power; digital subscripts represent engine section numbers, where 131 represents the third duct outlet, 142 represents the second duct outlet, 16 represents the rear duct outlet, cool242 represents the low-pressure turbine guide valve cooling air, cool261 represents the intermediate-pressure turbine guide valve cooling air, 29 represents the first duct outlet, 311 represents the hot end inlet of the heat exchanger, 4 represents the dual-variable combustion chamber outlet, 41 represents the high-pressure turbine guide valve outlet, 43 represents the high-pressure turbine outlet, 44 represents the high-pressure turbine guide valve outlet, 46 represents the intermediate-pressure turbine outlet, 48 represents the dual-variable combustion chamber outlet, 49 represents the low-pressure turbine guide valve outlet, 6 represents the low-pressure turbine outlet, 7 represents the afterburner outlet, and 9 represents the tail nozzle outlet; subscript H represents the high-pressure shaft, M represents the intermediate-pressure shaft, L represents the low-pressure shaft, and EX represents engine accessories.

[0065] Step 2: Combined with the common working equation, a high-throughput dual-variable cycle engine fusion model architecture is designed based on the Newton-Raphson algorithm and the double-delay deep deterministic policy gradient algorithm TD3.

[0066] In step 2, we first compare and analyze the mechanism of the deep reinforcement learning algorithm, and select an algorithm that meets the requirements for establishing a high-throughput dual-variable cycle engine fusion model and is suitable for environments with complex environments and high action dimensions.

[0067] Finally, the dual-delay deep deterministic policy gradient algorithm TD3 was selected, and it was combined with the Newton-Raphson algorithm to design a high-throughput dual-variable cycle engine fusion model architecture.

[0068] The designed high-throughput dual-variable cycle engine fusion model architecture includes two sets of iterative systems, inner and outer loops. When the initial iteration parameters are inaccurate, resulting in a large residual e of the engine common working equation, which is greater than the specified constant ε, the outer loop system based on the TD3 agent is used for iteration; when the residual e is less than the specified constant ε, the Newton-Raphson algorithm is used for inner loop iteration.

[0069] During the training process, since the initial iteration parameters are randomly given, the residual of the common working equation group is large, greater than the specified constant ε, and the fusion model is connected to the Actor of the outer loop system for training. t and the state S obtained by the engine flow calculation module t , S t+1 and reward r t The expected reward Q is stored in the playback buffer for the training of the Critic. t The error is updated to further optimize the Actor’s update strategy.

[0070] During the use phase, when the residual is large and greater than the specified constant ε, the trained Actor is called to output the initial iteration parameters until the residual e is less than the specified constant ε. At this time, the high-throughput dual-variable cycle engine fusion model is connected to the inner loop and iterated by the Newton-Raphson algorithm until the high-throughput dual-variable cycle engine fusion model converges.

[0071] Step 3: Design the system input-output structure and reward system based on the working mechanism of the high-throughput dual-variable cycle engine fusion model.

[0072] In step 3, by analyzing the working mechanism of the fusion model, it can be seen that the TD3-based agent contains two networks, Actor and Critic. In the fusion model, the Actor is mainly used to generate the initial iteration parameters, and the Critic is mainly used to judge the quality of the current initial iteration parameters and guide the Actor network update.

[0073] Design the input and output structures of the Actor network and the Critic network, because the quality of the initial iteration parameters of the engine depends largely on the working environment of the engine. T and current status.

[0074] The input parameters that determine the high-throughput dual-cycle engine environment mainly include altitude H, Mach number Ma, main combustion chamber fuel flow W f , Dual variable combustion chamber fuel flow W fD , afterburner fuel flow W fA , tail nozzle throat area A8, tail nozzle outlet area A9, intermediate pressure turbine guide vane angle A MT , Low-pressure turbine guide vane angle A LT , so the environmental parameters are as follows:

[0075] E T =[H t Ma t W f,t W fD,t W fA,t A 8,t A 9,t A MT,t A LT,t ]

[0076] Among them, the subscript t represents the current moment;

[0077] The current state of the high-throughput dual-variable cycle engine is mainly represented by the residual e of the common working equation, so the input of the Actor network is:

[0078] S t =[E T e t ]

[0079] Since the output of the Actor network is the initial iteration parameter I, the state S-action A pair of the input and output of the Actor network is:

[0080] S t =[E T e t ] → A t =I t

[0081] The input of the Critic network is the above state S-action A pair, and the output is the expected reward Q t , so the input and output structure of the Critic network is as follows:

[0082] S t ~A t →Q t

[0083] Finally, the setting of the agent reward system is mainly to guide the agent to generate accurate iteration parameters faster, so the convergence and convergence speed of the model need to be considered comprehensively. The convergence of the training phase is mainly determined by the sum of the absolute values ​​of the residuals of the common working equation The convergence speed is mainly determined by the number of required inner loop Newton-Raphson algorithm iterations K t The inner loop can converge. As the boundary, it can converge Reward t The settings are as follows:

[0084]

[0085] Step 4: According to the residual of the joint working equation and the designed reward system, the hyperparameters of the high-throughput dual-variable cycle engine fusion model are tuned.

[0086] In step 4, the Actor network and Critic network structures are first selected based on experience; then the agent hyperparameters are preliminarily given; then the system hyperparameters are adjusted based on the residual of the joint working equation and the reward size; finally, a high-throughput dual-variable cycle engine fusion model is obtained.

[0087] The present invention takes the high-throughput dual-variable cycle engine as the research object, combines the powerful information extraction ability of deep reinforcement learning and the characteristics of interaction with the environment, and establishes a fusion model based on the double-delay deep deterministic policy gradient TD3 algorithm. The model has good real-time performance, high precision and strong convergence. It can break through the limitation that the traditional model can only carry out simulation at specific points, reduce the number of iterations of the model under sudden changes in input parameters, and ensure convergence within the full envelope.

[0088] The implementation object of the embodiment of the present invention is a high-throughflow dual-cycle engine, which includes three rotating parts: a high-pressure shaft, an intermediate-pressure shaft and a low-pressure shaft. Figure 2 shown.

[0089] In a specific embodiment, step 1 is specifically:

[0090] Firstly, the aerodynamic thermodynamic working mechanism of each component of high-throughput dual variable cycle engine is analyzed, and the model of each component of high-throughput dual variable cycle engine is established by combining the modeling mechanism of each component of existing variable cycle engine.

[0091] Next, the high-throughflow dual-cycle engine is analyzed, including 7 rotating parts and 3 rotating shafts, including the front fan FFan, the rear fan RFan, the core drive fan CDFS, the high-pressure compressor Comp, the high-pressure turbine Hturb, the intermediate-pressure turbine Mturb and the low-pressure turbine Lturb, and the number of common working equations is determined to be 10;

[0092] Finally, combining the working principles of engine flow continuity, static pressure balance and power conservation, the common working equation is selected as follows:

[0093]

[0094] in, Indicates flow rate, P s represents static pressure, η represents turbine component efficiency, N represents power; digital subscripts represent engine section numbers, where 131 represents the third duct outlet, 142 represents the second duct outlet, 16 represents the rear duct outlet, cool242 represents the low-pressure turbine guide valve cooling air, cool261 represents the intermediate-pressure turbine guide valve cooling air, 29 represents the first duct outlet, 311 represents the hot end inlet of the heat exchanger, 4 represents the dual-variable combustion chamber outlet, 41 represents the high-pressure turbine guide valve outlet, 43 represents the high-pressure turbine outlet, 44 represents the high-pressure turbine guide valve outlet, 46 represents the intermediate-pressure turbine outlet, 48 represents the dual-variable combustion chamber outlet, 49 represents the low-pressure turbine guide valve outlet, 6 represents the low-pressure turbine outlet, 7 represents the afterburner outlet, and 9 represents the tail nozzle outlet; subscript H represents the high-pressure shaft, M represents the intermediate-pressure shaft, L represents the low-pressure shaft, and EX represents engine accessories.

[0095] In a specific embodiment, step 2 is specifically:

[0096] Firstly, we compared and analyzed the mechanism of deep reinforcement learning algorithms, and selected an algorithm that was suitable for complex environments and high action dimensions and that met the requirements for establishing a high-throughput dual-variable cycle engine fusion model. Finally, we selected the TD3 algorithm.

[0097] Next, we design a high-throughput dual-variable cycle engine fusion model architecture by combining the dual-delay deep deterministic policy gradient algorithm TD3 and the Newton-Raphson algorithm. Figure 3 shown.

[0098] The fusion model includes two sets of iterative systems, the inner and outer loops. When the initial iteration parameters are inaccurate, resulting in a large residual e of the engine joint working equation, the valve is opened upward, the Actor network is connected, and the outer loop system based on the TD3 agent is used for iteration; when the initial iteration variables given by the Actor satisfy the flow path calculation residual e less than the specified constant ε, the valve is opened downward and the inner loop iteration is performed using the Newton-Raphson algorithm.

[0099] During the training process, since the initial iteration parameters are randomly given and the residual of the joint working equations is large, the fusion model is connected to the Actor of the outer loop system for training. t and the state S obtained by the engine flow calculation module t , S t+1 and reward r t The expected reward Q is stored in the playback buffer for the training of the Critic. t The error is updated to further optimize the Actor’s update strategy.

[0100] During the use phase, when the residual is large, the trained Actor is called to output the initial iteration parameters until the residual e is less than the specified constant ε. At this time, the model is connected to the inner loop and iterated by the Newton-Raphson algorithm until the high-throughput dual-variable cycle engine fusion model converges.

[0101] In a specific embodiment, step 3 is specifically:

[0102] In step 3, the working mechanism of the fusion model is analyzed, and it can be seen that the TD3-based agent contains two networks, Actor and Critic. In the fusion model, Actor is mainly used to generate initial iteration parameters, and Critic is mainly used to judge the quality of the current initial iteration parameters and guide the update of Actor network.

[0103] Design the input and output structures of the Actor network and the Critic network, because the quality of the initial iteration parameters of the engine depends largely on the working environment of the engine. T and current status.

[0104] The input parameters that determine the high-throughput dual-cycle engine environment mainly include altitude H, Mach number Ma, main combustion chamber fuel flow W f , Dual variable combustion chamber fuel flow W fD , afterburner fuel flow W fA , tail nozzle throat area A8, tail nozzle outlet area A9, intermediate pressure turbine guide vane angle A MT , Low-pressure turbine guide vane angle ALT , so the environmental parameters are as follows:

[0105] E T =[H t Ma t W f,t W fD,t W fA,t A 8,t A 9,t A MT,t A LT,t ]

[0106] The subscript t represents the current time.

[0107] The current state of the high-throughput dual-variable cycle engine is mainly represented by the residual e of the common working equation, so the input of the Actor network is:

[0108] S t =[E T e t ]

[0109] Since the output of the Actor network is the initial iteration parameter I, the state S-action A pair of the input and output of the Actor network is:

[0110] S t =[E T e t ] → A t =I t

[0111] The input of the Critic network is the above state S-action A pair, and the output is the expected reward Q t , so the input and output structure of the Critic network is as follows:

[0112] S t ~A t →Q t

[0113] Finally, the setting of the agent reward system is mainly to guide the agent to generate accurate iteration parameters faster, so the convergence and convergence speed of the model need to be considered comprehensively. The convergence of the training phase is mainly determined by the sum of the absolute values ​​of the residuals of the common working equation The convergence speed is mainly determined by the number of required inner loop Newton-Raphson algorithm iterations K t The inner loop can converge. As the boundary, it can converge Reward t The settings are as follows:

[0114]

[0115] In a specific embodiment, step 4 is specifically:

[0116] First, based on experience, it is selected that the Actor network contains two hidden layers with 400 and 300 nodes respectively; the state path of the Critic network structure contains two hidden layers with 400 and 300 nodes respectively, the action path contains one hidden layer with 300 nodes, and the common path contains an addition layer and an output layer.

[0117] After the agent hyperparameters are initially given, the system hyperparameters are adjusted according to the residual of the joint working equation and the reward size. The final selected fusion model parameters are shown in Table 1:

[0118] Table 1. Fusion model hyper parameters

[0119]

[0120] Through training, a high-throughput dual-variable cycle engine fusion model is finally obtained.

[0121] Simulation Results Figure 4 It shows that the established fusion model can almost cover 70%-100% of the maximum speed in the full envelope, that is, from slow speed to maximum speed, which shows that it can work at various typical operating points for safe operation of the engine. Figure 5 It can be seen that the relative errors of the key parameters of the fusion model in the slow-speed state, subsonic cruise state and maximum afterburner state are no more than 1.5%, and the model accuracy is high. The real-time performance of the fusion model is tested on a host computer with a CPU i7-14700 and a main frequency of 2.10GHz, and the average single-step simulation time is no more than 0.157ms. The real-time performance is tested on a P2020 development board with a main frequency of 1GHz, and the average single-step simulation time of the model is no more than 2.949ms.

[0122] The high-throughput dual-variable cycle engine fusion model established by the high-throughput flow dual-variable cycle engine fusion model construction method based on reinforcement learning proposed in an embodiment of the present invention has fast convergence speed, high accuracy and good real-time performance. It can break through the limitation that traditional models can only carry out simulation at specific points, reduce the number of model iterations under sudden input parameters and ensure convergence within the full envelope.

[0123] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0124] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0125] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

Claims

1. A method for constructing a high-throughput dual-variable cycle engine fusion model based on reinforcement learning, characterized in that: The following steps are involved: Step 1: Based on the aerodynamic thermodynamic mechanism of the high-throughput dual-variable cycle engine, select the common working equation to achieve the performance matching of the whole machine; Step 2: combining the common working equation, designing a high-throughput dual-variable cycle engine fusion model architecture based on the Newton-Raphson algorithm and the double-delay deep deterministic policy gradient algorithm TD3; Step 3: Design the system input and output structure and reward system based on the working mechanism of the high-throughput dual-variable cycle engine fusion model; Step 4: According to the residual of the joint working equation and the designed reward system, the hyperparameters of the high-throughput dual-variable cycle engine fusion model are tuned.

2. The method according to claim 1, characterized in that Step 1 specifically includes: Step 1.1: Analyze the aerodynamic thermodynamic working mechanism of each component of the high-throughflow dual-variable cycle engine, and establish a mathematical model of typical components of the high-throughflow dual-variable cycle engine in combination with the working mechanism of each component of the existing variable cycle engine; Step 1.2: Analyze the typical components of the high-throughput dual-cycle engine: including the front fan FFan, the rear fan RFan, the core drive fan CDFS, the high-pressure compressor Comp, the high-pressure turbine Hturb, the intermediate-pressure turbine Mturb and the low-pressure turbine Lturb, and the working mechanism of the seven rotating parts and three rotating shafts, and determine the number of common working equations; Step 1.3: Combining the working principles of engine flow continuity, static pressure balance and power conservation, the common working equation is selected as: in, Indicates flow rate, P s represents static pressure, η represents mechanical efficiency of rotor shaft, N represents power; digital subscript represents engine section number, 131 represents outlet of third duct, 142 represents outlet of second duct, 16 represents outlet of rear duct, cool242 represents cooling air of low-pressure turbine guide vane, cool261 represents cooling air of intermediate-pressure turbine guide vane, 29 represents outlet of first duct, 311 represents inlet of hot end of heat exchanger, 4 represents outlet of combustion chamber, 41 represents outlet of high-pressure turbine guide vane, 43 represents outlet of high-pressure turbine, 44 represents outlet of high-pressure turbine guide vane, 46 represents outlet of intermediate-pressure turbine, 48 represents outlet of dual-variable combustion chamber, 49 represents outlet of low-pressure turbine guide vane, 6 represents outlet of low-pressure turbine, 7 represents outlet of afterburner, 9 represents outlet of tail nozzle; subscript H represents high-pressure shaft, M represents intermediate-pressure shaft, L represents low-pressure shaft, EX represents engine accessories.

3. The method according to claim 1, characterized in that Step 2 specifically includes: Step 2.1: Compare and analyze the mechanism of deep reinforcement learning algorithms, and select the algorithm required to establish the high-throughput dual-variable cycle engine fusion model; Step 2.2: Combine the double-delay deep deterministic policy gradient algorithm TD3 and the Newton-Raphson algorithm to design the high-throughput dual-variable cycle engine fusion model architecture.

4. The method according to claim 1, characterized in that: The high-throughput dual-variable cycle engine fusion model includes two sets of iterative systems, inner and outer loops. When the initial iteration parameters are inaccurate and the residual e of the engine common working equation is greater than the specified constant ε, the outer loop system based on the TD3 agent is used for iteration; when the residual e is less than the specified constant ε, the Newton-Raphson algorithm is used for inner loop iteration; During the training process, since the initial iteration parameters are randomly given, the residual e of the common working equation group is greater than the specified constant ε. The high-throughput dual-variable cycle engine fusion model is connected to the Actor of the outer loop system for training. The action A output by the Actor t and the state S obtained by the engine flow calculation module t , S t+1 and reward r t Stored together in the playback buffer for the training of Critic, Critic based on the expected return Q t The error is updated to further optimize the Actor's update strategy; During the use phase, when the residual e is greater than the specified constant ε, the trained Actor is called to output the initial iteration parameters until the residual e is less than the specified constant ε. At this time, the high-throughput dual-variable cycle engine fusion model is connected to the inner loop and iterated by the Newton-Raphson algorithm until the high-throughput dual-variable cycle engine fusion model converges.

5. The method according to claim 1, characterized in that Step 3 specifically includes: Step 3.1: Analyze the working mechanism of the high-throughput dual-variable cycle engine fusion model. The TD3-based intelligent agent includes two networks, Actor and Critic. In the high-throughput dual-variable cycle engine fusion model, Actor is used to generate initial iteration parameters, and Critic is used to judge the quality of the current initial iteration parameters and guide the Actor network update; Step 3.2: Design the input and output structures of the Actor network and Critic network; Step 3.3: The setting of the agent reward system is to guide the agent to generate accurate iteration parameters faster, so it is necessary to comprehensively consider the convergence and convergence speed of the high-throughput dual-variable cycle engine fusion model. The convergence of the training stage is mainly determined by the sum of the absolute values ​​of the residuals of the common working equations. The convergence rate is expressed as the number of required inner loop Newton-Raphson algorithm iterations K t Determine the flag bit that the inner loop can converge As the boundary, it can converge Reward t The settings are: The subscript t represents the current time.

6. The method according to claim 5, characterized in that The quality of the initial iteration parameters of the engine depends on the working environment E of the engine T and the current state, The input parameters that determine the high-throughput dual-cycle engine environment include altitude H, Mach number Ma, and main combustion chamber fuel flow W. f , Dual variable combustion chamber fuel flow W fD , afterburner fuel flow W fA , tail nozzle throat area A8, tail nozzle outlet area A9, intermediate pressure turbine guide vane angle A MT , Low-pressure turbine guide vane angle A LT , so the simulation input parameters of the high-throughput dual-variable cycle engine fusion model are as follows: HAVE BEEN T =[H t Ma t W f,t W fD,t W fA,t A 8,t A 9,t A MT,t A LT,t ] Among them, the subscript t represents the current moment; The current state of the high-throughput dual-variable cycle engine is represented by the residual e of the common working equation, so the input of the Actor network is: S t =[E T yes t ] Since the output of the Actor network is the initial iteration parameter I, the state S-action A pair of the input and output of the Actor network is: S t =[E T yes t ]→A t =I t The input of the Critic network is the above state S-action A pair, and the output is the expected reward Q t , so the input and output structure of the Critic network is as follows: S t A t →Q t 。 7. The method according to claim 6, characterized in that Step 4 is as follows: Step 4.1: Select the Actor network and Critic network structure based on experience; Step 4.2: Preliminary setting of agent hyperparameters; Step 4.3: Tune system hyperparameters according to the residual of the joint working equation and the reward size; Step 4.4: Obtain the high-throughput dual-variable cycle engine fusion model.

Images