Design method and system of intelligent controller for solid-liquid variable thrust engine

Abstract

The present invention provides a method and system for designing an intelligent controller for a solid-liquid variable thrust engine. The method comprises: using the actual thrust, thrust error, power of an electric pump, and valve opening of a variable venturi as observation inputs, and control instructions for the electric pump and variable venturi as controller outputs, to construct a target reinforcement learning controller. The thrust error is the error between the actual thrust and the desired thrust of the solid-liquid variable thrust engine. The electric pump is a booster electric pump for the oxidizer circuit of the solid-liquid variable thrust engine. The electric pump is a low-power battery-powered electric pump. Based on the target reinforcement learning controller, an intelligent controller for the solid-liquid variable thrust engine is designed. This invention alleviates the technical problem of difficult control in the prior art caused by the low power consumption of the electric pump and the low energy density constraints of the battery.

Description

Technical Field

[0001] The present invention relates to the technical field of rocket engine intelligent control, and in particular to a design method and system for an intelligent controller of a solid-liquid variable thrust engine. Background Art

[0002] A typical hybrid rocket engine generates thrust through a chemical reaction between solid fuel and liquid oxidizer. The solid fuel is poured into the combustion chamber in a specific configuration. The liquid oxidizer is pressurized by extrusion or pumping and then delivered into the combustion chamber through a hydraulic conveying system. There, it reacts chemically with the solid fuel, producing high-temperature, high-pressure gas. This gas is accelerated through the nozzle, forming a high-speed, low-temperature gas that is ejected backward, generating a forward reaction force.

[0003] The solid-liquid hybrid variable thrust rocket engine, as its inheritance and development, has the function of regulating the engine thrust according to demand. Generally, it adjusts the valve opening in the liquid transportation system or the power of the electric pump in real time to achieve the regulation of the oxidant flow, thereby affecting the chemical reaction process, controlling the energy released by the reaction, and ultimately achieving the function of controlling the engine thrust.

[0004] Electric pumps need to be equipped with a driver, but high-power drivers are heavy and difficult to use, and there are limitations with low-power drivers. Therefore, electric pumps need to carry power batteries. However, battery energy density is limited, and battery energy consumption becomes one of the constraints of engine control, which in turn leads to technical problems that make electric pumps difficult to control. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a method and system for designing an intelligent controller for a solid-liquid variable thrust engine, so as to alleviate the technical problems existing in the prior art that are difficult to control due to the low power drive of the electric pump and the low energy density constraint of the battery.

[0006] In a first aspect, an embodiment of the present invention provides a method for designing an intelligent controller for a solid-liquid variable thrust engine, comprising: using the actual thrust, thrust error, power of an electric pump, and valve opening of a variable venturi of the solid-liquid variable thrust engine as observation inputs, and using control instructions of the electric pump and the variable venturi as controller outputs to construct a target reinforcement learning controller; the thrust error is the error between the actual thrust and the expected thrust of the solid-liquid variable thrust engine; the electric pump is a booster electric pump for an oxidizer circuit of the solid-liquid variable thrust engine; and based on the target reinforcement learning controller, designing an intelligent controller for the solid-liquid variable thrust engine.

[0007] Furthermore, the method further includes: training the target reinforcement learning controller using a reinforcement learning algorithm.

[0008] Furthermore, the reward function of the reinforcement learning algorithm is: R = β1R e +β2R p ; Among them, β1 and β2 are reward weight coefficients, and β1+β2=1; R e is the error reward that is negatively correlated with the absolute value of the thrust error, R p Bonus for electric pump power related to battery power.

[0009] Furthermore, based on the target reinforcement learning controller, an intelligent controller of the solid-liquid variable thrust engine is designed, including: connecting the output end of the intelligent controller to the control input end of the electric pump and the control input end of the variable venturi, respectively, so that the intelligent controller controls the output liquid oxidizer flow of the electric pump and the variable venturi simultaneously based on the control instructions output in response to the state observation quantity feedback from the measurement module; the measurement module is used to obtain the state observation quantity of the solid-liquid variable thrust engine; the state observation quantity includes: the actual thrust of the solid-liquid variable thrust engine, the power of the electric pump and the valve opening of the variable venturi.

[0010] Furthermore, the intelligent controller responds to the control instructions output by the state observable fed back by the measurement module, including: a first control instruction for controlling the electric pump and a second control instruction for controlling the variable venturi; wherein the sum of the first liquid oxidizer flow rate output by the electric pump based on the first control instruction and the second liquid oxidizer flow rate output by the variable venturi based on the second control instruction is the liquid oxidizer flow rate required when the thrust of the solid-liquid variable thrust engine is the desired thrust.

[0011] In a second aspect, an embodiment of the present invention further provides a solid-liquid variable thrust engine intelligent controller design system, comprising: a construction module and a design module; wherein the construction module is used to construct a target reinforcement learning controller using the actual thrust, thrust error, power of the electric pump and valve opening of the variable venturi of the solid-liquid variable thrust engine as observation inputs, and the control instructions of the electric pump and the variable venturi as controller outputs; the thrust error is the error between the actual thrust and the expected thrust of the solid-liquid variable thrust engine; the electric pump is a booster electric pump for the oxidizer circuit of the solid-liquid variable thrust engine; the design module is used to design the intelligent controller of the solid-liquid variable thrust engine based on the target reinforcement learning controller.

[0012] Furthermore, the system further comprises: a training module for training the target reinforcement learning controller using a reinforcement learning algorithm; wherein the reward function of the reinforcement learning algorithm is: R = β1R e +β2R pβ1 and β2 are reward weight coefficients, and β1+β2=1; R e is the error reward that is negatively correlated with the absolute value of the thrust error, R p Bonus for electric pump power related to battery power.

[0013] Furthermore, the design module is also used to: connect the output end of the intelligent controller to the control input end of the electric pump and the control input end of the variable venturi respectively, so that the intelligent controller can simultaneously control the output liquid oxidizer flow of the electric pump and the variable venturi based on the control instructions output in response to the state observation quantity feedback from the measurement module; the measurement module is used to obtain the state observation quantity of the solid-liquid variable thrust engine; the state observation quantity includes: the actual thrust of the solid-liquid variable thrust engine, the power of the electric pump and the valve opening of the variable venturi.

[0014] In a third aspect, an embodiment of the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the first aspect when executing the computer program.

[0015] In a fourth aspect, an embodiment of the present invention further provides a computer-readable medium having a non-volatile program code executable by a processor, wherein the program code enables the processor to execute the method described in the first aspect.

[0016] The present invention provides a method and system for designing an intelligent controller for a solid-liquid variable thrust engine. By controlling the solid-liquid variable thrust engine through an intelligent controller based on reinforcement learning, the solid-liquid variable thrust engine can complete the thrust control task while satisfying the low-power constraint of an electric pump, one of the actuators. This improves the stability of the electric pump of the solid-liquid variable thrust engine without carrying a high-power electric pump, and alleviates the technical problem of difficult control caused by the low-power drive of the electric pump and the low energy density constraint of the battery in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 A flow chart of a method for designing an intelligent controller for a solid-liquid variable thrust engine provided by an embodiment of the present invention;

[0019] Figure 2 A schematic diagram of the interaction between an intelligent agent and an environment for reinforcement learning provided by an embodiment of the present invention;

[0020] Figure 3 A schematic structural diagram of a solid-liquid variable thrust controller provided in an embodiment of the present invention;

[0021] Figure 4 A schematic diagram of a solid-liquid variable thrust engine intelligent controller design system provided by an embodiment of the present invention;

[0022] Figure 5 A schematic diagram of another solid-liquid variable thrust engine intelligent controller design system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0024] Example 1:

[0025] Figure 1 This is a flow chart of a method for designing an intelligent controller for a solid-liquid variable thrust engine according to an embodiment of the present invention. Figure 1 As shown, the method specifically includes the following steps:

[0026] In step S102, a target reinforcement learning controller is constructed using the actual thrust of the solid-liquid variable thrust engine, the thrust error, the power of the electric pump, and the valve opening of the variable venturi as observation inputs, and the control instructions of the electric pump and the variable venturi as controller outputs. The thrust error is the error between the actual thrust and the desired thrust of the solid-liquid variable thrust engine. The electric pump is a booster electric pump for the oxidizer circuit of the solid-liquid variable thrust engine. The electric pump is a low-power battery-driven electric pump.

[0027] Step S104: Design an intelligent controller for the solid-liquid variable thrust engine based on the target reinforcement learning controller.

[0028] The present invention provides a method for designing an intelligent controller for a solid-liquid variable thrust engine. By controlling the solid-liquid variable thrust engine through an intelligent controller based on reinforcement learning, the solid-liquid variable thrust engine can complete the thrust control task while satisfying the low power constraint of an electric pump, one of the actuators. This improves the stability of the system of the electric pump of the solid-liquid variable thrust engine without carrying a high-power electric pump, and alleviates the technical problem of difficult control caused by the low power drive of the electric pump and the low energy density constraint of the battery in the prior art.

[0029] In the method provided by the embodiment of the present invention, the intelligent controller is trained using a reinforcement learning algorithm. Reinforcement learning can autonomously learn the optimal strategy by continuously interacting with the environment, and is considered to be one of the most important ways to achieve general artificial intelligence computing. The problem discussed in reinforcement learning is how an intelligent agent can maximize the reward it obtains in a complex and uncertain environment. In reinforcement learning, the interaction between the intelligent agent and the environment is as follows: Figure 2 shown.

[0030] like Figure 2 As shown, this process can be represented by a Markov decision process. The Markov decision process can be represented by a tuple (S, A, T, R, γ). Where S is a finite state space set; A is a finite behavior space set; and T is a state transition model:

[0031] T(s t+1 =s′|s t =s,a t =a) (1)

[0032] R is the reward function:

[0033] R(s t =s,a t =a) = E[r t |s t =s,a t =a] (2)

[0034] γ is the decay factor, γ∈[0,1].

[0035] Given a Markov decision process policy π:

[0036] π(a|s)=P(a t =a|s t =s) (3)

[0037] Define the reward function G t :

[0038] G t =R t+1 +γR t+2+…+γ T-t-1 R T (4)

[0039] The reward function is the benefit obtained after discounting the reward. Here we introduce the Q function, which is the action value function q π (s,a) represents the expectation of taking an action when executing strategy π in a certain state:

[0040] Q π (s,a)=E[G t |s t =s,A t =a] (5)

[0041] After summing the action value functions, we get the value function V π (s):

[0042]

[0043] The principle of updating the value function network is to minimize the loss function. The mathematical representation of the loss function is as follows:

[0044]

[0045] where y i =r i +γQ′(s i+1 ,μ′(s i+1 ∣θ μ′ )|θ Q′ ), you can see y i It is also a Q Parameter-dependent functions.

[0046] Assume that the objective function of training is:

[0047]

[0048] Where γ is the attenuation factor, that is, the farther the reward value is from the current action, the less influence it has on the network. The policy network is updated according to the gradient of the objective function, and its mathematical expression is as follows:

[0049]

[0050] Finally update the target network during training:

[0051] θ Q′ ←τθ Q +(1-τ)θ Q′

[0052] θ μ′ ←τθ μ +(1-τ)θ μ′(10)

[0053] Optionally, the method provided by the embodiment of the present invention further includes: training the target reinforcement learning controller using a reinforcement learning algorithm.

[0054] The reinforcement learning-based intelligent controller designed in this embodiment of the present invention observes as input the desired thrust Tr, thrust error e, valve opening δ, and electric pump power P. The intelligent controller outputs control commands for the electric pump and variable venturi. The reward function is designed as an incremental synthesis of multiple rewards, each related to the thrust error and the electric pump power (i.e., valve opening).

[0055] Specifically, the reward function of the reinforcement learning algorithm is: R = β1R e +β2R p β1 and β2 are reward weight coefficients, and β1+β2=1; R e is the error reward that is negatively correlated with the absolute value of the thrust error, that is, the smaller the thrust error, the greater the reward; R p The electric pump power reward is related to the battery power. Optionally, in the embodiment of the present invention, when the electric pump power is between 0.3 times and 0.7 times the maximum power, a positive reward is obtained, and in other cases, a negative reward is obtained. Within the positive reward range, the smaller the battery power, the greater the reward. At the same time, the thrust error is also used to limit the amplitude of the electric pump power reward. When the thrust error is small, the reward is larger, so that the electric pump power constraint problem is solved while giving priority to satisfying the smaller error. Preferably, in the embodiment of the present invention, R p <R e .

[0056] Figure 3 Schematic diagram of a solid-liquid variable thrust controller according to an embodiment of the present invention. Figure 3 As shown, specifically, step S104 also includes the following steps: connecting the output end of the intelligent controller to the control input end of the electric pump and the control input end of the variable venturi respectively, so that the intelligent controller can control the output liquid oxidant flow of the electric pump and the variable venturi simultaneously based on the control instructions output in response to the state observation quantity feedback from the measurement module.

[0057] The measurement module is used to obtain state observations of the solid-liquid variable thrust engine. These state observations include the actual thrust of the solid-liquid variable thrust engine, the power of the electric pump, and the valve opening of the variable venturi. Optionally, the state observations also include pipeline pressure, electric pump speed, etc.

[0058] Alternatively, as Figure 3As shown, the intelligent controller responds to the control instructions output by the state observation quantity fed back by the measurement module, including: a first control instruction u1 for controlling the electric pump and a second control instruction u2 for controlling the variable venturi; wherein, the sum of the first liquid oxidant flow rate w1 output by the electric pump based on the first control instruction u1 and the second liquid oxidant flow rate w2 output by the variable venturi based on the second control instruction u2 is the liquid oxidant flow rate required when the thrust of the solid-liquid variable thrust engine is the desired thrust.

[0059] In an embodiment of the present invention, the solid-liquid variable thrust engine needs to control the electric pump and the variable venturi at the same time, and the liquid oxidant flow rates w1 and w2 output by the electric pump and the variable venturi are respectively obtained according to the control instructions given by the intelligent controller.

[0060] According to the intelligent controller designed by the method provided in an embodiment of the present invention, under low power constraints, the output capacity of the engine electric pump is reduced, and the variable venturi and electric pump need to be coordinated and scheduled to meet the engine output requirements. The reinforcement learning intelligent controller comprehensively considers the control instruction size of the two, and can realize intelligent control of the solid-liquid variable thrust engine under low power constraints.

[0061] Based on the tracking of actual thrust against desired thrust, an intelligent controller is further employed to manage the battery energy of the electric pump. This function is implemented by the RP component. Under the combined conditions of small error and low battery output power, the intelligent controller receives a large reward, thereby managing battery power, or energy, and ultimately achieving a control method for a solid-liquid variable thrust engine that optimizes battery energy.

[0062] As can be seen from the foregoing description, the intelligent controller design method for a solid-liquid variable thrust engine provided by the present invention utilizes a reinforcement learning intelligent controller to control the solid-liquid hybrid variable thrust engine, enabling it to complete thrust control tasks while satisfying the low-power constraints of the electric pump, one of the actuators. This eliminates the need for a high-power electric pump for the solid-liquid variable thrust engine, reducing engine dead weight and improving system stability. Furthermore, intelligent control is used to manage the electric pump battery energy, achieving a control strategy that optimizes battery energy consumption and extending the engine's operating life.

[0063] Example 2:

[0064] Figure 4 Schematic diagram of a solid-liquid variable thrust engine intelligent controller design system according to an embodiment of the present invention. Figure 4 As shown, the system includes: a construction module 10 and a design module 20.

[0065] Specifically, module 10 is constructed to construct a target reinforcement learning controller using the actual thrust, thrust error, power of the electric pump and valve opening of the variable venturi of the solid-liquid variable thrust engine as observation inputs, and the control instructions of the electric pump and the variable venturi as controller outputs; the thrust error is the error between the actual thrust and the expected thrust of the solid-liquid variable thrust engine; the electric pump is a booster electric pump for the oxidizer circuit of the solid-liquid variable thrust engine; and the electric pump is a low-power battery-driven electric pump.

[0066] The design module 20 is used to design an intelligent controller for a solid-liquid variable thrust engine based on a target reinforcement learning controller.

[0067] The present invention provides a solid-liquid variable thrust engine intelligent controller design system. By controlling the solid-liquid variable thrust engine through an intelligent controller based on reinforcement learning, the solid-liquid variable thrust engine can complete the thrust control task while satisfying the low power constraint of an electric pump, one of the actuators. This improves the stability of the solid-liquid variable thrust engine electric pump without carrying a high-power electric pump, and alleviates the technical problem of difficult control caused by the low power drive of the electric pump and the low energy density constraint of the battery in the prior art.

[0068] Optionally, Figure 5 Schematic diagram of another solid-liquid variable thrust engine intelligent controller design system provided according to an embodiment of the present invention. Figure 5 As shown, the system further includes: a training module 30 for training the target reinforcement learning controller using a reinforcement learning algorithm; wherein the reward function of the reinforcement learning algorithm is: R = β1R e +β2R p β1 and β2 are reward weight coefficients, and β1+β2=1; R e is the error reward that is negatively correlated with the absolute value of the thrust error, R p Bonus for electric pump power related to battery power.

[0069] Optionally, the design module 20 is further configured to:

[0070] The output end of the intelligent controller is connected to the control input end of the electric pump and the control input end of the variable venturi respectively, so that the intelligent controller can simultaneously control the output liquid oxidizer flow rate of the electric pump and the variable venturi based on the control instructions output in response to the state observation quantity feedback from the measurement module; wherein the measurement module is used to obtain the state observation quantity of the solid-liquid variable thrust engine; the state observation quantity includes: the actual thrust of the solid-liquid variable thrust engine, the power of the electric pump and the valve opening of the variable venturi.

[0071] An embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method in the first embodiment are implemented.

[0072] An embodiment of the present invention further provides a computer-readable medium having a non-volatile program code executable by a processor, wherein the program code enables the processor to execute the method in the above-mentioned embodiment 1.

[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for designing an intelligent controller for a solid-liquid variable thrust engine, characterized in that: include: A target reinforcement learning controller is constructed using the actual thrust of a solid-liquid variable thrust engine, the thrust error, the power of an electric pump, and the valve opening of a variable venturi as observation inputs, and control instructions for the electric pump and the variable venturi as controller outputs. The thrust error is the error between the actual thrust and the desired thrust of the solid-liquid variable thrust engine. The electric pump is a booster electric pump for the oxidizer circuit of the solid-liquid variable thrust engine. Based on the target reinforcement learning controller, an intelligent controller of the solid-liquid variable thrust engine is designed, wherein the intelligent controller of the solid-liquid variable thrust engine is designed based on the target reinforcement learning controller, including: connecting the output end of the intelligent controller to the control input end of the electric pump and the control input end of the variable venturi respectively, so that the intelligent controller controls the output liquid oxidizer flow of the electric pump and the variable venturi simultaneously based on the control instructions output in response to the state observation quantity feedback from the measurement module; the measurement module is used to obtain the state observation quantity of the solid-liquid variable thrust engine; the state observation quantity includes: the actual thrust of the solid-liquid variable thrust engine, the power of the electric pump and the valve opening of the variable venturi.

2. The method according to claim 1, characterized in that The method further includes: training the target reinforcement learning controller using a reinforcement learning algorithm.

3. The method according to claim 2, characterized in that The reward function of the reinforcement learning algorithm is: R=β1R e +β2R p ; Among them, β1 and β2 are reward weight coefficients, and β1+β2=1; R e is the error reward that is negatively correlated with the absolute value of the thrust error, R p Bonus for electric pump power related to battery power.

4. The method according to claim 1, wherein The intelligent controller responds to the control instructions output by the state observable fed back by the measurement module, including: a first control instruction for controlling the electric pump and a second control instruction for controlling the variable venturi; wherein the sum of a first liquid oxidizer flow rate output by the electric pump based on the first control instruction and a second liquid oxidizer flow rate output by the variable venturi based on the second control instruction is the liquid oxidizer flow rate required when the thrust of the solid-liquid variable thrust engine is the desired thrust.

5. A solid-liquid variable thrust engine intelligent controller design system, characterized in that: include: Building blocks and design blocks; where The construction module is configured to construct a target reinforcement learning controller using the actual thrust of the solid-liquid variable thrust engine, the thrust error, the power of the electric pump, and the valve opening of the variable venturi as observation inputs, and the control instructions of the electric pump and the variable venturi as controller outputs; the thrust error is the error between the actual thrust and the desired thrust of the solid-liquid variable thrust engine; the electric pump is a booster electric pump for the oxidizer circuit of the solid-liquid variable thrust engine; The design module is used to design an intelligent controller of the solid-liquid variable thrust engine based on the target reinforcement learning controller; wherein, designing the intelligent controller of the solid-liquid variable thrust engine based on the target reinforcement learning controller includes: connecting the output end of the intelligent controller to the control input end of the electric pump and the control input end of the variable venturi, so that the intelligent controller controls the output liquid oxidizer flow of the electric pump and the variable venturi simultaneously based on the control instructions output in response to the state observation quantity feedback from the measurement module; the measurement module is used to obtain the state observation quantity of the solid-liquid variable thrust engine; the state observation quantity includes: the actual thrust of the solid-liquid variable thrust engine, the power of the electric pump and the valve opening of the variable venturi.

6. The system according to claim 5, characterized in that The system further includes a training module for training the target reinforcement learning controller using a reinforcement learning algorithm; wherein the reward function of the reinforcement learning algorithm is: R=β1R e +β2R p ; β1 and β2 are reward weight coefficients, and β1+β2=1; R e is the error reward that is negatively correlated with the absolute value of the thrust error, R p Bonus for electric pump power related to battery power.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

8. A computer-readable medium having a non-volatile program code executable by a processor, characterized in that The program code enables the processor to execute the method according to any one of claims 1 to 4.

Images