Genetic programming and reinforcement learning fused interpretable intelligent flow control method

Abstract

According to the interpretable intelligent flow control method, an automatic differential mechanism in a neural network is introduced into linear genetic programming, an explicit expression obtained by the linear genetic programming is automatically adjusted through a gradient descent algorithm, and the time needed by control law optimization can be greatly shortened; compared with a neural network control law, the explicit symbolic expression control law has higher interpretability, so that researchers can better understand a control mechanism and find new physical knowledge; the framework can set state sensors and different reward functions according to different control targets, and has very high universality.

Description

Technical Field

[0001] The present invention relates to the field of active flow control, and in particular to an interpretable intelligent flow control method integrating genetic programming and reinforcement learning. Background Art

[0002] Active flow control (AFC) improves the overall performance of the aircraft by inputting additional energy into the flow field through an actuator, thereby increasing lift and reducing flight drag. Most AFC research and applications are open-loop control under fixed control parameters. Compared with open-loop control, closed-loop control can adjust control parameters or control strategies according to real-time feedback from state sensors, thereby improving the control gain and robustness of AFC. Currently, intelligent closed-loop control methods represented by linear genetic programming (LGP) and deep reinforcement learning (DRL) have been widely used in the field of AFC, such as flow around a cylinder and airfoil separation control. However, in the existing intelligent flow control framework, there are still several problems that need to be solved. First, although the LGP method is simple and easy to use and can avoid falling into the local optimal strategy, the entire control law optimization process involves hundreds of iterations and often takes several hours (Li, R., Noack, BR, Cordier, L., Borée, J., Harambat, F. (2017). Dragreduction of a car model by linear genetic programming control. Experiments in Fluids 58, 103. doi: 10.1007 / s00348-017-2382-2). The high time cost makes it difficult to apply to engineering practice. Second, although the DRL method can shorten the strategy optimization process to a few minutes through online learning and achieve control benefits comparable to those of LGP, since the control law in DRL is approximated by a deep neural network, the "black box nature" of the neural network makes it difficult for researchers to further explore the physical explanation and flow mechanism behind the optimal control law (Yao Zhangyi, Shi Zhiwei, Dong Yizhang. Application of deep reinforcement learning in airfoil separation flow control [J]. Experimental Fluid Mechanics, 2022, 36(03): 55-64.). In order to further improve the effectiveness of closed-loop flow control, it is urgent to develop an intelligent learning framework with strong interpretability and fast learning speed. Summary of the invention

[0003] In view of the problems of slow optimization speed and poor interpretability of current intelligent control algorithms, the present invention proposes an interpretable intelligent flow control method integrating genetic programming and reinforcement learning, which includes two flow control strategy optimization stages, as follows:

[0004] Step 1: Genetic programming initial screening of symbolic expressions;

[0005] The process of genetic programming LGD initial screening symbolic expression is as follows: LGP uses basic unary and binary function operators to generate a fixed-depth binary tree containing N i The function set F of the initial control strategy is 1 ,f 2 ,…,f Ni}, where f 1 ,f 2 ,…,f Ni Respectively represent different initial control strategies, f(s t ) is a mapping from flow state to action, i.e. t =f(s t ), where a t and t They are the control command of the actuator and the sensor signal, the initial f(s t )The expression is randomly generated by LGP;

[0006] Each control strategy is applied to the flow field environment and measured by sensors to obtain the corresponding fitness;

[0007] The LGP algorithm evaluates the fitness of each control law, and according to the fitness ranking, performs operations such as replication, crossover, and mutation on the current generation of control strategies to obtain a new generation of control strategies; that is, the greater the fitness of an individual, the greater the probability that the individual will be selected;

[0008] After preliminary screening of several generations of control laws, an explicit symbolic control law with better performance was obtained. t * =f(s t ,θ), where θ is the numerical coefficient in the explicit expression, f(s t ,θ) is the expression form in which the numbers in the control law expression are further optimized as parameters after the initial screening stage of the control law; at this time, the corresponding digital coefficient θ is definitely not the optimal value and needs to be further optimized;

[0009] Step 2: Reinforce learning to fine-tune the control law parameters;

[0010] Use the reinforcement learning framework to adjust the internal parameters: achieve the best performance expression a t * =f(s t ) and the control law a t * The coefficient θ in is set as a trainable weight coefficient; solve the gradient of the root mean square loss function with respect to θ; combine the optimizer to optimize the parameter θ;

[0011] The reinforcement learning optimization framework includes an explicit symbolic expression and two neural networks; the explicit symbolic expression is the control strategy, which builds a bridge to communicate with the flow environment and determines the output of the actor;

[0012] Actor interacts with the real flow environment and t Give control command a t , the flow field migrates to the new state s t+1 , and by rewarding r t As the feedback information of the closed-loop system, [s t ,a t ,r t ,s t+1 ] is stored in the experience playback pool;

[0013] The two neural networks inside the critic Critic evaluate the current state s t and control instructions a t The action value Q under is evaluated, and the action value is calculated by fitting the neural network inside the critic, guiding the parameter update of the explicit symbolic expression control law;

[0014] The explicit symbolic expression control law infers the control command based on the current state. The Q network and the Target Q network are exactly the same, except that the Q network is used to estimate the current value Q, and the Target Q network is combined with the reward to estimate the expected value y i ; The difference between the two is the loss function of the network; get the target value of the value function in the current state:

[0015] y t =r t +γQ t (s t+1 ,f(s t+1 )) (1)

[0016] Among them, r t Represents the execution of action a in the current state t The rewards obtained, Q t Represents the value calculated by the Target Q network, γ represents the discount factor, and is the weight of future rewards. When training the Q network and the Target Q network, small batches of data should be randomly extracted from the experience replay pool to update the weights and bias parameters of the Q network. The Target Q network is updated using a soft update method, that is, some parameters of the Target Q network are updated; the loss function used for the Q network update is:

[0017]

[0018] Combined with the value gradient of Q network training, the policy gradient of the explicit symbolic expression control law during parameter update is obtained

[0019]

[0020] Among them, N m Represents the number of experience sequences during training, s i and a i They represent the i-th group of states and actions in the experience sequence, Q(s,a) represents the value of different actions a under state s, represents the gradient of the value function to action a in state s, Represents the gradient of the explicit symbolic expression control law to its coefficient θ; based on the obtained gradient information, the expression parameter θ is updated, and the parameter update process is executed repeatedly until the control effect no longer changes or the maximum time of the optimization process is reached.

[0021] In one embodiment of the present invention, the control law reasoning process is deployed on a field programmable gate array FPGA, the control law optimization process is performed on a computer host CPU, and the real-time control loop running on the FPGA and the strategy optimization loop running on the CPU are performed simultaneously; in the real-time control loop, the FPGA measures the velocity pulsation inside the shear layer through a hot wire anemometer, and converts the measurement results through an analog-to-digital converter to obtain the current flow field state s t ; Set the state s t Input into the explicit symbolic control law to obtain the control instruction a t ; The control instruction a t The digital-to-analog converter converts it into a control voltage signal, which is applied to the plasma actuator to disturb the flow field, causing the flow field to migrate to a new state s t+1 .

[0022] In another embodiment of the present invention, the setting of the fitness varies depending on the flow field and the control target. In actual flow control, parameters such as time-averaged lift, drag, area of ​​the separation zone, and pulsation of surface pressure are used as the fitness.

[0023] In a specific embodiment of the present invention, the method for the LGP algorithm to evaluate the fitness of each control law is: sorting the values ​​of fitness from large to small.

[0024] In another embodiment of the present invention, after preliminary screening of 3-5 generations of control laws, an explicit sign control law a with better performance is obtained. t * =f(s t ,θ).

[0025] In another specific embodiment of the present invention, after preliminary screening of five generations of control laws, an explicit sign control law a with better performance is obtained. t * =f(s t ,θ).

[0026] In another embodiment of the present invention, the better performing expression a is implemented by Python code. t * =f(s t ); use numerical differentiation method to solve the gradient of RMS loss function with respect to θ; combine Adam or RMSProp optimizer to realize the optimization of parameter θ.

[0027] In another specific embodiment of the present invention, the parameter update of the control law guided by the explicit symbolic expression is specifically as follows: if the value of taking a certain action in the current state is high, the Q network inside the critic will guide the Actor to increase the probability of the action accordingly; conversely, if the value is low, the corresponding action probability will be reduced.

[0028] In yet another specific embodiment of the present invention, the root mean square error loss is used to update the weight and bias parameters of the Q network.

[0029] The present invention introduces the automatic differentiation mechanism in neural networks into linear genetic programming, and automatically adjusts the explicit expression obtained by linear genetic programming through the gradient descent algorithm, which can greatly reduce the time required for control law optimization. Explicit symbolic expression of control laws has stronger interpretability than neural network control laws, which is convenient for researchers to better understand control mechanisms and discover new physical knowledge. The framework can set state sensors and different reward functions according to different control objectives, and has strong versatility.

[0030] The advantages of the present invention are as follows:

[0031] 1. Traditional genetic programming and reinforcement learning closed-loop flow control have the disadvantages of long optimization time and poor interpretability respectively. The present invention can greatly reduce the optimization time by combining the two frameworks, and the explicit symbolic expression of the control law has better interpretability than neural networks.

[0032] 2. The present invention proposes to optimize the explicit symbolic expression control law under the actor-critic framework, which can reduce the tens of thousands of training parameters of traditional neural network control by three orders of magnitude, reducing the computing time and computing cost required for control law optimization.

[0033] 3. In addition, the explicit symbolic expression can improve the flow control benefit of the LGD control law by adjusting its parameters θ through the gradient descent algorithm under the reinforcement learning framework, and can overcome the problem of poor training stability caused by the random initial parameters of the DRL control law. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram showing an interpretable intelligent flow control method combining genetic programming and reinforcement learning;

[0035] Figure 2 Show the parameter update process of the control law expressed explicitly with symbols in the reinforcement learning framework;

[0036] Figure 3 A schematic diagram showing the implementation method of the explainable intelligent flow control framework. DETAILED DESCRIPTION

[0037] The present invention will be described in detail below with reference to the accompanying drawings.

[0038] Figure 1 The present invention shows an interpretable intelligent flow control method that combines genetic programming with reinforcement learning. The method mainly includes two flow control strategy optimization stages: a genetic programming initial screening symbolic expression process and a reinforcement learning fine-tuning control law parameter process.

[0039] Step 1: Genetic programming initial screening of symbolic expressions;

[0040] The entire genetic programming (LGD) initial screening symbolic expression process is as follows: LGP uses basic unary and binary function operators (such as +, -, ×, sin, cos, x 2 ,e x ), by randomly generating a fixed-depth binary tree (the binary tree structure is well known to those skilled in the art), generating a binary tree containing N i The function set F of the initial control strategy is 1 ,f 2 ,…,f Ni}, where f 1 ,f 2 ,…,f Ni Respectively represent different initial control strategies, f(s t ) is essentially a mapping from flow state to action, i.e. t =f(s t ), where a t and t They are the control command of the actuator and the sensor signal, the initial f(s t ) Expressions are randomly generated by LGP and may contain operators (addition, subtraction, multiplication, division) and basic functions (exponential function, logarithmic function, trigonometric function, etc.).

[0041] Each control strategy is applied to the flow field environment and measured through sensors to obtain the corresponding fitness. It should be noted that the setting of fitness varies depending on the flow field and control objectives. In actual flow control, parameters such as time-averaged lift, drag, area of ​​separation zone, and pulsation of surface pressure are often used as fitness.

[0042] The LGP algorithm evaluates the fitness of each control law. The specific evaluation method is to sort the values ​​of fitness from large to small. According to the fitness sorting, the control strategy of the current generation is copied, crossed, mutated, and other operations are performed to obtain a new generation of control strategies. That is, the greater the fitness of the individual, the greater the probability of the individual being selected. Specifically, the copy operation will copy the individual to the next generation group, the crossover operation is to interchange the function operations under the nodes of two individuals, and the mutation operation is to generate a new function operation relationship on the node of the individual, which is known to those skilled in the art.

[0043] After preliminary screening of 3-5 generations of control laws (preferably 5 generations), the explicit symbolic control law with better performance can be obtained. t * =f(s t ,θ), where θ is the numerical coefficient in the explicit expression, f(s t ,θ) is the expression form in which the numbers in the control law expression are further optimized (emphasizing the optimization of parameters) after the initial screening stage of the control law (emphasizing the optimization of functional relationships). The control law performed outstandingly in the first 3-5 generations, indicating that its function form (binary tree structure) is reasonable, but the corresponding digital coefficient θ is definitely not the optimal value and needs to be further optimized.

[0044] Step 2: Reinforce learning to fine-tune the control law parameters;

[0045] The reinforcement learning framework is used to adjust these internal parameters. Specifically, the Python code is used to implement the better performing expression a t * =f(s t ) (this method is known to those skilled in the art), and the control law a t * The coefficient θ in is set as a trainable weight coefficient. The numerical differentiation method is used to solve the gradient of the root mean square loss function (the loss function expression is shown in equation (2) below) with respect to θ. Further combined with optimizers such as Adam and RMSProp (these optimizers are well known to those skilled in the art) to achieve the optimization of the parameter θ.

[0046] Obviously, in the above intelligent closed-loop control method, the total number of control strategies that need to be evaluated is much smaller than the number required by the classic LGP method (20N i -50N i), so a control law with higher benefits can be found in a shorter time.

[0047] Figure 2 Taking the classic DDPG algorithm as an example, the update and iteration process of the explicit symbolic control strategy under the reinforcement learning framework is described. Compared with the implicit neural network control law in traditional deep reinforcement learning, the control law in the present invention is an explicit symbolic expression with fewer parameters, so it can greatly reduce the time required for gradient calculation during training. The entire reinforcement learning optimization framework contains an explicit symbolic expression and two neural networks; the explicit symbolic expression is the control strategy, which builds a bridge to communicate with the flow environment and determines the output of the actor. The actor interacts with the real flow environment and responds to the real-time flow field state s t Give control command a t , the flow field migrates to the new state s t+1 , and by rewarding r t As the feedback information of the closed-loop system, [s t ,a t ,r t ,s t+1 ] is stored in the experience replay pool. The two neural networks inside the critic evaluate the current state s t and control instructions a t The action value Q (long-term reward) under the condition is evaluated. The action value is calculated by the neural network inside the critic through fitting, guiding the parameter update of the explicit symbolic expression control law. Specifically, if the value of taking an action in the current state is high, the Q network inside the critic will guide the Actor to increase the probability of the action accordingly. Conversely, if the value is low, the corresponding action probability will be reduced. The explicit symbolic expression control law can infer the control command based on the current state. The Q network and the Target Q network are exactly the same. The difference is that the Q network is used to estimate the current value Q, and the Target Q network is combined with the reward to estimate the expected value y i The difference between the two is the loss function of the network. In actual implementation, the target value of the value function in the current state is obtained:

[0048] y t =r t +γQ t (s t+1 ,f(s t+1 )) (1)

[0049] Among them, r t Represents the execution of action a in the current state t The rewards obtained, Q tRepresents the value calculated by the Target Q network, γ represents the discount factor, which is the weight of future rewards. When training the Q network and the Target Q network, a small batch of data should be randomly extracted from the experience replay pool, and then the root mean square error loss should be used to update the weights and bias parameters of the Q network (the parameter update method is well known to those skilled in the art). The Target Q network is updated using a soft update method, that is, some parameters of the TargetQ network are updated to ensure that they are updated slowly, thereby improving learning stability. The loss function used for Q network update is:

[0050]

[0051] Combined with the value gradient of Q network training, we can get the policy gradient of the explicit symbolic expression control law during parameter update.

[0052]

[0053] Among them, N m Represents the number of experience sequences during training, s i and a i They represent the i-th group of states and actions in the experience sequence, Q(s,a) represents the value of different actions a under state s, represents the gradient of the value function to action a in state s, Represents the gradient of the explicit symbolic control law to its coefficient θ. In the reinforcement learning optimization process, both the value and strategy methods are combined. The explicit symbolic control law helps the agent to process continuous actions, and the value function helps the explicit symbolic control law to update the control strategy. Based on the obtained gradient information, the expression parameter θ is updated, and the parameter update process is executed repeatedly until the control effect no longer changes or the maximum time (pre-set) of the optimization process is reached.

[0054] Figure 3This is a hardware implementation diagram of the present invention for rear step mixing enhancement. The sensor is selected as a hot wire anemometer, which is arranged inside the shear layer (at a position related to the velocity pulsation, and the specific position can be determined by PIV particle image velocimetry technology) to monitor instantaneous velocity pulsation. The plasma exciter is arranged at the corner edge of the rear step model in the wind tunnel test section to disturb the boundary layer through momentum effect or thermal effect. In order to reduce the control delay (the time required for control law reasoning) of receiving the flow field state and issuing control instructions, the control law reasoning process of the method of the present invention is deployed on a field programmable gate array (FPGA) to reduce the control delay (within 10us) and improve the control efficiency. The optimization process of the control law is carried out on the computer host CPU, and the real-time control loop running on the FPGA and the strategy optimization loop running on the CPU are carried out simultaneously. In the real-time control loop, the FPGA first measures the velocity pulsation inside the shear layer through a hot wire anemometer, and converts the measurement results through an analog-to-digital converter to obtain the current flow field state s t . Set the state s t Input into the explicit symbolic control law to obtain the control instruction a t The control instruction a t The digital-to-analog converter converts it into a control voltage signal, which is applied to the plasma actuator to disturb the flow field, causing the flow field to migrate to a new state s t+1 .

[0055] The strategy optimization loop is completed by the Python program and the Labview program. The Python program is used to execute genetic programming and gradient descent algorithms, while the Labview program is essentially an information transfer bridge (including experience information and strategy parameters). Specifically, Labview continuously receives the experience information collected by the FPGA through the FIFO (first-in, first-out sequence). t and a t , and the fitness or reward r is calculated from this information t , and then the fitness or experience sequence [s t ,a t ,r t ,s t+1 ] is transmitted to Python via TCP / IP protocol; Python updates the symbolic expression of the control strategy through genetic programming or gradient descent algorithm a t * =f(s t) or control law coefficient θ, and send the new control strategy symbolic expression or control law coefficient θ to the FPGA chip through FIFO through Labview to update the control strategy. Since the number of parameters of the symbolic control law is far less than that of the neural network control law, the execution time of the control law will be greatly shortened, reaching 10μs under typical conditions, which can better meet the needs of high-speed flow high-frequency closed-loop control.

[0056] The interpretable intelligent flow control framework combining genetic programming and reinforcement learning proposed in the present invention can set the objective function of genetic programming and the reward of reinforcement learning according to different control objectives, and can be matched with various types of sensors and actuators, and has strong versatility and interpretability.

Claims

1. An interpretable intelligent flow control method integrating genetic programming and reinforcement learning, characterized in that: The method includes two stages of flow control strategy optimization: genetic programming initial screening of symbolic expressions and reinforcement learning fine-tuning of control law parameters; the details are as follows: Step 1: Genetic programming initial screening of symbolic expressions; The process of genetic programming LGD initial screening symbolic expression is as follows: LGP uses basic unary and binary function operators to generate a fixed-depth binary tree containing N i The function set F of the initial control strategy is {f1,f2,…,f Ni }, where f1, f2, ..., f Ni Respectively represent different initial control strategies, f(s t ) is a mapping from flow state to action, i.e. t =f(s t ), where a t and t They are the control command of the actuator and the sensor signal, the initial f(s t )The expression is randomly generated by LGP; Each control strategy is applied to the flow field environment and measured by sensors to obtain the corresponding fitness; The LGP algorithm evaluates the fitness of each control law, and according to the fitness ranking, performs operations such as replication, crossover, and mutation on the current generation of control strategies to obtain a new generation of control strategies; that is, the greater the fitness of an individual, the greater the probability that the individual will be selected; After preliminary screening of several generations of control laws, an explicit symbolic control law with better performance was obtained. t * =f(s t ,θ), where θ is the numerical coefficient in the explicit expression, f(s t ,θ) is the expression form in which the numbers in the control law expression are further optimized as parameters after the initial screening stage of the control law; at this time, the corresponding digital coefficient θ is definitely not the optimal value and needs to be further optimized; Step 2: Reinforce learning to fine-tune the control law parameters; Use the reinforcement learning framework to adjust the internal parameters: achieve the best performance expression a t * =f(s t ) and the control law a t * The coefficient θ in is set as a trainable weight coefficient; solve the gradient of the root mean square loss function with respect to θ; combine the optimizer to optimize the parameter θ; The reinforcement learning optimization framework includes an explicit symbolic expression and two neural networks; the explicit symbolic expression is the control strategy, which builds a bridge to communicate with the flow environment and determines the output of the actor; Actor interacts with the real flow environment and t Give control command a t , the flow field migrates to the new state s t+1 , and by rewarding r t As the feedback information of the closed-loop system, [s t ,a t ,r t ,s t+1 ] is stored in the experience replay pool; The two neural networks inside the critic Critic evaluate the current state s t and control instructions a t The action value Q under is evaluated, and the action value is calculated by fitting the neural network inside the critic, guiding the parameter update of the explicit symbolic expression control law; The explicit symbolic expression control law infers the control command based on the current state. The Q network and the Target Q network are exactly the same, except that the Q network is used to estimate the current value Q, and the Target Q network is combined with the reward to estimate the expected value y i ; The difference between the two is the loss function of the network; get the target value of the value function in the current state: y t =r t +γQ t (s t+1 ,f(s t+1 )) (1) Among them, r t Represents the execution of action a in the current state t The rewards obtained, Q t Represents the value calculated by the Target Q network, γ represents the discount factor, and is the weight of future rewards. When training the Q network and the Target Q network, small batches of data should be randomly extracted from the experience replay pool to update the weights and bias parameters of the Q network. The Target Q network is updated using a soft update method, that is, some parameters of the Target Q network are updated; the loss function used for the Q network update is: Combined with the value gradient of Q network training, the policy gradient of the explicit symbolic expression control law during parameter update is obtained Among them, N m Represents the number of experience sequences during training, s i and a i They represent the i-th group of states and actions in the experience sequence, Q(s,a) represents the value of different actions a under state s, represents the gradient of the value function to action a in state s, Represents the gradient of the explicit symbolic expression control law to its coefficient θ; based on the obtained gradient information, the expression parameter θ is updated, and the parameter update process is executed repeatedly until the control effect no longer changes or the maximum time of the optimization process is reached.

2. The interpretable intelligent flow control method integrating genetic programming and reinforcement learning as claimed in claim 1, characterized in that: The control law reasoning process is deployed on the field programmable gate array FPGA, and the control law optimization process is carried out on the computer host CPU. The real-time control loop running on the FPGA and the strategy optimization loop running on the CPU are carried out simultaneously; in the real-time control loop, the FPGA measures the velocity pulsation inside the shear layer through a hot wire anemometer, and converts the measurement results through an analog-to-digital converter to obtain the current state of the flow field s t ; Set the state s t Input into the explicit symbolic control law and obtain the control instruction a t ; The control instruction a t The digital-to-analog converter converts it into a control voltage signal, which is applied to the plasma actuator to disturb the flow field, causing the flow field to migrate to a new state s t+1 .

3. The interpretable intelligent flow control method integrating genetic programming and reinforcement learning as claimed in claim 1, characterized in that: The setting of fitness varies depending on the flow field and control objectives. In actual flow control, parameters such as time-averaged lift, drag, area of ​​separation zone, and pulsation of surface pressure are used as fitness.

4. The interpretable intelligent flow control method integrating genetic programming and reinforcement learning as claimed in claim 1, characterized in that: The method by which the LGP algorithm evaluates the fitness of each control law is to sort them from large to small in terms of fitness value.

5. The interpretable intelligent flow control method integrating genetic programming and reinforcement learning as claimed in claim 1, characterized in that: After preliminary screening of 3-5 generations of control laws, the explicit symbolic control law a with better performance was obtained. t * =f(s t ,θ).

6. The interpretable intelligent flow control method integrating genetic programming and reinforcement learning as claimed in claim 5, characterized in that: After preliminary screening of five generations of control laws, the explicit symbolic control law a with better performance was obtained. t * =f(s t ,θ).

7. The interpretable intelligent flow control method integrating genetic programming and reinforcement learning as claimed in claim 1, characterized in that: Implementing the better performing expression a through Python code t * =f(s t ); use numerical differentiation method to solve the gradient of RMS loss function with respect to θ; combine Adam or RMSProp optimizer to realize the optimization of parameter θ.

8. The interpretable intelligent flow control method integrating genetic programming and reinforcement learning as claimed in claim 1, characterized in that: The parameter update of the control law guided by the explicit symbolic expression is as follows: if the value of taking a certain action in the current state is high, the Q network inside the critic will guide the Actor to increase the probability of the action accordingly; conversely, if the value is low, the corresponding action probability will be reduced.

9. The interpretable intelligent flow control method integrating genetic programming and reinforcement learning as claimed in claim 1, characterized in that: The weights and bias parameters of the Q network are updated using the root mean square error loss.

Images