An active flow control system based on masked deep neural network and deep reinforcement learning

Abstract

The present invention discloses an active flow control system based on a masked deep neural network and deep reinforcement learning. The main process of its implementation includes: step one, establishing a database; step two, constructing a training set and a test set according to a time series; step three, building and training a masked deep neural network; and step four, combining the masked deep neural network with a deep reinforcement learning algorithm to design an active flow control strategy. By establishing a flow field reduction model based on a masked deep neural network, a prediction of the unsteady flow field of the control body under arbitrary motion disturbances is provided, replacing the traditional computational fluid dynamics solution, and interacting with the deep reinforcement learning algorithm as an environment, a rapid solution of the active flow control strategy is achieved, solving the problems of long iteration cycles and high computational costs in the traditional active flow control strategy solution process, and facilitating the rapid design and analysis of active flow control systems.

Description

Technical Field

[0001] The present invention relates to the field of engineering fluid mechanics calculations, and in particular to an active flow control system based on masked deep neural networks and deep reinforcement learning. Background Art

[0002] Active flow control technology regulates the flow state by introducing external energy or control means into the fluid, and is widely used in engineering problems such as drag reduction, vibration reduction, lift increase, noise control, and heat transfer control. Existing research on fluid-related engineering problems relies on experiments and numerical simulations. Experiments are expensive and it is difficult to obtain more comprehensive flow field details, and are often used to verify the final design solution. In the design optimization stage of fluid-related engineering problems, it is necessary to rely heavily on numerical simulation technology for solution development and optimization. However, due to the complexity of the flow problem itself and the complexity of computational fluid dynamics solutions, relying on computational fluid dynamics technology to design effective active control strategies for flow systems faces problems such as large computational workload, long iterative design cycle, and high dependence on experience. It is necessary to combine emerging technologies such as reinforcement learning to optimize the design process and reduce design costs.

[0003] In recent years, deep reinforcement learning technology has developed rapidly in many fields and can be used to solve complex decision-making problems. For active flow control problems, deep reinforcement learning technology can rely on computational fluid dynamics as an environment and improve control strategies through interaction with the environment, providing a new method for the rapid optimization design of active flow control strategies. However, since each interaction between the deep reinforcement learning algorithm and the environment requires multiple computational fluid dynamics solutions, and the sampling data that can be provided by each interaction is limited, a large amount of sample collection is required to complete the optimization of the active flow control strategy throughout the entire process. In addition, for complex engineering flow cases, the complexity of the geometry and boundary conditions and the large grid size in the computational fluid dynamics solution process make the computational cost high, which seriously restricts the efficiency and cost of the active flow control strategy optimization process. There is an urgent need to develop corresponding methods to optimize the computational fluid dynamics solution part to improve the efficiency of the entire active flow control strategy development process and reduce the corresponding R&D costs.

[0004] To address the above problems, the present invention proposes an active flow control system based on masked deep neural networks and deep reinforcement learning. Masked deep neural networks are used instead of traditional computational fluid dynamics solutions as the interactive environment of deep reinforcement learning algorithms to solve the problems of long iteration cycles and high computational costs in the development process of traditional active flow control strategies. Summary of the Invention

[0005] (1) Technical issues to be resolved

[0006] The purpose of the present invention is to provide an active flow control system based on masked deep neural networks and deep reinforcement learning. It aims to provide unsteady flow field prediction of the control body under arbitrary motion disturbance by establishing a flow field reduction model based on masked deep neural networks, replacing the traditional computational fluid dynamics solution, and interacting with the deep reinforcement learning algorithm as an environment to achieve rapid solution of active flow control strategies, thereby solving the problems of long iteration cycle and high computational cost in the solution process of traditional active flow control strategies.

[0007] (2) Technical solution

[0008] In order to solve the above technical problems, the present invention provides an active flow control system based on a masked deep neural network and deep reinforcement learning, comprising the following steps:

[0009] Step 1: Create a database;

[0010] Step 2: Construct training set and test set according to time series;

[0011] Step 3: Build and train a masked deep neural network;

[0012] Step 4: Design active flow control strategy by combining masked deep neural network and deep reinforcement learning algorithm;

[0013] ① The establishment of a database includes:

[0014] Determine the parameters that need to be generalized for the active flow control problem under study, use a computational fluid dynamics solver to perform multi-time series numerical simulation calculations on the active flow control problem example under study, and obtain flow field data at each time step; wherein the spatial position of the active flow control problem example under study includes the flow field position and the solid field position; the time series is composed of time step 1 to time step t, and the value of time step t is a positive integer; the motion mode of the control body of the active flow control problem under study adopts a random motion mode within the control range;

[0015] Projecting the flow field data of each time step into a fluid matrix F, and projecting the solid field data of each time step into a solid matrix S; wherein the fluid matrix F and the solid matrix S have the same dimensions;

[0016] Assigning the flow field position corresponding to the solid matrix S to 1, and assigning the solid field position corresponding to the solid matrix S to 0; the database is composed of the fluid matrix F and the solid matrix S;

[0017] ② The construction of training sets and test sets according to time series includes:

[0018] The fluid matrix F is normalized to obtain a normalized fluid matrix F n , the normalized fluid matrix F n Perform dot multiplication with the solid matrix S to obtain the mask fluid matrix F m , that is, F m =F n ·S;

[0019] The mask fluid matrix at time step t-1 and the solid matrix S with time step t t Perform splicing processing on the time dimension to obtain the variable matrix T with a time step of t t ; The mask fluid matrix with time step t As the variable matrix T t The label matrix of each time step; the variable matrix T t and the variable matrix T at each time step t The label matrix is ​​spliced ​​to obtain the masked deep neural network training set and the masked deep neural network test set;

[0020] ③ The construction and training of the masked deep neural network includes:

[0021] The mask deep neural network is constructed by using a convolutional layer, a convolutional long-short time neural network layer and a deconvolutional layer; the convolutional layer is used to capture the spatial features of the flow field data at each time step and reduce the dimension; the convolutional long-short time neural network layer is based on the mask fluid matrix of the time step t-1 and the solid matrix S with time step t t Predict the low-dimensional flow field features with a time step of t; reconstruct the low-dimensional flow field features with a time step of t using the deconvolution layer to obtain a high-dimensional flow field with a time step of t The high-dimensional flow field and the solid matrix S at the time step t t Perform dot multiplication to obtain the predicted mask fluid matrix Right now The prediction mask fluid matrix Perform inverse normalization processing to obtain the real flow field;

[0022] The masked deep neural network training set and the masked deep neural network test set constructed in step 2 are used to train the masked deep neural network to obtain a usable masked deep neural network model; the usable masked deep neural network model can be used to calculate the fluid matrix F at time step t-1. t-1 and the solid matrix S with time step t t Complete the prediction of flow field data with time step t;

[0023] ④ The active flow control strategy design combining the masked deep neural network and the deep reinforcement learning algorithm includes:

[0024] Using a deep reinforcement learning algorithm and the available masked deep neural network model obtained in step 3 to build an active flow control system based on masked deep neural network and deep reinforcement learning;

[0025] At the beginning of the calculation, according to the active flow control problem to be solved, the fluid matrix F corresponding to the initial flow field is given 0 ; The intelligent agent of the deep reinforcement learning algorithm converts the fluid matrix F corresponding to the initial location of the flow field 0 As the state, apply the convolutional neural network to the fluid matrix F corresponding to the initial location of the flow field 0 Extract features and output the control action as a solid matrix S 1 ; The solid matrix S 1 The fluid matrix F corresponding to the initial location of the flow field 0 As the input of the available mask deep neural network model obtained in step 3, the fluid matrix F with time step 1 is 1 Make a prediction; the intelligent agent of the deep reinforcement learning algorithm is based on the fluid matrix F with a time step of 1 1 Get the control action for the solid matrix S 1 The corresponding reward completes an interaction and data collection, and the fluid matrix F with the time step of 1 1 As the state input of the agent of the deep reinforcement learning algorithm at time step 2;

[0026] Iterate according to the time sequence, and the agent of the deep reinforcement learning algorithm converts the fluid matrix F at time step t-1 t-1 As the state, apply the convolutional neural network to the fluid matrix F at time step t-1 t-1 Extract features and output the control action as a solid matrix S t ; The solid matrix S t The fluid matrix F at time step t-1 t-1 As the input of the available mask deep neural network model obtained in step 3, the fluid matrix F with time step t is t Make a prediction; the intelligent agent of the deep reinforcement learning algorithm is based on the fluid matrix F at the time step t t Get the control action for the solid matrix S t The corresponding reward completes an interaction and data collection, and the fluid matrix F with the time step t t As input to the state of the agent of the deep reinforcement learning algorithm at time step t+1;

[0027] Iterate the time sequence from time step 1 to time step t until the total number of iteration steps T is reached, thereby completing the design of the active flow control system based on masked deep neural network and deep reinforcement learning.

[0028] (3) Beneficial effects

[0029] The present invention provides an active flow control system based on a masked deep neural network and deep reinforcement learning, which has the following beneficial effects: It establishes a flow field reduction model based on a masked deep neural network, provides rapid prediction of the unsteady flow field of the control body under arbitrary motion disturbances, replaces traditional computational fluid dynamics solutions, and interacts with the deep reinforcement learning algorithm as an environment to achieve rapid solution of active flow control strategies. This effectively solves the high computational cost of traditional methods, achieves rapid solution of active flow control problems in engineering cases, and facilitates the rapid design and analysis of active flow control systems.

[0030] The method of the present invention can use Python for interactive control, is based on the Tensorflow library and the PyTorch library, and has the characteristics of simple code, short training time, and easy extension to existing programs.

[0031] The method of the present invention has demonstrated good prediction and control effects in multiple classic case tests, shortening the interaction time between the environment and the deep reinforcement learning algorithm to milliseconds, and has good generalization performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a flow chart of an active flow control system based on masked deep neural network and deep reinforcement learning in the present invention;

[0033] Figure 2 Schematic diagram of a masked deep neural network for an active flow control system based on a masked deep neural network and deep reinforcement learning according to the present invention;

[0034] Figure 3 Schematic diagram of the flow field grid of the pulsation baffle active control system according to an embodiment of the present invention;

[0035] Figure 4 This figure compares the control effects of the pulsating baffle active control system of an embodiment of the present invention using an active flow control system based on a masked deep neural network and deep reinforcement learning and a traditional active flow control system based on computational fluid dynamics and deep reinforcement learning. DETAILED DESCRIPTION

[0036] The following is a further detailed description of the specific implementation of the present invention, with reference to the accompanying drawings and examples, taking the active control problem of the pulsation baffle as an example. The following examples are only used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0037] like Figure 1 Shown is a flow chart of an active flow control system based on masked deep neural network and deep reinforcement learning according to the present invention.

[0038] The present invention provides an active flow control system based on a masked deep neural network and deep reinforcement learning, comprising the following steps:

[0039] Step 1: Create a database;

[0040] In this step, the parameters that need to be generalized for the active flow control problem under study are determined, and a computational fluid dynamics solver is used to perform multi-time series numerical simulation calculations on the active flow control problem example under study to obtain the flow field data of each time step; wherein, the spatial position of the active flow control problem example under study includes the flow field position and the solid field position; the time series is composed of time step 1 to time step t, and the value of time step t is a positive integer; the flow field data of each time step is projected onto the fluid matrix F, and the solid field data of each time step is projected onto the solid matrix S; wherein, the fluid matrix F and the solid matrix S have the same dimension; the flow field position corresponding to the solid matrix S is assigned a value of 1, and the solid field position corresponding to the solid matrix S is assigned a value of 0; the database is composed of the fluid matrix F and the solid matrix S.

[0041] like Figure 3 Figure 2 shows a schematic diagram of the flow field mesh for an active control system for a pulsating baffle according to an embodiment of the present invention. The flow field mesh uses an overlapping grid and is densified around the solid walls, ensuring good mesh quality even when the control volume moves. As a blind test example, the active control system for a pulsating baffle consists of three rectangular baffles. These baffles are placed horizontally within the fluid domain and have infinite extension in the flow direction, thus simplifying the computational problem for this system to two dimensions. The middle baffle measures 1m x 0.05m, while the upper and lower baffles each measure 0.4m x 0.05m. The initial positions of the upper and lower baffles are each 0.4m away from the middle baffle. The initial position of each baffle is used as its coordinate origin, with the upward direction defined as the positive direction. The positions of the upper and lower baffles are expressed as coordinates relative to their initial positions. The upper baffle's range of motion is specified as ±0.22m, and the lower baffle's range of motion is specified as ±0.3m. The entire computational domain is 2m x 4m.

[0042] As a simplification of the engineering problem, the middle baffle represents the physical component requiring force control. The upper baffle represents the disturbance source, simulating unsteady flow. The moving upper baffle generates unsteady pressure pulsations and varying forces on the middle baffle. The lower baffle, the controlled object, is controlled by a deep reinforcement learning agent. The control objective is to manipulate the lower baffle's motion within a specific time period to maintain a very small force on the middle baffle, thereby reducing vibration. To verify the reliability of the framework, the upper and lower baffles were designed symmetrically to facilitate comparison with human experience.

[0043] In the pulsating baffle active control system of the present invention, the parameter that requires generalization is the effect of different control actions performed by the lower baffle at different positions on the flow field. A computational fluid dynamics solver was used to perform multi-time series numerical simulations of the pulsating baffle active control system under study, obtaining flow field data for each time step. In each time step, the upper baffle periodically moves at a speed of 0.3 m / s and an amplitude of 0.22 m, while the lower baffle randomly selects an available speed from {-0.2 m / s, -0.3 m / s, -0.4 m / s, 0.2 m / s, 0.3 m / s, 0.4 m / s} as its speed. The calculation time step is 0.1 s, and the total calculation time is 500 s, resulting in 5000 flow field data points. The flow field data at each time step are projected onto the fluid matrix F, whose dimensions are (224, 224, 1). The solid field data at each time step are projected onto the solid matrix S, whose dimensions are (224, 224, 1). The fluid matrix F and the solid matrix S have the same dimensions. The flow field positions corresponding to the solid matrix S are assigned a value of 1, and the solid field positions corresponding to the solid matrix S are assigned a value of 0. The constructed database consists of the fluid matrix F and the solid matrix S.

[0044] Step 2: Construct training set and test set according to time sequence;

[0045] In this step, the fluid matrix F is normalized to obtain the normalized fluid matrix F n , the normalized fluid matrix F n Multiply with the solid matrix S to get the mask fluid matrix F m , that is, F m =F n S; the mask fluid matrix at time step t-1 and the solid matrix S with time step t t Perform splicing processing on the time dimension to obtain the variable matrix T with a time step of t t ; The mask fluid matrix with time step t As the variable matrix T t The label matrix of each time step; the variable matrix T tand the variable matrix T at each time step t The label matrices are concatenated to obtain the masked deep neural network training set and the masked deep neural network test set.

[0046] In the pulsation baffle active control system of the embodiment of the present invention, the maximum and minimum values ​​of the fluid matrix F are obtained to normalize the fluid matrix F to obtain the normalized fluid matrix F n ; Because the solid matrix only has 0 and 1, no normalization is required. The normalized fluid matrix F n Multiply with the solid matrix S to get the mask fluid matrix F m , that is, F m =F n S, this operation can make the solid boundary more obvious to enhance the boundary capture ability of the neural network. and the solid matrix S with time step t t Perform splicing processing on the time dimension to obtain the variable matrix T with a time step of t t , whose dimensions are (5000,2,224,224,1), the mask fluid matrix with time step t As the variable matrix T t The label matrix has a dimension of (5000, 224, 224, 1); the variable matrix T of each time step t and the variable matrix T at each time step t The label matrices are spliced ​​to obtain the masked deep neural network training set and the masked deep neural network test set of the pulsation baffle active control system according to an embodiment of the present invention.

[0047] Step 3: Build and train a masked deep neural network;

[0048] In this step, the masked deep neural network is constructed using convolutional layers, convolutional long-short-time neural network layers, and deconvolutional layers; the convolutional layers are used to capture the spatial features of the flow field data at each time step and reduce the dimension; the convolutional long-short-time neural network layers are used to generate the masked fluid matrix at time step t-1. and the solid matrix S with time step t t Predict the low-dimensional flow field features with a time step of t; use the deconvolution layer to reconstruct the low-dimensional flow field features with a time step of t to obtain the high-dimensional flow field with a time step of t High-dimensional flow field and the solid matrix S with time step t t Perform dot multiplication to obtain the predicted mask fluid matrix Right now Prediction mask fluid matrix The real flow field is obtained by performing inverse normalization processing.

[0049] The masked deep neural network training set and masked deep neural network test set constructed in step 2 are used to train the constructed masked deep neural network to obtain a usable masked deep neural network model; the usable masked deep neural network model can be used to calculate the fluid matrix F at time step t-1. t-1 and the solid matrix S with time step t t Complete the prediction of flow field data with time step t.

[0050] Figure 2 This is a schematic diagram of a masked deep neural network for an active flow control system based on a masked deep neural network and deep reinforcement learning according to the present invention. In the pulsating baffle active control system according to an embodiment of the present invention, the masked deep neural network constructed is divided into a convolution layer, a convolutional long-short-time neural network layer, and a deconvolution layer. The fluid matrix of the previous time step and the deformed solid matrix of the current time step are used as input, and the fluid matrix of the current time step is output. The convolution layer is used to capture the spatial characteristics of the flow field and reduce the dimension, and the low-dimensional flow field characteristics are used to predict the flow field changes to improve the computational efficiency. The residual connection structure of the residual neural network can overcome the gradient vanishing problem of the deep network; the mask operation therein can improve the prediction accuracy of the flow field.

[0051] Step 4: Design active flow control strategy by combining masked deep neural network and deep reinforcement learning algorithm;

[0052] In this step, the deep reinforcement learning algorithm and the available masked deep neural network model obtained in step 3 are used to build an active flow control system based on masked deep neural network and deep reinforcement learning.

[0053] At the beginning of the calculation, according to the active flow control problem to be solved, the fluid matrix F corresponding to the initial flow field is given 0 ; The intelligent agent of the deep reinforcement learning algorithm converts the fluid matrix F corresponding to the initial location of the flow field 0 As the state, the convolutional neural network is applied to the fluid matrix F corresponding to the initial position of the flow field 0 Extract features and output the control action as a solid matrix S 1 ; Solid matrix S 1 The fluid matrix F corresponding to the initial location of the flow field 0 As the input of the available mask deep neural network model obtained in step 3, the fluid matrix F with time step 1 is 1 Make predictions; the agent of the deep reinforcement learning algorithm is based on the fluid matrix F at time step 1 1 The control action is obtained as the solid matrix S 1 The corresponding reward completes an interaction and data collection, and the fluid matrix F with time step 1 1As the state input of the agent of the deep reinforcement learning algorithm at time step 2.

[0054] Iterate according to the time series, and the agent of the deep reinforcement learning algorithm will transform the fluid matrix F at time step t-1 t-1 As the state, apply the convolutional neural network to the fluid matrix F at time step t-1 t-1 Extract features and output the control action as a solid matrix S t ; Solid matrix S t and the fluid matrix F at time step t-1 t-1 As the input of the available mask deep neural network model obtained in step 3, the fluid matrix F at time step t is t Make predictions; the agent of the deep reinforcement learning algorithm is based on the fluid matrix F at time step t t The control action is obtained as the solid matrix S t The corresponding reward completes an interaction and data collection, and the fluid matrix F with time step t t As the state input of the agent of the deep reinforcement learning algorithm at time step t+1.

[0055] Iterate the time sequence from time step 1 to time step t until the total number of iteration steps T is reached, thereby completing the design of the active flow control system based on masked deep neural network and deep reinforcement learning.

[0056] like Figure 4 The figure shows a comparison of the control effects of the active flow control system based on masked deep neural network and deep reinforcement learning of the present invention and the active flow control system based on traditional deep reinforcement learning on the pulsating baffle active control system of the embodiment of the present invention. In the legend, "based on computational fluid dynamics" represents the solution result of the traditional active flow control system based on computational fluid dynamics and deep reinforcement learning, "based on masked deep neural network" represents the calculation result of the active flow control system based on masked deep neural network and deep reinforcement learning of the present invention, and "no control" represents the result of not applying control. Figure 4A comparison of the results shows that the lift per unit length of the controlled intermediate baffle is reduced to 90% of the original, significantly suppressing the system's vibration. While achieving the same control effect, the interactive response time between the environment and the deep reinforcement learning algorithm can reach seconds when rapidly predicting the flow field using the active flow control system based on a masked deep neural network and deep reinforcement learning of the present invention. The computing resource consumption is only 5% of that of a traditional active flow control system based on computational fluid dynamics and deep reinforcement learning. This demonstrates the significant advantages of the active flow control system based on a masked deep neural network and deep reinforcement learning of the present invention in solving active flow control problems.

[0057] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0058] In summary, this paper establishes a flow field reduction model based on a masked deep neural network to provide rapid prediction of the unsteady flow field of a control volume under arbitrary motion disturbances. This model replaces traditional computational fluid dynamics solutions and interacts with deep reinforcement learning algorithms as an environment to rapidly solve active flow control strategies. This effectively addresses the long iteration cycles and high computational overhead associated with traditional active flow control strategy development, providing a rapid solution for active flow control problems in engineering cases and facilitating the rapid design and analysis of active flow control systems.

Claims

1. An active flow control system based on masked deep neural network and deep reinforcement learning, characterized in that The steps include: Step 1: Create a database; Step 2: Construct training set and test set according to time series; Step 3: Build and train a masked deep neural network; Step 4: Design active flow control strategy by combining masked deep neural network and deep reinforcement learning algorithm; ① The establishment of a database includes: Determine the parameters that need to be generalized for the active flow control problem under study, use a computational fluid dynamics solver to perform multi-time series numerical simulation calculations on the active flow control problem example under study, and obtain flow field data at each time step; wherein the spatial position of the active flow control problem example under study includes the flow field position and the solid field position; the time series is composed of time step 1 to time step t, and the value of time step t is a positive integer; the motion mode of the control body of the active flow control problem under study adopts a random motion mode within the control range; Projecting the flow field data of each time step into a fluid matrix F, and projecting the solid field data of each time step into a solid matrix S; wherein the fluid matrix F and the solid matrix S have the same dimensions; Assigning the flow field position corresponding to the solid matrix S to 1, and assigning the solid field position corresponding to the solid matrix S to 0; the database is composed of the fluid matrix F and the solid matrix S; ② The construction of training sets and test sets according to time series includes: Normalizing the fluid matrix F to obtain a normalized fluid matrix Fn, and performing a dot product between the normalized fluid matrix Fn and the solid matrix S to obtain a mask fluid matrix Fm, that is, Fm=Fn·S; The mask fluid matrix at time step t-1 and the solid matrix S with time step t t Perform splicing processing on the time dimension to obtain the variable matrix T with a time step of t t ; The mask fluid matrix with time step t As the variable matrix T t The label matrix of each time step; the variable matrix T t and the variable matrix T at each time step t The label matrix is ​​spliced ​​to obtain the masked deep neural network training set and the masked deep neural network test set; ③ The construction and training of the masked deep neural network includes: The mask deep neural network is constructed by using a convolutional layer, a convolutional long-short time neural network layer and a deconvolutional layer; the convolutional layer is used to capture the spatial features of the flow field data at each time step and reduce the dimension; the convolutional long-short time neural network layer is based on the mask fluid matrix of the time step t-1 and the solid matrix S with time step t t Predict the low-dimensional flow field features with a time step of t; reconstruct the low-dimensional flow field features with a time step of t using the deconvolution layer to obtain a high-dimensional flow field with a time step of t The high-dimensional flow field and the solid matrix S at the time step t t Perform dot multiplication to obtain the predicted mask fluid matrix Right now The prediction mask fluid matrix Perform inverse normalization to obtain the true flow field; The masked deep neural network training set and the masked deep neural network test set constructed in step 2 are used to train the masked deep neural network to obtain a usable masked deep neural network model; the usable masked deep neural network model can be used to calculate the fluid matrix F at time step t-1. t-1 and the solid matrix S with time step t t Complete the prediction of flow field data with time step t; ④ The active flow control strategy design combining the masked deep neural network and the deep reinforcement learning algorithm includes: Using a deep reinforcement learning algorithm and the available masked deep neural network model obtained in step 3 to build an active flow control system based on masked deep neural network and deep reinforcement learning; At the beginning of the calculation, according to the active flow control problem to be solved, the fluid matrix F corresponding to the initial flow field is given 0 ; The intelligent agent of the deep reinforcement learning algorithm converts the fluid matrix F corresponding to the initial location of the flow field 0 As the state, apply the convolutional neural network to the fluid matrix F corresponding to the initial location of the flow field 0 Extract features and output the control action as a solid matrix S 1 ; The solid matrix S 1 The fluid matrix F corresponding to the initial location of the flow field 0 As the input of the available mask deep neural network model obtained in step 3, the fluid matrix F with time step 1 is 1 Make a prediction; the intelligent agent of the deep reinforcement learning algorithm is based on the fluid matrix F with a time step of 1 1 Get the control action for the solid matrix S 1 The corresponding reward completes an interaction and data collection, and the fluid matrix F with the time step of 1 1 As the state input of the agent of the deep reinforcement learning algorithm at time step 2; Iterate according to the time sequence, and the agent of the deep reinforcement learning algorithm converts the fluid matrix F at time step t-1 t-1 As the state, apply the convolutional neural network to the fluid matrix F at time step t-1 t-1 Extract features and output the control action as a solid matrix S t ; The solid matrix S t The fluid matrix F at time step t-1 t-1 As the input of the available mask deep neural network model obtained in step 3, the fluid matrix F with time step t is t Make a prediction; the intelligent agent of the deep reinforcement learning algorithm is based on the fluid matrix F at the time step t t Get the control action for the solid matrix S t The corresponding reward completes an interaction and data collection, and the fluid matrix F with the time step t t As input to the state of the agent of the deep reinforcement learning algorithm at time step t+1; Iterate the time sequence from time step 1 to time step t until the total number of iteration steps T is reached, thereby completing the design of the active flow control system based on masked deep neural network and deep reinforcement learning.

Images