A rotor airfoil optimization method and system based on deep reinforcement learning

Abstract

The application relates to a rotor airfoil optimization method and system based on deep reinforcement learning, and belongs to the field of airfoil design. Through a large number of interactive learning between an intelligent agent and an environment, an optimization strategy is obtained, a new airfoil with better performance can be obtained by dynamically optimizing a target airfoil, and the optimization process corresponding to the optimization strategy has physical interpretability; an agent model for predicting a rotor airfoil aerodynamic force hysteresis loop is established by using a deep neural network, and the agent model has better nonlinear prediction capability.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of airfoil design, in particular to a rotor airfoil optimization method and system based on deep reinforcement learning. BACKGROUND

[0002] Rotor is an important component of helicopter, which determines its flight performance. Dynamic stall is a common flow phenomenon of rotor, which can make the rotor drag coefficient rise and the rotor lift coefficient drop, greatly reducing the aerodynamic performance of the rotor. Therefore, how to suppress dynamic stall is a key problem that needs to be solved in rotor design. Airfoil is the basic element of rotor, which determines the aerodynamic performance of rotor. If a rotor airfoil can suppress dynamic stall, it will fundamentally alleviate the dynamic stall of rotor and improve its aerodynamic performance.

[0003] Optimization is an important means of airfoil design. In recent years, scholars have carried out related research on rotor airfoil aerodynamic optimization. In 2012, Mani et al. used the adjoint method to optimize the SC1095 rotor airfoil and obtained an airfoil that can suppress dynamic stall. In 2015, Wang Qing et al. based on the optimization algorithm of quadratic sequence planning, carried out research on dynamic stall optimization of rotor airfoil. In 2020, Yu Boping et al. used the method based on surrogate model to optimize the dynamic stall of rotor airfoil. It can be seen that: although these airfoil optimization methods have achieved certain optimization effect, they are realized through large-scale optimization, without learning experience from the airfoil optimization process, and the optimization strategy is not physically interpretable, which is also difficult to provide guidance for future airfoil design.

[0004] In recent years, the development of reinforcement learning has attracted widespread attention from many scholars, and it has been widely applied in computer games, robot control, natural language processing and other fields. This method can learn the strategy of maximizing the reward through the interaction between the agent and the environment. Compared with other machine learning methods, this method improves the "black box" property of the model, has the ability to accumulate experience from a large number of experiments and trial and error, and in principle, reinforcement learning itself is not limited by the nonlinear characteristics of the system. Recently, reinforcement learning has also been applied to the field of fluid mechanics. Some scholars have introduced reinforcement learning into the field of flow control. Rabault et al. used reinforcement learning algorithm to control the zero mass jet on the lower end of the cylinder, achieving the effect of suppressing Karman vortex street and reducing drag. Some scholars have introduced reinforcement learning into the study of fish flow behavior. Verma et al. used reinforcement learning algorithm to train the swimming position and body deformation of fish, improving the swimming efficiency of fish. The results show that reinforcement learning can learn the jet control strategy or fish swimming strategy well. It can be predicted that if reinforcement learning is applied to the dynamic stall optimization of rotor airfoil, the optimization strategy learned by reinforcement learning can be physically interpretable, which can better guide the design of rotor airfoil and truly play the "intelligent" advantage of the method. SUMMARY

[0005] The application aims to provide a rotor airfoil optimization method and system based on deep reinforcement learning, so that the optimization process has physical interpretability and improves nonlinear prediction capability.

[0006] To achieve the above-mentioned purpose, the application provides the following scheme:

[0007] A rotor airfoil optimization method based on deep reinforcement learning, comprising:

[0008] Parameterizing the helicopter rotor airfoil and establishing an airfoil database; each airfoil in the airfoil database is represented by a set of parameter values;

[0009] Determine the aerodynamic force hysteresis loop corresponding to each airfoil in the airfoil database;

[0010] Each airfoil parameter and the aerodynamic force hysteresis loop corresponding to each airfoil form a training data set;

[0011] Use the training data set to train a deep neural network to obtain a proxy model for predicting the aerodynamic force hysteresis loop;

[0012] With the goal of suppressing the dynamic stall of the rotor airfoil, use the proxy model to establish an environment required for reinforcement learning;

[0013] Use the established environment to train an agent using a reinforcement learning algorithm to obtain an optimized strategy and an optimized airfoil.

[0014] Optionally, the parameterization of the helicopter rotor airfoil and the establishment of the airfoil database specifically include:

[0015] The upper and lower surfaces of the helicopter rotor airfoil are represented by 6 parameters using a shape category function transformation method;

[0016] Use the Latin hypercube method to sample the values of each parameter within the design domain to obtain multiple airfoils and form an airfoil database; the design domain is the range of airfoil geometric shape changes.

[0017] Optionally, the determination of the aerodynamic force hysteresis loop corresponding to each airfoil in the airfoil database specifically includes:

[0018] Use a computational fluid dynamics solver to numerically simulate each airfoil in the airfoil database and calculate the aerodynamic force hysteresis loop corresponding to each airfoil; the aerodynamic force hysteresis loop includes lift coefficient hysteresis loop, drag coefficient hysteresis loop and moment coefficient hysteresis loop.

[0019] Optionally, the state in the reinforcement learning is 12 parameters of the airfoil, the action is the increase or decrease operation on the 12 parameters, and the reward is set as the reduction value of the peak value of the airfoil drag coefficient and the moment coefficient.

[0020] The environment required for establishing the reinforcement learning is that the airfoil airfoil-i obtains the array of rewards reward-i under the action action-i.

[0021] Optionally, the established environment is used to train the agent by using a reinforcement learning algorithm to obtain an optimized strategy and an optimized airfoil, and specifically includes the following steps.

[0022] The established environment is used to train the agent by using a reinforcement learning algorithm A3C, and after n rounds of training, an optimized strategy is obtained, and an optimized airfoil is obtained.

[0023] A rotor airfoil optimization system based on deep reinforcement learning, comprising:

[0024] A parameterization module is configured to parameterize the helicopter rotor airfoil and establish an airfoil database, wherein each airfoil in the airfoil database is represented by a set of parameter values.

[0025] A numerical simulation module is configured to perform numerical simulation on each airfoil in the airfoil database to obtain the corresponding aerodynamic force hysteresis loop of each airfoil.

[0026] A training data set construction module is configured to construct a training data set from the parameters of each airfoil and the corresponding aerodynamic force hysteresis loop of each airfoil.

[0027] An agent model establishment module is configured to train a deep neural network using the training data set to obtain an agent model for predicting the aerodynamic force hysteresis loop.

[0028] An environment establishment module is configured to establish an environment required for reinforcement learning by using the agent model to suppress the dynamic stall of the rotor airfoil.

[0029] An airfoil optimization module is configured to train an agent by using a reinforcement learning algorithm using the established environment to obtain an optimized strategy and an optimized airfoil.

[0030] An electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements the deep reinforcement learning-based rotor airfoil optimization method as described above when executing the computer program.

[0031] A computer-readable storage medium has a computer program stored thereon, and the computer program is executed to implement the deep reinforcement learning-based rotor airfoil optimization method as described above.

[0032] According to the specific embodiments of the present application, the following technical effects are disclosed:

[0033] The application discloses a rotor airfoil optimization method and system based on deep reinforcement learning, and the method comprises the following steps: obtaining an optimization strategy through a large number of interactive learning between an agent and an environment; and obtaining a new airfoil with better performance by dynamically stalling the target airfoil according to the optimization strategy, wherein the optimization process corresponding to the optimization strategy has physical interpretability; and the agent model for predicting the hysteresis loop of the rotor airfoil aerodynamic force is established by using a deep neural network, and the agent model has better nonlinear prediction capability. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below, and obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0035] Figure 1 The flowchart of the rotor airfoil optimization method based on deep reinforcement learning provided by the embodiments of the present application is shown in the figure.

[0036] Figure 2 The principle diagram of the rotor airfoil optimization method based on deep reinforcement learning provided by the embodiments of the present application is shown in the figure.

[0037] Figure 3 The calculation model diagram of the rotor airfoil OA212 dynamic stall provided by the embodiments of the present application is shown in the figure.

[0038] Figure 4 The geometric change diagram of the airfoil optimization process provided by the embodiments of the present application is shown in the figure.

[0039] Figure 5 The airfoil geometric change diagram of the reference airfoil OA212 provided by the embodiments of the present application is shown in the figure. Figure 5 (a) in the figure is the airfoil geometric change diagram of the 7th to 14th stage, Figure 5 (b) in the figure is the airfoil geometric change diagram of the 15th to 33rd stage, Figure 5 (c) in the figure is the airfoil geometric change diagram of the 37th to 41st stage, Figure 5 (d) in the figure is the airfoil geometric change diagram of the 42nd to 51st stage.

[0040] Figure 6 The comparison diagram of the reference airfoil OA212 and the optimized airfoil aerodynamic force coefficient hysteresis loop provided by the embodiments of the present application is shown in the figure. Figure 6 (a) in the figure is the comparison diagram of the reference airfoil OA212 and the lift coefficient hysteresis loop, Figure 6(b) in FIG. 1 is a comparison diagram of the baseline airfoil OA212 and the drag coefficient hysteresis loop, Figure 6 (c) in FIG. 1 is a comparison diagram of the baseline airfoil OA212 and the moment coefficient hysteresis loop.

[0041] Figure 7 The comparison diagram of the baseline airfoil OA212 and the optimized airfoil flow separation vortex provided by the embodiment of the present application is shown in FIG. 1. Figure 7 (a) in FIG. 1 is a surface streamline diagram of the baseline airfoil OA212 at an upstroke side angle of attack of 16.9°, Figure 7 (b) in FIG. 1 is a surface streamline diagram of the optimized airfoil at an upstroke side angle of attack of 16.9°. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0043] The purpose of the present application is to provide a rotor airfoil optimization method and system based on deep reinforcement learning, which has physical interpretability in the optimization process and improves the nonlinear prediction ability.

[0044] In order to make the above-mentioned purposes, characteristics and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0045] Dynamic stall: a phenomenon that occurs in the rotor of a helicopter in high-speed forward flight, heavy-load flight, etc. Dynamic stall has an adverse effect on the performance of the helicopter. Therefore, dynamic stall should be avoided as much as possible when designing the rotor.

[0046] Reinforcement learning (RL): a technique belonging to machine learning. Reinforcement learning discusses how an intelligent agent maximizes the rewards it can obtain in a complex and uncertain environment. By perceiving the state of the environment in response to the action, better actions are guided to obtain the maximum benefit. This method of learning in interaction is called reinforcement learning.

[0047] To solve the problem of poor interpretability of conventional optimization methods, the present application provides a rotor airfoil optimization method based on deep reinforcement learning, which provides a new optimization method for rotor dynamic stall optimization.

[0048] The rotor airfoil optimization method based on deep reinforcement learning provided by the present application is as follows: Figure 1 andFigure 2 As shown, comprising the following steps:

[0049] Step S1, parameterize the helicopter rotor airfoil and establish an airfoil database; each airfoil in the airfoil database is represented by a set of parameter values.

[0050] Parameterize the airfoil, sample in the design domain, and establish an airfoil database. The parameterization method uses the CST (Class-Shape Transform) method to represent the upper and lower surfaces with 6 parameters, and the sampling method uses the Latin hypercube method to obtain 520 airfoils. Among them, the design domain is the range of changes in the geometric shape of the airfoil.

[0051] Step S2, determine the aerodynamic force hysteresis loop corresponding to each airfoil in the airfoil database.

[0052] Perform computational fluid dynamics (CFD) numerical simulation calculation on all airfoils obtained in step S1 to obtain the hysteresis loop of the lift coefficient, drag coefficient and moment coefficient corresponding to each airfoil, thereby establishing an airfoil hysteresis loop database. The aerodynamic force hysteresis loop includes the lift coefficient hysteresis loop, the drag coefficient hysteresis loop and the moment coefficient hysteresis loop.

[0053] The aerodynamic force hysteresis loop corresponding to each airfoil can also be determined by wind tunnel experiment.

[0054] Step S3, construct a training data set from the parameters of each airfoil and the aerodynamic force hysteresis loop corresponding to each airfoil.

[0055] Organize the airfoil database and hysteresis loop database of step S1 and step S2 with corresponding labels to obtain a data set for deep neural network training.

[0056] Step S4, train a deep neural network using the training data set to obtain a proxy model for predicting aerodynamic force hysteresis loop.

[0057] A deep neural network is used to train a proxy model for predicting airfoil aerodynamic coefficient hysteresis loop. 520 airfoils are used as input data, and the corresponding aerodynamic force hysteresis loop is used as output. The neural network is built, the hyperparameters are adjusted for model training, and the proxy model is obtained after training convergence.

[0058] Step S5, to suppress the dynamic stall of the rotor airfoil, a proxy model is used to establish the environment required for reinforcement learning.

[0059] The state in the reinforcement learning is the parameterized variable of the airfoil geometry, and the action is the increase or decrease operation on the 12 parameters (6 parameters for the upper surface and 6 parameters for the lower surface, a total of 12 parameters). In order to suppress the dynamic stall of the rotor airfoil, the optimization of the airfoil needs to have lower peak values of the drag coefficient and the moment coefficient during the pitch oscillation process, so the reward of the reinforcement learning is set as the reduction value of the peak values of the airfoil drag coefficient and the moment coefficient. Finally, the environment of the reinforcement learning is established: the airfoil airfoil-i obtains the array of the reward reward-i under the action action-i.

[0060] In step S6, the established environment is used to train the agent by using the reinforcement learning algorithm to obtain the optimized strategy and the optimized airfoil.

[0061] The agent is trained by using the reinforcement learning algorithm A3C (Asynchronous Advantage Actor-Critic), n rounds of training are performed, the optimized strategy is obtained, and the optimized airfoil is obtained by applying the optimized strategy to the benchmark airfoil.

[0062] The OA212 airfoil is taken as an example to learn the optimized strategy. Figure 3 The dynamic stall calculation model of the airfoil OA212 is shown in the figure. The airfoil performs a simple harmonic motion with the 1 / 4 chord line as the rotation center in the flow field with a flow velocity v = 0.3Ma, and the angle of attack changes according to the law α = 10° + 8°sin(ωt). The chord length c of the airfoil is 0.57m, and the Reynolds number Re is 3.735×10 6 . Figure 4 The benchmark airfoil, the optimized airfoil and the airfoil generated in the optimization process. Figure 5 The airfoil geometry change diagram of the airfoil optimization process. Figure 6 The (a), (b) and (c) in the figure are respectively the comparison diagrams of the lift coefficient, the drag coefficient and the moment coefficient hysteresis loops of the airfoil before and after optimization. It can be seen that the overall lift coefficient of the optimized airfoil increases, the amplitude of the drag coefficient and the moment coefficient decreases, and the dynamic stall characteristics obviously weaken. Figure 6 The horizontal and vertical coordinates in the figure are respectively the dimensionless values of the airfoil horizontal and vertical coordinates. Figure 7 The surface streamlines of the benchmark airfoil and the optimized airfoil at the upstroke side angle of attack 16.9° are shown in the figure. It can be seen that the separation vortex of the optimized airfoil is obviously weakened compared with the benchmark airfoil, the leading edge separation position is pushed back, and the generation of the dynamic stall vortex of the airfoil is suppressed.

[0063] When the airfoil undergoes dynamic stall, the main characteristics are: the area of the closed loop surrounded by the lift coefficient hysteresis loop increases, and the peak values of the drag coefficient and the moment coefficient are larger. Therefore, the optimization goal is to reduce the peak values of the drag coefficient and the moment coefficient while keeping the lift coefficient unchanged. From the above characteristics, it can be concluded that the dynamic stall characteristics of the optimized airfoil are weakened.

[0064] The advantages of the present application are as follows:

[0065] 1. The present application is based on a rotor airfoil dynamic stall optimization strategy learning of deep reinforcement learning, a large number of interactive learning between an agent and an environment is carried out, an optimization strategy is obtained, and a new airfoil with better performance can be obtained by performing dynamic stall optimization on a target airfoil. The optimization process corresponding to the optimization strategy has physical interpretability, and can provide optimization reference for designers.

[0066] 2. The rotor airfoil aerodynamic force hysteresis loop proxy model established by the deep neural network adopted by the present application has better nonlinear prediction capability.

[0067] The present application also provides a rotor airfoil optimization system based on deep reinforcement learning, comprising:

[0068] A parameterization module is used to parameterize the helicopter rotor airfoil and establish an airfoil database; each airfoil in the airfoil database is represented by a set of parameter values;

[0069] A numerical simulation module is used to numerically simulate each airfoil in the airfoil database to obtain the aerodynamic force hysteresis loop corresponding to each airfoil;

[0070] A training data set forming module is used to form a training data set from the parameters of each airfoil and the aerodynamic force hysteresis loop corresponding to each airfoil;

[0071] A proxy model establishing module is used to train a deep neural network using the training data set to obtain a proxy model for predicting the aerodynamic force hysteresis loop;

[0072] An environment establishing module is used to establish an environment required for reinforcement learning by using the proxy model to suppress the dynamic stall of the rotor airfoil;

[0073] An airfoil optimization module is used to train an agent using a reinforcement learning algorithm to obtain an optimization strategy and an optimized airfoil using the established environment.

[0074] The rotor airfoil optimization system based on deep reinforcement learning provided by the embodiments of the present application has similar working principles and beneficial effects to the rotor airfoil optimization method based on deep reinforcement learning described in the above embodiments, and therefore will not be described in detail here. For specific content, please refer to the introduction of the above method embodiments.

[0075] The present application also provides an electronic device, characterized in that it comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the rotor airfoil optimization method based on deep reinforcement learning as described above when executing the computer program.

[0076] The computer program described above can be realized based on software, and embodied in the form of a software product. The computer software product is stored in a storage medium, and the storage medium can be a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk, or an optical disk, etc.

[0077] Further, the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed to realize the rotor airfoil optimization method based on deep reinforcement learning as described above.

[0078] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the system disclosed by the embodiments, since it corresponds to the method disclosed by the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part.

[0079] The principles and implementation manners of the present application are described by using specific examples in this paper, and the above embodiment description is only used to help understand the method of the present application and its core idea. Meanwhile, for the general technical personnel in the field, according to the idea of the present application, the specific implementation manner and application range will be changed. In conclusion, the content of the specification should not be understood as the limitation of the present application.

Claims

1. A rotor airfoil optimization method based on deep reinforcement learning, characterized in that, The application comprises the following steps: Parameterizing the airfoil of a helicopter rotor and establishing an airfoil database; specifically, the upper and lower surfaces of the airfoil of a helicopter rotor are represented by six parameters respectively by using a shape category function transformation method; a Latin hypercube method is used to sample the values of each parameter in the design domain to obtain multiple airfoils, which constitute the airfoil database; the design domain is the range of changes in the geometric shape of the airfoil; each airfoil in the airfoil database is represented by a set of parameter values; Determining the aerodynamic force hysteresis loop corresponding to each airfoil in the airfoil database; specifically, a computational fluid dynamics solver is used to numerically simulate each airfoil in the airfoil database to calculate the aerodynamic force hysteresis loop corresponding to each airfoil; the aerodynamic force hysteresis loop includes the lift coefficient hysteresis loop, the drag coefficient hysteresis loop and the moment coefficient hysteresis loop; An airfoil parameter and the aerodynamic force hysteresis loop corresponding to each airfoil are used to constitute a training data set; A deep neural network is trained using the training data set to obtain a proxy model for predicting the aerodynamic force hysteresis loop; An environment required by reinforcement learning is established by using the proxy model to suppress the dynamic stall of the airfoil of a rotor; An agent is trained using a reinforcement learning algorithm in the established environment to obtain an optimized strategy and an optimized airfoil.

2. The deep reinforcement learning-based rotor airfoil optimization method according to claim 1, wherein, In the reinforcement learning, the state is the 12 parameters of the airfoil, the action is the increase or decrease operation on the 12 parameters, and the reward is set as the reduction value of the peak values of the airfoil drag coefficient and the moment coefficient. The environment required by reinforcement learning is that the airfoil airfoil-i obtains the array of rewards reward-i under the action action-i.

3. The deep reinforcement learning-based rotor airfoil optimization method of claim 1, wherein, The agent is trained using the reinforcement learning algorithm A3C in the established environment, and after n rounds of training, an optimized strategy is obtained, and an optimized airfoil is obtained. The application comprises the following steps:

4. A deep reinforcement learning based rotor airfoil optimization system, characterized in that, A parameterization module is used to parameterize the airfoil of a helicopter rotor and establish an airfoil database; specifically, the upper and lower surfaces of the airfoil of a helicopter rotor are represented by six parameters respectively by using a shape category function transformation method; a Latin hypercube method is used to sample the values of each parameter in the design domain to obtain multiple airfoils, which constitute the airfoil database; the design domain is the range of changes in the geometric shape of the airfoil; each airfoil in the airfoil database is represented by a set of parameter values; A numerical simulation module is used to numerically simulate each airfoil in the airfoil database to obtain the aerodynamic force hysteresis loop corresponding to each airfoil; specifically, a computational fluid dynamics solver is used to numerically simulate each airfoil in the airfoil database to calculate the aerodynamic force hysteresis loop corresponding to each airfoil; the aerodynamic force hysteresis loop includes the lift coefficient hysteresis loop, the drag coefficient hysteresis loop and the moment coefficient hysteresis loop; A training data set construction module is used to construct a training data set from an airfoil parameter and the aerodynamic force hysteresis loop corresponding to each airfoil; A proxy model establishment module is used to train a deep neural network using the training data set to obtain a proxy model for predicting the aerodynamic force hysteresis loop. ​ An environment establishing module is configured to establish an environment required by reinforcement learning by using a proxy model to target dynamic stall of a rotor airfoil; An airfoil optimization module is configured to train an agent by using a reinforcement learning algorithm to obtain an optimized strategy and an optimized airfoil by using the established environment.

5. An electronic device, comprising: A computer program product comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the deep reinforcement learning based rotor airfoil optimization method according to any one of claims 1 to 3 when executing the computer program.

6. A computer readable storage medium characterized by A computer program product comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the deep reinforcement learning based rotor airfoil optimization method according to any one of claims 1 to 3 when executing the computer program.

Images