Multi-fan wind tunnel velocity field control method and system based on reinforcement learning

Abstract

The present invention belongs to the field of wind tunnel flow field control, and discloses a multi-fan wind tunnel velocity field control method and system based on reinforcement learning. The method includes: generating theoretical working condition samples based on wind tunnel aerodynamic design parameters, constructing a state-action mapping relationship by injecting velocity noise and actuator noise, and forming a noisy mapping data set; using the mapping data set to pre-train a deep neural network to learn fan adjustment strategies in a noisy environment; deploying the pre-trained network to a wind tunnel environment for real-time interaction, optimizing network parameters using the uniformity of the test section velocity field as a reward signal, and completing the optimization of the reinforcement learning controller; using the optimized deep neural network controller to close-loop adjust the fan speed to achieve uniform control of the test section velocity field. The present invention overcomes the drawbacks of traditional wind tunnel control methods that rely on precise models and expert knowledge and are cumbersome to adjust parameters, and realizes automatic optimization of controller parameters, reducing wind tunnel debugging time and resource consumption, while also having good anti-interference performance.

Description

Technical Field

[0001] The present invention relates to the field of wind tunnel flow field control, and in particular to a multi-fan wind tunnel velocity field control method and system based on reinforcement learning. Background Art

[0002] Wind tunnels are essential experimental equipment for aerodynamic research. They are specially designed pipes that simulate real-world atmospheric conditions by manually controlling the flow of gas within them. Wind tunnel testing plays a crucial role in aerodynamic research.

[0003] In traditional single-fan wind tunnels, to achieve low-turbulence, nearly isotropic turbulent flow in the test section, a series of straightening devices are often deployed within the larger, upstream stable section of the wind tunnel. These devices improve airflow uniformity and reduce airflow deflection and turbulence. These straightening devices include honeycombs and damping meshes. Honeycombs can split large-scale vortices into smaller ones, significantly reducing turbulence. Damping meshes, on the other hand, reduce airflow turbulence; the finer the mesh, the better the straightening effect.

[0004] Multi-fan wind tunnels are equipped with multiple fans in the power section, each with independently controlled speed. Through appropriate control strategies, they can actively simulate the target velocity field in the test section. Therefore, in addition to improving velocity field uniformity through rectifiers, multi-fan wind tunnels can also achieve a uniform velocity field consistent with the target speed in the test section by designing appropriate control strategies that use the actual velocity field distribution as feedback to adjust the speed of each fan in real time.

[0005] Existing multi-fan wind tunnel control mainly relies on model predictive control or PID control, but these methods have significant defects: 1) Strong model dependence: a precise fluid dynamics model is required, but the nonlinear and strongly coupled characteristics of the wind tunnel flow field make modeling difficult; 2) Manual parameter adjustment is time-consuming: control parameters need to be repeatedly adjusted manually, which cannot adapt to the real-time control requirements of complex dynamic wind fields; 3) Weak anti-interference ability: lack of adaptive adjustment capability to external interference (such as model posture changes, air source fluctuations).

[0006] In view of this, the present invention is proposed. Summary of the Invention

[0007] The present invention aims to solve at least one of the technical problems in the related art to a certain extent, and provides a multi-fan wind tunnel velocity field control method and system based on reinforcement learning.

[0008] In order to achieve the above object, the first technical solution adopted by the present invention is:

[0009] The multi-fan wind tunnel velocity field control method based on reinforcement learning includes:

[0010] Generate theoretical working condition samples based on wind tunnel aerodynamic design parameters, and construct a state-action mapping relationship by injecting velocity noise and actuator noise to form a noisy mapping data set.

[0011] Pre-training a deep neural network using the mapping dataset to learn a fan adjustment strategy in a noisy environment;

[0012] Deploy the pre-trained network to a wind tunnel environment for real-time interaction, use the velocity field uniformity of the test section as a reward signal to optimize network parameters and complete the optimization of the reinforcement learning controller.

[0013] The optimized deep neural network controller is used to adjust the fan speed in a closed loop to achieve uniform control of the velocity field in the test section.

[0014] Preferably, the construction of the noisy mapping dataset includes:

[0015] Generate theoretical fan speed distribution and corresponding theoretical test section velocity field for target operating conditions;

[0016] Add velocity noise simulation value to the velocity field of theoretical test section;

[0017] Adding actuator noise simulation values ​​to the theoretical fan speed distribution;

[0018] Generate a data set containing the noise velocity field, disturbed speed, and compensation adjustment.

[0019] Preferably, the speed noise is a random offset of the wind speed at the test section measurement point, and the actuator noise is a random offset of the fan speed.

[0020] Preferably, the input layer dimension of the deep neural network is the sum of the number of fans, the number of measuring points, the target wind speed and the total pressure parameter, and the output layer dimension is equal to the number of fans.

[0021] Preferably, the reward signal is positively correlated with the uniformity of the velocity field in the test segment, and the reward value is saturated when the sum of the absolute deviations of the velocity distribution is lower than a threshold.

[0022] Preferably, the reward function is designed as the ratio of the number of test segment velocity field measurement points to the sum of the absolute errors of the test segment velocity field.

[0023] Preferably, the optimization phase of the reinforcement learning controller uses a deep deterministic policy gradient algorithm, and the network update includes an experience replay mechanism.

[0024] Preferably, the construction of the environmental state in the optimization phase of the reinforcement learning controller relies on the total pressure parameter measured in real time, which together with the fan speed, the test section velocity field, and the target wind speed forms a four-dimensional state vector.

[0025] Preferably, the closed-loop adjustment of the fan speed comprises an iterative process:

[0026] Real-time collection of fan speed, total pressure and test section velocity field distribution;

[0027] Dynamically output speed adjustment instructions;

[0028] Verify uniformity specifications and cycle until achieved.

[0029] The second technical solution adopted in the present invention is:

[0030] Multi-fan wind tunnel velocity field control system based on reinforcement learning, including:

[0031] A construction module is used to generate theoretical working condition samples based on wind tunnel aerodynamic design parameters, construct a state-action mapping relationship by injecting velocity noise and actuator noise, and form a noisy mapping data set;

[0032] A training module, configured to use the mapping data set to pre-train a deep neural network to learn a fan adjustment strategy in a noisy environment;

[0033] The reinforcement learning module deploys the pre-trained network into the wind tunnel environment for real-time interaction, uses the uniformity of the velocity field in the test section as a reward signal to optimize network parameters and complete the optimization of the reinforcement learning controller.

[0034] The regulation module is used to adjust the fan speed in a closed loop using an optimized deep neural network controller to achieve uniform control of the velocity field in the test section.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] The present invention constructs a three-stage control method consisting of supervised learning pre-training, reinforcement learning optimization, and real-time closed-loop control. It creatively solves the problems of model dependence, manual parameter adjustment, and uniformity optimization in the control of the velocity field of a multi-fan wind tunnel. It can perform closed-loop control of the velocity field of a multi-fan wind tunnel without knowing the precise control model, and ultimately achieves a uniform velocity field in the test section through the speed distribution of multiple fans. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 2. A flow chart of a multi-fan wind tunnel velocity field control method based on reinforcement learning according to an embodiment of the present invention;

[0038] Figure 2 Schematic diagram of the structure of a multi-fan wind tunnel velocity field control system based on reinforcement learning according to one embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the implementation of this application clearer, the technical solutions in the embodiments of this application will be described in more detail below in conjunction with the drawings in the embodiments of this application. In the drawings, the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The described embodiments are part of the embodiments of this application, not all of the embodiments. The embodiments described below with reference to the drawings are exemplary and are intended to be used to explain this application, and should not be understood as limitations on this application. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0040] refer to Figure 1 The first embodiment of the present invention provides a multi-fan wind tunnel velocity field control method based on reinforcement learning, comprising the following steps:

[0041] S101: Generate theoretical working condition samples based on wind tunnel aerodynamic design parameters, construct a state-action mapping relationship by injecting velocity noise and actuator noise, and form a noisy mapping data set.

[0042] This step constructs a data set based on wind tunnel aerodynamic design parameters and simulates real working condition disturbances by adding random noise, which significantly improves the model generalization ability.

[0043] The construction of the above noisy mapping dataset includes:

[0044] Generate theoretical fan speed distribution and corresponding theoretical test section velocity field for target operating conditions;

[0045] Add velocity noise simulation value to the velocity field of theoretical test section;

[0046] Adding actuator noise simulation values ​​to the theoretical fan speed distribution;

[0047] Generate a data set containing the noise velocity field, disturbed speed, and compensation adjustment.

[0048] It should be noted that the velocity noise is the random offset of the wind speed at the test section measurement point, and the actuator noise is the random offset of the fan speed. The amplitudes of the velocity noise and actuator noise are adaptively set based on the wind tunnel measurement accuracy.

[0049] As an example, the specific calculation of the mapping data set is as follows:

[0050] Assuming that the number of fans in the power section is n and they are numbered from 1 to n, the number of velocity field measurement points in the test section is m and they are numbered from 1 to m, given the target wind speed v in the test section and the total pressure P of the wind tunnel operation, the theoretical speed value of each fan under this working condition can be determined according to the aerodynamic design parameters of the wind tunnel. , velocity field of the theoretical test section ,in Represents the speed of the nth fan, Represents the wind speed collected at the mth measuring point;

[0051] Randomly generate the velocity field noise value of the test section ,in Represents the noise value applied at the mth measurement point, and calculates the fan speed adjustment required to generate dV under the current working conditions ,in, Represents the speed adjustment required for the nth fan, and randomly generates the speed adjustment value of each fan based on this ,in, Represents the randomly generated n-th fan speed adjustment value; traverses different V and P to generate a mapping data set consisting of v, p, V+dv, R+dR2, and dR1+dR2.

[0052] S102: Pre-training a deep neural network using the mapping data set to learn a fan adjustment strategy in a noisy environment.

[0053] This step is pre-training, which uses a noisy mapping data set to initialize network parameters and train the deep neural network to learn the mapping relationship from input to output. This enables the deep neural network training to have a reasonable control strategy in the early stages, significantly shortening the training time compared to random initialization.

[0054] The deep neural network has an input dimension of n+m+2 and an output dimension of n. The input layer dimension is the sum of the number of fans, the number of measurement points, the target wind speed and the total pressure parameter, and the output layer dimension is equal to the number of fans. The network is pre-trained for supervised learning using a mean square error loss function and an Adam optimizer. The training is iterated until the network converges to obtain the pre-trained network parameters.

[0055] In some preferred embodiments, the deep neural network adopts a fully connected structure, the number of hidden layers is not less than 3, the number of neurons in each layer is not less than 128, the network update adopts the experience replay mechanism, and the replay buffer capacity is not less than 10 4 samples.

[0056] S103, deploying the pre-trained network to the wind tunnel environment for real-time interaction, optimizing network parameters using the uniformity of the velocity field in the test section as a reward signal, and completing the optimization of the reinforcement learning controller.

[0057] This step uses real-time data collected from the fan speed, total pressure, velocity field, and target wind speed as state inputs, reflecting system dynamics more comprehensively than existing solutions. By designing a reward function that maximizes velocity field uniformity while implicitly minimizing the adjustment amplitude, a balance between control accuracy and energy consumption is achieved. The construction of the environmental state in this step relies on the real-time measured total pressure parameter, which, together with the fan speed, test section velocity field, and target wind speed, forms a four-dimensional state vector.

[0058] As a preferred implementation, this stage employs a deep deterministic policy gradient algorithm, with network updates incorporating an experience replay mechanism. The reward signal is positively correlated with the uniformity of the velocity field during the test segment, and the reward value saturates when the sum of the absolute deviations of the velocity distribution falls below a threshold (e.g., approaches zero). For example, the reward function is designed to be the ratio of the number of velocity field measurement points during the test segment to the sum of the absolute errors in the velocity field during the test segment.

[0059] S104, uses the optimized deep neural network controller to close the loop to adjust the fan speed and achieve uniform control of the speed field in the test section.

[0060] This step collects the current fan speed, total pressure, test section velocity field, and target speed for each fan in real time. These data are fed into a deep neural network optimized through reinforcement learning, and the speed adjustment for each fan is output to the powertrain for speed adjustment. This step achieves dynamic compensation through an iterative acquisition-control-feedback process. Compared to the traditional rolling optimization of MPC, the strategy network of this invention can capture more complex nonlinear mapping relationships.

[0061] Specifically, closed-loop fan speed regulation involves an iterative process:

[0062] Real-time collection of fan speed, total pressure and test section velocity field distribution;

[0063] Dynamically output speed adjustment instructions;

[0064] Verify uniformity specifications and cycle until achieved.

[0065] The present invention utilizes deep neural networks and reinforcement learning to calculate fan speed adjustments in real time, thereby solving the problem of optimizing the uniformity of the velocity field in a multi-fan wind tunnel test section. In the pre-training phase, a dataset is constructed based on wind tunnel design data, and the weights of the reinforcement learning controller network are updated through supervised learning. In the reinforcement learning phase, the controller is continuously trained using the DDPG method through interaction between the deep neural network controller and the actual wind tunnel environment. The dynamic characteristics of the wind tunnel test section velocity field and the power section fans under all operating conditions are cyclically learned from the sampled data. This allows for precise control of the test section velocity field even when the wind tunnel model is inaccurate. Precise control is also possible even when the wind tunnel model changes or external disturbances occur. The method provided by the present invention overcomes the drawbacks of traditional wind tunnel control methods, which rely on precise models and expert knowledge and are cumbersome to adjust. Automatic optimization of controller parameters reduces wind tunnel debugging time and resource consumption, while also exhibiting good anti-interference performance. This method can also improve the uniformity of the wind tunnel test temperature field distribution and, to a certain extent, reduce the rectification requirements for devices such as honeycombs and damping nets.

[0066] refer to Figure 2 The second embodiment of the present invention provides a multi-fan wind tunnel velocity field control system 200 based on reinforcement learning, which includes a construction module 201, a training module 202, a reinforcement learning module 203, and an adjustment module 204. The specific functions of each module are described as follows:

[0067] A construction module 201 is used to generate theoretical working condition samples based on wind tunnel aerodynamic design parameters, construct a state-action mapping relationship by injecting velocity noise and actuator noise, and form a noisy mapping data set;

[0068] A training module 202 is configured to use the mapping data set to pre-train a deep neural network to learn a fan adjustment strategy in a noisy environment;

[0069] Reinforcement learning module 203 is used to deploy the pre-trained network to the wind tunnel environment for real-time interaction, optimize network parameters using the uniformity of the velocity field in the test section as a reward signal, and complete the optimization of the reinforcement learning controller;

[0070] The adjustment module 204 is used to use the optimized deep neural network controller to adjust the fan speed in a closed loop to achieve uniform control of the speed field in the test section.

[0071] It should be noted that the above explanation of the method embodiment is also applicable to the system of this embodiment and will not be repeated here.

[0072] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the quantity of the technical features indicated. Thus, features defined as "first" or "second" may explicitly or implicitly include at least one of these features.

[0073] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A multi-fan wind tunnel velocity field control method based on reinforcement learning, characterized in that: Include: Generate theoretical working condition samples based on wind tunnel aerodynamic design parameters, and construct a state-action mapping relationship by injecting velocity noise and actuator noise to form a noisy mapping data set. Pre-training a deep neural network using the mapping dataset to learn a fan adjustment strategy in a noisy environment; Deploy the pre-trained network to a wind tunnel environment for real-time interaction, use the velocity field uniformity of the test section as a reward signal to optimize network parameters and complete the optimization of the reinforcement learning controller. The optimized deep neural network controller is used to adjust the fan speed in a closed loop to achieve uniform control of the velocity field in the test section.

2. The multi-fan wind tunnel velocity field control method based on reinforcement learning according to claim 1, characterized in that: The construction of the noisy mapping dataset includes: Generate theoretical fan speed distribution and corresponding theoretical test section velocity field for target operating conditions; Add velocity noise simulation value to the velocity field of theoretical test section; Adding actuator noise simulation values ​​to the theoretical fan speed distribution; Generate a data set containing the noise velocity field, disturbed speed, and compensation adjustment.

3. The multi-fan wind tunnel velocity field control method based on reinforcement learning according to claim 1 or 2, characterized in that: Velocity noise is the random offset of wind speed at the test section measurement point, and actuator noise is the random offset of fan speed.

4. The multi-fan wind tunnel velocity field control method based on reinforcement learning according to claim 1, characterized in that: The input layer dimension of the deep neural network is the sum of the number of fans, the number of measurement points, the target wind speed and the total pressure parameter, and the output layer dimension is equal to the number of fans.

5. The multi-fan wind tunnel velocity field control method based on reinforcement learning according to claim 1, characterized in that: The reward signal is positively correlated with the uniformity of the velocity field in the test segment, and the reward value is saturated when the sum of the absolute deviations of the velocity distribution is lower than a threshold.

6. The multi-fan wind tunnel velocity field control method based on reinforcement learning according to claim 1 or 5, characterized in that: The reward function is designed as the ratio of the number of measurement points in the test segment velocity field to the sum of the absolute errors in the test segment velocity field.

7. The multi-fan wind tunnel velocity field control method based on reinforcement learning according to claim 1, characterized in that: The optimization phase of the reinforcement learning controller uses a deep deterministic policy gradient algorithm, and the network update includes an experience replay mechanism.

8. The multi-fan wind tunnel velocity field control method based on reinforcement learning according to claim 1, characterized in that: The construction of the environmental state in the optimization phase of the reinforcement learning controller depends on the real-time measured total pressure parameter, which together with the fan speed, the test section velocity field, and the target wind speed forms a four-dimensional state vector.

9. The multi-fan wind tunnel velocity field control method based on reinforcement learning according to claim 1, characterized in that: The closed-loop fan speed regulation includes an iterative process: Real-time collection of fan speed, total pressure and test section velocity field distribution; Dynamically output speed adjustment instructions; Verify uniformity specifications and cycle until achieved.

10. A multi-fan wind tunnel velocity field control system based on reinforcement learning, characterized in that: Include: A construction module is used to generate theoretical working condition samples based on wind tunnel aerodynamic design parameters, construct a state-action mapping relationship by injecting velocity noise and actuator noise, and form a noisy mapping data set; A training module, configured to use the mapping data set to pre-train a deep neural network to learn a fan adjustment strategy in a noisy environment; The reinforcement learning module deploys the pre-trained network into the wind tunnel environment for real-time interaction, uses the uniformity of the velocity field in the test section as a reward signal to optimize network parameters and complete the optimization of the reinforcement learning controller. The regulation module is used to adjust the fan speed in a closed loop using an optimized deep neural network controller to achieve uniform control of the velocity field in the test section.

Images