Bionic undulating fin resistance reduction device and method based on reinforcement learning
By designing a bionic wave fin drag reduction device based on reinforcement learning, and using the reinforcement learning algorithm DDPG to achieve intelligent drag reduction control during the movement of the wave fin, the problem of lack of active control algorithms and flow field measurement in the prior art is solved, and the adaptability and robustness of the drag reduction device are improved.
Patent Information
- Application Number
- CN202411969612.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-04-18
AI Technical Summary
The existing technology lacks the active control algorithm of bionic fluctuating fins during movement, and cannot achieve intelligent drag reduction, and cannot measure the flow field generated at the tail carefully, resulting in the feasibility of actively controlling drag reduction from the perspective of flow field evolution mechanism.
A bionic wave fin drag reduction device based on reinforcement learning is designed, including a transparent water tank, a bionic wave fin, a servo, a patch resistance sensor, a micro bubble cell, a high-frequency dual-pulse laser and a high-speed camera. Through the reinforcement learning algorithm DDPG, intelligent drag reduction control during the movement of the fluctuating fins is achieved and flow field characteristics are measured.
The intelligent drag reduction control of bionic fluctuation fins in different environments is realized, which improves the adaptability and robustness of the drag reduction device, can make real-time feedback adjustments according to environmental changes, and enhances the adaptability to complex and variable marine environments.
Smart Images
Figure CN119840798A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of bionic undulating fin drag reduction experimental technology measurement, and in particular relates to a bionic undulating fin drag reduction device and method based on reinforcement learning. Background Art
[0002] Underwater propulsion technology is a technology that studies the theory and methods of converting energy power into propulsion for surface and underwater navigation bodies. It is also an important core technology for surface and underwater marine equipment. In the military field, the performance of underwater propulsion directly determines the technical indicators of naval equipment such as rapidity, endurance, and stealth. It is urgent to break through underwater propulsion technology with low noise, high critical speed and strong maneuverability to support the modernization and upgrading of my country's naval equipment. However, the low cavitation and low noise performance of conventional propulsion has reached a bottleneck, and it is urgent to break through disruptive technological breakthroughs. The bionic wave fin has attracted widespread attention due to its comprehensive performance such as high efficiency, low noise, and low vibration, as well as the characteristics of high maneuverability, high intensity, and high precision operation tasks facing complex and changeable marine environments. Although there is a relatively in-depth study on the drag reduction of bionic wave fins in theory, it is mostly focused on the design of the shape contour, wave equation fitting, and the selection of drag reduction materials on the surface of the wave fin. There is a lack of an active control algorithm during the swimming process of the wave fin to achieve intelligent drag reduction for the wave fin. At the same time, there is no device to measure the fine flow field generated at the tail of the bionic undulating fin during swimming under active control conditions, which makes it impossible to explore the feasibility of active control of drag reduction from the perspective of flow field evolution mechanism.
[0003] In summary, the current experiments on the active drag reduction flow field measurement of bionic undulating fins are not perfect, and the measurement scheme is not mature. Therefore, forming a complete test system and a bionic undulating fin drag reduction method based on reinforcement learning is of great significance for the research on bionic undulating fin drag reduction. Summary of the invention
[0004] The purpose of the present invention is to provide a bionic undulating fin drag reduction device and method based on reinforcement learning, to achieve intelligent drag reduction during the movement of the undulating fin, and to provide technical support for exploring the interaction mechanism between the evolution law of the undulating fin flow field morphology and the undulating drag reduction from the perspective of experimental mechanism of the bionic undulating fin.
[0005] The purpose of the present invention is achieved through the following technical solutions:
[0006] A bionic undulating fin drag reduction device based on reinforcement learning comprises: a transparent water tank, in which a bionic undulating fin is placed, the bionic undulating fin is externally connected to a steering gear and a steering gear swing rod for controlling the bionic undulating fin to swing; a patch resistance sensor is installed on the bionic undulating fin for measuring the resistance encountered by the bionic undulating fin during movement; micro bubble holes are provided on both sides of the bionic undulating fin, the micro bubble holes are electrically connected to a motor control box, and the motor control box controls the ventilation or suction of the micro bubble holes; a high-frequency double-pulse laser, a camera synchronizer, and a reflective prism are placed under the transparent water tank, the laser emitted by the high-frequency double-pulse laser is reflected to the flow field area behind the bionic undulating fin via the reflective prism; the camera synchronizer is used to control the synchronous operation of the high-frequency double-pulse laser and the high-speed camera; the motor control box and the camera synchronizer are respectively connected to an active control system.
[0007] Furthermore, the material of the bionic wave fin is silicone with a density similar to that of water, high ductility, smooth surface and good hydrophobicity.
[0008] Furthermore, the high-frequency pulse laser emits a laser wavelength of 532 nm, and the thickness of the pulse laser sheet of the high-frequency pulse laser is 0.6 mm.
[0009] Furthermore, an air pump, a motor, a resistance data acquisition module and a single-chip control module are placed inside the motor control box, which are used to control the movement of the undulating fins, collect resistance data and external micro bubble holes for air suction or air ejection.
[0010] Furthermore, the high-speed camera is a 12-bit precision CCD camera, and the camera resolution is 2048 pixels×2048 pixels.
[0011] The present invention may also include: a drag reduction method of the bionic undulating fin drag reduction device based on reinforcement learning as described above, the method comprising the steps of:
[0012] During the experiment, a high-frequency double-pulse laser and a high-speed camera were turned on. The wavelength of the high-frequency double-pulse laser was 532nm, and the thickness of the pulse laser sheet was 0.6mm. The laser was irradiated through a reflecting prism to the flow field to be measured at the rear of the bionic undulating fin. At the same time, an electrical signal was sent out by the motor control box to drive the bionic undulating fin connected to the servo and the servo rocker to swing. After the movement of the bionic undulating fin stabilized, the patch resistance sensor fed back the resistance value on the undulating fin to the single-chip microcomputer module of the motor control box, and finally transmitted it to the active control processing system. The micro-bubble hole blowing and suction speed was fed back through the reinforcement learning algorithm in the active control processing system. In addition, the high-speed camera recorded the flow field characteristics of the micro-bubble hole blowing and suction.
[0013] Furthermore, the active control processing system is composed of a reinforcement learning algorithm DDPG, and the blowing and suction speed of the micro bubble hole is controlled by the resistance value fed back by the motor control box. The internal algorithm is that after obtaining the action a provided by the real strategy network, that is, the resistance value fed back by the motor control box, the active control system will return the state s' and reward value r at the next moment, where the state s is the resistance value f, and r is a function of the resistance value, as shown in the following formula:
[0014] r=α·f+b (1)
[0015] Its main purpose is to reduce r when the resistance value f increases;
[0016] The reward value obtained by performing action a at the current state s and the state s' at the next moment are combined into an experience tuple {s, a, r, s'}, and put it into the experience replay pool. After it is used up, N experience tuples are randomly sampled from the experience replay pool; then, s and a are sampled from the experience replay pool, and they are input into the real Q network to obtain the real Q value; then, s' is sampled from the experience replay pool and input into the target strategy network to obtain the action a' at the next moment, that is, to control the blowing and suction speed of the micro bubble hole; and it is input into the target Q network together with s' and r sampled from the experience replay pool to obtain the target Q value; the goal of the intelligent algorithm is to make the target Q value as close to 0 as possible, that is, to reduce the resistance value as much as possible; at the same time, the optimizer is used to update the real strategy network;
[0017] Gradient update uses a deterministic policy gradient method to implement a step-by-step intelligent agent decision-making strategy by minimizing the loss function of the value network and maximizing the Q value of the policy network for the output action to achieve a better action selection; that is, determine which micro-bubbles to blow or inhale, and determine the blowing and inhaling speed; then, use the soft update method to replace the parameters of the two real networks with the other two target networks;
[0018] In the next step, the updated real strategy network outputs a and adds random noise to improve the robustness of the system. Finally, action a is output to the active control system to form a loop process, and the above steps are repeated until the predetermined performance requirements are met.
[0019] Furthermore, the experience replay is to solve the sample correlation, break the time correlation, and make the reinforcement learning process training more stable; at the same time, it improves the utilization rate of the sampled data.
[0020] The beneficial effects of the present invention are:
[0021] The design of the bionic undulating fin drag reduction device and method of the present invention is different from that of traditional drag reduction devices. During the design process, the mutual coupling mechanism between the flow field and the resistance during the movement of the undulating fin is fully considered. From the perspective of experimental mechanism, the mutual influence between the evolution law of the undulating fin flow field morphology and the undulating drag reduction is better explored, providing technical support for further drag reduction research on bionic undulating fins.
[0022] The present invention is based on the DDPG reinforcement learning algorithm, establishes the bionic undulating fin drag reduction loss function, and improves the scoring system of the blowing and sucking actions of the micro-bubble holes, thereby further enhancing the adaptability of the bionic undulating fin drag reduction device in different environments, enabling it to perform real-time feedback adjustment according to environmental changes, increasing the adaptability and robustness of the drag reduction device, and having high-maneuverability, high-intensity, and high-precision operating task characteristics for complex and changeable marine environments. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Attached Figure 1 It is a structural schematic diagram of the present invention.
[0024] Attached Figure 2 It is a partial schematic diagram of the bionic undulating fin of the present invention.
[0025] Attached Figure 3 Schematic diagram of the patch resistance sensor of the present invention.
[0026] Attached Figure 4 It is a flow chart of the active control algorithm of bionic undulating fin drag reduction of the present invention.
[0027] Attached Figure 5 It is a diagram of the reinforcement learning training process of the bionic undulating fin of the present invention.
[0028] In the attached figure: 1. Motor control box, 2. Active control system, 3. Camera synchronizer, 4. High-frequency double-pulse laser, 5. Reflective prism, 6. High-speed camera, 7. Transparent water tank, 8. Bionic wave fin, 9. Micro bubble hole, 10. Patch resistance sensor. DETAILED DESCRIPTION
[0029] The present invention is further described below in conjunction with the accompanying drawings.
[0030] According to an embodiment of the present invention, a bionic undulating fin drag reduction device based on reinforcement learning is provided, as shown in the attached Figure 1 As shown, it includes: a motor control box 1, an active control system 2, a camera synchronizer 3, a high-frequency double-pulse laser 4, a reflective prism 5, a high-speed camera 6, a transparent water tank 7, a bionic wave fin 8, a micro bubble hole 9, and a patch resistance sensor 10.
[0031] The high-frequency double-pulse laser 4 is placed under the transparent water tank 7. The high-frequency pulse laser 4 emits a laser wavelength of 532nm, and the thickness of the pulse laser sheet is 0.6mm. The laser is reflected to the flow field area behind the bionic wave fin 8 through the reflective prism 5 under the transparent water tank 7.
[0032] As attached Figure 2 As shown, the bionic undulating fin 8 is composed of silica gel with a density similar to that of water, high ductility, smooth surface and good hydrophobicity, and is externally connected to a steering gear and a steering gear swing rod to control the swing of the bionic undulating fin 8; the patch resistance sensor 10 is placed on the bionic undulating fin 8 to measure the resistance encountered by the bionic undulating fin 8 during movement. At the same time, the patch resistance sensor 10 is externally connected to the motor control box 1 for resistance value feedback; the micro bubble holes 9 are distributed on both sides of the bionic undulating fin 8, and the micro bubble holes are externally connected to the motor control box 1, and the motor control box 1 controls the micro bubble holes 9 to ventilate or inhale.
[0033] The high-speed camera 6 is composed of a Flow Sense 4M Mk II 12-bit precision CCD camera from Dandy, and the camera resolution is 2048 pixels × 2048 pixels;
[0034] The motor control box 1 is internally provided with an air pump, a motor, a resistance data acquisition module and a single-chip control module, which are used to control the movement of the wave fin, collect resistance data and the external micro bubble holes 9 to inhale or eject air, and is externally connected to an active control system 2;
[0035] The camera synchronizer 3 is placed under the transparent water tank 7 and is used to control the high-frequency double-pulse laser 4 and the high-speed camera 6 to run synchronously.
[0036] The principle or method of use of the present invention is as follows: When conducting the test, turn on the high-frequency double-pulse laser 4 and the high-speed camera 6. The high-frequency double-pulse laser 4 has a laser wavelength of 532nm and a pulse laser sheet thickness of 0.6mm. The laser is irradiated to the flow field to be measured at the rear of the bionic wave fin through the reflecting prism 5. At the same time, the motor control box 5 sends an electrical signal to drive the bionic wave fin connected to the servo and the servo rocker to swing. After the bionic wave fin moves steadily, the patch resistance sensor will feed back the resistance value on the wave fin to the single-chip microcomputer module of the motor control box, and finally transmit it to the active control processing system 2. Through the reinforcement learning algorithm in the self-developed active control processing system 2, the blowing and suction speed of the micro bubble hole 9 is fed back. In addition, the high-speed camera 6 records the flow field characteristics of the micro bubble hole 9 for blowing and suction.
[0037] The present invention fully considers the mutual coupling mechanism between the flow field and the resistance during the movement of the undulating fin. From the perspective of experimental mechanism, it better explores the mutual influence between the evolution law of the flow field morphology of the undulating fin and the undulating drag reduction, and provides technical support for further drag reduction research of bionic undulating fins.
[0038] The active control processing system 2 described in this embodiment is composed of a reinforcement learning algorithm DDPG, which controls the blowing and suction speed of the micro bubble holes through the resistance value fed back by the motor control box 1. The main components of its internal algorithm are shown in the attached figure. Figure 4 After obtaining the action a provided by the real strategy network, that is, the resistance value fed back by the motor control box 1, the active control system 2 will return the state s' and reward value r at the next moment, where the state s is the resistance value f, and r is a function of the resistance value, as shown in the following formula:
[0039] r=α·f+b (1)
[0040] Its main purpose is to reduce r when the resistance value f increases. The reward value obtained by implementing action a at the current state s and the state s' at the next moment are combined into an experience tuple {s, a, r, s'}, and put it into the experience replay pool. After it is used up, N experience tuples are randomly sampled from the experience replay pool. The main function of experience replay is to solve the sample correlation, break the time correlation, and make the reinforcement learning process training more stable. At the same time, improve the utilization rate of sampled data. Subsequently, s and a are sampled from the experience replay pool and input into the real Q network to obtain the real Q value. Then, s' is sampled from the experience replay pool and input into the target strategy network to obtain the action a' at the next moment, that is, to control the blowing and suction speed of the micro bubble hole. And it is input into the target Q network together with s' and r sampled from the experience replay pool to obtain the target Q value. The goal of the intelligent algorithm is to make the target Q value as close to 0 as possible, that is, to reduce the resistance value as much as possible. At the same time, the optimizer is used to update the real strategy network. Gradient update uses a deterministic policy gradient method to implement a step-by-step intelligent agent decision-making strategy by minimizing the loss function of the value network and maximizing the Q value of the policy network for the output action to achieve a better action selection. That is, determine which microbubbles to blow or inhale, and determine the blowing and inhaling speed. Then, use the soft update method to replace the parameters of the other two target networks with the parameters of the two real networks. Next, use the updated real policy network to output a, and add random noise to improve the robustness of the system. Finally, action a is output to the active control system 2, forming a loop process, and repeating the above steps until the predetermined performance requirements are met.
[0041] The algorithm flow pseudo code is shown in Table 1:
[0042] Table 1 Algorithm flow pseudo code
[0043]
[0044]
[0045] The present invention is based on the DDPG reinforcement learning algorithm, establishes the bionic undulating fin drag reduction loss function, and improves the scoring system of the blowing and sucking actions of the micro-bubble holes, thereby further enhancing the adaptability of the bionic undulating fin drag reduction device in different environments, enabling it to perform real-time feedback adjustment according to environmental changes, increasing the adaptability and robustness of the drag reduction device, and having high-maneuverability, high-intensity, and high-precision operating task characteristics for complex and changeable marine environments.
[0046] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A bionic undulating fin drag reduction device based on reinforcement learning, characterized in that: include: A transparent water tank (7), wherein a bionic undulating fin (8) is placed in the transparent water tank (7), wherein the bionic undulating fin (8) is externally connected to a steering gear and a steering gear swing rod, and is used to control the bionic undulating fin to oscillate; a patch resistance sensor (10) is installed on the bionic undulating fin (8), and is used to measure the resistance encountered by the bionic undulating fin during movement; and micro bubble holes (9) are provided on both sides of the bionic undulating fin (8), wherein the micro bubble holes (9) are electrically connected to a motor control box (1), and the motor control box (1) controls the micro bubble holes (9) to move. ) ventilation or inhalation; a high-frequency double-pulse laser (4), a camera synchronizer (3), and a reflective prism (5) are placed below the transparent water tank (7); the laser emitted by the high-frequency double-pulse laser (4) is reflected to the flow field area behind the bionic wave fin (8) via the reflective prism (5); the camera synchronizer (3) is used to control the synchronous operation of the high-frequency double-pulse laser (4) and the high-speed camera (6); the motor control box (1) and the camera synchronizer (3) are respectively connected to the active control system (2).
2. The bionic undulating fin drag reduction device based on reinforcement learning according to claim 1 is characterized in that: The material of the bionic undulating fin (8) is silica gel having a density similar to that of water, high ductility, a smooth surface and good hydrophobicity.
3. The bionic undulating fin drag reduction device based on reinforcement learning according to claim 1 is characterized in that: The high-frequency pulse laser (4) emits a laser with a wavelength of 532 nm, and the thickness of the pulse laser sheet of the high-frequency pulse laser (4) is 0.6 mm.
4. The bionic undulating fin drag reduction device based on reinforcement learning according to claim 1 is characterized in that: An air pump, a motor, a resistance data acquisition module and a single-chip control module are placed inside the motor control box (1), which are used to control the movement of the wave fins, collect resistance data and external micro bubble holes for air suction or air ejection.
5. The bionic undulating fin drag reduction device based on reinforcement learning according to claim 1 is characterized in that: The high-speed camera is a 12-bit precision CCD camera with a camera resolution of 2048 pixels×2048 pixels.
6. A drag reduction method for a bionic undulating fin drag reduction device based on reinforcement learning as claimed in any one of claims 1 to 5, characterized in that: The method comprises the steps of: When conducting the test, a high-frequency double-pulse laser (4) and a high-speed camera (6) are turned on. The high-frequency double-pulse laser (4) has a laser wavelength of 532nm and a pulse laser sheet thickness of 0.6mm. The laser is irradiated to the flow field to be measured at the rear of the bionic wave fin through a reflecting prism (5). At the same time, an electric signal is sent from a motor control box (5) to drive the bionic wave fin connected to the steering gear and the steering gear swing rod to swing. After the bionic wave fin moves stably, the patch resistance sensor feeds back the resistance value on the wave fin to the single-chip microcomputer module of the motor control box, and finally transmits it to the active control processing system (2). The blowing and suction speed of the micro bubble hole (9) is fed back through the reinforcement learning algorithm in the active control processing system (2). In addition, the high-speed camera (6) records the flow field characteristics of the micro bubble hole (9) for blowing and suction.
7. The drag reduction method of the bionic undulating fin drag reduction device based on reinforcement learning according to claim 6, characterized in that: The active control processing system (2) is composed of a reinforcement learning algorithm DDPG, and controls the blowing and suction speed of the micro bubble hole through the resistance value fed back by the motor control box (1). Its internal algorithm is that after obtaining the action a provided by the real strategy network, that is, the resistance value fed back by the motor control box (1), the active control system (2) will return the state s and reward value r at the next moment, where the state s is the resistance value f, and r is a function of the resistance value, as shown in the following formula: r=α·f+b (1) Its main purpose is to reduce r when the resistance value f increases; The reward value obtained by performing action a at the current state s and the state s at the next moment are combined into an experience tuple {s, a, r, s , }, and put it into the experience replay pool. After it is used up, N experience tuples are randomly sampled from the experience replay pool. Then, s and a are sampled from the experience replay pool and input into the real Q network to get the real Q value. Then, s, are sampled from the experience replay pool and input into the target strategy network to get the next action a. , , that is, control the blowing and suction speed of the micro bubble hole; and input it together with s and r obtained by sampling in the experience playback pool into the target Q network to obtain the target Q value; the goal of the intelligent algorithm is to make the target Q value as close to 0 as possible, that is, to reduce the resistance value as much as possible; at the same time, the optimizer is used to update the real strategy network; Gradient update uses a deterministic policy gradient method to implement a step-by-step intelligent agent decision-making strategy by minimizing the loss function of the value network and maximizing the Q value of the policy network for the output action to achieve a better action selection; that is, determine which micro-bubbles to blow or inhale, and determine the blowing and inhaling speed; then, use the soft update method to replace the parameters of the two real networks with the other two target networks; Next, the updated real strategy network outputs a and adds random noise to improve the robustness of the system. Finally, action a is output to the active control system (2), forming a loop process and repeating the above steps until the predetermined performance requirements are met.
8. The drag reduction method of the bionic undulating fin drag reduction device based on reinforcement learning according to claim 7, characterized in that: The experience replay is to solve the sample correlation, break the time correlation, make the reinforcement learning process training more stable; at the same time, improve the utilization rate of the sampled data.