Intelligent hydrofoil noise reduction device and method based on reinforcement learning
By using a reinforcement learning-based intelligent hydrofoil device, the shape and installation position of the propeller are automatically optimized, solving the problems of high cost and low efficiency in traditional methods and achieving a highly efficient noise reduction effect for the propeller.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-12
- Publication Date
- 2026-03-13
AI Technical Summary
Existing technologies are costly and inefficient in reducing propeller noise. Traditional methods require manual intervention and are difficult to optimize multivariate problems efficiently.
By employing a reinforcement learning-based intelligent hydrofoil device, the design space is automatically explored through the adjustment of the shape parameters and optimization of the installation position of the intelligent hydrofoil, combined with Q-learning and ε-greedy strategies, reducing physical experiments and achieving the best noise reduction effect for the propeller.
Significantly reduces propeller noise, reduces reliance on physical experiments, improves optimization efficiency, reduces costs, and achieves the best noise reduction effect for propellers.
Smart Images
Figure CN121650848A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of propeller noise reduction technology, specifically relating to an intelligent hydrofoil noise reduction device and method based on reinforcement learning. Background Technology
[0002] Propeller noise, as one of the main noise sources of underwater vehicles, has always been a key concern for countries worldwide. Improving propulsive efficiency can be achieved by reducing hub vortices, while flow noise, caused by pressure pulsations in the flow field, also affects noise levels. Modifying the propeller shape and adding accessories around the propeller can alter the surrounding flow field, thereby improving hydrodynamic efficiency and reducing propeller noise. However, traditional methods, such as experimental testing, computational fluid dynamics (CFD) simulations, and experience-based design rules, require manual intervention, resulting in high costs and low efficiency.
[0003] Reinforcement learning (RL), a propeller-like optimization method, learns optimal policies through the interaction between an agent and its environment. Compared to traditional methods such as experimental testing, computational fluid dynamics (CFD) simulations, and experience-based design rules, RL can automatically explore different design spaces without human intervention. For complex multivariate problems, RL can handle multiple interrelated factors and optimize various performance metrics simultaneously. Furthermore, the model can continuously evolve and improve with the accumulation of new data. Through extensive trial and error in a virtual environment, the number of physical prototypes that need to be built and tested before actual manufacturing can be significantly reduced, thereby lowering costs and accelerating the development cycle. Summary of the Invention
[0004] The purpose of this invention is to provide an intelligent hydrofoil noise reduction device and method based on reinforcement learning, which effectively weakens hub vortices, improves flow field structure, significantly reduces propeller noise, and at the same time reduces dependence on physical experiments, improves optimization efficiency and reduces costs.
[0005] The objective of this invention is achieved through the following technical solution:
[0006] A noise reduction device for an intelligent hydrofoil based on reinforcement learning includes: a control box, a propeller, an intelligent hydrofoil, a telescopic device, a hub ring, a test water tank, a pressure sensor, a data center, and an active control system;
[0007] The propeller is connected to the control box and installed in a transparent water tank. The control box is used to adjust the rotation speed and the incoming flow speed.
[0008] The intelligent hydrofoil is connected to the propeller. A telescopic device is installed inside the intelligent hydrofoil. The telescopic device can change the hydrofoil installation spacing ratio and the hydrofoil length. A propeller hub ring that can change the circumferential installation position is installed at the root of the intelligent hydrofoil.
[0009] The array of pressure sensors is respectively installed on the surface of the propeller hub ring and the smart hydrofoil to collect unsteady pressure pulsation data. The pressure sensors are connected to a data center.
[0010] Furthermore, the control box includes a motor, a water pump, and a microcontroller control module, which controls the propeller speed and the shape parameters of the intelligent hydrofoil through the speed value provided by the motor and the extension value provided by the extension device.
[0011] Furthermore, the control box is externally connected to a data center, which transmits equipment operation information and noise data to the active control system. Control commands generated in the active control system are sent to the control box through the data center.
[0012] Furthermore, the active control processing system consists of a computer and a reinforcement learning algorithm based on Q-learning combined with an ε-greedy strategy. It collects unsteady pressure pulsation data through the pressure sensor and outputs commands to control the spacing ratio and length of the intelligent hydrofoil. By continuously adjusting, it achieves the optimal vortex-cutting angle, thereby realizing intelligent noise reduction of the propeller.
[0013] Furthermore, the axial mounting position of the intelligent hydrofoil is measured by a spacing ratio, which is defined as follows:
[0014] .
[0015] Furthermore, the circumferential installation position of the intelligent hydrofoil is based on the center position of the propeller blade, with rightward being positive and leftward being negative.
[0016] Furthermore, the intelligent hydrofoil is made of aluminum alloy.
[0017] The present invention may also include:
[0018] A noise reduction method for the above-mentioned reinforcement learning-based intelligent hydrofoil noise reduction device includes:
[0019] Initialize the Q-table and obtain the current state through experimental setup information. Set the learning rate Discount Factor and in ε-greedy Value, at each time step According to the current state :
[0020] If random number Select the action with the highest Q value. Otherwise from Randomly select an action , It is a probability value between 0 and 1, which gradually decreases as training progresses, so as to rely more on the learned knowledge;
[0021] Get new state The system checks if the termination condition is met. The termination condition is either traversing the Q-table or Q-value convergence. If the condition is not met, the data is returned to the experimental setup. If the termination condition is met, the optimal vortex shearing angle parameter settings for the intelligent hydrofoil under a fixed incoming flow velocity are obtained. At this point, the noise reduction effect reaches its optimal level.
[0022] Furthermore, the action After the command is sent to the control box via the data center, the experimental device performs the experiment, acquires noise data, and transmits it back, rewarding... The value was determined based on experimentally measured propeller noise levels.
[0023]
[0024] in, The weighted composite value is obtained by averaging the propeller noise measured by the hydrophone.
[0025] Perform the selected action When the next state is obtained and instant rewards ,
[0026] Update the Q value using the Bellman equation:
[0027]
[0028] Here, the learning rate The discount factor determines the extent to which new information affects the old Q value. This reflects the importance of future rewards. ; This represents the maximum expected reward that can be obtained by taking the best action under the new state.
[0029] The beneficial effects of this invention are as follows:
[0030] This invention demonstrates that reducing propeller noise by mitigating hub vortices is an effective method for improving propeller wake. The design of this invention's reinforcement learning-based intelligent hydrofoil noise reduction device differs from traditional noise reduction devices, fully considering the influence of numerous parameters on the noise reduction effect. From an experimental mechanism perspective, it better explores the impact of different parameters on propeller noise reduction. Furthermore, this device employs a reinforcement learning algorithm, Q-learning, combined with an ε-greedy strategy to iteratively apply the parameters. Compared to traditional experimental methods that change parameters one by one to obtain the optimal combination, the reinforcement learning method can iteratively select the optimal solution, effectively reducing costs and improving efficiency.
[0031] The active control processing system of this invention consists of a reinforcement learning algorithm Q-learning combined with an ε-greedy strategy. It mainly uses noise data collected by hydrophones to control the shape parameters of the intelligent hydrofoil to achieve the optimal vortex shearing angle, the optimal axial installation position, and the hydrofoil length under the current state, thereby achieving the best noise reduction effect for the propeller. Attached Figure Description
[0032] Appendix Figure 1 This is a schematic diagram of the structure of the present invention;
[0033] Appendix Figure 2 This is a flowchart of the active control algorithm for the intelligent hydrofoil of the present invention.
[0034] In the attached diagram: 1. Control box, 2. Propeller, 3. Hydrofoil, 4. Telescopic device, 5. Propeller hub ring, 6. Test water tank, 7. Pressure sensor, 8. Data center, 9. Active control system. Detailed Implementation
[0035] The present invention will now be further described with reference to the accompanying drawings.
[0036] This invention relates to a reinforcement learning-based intelligent hydrofoil noise reduction device, as shown in the attached figure. Figure 1 As shown, it includes: control box 1, propeller 2, intelligent hydrofoil 3, telescopic device 4, propeller hub ring 5, test water tank 6, pressure sensor 7, data center 8, active control system 9;
[0037] The propeller 2 is connected to the control box 1 and installed in a transparent water tank. The control box 1 is used to adjust the rotation speed and the incoming flow speed to meet the adjustment requirements of different working conditions.
[0038] The intelligent hydrofoil 3 is made of aluminum alloy and has a telescopic device 4 installed inside, which can change the shape parameters of the hydrofoil. The intelligent hydrofoil 3 is installed at the hub of the propeller 2, and the change of its shape parameters is controlled by the control box.
[0039] The axial installation position of the intelligent hydrofoil is measured by the spacing ratio, which is defined as follows:
[0040] .
[0041] The circumferential installation position of the intelligent hydrofoil is based on the center position of the propeller blades, with rightward movement being positive and leftward movement being negative. Adjustment is performed using the hub ring 5.
[0042] The hub ring 5 is located at the root of the intelligent hydrofoil 3 and can be rotated to change the circumferential installation position of the hydrofoil by the control box 1.
[0043] The array of pressure sensors 7 is mounted on the propeller hub and hydrofoil surface to collect unsteady pressure pulsation data. Since the pressure pulsation amplitude (RMS) on the propeller surface is proportional to the flow noise power, this data can directly reflect the propeller's noise reduction effect. The sensors transmit the collected signals to the data center 8.
[0044] The control box 1 contains a motor, a water pump, and a microcontroller control module. It primarily controls the propeller speed and hydrofoil shape parameters through the motor's rotational speed and the telescopic extension / retraction value provided by the telescopic device. It is externally connected to a data center.
[0045] The main function of the data center 8 is to transmit equipment operation information and noise data to the active control system. At the same time, the control commands generated by the active control processing system in the computer are also sent to the control box 1 through the data center.
[0046] The active control processing system consists of a computer and a reinforcement learning algorithm based on the ε-greedy strategy. It mainly uses the unsteady pressure pulsation data collected by the array of pressure sensors 7 to output commands to control the shape parameters and installation position of the intelligent hydrofoil 3. By continuously adjusting, it achieves the optimal vortex reduction angle under the working condition, thereby realizing intelligent noise reduction of the propeller.
[0047] In this embodiment, as shown in the appendix Figure 2 As shown, after selecting a certain operating condition (fixed incoming flow velocity and fixed propeller speed) as the experimental conditions, the state values were obtained through the experimental setup information. and action value ,state The pitch ratio of the smart hydrofoils and hydrofoil length value Composition, Action This includes minor adjustments to parameters within the range of pitch ratio changes, circumferential position changes, and hydrofoil length changes, while maintaining consistent motion. For all states... and actions Based on the obtained data And initialize it. Set the learning rate. Discount factor and in ε-greedy Value, at each time step According to the current state If random number Select the action with the highest Q value. Otherwise from Randomly select an action here It is a probability value between 0 and 1, which typically decreases gradually as training progresses (i.e., ε-decay) to rely more on learned knowledge. The action... After the command is sent to the control box via the data center, the experimental setup performs the experiment, acquires noise data, and transmits it back. (Reward) The value was determined based on experimentally measured propeller noise levels.
[0048]
[0049] in, The weighted average value is obtained by taking the propeller noise measured by the hydrophone.
[0050] Perform the selected action When the next state is obtained and instant rewards ,
[0051] Update the Q value using the Bellman equation:
[0052]
[0053] Here, The learning rate determines the extent to which new information affects the old Q value. It is a discount factor that reflects the importance of future rewards; . This represents the maximum expected reward that can be obtained by taking the best action under the new state.
[0054] Repeat the above steps until the termination condition is met: traversing the Q-table or Q-value convergence. At this point, the optimal vortex shearing angle parameter settings for the intelligent hydrofoil under a certain fixed inflow velocity are obtained. At this point, the noise reduction effect reaches its optimal level.
[0055] This invention employs reinforcement learning control by varying the hydrofoil installation spacing ratio, circumferential installation position, and hydrofoil length. It is worth noting that, based on the above approach, obtaining optimal hydrodynamic characteristics by adding variations such as changing the number of intelligent hydrofoil blades can also be achieved through the same reinforcement learning control. Besides the reinforcement learning algorithms and strategies used above, other algorithms and strategies can also achieve the aforementioned requirements.
[0056] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A smart hydrofoil noise reduction device based on reinforcement learning, characterized in that, include: Control box (1), propeller (2), smart hydrofoil (3), telescopic device (4), propeller hub ring (5), test water tank (6), pressure sensor (7), data center (8), active control system (9); The propeller (2) is connected to the control box (1) and installed in the transparent water tank. The rotation speed and the incoming flow speed are adjusted using the control box (1). The intelligent hydrofoil (3) is connected to the propeller (2). The intelligent hydrofoil (3) is equipped with a telescopic device (4). The telescopic device (4) can change the hydrofoil installation spacing ratio and hydrofoil length. The intelligent hydrofoil (3) is equipped with a hub ring (5) at the root to change the circumferential installation position. The array of pressure sensors (7) is installed on the surfaces of the rotor hub ring (5) and the smart hydrofoil (3) respectively, and is used to collect unsteady pressure pulsation data. The pressure sensors (7) are connected to the data center (8).
2. The intelligent hydrofoil noise reduction device based on reinforcement learning according to claim 1, characterized in that, The control box (1) includes a motor, a water pump and a microcontroller control module. The rotation speed of the propeller (2) and the shape parameters of the smart hydrofoil (3) are controlled by the rotation speed value provided by the motor and the extension value provided by the extension device (4).
3. The intelligent hydrofoil noise reduction device based on reinforcement learning according to claim 2, characterized in that, The control box is externally connected to a data center (8). The data center (8) transmits equipment operation information and noise data to the active control system (9). The control commands generated in the active control system (9) are sent to the control box (1) through the data center (8).
4. The intelligent hydrofoil noise reduction device based on reinforcement learning according to claim 3, characterized in that, The active control processing system (9) consists of a computer and a reinforcement learning algorithm based on Q-learning combined with an ε-greedy strategy. It collects unsteady pressure pulsation data through the pressure sensor (7) and outputs commands to control the spacing ratio and length of the intelligent hydrofoil (3). By continuously adjusting, it achieves the optimal vortex-cutting angle, thereby realizing intelligent noise reduction of the propeller.
5. The intelligent hydrofoil noise reduction device based on reinforcement learning according to claim 4, characterized in that, The axial installation position of the intelligent hydrofoil (3) is measured by the spacing ratio, which is defined as: 。 6. The intelligent hydrofoil noise reduction device based on reinforcement learning according to claim 5, characterized in that, The circumferential installation position of the intelligent hydrofoil (3) is based on the middle position of the propeller (2) blade, with rightward being positive and leftward being negative.
7. The intelligent hydrofoil noise reduction device based on reinforcement learning according to claim 6, characterized in that, The intelligent hydrofoil (3) is made of aluminum alloy.
8. A noise reduction method for an intelligent hydrofoil noise reduction device based on reinforcement learning as described in any one of claims 1-7, characterized in that, include: Initialize the Q-table and obtain the current state through experimental setup information. Set the learning rate Discount Factor and in ε-greedy Value at each time step According to the current state : If random number Select the action with the highest Q value. Otherwise from Randomly select an action , It is a probability value between 0 and 1, which gradually decreases as training progresses, so as to rely more on the learned knowledge; Get new state The system checks if the termination condition is met. The termination condition is either traversing the Q-table or Q-value convergence. If the condition is not met, the data is returned to the experimental setup. If the termination condition is met, the optimal vortex shearing angle parameter settings for the intelligent hydrofoil under a fixed incoming flow velocity are obtained. At this point, the noise reduction effect reaches its optimal level.
9. The noise reduction method for the intelligent hydrofoil noise reduction device based on reinforcement learning according to claim 8, characterized in that, Actions After the instruction is sent to the control box (1) via the data center (8), the experimental device performs the experiment to obtain noise data and transmits it back, rewarding... The value was determined based on experimentally measured propeller noise levels. in, The weighted composite value is obtained by averaging the propeller noise measured by the hydrophone. Perform the selected action When the next state is obtained and instant rewards , Update the Q value using the Bellman equation: Here, the learning rate The discount factor determines the extent to which new information affects the old Q value. This reflects the importance of future rewards. ; It represents the maximum expected reward that can be obtained by taking the best action in a new state.