Jet device control method and system for controlling wave surface wake of underwater vehicle based on synthetic jet

Abstract

The invention relates to the field of ocean engineering and flow control, in particular to a jet device control method and system for controlling wave-surface wake of an underwater vehicle based on synthetic jet. Wave wake state information obtained by processing a wave picture shot by a camera is mainly used for controlling the hole diameter of a water spraying hole, the water spraying flow speed, the water spraying frequency and the jet flow direction, so that the optimal inhibition effect on the wave wake is achieved. According to the method, the wave surface wake caused when the aircraft executes an underwater task can be effectively eliminated, and the concealment and the safety of the aircraft are improved. And meanwhile, a technical reference is provided for the concealment optimization design in the navigation of the vehicle. According to the scheme, through fusion of dynamic wave measurement, synthetic jet excitation and a nonlinear control algorithm, full-period suppression of the wakes of the induced wave surface of the underwater vehicle is achieved, and the method has more advantages compared with a traditional wave absorbing mode.

Description

Technical Field

[0001] This invention relates to the field of marine engineering and flow control, and in particular to a jet control method and system for controlling the wake of an underwater vehicle based on synthetic jet control. Background Technology

[0002] When an underwater vehicle moves underwater, its hull displaces water, affecting the free surface. This disturbance is particularly pronounced when it approaches the surface. Simultaneously, the high-speed rotation of the propeller generates a strong wake and a localized low-pressure area behind it, further exacerbating surface disturbance and generating wave wakes that propagate in all directions. This significantly impacts the vehicle's stealth, maneuverability, and safety. Traditional wave-damping methods, such as optimizing the hull shape, suffer from insufficient adaptability due to their fixed geometry. Their wave-damping efficiency is limited by structural design, unable to dynamically adapt to wave characteristics at different speeds or model scales, and may sacrifice other critical performance characteristics of the vehicle.

[0003] In existing technologies, synthetic jets have been widely used in flow control due to their zero-mass jet characteristics. By adjusting jet parameters such as the jet outlet aperture and velocity, local flow field disturbances under different conditions can be achieved. However, such devices are mostly designed for a single direction or a fixed frequency, lacking adaptability to complex operating conditions.

[0004] Real-time wavefront wake detection technology, as a crucial prerequisite for wavefront wake control, requires high-frequency sampling to extract spectral features. However, traditional wavefront wake monitoring suffers from low spatiotemporal resolution or poor anti-interference capabilities, making it difficult to accurately reflect wave phase and amplitude information. This places higher demands on existing detection systems.

[0005] In summary, existing wavefront wake control methods still face significant bottlenecks, including insufficient efficiency and adaptability of passive wave-damping devices and limited spatiotemporal resolution of real-time wavefield detection. This patent addresses these issues by proposing a novel solution involving real-time wavefield reconstruction, adaptive jet control, and distributed operation. This solution also holds significant importance for research on the stealth optimization of underwater vehicles. Summary of the Invention

[0006] The purpose of this invention is to address the wave wake caused by underwater vehicles. Traditional passive wave-damping devices suffer from poor dynamic adaptability, limited coverage, and insufficient high-frequency response. Existing active flow control technologies (such as fixed-frequency synthetic jets) lack real-time wavefield feedback and collaborative control mechanisms, making it difficult to achieve full-domain dynamic suppression of wave wakes. The purpose of this invention is to provide a device and method for controlling the wave wake of underwater vehicles through synthetic jets, thereby significantly improving wave wake suppression efficiency, shortening control response time, and providing a new reference for active flow control technology for underwater vehicles.

[0007] This invention proposes a jet ejector control method and system based on synthetic jet control of the wavefront wake of an underwater vehicle, the core of which includes the following technical solutions:

[0008] A jet control method based on synthetic jet control of the wavefront wake of an underwater vehicle:

[0009] Multiple jets are arranged at designated locations on the hull of the underwater vehicle, forming a jet array. When the vehicle generates a wake while navigating underwater, the following steps are performed simultaneously for each jet:

[0010] Step 1: Set the maximum number of iterations and the number of detection points; initialize the number of iterations and the experience replay pool.

[0011] Step 2: Obtain the wave wake image and still water surface image at the current moment, and perform image processing on the wave wake image at the current moment; based on the processed wave wake image at the current moment, obtain the actual state of the jet ejector at the current moment and the expected state of jet ejector control at the previous moment; calculate the reward value at the current moment based on the actual state of the jet ejector at the current moment.

[0012] Step 3: Input the current actual state of the ejector, the current reward value, and the expected state of the ejector control at the previous moment into the reinforcement learning network to obtain the current actual action value, the expected action value of the ejector control at the previous moment, the expected state of the ejector control at the next moment, the actual target value, and the expected target value.

[0013] Determine if the current iteration count has reached the maximum iteration count. If not, update the iteration count and set the current ejector actual state s. t The relevant state parameters and their corresponding values ​​are stored in the experience replay pool, and the reinforcement learning network parameters are updated before returning to step 2.

[0014] If so, then the current actual state s of the ejector will be... t The corresponding state parameters are stored in the experience replay pool, and after updating the reinforcement learning network parameters, the current actual action value 'a' is... t Add random noise and output it, causing the ejector to move according to the current actual action value a. t Perform the action to end this process.

[0015] Furthermore, the image processing in step 2 includes grayscale processing, Gaussian filtering, image correction, and generating a high-precision disparity map in stereo vision.

[0016] The actual state of the ejector at the current moment s t =[S,h1,h2,h3,...h i ], where S is the similarity between the wave wake image and the still water surface image at time t, hi Let be the wave height of the i-th detection point.

[0017] Furthermore, the method for calculating the reward value at the current moment in step 2 includes:

[0018] r t =w1·e mhmax +w2·S-λ·g(d·v·f·b)

[0019] Where w1, w2, and λ are weighting coefficients, m and e are constants with m being negative, and h max denoted as the maximum wave height among all detection points; g(d·v·f·b) is the energy function, where d is the nozzle diameter, v is the spray velocity, f is the spray frequency, and b is the spray direction.

[0020] Furthermore, the reinforcement learning network described in step 3 includes a display policy network, a target policy network, a real value network, and a target value network.

[0021] Furthermore, step 3 involves obtaining the current actual action value a. t The expected action value a′ of the ejector control at the previous moment t-1 The specific method is as follows: The current actual state s of the ejector t and the reward value r at the current moment t Input the display policy network to obtain the current actual action value 'a'. t and the desired state s of the ejector control at the next moment t ′;The desired state s′ of the ejector control at the previous moment. t-1 Input the target policy network to obtain the desired action value a′ of the ejector control at the previous time step. t-1 .

[0022] Furthermore, the method for updating the reinforcement learning network in step 3 is specifically as follows:

[0023] Step 3.1: Set the current actual action value a t Input the real-value network to obtain the actual target value Q. θ ; the desired action value a′ of the ejector control at the previous moment t Input the target value network to obtain the expected target value Q′ at the previous time step. t-1 .

[0024] Step 3.2: Based on the actual target value Q θ And the expected target value Q′ at the previous moment t-1 Calculate the optimal objective value Q target According to the optimal objective value Q target The parameters of the reinforcement learning network are updated by combining the experience replay pool.

[0025] Q target =r t +γ·Q′ t-1

[0026] Where γ is the discount factor.

[0027] A computer device / apparatus / system includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above-described method.

[0028] A computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0029] A computer program product includes a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0030] The beneficial effects of this invention are as follows:

[0031] Compared with the prior art, the design of the underwater vehicle wake suppression device and method of the present invention is different from the traditional wake suppression device. In the design process, it starts from the perspective of active control, and through real-time monitoring of the vehicle's wake and the combination of synthetic jet array and adaptive control algorithm, it can suppress the wake generated by different vehicles under different navigation conditions, and provide technical reference for the stealth optimization of vehicles during navigation. Attached Figure Description

[0032] Figure 1 This is an overall schematic diagram of an embodiment of the present invention.

[0033] Figure 2 This is a front view of the composite jet surrounding the underwater vehicle according to an embodiment of the present invention.

[0034] Figure 3 This is a top view of the composite jet surrounding the underwater vehicle according to an embodiment of the present invention.

[0035] Figure 4 This is an isometric view of the composite jet surrounding the underwater vehicle according to an embodiment of the present invention.

[0036] Figure 5 This is a flowchart of the algorithm structure of the present invention.

[0037] In the diagram, there are 1 test pool, 2 underwater vehicle, 3 laser, 4 high-speed camera, 5 active control and processing system, and 6 synthetic jet array. Detailed Implementation

[0038] A jet control method based on synthetic jet control of the wavefront wake of an underwater vehicle:

[0039] Multiple jets are arranged at designated locations on the hull of the underwater vehicle, forming a jet array. When the vehicle generates a wake while navigating underwater, the following steps are performed simultaneously for each jet:

[0040] Step 1: Set the maximum number of iterations and the number of detection points; initialize the number of iterations and the experience replay pool.

[0041] Step 2: Acquire the wave wake image and still water surface image at the current moment, and perform image processing on the wave wake image at the current moment; based on the processed wave wake image at the current moment, obtain the actual jet state s of the vehicle at the current moment. t The desired state s′ of the ejector control at the previous moment t-1 Based on the current actual state s of the ejector t Calculate the reward value r at the current time. t .

[0042] The image processing includes grayscale processing, Gaussian filtering, image correction, and generating a high-precision disparity map in stereo vision.

[0043] The actual state of the ejector at the current moment s t =[S,h1,h2,h3,...h i ], where S is the similarity between the wave wake image and the still water surface image at time t, h i Let be the wave height of the i-th detection point.

[0044] The reward value at the current moment satisfies r t :

[0045] r t =w1·e mhmax +w2·S-λ·g(d·v·f·b)

[0046] Where w1, w2, and λ are weighting coefficients, m and e are constants with m being negative, and h max denoted as the maximum wave height among all detection points; g(d·v·f·b) is the energy function, where d is the nozzle diameter, v is the spray velocity, f is the spray frequency, and b is the spray direction.

[0047] Step 3: Set the current actual state s of the ejector. t Current reward value r t The desired state s′ of the ejector control at the previous moment t-1 The input is given to the reinforcement learning network to obtain the current actual action value 'a'. t The desired action value a′ of the ejector control at the previous momentt-1 The desired state s′ of the ejector control at the next moment t Actual target value Q θ And the expected target value Q′.

[0048] Determine if the current iteration count has reached the maximum iteration count. If not, update the iteration count and set the current ejector actual state s. t The relevant state parameters and their corresponding values ​​are stored in the experience replay pool, and the reinforcement learning network parameters are updated before returning to step 2.

[0049] If so, then the current actual state s of the ejector will be... t The corresponding state parameters are stored in the experience replay pool, and after updating the reinforcement learning network parameters, the current actual action value 'a' is... t Add random noise and output it, causing the ejector to move according to the current actual action value a. t Perform the action to end this process.

[0050] The reinforcement learning network includes a visible policy network, a target policy network, a real value network, and a target value network.

[0051] Step 3 describes obtaining the current actual action value a. t The expected action value a′ of the ejector control at the previous moment t-1 The specific method is as follows:

[0052] The current actual state of the ejector, s. t and the reward value r at the current moment t Input the display policy network to obtain the current actual action value 'a'. t and the desired state s′ of the ejector control at the next moment t The desired state s′ of the ejector control at the previous moment. t-1 Input the target policy network to obtain the desired action value a′ of the ejector control at the previous time step. t-1 .

[0053] The method for updating the reinforcement learning network in step 3 is as follows:

[0054] Step 3.1: Set the current actual action value a t Input the real-value network to obtain the actual target value Q. θ ; the desired action value a′ of the ejector control at the previous moment t Input the target value network to obtain the expected target value Q′ at the previous time step. t-1 .

[0055] Step 3.2: Based on the actual target value Q θ And the expected target value Q′ at the previous moment t-1Calculate the optimal objective value Q target According to the optimal objective value Q target The parameters of the reinforcement learning network are updated by combining the experience replay pool.

[0056] Q target =r t +γ·Q′ t-1

[0057] Where γ is the discount factor.

[0058] Example

[0059] Combination Figures 1 to 5 The underwater vehicle 2 locks its attitude in the test pool 1 using a physical device, with an incoming current positioned in front of it.

[0060] The laser 3 is positioned directly above the spacecraft and is equipped with a scattering lens. Through the lens, the line light source can be diffused into speckle.

[0061] The high-speed camera 4 is positioned diagonally above the vehicle, at an angle of 60° to the water surface, to acquire wave wake image data in real time.

[0062] The active control processing system 5 includes a real-time wavefront wake detection system and a real-time control system.

[0063] The real-time wavefront wake detection system extracts wave height information from the real-time wavefront wake image data acquired by the high-speed camera 4 based on the semi-global matching (SGM) algorithm, and compares it with calm wavefronts using the SSIM (Structural Similarity Index Measure). The specific implementation steps are as follows:

[0064] The wavefront image captured by high-speed camera 4 is converted to grayscale, and then Gaussian filtering is applied to the grayscale image, as shown in the following formula:

[0065]

[0066] x and y are spatial coordinates, representing the coordinate offset of the current point in the convolution kernel relative to the center point;

[0067] σ is the standard deviation, which controls the width of the Gaussian distribution;

[0068] These are normalization coefficients to ensure that the sum of the convolution kernel weights is 1;

[0069] To calculate the weight decay at the distance from the center point.

[0070] The Gaussian-filtered image data is corrected using a rotation matrix R and a translation vector T to ensure horizontal alignment of the epipolar lines.

[0071] I' L =remap(I L ,R1,K rect )

[0072] I' R =remap(I R ,R2,K rect )

[0073] I L A matrix of wavefront images captured by the left camera;

[0074] I R A matrix of wavefront images captured simultaneously by the right camera;

[0075] K rect This is the corrected intrinsic parameter matrix;

[0076] R1 and R2 are rotation matrices for the left and right views;

[0077] I' L The wavefront image matrix captured by the left camera after correction;

[0078] I' R This is the wavefront image matrix captured synchronously by the right camera after correction.

[0079] The Semi-Global Matching (SGM) algorithm is used to generate a high-precision disparity map in stereo vision and calculate the optimal disparity value d(x,y) of pixel (x,y) to minimize the energy function.

[0080]

[0081] d(x,y) is the disparity value of pixel (x,y);

[0082] C AD (p,d) represents the absolute difference and cost;

[0083] λ is the weight coefficient of the data item;

[0084] P1 and P2 are smoothing penalty coefficients;

[0085] N p Let p be the neighborhood of pixel p;

[0086] I[·] is an indicator function.

[0087] Based on the principles of triangulation, the three-dimensional coordinates are calculated, and the Z-axis relative to the still water surface is obtained. ref The vertical deviation wave height h:

[0088]

[0089] h(x,y)=Z ref -Z(x,y)f is the camera focal length;

[0090] B represents the binocular baseline distance.

[0091] The real-time control system is based on a multi-agent deep deterministic policy gradient algorithm. It primarily uses wave wake state information obtained from processing wavefront images captured by a camera to control the nozzle diameter, water flow velocity, frequency, and direction, thereby achieving optimal suppression of wave wakes. Its main internal algorithm components are as follows: Figure 5 As shown. Figure 5 As described, the environmental component consists of a water spray control system, a test pool, a boat model, and a high-speed camera. The agent component comprises a policy network and a value network, each independently optimized for N water spray nozzles. The specific implementation steps are as follows:

[0092] Regarding actions provided by the real-world policy network. After the camera captures the wave height and wavefront similarity, the active control system returns to the state s' of the next moment. t and reward value Where state s t Includes the similarity S between the wave surface and the still water surface, and the wave height h at each detection point. i Information is provided here, with three detection points set up as shown in the following formula:

[0093] s t =[S,h1,h2,h3]

[0094]

[0095] h max =max(h1,h2,h3)

[0096] In this formula, α is a negative value, meaning that it can be achieved when the maximum wave height h is... max Increasing the value of 'r' decreases the value of 'S', where 'S' is the similarity between the current wavefront image and the still water surface image, and 'd' is the similarity between 'r' and 'S'. i v i f i b i These are the nozzle diameter, spray velocity, spray frequency, and spray direction, respectively, expressed as g(d). i ·v i ·f i ·b iThe function serves as an indicator of energy consumption. Current state s t Implementation of actions Rewards earned With the next state s' t Combined into an empirical tuple The data is then placed into the experience replay pool. The main purpose of experience replay is to resolve sample correlations, break temporal correlations, and make the reinforcement learning process more stable. Simultaneously, it improves the utilization rate of the sampled data. Subsequently, samples are taken from the experience replay pool. t and Inputting it into the real Q-value network yields the real Q-value: Then, sample s' from the experience replay pool. t The input is fed into the target policy network to obtain the next action a', which controls the water spray state of different nozzles. This action is then compared with the sampled data from the experience replay pool to obtain s'. t and Together, these values ​​are input into the target Q-value network to obtain the target Q-value: Q'(s') t The target Q value participates in the update of the target Q' value.

[0097]

[0098] Q target The target Q value used for network updates;

[0099] r represents the immediate reward that the current action a receives in state s;

[0100] γ is a discount factor used to balance the importance of current rewards and future rewards;

[0101] Q θ (s' t (a') represents the target Q-value calculated by the target network.

[0102] The mean squared error (MSE) between the predicted value of the actual Q-value network and the Q-value of the target value network is used as the loss function to optimize the value network:

[0103]

[0104] The Q value is calculated from the real-world value network;

[0105] This intelligent algorithm enables the target value network Q-value Q to be optimized. target The wavefront wake height should be minimized as much as possible, ideally as close to zero. Simultaneously, gradient descent is used to minimize the loss function, and the parameters of the real-valued Q-network are updated.

[0106] In this algorithm, the policy network (Actor) is updated less frequently than the value network (Critic). The Actor parameters are updated using gradient ascent to maximize the Q-value of the real-world value network for the current state and action.

[0107]

[0108] in, Let be the gradient of the policy objective function with respect to θ, and E be the expected value. This indicates the effect of action 'a' on the current target Q value. Let θ represent the effect of θ on action a, where θ is the policy network parameter and a is the optimal action output by the policy network.

[0109] The value network (Critic) update also uses the gradient ascent method:

[0110]

[0111] This is the gradient of the value objective function with respect to the policy parameter φ, used to update the value network (Critic).

[0112] This refers to averaging the gradients of N samples (states s) to reduce variance and improve training stability.

[0113] This represents the gradient of the Q-value obtained from the Critic network with respect to action a.

[0114] This represents the gradient of the value network (Critic) with respect to its own parameter φ.

[0115] By minimizing the loss function of the value network and maximizing the Q-value of the policy network for the output action, a progressive agent's decision-making strategy is achieved to obtain better action selection. Specifically, this involves determining which nozzles will spray water and simultaneously determining the spraying speed. Then, a soft update method is used to replace the parameters of the target network with the parameters of the real network.

[0116] θ' i ←τθ i +(1-τ)θ' i

[0117] φ'←τφ+(1-τ)φ'

[0118] θ i These are the parameters of the real-world policy network, θ' i These are the parameters of the target policy network.

[0119] φ is the parameter of the actual value network, and φ' is the parameter of the target value network.

[0120] Next, the updated real-world policy network is output, with random noise added to improve system robustness. Finally, action 'a' is output to the active control system, forming a loop process that repeats the above steps until the predetermined performance requirements are met.

[0121] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for controlling the wave surface wake of an underwater vehicle based on synthetic jet control, comprising: arranging a plurality of jet devices at designated positions on the outer shell of the underwater vehicle, the plurality of jet devices forming a jet array; when the vehicle is navigating underwater and generating a wave surface wake, simultaneously performing the following steps for each jet device: Step 1: setting a maximum number of iterations and a number of detection points; initializing the number of iterations and an experience replay pool; Step 2: obtaining a wave surface wake image and a still water surface image at the current time, processing the wave surface wake image at the current time; based on the processed wave surface wake image at the current time, obtaining an actual state of the jet device at the current time, an expected state of the jet device controlled at the previous time, and a reward value at the current time based on the actual state of the jet device at the current time; Step 3: inputting the actual state of the jet device at the current time, the reward value at the current time, and the expected state of the jet device controlled at the previous time into a reinforcement learning network to obtain an actual action value at the current time, an expected action value of the jet device controlled at the previous time, an expected state of the jet device controlled at the next time, an actual target value, and an expected target value; The image processing of Step 2 includes grayscale processing, Gaussian filtering processing, image correction processing, and generating a high-precision disparity map in stereo vision. determining whether the iteration number at the current time reaches the maximum iteration number, if not, updating the iteration number, and storing the actual state s of the fluidic device at the current time and the corresponding related state parameters into the experience replay pool, and updating the reinforcement learning network parameters and returning to step 2 t and the corresponding related state parameters into the experience replay pool, and updating the reinforcement learning network parameters and returning to step 2 If so, then the current actual state s of the ejector will be... t The corresponding state parameters are stored in the experience replay pool, and after updating the reinforcement learning network parameters, the current actual action value 'a' is... t Add random noise and output it, causing the ejector to move according to the current actual action value a. t Perform the action to end this process.

2. The method of claim 1, wherein: The method for calculating the reward value at the current time of Step 2 includes: The actual state of the ejector at the current moment s t =[S,h1,h2,h3,...h i ], where S is the similarity between the wave wake image and the still water surface image at time t, h i Let be the wave height of the i-th detection point.

3. The method of claim 2, wherein the method further comprises: The reinforcement learning network of Step 3 includes a display policy network, a target policy network, a real value network, and a target value network. r t = w1·e mhmax + w2·S - λ·g(d·v·f·b) where w1, w2 and λ are weight coefficients, m and e are constants and m is negative, h max is the maximum wave height among the wave heights of all detection points; g(d·v·f·b) is an energy function, d is the water jet hole diameter, v is the water jet speed, f is the water jet frequency, and b is the water jet direction.

4. The method of claim 3, wherein: The method for updating the reinforcement learning network of Step 3 is:

5. The method of claim 4, wherein the method further comprises: The current actual action value a is obtained by inputting the current time jet flow device actual state s t and the last time jet flow device control expected action value a′ t-1 into the display strategy network. t The current time reward value r t is input into the display strategy network to obtain the current actual action value a t . The next time jet flow device control expected state s t ′ is obtained by inputting the last time jet flow device control expected state s t ′ -1 into the target strategy network. t-1 The last time jet flow device control expected action value a′ is obtained by inputting the last time jet flow device control expected state s t ′ -1 into the target strategy network.

6. The method of claim 4, wherein the method further comprises: wherein γ is a discount factor. Step 3.1: input the current actual action value a t input the real value network to obtain the actual target value Q θ ; input the last time jet controller expected action value a' t input the target value network to obtain the last time expected target value Q' t-1 ; Step 3.2: according to the actual target value Q θ and the expected target value Q' of the last moment t-1 , calculate the optimal target value Q target ; according to the optimal target value Q target , update the parameters of the reinforcement learning network in combination with the experience replay pool; Q target = r t + γ · Q' t-1 The processor executes the computer program to implement the steps of the method of any one of claims 1 to 7.

7. A computer apparatus / device / system comprising a memory, a processor, and a computer program stored on the memory, characterized in that: The computer program / instructions, when executed by the processor, implement the steps of the method of any one of claims 1 to 7.

8. A computer readable storage medium having stored thereon computer programs / instructions, characterized in that: The computer program / instructions, when executed by the processor, implement the steps of the method of any one of claims 1 to 7.

9. A computer program product comprising computer programs / instructions, characterized in that: ​

Images