Focal length adjusting method and device of electro-hydraulic focusable lens, equipment and medium

Abstract

The invention discloses a focal length adjusting method and device of an electro-hydraulic focusable lens, equipment and a medium, and relates to the technical field of lens focusing, and the method comprises the steps: obtaining a scene image where a dynamic target is located through a camera, and selecting an area containing the dynamic target as a target area needing focusing; calculating the image definition of the focusing target area; when the image definition is smaller than the image definition threshold value, the focal length of the electro-hydraulic focus-adjustable lens is calculated; and inputting the image definition and the focal length into the electro-hydraulic focusable lens control network model to obtain a focusing current value, adjusting the focal length of the electro-hydraulic focusable lens through the focusing current value, determining the adjusted image definition, and enabling the adjusted image definition to be greater than or equal to an image definition threshold value, and determining an optimal focusing current value based on the adjusted image definition, and performing focal length adjustment on the electro-hydraulic focusable lens according to the optimal focusing current value. According to the method and the device, the optimal focusing of the electro-hydraulic focusable lens can be quickly and accurately realized.

Description

Technical Field

[0001] The present application relates to the technical field of lens focusing, and in particular to a method, device, equipment and medium for adjusting the focal length of an electro-hydraulic adjustable focus lens. Background Art

[0002] In the application scenarios of unmanned boats, clear image acquisition is crucial. As an advanced imaging device, the electro-hydraulic adjustable focus lens can obtain clear images by adjusting the focal length of the lens. However, traditional focusing methods often rely on preset parameters or manual adjustments, and it is difficult to quickly and accurately achieve optimal focusing for dynamic targets in dynamic environments.

[0003] Therefore, the traditional method of focusing the electro-hydraulic adjustable focus lens has the problems of slow focusing speed and low accuracy. Summary of the invention

[0004] The purpose of the present application is to provide a method, device, equipment and medium for adjusting the focal length of an electro-hydraulic adjustable focus lens, which can quickly and accurately achieve optimal focusing of the electro-hydraulic adjustable focus lens.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a method for adjusting the focal length of an electro-hydraulic adjustable focus lens, the method for adjusting the focal length of an electro-hydraulic adjustable focus lens comprising:

[0007] Acquire a scene image where a dynamic target is located through a camera, and select an area containing the dynamic target as a target area that needs to be focused; wherein the dynamic target is a movable target on the sea surface;

[0008] Calculating the image clarity of the focus target area;

[0009] When the image clarity is less than the image clarity threshold, calculating the focal length of the electro-hydraulic adjustable focus lens;

[0010] Constructing a focal length adjustment model based on reinforcement learning, wherein the focal length adjustment model based on reinforcement learning includes a policy network, a target policy network, a value network, and a target value network, and determining the trained policy network as an electro-hydraulic adjustable focus lens control network model;

[0011] The image clarity and the focal length are input into the control network model of the electro-hydraulic adjustable focus lens to obtain a focus current value, and the focal length of the electro-hydraulic adjustable focus lens is adjusted by the focus current value, and the adjusted image clarity is determined so that the adjusted image clarity is greater than or equal to an image clarity threshold value.

[0012] An optimal focus current value is determined based on the adjusted image clarity, and the focal length of the electro-hydraulic focus-adjustable lens is adjusted according to the optimal focus current value.

[0013] Furthermore, before obtaining the scene image where the dynamic target is located through the camera, it also includes:

[0014] Constructing a target tracking model based on reinforcement learning, wherein the target tracking model based on reinforcement learning includes a tracking strategy network, a tracking target strategy network, a tracking value network, and a tracking target value network, and determining the trained tracking strategy network as the unmanned boat target tracking network model;

[0015] Acquiring the distance between the dynamic target and the unmanned boat by using a radar on the unmanned boat;

[0016] When the distance is greater than or equal to the control distance of the electro-hydraulic adjustable focus lens, the distance is adjusted using the unmanned boat target tracking network model;

[0017] When the adjusted distance is less than the control distance of the electro-hydraulic adjustable focus lens, the unmanned boat is controlled to maintain the same speed and acceleration as the dynamic target, so that the unmanned boat and the dynamic target remain relatively still.

[0018] Furthermore, the distance is adjusted using the unmanned boat target tracking network model, including:

[0019] Obtaining the state quantity at the current moment; the state quantity includes the speed of the dynamic target, the acceleration of the dynamic target, the speed of the unmanned boat, the acceleration of the unmanned boat, and the angle between the unmanned boat bow direction and the line connecting the unmanned boat and the dynamic target;

[0020] The state quantity at the current moment is input into the unmanned boat target tracking network model to obtain the propeller control quantity of the unmanned boat, and the movement of the unmanned boat is controlled according to the propeller control quantity, thereby adjusting the distance so that the distance is less than the control distance of the electro-hydraulic adjustable focus lens.

[0021] Furthermore, the training process of the target tracking model based on reinforcement learning includes:

[0022] Constructing an experience replay pool, wherein the experience replay pool stores a plurality of experience tuples, wherein the experience tuples include a state quantity at time k, an action quantity at time k, a state quantity at time k+1, and a reward, wherein the action quantity is a propeller control quantity;

[0023] Sampling the experience tuples in the experience replay pool to obtain sampled experience tuples;

[0024] Inputting the state quantity at time k+1 in the sampled experience tuple into the tracking target strategy network to obtain the action quantity at time k+1;

[0025] Based on the target strategy smoothing regularization, the action amount at the k+1 moment is added with noise; the state amount at the k+1 moment and the action amount at the k+1 moment after adding noise are input into the tracking target value network to obtain the state value target value;

[0026] Input the state quantity at time k and the action quantity at time k in the sampled experience tuple into the tracking value network to output the state value evaluation value;

[0027] Minimize the error between the state value evaluation value and the state value target value using a gradient descent algorithm, and update the parameters of the tracking value network;

[0028] Input the state quantity at time k in the sampled experience tuple into the tracking strategy network to obtain the predicted action quantity; input the state quantity at time k and the predicted action quantity into the tracking value network to obtain the predicted state value evaluation value;

[0029] The predicted state value evaluation value is maximized by using a gradient ascent algorithm, the parameters in the tracking strategy network are updated, the parameters of the tracking target strategy network are updated according to the updated parameters of the tracking strategy network, and the parameters of the tracking target value network are updated according to the updated parameters of the tracking value network.

[0030] Furthermore, the training process of the focus adjustment model based on reinforcement learning includes:

[0031] Construct an electro-hydraulic experience replay pool, the electro-hydraulic experience replay pool includes multiple electro-hydraulic experience tuples, the electro-hydraulic experience tuples include the electro-hydraulic state quantity at time G, the electro-hydraulic action quantity at time G, the electro-hydraulic state quantity at time G+1, and the electro-hydraulic reward; wherein the electro-hydraulic state quantity is image clarity and focal length, the image clarity includes overall image clarity, image edge clarity, and image contrast, and the electro-hydraulic action quantity is the focusing current value;

[0032] Sampling the electro-hydraulic experience tuple in the electro-hydraulic experience playback pool to obtain a sampled electro-hydraulic experience tuple;

[0033] Inputting the electro-hydraulic state quantity at time G+1 in the sampled electro-hydraulic experience tuple into the target strategy network to obtain the electro-hydraulic action quantity at time G+1;

[0034] Based on the target strategy smoothing regularization, the electro-hydraulic action amount at the G+1 moment is added with noise; the electro-hydraulic state amount at the G+1 moment and the electro-hydraulic action amount at the G+1 moment after adding noise are input into the target value network to obtain the electro-hydraulic state value target value;

[0035] Input the electro-hydraulic state quantity at time G and the electro-hydraulic action quantity at time G into the value network to output the electro-hydraulic state value evaluation value;

[0036] Minimizing the error between the electro-hydraulic state value evaluation value and the electro-hydraulic state value target value by using a gradient descent algorithm, and updating the parameters of the value network;

[0037] Inputting the electro-hydraulic state quantity at time G in the sampled electro-hydraulic experience tuple into the strategy network to obtain the predicted electro-hydraulic action quantity;

[0038] Input the electro-hydraulic state quantity and the predicted electro-hydraulic action quantity at time G in the sampled experience tuple into the value network to obtain the predicted electro-hydraulic state value evaluation value;

[0039] The predicted electro-hydraulic state value evaluation value is maximized by using a gradient ascent algorithm, the parameters of the strategy network are updated, the parameters of the target strategy network are updated according to the updated parameters of the strategy network, and the parameters of the target value network are updated according to the updated parameters of the value network.

[0040] Furthermore, the calculation formula of the electro-hydraulic reward is:

[0041]

[0042] Among them, r st For electro-hydraulic rewards, 11 ,ω 12 ,ω 13 ,ω 14 is a hyperparameter, f t-1 (a) are the image contrast, overall image clarity, image edge clarity and focal length at time G-1, a is the focus target area, f t (a) are the image contrast, overall image clarity, image edge clarity and focal length at time G, respectively, and |||| is the binary norm.

[0043] Further, determining an optimal focus current value based on the adjusted image clarity, and adjusting the focal length of the electro-hydraulic focus adjustable lens according to the optimal focus current value, including:

[0044] Adjusting the focal length in the direction of increasing image clarity according to a preset focusing current step length to obtain a re-adjusted focal length, and determining the re-adjusted image clarity;

[0045] Comparing the image clarity after adjustment with the image clarity after re-adjustment, and determining the optimal focus current value according to the comparison result;

[0046] The focal length of the electro-hydraulic adjustable focus lens is adjusted according to the optimal focusing current value.

[0047] In a second aspect, the present application provides a focal length adjustment device for an electro-hydraulic adjustable focus lens, wherein the focal length adjustment system of the electro-hydraulic adjustable focus lens comprises:

[0048] A target area determination module is used to obtain a scene image where a dynamic target is located through a camera, and select an area containing the dynamic target as a target area that needs to be focused; wherein the dynamic target is a movable target on the sea surface;

[0049] A first calculation module, used to calculate the image clarity of the focus target area;

[0050] A second calculation module, used for calculating the focal length of the electro-hydraulic adjustable focus lens when the image clarity is less than the image clarity threshold;

[0051] A model building module, used to build a focus adjustment model based on reinforcement learning, wherein the focus adjustment model based on reinforcement learning includes a policy network, a target policy network, a value network, and a target value network, and the trained policy network is determined as an electro-hydraulic adjustable focus lens control network model;

[0052] A first focal length adjustment module, used for inputting the image clarity and the focal length into the electro-hydraulic adjustable focus lens control network model, obtaining a focusing current value, adjusting the focal length of the electro-hydraulic adjustable focus lens by the focusing current value, and determining the adjusted image clarity, so that the adjusted image clarity is greater than or equal to an image clarity threshold;

[0053] The second focal length adjustment module is used to determine the optimal focusing current value based on the adjusted image clarity, and to adjust the focal length of the electro-hydraulic adjustable focus lens according to the optimal focusing current value.

[0054] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the focal length adjustment method of the electro-hydraulic adjustable focus lens described in any one of the above.

[0055] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the above-described methods for adjusting the focal length of an electro-hydraulic focus-adjustable lens.

[0056] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0057] The present application provides a method, device, equipment and medium for adjusting the focal length of an electro-hydraulic adjustable focus lens. The present application determines a focus target area based on an image of a scene where a dynamic target is located, and when the image clarity of the focus target area is less than an image clarity threshold, uses an electro-hydraulic adjustable focus lens control network model to adjust the focal length so that the image clarity of the focus target area meets the requirements, thereby achieving automatic adjustment of the focal length without manual intervention, and then uses the adjusted image clarity to determine an optimal focusing current value, and adjusts the focal length again according to the optimal focusing current value, further optimizing the image clarity and achieving optimal focusing of the electro-hydraulic adjustable focus lens. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0059] Figure 1 This is an application environment diagram of a method for adjusting the focal length of an electro-hydraulic adjustable focus lens in one embodiment of the present application;

[0060] Figure 2 This is a schematic flow chart of a method for adjusting the focal length of an electro-hydraulic adjustable focus lens in one embodiment of the present application;

[0061] Figure 3 A schematic diagram of a flow chart of a method for determining an optimal focusing current value provided in an embodiment of the present application;

[0062] Figure 4 A schematic diagram of functional modules of a focal length adjustment device for an electro-hydraulic adjustable focus lens provided in one embodiment of the present application;

[0063] Figure 5 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0064] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0065] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0066] The focal length adjustment method of the electro-hydraulic adjustable focus lens provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the scene image where the dynamic target is located to the server 104. After the server 104 receives the scene image where the dynamic target is located, for the scene image where the dynamic target is located, the server 104 selects the area containing the dynamic target as the target area that needs to be focused; calculates the image clarity of the focus target area; when the image clarity is less than the image clarity threshold, calculates the focal length of the electro-hydraulic adjustable focus lens; inputs the image clarity and the focal length into the electro-hydraulic adjustable focus lens control network model to obtain the focusing current value, and adjusts the focal length of the electro-hydraulic adjustable focus lens by the focusing current value, determines the adjusted image clarity, and makes the adjusted image clarity greater than or equal to the image clarity threshold; determines the optimal focusing current value based on the adjusted image clarity, and adjusts the focal length of the electro-hydraulic adjustable focus lens according to the optimal focusing current value. The server 104 may feed back the adjusted focal length to the terminal 102. In addition, in some embodiments, the focal length adjustment method of the electro-hydraulic adjustable focus lens may also be implemented by the server 104 or the terminal 102 alone, such as the terminal 102 may directly process the scene image where the dynamic target is located, or the server 104 may obtain the scene image where the dynamic target is located from the data storage system, and adjust the focal length of the electro-hydraulic adjustable focus lens according to the scene image where the dynamic target is located.

[0067] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, and tablet computers. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0068] In an exemplary embodiment, Figure 2 As shown, a method for adjusting the focal length of an electro-hydraulic adjustable focus lens is provided. The method is executed by a computer device, and specifically can be executed by a computer device such as a terminal or a server alone, or can be executed by a terminal and a server together. In an embodiment of the present application, the method for adjusting the focal length of an electro-hydraulic adjustable focus lens includes the following steps 101 to 106, wherein:

[0069] Step 101, obtain a scene image where a dynamic target is located through a camera, and select an area containing the dynamic target as a target area that needs to be focused; wherein the dynamic target includes various ships, cruise ships, unmanned boats and other movable targets on the sea surface.

[0070] Step 102: Calculate the image clarity of the focus target area.

[0071] Step 103: When the image clarity is less than the image clarity threshold, the focal length of the electro-hydraulic adjustable focus lens is calculated.

[0072] Step 104: construct a focus adjustment model based on reinforcement learning, wherein the focus adjustment model based on reinforcement learning includes a policy network, a target policy network, a value network, and a target value network, and the trained policy network is determined as an electro-hydraulic adjustable focus lens control network model.

[0073] Step 105: input the image clarity and the focal length into the electro-hydraulic adjustable focus lens control network model to obtain a focusing current value, and adjust the focal length of the electro-hydraulic adjustable focus lens according to the focusing current value, and determine the adjusted image clarity so that the adjusted image clarity is greater than or equal to an image clarity threshold.

[0074] Step 106 : determining an optimal focus current value based on the adjusted image clarity, and adjusting the focal length of the electro-hydraulic focus adjustable lens according to the optimal focus current value.

[0075] By implementing the above steps 101 to 106, the present application can improve the efficiency of focal length adjustment and achieve optimal focusing of the electro-hydraulic adjustable focus lens.

[0076] In another exemplary embodiment of the present application, before step 101, the following further includes: step 100, controlling the unmanned boat so that the unmanned boat and the dynamic target remain relatively still. Step 100 specifically includes:

[0077] Step 1001: construct a target tracking model based on reinforcement learning, wherein the target tracking model based on reinforcement learning includes a tracking strategy network, a tracking target strategy network, a tracking value network, and a tracking target value network, and the trained tracking strategy network is determined as the unmanned boat target tracking network model.

[0078] Step 1002: Acquire the distance between the dynamic target and the unmanned boat through the radar on the unmanned boat.

[0079] Step 1003: When the distance is greater than or equal to the control distance of the electro-hydraulic adjustable focus lens, the distance is adjusted using the unmanned boat target tracking network model.

[0080] Step 1004: When the adjusted distance is less than the control distance of the electro-hydraulic adjustable focus lens, the unmanned boat and the dynamic target are controlled to maintain the same speed and acceleration, so that the unmanned boat and the dynamic target remain relatively still.

[0081] Reinforcement Learning (RL) is a machine learning method that learns the optimal strategy based on reward signals by interacting with the environment. Compared with traditional control methods, reinforcement learning does not require a clear model and has strong robustness and adaptability. Therefore, applying reinforcement learning to unmanned boat control can enable the unmanned boat to autonomously learn the optimal motion control strategy in a complex marine environment, so that the unmanned boat and the dynamic target remain relatively stationary.

[0082] In another exemplary embodiment of the present application, the target tracking model based on reinforcement learning constructed in step 1001 includes a tracking strategy network, a tracking target strategy network, a tracking value network and a tracking target value network, specifically including a tracking strategy network as an Actor network, a tracking target strategy network as a TargetActor network, a tracking value network as a Critic1 network and a Critic2 network, and a tracking target value network as a Target Critic1 network and a Target Critic2 network.

[0083] The network architecture of the Actor network and the TargetActor network is the same, but their input and output are different. Specifically, the input of the Actor network is the state quantity at the current moment, and the output of the Actor network is the action quantity at the current moment. The input of the TargetActor network is the state quantity at the next moment, and the output of the TargetActor network is the action quantity at the next moment.

[0084] The four Critic networks can be divided into two parts: one is the Critic1 network and the Critic2 network, which are exactly the same; the other is the Target Critic1 network and the Target Critic2 network, which are also exactly the same. The Critic network and the Target Critic network mainly differ in input and output. The input of the Critic network is the state-action pair at the current moment, and the output is the state value evaluation value of the current state-action pair. The state quantity and action quantity at the same moment are a state-action pair; the input of the Target Critic network is the state-action pair at the next moment, and the output is the state value target value of the state-action pair at the next moment.

[0085] The Actor network mainly consists of 1 input layer, 4 hidden layers and 1 output layer. The input layer contains 6 nodes, corresponding to 6 parameters of the state quantity; the hidden layer is composed of linear layers, and the four hidden layers contain 1024, 1024, 512 and 256 nodes respectively; the output layer contains 2 nodes, corresponding to 2 parameters of the action quantity.

[0086] The Critic network architecture consists of two branches. The first branch is similar to the Actor network, consisting of 1 input layer and 4 hidden layers. The input layer contains 6 nodes, corresponding to the 6 parameters of the state quantity; the hidden layer is composed of linear layers, and the four hidden layers contain 1024, 1024, 512 and 256 nodes respectively. The second branch consists of 1 input layer and 2 hidden layers. The input layer is the output layer of the Actor network, containing 2 nodes; the two hidden layers contain 512 and 256 nodes respectively. The two branches have the same size of 256-node hidden layers. The outputs of the two 256-node hidden layers are added and normalized and then output to the output layer. The output layer contains 1 node, so the output of the output layer is the evaluation value or target value of the state-action pair.

[0087] In another exemplary embodiment of the present application, the target tracking model training process based on reinforcement learning includes the following steps:

[0088] S1. Construct an experience replay pool, in which multiple experience tuples are stored. The experience tuples include a state quantity at time k, an action quantity at time k, a state quantity at time k+1, and a reward. The action quantity is a propeller control quantity.

[0089] Create a simulation environment for unmanned boat target tracking, which mainly includes unmanned boats, radars, dynamic targets, marine environments and other components. Randomly define the starting positions of unmanned boats and dynamic targets in the simulation environment for unmanned boat target tracking. Use the radar on the unmanned boat to obtain the current position of the dynamic target, calculate the current speed and acceleration of the dynamic target, use the unmanned boat inertial navigation to confirm the current speed and acceleration of the unmanned boat, and obtain the distance between the unmanned boat and the dynamic target, as well as the angle between the unmanned boat bow direction and the line connecting the unmanned boat and the dynamic target.

[0090] The obtained parameters are used as state quantities Among them, the speed of the unmanned boat v u , the acceleration of the unmanned boat is a u , the speed v of the dynamic target d , the acceleration of the dynamic target a d , the distance Δl between the unmanned boat and the dynamic target, the angle between the current heading angle of the unmanned boat and the line connecting the unmanned boat and the dynamic target Input into the target tracking model based on reinforcement learning to obtain the action amount a that controls the movement of the unmanned boat t , the amount of action performed in, They represent the control quantities of the left and right propellers respectively, and the state quantity at the next moment is obtained. The control quantities of the left and right propellers are sent to the unmanned boat to realize the tracking of dynamic targets. At the same time, the reward is calculated according to the obtained state quantity. The target tracking model based on reinforcement learning is an improved twin delayed deep deterministic policy gradient (Twin Delayed Deep Deterministic policy gradient, TD3) reinforcement learning algorithm neural network.

[0091] Loop through the steps and pack the state s at time k t , the action amount a at time k t , the reward r at time k t And the state quantity s at time k+1 t ', forming a set of experience tuples (s t ,a t ,r t ,s t ') Store it in the experience replay pool, which stores multiple experience tuples. Get multiple sets of experience tuples and store them in the experience replay pool. If the distance between the unmanned boat and the dynamic target is less than the specified distance (the specified distance is the control distance of the electro-hydraulic adjustable focus lens) and lasts for more than 30 seconds, or the distance fails to be less than the specified distance within 60 seconds, the current round of training ends. Repeat the steps. If K rounds have been repeated, execute step S2 and recount. If the number of repeated rounds is less than K rounds, continue to repeat the steps.

[0092] S2. Sampling the experience tuples in the experience replay pool to obtain sampled experience tuples.

[0093] The priority experience replay algorithm is used to sample the experience tuples in the experience replay pool, and n groups of experience tuples are taken from the experience replay pool.

[0094] Specifically, formula (1) is used to implement sampling of the experience replay pool:

[0095]

[0096] Wherein, P(c) represents the priority of a group of experience tuples selected from the experience playback pool, c represents the sequence number of the currently extracted experience tuple, Indicates the priority of the currently extracted experience tuple, represents the priority of the extracted experience tuple; α represents the preset parameter used to adjust the priority sampling degree of data samples.

[0097] S3. Input the state quantity at time k+1 in the sampled experience tuple into the target tracking strategy network to obtain the action quantity at time k+1.

[0098] S4. Add noise to the action amount at time k+1 based on target strategy smoothing regularization; input the state amount at time k+1 and the action amount at time k+1 after adding noise into the tracking target value network to obtain the state value target value.

[0099] Specifically, the action volume at time k+1 obtained by the target tracking strategy network is added with noise, and the calculation formula for adding noise is:

[0100] a t '=μ'(s t '|θ μ' )+∈ (2)

[0101] ∈=clip(N(0,σ),-b,b) (3)

[0102] Among them, θ μ' are the network parameters of the TargetActor network, μ'() represents the TargetActor network, ∈ represents the random noise parameter; the clip function indicates that when N(0,σ)<-b, ∈=-b, when N(0,σ)>b, ∈=b, otherwise, ∈=N(0,σ); N(0,σ) means that it satisfies the normal distribution, -b and b represent fixed parameters, b>0.

[0103] Based on the idea of ​​dual networks, the Target Critic network is used to calculate the state-action pair (s t ',a t ') state value target value y t , state value target value y t The calculation formula is as follows:

[0104]

[0105] Among them, r t is the reward function, γ is the discounted return rate, is the network parameter of Target Critic1 or Target Critic2 network, Q i '() represents the Target Critic1 or Target Critic2 network.

[0106] Reward t The calculation formula is:

[0107]

[0108] Among them, ω1, ω2, ω3, ω4, ω5 are hyperparameters, which are adjusted according to the actual situation, a is the amplitude height, σ is the standard deviation, b is the position parameter, and r 2tis the reward function within the control distance of the electro-hydraulic adjustable focus lens, and the speed v of the unmanned boat u , the acceleration of the unmanned boat is a u , the speed v of the dynamic target d , the acceleration of the dynamic target a d , the distance Δl between the unmanned boat and the dynamic target, the angle between the current heading angle of the unmanned boat and the line connecting the unmanned boat and the dynamic target

[0109]

[0110] Wherein, R is the control distance of the electro-hydraulic focusable lens.

[0111] S5. Input the state quantity at time k and the action quantity at time k in the sampled experience tuple into the tracking value network to output the state value evaluation value.

[0112] S6. Use the gradient descent algorithm to minimize the error between the state value evaluation value and the state value target value, and update the parameters of the tracking value network.

[0113] Specifically, the gradient descent algorithm is used to minimize the error between the state value evaluation value and the state value target value, update the parameters of the tracking value network, and the error between the state value evaluation value and the state value target value The calculation formula is:

[0114]

[0115] Among them, Q i () represents the Critic1 or Critic2 network, s t and a t They represent the state quantity and action quantity at the current moment respectively. Get the network parameters of the Critic2 network for Critic1.

[0116] S7. Input the state quantity at time k in the sampled experience tuple into the tracking strategy network to obtain the predicted action quantity; input the state quantity at time k and the predicted action quantity into the tracking value network to obtain the predicted state value evaluation value.

[0117] S8. Use the gradient ascent algorithm to maximize the predicted state value evaluation value, update the parameters in the tracking strategy network, and update the parameters of the tracking target strategy network according to the updated parameters of the tracking strategy network, and update the parameters of the tracking target value network according to the updated parameters of the tracking value network.

[0118] Specifically, after the Ctitic1 and Critic2 networks are updated for d steps, the Actor network update is started, and the state quantity s at time k is calculated using the Actor network. t The predicted action amount a tnew , predict the action amount a tnew The calculation formula is:

[0119] a tnew =μ(s t |θ μ ) (8)

[0120] Among them, s t is the state quantity at time k, θ μ is the network parameter of the Actor network, and μ() represents the Actor network.

[0121] There is no need to add noise after calculating the predicted action amount, because here we hope that the Actor network can be updated in the direction of the maximum value, and adding noise does not make sense.

[0122] Use Critic1 or Critic2 network to calculate the state action pair (s t ,a tnew )’s state value evaluation value q t , state value evaluation value q t The calculation formula is as follows, assuming that the Critic1 network is used:

[0123] q t =Q1(s t， a tnew |θ Q1 ) (9)

[0124] Among them, θ Q1 are the network parameters of the Critic1 network, and Q1() represents the Critic1 network.

[0125] Use the gradient ascent algorithm to maximize the state value evaluation value and update the parameters in the Actor network. The reason why either Critic1 or Critic2 can be used to calculate the Q value is mainly because the purpose of the Actor network is to maximize the cumulative expected return, and there is no need to use the minimum value.

[0126] The TD3 algorithm uses a soft update method to update the parameters of the tracking target strategy network and the tracking target value network. The parameter update process of the TargetActor network is as follows:

[0127] θ μ' =τθ μ +(1-τ)θ μ' (10)

[0128] Among them, τ is the learning rate (momentum), θ μ is the network parameter of the Actor network, θ μ' The network parameters for the TargetActor network.

[0129] The Target Critic1 and Target Critic2 network update process is as follows:

[0130]

[0131] in, is the network parameter of Target Critic1 or Target Critic2 network, is the network parameter of Critic1 or Critic2 network, τ is the learning rate (momentum), τ∈(0,1), and is usually taken as 0.005.

[0132] During the training process, when the number of games of the target tracking model based on reinforcement learning reaches the predetermined number of stable cycles M, the game number-reward curve is checked. If the reward has stabilized in cycle M, the cycle ends, the training of the target tracking model based on reinforcement learning is completed, and the best network model is obtained. If the reward has not stabilized, the training process is repeated, and the reward function in the game number-reward curve is checked to see if it is stable after each preset number of repetitions until the reward stabilizes.

[0133] In another exemplary embodiment of the present application, step 1001 is used to obtain a target tracking model based on reinforcement learning, and the trained tracking strategy network is determined as the unmanned boat target tracking network model, and then step 1002 is executed. Step 1002 specifically includes using the radar carried by the unmanned boat to obtain the direction of the dynamic target, calculating the speed and acceleration of the dynamic target, using the unmanned boat's own inertial navigation to determine the speed and acceleration of the unmanned boat, and obtaining the distance between the unmanned boat and the dynamic target and the angle between the direction of the unmanned boat's bow and the line connecting the unmanned boat and the dynamic target.

[0134] In another exemplary embodiment of the present application, step 1003 uses the unmanned boat target tracking network model to adjust the distance, specifically including steps 10031 to 10032:

[0135] Step 10031, obtain the state quantity at the current moment; the state quantity includes the speed of the dynamic target, the acceleration of the dynamic target, the speed of the unmanned boat, the acceleration of the unmanned boat, and the angle between the bow direction of the unmanned boat and the line connecting the unmanned boat and the dynamic target.

[0136] Specifically, in the above step 1002, while obtaining the distance between the radar dynamic target on the unmanned boat and the unmanned boat, the speed of the dynamic target, the acceleration of the dynamic target, the speed of the unmanned boat, the acceleration of the unmanned boat, and the angle between the direction of the unmanned boat bow and the line connecting the unmanned boat and the dynamic target are obtained.

[0137] Step 10032: input the current state quantity into the unmanned boat target tracking network model to obtain the propeller control quantity of the unmanned boat, control the movement of the unmanned boat according to the propeller control quantity, and then adjust the distance so that the distance is less than the control distance of the electro-hydraulic adjustable focus lens.

[0138] The present application controls the distance between the unmanned boat and the dynamic target through an unmanned boat target tracking network model, thereby realizing automatic control and quickly adjusting the distance between the unmanned boat and the dynamic target. At the same time, by controlling the speed and acceleration of the unmanned boat and the dynamic target, the unmanned boat and the dynamic target remain relatively still. When the unmanned boat and the dynamic target remain relatively still, the scene image of the dynamic target is obtained through the camera, and then the focal length of the electro-hydraulic adjustable focus lens is adjusted.

[0139] In another exemplary embodiment of the present application, step 101 specifically includes:

[0140] The electro-hydraulic focusable lens is placed on a designated camera and mounted on a specific controllable three-degree-of-freedom gimbal to ensure that the dynamic target is always in the center of the image. The camera acquires the scene image where the dynamic target is located, and uses computer vision technology to automatically select the area containing the dynamic target as the target area that needs to be focused.

[0141] In another exemplary embodiment of the present application, step 102 uses the image clarity formula (12) to calculate the image clarity of the focus target area. In this embodiment, the EOG function is used as the evaluation index of the image clarity. The image clarity formula is as follows:

[0142] E0=E EOG =∑ Hight ∑ Widh [I(x+1,y)-I(x,y)] 2 +[I(x,y+1)-I(x,y)] 2 (12)

[0143] Wherein, E0 is the image clarity of the focus target area, and I(x,y) represents the pixel value of the image at the coordinate (x,y).

[0144] In another exemplary embodiment of the present application, in step 103, when calculating the focal length, formula (13) may be used for calculation:

[0145] The focal length is calculated using the lens imaging formula, the specific formula is:

[0146]

[0147] Among them, f is the focal length, u is the object distance, and v is the image distance.

[0148] In another exemplary embodiment of the present application, the reinforcement learning-based focus adjustment model constructed in step 104 specifically includes a policy network, a target policy network, a value network and a target value network, specifically including an Actor network, a TargetActor network, a Critic1 network, a Target Critic1 network, a Critic2 network and a Target Critic2 network.

[0149] The Actor network, TargetActor network, Critic1 network, Target Critic1 network, Critic2 network and Target Critic2 network in the focal length adjustment model based on reinforcement learning have the same structure as the target tracking model based on reinforcement learning, except that the input and output quantities of the Actor network, TargetActor network, Critic1 network, TargetCritic1 network, Critic2 network and Target Critic2 network are changed respectively. Among them, the nodes of the input layer of the Actor network become 4 nodes, corresponding to 4 parameters of the electro-hydraulic state quantity; the node of the output layer becomes 1 node, corresponding to 1 parameter of the electro-hydraulic action quantity.

[0150] The number of nodes in the first branch input layer of the Critic network architecture is changed to 4, corresponding to the 4 electro-hydraulic state quantities. The number of nodes in the second branch input layer is changed to 1, corresponding to the output of the Actor network.

[0151] In another exemplary embodiment of the present application, the training process of the focus adjustment model based on reinforcement learning includes the following steps:

[0152] S11: Construct an electro-hydraulic experience replay pool, which contains multiple electro-hydraulic experience tuples, and the electro-hydraulic experience tuples contain the electro-hydraulic state quantity at time G, the electro-hydraulic action quantity at time G, the electro-hydraulic state quantity at time G+1, and the electro-hydraulic reward; wherein, the electro-hydraulic state quantity is image clarity and focal length, the image clarity includes overall image clarity, image edge clarity, and image contrast, and the electro-hydraulic action quantity is the focusing current value.

[0153] A simulation environment for training a focal length adjustment model based on reinforcement learning is established, including dynamic targets, electro-hydraulic adjustable focus lenses, corresponding cameras and other equipment. The focus target area obtained in the above step 101 is obtained, and then the image clarity and focal length of the focus target area are calculated. The image clarity includes the overall image clarity, edge clarity and image contrast. The overall image clarity, edge clarity, image contrast and focal length are input as electro-hydraulic state quantities into the focal length adjustment model based on reinforcement learning to obtain the electro-hydraulic action quantity, execute the electro-hydraulic action quantity, and realize the focal length adjustment of the electro-hydraulic adjustable focus lens. At the same time, the electro-hydraulic reward is calculated according to the obtained electro-hydraulic state quantity. The focal length adjustment model based on reinforcement learning is an improved twin-delayed deep deterministic policy gradient (TD3) reinforcement learning algorithm neural network.

[0154] The steps are executed in a loop, and the electro-hydraulic state quantity at time G, the electro-hydraulic action quantity at time G, the electro-hydraulic reward function at time G, and the electro-hydraulic state quantity at time G+|1 are packaged to form a set of electro-hydraulic experience tuples, which are stored in the electro-hydraulic experience replay pool. The electro-hydraulic state quantity is the image clarity and focal length. The image clarity includes the overall image clarity, the image edge clarity, and the image contrast. The electro-hydraulic action quantity is the focusing current value. Multiple sets of electro-hydraulic experience tuples are obtained and stored in the electro-hydraulic experience replay pool. If the image clarity exceeds the image clarity threshold, the current round of training ends. Repeat the steps. If K rounds have been repeated, execute step S22 and recount. If the number of repeated rounds is less than K rounds, continue to repeat the steps.

[0155] S22. Sampling the electro-hydraulic experience tuples in the electro-hydraulic experience playback pool to obtain sampled electro-hydraulic experience tuples.

[0156] The specific implementation of this process is the same as that of the above step S2 and will not be repeated here.

[0157] S33, inputting the electro-hydraulic state quantity at time G+1 in the sampled electro-hydraulic experience tuple into the target strategy network to obtain the electro-hydraulic action quantity at time G+1.

[0158] S44. Add noise to the electro-hydraulic action quantity at time G+1 based on target strategy smoothing regularization; input the electro-hydraulic state quantity at time G+1 and the electro-hydraulic action quantity at time G+1 after adding noise into the target value network to obtain the electro-hydraulic state value target value.

[0159] S55. Input the electro-hydraulic state quantity at time G and the electro-hydraulic action quantity at time G into the value network to output the electro-hydraulic state value evaluation value.

[0160] S66. Use a gradient descent algorithm to minimize the error between the electro-hydraulic state value assessment value and the electro-hydraulic state value target value, and update the parameters of the value network.

[0161] S77, inputting the electro-hydraulic state quantity at time G in the sampled electro-hydraulic experience tuple into the strategy network to obtain the predicted electro-hydraulic action quantity; inputting the electro-hydraulic state quantity at time G in the sampled experience tuple and the predicted electro-hydraulic action quantity into the value network to obtain the predicted electro-hydraulic state value evaluation value;

[0162] S88. Use the gradient ascent algorithm to maximize the predicted electro-hydraulic state value assessment value, update the parameters of the strategy network, and update the parameters of the target strategy network according to the updated parameters of the strategy network, and update the parameters of the target value network according to the updated parameters of the value network.

[0163] In another exemplary embodiment of the present application, the process of updating the parameters of the value network through the above steps S33 to S66 is the same as the updating process of the tracking value network parameters, except that the input and output of the tracking target strategy network are changed to the input and output of the corresponding target strategy network; the input and output of the tracking target value network are changed to the input and output of the corresponding target value network; the input of the tracking value network is changed to the input of the value network, and the output of the corresponding value network, i.e., the electro-hydraulic state value target value, is obtained. The reward in the calculation formula (4) of the state value target value is changed to the electro-hydraulic reward to obtain the calculation formula of the electro-hydraulic state value target value. The calculation formula (14) of the electro-hydraulic reward is:

[0164]

[0165] Among them, r st For electro-hydraulic rewards, 11 ,ω 12 ,ω 13 ,ω 14 is a hyperparameter, f t-1 (a) are the image contrast, overall image clarity, image edge clarity and focal length at time G-1, a is the focus target area, f t (a) are the image contrast, overall image clarity, image edge clarity and focal length at time G, respectively, and |||| is the binary norm.

[0166] The parameter update process of the policy network, target policy network and target value network through the above steps S77 to S88 is the same as the update process of the above steps S7 to S8, except that the input and output of the tracking strategy network are changed to the input and output of the policy network; the parameter update of the target policy network and the target value network is carried out according to the corresponding updated parameters of the policy network and the updated parameters of the value network, and the specific process is not repeated here.

[0167] During the training process, when the number of training rounds of the focal length adjustment model based on reinforcement learning reaches the predetermined number of stable cycles M, the number of rounds-electro-hydraulic reward curve is checked. If the electro-hydraulic reward has stabilized at cycle M, the training of the focal length adjustment model based on reinforcement learning is completed and the best network model is obtained. If the electro-hydraulic reward has not stabilized, the training process is repeated for training. After each preset number of repetitions, check whether the electro-hydraulic reward in the number of rounds-electro-hydraulic reward curve is stable until the electro-hydraulic reward stabilizes.

[0168] In another exemplary embodiment of the present application, Figure 3 As shown, step 106 specifically includes:

[0169] Step 1061 , adjusting the focal length in the direction of increasing the image clarity according to a preset focus current step length, obtaining a re-adjusted focal length, and determining the re-adjusted image clarity.

[0170] In this embodiment, the preset focus current step length is selected as 2 mA, the adjusted image clarity is adjusted again in the direction of increasing image clarity according to the preset focus current step length, and the re-adjusted image clarity is determined.

[0171] Step 1062: Compare the adjusted image clarity with the re-adjusted image clarity, and determine the optimal focus current value according to the comparison result.

[0172] Specifically, the image clarity after adjustment is E1, and the image clarity after adjustment again is E2. Compare E1 and E1. If E1≤E2, return to step 1061. If E1>E2, the focusing current value corresponding to E1 is the optimization result of a single hill climbing, and the focusing current value corresponding to E1 is taken as the optimal focusing current value.

[0173] Step 1063: adjust the focal length of the electro-hydraulic adjustable focus lens according to the optimal focusing current value.

[0174] Specifically, the focal length of the electro-hydraulic adjustable focus lens is adjusted according to the optimal focusing current value I obtained in step 1062 .

[0175] The present application uses a camera to obtain a scene image where a dynamic target is located, and based on this, obtains the image clarity of the focus target area. When the image clarity does not meet the clarity requirements, the focal length is adjusted using an electro-hydraulic adjustable focus lens control network model. The electro-hydraulic adjustable focus lens control network model is used to automatically adjust the focal length, and an image whose image clarity meets the requirements is quickly obtained, and autonomous focusing is achieved without human intervention. Thereafter, the image clarity is adjusted again using a single hill climbing optimization algorithm to determine the optimal focusing current value, further optimize the image clarity, and achieve optimal focusing of the electro-hydraulic adjustable focus lens.

[0176] Based on the same inventive concept, the embodiment of the present application also provides a focal length adjustment device for implementing the above-mentioned electro-hydraulic adjustable focus lens. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in the embodiments of the focal length adjustment device for one or more electro-hydraulic adjustable focus lenses provided below can refer to the limitations of the focal length adjustment method for the electro-hydraulic adjustable focus lens above, and will not be repeated here.

[0177] In an exemplary embodiment, Figure 4 As shown, a focal length adjustment device of an electro-hydraulic adjustable focus lens is provided, comprising:

[0178] The target area determination module 31 is used to obtain a scene image of a dynamic target through a camera, and select an area containing the dynamic target as a target area to be focused; wherein the dynamic target is a movable target on the sea surface.

[0179] The first calculation module 32 is used to calculate the image clarity of the focus target area.

[0180] The second calculation module 33 is used to calculate the focal length of the electro-hydraulic adjustable focus lens when the image clarity is less than the image clarity threshold.

[0181] The model building module 34 is used to build a focus adjustment model based on reinforcement learning, wherein the focus adjustment model based on reinforcement learning includes a policy network, a target policy network, a value network and a target value network, and the trained policy network is determined as an electro-hydraulic adjustable focus lens control network model.

[0182] The first focal length adjustment module 35 is used to input the image clarity and the focal length into the electro-hydraulic adjustable focus lens control network model to obtain a focusing current value, and adjust the focal length of the electro-hydraulic adjustable focus lens by the focusing current value, and determine the adjusted image clarity so that the adjusted image clarity is greater than or equal to an image clarity threshold; wherein the electro-hydraulic adjustable focus lens control network model includes a strategy network, a target strategy network, a value network and a target value network.

[0183] The second focal length adjustment module 36 is used to determine an optimal focusing current value based on the adjusted image clarity, and to adjust the focal length of the electro-hydraulic adjustable focus lens according to the optimal focusing current value.

[0184] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store scene images where dynamic targets are located. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for adjusting the focal length of an electro-hydraulic adjustable focus lens is implemented.

[0185] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0186] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0187] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0188] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0189] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0190] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0191] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0192] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for adjusting the focal length of an electro-hydraulic adjustable focus lens, characterized in that: The focal length adjustment method of the electro-hydraulic adjustable focus lens comprises: Acquire a scene image where a dynamic target is located through a camera, and select an area containing the dynamic target as a target area that needs to be focused; wherein the dynamic target is a movable target on the sea surface; Calculating the image clarity of the focus target area; When the image clarity is less than the image clarity threshold, calculating the focal length of the electro-hydraulic adjustable focus lens; Constructing a focal length adjustment model based on reinforcement learning, wherein the focal length adjustment model based on reinforcement learning includes a policy network, a target policy network, a value network, and a target value network, and determining the trained policy network as an electro-hydraulic adjustable focus lens control network model; Inputting the image clarity and the focal length into an electro-hydraulic adjustable focus lens control network model to obtain a focusing current value, and adjusting the focal length of the electro-hydraulic adjustable focus lens by the focusing current value, and determining the adjusted image clarity, so that the adjusted image clarity is greater than or equal to an image clarity threshold; An optimal focus current value is determined based on the adjusted image clarity, and the focal length of the electro-hydraulic focus-adjustable lens is adjusted according to the optimal focus current value.

2. The method for adjusting the focal length of an electro-hydraulic adjustable focus lens according to claim 1, characterized in that: Before obtaining the scene image where the dynamic target is located through the camera, it also includes: Constructing a target tracking model based on reinforcement learning, wherein the target tracking model based on reinforcement learning includes a tracking strategy network, a tracking target strategy network, a tracking value network, and a tracking target value network, and determining the trained tracking strategy network as the unmanned boat target tracking network model; Acquiring the distance between the dynamic target and the unmanned boat by using a radar on the unmanned boat; When the distance is greater than or equal to the control distance of the electro-hydraulic adjustable focus lens, the distance is adjusted using the unmanned boat target tracking network model; When the adjusted distance is less than the control distance of the electro-hydraulic adjustable focus lens, the unmanned boat is controlled to maintain the same speed and acceleration as the dynamic target, so that the unmanned boat and the dynamic target remain relatively still.

3. The focal length adjustment method of the electro-hydraulic adjustable focus lens according to claim 2, characterized in that: The distance is adjusted using the unmanned boat target tracking network model, including: Obtaining the state quantity at the current moment; the state quantity includes the speed of the dynamic target, the acceleration of the dynamic target, the speed of the unmanned boat, the acceleration of the unmanned boat, and the angle between the unmanned boat bow direction and the line connecting the unmanned boat and the dynamic target; The state quantity at the current moment is input into the unmanned boat target tracking network model to obtain the propeller control quantity of the unmanned boat, and the movement of the unmanned boat is controlled according to the propeller control quantity, thereby adjusting the distance so that the distance is less than the control distance of the electro-hydraulic adjustable focus lens.

4. The focal length adjustment method of the electro-hydraulic adjustable focus lens according to claim 3, characterized in that: The training process of the target tracking model based on reinforcement learning includes: Constructing an experience replay pool, wherein the experience replay pool stores a plurality of experience tuples, wherein the experience tuples include a state quantity at time k, an action quantity at time k, a state quantity at time k+1, and a reward, wherein the action quantity is a propeller control quantity; Sampling the experience tuples in the experience replay pool to obtain sampled experience tuples; Inputting the state quantity at time k+1 in the sampled experience tuple into the tracking target strategy network to obtain the action quantity at time k+1; Based on the target strategy smoothing regularization, the action amount at the k+1 moment is added with noise; the state amount at the k+1 moment and the action amount at the k+1 moment after adding noise are input into the tracking target value network to obtain the state value target value; Input the state quantity at time k and the action quantity at time k in the sampled experience tuple into the tracking value network to output the state value evaluation value; Minimize the error between the state value evaluation value and the state value target value using a gradient descent algorithm, and update the parameters of the tracking value network; Input the state quantity at time k in the sampled experience tuple into the tracking strategy network to obtain the predicted action quantity; input the state quantity at time k and the predicted action quantity into the tracking value network to obtain the predicted state value evaluation value; The predicted state value evaluation value is maximized by using a gradient ascent algorithm, the parameters in the tracking strategy network are updated, the parameters of the tracking target strategy network are updated according to the updated parameters of the tracking strategy network, and the parameters of the tracking target value network are updated according to the updated parameters of the tracking value network.

5. The method for adjusting the focal length of an electro-hydraulic adjustable focus lens according to claim 1, characterized in that: The training process of the focus adjustment model based on reinforcement learning includes: Construct an electro-hydraulic experience replay pool, the electro-hydraulic experience replay pool includes multiple electro-hydraulic experience tuples, the electro-hydraulic experience tuples include the electro-hydraulic state quantity at time G, the electro-hydraulic action quantity at time G, the electro-hydraulic state quantity at time G+1, and the electro-hydraulic reward; wherein the electro-hydraulic state quantity is image clarity and focal length, the image clarity includes overall image clarity, image edge clarity, and image contrast, and the electro-hydraulic action quantity is the focusing current value; Sampling the electro-hydraulic experience tuple in the electro-hydraulic experience playback pool to obtain a sampled electro-hydraulic experience tuple; Inputting the electro-hydraulic state quantity at time G+1 in the sampled electro-hydraulic experience tuple into the target strategy network to obtain the electro-hydraulic action quantity at time G+1; Based on the target strategy smoothing regularization, the electro-hydraulic action amount at the G+1 moment is added with noise; the electro-hydraulic state amount at the G+1 moment and the electro-hydraulic action amount at the G+1 moment after adding noise are input into the target value network to obtain the electro-hydraulic state value target value; Input the electro-hydraulic state quantity at time G and the electro-hydraulic action quantity at time G into the value network to output the electro-hydraulic state value evaluation value; Minimizing the error between the electro-hydraulic state value evaluation value and the electro-hydraulic state value target value by using a gradient descent algorithm, and updating the parameters of the value network; Inputting the electro-hydraulic state quantity at time G in the sampled electro-hydraulic experience tuple into the strategy network to obtain the predicted electro-hydraulic action quantity; Input the electro-hydraulic state quantity and the predicted electro-hydraulic action quantity at time G in the sampled experience tuple into the value network to obtain the predicted electro-hydraulic state value evaluation value; The predicted electro-hydraulic state value evaluation value is maximized by using a gradient ascent algorithm, the parameters of the strategy network are updated, the parameters of the target strategy network are updated according to the updated parameters of the strategy network, and the parameters of the target value network are updated according to the updated parameters of the value network.

6. The method for adjusting the focal length of an electro-hydraulic adjustable focus lens according to claim 5, characterized in that: The calculation formula of the electro-hydraulic reward is: Among them, r st For electro-hydraulic rewards, 11 ,ω 12 ,ω 13 ,ω 14 is a hyperparameter, are the image contrast, overall image clarity, image edge clarity and focal length at G-1, a is the focus target area, f t (a) are the image contrast, overall image clarity, image edge clarity and focal length at time G, respectively, and || || is the binary norm.

7. The method for adjusting the focal length of an electro-hydraulic adjustable focus lens according to claim 1, characterized in that: Determining an optimal focus current value based on the adjusted image clarity, and adjusting the focal length of the electro-hydraulic focus-adjustable lens according to the optimal focus current value, including: Adjusting the focal length in the direction of increasing image clarity according to a preset focusing current step length to obtain a re-adjusted focal length, and determining the re-adjusted image clarity; Comparing the image clarity after adjustment with the image clarity after re-adjustment, and determining the optimal focus current value according to the comparison result; The focal length of the electro-hydraulic adjustable focus lens is adjusted according to the optimal focusing current value.

8. A focal length adjustment device for an electro-hydraulic adjustable focus lens, characterized in that: The focal length adjustment system of the electro-hydraulic focus-adjustable lens comprises: A target area determination module is used to obtain a scene image where a dynamic target is located through a camera, and select an area containing the dynamic target as a target area that needs to be focused; wherein the dynamic target is a movable target on the sea surface; A first calculation module, used to calculate the image clarity of the focus target area; A second calculation module, used for calculating the focal length of the electro-hydraulic adjustable focus lens when the image clarity is less than the image clarity threshold; A model building module, used to build a focus adjustment model based on reinforcement learning, wherein the focus adjustment model based on reinforcement learning includes a policy network, a target policy network, a value network, and a target value network, and the trained policy network is determined as an electro-hydraulic adjustable focus lens control network model; A first focal length adjustment module, used for inputting the image clarity and the focal length into the electro-hydraulic adjustable focus lens control network model, obtaining a focusing current value, adjusting the focal length of the electro-hydraulic adjustable focus lens by the focusing current value, and determining the adjusted image clarity, so that the adjusted image clarity is greater than or equal to an image clarity threshold; The second focal length adjustment module is used to determine the optimal focusing current value based on the adjusted image clarity, and to adjust the focal length of the electro-hydraulic adjustable focus lens according to the optimal focusing current value.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method for adjusting the focal length of the electro-hydraulic focus-adjustable lens according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for adjusting the focal length of the electro-hydraulic focus-adjustable lens according to any one of claims 1 to 7 are implemented.

Images