A precise movement correction control system and method for an unmanned workboat

Abstract

This invention relates to a precise motion correction control system and method for an unmanned workboat, belonging to the field of unmanned vessels. The system comprises a workboat hull, an accelerometer, a spirit level, and a central processing unit installed on the hull. The method comprises the following steps: S1: initialization; S2: receiving client instructions to perform operations; S3: real-time sensing of the hull state using the installed accelerometer, spirit level, and pressure sensor; S4: force analysis of the hull based on empirical formulas and the sensed hull state, calculating the theoretical action of the water pump; S5: selecting a candidate experience pool that meets the error limit requirements using a KNN network; S6: calculating the optimal action using a DQN network; S7: sending corresponding control instructions to the water pump according to the optimal action and executing them; S8: repeating steps S3 to S7 until the operation is completed. This invention can effectively reduce the action solution space, reduce the complexity of the solution, and improve control accuracy, thereby enhancing the adaptability of unmanned workboats to complex environments.

Description

Technical Field

[0001] The present invention relates to a precise movement correction control system and method for an unmanned work boat, belongs to the field of unmanned ships, and is particularly suitable for precise movement correction control of an unmanned work boat. Background Art

[0002] With the continuous advancement of technology, unmanned vessels (UAVs) have demonstrated broad application potential across multiple industries, particularly in marine resource exploration, environmental monitoring, marine scientific research, and even the military. Compared to traditional human-controlled vessels, UAVs can operate in extreme or hazardous environments, significantly improving operational safety and economic efficiency.

[0003] The development of unmanned vessels is currently experiencing rapid growth. These vessels are becoming increasingly intelligent, assisted by advanced propulsion systems, navigation technologies, communications equipment, and artificial intelligence. For example, using advanced sensors and GPS systems, unmanned vessels can self-locate and navigate autonomously. Furthermore, advancements in machine learning and data analytics enable them to more efficiently process complex data and make operational decisions.

[0004] Despite a series of progress, there are still many challenges in the research, development and application of unmanned operation vessels. One of the most core issues is how to achieve precise speed control in complex marine environments. Precise speed control is the key to ensuring that unmanned operation vessels can effectively complete tasks, such as synchronous measurement, docking operations and other complex operations. The hydrological and meteorological conditions in different sea areas vary significantly, and existing technologies often cannot meet the needs. In summary, although unmanned operation vessel technology has developed rapidly, there is still room for further improvement in high-precision power control technology. Future research work needs to optimize the response speed, accuracy and adaptability of precise control systems to complex environments to meet the growing application needs and improve the performance of unmanned operation vessels in various operation scenarios. Technological breakthroughs in this area will directly affect the development prospects of the unmanned operation vessel industry and promote its application in a wider range. Summary of the Invention

[0005] In view of this, the present invention provides a precise movement correction control system and method for an unmanned work vessel. By utilizing theoretical force analysis and combining it with error analysis, the solution space of the unmanned work vessel's control action is constrained to reduce the complexity of the solution and improve the control accuracy. At the same time, reinforcement learning is used to improve adaptability in complex environments.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A precise motion correction control system for an unmanned workboat comprises a workboat hull and an acceleration sensor, a spirit level, and a central processing unit mounted on the hull; the acceleration sensor and spirit level are respectively connected to the central processing unit; six controllable water outlets are horizontally symmetrically and evenly distributed on both sides of the workboat hull; each controllable water outlet comprises an independently controllable water pump and a pressure sensor mounted on the water outlet; the water pump and pressure sensor are connected to the central processing unit.

[0008] Preferably, the water pump can also calculate the pressure the water pump is subjected to by measuring the back electromotive force, thereby replacing the function of the pressure sensor.

[0009] A precise movement correction control method for an unmanned workboat comprises the following steps:

[0010] S1: Initialize the parameters and error limits of the reinforcement learning network (Deep Q-Network, DQN), experience pool (Reply Buffer), and K-Nearest Neighbor (KNN) network in the central processing unit of the unmanned operation vessel;

[0011] S2: The unmanned operation vessel receives instructions from the client and performs operations;

[0012] S3: Detects the hull status in real time through the installed acceleration sensor, level gauge and pressure sensor, and sends it to the central processing unit;

[0013] S4: The central processing unit performs a force analysis on the hull based on empirical formulas and the sensed hull state, and calculates the theoretical action of the water pump;

[0014] S5: The central processing unit uses the KNN network to select a subset that meets the error limit requirements from the experience pool as a candidate experience pool;

[0015] S6: The central processing unit uses the DQN network to calculate the optimal action based on the real-time perception of the hull state and the candidate experience pool;

[0016] S7: The central processing unit sends corresponding control instructions to the water pump according to the optimal action and executes them;

[0017] S8: Repeat steps S3 to S7 until the job is completed.

[0018] Furthermore, the reinforcement learning network is composed of a tuple containing three elements (S, A, R), where R is a reward function; the hull state is represented as a state space S; the pressure adjustment amounts of all water pumps are regarded as an action space A; and the KNN network is a binary classification network.

[0019] Furthermore, the error limit in step S1 is a relative error ratio.

[0020] Furthermore, the state of the reinforcement learning network is the hull state The reinforcement learning action is a=[Δf i ] i＝1,,6 ; Among them, Oxyz is the Cartesian coordinate system established for the hull of the unmanned operation ship; a x is the acceleration of the unmanned operation ship in the x direction, a y 、a z Similarly, it can be directly measured by an acceleration sensor; θ is the azimuth angle of the unmanned operating ship in the xoy plane and the x direction, θ∈[-π,π], which can be directly measured by a level; is the polar angle of the unmanned operating vessel in the z direction, It can be directly measured by a level; f1,…,f8 are the pressures of the six controllable water outlets, which can be directly measured by a pressure sensor; Δf i is the pressure adjustment of the i-th water pump.

[0021] Furthermore, the step S4 is specifically as follows:

[0022] S401: Based on the opening unit direction vector of the controllable water outlet Calculate the combined force of the water pump on the unmanned operating vessel Among them, the unit direction vector of the opening is is the design parameter, is a known quantity;

[0023] S402: Based on the distance l between the i-th controllable water outlet and the geometric center of the hull i , establish the torque balance constraint: in,

[0024] S403: According to the force balance: F z -ma z =0; where is the propulsion acceleration of the unmanned operation vessel in the x-direction and y-direction when performing operations, which is a known quantity; m is the mass of the unmanned operation vessel, which is also a known quantity;

[0025] S404: Solve the equations of step S402 and step S403 to calculate the theoretical action of the water pump in, is the theoretical pressure adjustment of the i-th water pump.

[0026] Preferably, since the mass of the unmanned work boat may not be fixed, the mass of the unmanned work boat can be indirectly calculated by a depth sensor, and the buoyancy is calculated by establishing a relationship between the draft depth and displacement volume of the unmanned work boat, and then the mass is calculated; the depth sensor is installed on the hull and connected to the central processing unit.

[0027] Furthermore, the step S5 is specifically as follows:

[0028] S501: The central processing unit uses the KNN network to randomly select M state-action transition pairs (s t ,a t ,r t ,s t+1 ); where s t is the state at time t, s t+1 is the state at time t+1, a t is the action at time t, r t is the reward at time t;

[0029] S502: According to the state error limit And the action error limit state-action transfer pairs, and use the M state-action transfer pairs in the KNN network to select the state-action transfer pairs that meet the conditions as the candidate experience pool.

[0030] Preferably, the upper limit parameters of the state error limit and the action error limit are different.

[0031] The beneficial effects of the present invention are as follows: the present invention provides a precise movement correction control system and method for an unmanned work boat, which realizes the posture vector control of the unmanned work boat by installing a controllable water outlet on the unmanned work boat; at the same time, a K-nearest neighbor candidate experience pool is established in combination with dynamic information, which can quickly and accurately select actions, and then assist the reinforcement learning network to make timely and accurate feedback, thereby realizing precise control of the unmanned work boat. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to illustrate the purpose and technical solution of the present invention, the present invention provides the following drawings:

[0033] Figure 1 This is a framework diagram of a precise motion correction control system for an unmanned workboat according to embodiment 1 of the present invention; the arrows indicate the direction of signal data transmission; 1 is the hull, 11 is the controllable water outlet, 2 is the acceleration sensor, 3 is the level, 4 is the central processing unit, and 5 is the depth sensor;

[0034] Figure 2Schematic diagram of the controllable water outlet of the unmanned work boat according to embodiment 1 of the present invention; wherein 1 is the hull and 11 is the controllable water outlet;

[0035] Figure 3 This is a flow chart of Example 2 of the present invention;

[0036] Figure 4 This is a reinforcement learning network diagram of Example 2 of the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose and technical solution of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0038] Example 1: Monitoring and sampling of blue algae blooms in waters with strong winds and waves. In order to reduce labor costs and realize automated sampling, an unmanned operation vessel is required to complete the operation.

[0039] In order to ensure the smoothness of the operation process and the accuracy of control, the present invention provides a "precision movement correction control system for unmanned operation vessels". Figure 1 , consisting of a workboat hull (1) and an acceleration sensor (2), a level (3), and a central processing unit (4) installed on the hull (1); the acceleration sensor (2) and the level (3) are respectively connected to the central processing unit (4); the acceleration sensor (2) is a gyroscope; six controllable water outlets (11) are evenly and horizontally distributed on both sides of the workboat hull (1), such as Figure 2 As shown; each of the controllable water outlets (11) is equipped with an independently controllable water pump (111) and a pressure sensor (112); the water pump (111) and the pressure sensor (112) are connected to the central processing unit (4).

[0040] When the pressure sensor (112) is abnormal, the water pump (111) can also calculate the pressure the water pump withstands by measuring the back electromotive force, thereby replacing the function of the pressure sensor (112).

[0041] Since the mass of the unmanned operation boat is unknown, a depth sensor (5) is installed on the hull and connected to the central processing unit (4). The mass of the unmanned operation boat can be indirectly calculated through the depth sensor (5). By establishing an interpolation function by establishing the relationship between the draft depth and displacement volume of the unmanned operation boat, the buoyancy and mass can be calculated.

[0042] Example 2: In view of the scenario and system of Example 1, the present invention provides a "precision movement correction control method for an unmanned work vessel". Figure 2 , which includes the following steps:

[0043] S1: Initialize the reinforcement learning network, experience pool, parameters of the K nearest neighbor network, and error limits in the central processing unit (4) of the unmanned operation ship.

[0044] The reinforcement learning network comprises a tuple of three elements (S, A, R), where R is a reward function; the hull state is represented as a state space S; and the pressure adjustment values ​​of all water pumps are regarded as an action space A.

[0045] The reinforcement learning network is a DDPG deep reinforcement learning network based on the Actor-Critic framework, which consists of an actor network μ(s|θ μ ), critic network Q(s,a|θ Q ), actor target network μ′(s|θ μ′ ), critic target network Q′(s,a|θ Q′ ) is composed of; where: θ μ ,θ Q ,θ μ′ ,θ Q′ are the weight coefficients of the four networks respectively; Markov Decision Process (MDP) is used as the gradient strategy.

[0046] The state of the reinforcement learning network is the hull state The reinforcement learning action is a=[Δf i ] i＝1,,6 ∈A; where Oxyz is the Cartesian coordinate system established for the unmanned operation vessel; a x is the acceleration of the unmanned operation ship in the x direction, a y 、a z Similarly, it can be directly measured by an acceleration sensor; θ is the azimuth angle of the unmanned operating ship in the xoy plane and the x direction, θ∈[-π,π], which can be directly measured by a level; is the polar angle of the unmanned operating vessel in the z direction, It can be directly measured by a level; f1,…,f8 are the pressures of the six controllable water outlets, which can be directly measured by a pressure sensor; Δf i is the pressure adjustment of the i-th water pump.

[0047] The KNN network is a two-class network; the experience pool is a historical state-action transfer pair (s t ,a t ,r t ,s t+1 ).

[0048] S2: The unmanned operation vessel receives instructions from the client and performs operations.

[0049] S3: The hull status is sensed in real time through the installed acceleration sensor (2), level meter (3) and pressure sensor (112), and sent to the central processing unit (4).

[0050] S4: The central processing unit (4) performs a force analysis on the hull (1) based on the empirical formula and the sensed hull state, and calculates the theoretical action of the water pump (111). Specifically:

[0051] S401: According to the opening unit direction vector of the controllable water outlet (11) Calculate the resultant force of the water pump (111) on the unmanned operation boat Among them, the unit direction vector of the opening is is the design parameter, is a known quantity;

[0052] S402: Based on the distance l between the i-th controllable water outlet (11) and the geometric center of the hull (1) i , establish the torque balance constraint: in,

[0053] S403: According to the force balance: F z -ma z =0; where is the propulsion acceleration of the unmanned operation vessel in the x-direction and y-direction when performing operations, which is a known quantity; m is the mass of the unmanned operation vessel, which is also a known quantity;

[0054] S404: Solve the equations of step S402 and step S403 to calculate the theoretical action of the water pump (111) in, is the theoretical pressure adjustment of the i-th water pump (111).

[0055] S5: The central processing unit (4) uses the KNN network to select a subset that meets the error limit requirements from the experience pool as a candidate experience pool. Specifically:

[0056] S501: The central processing unit (4) uses the KNN network to randomly select M state-action transition pairs (s t ,a t ,r t ,s t+1 ); where s t is the state at time t, s t+1 is the state at time t+1, a t is the action at time t, r tis the reward at time t;

[0057] S502: According to the state error limit And the action error limit The state-action transition pairs in the KNN network are used to select the state-action transition pairs that meet the conditions as the candidate experience pool. Usually, ε1 and ε2 should be set to less than 30%.

[0058] S6: The central processing unit (4) uses the DQN network to calculate the optimal action based on the real-time perception of the hull state and the candidate experience pool. Specifically:

[0059] S601: Using loss function Update the critic network Q(s,a|θ) Q ) parameter θ Q ; Among them, y t =r t +λQ′(s t+1 ,μ′(s t+1 |θ μ′ )|θ Q′ ), N is the number of state-action transfer pairs, λ is a hyperparameter, and t is the corresponding current moment s in the candidate experience pool t The subscript of the state-action transition pair;

[0060] S602: Update the actor network μ(s|θ using the gradient strategy μ )’s network parameters θ μ ;

[0061] S603: Update the actor target network μ′(s|θ μ′ ), critic target network Q′(s,a|θ Q′ )’s network parameters θ μ′ ←τθ μ +(1-τ)θ μ′ ,θ Q′ ←τθ Q +(1-τ)θ Q′ ; where τ is a hyperparameter.

[0062] S604: Using the updated actor network μ(s|θ μ ) calculates the optimal action.

[0063] S7: The central processing unit (4) sends corresponding control instructions to the water pump (111) according to the optimal action and executes them.

[0064] S8: Repeat steps S3 to S7 until the job is completed.

[0065] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made in form and details without departing from the scope defined by the claims of the present invention.

Claims

1. A precise movement correction control method for an unmanned workboat, characterized in that: The method comprises the following steps: S1: Initialize the parameters and error limits of the reinforcement learning network (Deep Q-Network, DQN), experience pool (Reply Buffer), and K-Nearest Neighbor (KNN) network in the central processing unit (4) of the unmanned operation ship; S2: The unmanned operation vessel receives instructions from the client and performs operations; S3: The hull status is sensed in real time through the installed acceleration sensor (2), level meter (3) and pressure sensor (112), and sent to the central processing unit (4); S4: The central processing unit (4) performs a force analysis on the hull (1) based on an empirical formula and the sensed hull state, and calculates the theoretical action of the water pump (111); S5: The central processing unit (4) uses the KNN network to select a subset that meets the error limit requirements from the experience pool as a candidate experience pool; S6: The central processing unit (4) uses the DQN network to calculate the optimal action based on the real-time perception of the hull state and the candidate experience pool; S7: The central processing unit (4) sends corresponding control instructions to the water pump (111) according to the optimal action and executes them; S8: Repeat steps S3 to S7 until the operation is completed; The reinforcement learning network is composed of a tuple containing three elements (S, A, R), wherein R is a reward function; the hull state is represented as a state space S; the pressure adjustment amounts of all water pumps (111) are regarded as an action space A; the KNN network is a binary classification network; The state of the reinforcement learning network is the hull state The reinforcement learning action is a=[△f i ] i＝1,…,6 ; Among them, Oxyz is the Cartesian coordinate system established for the hull of the unmanned operation ship; a x is the acceleration of the unmanned operation ship in the x direction, a y 、a z Similarly, it is directly measured by the acceleration sensor (2); θ is the azimuth angle of the unmanned operating ship in the xoy plane and the x direction, θ∈[-π,π], which is directly measured by the level meter (3); is the polar angle of the unmanned operating vessel in the z direction, is directly measured by the level meter (3); f1, ..., f8 are the pressures of the six controllable water outlets (11), which are directly measured by the pressure sensor (112); △f i is the pressure adjustment amount of the i-th water pump (111); The error limit in step S1 is a relative error ratio. The step S4 is specifically as follows: S401: According to the opening unit direction vector of the controllable water outlet (11) Calculate the resultant force of the water pump (111) on the unmanned operation boat Among them, the unit direction vector of the opening is is the design parameter, is a known quantity; S402: Based on the distance l between the i-th controllable water outlet (11) and the geometric center of the hull (1) i , establish the torque balance constraint: in, S403: According to the force balance: |F| z -ma z =0; where is the propulsion acceleration of the unmanned operation vessel in the x-direction and y-direction when performing operations, which is a known quantity; m is the mass of the unmanned operation vessel, which is also a known quantity; S404: Solve the equations of step S402 and step S403 to calculate the theoretical action of the water pump (111) in, is the theoretical pressure adjustment of the i-th water pump (111).

2. The precise movement correction control method of an unmanned workboat according to claim 1, characterized in that: The step S5 is specifically as follows: S501: The central processing unit (4) uses the KNN network to randomly select M state-action transition pairs (s t ,a t ,r t ,s t+1 ); where s t is the state at time t, s t+1 is the state at time t+1, a t is the action at time t, r t is the reward at time t; S502: According to the state error limit And the action error limit state-action transfer pairs, and use the M state-action transfer pairs in the KNN network to select the state-action transfer pairs that meet the conditions as the candidate experience pool.

3. The precise movement correction control method of an unmanned workboat according to claim 2, characterized in that: The upper bound parameters of the state error limit and the action error limit are different.

4. A precise movement correction control system for an unmanned work vessel applied to a precise movement correction control method for an unmanned work vessel according to any one of claims 1 to 3, characterized in that: The invention comprises a workboat hull (1), an acceleration sensor (2), a level gauge (3), and a central processing unit (4) installed on the hull (1); the acceleration sensor (2) and the level gauge (3) are respectively connected to the central processing unit (4); six controllable water outlets (11) are horizontally symmetrically and evenly distributed on both sides of the workboat hull (1); each of the controllable water outlets (11) is equipped with an independently controllable water pump (111) and a pressure sensor (112); The water pump (111) and the pressure sensor (112) are connected to the central processing unit (4).

5. The precise movement correction control system for an unmanned work vessel according to claim 4, characterized in that: The water pump (111) also calculates the pressure the water pump (111) bears by measuring the back electromotive force, thereby replacing the function of the pressure sensor (112).

6. The precise movement correction control system for an unmanned work vessel according to claim 4, characterized in that: Since the mass of the unmanned operation boat may not be fixed, the mass of the unmanned operation boat is indirectly calculated through a depth sensor (5). The buoyancy is calculated by establishing a relationship between the draft and displacement volume of the unmanned operation boat, and then the mass is calculated. The depth sensor (5) is installed on the hull (1) and is connected to the central processing unit (4).

Images