Accurate movement correction control system and method for unmanned workboat

Abstract

The invention relates to a precise movement correction control system and method for an unmanned workboat, and belongs to the field of unmanned boats. The system is composed of a workboat body, an acceleration sensor, a gradienter and a central processing unit, the method comprises the following steps: S1, initializing; s2, receiving a client instruction to perform operation; s3, the state of the ship body is sensed in real time through an installed acceleration sensor, a gradienter and a pressure sensor; s4, carrying out stress analysis on the ship body according to an empirical formula and the sensed ship body state, and calculating a theoretical action of the water pump; s5, selecting a candidate experience pool meeting an error limit requirement by using a KNN network; s6, calculating an optimal action by using a DQN network; s7, a corresponding control instruction is sent to the water pump according to the optimal action and executed; and S8, repeating the steps S3 to S7 until the operation is completed. According to the method, the action solution space can be effectively reduced, the solution complexity is reduced, the control precision is improved, and the adaptability of the unmanned workboat to the complex environment is improved.

Description

Technical Field

[0001] The present invention relates to a precise movement correction control system and method for an unmanned operating vessel, belongs to the field of unmanned vessels, and is particularly suitable for precise movement correction control of an unmanned operating vessel. Background Art

[0002] With the continuous advancement of science and technology, unmanned operation ships have shown wide application potential in many industries, especially in marine resource exploration, environmental monitoring, marine scientific research and even military fields. Compared with traditional manually operated ships, unmanned operation ships can operate in extreme or dangerous environments, greatly improving operational safety and economic benefits.

[0003] Currently, the development of unmanned ships is in a rapid growth stage. With the assistance of power systems, navigation technology, communication equipment and artificial intelligence, these unmanned ships are becoming more and more intelligent. For example, by using advanced sensors and GPS systems, unmanned ships can self-position and navigate autonomously. In addition, with the help of advances in machine learning and data analysis technology, unmanned ships can process complex data more efficiently and make operational decisions.

[0004] Despite a series of progress, there are still many challenges in the research and development and application of unmanned ships. One of the most core issues is how to achieve precise speed control in complex marine environments. Precise speed control is the key to ensuring that unmanned ships can effectively complete tasks, such as complex operations such as synchronous measurement and docking operations. The hydrological and meteorological conditions in different sea areas vary significantly, and existing technologies often cannot meet the needs. In summary, although the technology of unmanned ships has developed rapidly, there is still room for further improvement in high-precision power control technology. Future research work needs to optimize the response speed, accuracy and adaptability of precise control systems to complex environments to meet the growing application needs and improve the performance of unmanned ships in various operating scenarios. Technological breakthroughs in this area will directly affect the development prospects of the unmanned ship industry and promote its application in a wider range. Summary of the invention

[0005] In view of this, the present invention provides a precise movement correction control system and method for an unmanned work boat. By utilizing theoretical force analysis and combining it with error analysis, the solution space of the control action of the unmanned work boat is constrained to reduce the complexity of the solution and improve the control accuracy. At the same time, reinforcement learning is used to improve the adaptability in complex environments.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A precise movement correction control system for an unmanned work boat, comprising a work boat hull and an acceleration sensor, a level meter, and a central processing unit installed on the hull; the acceleration sensor and the level meter are respectively connected to the central processing unit; six controllable water outlets are horizontally symmetrically and evenly distributed on both sides of the work boat hull; each of the controllable water outlets is equipped with an independently controllable water pump and a pressure sensor; the water pump and the pressure sensor are connected to the central processing unit.

[0008] Preferably, the water pump can also calculate the pressure the water pump is subjected to by measuring the back electromotive force, thereby replacing the function of the pressure sensor.

[0009] A precise movement correction control method for an unmanned operating vessel comprises the following steps:

[0010] S1: Initialize the parameters and error limits of the reinforcement learning network (Deep Q-Network, DQN), experience pool (Reply Buffer), and K-Nearest Neighbor (KNN) network in the central processing unit of the unmanned operation ship;

[0011] S2: The unmanned operation ship receives instructions from the client to perform operations;

[0012] S3: The hull status is sensed in real time through the installed acceleration sensor, level gauge and pressure sensor, and sent to the central processing unit;

[0013] S4: The central processing unit performs a force analysis on the hull based on the empirical formula and the sensed hull state, and calculates the theoretical action of the water pump;

[0014] S5: The central processing unit uses the KNN network to select a subset that meets the error limit requirements from the experience pool as a candidate experience pool;

[0015] S6: The central processing unit uses the DQN network to calculate the optimal action based on the real-time perception of the hull state and the candidate experience pool;

[0016] S7: The central processing unit sends corresponding control instructions to the water pump according to the optimal action and executes them;

[0017] S8: Repeat steps S3 to S7 until the operation is completed.

[0018] Furthermore, the reinforcement learning network is composed of a tuple containing three elements (S, A, R), where R is a reward function; the hull state is represented as a state space S; the pressure adjustment amounts of all water pumps are regarded as an action space A; and the KNN network is a binary classification network.

[0019] Furthermore, the error limit in step S1 is a relative error ratio.

[0020] Furthermore, the state of the reinforcement learning network is the hull state The reinforcement learning action is a = [Δf i ] i＝1,,6 ; Oxyz is the Cartesian coordinate system established for the hull of the unmanned operation ship; a x is the acceleration of the unmanned operating ship in the x direction, a y 、a z Similarly, it can be directly measured by an acceleration sensor; θ is the azimuth angle of the unmanned operating ship in the xoy plane and the x direction, θ∈[-π,π], which can be directly measured by a level meter; is the polar angle of the unmanned operating ship in the z direction, It can be directly measured by a level meter; f1,…,f8 are the pressures of the six controllable water outlets, which can be directly measured by a pressure sensor; Δf i is the pressure adjustment of the i-th water pump.

[0021] Further, the step S4 is specifically as follows:

[0022] S401: According to the opening unit direction vector of the controllable water outlet Calculate the combined force of the water pump on the unmanned operating vessel Among them, the opening unit direction vector is the design parameter, is a known quantity;

[0023] S402: Based on the distance l between the ith controllable water outlet and the geometric center of the hull i , establish the torque balance constraint: in,

[0024] S403: According to the force balance: F z -ma z =0; where is the propulsion acceleration of the unmanned operation ship in the x-direction and y-direction when performing operations, which is a known quantity; m is the mass of the unmanned operation ship, which is a known quantity;

[0025] S404: Solve the equations of step S402 and step S403 to calculate the theoretical action of the water pump in, is the theoretical pressure adjustment of the ith water pump.

[0026] Preferably, since the mass of the unmanned operating ship may not be fixed, the mass of the unmanned operating ship can be indirectly calculated through a depth sensor, and the buoyancy is calculated by establishing a relationship between the draft and displacement volume of the unmanned operating ship, and then the mass is calculated; the depth sensor is installed on the hull and connected to the central processing unit.

[0027] Further, the step S5 is specifically as follows:

[0028] S501: The central processing unit uses the KNN network to randomly select M state-action transition pairs (s t ,a t ,r t ,s t+1 ), where s t is the state at time t, s t+1 is the state at time t+1, a t is the action at time t, r t is the reward at time t;

[0029] S502: According to the state error limit And the action error limit The state-action transfer pairs in the KNN network are used to select the state-action transfer pairs that meet the conditions as the candidate experience pool.

[0030] Preferably, the upper limit parameters of the state error limit and the action error limit are different.

[0031] The beneficial effects of the present invention are as follows: the present invention provides a precise movement correction control system and method for an unmanned work boat, which realizes the posture vector control of the unmanned work boat by installing a controllable water outlet on the unmanned work boat; at the same time, a K nearest neighbor candidate experience pool is established in combination with dynamic information, which can quickly and accurately select actions, thereby assisting the reinforcement learning network to make timely and accurate feedback, thereby realizing precise control of the unmanned work boat. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to make the purpose and technical solution of the present invention, the present invention provides the following drawings for explanation:

[0033] Figure 1 This is a framework diagram of a precise movement correction control system for an unmanned working boat according to Embodiment 1 of the present invention; wherein the direction of the arrow is the direction of signal data transmission; wherein 1 is the hull, 11 is a controllable water outlet, 2 is an acceleration sensor, 3 is a level meter, 4 is a central processing unit, and 5 is a depth sensor;

[0034] Figure 2Schematic diagram of a controllable water outlet of an unmanned working boat according to Embodiment 1 of the present invention; wherein 1 is a hull, and 11 is a controllable water outlet;

[0035] Figure 3 This is a flow chart of Embodiment 2 of the present invention;

[0036] Figure 4 This is a reinforcement learning network diagram of Example 2 of the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose and technical solution of the present invention more clearly understood, the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0038] Example 1: Monitoring and sampling of blue algae blooms in waters with strong winds and waves. In order to reduce labor costs and realize automated sampling, an unmanned operation ship is required to complete the operation.

[0039] In order to ensure the smoothness of the operation process and the accuracy of control, the present invention provides "a precise movement correction control system for an unmanned operation ship". Figure 1 The invention comprises a work boat hull (1), an acceleration sensor (2), a level gauge (3), and a central processing unit (4) installed on the hull (1); the acceleration sensor (2) and the level gauge (3) are respectively connected to the central processing unit (4); the acceleration sensor (2) is a gyroscope; six controllable water outlets (11) are evenly and horizontally distributed on both sides of the work boat hull (1), such as Figure 2 As shown; each of the controllable water outlets (11) is equipped with an independently controllable water pump (111) and a pressure sensor (112); the water pump (111) and the pressure sensor (112) are connected to the central processing unit (4).

[0040] When the pressure sensor (112) is abnormal, the water pump (111) can also calculate the pressure the water pump is subjected to by measuring the back electromotive force, thereby replacing the function of the pressure sensor (112).

[0041] Since the mass of the unmanned operation boat is unknown, a depth sensor (5) is installed on the hull and connected to the central processing unit (4). The mass of the unmanned operation boat can be indirectly calculated through the depth sensor (5). By establishing the relationship between the draft depth and displacement volume of the unmanned operation boat, an interpolation function is established, and the buoyancy and mass are then calculated.

[0042] Embodiment 2: In view of the scenario and system of Embodiment 1, the present invention provides "a precise movement correction control method for an unmanned working ship". Figure 2 , including the following steps:

[0043] S1: Initialize the reinforcement learning network, experience pool, parameters of the K nearest neighbor network, and error limits in the central processing unit (4) of the unmanned operation ship.

[0044] The reinforcement learning network comprises a tuple of three elements (S, A, R), wherein R is a reward function; the hull state is represented as a state space S; and the pressure adjustment amounts of all water pumps are regarded as an action space A.

[0045] The reinforcement learning network is a DDPG deep reinforcement learning network based on the Actor-Critic framework, which consists of an actor network μ(s|θ μ ), critic network Q(s,a|θ Q ), actor target network μ′(s|θ μ′ ), critic target network Q′(s,a|θ Q′ ); where: θ μ ,θ Q ,θ μ′ ,θ Q′ are the weight coefficients of the four networks respectively; Markov Decision Process (MDP) is used as the gradient strategy.

[0046] The state of the reinforcement learning network is the hull state The reinforcement learning action is a = [Δf i ] i＝1,,6 ∈A; Oxyz is the Cartesian coordinate system established for the hull of the unmanned operation ship; a x is the acceleration of the unmanned operating ship in the x direction, a y 、a z Similarly, it can be directly measured by an acceleration sensor; θ is the azimuth angle of the unmanned operating ship in the xoy plane and the x direction, θ∈[-π,π], which can be directly measured by a level meter; is the polar angle of the unmanned operating ship in the z direction, It can be directly measured by a level meter; f1,…,f8 are the pressures of the six controllable water outlets, which can be directly measured by a pressure sensor; Δf i is the pressure adjustment of the i-th water pump.

[0047] The KNN network is a binary classification network; the experience pool is a historical state-action transfer pair (s t ,a t ,r t ,s t+1 ).

[0048] S2: The unmanned operation vessel receives instructions from the client and performs operations.

[0049] S3: The hull status is sensed in real time through the installed acceleration sensor (2), level meter (3) and pressure sensor (112), and sent to the central processing unit (4).

[0050] S4: The central processing unit (4) performs a force analysis on the hull (1) according to the empirical formula and the sensed hull state, and calculates the theoretical action of the water pump (111). Specifically:

[0051] S401: According to the opening unit direction vector of the controllable water outlet hole (11), Calculate the combined force of the water pump (111) on the unmanned operation boat Among them, the opening unit direction vector is the design parameter, is a known quantity;

[0052] S402: Based on the distance l between the i-th controllable water outlet (11) and the geometric center of the hull (1), i , establish the torque balance constraint: in,

[0053] S403: According to the force balance: F z -ma z =0; where is the propulsion acceleration of the unmanned operation ship in the x-direction and y-direction when performing operations, which is a known quantity; m is the mass of the unmanned operation ship, which is a known quantity;

[0054] S404: Solve the equations of step S402 and step S403 to calculate the theoretical action of the water pump (111) in, is the theoretical pressure adjustment value of the i-th water pump (111).

[0055] S5: The central processing unit (4) uses the KNN network to select a subset that meets the error limit requirements from the experience pool as a candidate experience pool. Specifically:

[0056] S501: The central processing unit (4) uses the KNN network to randomly select M state-action transition pairs (s t ,a t ,r t ,s t+1 ), where s t is the state at time t, s t+1 is the state at time t+1, a t is the action at time t, r tis the reward at time t;

[0057] S502: According to the state error limit And the action error limit The state-action transfer pairs in the KNN network are used to select the state-action transfer pairs that meet the conditions as the candidate experience pool. Usually, ε1 and ε2 should be set to less than 30%.

[0058] S6: The central processing unit (4) uses the DQN network to calculate the optimal action based on the real-time perception of the ship state and the candidate experience pool. Specifically:

[0059] S601: Using loss function Update the critic network Q(s,a|θ Q ) parameter θ Q ; Among them, y t =r t +λQ′(s t+1 ,μ′(s t+1 |θ μ′ )|θ Q′ ), N is the number of state-action transition pairs, λ is a hyperparameter, and t is the corresponding current time s in the candidate experience pool t The subscript of the state-action transition pair;

[0060] S602: Update the actor network μ(s|θ) using the gradient strategy μ )’s network parameters θ μ ;

[0061] S603: Update actor target network μ′(s|θ μ′ ), critic target network Q′(s,a|θ Q′ )’s network parameters θ μ′ ←τθ μ +(1-τ)θ μ′ ,θ Q′ ←τθ Q +(1-τ)θ Q′ ; where τ is a hyperparameter.

[0062] S604: Using the updated actor network μ(s|θ μ ) calculates the optimal action.

[0063] S7: The central processing unit (4) sends corresponding control instructions to the water pump (111) according to the optimal action and executes them.

[0064] S8: Repeat steps S3 to S7 until the operation is completed.

[0065] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made in form and details without departing from the scope defined by the claims of the present invention.

Claims

1. A precise movement correction control system for an unmanned working vessel, characterized in that: The invention comprises a working boat hull (1), an acceleration sensor (2), a level gauge (3), and a central processing unit (4) installed on the hull (1); the acceleration sensor (2) and the level gauge (3) are respectively connected to the central processing unit (4); six controllable water outlets (11) are horizontally symmetrically and evenly distributed on both sides of the working boat hull (1); each of the controllable water outlets (11) is equipped with an independently controllable water pump (111) and a pressure sensor (112); the water pump (111) and the pressure sensor (112) are connected to the central processing unit (4).

2. The precise movement correction control system of an unmanned working vessel according to claim 1, characterized in that: The water pump (111) can also calculate the pressure the water pump (111) is subjected to by measuring the back electromotive force, thereby replacing the function of the pressure sensor (112).

3. The precise movement correction control system of an unmanned working vessel according to claim 1, characterized in that: Since the mass of the unmanned operating boat may not be fixed, the mass of the unmanned operating boat can be indirectly calculated by a depth sensor (5). The buoyancy is calculated by establishing a relationship between the draft and displacement volume of the unmanned operating boat, and then the mass is calculated. The depth sensor (5) is installed on the hull (1) and is connected to the central processing unit (4).

4. A precise movement correction control method for an unmanned work vessel according to any one of claims 1 to 3, characterized in that: The method comprises the following steps: S1: Initialize the parameters and error limits of the reinforcement learning network (Deep Q-Network, DQN for short), the experience pool (Reply Buffer), and the K-Nearest Neighbor (KNN) network in the central processing unit (4) of the unmanned operation ship; S2: The unmanned operation ship receives instructions from the client to perform operations; S3: The hull state is sensed in real time through the installed acceleration sensor (2), level meter (3) and pressure sensor (112), and sent to the central processing unit (4); S4: the central processing unit (4) performs a force analysis on the hull (1) according to an empirical formula and the sensed hull state, and calculates the theoretical action of the water pump (111); S5: The central processing unit (4) uses the KNN network to select a subset that meets the error limit requirement from the experience pool as a candidate experience pool; S6: The central processing unit (4) uses the DQN network to calculate the optimal action based on the real-time perception of the hull state and the candidate experience pool; S7: the central processing unit (4) sends corresponding control instructions to the water pump (111) according to the optimal action and executes them; S8: Repeat steps S3 to S7 until the operation is completed; The reinforcement learning network is composed of a tuple containing three elements (S, A, R), wherein R is a reward function; the hull state is represented as a state space S; the pressure adjustment amounts of all water pumps (111) are regarded as an action space A; the KNN network is a binary classification network; The state of the reinforcement learning network is the hull state The reinforcement learning action is a = [Δf i ] i＝1,,6 ; Oxyz is the Cartesian coordinate system established for the hull of the unmanned operation ship; a x is the acceleration of the unmanned operating ship in the x direction, a y 、a z Similarly, it can be directly measured by the acceleration sensor (2); θ is the azimuth angle of the unmanned operating ship in the xoy plane and the x direction, θ∈[-π,π], which can be directly measured by the level meter (3); is the polar angle of the unmanned operating ship in the z direction, can be directly measured by a level meter (3); f1, f8 are the pressures of the six controllable water outlets (11), which can be directly measured by a pressure sensor (112); Δf i is the pressure adjustment amount of the i-th water pump (111); The error limit described in step S1 is a relative error ratio.

5. The precise movement correction control method of an unmanned working ship according to claim 4 is characterized in that: The step S4 is specifically as follows: S401: according to the opening unit direction vector of the controllable water outlet hole (11) Calculate the combined force of the water pump (111) on the unmanned operation boat Among them, the opening unit direction vector is the design parameter, is a known quantity; S402: Based on the distance l between the i-th controllable water outlet (11) and the geometric center of the hull (1), i , establish the torque balance constraint: in, S403: According to the force balance: F z -ma z =0; where is the propulsion acceleration of the unmanned operation ship in the x-direction and the y-direction when performing the operation, which is a known quantity; m is the mass of the unmanned operation ship, which is a known quantity; S404: Solve the equations of step S402 and step S403 to calculate the theoretical action of the water pump (111) in, is the theoretical pressure adjustment value of the i-th water pump (111).

6. The precise movement correction control method of an unmanned working ship according to claim 4, characterized in that: The step S5 is specifically as follows: S501: The central processing unit (4) uses the KNN network to randomly select M state-action transition pairs (s t ,a t ,r t ,s t+1 ), where s t is the state at time t, s t+1 is the state at time t+1, a t is the action at time t, r t is the reward at time t; S502: According to the state error limit And the action error limit The state-action transfer pairs in the KNN network are used to select the state-action transfer pairs that meet the conditions as the candidate experience pool.

7. The precise movement correction control method of an unmanned working ship according to claim 6 is characterized in that: The upper limit parameters of the state error limit and the action error limit are different.

Images