A cable-controlled robot intelligent control system applicable to seabed multi-sample collection

Abstract

A cable-controlled robot intelligent control system applicable to submarine multi-sample collection is provided, a coordinate system is established, a ROV linear velocity matrix in a body coordinate system is converted into a spatial position change rate matrix in an inertial coordinate system, and a first conversion relationship model is constructed; a second conversion relationship model between angular velocity and attitude angle change rate is established through the geometric relationship between angular velocity and attitude angle; a kinematics model containing the spatial position, attitude angle, linear velocity and angular velocity of the ROV is obtained through the first conversion relationship model and the second conversion relationship model; the A* algorithm is introduced based on the kinematics model and the dynamics model of the ROV; the artificial potential field method is introduced based on the kinematics model and the dynamics model of the ROV, and the ROV obstacle avoidance path planning is carried out; the simulation of the ROV local path planning based on the artificial potential field algorithm is carried out through the use of python programming, and the ROV can complete the local path planning in an unknown environment.

Description

Technical Field

[0001] This invention relates to the field of underwater robot control technology, and in particular to an intelligent control system for a cable-controlled robot applicable to multi-sample collection on the seabed. Background Technology

[0002] With the development of marine exploration and intelligent control technologies, remotely operated vehicles (ROVs) have emerged. ROVs are characterized by their large operating depth, strong environmental adaptability, and ability to be remotely and precisely controlled. They can be equipped with various sensors and tools to perform diverse tasks in complex underwater environments, such as seabed topography mapping, marine ecological monitoring, and marine resource exploration. This has driven in-depth research in the marine scientific research field on the autonomous navigation and efficient task execution capabilities of ROVs. Global path planning for ROVs, as one of the core technologies for achieving autonomous navigation and efficient task execution, requires planning an optimal or collision-free path from the starting point to the target point in a three-dimensional marine environment, based on prior maps or real-time sensing data, while simultaneously satisfying a balance between safety, energy efficiency, and time cost under multiple constraints.

[0003] In related technologies, global path planning for ROVs typically employs rule-based algorithms such as Dijkstra's algorithm and A* algorithm, performing path planning in two-dimensional or simplified three-dimensional environments. However, while this current approach based on simple rule algorithms can achieve path planning to a certain extent, it often neglects the dynamic characteristics of ROVs, the dynamic changes in the marine environment, and comprehensive optimization under multiple constraints. This can lead to problems such as high energy consumption, long travel times, and increased risk of collisions in practical applications. Therefore, there is an urgent need to provide a navigation control method for cable-controlled underwater robots to meet the demands of modern marine scientific research for efficient and safe operation of cable-controlled underwater robots. Summary of the Invention

[0004] Therefore, it is necessary to address the problems of high energy consumption, long time, and easy collision in traditional ROV global path planning methods, which make it difficult to meet the requirements of efficient and safe operation of cable-controlled underwater robots, and to provide an intelligent control system for cable-controlled robots that can be applied to seabed multi-sample collection.

[0005] The technical solution adopted in this invention is as follows:

[0006] A cable-controlled robot intelligent control system applicable to multi-sample seabed collection includes the following steps:

[0007] S1. Establish an inertial coordinate system E-xyz to describe the ROV's trajectory and attitude, and establish a body coordinate system Ox to describe the ROV's forces. b y b zb ;

[0008] S2. Using the transformation matrix, the ROV linear velocity matrix in the body coordinate system is converted into the spatial position change rate matrix in the inertial coordinate system, thereby constructing the first transformation relationship model;

[0009] By establishing the geometric relationship between angular velocity and attitude angle, a second conversion relationship model between angular velocity and the rate of change of attitude angle is established;

[0010] By combining the first and second transformation relationship models, a kinematic model is obtained that includes the spatial position, attitude angle, linear velocity, and angular velocity of the ROV.

[0011] S3. Assuming the ROV is a rigid body, according to the theorem of rigid body motion and the theorem of moment of inertia, the first dynamic model of the ROV can be expressed as:

[0012]

[0013] Among them, M RB Here is the mass matrix of the ROV; C RB (V) is the Coriolis force and centripetal force matrix of the ROV, which is related to the velocity of the object, and C RB (V)=-C RB T (V); F represents the resultant force (torque) acting on the ROV.

[0014] S4: Based on the kinematic and dynamic models of ROV, the A* algorithm is introduced;

[0015] In the A* algorithm, a corner constraint function p(θ) is introduced to make the ROV turn at an obtuse angle;

[0016] Increase the interval step size to reduce the number of nodes required to compute the optimal path;

[0017] The effect of enhancing the estimated cost of the target node is such that the ROV always tends to search towards the endpoint.

[0018] S5: Based on the kinematic and dynamic models of ROV, the artificial potential field method is introduced to plan ROV obstacle avoidance paths;

[0019] S6: Simulations were conducted using Python programming to perform local path planning for ROVs based on an artificial potential field algorithm. Under the influence of the designed algorithm, ROVs were able to complete local path planning in unknown environments.

[0020] Its further technical solution lies in:

[0021] In S2, the kinematic model of the ROV is obtained as follows:

[0022]

[0023] The linear velocity and angular velocity of the ROV at the origin O in the body coordinate system can be represented in matrix form as: V = [V1 V2] T The matrix representation of the linear velocity of the ROV motion is: V1 = [uvw] T The matrix representation of angular velocity is: V² = [pqr] T The matrix representing the position and attitude of the ROV in the inertial coordinate system is: η = [pθ] T The ROV spatial location matrix is ​​represented as: p = [xyz] T The attitude angle of the ROV is: θ=[φ θ ψ] T ;

[0024] in, It is the transformation matrix from the ROV body coordinate system to the inertial coordinate system, and

[0025] The attitude angle can be expressed as:

[0026]

[0027] In S3, considering the static forces, propulsion system thrust, and hydrodynamic forces acting on the ROV during navigation, the dynamic model of the ROV is obtained based on the first dynamic model:

[0028]

[0029] Where: M RB Here is the mass matrix of the ROV; C RB (V) represents the Coriolis force and centripetal force matrix of the ROV, where V is the velocity of the ROV. F is the acceleration of the ROV. I For inertial hydrodynamics, F D For viscous hydrodynamics, F R F is the torque generated by the combined action of gravity W and buoyancy B on the object. Thr This refers to the torque of the ROV thruster.

[0030] In S4, the specific steps of applying the A* algorithm to ROV path planning are as follows:

[0031] The A* algorithm creates two lists during its search: an open list and a closed list. The open list stores nodes to be considered, each with an estimated cost representing the path cost from the starting node to the current node. At each step of the search, the A* algorithm expands upon the node with the lowest estimated cost from the open list. The closed list stores nodes that have already been considered. Once a node is added to the closed list, it is not expanded again, preventing the same nodes from being considered repeatedly and improving the algorithm's efficiency. When a node is selected as the current node, it moves from the open list to the closed list, while adjacent nodes that have not yet been considered are added to the open list. By maintaining these open and closed lists, the A* algorithm can systematically search the nodes in the grid and find the shortest path.

[0032] The planning steps of the A* algorithm are as follows:

[0033] Initialization: Add the starting node to the open list and set its estimated cost to 0;

[0034] Loop search: when the open list is not empty;

[0035] Path backtracking: When the target node is found, the shortest path can be determined by tracing back to the starting node along the parent node of each node.

[0036] The execution steps of the loop search are as follows:

[0037] Select the node with the lowest estimated cost from the open list and mark it as the current node;

[0038] Remove the current node from the open list and add it to the closed list;

[0039] If the current node is the target node, it means that the shortest path has been found, and the algorithm ends.

[0040] Traverse the adjacent nodes of the current node;

[0041] If an adjacent node is in the closed list, ignore it; if an adjacent node is not in the open list, add it to the open list and calculate its estimated cost, actual cost, and heuristic estimate; if an adjacent node is already in the open list, check if the path to that node through the current node is shorter, and if so, update the node's parent node and cost value.

[0042] The beneficial effects of this invention are as follows:

[0043] This invention incorporates the static forces, propulsion system thrust, and hydrodynamic forces (including inertial and viscous hydrodynamic models) experienced by ROVs during navigation. This enhances their adaptability to complex underwater environments. In the marine environment, the external forces experienced by ROVs are complex and variable, and hydrodynamic disturbances (such as velocity variations, eddies, and additional inertia) significantly affect their attitude and speed. Incorporating a complete hydrodynamic model helps the control system accurately estimate the impact of disturbances based on current speed and acceleration, improving system robustness. For example, the inertial hydrodynamic term characterizes the additional mass effect of the fluid on acceleration, while the viscous damping term reflects the changing trend of drag experienced by the ROV at different speeds. It also enables more rational propulsion system allocation and energy consumption optimization. The introduction of a thruster model helps to rationally allocate thrust in multi-thrust systems, avoiding energy waste due to thrust overload or excessive redundancy. Considering static forces (such as the torques of buoyancy and gravity) helps maintain attitude balance, reducing the burden of attitude adjustment on the thrusters, thereby reducing energy consumption and improving mission continuity. Furthermore, to improve the consistency between simulation results and the real system, more accurate dynamic modeling is introduced during the path planning and simulation verification stages to make the simulation environment closer to the real operating environment. This helps to evaluate the effectiveness and stability of the planning algorithm under complex real-world conditions and shortens the debugging cycle from simulation to actual deployment.

[0044] This invention can provide path feasibility constraints (the role of the kinematic model). The kinematic model describes the relationship between the position, attitude, and velocity of the ROV, reflecting its pose evolution and nonholonomic constraints (such as non-sideslip constraints, maximum turning radius, etc.). In global path planning, it mainly plays the following roles:

[0045] (1) Limit the path search space: ensure that the path is physically reachable and avoid generating trajectories that violate the motion characteristics of ROV (such as sudden turns, lateral slips, etc.).

[0046] (2) Constraining turning radius and maximum speed: Assisting in planning to generate feasible paths that satisfy ROV maneuverability.

[0047] This invention can evaluate the cost and safety of path execution (the role of the dynamic model). The dynamic model describes the motion response of an ROV under forces, including factors such as propulsion, hydrodynamics, buoyancy, mass, and added mass, reflecting the system's dynamic response capability and control load. It plays the following role in path planning:

[0048] (1) Evaluate the “controllability” and energy consumption cost of different paths: The dynamic model can predict the required propulsion force and energy consumption of each segment along the path, so as to add energy efficiency index to the path cost function and achieve path optimization.

[0049] (2) Avoiding uncontrollable trajectories: Although some paths may be kinematically feasible, the ROV may be unable to track them stably due to factors such as hydrodynamic interference or excessive acceleration. The dynamic model can predict this risk in advance and avoid planning unexecutable trajectories. Attached Figure Description

[0050] Figure 1 The coordinate system and ROV motion parameters of this invention are shown.

[0051] Figure 2 This is a topographic map of the seabed in the operating area of ​​this invention.

[0052] Figure 3 This is a grid map for the present invention.

[0053] Figure 4 This is the force model of the artificial potential field method of the present invention.

[0054] Figure 5 This invention relates to ROV local path planning based on the dynamic potential field method.

[0055] Figure 6 This is a diagram showing the distance change between the ROV and dynamic obstacles in this invention. Detailed Implementation

[0056] The specific embodiments of the present invention will now be described with reference to the accompanying drawings.

[0057] like Figures 1-6 As shown in this embodiment, a cable-controlled robot intelligent control system applicable to multi-sample seabed collection mainly includes the following steps:

[0058] S1. Establish an inertial coordinate system E-xyz to describe the ROV's trajectory and attitude, and establish a body coordinate system Ox to describe the ROV's forces. b y b z b ;

[0059] S2. Using the transformation matrix, the ROV linear velocity matrix in the body coordinate system is converted into the spatial position change rate matrix in the inertial coordinate system, thereby constructing the first transformation relationship model;

[0060] By establishing the geometric relationship between angular velocity and attitude angle, a second conversion relationship model between angular velocity and the rate of change of attitude angle is established;

[0061] By combining the first and second transformation relationship models, a kinematic model is obtained that includes the spatial position, attitude angle, linear velocity, and angular velocity of the ROV.

[0062] S3. Assuming the ROV is a rigid body, according to the theorem of rigid body motion and the theorem of moment of inertia, the first dynamic model of the ROV can be expressed as:

[0063]

[0064] Among them, M RB Here is the mass matrix of the ROV; C RB (V) is the Coriolis force and centripetal force matrix of the ROV, which is related to the velocity of the object, and C RB (V)=-C RB T (V); F represents the resultant torque acting on the ROV;

[0065] S4: Based on the kinematic and dynamic models of ROV, the A* algorithm is introduced;

[0066] In the A* algorithm, a corner constraint function p(θ) is introduced to make the ROV turn at an obtuse angle;

[0067] Increase the interval step size to reduce the number of nodes required to compute the optimal path;

[0068] The effect of enhancing the estimated cost of the target node is such that the ROV always tends to search towards the endpoint.

[0069] S5: Based on the kinematic and dynamic models of ROV, the artificial potential field method is introduced to plan ROV obstacle avoidance paths;

[0070] S6: Simulations were conducted using Python programming to perform local path planning for ROVs based on an artificial potential field algorithm. Under the influence of the designed algorithm, ROVs were able to complete local path planning in unknown environments.

[0071] In S2, the kinematic model of the ROV is obtained as follows:

[0072]

[0073] The linear velocity and angular velocity of the ROV at the origin O in the body coordinate system can be represented in matrix form as: V = [V1 V2] T The matrix representation of the linear velocity of the ROV motion is: V1 = [uvw] T The matrix representation of angular velocity is: V² = [pqr] T The matrix representing the position and attitude of the ROV in the inertial coordinate system is: η = [p θ] T The ROV spatial location matrix is ​​represented as: p = [xyz] T The attitude angle of the ROV is: θ=[φ θ ψ]T ;

[0074] in, It is the transformation matrix from the ROV body coordinate system to the inertial coordinate system, and

[0075] The attitude angle can be expressed as:

[0076]

[0077] In S3, considering the static forces, propulsion system thrust, and hydrodynamic forces acting on the ROV during navigation, the dynamic model of the ROV is obtained based on the first dynamic model:

[0078]

[0079] Where: M RB Here is the mass matrix of the ROV; C RB (V) represents the Coriolis force and centripetal force matrix of the ROV, where V is the velocity of the ROV. F is the acceleration of the ROV. I For inertial hydrodynamics, F D For viscous hydrodynamics, F R F is the torque generated by the combined action of gravity W and buoyancy B on the object. Thr This refers to the torque of the ROV thruster.

[0080] In S4, the specific steps of applying the A* algorithm to ROV path planning are as follows:

[0081] During its operation, the A* algorithm creates two lists for storing nodes during the search: an open list and a closed list. The open list stores nodes to be considered, each with an estimated cost value, representing the estimated path cost from the starting node to the current node. In each search step, the A* algorithm selects the node with the lowest estimated cost from the open list for expansion. The closed list stores nodes that have already been considered. Once a node is added to the closed list, it will not be expanded again, avoiding the reconsideration of the same node during the search and improving the algorithm's efficiency. When a node is selected as the current node, it moves from the open list to the closed list, while adjacent nodes that have not yet been considered are added to the open list. By maintaining the open and closed lists, the A* algorithm can systematically search the nodes in the grid and find the shortest path.

[0082] The planning steps of the A* algorithm are as follows:

[0083] Initialization: Add the starting node to the open list and set its estimated cost to 0;

[0084] Loop search: when the open list is not empty;

[0085] Path backtracking: When the target node is found, the shortest path can be determined by tracing back to the starting node along the parent node of each node.

[0086] The execution steps of the loop search are as follows:

[0087] Select the node with the lowest estimated cost from the open list and mark it as the current node;

[0088] Remove the current node from the open list and add it to the closed list;

[0089] If the current node is the target node, it means that the shortest path has been found, and the algorithm ends.

[0090] Traverse the adjacent nodes of the current node;

[0091] If an adjacent node is in the closed list, ignore it; if an adjacent node is not in the open list, add it to the open list and calculate its estimated cost, actual cost, and heuristic estimate; if an adjacent node is already in the open list, check if the path to that node through the current node is shorter, and if so, update the node's parent node and cost value.

[0092] In actual work process:

[0093] first:

[0094] To better describe the motion and stress states of ROVs, establishing a suitable coordinate system is essential and important. This study adopts the system recommended by the International Tank Conference (ITTC) and the terminology bulletin of the Society of Naval Architects and Marine Engineers (SNAME), establishing two coordinate systems: an inertial coordinate system to describe the ROV's trajectory and attitude; and a body-dependent coordinate system to describe the ROV's stress states. (e.g.) Figure 1 (As shown)

[0095] The inertial coordinate system E-xyz is also called the fixed coordinate system (or simply "fixed system"). The inertial coordinate system is fixed at an arbitrary point E on the Earth. The positive direction of the Ez axis points to the Earth's center, and the Ex and Ey axes lie in the horizontal plane and are perpendicular to each other, making the coordinate system E-xyz a right-handed coordinate system.

[0096] Body coordinate system Ox b y b z b Also known as a moving coordinate system (or simply "moving frame"). The body coordinate system is fixed at a point O on the ROV, Ox b The axis is positive when it moves forward along the longitudinal direction of the ROV; Oy bThe axis pointing to the starboard side of the ROV is positive; Oz b Axis perpendicular to Ox b y b Pointing downwards in a plane is considered positive.

[0097] The linear velocity and angular velocity of the ROV at the origin O in the body coordinate system can be represented in matrix form as: V = [V1 V2] T The matrix representation of the linear velocity of the ROV motion is: V1 = [uvw] T The matrix representation of angular velocity is: V² = [pqr] T The matrix representing the position and attitude of the ROV in the inertial coordinate system is: η = [p θ] T The ROV spatial location matrix is ​​represented as: p = [xyz] T The attitude angle of the ROV is: θ=[φ θ ψ] T The force and torque expression matrix for the ROV in body coordinates is: F = [F1 F2] T The force matrix is: F1 = [XYZ] T The moment matrix acting on the ROV is: F2 = [KMN] T All motion parameters are positive if they point in the positive direction of the coordinate axis or rotate around the positive direction of the coordinate axis according to the right-hand rule, and negative otherwise. The specific motion definitions of the ROV under different degrees of freedom are shown in Table 1.

[0098] Table 1 Definition of ROV motion parameters

[0099]

[0100] Secondly:

[0101] The ROV's body coordinate system can be aligned with the inertial coordinate system through three rotations around its coordinate axes. Therefore, the transformation relationship between the ROV's linear velocity in the body coordinate system and its spatial position relative to the inertial coordinate system is as follows:

[0102]

[0103] in, It is the transformation matrix from the ROV body coordinate system to the inertial coordinate system, and The attitude angle can be expressed as:

[0104]

[0105] The relationship between ROV linear velocity and spatial attitude can be obtained:

[0106]

[0107] Similarly, the relationship between ROV angular velocity and attitude angle is as follows:

[0108]

[0109] Wherein, the transformation matrix

[0110] Therefore, the formula can be expanded as follows:

[0111]

[0112] The kinematic model of the ROV is further obtained as follows:

[0113]

[0114] Subsequently:

[0115] To facilitate the study of the problem, the following settings and assumptions were made for the ROV. It is assumed that the ROV is a rigid body with an unchanged geometric shape, and that its mass and center of gravity are constant. The influence of the umbilical cable and other operating tools, as well as the influence of the seabed on the ROV's hydrodynamics, are not considered. The center of gravity of the ROV is taken as the origin of the body coordinate system, and the three coordinate axes of the ROV are approximately considered as the central principal axes of inertia.

[0116] According to the theorems of rigid body motion and moment of inertia, the dynamic model of a ROV can be expressed as:

[0117]

[0118] Among them, M RB Here is the mass matrix of the ROV; C RB (V) is the Coriolis force and centripetal force matrix of the ROV, which is related to the velocity of the object, and C RB (V)=-C RB T (V); F represents the resultant force (torque) acting on the ROV. For an ROV with its center of gravity as the origin and assuming the three coordinate axes as the principal axes of inertia, the mass matrix can be simplified to:

[0119]

[0120] Where m is the mass of the ROV, I g Let be the inertia matrix of the ROV, and we have .

[0121] I g =diag[I x I y I z ]

[0122]

[0123]

[0124] Coriolis force and centripetal force matrix C RB (V) can be simplified to:

[0125]

[0126] S is the vector multiplication operator.

[0127]

[0128] M RB C RB Substituting the expression for (V) and expanding it, we can obtain the general equation for the motion of the ROV under force:

[0129]

[0130] ROV stress analysis:

[0131] Generally, the forces (torques) acting on an ROV can be roughly divided into several categories: static forces (torques) F generated by the combined action of the object's gravity W and buoyancy B. R The hydrodynamic force (torque) F caused by the movement of the ROV in the flow field. H ROV thrust (torque) F thr Accurate calculation and analysis of the above torques are the foundation for ROV motion control and manipulation simulation.

[0132] The hydrodynamic force (torque) F acting on the ROV H The dynamics (torque) of an ROV are closely related to its geometry, velocity, acceleration, and direction. In infinitely deep and wide still water, without considering the effects of flow boundaries, current, and internal waves, the hydrodynamic force (torque) of a given ROV with a fixed shape depends only on its velocity V and acceleration. ROV hydrodynamic (torque) F H Based on whether it is caused by steady motion or unsteady hydrodynamic forces, it can be divided into inertial hydrodynamic forces F I and viscous hydrodynamic F D Two categories. The inertial hydrodynamics of an ROV is related not only to its acceleration but also, to some extent, to its velocity. For ease of calculation and study, the velocity-related inertial hydrodynamic component is incorporated into the viscous hydrodynamic component F. D Therefore, hydrodynamics can be expressed as:

[0133]

[0134] The kinematic model of ROV can be represented as:

[0135]

[0136] Static force is a restoring force resulting from the combined effects of gravity and buoyancy of the ROV. When the center of gravity is at the origin of the ROV's body coordinate system, according to the transformation relationship between the inertial coordinate system and the body coordinate system, the static force (torque) generated by the combined effects of gravity and buoyancy in the ROV's body coordinate system is expressed as:

[0137]

[0138] Static (torque) F k (η) is related to the ROV's motion posture. Where r b r represents the radius vector of the ROV's center of buoyancy relative to the body coordinate system. b =[x b y b , z b ] T When the magnitudes of gravity and buoyancy of the ROV are equal, the ROV's still water (torque) F k (η) can be further simplified to:

[0139]

[0140] A 3D map was constructed based on seabed topographic data from the ISA DeepData database, and a rectangular area between 21.405°N and 21.445°N, 159.645°E and 159.600°E, and with a depth of 1.280km to 1.340km was selected as the benchmark test scenario for the static global path planning algorithm.

[0141] based on Figure 2 Terrain data is used to generate a accessibility map, which is then simplified into a binary matrix. The discrete depth values ​​of this area are represented as matrix D. The relative depth is calculated and normalized using the following formula, yielding the dimensionless quantity N:

[0142]

[0143] Where max(D) and min(D) represent the maximum and minimum depths, respectively, and the dimensionless quantity N ranges from [0, 1]. Defining the threshold ε∈(0, 1), we can obtain...

[0144]

[0145] This is the accessibility classifier. Assume the binary matrix of the accessibility map is M0, and the obstacle avoidance zone matrix is ​​M. rafe The radial width of the obstacle avoidance zone can be changed by adjusting the number of iterations in the following operation:

[0146]

[0147] M rafe =M rfan -M0

[0148] The processed accessibility map is rasterized to obtain the following output. Black areas represent prohibited areas, blue areas represent obstacle avoidance zones, and white areas represent feasible regions. The side length of each square is equal to the scale value of the actual map, and the red and green squares represent the start and end points of the path, respectively.

[0149] Algorithm Design:

[0150] Based on the grid diagram model described above, the decision-making process of underwater robots can be modeled as a Markov Decision Process (MDP). First, define the basic variables:

[0151] 1) State Space This indicates that the Agent's two-dimensional coordinates in the raster graph are unknown, and each state s t =(x t y t The row and column indices of the corresponding map cells are ∈S.

[0152] 2) The action space A = {0, 1, ..., 7} defines eight possible movement directions, including four basic directions (up, right, down, left) and four diagonal directions.

[0153] 3) Environmental dynamics characteristics are determined by the transition probability P(s) t+1 |s t a t ) and immediate reward function r(s) t a t The system is described in conjunction with other rules, where collisions with obstacles will result in negative rewards, reaching the target position will result in positive rewards, and other situations will incur time penalties.

[0154] The goal of the agent is to learn the optimal policy π through interaction with the environment. * : S→A, maximizing the expected value of the cumulative discount reward:

[0155]

[0156] Where γ∈

[01] is the discount factor. To achieve this goal, this paper adopts the Double Deep Q-Network (Double DQN) algorithm, whose mathematical model can be decomposed into the following key steps:

[0157] (1) Q-function approximation and Bellman equation

[0158] Traditional Q-learning approximates the optimal policy by iteratively updating the action-value function Q(s, a), and its update rule follows the Bellman optimality equation:

[0159]

[0160] However, in high-dimensional state spaces, the tabular form of the Q-function is difficult to store and update efficiently. To address this, DQN introduces a deep neural network Q(s, a: θ) as a function approximator, where θ represents the network parameters. The update objective of the Q-function then transforms into minimizing the temporal difference error:

[0161]

[0162] Where θ - The target network parameters are periodically synchronized from an online network to stabilize the training process.

[0163] (2) Double DQN Improvement Mechanism

[0164] Traditional DQN relies on the target network for both action selection and value estimation when calculating the target Q-value, which can lead to an overestimation bias. Double DQN addresses this issue by decoupling action selection and value evaluation.

[0165]

[0166] That is, the optimal action for the next state is selected by the current online network θ, and the corresponding Q value is evaluated by the target network θ-θ-.

[0167] (3) Experience replay and network updates

[0168] The transfer samples (s) accumulated by the agent during the exploration process t a t r t s t+1 d t Stored in the experience replay buffer Where d t This indicates the end of a round. During training, mini-batch samples are randomly sampled for parameter updates, breaking temporal correlations and improving data efficiency. The online network parameters θ are minimized using gradient descent to minimize the loss function:

[0169]

[0170] The target network uses a soft update strategy to gradually track the online network: θ - ←τθ+(1-τ)θ - , where τ << 1 is the smoothing coefficient.

[0171] (4) Balancing Exploration and Exploitation

[0172] To strike a balance between exploring new actions and utilizing existing knowledge, the algorithm employs an e-greedy strategy for action selection. For any action a∈A, its probability of being selected is:

[0173]

[0174] Where δ(·) is an indicator function, taking the value 1 when the condition within the parentheses is met, and 0 otherwise; ∈ represents the exploration rate, which varies according to the following alkali decay rate:

[0175] ∈(k)=∈ end +(∈ start -∈ end )·e -k∫decay_steps

[0176] This formula clearly expresses two sources of probability:

[0177] 1) Exploration: All actions are gained. The base probability is guaranteed to allow random exploration with a total probability of at least ∈.

[0178] 2) Exploitation: The optimal action gains an additional probability bonus of (1-∈), ensuring that the policy prioritizes the current optimal action when exploiting it.

[0179] (5) Network architecture design

[0180] The Q network employs a fully connected structure, with the input layer dimension corresponding to the state space dimension |S=2, and the output layer dimension matching the action space dimension |A|=8. To improve function approximation capability, the network introduces residual connection modules:

[0181] ResBlock(x)=x+W2σ(W1x+b1)+b2

[0182] Where σ is the Leaky ReLU activation function, and W1 and W2 are trainable weight matrices.

[0183] Parameter initialization uses an orthogonal initialization method:

[0184]

[0185] Where n l Using the l-th layer as the input dimension, this initialization strategy, combined with the adjustment of the gain coefficient of the activation function, can ensure the stability of the signal amplitude during forward propagation.

[0186] (6) Path planning specific design

[0187] Considering the characteristics of the grid environment, the reward function is designed as follows:

[0188]

[0189] The sparse reward setting forces the agent to quickly find a path to the goal, while the time penalty of -0.1 encourages the selection of the shortest path. The action space is expanded to eight directions, allowing diagonal movement, which reduces the path length by up to approximately 29% compared to four-direction movement.

[0190] (7) Training Dynamics Analysis

[0191] Define local average reward As a performance metric, where N is the sliding window size. When K consecutive times without exceeding the threshold δ An early stopping mechanism is triggered, saving the current optimal model parameters. The magnitude is updated based on gradient pruning constraint parameters. Where ξ is the clipping threshold.

[0192] The specific execution process of the A* algorithm is as follows: First, the positions of the starting node and the target node are determined as the start and end points of path planning. Next, two lists are initialized: an open list (Open) to store nodes to be examined, and a closed list (Close) to record nodes that have already been examined. Then, the A* algorithm begins running, traversing the nodes surrounding the starting node and selecting the node with the minimum cost as the next search target based on certain evaluation criteria. This process is repeated as path nodes are gradually determined. The algorithm continuously selects nodes from the open list, examines their surrounding nodes, and updates the open and closed lists until it finally extends to the target location and finds the optimal path from the starting node to the target node.

[0193] The traditional A* algorithm suffers from problems such as sharp path inflection points, weak search directionality, and excessive number of computational nodes during path planning. To address these issues, an improved A* algorithm is proposed. First, an angle constraint is introduced into the evaluation function, changing right-angle turns to obtuse angles, resulting in smoother path inflection points. Second, the distance function is improved to enhance the directionality of path search, while redundant nodes are removed, significantly reducing path planning time. Finally, an improved step size method is proposed to optimize the number of path nodes obtained by the improved A* algorithm.

[0194] Under given constraints and evaluation criteria, the artificial potential field method can plan the optimal or satisfactory route for ROVs in advance or in real-time based on obstacle information within the planning space. This algorithm does not require finding a global path and is fast and efficient, making it particularly suitable for trajectory generation tasks requiring high real-time performance and high safety. The route calculated by this method may not be the shortest, but it is relatively smooth and safe.

[0195] The artificial potential field method is a virtual method. Its basic idea is to represent the environment as a virtual potential field, with the target point forming an gravitational potential field and obstacles forming a repulsive potential field. The ROV moves towards the target under the combined influence of the gravitational potential field of its target and the repulsive potential field around the obstacles. The gravitational force (F) exerted by the ROV on the gravitational potential field... att The repulsive force (F) of the repulsive potential field rep The combined effect of these forces generates a resultant force (F). total The combined forces drive the ROV to move toward the target.

[0196] To verify the effectiveness of the artificial potential field algorithm in ROV local path planning, a simulation of ROV local path planning based on the artificial potential field algorithm was performed using Python programming. Figure 5 As shown, the blue solid line represents the ROV local planning path based on the artificial potential field algorithm. Figure 6 The figure shows the distance changes between the ROV and various obstacles during the dynamic obstacle avoidance process. As can be seen from the simulation results above, under the action of the designed algorithm, the ROV can complete local path planning in an unknown environment.

[0197] The above description is an explanation of the present invention and not a limitation thereof. The scope of the present invention is defined by the claims. Within the scope of protection of the present invention, any form of modification may be made.

Claims

1. An intelligent control system for a cable-controlled robot applicable to multi-sample seabed collection, characterized in that: Includes the following steps: S1. Establish an inertial coordinate system to describe the ROV's trajectory and attitude. Establish a body coordinate system to describe the stress state of the ROV. ; S2. Using the transformation matrix, the ROV linear velocity matrix in the body coordinate system is converted into the spatial position change rate matrix in the inertial coordinate system, thereby constructing the first transformation relationship model; By establishing the geometric relationship between angular velocity and attitude angle, a second conversion relationship model between angular velocity and the rate of change of attitude angle is established; By combining the first and second transformation relationship models, a kinematic model is obtained that includes the spatial position, attitude angle, linear velocity, and angular velocity of the ROV. S3. Assuming the ROV is a rigid body, according to the theorem of rigid body motion and the theorem of moment of inertia, the first dynamic model of the ROV is expressed as: in, The mass matrix of the ROV; The Coriolis force and centripetal force matrix of the ROV are related to the velocity of the object, and ; This represents the resultant torque acting on the ROV; S4: Based on the kinematic and dynamic models of ROV, the A* algorithm is introduced; In the A* algorithm, a corner constraint function is introduced. This causes the ROV to turn at an obtuse angle. Increase the interval step size to reduce the number of nodes required to compute the optimal path; The effect of enhancing the estimated cost of the target node is such that the ROV always tends to search towards the endpoint. S5: Based on the kinematic and dynamic models of ROV, the artificial potential field method is introduced to plan ROV obstacle avoidance paths; S6: Simulation was performed using Python programming based on the artificial potential field algorithm for ROV local path planning. Under the action of the designed algorithm, ROV can complete local path planning in an unknown environment. In S2, the kinematic model of the ROV is obtained as follows: ROV body coordinate system origin O The linear velocity and angular velocity of the motion in the body coordinate system can be expressed in matrix form as follows: The matrix representation of the linear velocity of the ROV motion is as follows: The matrix representation of angular velocity is: The matrix representing the position and attitude of the ROV in the inertial coordinate system is as follows: The matrix representation of the ROV spatial location is as follows: The ROV's attitude angles are: ; in, ; It is the transformation matrix from the ROV body coordinate system to the inertial coordinate system, and , The attitude angle can be expressed as: In S3, considering the static forces, propulsion system thrust, and hydrodynamic forces acting on the ROV during navigation, the dynamic model of the ROV is obtained based on the first dynamic model: in: The mass matrix of the ROV; The Coriolis force and centripetal force matrix of the ROV. V For the speed of ROV, For the acceleration of the ROV, For inertial hydrodynamics, For viscous hydrodynamics, The force of gravity acting on an object W and buoyancy B The torque generated by the combined action This refers to the torque of the ROV thruster.

2. The intelligent control system for a cable-controlled robot applicable to multi-sample seabed collection as described in claim 1, characterized in that: In S4, the specific steps of applying the A* algorithm to ROV path planning are as follows: During its operation, the A* algorithm creates two lists for storing nodes during the search: an open list and a closed list. The open list stores nodes to be considered, each with an estimated cost value, representing the estimated path cost from the starting node to the current node. In each search step, the A* algorithm selects the node with the lowest estimated cost from the open list for expansion. The closed list stores nodes that have already been considered. Once a node is added to the closed list, it will not be expanded again, avoiding the reconsideration of the same node during the search and improving the algorithm's efficiency. When a node is selected as the current node, it moves from the open list to the closed list, while adjacent nodes that have not yet been considered are added to the open list. By maintaining the open and closed lists, the A* algorithm can systematically search the nodes in the grid and find the shortest path.

3. The intelligent control system for a cable-controlled robot applicable to multi-sample seabed collection as described in claim 2, characterized in that: The planning steps of the A* algorithm are as follows: Initialization: Add the starting node to the open list and set its estimated cost to 0; Loop search: when the open list is not empty; Path backtracking: When the target node is found, the shortest path is determined by tracing back to the starting node along the parent node of each node.

4. The intelligent control system for a cable-controlled robot applicable to multi-sample seabed collection as described in claim 3, characterized in that: The execution steps of the loop search are as follows: Select the node with the lowest estimated cost from the open list and mark it as the current node; Remove the current node from the open list and add it to the closed list; If the current node is the target node, it means that the shortest path has been found, and the algorithm ends. Traverse the adjacent nodes of the current node; If an adjacent node is in the closed list, ignore it; If a neighboring node is not in the open list, it will be added to the open list, and its estimated cost, actual cost, and heuristic estimate will be calculated. If a neighboring node is already in the open list, it will be checked whether the path to that node through the current node is shorter. If so, the parent node and cost value of that node will be updated.

Images