An Autonomous Visual Navigation Method and System for Underwater Robots Based on Empirical Knowledge Transfer

Abstract

This invention belongs to the field of robot navigation technology, and relates to an autonomous visual navigation method and system for underwater robots based on experiential knowledge transfer. First, visual, sonar, and inertial navigation data are collected to establish a high-precision 3D environment model. Second, a blind-spot-free and distortion-free gaze point reconstruction algorithm and a temporal modeling network are used to extract human visual attention and operational behavior features, generating experiential feature vectors. A human strategy template is then constructed using an imitation learning network. Using this strategy template as a priori, multi-objective adaptive optimization is performed based on four reward categories: navigation accuracy, obstacle avoidance, energy consumption optimization, and experience consistency. A two-layer experiential memory mechanism is combined to achieve cross-task knowledge transfer. A global path is generated using a voxelized map combined with Regression-Regression Mapping (RRT) and an energy-weighted Dijkstra algorithm. This method effectively improves the autonomous perception, decision-making, and navigation capabilities of underwater robots in complex deep-sea environments, achieving a leapfrog development from human-robot collaboration to intelligent autonomy.

Description

Technical Field

[0001] This invention belongs to the field of robot navigation technology, and relates to an autonomous visual navigation method and system for underwater robots based on the transfer of experiential knowledge. Background Technology

[0002] Currently, underwater robots, as core equipment for deep-sea operations, have been initially applied in tasks such as scientific research, resource exploration, equipment maintenance, and ecological monitoring. However, due to factors such as environmental complexity, limited communication, and insufficient algorithmic intelligence, existing underwater robots still mainly rely on remote manual control or semi-autonomous control modes, lacking fully autonomous perception and decision-making capabilities for complex environments.

[0003] On the one hand, the deep-sea environment possesses extreme characteristics such as high pressure, low temperature, low light, and electromagnetic shielding. Traditional visual navigation methods are prone to target loss, path deviation, and positioning inaccuracy under low visibility and high noise conditions, severely limiting the operational stability and reliability of underwater robots. On the other hand, underwater communication bandwidth is limited and latency is high, hindering real-time human-machine interaction and making it difficult to support remote command and dynamic control under complex tasks. Furthermore, existing autonomous navigation algorithms generally rely on static rules or single-scene training, lacking the ability to adapt to environmental changes and task diversity, and are unable to cope with complex tasks involving multiple targets, long time periods, and high uncertainties in real-world ocean environments.

[0004] In terms of human-machine collaboration, although the introduction of virtual reality (VR) and augmented reality (AR) technologies has improved the immersion and safety of operation, existing systems mostly remain at the stage of "human-controlled machines," failing to fully utilize human experience, gaze patterns, and decision-making logic to guide robot learning. The visual attention mechanisms, risk assessment experience, and strategic judgments accumulated by humans in complex environments have not yet been effectively digitally expressed and transferred to machines, causing robots to still rely heavily on trial and error in unknown scenarios, resulting in low learning efficiency and high costs. Summary of the Invention

[0005] The purpose of this invention is to address the problems existing in the prior art and provide an autonomous visual navigation method and system for underwater robots based on experiential knowledge transfer. This method establishes a "human-machine co-intelligence" knowledge transfer framework by integrating the operational experience of human seafarers with the self-learning capabilities of AI agents. The system utilizes a self-evolutionary mechanism that combines multimodal sensor fusion, blind-spot-free and distortion-free gaze acquisition, and reinforcement learning with imitation learning. This enables the robot to extract quantifiable experiential features from human gaze trajectories, operational habits, and strategic preferences, completing an intelligent evolution from "experience imitation" to "autonomous reasoning."

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] An autonomous visual navigation method for underwater robots based on experiential knowledge transfer includes the following steps:

[0008] (1) Collect visual data, sonar data and inertial navigation data; realize the fusion of visual data, sonar data and inertial navigation data through a timestamp synchronization mechanism to construct a three-dimensional environment model. ;

[0009] in, For visual frames, For sonar images, For inertial navigation information, For environmental parameters;

[0010] Geometric registration functions are used to achieve spatial consistency of multimodal data;

[0011] Gazing points are collected using virtual reality devices that integrate eye tracking. fixation time and the trajectory of eye movement;

[0012] (2) Perform distortion-free fixation point reconstruction on the acquired equidistant rectangular projection images, and divide the sphere into... Sub-blocks, and through a mapping matrix Minimize distortion error to obtain spherical fixation point data with fixation error less than a threshold:

[0013] ;

[0014] in, Represents the pixel coordinates in spherical coordinates; This represents the pixel point corresponding to the ERP plane coordinates; This represents a nonlinear transformation from the ERP plane to spherical coordinates; The global distortion loss function is solved using gradient descent.

[0015] Simultaneously record the action sequence and time interval to construct a behavior sequence:

[0016] ;

[0017] Temporal features are extracted using a Long Short-Term Memory (LSTM) network to generate empirical feature vectors.

[0018] ;

[0019] By imitating learning networks to establish templates of human operational strategies, these strategies can be used as initial prior strategies for reinforcement learning.

[0020] ;

[0021] (3) Robot in environmental conditions Select action Receive rewards and update your strategy according to the following rules:

[0022] ;

[0023] in, For a moment Strategies; For a moment The reward function; This represents the gradient operator with respect to the policy parameters; This represents the expectation operation; The learning rate;

[0024] A two-tiered experience memory mechanism is established: short-term experience replay is used for near-task updates, and a long-term experience database is used for cross-task migration; its parameter fusion rules are as follows:

[0025] ;

[0026] in, These are empirical weighting coefficients; Parameters for human experience networks; To reinforce the learning of network parameters; For fusion parameters;

[0027] (4) Generating voxelized 3D maps based on multimodal data:

[0028] ;

[0029] The global path is generated by combining a fast expanding random tree with an energy-weighted Dijkstra algorithm, and its path cost function is:

[0030] ;

[0031] in, For distance, For energy consumption, These are the weighting coefficients for distance and energy consumption, respectively. These are two nodes on the path.

[0032] Preferably, local obstacle avoidance employs model predictive control for attitude adjustment to achieve real-time trajectory correction; the cost function of the model predictive controller is:

[0033] ;

[0034] in, For reference path nodes; These are the weight matrices for the state error and the control input, respectively. For prediction time windows; Control input; This is the robot's state vector.

[0035] Preferably, the reward function is:

[0036] ;

[0037] in, Rewards for navigation accuracy Rewards for safe obstacle avoidance Incentives for energy consumption optimization Rewards for consistency of experience; , , , They are respectively , , , The weighting coefficients.

[0038] Preferably, after the mission is completed, the sailors generate a delayed reward based on the video playback:

[0039] ;

[0040] in, Indicates the accuracy of the navigation path completion; Indicates the safety factor of the operation; Indicates energy efficiency index; Indicates data quality score;

[0041] The system performs closed-loop parameter updates:

[0042] ;

[0043] in, For the first The policy network trained after the sub-task; For the updated policy network; The learning rate is used to control the intensity of updates. This indicates that the strategy parameters are adjusted according to the direction of the delayed reward gradient.

[0044] Preferably, when communication is interrupted for more than a set time, the robot automatically switches to a local strategy. :

[0045] ;

[0046] in, For local reinforcement learning value function;

[0047] It can independently complete navigation tasks based on the characteristics of the current environment.

[0048] Preferably, the local policy network that automatically switches when communication is interrupted is initialized with the policy parameter that has the highest delay reward in historical tasks; the weights of each indicator in the delay reward function satisfy:

[0049] .

[0050] This invention also provides an autonomous visual navigation system for underwater robots based on experience knowledge transfer. The system comprises a multimodal environment perception and data acquisition module, an experience extraction and modeling module, a reinforcement learning and experience transfer module, an autonomous navigation and path planning module, and a policy optimization module. The multimodal environment perception and data acquisition module is used for multimodal environment perception and data acquisition, and for processing the acquired data. The experience extraction and modeling module is used for extracting and modeling human experience features. The reinforcement learning and experience transfer module is used for reinforcement learning and experience transfer for the robot. The autonomous navigation and path planning module is used for path planning and autonomous visual navigation for the robot. The policy optimization module is used for policy verification and iterative optimization. During system operation, the steps of the method are implemented.

[0051] Compared with existing technologies, the advantages of this invention are: This invention constructs a complete closed loop from human experience acquisition and knowledge modeling to reinforcement learning optimization, enabling underwater robots not only to autonomously complete visual navigation, path planning, and risk avoidance in complex environments, but also to achieve strategy transfer and capability self-evolution in long-term missions, thereby realizing truly fully autonomous deep-sea operations. This significantly improves the intelligence, safety, and continuous operation capabilities of deep-sea operations. Attached Figure Description

[0052] Figure 1 This is a flowchart of the fully autonomous visual navigation method for underwater robots based on experience knowledge transfer in the implementation of this invention;

[0053] Figure 2 This is a diagram of the fully autonomous visual navigation system for an underwater robot based on experience-based knowledge transfer, as implemented in this invention.

[0054] Figure 3 This is a flowchart of the self-evolutionary training closed-loop mechanism. Detailed Implementation

[0055] To facilitate understanding of the present invention, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and specific examples. The following examples or drawings are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0056] This invention provides an autonomous visual navigation method for underwater robots based on empirical knowledge transfer, such as... Figure 1 and Figure 2As shown, it includes the following steps:

[0057] I. Multimodal Environment Perception and Data Acquisition Steps

[0058] This step is performed by the multimodal environment perception and data acquisition module.

[0059] In the implementation of this invention, the underwater robot is equipped with a multimodal sensing system, including:

[0060] (1) A 360° panoramic visual acquisition device for obtaining underwater all-around image information;

[0061] (2) A three-dimensional multibeam sonar imaging device for constructing acoustic depth maps in low visibility environments;

[0062] (3) Inertial Measurement Unit (IMU), used for attitude and motion parameter detection;

[0063] (4) Environmental sensors such as water pressure, temperature, and turbidity are used to record changes in the external environment.

[0064] The data from the aforementioned multiple sensors are registered and fused through a timestamp synchronization mechanism to construct a high-precision, spatiotemporally consistent environmental model. The multimodal data fusion process can be represented as follows:

[0065] ;

[0066] in: Indicates time The fused dataset; This represents an image frame acquired by a vision sensor; This represents an acoustic intensity map or depth slice acquired by a sonar sensor. This represents the attitude and acceleration information output by the inertial navigation unit; This indicates environmental parameters (including water pressure, temperature, light intensity, etc.).

[0067] To achieve geometric registration of data from different modalities in a unified coordinate system, the system employs a three-dimensional coordinate transformation model, the expression of which is:

[0068] ;

[0069] in: This is the point vector in the original coordinate system; A point in the transformed target coordinate system; For the yaw angle Pitch angle Roll angle The rotation matrix formed; This is a translation vector used to achieve translation correction of the origin of the coordinate system.

[0070] The aforementioned geometric transformations ensure a strict correspondence between the visual image, sonar depth map, and inertial navigation attitude data in three-dimensional space, thereby generating a multimodal fused spatial point cloud and semantic map. The system further utilizes an external calibration matrix. Perform photoacoustic joint correction:

[0071] ;

[0072] in, This represents the set of points projected onto the visual coordinate system, used to achieve joint image-sonar alignment.

[0073] During the data acquisition phase, seafarers operate remotely by wearing integrated eye-tracking virtual reality (VR) devices. The system collects the following human-computer interaction information in real time:

[0074] gaze point coordinates: This indicates the sailor's real-time gaze position within the VR field of view;

[0075] Duration of gaze: , used to characterize the intensity of the region of interest;

[0076] Eye movement trajectory: It consists of a continuous sequence of gaze points and is used to characterize attentional dynamics;

[0077] Voice and gesture commands: It is used to assist in robot task control.

[0078] Therefore, the multi-source information flow output at this stage can be comprehensively represented as:

[0079] ;

[0080] in, Indicates time The comprehensive information state set includes three categories of information: perception, environment, and human-computer interaction.

[0081] This step enables the simultaneous fusion of visual, acoustic, inertial, and cognitive information about the underwater environment, providing a unified, high-precision data foundation for subsequent experience modeling and knowledge transfer.

[0082] II. Steps for Extracting and Modeling Human Experience Features

[0083] This step is performed by the experience extraction and modeling module.

[0084] In this invention, the system constructs a gaze point reconstruction algorithm without blind spots or distortion to achieve geometric correction and behavioral modeling of human visual attention information. The core objective of this module is to transform the visual attention distribution and action patterns of seafarers during virtual operations into high-dimensional experiential feature representations that can be learned by AI.

[0085] First, the system performs spherical geometry correction on the equirectangular projection (ERP) images captured by the VR device. To reduce stretching distortion in the polar regions (top and bottom), the system divides the sphere according to latitude and longitude rules. A number of sub-blocks, where the default value is... Each sub-block corresponds to a latitude and longitude region on the sphere.

[0086] For each sub-block Establish a mapping matrix from a sphere to a plane. Geometric consistency correction is achieved by minimizing the global distortion error function. The distortion optimization objective function is defined as follows:

[0087] ;

[0088] in: Represents the pixel coordinates in spherical coordinates; This represents the pixel point corresponding to the ERP plane coordinates; This represents a nonlinear transformation from the ERP plane to spherical coordinates; The global distortion loss function is solved using gradient descent; the optimization objective is to minimize the fixation point mapping error. .

[0089] The corrected gaze point data is mapped back to the true spherical field of view coordinate system, resulting in a high-precision, distortion-free gaze point trajectory set:

[0090] .

[0091] At the same time, the system records the seafarer's actions. (e.g., grabbing, rotating, labeling, etc.) and the time interval between actions And form a temporal behavior sequence:

[0092] ;

[0093] in: For a moment The location of the fixation point; The category of actions performed by the seaman at that moment; The time interval between consecutive actions; For length is Behavioral data sequences.

[0094] To extract the temporal dependence features of seafarers during observation and operations, the system employs a Long Short-Term Memory (LSTM) network for behavioral sequence modeling. The network input is a sequence. The output is an empirical feature vector. Its expression is as follows:

[0095] ;

[0096] The LSTM unit captures the dynamic correlation between gaze and action through input gate, forget gate and output gate mechanisms; The dimension is The feature vector is an experience embedding vector; this feature vector is used to describe the seaman's attention shift, operational rhythm and decision preferences during mission execution.

[0097] Subsequently, the system introduces a Behavioral Cloning Network (BCN) to learn and map human operational strategies. This network is based on a conditional probability model and maps states... With experience characteristics As input, a human policy template is obtained by maximizing the action selection probability:

[0098] ;

[0099] in: Indicates the state Seafarer decision-making strategies; This indicates the execution of actions under empirical feature constraints. The probability of.

[0100] The model is trained using cross-entropy loss to minimize the difference between AI-predicted actions and actual human actions.

[0101] Ultimately, the resulting strategy template Used as the initial prior policy network in the reinforcement learning phase This provides robots with behavioral references inspired by human experience. Through this mechanism, robots can acquire the visual attention and operational patterns of human experts at an early stage, thereby significantly improving their learning efficiency and task adaptability in complex underwater environments.

[0102] III. Strengthening Learning and Experience Transfer Steps

[0103] This step is performed by the reinforcement learning and experience transfer module.

[0104] In this invention, the underwater robot, based on multimodal environment perception and experience feature modeling, achieves autonomous strategy optimization and long-term self-evolution by combining reinforcement learning (RL) with experience knowledge transfer mechanism.

[0105] First, let the current environment state be... Underwater robots rely on their strategy network Select Action After performing this action, the environment returns an instant reward and enters the next state. .

[0106] During the execution of continuous tasks, the parameters of the reinforcement learning model Adaptive optimization is performed according to the following gradient update rule:

[0107] ;

[0108] in: Indicates time The strategy network; The learning rate (ranging from 0.0001 to 0.01) is used to control the policy update step size. For a moment The comprehensive reward function; This represents the gradient operator with respect to the policy parameters; This indicates the expected operation.

[0109] To balance task performance and energy consumption stability, this invention designs the instantaneous reward function as a multi-objective weighted combination:

[0110] ;

[0111] in: A navigation accuracy reward is given to encourage the robot to find the shortest path to the target point. The obstacle avoidance reward is used to penalize behavior that involves getting too close to obstacles. This is an energy consumption optimization term used to constrain the power consumption of the thrusters; This is an experience-consistent reward used to measure the correlation between robot behavior and human experience. The degree of similarity; Let be the weighting coefficient, satisfying It can be dynamically adjusted according to task priority.

[0112] In practical implementation, each sub-reward can be defined as:

[0113] ;

[0114] ;

[0115] ;

[0116] ;

[0117] in: For the current pose, The target pose; Distance to the nearest obstacle This is a safety threshold; For current energy consumption, This represents the maximum energy consumption for the task. Embed the policy features of the AI ​​in the current state; Cosine similarity is used to measure the degree of matching between AI behavior and human experience.

[0118] To achieve cross-task knowledge transfer and long-term learning stability, this invention designs a two-layer experience memory mechanism:

[0119] 1. Short-term experience replay (RB)

[0120] Used to store the state-action-reward-next state quadruple of the most recent task. To support offline batch training:

[0121] ;

[0122] in, For short-term memory length, As a reward.

[0123] 2. Long-term experience bank (EB)

[0124] This is used to preserve high-quality samples that have been manually evaluated or confirmed through incentive rewards, forming a transferable set of expert experience:

[0125] ;

[0126] in, This is a high-quality sample threshold used to filter representative operational behaviors.

[0127] During parameter updates, the system employs an experience-based strategy weighting mechanism that integrates human experience with network parameters. With reinforcement learning network parameters :

[0128] ;

[0129] in: Fusion network parameters; For empirical weighting coefficients; when When the size is large, the model tends to follow human strategies more; when When the model is smaller, it tends to explore autonomously.

[0130] This fusion mechanism ensures that the policy network converges quickly in the early stages and has adaptive evolution capabilities in the later stages.

[0131] Through the aforementioned reinforcement learning and experience transfer mechanisms, underwater robots can continuously learn and optimize strategies in complex and dynamic deep-sea environments, gradually forming a highly robust navigation system that combines human experience characteristics with self-evolution capabilities.

[0132] IV. Autonomous Visual Navigation and Path Planning Steps

[0133] This step is performed by the autonomous navigation and path planning module.

[0134] In this invention, the underwater robot uses multimodal fusion data to construct a three-dimensional voxelized environment map and combines global path planning and local obstacle avoidance control strategies to achieve autonomous navigation in the deep sea environment.

[0135] First, the system is based on a vision and sonar fusion dataset. A 3D environment model is generated through depth estimation and point cloud reconstruction. To facilitate path planning, the environment is discretized into a voxel map.

[0136] ;

[0137] in: For the first Individual factors; Indicate its spatial coordinates; This represents the total number of voxels in the map.

[0138] Each voxel contains environmental feature labels (accessible, obstacles, unknown areas, etc.); the voxel resolution is set to... To balance accuracy and computational load.

[0139] 1. Global Path Planning

[0140] Robot at the starting point With the target point The system plans the optimal path between these paths. It combines Rapidly-exploring Random Tree (RRT) and energy-weighted Dijkstra's algorithm to simultaneously optimize path length and energy cost.

[0141] For any adjacent nodes The path cost function is defined as follows:

[0142] ;

[0143] in: For the node To the node The overall cost; The distance is Euclidean.

[0144] The estimated energy consumption can be calculated from propulsion power and drag coefficient:

[0145] ;

[0146] in, To increase power, This is the hydrodynamic drag coefficient; They are slave nodes Movement to node The start and end times, For a moment The robot's forward speed; is a weighting coefficient used to balance shortest path and optimal energy.

[0147] The RRT algorithm is used to quickly explore the feasible space and generate a set of candidate paths. .

[0148] Subsequently, Dijkstra's algorithm is based on the cost function. Perform global optimization on the candidate paths to find the optimal path:

[0149] .

[0150] The output is a sequence of nodes. Each node contains three-dimensional pose information and an energy budget.

[0151] 2. Local obstacle avoidance and model predictive control (MPC)

[0152] During the robot's execution of the global path, local obstacle avoidance control is achieved through real-time trajectory correction via Model Predictive Control (MPC).

[0153] The kinematic equations of a robot can be described as follows:

[0154] ;

[0155] in: This is the robot's state vector; For control inputs, including forward speed Vertical velocity With angular velocity .

[0156] The control objective of MPC is to minimize the prediction time window. The cost function for trajectory deviation and energy loss within the range is defined as:

[0157] ;

[0158] in: For reference path nodes; These are the weight matrices for the state error and the control input, respectively.

[0159] The optimization objective is to find the optimal control sequence:

[0160] .

[0161] And only the first control input is executed. This enables rolling optimization control.

[0162] This control method enables the robot to correct its attitude when disturbed by ocean currents or encounters dynamic obstacles, ensuring track stability and maintaining a safe distance.

[0163] 3. Autonomous policy switching under communication interruption

[0164] Considering the limited bandwidth and significant latency of underwater acoustic communication in the deep-sea environment, this invention incorporates an autonomous switching mechanism in the system's communication module. When a communication interruption lasts for more than 30 seconds, the system automatically activates the local policy network. :

[0165] ;

[0166] in: For local reinforcement learning value function; It indicates the current local environmental state of the robot, including the distribution of obstacles in the vicinity and the remaining energy; For state The action to output.

[0167] After communication is restored, the robot synchronizes the task log and status trajectory back to the master control terminal. The master control system uses the delayed reward mechanism to update the global policy network parameters, thereby realizing a dynamic closed loop of "self-decision-relearning after link failure".

[0168] Through the aforementioned autonomous visual navigation and path planning mechanisms, underwater robots can achieve adaptive trajectory optimization and risk avoidance in complex terrain and strong interference environments, and possess dual-layer control capabilities of global planning and local correction, thereby ensuring the stability and safety of long-term deep-sea missions.

[0169] V. Strategy Verification and Iterative Optimization Steps

[0170] This step is performed by the strategy optimization module.

[0171] In this invention, after completing deep-sea operations, the underwater robot verifies and continuously optimizes its strategy through a human-machine collaborative delayed reward mechanism. This mechanism constitutes a closed-loop self-evolving training system of "collection-learning-verification-relearning".

[0172] First, after completing a full task, the robot transmits multimodal data (images, sonar, control signals, energy consumption information, etc.) recorded during the operation back to the main control system. Seafarers then comprehensively evaluate the operation results in a virtual playback interface, generating a delayed reward signal. The reward function is calculated based on multiple dimensions, including task completion, operational safety, energy efficiency, and data validity, and is defined as follows:

[0173] ;

[0174] in:

[0175] ): This represents the accuracy of the navigation path completion, defined as the inverse ratio of the error in the robot's arrival at the target point:

[0176] ;

[0177] in, The end point of the mission. For the target point, This represents the maximum tolerable error.

[0178] Operational safety factor, based on the minimum obstacle distance during the task. calculate:

[0179] ;

[0180] in, To set a safety threshold.

[0181] Energy efficiency index, defined as the inverse ratio of energy consumption per unit task distance:

[0182] ;

[0183] in, This represents actual energy consumption. This is the maximum permissible energy budget.

[0184] Data quality scoring is a comprehensive evaluation based on the clarity of the acquired images, sonar coverage, and information completeness.

[0185] After calculation using the above function, the delayed reward It reflects the overall performance of the task and serves as a guiding signal for global model optimization.

[0186] 1. Delayed Feedback Updates in Policy Networks

[0187] After receiving the delayed reward, the system performs gradient optimization updates on the reinforcement learning policy network parameters. The update rules are as follows:

[0188] ;

[0189] in: For the first The policy network trained after the sub-task; For the updated policy network; The learning rate has a range of values. This is used to control the intensity of the update. This indicates that the policy parameters are adjusted according to the direction of the delayed reward gradient;

[0190] The updated direction allows future strategies to achieve a higher expected reward in similar tasks:

[0191] .

[0192] 2. Self-evolutionary training closed-loop mechanism

[0193] The system continuously accumulates experience and feedback data during multiple rounds of task execution, forming an iterative training loop, such as... Figure 3 As shown.

[0194] In the In each round of tasks, the robot's strategy is... Updated parameters are generated after task execution and reward feedback. Its update formula can be further expressed as:

[0195] ;

[0196] in: Time discount factor; The first term of the learning rate (policy update step size coefficient) is the cumulative optimization term for immediate rewards; the second term is the long-term guidance term for delayed rewards. This is the delayed reward impact coefficient, used to balance short-term behavior and long-term performance.

[0197] 3. Formation of strategy stability and robustness

[0198] After multiple rounds of task loops and delay feedback optimization, the system automatically filters out high-reward strategy fragments and adds corresponding samples to the experience base.

[0199] Once the strategy converges, the robot can demonstrate autonomous decision-making capabilities approaching those of human experts even in unknown environments.

[0200] Finally, after After self-evolutionary training, the robot develops a stable and highly robust set of autonomous navigation strategies:

[0201] ;

[0202] .

[0203] This strategy set has the following characteristics:

[0204] 1. Stability: Under different deep-sea topography and environmental disturbances, the navigation error variance is less than 5%;

[0205] 2. Adaptability: The control strategy can be dynamically adjusted according to the task objectives and environmental characteristics;

[0206] 3. Transferability: It can be migrated to different sea areas or equipment platforms while maintaining high-performance execution.

[0207] Through the verification and iterative optimization of the above strategies, this invention constructs an intelligent closed-loop system with self-evolution capabilities, enabling underwater robots to continuously improve their perception, decision-making, and execution performance in long-term missions, ultimately achieving fully autonomous navigation and mission optimization in deep-sea environments.

Claims

1. An autonomous visual navigation method for an underwater robot based on experience knowledge transfer, characterized in that, The method comprises the following steps: (1) collecting visual data, sonar data and inertial navigation data; The visual data, sonar data and inertial navigation data are fused through a timestamp synchronization mechanism to construct a three-dimensional environment model; and a geometric registration function is used to realize spatial consistency of multi-modal data; Collecting a gaze point by a virtual reality device integrated with eye movement tracking , gaze time and gaze shift trajectory; (2) performing a non-distortion gaze point reconstruction on the collected equi-rectangular projection images, dividing a sphere into sub-blocks, and minimizing distortion errors by a mapping matrix to obtain spherical gaze point data with gaze errors less than a threshold value: ； wherein, represents the pixel point coordinate in the spherical coordinate; represents the corresponding pixel point in the ERP plane coordinate; represents the nonlinear transformation from the ERP plane to the spherical coordinate; is the global distortion loss function, which is solved by gradient descent; Meanwhile, action sequences and time intervals are recorded to construct behavior sequences: ； wherein, is the gaze point position at time ; is the action class performed at time ; is the time interval between consecutive actions; is the behavior data sequence with a time length of ; The time sequence features of the behavior data sequences are extracted by a long short-term memory network (LSTM) to generate experience feature vectors: ； wherein, represents an empirical embedding vector of dimension ​ A human operation strategy template is established through an imitation learning network as an initial prior strategy for reinforcement learning: ； wherein, represents a seafarer decision strategy in a state ; represents a probability of performing an action under experience feature constraints; (3) Robot in environment state Select an action Obtain a reward and update the policy according to the following rules: ； wherein, is a policy network at time ; is a reward function at time ; denotes a gradient operator on the policy parameters; denotes an expectation operation; is a learning rate; A double-layer experience memory mechanism is established, short-term experience replay is used for near-task updating, and a long-term experience library is used for cross-task transfer; and the parameter fusion rule is: ； wherein, is an empirical weight coefficient; is a human experience network parameter; is a reinforcement learning network parameter; is a fusion parameter; (4) generating a voxelized three-dimensional map based on multi-modal data: ； wherein, is the first voxel; denotes its spatial coordinates; is the total number of voxels in the map; A global path is generated by combining a rapidly-exploring random tree and an energy-weighted Dijkstra algorithm, and the path cost function is: ； wherein, is a distance, is an energy consumption, are weight coefficients of distance and energy consumption, respectively; are two adjacent nodes on the path.

2. The method of autonomous visual navigation of an underwater robot based on experience knowledge transfer of claim 1, wherein, Local obstacle avoidance adopts model predictive control for attitude adjustment to realize real-time trajectory correction; the cost function of the model predictive controller is: ； wherein, is a reference path node; are weight matrices for state error and control input, respectively; is a prediction time window; is a control input; is a robot state vector.

3. The method of autonomous visual navigation for an underwater robot based on experience knowledge transfer of claim 1, wherein, The reward function is: ； wherein, is a navigation accuracy reward, is a safety obstacle avoidance reward, is an energy consumption optimization reward, is an experience consistency reward; , , , are weight coefficients of , , , respectively.

4. The method of autonomous visual navigation for an underwater robot based on experience knowledge transfer of claim 1, wherein, After the task is completed, the sailor generates a delay reward according to video playback: ； wherein, represents a navigation path completion accuracy; represents a work safety coefficient; represents an energy consumption efficiency index; represents a data quality score; The system performs closed-loop parameter updating: ； wherein, is the first task-trained policy network; is the updated policy network; is the learning rate used to control the update strength; denotes adjusting the policy parameters according to the delayed reward gradient direction.

5. The method of autonomous visual navigation of an underwater robot based on experience knowledge transfer of claim 4, wherein, When the communication interruption exceeds a set time, the robot automatically switches to a local strategy : ； wherein, is a local reinforcement learning value function; According to the current environmental characteristics, the navigation task is independently completed.

6. The method of claim 5, wherein, The local strategy network automatically switched during the communication interruption is initialized by the strategy parameters with the highest delay reward in the historical tasks; the weight of each index in the delay reward function satisfies: 。 7. An autonomous vision navigation system for an underwater robot based on experience knowledge transfer, characterized in that, The system comprises a multi-modal environment perception and data acquisition module, an experience extraction and modeling module, a reinforcement learning and experience transfer module, an autonomous navigation and path planning module, and a strategy optimization module; the multi-modal environment perception and data acquisition module is used for multi-modal environment perception and data acquisition, and processes the collected data; the experience extraction and modeling module is used for human experience feature extraction and modeling; The reinforcement learning and experience transfer module is used for reinforcement learning and experience transfer of the robot; the autonomous navigation and path planning module is used for path planning and autonomous visual navigation of the robot; the strategy optimization module is used for strategy verification and iterative optimization; when the system is running, the steps of the method in any one of claims 1-6 are realized.

Images