A communication, guidance, and sensing collaborative system and method for autonomous precision operation of underwater robots near targets

By establishing a collaborative state among the underwater robot end, communication link, and shore-based monitoring terminal, the problem of independent perception, guidance, and communication modules in near-target operations of underwater robots is solved. Closed-loop collaboration is achieved under low visibility and strong disturbance conditions, reducing the risk of misoperation and prioritizing current action judgment and safety supervision.

CN123123394APending Publication Date: 2026-11-17WUHAN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611395726.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-09
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In existing underwater robots, the perception, guidance and control and communication modules are independent of each other in near-target autonomous precision operations. This leads to problems such as target pose uncertainty due to optical image degradation, high collision risk, communication delay and insufficient information.

Method used

Establish data connections between the underwater robot, communication link, and shore-based monitoring terminal to form a communication-guided-sensing collaborative state. Determine operational conditions based on perception reliability, generate control actions to be executed, prioritize the transmission of key visual evidence, and utilize shore-based monitoring commands to form a closed-loop collaboration.

Benefits of technology

It enables closed-loop collaboration of underwater robots under low visibility and strong disturbance conditions, reduces the risk of misoperation, prioritizes current action judgment and safety supervision, avoids module disconnection, and forms an interpretable autonomous precision operation system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123123394A_ABST
    Figure CN123123394A_ABST
Patent Text Reader

Abstract

This invention discloses a communication, guidance, and perception collaborative system and method for autonomous precision operation of an underwater robot near a target. The system includes: the robot collecting target environment, body, and link information to form a collaborative state encompassing the target's relative pose, perception reliability of key areas, end-effector alignment error, and link quality; generating a work permit or active re-observation command based on perception reliability; generating key visual evidence based on precise guidance actions; and determining the feedback content and protection strength based on precision operation risks and link quality. The shore-based system returns commands for continuation, re-observation, target correction, remote assistance, or safe exit based on the collaborative state, guidance actions, and key visual evidence, updating subsequent perception, guidance, and communication states. This method establishes a closed loop between perception, precise guidance, evidence feedback, and shore-based feedback, improving the safety and reliability of near-target alignment, contact, and operation under low-bandwidth underwater acoustic conditions.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the fields of intelligent underwater robot operation, underwater machine vision, underwater communication, autonomous control and remote supervision, and specifically to a communication, guidance, control and perception collaborative system and method for autonomous precision operation of underwater robots near targets. In particular, it is a communication, guidance and control and perception collaborative system and method that uses the operation status as a link to enable the interaction of perception results, guidance actions, communication resources and shore-based commands, and is aimed at autonomous precision operation of underwater robots near targets. Background Technology

[0002] Tasks such as underwater resource surveys, marine engineering maintenance, coral reef sampling, pipeline inspection, and underwater structure detection typically require underwater robots to perform autonomous alignment, stable tracking, precise grasping, localized inspection, or sample collection upon approaching the target. Near-target operations are not a single perception or control process, but rather a closed loop continuously comprised of target observation, operational condition assessment, action generation, information transmission, and shore-based monitoring; changes in the state of any stage can alter the subsequent operational methods.

[0003] Light attenuation, backscattering, suspended particles, non-uniform illumination, and color shifts in the underwater environment can lead to low contrast, blurring, color cast, and loss of detail in optical images. Existing systems often send processed images directly to the guidance and control module without considering whether there is reliable observation data for the target's critical areas as a condition for action permission. As a result, they may perform approach or contact actions when the target's pose, contact boundaries, or local obstacle relationships are still uncertain, causing alignment deviations, collisions, slippage, or target damage.

[0004] Current underwater robots primarily adjust their movements based on target pose or mission status, failing to fully utilize the perceptual reliability of degraded underwater images to assess operational conditions. When observations of critical target areas are unreliable, there is a lack of collaborative mechanisms for coordinated action constraints, active re-observation, information feedback, and shore-based confirmation. Under conditions of ocean current disturbances, partial obstruction, or changes in target status, even if a single control algorithm can output a movement, it is difficult to guarantee that the movement has sufficient perceptual basis and a continuously effective safety boundary.

[0005] On the other hand, long-distance underwater communication mainly relies on bandwidth-limited and time-varying underwater acoustic links. Continuously transmitting complete high-definition images would cause delays in critical states; transmitting only compressed images based on texture complexity or target area could not guarantee that the transmitted information was sufficient to explain the robot's current intended action. If the communication module does not understand the operational phase, the action to be performed, and its risks, it cannot prioritize the transmission of information necessary for shore-based confirmation of the action under limited link resources.

[0006] In addition, existing underwater robots typically adopt a modular architecture. Although there is data transmission or local feedback between perception, guidance and control and communication, each module processes data separately based on its own state. There is still a lack of a cross-module collaboration mechanism driven by perception reliability, operational risk and communication link status.

[0007] Therefore, there is an urgent need for a communication, guidance, and sensing collaborative system and method for near-target autonomous precision operations. Summary of the Invention

[0008] The purpose of this invention is to address the problem that sensing, guidance, communication, and shore-based monitoring are independent in the prior art, and to provide a communication, guidance, and sensing collaborative system and method for autonomous precision operation of underwater robots near targets.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A communication, guidance, and perception coordination method for autonomous precision operation of an underwater robot near a target includes the following steps: Step 1: Establish data connection between the underwater robot end, communication link and shore end or mother ship end monitoring terminal, collect target environment, robot body and communication link information, and form a communication guidance and control perception collaborative state; Step 2: Determine whether the operation conditions are met based on the communication guidance and sensing coordination status; if the conditions are met, generate an operation permit; if not, prohibit new contact actions and trigger a re-observation. Step 3: After obtaining the work permit, generate the control action to be executed and write the control action to be executed and its uncertainty into the communication control perception coordination state; when the perception state changes or the safety constraints are not met, restrict, suspend or cancel the control action to be executed. Step 4: Determine key visual evidence based on the control action to be executed; Step 5: Determine the content to be transmitted, the transmission order, the encoding accuracy, and the protection strength based on the importance of the key visual evidence, the current operational risk, the operational stage, and the quality of the communication link; when communication resources are limited, prioritize the transmission of status metadata and key visual evidence necessary to support the current action judgment. Step 6: The shore-based or mother ship-based monitoring terminal generates a monitoring instruction based on the received communication guidance and perception coordination status, the guidance and control actions to be executed, and key visual evidence. The underwater robot updates the communication guidance and perception coordination status based on the monitoring instruction and re-enters the perception judgment, guidance and control execution, communication feedback, or safety handling process, forming a closed-loop interaction.

[0010] This communication-guided perception-coordinated method establishes causal connections between perception credibility and operational permission, between actions to be executed and transmitted evidence, between operational risks and link quality and autonomy level, and between shore-based monitoring commands and subsequent observations and actions, enabling underwater robots to form closed-loop coordination under conditions of low visibility, strong disturbances, and low bandwidth.

[0011] Furthermore, the operation includes at least one stage of target approach, end-effector alignment, controlled contact, and target manipulation; the operation conditions include at least the target critical operation area perception confidence reaching a preset threshold, the target pose uncertainty and the position and attitude errors of the end effector being lower than corresponding thresholds, the local collision risk being lower than a preset threshold, and the predicted task success probability being higher than a preset threshold.

[0012] Furthermore, the underwater robot acquires the original underwater image and inputs it into an image inpainting network that includes a low-frequency recovery branch, a high-frequency detail recovery branch, and a spatial reliability fusion unit. This network corrects color cast, illumination, fogging, and contrast, and compensates for target contact boundaries, texture, and local blurring, resulting in the inpainted image. , in, I o , I r These are the original underwater image and the restored image, respectively. R l , R h Residual recovery for low and high frequencies, respectively. W l , W h These are spatial reliability weights; the perceived credibility of the target's key operational region is generated based on multi-scale feature consistency, residual amplitude, and multi-frame observation consistency.

[0013] Furthermore, the steps for generating the control actions to be executed include: generating basic control actions through expert-based basic strategies. Residual compensation actions with limited amplitude are generated through an online residual strategy. The residual gating coefficients are determined based on perceived credibility, policy uncertainty, and security constraints. To obtain the final control action : , When joint limits, end contact force, alignment error, collision distance, or target protection constraints do not meet safety requirements, Set to 0.

[0014] Furthermore, the steps for determining the key visual evidence include: The generated control actions to be executed are read, and the repaired image is divided into target body region, contact boundary region, robotic arm region, obstacle region, abnormal region and background region to form candidate visual units that may be related to the current action; Candidate visual units are occluded, downsampled, or have their features set to zero, respectively. Calculate the action counterfactual contribution A of the i-th candidate visual unit. i : , Among them, a t P t R t The actions, success probabilities, and risks corresponding to the original input. , and Let represent the control action, task success probability, and operational risk after removing the i-th candidate region, respectively; A i Candidate visual units that exceed a preset threshold are identified as key visual evidence.

[0015] Furthermore, the steps for acquiring the key visual evidence necessary to support the current action determination are as follows: Read the generated key visual evidence and its counterfactual contribution to action, extract global frequency domain features from the evidence to be returned, and obtain multi-scale time-frequency features of low frequency and high frequency through two-dimensional discrete wavelet transform. Based on the current underwater acoustic channel state, the global frequency domain features and multi-scale time-frequency features are fused to obtain fused features, which are then mapped to discrete symbols s in a preset modulation set: , In the formula, F m Represents the fused feature vector. Represents the feature transformation / encoding function. Indicates a quantizer; Based on the counterfactual contribution of actions, current operational risks, and communication link quality, different visual evidence is assigned transmission priorities and redundancy protection strengths. When communication links are limited, priority is given to ensuring reliable backhaul of state metadata, target boundaries, contact areas, and obstacle relationships. When communication links are improved, high-resolution details and complete scene information are supplemented.

[0016] Furthermore, the steps for assessing operational risks include: By summarizing the perception reliability of the communication-guided perception coordination state, the action uncertainty of the guidance action to be executed, the key visual evidence, and the communication link quality, the current operational risk is calculated. Represented as: , in, This represents the average credibility of the target's key areas. This represents the uncertainty of the target pose or control strategy. Indicates the risk of a localized collision. The terms α, β, γ, δ, and ε represent the probability of success in predicting a task. ‌ These represent the weighting coefficients; When the job risk is below the first threshold, the robot autonomously executes the job according to the job permission and sends back status metadata; when the job risk is between the first and second thresholds, it pauses new contact actions and returns to the execution to re-observe. When the operational risk is between the second and third thresholds, the underwater robot maintains a safe posture, transmits key visual evidence to support the judgment of the action to be guided, and waits for confirmation or remote assistance from the shore. When the operational risk exceeds the third threshold, the current action is stopped and the robotic arm is withdrawn, the robot moves backward, or it exits safely.

[0017] A communication, guidance, and perception coordination system for autonomous precision operation of an underwater robot near a target is provided to realize the communication, guidance, and perception coordination method for autonomous precision operation of an underwater robot near a target as described above. The system includes an underwater robot end, a communication link, and a monitoring terminal at the shore end or the mother ship end. The underwater robot terminal includes at least a multi-source perception and visual repair module, a target state estimation and active observation module, a guidance and control decision and safety execution module, a collaborative state and operation risk assessment module, a key visual evidence generation module, and a communication coding and scheduling module. These modules are connected through a unified communication, guidance, perception, and collaborative state mechanism: perception results are used to determine operation permits, operation permits are used to constrain guidance and control actions, actions to be executed are used to determine key visual evidence, operation risks and communication link quality are used to determine feedback and control methods, and supervision instructions from the monitoring terminal are used to update subsequent perception, guidance, or safety states.

[0018] Furthermore, the collaborative state is established with the operation risk assessment module to establish a communication-guided-sensing collaborative state z. t : , Where, x t For the target's relative pose and end-effector alignment state, Σ t Let c be the target pose uncertainty. t To perceive credibility for the target's critical operational areas, u t To reduce the uncertainty of fine-tuning control actions, d t For environmental disturbances and contact states, q t For communication link quality, h t This is the precision operation stage.

[0019] Furthermore, the key visual evidence includes a state metadata layer, a minimum visual evidence layer for the operation, and a detail enhancement layer; the minimum visual evidence layer for the operation includes a predetermined operation position where the counterfactual contribution of the action exceeds a threshold, the target contact boundary, the relative positional relationship between the end effector and the target, the relationship of local obstacles, or the sonar structure, and the communication coding and scheduling module selects the data level to be transmitted back based on the operation risk and the quality of the communication link.

[0020] Compared with existing technologies, the beneficial effects of this invention are: 1. The communication, guidance, control, and perception coordination method for autonomous precision operation of underwater robots near targets connects perception, guidance, communication, and shore-based supervision in a unified and coordinated state, transforming each link from fixed serial operation to event-triggered closed-loop operation; 2. Perception reliability is used as a prerequisite for guidance actions, triggering active re-observation when observation data is insufficient, reducing the risk of misoperation caused by unreliable visual information; 3. Key visual evidence is determined by the dependence of the guidance actions to be executed on visual information, allowing limited bandwidth to prioritize current action judgment and safety supervision; 4. Through the setting of the steps in this method, the operational risk and communication link quality jointly determine the autonomous execution, re-observation, shore-based confirmation, remote assistance, or safe exit state, avoiding the disconnect between the communication module and operational decisions; enabling shore-based supervision commands to update subsequent perception, guidance, and communication processes, thereby forming an interpretable, superviseable, and safely degradeable autonomous precision operation system for near targets. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating a communication, guidance, perception, and collaborative system and method for autonomous precision operation of an underwater robot near a target, according to the present invention. Figure 2 This is a schematic diagram of the multi-source perception, visual integrated repair, and repair credibility generation process in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the expert prior guidance and online residual correction module in an embodiment of the present invention; Figure 4 This is a schematic diagram of the key visual evidence generation process based on the counterfactual contribution of actions in an embodiment of the present invention; Figure 5 This is a schematic diagram of the underwater acoustic joint coding and discrete symbol output process of global frequency domain and wavelet time-frequency domain fusion in an embodiment of the present invention; Figure 6 This is a schematic diagram of the closed-loop interaction process between the underwater robot and the shore-based monitoring terminal in an embodiment of the present invention. Detailed Implementation

[0022] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Example 1: A communication, guidance, and perception coordination method for autonomous precision operation of an underwater robot near a target, combining... Figures 1-6 As shown, it includes the following steps: Step 1: Establish data connections between the underwater robot, the communication link, and the shore-based or mother ship-based monitoring terminal; collect information on the target environment, the robot itself, and the communication link to form a communication-guided-perception-coordinated state; the communication-guided-perception-coordinated state includes at least the coordinated state of the target's relative pose, the perception reliability of the target's key operating areas, the alignment error of the end effector, the uncertainty of the guidance and control actions, and the quality of the communication link. This coordinated state serves as the basis for perception judgment, guidance and control decision-making, communication scheduling, and shore-based monitoring within the same work cycle. Step 2: Determine whether the operation conditions are met based on the communication guidance and sensing coordination status; if the conditions are met, generate an operation permit; if the conditions are not met, prohibit new contact actions and trigger re-observation, i.e., adjust the observation position, attitude, lighting or imaging parameters and then re-observe. Specifically, the target state and the perception reliability of key areas of the target are obtained based on multi-source sensing information, and the (precision) operation conditions are determined based on the perception reliability, target pose uncertainty and alignment error. Step 3: After obtaining the work permit, generate the fine-tuning actions to be executed for target approach, end-effector alignment, controlled contact, or target operation based on the target's relative pose, alignment error, underwater robot body state, and safety constraints. Write the fine-tuning actions and their uncertainties into the communication guidance and sensing coordination state. If the sensing state changes or the safety constraints are not met, restrict, pause, or cancel the fine-tuning actions to be executed. Step 4: Based on the control action to be executed, determine key visual evidence; perform counterfactual perturbation on candidate visual units related to the control action to be executed (fine control action), compare the control action before and after the perturbation, the predicted success probability of the task and the operational risk, and identify key operating areas, contact boundaries, end effector status, local obstacle relationships or abnormal information that lead to changes in action, a decrease in success probability or an increase in risk as key visual evidence; Step 5: Determine the content to be transmitted, the transmission order, the encoding accuracy, and the protection strength based on the importance of the key visual evidence, the current operational risk, the operational stage, and the quality of the communication link; when communication resources are limited, prioritize the transmission of status metadata and key visual evidence necessary to support the current action judgment. Step 6: The shore-based or mother ship-based monitoring terminal generates a monitoring command based on the received communication, guidance, perception, and coordination status, the guidance and control actions to be executed, and key visual evidence. The underwater robot updates the communication, guidance, perception, and coordination status based on the monitoring command and re-enters the perception judgment, guidance and control execution, communication feedback, or safety handling process, forming a closed-loop interaction. That is, the underwater robot updates the coordination status based on the monitoring command and returns to the corresponding process in steps 2 to 5, forming a closed loop of communication, guidance and control, perception, and shore-based monitoring.

[0024] The communication, guidance, control, and perception collaboration method for autonomous precision operation of underwater robots near targets connects perception, guidance, communication, and shore-based supervision in a unified collaborative state, transforming each link from a fixed, sequential operation to an event-triggered closed-loop operation. Perception reliability is used as a prerequisite for guidance actions, triggering active re-observation when observation data is insufficient, reducing the risk of misoperation caused by unreliable visual information. The dependence of the guidance actions to be executed on visual information is used to determine key visual evidence, ensuring that limited bandwidth is prioritized for current action judgment and safety supervision.

[0025] By setting up the above steps, the operational risk and the quality of the communication link jointly determine the autonomous execution, re-observation, shore-based confirmation, remote assistance, or safe exit status, avoiding the disconnect between the communication module and operational decisions; and enabling shore-based monitoring commands to update subsequent sensing, guidance, and communication processes in reverse, thereby forming an interpretable, superviseable, and safely degradeable near-target autonomous precision operation system.

[0026] Further, in step 2, the perception processing simultaneously outputs the target observation result and the perception confidence level corresponding to the target observation result; the perception processing may employ image enhancement, image inpainting, multi-frame fusion, multimodal fusion, or a combination thereof. In a preferred embodiment, underwater image inpainting includes low-frequency restoration and high-frequency detail restoration, and is fused using spatial reliability weights. The operation (precision operation) includes at least one stage of target approach, end-effector alignment, controlled contact, and target manipulation; the operation conditions include at least the perception confidence level of the target's critical region reaching a preset threshold, the target pose uncertainty being lower than a preset threshold, the continuity of the target contact boundary meeting preset conditions, the local collision risk being lower than a preset threshold, and the predicted task success probability being higher than a preset threshold.

[0027] Furthermore, in step 3, the generation of the control action to be executed can employ model control, visual servoing, teaching learning, reinforcement learning, or a combination thereof. In a preferred embodiment, a basic action is generated using an expert-based strategy, and limited-amplitude compensation is performed using a safety-constrained online residual strategy. When the perception reliability decreases, the uncertainty of the control strategy increases, or the safety constraints are not met, the compensation amplitude is reduced, and the action to be executed is paused or canceled.

[0028] Furthermore, in step 4, key visual evidence is determined according to the dependency relationship between visual information and the action to be performed; the dependency relationship can be calculated based on the differences in guided actions before and after the candidate visual region is missing, changes in the predicted success probability of the task, and changes in operational risk. The stronger the dependency relationship, the higher the priority of the candidate visual region being transmitted back.

[0029] Furthermore, in step 5, the communication scheduler selects the data level, transmission order, encoding precision, and protection strength to be transmitted back based on the importance of the key visual evidence, operational risk, and communication link quality. In a preferred embodiment, the global frequency domain features and multi-scale wavelet time-frequency features of the key visual evidence are extracted, the fused features are mapped to discrete symbols, and unequal importance protection is implemented for different visual evidence.

[0030] Furthermore, in step 6, the closed-loop state includes at least the autonomous execution state, the active re-observation state, the shore-based confirmation or remote assistance state, and the safe exit state; when the operational risk increases or the communication link is insufficient to support the current supervision needs, new contact actions are restricted and re-observation or shore-based confirmation is triggered; after the shore-based supervision command is returned, the underwater robot updates the observation target, action permission, communication content, or safety status for the next cycle accordingly.

[0031] In this embodiment, step 1 includes the following specific steps: Step 1.1: Set up a multi-source perception and visual repair module, a target state estimation and active observation module, a guidance and control decision and safety execution module, a collaborative state and operation risk assessment module, a key visual evidence generation module, and a communication coding and scheduling module on the underwater robot end; each module interacts with data through collaborative state, rather than running independently. Step 1.2: Set up a monitoring terminal on the shore or on the mother ship to receive the status information, pending actions, key visual evidence and operational risks of the underwater robot through the communication link, and return instructions to the underwater robot to continue execution, re-observe, correct the target, restrict actions, provide remote assistance or safely exit. Step 1.3: Write the target status, perceived credibility, guidance and control actions and their uncertainties, operation stage, operation risk and communication link quality into the collaborative status; when any status quantity exceeds the preset conditions, it triggers subsequent operation permission, re-observation, evidence transmission, shore confirmation or safety handling events.

[0032] Combination Figure 2 As shown, step 2 further includes: Step 2.1: The underwater robot acquires raw underwater images and uses underwater cameras, imaging sonar, inertial measurement units, depth gauges, and robotic arm sensors to acquire information about the target environment and the robot itself. It performs time synchronization and coordinate unification of data from different sensors to obtain target observation results and their perception reliability. Step 2.2: The original underwater image is input into an image inpainting network (underwater image integrated inpainting network) that includes a low-frequency recovery branch, a high-frequency detail recovery branch, and a spatial reliability fusion unit. The low-frequency recovery branch corrects color cast, uneven illumination, fogging, and reduced contrast, while the high-frequency detail branch compensates for target boundaries, textures, and local blurring. This image inpainting network serves as a preferred implementation for obtaining target observation results and perceived credibility, resulting in the following inpainted image: , in, I o , I r These are the original underwater image and the restored image, respectively. R l , R h Residual recovery for low and high frequencies, respectively. W l , W h These are the spatial reliability weights; Step 2.3: Generate the perceived credibility of the target's key operational region based on multi-scale feature consistency, residual amplitude, and multi-frame observation consistency (repair credibility map). The repaired image, repaired confidence map, sonar geometry information, and robot state are used together for target pose estimation.

[0033] When the reliability of the repair of the key area of ​​the target is lower than the preset threshold, or the uncertainty of the target pose exceeds the preset threshold, no work permit is generated. The robot pauses new contact actions and adjusts the observation position, attitude, lighting direction or camera parameters before re-acquiring target information. When the work conditions are met, the robot enters the guidance action generation process.

[0034] In this embodiment, expert prior guidance and online residual correction are used as a preferred guidance and control method, combined with Figure 3As shown, step 3 further includes: Step 3.1: Train the expert's basic strategy using the sensory-motor teaching data collected through manual teleoperation. Based on the current target state Generate basic control actions : ; Step 3.2, utilize online residual strategy Based on the target pose error, ocean current disturbance, repair reliability, and end-point contact status, a residual compensation action with a finite amplitude is generated. : ; Step 3.3: Determine the residual gating coefficients based on perceived credibility, policy uncertainty, and security constraints. To obtain the final control action : , When joint limits, end contact force, alignment error, collision distance, or target protection constraints do not meet safety requirements, Set to 0.

[0035] This embodiment uses action counterfactual contribution to characterize the dependency relationship between visual information and the action to be performed, combined with... Figure 4 As shown, step 4 further includes the following steps: Step 4.1: Read the control action to be executed generated in step 3, divide the repaired image into target body region, contact boundary region, robotic arm region, obstacle region, abnormal region and background region, and form candidate visual units (candidate regions) that may be related to the current action. Step 4.2: Occlude, downsample, or zero out the features of the candidate visual units respectively, re-input the counterfactual input into the guidance strategy and risk assessment model, and compare the actions, task success probability and operational risk before and after the missing area to determine whether the shore-side confirmation of the current action depends on the area. Step 4.3, calculate the action counterfactual contribution A of the i-th candidate visual unit. i : , Among them, a t P t R t The actions, success probabilities, and risks corresponding to the original input. , and Let represent the control action, task success probability, and operational risk after removing the i-th candidate region, respectively; The larger the value, the more important the area is to current control decisions; Step 4.4, place A i Candidate visual units exceeding a preset threshold are identified as key visual evidence. This means that target areas, contact boundaries, obstacle relationships, and necessary sonar information whose action counterfactual contribution exceeds a preset threshold are identified as key visual evidence and sent to the communication encoding and scheduling module along with the action to be executed, operational risks, and status metadata.

[0036] In step 5, communication scheduling and underwater acoustic coding are implemented based on key visual evidence, operational risks, and link quality, such as... Figure 5 As shown; this embodiment uses joint frequency domain and wavelet time-frequency domain coding as a preferred implementation, and this step further includes: Step 5.1: Read the key visual evidence and its action counterfactual contribution generated in Step 4, extract global frequency domain features from the evidence to be returned, and obtain multi-scale time-frequency features of low frequency and high frequency through two-dimensional discrete wavelet transform; , In the formula, This represents the spatial distribution characteristics of four sub-bands: low-frequency approximation, horizontal high-frequency detail, vertical high-frequency detail, and diagonal high-frequency detail. Represents a two-dimensional discrete wavelet transform; Step 5.2: Based on the current underwater acoustic channel state, utilize the attention mechanism to analyze the global frequency domain features. Wavelet time-frequency features To merge: , In the formula, α f With α w These are the weighting coefficients; Step 5.3: Map the fused features to discrete symbols s in a preset modulation set for transmission via the underwater acoustic link. , In the formula, F m Represents the fused feature vector. Represents the feature transformation / encoding function. Indicates a quantizer; Step 5.4: Based on the counterfactual contribution of the action, the current operational risk, and the quality of the communication link, assign transmission priority and redundancy protection strength to different visual evidence; when the communication link is limited, prioritize the reliable transmission of status metadata, target boundaries, contact areas, and obstacle relationships; when the communication link is improved, supplement with high-resolution details and complete scene information.

[0037] Combination Figure 6 As shown, step 6 further includes the following steps: Step 6.1: Summarize the perception reliability of the communication guidance and perception coordination status mentioned in Step 2, the action uncertainty of the guidance action to be executed mentioned in Step 3, the key visual evidence mentioned in Step 4, and the communication link quality mentioned in Step 5, and calculate the current operation risk. Represented as: , in, This represents the average credibility of the target's key areas. This represents the uncertainty of the target pose or control strategy. Indicates the risk of a localized collision. The terms α, β, γ, δ, and ε represent the probability of success in predicting a task. ‌ These represent the weighting coefficients; Step 6.2: When the job risk is below the first threshold, the robot autonomously executes the job according to the job permission and sends back the status metadata; when the job risk is between the first threshold and the second threshold, the new contact action is paused and the robot returns to the execution and re-observes. Step 6.3: When the operational risk is between the second and third thresholds, the underwater robot maintains a safe posture, transmits key visual evidence supporting the judgment of the action to be guided in step 5, and waits for confirmation from the shore or remote assistance; when the operational risk exceeds the third threshold, the current action is stopped and the robotic arm is withdrawn, the robot retreats, or it safely exits. Step 6.4: The shore-based monitoring terminal sends instructions to continue execution, re-observe, correct the target, restrict actions, provide remote assistance, or terminate the operation based on the received coordination status, actions to be performed, and key visual evidence. The underwater robot updates the coordination status according to the instructions and returns to steps 2 and 3 or enters the safety handling process, thereby forming a closed loop between perception, guidance and control, communication and shore-based monitoring.

[0038] In precise near-target operations, the underwater robot first establishes the target state and perception credibility. Only when reliable observational evidence is available in the target's critical areas does it generate a work permit. After the guidance and control module generates the action to be executed, the system extracts key visual evidence needed for shore-based judgment based on this action and arranges data transmission according to operational risk and link quality. When the shore-based system returns instructions for re-observation, target correction, or action restriction, these instructions initiate the next operational cycle, altering the observed target, action permit, or safety status. Low- and high-frequency image inpainting, expert prior knowledge and online residual correction, action counterfactual analysis, and joint frequency domain and wavelet coding are all preferred methods for achieving the aforementioned closed loop without altering the causal connections between each stage.

[0039] Example 2: A communication, guidance, and perception collaborative system for autonomous precision operation of an underwater robot near a target is provided, which is used to realize the communication, guidance, and perception collaborative method for autonomous precision operation of an underwater robot near a target as described above.

[0040] The system includes an underwater robot end, a communication link, and a monitoring terminal on the shore or mother ship. The underwater robot end includes at least a multi-source perception and visual repair module, a target state estimation and active observation module, a guidance and control decision and safety execution module, a collaborative state and operation risk assessment module, a key visual evidence generation module, and a communication coding and scheduling module, which are connected through a unified communication, guidance, perception, and collaborative state: the perception results are used to determine the operation permission, the operation permission is used to constrain the guidance and control actions, the actions to be executed are used to determine the key visual evidence, the operation risk and the communication link quality are used to determine the feedback and control methods, and the monitoring instructions of the monitoring terminal are used to update the subsequent perception, guidance, or safety status.

[0041] Furthermore, the collaborative state establishes a communication-guided perception collaborative state with the operation risk assessment module. This collaborative state is used to jointly characterize the target geometric state, perception reliability, action uncertainty, operation stage, environmental disturbances, and communication link quality within the same operation cycle, and serves as a unified data interface for transmitting states and triggering events between various links. One form of its expression is as follows: , Where, x t For the target's relative pose and end-effector alignment state, Σ t Let c be the target pose uncertainty. t To perceive credibility for the target's critical operational areas, u t To reduce the uncertainty of fine-tuning control actions, d t For environmental disturbances and contact states, q t For communication link quality, h t This is the precision operation stage.

[0042] The multi-source perception and visual restoration module acquires underwater optical images, sonar data, inertial measurement data, robot motion state, and robotic arm state, outputting target observation results and corresponding perception reliability. The perception reliability indicates whether the position, boundary, structure, or texture in the target's key regions is supported by the current valid observations, and is written into the communication guidance and perception coordination state along with the target observation results, rather than merely serving as an image quality evaluation result. Image enhancement, image restoration, multi-frame fusion, or multi-modal fusion can all be used as implementation methods for this module.

[0043] The target state estimation and active observation module receives target observation results and perception reliability, estimates target pose, key operational areas, and local obstacle relationships, and outputs a state allowing operation or re-observation. Operation permission is only sent to the guidance decision and safety execution module when the reliability of the target's key areas and the accuracy of the target state meet preset operation conditions; if the operation conditions are not met, new contact actions are prohibited, and a re-observation command is generated to change distance, viewing angle, attitude, lighting, or imaging parameters.

[0044] In a preferred embodiment of active re-observation, the comprehensive observation gain can be calculated for the candidate observation pose v: , in, This represents the increase in visual credibility of the target's key region due to the candidate observation pose. This represents the increment in the predicted success probability of the task; Indicates the cost of movement or energy consumption; The system represents the collision risk; ω1 to ω4 are configurable weights. The system selects the candidate observation pose with the highest overall observation benefit to perform active re-observation until the target critical area meets the conditions for operational feasibility or reaches the safe stopping condition.

[0045] The guidance and control decision-making and safety execution module generates an action to be executed based on the target state, robot state, and safety constraints only after receiving a work permit. The action to be executed and its uncertainty are simultaneously written into the communication guidance and control perception collaborative state, serving as the basis for the key visual evidence generation module to determine the content to be transmitted back. In a preferred embodiment, the guidance action is generated jointly by a basic guidance and control strategy and a constrained residual compensation strategy, the expression of which is: , in, πR represents the basic guidance and control strategy obtained through expert instruction; πR represents the residual compensation strategy. It represents the residual gating coefficient determined based on perceived credibility, policy uncertainty, and safety constraints; sat represents the safety limit on motion amplitude, velocity, acceleration, or joint torque.

[0046] The collaborative status and operational risk assessment module receives perceived reliability, target state uncertainty, uncertainty of the action to be performed, local collision relationships, operational stage, and communication link quality, and outputs the current operational risk and corresponding state transition events. The operational risk is not only used for display but also to determine whether to continue autonomous execution, trigger re-observation, request shore-based confirmation, enter remote assistance, or perform a safe exit. In one implementation, the operational risk value can be expressed as: ,

[0047] in, This represents the inherent risk of the task, determined by the target's vulnerability, the contact phase, the robotic arm's load, or the relationship with local obstacles; α, β, γ, δ, and ε are weights. The collaborative state and operational risk assessment module controls the state transitions between autonomous execution, active re-observation, shore-based confirmation, remote assistance, and safe pause based on risk thresholds.

[0048] The key visual evidence generation module receives the action to be executed, target observation results, and current operational risks, and determines the target area, contact boundary, robotic arm status, obstacle relationships, or abnormal information required by the shore-side to judge the action to be executed. The key visual evidence is determined by the dependency relationship between visual information and the action to be executed. In a preferred embodiment, the counterfactual contribution of candidate region i to the action can be calculated by performing occlusion, replacement, downsampling, compression, or feature masking on candidate regions (candidate visual units) and comparing the changes in action, success probability, and safety risk before and after processing. ,

[0049] in, and These represent the control actions before and after the disturbance, respectively. and These represent the probability of task success before and after the perturbation, respectively. and These represent the operational risks before and after the disturbance, respectively. , and These represent the corresponding weighting coefficients. The higher the counterfactual contribution of an action, the more important that region is to the robot's current action and safety decisions.

[0050] Furthermore, the key visual evidence generation module organizes hierarchical visual evidence packages according to the current operation stage and the actions to be performed. Each hierarchical visual evidence package includes at least a state metadata layer, a minimum visual evidence layer supporting action judgment, and a detail enhancement layer for result verification. The state metadata layer describes what the robot is about to perform and the current risks; the minimum visual evidence layer describes the observational basis upon which the action relies; and the detail enhancement layer is used to supplement color, texture, background, and complete scene information when link resources permit.

[0051] The communication coding and scheduling module receives hierarchical visual evidence packets, current operational risk, operational stage, and communication link quality to determine whether to transmit back data in this cycle, which data to transmit, and what priority, coding precision, and protection strength to use. Changes in communication link quality not only alter the bitrate but also participate in the autonomous level switching in conjunction with operational risk: when the risk is low and the link is limited, priority is given to transmitting status metadata; when shore-side confirmation is required, priority is given to transmitting the minimum visual evidence supporting the current action judgment; and when the link improves, supplementary detailed enhancement data is provided.

[0052] In a preferred embodiment of underwater acoustic communication, the communication coding and scheduling module fuses the global frequency domain features and discrete wavelet time-frequency features of key visual evidence, and jointly optimizes compression mapping, channel protection, and modulation symbol generation to output transmittable discrete symbols corresponding to the actual modulation set. For visual evidence more strongly dependent on the action to be performed, higher transmission priority, code rate, or redundancy protection is assigned. The above coding method is used to achieve reliable backhaul in the closed loop, but it does not limit the communication coding and scheduling module to adopt other coding methods that can maintain the readability of key evidence.

[0053] The shore-based or mothership-based monitoring terminal receives status metadata and key visual evidence. Combining this with the robot's pending actions and current operational risks, it generates commands to continue execution, re-observe, correct the target, restrict actions, provide remote assistance, or safely exit. These commands are then sent back to the underwater robot, updating the communication, guidance, and perception coordination status. This allows subsequent cycles to re-enter the perception condition judgment, guidance action generation, evidence transmission, or safety handling process until the operation is completed or a safe state is reached.

[0054] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A communication, guidance, and sensing collaborative method for autonomous precision operation of an underwater robot near a target, characterized in that: Includes the following steps: Establish data connections between the underwater robot end, communication link, and shore-based or mother ship-based monitoring terminal; collect information on the target environment, robot body, and communication link to form a communication, guidance, control, perception, and collaborative state. The system determines whether the operation conditions are met based on the communication-guided sensing and coordination status; if the conditions are met, an operation permit is generated; if the conditions are not met, new contact actions are prohibited, and a re-observation is triggered. After obtaining the work permit, a control action to be executed is generated, and the control action to be executed and its uncertainty are written into the communication control perception coordination state; when the perception state changes or the safety constraints are not met, the control action to be executed is restricted, suspended or canceled. Based on the control actions to be executed, key visual evidence is determined; Based on the importance of the key visual evidence, the current operational risk, the operational stage, and the quality of the communication link, determine the content to be transmitted, the transmission order, the encoding accuracy, and the protection strength; when communication resources are limited, prioritize the transmission of status metadata and key visual evidence necessary to support the current action judgment. The shore-based or mother ship-based monitoring terminal generates monitoring instructions based on the received communication guidance and perception coordination status, the guidance and control actions to be executed, and key visual evidence. The underwater robot updates the communication guidance and perception coordination status based on the monitoring instructions and re-enters the perception judgment, guidance and control execution, communication feedback, or safety handling process, forming a closed-loop interaction.

2. The communication, guidance, and perception coordination method for autonomous precision operation of an underwater robot near a target as described in claim 1, characterized in that, The operation includes at least one stage of target approach, end-effector alignment, controlled contact, and target manipulation; the operation conditions include at least the target critical operation area perception confidence level reaching a preset threshold, the target pose uncertainty and the position and attitude errors of the end effector being lower than the corresponding thresholds, the local collision risk being lower than a preset threshold, and the predicted task success probability being higher than a preset threshold.

3. The communication, guidance, and perception coordination method for autonomous precision operation of an underwater robot near a target as described in claim 1, characterized in that, An underwater robot acquires raw underwater images, which are then input into an image inpainting network comprising a low-frequency recovery branch, a high-frequency detail recovery branch, and a spatial reliability fusion unit. This network corrects color cast, illumination, fogging, and contrast, and compensates for target contact boundaries, texture, and local blurring, resulting in a restored image. , in, I o , I r These are the original underwater image and the restored image, respectively. R l , R h Residual recovery for low and high frequencies, respectively. W l , W h These are spatial reliability weights; the perceived credibility of the target's key operational region is generated based on multi-scale feature consistency, residual amplitude, and multi-frame observation consistency.

4. The communication, guidance, and perception coordination method for autonomous precision operation of an underwater robot near a target according to claim 1, characterized in that, The steps for generating the control actions to be executed include: generating basic control actions through expert-based basic strategies. Residual compensation actions with limited amplitude are generated through an online residual strategy. The residual gating coefficients are determined based on perceived credibility, policy uncertainty, and security constraints. To obtain the final control action : , When joint limits, end contact force, alignment error, collision distance, or target protection constraints do not meet safety requirements, Set to 0.

5. The communication, guidance, and perception coordination method for autonomous precision operation of an underwater robot near a target according to claim 1, characterized in that, The steps for determining the key visual evidence include: The generated control actions to be executed are read, and the repaired image is divided into target body region, contact boundary region, robotic arm region, obstacle region, abnormal region and background region to form candidate visual units that may be related to the current action; Candidate visual units are occluded, downsampled, or have their features set to zero, respectively. Calculate the action counterfactual contribution A of the i-th candidate visual unit. i : , Among them, a t P t R t The actions, success probabilities, and risks corresponding to the original input. , and Let represent the control action, task success probability, and operational risk after removing the i-th candidate region, respectively; A i Candidate visual units that exceed a preset threshold are identified as key visual evidence.

6. The communication, guidance, and perception coordination method for autonomous precision operation of an underwater robot near a target according to claim 1, characterized in that, The steps for obtaining the key visual evidence necessary to support the current action determination are as follows: Read the generated key visual evidence and its counterfactual contribution to action, extract global frequency domain features from the evidence to be returned, and obtain multi-scale time-frequency features of low frequency and high frequency through two-dimensional discrete wavelet transform. Based on the current underwater acoustic channel state, the global frequency domain features and multi-scale time-frequency features are fused to obtain fused features, which are then mapped to discrete symbols s in a preset modulation set: , In the formula, F m Represents the fused feature vector. Represents the feature transformation / encoding function. Indicates a quantizer; Based on the counterfactual contribution of actions, current operational risks, and communication link quality, different visual evidence is assigned transmission priorities and redundancy protection strengths. When communication links are limited, priority is given to ensuring reliable backhaul of state metadata, target boundaries, contact areas, and obstacle relationships. When communication links are improved, high-resolution details and complete scene information are supplemented.

7. The communication, guidance, and perception coordination method for autonomous precision operation of an underwater robot near a target according to claim 1, characterized in that, The steps for assessing operational risks include: By summarizing the perception reliability of the communication-guided perception coordination state, the action uncertainty of the guidance action to be executed, the key visual evidence, and the communication link quality, the current operational risk is calculated. Represented as: , in, This represents the average credibility of the target's key areas. This represents the uncertainty of the target pose or control strategy. Indicates the risk of a localized collision. The terms α, β, γ, δ, and ε represent the probability of success in predicting a task. ‌ These represent the weighting coefficients; When the job risk is below the first threshold, the robot autonomously executes the job according to the job permission and sends back status metadata; when the job risk is between the first and second thresholds, it pauses new contact actions and returns to the execution to re-observe. When the operational risk is between the second and third thresholds, the underwater robot maintains a safe posture, transmits key visual evidence to support the judgment of the action to be guided, and waits for confirmation or remote assistance from the shore. When the operational risk exceeds the third threshold, the current action is stopped and the robotic arm is withdrawn, the robot moves backward, or it exits safely.

8. A communication, guidance, and perception collaborative system for autonomous precision operation of an underwater robot near a target, used to implement the communication, guidance, and perception collaborative method for autonomous precision operation of an underwater robot near a target as described in any one of claims 1 to 7, characterized in that, The system includes an underwater robot terminal, a communication link, and a monitoring terminal on the shore or mother ship. The underwater robot terminal includes at least a multi-source perception and visual repair module, a target state estimation and active observation module, a guidance and control decision and safety execution module, a collaborative state and operation risk assessment module, a key visual evidence generation module, and a communication coding and scheduling module. These modules are connected through a unified communication, guidance, perception, and collaborative state mechanism: perception results are used to determine operation permits, operation permits are used to constrain guidance and control actions, actions to be executed are used to determine key visual evidence, operation risks and communication link quality are used to determine feedback and control methods, and supervision instructions from the monitoring terminal are used to update subsequent perception, guidance, or safety states.

9. The communication, guidance, and perception collaborative system for autonomous precision operation of an underwater robot near a target as described in claim 8, characterized in that, The collaborative status is established with the operation risk assessment module to establish a communication-guided perception-collaborative status: , Where, x t For the target's relative pose and end-effector alignment state, Σ t Let c be the target pose uncertainty. t To perceive credibility for the target's critical operational areas, u t To reduce the uncertainty of fine-tuning control actions, d t For environmental disturbances and contact states, q t For communication link quality, h t This is the precision operation stage.

10. The communication, guidance, and perception collaborative system for autonomous precision operation of an underwater robot near a target according to claim 8, characterized in that, The key visual evidence includes a state metadata layer, a minimum visual evidence layer for the operation, and a detail enhancement layer. The minimum visual evidence layer for the operation includes a predetermined operation position where the counterfactual contribution of the action exceeds a threshold, the target contact boundary, the relative positional relationship between the end effector and the target, the relationship of local obstacles, or the sonar structure. The communication coding and scheduling module selects the data level to be transmitted back based on the operation risk and the quality of the communication link.