A control method of an integrated fishing net repairing robot and the fishing net repairing robot

Abstract

This invention relates to the field of marine ranching technology, and provides an integrated control method and robot for repairing fishing nets. The method includes: based on an uncertain heatmap, using an adaptive focusing scanning algorithm to guide a 3D lidar to collect point cloud data of a target area, forming a 3D topography of the target area; acquiring the spectral signal of the fishing net in the target area to determine the net material type, optimizing the tactile probe parameters to control the tactile probe's movement, collecting the vibration response signal of the fishing net to obtain a fishing net health index; determining whether to issue a repair command through multi-source fusion decision-making; and upon receiving the repair command, determining the location of the damaged point based on the 3D topography and the geometric contour of the damaged area, obtaining the robot's planned path through an A search algorithm, and precisely moving to the actual working area to perform the repair operation through adaptive PID control.

Description

Technical Field

[0001] This invention belongs to the field of marine ranching technology, and particularly relates to an integrated control method for a fishing net repair robot and the fishing net repair robot. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] As global fisheries transform towards intelligent and intensive operations, marine ranching, as a core model for the coordinated development of efficient aquaculture and ecological protection, has become an important growth point for the marine economy. Among them, nearshore marine ranches have taken the lead in achieving large-scale development due to their proximity to load centers and convenient operation and maintenance; while deep-sea marine ranches, although richer in wind energy and fishery resources, suffer from frequent damage to fishing nets, which are the core aquaculture facilities, due to long-term effects such as water erosion, organism attachment, and aging of net fibers.

[0004] Damaged fishing nets not only lead to the escape of farmed organisms and economic losses, but can also cause ecological pollution; therefore, efficient maintenance is crucial. Currently, fishing net repair mainly relies on two methods: Firstly, manual diving operations are limited by water depth (usually ≤10m) and water quality visibility, resulting in low efficiency (the repair area per day is less than 20m²). 2 Problems include high safety risks (water flow disturbances can easily cause diving accidents); Secondly, while traditional underwater robot operations can overcome the limitations of artificial environments, they also face technical bottlenecks: they mostly rely on monocular vision or simple sonar detection, which is insufficient in nearshore turbid waters (turbidity > 50 NTU) and uneven lighting conditions to detect minor damage (area < 10 cm²). 2 The recognition rate for special damage such as interface peeling of composite material fishing nets (such as nylon-aramid blends) is less than 80%, and the material properties of the fishing net cannot be obtained at the same time, resulting in a high rate of missed detection and false detection, and the problem of repairs being out of place in the actual working area is easy to occur. Summary of the Invention

[0005] To address the technical problems mentioned above, this invention provides an integrated control method and robot for fishing net repair. By combining an uncertainty perception network with an adaptive focusing scanning algorithm, it achieves accurate identification of minor damage and special damage to composite material fishing nets in turbid water and uneven lighting conditions. Simultaneously, it acquires the material properties and health index of the fishing net and determines whether to issue a repair command through multi-source fusion decision-making. This solves the problems of missed detection, false detection, and limited information in traditional technologies, providing comprehensive data support for path planning and enabling precise movement to the actual working area to perform repair operations.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: The first aspect of the present invention provides an integrated fishing net repair robot control method, comprising: The fishing net image of the target area is acquired, and the geometric contour of the damaged area and the uncertainty heat map are extracted through an uncertainty perception network. Based on the uncertainty heatmap, an adaptive focusing scanning algorithm is used to guide the 3D LiDAR to collect point cloud data of the target area and form the 3D shape of the target area. The spectral signal of the fishing net in the target area is acquired, and the material category of the fishing net is obtained through a lightweight material classification network. Based on the material category, the parameters of the tactile probe are optimized to control the action of the tactile probe, and the vibration response signal of the fishing net is collected. The health index of the fishing net is obtained through vibration spectrum analysis. Based on the geometric contour of the damaged area, the uncertainty heat map, the three-dimensional shape of the target area, the fishing net material type, and the fishing net health index, a multi-source fusion decision is made to determine whether to issue a repair command. Upon receiving the repair instruction, the location of the damage point is determined based on the three-dimensional topography and the geometric contour of the damaged area, using A... The search algorithm obtains the robot's planned path, and through adaptive PID control, it moves the robot to the target damage point to perform the repair operation.

[0007] Furthermore, the adaptive focusing scanning algorithm includes: mapping pixel clusters with values ​​below a threshold in the uncertainty heatmap to the area to be inspected in three-dimensional space, controlling the three-dimensional LiDAR scanning head to automatically perform multi-angle tilt scanning in the area to be inspected and increasing the point cloud sampling frequency; at the same time, a fast scanning mode is adopted for the non-inspection area.

[0008] Furthermore, the A The costs employed by the search algorithm include cumulative cost and heuristic estimation cost.

[0009] Furthermore, the cumulative cost is: g(s') = g(s) + state transition cost; where g(s) is the cumulative cost of the current node s, s' is the neighbor node of the current node s, and the state transition cost is the cost of water navigation, net surface travel, attachment action cost, or detachment action cost.

[0010] Furthermore, the cost of the heuristic estimation is: h(s') = ρ × ||p s' -p goal ||;where ρ is the heuristic weight coefficient, p s' Let p be the position of the neighboring node s'. goal This indicates the location of the damage point.

[0011] Furthermore, the neighbor nodes are generated based on the robot's current attachment state: If the robot's current attachment state is underwater, two types of neighbor nodes can be generated: one is the neighbor node that maintains the underwater state; the other is the attachment state switching node, which generates the mesh state node. If the robot is currently attached to a net surface, it can generate two types of neighbor nodes: one is the neighbor node that maintains the net surface state; the other is the node that switches from the state to the state that generates the underwater state node.

[0012] Furthermore, the robot planning path includes several planning nodes, and each planning node is defined as including the robot's three-dimensional spatial position, robot posture, and attachment status with the fishing net.

[0013] Furthermore, when performing the repair operation, the repair temperature is: T_ref=T0+k1×(T_m-T0); where T0 is the initial setting value, k1 is the material category correlation coefficient, and T_m is the optimal repair temperature corresponding to the material category.

[0014] Furthermore, when performing the repair operation, the suture tension is: F_ref=F0-k2×S; where k2 is the damage area correction coefficient, S is the damage area, and F0 is the basic suture tension.

[0015] A second aspect of the present invention provides a fully submersible fishing net repair robot, which includes a robot body and a host computer mounted on the robot body, and the host computer is configured with an integrated fishing net repair robot control method as described in the first aspect.

[0016] Compared with the prior art, the beneficial effects of the present invention are: This invention achieves accurate identification of minor damage and special damage to composite fishing nets in turbid water and uneven lighting conditions by combining an uncertainty perception network with an adaptive focusing scanning algorithm. At the same time, it acquires the material properties and health index of the fishing net and determines whether to issue a repair command through multi-source fusion decision-making. This solves the problems of missed detection, false detection and limited information in traditional technologies, and provides comprehensive data support for path planning, enabling precise movement to the actual working area to perform repair operations. Attached Figure Description

[0017] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0018] Figure 1 This is an overall structural diagram of an integrated fishing net repair robot according to Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the intelligent adaptive repair execution subsystem structure according to Embodiment 1 of the present invention; Figure 3This is a structural diagram of the uncertainty-aware network of Embodiment 2 of the present invention; Figure 4 This is a structural diagram of the auxiliary output head according to Embodiment 2 of the present invention; Figure 5 This is a flowchart of the multi-source fusion decision-making process according to Embodiment 2 of the present invention; Figure 6 This is the global A of Embodiment 2 of the present invention. Search flowchart; Figure 7 This is a flowchart of the adaptive PID control according to Embodiment 2 of the present invention; Figure 8 This is a schematic diagram of the repair parameter adjustment logic in Embodiment 2 of the present invention; Figure 9 This is a schematic diagram of intelligent task scheduling according to Embodiment 2 of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0020] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0021] Terminology Explanation: Multimodal perception: refers to a perception method that integrates multiple sensor data such as vision, laser, touch, and spectrum.

[0022] Hierarchical reinforcement learning: refers to intelligent algorithms that divide a task into two layers: global planning and local control.

[0023] Example 1 This embodiment provides a fully submersible fishing net repair robot.

[0024] Existing technologies attempt to integrate multiple sensors or optimize control algorithms, but have not formed a fully intelligent closed loop of perception-planning-execution. In particular, there are shortcomings in core aspects such as multimodal data fusion, autonomous decision-making in dynamic environments, and self-adaptation of repair parameters. Therefore, developing a fishing net repair robot with efficient perception, intelligent planning, and adaptive repair capabilities is of great significance for reducing the operation and maintenance costs of marine ranches and improving operational reliability.

[0025] This embodiment provides a fully submersible fishing net repair robot, and an integrated fishing net repair robot control method based on this robot. Figure 1As shown, the system includes a robot body and a multimodal intelligent perception subsystem, a hierarchical reinforcement learning path planning subsystem, and an intelligent adaptive repair execution subsystem mounted on the robot body. It addresses the problems of weak perception capabilities, poor dynamic adaptability, and lack of adaptive repair in traditional fishing net repair techniques, enabling efficient and intelligent maintenance of fishing nets in complex nearshore environments.

[0026] This embodiment provides a fully submersible fishing net repair robot. Through a multimodal perception subsystem mounted on the robot body, it collects geometric features, material properties, and three-dimensional environmental parameters of the damaged area in real time during operation mode where it is in close proximity to the fishing net. The perception data is then fused by a hierarchical reinforcement learning algorithm built into the hierarchical reinforcement learning path planning subsystem to generate a dynamically optimized operation path and repair strategy. Finally, the intelligent adaptive repair execution subsystem completes the precise operation.

[0027] like Figure 1 As shown, the multimodal perception subsystem integrates a high-resolution underwater vision sensor (supporting wide dynamic range imaging), a three-dimensional lidar (to achieve structured light scanning), an array-type tactile sensor (i.e., a tactile probe used to sense contact pressure distribution), and a spectral probe (to identify fiber materials). Through a flexible adjustment mechanism mounted on the robot, the array-type tactile sensor can stably adhere to the surface of the fishing net, achieving high-quality data acquisition.

[0028] The hierarchical reinforcement learning path planning subsystem includes: a high-level task planning unit, which prioritizes and plans global paths for damaged points based on perception results; and a low-level motion control unit, which is responsible for real-time obstacle avoidance, attitude adjustment, and motion compensation.

[0029] The intelligent adaptive repair execution subsystem consists of modular tool heads that support various processes such as stitching, welding, and spraying, and can dynamically adjust operation parameters based on real-time sensing data.

[0030] like Figure 2 As shown, the intelligent adaptive repair execution subsystem includes a robotic arm and a magnetic quick-change interface mounted on the robotic arm. It also includes a variety of tool heads: a precision suturing tool head (supporting 0.3-1.2mm needle switching), an ultrasonic welding tool head (amplitude adjustable from 5-20μm), and an intelligent spraying tool head (spraying thickness controllable from 0.1-1mm). The tool heads are connected to the robotic arm via the magnetic quick-change interface, with a switching time of ≤3 seconds, and can adapt to different damage types and fishing net materials.

[0031] During execution, the robot autonomously adjusts its tool head based on real-time multimodal perception data: for example, when it detects minor damage (area < 10 cm²), it automatically activates a precision sewing tool and reduces tension; when it identifies composite materials, it matches the material-parameter library based on spectral data and adaptively adjusts the welding temperature; when it senses strong water flow (flow velocity > 1.0 m / s), the lower-level controller automatically increases the thruster power to stabilize the posture.

[0032] This embodiment provides a fully submersible fishing net repair robot that deeply integrates a multimodal perception module (vision, lidar, tactile, and spectral), a hierarchical reinforcement learning controller, an adaptive repair actuator, and a propulsion system into a single body.

[0033] The fully submersible fishing net repair robot provided in this embodiment has the following workflow: First, it directly connects to the marine ranch operation and maintenance network through its built-in dual-mode communication unit, employing a dual-mode communication strategy of "wide-area wake-up" and "local operation": it receives task commands via a low-power wide-area link, wakes up and guides the robot to autonomously navigate to the target location; upon arrival, the robot independently completes subsequent tasks using its own perception-decision-execution closed loop, with local communication used only for status feedback and environmental data supplementation. All perception, decision-making, and control calculations are completed in a closed loop within the onboard computing unit, and the entire repair process is driven by its own power source.

[0034] The integrated architecture of this embodiment eliminates the complex coordination problems of multi-terminal systems, greatly improves the system's reliability, response speed and deployment robustness, and realizes closed-loop autonomous operation of the entire process from remote wake-up and autonomous navigation to precision operation in a dynamic marine environment.

[0035] Example 2 This embodiment provides an integrated fishing net repair robot control method. By deeply integrating perception, decision-making, and execution into a single robot platform and relying on its built-in intelligent algorithm to achieve closed-loop control, the method has been verified to improve the fishing net damage recognition rate to over 95% and the repair qualification rate to over 90% without the need for external real-time intervention. The external communication link is only used for task issuance and status feedback and does not need to bear the real-time control data flow, thereby significantly reducing the stringent requirements for communication bandwidth and reliability and reducing the investment costs of system deployment and communication links.

[0036] This embodiment provides an integrated fishing net repair robot control method, applied to a fully submersible fishing net repair robot described in Embodiment 1. During operation, the method includes the following steps: Step 1: After the robot arrives at the target location, the vision sensor acquires images of the fishing net in the target area (i.e., underwater damage images). Through the uncertainty perception network, the robot extracts the preliminary geometric contours of the damaged area (i.e., preliminary contour segmentation map) and uncertainty heatmaps.

[0037] Uncertainty-aware networks are dual-output deep neural networks with self-diagnostic capabilities, used to address the reliability issues in feature extraction caused by uneven underwater image quality.

[0038] like Figure 3 As shown, the uncertainty-aware network, based on the encoder-decoder architecture, connects two output heads in parallel at the end of the decoder: the main output head generates a preliminary contour segmentation map of the damaged area, while the auxiliary output head simultaneously generates an uncertainty heatmap of the same resolution.

[0039] like Figure 4 As shown, in the internal structure design of the auxiliary output head, a parallel lightweight quantum model array mechanism is innovatively introduced: the auxiliary output head is not a single convolutional layer, but is composed of multiple lightweight sub-models with slight differences in structure (e.g., 3-5 sub-networks with different convolutional kernel sizes, different Dropout rates, and different initialization methods) in parallel. These lightweight sub-models share the feature map output by the same decoder, but independently predict the probability of damage in the same region.

[0040] like Figure 4 As shown, the heatmap generation mechanism adopts the principle of prediction ensemble and inconsistency measurement. That is, multiple lightweight sub-models are used to predict the probability of damage to the same area, and the statistical difference of their prediction results is calculated. The larger the difference of a pixel, the greater the disagreement between different lightweight sub-models in judging whether the area is a damage edge, and the higher the uncertainty value.

[0041] Specifically, the uncertainty calculation module calculates the variance or entropy of the prediction results of each lightweight sub-model pixel by pixel. The larger the variance, the more significant the discrepancy between the lightweight sub-models, and the lower the reliability of feature extraction in the corresponding region. This allows the network to not only output "what it is," but also assess "how certain it is," accurately locating ambiguous and suspicious areas caused by water turbidity, mesh shadows, fiber reflections, or interference from attached materials. The innovative value of this design lies in: by deliberately introducing model diversity and measuring their disagreement, uncertainty is transformed from an implicit posterior probability into an explicit, interpretable, structured output. Moreover, a single forward propagation can simultaneously obtain the segmentation result and the uncertainty heatmap, and the computational efficiency meets the real-time requirements of underwater robots. This heatmap serves as the core guiding signal for subsequent multimodal fusion, clearly identifying specific regions in the image where features are unreliable due to occlusion, reflection, or blurring.

[0042] Step 2: Based on the uncertainty heatmap, an adaptive focusing scanning algorithm is used to guide the 3D LiDAR to collect point cloud data of the target area, thereby forming the 3D shape and depth information of the target area.

[0043] This embodiment proposes an adaptive focusing scanning algorithm that plans the scanning path and parameters in real time based on the spatial guidance of the uncertain heatmap.

[0044] The core of the adaptive focusing scanning algorithm lies in combining the two-dimensional uncertainty heatmap with the physical scanning model of the LiDAR to generate a dynamic focusing scanning strategy: First, the high uncertainty pixel clusters in the uncertainty heatmap are mapped to the area to be inspected in the three-dimensional working space; then, the three-dimensional LiDAR scanning head is controlled to automatically perform multi-angle tilt scanning in the area to be inspected and increase the point cloud sampling frequency (for example, from the conventional 100 points / square centimeter to 500 points / square centimeter). At the same time, a low-density fast scanning mode is adopted for non-inspection areas with high visual confidence.

[0045] Through this precise resource deployment based on semantic prior knowledge, the final output of 3D LiDAR is no longer a uniform but potentially redundant point cloud, but a 3D data with varying density and a high correlation between information density and visual suspicious points. After subsequent processing, high-precision 3D topography and depth information that integrates visual prior knowledge can be obtained, fundamentally solving the problem of independent processing of visual and point cloud data and the fragmentation of spatial information in traditional methods.

[0046] Step 3: After the robot arrives at the target location, the spectral probe collects the spectral signal of the fishing net in the target area. Through a lightweight material classification network, the material category of the fishing net is obtained. Based on the material category, the tactile parameter adaptive mapping model is triggered to optimize the tactile probe parameters so that the tactile probe can move and collect vibration response signals. Through vibration spectrum analysis, the health index is obtained.

[0047] In the material identification stage, firstly, reflectance spectrum analysis is performed on the fiber sampling points, and the initial material classification is completed through a pre-trained lightweight classification network. Then, the initial material classification result triggers the tactile parameter adaptive mapping model. The tactile parameter adaptive mapping model embeds a material-tactile parameter dictionary, which presets the mapping relationship between different material categories (nylon, polyethylene, etc.) and their optimal tactile diagnostic parameter sets. For example, for "nylon" material, the tactile parameter adaptive mapping model will call the preset high-frequency micro-amplitude excitation parameter set (such as setting the contact force to 0.5N and the excitation frequency to 200Hz) to configure the precise action of driving the tactile probe. The tactile probe uses these optimized parameters to actively "tape" the test point and collect its vibration response signal. Subsequently, by analyzing the shift, broadening and harmonic composition changes of characteristic peaks in the response spectrum, the microstructural changes of the material (such as molecular chain relaxation and microcrack generation) are quantitatively evaluated. In the end, not only is the material category confirmed, but a health index with a continuous value from 0 (new) to 1 (severe aging) is also output.

[0048] This process transforms tactile perception from a passive reading of surface hardness through "ubiquitous touch" to "feature-enhanced diagnosis" that actively stimulates and analyzes the internal state of materials based on spectral prior knowledge, thus enabling quantitative early detection of material performance degradation.

[0049] Step 4: The precise geometric features guided by the above vision (including three-dimensional shape and depth information, preliminary contour segmentation map and uncertainty heat map of the corresponding area) and the tactile diagnostic results guided by the spectrum (including material category and its health index) will be sent to the multi-source fusion decision module based on improved evidence theory to determine whether to proceed with the next repair or return to re-perception.

[0050] (1) The core innovation of the multi-source fusion decision-making module based on improved evidence theory lies in directly quantifying the inherent uncertainty of the perception process into evidence weights. For example... Figure 5 As shown, this module achieves adaptive fusion of multi-source information through the following mechanism: (A) Construct a basic probability assignment function (BPA) for each type of evidence (3D topography and depth information, preliminary contour segmentation map, material category, etc.), and assign it to the quality assessment index (mean of uncertainty heatmap, health index) output by the previous steps as input, and dynamically calculate the credibility of each piece of evidence through nonlinear mapping.

[0051] (B) The improved DS evidence theory rules are adopted for integration, and the core improvement is the introduction of a mechanism for tracing and re-evaluating conflicting evidence.

[0052] (C) Output three types of deterministic decisions (damage type, material properties, and damage severity) and a unified joint decision confidence level C, which serve as the basis for subsequent control layer judgments.

[0053] (2) The basic probability allocation function is not constructed equally, but rather the quality assessment index output from the previous steps is used as input, and the credibility of each piece of evidence is dynamically calculated through a nonlinear mapping layer.

[0054] Specifically: the credibility α of visual evidence geo Uncertainty heatmap calculation is based on the output of the uncertainty-aware network in step 1. The value of each pixel in the heatmap represents the degree of disagreement among multiple lightweight sub-models regarding whether a point is a broken edge; greater disagreement indicates higher visual uncertainty. Let the mean of the heatmap be... The credibility of visual evidence is then defined as α.geo =1 ,when A higher value indicates that the features in the area are unreliable due to interference from water turbidity, shadows, reflections, etc., and the credibility weight of the evidence of its complete outline is automatically reduced.

[0055] Credibility α of tactile evidence tac The health index H (ranging from 0 to 1, where 0 indicates severe aging and 1 indicates brand new) is obtained through spectral analysis of the vibration response signal collected by the tactile probe in step 3. The credibility of the tactile evidence is defined as α. tac =H. When the health index indicates severe aging of the material (lower H), the certainty of evidence regarding its material strength is weakened.

[0056] Credibility α of spectroscopic evidence spec The material classification results of the spectral probe are less affected by the turbidity of the water body, and the default confidence level is α. spec =0.9. If subsequent cross-validation reveals a conflict between the classification results and other evidence, adjustments can be made dynamically based on the validation results.

[0057] (3) After obtaining the credibility of each piece of evidence, the module uses the improved DS evidence theory rules for fusion. The standard DS fusion rules are well-known in the field and will not be described in detail here; the core improvement of this embodiment lies in the introduction of the following two mechanisms: Mechanism 1: Tracing and Re-evaluating Conflicting Evidence: When the conclusions of different pieces of evidence are highly conflicting (e.g., geometric features indicate extensive tearing while tactile health indicates material toughness), the following processing flow is automatically triggered: ① Retrospection: Locate the source of evidence that caused the conflict, tracing back to the original sensor data and quality indicators; ② Cross-validation: Instruct relevant sensors to add sampling points in the conflict area for verification, for example, schedule the spectral module to perform secondary spectral analysis on the conflict area, or instruct the vision module to re-acquire multi-angle images; ③ Dynamic adjustment: Based on the cross-validation results, the BPA (Basic Probability Assignment Function) of the relevant evidence is adjusted using a discount operation: ′(A)=α m(A) (under the assumption of non-universal set) and m′(Θ) = 1 α+α m(Θ) (hypothesis for the whole set) where the confidence coefficient α is determined by the confidence level of the cross-validation results (range 0~1).

[0058] Mechanism 2: Weighted average iterative fusion To avoid the counterintuitive problems of traditional DS rules when dealing with highly conflicting evidence, this module uses a weighted average method for iterative fusion: ① Calculate the weighted average BPA of all evidence, wi=α i / j Where i is the credibility of the i-th piece of evidence (as mentioned above: α) geo =1 (Visual), α tac =H (tactile sensation), α spec =0.9 (spectral density, dynamically adjustable)); j The sum of confidence coefficients for all evidence is used to normalize the weights; N is the total number of pieces of evidence involved in the fusion, a positive integer (usually 3-5), determined based on the type of sensor actually activated, such as visual evidence, LiDAR evidence, tactile evidence, spectral evidence, etc. ② Using the weighted average BPA as new evidence, iterative self-fusion was performed using the classic DS rule. 1 time (n is the number of pieces of evidence); ③ After obtaining the final fused BPA, select the proposition corresponding to the highest trust level as the decision output: if the damaged trust level is significantly higher than the intact trust level, then perform repair; otherwise, return to re-perceive.

[0059] (4) Based on the above fusion results, the module outputs the following three types of deterministic decisions and a unified joint decision confidence level: ① Damage Type Decision: Determined by visually-driven evidence. After spatially aligning the preliminary contour segmentation map output by the uncertainty perception network with the high-precision 3D topography output by the 3D LiDAR, a geometric morphology classifier (based on features such as contour moments, porosity, and edge roughness) is used to distinguish between regular holes and irregular tears. If the mean value of the uncertainty heatmap for a certain region is higher than a threshold (example value: 0.6), the weight of the contour features of that region in the classification is reduced, and the judgment is instead based on the continuity of 3D depth information.

[0060] ② Material Property Decision: Determined by tactile evidence. The spectral probe obtains the initial material category (e.g., nylon, polyethylene) through a lightweight material classification network. The tactile parameter adaptive mapping model drives the tactile probe to collect vibration response signals based on this category, and outputs a health index H (a continuous value from 0 to 1) after spectral analysis. The fusion module binds the material category with the health index in the output, for example, "Nylon-6, health 0.3".

[0061] ③ Quantitative scoring of damage severity: calculated by combining three types of information.

[0062] Geometric Dimension Score size Definition: Damaged area A (unit: cm²) and maximum aperture D max Weighted sum of (unit: cm): Ssize =W s1 A / A max +W s2 D max / D max，ref ;where A max To determine the maximum damaged area, D max，ref For reference to the maximum aperture, W s1， W s2 Preset weights (example values: 0.6, 0.4).

[0063] Depth rating S depth Definition: Average depth (Unit: mm) and depth variance Weighted sum: S depth = w d1 +w d2 ;in, To calculate the maximum depth, To calculate the maximum depth variance, w d1， w d2 Preset weights.

[0064] Material health rating S health Definition: S health =1-H; where H is the health index.

[0065] The overall score is obtained through weighted fusion: S severity =w1S size +w2S depth +w3S health The weights w1, w2, and w3 can be dynamically adjusted based on the uncertainty heatmap. If visual uncertainty is high, w1 and w2 are decreased, and w3 is increased, and vice versa. The final score is normalized to the range of 0 to 1, with higher values ​​indicating more severe damage.

[0066] ④ Joint decision confidence level C.

[0067] After obtaining the final fused BPA, it is converted into a Pignistic probability. Then calculate the confidence level: ; Where N is the total number of damage types. The formula means that the more concentrated the probability distribution, the higher C, and the more reliable the decision. If C ≥ 0.7, repair is performed; otherwise, re-sensing or manual intervention is requested. iThis represents the i-th type of damage (e.g., regular holes, irregular tears, intact, etc.). The formula first calculates the Shannon entropy of the Pignistic probability distribution to measure the ambiguity of the decision; then it normalizes by dividing by the maximum possible entropy log₂N to ensure the confidence value is unaffected by the number of propositions; finally, it subtracts the normalized entropy value from 1, resulting in higher confidence when the entropy is lower (the more explicit the decision). This quantitative indicator directly reflects the overall reliability of the current fusion decision, providing the subsequent control layer with a key criterion for deciding whether to "perform repair" or "trigger re-sensing," achieving a complete closed-loop management from perceived uncertainty to decision credibility. This confidence level is the linchpin connecting perception and control.

[0068] (5) The closed-loop mechanism of perception-decision-control.

[0069] In this embodiment, the real-time force and position feedback generated during the repair process will be fed back to the fusion decision module as new evidence to dynamically update the health index and confidence assessment.

[0070] (1) Collection and transformation of feedback data: ① The six-dimensional force sensor acquires contact force and friction force data at a frequency of 1kHz. ② The joint encoder acquires end-effector position and attitude data at a frequency of 2kHz; ③ Pack a feedback data packet every 50ms and convert it into standardized evidence items through the rule engine (e.g., judge whether the resistance is normal or abnormal based on the deviation ratio of axial force and expected repair force, and judge whether the positioning is accurate, biased or inaccurate based on the error between the end position and the planned path).

[0071] (2) Complete closed loop: This forms a complete adaptive closed loop of perception → decision-making → control → perception optimization: ① When the confidence level is sufficient, issue a repair command; ② When the confidence level is insufficient, targeted re-sensing is triggered based on the credibility analysis of each piece of evidence (such as scheduling the spectrometer to re-test the low-confidence area). ③ Execution feedback serves as new evidence feedback, enabling rolling updates of confidence levels.

[0072] This system achieves synergistic optimization of sensing accuracy, decision reliability, and repair efficiency in complex underwater environments through guided collaboration between sensors and closed-loop control of confidence transfer.

[0073] Step 5: After receiving the repair instruction, based on the three-dimensional topography and depth information and the preliminary contour segmentation map, determine the three-dimensional spatial location of the damage point as the target damage point, and obtain the robot's planned path (containing several planning nodes) through the cross-media spatiotemporal AI path planning algorithm.

[0074] For the global task of underwater robots navigating from open water to the surface of flexible fishing nets, an innovative cross-medium spatiotemporal AI path planning algorithm is proposed. This algorithm overcomes the limitations of traditional methods that separate underwater navigation from net movement planning. It constructs a unified hierarchical state space in which each planning node is defined as a hybrid state including the robot's three-dimensional spatial position, robot posture, and attachment state with the fishing net. This allows the algorithm to autonomously decide when to navigate freely in the water, when and where to attach to the fishing net in what posture, and how to move along the net surface after attachment in a continuous search.

[0075] The core innovation of the cross-media spatiotemporal AI path planning algorithm lies in its adaptive cost function and cross-layer action model.

[0076] The cost function can dynamically switch the dominant cost term according to the robot's current state (in water or on a net): when in water, it focuses on optimizing water flow resistance, visibility and static obstacle avoidance; when on a net, it shifts to optimizing pressure deformation on the fishing net, avoidance of biological attachments and its own stability.

[0077] Specifically, the actual movement cost function for navigation in water is defined as: g water (i,j)=w1×d ij +w2×∫F current (t)dt+w3×R obs (j); Where; d ij Let F be the Euclidean distance between the two planning nodes before and after the move. current (t)dt represents the integral cost of the water flow resistance, R obs (j) represents the static obstacle risk coefficient of the j-th planning node, which can be obtained by calculating the local point cloud density based on the lidar point cloud data obtained in step 2, or by querying a preset obstacle map. w1, w2, and w3 are weights.

[0078] The actual movement cost function of the network travel state is defined as: g net (i,j)=μ1×l ij +μ2×σ net (j)+μ3×B attach (j)+μ4×δ stability (j); Among them, l ij σ is the geodesic distance along the network surface between the two planning nodes before and after the move. net (j) represents the stress cost of the fishing net deformation at the j-th planning node, B attach (j) represents the risk coefficient of biological attachment at the j-th planning node, δ stability(j) represents the predicted sway amplitude of the j-th planning node, with μ1, μ2, μ3, and μ4 as weights.

[0079] The explicit modeling and optimal connection of key attachment / detachment actions are achieved by abstracting this physical interaction process into key nodes in the planning state space and assigning them a multi-dimensional cost for dynamic computation. Specifically, the cross-media spatiotemporal AI path planning algorithm defines two fishing net attachment states for the robot: free in water and attached to the net surface, and establishes the switching between states as a special planning action.

[0080] When the cross-media spatiotemporal AI path planning algorithm considers performing the attachment action, it invokes a cost evaluation model in real time. This cost evaluation model comprehensively calculates three factors: first, the action execution cost, which assesses the adjustment difficulty required to complete the grasp or adhesion based on the relative position difference between the current robot and the target fishing net point; second, the landing point stability cost, which predicts the swaying amplitude in the short term by analyzing the historical motion data of the net where the target point is located, thus avoiding dynamically unstable areas; and third, the crucial net surface path look-ahead cost, where the algorithm quickly simulates all possible net surface paths from the attachment point to the final target, estimating their passage difficulty and risk. The weighted sum of these three costs constitutes the total cost function of this attachment action.

[0081] The above three costs are weighted and summed to form the total cost function of the attachment action: C attach (k)=α×C exec (k)+β×C stability (k)+γ×C lookahead (k); Where α, β, and γ are weights, and the specific calculation formula for each sub-cost is as follows: Action execution cost: C exec (k)=||p robot -p k ||+λ×Δθ; where Δθ is the attitude angle deviation, p robot For p robot =(x robot ,y robot ,z robot) Let p be the robot's current position coordinates in three-dimensional space. k For p k =(x k ,y k ,z k ) is the first The three-dimensional spatial coordinates of the candidate attachment points on the fishing net surface, where λ is the weighting coefficient of the attitude deviation term, used to balance the proportion of position distance and attitude adjustment in the cost, and the value range is usually 0.1-0.5.

[0082] Landing point stability cost: C stability (k)=(1 / T)×∫||v net (k,τ)||dτ; where the integration interval is from tT to t, T represents the length of the backtracking time window for historical motion data (e.g., T=5 seconds), representing the time range of historical data used to evaluate net stability, t represents the current moment (i.e., the time point when the algorithm performs the evaluation), and τ represents the integration time variable, taking values ​​from tT to t, representing a certain moment within the historical time window, v net (k,τ) represents the state where At any given moment, the instantaneous velocity vector (unit: m / s) of the fishing net surface area where the candidate attachment point k is located can be obtained by continuously tracking the marked points on the net surface using a visual sensor or inertial measurement unit. Path look-ahead cost for mesh surfaces: C lookahead (k)=A local (k,goal); where A local (k,goal) represents the local execution of a function on the two-dimensional manifold of the fishing net surface, starting from the candidate attachment point k and ending at the final target point goal. The estimated cost of the optimal path obtained by the path search algorithm.

[0083] The cost function for detaching from the action is: C detach (s')=η1×E release +η2×R detach (s') where E release To reduce the energy consumption of the mechanism, R detach (s') represents the environmental risk coefficient at the break point, and η1 and η2 are the weights.

[0084] Building upon the traditional A framework, this algorithm achieves its core innovation through reconstructing the state space and using a dynamic cost function, such as... Figure 6 As shown in the figure. The algorithm defines the robot's three-dimensional spatial position, attitude, and attachment state as hybrid state nodes, and uses the A search framework for global path search.

[0085] During node expansion, the corresponding cost function is called to calculate the transition cost based on the transition type between the current node and its neighboring nodes: In water → In water: Using the underwater navigation cost function g water ; Mesh → Mesh: Using the mesh traversal cost function g net ; In water → onto the net (attachment): using the total cost function C of the attachment action. attach ; Net surface → Water (detachment): Employing the detachment action cost function C detach .

[0086] The heuristic function uses a weighted Euclidean distance: h(s') = ρ × ||p s' -p goal ||, where ρ is the heuristic weight coefficient (ρ≤1 to ensure acceptability), p s' Let p be the position of the neighboring node s'. goal The target node location (damage point location) is defined. To cope with the dynamic environment, the algorithm introduces spatiotemporal joint optimization, dynamically updating terms such as water flow resistance integral and sway prediction in the cost function through a short-term prediction model, making the algorithm tend to select a more stable trajectory within the future time window. Ultimately, this algorithm generates a globally spatiotemporally optimal trajectory from the starting point to the damage point. This trajectory not only indicates the route in three-dimensional space but also intelligently plans state transition points and net surface travel routes, enabling the robot to make high-level decisions such as detouring or using the net. For example, the algorithm may proactively choose a slightly farther but more stable edge point from the waterway for attachment. It should be noted that attachment here is only a state transition node in path planning, not the end point of the task. After attachment, the robot still needs to continue traveling along the net surface until it reaches the true task endpoint (target damage point) before starting the repair task. This design achieves the effect of a small increase in local cost in exchange for a large overall optimization, significantly improving the efficiency, safety, and autonomy of complex underwater global task execution. This trajectory serves as the tracking benchmark for the bottom-level controller in step 6.

[0087] Step 6: After obtaining the robot's planned path (i.e., the desired trajectory), adaptive PID (proportional-integral-derivative) control is used to move the robot to the target damage point. To accurately execute the desired trajectory amidst dynamic water flow and net swaying, a high-performance underlying motion control unit is required. In this embodiment, a robust adaptive PID controller based on a disturbance observer and feedforward compensation is designed for the underlying motion control unit, such as... Figure 7 As shown, this controller introduces an extended state observer to perform real-time estimation and active compensation of the total disturbance based on the traditional PID feedback loop.

[0088] ① System model definition: Let e ​​be the position error between the robot's actual position and the desired trajectory. x, The system state equation can be described as: e x =v,v=f+b u+d, where u is the control input (thrust command for the thruster); d is the lumped disturbance, including water flow impact, fishing net swaying, model uncertainty, and unmodeled dynamics; f is the known dynamic part of the system; and b is the control gain.

[0089] ② Extended State Observer Design: Expand the lumped disturbance d into new state variables and design a third-order extended state observer. Where: z1, z2, and z3 are the estimated values ​​of the error, the error derivative, and the total disturbance, respectively; ω0 is the observer bandwidth, which determines the speed of disturbance estimation; and b0 is the estimated control gain. The observer input is the position error e. x The control variable u is used to output the total disturbance estimate z3.

[0090] ③ The control law based on disturbance compensation is designed as follows: The adaptive PID control law is: u0 = K p (γ) e x +K i (γ) ∫e x dt+K d (γ) e x In the formula, u0 is the PID control quantity, and z3 is the estimated value of the total disturbance. K is the estimated control gain. p ,K i, K d The fuzzy logic controller dynamically adjusts based on path curvature and tracking error, with γ being an adaptive coefficient. This controller directly adds the disturbance estimate from the observer output as a feedforward compensation to the PID output, achieving active cancellation of the disturbance.

[0091] ④ The real-time control process of the underlying motion control unit is as follows: a. Sensors acquire the robot's current pose and calculate the error e between the pose and the desired trajectory. x ; b. The error e x The control quantity u from the previous moment is input into the extended state observer to obtain the total disturbance estimate z3; c. Dynamically update the PID parameter k using a fuzzy rule table based on the path curvature and error rate of change. p ,k i ,k d ; d. Calculate the PID control quantity u0 and the disturbance compensation quantity, and synthesize the final control command. ; e. Distribute control commands to each actuator to drive the robot's movement; f. Repeat steps a~e to achieve real-time closed-loop control.

[0092] Furthermore, a predictive fault-tolerant module is integrated. By monitoring and predicting health indicators such as thruster current and motor temperature online, it adjusts the thrust distribution strategy of individual thrusters in advance before their performance deteriorates or they fail, maintaining system controllability. This innovative design enables the lower-level control unit to not only maintain path tracking accuracy at ≤5cm, but also significantly improves the system's robustness, adaptability, and safety under strong disturbances and gradual changes in actuator performance, providing a solid and reliable execution foundation for high-level intelligent planning.

[0093] Step 7: After the robot moves to the target damage point, it begins the repair process, such as... Figure 8 As shown.

[0094] Existing technical solutions for repair lack adaptability: the actuator operates with fixed parameters, which cannot adjust the welding temperature and sewing tension according to the material of the fishing net (such as nylon with a melting point of 250℃ and polyethylene with a melting point of 130℃), and it is also difficult to adapt to different mesh sizes (from 2mm precision aquaculture nets to 100mm shellfish aquaculture nets). The repair qualification rate is only 75%-85%, and the equipment has poor versatility.

[0095] This embodiment proposes a repair parameter adjustment logic: Let the repair temperature command value be T_ref, the initial setting value be T0, and the correction be made according to the material type identified by the spectrum: T_ref=T0+k1×(T_m-T0), where k1 is the material correction coefficient (0.6-0.9), and T_m is the optimal repair temperature for the material; The suture tension command value F_ref=F0-k2×S, where k2 is the damage area correction coefficient (0.01-0.05), S is the damage area, and F0 is the basic suture tension.

[0096] As can be seen from the control logic above, when a composite fishing net is detected, T_ref will approach the optimal repair temperature T_m of that material to avoid the net wires becoming brittle due to excessive temperature or the welds becoming weak due to excessive temperature. When a minor damage (S < 10 cm²) is detected, F_ref will decrease to prevent the net wires from breaking due to excessive tension. Since the repair requirements (damage type and quantity) of the fishing net are dynamically changing, the robot can maintain a repair pass rate of over 90% by adjusting the control parameters in real time. The auxiliary communication link is only responsible for transmitting initial instructions and operation data and does not need to participate in real-time control. Therefore, this link can be designed as a low-bandwidth, low-cost communication solution.

[0097] Step 8: If there are multiple damage points, a dynamic priority queue is obtained through an intelligent task scheduling model based on multi-objective weighted optimal decision-making. Steps 5, 6 and 7 are then executed according to the order in the priority queue.

[0098] Based on multimodal sensing data, an intelligent task scheduling model based on multi-objective weighted optimal decision-making is proposed. This model surpasses traditional static ranking methods based on fixed rules (such as area only). Its core lies in constructing a three-layer progressive evaluation architecture, which comprehensively scores the scheduling priority of each damage point, such as... Figure 9 As shown: First, a structural urgency assessment is conducted, taking into account the damaged area, the stress location within the fishing net topology, and the surrounding stress concentration as detected by lidar scanning, to calculate the vulnerability index that the damaged point may lead to a chain of structural failures. Second, the timeliness of the operation is assessed. Based on the water flow prediction data, the best window of opportunity for treating the damaged points is evaluated, and priority is given to treating high-risk points before they deteriorate or when environmental conditions are favorable. Thirdly, resource efficiency assessment involves obtaining structural urgency and operational timeliness scores for each breach point, and then comprehensively revising the results by combining these scores with the pre-planning path. This includes pre-planning paths for candidate breach points, calculating actual reachability and movement path costs, and incorporating cost as a penalty into the overall score. This ensures that scheduling results prioritize breach points with lower path costs. The final output is a dynamically optimized priority queue.

[0099] In addition, as an optional implementation, this model also includes a fourth layer: an online learning dimension. By continuously recording actual repair data, the evaluation weights of different damage characteristics are adjusted using a combination of offline pre-training and online incremental updates, allowing the decision-making model to gradually adapt to the current operating environment.

[0100] Ultimately, the intelligent task scheduling model outputs a dynamic priority queue that updates dynamically based on environmental changes and new discoveries. This queue serves as the target input for path planning. A collaborative mechanism of one-way transmission and limited retries is employed between scheduling and planning: after the scheduling model outputs the priority queue, the path planning algorithm attempts to generate a global trajectory in that order. If the planning algorithm returns an "infeasible" flag (e.g., due to a break in the mesh between two damaged points preventing passage), the scheduling model removes the task and recalculates the second-best priority queue, retrying a maximum of three times. If the number of retries is exceeded, manual intervention is requested. The command robot not only knows which task to repair first, but also more accurately determines when to repair and which tasks to repair along the route, achieving a leap from simple sorting to global multi-objective adaptive scheduling, significantly improving the overall efficiency and reliability of complex underwater operations.

[0101] This embodiment provides an integrated fishing net repair robot control method. The robot body dynamically adjusts repair parameters and motion strategies based on multimodal perception data, which can improve the fishing net damage recognition rate to over 95% and the repair qualification rate to over 90%. The auxiliary communication link is only responsible for black start and data transmission, and can be designed as a low-bandwidth scheme to reduce investment costs. This control method requires little modification to the existing marine ranch operation and maintenance network, is easy to implement, and can significantly improve the efficiency of fishing net repair in complex environments.

[0102] This embodiment provides an integrated fishing net repair robot control method that uses a multimodal intelligent sensing subsystem (vision, laser, touch, and spectroscopy) to simultaneously identify the damage type, material properties, and degree of damage, increasing the damage recognition rate to over 95% and solving the problem of low recognition rate in traditional single-modal sensing under turbid water conditions and complex damage scenarios.

[0103] This embodiment provides an integrated control method for a fishing net repair robot, which adopts a hierarchical reinforcement learning path planning subsystem. The higher layer is responsible for global task sorting, while the lower layer is responsible for dynamic obstacle avoidance and attitude adjustment. Under scenarios of water flow disturbance (flow velocity > 1.0 m / s) and fishing net deformation, the operation interruption rate is reduced to below 5%, solving the problem of poor dynamic adaptability of traditional static path planning.

[0104] This embodiment provides an integrated fishing net repair robot control method. The intelligent adaptive repair execution subsystem supports modular tool head quick change and parameter self-adaptation. It can adapt to 6 types of mainstream fishing net materials and mesh sizes of 2-100mm, and improve the repair qualification rate to over 90%. It solves the problems of poor versatility and unstable repair quality of traditional fixed parameter operations.

[0105] This embodiment provides an integrated fishing net repair robot control method. The auxiliary communication link is only responsible for black start and data transmission and does not need to participate in real-time control. It can be designed as a low-bandwidth scheme. Compared with the traditional scheme that requires real-time transmission of control commands, the investment cost of the communication link is reduced by more than 40%.

[0106] The integrated fishing net repair robot control method provided in this embodiment requires minimal modification to the existing marine ranch operation and maintenance network, can be directly connected to the shore-based / mother ship control system, has low integration difficulty, and is easy to promote and apply.

[0107] It should be noted that each module in this embodiment corresponds one-to-one with each step in Embodiment 1, and their specific implementation processes are the same, so they will not be repeated here.

[0108] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A control method for an integrated fishing net repair robot, characterized in that, include: The fishing net image of the target area is acquired, and the geometric contour of the damaged area and the uncertainty heat map are extracted through an uncertainty perception network. Based on the uncertainty heatmap, an adaptive focusing scanning algorithm is used to guide the 3D LiDAR to collect point cloud data of the target area and form the 3D shape of the target area. The spectral signal of the fishing net in the target area is acquired, and the material category of the fishing net is obtained through a lightweight material classification network. Based on the material category, the parameters of the tactile probe are optimized to control the action of the tactile probe, and the vibration response signal of the fishing net is collected. The health index of the fishing net is obtained through vibration spectrum analysis. Based on the geometric contour of the damaged area, the uncertainty heat map, the three-dimensional shape of the target area, the fishing net material type, and the fishing net health index, a multi-source fusion decision is made to determine whether to issue a repair command. Upon receiving the repair instruction, the location of the damage point is determined based on the three-dimensional topography and the geometric contour of the damaged area, using A... The search algorithm obtains the robot's planned path, and through adaptive PID control, it controls the robot to move to the target damage point and perform the repair operation.

2. The integrated fishing net repair robot control method as described in claim 1, characterized in that, The adaptive focusing scanning algorithm includes: mapping pixel clusters with values ​​below a threshold in the uncertainty heatmap to the area to be inspected in three-dimensional space; controlling the three-dimensional LiDAR scanning head to automatically perform multi-angle tilt scanning in the area to be inspected and increasing the point cloud sampling frequency; and using a fast scanning mode for areas not to be inspected.

3. The integrated fishing net repair robot control method as described in claim 1, characterized in that, The A The costs employed by the search algorithm include cumulative cost and heuristic estimation cost.

4. The integrated fishing net repair robot control method as described in claim 3, characterized in that, The cumulative cost is: g(s') = g(s) + state transition cost; where g(s) is the cumulative cost of the current node s, s' is the neighbor node of the current node s, and the state transition cost is the cost of water navigation, net travel, attachment action cost, or detachment action cost.

5. The integrated fishing net repair robot control method as described in claim 3, characterized in that, The heuristic estimation cost is: h(s') = ρ × ||p s' -p goal ||; Where ρ is the heuristic weight coefficient, p s' Let p be the position of the neighboring node s'. goal This indicates the location of the damage point.

6. The integrated fishing net repair robot control method as described in claim 4 or 5, characterized in that, The neighbor nodes are generated based on the robot's current attachment state: If the robot's current attachment state is underwater, two types of neighbor nodes can be generated: one is the neighbor node that maintains the underwater state; the other is the attachment state switching node, which generates the mesh state node. If the robot is currently attached to a net surface, it can generate two types of neighbor nodes: one is the neighbor node that maintains the net surface state; the other is the node that switches from the state to the state that generates the underwater state node.

7. The integrated fishing net repair robot control method as described in claim 1, characterized in that, The robot planning path includes several planning nodes, and each planning node is defined as including the robot's three-dimensional spatial position, robot posture, and attachment status with the fishing net.

8. The integrated fishing net repair robot control method as described in claim 1, characterized in that, When performing a repair operation, the repair temperature is: T_ref=T0+k1×(T_m-T0); where T0 is the initial setting value, k1 is the material category correlation coefficient, and T_m is the optimal repair temperature corresponding to the material category.

9. The integrated fishing net repair robot control method as described in claim 1, characterized in that, When performing the repair operation, the suture tension is: F_ref=F0-k2×S; where k2 is the damage area correction coefficient, S is the damage area, and F0 is the basic suture tension.

10. A fully submersible fishing net repair robot, characterized in that, The system includes a robot body and a host computer mounted on the robot body, wherein the host computer is configured with an integrated fishing net repair robot control method as described in any one of claims 1-9.

Images