A method for umbilical cable state visual detection based on skeleton prior constraint

Abstract

This invention discloses a visual detection method for underwater umbilical cables based on skeleton prior constraints, belonging to the field of non-destructive testing technology. The method first acquires and preprocesses underwater umbilical cable image samples, constructs a dual-stream feature extraction network with topological and texture branches, extracts continuous topological features and damage detail enhancement features respectively, and adaptively fuses them; then, it predicts the umbilical cable skeleton probability map, selects skeleton anchor points, and adaptively sparsely samples tangentially, normally, and with the anchor points as the center, completes the cable features in occluded and discontinuous regions through cross-scale feature interaction based on topological constraints; the enhanced features are input into the detection head, and the output shows the entanglement and damage status results; during the training phase, a joint optimization model using classification, bounding box regression, and topological continuity loss is adopted, and finally deployed on edge devices to achieve real-time detection and early warning. This invention can stably maintain the cable topological continuity, accurately identify minor damage, has low computational cost, is adaptable to edge computing power, and significantly improves the robustness of detection in complex underwater scenes.

Description

Technical Field

[0001] This invention relates to the field of nondestructive testing technology, specifically to a visual inspection method for the state of umbilical cables based on skeleton prior constraints. Background Technology

[0002] Umbilical cables are critical components connecting underwater work platforms, ROVs, or other underwater equipment to surface equipment, providing functions such as power supply, communication, and fluid transmission. During cable deployment, towing, and operations in complex sea conditions, umbilical cables are prone to tangling, abnormal bending, and sheath damage. If these issues are not detected in time, they can lead to signal interruptions, decreased insulation performance, or even structural damage.

[0003] In the prior art, patent CN118447363A provides a method for submarine cable target detection based on an improved deep network. This patent uses YOLOv5 as its basic architecture to build a submarine cable detection network SCINet. The backbone network is replaced with ConvNeXt v2 to improve the feature extraction capability in complex underwater environments; the neck module is replaced with a C2f module to optimize multi-scale feature fusion and gradient flow; the detection head adopts a 4-layer stacked DyHead to improve the characterization capability of the detection head. The core objective is to solve the problem of general target localization of submarine cables under underwater scattering and silt obstruction. It only realizes the detection of submarine cable position and does not involve the subdivision of abnormal states such as entanglement and damage.

[0004] Patent CN120672672A provides a method for detecting umbilical cable anomalies. This patent targets three types of anomalies in underwater robot umbilical cables: entanglement, damage, and bending. It constructs a deep learning detection model: the backbone network uses WTConv wavelet convolution and a dual-domain selection mechanism to extract features; a hybrid encoder integrates hybrid dilated residual attention to suppress noise; and the decoder optimizes the bounding box accuracy through fine-grained localization and global optimal localization self-distillation to achieve umbilical cable anomaly detection. However, it lacks a dedicated module for enhancing details of minute defects, making it prone to missing minor damage such as cracks and peeling under underwater turbidity and background interference.

[0005] However, the existing technology has the following drawbacks:

[0006] The system lacks a dedicated structure designed for the slender, continuous, and highly topologically interconnected characteristics of umbilical cables / submarine cables. Under conditions of occlusion or entanglement, cable features are prone to breakage, making it impossible to recover the continuous structure. It only employs conventional loss methods such as classification, regression, and distillation, without constraining the continuity of the cable skeleton and the consistency between the detection frame and the cable's orientation, resulting in large detection errors in entangled / bending areas. Furthermore, it utilizes global feature computation and full-scale attention, leading to high computational costs and a large number of parameters. Without sparsity optimization, it is difficult to deploy in real-time on edge computing-constrained devices such as ROVs and offshore platforms.

[0007] Therefore, it is necessary to propose a dedicated visual inspection scheme for underwater umbilical cable scenarios, which can simultaneously improve the ability to maintain continuous topology, the ability to complete features in occluded scenarios, and the ability to detect minor damage under the condition of limited edge computing power. Summary of the Invention

[0008] To address the above problems, this invention proposes a visual detection method for the state of underwater umbilical cables based on skeleton prior constraints. The method includes the following steps: S1. Acquire underwater image samples containing umbilical cables, and annotate and preprocess the image samples to obtain a training sample set; S2. Construct a two-stream feature extraction network and extract multi-scale features. The two-stream feature extraction network includes: S21. Topological branching: Based on the local orientation and curvature information of the input image or shallow features, construct a convolution operator with enhanced response along the extension direction of the umbilical cable to extract topological features that preserve the continuous structure. S22. Texture Branch: Perform high-frequency enhancement and gating filtering on the input features to highlight local detail features corresponding to cracks, peeling or abnormal textures on the umbilical cable surface; S23. Fusion step: Based on the structural complexity and skeleton confidence at the spatial location, the topological features and the local detail features are adaptively fused to obtain multi-scale fused features; S3. Predict the umbilical cable skeleton probability map based on the multi-scale fusion features, and select skeleton anchor points based on the skeleton probability map; S4. Using the skeleton anchor point as the center, generate sparse sampling positions according to the skeleton tangential direction, normal direction and rate adaptive offset, and perform cross-scale feature interaction of topological constraints only in the neighborhood of the anchor point to obtain enhanced features, wherein the enhanced features are used to complete the cable representation of the occluded area or discontinuous area. S5. Input the enhanced features into the detection head and output the detection results of the umbilical cable entanglement or damage status; S6. During the training phase, classification loss, bounding box regression loss, and topological continuity loss are used to jointly optimize the dual-stream feature extraction network, the skeleton prediction branch, the topologically constrained cross-scale feature interaction module, and the detection head. S7. Deploy the trained model on an edge computing device to perform inference on real-time acquired underwater images and output early warning information.

[0009] The topology branch output in step S2 is not only used for target detection, but also serves as one of the inputs for skeleton probability map prediction in step S3. The skeleton anchor points obtained in step S3 are used simultaneously for the sparse sampling position generation in step S4 and the fusion weight correction in step S23, thereby forming a skeleton-guided collaborative detection closed loop.

[0010] The topological continuity loss in step S6 simultaneously constrains the continuity and smoothness of the skeleton prediction results, as well as the consistency between the detection results and the skeleton orientation, in order to reduce the fracture characterization of the umbilical cable in the entanglement and obstruction areas.

[0011] Based on the above method, the present invention also provides an underwater umbilical cable state visual inspection system based on skeleton prior constraints, comprising: The sample processing module is used to acquire and preprocess underwater image samples containing umbilical cables; The dual-stream feature extraction module is used to extract topological features that preserve the continuous topological structure and texture features that enhance the response to damaged details; The skeleton prediction module is used to generate a skeleton probability map and select skeleton anchor points. The sparse feature interaction module is used to generate sparse sampling locations based on skeleton anchor points and perform cross-scale feature interactions with topological constraints. The detection output module is used to output the detection results of the umbilical cable entanglement status and / or damage status; The joint optimization module is used to perform joint optimization of classification loss, bounding box regression loss, and topological continuity loss during the training phase; The deployment and early warning module is used to perform model inference on edge computing devices and output early warning information.

[0012] Compared with the prior art, the beneficial effects of the present invention include: (1) By relying on the direction-curvature guided topology branch and skeleton prior constraints, the continuous directional structure of the umbilical cable can be stably maintained. Through cross-scale feature interaction of topology constraints, the cable features of occluded and discontinuous areas can be automatically completed, thus completely solving the problem of cable feature breakage and detection failure in complex underwater scenarios.

[0013] (2) The texture enhancement branch accurately highlights the details of minor damage such as cracks and peeling through high-frequency enhancement and gating screening. It is adaptively integrated with the topology branch and can still stably capture minor defects under interference such as turbidity and uneven lighting, thus solving the pain point of minor damage being easily missed in general solutions.

[0014] (3) By adopting the skeleton anchor point sparse sampling mechanism, feature interaction is only performed in the neighborhood of the anchor point, and the amount of computation is compressed to a minimum. With model quantization optimization, high-speed inference can be achieved on edge computing-limited devices such as underwater robots and marine operation platforms to meet the real-time detection requirements.

[0015] (4) Introducing joint optimization of topological continuity loss to constrain the smoothness of the skeleton and the consistency of the detection frame-cable direction, reducing the positioning error of the entanglement and bending areas, and improving the detection stability and robustness under complex underwater conditions compared with conventional detection schemes. Attached Figure Description

[0016] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0017] Figure 1 This is an overall flowchart of the method of the present invention; Figure 2 This is a schematic diagram of the structure of the dual-stream feature extraction network of the method of the present invention; Figure 3 This is a schematic diagram of the processing flow of the direction-curvature guided topology branching method of the present invention; Figure 4 This is a schematic diagram of the processing flow of the texture enhancement branch in the method of the present invention; Figure 5 This is a schematic diagram illustrating the skeleton anchor point generation and sparse sampling position construction of the method of the present invention; Figure 6 This is a schematic diagram of cross-scale feature interaction of topological constraints in the method of the present invention. Detailed Implementation

[0018] The overall flowchart of the visual detection method for umbilical cable state based on skeleton prior constraints described in this invention is as follows: Figure 1 As shown, the specific implementation is as follows: Example 1 Images of umbilical cables are acquired, including those depicting clear water, turbid water, strong reflection, low light, partial occlusion, entanglement, and surface damage. Enclosed boxes are drawn for entangled and damaged areas in the images, and a skeleton is drawn for the umbilical cable centerline. Preprocessing may include size normalization, color perturbation, scattering noise simulation, blurring simulation, and geometric enhancement to maintain topological continuity.

[0019] Example 2 Based on the preprocessed data from Example 1, the detection is performed according to the following steps: S1. Acquire underwater image samples containing umbilical cables, and annotate and preprocess the image samples to obtain a training sample set; S2. Construct a two-stream feature extraction network and extract multi-scale features. The two-stream feature extraction network includes: S21. Topological branching: Based on the local orientation and curvature information of the input image or shallow features, construct a convolution operator with enhanced response along the extension direction of the umbilical cable to extract topological features that preserve the continuous structure. S22. Texture Branch: Perform high-frequency enhancement and gating filtering on the input features to highlight local detail features corresponding to cracks, peeling or abnormal textures on the umbilical cable surface; S23. Fusion step: Based on the structural complexity and skeleton confidence at the spatial location, the topological features and the local detail features are adaptively fused to obtain multi-scale fused features; S3. Predict the umbilical cable skeleton probability map based on the multi-scale fusion features, and select skeleton anchor points based on the skeleton probability map; S4. Using the skeleton anchor point as the center, generate sparse sampling positions according to the skeleton tangential direction, normal direction and / or curvature adaptive offset, and perform cross-scale feature interaction of topological constraints only in the neighborhood of the anchor point to obtain enhanced features, wherein the enhanced features are used to complete the cable representation of the occluded area or discontinuous area. S5. Input the enhanced features into the detection head and output the detection results of the umbilical cable entanglement or damage status; S6. During the training phase, classification loss, bounding box regression loss, and topological continuity loss are used to jointly optimize the dual-stream feature extraction network, the skeleton prediction branch, the topologically constrained cross-scale feature interaction module, and the detection head. S7. Deploy the trained model on an edge computing device to perform inference on real-time acquired underwater images and output early warning information.

[0020] Example 3: Two-stream Feature Extraction Network Topological branching generates directional response-enhanced convolutions based on local orientation and curvature information. For example... Figure 3 The processing flow shown first utilizes edge responses, gradient directions, or structure tensors to obtain local principal directions. And estimate the curvature based on the directional changes of adjacent positions. .based on and Generate directional convolution kernels and with the basic convolution kernel By weight By merging, we obtain the convolution kernel. ,Right now: ; in, It adaptively adjusts to changes in curvature to enhance the response to continuous orientation in bending regions.

[0021] Texture branching enhances input features at high frequencies. For example... Figure 4 As shown, high-frequency features are obtained by employing one or more of the following methods: frequency domain high-pass, Laplace enhancement, and differential response enhancement. Then through gating mapping High-frequency features are filtered to output texture enhancement features. This branch is used to highlight cracks, spalling boundaries, and abnormal sheath textures.

[0022] Fusion steps based on spatial attention maps Topological features and texture enhancement features By fusing, we obtain: ; Preferably, In addition to relying on local complexity, the generation of the algorithm is also corrected by combining the skeleton prediction confidence. For position p, let the local complexity weight be... The skeleton prediction confidence level is The corrected fusion weights can then be expressed as: ; in, To adjust the parameters; When a location has both high local complexity and high skeleton confidence, it indicates that the region is likely located near the umbilical cable trunk and contains key details such as entanglement, bending, or damage. Therefore, the proportion of texture enhancement features in the fusion result should be increased. When a location has high local complexity but low skeleton confidence, it is considered that the region is more likely to originate from background noise, suspended particles, or reflective interference. The weight of texture enhancement features should be reduced to enhance the ability of the fusion features to discriminate the real umbilical cable structure.

[0023] Example 4: Cross-scale feature interaction between skeleton anchor point generation and topological constraints like Figure 5 As shown, shallow features and fused features are used to predict the skeleton probability map. Thresholding and non-maximum suppression are applied to the skeleton probability map to obtain the skeleton anchor point set. .

[0024] Preferably, the number of skeleton anchor points is controlled within the range of 1% to 10% of the total number of image pixels to reduce the amount of computation.

[0025] like Figure 6 For each skeleton anchor point Based on the tangential direction at that position , normal direction and curvature Several sampling locations are generated. These locations cover both the neighborhood of the visible cable and directional regions where occlusion or continuous extension may occur. Subsequently, query features, key features, and value features for the corresponding locations are extracted from feature maps at different scales, and cross-scale feature interaction is performed to obtain enhanced features. After backfilling the feature map with the enhanced features corresponding to all anchor points, the enhanced features facing the detection head are obtained. F' .

[0026] Example 5: Joint Optimization The total loss during the training phase can be expressed as: ; in, For classifying losses, it is used to distinguish between entanglement, breakage, or background; The bounding box regression loss is used to constrain the positional and scale differences between the predicted bounding box and the labeled bounding box; Topological continuity loss is used to constrain the continuity and smoothness of the skeleton results, and to constrain the consistency between the detection results and the skeleton orientation. , , These are the weight coefficients corresponding to the classification loss, bounding box regression loss, and topological continuity loss, respectively, used to balance the contributions of the three training objectives—class discrimination, target localization, and topological constraints—in the overall optimization process.

[0027] In one alternative implementation, It includes an adjacent skeleton point smoothing term and a frame-skeleton consistency term. The adjacent skeleton point smoothing term is used to constrain the direction and spacing changes of adjacent skeleton points; the frame-skeleton consistency term is used to constrain the consistency between the main axis direction of the detection frame and the local direction of the corresponding skeleton.

[0028] In another alternative implementation, the box regression loss It may include a loss term based on the distance between the predicted box and the ground truth box probability distribution. For example, the distance can be calculated after approximating the predicted box and the ground truth box as two-dimensional Gaussian distributions to improve the robustness to localization perturbations of slightly damaged targets.

[0029] Example 6: Detection Head and Deployment The detection head can be either a query-based decoupled detection head or an anchor-box detection head. Preferably, the detection head outputs the class probability and bounding box parameters respectively, and uses the topological completion information in the enhanced features for the state determination of the entangled region.

[0030] After model training is complete, graph optimization, operator fusion, and fixed-point quantization can be performed, and the model can be deployed on edge computing devices. Once the edge device acquires real-time images, it inputs these images into the trained model and outputs detection results for the umbilical cable's entanglement and damage states. When the detection results meet preset thresholds or consecutive frame triggering conditions, alarm information or maintenance prompts are output.

[0031] The embodiments described above are merely illustrative of implementation methods of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A visual detection method for the state of underwater umbilical cables based on skeleton prior constraints, characterized in that, Includes the following steps: S1. Acquire underwater image samples containing umbilical cables, and annotate and preprocess the image samples to obtain a training sample set; S2. Construct a two-stream feature extraction network and extract multi-scale features: S3. Predict the umbilical cable skeleton probability map based on multi-scale fusion features, and select skeleton anchor points based on the skeleton probability map; S4. Using the skeleton anchor point as the center, generate sparse sampling positions according to the skeleton tangential direction, normal direction and rate adaptive offset. Perform cross-scale feature interaction of topological constraints only in the neighborhood of the anchor point to obtain enhanced features. The enhanced features are used to complete the cable representation of the occluded area or discontinuous area. S5. Input the enhanced features from step S4 into the detection head and output the detection results of the umbilical cable entanglement or damage status; S6. During the training phase, classification loss, bounding box regression loss, and topological continuity loss are used to jointly optimize the dual-stream feature extraction network, the skeleton prediction branch, the topologically constrained cross-scale feature interaction module, and the detection head. S7. Deploy the trained model on an edge computing device to perform inference on real-time acquired underwater images and output early warning information.

2. The underwater umbilical cable state visual detection method based on skeleton prior constraints according to claim 1, characterized in that, The construction of the dual-stream feature extraction network specifically includes topology branching, texture branching, and fusion steps.

3. The underwater umbilical cable state visual detection method based on skeleton prior constraints according to claim 2, characterized in that, The topological branch constructs a convolution operator with enhanced response along the umbilical cable extension direction based on the local orientation and curvature information of the input image or shallow features, in order to extract topological features that preserve the continuous structure.

4. The underwater umbilical cable state visual detection method based on skeleton prior constraints according to claim 2, characterized in that, The texture branch uses one or more of the following methods to obtain high-frequency features: frequency domain high-pass, Laplacian enhancement, and differential response enhancement. Then, the high-frequency features are filtered through gating mapping to output texture enhancement features.

5. The underwater umbilical cable state visual detection method based on skeleton prior constraints according to claim 2, characterized in that, The fusion step is based on spatial attention maps. Topological features and texture enhancement features By fusing, we obtain: ； In addition to relying on local complexity, the generation also incorporates skeleton prediction confidence for correction, thereby increasing the weight of local details in entangled nodes, bending regions, and broken regions.

6. The underwater umbilical cable state visual detection method based on skeleton prior constraints according to claim 5, characterized in that, The method for correction based on skeleton prediction confidence is as follows: For location p Let the local complexity weight be... The skeleton prediction confidence level is The corrected fusion weights can then be expressed as: ； in, To adjust the parameters; When a location has both high local complexity and high skeleton confidence, it indicates that the region is likely located near the umbilical cable trunk and has key details such as entanglement, bending or damage. Therefore, the proportion of texture enhancement features in the fusion result should be increased. When a certain location has high local complexity but low skeleton confidence, it is considered that the region is more likely to originate from background noise, suspended particles, or reflective interference. The weight of texture enhancement features is reduced to enhance the ability of fused features to distinguish the real umbilical cable structure.

7. The underwater umbilical cable state visual detection method based on skeleton prior constraints according to claim 1, characterized in that, The sparse sampling locations include at least a first set of sampling points distributed along the tangential direction of the skeleton and a second set of sampling points distributed along the normal direction of the skeleton. The first set of sampling points is used to characterize the continuous extension area of ​​the umbilical cable, and the second set of sampling points is used to characterize the occlusion boundary or the local damage neighborhood.

8. The underwater umbilical cable state visual detection method based on skeleton prior constraints according to claim 1, characterized in that, The sum of the classification loss, box regression loss, and topological continuity loss in step S6 can be expressed as: ； in, For classifying losses, it is used to distinguish between entanglement, breakage, or background; The bounding box regression loss is used to constrain the positional and scale differences between the predicted bounding box and the labeled bounding box; It is a topological continuity loss used to constrain the continuity and smoothness of the skeleton results, and to constrain the consistency between the detection results and the skeleton orientation.

9. The underwater umbilical cable state visual detection method based on skeleton prior constraints according to claim 8, characterized in that, The topological continuity loss includes an adjacent skeleton point smoothing term and a box-skeleton consistency term. The adjacent skeleton point smoothing term is used to constrain the direction and spacing changes of adjacent skeleton points, and the box-skeleton consistency term is used to constrain the consistency between the main axis direction of the detection box and the local direction of the corresponding skeleton.

10. The underwater umbilical cable state visual detection method based on skeleton prior constraints according to claim 8, characterized in that, The bounding box regression loss includes a loss term based on the distance between the predicted bounding box and the ground truth bounding box probability distribution, wherein the predicted bounding box and the ground truth bounding box are represented as a two-dimensional probability distribution to improve robustness to perturbations in the localization of slightly damaged targets.

11. A system for implementing the underwater umbilical cable state visual detection method based on skeleton prior constraints as described in any one of claims 1-10, characterized in that, include: The sample processing module is used to acquire and preprocess underwater image samples containing umbilical cables; The dual-stream feature extraction module is used to extract topological features that preserve the continuous topological structure and texture features that enhance the response to damaged details; The skeleton prediction module is used to generate a skeleton probability map and select skeleton anchor points. The sparse feature interaction module is used to generate sparse sampling locations based on skeleton anchor points and perform cross-scale feature interactions with topological constraints. The detection output module is used to output the detection results of the umbilical cable entanglement status and / or damage status; The joint optimization module is used to perform joint optimization of classification loss, bounding box regression loss, and topological continuity loss during the training phase; The deployment and early warning module is used to perform model inference on edge computing devices and output early warning information.

12. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program, which, when executed by the processor, implements the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method of any one of claims 1 to 10.

Images