A salvage ROV umbilical cable terminal status recognition and early warning method

Abstract

A salvage ROV umbilical cable terminal state recognition and early warning method of the present invention comprises: collecting images of the umbilical cable terminal in various states; performing coarse segmentation on the above-mentioned umbilical cable terminal video image using a segmentation and positioning model; performing fine classification on the image using an umbilical cable terminal state recognition model; and classifying and identifying the salvage ROV umbilical cable terminal image read in real time and issuing an early warning. The present invention uses a "coarse to fine" recognition algorithm of first segmentation and then detection to first locate the umbilical cable terminal area and then realize refined recognition. Compared with directly recognizing the entire image, the present invention can reduce the impact of complex underwater environments and improve recognition accuracy. In addition, since underwater images will present a large area of homogeneous areas, the segmentation algorithm can better segment the umbilical cable terminal position. The method can realize automatic recognition and early warning of the umbilical cable terminal state, greatly reducing the human judgment cost during the salvage operation and improving the intelligence level.

Description

Technical Field

[0001] The present invention relates to the field of underwater salvage robots, and in particular to a method for identifying and warning the terminal state of an umbilical cable of a salvage ROV. The method can also be applied to the fields of operational and observation-level ROVs. Background Art

[0002] In recent years, salvage underwater robots (ROVs) have been widely used to salvage and recover sunken underwater targets in deepwater, offshore, and test sites. A salvage ROV typically consists of an underwater ROV body, an umbilical cable, and a surface monitoring system. The umbilical cable, which provides power and data transmission underwater and on the surface, is a crucial component of the entire system.

[0003] However, umbilical cables are prone to failure during operation due to the complexity and uncertainty of the underwater environment. In particular, the terminal position is more prone to various types of failures during the salvage process. Severe failures can lead to the loss of ROVs and abnormal surface power distribution, posing a significant threat to human and property safety. Therefore, it is necessary to take necessary measures to monitor the terminal position of the umbilical cable:

[0004] (1) Manual observation and judgment: The umbilical cable image is observed in real time by the underwater camera on the ROV, and its status is judged manually. However, this measure has a low level of automation and consumes a lot of manpower, which reduces the efficiency of the salvage operation. At present, most salvage ROVs adopt this method.

[0005] (2) External observation and judgment: During the salvage operation, an independent observation-level ROV is used to observe and monitor the umbilical cable. However, such measures require a separate ROV operation, which not only increases the cost but also increases the difficulty of collaborative operations of multiple ROVs.

[0006] (3) Umbilical cable self-monitoring: By designing a strain detection system, the strain information of the umbilical cable can be monitored in real time, and the status of the umbilical cable can be analyzed. However, this method is costly and the system integration is complex. Summary of the Invention

[0007] Based on this, the present invention provides a salvage ROV umbilical cable end status identification and early warning method, which can automatically identify the normal, stretched, bent, interfered, twisted, entangled, knotted, damaged, broken and other states of the umbilical cable end, and realize automatic early warning according to the identification results, solving the defects of the above-mentioned manual observation and judgment. At the same time, the system of the present invention only needs to add an operating terminal to the original ROV, which improves feasibility.

[0008] In order to achieve the invention purpose of this application, this application adopts the following technical solutions:

[0009] The present invention provides a salvage ROV umbilical cable terminal state identification and early warning method, which includes the following steps:

[0010] (1) Collect images of the umbilical cable end in various states

[0011] (a) capturing video images of the end of the salvage ROV umbilical cable using an underwater camera and storing the video images, and performing flipping, rotation, and color changes on each state image of the captured video images to increase the number of the captured video images;

[0012] (2) Use the segmentation and positioning model to perform coarse segmentation on the video image at the end of the umbilical cable

[0013] (b) Pre-segment the above image using the SAM model to obtain an initial segmentation result that is independent of the category. Then, the above segmentation result is manually optimized and labeled, and the labels are binary images of the umbilical cable foreground and other backgrounds to obtain the image to be trained;

[0014] (c) Use the above images to train the segmentation and positioning model

[0015] (3) Use the umbilical cable end state recognition model to perform fine classification of images

[0016] (d) Use the post-processing module to first annotate the image from step (b) above to obtain sub-image blocks of the umbilical cable end. Then manually annotate these sub-image blocks into nine categories: normal, stretched, bent, interfered, twisted, entangled, knotted, damaged, and broken. Then use these as input data to train the umbilical cable end state recognition model to obtain classification results of 1-9.

[0017] (IV) Classify and identify the real-time image of the salvage ROV umbilical cable end and issue an early warning

[0018] Reading a real-time umbilical cable end image, and sequentially classifying the real-time umbilical cable end image using the trained segmentation and positioning model, the post-processing module, and the trained umbilical cable end state recognition model, according to the classification result, when the output of the umbilical cable end state recognition model is 2-3, a first-level alarm is issued to the umbilical cable end; when the output of the umbilical cable end state recognition model is 4-5, a second-level alarm is issued to the umbilical cable end; when the output of the umbilical cable end state recognition model is 4-6, a third-level alarm is issued to the umbilical cable end; when the output of the umbilical cable end state recognition model is 7-9, a fourth-level alarm is issued to the umbilical cable end;

[0019] When the segmentation and positioning model is trained with the image to be trained, when the loss function of the segmentation and positioning model converges, the segmentation and positioning model has been trained. The loss function L is calculated using the following formula (1):

[0020]

[0021] Where N represents the total number of predicted pixels, C represents the number of categories, C = 0 represents underwater background, C = 1 represents umbilical cable; 1{·} is an indicator function, which takes 1 if the condition is met and 0 if it is not met, y i Represents the true category label of the i-th pixel, taking y i ∈{0,1,...,C-1},p ij Indicates the probability that the i-th pixel is predicted to be category j; t is the threshold, which is 0.7. Since p ij The smaller it is, the more difficult it is to classify pixel i. ij Pixels with a value less than 0.7 are considered difficult examples, which makes the model pay more attention to the learning of difficult examples. j represents the class weight of the jth class, which is expressed as formula (2):

[0022]

[0023] Among them, hist j It represents the ratio of the number of pixels in the jth category to the total number of pixels in all categories. The smaller the ratio, the higher the weight θ. j The larger the value, the more γ is used to control the weight range. When γ=1.10, θ j ∈[1,10.5].

[0024] The present invention provides a salvage ROV umbilical cable terminal state recognition and early warning method, wherein: the post-processing module processes the above image and includes the following steps:

[0025] (I) Binarize the segmented result into foreground image and background image;

[0026] (II) Perform morphological closing operation on the processed foreground image to connect the umbilical cable break sub-regions caused by segmentation errors;

[0027] (III) Perform contour search on the foreground image after the closing operation, calculate the area, perimeter and image moment characteristics of each contour, and remove small area useless contours formed by noise;

[0028] (IV) Find the horizontal circumscribed rectangle of each contour;

[0029] Find the horizontal circumscribed rectangle of each contour, then merge all the circumscribed rectangles to the maximum extent, and expand the merged rectangle by 1.5-2 times to obtain a sub-image block that retains the information of the umbilical cable and the surrounding environment.

[0030] The present invention provides a method for identifying and warning the end state of a salvage ROV umbilical cable, wherein: the normal state is manifested as the umbilical cable being normally fixed to the ROV frame through a load-bearing head; the stretching state is manifested as the umbilical cable being stretched due to the ROV diving too fast; the bending state is manifested as the umbilical cable being paid out too long; the interference state is manifested as the umbilical cable interfering with the suspension cable or the ROV frame; the twisted state is manifested as the twisting force generated by the water flow at the end of the umbilical cable; the entanglement state is manifested as the umbilical cable being entangled with the ROV frame, the salvage target or the underwater object; the knotting state is manifested as the umbilical cable being knotted with the ROV frame; the damage state is manifested as the surface of the umbilical cable being damaged; and the fracture state is manifested as the fracture of the umbilical cable.

[0031] The present invention provides a method for identifying and warning the terminal state of a salvage ROV umbilical cable, wherein: the segmentation and positioning model includes: a local feature extraction module, a context feature extraction module, a feature fusion module and a segmentation module connected in sequence, the local feature extraction module is composed of at least three residual units connected in series, the outputs of the local feature extraction module and the context feature extraction module are used as inputs of a feature fusion module, and multi-scale fusion is achieved step by step. The segmentation module inputs the fusion features output by the feature fusion module into a preset FCN-based semantic segmentation model to complete the segmentation of the umbilical cable foreground and other backgrounds.

[0032] The present invention provides a salvage ROV umbilical cable terminal state recognition and early warning method, wherein: the first residual unit replaces the original first convolution kernel with a step size of 2 and a width of 7 in the initial stage with two groups of second convolution kernels with a step size of 1 and a width of 3 and one group of third convolution kernels with a step size of 2 and a width of 3, which are used to expand the image feature receptive field and reduce the number of parameters.

[0033] The present invention provides a salvage ROV umbilical cable terminal state recognition and early warning method, wherein: the context feature extraction module includes: a global feature extraction unit and a cross feature extraction unit; the global feature extraction unit performs global pooling processing on the input feature map, and performs a fourth convolution kernel processing with a width of 1 on it, and upsamples the convolution result to the size of the input feature map to obtain a global feature map for obtaining image maximum receptive field information; the cross feature extraction unit performs horizontal and vertical convolution on the input feature map respectively to obtain a cross-enhanced feature map for enhancing the edge features of the umbilical cable; and then the global feature map and the cross-enhanced feature map are accumulated pixel by pixel to enhance the context features.

[0034] The present invention provides a salvage ROV umbilical cable terminal state identification and early warning method, wherein:

[0035] The present invention provides a salvage ROV umbilical cable terminal state recognition and early warning method, wherein: the umbilical cable terminal state recognition model is a MobileNetV2 multi-category image classification model.

[0036] The present invention provides a salvage ROV umbilical cable terminal status identification and early warning method, wherein when a first-level early warning is performed, the operator displays an alarm message on the water surface or uses a surface umbilical cable winch to quickly reel in and release the cable; when a second-level early warning is performed, the operator displays an alarm message on the water surface or suspends the ROV operation and immediately fine-tunes the attitude and position; when a third-level early warning is performed, the operator issues an audible and visual alarm or suspends the ROV operation and immediately recovers the ROV; when a fourth-level early warning is performed, the operator cuts off the underwater high-voltage power supply or stops the salvage operation.

[0037] The working principle of the present invention is as follows: In view of the problems such as stretching, bending, interference, twisting, entanglement, knotting, damage, and breakage that may occur during the operation of the salvage ROV, the present invention designs a salvage ROV umbilical cable terminal state recognition and early warning method. First, the video image of the salvage ROV umbilical cable monitoring underwater camera is collected, and the umbilical cable area and image state in the image are marked. The marked training set is used to train the umbilical cable terminal segmentation and positioning model to realize pixel-level recognition of the umbilical cable terminal and locate its image area. The umbilical cable terminal state recognition model is trained using the marked images of different states to identify the segmented area and confirm the specific state. Secondly, the threat level is divided according to the identified umbilical cable terminal state, and corresponding early warning measures are executed according to the level to effectively prevent accidents. Finally, the method of the present invention can be directly applied to the existing salvage ROV system. It only needs to transmit the data of the umbilical cable monitoring underwater camera to the newly added operation terminal.

[0038] Compared with the existing methods, the present invention has the following beneficial effects:

[0039] (1) The present invention uses artificial intelligence technology to automatically identify and warn the terminal status of the umbilical cable without the participation of operators, greatly reducing the cost of manual judgment during the salvage operation and improving the level of intelligence;

[0040] (2) The present invention designs a "coarse-to-fine" recognition algorithm that first segments and then detects. It first locates the end area of the umbilical cable and then realizes refined recognition. Compared with directly recognizing the entire image, it can reduce the impact of the complex underwater environment and improve the recognition accuracy. In addition, since underwater images tend to present a large area of homogeneous areas, the segmentation algorithm can better segment the end position of the umbilical cable;

[0041] (3) The early warning module of the present invention divides the threat levels of stretching, bending, interference, twisting, entanglement, knotting, breakage and fracture into the first warning level, the second warning level, the third warning level and the fourth warning level, and designs different early warning measures for each level, thereby effectively ensuring the safety of the salvage operation.

[0042] (4) The method of the present invention only requires adding an operating terminal to the existing salvage ROV system, without the need for an expensive GPU server. This not only facilitates the integration of the existing ROV system but also significantly reduces the operating cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 Flowchart of the salvage ROV umbilical cable terminal status identification and early warning method of the present invention

[0044] Figure 2 This is the flow chart of the post-processing module of the present invention

[0045] Figure 3 Structural composition diagram of the umbilical cable end segmentation and positioning model of the present invention;

[0046] Figure 4 Structural diagram of the improved local feature extraction module in the initial stage of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concept of the present invention. It should be noted that, in the absence of conflict, the embodiments in this application and the features described in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0048] like Figure 1 As shown, steps (1) to (3) are the training model stage, and step (4) is the reasoning stage. The method for identifying and warning the terminal state of a salvaging ROV umbilical cable of the present invention includes the following steps:

[0049] (1) Collect images of the umbilical cable end in various states

[0050] (a) capturing video images of the end of the salvage ROV umbilical cable using an underwater camera and storing the video images, flipping, rotating, and color-changing the captured video images to increase the number of captured video images, and resizing the captured images to a size of 1920×1080;

[0051] (2) Use the segmentation and positioning model to perform coarse segmentation on the video image at the end of the umbilical cable

[0052] (b) Pre-segment the above image using the SAM model to obtain an initial segmentation result that is independent of the category. Then, the above segmentation result is manually optimized and labeled, and the labels are binary images of the umbilical cable foreground and other backgrounds to obtain the image to be trained;

[0053] (c) Use the above images to train the segmentation and positioning model

[0054] (3) Use the umbilical cable end state recognition model to perform fine classification of images

[0055] (d) Use an image cropping tool such as the post-processing module to process the image in step (b) above to obtain sub-image blocks, unify the size of the image blocks to 224×224, and assign them nine status labels, such as normal, stretched, bent, interference, twisted, entangled, knotted, damaged, and broken. These labels are then used as input data to train the umbilical cable end state recognition model, which is a MobileNetV2 multi-category image classification model.

[0056] Normal performance means that the umbilical cable is normally fixed to the ROV frame through the bearing head, with a score of 1;

[0057] Stretching is manifested by the umbilical cable being stretched due to the ROV diving too fast, with a score of 2;

[0058] The bending is manifested by the umbilical cable being too long, with a score of 3;

[0059] Interference occurs when the umbilical cable interferes with the gondola or ROV frame, with a score of 4;

[0060] The twisting state is manifested as the twisting force generated by the water flow at the end of the umbilical cable, with a score of 5;

[0061] Entanglement occurs when the umbilical cable becomes entangled with the ROV frame, salvage target, or underwater object, with a score of 6.

[0062] The knotting performance is that the umbilical cable is knotted with the ROV frame, with a score of 7;

[0063] The damage is manifested as surface damage to the umbilical cable, with a score of 8;

[0064] The fracture is manifested by the umbilical cable breaking, with a score of 9;

[0065] (IV) Classify and identify the real-time image of the salvage ROV umbilical cable end and issue an early warning

[0066] Reading a real-time umbilical cable end image, and sequentially classifying the real-time umbilical cable end image using the trained segmentation and positioning model, the post-processing module, and the trained umbilical cable end state recognition model, according to the classification result, when the output of the umbilical cable end state recognition model is 2-3, a first-level alarm is issued to the umbilical cable end; when the output of the umbilical cable end state recognition model is 4-5, a second-level alarm is issued to the umbilical cable end; when the output of the umbilical cable end state recognition model is 6-7, a third-level alarm is issued to the umbilical cable end; when the output of the umbilical cable end state recognition model is 8-9, a fourth-level alarm is issued to the umbilical cable end;

[0067] When a first-level warning is issued, the operator displays an alarm message on the surface or quickly reels in and releases the umbilical cable with a surface winch; when a second-level warning is issued, the operator displays an alarm message on the surface or suspends the ROV operation and immediately fine-tunes the attitude and position; when a third-level warning is issued, the operator issues an audible and visual alarm or suspends the ROV operation and immediately recovers it; when a fourth-level warning is issued, the operator cuts off the underwater high-voltage power or stops the salvage operation.

[0068] like Figure 2 As shown, in step (3) and step (4) of the above method, the post-processing module processes the above image and includes the following steps:

[0069] (I) Binarize the segmented result into foreground image and background image;

[0070] (II) Perform morphological closing operation on the processed foreground image to connect the umbilical cable break sub-regions caused by segmentation errors;

[0071] (III) Perform contour search on the foreground image after the closing operation, calculate the area, perimeter and image moment characteristics of each contour, and remove small area useless contours formed by noise;

[0072] (IV) Find the horizontal circumscribed rectangle of each contour;

[0073] Find the horizontal circumscribed rectangle of each contour, then merge all the circumscribed rectangles to the maximum extent, and expand the merged rectangle by 1.5-2 times to obtain a sub-image block that retains the information of the umbilical cable and the surrounding environment.

[0074] like Figure 3As shown, the segmentation and positioning model includes: a local feature extraction module 101, a context feature extraction module 102, a feature fusion module 103 and a segmentation module 104 connected in sequence. The local feature extraction module is composed of four residual units ResNet connected in series. The outputs of the local feature extraction module and the context feature extraction module are used as inputs of the feature fusion module to achieve multi-scale fusion step by step. The segmentation module completes the segmentation of the umbilical cable foreground and other backgrounds based on the fusion features output by the feature fusion module.

[0075] like Figure 4 As shown, the local feature extraction module 101 modifies the original 7x7 convolution with a stride of 2 in the initial stage into two groups of 3x3 convolutions with a stride of 1 and one group of 3x3 convolutions with a stride of 2, and then extracts four feature maps [C1, C2, C3, C4] with resolutions of 1 / 32, 1 / 16, 1 / 8, and 1 / 4 of the original image resolution.

[0076] The context feature extraction module of the segmentation and positioning model includes: a global feature extraction unit and a cross feature extraction unit; the global feature extraction unit performs global pooling processing on the input feature map, and performs a fourth convolution kernel processing with a width of 1 on it, and upsamples the convolution result to the size of the input feature map to obtain a global feature map for obtaining the maximum receptive field information of the image; the cross feature extraction unit performs horizontal and vertical convolution on the input feature map respectively to obtain a cross-enhanced feature map for enhancing the edge features of the umbilical cable; then the global feature map and the cross-enhanced feature map are accumulated pixel by pixel to enhance the context features.

[0077] The context feature extraction module 102 includes a global feature extraction unit and a cross feature extraction unit. The specific extraction steps are as follows:

[0078] A1, perform two-dimensional global pooling on the input feature map C1, so that the output feature map size becomes 1x1 pixels to obtain the maximum image receptive field information;

[0079] A2. Perform 1x1 convolution on the output feature map, including convolution, batch normalization and activation layers, to obtain a global feature map.

[0080] A3, upsample the global feature map to the input feature map size;

[0081] A4, perform horizontal and vertical convolution on the input feature map C1 to obtain the umbilical cable cross-enhanced feature map;

[0082] A5. Accumulate the upsampled global feature map and the cross-enhanced feature map pixel by pixel to obtain the context feature P1.

[0083] Furthermore, in step A4, the horizontal and vertical convolutions are 1xn and mx1 respectively, which are used to enhance the edge features of the umbilical cable in various directions. In this embodiment, m=n=7.

[0084] Specifically, the feature fusion module 103 consists of a lateral connection unit and an upsampling unit. It upsamples the above feature map [P1, C2, C3, C4] to the size of the C4 feature map, then performs a Cat operation on it, and finally uses a 3x3 convolution to complete the feature fusion to obtain the feature map P.

[0085] Specifically, the segmentation module 104 uses 3x3 convolution to reduce the number of channels of the feature map P to 2, and then uses Softmax to complete the category prediction.

[0086] The segmentation and positioning model is trained with the images to be trained. When the loss function of the segmentation and positioning model converges, the segmentation and positioning model has been trained. The loss function L is calculated using the following formula (1):

[0087]

[0088] Where N represents the total number of predicted pixels, C represents the number of categories, C = 0 represents underwater background, C = 1 represents umbilical cable; 1{·} is an indicator function, which takes 1 if the condition is met and 0 if it is not met, y i Represents the true category label of the i-th pixel, taking y i ∈{0,1,...,C-1},p ij Indicates the probability that the i-th pixel is predicted to be category j; t is the threshold, which is 0.7. Since p ij The smaller it is, the more difficult it is to classify pixel i. ij Pixels with a value less than 0.7 are considered difficult examples, which makes the model pay more attention to the learning of difficult examples. j represents the class weight of the jth class, which is expressed as formula (2):

[0089]

[0090] Among them, hist j It represents the ratio of the number of pixels in the jth category to the total number of pixels in all categories. The smaller the ratio, the higher the weight θ. j The larger the value, the more γ is used to control the weight range. When γ=1.10, θ j ∈[1,10.5].

[0091] The segmentation model was trained using the stochastic gradient descent optimization algorithm and the loss function. Specifically, the Adam optimizer was used in conjunction with the backpropagation training model, with an initial learning rate of 4e-4, a batch size of 8, a weight decay coefficient of 0.0001, and a maximum number of iterations of 8000. The cosine learning rate decay strategy was used to gradually reduce the learning rate to achieve better model convergence.

[0092] The present invention has been described in detail above. The principles of the present invention have been described herein. The above description of the working principles is only intended to help understand the core ideas of the present invention. It should be noted that those skilled in the art may improve and modify the present invention without departing from the principles of the present invention, and such improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A salvage ROV umbilical cable terminal status identification and early warning method, characterized by: The following steps are involved: (1) Collect images of the umbilical cable end in various states (a) capturing video images of the end of the salvage ROV umbilical cable using an underwater camera and storing the video images, and performing flipping, rotation, and color changes on each state image of the captured video images to increase the number of the captured video images; (2) Use the segmentation and positioning model to perform coarse segmentation on the video image at the end of the umbilical cable (b) Pre-segment the above image using the SAM model to obtain an initial segmentation result that is independent of the category. Then, the above segmentation result is manually optimized and labeled, and the labels are binary images of the umbilical cable foreground and other backgrounds to obtain the image to be trained; (c) Use the above images to train the segmentation and positioning model (3) Use the umbilical cable end state recognition model to perform fine classification of images (d) Use the post-processing module to first annotate the image from step (b) above to obtain sub-image blocks of the umbilical cable end. Then manually annotate these sub-image blocks into nine categories: normal, stretched, bent, interfered, twisted, entangled, knotted, damaged, and broken. Then use these as input data to train the umbilical cable end state recognition model to obtain classification results of 1-9. (IV) Classify and identify the real-time image of the salvage ROV umbilical cable end and issue an early warning Reading a real-time umbilical cable end image, and sequentially classifying the real-time umbilical cable end image using the trained segmentation and positioning model, the post-processing module, and the trained umbilical cable end state recognition model, according to the classification result, when the output of the umbilical cable end state recognition model is 2-3, a first-level alarm is issued to the umbilical cable end; when the output of the umbilical cable end state recognition model is 4-5, a second-level alarm is issued to the umbilical cable end; when the output of the umbilical cable end state recognition model is 6-7, a third-level alarm is issued to the umbilical cable end; when the output of the umbilical cable end state recognition model is 8-9, a fourth-level alarm is issued to the umbilical cable end; The loss function L used to train the segmentation and localization model is calculated using the following formula (1): Where N represents the total number of predicted pixels, C represents the number of categories, C = 0 represents underwater background, C = 1 represents umbilical cable; 1{·} is an indicator function, which takes 1 if the condition is met and 0 if it is not met, y i Represents the true category label of the i-th pixel, taking y i ∈{0,1,...,C-1},p ij Indicates the probability that the i-th pixel is predicted to be category j; t is the threshold, which is 0.

7. Since p ij The smaller it is, the more difficult it is to classify pixel i. ij Pixels with a value less than 0.7 are considered difficult examples, which makes the model pay more attention to the learning of difficult examples. j represents the class weight of the jth class, which is expressed as formula (2): Among them, hist j It represents the ratio of the number of pixels in the jth category to the total number of pixels in all categories. The smaller the ratio, the higher the weight θ. j The larger the value, the more γ is used to control the weight range. When γ=1.10, θ j ∈[1,10.5].

2. The salvage ROV umbilical cable terminal status identification and early warning method according to claim 1, characterized in that: The post-processing module processes the above image and includes the following steps: (I) Binarize the segmented result into foreground image and background image; (II) Perform morphological closing operation on the processed foreground image to connect the umbilical cable break sub-regions caused by segmentation errors; (III) Perform contour search on the foreground image after the closing operation, calculate the area, perimeter and image moment characteristics of each contour, and remove small area useless contours formed by noise; (IV) Find the horizontal circumscribed rectangle of each contour; Find the horizontal circumscribed rectangle of each contour, then merge all the circumscribed rectangles to the maximum extent, and expand the merged rectangle by 1.5-2 times to obtain a sub-image block that retains the information of the umbilical cable and the surrounding environment.

3. A salvage ROV umbilical cable terminal status identification and early warning method according to claim 2, characterized in that: The normal manifestation is that the umbilical cable is normally fixed to the ROV frame through the bearing head; the stretching manifestation is that the ROV dives too fast and the umbilical cable is stretched; the bending manifestation is that the umbilical cable is paid out too long; the interference manifestation is that the umbilical cable interferes with the suspension cable or the ROV frame; the twisted state is manifested as the twisting force generated by the water flow at the end of the umbilical cable; the entanglement manifestation is that the umbilical cable is entangled with the ROV frame, the salvage target or the underwater object; the knotting manifestation is that the umbilical cable is knotted with the ROV frame; the damage manifestation is that the surface of the umbilical cable is damaged; the breakage manifestation is that the umbilical cable is broken.

4. A salvage ROV umbilical cable terminal status identification and early warning method according to claim 3, characterized in that: The segmentation and positioning model includes: a local feature extraction module, a context feature extraction module, a feature fusion module and a segmentation module connected in sequence. The local feature extraction module is composed of at least three residual units connected in series. The outputs of the local feature extraction module and the context feature extraction module are used as inputs of the feature fusion module to achieve multi-scale fusion step by step. The segmentation module inputs the fusion features output by the feature fusion module into a preset FCN-based semantic segmentation model to complete the segmentation of the umbilical cable foreground and other backgrounds.

5. A salvage ROV umbilical cable terminal status identification and early warning method according to claim 4, characterized in that: The first residual unit changes the original first convolution kernel with a stride of 2 and a width of 7 in the initial stage to two sets of second convolution kernels with a stride of 1 and a width of 3 and one set of third convolution kernels with a stride of 2 and a width of 3, which are used to expand the image feature receptive field and reduce the number of parameters.

6. A salvage ROV umbilical cable terminal status identification and early warning method according to claim 5, characterized in that: The context feature extraction module includes: a global feature extraction unit and a cross feature extraction unit; the global feature extraction unit performs global pooling processing on the input feature map, and performs a fourth convolution kernel processing with a width of 1 on it, and upsamples the convolution result to the size of the input feature map to obtain a global feature map for obtaining the maximum receptive field information of the image; the cross feature extraction unit performs horizontal and vertical convolution on the input feature map respectively to obtain a cross-enhanced feature map for enhancing the edge features of the umbilical cable; then the global feature map and the cross-enhanced feature map are accumulated pixel by pixel to enhance the context features.

7. The method for identifying and warning the terminal state of a salvage ROV umbilical cable according to claim 1, characterized in that: The umbilical cable terminal state recognition model is a MobileNetV2 multi-category image classification model.

8. The method for identifying and warning the terminal state of a salvage ROV umbilical cable according to claim 7, characterized in that: When a first-level warning is issued, the operator displays an alarm message on the surface or quickly reels in and releases the umbilical cable with a surface winch; when a second-level warning is issued, the operator displays an alarm message on the surface or suspends the ROV operation and immediately fine-tunes the attitude and position; when a third-level warning is issued, the operator issues an audible and visual alarm or suspends the ROV operation and immediately recovers it; when a fourth-level warning is issued, the operator cuts off the underwater high-voltage power or stops the salvage operation.

Images