A method, system, and storage medium for ARV underwater tubing curvature detection using feature pre-fusion

Abstract

This invention discloses a feature pre-fusion method, system, and storage medium for underwater oil pipe curvature detection using an ARV (Unmanned Underwater Vehicle). The invention incorporates a feature pre-fusion module into the baseline network RTMDet. The underwater oil pipe is detected and segmented using the RTMDet network mounted on the ARV. The segmented oil pipe image is converted into a binary image, and the upper boundary of the oil pipe is extracted and transformed into a series of discrete point coordinates. A polynomial least squares method is used to fit the coordinate points to obtain a fitted curve, and the curvature of the underwater oil pipe is calculated using the fitted curve formula. This invention solves the problems of pipe twisting and rupture caused by excessive curvature during underwater oil pipe laying, providing a basis for monitoring the curvature of the oil pipe during laying, thus facilitating better completion of the oil pipe laying work.

Description

Technical Field

[0001] This invention belongs to the field of unmanned underwater vehicles, specifically relating to a feature pre-fusion method, system, and storage medium for detecting the curvature of underwater oil pipes in ARVs. Background Technology

[0002] Submarine pipeline transportation of marine oil and gas resources is the most economical, fastest, and most reliable mode of maritime transport. However, the complex marine environment necessitates specialized technical teams and equipment, resulting in high costs for pipeline laying and installation. In traditional pipeline laying operations, the pipeline is inspected for defects and sealed before being lowered to the seabed by the pipeline-laying vessel. This process can lead to pipeline damage and rupture due to excessive bending or curvature, resulting in significant manpower and material costs for repairs. Therefore, effective monitoring of pipeline curvature during laying is a pressing issue that needs to be addressed.

[0003] In recent years, with the development of deep learning theory and algorithms, object detection algorithms have seen significant improvements in both speed and accuracy. Some deep learning-based object detection algorithms have also been integrated into the environmental perception systems of underwater unmanned vehicles. For example, deep learning object detection frameworks can be used to detect and identify pipeline images. Further image processing can then enable real-time monitoring of pipeline curvature during pipeline laying. Summary of the Invention

[0004] The purpose of this invention is to provide a feature pre-fusion method, system and storage medium for detecting the curvature of underwater oil pipes in ARVs, to solve the problem of pipeline rupture and damage caused by excessive curvature during the laying of underwater oil pipes in existing engineering projects. A real-time oil pipe curvature monitoring method is designed for use in underwater unmanned vehicles to complete the oil pipe curvature monitoring task.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A feature pre-fusion method for detecting the curvature of underwater oil pipes in ARVs, the specific steps of which are as follows:

[0007] Step 1: Create a dataset by taking underwater oil pipeline images in the test pool; take images of the simulated oil pipeline laying process using an underwater ARV, and label the captured images with target locations and segmentation to construct an underwater oil pipeline detection dataset;

[0008] Step 2: Perform data augmentation preprocessing on the underwater oil pipe dataset. The underwater oil pipe detection dataset is augmented by random cropping, random splicing, random scaling, saturation transformation, contrast enhancement, and Mixup image operations.

[0009] Step 3: Construct the backbone network of the underwater oil pipe detection model RTMDet and add a feature pre-fusion module; use the RTMDet network as the network baseline, and generate feature maps of different sizes and depths in the backbone network. Before inputting them into the Neck network, new feature maps are generated by the added pre-fusion module, and then input into the Neck network for further processing;

[0010] Step 4: Training and predicting the network model and image processing; input the underwater oil pipe detection dataset to complete network training and use the test set to generate prediction results. Binarize the prediction results to generate a black and white image with the outline boundary of the oil pipe.

[0011] Step 5: Extract and discretize the contour boundary; extract the upper boundary contour of the black and white image with the oil pipe contour boundary, extract the points located on the upper boundary line at fixed intervals starting from the left side of the boundary line, and record the coordinates of the points.

[0012] Step 6: Solve for the fitted curve and radius of curvature; using the recorded point coordinates, obtain the fitted curve of the fitted coordinate points and the fitting formula of the fitted curve, and use the radius of curvature formula to solve for the radius of curvature value at each coordinate point.

[0013] Step 7: Compare the radius of curvature value at each coordinate point with the critical radius of curvature R0 of the oil pipe during the oil pipe laying process. If it is greater than R0, it meets the safety operation requirements; if it is less than R0, the work vessel needs to be prompted to adjust the operation mode and return to Step 1 for re-monitoring.

[0014] Furthermore, the Mixup image operation in step two is as follows: each time, after performing operations such as random splicing, saturation transformation, and contrast enhancement on two images, different coefficients are multiplied by the two images respectively, and then superimposed to obtain a new image.

[0015] Furthermore, in step three, the pre-fusion module is a newly added module located between the Backbone and Neck of the baseline RTMDet network model. The network incorporates three pre-fusion modules from shallow to deep. Each pre-fusion module takes the network feature maps of three different stages in the Backbone as input, processes them, and then outputs a feature map. Finally, the three feature maps of different sizes and depths generated by the three pre-fusion modules are used as inputs to the next stage, the Neck, for further processing.

[0016] Furthermore, the pre-fusion module first integrates i-1 th Feature map, i th Feature map and i+1 th The feature map is used as input for i thThe feature map is upsampled to size C, 4H, 4W using deconvolution and then compared with i-1. th Feature maps are summed, and the result is subjected to a non-linear transformation using DW convolution. In depthwise separable convolution, a dilation coefficient of 2 is used to expand the receptive field of the feature map during the non-linear transformation. Simultaneously, depthwise separable convolution effectively reduces model parameters and computational overhead. The feature map output from the depthwise separable convolution is then reduced to a size of 2C, 2H, 2W using the Down-Focus module designed above. Similarly, for i+1 with a size of 4C, H, W... th We enlarged the feature maps to 2C, 2H, and 2W by applying depthwise separable nonlinear convolution and up-focus operations respectively, and then processed the i-1... th Feature map, i+1 th Feature map and i th The feature maps are stacked and finally transformed back to i using a depthwise separable convolution operation. th Scale and output the feature map.

[0017] Furthermore, in step four, the binarization process refers to setting the pixel values ​​of the mask portion to 255 and the values ​​of the non-mask portion to 0 in the oil pipe mask image obtained by the network prediction, thereby obtaining a black and white image with white oil pipe segmentation outlines.

[0018] Furthermore, in step six, the fitting curve refers to the fitting using the polynomial least squares method. The least squares model is obtained by constructing a polynomial model and summing the squares of the differences between the polynomial model and the y-coordinates of each discrete point. The matrix solution of the model parameters is obtained based on the fact that the partial derivatives of each model parameter at the extreme point are 0.

[0019] The equation of the fitted curve is:

[0020] y = a0 + a1x + a2x 2 +…+a k x k =XA

[0021] in, x and y represent the coordinate variables of the fitted curve, a0, a1, ..., a k These are the coefficients of the polynomial fitting curve, where k is the polynomial order, (x... i ,y i The coordinates of the oil pipe boundary points (i = 1, 2, ..., n) are obtained through the aforementioned steps.

[0022] The radius of curvature R is expressed as:

[0023]

[0024] in:

[0025]

[0026] Where y' and y” represent the first and second derivatives of the polynomial curve, respectively.

[0027] Substituting the above formula into the radius of curvature, we can solve for the coordinates of the point (x). i ,y i The radii of curvature of the tubing boundary represented by i = 1, 2, ..., n are as follows:

[0028]

[0029] A computer device / equipment / system includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of a feature pre-fusion ARV underwater tubing curvature detection method.

[0030] A computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the steps of a feature pre-fusion method for detecting the curvature of an ARV underwater tubing.

[0031] A computer program product includes a computer program / instructions that, when executed by a processor, implement the steps of a feature pre-fusion method for detecting the curvature of an ARV underwater tubing.

[0032] The beneficial effects of this invention are as follows:

[0033] This invention improves the accuracy of real-time underwater pipeline detection by enriching the contextual information in the feature maps of the neural network through the use of a pre-fusion module. By designing a post-processing curvature algorithm to predict images using a network model, the curvature at different points in the pipeline laying is accurately obtained. Comparing this curvature with the critical curvature allows for real-time monitoring of the pipeline laying status. Compared to traditional pipeline laying methods, this effectively avoids pipeline rupture and damage caused by excessive curvature during laying, significantly reducing rework and improving the efficiency of pipeline laying. Attached Figure Description

[0034] Figure 1 This is a flowchart of an ARV underwater tubing curvature detection method based on feature pre-fusion according to the present invention;

[0035] Figure 2 This is a partial network structure diagram of the present invention;

[0036] Figure 3 This is a schematic diagram of the feature map pre-fusion module structure of the present invention;

[0037] Figure 4 This is a binarized image of the oil pipe detection results of the present invention;

[0038] Figure 5 This is a discrete point diagram of the upper boundary curve for oil pipe detection according to the present invention;

[0039] Figure 6 This is a curve fitting diagram for the oil pipe detection according to the present invention. Detailed Implementation

[0040] The present invention will now be further described with reference to the accompanying drawings.

[0041] As attached Figure 1 The flowchart shown is for the ARV underwater tubing curvature detection method based on feature prefusion, including the following steps:

[0042] Step 1: Create a dataset by taking underwater pipeline images in the test pool; take images of the simulated pipeline laying process using an underwater ARV, and label the captured images with target locations and segmentation to construct an underwater pipeline detection dataset.

[0043] Step 2: Perform data augmentation preprocessing on the underwater oil pipe dataset. The underwater oil pipe detection dataset is augmented by image operations such as random cut, random paste, random scaling, saturation transformation, contrast enhancement, and Mixup.

[0044] Step 3: Construct the backbone network of the underwater oil pipe detection model RTMDet and add a feature pre-fusion module; using the RTMDet network as the network baseline, five feature maps of different sizes and depths will be generated in the backbone network. Before inputting them into the Neck network, new feature maps will be generated by the added pre-fusion module, and then input into the Neck network for further processing.

[0045] Step 4: Training the network model and image processing for prediction; input the underwater oil pipe detection dataset to complete network training and use the test set to generate prediction results. Binarize the prediction results to generate a black and white image with the outline boundary of the oil pipe.

[0046] Step 5: Extract the contour boundary and discretize; extract the upper boundary contour of the black and white image with the oil pipe contour boundary, extract the points located on the upper boundary line at fixed intervals starting from the left side of the boundary line, and record the coordinates of the points.

[0047] Step 6: Solve for the fitted curve and radius of curvature; using the recorded point coordinates, obtain the fitted curve of the fitted coordinate points and the fitting formula of the fitted curve, and use the radius of curvature formula to solve for the radius of curvature value at each coordinate point.

[0048] Step 7: Compare the radius of curvature value at each coordinate point with the critical radius of curvature R0 of the pipeline during the pipeline laying process. If it is greater than R0, it meets the safety operation requirements; if it is less than R0, the work vessel needs to be prompted to adjust the operation mode and return to Step 1 for re-monitoring to complete the ARV underwater pipeline curvature detection.

[0049] Further details are attached. Figure 2 As shown, this illustrates the location of the pre-fusion module within the network model. The original RTMDet network model directly transmits the three deep feature maps from the Backbone to the Neck as feature input, ignoring the two shallow feature maps, which actually contain rich detail information and can greatly improve the target localization capability. Therefore, a pre-fusion module is proposed and added between the Neck and Backbone as a preprocessing layer for the Neck layer. Each module receives the Backbone output feature maps from three different stages and performs fusion processing. The feature maps output by the three parallel modules are then fused in a top-down manner. Finally, a DW convolution is used to output a feature map of the same size as the original Backbone, which is then passed to the Neck layer for further processing.

[0050] Further details are attached. Figure 3 The diagram illustrates the specific components of the pre-fusion module. First, i-1... th Feature map, i th Feature map and i+1 th The feature map is used as input for i th For the feature map, we use deconvolution to upsample the feature map to a size of (C, 4H, 4W) and then combine it with i-1. th The feature maps are summed, and the result is subjected to a non-linear transformation using DW convolution. In depthwise separable convolution, we use a dilation coefficient of 2 for the convolution kernel, which allows the feature map to obtain a larger receptive field during the non-linear transformation. At the same time, using depthwise separable convolution can effectively reduce model parameters and reduce the extra overhead in the model operation. The feature map output by the depthwise separable convolution is then reduced to a size of (2C, 2H, 2W) by the Down-Focus module designed above. Similarly, for i+1 of size (4C, H, W)... th We enlarged the feature map to (2C, 2H, 2W) by applying depthwise separable nonlinear convolution and UP-Focus operations, and then processed the i-1... th Feature map, i+1 th Feature map and i th The feature maps are stacked and finally transformed back to i using a depthwise separable convolution operation. th Size and output of the feature map. (Self-designed)

[0051] Further details are attached. Figure 4 As shown, the oil pipe mask image predicted by the network model is binarized to obtain a black and white image with white oil pipe segmentation outlines.

[0052] Further details are attached. Figure 5 As shown, the coordinates of discrete points on the upper boundary of a binary image with tubing segmentation are extracted and given as red dots in the image.

[0053] Furthermore, the polynomial least squares method is used to fit the discrete points and the fitted curve is given. The formula derivation is as follows:

[0054] Assume the coordinates of the discrete point are {(x i ,y i ),i=0,1,2,3,…n}

[0055] Let the polynomial model be:

[0056] y = a0 + a1x + a2x 2 +…+a k x k

[0057] Based on the least squares formula and the model assumptions, we can obtain:

[0058]

[0059] To minimize the result, the parameters a0, a1, a2, ..., a k The partial derivatives should satisfy the following conditions. Therefore, we obtain n equations:

[0060]

[0061] Simplifying and rearranging the above equations, we obtain the matrix equation as follows:

[0062]

[0063] To solve for the parameters a0, a1, a2, ..., a k First, assume that matrix X is a Vandermonde matrix, then:

[0064]

[0065] but:

[0066]

[0067] At the same time:

[0068]

[0069] From the above, we can conclude that:

[0070] XX T A = XY

[0071] Solving for the parameters a0, a1, a2, ..., a k for:

[0072]

[0073] Right now:

[0074]

[0075] Furthermore, plotting the obtained polynomial fitting curve in a graph yields the following result. Figure 6 The image shown, and its line fitting equation, are attached. Figure 6 As shown.

[0076] Furthermore, the radius of curvature at any point can be obtained using the formula for calculating the radius of curvature, as follows:

[0077] The equation of the fitted curve is:

[0078] y = a0 + a1x + a2x 2 +…+a k x k =AX

[0079] The formula for radius of curvature is:

[0080]

[0081] in:

[0082]

[0083] Substituting the above formula into the radius of curvature, we can solve for the coordinates of the point (x). i ,y i The radius of curvature at point () is shown below:

[0084]

[0085] Furthermore, by comparing the radius of curvature at each coordinate point obtained with the critical radius of curvature, it can be determined whether the curvature during the pipeline laying process meets the laying requirements.

[0086] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An ARV underwater tubing curvature detection method with feature pre-fusion, characterized in that: The specific steps are as follows: Step one: underwater oil pipe image shooting in the test pool to make a data set; underwater ARV is used to shoot images simulating the process of oil pipe laying, and the shooting images are labeled for target position and segmentation, to build an underwater oil pipe detection data set; Step two: data expansion preprocessing of the underwater oil pipe detection data set to expand the underwater oil pipe detection data set; data expansion preprocessing includes random cutting, random splicing, random scaling, saturation transformation, contrast enhancement and Mixup image operation; Step three: building the backbone network of the underwater oil pipe detection model RTMDet and adding a feature pre-fusion module; RTMDet network is used as the network baseline, different size and depth feature maps are generated in the backbone network, and new feature maps are generated through the added pre-fusion module before being input into the Neck network for further processing; Step four: network model training and prediction image processing; input the underwater oil pipe detection data set to complete network training and use the test set to produce prediction results, and the prediction results are binarized to produce a black and white image with oil pipe contour boundary; Step five: extracting the contour boundary and discretizing; the upper boundary contour on the black and white image with oil pipe contour boundary is extracted, and points on the upper boundary line are extracted every fixed distance from the left side of the boundary line, and the point coordinates are recorded; Step six: solving the fitting curve and the radius of curvature; the fitting curve of the recorded point coordinates is solved, the fitting formula of the fitting curve is obtained, and the radius of curvature value at each coordinate point is solved by using the formula of the radius of curvature; The fitting curve refers to the polynomial least squares fitting, a polynomial model is constructed, and the least squares model is obtained by minimizing the sum of squares of the difference between each discrete point and the y coordinate, and the matrix of model parameters is obtained according to the partial derivative of each model parameter being 0; The fitting curve equation is: wherein, , and denote the coordinate variables of the fitting curve, is the coefficient of the polynomial fitting curve, is the polynomial order, is the coordinate of the tubing boundary point obtained by the foregoing steps; radius of curvature is represented as: Wherein: wherein and denote the first and second derivative of the polynomial curve, respectively; Substitute the above equation into the radius of curvature, and solve the coordinate point The oil pipe boundary curvature radius is represented as follows: Step seven: compare the curvature radius value at each coordinate point with the critical curvature radius of the tubing during the tubing laying process If greater than , it meets the safety operation requirements; if less than , the workboat needs to be prompted to adjust the operation mode and return to step one for re-monitoring.

2. The method of claim 1, wherein: The Mixup image operation in step two is that after random splicing, saturation transformation and contrast enhancement operations are performed on two images, a new image is obtained by multiplying the two images with different coefficients and then superimposing them.

3. The method of claim 1, wherein: The pre-fusion module in step three is a newly added module between the Backbone and the Neck of the baseline RTMDet network model, three pre-fusion modules are added from shallow to deep in the network, each pre-fusion module takes the network feature maps of three different stages in the Backbone as input, processes them to obtain a feature map output, and finally, the three different size and depth feature maps generated by the three pre-fusion modules are input into the next stage Neck for further processing.

4. The method of claim 3, wherein: The pre-fusion module first upsamples feature maps, feature maps and feature maps as inputs, for feature maps, the feature maps are upsampled to the size of C, 4H, 4W by using the deconvolution operation and added to feature maps, the result is nonlinearly transformed by using the DW convolution, in the depth separable convolution, the dilated convolution kernel with the dilated coefficient of 2 is used, so that the feature maps can obtain a larger receptive field when nonlinearly transformed, at the same time, the depth separable convolution can effectively reduce the model parameters and reduce the additional overhead in the model operation process, the feature maps output by the depth separable convolution are then reduced to the size of 2C, 2H, 2W by the Down-Focus module designed above, and similarly, for feature maps, the depth separable nonlinear convolution and the UP-Focus operation are used to enlarge the feature maps to 2C, 2H, 2W, then the above processed feature maps, feature maps and feature maps are stacked, and finally the size of feature maps is transformed back by the depth separable convolution operation and output.

5. The method of claim 1, wherein: The binarization processing in step four refers to setting the pixel value of the Mask part to 255 and the value of the non-Mask part to 0 in the oil pipe Mask image predicted by the network, to obtain a black and white image with a white oil pipe segmentation contour.

6. A computer system comprising a memory, a processor, and a computer program stored on the memory, wherein: The processor executes the computer program to implement the steps of the method of any one of claims 1 to 5.

7. A computer readable storage medium having stored thereon a computer program, characterized in that: The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 5.

8. A computer program product comprising a computer program, characterized in that: The computer program, which is executed by a processor, implements the steps of the method according to any one of claims 1 to 5.

Images