Characteristic pre-fusion ARV underwater oil pipe curvature detection method and system and storage medium

Abstract

The invention discloses a feature pre-fusion ARV underwater oil pipe curvature detection method and system and a storage medium, and belongs to the field of unmanned underwater vehicles. A feature pre-fusion module is added in a baseline network RTMDet, the underwater oil pipe is detected and segmented through an ARV carrying RTMDet network, the segmented oil pipe picture is converted into a binary picture, the upper boundary of the oil pipe is extracted and converted into a series of discrete point coordinates, coordinate points are fitted through a polynomial least square method, a fitting curve is obtained, and the underwater oil pipe is subjected to underwater oil pipe detection and segmentation. And the curvature of the underground oil pipe is solved through a fitting curve formula. The problems of pipeline distortion, breakage and the like caused by overlarge curvature in the underwater oil pipe laying process are solved, a basis is provided for curvature monitoring of the oil pipe in the oil pipe laying process, and the oil pipe laying work can be better completed.

Description

Technical Field

[0001] The present invention belongs to the field of unmanned underwater vehicles, and in particular relates to a feature pre-fusion ARV underwater oil pipe curvature detection method, system and storage medium. Background Art

[0002] Submarine pipeline transportation of marine oil and gas resources is the most economical, fast and reliable mode of transportation in marine transportation. Due to the complex operating environment of the ocean, professional technical teams and technical equipment are required, resulting in high costs for laying and installing pipelines. In traditional pipeline laying operations, the offshore pipe-laying vessel performs defect detection and connection sealing on the pipeline before lowering it to the seabed, and then lowers it to the seabed. This will cause the pipeline to be damaged and ruptured due to excessive bending curvature during the laying process, resulting in considerable manpower and material resources spent on repairing the pipeline. Therefore, it is an urgent problem to monitor the curvature of the pipeline during the laying process.

[0003] In recent years, with the development of deep learning theory and algorithms, target detection algorithms have greatly improved in speed and accuracy. Some deep learning-based target detection algorithms have also been integrated into the environmental perception system of underwater unmanned vehicles. The deep learning target detection framework can be used to detect and identify pipeline images, and the images can be further processed on this basis to achieve real-time monitoring of the curvature of the pipeline during the laying process. Summary of the invention

[0004] The purpose of the present invention is to provide a feature pre-fused ARV underwater oil pipeline curvature detection method, system and storage medium to solve the pipeline rupture and damage problems caused by excessive oil pipeline curvature during the laying process of submarine underwater oil pipelines in existing projects. A real-time oil pipeline curvature monitoring method is designed for underwater unmanned vehicles to complete the oil pipeline curvature monitoring task.

[0005] The purpose of the present invention is achieved through the following technical solutions:

[0006] A feature pre-fusion ARV underwater oil pipe curvature detection method, the specific steps are as follows:

[0007] Step 1: Take underwater oil pipeline images in the test pool to create a data set; use an underwater ARV to take images of the simulated pipeline laying process, and annotate the captured images with target locations and segmentation labels to build an underwater oil pipeline detection data set;

[0008] Step 2: Perform data expansion preprocessing on the underwater oil pipeline dataset to expand the underwater oil pipeline detection dataset; data expansion preprocessing includes random cutting, random splicing, random scaling, saturation transformation, contrast enhancement and Mixup image operations;

[0009] Step 3: Construct the backbone network of the underwater oil pipeline detection model RTMDet and add a feature pre-fusion module; use the RTMDet network as the network baseline, and generate feature maps of different sizes and depths in the backbone network. Before inputting them into the Neck network, a new feature map is first generated by the added pre-fusion module, and then input into the Neck network for further processing;

[0010] Step 4: Network model training and prediction image processing: Input the underwater oil pipeline detection data set to complete network training and use the test set to generate prediction results, binarize the prediction results to generate a black and white image with the outline of the oil pipeline;

[0011] Step 5: Extract the contour boundary and discretize it; extract the upper boundary contour from the black-and-white image with the oil pipe contour boundary, extract points on the upper boundary line at a fixed distance from the left side of the boundary line, and record the point coordinates;

[0012] Step 6: Solve the fitting curve and curvature radius; obtain the fitting curve of the fitting coordinate point through the recorded point coordinates, and obtain the fitting formula of the fitting curve, and use the curvature radius formula to solve and obtain the curvature radius value at each coordinate point;

[0013] Step 7: Compare the curvature radius value at each coordinate point with the critical curvature radius R of the oil pipe during the oil pipe laying process. 0 Compare, if it is greater than R 0 , it meets the safety operation requirements; if it is less than R 0 , it is necessary to prompt the operating vessel to adjust the operating mode and return to step 1 for re-monitoring.

[0014] Furthermore, the Mixup image operation in step 2 is: after randomly splicing, saturation conversion, contrast enhancement and other operations are performed on two images each time, different coefficients are used to multiply the two images respectively and then superimpose them to obtain a new image.

[0015] Furthermore, the pre-fusion module in the step three is a newly added module between the Backbone and Neck of the baseline RTMDet network model. Three pre-fusion modules are added to the network from shallow to deep. Each pre-fusion module takes the network feature maps of three different stages in the Backbone as input, and obtains a feature map output after processing. Finally, the three feature maps of different sizes and depths generated by the three pre-fusion modules are used as the input of the Neck of the next stage for further processing.

[0016] Furthermore, the pre-fusion module first converts i-1 th Feature map, i th Feature map and i+1 th The feature map is used as input, for ith The feature map is upsampled to the size of C, 4H, 4W by deconvolution operation and combined with i-1 th The feature maps are added, and the result is transformed nonlinearly using DW convolution. In the depthwise separable convolution, the dilated convolution kernel with a dilation factor of 2 is used, so that the feature map can obtain a larger receptive field during nonlinear transformation. At the same time, the use of depthwise separable convolution can effectively reduce model parameters and reduce the additional overhead in the model operation process. The feature map output by the depthwise separable convolution is then reduced to the size of 2C, 2H, and 2W through the Down-Focus module designed above. Similarly, for i+1 with a size of 4C, H, and W, th We use depth-separable nonlinear convolution and UP-Focus operations to enlarge the feature map to 2C, 2H, and 2W, and then use the above processed i-1 th Feature map, i+1 th Feature map and i th The feature maps are stacked and finally transformed back to i through a depth-wise separable convolution operation. th The size of the feature map is output.

[0017] Furthermore, the binarization processing in step 4 refers to setting the pixel values ​​of the masked part to 255 and the values ​​of the non-masked part to 0 in the oil pipeline mask image obtained by network prediction, so as to obtain a black and white image with a white oil pipeline segmentation outline.

[0018] Furthermore, the fitting curve in step 6 refers to polynomial least squares fitting, a polynomial model is constructed and the least squares model is obtained by the sum of the squares of the differences between the y coordinates of each discrete point, and the matrix solution of the model parameters is obtained based on the partial derivative of each model parameter at the extreme point being 0;

[0019] The fitting curve equation is:

[0020] y=a 0 +a 1 x+a 2 x 2 +…+a k x k =XA

[0021] in, x and y represent the coordinate variables of the fitting curve, a 0 ,a 1 ,…,a k is the coefficient of the polynomial fitting curve, k is the polynomial order, (x i ,y i ), i = 1, 2, ..., n are the coordinates of the pipeline boundary points obtained through the above steps.

[0022] The radius of curvature R is expressed as:

[0023]

[0024] in:

[0025]

[0026] where y' and y" represent the first and second order derivatives of the polynomial curve, respectively.

[0027] Substitute the above formula into the radius of curvature and solve the coordinate point (x i ,y i ), i=1,2,…,n represents the radius of curvature of the tubing boundary, as follows:

[0028]

[0029] A computer device / equipment / system comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of a feature pre-fusion ARV underwater oil pipe curvature detection method.

[0030] A computer-readable storage medium stores a computer program / instruction, which, when executed by a processor, implements the steps of a feature pre-fusion ARV underwater oil pipe curvature detection method.

[0031] A computer program product includes a computer program / instruction, which, when executed by a processor, implements the steps of a feature pre-fusion ARV underwater oil pipe curvature detection method.

[0032] The beneficial effects of the present invention are:

[0033] The present invention enriches the context information in the neural network feature map by using a pre-fusion module, thereby improving the accuracy of real-time detection of underwater oil pipelines. By designing a post-processing curvature algorithm for predicting images using a network model, the curvature of different points of the laid oil pipeline is accurately obtained, and the curvature is compared with the critical curvature to monitor the laying status of the oil pipeline in real time. Compared with the original oil pipeline laying method, it can effectively avoid the occurrence of pipeline rupture and damage caused by excessive curvature of the oil pipeline during the laying process, effectively reduce the rework process during the laying of the oil pipeline, and improve the efficiency of the oil pipeline laying. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a flow chart of a feature pre-fusion ARV underwater oil pipe curvature detection method of the present invention;

[0035] Figure 2 It is a schematic diagram of a partial network structure of the present invention;

[0036] Figure 3 It is a structural schematic diagram of a feature graph pre-fusion module of the present invention;

[0037] Figure 4 It is a binary image of the oil pipe detection result of the present invention;

[0038] Figure 5 It is a discrete point diagram of the upper boundary curve of the oil pipe detection of the present invention;

[0039] Figure 6 It is the oil pipe detection curve fitting diagram of the present invention. DETAILED DESCRIPTION

[0040] The present invention is further described below in conjunction with the accompanying drawings.

[0041] As attached Figure 1 FIG. 1 is a flow chart of a feature pre-fusion ARV underwater oil pipe curvature detection method, comprising the following steps:

[0042] Step 1: Take underwater oil pipeline images in the test pool to create a dataset; use an underwater ARV to take images of the simulated pipeline laying process, and annotate the captured images with target positions and segmentation labels to construct an underwater oil pipeline detection dataset.

[0043] Step 2: Perform data expansion preprocessing on the underwater oil pipeline dataset to expand the underwater oil pipeline detection dataset; data expansion preprocessing includes image operations such as random cutting, random pasting, random scaling, saturation transformation, contrast enhancement and Mixup.

[0044] Step 3: Construct the backbone network of the underwater oil pipeline detection model RTMDet and add a feature pre-fusion module; use the RTMDet network as the network baseline, and five feature maps of different sizes and depths will be generated in the backbone network. Before inputting them into the Neck network, a new feature map will be generated by the added pre-fusion module, and then input into the Neck network for further processing.

[0045] Step 4: Network model training and prediction image processing: Input the underwater oil pipeline detection data set to complete network training and use the test set to generate prediction results. The prediction results are binarized to generate a black and white image with the outline of the oil pipeline.

[0046] Step 5: Extract the contour boundary and discretize it; extract the upper boundary contour in the black-and-white image with the oil pipeline contour boundary, extract points on the upper boundary line at a fixed distance from the left side of the boundary line, and record the point coordinates.

[0047] Step 6: Solve the fitting curve and curvature radius; obtain the fitting curve of the fitting coordinate point through the recorded point coordinates, and obtain the fitting formula of the fitting curve, and use the curvature radius formula to solve and obtain the curvature radius value at each coordinate point;

[0048] Step 7: Compare the curvature radius value at each coordinate point with the critical curvature radius R of the oil pipe during the oil pipe laying process. 0 Compare, if it is greater than R 0 , it meets the safety operation requirements; if it is less than R 0 , it is necessary to prompt the operating vessel to adjust the operating mode and return to step 1 for re-monitoring to complete the ARV underwater oil pipe curvature detection.

[0049] Further, as attached Figure 2 As shown in the figure, the network model position of the pre-fusion module is shown. The original RTMDet network model directly transmits the three deep feature maps of BackBone to Neck as feature input, while ignoring the two shallow feature maps. However, in fact, it has rich detail information and can greatly improve the positioning ability of the target. Therefore, a pre-fusion module is proposed and added between Neck and BackBone as a preprocessing layer of the Neck layer. Each module receives the output feature maps of BackBone at three different stages and performs fusion processing. The feature maps output by the three parallel modules are then fused in a top-down manner. Finally, the feature map size of the same as the original BackBone is output through DW convolution and passed to the Neck layer for further processing.

[0050] Further, as attached Figure 3 As shown in the figure, the specific composition of the pre-fusion module is shown. First, i-1 th Feature map, i th Feature map and i+1 th The feature map is used as input, for i th Feature map, we use the deconvolution operation to upsample the feature map to the size of (C, 4H, 4W) and combine it with i-1 th The feature maps are added and the result is transformed nonlinearly using DW convolution. In the depthwise separable convolution, we use a dilated convolution kernel with a dilation factor of 2, so that the feature map can obtain a larger receptive field during nonlinear transformation. At the same time, the use of depthwise separable convolution can effectively reduce model parameters and reduce the additional overhead in the model operation process. The feature map output by the depthwise separable convolution is then reduced to the size of (2C, 2H, 2W) through the Down-Focus module designed above. Similarly, for i+1 with a size of (4C, H, W) thWe use depth-separable nonlinear convolution and UP-Focus operations to enlarge the feature map to (2C, 2H, 2W), and then use the above processed i-1 th Feature map, i+1 th Feature map and i th The feature maps are stacked and finally transformed back to i through a depth-wise separable convolution operation. th The size of the feature map and output. Designed by myself

[0051] Further, as attached Figure 4 As shown, the oil pipeline Mask image predicted by the network model is binarized to obtain a black and white image with a white oil pipeline segmentation outline.

[0052] Further, as attached Figure 5 As shown, the coordinates of the discrete points on the upper half of the boundary of the binary image with oil pipe segmentation are extracted and given as red points in the figure.

[0053] Furthermore, the polynomial least squares method is used to fit the discrete points and give the fitting curve. The formula is derived as follows:

[0054] Assume that the coordinates of the discrete points are {(x i ,y i ),i=0,1,2,3,…n}

[0055] Assume the polynomial model is:

[0056] y=a 0 +a 1 x+a 2 x 2 +…+a k x k

[0057] According to the least squares formula and model assumptions, we can get:

[0058]

[0059] To minimize the result, the parameter a 0 ,a 1 ,a 2 ,…,a k All partial derivatives should satisfy So we get n equations:

[0060]

[0061] Simplifying the above equation, we can get the matrix equation as follows:

[0062]

[0063] In order to solve the parameter a 0 ,a 1 ,a 2 ,…,a k , first assume that the matrix X is a Vandermonde matrix, then:

[0064]

[0065] but:

[0066]

[0067] At the same time:

[0068]

[0069] From the above, we can get:

[0070] XX T A=XY

[0071] Solve for parameter a 0 ,a 1 ,a 2 ,…,a k for:

[0072]

[0073] Right now:

[0074]

[0075] Furthermore, the obtained polynomial fitting curve can be plotted in a graph as shown in the attached figure. Figure 6 The image shown and its line fitting equation are shown in the attached Figure 6 shown.

[0076] Furthermore, the radius of curvature at any point can be obtained by solving the formula for the radius of curvature. The formula is as follows:

[0077] The fitting curve equation is:

[0078] y=a 0 +a 1 x+a 2 x 2 +…+a k x k =AX

[0079] The formula for the radius of curvature is:

[0080]

[0081] in:

[0082]

[0083] Substituting the above formula into the radius of curvature, we can solve the coordinate point (x i ,y i ) is as follows:

[0084]

[0085] Furthermore, by comparing the obtained curvature radius at each coordinate point with the critical curvature radius, it can be determined whether the curvature during the laying process of the oil pipe meets the laying requirements.

[0086] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A feature pre-fusion ARV underwater oil pipe curvature detection method, characterized by: The specific steps are as follows: Step 1: Take underwater oil pipeline images in the test pool to create a data set; use an underwater ARV to take images of the simulated pipeline laying process, and annotate the captured images with target locations and segmentation labels to build an underwater oil pipeline detection data set; Step 2: Perform data expansion preprocessing on the underwater oil pipeline dataset to expand the underwater oil pipeline detection dataset; data expansion preprocessing includes random cutting, random splicing, random scaling, saturation transformation, contrast enhancement and Mixup image operations; Step 3: Construct the backbone network of the underwater oil pipeline detection model RTMDet and add a feature pre-fusion module; use the RTMDet network as the network baseline, and generate feature maps of different sizes and depths in the backbone network. Before inputting them into the Neck network, a new feature map is first generated by the added pre-fusion module, and then input into the Neck network for further processing; Step 4: Network model training and prediction image processing: Input the underwater oil pipeline detection data set to complete network training and use the test set to generate prediction results, binarize the prediction results to generate a black and white image with the outline of the oil pipeline; Step 5: Extract the contour boundary and discretize it; extract the upper boundary contour from the black-and-white image with the oil pipe contour boundary, extract points on the upper boundary line at a fixed distance from the left side of the boundary line, and record the point coordinates; Step 6: Solve the fitting curve and curvature radius; obtain the fitting curve of the fitting coordinate point through the recorded point coordinates, and obtain the fitting formula of the fitting curve, and use the curvature radius formula to solve and obtain the curvature radius value at each coordinate point; Step 7: Compare the curvature radius value at each coordinate point with the critical curvature radius R0 of the oil pipeline during the oil pipeline laying process. If it is greater than R0, it meets the safety operation requirements; if it is less than R0, it is necessary to prompt the operating vessel to adjust the operation mode and return to step 1 for re-monitoring.

2. The feature pre-fusion ARV underwater oil pipe curvature detection method according to claim 1, characterized in that: The Mixup image operation in step 2 is: after randomly splicing, saturation conversion, contrast enhancement and other operations on two images each time, different coefficients are used to multiply the two images respectively and then superimpose them to obtain a new image.

3. The feature pre-fusion ARV underwater oil pipe curvature detection method according to claim 1, characterized in that: The pre-fusion module in step 3 is a newly added module in the Backbones of the baseline RTMDet network model. e Between Backbon and Neck, the network adds three pre-fusion modules from shallow to deep. Each pre-fusion module combines Backbon e The network feature maps of three different stages are taken as input, and a feature map output is obtained after processing. Finally, the three feature maps of different sizes and depths generated by the three pre-fusion modules are taken as the input of the next stage Neck for further processing.

4. The feature pre-fusion ARV underwater oil pipe curvature detection method according to claim 3 is characterized in that: The pre-fusion module first converts i-1 th Feature map, i th Feature map and i+1 th The feature map is used as input, for i th The feature map is upsampled to the size of C, 4H, 4W by deconvolution operation and combined with i-1 th The feature maps are added, and the result is transformed nonlinearly using DW convolution. In the depthwise separable convolution, the dilated convolution kernel with a dilation factor of 2 is used, so that the feature map can obtain a larger receptive field during nonlinear transformation. At the same time, the use of depthwise separable convolution can effectively reduce model parameters and reduce the additional overhead in the model operation process. The feature map output by the depthwise separable convolution is then reduced to the size of 2C, 2H, and 2W through the Down-Focus module designed above. Similarly, for i+1 with a size of 4C, H, and W, th We use depth-separable nonlinear convolution and UP-Focus operations to enlarge the feature map to 2C, 2H, and 2W, and then use the above processed i-1 th Feature map, i+1 th Feature map and i th The feature maps are stacked and finally transformed back to i through a depth-wise separable convolution operation. th The size of the feature map is output.

5. The feature pre-fusion ARV underwater oil pipe curvature detection method according to claim 1 is characterized in that: The binarization process in step 4 refers to setting the pixel values ​​of the masked part to 255 and the values ​​of the non-masked part to 0 in the oil pipeline mask image obtained by network prediction, so as to obtain a black and white image with a white oil pipeline segmentation outline.

6. The feature pre-fusion ARV underwater oil pipe curvature detection method according to claim 1, characterized in that: The fitting curve in step 6 refers to polynomial least squares fitting, which constructs a polynomial model and obtains a least squares model by the sum of the squares of the differences between the y coordinates of each discrete point, and obtains a matrix solution of the model parameters based on the partial derivative of each model parameter at the extreme point being 0; The fitting curve equation is, y=a0+a1x+a2x 2 +...+a k x k =XA in, x and y represent the coordinate variables of the fitting curve, a0, a1, ..., a k is the coefficient of the polynomial fitting curve, k is the polynomial order, (x i ,y i ), i = 1, 2, ..., n are the coordinates of the boundary points of the oil pipe obtained through the above steps. The radius of curvature R is expressed as: in: where y′ and y″ represent the first and second order derivatives of the polynomial curve, respectively. Substitute the above formula into the radius of curvature and solve the coordinate point (x i ,y i ), i = 1, 2, ..., n represents the radius of curvature of the tubing boundary, as follows:

7. A computer device / equipment / system comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer program product comprising a computer program / instructions, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images