Methods, devices and related products for real-time detection and positioning of submarine pipelines

Abstract

This application relates to the field of computer technology and discloses a method, device, and related products for real-time detection and positioning of subsea pipelines. The method includes: using a trained deep learning subsea pipeline detection model to detect underwater images to be inspected, obtaining subsea pipeline detection results; using a trained deep learning operational device detection model to detect underwater images to be inspected, obtaining operational device detection results; generating a set of subsea pipeline contour points from the underwater images to be inspected based on the subsea pipeline detection results and the operational device detection results; fitting the points in the subsea pipeline contour point set to obtain a straight line model; and calculating the positioning data of the subsea pipeline based on the straight line model. This application can generate more robust, more accurate, and directly controllable underwater subsea pipeline positioning data, ensuring stable and reusable positioning data in both the unmanned underwater vehicle search and operational device release phases, thereby significantly improving the efficiency and success rate of close-range operations.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a method, apparatus, and related products for real-time detection and positioning of submarine pipelines. Background Technology

[0002] In recent years, with the rapid development of marine engineering and intelligent equipment, the scale of laying linear infrastructure such as submarine cables, pipelines, and moorings has been continuously expanding, posing a severe challenge to their long-term safe and stable operation in the complex marine environment. Displacement, exposure, or damage to these submarine pipelines will directly affect energy transmission, communication security, and ecological safety. Therefore, achieving real-time and accurate detection and positioning of submarine pipelines is a primary prerequisite for conducting inspections, maintenance, and emergency repairs, and a core requirement for improving the intelligence level of marine engineering operation and maintenance and reducing the risks of human intervention.

[0003] In related technologies, image processing techniques based on edge or line detection are generally used to extract gradient features from the image and use Hough transform or its improved algorithm to identify line segment structures, thereby inferring the location and shape of the submarine pipeline, and thus realizing the detection and positioning of the submarine pipeline.

[0004] However, in real underwater operating environments, submarine pipelines often exhibit distortion and obstruction in images, preventing them from appearing as clear, continuous, and nearly straight lines. This results in low accuracy in the detection and positioning of submarine pipelines. Therefore, it is urgent to solve this technical problem. Summary of the Invention

[0005] In view of the above situation, this application provides a method, device and related products for real-time detection and positioning of submarine pipelines, which aims to solve the above problems or at least partially solve the above problems.

[0006] In a first aspect, embodiments of this application provide a method for real-time detection and positioning of subsea pipelines. The method is implemented based on a real-time subsea pipeline detection and positioning system, which includes: an image acquisition device, a target detection board, and an intelligent planning board. The method includes: Acquire underwater images to be detected; A trained deep learning subsea pipeline detection model is used to detect the underwater image to be detected, and the subsea pipeline detection results are obtained; the subsea pipeline detection results include at least a subsea pipeline mask; The underwater image to be detected is detected using a trained deep learning detection model for the operating device, and the detection result of the operating device is obtained. Based on the detection results of the subsea pipeline and the detection results of the working device, a set of subsea pipeline contour points is generated for the underwater image to be detected. A straight line model is obtained by fitting the points in the set of points representing the submarine pipeline outline. Based on the linear model, the positioning data of the subsea pipeline is calculated; the positioning data includes at least: the slope and intercept of the linear model, the orientation angle of the subsea pipeline, and the normal offset data of the subsea pipeline relative to the imaging center of the underwater image to be detected.

[0007] Secondly, embodiments of this application also provide a real-time detection and positioning device for submarine pipelines. The device is based on a real-time detection and positioning system for submarine pipelines, the system comprising: an image acquisition device, a target detection board, and an intelligent planning board; the device comprises: The acquisition module is used to acquire underwater images to be detected. The subsea pipeline detection module is used to detect the underwater image to be detected using a trained deep learning subsea pipeline detection model, and obtain the subsea pipeline detection result; the subsea pipeline detection result includes at least a subsea pipeline mask. The operation device detection module is used to detect the underwater image to be detected using a trained deep learning operation device detection model, and obtain the operation device detection result. The generation module is used to generate a set of submarine pipeline contour points for the underwater image to be detected based on the detection results of the submarine pipeline and the detection results of the working device. The fitting module is used to fit the points in the set of points on the outline of the subsea pipeline to obtain a straight line model; The positioning module is used to calculate the positioning data of the subsea pipeline based on the straight line model; the positioning data includes at least: the slope and intercept of the straight line model, the orientation angle of the subsea pipeline, and the normal offset data of the subsea pipeline relative to the imaging center of the underwater image to be detected.

[0008] Thirdly, embodiments of this application also provide a real-time detection and positioning system for submarine pipelines, the system comprising: Image acquisition equipment used to acquire underwater images; The target detection board is used to perform the steps of the real-time detection and positioning method for subsea pipelines as described in the first aspect, and to generate positioning data for the subsea pipelines. The intelligent planning board is used to generate control commands based on the positioning data sent by the target detection board and send them to the controlled object, which is an unmanned underwater vehicle or a working device.

[0009] Fourthly, embodiments of this application also provide an electronic device, including: a processor; and a memory arranged to store computer-executable instructions, which, when executed, cause the processor to perform the steps of the above-described real-time detection and positioning method for submarine pipelines.

[0010] Fifthly, embodiments of this application also provide a computer-readable storage medium storing one or more programs, which, when executed by an electronic device including multiple applications, cause the electronic device to perform the steps of the above-described real-time detection and positioning method for submarine pipelines.

[0011] Using the above technical solutions, the real-time detection and positioning method, device, and related products for subsea pipelines provided in this application first acquire images of the pipeline to be detected using an underwater optical camera, and then identify the pipeline using a trained deep learning subsea pipeline detection model. This model is based on data-driven feature abstraction capabilities and does not rely on the fragile assumption that the target must present a clear and continuous straight line. It can effectively overcome problems such as underwater imaging distortion and complex background interference, and output a subsea pipeline mask containing semantic information, fundamentally solving the core defects of traditional methods such as sensitivity to geometric shape and poor environmental adaptability. At the same time, the system uses a detection model for the operating device to perform parallel detection on the image to cope with the occlusion scene of the operating device during operation, ensuring that the target features can be stably extracted in both the search and disposal stages. Based on the detection results of the two models, the system generates a set of subsea pipeline contour points. Then, the contour points are geometrically fitted to obtain a straight line model, and finally, positioning data such as slope, intercept, azimuth angle, and normal offset are calculated. These parameters can be directly used as real-time guidance quantities for the unmanned underwater vehicle control system, realizing closed-loop operation of detection and guidance simultaneously. This embodiment can obtain more robust, more accurate and directly controllable underwater subsea pipeline positioning data, ensuring that stable and reusable positioning data can be obtained in both the search and disposal phases, thereby significantly improving the efficiency and success rate of close-range operations.

[0012] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0013] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 A schematic diagram of the structure of the real-time detection and positioning system for submarine pipelines provided in an embodiment of this application is shown; Figure 2 A flowchart illustrating the real-time detection and positioning method for submarine pipelines provided in an embodiment of this application is shown. Figure 3 This illustration shows a schematic diagram of the process for detecting and locating submarine pipelines when there is no obstruction from operating equipment, as provided in an embodiment of this application. Figure 4 This paper shows a schematic diagram of the structure of the real-time detection and positioning device for submarine pipelines provided in an embodiment of this application; Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0015] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0016] It should be noted that the terms "first," "second," etc., in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such use can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the term "comprising" and its variations should be interpreted as open-ended terms meaning "including but not limited to."

[0017] As previously described, existing technologies typically utilize image processing techniques based on edge or line detection. These techniques extract gradient features from images and employ Hough transform or its improved algorithms to identify line segment structures, thereby inferring the orientation and shape of subsea pipelines and achieving their detection and positioning. However, in real underwater operating environments, subsea pipelines often exhibit distortion or occlusion in images, preventing them from appearing as clear, continuous, and nearly ideal straight lines. This results in low accuracy in subsea pipeline detection and positioning. Therefore, this invention proposes a method, apparatus, and related products for real-time detection and positioning of subsea pipelines. Specific embodiments are described in detail below.

[0018] To facilitate understanding of this embodiment, a detailed description of the real-time detection and positioning method for subsea pipelines disclosed in this application embodiment will be provided first. The real-time detection and positioning method for subsea pipelines provided in this application embodiment is implemented based on a real-time detection and positioning system for subsea pipelines, such as... Figure 1 As shown, the real-time detection and positioning system 100 for submarine pipelines includes: Image acquisition device 101 is used to acquire underwater images; The target detection board 102 is used to execute the steps of the real-time detection and positioning method for submarine pipelines provided in any of the following embodiments, and generate positioning data of the submarine pipelines. The intelligent planning board 103 is used to generate control commands based on the positioning data sent by the target detection board and send them to the controlled object, which is an unmanned underwater vehicle or a working device.

[0019] In this embodiment, the image acquisition device is used to acquire real-time underwater image sequences. In practice, exemplarily, the image acquisition device is an underwater optical camera.

[0020] The target detection board is equipped with the real-time detection and localization method for subsea pipelines provided in this application embodiment, which is used to segment, detect, and geometrically fit subsea pipelines in underwater images to generate localization data for the subsea pipelines. In some possible implementations, the real-time detection and localization method for subsea pipelines can be implemented by the processor of the target detection board calling computer-readable instructions stored in the memory.

[0021] The intelligent planning board receives the results from the target detection board, i.e., the positioning data, and converts the structured positioning data into control commands for the unmanned underwater vehicle (UUV) / operation device. The intelligent planning board interfaces with the AUV / operation device controller, thereby controlling the UUV / operation device to adjust its course and position. The operation device refers to the terminal execution device mounted on the UUV, used to perform subsequent specific operations on the seabed pipeline. Its core function is to be released after the UUV completes target search and positioning, and then approach the target to carry out the operational task.

[0022] The image acquisition device, target detection board, and intelligent planning board exchange data using low-latency communication, which ensures the consistency and timeliness of the guidance quantity and the corresponding image frame.

[0023] Figure 2 This document illustrates a flowchart of a real-time detection and positioning method for submarine pipelines provided in an embodiment of this application. Figure 2 It can be seen that the embodiments of this application include at least steps S201-S206: S201: Acquire the underwater image to be detected; S202: Using a trained deep learning subsea pipeline detection model, the underwater image to be detected is analyzed to obtain subsea pipeline detection results; the subsea pipeline detection results include at least a subsea pipeline mask. S203: Using a trained deep learning detection model for the underwater image to be detected, the detection result of the underwater image is obtained. S204: Based on the detection results of the subsea pipeline and the detection results of the working device, generate a set of subsea pipeline contour points for the underwater image to be detected; S205: Fit the points in the set of points representing the submarine pipeline outline to obtain a straight line model; S206: Based on the straight line model, calculate the positioning data of the submarine pipeline; the positioning data includes at least: the slope and intercept of the straight line model, the orientation angle of the submarine pipeline, and the normal offset data of the submarine pipeline relative to the imaging center of the underwater image to be detected.

[0024] As can be seen, the real-time detection and positioning method for subsea pipelines provided in this embodiment first acquires the image to be detected using an underwater optical camera, and then uses a trained deep learning subsea pipeline detection model for identification. This model is based on data-driven feature abstraction capabilities and does not rely on the fragile assumption that the target must present a clear and continuous straight line. It can effectively overcome problems such as underwater imaging distortion and complex background interference, and outputs a subsea pipeline mask containing semantic information, fundamentally solving the core defects of traditional methods such as sensitivity to geometric shape and poor environmental adaptability. At the same time, the image is detected in parallel using a detection model for the operating device to cope with the occlusion scene of the operating device during operation, ensuring that the target features can be stably extracted in both the search and disposal stages. Based on the detection results of the two models, the system generates a set of subsea pipeline contour points. Then, the contour points are geometrically fitted to obtain a straight line model, and finally, positioning data such as slope, intercept, direction angle, and normal offset are calculated. These parameters can be directly used as real-time guidance quantities for the unmanned underwater vehicle control system to realize closed-loop operation of detection and guidance simultaneously. This embodiment can obtain more robust, more accurate and directly controllable underwater subsea pipeline positioning data, ensuring that stable and reusable positioning data can be obtained in both the search and disposal phases, thereby significantly improving the efficiency and success rate of close-range operations.

[0025] The following provides a detailed explanation of S201-S206.

[0026] Regarding the above S201-202: First, the implementing entity of this application receives the underwater image to be detected from the image acquisition device, i.e., the current frame image. Next, the trained deep learning subsea pipeline detection model is used to process the underwater image to be detected, and the subsea pipeline detection results are obtained.

[0027] In some embodiments, before using a trained deep learning subsea pipeline detection model to detect the underwater image to be detected and obtaining the subsea pipeline detection result, the method further includes: Obtain a subsea pipeline dataset; the samples in the subsea pipeline dataset are underwater images containing subsea pipelines, and the sample features include subsea pipeline bounding box data, category labels, and subsea pipeline contour point sets. Using the aforementioned submarine pipeline dataset, an initial deep learning object detection model is trained to obtain the trained deep learning submarine pipeline detection model.

[0028] In this embodiment, to obtain a trained deep learning subsea pipeline detection model, it is first necessary to acquire a subsea pipeline dataset. In practice, underwater scene image acquisition can begin, collecting raw underwater images containing various types of subsea pipelines. The acquired raw images are then preprocessed, with image quality improved and image specifications standardized through noise reduction, contrast enhancement, and size normalization to meet the input requirements of subsequent model training. Subsequently, the preprocessed images are manually and precisely labeled. First, the bounding box data corresponding to each subsea pipeline is labeled, then the category label for each subsea pipeline is determined. Next, the contour point set of the subsea pipeline is labeled point by point, ensuring that each image possesses complete sample features. Finally, all labeled sample underwater images are divided into training, validation, and test sets according to a preset ratio, and then integrated to form a subsea pipeline dataset. For example, raw underwater images of oil pipelines in underwater port areas can be collected. First, the images are processed by Gaussian denoising and histogram equalization. Then, the rectangular bounding box of the oil pipeline is marked, and the category label is marked as underwater oil pipeline. Then, the contour point set of the pipeline edge is marked sequentially along the direction of the oil pipeline. The marked image is used as a single sample. After batch completing the image collection and annotation of submarine pipelines such as underwater cables and fishing nets, the training set, validation set and test set are divided in a ratio of 7:2:1 to generate a submarine pipeline dataset.

[0029] For initial deep learning object detection models, a pre-trained YOLOv8 segmentation network can be chosen as a starting point. This model has been trained on general large-scale datasets and has strong basic capabilities in feature extraction and object detection. Specifically, YOLOv8-seg, a version that supports instance segmentation, can be used.

[0030] After obtaining the subsea pipeline dataset and determining the initial deep learning object detection model, loss functions, such as cross-entropy loss, can be used on the training set for iterative optimization to adjust model parameters and improve the detection and segmentation accuracy of subsea pipelines. At the same time, performance is monitored through the validation set to prevent overfitting. After training, the model is evaluated on an independent test set, and fine-tuned according to the results until it meets the real-time detection requirements. Finally, a trained deep learning subsea pipeline detection model is obtained, which can be deployed to an object detection board for actual scanning tasks.

[0031] This embodiment first constructs a submarine pipeline dataset containing sample underwater images, and the sample features cover submarine pipeline bounding box data, category labels, and submarine pipeline contour point sets. Then, the initial deep learning object detection model is trained using this submarine pipeline dataset, allowing the model to fully learn the feature patterns of underwater submarine pipelines during the training process, thereby optimizing the parameters and improving the performance of the initial model, and finally obtaining a well-trained deep learning submarine pipeline detection model.

[0032] Next, the trained model is used to detect submarine pipelines in the underwater image to be detected. By using the underwater submarine pipeline features learned by the model, the target in the image is identified and detected. Finally, accurate underwater submarine pipeline detection results are obtained.

[0033] Regarding the above S203: After acquiring the underwater image to be detected, a trained deep learning detection model for the underwater device is used to detect the underwater image and obtain the detection result of the device.

[0034] In some embodiments, before using a trained deep learning detection model to detect the underwater image to be detected and obtaining the detection result of the underwater device, the method further includes: Obtain the operational equipment dataset; the samples in the operational equipment dataset are underwater images containing operational equipment, and the sample features include operational equipment bounding box data and category labels; Using the aforementioned job device dataset, the initial deep learning job device detection model is trained to obtain a trained deep learning job device detection model.

[0035] In this embodiment, to obtain a trained deep learning detection model for a work device, it is first necessary to acquire a dataset of work devices. In practice, underwater images containing the work devices can be acquired first. Then, the acquired raw images are preprocessed to improve image quality and standardize image specifications through methods such as noise reduction, contrast enhancement, and size normalization, adapting to the input requirements of subsequent model training. Subsequently, the preprocessed images are manually labeled to create training samples. Specifically, underwater cameras can be used to capture images of the work devices under different postures, lighting conditions, and occlusion conditions in actual work scenarios or experimental pools. Then, annotation tools are used to mark the bounding box positions of the work devices in each image and assign them uniform category labels. Finally, all annotation information and the raw images are organized into a structured dataset. The dataset is then divided into training, validation, and test sets according to a preset ratio and integrated to form the work device dataset.

[0036] The initial deep learning task device detection model, such as the pre-trained YOLOv5 object detection network, can be used as the base model. This model has high real-time performance and accuracy in general object detection tasks, and its structure supports fast output of bounding boxes and category information.

[0037] After obtaining the dataset of work equipment and determining the initial deep learning model for detecting work equipment, the model can be trained using the dataset. The specific steps are as follows: load the pre-trained YOLOv5 model weights and initialize the network parameters using transfer learning; then iteratively optimize the model on the training set using a composite loss function that includes bounding box loss and classification loss, adjusting the model parameters to improve the accuracy and robustness of work equipment detection, while monitoring model performance on the validation set to prevent overfitting; after training, evaluate the model's precision and recall on an independent test set, and perform hyperparameter tuning or data augmentation based on the results; finally, deploy the model that meets the requirements of real-time performance and accuracy onto the object detection board for real-time detection of work equipment in actual operations.

[0038] During implementation, the segmentation threshold and non-maximum suppression strategy can be adaptively set according to scene noise and suspended particle density.

[0039] This embodiment first acquires a dataset of underwater work equipment, which uses sample underwater images containing work equipment as samples. It also sets two types of sample features: bounding box data and category labels for the work equipment, which can provide annotation information on the location and category features of the underwater work equipment for model training. Then, the dataset of underwater work equipment is used to train the initial deep learning work equipment detection model, allowing the model to fully learn the visual feature patterns of underwater work equipment during the training process, complete the iterative optimization of the model network parameters and performance improvement, and achieve targeted training and capability adaptation of the initial model.

[0040] Finally, by using the trained deep learning detection model for underwater equipment, accurate detection results can be obtained for the underwater images to be detected.

[0041] When an underwater pipeline is detected in an underwater image, the pipeline may be divided into two segments due to the obstruction caused by the equipment. In this case, the trained deep learning underwater pipeline detection model will perform real-time inference on the current frame image to obtain a preliminary segmentation result containing multiple instances. Then, for these multi-instance segmentation results, the data is first filtered according to confidence level, for example, the top two instances with the highest confidence level can be selected. The segmentation results of these two instances are then merged to finally generate an underwater pipeline mask.

[0042] Regarding the above S204: After obtaining the detection results of the subsea pipeline and the detection results of the operating equipment, a set of subsea pipeline contour points for the underwater image to be detected can be generated based on these two results.

[0043] It should be noted that the usage phase of this application embodiment covers both the unmanned underwater vehicle (UUV) search phase and the operation device release phase. During the UUV search phase, only seabed pipelines exist in the underwater image to be detected; during the operation device release phase, both seabed pipelines and the operation device exist in the underwater image to be detected.

[0044] In some embodiments, if the detection result of the working device indicates that there is no working device in the underwater image to be detected, then generating the set of submarine pipeline contour points in the underwater image to be detected based on the submarine pipeline detection result and the working device detection result includes: Based on the aforementioned subsea pipeline mask, an initial set of subsea pipeline outline points is generated; Calculate the distance data from each point in the initial set of subsea pipeline contour points to the imaging center of the underwater image to be detected; The distance data are sorted in ascending order, and the set of points corresponding to the first preset number of distance data is determined as the set of points representing the outline of the submarine pipeline.

[0045] Research has revealed that the refraction of the end window and the characteristics of the wide-angle lens in underwater optical cameras can cause distortion problems such as stretching and warping in the edge areas of the captured images. Based on this, this embodiment is proposed.

[0046] In this embodiment, all points with a pixel value of 1 are first selected from the subsea pipeline mask to form an initial subsea pipeline contour point set. This point set is a pixel-level representation of the subsea pipeline.

[0047] Next, the imaging center of the underwater image to be detected... For reference, calculate the distance data from each point in the initial subsea pipeline outline point set to the center, such as Euclidean distance. ,right After sorting, select the m points with the smallest distance, where, , This yields a center-priority set of submarine pipeline outline points. .

[0048] In practice, m can also be configured according to the field of view and the lens distortion level.

[0049] The calculation method for the submarine pipeline contour point set provided in this embodiment first generates an initial submarine pipeline contour point set based on the submarine pipeline mask. Considering that the initial submarine pipeline contour point set contains points with severe edge distortion and that end window refraction will also cause linear distortion, if the full point set is directly used for linear fitting, the fitting result will deviate from the actual direction of submarine pipelines such as submarine cables and pipelines, thereby affecting the positioning accuracy and guidance reliability of submarine pipelines. Since the imaging of the central region of the image is relatively stable, this embodiment calculates the distance data from each point in the initial submarine pipeline contour point set to the imaging center of the underwater image to be detected, establishing the spatial positional relationship between each contour point and the imaging center, providing a quantifiable judgment basis for subsequent contour point selection. Then, the distance data is sorted in ascending order, and the first preset number of points are selected to form the final submarine pipeline contour point set, which is a submarine pipeline contour point set that is closer to the actual direction.

[0050] In other embodiments, if the detection result of the working device indicates the presence of a working device in the underwater image to be detected, and the detection result of the working device includes at least the bounding box data of the working device, then generating the set of submarine pipeline contour points in the underwater image to be detected based on the detection result of the submarine pipeline and the detection result of the working device includes: Based on the aforementioned subsea pipeline mask, an initial set of subsea pipeline outline points is generated; Calculate the distance data from each point in the initial subsea pipeline outline point set to the center point of the working device boundary frame; The distance data are sorted in ascending order, and the set of points corresponding to the first preset number of distance data is determined as the set of points representing the outline of the submarine pipeline.

[0051] In this embodiment, firstly, all points with a pixel value of 1 are selected from the subsea pipeline mask to form an initial subsea pipeline outline point set. This point set is a pixel-level representation of the subsea pipeline.

[0052] This embodiment takes the center position of the working device as the reference point. Abstracting the model avoids interference from the distant contours of the working device on subsequent fitting of the subsea pipeline. Specifically, as a working device with physical volume, the structure of its non-operational areas (such as the shell far from the operation point or the distal end of the robotic arm not involved in the operation) will form contours in the image. These contours, unrelated to the subsea pipeline, are called "distant contours." If the algorithm attempts to identify the complete contour of the working device to distinguish the target from the working device, underwater imaging distortion, suspended particles, and other interference can easily lead to inaccurate contour recognition of the working device, causing these distant contours to be misjudged as subsea pipeline contour points and included in the fitting, resulting in fitting deviation. However, this embodiment uses the center position of the working device for abstraction, eliminating the need to distinguish between the working device contour and the subsea pipeline contour. The center of the working device is used as the reference benchmark for the core operation area. Subsequently, by calculating the distance from the initial contour points to the center and filtering nearby points, the effective area of ​​the subsea pipeline that is not obscured and has clear imaging is further locked, fundamentally eliminating the possibility of the distant contours of the working device being mistakenly included in the fitting, thereby avoiding its interference with the fitting.

[0053] Next, the distance data from each point in the initial subsea pipeline outline point set to the center of the operating device is calculated, such as the Euclidean distance. After sorting the distances, the m points with the smallest distances are selected. , This yields a center-priority set of submarine pipeline outline points. .

[0054] The method for generating the submarine pipeline contour point set provided in this embodiment first generates an initial submarine pipeline contour point set based on a submarine pipeline mask. Then, it calculates the distance data from each point in the initial contour point set to the center point of the bounding box of the working device, establishing a spatial relationship between the contour points and the center of the working device. By sorting the distance data in ascending order and selecting the first preset number of points, distant points affected by occlusion interference can be selectively removed, while retaining the visible area contour points near the working device that provide more stable imaging. Finally, this embodiment can obtain a submarine pipeline contour point set that is more resistant to occlusion interference and closely matches the actual route.

[0055] Regarding the above S205-206: After obtaining the point set of the submarine pipeline outline, this point set is then used in a straight line model. L2 fitting is performed to obtain the estimated parameters. and .

[0056] Then, the orientation angle of the subsea pipeline in the coordinate system of the image to be detected and its normal offset relative to the imaging center are calculated. .in, The calculation formula is:

[0057] The calculation formula is:

[0058] The four parameters obtained The data is continuously output to the intelligent planning board and converted into heading correction and lateral displacement commands for the unmanned underwater vehicle. During implementation, the positioning data can be uniformly encapsulated on the communication side so that the upper-level control system can directly calculate it into heading / attitude / position control quantities.

[0059] When the current frame is missing, the result of the most recent historical frame can be used to maintain the continuity of the guidance.

[0060] Figure 3 This illustration shows a schematic diagram of the subsea pipeline detection and positioning process when there is no obstruction from operating equipment, as provided in an embodiment of this application. See also... Figure 3 As shown, during implementation, the relevant detection information of the subsea pipeline is first identified in the underwater image to be detected using a deep learning model, namely the YOLOv8 model. Based on the detection results, invalid interference information is filtered out, and key feature points that can characterize the outline of the subsea pipeline are accurately extracted. Finally, least squares parametric regression can be performed, and linear fitting is performed using the extracted key points to obtain the linear fitting result, thereby obtaining a straight line model that can be used to calculate positioning data such as slope, intercept, and orientation angle of the subsea pipeline.

[0061] In some embodiments, if the detection result of the working device indicates that there is a working device in the underwater image to be detected, the positioning data also includes the normal offset data of the subsea pipeline relative to the center position of the working device.

[0062] In this embodiment, if a working device is present, the parameters The calculation method is the same as when there is no working device, and will not be repeated here. In addition, when there is a working device, the normal offset data of the subsea pipeline relative to the center position of the working device is calculated according to the following formula:

[0063] These five parameters are output to the intelligent planning board, providing directional and position guidance information for both the unmanned underwater vehicle and the operating device. This stage also supports historical rollback and lightweight timing smoothing to ensure that guidance is not jittered or lost during the release operation.

[0064] Those skilled in the art will understand that in the above-described method of the specific embodiments, the order in which the steps are written does not imply a strict execution order, but constitutes no limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0065] It should be noted that in practical applications, all the above-described possible implementation methods can be combined in any way to form possible embodiments of this application, and will not be described in detail here. The information (including but not limited to device information, user information, etc.) and data (including but not limited to data used for analysis, storage, and display) involved in this application are all information and data authorized by the user or fully authorized by all parties. The software tools or components appearing in the embodiments of this application are merely illustrative examples and do not represent actual use.

[0066] Based on the same concept, this application also provides a real-time detection and positioning device for submarine pipelines, which corresponds one-to-one with the real-time detection and positioning method for submarine pipelines in the above embodiments. Figure 4 A schematic diagram of the structure of the real-time detection and positioning device for submarine pipelines provided in an embodiment of this application is shown. See also: Figure 4 As shown, the real-time detection and positioning device 400 for subsea pipelines provided in this application embodiment is based on a real-time detection and positioning system for subsea pipelines. The system includes: an image acquisition device, a target detection board, and an intelligent planning board; the real-time detection and positioning device 400 for subsea pipelines includes: Acquisition module 401 is used to acquire underwater images to be detected; The subsea pipeline detection module 402 is used to detect the underwater image to be detected using a trained deep learning subsea pipeline detection model to obtain subsea pipeline detection results; the subsea pipeline detection results include at least a subsea pipeline mask. The operation device detection module 403 is used to detect the underwater image to be detected using a trained deep learning operation device detection model, and obtain the operation device detection result. The generation module 404 is used to generate a set of submarine pipeline contour points for the underwater image to be detected based on the submarine pipeline detection results and the detection results of the working device. The fitting module 405 is used to fit the points in the set of points on the outline of the submarine pipeline to obtain a straight line model. The positioning module 406 is used to calculate the positioning data of the submarine pipeline based on the straight line model; the positioning data includes at least: the slope and intercept of the straight line model, the orientation angle of the submarine pipeline, and the normal offset data of the submarine pipeline relative to the imaging center of the underwater image to be detected.

[0067] In some embodiments, the apparatus further includes a first training module for: Obtain a subsea pipeline dataset; the samples in the subsea pipeline dataset are underwater images containing subsea pipelines, and the sample features include subsea pipeline bounding box data, category labels, and subsea pipeline contour point sets. Using the aforementioned submarine pipeline dataset, an initial deep learning object detection model is trained to obtain the trained deep learning submarine pipeline detection model.

[0068] In some embodiments, the apparatus further includes a second training module for: Obtain the operational equipment dataset; the samples in the operational equipment dataset are underwater images containing operational equipment, and the sample features include operational equipment bounding box data and category labels; Using the aforementioned job device dataset, the initial deep learning job device detection model is trained to obtain a trained deep learning job device detection model.

[0069] In some embodiments, in the above-described apparatus, if the detection result of the working device indicates that the working device does not exist in the underwater image to be detected, then the generation module is used to: Based on the aforementioned subsea pipeline mask, an initial set of subsea pipeline outline points is generated; Calculate the distance data from each point in the initial set of subsea pipeline contour points to the imaging center of the underwater image to be detected; The distance data are sorted in ascending order, and the set of points corresponding to the first preset number of distance data is determined as the set of points representing the outline of the submarine pipeline.

[0070] In some embodiments, in the above-described apparatus, if the detection result of the working device indicates the presence of a working device in the underwater image to be detected, and the detection result of the working device includes at least the bounding box data of the working device, then the generation module is used to: Based on the aforementioned subsea pipeline mask, an initial set of subsea pipeline outline points is generated; Calculate the distance data from each point in the initial subsea pipeline outline point set to the center point of the working device boundary frame; The distance data are sorted in ascending order, and the set of points corresponding to the first preset number of distance data is determined as the set of points representing the outline of the submarine pipeline.

[0071] In some embodiments, in the above-described apparatus, if the detection result of the working device indicates that a working device exists in the underwater image to be detected, the positioning data also includes normal offset data of the subsea pipeline relative to the center position of the working device.

[0072] This invention provides a real-time detection and positioning device for subsea pipelines. First, an underwater optical camera acquires an image of the pipeline to be detected. Then, a trained deep learning subsea pipeline detection model is used for identification. This model, based on data-driven feature abstraction capabilities, does not rely on the fragile assumption that the target must present a clear, continuous straight line. It effectively overcomes problems such as underwater imaging distortion and complex background interference, outputting a subsea pipeline mask containing semantic information. This fundamentally solves the core shortcomings of traditional methods, such as sensitivity to geometric shapes and poor environmental adaptability. Simultaneously, a detection model for operational devices is used to perform parallel image detection to handle scenarios where operational devices obstruct the view, ensuring stable extraction of target features in both the search and handling stages. Based on the detection results of the two models, the system generates a set of subsea pipeline contour points. Subsequently, geometric fitting is performed on the contour points to obtain a straight line model. Finally, positioning data such as slope, intercept, azimuth angle, and normal offset are calculated. These parameters can be directly used as real-time guidance parameters for the unmanned underwater vehicle control system, realizing closed-loop operation of simultaneous detection and guidance. This embodiment can obtain more robust, more accurate and directly controllable underwater subsea pipeline positioning data, ensuring that stable and reusable positioning data can be obtained in both the search and disposal phases, thereby significantly improving the efficiency and success rate of close-range operations.

[0073] Specific limitations regarding the real-time detection and positioning device for subsea pipelines can be found in the limitations of the real-time detection and positioning method for subsea pipelines mentioned above, and will not be repeated here. Each module in the aforementioned real-time detection and positioning device for subsea pipelines can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0074] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Figure 5 As shown, at the hardware level, this electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or it may include non-volatile memory, such as at least one disk drive. Of course, this electronic device may also include other hardware required for other business operations.

[0075] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0076] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0077] The processor reads the corresponding computer program from non-volatile memory into main memory and then runs it, forming a real-time detection and positioning device for subsea pipelines at the logical level. The processor executes the program stored in memory and specifically performs the aforementioned methods.

[0078] The processor may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0079] This electronic device can execute the real-time detection and positioning method for submarine pipelines provided in several embodiments of this application, and realize a real-time detection and positioning device for submarine pipelines. Figure 4 The functions of the embodiments shown are not described in detail here.

[0080] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform the real-time detection and positioning method for submarine pipelines provided in several embodiments of this application.

[0081] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0082] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0083] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0084] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0085] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0086] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0087] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0088] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0089] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0090] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for real-time detection and positioning of subsea pipelines, characterized in that, The method is based on a real-time detection and positioning system for subsea pipelines, the system comprising: image acquisition equipment, target detection board, and intelligent planning board; the method comprises: Acquire underwater images to be detected; A trained deep learning subsea pipeline detection model is used to detect the underwater image to be detected, and the subsea pipeline detection results are obtained; the subsea pipeline detection results include at least a subsea pipeline mask; The underwater image to be detected is detected using a trained deep learning detection model for the operating device, and the detection result of the operating device is obtained. Based on the detection results of the subsea pipeline and the detection results of the working device, a set of subsea pipeline contour points is generated for the underwater image to be detected. A straight line model is obtained by fitting the points in the set of points representing the submarine pipeline outline. Based on the straight line model, the positioning data of the subsea pipeline is calculated; the positioning data includes at least: the slope and intercept of the straight line model, the orientation angle of the subsea pipeline, and the normal offset data of the subsea pipeline relative to the imaging center of the underwater image to be detected; If the detection result of the working device indicates the presence of a working device in the underwater image to be detected, and the detection result of the working device includes at least the bounding box data of the working device, then generating the set of submarine pipeline contour points in the underwater image to be detected based on the detection result of the submarine pipeline and the detection result of the working device includes: Based on the aforementioned subsea pipeline mask, an initial set of subsea pipeline outline points is generated; Calculate the distance data from each point in the initial subsea pipeline outline point set to the center point of the working device boundary frame; The distance data are sorted in ascending order, and the set of points corresponding to the first preset number of distance data is determined as the set of points representing the outline of the submarine pipeline.

2. The method according to claim 1, characterized in that, Before using a trained deep learning subsea pipeline detection model to detect the underwater image to be detected and obtaining the subsea pipeline detection result, the method further includes: Obtain a subsea pipeline dataset; the samples in the subsea pipeline dataset are underwater images containing subsea pipelines, and the sample features include subsea pipeline bounding box data, category labels, and subsea pipeline contour point sets. Using the aforementioned submarine pipeline dataset, an initial deep learning object detection model is trained to obtain the trained deep learning submarine pipeline detection model.

3. The method according to claim 1, characterized in that, Before using a trained deep learning-based detection model to detect the underwater image to be detected and obtaining the detection result of the underwater equipment, the method further includes: Obtain the operational equipment dataset; the samples in the operational equipment dataset are underwater images containing operational equipment, and the sample features include operational equipment bounding box data and category labels; Using the aforementioned job device dataset, the initial deep learning job device detection model is trained to obtain a trained deep learning job device detection model.

4. The method according to claim 1, characterized in that, If the detection result of the working device indicates that there is no working device in the underwater image to be detected, then the step of generating a set of submarine pipeline contour points in the underwater image to be detected based on the detection result of the submarine pipeline and the detection result of the working device includes: Based on the aforementioned subsea pipeline mask, an initial set of subsea pipeline outline points is generated; Calculate the distance data from each point in the initial set of subsea pipeline contour points to the imaging center of the underwater image to be detected; The distance data are sorted in ascending order, and the set of points corresponding to the first preset number of distance data is determined as the set of points representing the outline of the submarine pipeline.

5. The method according to claim 1, characterized in that, The positioning data also includes the normal offset data of the subsea pipeline relative to the center position of the working device.

6. A real-time detection and positioning device for submarine pipelines, characterized in that, The device is based on a real-time detection and positioning system for subsea pipelines, which includes: an image acquisition device, a target detection board, and an intelligent planning board; the device includes: The acquisition module is used to acquire underwater images to be detected. The subsea pipeline detection module is used to detect the underwater image to be detected using a trained deep learning subsea pipeline detection model, and obtain the subsea pipeline detection result; the subsea pipeline detection result includes at least a subsea pipeline mask. The operation device detection module is used to detect the underwater image to be detected using a trained deep learning operation device detection model, and obtain the operation device detection result. The generation module is used to generate a set of submarine pipeline contour points for the underwater image to be detected based on the detection results of the submarine pipeline and the detection results of the working device. The fitting module is used to fit the points in the set of points on the outline of the subsea pipeline to obtain a straight line model; The positioning module is used to calculate the positioning data of the subsea pipeline based on the straight line model; the positioning data includes at least: the slope and intercept of the straight line model, the orientation angle of the subsea pipeline, and the normal offset data of the subsea pipeline relative to the imaging center of the underwater image to be detected; If the detection result of the working device indicates that a working device exists in the underwater image to be detected, and the detection result of the working device includes at least the bounding box data of the working device, then the generation module is used to: Based on the aforementioned subsea pipeline mask, an initial set of subsea pipeline outline points is generated; Calculate the distance data from each point in the initial subsea pipeline outline point set to the center point of the working device boundary frame; The distance data are sorted in ascending order, and the set of points corresponding to the first preset number of distance data is determined as the set of points representing the outline of the submarine pipeline.

7. A real-time detection and positioning system for submarine pipelines, characterized in that, The system includes: Image acquisition equipment used to acquire underwater images; The target detection board is used to perform the steps of the real-time detection and positioning method for submarine pipelines as described in any one of claims 1-5, and to generate positioning data for the submarine pipelines. The intelligent planning board is used to generate control commands based on the positioning data sent by the target detection board and send them to the controlled object, which is an unmanned underwater vehicle or a working device.

8. An electronic device, comprising: processor; as well as A memory configured to store computer-executable instructions, characterized in that, when executed, the executable instructions cause the processor to perform the steps of the real-time detection and positioning method for submarine pipelines as described in any one of claims 1-5.

9. A computer-readable storage medium storing one or more programs, characterized in that, When the one or more programs are executed by an electronic device including multiple applications, the electronic device performs the steps of the real-time detection and positioning method for submarine pipelines as described in any one of claims 1-5.

Images