Submarine pipeline side scan sonar detection method and device and electronic equipment

Abstract

The application relates to a submarine pipeline side-scan sonar detection method and device and electronic equipment, and relates to the target detection technical field of a multifunctional underwater robot, which comprises the following steps: inputting a side-scan sonar image into a deep learning model for feature extraction, determining a pipeline prediction result and a pipeline prediction confidence according to a predicted feature tensor, identifying a pipeline target in the side-scan sonar image, carrying out informatization processing on the pipeline target, and generating a pipeline information dictionary; carrying out post-processing optimization on the pipeline information dictionary, carrying out coordinate conversion on pixel-level pipeline target image coordinates in the pipeline information dictionary after the post-processing optimization in combination with real-time navigation data, and calculating longitude and latitude coordinates of the pipeline target, so that the application can improve the automation level of detection, effectively eliminate misdetected targets that do not conform to pipeline characteristics, reduce the false alarm rate, and capture weak targets or subtle features of buried pipelines from complex sonar images.

Description

Technical Field

[0001] This application relates to the field of target detection technology for multifunctional underwater robots, specifically to a side-scan sonar detection method, device, and electronic equipment for submarine pipelines. Background Technology

[0002] Submarine pipelines, including oil pipelines, communication cables, and monitoring cables, are important targets in underwater environments. The safe and stable operation of these pipelines is not only directly related to economic development and environmental protection, but also a critical infrastructure for maintaining global energy and information interconnectivity. In recent years, underwater competition targeting submarine pipelines has intensified, with frequent incidents of sabotage and reconnaissance against them.

[0003] Currently, submarine pipeline target detection is mainly achieved through equipment such as side-scan sonar, optical cameras, magnetometers, and lidar. However, although these existing technologies can acquire relatively rich information under specific conditions, they still have significant limitations in practical applications. For example, optical cameras are limited by water turbidity and lighting conditions, and their imaging quality drops sharply in deep sea or poor water quality environments, with a short effective detection range; magnetometers, while sensitive to ferromagnetic targets, are easily affected by changes in the geomagnetic field and magnetic interference from seabed rocks, and have difficulty accurately locating pipelines made of non-magnetic materials (such as communication optical cables); lidar, although capable of building high-precision 3D models, is expensive and its underwater detection range is also limited, making it difficult to meet the needs of large-scale rapid scanning.

[0004] In contrast, side-scan sonar has the advantages of high accuracy, low power consumption, and low cost, and is widely recognized as an effective means for detecting large-scale submarine pipeline targets. However, existing side-scan sonar detection technologies usually rely on manual interpretation or traditional image processing algorithms. When facing complex seabed environments (such as harsh sea conditions, complex seabed sediments, and strong interference noise), they often suffer from low automation, high false alarm rates, and insufficient ability to identify weak targets or buried pipelines, making it difficult to meet the current urgent need for efficient and intelligent detection of submarine pipelines. Summary of the Invention

[0005] In view of this, this application provides a side-scan sonar detection method, device and electronic equipment for submarine pipelines. The main purpose is to solve the problems that existing side-scan sonar detection technologies often suffer from low automation, high false alarm rate and insufficient ability to identify weak targets or buried pipelines when facing complex seabed environments, which makes it difficult to meet the current urgent need for efficient and intelligent detection of submarine pipelines.

[0006] In a first aspect, this application provides a side-scan sonar detection method for subsea pipelines, including: Acquire side-scan sonar images and real-time navigation data of the underwater robot, wherein the real-time navigation data includes at least altitude, heading angle, coordinates and side-scan sonar image resolution; The side-scan sonar image is input into a deep learning model to extract features and obtain a prediction feature tensor. The pipeline prediction result and pipeline prediction confidence are then determined based on the prediction feature tensor. Based on the pipeline prediction results and / or the pipeline prediction confidence, pipeline targets in the side-scan sonar image are identified, and the pipeline targets are processed by information based on the solution information of the target positions of the pipeline targets in the image coordinate system to generate a pipeline information dictionary, which is a dictionary containing pixel-level pipeline target image coordinates. The pipeline information dictionary is post-processed and optimized using the prior information of the pipeline target. The pixel-level pipeline target image coordinates in the post-processed and optimized pipeline information dictionary are then combined with the real-time navigation data to perform coordinate transformation, thereby calculating the longitude and latitude coordinates of the pipeline target.

[0007] Secondly, this application provides a side-scan sonar detection device for subsea pipelines, comprising: The acquisition module is used to acquire side-scan sonar images and real-time navigation data of the underwater robot, wherein the real-time navigation data includes at least altitude, heading angle, coordinates and side-scan sonar image resolution; The extraction module is used to input the side-scan sonar image into a deep learning model to extract features and obtain a predicted feature tensor, and to determine the pipeline prediction result and pipeline prediction confidence based on the predicted feature tensor. The generation module is used to identify pipeline targets in the side-scan sonar image based on the pipeline prediction results and / or the pipeline prediction confidence, and to perform information processing on the pipeline targets based on the solution information of the target positions of the pipeline targets in the image coordinate system to generate a pipeline information dictionary, wherein the pipeline information dictionary is a dictionary containing pixel-level pipeline target image coordinates; The calculation module is used to perform post-processing optimization on the pipeline information dictionary using the prior information of the pipeline target, and to combine the pixel-level pipeline target image coordinates in the post-processed pipeline information dictionary with the real-time navigation data to perform coordinate transformation and calculate the longitude and latitude coordinates of the pipeline target.

[0008] Thirdly, this application provides an electronic device, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the computer program to implement the subsea pipeline side-scan sonar detection method described in the first aspect.

[0009] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the subsea pipeline side-scan sonar detection method described in the first aspect.

[0010] By employing the above technical solution, this application provides a method, apparatus, and electronic device for detecting subsea pipelines using side-scan sonar. Compared with existing technologies, this application can acquire side-scan sonar images and real-time navigation data of an underwater robot. The real-time navigation data includes at least altitude, heading angle, coordinates, and side-scan sonar image resolution. The side-scan sonar images are input into a deep learning model for feature extraction to obtain a predicted feature tensor. Based on the predicted feature tensor, pipeline prediction results and pipeline prediction confidence are determined. Based on the pipeline prediction results and / or pipeline prediction confidence, pipeline targets in the side-scan sonar images are identified. The pipeline targets are then processed using information about their positions in the image coordinate system to generate a pipeline information dictionary containing pixel-level pipeline target image coordinates. Prior information about the pipeline targets is used to post-process and optimize the pipeline information dictionary. The pixel-level pipeline target image coordinates in the post-processed and optimized pipeline information dictionary are then combined with real-time navigation data for coordinate transformation to calculate the longitude and latitude coordinates of the pipeline targets.

[0011] By employing the above technical solution, this application extracts features from a deep learning model by inputting side-scan sonar images, obtains predicted feature tensors, and then determines the pipeline prediction results and pipeline prediction confidence. The deep learning model can automatically learn the feature patterns of subsea pipelines in sonar images, eliminating the need for manual setting of complex feature extraction rules, reducing manual intervention, and greatly improving the automation level of detection.

[0012] From acquiring side-scan sonar images and real-time navigation data of underwater robots to finally calculating the longitude and latitude coordinates of the pipeline target, the entire process covers multiple stages such as feature extraction, target recognition, information processing, post-processing optimization, and coordinate transformation, forming a complete automated detection process that improves detection efficiency.

[0013] The generated pipeline information dictionary is post-processed and optimized using prior information about pipeline targets. Prior information can include knowledge of common pipeline shapes, routes, and size ranges. By combining this prior information with the detection results, false detection targets that do not conform to pipeline characteristics can be effectively eliminated, thereby reducing the false alarm rate and improving the accuracy of the detection results.

[0014] Deep learning models possess powerful feature learning and representation capabilities, enabling them to capture subtle features of faint targets or buried pipelines from complex sonar images. By training on a large number of sonar images containing pipelines in different states, the model can learn the essential characteristics of these targets, accurately identifying pipeline targets even in situations with poor image quality and weak target signals.

[0015] This application not only utilizes side-scan sonar images but also incorporates real-time navigation data from underwater robots, such as altitude, heading angle, coordinates, and side-scan sonar image resolution. The fusion of this multi-source data provides the model with richer information, helping it to better understand image background and target features, further improving its ability to identify faint targets or buried pipelines, and meeting the urgent need for efficient and intelligent detection of subsea pipelines.

[0016] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A schematic flowchart of a side-scan sonar detection method for subsea pipelines provided in this application embodiment; Figure 2 A hardware system diagram of a side-scan sonar detection method for subsea pipelines provided in this application embodiment; Figure 3 A schematic flowchart of another side-scan sonar detection method for subsea pipelines provided in this application embodiment; Figure 4 A deep learning network structure diagram based on the Transformer deep neural network model is provided for embodiments of this application; Figure 5 This application provides an example of a diagram showing the wire breakage before merging. Figure 6 This application provides an example of a wire breakage merging diagram; Figure 7 An image before short line removal provided in an embodiment of this application; Figure 8 An image showing the result after short line removal, provided as an embodiment of this application; Figure 9 A schematic diagram illustrating the principle of sonar image coordinate transformation to geodetic coordinates provided in this application embodiment; Figure 10 This is a schematic diagram of the structure of a side-scan sonar detection device for a subsea pipeline provided in an embodiment of this application. Detailed Implementation

[0020] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0021] The following description, with reference to the accompanying drawings, describes a method, apparatus, and electronic device for detecting subsea pipelines using side-scan sonar according to embodiments of this application.

[0022] This application provides a side-scan sonar detection method, device, and electronic equipment for submarine pipelines. The main purpose is to solve the problems that existing side-scan sonar detection technologies often suffer from low automation, high false alarm rate, and insufficient ability to identify weak targets or buried pipelines when facing complex seabed environments, making it difficult to meet the current urgent technical needs for efficient and intelligent detection of submarine pipelines.

[0023] like Figure 1 As shown, an embodiment of this application provides a side-scan sonar detection method for subsea pipelines, including: Step 101: Acquire side-scan sonar images and real-time navigation data of the underwater robot. The real-time navigation data includes at least altitude, heading angle, coordinates, and side-scan sonar image resolution.

[0024] This embodiment is applied to the detection of subsea pipelines. An underwater robot (AUV) equipped with side-scan sonar operates in the seabed area according to the preset task stages (surface cruising stage, diving stage, underwater rough operation stage, underwater fine operation stage, and surfacing stage), and obtains subsea pipeline information in real time through the detection method of this application.

[0025] like Figure 2As shown, the hardware system of a side-scan sonar detection method for subsea pipelines provided in this embodiment consists of an intelligent planning board, a target detection board, a side-scan sonar (i.e., side-scan sonar, SSS), a switch, and an inertial navigation unit (IMU). Figure 2 (not marked in the text), altimeter ( Figure 2 (not marked in the text) and Global Positioning System (i.e., GPS sensors, Figure 2 (Not marked in the text)

[0026] like Figure 3 As shown, the target detection board has functions such as acoustic image preprocessing, deep learning feature extraction, sonar information conversion, post-processing, and target coordinate transformation.

[0027] Among them, the acoustic image preprocessing function is used to denoise, normalize, and equalize sonar data; The deep learning feature extraction function is used to extract sonar image features from preprocessed sonar data to achieve pixel-level line target detection; The sonar information conversion function is used to process the graphical detection results into information. The post-processing function is used to filter targets based on prior information, including removing seabed lines, connecting broken lines, and merging short lines; The target coordinate transformation function is used to transform the target from the image coordinate system to the geodetic coordinate system.

[0028] The workflow can be as follows: During the surface phase, the GPS sensor acquires absolute position information; during the descent phase, the working status of the side-scan sonar system is tested; the working status of the AUV and various sensors is checked; during the rough operation phase, acoustic image acquisition and data acquisition from various sensors are performed; linear target identification and positioning are conducted; during the fine operation phase, acoustic image acquisition and linear target tracking and evidence collection are performed; during the ascent phase, the GPS is recalibrated and data is stored.

[0029] like Figure 2 As shown, the side-scan sonar sends its images to the intelligent planning board via TCP protocol. The intelligent planning board then sends the side-scan sonar images (i.e., side-scan sonar images), along with real-time navigation data such as the altitude, heading, coordinates, and side-scan sonar image resolution of the acquired autonomous underwater vehicle (AUV), to the target detection board. The target detection board, with a computing power of 40 TOPS, 8GB of video memory, a power consumption of 7W-15W, and a size of 69.6mm × 45mm, is responsible for online detection, information processing, post-processing, and target coordinate transformation of the side-scan sonar images. It also sends the pipeline's coordinates, orientation, and pixel-level confidence level back to the intelligent planning board. The side-scan sonar, target detection board, and intelligent planning board communicate via a switch.

[0030] During the underwater robot's mission, side-scan sonar continuously acquires side-scan sonar images of the seabed area. Simultaneously, the intelligent planning board acquires real-time navigation data such as the underwater robot's altitude, heading angle, coordinates, and side-scan sonar image resolution. For example, during the precision underwater operation phase, the underwater robot navigates the target area at a low speed, and the side-scan sonar acquires side-scan sonar images at a certain frequency (e.g., 10 frames per second). The intelligent planning board simultaneously acquires the underwater robot's real-time navigation data, including an altitude of 50 meters, a heading angle of 30 degrees, coordinates (longitude 120.12345, latitude 30.54321), and a side-scan sonar image resolution of 0.01 meters per pixel.

[0031] Step 102: Input the side-scan sonar image into the deep learning model to extract features and obtain the predicted feature tensor. Then, determine the pipeline prediction result and pipeline prediction confidence based on the predicted feature tensor.

[0032] In this embodiment, the deep learning model can be the Transformer deep neural network model. The Transformer model has advantages such as high accuracy and good description of long-range features, and can effectively extract multi-scale, long-range feature information from sonar images.

[0033] Specifically, the side-scan sonar images are input into a deep learning model for feature extraction to obtain a predicted feature tensor, and the pipeline prediction result and pipeline prediction confidence are determined based on the predicted feature tensor, which may include: Side-scan sonar images are input into a deep learning model for feature extraction to obtain feature information at different scales. The feature information at different scales is then fused within the same scale to obtain multi-scale pyramid features. Decode the multi-scale pyramid features to generate a prediction feature tensor; The pipeline prediction results are obtained by performing a maximum index calculation on the prediction feature tensor. The prediction feature tensor is normalized to obtain the pipeline prediction confidence.

[0034] In this embodiment, as Figure 4As shown, this application proposes to use the Transformer model to achieve pipeline target detection based on side-scan sonar. During network training, weights pre-trained on the ImageNet dataset are first loaded, and then fine-tuned using precisely labeled sonar images containing pipelines to obtain weights suitable for pipeline target detection in sonar images. During inference, the side-scan sonar image to be detected can be input into the deep learning feature extraction network of the Transformer model. The Transformer module is used to extract feature information at different scales. For example, three feature maps of different scales are extracted, with sizes of [64, 64], [32, 32], and [16, 16], respectively. Downsampling is achieved through image patch merging to make the feature maps of different scales have a uniform size for subsequent processing.

[0035] Next, the pyramid pooling module can be used to achieve feature fusion within the same scale, and the features of different scales output by pyramid pooling can be fused to construct multi-scale pyramid features.

[0036] Then, the multi-scale pyramid features are decoded to generate the final prediction feature tensor.

[0037] Finally, the maximum value indexing operation (argmax operation) is performed on the predicted feature tensor to obtain the pipeline prediction result; at the same time, the predicted feature tensor is normalized (Softmax) to obtain the pipeline prediction confidence.

[0038] In this embodiment, the prediction feature tensor can be a tensor of size [2, H, W], where H and W are the height and width of the side-scan sonar image, respectively. Assume H = 256 and W = 512. Regarding the representation of different target categories, this technical solution targets both pipeline targets and background. Therefore, the number of channels in the prediction feature tensor can be set to 2. A channel count of 0 indicates a confidence level of predicting the target as background, while a channel count of 1 indicates a confidence level of predicting the target as a pipeline. The pipeline prediction result can be a mask image of size [H, W], consisting of predictions composed of 0s (representing background) and 1s (representing pipelines).

[0039] Step 103: Based on the pipeline prediction results and / or pipeline prediction confidence, identify pipeline targets in the side-scan sonar image, and perform information processing on the pipeline targets based on the solution information of the target positions in the image coordinate system to generate a pipeline information dictionary, which is a dictionary containing pixel-level pipeline target image coordinates.

[0040] The pipeline information is generated by the information conversion module. Since the deep neural network outputs a recognition mask for pipeline categories, it cannot directly distinguish different pipelines in the image (e.g., multiple pipelines arranged in parallel). Therefore, it is necessary to perform information processing on the pipeline targets in the side-scan sonar images. The digitized pipeline information is stored in the form of a dictionary. The pipeline information dictionary contains pixel-level pipeline target image coordinates for convenient subsequent tracking.

[0041] Specifically, the steps of information processing are as follows: Based on the pipeline prediction results and / or pipeline prediction confidence, the pipeline information for the current detection period (current ping) and the pipeline information for the previous detection period (previous ping) of the side-scan sonar image can be recorded. The pipeline information for both the current and previous detection periods can include: the left edge point of the pipeline (cable_left), the right edge point of the pipeline (cable_right), and the pipeline width (cable_width); for example, the current ping's cable_left is (100, 200), cable_right is (150, 200), and cable_width is 50; the previous ping's cable_left is (95, 200), cable_right is (145, 200), and cable_width is 50.

[0042] The detection period corresponds to one scan line in the sonar image; The left edge point of the cable (cable_left) is the location in the side-scan sonar image where the pixel value changes from 0 to 1. The right edge point of the cable (cable_right) is the location in the side-scan sonar image where the pixel value changes from 1 to 0. Cable width (cable_width): calculated as cable_right - cable_left.

[0043] The pipeline information of the current detection cycle can be compared with the pipeline information of the previous detection cycle. The preset horizontal threshold (hori_treshhold) can be set to 10 pixels and the preset width threshold is 5 pixels.

[0044] The position difference between cable_left in the current detection cycle and cable_left in the previous detection cycle can be calculated. If the position difference is less than a preset horizontal threshold, it is determined that the target pipeline in the current detection cycle and the target pipeline in the previous detection cycle belong to the same pipeline.

[0045] Furthermore, during the comparison, in addition to comparing the left edge point, the comparison of the right edge point and the pipeline width can be further combined. That is, if the position difference between the right edge point of the pipeline in the current detection cycle and the right edge point of the pipeline in the previous detection cycle is less than a preset horizontal threshold, and / or the width difference between the pipeline width in the current detection cycle and the pipeline width in the previous detection cycle is less than a preset width threshold, then it is determined that the target pipeline in the current detection cycle and the target pipeline in the previous detection cycle belong to the same pipeline, thereby improving the robustness of information matching.

[0046] If it is determined that the target pipeline in the current detection cycle belongs to the same pipeline as the target pipeline in the previous detection cycle, the pipeline information of the current cycle and the pipeline information of the previous cycle are merged in the pipeline information dictionary. If the position difference is greater than or equal to a preset horizontal threshold, or the width difference is greater than or equal to a preset width threshold, the target pipeline in the current detection cycle is considered not to belong to the same pipeline as the target pipeline in the previous detection cycle, and the pipeline information of the current detection cycle is added to the pipeline information dictionary (i.e., a new pipeline entry is added to the pipeline information dictionary).

[0047] The pipeline information dictionary is stored in dictionary form. The pipeline name can be named "Cable_start_h_start_w". The second-level dictionary represents the coordinates of the left and right points and the width of the candidate pipeline. The coordinates can be represented by a two-dimensional list, and the length of the list represents the number of pipelines.

[0048] Step 104: Optimize the pipeline information dictionary by using prior information about the pipeline target, and combine the pixel-level pipeline target image coordinates in the post-optimized pipeline information dictionary with real-time navigation data to perform coordinate transformation and calculate the longitude and latitude coordinates of the pipeline target.

[0049] The post-processing module can be used to remove false detections from the inspection results and optimize the pipeline target detection results. Post-processing optimization can include at least the removal of submarine lines, disconnection of broken lines, and removal of short lines.

[0050] In this embodiment, seabed lines in side-scan sonar images have a linear highlighting feature, which can easily be misdetected as pipelines by deep neural networks. Therefore, seabed line removal processing is required to remove them, specifically including: The seabed coordinates can be calculated using the formula for calculating seabed coordinates, based on the resolution of the side-scan sonar image and the height of the underwater robot. The height of the underwater robot can be obtained from its altimeter, and the resolution of the side-scan sonar image can be calculated as the ratio of the side-scan sonar range to the image width. The formula for calculating seabed coordinates is as follows:

[0051] In the formula, The coordinates of the seabed line are... The height of the underwater robot, This represents the resolution of the side-scan sonar image.

[0052] If the difference between the coordinates of the left edge point of the pipeline and the seabed line in the current detection cycle is less than the preset seabed line threshold (sealine_treshhold), then the candidate pipeline in the current detection cycle is determined to be the seabed line. The amount of data identified as submarine lines among the candidate pipelines in the statistical pipeline information dictionary; If the proportion of data identified as submarine lines to the total data of candidate pipelines exceeds a preset threshold, the entire pipeline will be removed. The preset threshold can be set to 60%.

[0053] Because pipelines may be buried or the identification accuracy may be insufficient, the identification results for the same pipeline may be intermittent, requiring the connection of these broken lines. Specific steps may include: Calculate the distance between the end position of the current pipeline and the start position of other pipelines in the pipeline information dictionary; If the distance is less than the preset connection threshold (connection_treshhold), it is determined that the current pipeline belongs to the same pipeline as other pipelines, and the pipeline information of the current pipeline and other pipelines is merged in the pipeline information dictionary, such as... Figure 5 and Figure 6 The image shows the result of reconnecting the broken wire.

[0054] To meet tracking requirements, excessively short pipes and point noise need to be removed. Specific steps may include: Obtain the pipeline length of the candidate pipeline (i.e., the cumulative length or geometric length of the detection cycle). If the pipeline length is less than a preset length threshold (length_treshhold), the candidate pipeline is removed. Figure 7 The image shown is before the short line was removed. Figure 8 The image shows the result after the short lines have been removed.

[0055] After the above post-processing, pixel-level pipeline target image coordinates are obtained from the optimized pipeline information dictionary. Next, the optimized pixel-level pipeline target image coordinates can be combined with real-time navigation data for coordinate transformation, such as... Figure 9 The diagram shown illustrates the principle of converting sonar image coordinates to geodetic coordinates.

[0056] In this embodiment, the pixel-level pipeline target image coordinates in the post-processed optimized pipeline information dictionary are combined with real-time navigation data for coordinate transformation to calculate the longitude and latitude coordinates of the pipeline target. Specifically, this may include: Based on the x-coordinate of the pipeline target's pixels and the x-coordinate of the image center in the side-scan sonar image, the pixel offset between the pipeline target and the image center is calculated. Using the pixel offset and the resolution of the side-scan sonar image, the slant distance between the pipeline target and the underwater robot is calculated. The specific formula is as follows:

[0057] In the formula, The slant distance between the pipeline target and the underwater robot. This is the pixel offset (i.e., distance) between the pipeline target and the center of the image. Let x be the x-coordinate of the pipeline target in the side-scan sonar image. Let x be the x-coordinate of the pipeline target's center in the side-scan sonar image. This refers to the resolution of the side-scan sonar image; Using slant range, the underwater robot's height, and its heading angle, the horizontal and vertical distances between the pipeline target and the underwater robot are calculated using geometric formulas, as shown below:

[0058]

[0059] In the formula, The horizontal distance between the pipeline target and the underwater robot. The slant distance between the pipeline target and the underwater robot. The height of the underwater robot, The heading angle of the underwater robot. Let be the coordinate components of the pipeline target relative to the underwater robot in the horizontal direction. The coordinate components of the pipeline target relative to the underwater robot in the vertical direction; Based on the horizontal and vertical coordinate components and the current longitude coordinates of the underwater robot, the longitude and latitude changes of the pipeline target and the underwater robot are calculated, as shown in the following formulas:

[0060] In the formula, The longitude change between the pipeline target and the underwater robot. This represents the change in latitude between the pipeline target and the underwater robot. Let be the coordinate components of the pipeline target relative to the underwater robot in the horizontal direction. Let be the coordinate components of the pipeline target relative to the underwater robot in the vertical direction. This represents the current longitude coordinates of the underwater robot. Based on the changes in longitude and latitude, and the current longitude and latitude coordinates of the underwater robot, the longitude and latitude coordinates of the pipeline target are obtained, as shown in the following formula:

[0061] In the formula, The longitude coordinates of the pipeline target. The latitude coordinates of the pipeline target. The longitude change between the pipeline target and the underwater robot. This represents the change in latitude between the pipeline target and the underwater robot. The current longitude coordinates of the underwater robot. This represents the current latitude coordinates of the underwater robot.

[0062] In summary, the side-scan sonar detection method for subsea pipelines provided in this application, compared with the existing technology, can acquire side-scan sonar images and real-time navigation data of an underwater robot. The real-time navigation data includes at least altitude, heading angle, coordinates, and side-scan sonar image resolution. The side-scan sonar images are input into a deep learning model for feature extraction to obtain a predicted feature tensor. Based on the predicted feature tensor, pipeline prediction results and pipeline prediction confidence are determined. Based on the pipeline prediction results and / or pipeline prediction confidence, pipeline targets in the side-scan sonar images are identified. The pipeline targets are then processed using information about their positions in the image coordinate system to generate a pipeline information dictionary containing pixel-level pipeline target image coordinates. Prior information about the pipeline targets is used to post-process and optimize the pipeline information dictionary. The pixel-level pipeline target image coordinates in the post-processed and optimized pipeline information dictionary are then combined with the real-time navigation data for coordinate transformation to calculate the longitude and latitude coordinates of the pipeline targets.

[0063] By employing the above technical solution, this application extracts features from a deep learning model by inputting side-scan sonar images, obtains predicted feature tensors, and then determines the pipeline prediction results and pipeline prediction confidence. The deep learning model can automatically learn the feature patterns of subsea pipelines in sonar images, eliminating the need for manual setting of complex feature extraction rules, reducing manual intervention, and greatly improving the automation level of detection.

[0064] From acquiring side-scan sonar images and real-time navigation data of underwater robots to finally calculating the longitude and latitude coordinates of the pipeline target, the entire process covers multiple stages such as feature extraction, target recognition, information processing, post-processing optimization, and coordinate transformation, forming a complete automated detection process that improves detection efficiency.

[0065] The generated pipeline information dictionary is post-processed and optimized using prior information about pipeline targets. Prior information can include knowledge of common pipeline shapes, routes, and size ranges. By combining this prior information with the detection results, false detection targets that do not conform to pipeline characteristics can be effectively eliminated, thereby reducing the false alarm rate and improving the accuracy of the detection results.

[0066] Deep learning models possess powerful feature learning and representation capabilities, enabling them to capture subtle features of faint targets or buried pipelines from complex sonar images. By training on a large number of sonar images containing pipelines in different states, the model can learn the essential characteristics of these targets, accurately identifying pipeline targets even in situations with poor image quality and weak target signals.

[0067] This application not only utilizes side-scan sonar images but also incorporates real-time navigation data from underwater robots, such as altitude, heading angle, coordinates, and side-scan sonar image resolution. The fusion of this multi-source data provides the model with richer information, helping it to better understand image background and target features, further improving its ability to identify faint targets or buried pipelines, and meeting the urgent need for efficient and intelligent detection of subsea pipelines.

[0068] Based on the above Figure 1 The specific implementation of the method shown in this embodiment provides a side-scan sonar detection device for subsea pipelines, such as... Figure 10 As shown, the device includes: an acquisition module 31, an extraction module 32, a generation module 33, and a solution module 34; The acquisition module 31 is used to acquire side-scan sonar images and real-time navigation data of the underwater robot, wherein the real-time navigation data includes at least altitude, heading angle, coordinates and side-scan sonar image resolution; The extraction module 32 is used to input the side-scan sonar image into a deep learning model to extract features and obtain a predicted feature tensor, and to determine the pipeline prediction result and pipeline prediction confidence based on the predicted feature tensor. The generation module 33 is used to identify pipeline targets in the side-scan sonar image based on the pipeline prediction results and / or the pipeline prediction confidence, and to perform information processing on the pipeline targets based on the solution information of the target positions of the pipeline targets in the image coordinate system to generate a pipeline information dictionary, wherein the pipeline information dictionary is a dictionary containing pixel-level pipeline target image coordinates; The calculation module 34 is used to perform post-processing optimization on the pipeline information dictionary using the prior information of the pipeline target, and to combine the pixel-level pipeline target image coordinates in the post-processed pipeline information dictionary with the real-time navigation data to perform coordinate transformation and calculate the longitude and latitude coordinates of the pipeline target.

[0069] In specific application scenarios, the extraction module 32 can be used to input the side-scan sonar image into the deep learning model for feature extraction, obtain feature information at different scales, and fuse the feature information at different scales within the same scale to obtain multi-scale pyramid features; decode the multi-scale pyramid features to generate the prediction feature tensor; perform a maximum value indexing operation on the prediction feature tensor to obtain the pipeline prediction result; and normalize the prediction feature tensor to obtain the pipeline prediction confidence.

[0070] In specific application scenarios, the generation module 33 can be used to record pipeline information of the current detection cycle and pipeline information of the previous detection cycle of the side-scan sonar image based on the pipeline prediction results and / or the pipeline prediction confidence. The pipeline information of the current detection cycle includes the left edge point, the right edge point, and the pipeline width; the pipeline information of the previous detection cycle includes the left edge point, the right edge point, and the pipeline width. The generation module 33 compares the pipeline information of the current detection cycle with the pipeline information of the previous detection cycle. If the position difference between the left edge point of the pipeline in the current detection cycle and the left edge point of the pipeline in the previous detection cycle is less than a preset horizontal threshold, then it is determined that the target pipeline in the current detection cycle and the target pipeline in the previous detection cycle belong to the same pipeline. And / or, if the position difference between the right edge point of the pipeline in the current detection cycle and the left edge point of the pipeline in the previous detection cycle is less than a preset horizontal threshold, then it is determined that the target pipeline in the current detection cycle and the target pipeline in the previous detection cycle belong to the same pipeline. If the position difference between the right edge point of the pipeline in the current detection cycle and the target pipeline in the previous detection cycle is less than a preset horizontal threshold, then the target pipeline in the current detection cycle is determined to be the same pipeline as the target pipeline in the previous detection cycle; and / or, if the width difference between the pipeline width in the current detection cycle and the pipeline width in the previous detection cycle is less than a preset width threshold, then the target pipeline in the current detection cycle is determined to be the same pipeline as the target pipeline in the previous detection cycle; if the target pipeline in the current detection cycle and the target pipeline in the previous detection cycle are the same pipeline, then the pipeline information of the current detection cycle and the pipeline information of the previous detection cycle are merged in the pipeline information dictionary; if the target pipeline in the current detection cycle and the target pipeline in the previous detection cycle are not the same pipeline, then the pipeline information of the current detection cycle is added to the pipeline information dictionary.

[0071] In specific application scenarios, the post-processing optimization includes at least seabed line removal, broken line connection, and short line removal; the calculation module 34 can be used to calculate the seabed line coordinates based on the side-scan sonar image resolution and the height of the underwater robot; if the difference between the left edge point of the pipeline in the current detection cycle and the seabed line coordinates is less than a preset seabed line threshold, then the candidate pipeline in the current detection cycle is determined to be a seabed line; the amount of data identified as seabed lines in the candidate pipelines in the pipeline information dictionary is counted; if the proportion of the amount of data identified as seabed lines to the total amount of data in the candidate pipelines exceeds a preset proportion threshold, then the entire pipeline is removed.

[0072] In specific application scenarios, the calculation module 34 can be used to calculate the distance between the end position of the current pipeline and the start position of other pipelines in the pipeline information dictionary; if the distance is less than a preset connection threshold, it is determined that the current pipeline and the other pipelines belong to the same pipeline, and the pipeline information of the current pipeline and the other pipelines is merged in the pipeline information dictionary.

[0073] In a specific application scenario, the calculation module 34 can be used to obtain the pipeline length of the candidate pipeline; if the pipeline length is less than a preset length threshold, the candidate pipeline is removed.

[0074] In specific application scenarios, the calculation module 34 can be used to calculate the pixel offset between the pipeline target and the image center based on the pixel abscissa and the image center abscissa of the pipeline target in the side-scan sonar image; calculate the slant distance between the pipeline target and the underwater robot using the pixel offset and the resolution of the side-scan sonar image; calculate the horizontal and vertical coordinate components of the pipeline target relative to the underwater robot using the slant distance, the height of the underwater robot, and the heading angle of the underwater robot through geometric relationship formulas; calculate the longitude and latitude changes between the pipeline target and the underwater robot based on the horizontal and vertical coordinate components and the current longitude coordinate of the underwater robot; and obtain the longitude and latitude coordinates of the pipeline target based on the longitude change, the latitude change, the current longitude coordinates and the current latitude coordinates of the underwater robot.

[0075] It should be noted that other corresponding descriptions of the functional units involved in the side-scan sonar detection device for subsea pipelines provided in this embodiment can be found in [reference needed]. Figure 1 The corresponding descriptions in [the document] will not be repeated here.

[0076] Based on the above, Figure 1Accordingly, this embodiment also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method. Figure 1 The method shown.

[0077] Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as CD-ROM, USB flash drive, mobile hard drive, etc.) and includes several instructions to cause a computer device (such as personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of this application.

[0078] Based on the above, Figure 1 The method shown, and Figure 10 To achieve the above objectives, the present application also provides an electronic device, comprising a storage medium and a processor; the storage medium for storing a computer program; and the processor for executing the computer program to implement the above-described virtual device embodiments. Figure 1 The method shown.

[0079] Optionally, the aforementioned physical devices may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0080] Those skilled in the art will understand that the physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0081] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned physical device, supporting the operation of the subsea pipeline side-scan sonar detection program and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software within the subsea pipeline side-scan sonar detection physical device.

[0082] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platform, or it can be implemented by hardware. By applying the scheme of this embodiment, compared with the existing technology, this application can obtain side-scan sonar images and real-time navigation data of an underwater robot, wherein the real-time navigation data includes at least altitude, heading angle, coordinates, and side-scan sonar image resolution; input the side-scan sonar images into a deep learning model for feature extraction to obtain a predicted feature tensor, and determine the pipeline prediction result and pipeline prediction confidence based on the predicted feature tensor; identify pipeline targets in the side-scan sonar images based on the pipeline prediction result and / or pipeline prediction confidence, and perform information processing on the pipeline targets based on the target position solution information in the image coordinate system to generate a pipeline information dictionary, which is a dictionary containing pixel-level pipeline target image coordinates; perform post-processing optimization on the pipeline information dictionary using the prior information of the pipeline targets, and combine the pixel-level pipeline target image coordinates in the post-processed optimized pipeline information dictionary with real-time navigation data to perform coordinate transformation and calculate the longitude and latitude coordinates of the pipeline targets.

[0083] By employing the above technical solution, this application extracts features from a deep learning model by inputting side-scan sonar images, obtains predicted feature tensors, and then determines the pipeline prediction results and pipeline prediction confidence. The deep learning model can automatically learn the feature patterns of subsea pipelines in sonar images, eliminating the need for manual setting of complex feature extraction rules, reducing manual intervention, and greatly improving the automation level of detection.

[0084] From acquiring side-scan sonar images and real-time navigation data of underwater robots to finally calculating the longitude and latitude coordinates of the pipeline target, the entire process covers multiple stages such as feature extraction, target recognition, information processing, post-processing optimization, and coordinate transformation, forming a complete automated detection process that improves detection efficiency.

[0085] The generated pipeline information dictionary is post-processed and optimized using prior information about pipeline targets. Prior information can include knowledge of common pipeline shapes, routes, and size ranges. By combining this prior information with the detection results, false detection targets that do not conform to pipeline characteristics can be effectively eliminated, thereby reducing the false alarm rate and improving the accuracy of the detection results.

[0086] Deep learning models possess powerful feature learning and representation capabilities, enabling them to capture subtle features of faint targets or buried pipelines from complex sonar images. By training on a large number of sonar images containing pipelines in different states, the model can learn the essential characteristics of these targets, accurately identifying pipeline targets even in situations with poor image quality and weak target signals.

[0087] This application not only utilizes side-scan sonar images but also incorporates real-time navigation data from underwater robots, such as altitude, heading angle, coordinates, and side-scan sonar image resolution. The fusion of this multi-source data provides the model with richer information, helping it to better understand image background and target features, further improving its ability to identify faint targets or buried pipelines, and meeting the urgent need for efficient and intelligent detection of subsea pipelines.

[0088] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the term "comprising" or any other variations thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0089] The above are merely specific embodiments of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to these embodiments, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A side-scan sonar detection method for subsea pipelines, characterized in that, The method includes: Acquire side-scan sonar images and real-time navigation data of the underwater robot, wherein the real-time navigation data includes at least altitude, heading angle, coordinates and side-scan sonar image resolution; The side-scan sonar image is input into a deep learning model to extract features and obtain a prediction feature tensor. The pipeline prediction result and pipeline prediction confidence are then determined based on the prediction feature tensor. Based on the pipeline prediction results and / or the pipeline prediction confidence, pipeline targets in the side-scan sonar image are identified, and the pipeline targets are processed by information based on the solution information of the target positions of the pipeline targets in the image coordinate system to generate a pipeline information dictionary, which is a dictionary containing pixel-level pipeline target image coordinates. The pipeline information dictionary is post-processed and optimized using the prior information of the pipeline target. The pixel-level pipeline target image coordinates in the post-processed and optimized pipeline information dictionary are then combined with the real-time navigation data to perform coordinate transformation, and the longitude and latitude coordinates of the pipeline target are calculated. The step of identifying pipeline targets in the side-scan sonar image based on the pipeline prediction result and / or the pipeline prediction confidence level, and performing information processing on the pipeline targets based on the calculated target position information in the image coordinate system to generate a pipeline information dictionary, specifically includes: Based on the pipeline prediction results and / or the pipeline prediction confidence, the pipeline information of the current detection cycle and the pipeline information of the previous detection cycle of the side-scan sonar image are recorded. The pipeline information of the current detection cycle includes the left edge point of the pipeline, the right edge point of the pipeline, and the pipeline width. The pipeline information of the previous detection cycle includes the left edge point of the pipeline, the right edge point of the pipeline, and the pipeline width. Compare the pipeline information of the current detection cycle with the pipeline information of the previous detection cycle; If the positional difference between the left edge point of the pipeline in the current detection cycle and the left edge point of the pipeline in the previous detection cycle is less than a preset horizontal threshold, then it is determined that the target pipeline in the current detection cycle and the target pipeline in the previous detection cycle belong to the same pipeline; and / or, If the positional difference between the right edge point of the pipeline in the current detection cycle and the right edge point of the pipeline in the previous detection cycle is less than a preset horizontal threshold, then it is determined that the target pipeline in the current detection cycle and the target pipeline in the previous detection cycle belong to the same pipeline; and / or, If the difference between the pipeline width in the current detection cycle and the pipeline width in the previous detection cycle is less than a preset width threshold, then the target pipeline in the current detection cycle and the target pipeline in the previous detection cycle are determined to be the same pipeline. If the target pipeline in the current detection cycle and the target pipeline in the previous detection cycle belong to the same pipeline, then the pipeline information of the current detection cycle and the pipeline information of the previous detection cycle are merged in the pipeline information dictionary; If the target pipeline in the current detection cycle is not the same pipeline as the target pipeline in the previous detection cycle, then the pipeline information for the current detection cycle is added to the pipeline information dictionary.

2. The method according to claim 1, characterized in that, The step of inputting the side-scan sonar image into a deep learning model for feature extraction to obtain a predicted feature tensor, and determining the pipeline prediction result and pipeline prediction confidence based on the predicted feature tensor, specifically includes: The side-scan sonar image is input into the deep learning model for feature extraction to obtain feature information at different scales. The feature information at different scales is then fused within the same scale to obtain multi-scale pyramid features. The multi-scale pyramid features are decoded to generate the predicted feature tensor; The pipeline prediction result is obtained by performing a maximum index calculation operation on the predicted feature tensor. The predicted feature tensor is normalized to obtain the pipeline prediction confidence level.

3. The method according to claim 2, characterized in that, The post-processing optimization includes at least the removal of submarine lines, connection of broken lines, and removal of short lines.

4. The method according to claim 3, characterized in that, The removal of the seabed line specifically includes: Calculate the seabed coordinates based on the resolution of the side-scan sonar image and the height of the underwater robot; If the difference between the coordinates of the left edge point of the pipeline in the current detection cycle and the coordinates of the seabed line is less than a preset seabed line threshold, then the candidate pipeline in the current detection cycle is determined to be the seabed line. The number of candidate pipelines identified as submarine lines in the pipeline information dictionary is counted. If the proportion of data identified as the submarine line to the total data of the candidate pipelines exceeds a preset threshold, the entire pipeline will be removed.

5. The method according to claim 4, characterized in that, The disconnection connection specifically includes: Calculate the distance between the end position of the current pipeline and the start position of other pipelines in the pipeline information dictionary; If the distance is less than a preset connection threshold, the current pipeline is determined to belong to the same pipeline as the other pipelines, and the pipeline information of the current pipeline and the other pipelines is merged in the pipeline information dictionary.

6. The method according to claim 5, characterized in that, The removal of short lines specifically includes: Obtain the pipeline length of the candidate pipeline; If the pipeline length is less than a preset length threshold, the candidate pipeline is removed.

7. The method according to claim 1, characterized in that, The step of combining the pixel-level pipeline target image coordinates in the post-processed and optimized pipeline information dictionary with the real-time navigation data to perform coordinate transformation and calculate the longitude and latitude coordinates of the pipeline target specifically includes: Calculate the pixel offset between the pipeline target and the image center based on the pixel x-coordinate and the image center x-coordinate of the pipeline target in the side-scan sonar image. The slant distance between the pipeline target and the underwater robot is calculated using the pixel offset and the resolution of the side-scan sonar image. Using the slant distance, the height of the underwater robot, and the heading angle of the underwater robot, the coordinate components of the pipeline target relative to the underwater robot in the horizontal direction and the coordinate components in the vertical direction are calculated using geometric formulas. Based on the coordinate components in the horizontal direction and the coordinate components in the vertical direction, as well as the current longitude coordinates of the underwater robot, calculate the longitude and latitude changes of the pipeline target and the underwater robot. The longitude and latitude coordinates of the pipeline target are obtained based on the longitude change, the latitude change, and the current longitude and latitude coordinates of the underwater robot.

8. A side-scan sonar detection device for subsea pipelines, characterized in that, include: The acquisition module is used to acquire side-scan sonar images and real-time navigation data of the underwater robot, wherein the real-time navigation data includes at least altitude, heading angle, coordinates and side-scan sonar image resolution; The extraction module is used to input the side-scan sonar image into a deep learning model to extract features and obtain a predicted feature tensor, and to determine the pipeline prediction result and pipeline prediction confidence based on the predicted feature tensor. The generation module is used to identify pipeline targets in the side-scan sonar image based on the pipeline prediction results and / or the pipeline prediction confidence, and to perform information processing on the pipeline targets based on the solution information of the target positions of the pipeline targets in the image coordinate system to generate a pipeline information dictionary, wherein the pipeline information dictionary is a dictionary containing pixel-level pipeline target image coordinates; The calculation module is used to perform post-processing optimization on the pipeline information dictionary using the prior information of the pipeline target, and to combine the pixel-level pipeline target image coordinates in the post-processed and optimized pipeline information dictionary with the real-time navigation data to perform coordinate transformation and calculate the longitude and latitude coordinates of the pipeline target. The generation module is specifically used to record the pipeline information of the current detection cycle and the pipeline information of the previous detection cycle of the side-scan sonar image based on the pipeline prediction result and / or the pipeline prediction confidence. The pipeline information of the current detection cycle includes the left edge point of the pipeline, the right edge point of the pipeline, and the pipeline width. The pipeline information of the previous detection cycle includes the left edge point of the pipeline, the right edge point of the pipeline, and the pipeline width. Compare the pipeline information of the current detection cycle with the pipeline information of the previous detection cycle; If the positional difference between the left edge point of the pipeline in the current detection cycle and the left edge point of the pipeline in the previous detection cycle is less than a preset horizontal threshold, then it is determined that the target pipeline in the current detection cycle and the target pipeline in the previous detection cycle belong to the same pipeline; and / or, If the positional difference between the right edge point of the pipeline in the current detection cycle and the right edge point of the pipeline in the previous detection cycle is less than a preset horizontal threshold, then it is determined that the target pipeline in the current detection cycle and the target pipeline in the previous detection cycle belong to the same pipeline; and / or, If the difference between the pipeline width in the current detection cycle and the pipeline width in the previous detection cycle is less than a preset width threshold, then the target pipeline in the current detection cycle and the target pipeline in the previous detection cycle are determined to be the same pipeline. If the target pipeline in the current detection cycle and the target pipeline in the previous detection cycle belong to the same pipeline, then the pipeline information of the current detection cycle and the pipeline information of the previous detection cycle are merged in the pipeline information dictionary; If the target pipeline in the current detection cycle is not the same pipeline as the target pipeline in the previous detection cycle, then the pipeline information for the current detection cycle is added to the pipeline information dictionary.

9. An electronic device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

Images