Underwater pipeline detection method, device, equipment and storage medium

Abstract

The present invention provides an underwater pipeline detection method, device, equipment and storage medium, which relates to the field of underwater engineering technology. The method includes: obtaining a waveform image, an initial optical image and an initial acoustic image of an underwater pipeline; performing denoising and image enhancement processing on the initial optical image to obtain a target optical image; performing image segmentation processing on the initial acoustic image to obtain a target acoustic image; performing fusion processing on the waveform image, the target optical image and the target acoustic image to obtain information of the pipeline to be detected; performing identification processing on the pipeline information to be detected to obtain underwater pipeline detection results. The present invention provides a basis for the maintenance and repair of underwater pipelines, effectively obtains defect information of underwater pipelines, realizes monitoring thereof, improves the accuracy and efficiency of underwater pipeline detection, and ensures the safety and reliability of underwater facilities.

Description

Technical Field

[0001] The present invention relates to the field of underwater engineering technology, and in particular to an underwater pipeline detection method, device, equipment and storage medium. Background Art

[0002] In the current situation of oil resource shortage, the frequent breakage and leakage of oil pipelines have caused a large amount of oil waste. Therefore, the detection of submarine pipeline damage has become a research focus.

[0003] With the rapid development of deep learning and computer vision, underwater robots can better perceive the seabed environment and perform complex tasks such as submarine pipeline maintenance and target tracking. At present, the existing detection methods can accurately detect magnetic anomalies caused by ferromagnetic materials in different medium environments. Because of its advantages of continuous and rapid measurement of small changes in the geomagnetic field, it is widely used in pipeline non-destructive testing. However, although the existing detection methods can identify the size and depth of pipeline defects, the information transmitted by the sensors is not comprehensive enough to perform comprehensive detection of underwater pipeline defects. Summary of the invention

[0004] The problem solved by the present invention is how to obtain and process more effective pipeline defect information to achieve comprehensive detection of underwater pipelines.

[0005] In order to solve the above problems, the present invention provides an underwater pipeline detection method, device, electronic equipment and storage medium.

[0006] In a first aspect, the present invention provides an underwater pipeline detection method, comprising:

[0007] Acquiring a waveform image, an initial optical image, and an initial acoustic image of the underwater pipeline;

[0008] The initial optical image is subjected to denoising and image enhancement processing to obtain a target optical image, including:

[0009] Constructing a denoising model, and performing denoising processing on the initial optical image according to the denoising model to obtain a temporary optical image;

[0010] Performing color correction and contrast stretching processing on the temporary optical image to obtain a first optical sub-image and a second optical sub-image respectively;

[0011] Performing weighted processing and Laplace pyramid decomposition processing on the first optical sub-image and the second optical sub-image to obtain the target optical image;

[0012] Performing image segmentation processing on the initial acoustic image to obtain a target acoustic image;

[0013] fusing the waveform image, the target optical image and the target acoustic image to obtain information of the pipeline to be detected;

[0014] The information of the pipeline to be detected is identified and processed to obtain an underwater pipeline detection result.

[0015] Optionally, acquiring a waveform image of an underwater pipeline includes:

[0016] Magnetizing the underwater pipeline and performing leakage magnetic field detection to obtain a three-dimensional graph of leakage magnetic field data;

[0017] The waveform image is obtained according to the three-dimensional graph of the leakage magnetic field data.

[0018] Optionally, the process of constructing the denoising model includes:

[0019] Get the initial denoising model generator;

[0020] Removing the batch normalization layer of the initial denoising model generator and adding two layers of residual structure to obtain a temporary denoising model;

[0021] The temporary denoising model is tuned according to the loss function to obtain the denoising model.

[0022] Optionally, performing weighted processing and Laplace pyramid decomposition processing on the first optical sub-image and the second optical sub-image to obtain the target optical image includes:

[0023] Obtaining a local entropy weight map and a gradient weight map according to the first optical sub-image and the second optical sub-image;

[0024] Adding the local entropy weight map and the gradient weight map to obtain a normalized weight map;

[0025] According to the Laplacian pyramid decomposition, the normalized weight map, the first optical sub-image and the second optical sub-image are fused to obtain the target optical image.

[0026] Optionally, performing image segmentation processing on the initial acoustic image to obtain a target acoustic image includes:

[0027] Marking an observation area in the initial acoustic image to obtain a random field, wherein the random field includes pixel values ​​and position information of the initial acoustic image;

[0028] Modeling is performed according to the pixel values ​​and the position information to obtain a shadow area, an underwater area, and a target area;

[0029] Obtaining a joint probability distribution according to the shadow area, the bottom area and the target area;

[0030] According to the joint probability distribution, a maximum posterior probability is obtained;

[0031] The initial acoustic image is subjected to image segmentation processing according to the maximum a posteriori probability to obtain the target acoustic image.

[0032] Optionally, fusing the waveform image, the target optical image and the target acoustic image to obtain the information of the pipeline to be detected includes:

[0033] The waveform image, the target optical image and the target acoustic image are subjected to a particle filtering algorithm to obtain a group of initial particles to represent the initial position and state of the defect target in the underwater pipeline;

[0034] Constructing a dynamic model and obtaining process noise, updating the state of the initial particle according to the dynamic model, and adding the process noise to obtain a temporary particle;

[0035] Obtaining a preset ratio, and evenly distributing the temporary particles according to the preset ratio;

[0036] Constructing a measurement model and obtaining a measurement value, calculating the weight of the temporary particle according to the measurement value and normalizing it to obtain a temporary weight, wherein the measurement value is obtained according to the current initial optical image and the initial acoustic image;

[0037] Resampling is performed according to the temporary weight to obtain newly generated particles, and the above steps are repeated to obtain the target weight;

[0038] According to the target weight, the waveform image, the target optical image and the target acoustic image are fused to obtain the pipeline information to be detected.

[0039] Optionally, the identifying and processing the information of the pipeline to be detected to obtain the underwater pipeline detection result includes:

[0040] Performing edge information extraction and target detection on the target acoustic image in the pipeline information to be detected, and performing multi-level feature extraction on the target acoustic image using a deep residual network to obtain low-level features containing edge semantic information and high-level features containing position information;

[0041] Combining the low-level features containing edge semantic information with the high-level features containing position information, extracting edge information, and integrating multi-level backbone features;

[0042] According to the multi-level backbone features, the target acoustic image is gradually aggregated into multi-level fusion features, and converted into a binary image;

[0043] Combined with the activation function, feature extraction is performed on the binary image to obtain target feature information;

[0044] The target feature information is combined with the waveform image and the target optical image for identification to obtain the underwater pipeline detection result.

[0045] In a second aspect, the present invention provides an underwater pipeline detection device, comprising:

[0046] An acquisition unit, used for acquiring a waveform image, an initial optical image and an initial acoustic image of the underwater pipeline;

[0047] A processing unit, used for performing denoising and image enhancement processing on the initial optical image to obtain a target optical image, comprising:

[0048] Constructing a denoising model, and performing denoising processing on the initial optical image according to the denoising model to obtain a temporary optical image;

[0049] Performing color correction and contrast stretching processing on the temporary optical image to obtain a first optical sub-image and a second optical sub-image respectively;

[0050] Performing weighted processing and Laplace pyramid decomposition processing on the first optical sub-image and the second optical sub-image to obtain the target optical image;

[0051] Performing image segmentation processing on the initial acoustic image to obtain a target acoustic image;

[0052] fusing the waveform image, the target optical image and the target acoustic image to obtain information of the pipeline to be detected;

[0053] The identification unit is used to identify and process the information of the pipeline to be detected to obtain an underwater pipeline detection result.

[0054] In a third aspect, the present invention provides an electronic device, including a memory and a processor;

[0055] The memory is used to store computer programs;

[0056] The processor is used to implement the underwater pipeline detection method as described in the first aspect when executing the computer program.

[0057] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the underwater pipeline detection method as described in the first aspect is implemented.

[0058] The underwater pipeline detection method, device, electronic device and storage medium of the present invention have the following beneficial effects: the underwater pipeline detection method of the present invention first obtains a waveform image, an initial optical image and an initial acoustic image of the underwater pipeline, wherein these images can be obtained by a sensor mounted on an autonomous underwater robot or a remote-controlled underwater robot. Due to the particularity of the underwater environment, the obtained image often contains noise and interference. Therefore, it is necessary to perform denoising and image enhancement processing on the initial optical image to improve the image quality; for the initial acoustic image, it is necessary to perform image segmentation processing to separate the pipeline and other important features from the background, fuse the preprocessed optical image and the segmented acoustic image with the waveform image, combine multiple sensor data, provide more comprehensive pipeline information, improve the robustness of the detection system, reduce the limitations of a single data source, and extract and enhance the features in the image; through image processing and feature recognition, determine the position of the pipeline and the specific position of the defect, analyze the detection results, and form a detailed report to provide a basis for the maintenance and repair of the underwater pipeline, effectively obtain the defect information of the underwater pipeline, and realize the monitoring thereof, improve the accuracy and efficiency of underwater pipeline detection, and ensure the safety and reliability of underwater facilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A schematic diagram of a flow chart of an underwater pipeline detection method according to an embodiment of the present invention;

[0060] Figure 2 A schematic diagram of a camera and sonar coordinate system of an underwater pipeline detection method according to an embodiment of the present invention;

[0061] Figure 3 A schematic diagram of a denoising model of an underwater pipeline detection method according to an embodiment of the present invention;

[0062] Figure 4 A particle filtering step diagram of an underwater pipeline detection method according to an embodiment of the present invention;

[0063] Figure 5 The figure is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be interpreted as being limited to the embodiments described herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not intended to limit the scope of protection of the present invention.

[0065] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0066] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to"; the term "based on" means "based at least in part on"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments"; the term "optionally" means "optional embodiments". The relevant definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc. mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0067] It should be noted that the modifications of "one" and "plurality" mentioned in the present invention are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0068] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only used for illustrative purposes, and are not used to limit the scope of these messages or information.

[0069] like Figure 1 As shown, an underwater pipeline detection method provided by an embodiment of the present invention includes:

[0070] Step S1, acquiring a waveform image, an initial optical image and an initial acoustic image of an underwater pipeline;

[0071] Specifically, this embodiment takes into account the advantages of optical images that can clearly see the surface and acoustic images that can see the exposed conditions of underwater pipelines. The optical image has a better imaging effect underwater. Although there will be water refraction and other noise when shooting through an optical lens, the image content is richer after some processing, but the shooting distance is also required not to be too far. The acoustic image is generated by underwater sonar, that is, the acoustic image is also the sonar image, which can only express partial information and cannot fully express underwater information, which is not conducive to the identification of underwater targets. Therefore, the fusion of optical image and acoustic image information is more accurate for underwater target identification, and combined with the waveform image, the defect location is determined.

[0072] Specifically, before acquiring the image, the present embodiment first performs camera calibration. First, the camera coordinate system is established. ,in, It points to the right. It points downward. It points to the front of the camera and is consistent with the direction of the camera's image plane, pixel plane, and the Z axis of the world coordinate system. There is a conversion relationship between the coordinate systems, and the conversion between the coordinate systems is achieved through matrix operations. The conversion matrix contains the rotation submatrix R and the translation submatrix T. The conversion formula is given by the following formula:

[0073] ,

[0074] in, is the optical axis of the lens, perpendicular to the image plane. is the world coordinate system, is the camera coordinate system.

[0075] The image plane coordinate system is a system that maps the actual object to the plane where the image is located through the camera itself. The coordinate plane is parallel to the optical lens of the camera and is located at the focal length f. The unit of the image plane coordinate system is millimeter level. In the pinhole imaging model, there is a similar triangle conversion relationship between the camera coordinate system and the image coordinate system, as shown in the following formula:

[0076] ,

[0077] in, is the coordinate of the pixel in the image coordinate system, f is the focal length, and c is the reference of the camera.

[0078] Change the above formula to:

[0079] ,

[0080] The plane of optical imaging is called the image plane. The image presented by the image plane is composed of analog quantities, which need to be sampled and quantized to obtain the pixel values ​​on the pixel plane. There is a scaling and translation relationship between the pixel plane and the imaging plane, which is achieved through the corresponding formula. On-axis zoom times, in On-axis zoom times, the origin is shifted to .

[0081] ,

[0082] in, is the horizontal axis of the pixel coordinate system, is the vertical axis of the pixel coordinate system, is the origin of the image coordinate system below the origin of the pixel coordinate system Axis and Axis position, for The number of pixels per unit length in the axis direction, for The number of pixels per unit length in the axis direction.

[0083] By summarizing the above formulas, we can come up with the formula for transforming objects from the world coordinate system to the camera coordinate system:

[0084] ,

[0085] By summarizing the above formula, the homogeneous coordinates of the midpoint in space are expressed as Projection to image plane point coordinates , we can get the corresponding summary formula:

[0086] ,

[0087] Where P is The projection matrix is ​​decomposed into:

[0088] ,

[0089] ,

[0090] The matrix A can be described as:

[0091] ,

[0092] Among them, parameter A represents the camera intrinsic parameter matrix, Represents the scale factor. represents the principal coordinate point, f is the effective focal length, is the aspect ratio, is the tilt factor, where R and T represent the rotation and translation matrices from the world coordinate system to the camera coordinate system. represents the zero vector. Therefore, the rotation matrix, translation matrix, zero vector and parameter 1 together Construct a matrix to describe the conversion relationship between the two.

[0093] In order to better restore the true scale ratio, calibration must be performed. Zhang Zhengyou's calibration algorithm can calibrate the camera parameters and obtain higher accuracy. The calibration data is obtained through the calibration plate, and then the collected data is calibrated to obtain the camera intrinsic parameters and two distortion parameters, which are radial distortion parameters and tangential distortion parameters.

[0094] First, determine the coordinates of the origin of the coordinate system and convert the world coordinates Set it to 0, and set one corner of the extracted calibration plate grid to (0,0), and then get an H matrix:

[0095] ,

[0096] in, is the homography matrix, is the scale factor, parameter A represents the internal parameter matrix, and R represents the rotation matrix ,and and Orthogonal, while satisfying the relationship , and the camera parameter constraint equation is expressed as:

[0097] ,

[0098] At the same time The following formula is obtained:

[0099] ,

[0100] in, express The number of pixels per unit length in the axis direction, express The number of pixels per unit length in the axial direction, B is a symmetric matrix with 6 dimensions:

[0101] ,

[0102] At the same time, the relationship between B and H matrices can be established to obtain the formula:

[0103] ,

[0104] For the above formula , which is defined as follows:

[0105] ,

[0106] The above results The formula is sorted to get the standard formula:

[0107] ,

[0108] In this way, the correspondence between reprojection and camera parameters is obtained through the above formula.

[0109] Secondly, the calibration and data fusion method between the imaging sonar and underwater camera sensors. Since the working methods and working environments of underwater robot sensors are different, the two sensors can complement each other. Their coordinate system is Figure 2 shown.

[0110] The transformation from the world coordinate system to the camera coordinate system requires translation and rotation operations to establish the relationship between the two. The transformation matrix is a The matrix of .

[0111] ,

[0112] Similarly, through the transformation matrix ( ) Convert the world coordinate system to the sonar coordinate system. The conversion formula is:

[0113] ,

[0114] In summary:

[0115] ,

[0116] The above formula can be derived:

[0117] ,

[0118] in, is the coordinate in the camera coordinate system, is the coordinate in the sonar coordinate system.

[0119] The intrinsic parameters of the camera have been derived in the camera modeling, so we only need to solve Then, we can further transform it to the pixel plane by transforming the camera internal parameter matrix:

[0120] ,

[0121] Next, we will deduce the above summary in detail. Through the previous modeling of the camera, we can deduce the three-dimensional transformation relationship between the calibration plate and the camera:

[0122] ,

[0123] Where R represents the rotation matrix, T represents the translation matrix, is the world coordinate system, is the camera coordinate system.

[0124] According to the sonar model, the three-dimensional transformation relationship between the centroid of the sonar calibration plate and the sonar coordinate system can be derived:

[0125] ,

[0126] in, , ,in is the distance from the object to the sonar center, Objects and The angle of the plane, Objects in Projection of the plane and The angle between the axes, the rotation and translation matrices of the calibration plate centroid coordinate system and the calibration plate coordinate system are:

[0127] ,

[0128] Among them, point With point m and n represent the number of checkerboard grids in the horizontal physical direction and vertical direction of the calibration plate respectively: and Respectively represent the physical dimensions of the chessboard in the horizontal and vertical directions, where the following conversion relationship exists between the calibration plate centroid coordinate system and the calibration plate plane coordinate system:

[0129] ,

[0130] in, is the calibration plate centroid coordinate system, is the plane coordinate system of the calibration plate.

[0131] By summarizing the formula, we can get the conversion relationship between sonar and camera:

[0132] ,

[0133] in, is the sonar coordinate system, is the camera coordinate system.

[0134] The above formula is further processed, and the matrices R and T and related parameters are substituted into the above formula to obtain the transformation matrix. Then the obtained transformation formula is substituted into the following formula to realize the transformation from target coordinates to pixel coordinates.

[0135] The above theoretical derivation is based on the fact that the target object is a calibration plate. It can be applied to the case where a non-calibration plate is used as the target object for calibration. That is, when only the target centroid position coordinates appear, it is sufficient to improve the above formula. The vector is changed to the length and width of the minimum enclosing rectangle.

[0136] Step S2, performing denoising and image enhancement processing on the initial optical image to obtain a target optical image, including:

[0137] Constructing a denoising model, and performing denoising processing on the initial optical image according to the denoising model to obtain a temporary optical image;

[0138] Performing color correction and contrast stretching processing on the temporary optical image to obtain a first optical sub-image and a second optical sub-image respectively;

[0139] Performing weighted processing and Laplace pyramid decomposition processing on the first optical sub-image and the second optical sub-image to obtain the target optical image;

[0140] Specifically, due to the complexity of the underwater environment, the acquired optical images often contain noise and distortion. Denoising is a key step in image preprocessing, which can be achieved through but not limited to wavelet transform, filter and other methods. The purpose is to reduce random noise in the image and retain valuable signals. Image enhancement further improves the image quality and makes the defect features of the pipeline surface more obvious. Usually, techniques including but not limited to contrast enhancement and edge enhancement can be used.

[0141] Step S3, performing image segmentation processing on the initial acoustic image to obtain a target acoustic image;

[0142] Specifically, acoustic images provide information about the internal structure of the pipeline, but it is often necessary to separate the pipeline structure from the complex background. Image segmentation techniques, such as threshold segmentation, edge detection, and region growing, can be used to identify and separate the outline of the pipeline and potential defect areas.

[0143] Step S4, fusing the waveform image, the target optical image and the target acoustic image to obtain information of the pipeline to be detected;

[0144] Specifically, the fusion of waveform images, optical images and acoustic images can provide more comprehensive pipeline information. Fusion algorithms can include IHS (intensity-chroma-saturation) fusion, PCA (principal component analysis) fusion, frequency domain fusion, etc., which combine the advantages of different images to generate a fused image containing more information. Based on the fused image, machine learning and deep learning algorithms are applied to identify pipeline defects, including but not limited to the use of convolutional neural networks (CNN) to identify defects such as cracks, corrosion, and deformation. By training models to identify different defect patterns, the defect detection process can be automated.

[0145] Step S5, identifying and processing the information of the pipeline to be detected to obtain an underwater pipeline detection result.

[0146] Specifically, the test results are analyzed in detail and a report is generated, which includes classifying, locating and assessing the severity of the detected defects, providing a basis for subsequent maintenance or intervention.

[0147] The underwater pipeline detection method in this embodiment first obtains a waveform image, an initial optical image and an initial acoustic image of the underwater pipeline, wherein these images can be obtained by a sensor carried on an autonomous underwater robot or a remote-controlled underwater robot. Due to the particularity of the underwater environment, the obtained image often contains noise and interference. Therefore, it is necessary to perform denoising and image enhancement processing on the initial optical image to improve the image quality; for the initial acoustic image, it is necessary to perform image segmentation processing to separate the pipeline and other important features from the background, and fuse the preprocessed optical image and the segmented acoustic image with the waveform image, combine multiple sensor data, provide more comprehensive pipeline information, improve the robustness of the detection system, reduce the limitations of a single data source, and extract and enhance the features in the image; through image processing and feature recognition, determine the location of the pipeline and the specific location of the defect, analyze the detection results, and form a detailed report to provide a basis for the maintenance and repair of the underwater pipeline, effectively obtain the defect information of the underwater pipeline, and realize its monitoring, improve the accuracy and efficiency of underwater pipeline detection, and ensure the safety and reliability of underwater facilities.

[0148] Optionally, acquiring a waveform image of an underwater pipeline includes:

[0149] Magnetizing the underwater pipeline and performing leakage magnetic field detection to obtain a three-dimensional graph of leakage magnetic field data;

[0150] The waveform image is obtained according to the three-dimensional graph of the leakage magnetic field data.

[0151] Specifically, first, the underwater pipeline is magnetized using a magnetizing device. Magnetization can be performed by DC magnetization, AC magnetization, permanent magnet magnetization, or composite magnetization. During the magnetization process, the magnetic induction lines will form a closed loop inside the pipeline material; when there are defects in the pipeline material (such as cracks, corrosion, deformation, etc.), these defects will interrupt the continuity of the magnetic induction lines, causing some magnetic induction lines to leak out from the pipeline surface, forming a leakage magnetic field. The intensity and distribution of the leakage magnetic field are related to the type, size, shape, and location of the defect; a highly sensitive magnetic induction sensor (such as a Hall sensor, a magnetoresistive sensor, etc.) is used to move along the surface of the pipeline to collect leakage magnetic field data. These data usually include information such as the intensity, direction, and distribution of the leakage magnetic field; the collected leakage magnetic field data needs to be preprocessed, including denoising, filtering, signal enhancement, etc., to improve the signal-to-noise ratio and readability of the data. Then, the leakage magnetic field data is analyzed by data analysis techniques (such as finite element analysis, signal processing algorithms, etc.) to extract defect feature information; based on the processed leakage magnetic field data, a three-dimensional model of the pipeline is constructed using computer graphics technology, and the leakage magnetic field data is mapped to the three-dimensional model to form a three-dimensional graph of the leakage magnetic field data. This three-dimensional graph can intuitively display the distribution of the leakage magnetic field on the pipeline surface; finally, a waveform image is generated based on the leakage magnetic field data of the three-dimensional graph. The waveform image is a two-dimensional representation that converts the changes in the leakage magnetic field data into a waveform curve, which is convenient for analyzing and identifying defects. The peaks, troughs and abnormal changes in the waveform image usually correspond to defects in the pipeline. Through the above steps, comprehensive detection of underwater pipelines can be achieved from magnetization, leakage magnetic field data acquisition, data processing and analysis, to the final generation of waveform images. This method can effectively identify and evaluate pipeline defects and provide an important basis for the maintenance and repair of underwater pipelines.The formation process of leakage magnetic field is roughly as follows: first, the defects or changes in the organizational structure on the surface and near the surface of the ferromagnetic material cause the magnetic permeability of the space to change, increasing the magnetic resistance at the defect or structural change, and then causing the original magnetic field line flow direction to change. Specifically, the ferromagnetic material cannot accommodate the magnetic flux of the previous size, causing part of the magnetic flux to bypass the defect or directly pass through the defect. This part of the magnetic flux first enters the air above the defect surface or structural change, and finally returns to the ferromagnetic material. The process of magnetic flux entering the air from the material forms a leakage magnetic field. The magnetic field coil power supply is selected to output a stable direct The galvanic drive coil generates a magnetic field. The sensor of the detector is composed of 15 Hall elements evenly distributed along the circumference of the pipeline. A total of 15 channels of test data are collected to complete a scanning test. After analyzing and processing the test data, a three-dimensional graph of the leakage magnetic field data is obtained. The pipeline is a ferromagnetic material and produces corresponding magnetic field changes under the action of the excitation field. By collecting the magnetic field signal, the internal situation of the ferromagnetic material can be obtained. In order to better analyze the specific situation, the device is simplified and designed as a permanent magnet excitation device to magnetize different types of defects, obtain the magnetic field characteristics of the corresponding defects, and obtain information on the four defect types of corrosion, holes, deformation, and cracks.

[0152] Optionally, the process of constructing the denoising model includes:

[0153] Get the initial denoising model generator;

[0154] Removing the batch normalization layer of the initial denoising model generator and adding two layers of residual structure to obtain a temporary denoising model;

[0155] The temporary denoising model is tuned according to the loss function to obtain the denoising model.

[0156] Specifically, an initial denoising model generator is obtained. This generator can be a network based on deep learning, such as a convolutional neural network (CNN). The goal of the denoising model is to reduce random noise in the image while retaining important structural information. The batch normalization layer in the initial denoising model generator is removed because the batch normalization layer may suppress some useful signals in the denoising process. Then, two layers of residual structures are added to help the model learn more complex features and prevent the problem of gradient disappearance in deep networks. The residual structure allows the network to learn the identity mapping by adding jump connections, making the training of deep networks easier. The temporary denoising model is tuned using a loss function. The loss function is an indicator that measures the difference between the model prediction and the actual image. Common loss functions include mean square error (MSE) and structural similarity index (SSIM). By optimizing the loss function, the model parameters can be adjusted to make it better denoised. The denoised temporary optical image is color corrected and contrast stretched. Color correction is to eliminate the color cast of underwater images to make the colors more realistic. Contrast stretching is to enhance the contrast of the image and make the details clearer. The first optical sub-image and the second optical sub-image that have been color corrected and contrast stretched are weighted. This can be achieved by calculating the weighted average of the two images. The weight can be determined based on the local features of the image. Then, Laplace pyramid decomposition is performed, which helps to further enhance the details and edge information of the image. As a result, this embodiment obtains target optical images with higher clarity and less noise. These images provide more accurate visual information for subsequent underwater pipeline detection. The combination of these processing steps ensures that the image retains as much useful information as possible during the denoising and enhancement process.

[0157] Among them, the proposed basic network architecture is described, such as Figure 3 As shown in the figure, in order to further improve the denoising quality of the model, two modifications were made to the structure of the generator G: the batch normalization (BN) layer was removed, and the residual-in-residual dense block (RRDB) structure was introduced, which successfully improved the performance of the denoising model. First, removing the batch normalization layer helps to avoid the model from over-normalizing the input data, thereby retaining more image detail information. This prevents the noise introduced by batch normalization from having a negative impact on the denoising results and reduces the amount of computation required for the model. Secondly, the introduction of the RRDB structure further improves the capabilities of the generator. RRDB combines the characteristics of multi-level residual networks and dense connections, allowing information to be better transmitted and reused in the network. Through multi-level residual connections, the model can learn the feature representation of the image more deeply, thereby improving the denoising effect. Dense connections help promote the flow of feature information, allowing the model to better capture the details and textures in the image.

[0158] Specifically, the discriminator in the standard GAN is replaced by the relativistic average discriminator (RaD), which is expressed as In SRGAN, the standard discriminator can be expressed as:

[0159] ,

[0160] in is the Sigmoid function, is the discriminator output without converting the image into a discriminant value. The formula for the relativistic average discriminator is:

[0161] ,

[0162] in Express The discriminator output expectation. The loss function of the discriminator is defined as follows:

[0163] ,

[0164] The loss function of the generator is:

[0165] ,

[0166] in , represents the input low-resolution noisy image, , represents the real image. Observing the above formula, we find that the adversarial loss of the generator contains and ,Therefore, during the adversarial network training process, the generator can better adapt to the gradient changes of generated images and real images.

[0167] Among them, when using the activated features to reconstruct the real image, it may cause the problem of inconsistent brightness, because the activated features may not be able to completely restore the brightness of the original image. To solve this problem, this paper uses the features before the activation layer for reconstruction, which can more accurately retain the brightness information of the original image, thereby improving the reconstruction quality and consistency. Therefore, the total loss function of the generator is:

[0168] ,

[0169] in, is the content loss that evaluates the reconstructed image With real image The 1-norm between is the perceptual loss, and and is a coefficient used to balance different loss terms.

[0170] Specifically, the first version of the fusion process is color correction, which achieves image enhancement through histogram equalization. In underwater images, it is applied to three color channels respectively, and color correction is achieved by evenly distributing the color intensity values ​​in a certain color range throughout the entire color range. The second version of the fusion process is an enhanced image based on contrast stretching. There are limitations in using dark channel priors, and the enhancement effect is different from the transmission image. There is a direct relationship, The quality of directly determines the enhancement effect, so the transmission map is estimated by DehazeNet It is used to improve the robustness of the algorithm. Its specific implementation is as follows:

[0171] Among them, the DehazeNet transmission graph estimation network: feature extraction is different from the traditional convolutional neural network. In the first layer of the convolution layer with Maxout as the activation function, its expression function is as follows:

[0172] ,

[0173] in and represent weight and bias respectively, represents the network input, represents the convolution operation, express Affine feature maps.

[0174] The original image is initially enhanced by the above two methods to obtain two sub-images. Figure 4 The structure shows the final enhancement of the original image.

[0175] ,

[0176] in , represents a DCP-enhanced image, , Represent the maximum and minimum pixel values ​​of each color channel, and Max and Min represent the maximum and minimum intensity values ​​of the image, respectively.

[0177] Optionally, performing weighted processing and Laplace pyramid decomposition processing on the first optical sub-image and the second optical sub-image to obtain the target optical image includes:

[0178] Obtaining a local entropy weight map and a gradient weight map according to the first optical sub-image and the second optical sub-image;

[0179] Adding the local entropy weight map and the gradient weight map to obtain a normalized weight map;

[0180] According to the Laplacian pyramid decomposition, the normalized weight map, the first optical sub-image and the second optical sub-image are fused to obtain the target optical image.

[0181] Specifically, the local entropy weight map: The local entropy value gives an estimate of the information change in the local area of ​​the image, thereby reducing the impact of noise. Local entropy is used as an indicator in many applications, such as region extraction. The local entropy is converted from RGB image to HSV format and the MATLAB function entropyfilt is used to calculate the local entropy. The default neighborhood value of this value is calculated by The image component V is represented by . This weight map can only estimate one aspect, namely the information content of the image, so another weight map is needed to estimate the contrast of the image with another weight.

[0182] Gradient weight map: The gradient weight map estimates the variance of the image color saturation. The gradient weight map is calculated using the directional gradient of the HSV image saturation (S) component. The gradient weight map is calculated using the following formula ( ):

[0183] ,

[0184] ,

[0185] ,

[0186] in, is the horizontal image saturation value, is the vertical image saturation value. are the pixel values ​​to the left and right of the image saturation center, respectively. They are the pixel values ​​above and below the saturation center of the image, respectively.

[0187] Finally, for each input k, a weight map is obtained by adding the two weight maps ( ). The final normalized weight map for each input image k is derived using Eq. ( ).

[0188] ,

[0189] ,

[0190] in refers to the weight map of the two input images, and adds a constant of 0.001 to prevent division by zero errors. The image component V is represented by . Fusion based on multi-scale pyramid decomposition can complete image fusion by simply adding the two versions of the input image using the equation, but this addition will cause artifacts in the final image. Therefore, in order to avoid artifacts, this paper chooses to perform fusion based on multi-scale Laplacian pyramid decomposition, which is obtained by the formula:

[0191] ,

[0192] The multi-scale Laplacian pyramid decomposition of each version of the input image is performed on five levels. Now, the corresponding levels of each version are fused to form a fused pyramid using the following equation.

[0193] ,

[0194] in, is the final weight map, is the weight coefficient, is the fused pyramid level l, represents the Gaussian pyramid at level l, represents the Laplacian at level l.

[0195] Optionally, performing image segmentation processing on the initial acoustic image to obtain a target acoustic image includes:

[0196] Marking an observation area in the initial acoustic image to obtain a random field, wherein the random field includes pixel values ​​and position information of the initial acoustic image;

[0197] Modeling is performed according to the pixel values ​​and the position information to obtain a shadow area, an underwater area, and a target area;

[0198] Obtaining a joint probability distribution according to the shadow area, the bottom area and the target area;

[0199] According to the joint probability distribution, a maximum posterior probability is obtained;

[0200] The initial acoustic image is subjected to image segmentation processing according to the maximum a posteriori probability to obtain the target acoustic image.

[0201] Specifically, the area observed in the sonar image is set as , and marked as , then the random field is defined as ,in , and ,in Represented as the pixel value of the sonar image, Indicates location. The value range is . Our grayscale value of 255 represents white, and the grayscale value of 0 represents black. Mainly by Composition, among which represents the shaded area, It represents the underwater area. represents the target area. Therefore, the joint probability distribution The MRF distribution is expressed as:

[0202] ,

[0203] The goal of MAP-MRF is to maximize the posterior probability , this paper derives from the Bayesian formula:

[0204] ,

[0205] ,

[0206] in, is the joint probability of the random field, yes The conditional probability distribution of Yes Mark The probability of It can be maximized Or the minimized value is According to the theory of HammersleyClifford and the relationship between MRF and Gibbs distribution, the formula is:

[0207] ,

[0208] in is the partition function, is the energy function, which is defined as follows:

[0209] ,

[0210] Among them, t represents a neighboring pixel position of s, represents the neighborhood of s, represents the Markov model parameters for different position combinations. represents the Kronecker function, which is defined as follows:

[0211] ,

[0212] Probability The Nakagami distribution is usually assumed and for simplicity is defined as follows:

[0213] ,

[0214] in, yes The factorial of kth iteration The conditional distribution probability of represents the distribution parameter, and L is updated according to the above formula: In MRF, k-means clustering is used to obtain the initial three-category segmentation result L. At the kth iteration, Initial mark is , and is updated according to the following formula: ,

[0215] .

[0216] Specifically, the observed area in the initial acoustic image is marked and a random field is created. A random field is a statistical model that contains the pixel values ​​and position information of an image and can capture the spatial dependencies between pixels, which is particularly important for image segmentation. Based on the pixel values ​​and position information in the random field, the shadow area, underwater area, and target area in the image are modeled, and a joint probability distribution between the shadow area, underwater area, and target area is established. The joint probability describes the relationship between multiple random variables, which is crucial for understanding and segmenting complex structures in images. Based on the joint probability distribution, the maximum a posteriori probability (MAP) is calculated. The maximum a posteriori probability is an estimate of the maximum probability of a hypothesis being true given the observed data. In image segmentation, this means that we need to find the region label configuration that is most likely to produce the observed data. This usually involves Bayesian estimation, in which we need to consider the prior probability and likelihood function, and segment the initial acoustic image according to the maximum a posteriori probability to obtain the target acoustic image.

[0217] Optionally, fusing the waveform image, the target optical image and the target acoustic image to obtain the information of the pipeline to be detected includes:

[0218] The waveform image, the target optical image and the target acoustic image are subjected to a particle filtering algorithm to obtain a group of initial particles to represent the initial position and state of the defect target in the underwater pipeline;

[0219] Constructing a dynamic model and obtaining process noise, updating the state of the initial particle according to the dynamic model, and adding the process noise to obtain a temporary particle;

[0220] Obtaining a preset ratio, and evenly distributing the temporary particles according to the preset ratio;

[0221] Constructing a measurement model and obtaining a measurement value, calculating the weight of the temporary particle according to the measurement value and normalizing it to obtain a temporary weight, wherein the measurement value is obtained according to the current initial optical image and the initial acoustic image;

[0222] Resampling is performed according to the temporary weight to obtain newly generated particles, and the above steps are repeated to obtain the target weight;

[0223] According to the target weight, the waveform image, the target optical image and the target acoustic image are fused to obtain the pipeline information to be detected.

[0224] Specifically, according to the prior distribution of the initial state, a set of particles is sampled, which represent the initial position and state of possible defect targets in the underwater pipeline; the state of each particle is updated according to the dynamic model of the system. The dynamic model describes the evolution of the system state over time. For example, pipeline defects may change with water flow or mechanical stress; in the prediction process, process noise is added to each particle to simulate the uncertainty and external interference of the system; since the difference in particle weights may cause some particles to be less important, the resampling step samples according to the weights of the particles to avoid the problem of weight degradation. This usually involves removing particles with low weights, increasing the replication of particles with high weights, and updating the weights of particles using current measurements (such as features of optical images and acoustic images). This usually involves calculating the match between the predicted observations and the actual measurements of each particle. The weights of all particles are normalized to ensure that their sum is 1, and the prediction, resampling and measurement update steps are repeated until the required detection accuracy is achieved or the processing of all measurement data is completed. Finally, the final position and state of the pipeline defect are estimated based on the weights and states of the particles, and the information of different images is fused together to obtain the comprehensive information of the pipeline to be inspected.

[0225] Specifically, the particle filter algorithm is divided into six steps: Figure 4 As shown:

[0226] Step 1: Initialization, generate N particles, first according to the set initial distribution Generate N particles. If the initial distribution is not set, they are uniformly distributed in a limited space by default.

[0227] Step 2: Dynamic model update,The time evolution of the particles is updated through the dynamic model, and the process noise is added.

[0228] ,

[0229] Step 3: Redistribution: evenly distribute particles according to a certain ratio.

[0230] Step 4: Measurement model update, given measurement values Calculate the weight of each particle and normalize it.

[0231] Step 5: Resampling by giving weights Resample by replacing to generate N new particles,

[0232] Step 6: Go to step 2 and re-enter.

[0233] in is the state of the kth particle from time n-1 to time n, is the state transition model, is to control input noise.

[0234] Dynamic model establishment: The state matrix and dynamic model are shown in the equation, which represents the position and velocity of the object in the sonar frame. According to the above step 1, the initialized N particles are evenly distributed in the workspace. However, some simple velocity displacement matrices are added with some noise to update the position of each particle.

[0235] ,

[0236] in is the position coordinate at the next moment, The previous position coordinates.

[0237] Since the objects we want to track are stationary relative to the world coordinate system, we can incorporate the onboard odometry into the object motion. , we can predict the next position based on the previous position coordinates and the current speed. To do this, we first convert the Convert it to the underwater robot coordinate system and convert it into polar coordinates. It can be obtained by the following formula:

[0238] ,

[0239] in is the position of the object to the origin, is the angle with the xz plane, is the angle with the yz plane, is the displacement in the y direction, is the displacement in the z direction.

[0240] Then, the polar coordinate representation of the object in the underwater robot frame is differentiated in the equation, where According to the constant speed acquisition, the Jacobian matrix is ​​generated by the above formula:

[0241] ,

[0242] Next, the parameters in the Jacobian matrix are equivalent to sensor data, where is equivalent to a negative mean velocity component, , denote the pitch angular velocity equivalent to pitch and yaw respectively.

[0243] ,

[0244] in, is the pitch angular velocity, is the pitch velocity of the yaw.

[0245] ,

[0246] The newly estimated state Convert to sonar frames According to the following formula:

[0247] ,

[0248] Measurement model: Through the above description, this paper has predicted the new position of the object through the dynamic model , we now correct this problem using the sonar and camera measurements. To do this, since the calibration data is known, we can project the data into the respective image pixels, where the camera parameters Match it to the sonar image pixel coordinates provided by the sonar API. Then Convert to the camera coordinate system and project the data into the camera according to the following formula. In this way, two sensors can be extracted from the particle state: and The pixel coordinates of the point.

[0249] ,

[0250] ,

[0251] By obtaining the sonar pixel coordinates , and tests the pixel against several features, such as the brightness / intensity at that point and the long / short axis of the shape of the object where the pixel is located, to calculate the weight of the particle in the sonar image, and to extract shape information by performing contour analysis, morphology, and principal component analysis-oriented boundary fitting. This is done by calculating the log-likelihood of the particle / state, in which case the measurement of the above features follows a Gaussian distribution. It is calculated by the following formula, where d represents the difference between the measured value and the actual value.

[0252] ,

[0253] ,

[0254] The sonar particle weight is given by:

[0255] ,

[0256] Obtained through By testing pixels for multiple features, such as hue or shape ratio, the optical image particle weight calculation can be completed, which is calculated using the formula:

[0257] ,

[0258] in The intensity corresponds to value, The main axis corresponds to value, The secondary axis corresponds to value, Particle weight probability.

[0259] It is also possible to use the Lucas-Kanade (optical flow) feature tracking method to calculate the pixel velocity and give higher weight to the projected particles that match that velocity, but it was found that using the features described above was sufficient. These weights are finally added together and the algorithm can give more weight to either sensor during the fusion process and normalize them as per step 4. Finally, resampling with replacement is performed as per step 5 and the process is repeated.

[0260] ,

[0261] .

[0262] Specifically, the larger the volume of the corrosion defect, the more obvious the defect signal detected by the leakage magnetic sensor, and the larger the voltage amplitude at the defect. By comparing the positions of the channels where the different signal amplitudes are located and the number of channels with amplitude fluctuations, the location of the defect can also be analyzed. If the leakage magnetic detection signal with the largest amplitude is collected, then the defect is directly below this channel. Use multiple channels for detection and place the detection channels evenly. Compare the measurement range of the optical sensor and the distance of the placed detection channels to get the approximate position of the sensor in the image, and then get the position and size of the defect. Defects of buried pipelines can also be detected.

[0263] Optionally, the identifying and processing the information of the pipeline to be detected to obtain the underwater pipeline detection result includes:

[0264] Performing edge information extraction and target detection on the target acoustic image in the pipeline information to be detected, and performing multi-level feature extraction on the target acoustic image using a deep residual network to obtain low-level features containing edge semantic information and high-level features containing position information;

[0265] Combining the low-level features containing edge semantic information with the high-level features containing position information, extracting edge information, and integrating multi-level backbone features;

[0266] According to the multi-level backbone features, the target acoustic image is gradually aggregated into multi-level fusion features, and converted into a binary image;

[0267] Combined with the activation function, feature extraction is performed on the binary image to obtain target feature information;

[0268] The target feature information is combined with the waveform image and the target optical image for identification to obtain the underwater pipeline detection result.

[0269] Specifically, this embodiment extracts edge information from the target acoustic image (sonar image), which can be achieved through, but not limited to, image processing techniques such as Sobel operator, Canny operator, etc. These operators can identify edges in images and provide a basis for subsequent target detection. Target detection is the process of identifying specific targets in images, which can be achieved through deep learning methods, such as using convolutional neural networks (CNN) for feature learning; using deep residual networks (such as ResNet) to perform multi-level feature extraction on sonar images. The deep residual network solves the gradient disappearance problem in deep network training by introducing jump connections, and can learn low-level features (such as edges, textures) and high-level features (such as position, shape) of the image. Low-level features contain edge semantic information, while high-level features contain the location information of the target; the extracted low-level features and high-level features are combined, and this step can be achieved through feature fusion technology, such as weighted summation of feature maps at different levels or using more complex fusion strategies. In this way, richer edge information can be extracted and multi-level trunk features can be integrated; multi-level fusion features are converted into binary images by step-by-step aggregation. This step can be achieved through image binarization techniques, such as threshold segmentation, maximum stable extreme region (MSER) ​​and other methods. Binary images can clearly distinguish between targets and backgrounds, which is convenient for subsequent processing; combined with activation functions (such as ReLU), binary images are used to extract features to obtain target feature information. Activation functions are used to introduce nonlinearity in neural networks, allowing the model to learn more complex feature representations. Finally, the extracted target feature information is identified, which usually involves the design of classifiers, such as support vector machines (SVMs), random forests, etc., to obtain the final underwater pipeline detection results.

[0270] Specifically, the edge information of the target acoustic image, that is, the edge information of the sonar image in this embodiment is an important information, which can help determine the position and boundary of the target object. By detecting and extracting the edge contour of the target object, the position of the target can be accurately located and the target can be quantitatively analyzed. Edge information can provide key features such as the shape, size and position of the target object to help determine the exact position of the target. The detection network is improved, and an edge guidance module is introduced to achieve efficient and accurate sonar image target detection. The model is divided into an edge information guidance module and an improved YOLOv7 target detection module. For the input image, after multi-level feature extraction through the Res2Net-50 network, the low-level features containing edge semantic information are combined with the high-level features containing position information and sent to the edge information guidance module. The edge guidance module consists of three parts, namely, the edge-aware module (EAM), the edge-guidance feature module (EFM) and the context feature fusion module (CAM). By using the edge-aware module (EAM), the semantic information of the edge of the object in the image is obtained. It extracts the semantic information of the edge of the object from the low-level detail features of the image ( ) and advanced location information features ( ) to extract these edge information. The multi-level backbone features are integrated through the edge-guided feature module (EFM) ) and the context feature fusion module (CAM) to enhance the representation ability of the target boundary. Finally, multiple context feature fusion modules are used to gradually aggregate multi-level fusion features in a top-down manner, and distinguish the target and the background in a black-and-white binary manner. The image after the edge guidance module is converted into a binary image as the input of the target detection model. After multiple convolutional layer feature extraction, Leaky ReLU is used as the activation function to extract feature representations at different levels, and the multi-level splicing module is used to splice the features of different layers together.

[0271] An underwater pipeline detection device provided by an embodiment of the present invention includes:

[0272] An acquisition unit, used for acquiring a waveform image, an initial optical image and an initial acoustic image of the underwater pipeline;

[0273] A processing unit, used for performing denoising and image enhancement processing on the initial optical image to obtain a target optical image, comprising:

[0274] Constructing a denoising model, and performing denoising processing on the initial optical image according to the denoising model to obtain a temporary optical image;

[0275] Performing color correction and contrast stretching processing on the temporary optical image to obtain a first optical sub-image and a second optical sub-image respectively;

[0276] Performing weighted processing and Laplace pyramid decomposition processing on the first optical sub-image and the second optical sub-image to obtain the target optical image;

[0277] Performing image segmentation processing on the initial acoustic image to obtain a target acoustic image;

[0278] fusing the waveform image, the target optical image and the target acoustic image to obtain information of the pipeline to be detected;

[0279] The identification unit is used to identify and process the information of the pipeline to be detected to obtain an underwater pipeline detection result.

[0280] The underwater pipeline detection device of this embodiment is used to implement the underwater pipeline detection method described above. Its advantages over the prior art are the same as the advantages of the underwater pipeline detection method described above over the prior art, and will not be repeated here.

[0281] like Figure 5 As shown, an electronic device 500 provided by an embodiment of the present invention includes a memory 510 and a processor 520; the memory 510 is used to store a computer program; the processor 520 is used to implement the underwater pipeline detection method as described above when executing the computer program.

[0282] In other words, an electronic device 500 includes a memory 510 and a processor 520 coupled to the memory 510; the memory 510 is configured to store a computer program; and the processor 520 is configured to perform the following operations when executing the computer program:

[0283] Acquiring a waveform image, an initial optical image, and an initial acoustic image of the underwater pipeline;

[0284] Performing denoising and image enhancement processing on the initial optical image to obtain a target optical image;

[0285] Performing image segmentation processing on the initial acoustic image to obtain a target acoustic image;

[0286] fusing the waveform image, the target optical image and the target acoustic image to obtain information of the pipeline to be detected;

[0287] The information of the pipeline to be detected is identified and processed to obtain an underwater pipeline detection result.

[0288] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the underwater pipeline detection method as described above is implemented.

[0289] In other words, a non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor performs the following operations:

[0290] Acquiring a waveform image, an initial optical image, and an initial acoustic image of the underwater pipeline;

[0291] Performing denoising and image enhancement processing on the initial optical image to obtain a target optical image;

[0292] Performing image segmentation processing on the initial acoustic image to obtain a target acoustic image;

[0293] fusing the waveform image, the target optical image and the target acoustic image to obtain information of the pipeline to be detected;

[0294] The information of the pipeline to be detected is identified and processed to obtain an underwater pipeline detection result.

[0295] An electronic device 500 that can be used as a server or client of the present invention will now be described, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device 500 is intended to represent various forms of digital electronic computer equipment, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 500 can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the present invention described and / or required herein.

[0296] The electronic device 500 includes a computing unit, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) or a computer program loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for the operation of the device can also be stored. The computing unit, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0297] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc. In the present application, the unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment of the present invention. In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0298] Although the present invention is disclosed as above, the protection scope of the present invention is not limited thereto. Those skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will fall within the protection scope of the present invention.

Claims

1. A method for underwater pipeline detection, characterized in that: include: Acquiring a waveform image, an initial optical image, and an initial acoustic image of the underwater pipeline, wherein acquiring the waveform image of the underwater pipeline includes: Magnetizing the underwater pipeline and performing leakage magnetic field detection to obtain a three-dimensional graph of leakage magnetic field data; Obtaining the waveform image according to the three-dimensional graph of the leakage magnetic field data; The initial optical image is subjected to denoising and image enhancement processing to obtain a target optical image, including: Constructing a denoising model, and performing denoising processing on the initial optical image according to the denoising model to obtain a temporary optical image; Performing color correction and contrast stretching processing on the temporary optical image to obtain a first optical sub-image and a second optical sub-image respectively; Performing weighted processing and Laplace pyramid decomposition processing on the first optical sub-image and the second optical sub-image to obtain the target optical image; Performing image segmentation processing on the initial acoustic image to obtain a target acoustic image; fusing the waveform image, the target optical image and the target acoustic image to obtain information of the pipeline to be detected; The information of the pipeline to be detected is identified and processed to obtain an underwater pipeline detection result.

2. The underwater pipeline detection method according to claim 1, characterized in that: The construction process of the denoising model includes: Get the initial denoising model generator; Removing the batch normalization layer of the initial denoising model generator and adding two layers of residual structure to obtain a temporary denoising model; The temporary denoising model is tuned according to the loss function to obtain the denoising model.

3. The underwater pipeline detection method according to claim 2, characterized in that: The step of performing weighted processing and Laplace pyramid decomposition processing on the first optical sub-image and the second optical sub-image to obtain the target optical image includes: Obtaining a local entropy weight map and a gradient weight map according to the first optical sub-image and the second optical sub-image; Adding the local entropy weight map and the gradient weight map to obtain a normalized weight map; According to the Laplacian pyramid decomposition, the normalized weight map, the first optical sub-image and the second optical sub-image are fused to obtain the target optical image.

4. The underwater pipeline detection method according to claim 1, characterized in that: The performing image segmentation processing on the initial acoustic image to obtain a target acoustic image includes: Marking an observation area in the initial acoustic image to obtain a random field, wherein the random field includes pixel values ​​and position information of the initial acoustic image; Modeling is performed according to the pixel values ​​and the position information to obtain a shadow area, an underwater area, and a target area; Obtaining a joint probability distribution according to the shadow area, the bottom area and the target area; According to the joint probability distribution, a maximum posterior probability is obtained; The initial acoustic image is subjected to image segmentation processing according to the maximum a posteriori probability to obtain the target acoustic image.

5. The underwater pipeline detection method according to claim 4, characterized in that: The fusing the waveform image, the target optical image and the target acoustic image to obtain the pipeline information to be detected includes: The waveform image, the target optical image and the target acoustic image are subjected to a particle filtering algorithm to obtain a group of initial particles to represent the initial position and state of the defect target in the underwater pipeline; Constructing a dynamic model and obtaining process noise, updating the state of the initial particle according to the dynamic model, and adding the process noise to obtain a temporary particle; Obtaining a preset ratio, and evenly distributing the temporary particles according to the preset ratio; Constructing a measurement model and obtaining a measurement value, calculating the weight of the temporary particle according to the measurement value and normalizing it to obtain a temporary weight, wherein the measurement value is obtained according to the current initial optical image and the initial acoustic image; Resampling is performed according to the temporary weight to obtain a newly generated particle, and the newly generated particle is used as the current initial particle, and the above steps are repeated to obtain a target weight; According to the target weight, the waveform image, the target optical image and the target acoustic image are fused to obtain the information of the pipeline to be detected.

6. The underwater pipeline detection method according to claim 5, characterized in that: The step of identifying and processing the pipeline information to be detected to obtain underwater pipeline detection results includes: Performing edge information extraction and target detection on the target acoustic image in the pipeline information to be detected, and performing multi-level feature extraction on the target acoustic image using a deep residual network to obtain low-level features containing edge semantic information and high-level features containing position information; Combining the low-level features containing edge semantic information with the high-level features containing position information, extracting edge information, and integrating multi-level backbone features; According to the multi-level backbone features, the target acoustic image is gradually aggregated into multi-level fusion features, and converted into a binary image; Combined with the activation function, feature extraction is performed on the binary image to obtain target feature information; The target feature information is combined with the waveform image and the target optical image for identification to obtain the underwater pipeline detection result.

7. An underwater pipeline detection device, characterized in that: include: The acquisition unit is used to acquire a waveform image, an initial optical image and an initial acoustic image of the underwater pipeline, wherein acquiring the waveform image of the underwater pipeline includes: Magnetizing the underwater pipeline and performing leakage magnetic field detection to obtain a three-dimensional graph of leakage magnetic field data; Obtaining the waveform image according to the three-dimensional graph of the leakage magnetic field data; A processing unit, used for performing denoising and image enhancement processing on the initial optical image to obtain a target optical image, comprising: Constructing a denoising model, and performing denoising processing on the initial optical image according to the denoising model to obtain a temporary optical image; Performing color correction and contrast stretching processing on the temporary optical image to obtain a first optical sub-image and a second optical sub-image respectively; Performing weighted processing and Laplace pyramid decomposition processing on the first optical sub-image and the second optical sub-image to obtain the target optical image; Performing image segmentation processing on the initial acoustic image to obtain a target acoustic image; fusing the waveform image, the target optical image and the target acoustic image to obtain information of the pipeline to be detected; The identification unit is used to identify and process the information of the pipeline to be detected to obtain an underwater pipeline detection result.

8. An electronic device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to implement the underwater pipeline detection method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the underwater pipeline detection method according to any one of claims 1 to 6 is implemented.

Images