A pipe rubber lining defect identification and review method based on deep learning

Abstract

This invention discloses a deep learning-based method for identifying and verifying defects in pipe linings, comprising the following steps: acquiring and generating a multimodal dataset, establishing a spatially perceptual coordinate system; mapping the multimodal dataset to a cylindrical parameter domain, and simultaneously establishing an initial Sheaf graph structure; running the YOLOv11s-seg algorithm for defect detection and instance segmentation; constructing a Bayesian Sheaf neural network on the initial Sheaf graph structure, performing consistency propagation and uncertainty estimation to obtain a consistent defect set; performing local rotation scanning and axial micro-movement resampling at the locations of the consistent defect set to generate a verification dataset; fusing the verification dataset and the consistent defect set to complete 3D reconstruction and dimension calculation; writing the data into a database, completing pose consistency comparison and archiving, and outputting the final detection result set. This invention improves the accuracy of rubber lining defect identification and verification consistency.

Description

Technical Field

[0001] This invention relates to the field of pipeline inspection technology, and in particular to a method for identifying and verifying pipeline rubber lining defects based on deep learning. Background Technology

[0002] During long-term operation, rubber linings in pipelines are susceptible to corrosion from water flow, particle erosion, and biofouling, leading to defects such as bulging, delamination, cracking, and localized peeling. Failure to detect and address these defects promptly will directly impact the pipeline's corrosion resistance and structural safety. Currently, commonly used inspection methods include manual endoscopic inspection, ultrasonic testing, and video surveillance.

[0003] Traditional manual inspection relies on the subjective experience of operators, resulting in problems such as low recognition accuracy, poor verification consistency, and high false negative rate. Although image processing-based visual inspection methods have achieved automated recognition, under complex lighting conditions, curved surface reflections, and multi-scale defects in pipelines, the detection model is prone to problems such as feature blurring, boundary misjudgment, and lack of depth information. Point cloud detection technology can provide geometric shape information, but its registration accuracy with image features is limited, making it difficult to achieve spatial consistency verification.

[0004] Furthermore, existing deep learning detection methods are mostly trained on planar scenes, lacking topological modeling and spatial alignment mechanisms for unfolded images of cylindrical pipes, and failing to fully utilize the geometric relationships of multimodal data. Uncertainty estimation and verification decisions also lack a closed loop, resulting in some areas with ambiguous boundaries being unable to be verified through automated resampling. Existing pipe lining inspection technologies generally suffer from insufficient recognition accuracy, inconsistent spatial verification, and a disconnect between inspection equipment and algorithms. There is an urgent need for an intelligent detection and verification method that integrates multimodal data and possesses spatial topological constraints and uncertainty self-correction capabilities to improve the reliability and automation level of lining defect identification and verification.

[0005] Therefore, how to provide a deep learning-based method for identifying and verifying defects in pipe lining is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a deep learning-based method for identifying and verifying defects in pipe linings. This invention introduces a Bayesian Sheaf neural network structure, fusing panoramic images, point clouds, and pose data to construct a Sheaf graph topology model in the cylindrical parameter domain, enabling uncertainty estimation and consistency propagation of defect detection results. Through multimodal data fusion and 3D reconstruction mechanisms, combined with intelligent re-sampling using mobile and traction-lifting detection equipment, automatic identification, spatial verification, and dimensional quantification of lining defects are achieved. This method exhibits higher identification accuracy and verification reliability in complex curved surfaces and high-reflectivity environments, effectively improving the intelligence and precision of pipe lining inspection.

[0007] A method for identifying and verifying defects in pipe lining rubber based on deep learning according to an embodiment of the present invention includes the following steps:

[0008] The system acquires raw inspection data of the inner wall of the pipeline, performs rotational scanning, establishes a spatial sensing coordinate system, and simultaneously forms a multimodal data set.

[0009] The multimodal dataset is mapped to the cylindrical parameter domain to generate a cylindrical unfolded image. Based on the cylindrical unfolded image and the spatial perception coordinate system, a topological adjacency relationship is constructed to establish the initial structure of the Sheaf graph.

[0010] Based on the cylindrical unfolded image, the YOLOv11s-seg algorithm is used to perform defect detection and instance segmentation to obtain defect mask and defect feature map, which are then fused with point cloud data to generate a three-dimensional defect candidate set.

[0011] A Bayesian Sheaf neural network is constructed on the initial structure of the Sheaf graph. The defect feature map, defect mask and topological adjacency relationship are input, consistency propagation and uncertainty estimation are performed, and the defect confidence distribution, uncertainty index and boundary consistency results are output. The three-dimensional defect candidate set is updated to obtain the consistent defect set.

[0012] Perform local rotational scanning and axial micro-movement re-sampling at the corresponding locations of the consistency defect set, and then transmit the verification dataset back and merge it into a spatial sensing coordinate system.

[0013] By integrating the verification data set and the consistency defect set, the three-dimensional reconstruction and size calculation are completed, and the three-dimensional defect model set and defect parameter table are output.

[0014] Write the set of 3D defect models and the defect parameter table into the management database, complete the pose consistency comparison with the historical records and the entry archiving, and output the final detection result set.

[0015] Optionally, the generation of the multimodal data set and the spatially aware coordinate system includes:

[0016] Start the depth camera and rotating gimbal, load the unfolding and rotating scanning parameters, set the circumferential step size and axial step size, and enter continuous acquisition mode;

[0017] Acquire a sequence of circular scan images and simultaneously record pose data, then bind each frame of the circular scan sequence image with the corresponding pose data at the frame level;

[0018] Point cloud data is generated by distance measurement using a depth camera and synchronized with pose data. Invalid and isolated points are removed before the point cloud data is output.

[0019] Based on the pose data, the coordinate transformation relationship from the device coordinates to the spatial perception coordinate system is calculated, and the circumferential scan sequence images and point cloud data are mapped to the spatial perception coordinate system and aligned.

[0020] The panoramic scan sequence images, point cloud data, and pose data are packaged and registered in the spatial perception coordinate system according to the frame-level binding relationship to form a multimodal data set.

[0021] Optionally, the construction of the initial structure of the Sheaf graph includes:

[0022] In the spatial perception coordinate system, the multimodal data set is registered and time-synchronized, and the point cloud data and pose data corresponding to the ring scan sequence images are preserved;

[0023] Perform cylindrical parameter domain mapping to convert the circumferential scan sequence images and point cloud data to the cylindrical parameter domain, generate the cylindrical unfolded image and maintain the coordinate correspondence with the spatial perception coordinate system;

[0024] Adjacency rules are established based on the cylindrical unfolded image and the spatial perception coordinate system. The connection relationship between data units is calculated according to the angular adjacency and axial adjacency criteria, and a topological adjacency relationship is constructed.

[0025] An initial structure of a Sheaf graph is created on the cylindrical parameter domain based on topological adjacency relationships. The nodes and edges of the initial structure of the Sheaf graph are defined and aligned with a spatially aware coordinate system.

[0026] Optionally, the generation of the three-dimensional defect candidate set includes:

[0027] Load the YOLOv11s-seg algorithm configuration file, set the input size, confidence threshold and segmentation threshold, input the cylinder unfolded image into the YOLOv11s-seg algorithm and initialize the detection backbone;

[0028] Run the YOLOv11s-seg algorithm to perform feature extraction and multi-scale fusion, generate a set of feature maps containing the response of the defect region, and output the candidate set of bounding boxes, classification probability distribution and instance segmentation results;

[0029] Extract the defect mask from the instance segmentation results, generate the defect feature map by combining the feature map set, and establish a corresponding mask and feature pairing relationship for each defect target;

[0030] Based on the mapping relationship between the cylindrical parameter domain and the spatial perception coordinate system, the defect mask and the defect feature map are aligned to the spatial perception coordinate system;

[0031] The aligned defect mask and defect feature map are fused with point cloud data, and a 3D defect candidate set is generated using depth information and boundary matching constraints.

[0032] Optionally, the generation of the consistency defect set includes:

[0033] A Bayesian Sheaf neural network is constructed on the initial structure of the Sheaf graph. The set of nodes and the set of edges are defined, the topological adjacency relationship is mapped into a structured dependency matrix, and local Sheaf units and adjacency constraints are established.

[0034] The defect mask and defect feature map are respectively input into the node encoding layer and edge association layer of the Bayesian Sheaf neural network. Feature normalization and edge weight initialization are performed. The structured dependency matrix is ​​called to determine the edge constraint weights and neighborhood propagation direction. The input features of each node are represented as node observation vectors, and the features of each edge are represented as edge constraint vectors.

[0035] The consistency propagation process on the Sheaf structure is executed. The multi-layer message passing results are calculated based on the node observation vector and the edge constraint vector. The posterior distribution of node features is updated in each layer to form a feature propagation path with Bayesian uncertainty estimation.

[0036] Based on the feature propagation path, posterior inference is performed on the node distribution results output by the Bayesian Sheaf neural network to generate defect confidence distribution and uncertainty index, and boundary consistency results are generated based on the edge-level consistency calculation results.

[0037] The three-dimensional defect candidate set is updated based on the defect confidence distribution, uncertainty index and boundary consistency results, and a consistent defect set is generated and bound to the spatial sensing coordinate system.

[0038] Optionally, the generation of the review data set includes:

[0039] Based on the uncertainty index and the boundary consistency results, the set of review targets is determined from the set of consistency defects. A set of review instructions is generated and the target position, circumferential step distance, axial step distance, number of re-sampling frames and exposure time are marked.

[0040] The set of verification instructions is sent to the mobile intelligent equipment for inspecting rubber-lined pipes and the traction-lifting intelligent equipment for inspecting rubber-lined pipes. Based on the set of verification instructions, the attitude alignment and position correction are completed in the spatial perception coordinate system.

[0041] Perform local rotational scanning and axial micro-movement re-acquisition to acquire and verify ring scan images, verification point clouds and pose data, and record the acquisition sequence in the time order of the spatial perception coordinate system;

[0042] Frame-level binding, noise removal, and registration operations are performed on the verification ring scan image, verification point cloud, and pose data to generate a verification dataset that is then merged and mapped to a spatial perception coordinate system.

[0043] Establish a correspondence between the set of reviewed data and the set of consistency defects, and then return it to the process.

[0044] Optionally, the generation of the three-dimensional defect model set and defect parameter table includes:

[0045] Spatially align and temporally register the verification data set and the consistency defect set, and perform point cloud registration and image projection mapping based on the pose information of the spatial perception coordinate system to construct the verification data fusion body;

[0046] Based on the verification data fusion, feature correspondence and depth interpolation are performed. The verification ring scan image, verification point cloud and defect feature map are fused to generate a unified three-dimensional point cloud data set. The defect boundary area is refined and resampled to form a standardized data structure for three-dimensional reconstruction.

[0047] The 3D reconstruction module is invoked to perform spatial surface fitting and topological reconstruction on the standardized data structure, generating a set of 3D defect models, and calculating the spatial dimension parameters, volume parameters and boundary curvature parameters of each defect model to generate a defect parameter table;

[0048] The set of 3D defect models and the defect parameter table are mapped to a spatially perceptual coordinate system. Pose calibration and global index numbering are performed on each defect model to complete its positioning and labeling in the spatially perceptual coordinate system.

[0049] Optionally, the generation of the final detection result set includes:

[0050] A data writing mapping relationship is established based on the spatial perception coordinate system and the time index. The data structure processing of the three-dimensional defect model set and defect parameter table is carried out to generate the data object to be written.

[0051] Call the management database interface module to write the data objects to be written into the database in time index order, establish the association between the time index field and the pose field, and complete the database entry operation of the 3D defect model set and defect parameter table.

[0052] Based on the historical data in the management database, a pose consistency comparison is performed. Spatial registration and difference calculation are carried out between the current set of 3D defect models and the historical set of 3D defect models to generate a pose deviation matrix and a set of difference reports.

[0053] Bind the pose deviation matrix and the difference report set to the time index of the current detection task, update the database entry status and version number, and complete the entry archiving;

[0054] Output the final set of detection results based on the item archiving results.

[0055] The beneficial effects of this invention are:

[0056] First, this invention constructs an initial structure of the Sheaf diagram under the cylindrical parameter domain, and uniformly maps the circumferential scan image, point cloud data and pose information, so that the geometric features and spatial relationships of the rubber lining defect are consistently expressed, thereby significantly improving the stability of defect identification and spatial alignment accuracy.

[0057] Secondly, this invention utilizes the probabilistic reasoning capability of the Bayesian Sheaf neural network to achieve uncertainty estimation and consistency propagation during the defect identification process. It can automatically identify fuzzy boundaries and low-confidence regions and generate verification instructions to guide the detection equipment to perform local high-density resampling, effectively reducing the probability of false detection and missed detection.

[0058] Furthermore, this invention reconstructs and calculates the dimensions of a three-dimensional defect model by fusing a verification data set and a consistency defect set. It also uses a spatially perceptual coordinate system to compare the pose consistency with historical data, resulting in a fully traceable detection result. This method exhibits higher recognition accuracy and verification consistency in environments with complex pipeline surfaces and high humidity / reflective conditions, significantly improving the intelligence, reliability, and automation level of pipeline rubber lining defect detection. Attached Figure Description

[0059] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0060] Figure 1 This is an overall flowchart of a deep learning-based method for identifying and verifying defects in pipe lining rubber proposed in this invention.

[0061] Figure 2 This is a schematic diagram of the consistency propagation and uncertainty estimation of the Bayesian Sheaf neural network on the initial structure of the Sheaf graph in this invention;

[0062] Figure 3 This is a schematic diagram of the three-dimensional defect verification and reconstruction structure based on multimodal fusion in this invention. Detailed Implementation

[0063] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0064] refer to Figures 1-3 A deep learning-based method for identifying and verifying defects in pipe rubber lining includes the following steps:

[0065] The system acquires raw inspection data of the inner wall of the pipeline, including circumferential scan sequence images, point cloud data and pose data. It performs unfolding and rotational scanning based on a depth camera and a rotating gimbal to establish a spatial perception coordinate system and form a multimodal data set.

[0066] The multimodal dataset is mapped to the cylindrical parameter domain to generate a cylindrical unfolded image. Based on the cylindrical unfolded image and the spatial perception coordinate system, a topological adjacency relationship is constructed to establish the initial structure of the Sheaf graph. The cylindrical unfolded image and the initial structure of the Sheaf graph are then output.

[0067] Based on the cylindrical unfolded image, the YOLOv11s-seg algorithm is run to perform defect detection and instance segmentation, and defect masks and defect feature maps are obtained. The defect masks and defect feature maps are aligned to the spatial perception coordinate system and fused with point cloud data to generate a three-dimensional defect candidate set.

[0068] A Bayesian Sheaf neural network is constructed on the initial structure of the Sheaf graph. The defect feature map, defect mask and topological adjacency relationship are input, consistency propagation and uncertainty estimation are performed, and the defect confidence distribution, uncertainty index and boundary consistency results are output. The three-dimensional defect candidate set is updated to obtain the consistent defect set.

[0069] Based on the uncertainty index and the boundary consistency results, a set of verification instructions is generated. The mobile intelligent equipment for detecting rubber-lined pipes and the traction lifting intelligent equipment for detecting rubber-lined pipes are scheduled to perform local rotation scanning and axial micro-movement re-sampling at the corresponding positions of the consistency defect set. The verification dataset is then transmitted back and mapped to the spatial sensing coordinate system.

[0070] By integrating the verification data set and the consistency defect set, the three-dimensional reconstruction and size calculation are completed, and the three-dimensional defect model set and defect parameter table are output. The three-dimensional defect model set and defect parameter table are then labeled in the spatial perception coordinate system.

[0071] Based on the spatial perception coordinate system and time index, the three-dimensional defect model set and defect parameter table are written into the management database, and the pose consistency comparison with the historical records and the entry archiving are completed, and the final detection result set is output.

[0072] In this embodiment, the generation of the multimodal data set and the spatially aware coordinate system includes:

[0073] Start the depth camera and rotating gimbal, load the unfolding and rotating scanning parameters, set the circumferential step size and axial step size, and enter continuous acquisition mode;

[0074] The depth camera is an imaging device used to simultaneously acquire images and depth information of the inner wall of a pipe. It has active structured light and passive stereo ranging capabilities, and can output a sequence of circumferential scan images with a depth channel in low-light and high-reflection environments. Its output data includes grayscale images, depth maps, and camera intrinsic and extrinsic parameter matrices, which are used to generate point cloud data. The rotating gimbal is a two-degree-of-freedom rotary actuator installed at the front end of the inspection equipment. It controls the rotation angle and attitude stability around the pipe axis through a built-in servo motor, realizing the circumferential rotational scanning and pitch angle adjustment of the depth camera. Its angle step and rotation speed can be precisely controlled according to the set values ​​in the unfolding rotational scanning parameters. The unfolding rotational scanning parameters are a set of control parameters used to define the unfolding scanning path of the cylindrical curved surface, including circumferential angle step, axial step, scanning frame rate, exposure time, and synchronous trigger delay, etc., which are used to coordinate the matching relationship between the sampling frequency of the depth camera and the angle step of the rotating gimbal, so that the depth camera completes one frame of circumferential scanning acquisition at each fixed angle position, thereby realizing continuous unfolding full-coverage imaging of the inner wall of the pipe.

[0075] Acquire a sequence of circular scan images and simultaneously record pose data, then bind each frame of the circular scan sequence image with the corresponding pose data at the frame level;

[0076] Point cloud data is generated by distance measurement using a depth camera and synchronized with pose data. Invalid and isolated points are removed before the point cloud data is output.

[0077] Based on the pose data, the coordinate transformation relationship from the device coordinates to the spatial perception coordinate system is calculated, and the circumferential scan sequence images and point cloud data are mapped to the spatial perception coordinate system and aligned.

[0078] The panoramic scan sequence images, point cloud data, and pose data are packaged and registered in the spatial perception coordinate system according to the frame-level binding relationship to form a multimodal data set.

[0079] In this embodiment, the construction of the initial structure of the Sheaf graph includes:

[0080] In the spatial perception coordinate system, the multimodal data set is registered and time-synchronized, and the point cloud data and pose data corresponding to the ring scan sequence images are preserved;

[0081] Perform cylindrical parameter domain mapping to convert the circumferential scan sequence images and point cloud data to the cylindrical parameter domain, generate the cylindrical unfolded image and maintain the coordinate correspondence with the spatial perception coordinate system;

[0082] During the mapping process, the radial invariance and axial linear mapping principle of the cylindrical surface are maintained. Distance-preserving mapping is achieved through the geometric constraint function from polar coordinates to Cartesian coordinates, ensuring the spatial scale consistency between the ring scan sequence images and point cloud data after unfolding.

[0083] Adjacency rules are established based on the cylindrical unfolded image and the spatial perception coordinate system. The connection relationship between data units is calculated according to the angular adjacency and axial adjacency criteria, and a topological adjacency relationship is constructed.

[0084] When calculating topological adjacency relationships, an angle difference-based approach is used. axial distance Weighted adjacency function:

[0085] ;

[0086] in, This represents the adjacency weight between nodes i and j. and These represent the angular and axial smoothing parameters, respectively, used to control the local topological coupling strength, thereby forming a continuous topological coupling structure within the cylindrical parameter domain;

[0087] An initial structure of a Sheaf graph is created on the cylindrical parameter domain based on topological adjacency relationships. The nodes and edges of the initial structure of the Sheaf graph are defined and aligned with the spatially aware coordinate system.

[0088] During the construction process, a local function space is defined for each node to store the local feature vectors of the cylinder unfolded image and point cloud data, and a constraint mapping is established between adjacent nodes to describe the transition constraints of the features between nodes in the local cross-sectional space, thereby forming a Sheaf structure with topological consistency propagation capability.

[0089] The initial set of nodes, edges, and constraint mappings of the Sheaf graph together constitute the input structure of the subsequent Bayesian Sheaf neural network, providing a structured topological dependency basis for subsequent consistency propagation and uncertainty estimation.

[0090] Output the unfolded image of the cylinder and the initial structure of the Sheaf diagram, and record the correspondence with the spatially perceived coordinate system.

[0091] In this embodiment, the generation of the three-dimensional defect candidate set includes:

[0092] Load the YOLOv11s-seg algorithm configuration file, set the input size, confidence threshold and segmentation threshold, input the cylinder unfolded image into the YOLOv11s-seg algorithm and initialize the detection backbone;

[0093] Among them, the YOLOv11s-seg algorithm, "s" means small version, with fewer parameters and faster speed, suitable for embedded deployment, and "seg" means that this version integrates the instance segmentation function, which can output the target bounding box and pixel-level segmentation mask at the same time.

[0094] The YOLOv11s-seg algorithm is a real-time object detection and instance segmentation algorithm based on deep learning. It is an improved version of the YOLO series network. This algorithm adds a segmentation branch to the YOLOv11 structure, realizing simultaneous inference of defect detection and pixel-level segmentation. The algorithm consists of a backbone network, a Neck feature fusion layer, and a Head detection branch. The backbone adopts an improved CSPNet structure to extract multi-scale visual features and maintain gradient stability. The Neck layer fuses FPN and PAN structures to enhance the information transmission between features of different scales and improve the ability to identify small defects and blurred edge regions. The Head branch includes a detection output layer and a segmentation output layer. The detection layer is responsible for generating defect category, confidence, and bounding box information, while the segmentation layer generates the corresponding pixel-level mask.

[0095] In actual detection, the YOLOv11s-seg algorithm receives the unfolded image of the cylinder as input and outputs the spatial boundary and classification results of the defect area through multi-scale feature fusion and end-to-end prediction. To adapt to the uneven lighting, complex texture and curved surface reflection of the pipe lining surface, the algorithm training process introduces Mosaic enhancement, soft label learning and multi-scale input strategies, thereby significantly improving the accuracy of defect identification and boundary segmentation while ensuring detection speed.

[0096] Run the YOLOv11s-seg algorithm to perform feature extraction and multi-scale fusion, generate a set of feature maps containing the response of the defect region, and output the candidate set of bounding boxes, classification probability distribution and instance segmentation results;

[0097] Extract the defect mask from the instance segmentation results, generate the defect feature map by combining the feature map set, and establish a corresponding mask and feature pairing relationship for each defect target;

[0098] The process of extracting defect masks is as follows: Class filtering and confidence ranking are performed on the instance segmentation results output by the YOLOv11s-seg algorithm. Candidate targets with confidence scores below a threshold are removed. Binary segmentation maps corresponding to each defect target are extracted from the filtering results. Boundary regions are corrected through morphological closing operations and connected component analysis to obtain a set of defect masks with continuous boundaries and noise reduction. The process of generating defect feature maps is as follows: Based on the positional mapping relationship of each defect mask in the feature map set, convolutional feature blocks of the corresponding feature layers are extracted. Spatial pooling and channel-weighted fusion are performed on the convolutional feature blocks to generate local defect feature descriptions. The fusion results of multi-scale feature layers are standardized to output a defect feature map containing defect texture features, boundary gradient features, and intensity response features. This map is then bound to the corresponding defect mask to form a mask-feature pairing set.

[0099] Based on the mapping relationship between the cylindrical parameter domain and the spatial perception coordinate system, the defect mask and the defect feature map are aligned to the spatial perception coordinate system;

[0100] The aligned defect mask and defect feature map are fused with point cloud data. A three-dimensional defect candidate set is generated using depth information and boundary matching constraints. The three-dimensional defect candidate set is then bound to a spatial perception coordinate system, and the defect mask and defect feature map are output.

[0101] In this embodiment, the generation of the consistency defect set includes:

[0102] A Bayesian Sheaf neural network is constructed on the initial structure of the Sheaf graph. The set of nodes and the set of edges are defined, the topological adjacency relationship is mapped into a structured dependency matrix, and local Sheaf units and adjacency constraints are established.

[0103] When establishing local Sheaf units, a Gaussian prior distribution is defined for the feature variables of each node, and prior parameters are passed between adjacent nodes through constraint mapping to realize the probability constraint of the local function space. The prior distribution of each node forms a joint probability graph structure with the edge-level adjacency relationship under the control of the structured dependency matrix, providing the initial state for subsequent Bayesian propagation.

[0104] The defect mask and defect feature map are respectively input into the node encoding layer and edge association layer of the Bayesian Sheaf neural network. Feature normalization and edge weight initialization are performed. The structured dependency matrix is ​​called to determine the edge constraint weights and neighborhood propagation direction. The input features of each node are represented as node observation vectors, and the features of each edge are represented as edge constraint vectors.

[0105] A random perturbation term is introduced during the node encoding process to simulate sampling uncertainty, so that the node observation vector contains both feature mean and variance information, thereby maintaining the random inference characteristics of the Bayesian layer.

[0106] The consistency propagation process on the Sheaf structure is executed. The multi-layer message passing results are calculated based on the node observation vector and the edge constraint vector. The posterior distribution of node features is updated in each layer to form a feature propagation path with Bayesian uncertainty estimation.

[0107] During the propagation process, a variational Bayesian update mechanism is adopted to calculate new posterior parameters based on the posterior distribution of the previous layer nodes and the edge constraint weights. By restricting the mapping, the distribution consistency between nodes is maintained, thereby realizing probability propagation under topological constraints.

[0108] Based on the feature propagation path, posterior inference is performed on the node distribution results output by the Bayesian Sheaf neural network to generate defect confidence distribution and uncertainty index, and boundary consistency results are generated based on the edge-level consistency calculation results.

[0109] In the posterior inference process, the mean of the posterior distribution of nodes is used as the confidence center, the trace of the covariance is used as the uncertainty measure, and the boundary consistency score is calculated through the edge constraint residual to achieve probability-based boundary fusion and reliability quantification.

[0110] The three-dimensional defect candidate set is updated based on the defect confidence distribution, uncertainty index and boundary consistency results. Defect targets with low confidence or boundary conflicts are eliminated or merged to generate a consistent defect set and bind it to the spatial perception coordinate system.

[0111] During the update process, the a posteriori variance is used as the confidence adjustment factor, and the threshold for retaining or removing defect targets is determined by Bayesian confidence intervals to ensure that the output consistent defect set has both high confidence and structural consistency.

[0112] In this embodiment, the generation of the review data set includes:

[0113] Based on the uncertainty index and the boundary consistency results, the set of review targets is determined from the set of consistency defects. A set of review instructions is generated and the target position, circumferential step distance, axial step distance, number of re-sampling frames and exposure time are marked.

[0114] When generating the set of review instructions, the uncertainty index of each defect target is calculated. Consistency score with boundaries Based on the weighted fusion function:

[0115] ;

[0116] Determine the priority of re-mining, among which Indicates the review priority of the defective target. and These are weighting coefficients used to balance uncertainty and consistency; according to The value is dynamically adjusted to adjust the circumferential angle step, axial step and number of re-sampling frames to achieve adaptive allocation of verification density and form a dynamic verification scheduling strategy based on uncertainty.

[0117] The set of verification instructions is sent to the mobile intelligent equipment for inspecting rubber-lined pipes and the traction-lifting intelligent equipment for inspecting rubber-lined pipes. Based on the set of verification instructions, the attitude alignment and position correction are completed in the spatial perception coordinate system.

[0118] The mobile intelligent pipeline lining inspection equipment is a self-driven inspection unit used for autonomous mobile inspection and local verification of pipelines. The equipment consists of a self-driven chassis module, an inspection execution unit, a rotating gimbal, and a control and communication module. The self-driven chassis module adopts multi-wheel differential drive and an active centering mechanism to achieve smooth propulsion and posture stability within the pipe. The inspection execution unit integrates a depth camera, a light source array, and a folding rotating gimbal, which can perform circumferential scanning and axial micro-movement re-sampling in horizontal or inclined pipe sections. The control and communication module is responsible for receiving the set of verification instructions and adjusting the running trajectory according to the pose information in the spatial perception coordinate system to achieve automatic positioning and posture correction, and complete local high-density re-sampling of the area corresponding to the set of consistency defects.

[0119] The traction-lift intelligent equipment for inspecting rubber-lined pipes is a traction-lifting inspection unit used for the inspection and verification of vertical or high-angle pipe sections. The equipment consists of a traction drive system, an inspection carrier module, and an attitude control system. The traction drive system controls the lifting and lowering movement of the inspection carrier within the pipe through an electric winch mechanism and a tension feedback device. The inspection carrier module is equipped with a depth camera and a rotating gimbal at its front end, enabling it to perform vertical rotational scanning and fixed-point hovering re-sampling. The attitude control system utilizes attitude sensors and a laser ranging unit to achieve height positioning and attitude stabilization. Based on the set of verification instructions, the equipment completes vertical positioning and rotational verification scanning, and corrects the inspection path using the positional information of the spatial perception coordinate system, achieving accurate re-sampling and data transmission of defect areas in vertical sections.

[0120] Perform local rotational scanning and axial micro-movement re-acquisition to acquire and verify ring scan images, verification point clouds and pose data, and record the acquisition sequence in the time order of the spatial perception coordinate system;

[0121] During the re-sampling process, the scanning speed and the number of re-sampling frames are adjusted based on real-time pose feedback and uncertainty change rate, forming a self-adjusting re-sampling mechanism based on uncertainty change.

[0122] Frame-level binding, noise removal and registration operations are performed on the verification ring scan image, verification point cloud and pose data to generate a verification dataset and merge it to the spatial perception coordinate system. The verification dataset contains real-time confidence records and verification path parameters.

[0123] The data set for review is correlated with the set of consistency defects and fed back to the workflow to provide input data for 3D reconstruction and size calculation. The confidence level improvement of each review task is recorded to form a review-feedback closed loop, providing a basis for dynamic adjustment for subsequent inspection tasks.

[0124] In this embodiment, the generation of the three-dimensional defect model set and defect parameter table includes:

[0125] Spatially align and temporally register the verification data set and the consistency defect set, and perform point cloud registration and image projection mapping based on the pose information of the spatial perception coordinate system to construct the verification data fusion body;

[0126] Based on the verification data fusion, feature correspondence and depth interpolation are performed. The verification ring scan image, verification point cloud and defect feature map are fused to generate a unified three-dimensional point cloud data set. The defect boundary area is refined and resampled to form a standardized data structure for three-dimensional reconstruction.

[0127] The 3D reconstruction module is invoked to perform spatial surface fitting and topological reconstruction on the standardized data structure, generating a set of 3D defect models, and calculating the spatial dimension parameters, volume parameters and boundary curvature parameters of each defect model to generate a defect parameter table;

[0128] The 3D reconstruction module is specifically a spatial geometric reconstruction unit based on the fusion of point cloud data and image features. It is used to transform standardized data structures into continuous 3D surface models. This module consists of a point cloud registration submodule, a surface reconstruction submodule, a geometric parameter calculation submodule, and a topology optimization submodule.

[0129] The point cloud registration submodule performs pose correction and global registration on multi-source point clouds from the verification data fusion body based on the pose information of the spatial perception coordinate system, forming a fused point cloud set with spatial consistency; the surface reconstruction submodule performs spatial surface fitting using the fused point cloud set, generating a continuous surface mesh structure through voxel interpolation and surface normal estimation; the geometric parameter calculation submodule calculates the spatial size parameters, volume parameters, and boundary curvature parameters of each defect model based on the spatial discrete data of the surface mesh; the topology optimization submodule performs noise constraint and hole filling on the surface mesh to maintain the boundary continuity and geometric accuracy of the defect region. The three-dimensional defect model set and defect parameter table output by this module are used for subsequent positioning annotation and historical comparison.

[0130] The three-dimensional defect model set and defect parameter table are mapped to the spatial perception coordinate system. The pose calibration and global index number are performed on each defect model to complete the positioning and annotation in the spatial perception coordinate system.

[0131] The annotated 3D defect model set and defect parameter table are output to the management database to provide input data for historical record comparison and entry archiving.

[0132] In this embodiment, the generation of the final detection result set includes:

[0133] A data writing mapping relationship is established based on the spatial perception coordinate system and the time index. The data structure processing of the three-dimensional defect model set and defect parameter table is carried out to generate the data object to be written.

[0134] Call the management database interface module to write the data objects to be written into the database in time index order, establish the association between the time index field and the pose field, and complete the database entry operation of the 3D defect model set and defect parameter table.

[0135] Based on the historical data in the management database, a pose consistency comparison is performed. Spatial registration and difference calculation are carried out between the current set of 3D defect models and the historical set of 3D defect models to generate a pose deviation matrix and a set of difference reports.

[0136] Bind the pose deviation matrix and the difference report set to the time index of the current detection task, update the database entry status and version number, and complete the entry archiving;

[0137] Based on the item archiving results, the final detection result set is output, and a detection result record file containing time index, pose comparison results and defect parameters is generated for subsequent traceability analysis and statistical management.

[0138] In this embodiment, to adapt to different types of on-site inspection tasks, the method of the present invention can also be integrated into a handheld pipe lining inspection device. By configuring a lightweight sensing module and a portable embedded computing unit, the method can meet the inspection requirements of portable use scenarios while maintaining the original data acquisition and spatial perception capabilities. Specifically, it includes: integrating a handheld rotating scanning component and a high-resolution depth camera to acquire circumferential scan images and point cloud data of local areas of the pipe inner wall; configuring an embedded AI computing module to run the YOLOv11s-seg algorithm and the Bayesian Sheaf neural network model to achieve defect identification and confidence analysis at the edge; and transmitting the processing results back to the management database via a wireless network to perform 3D reconstruction and historical comparison operations, further improving on-site verification efficiency and inspection flexibility.

[0139] Example 1: To verify the feasibility of this invention in practice, it was applied to the inspection of a 10-meter-long, 2.6-meter-diameter rubber-lined steel pipe section. This pipeline operates in a high-salinity seawater environment for extended periods, resulting in varying degrees of blistering, delamination, and cracking of the rubber lining. The deep learning-based pipeline rubber lining defect identification and verification method proposed in this invention was deployed in an intelligent inspection equipment system. The entire pipe section was inspected and verified collaboratively using mobile and traction-lift inspection equipment.

[0140] During the inspection, the depth camera and rotating gimbal performed a folding and rotating scan, with an angular resolution of 0.5°, an axial step size of 2 mm, and a scan rate of 200 frames per minute. A total of 62,400 frames of angular scan images were obtained through the folding and rotating scan, along with approximately 9.8 GB of point cloud data. After registration and time synchronization, a multimodal dataset was formed. The cylinder unfolded image generated during the cylinder parameter domain mapping process has a width corresponding to a 360° circumference and a height corresponding to the axial length of the pipe, achieving a resolution of 0.1 mm.

[0141] Multimodal data was input into a Bayesian Sheaf neural network structure, establishing a topological adjacency relationship containing 12,800 nodes and 31,200 edges on the initial Sheaf graph structure. The feature vector corresponding to each node's image pixel region has a 64-dimensional dimension, and the edge constraint vector contains neighborhood feature covariance and spatial distance information. An uncertainty propagation mechanism was employed during network training, enabling the system to dynamically adjust the node confidence distribution during detection. A set of verification instructions was generated for regions with uncertainty indices higher than 0.35. A total of 72 local rotation scans were performed by the mobile detection equipment, and 18 vertical segment verifications were performed by the traction-lift detection equipment.

[0142] During the review process, a total of 11,200 frames of ring scan images and 1.6 GB of point cloud data were acquired and aligned with the consistency defect set using a spatially aware coordinate system. After fusing the review data set, the 3D reconstruction module performed spatial surface fitting and topological reconstruction, generating a total of 94 3D defect models. The defect depths ranged from 0.3 to 2.7 mm, and the areas ranged from 28 to 940 square millimeters.

[0143] To compare the performance differences between the method of this invention and traditional visual detection methods, the same pipe section area was selected for the experiment. The manual video detection method, the conventional convolutional neural network (CNN) method and the method of this invention were compared. The detection performance data are shown in Table 1.

[0144] Table 1. Performance Comparison of Different Detection Methods

[0145] Defect identification accuracy (%) 80.3 86.7 90.8 False negative rate (%) 10.5 7.2 4.4 False positive rate (%) 7.8 5.4 3.8 Boundary positioning error (mm) 3.2 1.9 0.6 3D verification accuracy (%) none 82.6 89.3 Processing time (min / 10m) 48.5 24.7 16.2

[0146] As shown in Table 1, the method of this invention significantly outperforms traditional detection methods in terms of recognition accuracy, spatial verification capability, and detection efficiency. Compared with manual video detection, the defect recognition accuracy increased from 80.3% to 90.8%, an improvement of approximately 10.5 percentage points; the false negative rate decreased from 10.5% to 4.4%, and the false positive rate decreased from 7.8% to 3.8%, indicating that the robustness of this invention in defect recognition under complex backgrounds is significantly enhanced. Compared with conventional CNN visual detection, the defect recognition accuracy increased by 4.7 percentage points, the false negative rate and false positive rate decreased by approximately 2.8% and 1.6%, respectively, and the boundary localization error decreased from 1.9 mm to 0.6 mm, a reduction of approximately 68%, demonstrating that the Bayesian Sheaf neural network has significant advantages in defect boundary modeling and spatial consistency maintenance.

[0147] Furthermore, this invention achieves a 3D verification accuracy of 89.3%, which is approximately 6.7 percentage points higher than conventional CNN methods, while manual detection cannot provide quantifiable verification results. Processing time is reduced from 24.7 minutes for conventional CNN methods to 16.2 minutes, improving detection efficiency by approximately 34%. This performance improvement stems from the node and edge-level consistency propagation mechanism introduced in this invention on the Sheaf graph topology, which enables uncertainty reasoning and spatial dependency constraints of features in multimodal inputs, reducing the propagation impact of misclassified regions.

[0148] The fundamental reason for the performance improvement lies in the fact that the Bayesian Sheaf neural network establishes a probabilistic relationship between node observations and edge constraints in the defect identification stage, enabling the model to simultaneously consider the similarity of the feature space and the continuity of the structural space. Through an uncertainty-driven intelligent verification mechanism, low-confidence areas are automatically calibrated and resampled, further enhancing the reliability of defect identification. In the 3D reconstruction and dimensional calculation stages, a spatially perceptible coordinate system is used to perform global constraint matching, ensuring the geometric consistency and measurement accuracy of the defect model. This invention achieves comprehensive performance with high identification accuracy, strong verification consistency, and excellent detection efficiency in complex pipeline environments, providing a reliable automated intelligent detection method for marine pipeline maintenance.

[0149] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for identifying and verifying defects in pipe lining based on deep learning, characterized in that, The steps include the following: The system acquires raw inspection data of the inner wall of the pipeline, performs rotational scanning, establishes a spatial sensing coordinate system, and simultaneously forms a multimodal data set. The multimodal dataset is mapped to the cylindrical parameter domain to generate a cylindrical unfolded image. Based on the cylindrical unfolded image and the spatial perception coordinate system, a topological adjacency relationship is constructed to establish the initial structure of the Sheaf graph. Based on the cylindrical unfolded image, the YOLOv11s-seg algorithm is used to perform defect detection and instance segmentation to obtain defect mask and defect feature map, which are then fused with point cloud data to generate a three-dimensional defect candidate set. A Bayesian Sheaf neural network is constructed on the initial structure of the Sheaf graph. The defect feature map, defect mask and topological adjacency relationship are input, consistency propagation and uncertainty estimation are performed, and the defect confidence distribution, uncertainty index and boundary consistency results are output. The three-dimensional defect candidate set is updated to obtain the consistent defect set. Perform local rotational scanning and axial micro-movement re-sampling at the corresponding locations of the consistency defect set, and then transmit the verification dataset back and merge it into a spatial sensing coordinate system. By integrating the verification data set and the consistency defect set, the three-dimensional reconstruction and size calculation are completed, and the three-dimensional defect model set and defect parameter table are output. Write the set of 3D defect models and the defect parameter table into the management database, complete the pose consistency comparison with the historical records and the entry archiving, and output the final detection result set.

2. The method for identifying and verifying defects in pipe lining based on deep learning according to claim 1, characterized in that, The generation of the multimodal data set and the spatially sensed coordinate system includes: Start the depth camera and rotating gimbal, load the unfolding and rotating scanning parameters, set the circumferential step size and axial step size, and enter continuous acquisition mode; Acquire a sequence of circular scan images and simultaneously record pose data, then bind each frame of the circular scan sequence image with the corresponding pose data at the frame level; Point cloud data is generated by distance measurement using a depth camera and synchronized with pose data. Invalid and isolated points are removed before the point cloud data is output. Based on the pose data, the coordinate transformation relationship from the device coordinates to the spatial perception coordinate system is calculated, and the circumferential scan sequence images and point cloud data are mapped to the spatial perception coordinate system and aligned. The panoramic scan sequence images, point cloud data, and pose data are packaged and registered in the spatial perception coordinate system according to the frame-level binding relationship to form a multimodal data set.

3. The method for identifying and verifying defects in pipe lining based on deep learning according to claim 1, characterized in that, The construction of the initial structure of the Sheaf graph includes: In the spatial perception coordinate system, the multimodal data set is registered and time-synchronized, and the point cloud data and pose data corresponding to the ring scan sequence images are preserved; Perform cylindrical parameter domain mapping to convert the circumferential scan sequence images and point cloud data to the cylindrical parameter domain, generate the cylindrical unfolded image and maintain the coordinate correspondence with the spatial perception coordinate system; Adjacency rules are established based on the cylindrical unfolded image and the spatial perception coordinate system. The connection relationship between data units is calculated according to the angular adjacency and axial adjacency criteria, and a topological adjacency relationship is constructed. An initial structure of a Sheaf graph is created on the cylindrical parameter domain based on topological adjacency relationships. The nodes and edges of the initial structure of the Sheaf graph are defined and aligned with a spatially aware coordinate system.

4. The method for identifying and verifying defects in pipe lining based on deep learning according to claim 1, characterized in that, The generation of the three-dimensional defect candidate set includes: Load the YOLOv11s-seg algorithm configuration file, set the input size, confidence threshold and segmentation threshold, input the cylinder unfolded image into the YOLOv11s-seg algorithm and initialize the detection backbone; Run the YOLOv11s-seg algorithm to perform feature extraction and multi-scale fusion, generate a set of feature maps containing the response of the defect region, and output the candidate set of bounding boxes, classification probability distribution and instance segmentation results; Extract the defect mask from the instance segmentation results, generate the defect feature map by combining the feature map set, and establish a corresponding mask and feature pairing relationship for each defect target; Based on the mapping relationship between the cylindrical parameter domain and the spatial perception coordinate system, the defect mask and the defect feature map are aligned to the spatial perception coordinate system; The aligned defect mask and defect feature map are fused with point cloud data, and a 3D defect candidate set is generated using depth information and boundary matching constraints.

5. The method for identifying and verifying defects in pipe lining based on deep learning according to claim 1, characterized in that, The generation of the consistency defect set includes: A Bayesian Sheaf neural network is constructed on the initial structure of the Sheaf graph. The set of nodes and the set of edges are defined, the topological adjacency relationship is mapped into a structured dependency matrix, and local Sheaf units and adjacency constraints are established. The defect mask and defect feature map are respectively input into the node encoding layer and edge association layer of the Bayesian Sheaf neural network. Feature normalization and edge weight initialization are performed. The structured dependency matrix is ​​called to determine the edge constraint weights and neighborhood propagation direction. The input features of each node are represented as node observation vectors, and the features of each edge are represented as edge constraint vectors. The consistency propagation process on the Sheaf structure is executed. The multi-layer message passing results are calculated based on the node observation vector and the edge constraint vector. The posterior distribution of node features is updated in each layer to form a feature propagation path with Bayesian uncertainty estimation. Based on the feature propagation path, posterior inference is performed on the node distribution results output by the Bayesian Sheaf neural network to generate defect confidence distribution and uncertainty index, and boundary consistency results are generated based on the edge-level consistency calculation results. The three-dimensional defect candidate set is updated based on the defect confidence distribution, uncertainty index and boundary consistency results, and a consistent defect set is generated and bound to the spatial sensing coordinate system.

6. The method for identifying and verifying defects in pipe lining based on deep learning according to claim 1, characterized in that, The generation of the review data set includes: Based on the uncertainty index and the boundary consistency results, the set of review targets is determined from the set of consistency defects. A set of review instructions is generated and the target position, circumferential step distance, axial step distance, number of re-sampling frames and exposure time are marked. The set of verification instructions is sent to the mobile intelligent equipment for inspecting rubber-lined pipes and the traction-lifting intelligent equipment for inspecting rubber-lined pipes. Based on the set of verification instructions, the attitude alignment and position correction are completed in the spatial perception coordinate system. Perform local rotational scanning and axial micro-movement re-acquisition to acquire and verify ring scan images, verification point clouds and pose data, and record the acquisition sequence in the time order of the spatial perception coordinate system; Frame-level binding, noise removal, and registration operations are performed on the verification ring scan image, verification point cloud, and pose data to generate a verification dataset that is then merged and mapped to a spatial perception coordinate system. Establish a correspondence between the set of reviewed data and the set of consistency defects, and then return it to the process.

7. The method for identifying and verifying defects in pipe lining based on deep learning according to claim 1, characterized in that, The generation of the three-dimensional defect model set and defect parameter table includes: Spatially align and temporally register the verification data set and the consistency defect set, and perform point cloud registration and image projection mapping based on the pose information of the spatial perception coordinate system to construct the verification data fusion body; Based on the verification data fusion, feature correspondence and depth interpolation are performed. The verification ring scan image, verification point cloud and defect feature map are fused to generate a unified three-dimensional point cloud data set. The defect boundary area is refined and resampled to form a standardized data structure for three-dimensional reconstruction. The 3D reconstruction module is invoked to perform spatial surface fitting and topological reconstruction on the standardized data structure, generating a set of 3D defect models, and calculating the spatial dimension parameters, volume parameters and boundary curvature parameters of each defect model to generate a defect parameter table; The set of 3D defect models and the defect parameter table are mapped to a spatially perceptual coordinate system. Pose calibration and global index numbering are performed on each defect model to complete its positioning and labeling in the spatially perceptual coordinate system.

8. The method for identifying and verifying defects in pipe lining based on deep learning according to claim 1, characterized in that, The generation of the final detection result set includes: A data writing mapping relationship is established based on the spatial perception coordinate system and the time index. The data structure processing of the three-dimensional defect model set and defect parameter table is carried out to generate the data object to be written. Call the management database interface module to write the data objects to be written into the database in time index order, establish the association between the time index field and the pose field, and complete the database entry operation of the 3D defect model set and defect parameter table. Based on the historical data in the management database, a pose consistency comparison is performed. Spatial registration and difference calculation are carried out between the current set of 3D defect models and the historical set of 3D defect models to generate a pose deviation matrix and a set of difference reports. Bind the pose deviation matrix and the difference report set to the time index of the current detection task, update the database entry status and version number, and complete the entry archiving; Output the final set of detection results based on the item archiving results.

Images