An underground pipeline disease intelligent detection and evaluation method and system

Abstract

The present application relates to a kind of underground pipeline disease intelligent detection evaluation method and system, comprising the following steps: S1: obtaining underground pipeline multi-source heterogeneous data, constructs underground pipeline detection knowledge graph;S2: according to detection task instruction, based on multimodal detection robot, obtains pipe multimodal original data set;S3: the obtained pipe multimodal original data set is preprocessed, and structured feature data set is obtained;S4: constructing intelligent identification model, defect detection is carried out, and the final defect intelligent identification result is output through multimodal data fusion decision algorithm;S5: according to defect intelligent identification result, each defect identified is positioned in three-dimensional space, and automatically labeled on digital pipe network map, quantifies the RBI risk index of each defect;S6: according to the RBI risk index of each defect, call underground pipeline detection knowledge graph intelligent matching best trenchless repair scheme, obtain detection report.The present application effectively improves the speed of underground pipeline disease detection and the consistency of evaluation.

Description

Technical Field

[0001] This invention relates to the field of pipeline inspection, and in particular to an intelligent detection and assessment method and system for underground pipeline defects. Background Technology

[0002] Urban underground pipeline networks serve as the lifeline of modern urban operations, undertaking critical functions such as water supply, sewage disposal, and gas transmission. However, with the aging of urban pipeline networks, external interference in densely constructed environments, and material corrosion, various pipeline defects such as cracks, corrosion, misalignment, and leakage are becoming increasingly prominent, seriously threatening urban operational safety and residents' quality of life. Globally, the maintenance and upgrading of underground pipelines has become a major challenge in urban infrastructure management.

[0003] Traditional pipeline inspection mainly relies on manual visual inspection or simple CCTV systems, which suffers from high missed detection rates, strong subjectivity, and low efficiency. Even with the application of robotic inspection, single sensor modalities are insufficient to cope with the complex internal pipe environment and the need to identify multiple types of defects. Existing inspection systems generally suffer from the problem of "information silos," making it difficult to effectively integrate multi-source heterogeneous data such as images, lasers, and sound waves, and lacking the ability to comprehensively capture and three-dimensionally perceive defect features. Summary of the Invention

[0004] To address the aforementioned problems, the present invention aims to provide an intelligent detection and evaluation method and system for underground pipeline defects, which effectively improves the speed of underground pipeline defect detection and the consistency of evaluation.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for intelligent detection and assessment of underground pipeline defects includes the following steps:

[0007] S1: Acquire multi-source heterogeneous data on underground pipelines and construct a knowledge graph for underground pipeline detection;

[0008] S2: Based on the detection task instructions, obtain the original multimodal dataset of the pipeline using the multimodal detection robot;

[0009] S3: Preprocess the acquired pipeline multimodal raw dataset to obtain a structured feature dataset;

[0010] S4: Combine improved YOLOv8 detection and 3D U-Net point cloud verification to build an intelligent recognition model, perform defect detection based on structured feature dataset, and output the final intelligent defect recognition result through multimodal data fusion decision algorithm;

[0011] S5: Based on the intelligent defect identification results, each identified defect is located in three-dimensional space and automatically marked on the digital pipeline network map. Combining structured features and underground pipeline detection knowledge graph, the RBI risk index of each defect is quantified.

[0012] S6: Based on the RBI risk index of each defect, the underground pipeline inspection knowledge graph is called to intelligently match the best trenchless repair solution and obtain the inspection report.

[0013] Furthermore, multi-source heterogeneous data on underground pipelines are acquired to construct a knowledge graph for underground pipeline detection, as detailed below:

[0014] Acquire multi-source heterogeneous data on underground pipelines, including basic structural information, operation and maintenance data, defect detection data, and environmental data;

[0015] Preprocessing of multi-source heterogeneous underground pipeline data involves unifying formats and converting them into structured forms based on their sources; establishing a data warehouse to manage all raw data by pipeline unit, time, and space through partitioning and tagging.

[0016] Based on preprocessed multi-source heterogeneous data, entities are extracted, and deep learning entity recognition NER and dependency parsing methods are applied to extract pipeline components, disease types, events, and relationships.

[0017] Finally, a knowledge graph for underground pipeline detection based on a graph database is generated.

[0018] Furthermore, the pipeline multimodal raw dataset includes optical video streams, discrete point clouds, gas concentration and temperature and humidity curves, real-time attitude (IMU) and location information; and all data streams are associated with the same detection task number, current spatial label and time label.

[0019] Further preprocessing includes data synchronization and calibration, data cleaning, mapping and normalization, and feature extraction, as detailed below:

[0020] The data synchronization and calibration process uses timestamps and spatial tags to synchronize and align different data streams, ensuring that multimodal data information at the same location and time can be correlated during the analysis process.

[0021] The data cleaning process checks the integrity and validity of each modality's data, and marks and removes outliers, invalid frames, and false sensor alarms.

[0022] The mapping normalization process, based on IMU and encoder data, achieves three-dimensional spatial positioning and path reconstruction for all data, accurately mapping the data to specific spatial segments and time nodes of the pipeline.

[0023] Feature extraction algorithms are used to transform raw data into quantitative or qualitative structured feature items, including extracting the location, type, and area visual features of pipe surface defects from optical video streams; extracting structural parameters such as cross-sectional contour anomalies, pipe diameter changes, local collapses, and wall thickness values ​​from discrete point cloud data; extracting gas abrupt change zones, dangerous concentration time periods, and humidity anomaly points from gas concentration and temperature-humidity curves; and extracting parameters such as the robot's current position, motion state, and detection segment length from IMU trajectories.

[0024] The extracted features are processed according to a unified structured data standard to output a structured feature dataset. Each data entry is associated with its task number, spatial location, time label, original data type, extracted feature parameters, and signal source.

[0025] Furthermore, the intelligent recognition model includes an improved YOLOv8 image defect detection submodule and a 3D U-Net point cloud defect verification submodule, as detailed below:

[0026] The improved YOLOv8 image defect detection submodule takes structured image data from the structured feature dataset as input, uses YOLOv8 as the backbone target detection network, and introduces the CBAM attention mechanism. The output is a defect candidate box for each image, which includes spatial location, confidence score, and category label.

[0027] The 3D U-Net point cloud defect verification submodule takes structured point cloud data from the structured feature dataset as input, and uses 3D U-Net to perform voxel segmentation and abnormal region extraction on the spatial point cloud of the pipe cross-section. This enables the 3D localization and contour segmentation of spatial defects such as volumetric deformation, cracks, and cavities, and outputs a point cloud spatial defect mask M. j ;

[0028] The outputs of the two models are automatically aligned with candidate defect entries in the structured feature dataset based on spatial and temporal labels, achieving cross-modal data correlation.

[0029] Furthermore, the improved YOLOv8 image defect detection submodule is based on the YOLOv8 network and includes a basic backbone network, a Neck network, and a Head network, as detailed below:

[0030] CSPDarknet53 is used as the backbone network, and residual connections and dense connections are introduced to enhance feature propagation capabilities; and the CBAM attention mechanism, including channel attention module and spatial attention module, is embedded in the backbone network.

[0031] Neck network enhancement introduces FPN, which fuses features of different scales of the backbone network through lateral and upsampling operations. It also adopts PANet structure, which downsamples the output of each layer of FPN from bottom to top and integrates it into the previous layer layer by layer, so as to achieve bidirectional fusion of high-resolution features and strong semantic features.

[0032] The Head network employs a decoupled head design, separating classification and regression tasks to improve detection accuracy. It introduces the FocalLoss loss function to address the imbalance between positive and negative samples and uses GIOU Loss as the bounding box regression loss to improve localization accuracy.

[0033] Furthermore, CSPDarknet53 is used as the backbone network, and residual connections and dense connections are introduced to enhance feature propagation capabilities; and the CBAM attention mechanism is embedded in the backbone network, including channel attention modules and spatial attention modules, as detailed below:

[0034] CSPDarknet53 serves as the backbone network, encoding the input image into multi-level high-level semantic features;

[0035] Residual connection: By performing a cross-layer addition operation, the output x of the previous layer is superimposed with the result F(x) of the current convolution operation, which alleviates the gradient vanishing problem in deep networks. The formula is as follows: y=F(x)+x;

[0036] Where y is the output after the cross-layer addition operation;

[0037] Dense connections: Concatenate the outputs of all preceding layers into the current layer to improve feature reuse and transfer capabilities. The formula is: x l =H l ([x0,x1,…,x l-1 ])

[0038] in, l H represents the current layer. l For the transformation operation of the current layer, [ ] indicates concatenation;

[0039] A CBAM module is inserted into the backbone network to perform saliency enhancement on the extracted feature maps, including channel attention module and spatial attention module;

[0040] The channel attention module is as follows:

[0041] Input: Feature map F;

[0042] Feature pooling: Apply global average pooling (AvgPool) and global max pooling (MaxPool) to the feature map F respectively. avg =AvgPool(F);

[0043] Fmax =MaxPool(F);

[0044] Among them, F avg F represents the feature vector after global average pooling. max The feature vector after global max pooling;

[0045] Multilayer perceptron (shared weights): M c (F)=σ(MLP(F avg )+MLP(F max ));

[0046] Where MLP is a two-layer fully connected layer, σ is a Sigmoid; M c (F) is the channel attention weight vector;

[0047] Output: Weighted channel features F′

[0048] F′=M c (F)⊙F

[0049] Where ⊙ represents dot product of each channel;

[0050] The spatial attention module is as follows:

[0051] Input: Weighted channel features F′;

[0052] Pooling operations: Perform max pooling (Maxchannel) and mean pooling (Meanchannel) along the channel dimension: F avg =Meanchannel(F′);

[0053] F max =Maxchannel(F′);

[0054] Among them, F avg ′ represents the result of mean pooling along the channel dimension; F max ′ represents the result of max pooling along the channel dimension;

[0055] Then perform the concatenation and convolution operations: Ms(F′) = σ(f 7×7 (Concat[Favg′;Fmax′]))

[0056] Among them, f 7×7 It is a 7×7 convolution, σ is the Sigmoid function; Concat is the concatenation operation; Ms(F′) is the spatial attention weight map;

[0057] Output: Weighted spatial feature F′′: F′′=Ms(F′)⊙F′.

[0058] Furthermore, the multimodal data fusion decision algorithm is as follows:

[0059] For the defect regions detected by the improved YOLOv8 image defect detection submodule and the 3D U-Net point cloud defect verification submodule, the spatial center point, interval overlap, and timestamp of the defect regions are determined, and those that meet the following criteria are fused: ; where δ s and δ t These are the spatial and temporal error tolerance thresholds, respectively; (x YOLO ,y YOLO ,z YOLO (x) represents the coordinates of the defect detected by the improved YOLOv8 submodule in three-dimensional space; U-Net ,y U-Net ,z U-Net ( ) represents the coordinates of the defects detected by the 3D U-Net point cloud verification submodule in three-dimensional space; t is the Euclidean distance between spatial coordinates; YOLO The timestamp for the detection of defects identified by the improved YOLOv8 submodule;

[0060] t U-Net This is the detection timestamp of the defect identified by the 3D U-Net point cloud verification submodule;

[0061] Combining the confidence scores of each modality output, a weighted fusion confidence score is used as the final confidence score C for each candidate defect. final :

[0062] C final= w img ×C img +w pc ×C pc ;

[0063] Among them, C img To improve the confidence level of defect detection in the YOLOv8 image defect detection submodule; C pc The confidence level of the 3D U-Net point cloud defect verification submodule for determining spatial anomalies in the same region; w img w pc Assign confidence weights to each modality;

[0064] The fused defect region is extracted, and its attributes are summarized, including spatial location, defect type, geometric attributes, detection time, and final confidence level C. final .

[0065] Furthermore, based on the intelligent defect identification results, each identified defect is located in three-dimensional space and automatically labeled on the digital pipeline network map. Combining structured features and the underground pipeline detection knowledge graph, the RBI risk index of each defect is quantified, as follows:

[0066] The identified defects are accurately mapped onto the digital pipeline map based on their spatial location information in the image and point cloud, combined with odometer, IMU and digital pipeline map, including the pipe segment to which the defect belongs, the start and end distance, the circumferential angle and the three-dimensional coordinates.

[0067] Each defect is automatically archived into the digital pipeline system with a unique number, spatial location, detection time, and metadata.

[0068] Structured features are extracted from the multimodal recognition results and semantically associated with the underground pipeline detection knowledge graph. Intelligent matching is performed on the historical occurrence patterns, failure mechanisms, consequences, and best repair experiences for each type of defect.

[0069] Based on structured features and knowledge graph reasoning results, risk-based testing theory is used to quantify the probability of failure (PoF) and the failure consequence (CoF) of each defect, and to comprehensively output the RBI risk index RB for each defect. Iindex The formula is as follows: RB Iindex =PoF×CoF;

[0070] The probability of failure (PoF) comprehensively considers defect size, growth rate, structural sensitivity, and operating environment factors, and is expressed using a normalized weighted summation.

[0071] PoF=α1S size +α2S growth +α3S location +α4S material +α5S history ;

[0072] Among them, S size Scoring for defect size; S growth Score the defect growth rate; location Sensitivity level of the defect's structure / location; S material Score the vulnerability of the pipe material; history The historical frequency of occurrence and factors for which repairs have not been completed; α1, α2, α3, α4, and α5 are the corresponding weighting coefficients, α1+α2+α3+α4+α5=1;

[0073] The failure consequences (COF) are calculated using a weighted average of multiple factors, including safety impact, environmental impact, economic loss, and negative social effects.

[0074] CoF=β1C safety +β2Cenvironment +β3C economic +β4C reputation ;

[0075] Among them, C safety For safety impact rating (e.g., the degree of threat to the pipeline); C environment For environmental impact assessment (such as pollution and secondary damage); C economic Score the economic losses (estimate direct and indirect losses); C reputation The social / public opinion impact score β is a weighted coefficient.

[0076] Finally, all detected defects will be mapped onto a digital pipeline network map based on the 3D positioning results to form a defect distribution.

[0077] An intelligent detection and assessment system for underground pipeline defects includes a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it specifically performs the steps in the intelligent detection and assessment method for underground pipeline defects as described above.

[0078] The present invention has the following beneficial effects:

[0079] 1. This invention makes full use of multi-source heterogeneous data to automatically construct a knowledge graph of underground pipelines, improve the intelligence of disease detection and the level of full-process automation, and realize the scientific quantification of disease risk and efficient repair decision-making.

[0080] 2. The improved YOLOv8 image defect detection submodule of this invention uses CSPDarknet53 as the backbone network. By introducing residual connections and dense connections, the information flow and feature reuse capabilities of the deep network are greatly enhanced. A CBAM attention module is embedded in the backbone network, organically combining channel attention and spatial attention. This enables the model to accurately learn and focus on important features of different channels and spatial regions in pipeline defect images, effectively improving the detection sensitivity and expressive power for small defects and abnormal targets in complex backgrounds. In the feature fusion part, PANet structure and FPN are introduced to achieve full interaction and multi-scale fusion of high and low-level features. The receptive field is expanded through the SPP module, significantly improving the model's adaptability to defect targets of different scales (including large volume and micro-sized).

[0081] 3. This invention utilizes a multimodal data fusion decision-making algorithm to effectively combine visual and spatial information for high-precision intelligent identification of pipeline defects. Simultaneously, by combining structured features and industry knowledge graphs, it scientifically assesses the failure probability and consequences of each defect based on Risk By Inspection (RBI) theory, thereby quantifying the risk index. Attached Figure Description

[0082] Figure 1This is a flowchart of the method of the present invention;

[0083] Figure 2 This is a schematic diagram of an improved YOLOv8 image defect detection submodule in one embodiment of the present invention;

[0084] Figure 3 This is a schematic diagram of the defect three-dimensional localization and RBI risk assessment process in one embodiment of the present invention. Detailed Implementation

[0085] The following is in conjunction with the appendix Figure 1-3 The present invention will be further described in detail with reference to specific embodiments:

[0086] A method for intelligent detection and assessment of underground pipeline defects includes the following steps:

[0087] S1: Acquire multi-source heterogeneous data on underground pipelines and construct a knowledge graph for underground pipeline detection;

[0088] S2: Based on the detection task instructions, obtain the original multimodal dataset of the pipeline using the multimodal detection robot;

[0089] S3: Preprocess the acquired pipeline multimodal raw dataset to obtain a structured feature dataset;

[0090] S4: Combine improved YOLOv8 detection and 3D U-Net point cloud verification to build an intelligent recognition model, perform defect detection based on structured feature dataset, and output the final intelligent defect recognition result through multimodal data fusion decision algorithm;

[0091] S5: Based on the intelligent defect identification results, each identified defect is located in three-dimensional space and automatically marked on the digital pipeline network map. Combining structured features and underground pipeline detection knowledge graph, the RBI risk index of each defect is quantified.

[0092] S6: Based on the RBI risk index of each defect, the underground pipeline inspection knowledge graph is called to intelligently match the best trenchless repair solution and obtain the inspection report.

[0093] In this embodiment, multi-source heterogeneous data of underground pipelines are acquired to construct an underground pipeline detection knowledge graph, as detailed below:

[0094] Acquire multi-source heterogeneous data on underground pipelines, including basic structural information, operation and maintenance data, defect detection data, and environmental data;

[0095] Basic structural information includes pipeline number, coordinates, pipe diameter, material, burial depth, construction time, and area; operation and maintenance data includes historical maintenance records, emergency events, and inspection logs; disease detection data includes multimodal raw detection data such as high-definition images / videos (optical, infrared), sound waves, ultrasound, lidar point clouds, gas, temperature, and IMU trajectory; environmental data includes external environment such as surrounding geology, water level, soil layer distribution, municipal roads, and important facilities.

[0096] For multi-source heterogeneous underground pipeline data preprocessing, format unification and structure conversion are performed according to the source; for text and tabular data, field mapping standardization is performed, and data cleaning tools (such as OpenRefine) are used to handle missing, abnormal, and duplicate data; for images / videos / point clouds, file archiving, naming conventions, and time-space label association are performed; for unstructured descriptions (maintenance logs, etc.), NLP information extraction methods are used to extract entity, event, and time-series information; a data warehouse is established, and all raw data are partitioned and labeled according to pipeline unit, time, and space for management.

[0097] Based on preprocessed multi-source heterogeneous data, entities are extracted, and deep learning entity recognition NER and dependency parsing methods are applied to extract pipeline components, disease types, events, and relationships.

[0098] Entities: Pipelines, manholes, joints, materials, defects, inspection tasks, inspection equipment, inspection parameters, repair processes, historical events…

[0099] Relationships: Pipeline - Existence - Defect; Defect - Association - Detection Results; Defect - Recommendation - Repair Process; Pipeline - Location - Geographic Point; Defect - Impact - Event…

[0100] Finally, a knowledge graph for underground pipeline detection is generated based on graph databases (such as Neo4j and ArangoDB).

[0101] In this embodiment, the pipeline multimodal raw dataset includes optical video stream, discrete point cloud, gas concentration and temperature and humidity curves, real-time attitude (IMU) and location information; and all data streams are associated with the same detection task number, current spatial label (such as pipeline ID, start and end coordinates, relative mileage) and time label (such as acquisition timestamp, batch number).

[0102] In this embodiment, preprocessing includes data synchronization and calibration, data cleaning, mapping normalization, and feature extraction, as detailed below:

[0103] The data synchronization and calibration process uses timestamps and spatial tags to synchronize and align different data streams, ensuring that multimodal data information at the same location and time can be correlated during the analysis process.

[0104] The data cleaning process checks the integrity and validity of each modal data, and marks and removes outliers, invalid frames, and false sensor alarms; for example, it automatically cleans blurry frames in optical video streams, isolated noise points in point clouds, and distorted segments of gas curves.

[0105] The mapping normalization process, based on IMU and encoder data, achieves three-dimensional spatial positioning and path reconstruction for all data, accurately mapping the data to specific spatial segments and time nodes of the pipeline.

[0106] Feature extraction algorithms are used to transform raw data into quantitative or qualitative structured feature items, including extracting the location, type, and area visual features of pipe surface defects (cracks, holes, discoloration areas, etc.) from optical video streams; extracting structural parameters such as cross-sectional contour anomalies, pipe diameter changes, local collapses, and wall thickness values ​​from discrete point cloud data; extracting gas abrupt change areas, dangerous concentration time periods, and humidity anomaly points from gas concentration and temperature-humidity curves; and extracting parameters such as the robot's current position, motion state, and detection segment length from the IMU trajectory.

[0107] The extracted features are processed according to a unified structured data standard (such as database tables, JSON / XML documents, etc.) to output a structured feature dataset. Each data entry is associated with its task number, spatial location, time label, original data type, extracted feature parameters, and signal source.

[0108] In this embodiment, the intelligent recognition model includes an improved YOLOv8 image defect detection submodule and a 3D U-Net point cloud defect verification submodule, as detailed below:

[0109] The improved YOLOv8 image defect detection submodule takes structured image data from the structured feature dataset as input, uses YOLOv8 as the backbone target detection network, and introduces the CBAM attention mechanism to enhance the model's ability to focus on and distinguish features in complex scenes such as cracks, corrosion, and micro-defects. The output is a defect candidate box for each image, which includes spatial location, confidence score, and category label.

[0110] The 3D U-Net point cloud defect verification submodule takes structured point cloud data from the structured feature dataset as input, and uses 3D U-Net to perform voxel segmentation and abnormal region extraction on the pipe section / spatial point cloud. This enables the 3D localization and contour segmentation of spatial defects such as volumetric deformation, cracks, and cavities, and outputs a point cloud spatial defect mask M. j It can quantify features such as shape, volume, and spatial coordinate range;

[0111] The outputs of the two models are automatically aligned with candidate defect entries in the structured feature dataset based on spatial and temporal labels, achieving cross-modal data correlation.

[0112] In this embodiment, the improved YOLOv8 image defect detection submodule is based on an improved YOLOv8 network, including a basic backbone network, a Neck network, and a Head network, as detailed below:

[0113] CSPDarknet53 is used as the backbone network, and residual connections and dense connections are introduced to enhance feature propagation capabilities; and the CBAM attention mechanism, including channel attention module and spatial attention module, is embedded in the backbone network.

[0114] Neck network enhancement introduces FPN, which fuses features of different scales of the backbone network (such as P3, P4, P5) through lateral and upsampling operations. The PANet structure is adopted, which downsamples the output of each layer of FPN from bottom to top and integrates it into the previous layer layer by layer, so as to achieve bidirectional fusion of high-resolution features and strong semantic features.

[0115] The Head network employs a decoupled head design, separating classification and regression tasks to improve detection accuracy. It introduces the FocalLoss loss function to address the imbalance between positive and negative samples and uses GIOU Loss as the bounding box regression loss to improve localization accuracy.

[0116] In this embodiment, CSPDarknet53 is used as the backbone network, and residual connections and dense connections are introduced to enhance feature propagation capabilities. Furthermore, a CBAM attention mechanism, including a channel attention module and a spatial attention module, is embedded in the backbone network, as detailed below:

[0117] CSPDarknet53 serves as the backbone network, encoding the input image into multi-level high-level semantic features;

[0118] Residual connection: By performing a cross-layer addition operation, the output x of the previous layer is superimposed with the result F(x) of the current convolution operation, which alleviates the gradient vanishing problem in deep networks. The formula is as follows: y=F(x)+x;[1]

[0119] Where y is the output after the cross-layer addition operation;

[0120] Dense connections: Concatenate the outputs of all preceding layers into the current layer to improve feature reuse and transfer capabilities. The formula is: x l =H l ([x0,x1,…,x l-1 ])

[0121] in, l Indicates the current layer, H l For the transformation operation of the current layer, [ ] indicates concatenation;

[0122] A CBAM module is inserted into the backbone network to perform saliency enhancement on the extracted feature maps, including channel attention module and spatial attention module;

[0123] The channel attention module is as follows:

[0124] Input: Feature map F;

[0125] Feature pooling: Apply global average pooling (AvgPool) and global max pooling (MaxPool) to the feature map F respectively. avg =AvgPool(F);

[0126] F max =MaxPool(F);

[0127] Among them, F avg F represents the feature vector after global average pooling. max The feature vector after global max pooling;

[0128] Multilayer perceptron (shared weights): M c (F)=σ(MLP(F avg )+MLP(F max ));

[0129] Where MLP is a two-layer fully connected layer, σ is a Sigmoid; M c (F) is the channel attention weight vector;

[0130] Output: Weighted channel features F′

[0131] F′=M c (F)⊙F

[0132] Where ⊙ represents dot product of each channel;

[0133] The spatial attention module is as follows:

[0134] Input: Weighted channel features F′;

[0135] Pooling operations: Perform max pooling (Maxchannel) and mean pooling (Meanchannel) along the channel dimension: F avg =Meanchannel(F′);

[0136] F max =Maxchannel(F′);

[0137] Among them, F avg ′ represents the result of mean pooling along the channel dimension; F max ′ represents the result of max pooling along the channel dimension;

[0138] Then perform the concatenation and convolution operations: Ms(F′) = σ(f 7×7 (Concat[Favg′;Fmax′]))

[0139] Among them, f 7×7 It is a 7×7 convolution, σ is the Sigmoid function; Concat is the concatenation operation; Ms(F′) is the spatial attention weight map;

[0140] Output: Weighted spatial feature F′′: F′′=Ms(F′)⊙F′.

[0141] In this embodiment, the multimodal data fusion decision algorithm is as follows:

[0142] For the defect regions detected by the improved YOLOv8 image defect detection submodule and the 3D U-Net point cloud defect verification submodule, their spatial center point, interval overlap, and timestamp are determined, and those that meet the following criteria are fused: ; where δ s and δ t These are the spatial and temporal error tolerance thresholds, respectively; (x YOLO ,y YOLO ,z YOLO (x) represents the coordinates of the defect detected by the improved YOLOv8 submodule in three-dimensional space; U-Net ,y U-Net ,z U-Net ( ) represents the coordinates of the defects detected by the 3D U-Net point cloud verification submodule in three-dimensional space; t is the Euclidean distance between spatial coordinates; YOLO This is the detection timestamp of the defect being identified by the improved YOLOv8 submodule;

[0143] t U-Net This is the detection timestamp of the defect identified by the 3D U-Net point cloud verification submodule;

[0144] Combining the confidence scores of each modality output, a weighted fusion confidence score is used as the final confidence score C for each candidate defect. final :

[0145] C final= w img ×C img +w pc ×C pc ;

[0146] Among them, C img To improve the confidence level of defect detection in the YOLOv8 image defect detection submodule; C pcThe confidence level of the 3D U-Net point cloud defect verification submodule for determining spatial anomalies in the same region; w img w pc The confidence weights for each modality (w) img +w pc =1, adjusted through historical performance evaluation or dynamically adaptive allocation);

[0147] The fused defect region is extracted, and its summarized attributes include spatial location (intersection of spatial intervals or weighted center), defect type (selected by model discrimination category or voting mechanism), geometric attributes (from point cloud or image segmentation results), detection time (average of multimodal detection or earliest / latest time), and final confidence C. final .

[0148] In this embodiment, based on the intelligent defect identification results, each identified defect is located in three-dimensional space and automatically marked on the digital pipeline network map. Combining structured features and the underground pipeline detection knowledge graph, the RBI risk index of each defect is quantified, as follows:

[0149] The identified defects are accurately mapped onto the digital pipeline map based on their spatial location information in the image and point cloud, combined with the odometer, IMU and digital pipeline map, including the pipeline segment to which the defect belongs, the start and end distance, the circumferential angle and the three-dimensional coordinates (latitude, longitude and elevation).

[0150] Each defect is automatically archived into the digital pipeline network system with a unique number, spatial location, and detection time metadata, so that the defects are "visible, searchable, and traceable".

[0151] Structured features (such as defect type, size, depth, distribution pattern, historical frequency of occurrence, etc.) are extracted from the multimodal recognition results and semantically associated with the underground pipeline detection knowledge graph. Intelligent matching is performed on the historical occurrence pattern, failure mechanism, consequences and impact of each type of defect and the best repair experience.

[0152] Based on structured features and knowledge graph reasoning results, risk-based testing (RBI) theory is used to quantify the probability of failure (PoF) and the failure consequence (CoF) of each defect, and to output the RBI risk index RB for each defect. Iindex The formula is as follows: RB Iindex =PoF×CoF;

[0153] The probability of failure (PoF) comprehensively considers defect size, growth rate, structural sensitivity, and operating environment factors, and is expressed using a normalized weighted summation.

[0154] PoF=α1S size +α2S growth +α3S location+α4S material +α5S history ;

[0155] Among them, S size Scoring for defect size (e.g., damaged area or crack length / width normalized score); S growth Score the defect growth rate (based on historical monitoring data); location Sensitivity level of the defect's structure / location; S material Score the vulnerability of the pipe material; history The historical frequency of occurrence and factors for which repairs have not been completed; α1, α2, α3, α4, and α5 are the corresponding weighting coefficients, α1+α2+α3+α4+α5=1;

[0156] The failure consequences (COF) are calculated using a weighted average of multiple factors, including safety impact, environmental impact, economic loss, and negative social effects.

[0157] CoF=β1C safety +β2C environment +β3C economic +β4C reputation ;

[0158] Among them, C safety For safety impact rating (e.g., the degree of threat to the pipeline); C environment For environmental impact assessment (such as pollution and secondary damage); C economic Score the economic losses (estimate direct and indirect losses); C reputation The social / public opinion impact score β is a weighted coefficient.

[0159] Finally, all detected defects will be mapped onto a digital pipeline network map based on the 3D positioning results to form a defect distribution.

[0160] An intelligent detection and assessment system for underground pipeline defects includes a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it specifically performs the steps of the intelligent detection and assessment method for underground pipeline defects described above.

[0161] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0162] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0163] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0164] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0165] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for intelligent detection and assessment of underground pipeline defects, characterized in that, Includes the following steps: S1: Acquire multi-source heterogeneous data on underground pipelines and construct a knowledge graph for underground pipeline detection; S2: Based on the detection task instructions, obtain the original multimodal dataset of the pipeline using the multimodal detection robot; S3: Preprocess the acquired pipeline multimodal raw dataset to obtain a structured feature dataset; S4: Combine improved YOLOv8 detection and 3D U-Net point cloud verification to build an intelligent recognition model, perform defect detection based on structured feature dataset, and output the final intelligent defect recognition result through multimodal data fusion decision algorithm; S5: Based on the intelligent defect identification results, each identified defect is located in three-dimensional space and automatically marked on the digital pipeline network map. Combining structured features and underground pipeline detection knowledge graph, the RBI risk index of each defect is quantified. S6: Based on the RBI risk index of each defect, the underground pipeline inspection knowledge graph is called to intelligently match the best trenchless repair solution and obtain the inspection report; The intelligent recognition model includes an improved YOLOv8 image defect detection submodule and a 3D U-Net point cloud defect verification submodule, as detailed below: The improved YOLOv8 image defect detection submodule takes structured image data from the structured feature dataset as input, uses YOLOv8 as the backbone target detection network, and introduces the CBAM attention mechanism. The output is a defect candidate box for each image, which includes spatial location, confidence score, and category label. The 3D U-Net point cloud defect verification submodule takes structured point cloud data from the structured feature dataset as input, and uses 3D U-Net to perform voxel segmentation and abnormal region extraction on the spatial point cloud of the pipe cross-section. This enables the 3D localization and contour segmentation of spatial defects such as volumetric deformation, cracks, and cavities, and outputs a point cloud spatial defect mask M. j ; The outputs of the two models are automatically aligned with candidate defect entries in the structured feature dataset based on spatial and temporal labels, achieving cross-modal data correlation. The improved YOLOv8 image defect detection submodule is based on the YOLOv8 network and includes a basic backbone network, a Neck network, and a Head network, as detailed below: CSPDarknet53 is used as the backbone network, and residual connections and dense connections are introduced to enhance feature propagation capabilities; and the CBAM attention mechanism, including channel attention module and spatial attention module, is embedded in the backbone network. Neck network enhancement introduces FPN, which fuses features of different scales of the backbone network through lateral and upsampling operations. It also adopts PANet structure, which downsamples the output of each layer of FPN from bottom to top and integrates it into the previous layer layer by layer, so as to achieve bidirectional fusion of high-resolution features and strong semantic features. The Head network employs a decoupled head design, separating classification and regression tasks to improve detection accuracy. It introduces the Focal Loss loss function to address the imbalance between positive and negative samples and uses GIOU Loss as the bounding box regression loss to improve localization accuracy.

2. The intelligent detection and evaluation method for underground pipeline defects according to claim 1, characterized in that, The acquisition of multi-source heterogeneous data on underground pipelines and the construction of an underground pipeline detection knowledge graph are detailed below: Acquire multi-source heterogeneous data on underground pipelines, including basic structural information, operation and maintenance data, defect detection data, and environmental data; Preprocessing of multi-source heterogeneous data on underground pipelines involves standardizing the format and converting it into a structured form according to the source. Establish a data warehouse and manage all raw data by partitioning and tagging it according to pipeline units, time, and space. Based on preprocessed multi-source heterogeneous data, entities are extracted, and deep learning entity recognition NER and dependency parsing methods are applied to extract pipeline components, disease types, events, and relationships. Finally, a knowledge graph for underground pipeline detection based on a graph database is generated.

3. The intelligent detection and evaluation method for underground pipeline defects according to claim 1, characterized in that, The pipeline multimodal raw dataset includes optical video streams, discrete point clouds, gas concentration and temperature and humidity curves, real-time attitude and location information; and all data streams are associated with the same detection task number, current spatial label and time label.

4. The intelligent detection and evaluation method for underground pipeline defects according to claim 3, characterized in that, The preprocessing includes data synchronization and calibration, data cleaning, mapping normalization, and feature extraction, as detailed below: The data synchronization and calibration process uses timestamps and spatial tags to synchronize and align different data streams, ensuring that multimodal data information at the same location and time can be correlated during the analysis process. The data cleaning process checks the integrity and validity of each modality's data, and marks and removes outliers, invalid frames, and false sensor alarms. The mapping normalization process, based on IMU and encoder data, achieves three-dimensional spatial positioning and path reconstruction for all data, accurately mapping the data to specific spatial segments and time nodes of the pipeline. Feature extraction algorithms are used to transform raw data into quantitative or qualitative structured feature items, including extracting the location, type, and area visual features of pipe surface defects from optical video streams; extracting structural parameters such as cross-sectional contour anomalies, pipe diameter changes, local collapses, and wall thickness values ​​from discrete point cloud data; extracting gas abrupt change zones, dangerous concentration time periods, and humidity anomaly points from gas concentration and temperature-humidity curves; and extracting parameters such as the robot's current position, motion state, and detection segment length from IMU trajectories. The extracted features are processed according to a unified structured data standard to output a structured feature dataset. Each data entry is associated with its task number, spatial location, time label, original data type, extracted feature parameters, and signal source.

5. The intelligent detection and evaluation method for underground pipeline defects according to claim 1, characterized in that, The system employs CSPDarknet53 as its backbone network, introducing residual connections and dense connections to enhance feature propagation capabilities. Furthermore, it embeds a CBAM attention mechanism within the backbone network, including channel attention and spatial attention modules, as detailed below: CSPDarknet53 serves as the backbone network, encoding the input image into multi-level high-level semantic features; Residual connection: By performing a cross-layer addition operation, the output x of the previous layer is superimposed with the result F(x) of the current convolution operation, which alleviates the gradient vanishing problem in deep networks. The formula is as follows: y=F(x)+x; Where y is the output after the cross-layer addition operation; Dense connections: Concatenate the outputs of all preceding layers into the current layer to improve feature reuse and transfer capabilities. The formula is: x l =H l ([x0,x1,…,x l-1 ]) in, l H represents the current layer. l For the transformation operation of the current layer, [ ] indicates concatenation; A CBAM module is inserted into the backbone network to perform saliency enhancement on the extracted feature maps, including channel attention module and spatial attention module; The channel attention module is as follows: Input: Feature map F; Feature pooling: Apply global average pooling (AvgPool) and global max pooling (MaxPool) to the feature map F respectively: F avg =AvgPool(F); F max =MaxPool(F); Among them, F avg F represents the feature vector after global average pooling. max The feature vector after global max pooling; Multilayer perceptron: M c (F)=σ(MLP(F avg )+MLP(F max )); Where MLP is a two-layer fully connected layer, σ is a Sigmoid; M c (F) is the channel attention weight vector; Output: Weighted channel features F′ = M c (F)⊙F Where ⊙ represents dot product of each channel; The spatial attention module is as follows: Input: Weighted channel features F′; Pooling operations: Perform max pooling (Maxchannel) and mean pooling (Meanchannel) along the channel dimension respectively: F avg =Meanchannel(F′); F max ′=Maxchannel(F′); Among them, F avg ′ represents the result of mean pooling along the channel dimension; F max ′ represents the result of max pooling along the channel dimension; Then perform the concatenation and convolution operations: Ms(F′) = σ(f 7×7 (Concat[Favg′;Fmax′])) Among them, f 7×7 It is a 7×7 convolution, σ is the Sigmoid function; Concat is the concatenation operation; Ms(F′) is the spatial attention weight map; Output: Weighted spatial feature F′′: F′′=Ms(F′)⊙F′.

6. The intelligent detection and evaluation method for underground pipeline defects according to claim 5, characterized in that, The multimodal data fusion decision algorithm is as follows: For the defect regions detected by the improved YOLOv8 image defect detection submodule and the 3D U-Net point cloud defect verification submodule, the spatial center point, interval overlap, and timestamp of the defect regions are determined, and those that meet the following criteria are fused: ; where δ s and δ t These are the spatial and temporal error tolerance thresholds, respectively; (x YOLO ,y YOLO ,z YOLO (x) represents the coordinates of the defect detected by the improved YOLOv8 submodule in three-dimensional space; U-Net ,y U-Net ,z U-Net ( ) represents the coordinates of the defects detected by the 3D U-Net point cloud verification submodule in three-dimensional space; t is the Euclidean distance between spatial coordinates; YOLO The timestamp for the detection of defects identified by the improved YOLOv8 submodule; t U-Net This is the detection timestamp of the defect identified by the 3D U-Net point cloud verification submodule; Combining the confidence scores of each modality output, a weighted fusion confidence score is used as the final confidence score C for each candidate defect. final : C final= w img ×C img +w pc ×C pc ; Among them, C img To improve the confidence level of defect detection in the YOLOv8 image defect detection submodule; C pc The confidence level of the 3D U-Net point cloud defect verification submodule for determining spatial anomalies in the same region; w img w pc Assign confidence weights to each modality; The fused defect region is extracted, and its attributes are summarized, including spatial location, defect type, geometric attributes, detection time, and final confidence level C. final .

7. The intelligent detection and evaluation method for underground pipeline defects according to claim 1, characterized in that, Based on the intelligent defect identification results, each identified defect is located in three-dimensional space and automatically marked on the digital pipeline network map. Combining structured features and underground pipeline detection knowledge graph, the RBI risk index of each defect is quantified, as follows: The identified defects are accurately mapped onto the digital pipeline map based on their spatial location information in the image and point cloud, combined with odometer, IMU and digital pipeline map, including the pipe segment to which the defect belongs, the start and end distance, the circumferential angle and the three-dimensional coordinates. Each defect is automatically archived into the digital pipeline system with a unique number, spatial location, detection time, and metadata. Structured features are extracted from the multimodal recognition results and semantically associated with the underground pipeline detection knowledge graph. Intelligent matching is performed on the historical occurrence patterns, failure mechanisms, consequences, and best repair experiences for each type of defect. Based on structured features and knowledge graph reasoning results, risk-based testing theory is used to quantify the probability of failure (PoF) and the failure consequence (CoF) of each defect, and to comprehensively output the RBI risk index RB for each defect. Iindex The formula is as follows: RB Iindex =PoF×CoF; The probability of failure (PoF) comprehensively considers defect size, growth rate, structural sensitivity, and operating environment factors, and is expressed using a normalized weighted summation. PoF=α1S size +α2S growth +α3S location +α4S material +α5S history ; Among them, S size Scoring for defect size; S growth Score the defect growth rate; location Sensitivity level of the defect's location; S material Score the vulnerability of the pipe material; history The historical frequency of occurrence and factors for which repairs have not been completed; α1, α2, α3, α4, and α5 are the corresponding weighting coefficients, α1+α2+α3+α4+α5=1; The failure consequences (COF) are calculated using a weighted average of multiple factors, including safety impact, environmental impact, economic loss, and negative social effects. CoF=β1C safety +β2C environment +β3C economic +β4C reputation ; Among them, C safety Score for safety impact; C environment Environmental impact score; C economic Rate the economic loss; C reputation The public opinion influence score β is a weighting coefficient. Finally, all detected defects will be mapped onto a digital pipeline network map based on the 3D positioning results to form a defect distribution.

8. An intelligent detection and assessment system for underground pipeline defects, characterized in that, It includes a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it specifically performs the steps in the intelligent detection and evaluation method for underground pipeline defects as described in any one of claims 1-7.

Images