Underground pipeline intelligent detection and mapping method based on image recognition

Abstract

The invention discloses an underground pipeline intelligent detection and mapping method based on image recognition, and the method comprises the following steps: 1, obtaining continuous video images, associating feature matching pairs of adjacent key frames, and forming a pose parameter set; 2, inputting the key frame into an improved YOLO-World detection network, and outputting a detection result set; 3, obtaining a geometric consistency matching set according to the detection result set; 4, performing multi-view triangularization on the geometric consistent matching set to form a weight factor; 5, introducing a weight factor, and executing incremental beam adjustment optimization on the pose parameter set and the three-dimensional sparse point set to obtain a sparse semantic point cloud; and step 6, outputting an underground pipe network topology map. According to the invention, high-precision and high-robustness intelligent identification and topological mapping in a complex underground pipeline environment are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and image processing, and particularly relates to an underground pipeline intelligent detection and mapping method based on image recognition. BACKGROUND

[0002] In the existing underground pipe network modeling and identification technology, usually relies on manual inspection, two-dimensional design drawings or manual modeling method based on traditional three-dimensional point cloud data, these ways have great limitations in precision, efficiency and intelligent degree. The two-dimensional drawing is difficult to completely and accurately reflect the spatial structure of the actual underground pipeline, and there is deviation with the actual construction result; although the dense point cloud obtained based on traditional laser radar or photogrammetry can provide three-dimensional spatial information, the data volume is large, the semantic information is missing, and the automatic processing is difficult, which is difficult to quickly land in practical engineering application. The existing method ignores the cross-frame geometric constraint and semantic consistency, resulting in low recognition rate and inaccurate positioning in the identification and modeling of complex structures such as valves, tees and flanges.

[0003] The pipe network identification method based on visual SLAM (Simultaneous Localization and Mapping) and deep learning detection algorithm gradually rises, trying to automatically extract targets and reconstruct spatial structures through image sequences and deep learning models. The existing method generally lacks explicit modeling of pipe segment geometric features, and it is difficult to establish a closed-loop process from image detection results to semantic sparse point cloud to topological graph structure. Most methods based on YOLO target detection network only output bounding boxes and class information, lack direct regression ability of target geometric structure parameters (such as cylinder axis, radius, orientation, etc.), which leads to the inability of subsequent spatial reconstruction process to accurately fit the geometric shape of the pipe segment. The existing SLAM system cannot fully combine the semantic labels and instance identification information of the detection results when constructing sparse point clouds, which limits the interpretability and topological structure expression ability of the point cloud.

[0004] Based on the above problems, the present application proposes an underground pipe network three-dimensional reconstruction method integrating detection, association, modeling and topology generation, which realizes the automatic processing flow from image input to underground pipe network topological graph output by introducing cylindrical geometric prior parameter modeling mechanism, sparse point cloud reconstruction process based on graph optimization and semantic instance guided pipe segment axis fitting and topology construction strategy, breaking through the technical bottleneck of existing technology in structure expression, semantic consistency and reconstruction accuracy. SUMMARY

[0005] One purpose of the present application is to propose a kind of underground pipe network three-dimensional modeling and topology generation method based on semantic detection and sparse graph optimization, the present application fully integrates improved YOLO-World detection network, cross-frame semantic instance matching, sparse point cloud three-dimensional reconstruction and graph structure optimization algorithm, detailedly describes the complete process of extracting semantic geometric information from image key frame, constructing sparse semantic point cloud and generating topological graph, with the advantages of high modeling precision, complete semantic expression and high automation degree.

[0006] According to the underground pipeline intelligent detection and mapping method based on image recognition provided by the embodiment of the present application, the following steps are included:

[0007] Step one: acquire continuous video images, select key frames, use LightGlue to associate feature matching pairs of adjacent key frames, establish pose graph, form pose parameter set;

[0008] Step two: input key frame into improved YOLO-World detection network, the improved YOLO-World detection network

[0009] The detection head of YOLO-World detection network includes cylinder geometric prior branch, and outputs a detection result set containing cylinder geometric prior parameters;

[0010] Step three: according to the detection result set, perform geometric consistency screening on feature matching pairs to obtain a geometric consistent matching set;

[0011] Step four: perform multi-view triangulation on the geometric consistent matching set to generate a three-dimensional sparse point set, calculate re-projection residual, and form a weight factor;

[0012] Step five: introduce the weight factor, perform incremental bundle adjustment optimization on the pose parameter set and the three-dimensional sparse point set to obtain a sparse semantic point cloud;

[0013] Step six: group the sparse semantic point cloud, fit the cylinder axis based on RANSAC and perform segmented clustering to generate a continuous three-dimensional center line segment set, generate graph edges according to adjacent center line segments and output underground pipe network topological map.

[0014] Optionally, the continuous video images are acquired, key frames are selected, feature matching pairs of adjacent key frames are associated using LightGlue, pose graph is established, and pose parameter set is formed, specifically as follows:

[0015] Acquire continuous video images and record time stamp, read camera intrinsic parameters, perform distortion correction on continuous video images to obtain a sequence of undistorted images;

[0016] According to time interval threshold and parallax threshold, key frames are selected from the sequence of undistorted images to form a sequence of key frames;

[0017] The SuperPoint feature extraction network is used to extract local features and calculate descriptor vectors for each keyframe in the keyframe sequence to obtain the keyframe feature set. Each pair of adjacent keyframes is selected sequentially from the keyframe sequence, and the keyframe feature set corresponding to the first keyframe and the keyframe feature set corresponding to the second keyframe are respectively used as the adjacent keyframe feature set.

[0018] LightGlue is used to perform feature matching on the feature sets of adjacent keyframes, and a set of feature matching pairs is output. The five-point method is used to solve the relative pose between adjacent keyframes based on the set of feature matching pairs.

[0019] Using the first keyframe as a reference coordinate system, the initial pose parameters of each keyframe are obtained by accumulating the pose chain; the pose chain refers to the ordered sequence of relative poses between adjacent keyframes arranged in chronological order in the keyframe sequence.

[0020] A pose graph is constructed using the initial pose parameters of each keyframe as nodes and the relative poses between adjacent keyframes as edges. The initial pose parameters of each keyframe are then aggregated to form a pose parameter set.

[0021] Optionally, the keyframes are input into the improved YOLO-World detection network, the improved...

[0022] The YOLO-World detection network's detection head includes a cylindrical geometry prior branch, which outputs detection results containing the cylindrical geometry prior parameters, specifically:

[0023] Load the improved YOLO-World detection network and set the target category vocabulary, classification confidence threshold, and non-maximum suppression threshold;

[0024] Size and pixel normalization are performed on keyframes to adjust them to a preset input resolution. The processed keyframes are input into the convolutional backbone to extract multi-scale backbone features. Multi-scale fusion with upsampling, downsampling and lateral connection is performed based on the feature pyramid network and the path aggregation network to generate a shared feature map.

[0025] The detection head branches run in parallel at the same spatial location sharing the feature map:

[0026] The category branch outputs a category probability vector, and the maximum component of the category probability vector is taken as the classification confidence.

[0027] The bounding box branch outputs bounding box parameters, and the bounding box branch regresses the bounding box parameters and generates a set of candidate bounding boxes.

[0028] The cylinder geometry prior branch outputs the cylinder geometry prior parameters;

[0029] The cylindrical geometry prior branch includes a feature aggregation unit, a depthwise separable convolution stack, and a one-dimensional convolution output unit:

[0030] First, generate a 7-channel parameter map for each shared feature map by sequentially performing a 1×1 convolution, two layers of 3×3 depthwise separable convolutions, and a 1×1 convolution.

[0031] Based on spatial location, establish a correspondence between the 7-channel parametric map and candidate bounding boxes generated at the same scale, and perform average pooling within a fixed window centered on the candidate bounding boxes to read out the original parameter vector:

[0032] Normalize the first and second dimensions to obtain the axial direction vector of the pipe segment;

[0033] The third dimension is constrained by nonnegativity and converted to pixel units based on the preset input resolution of the improved YOLO-World detection network and the downsampling ratio of the shared feature map to obtain the approximate radius of the image domain.

[0034] The fourth and fifth dimensions are used as the major and minor axis parameters of the projected ellipse, respectively, and adjusted in order of magnitude.

[0035] Use the sixth dimension as the orientation parameter of the projected ellipse;

[0036] The seventh dimension is normalized and used as the geometric confidence level;

[0037] When the same candidate bounding box obtains the original parameter vector on the shared feature map at the same time, the original parameter vector is weighted and fused according to the classification confidence to obtain the cylindrical geometric prior parameters of the candidate bounding box;

[0038] The prior geometric parameters of the cylinder include the axial direction vector of the pipe segment, the approximate radius of the image domain, and the parameters of the projected ellipse.

[0039] Candidate detection results are filtered based on classification confidence threshold and redundant suppression is performed based on non-maximum suppression threshold, retaining detection entries that meet the threshold conditions; the candidate detection results include class probability vector, classification confidence, bounding box parameters, cylindrical geometric prior parameters, and geometric confidence.

[0040] Perform instance association on the detection entries;

[0041] When the distance between the center point pixels is not greater than the preset distance threshold and the angle between the axial direction vectors is not greater than the preset angle threshold, it is determined as a matching pair. A one-to-one greedy strategy is used to select matching pairs from high to low until there are no matching pairs to choose from.

[0042] The same instance identifier is assigned to the successfully matched detection entries, a new instance identifier is assigned to the unmatched detection entries, and the instance identifier is terminated when there are consecutive unmatched entries for a preset number of frames.

[0043] The output detection result set is as follows: the detection entries that have been filtered and retained after redundancy suppression and assigned instance identifiers through instance association are summarized and stored in keyframe time order. Each detection entry includes a category label and classification confidence, bounding box parameters, cylindrical geometric prior parameters, and geometric confidence.

[0044] Optionally, based on the detection result set, geometric consistency screening is performed on the feature matching pairs to obtain a geometrically consistent matching set, specifically as follows:

[0045] Spatial matching is performed between the coordinates of the first keyframe feature points and the coordinates of the second keyframe feature points of each feature matching pair and the bounding box parameters of the second keyframe, and the matching point pairs that fall inside the corresponding bounding box in the second keyframe are selected.

[0046] Read the pipe segment axial direction vector and the approximate radius of the image domain from the prior cylindrical geometry parameters corresponding to each pair of retained matching points, and calculate the normal distance from the matching point pair to the axial direction line based on the coordinates of the feature points of the second keyframe and the center abscissa and center ordinate.

[0047] Remove matching point pairs whose normal distance is greater than the first distance threshold, and retain matching point pairs that satisfy the position constraints;

[0048] For matching point pairs that satisfy the constraints, calculate the angle between the matching vector and the axial direction vector of the pipe segment in the corresponding bounding box, filter out matching point pairs with an angle greater than the first angle threshold, and retain matching point pairs that satisfy the angle constraints.

[0049] The set of matching point pairs that simultaneously satisfy both position and angle constraints is defined as the geometrically consistent matching set.

[0050] Optionally, the step of performing multi-view triangulation on the geometrically consistent matching set to generate a three-dimensional sparse point set, calculating the reprojection residual, and forming a weighting factor specifically involves:

[0051] Map the coordinates of the first keyframe feature points and the second keyframe feature points of any geometrically consistent matching point pair to the camera's normalized image plane coordinates. Combine the pose parameters of the two frames to perform linear triangulation to obtain the corresponding three-dimensional spatial point coordinates. Repeat the above process for all geometrically consistent matching point pairs to generate a three-dimensional sparse point set.

[0052] The category label and instance identifier recorded in the detection entries of the first keyframe and the second keyframe of the feature matching point pair are respectively assigned to the corresponding three-dimensional sparse points to form a semantic instance sparse point set.

[0053] Read the depth value used in the second keyframe of linear triangulation for each 3D sparse point, as well as the image domain approximate radius in the corresponding cylindrical geometry prior parameters;

[0054] The approximate radius in the image domain is combined with the participating depth value and scaled to obtain the estimated diameter value;

[0055] The estimated diameter value is averaged with the major axis and minor axis parameters of the projected ellipse to obtain the projected diameter value. The difference between the estimated diameter value and the projected diameter value is calculated as the diameter consistency residual.

[0056] Perform forward projection based on the pose parameters of the keyframe corresponding to each 3D sparse point, and calculate the Euclidean distance between the projected pixel position and the original feature point coordinates as the reprojection residual.

[0057] The diameter consistency residual and the reprojection residual are combined in a weighted manner to form a weighting factor.

[0058] Optionally, the introduction of weighting factors to perform incremental bundle adjustment optimization on the pose parameter set and the 3D sparse point set to obtain a sparse semantic point cloud is specifically as follows:

[0059] The spatial coordinates of 3D sparse points are used as structural variable nodes, the pose parameters of each key frame are used as pose variable nodes, and the relative poses between key frames are used to construct edge constraints, thus establishing variable nodes and edge constraints for the local graph structure.

[0060] By applying weighting factors as confidence factors to the reprojection error and diameter consistency error respectively, a weighted minimization objective function is established for the observation error term in the local graph structure.

[0061] An incremental optimization method is used to sequentially add keyframes and corresponding 3D sparse points to the local graph structure, iteratively update the pose variable nodes and structure variable nodes, and perform nonlinear least squares optimization.

[0062] In each round of incremental optimization, a robust kernel function is introduced to jointly optimize the spatial coordinates of the pose variable nodes and the 3D sparse points of the key frame until the convergence condition is met.

[0063] The optimized pose parameters of each keyframe and the 3D sparse points of the semantic instance sparse point set are output as a sparse semantic point cloud.

[0064] Optionally, the step of grouping the sparse semantic point cloud, performing cylindrical axis fitting and piecewise clustering based on RANSAC, generating a continuous three-dimensional centerline segment set, generating graph edges based on adjacent centerline segments, and outputting an underground pipeline network topology map specifically involves:

[0065] The 3D sparse points in the sparse semantic point cloud are grouped according to the instance identifier to form a semantic instance sparse point set;

[0066] For each set of sparse points of semantic instances, the coordinates of the points and the corresponding axial direction vector of the pipe segment are read. The outliers are iteratively eliminated using a RANSAC-based straight line fitting method to obtain candidate cylindrical axes.

[0067] Based on the candidate cylindrical axis, the three-dimensional sparse points in the semantic instance sparse point set are segmented and clustered into approximately straight line segments to generate a continuous three-dimensional center line segment set.

[0068] Read the spatial relationship between adjacent line segments in a continuous three-dimensional centerline segment set, and generate the corresponding graph edge when there are intersections or adjacent endpoints;

[0069] Read the 3D positions of valves, tees, elbows, and flanges marked with detection entries from the sparse semantic point cloud, and generate graph nodes at the corresponding positions;

[0070] The graph structure consisting of graph nodes and graph edges is stored as an underground pipeline network topology map, and the nodes, edges and attribute fields are recorded in the underground pipeline network topology map.

[0071] The beneficial effects of this invention are:

[0072] This invention proposes an intelligent detection and mapping method for underground pipelines based on image recognition, possessing significant technical advantages and application value. By introducing keyframe selection and a robust feature matching mechanism based on LightGlue, it can effectively improve image registration accuracy in complex underground environments such as low-texture and repetitive textures, providing a stable pose map and feature pair foundation for subsequent 3D reconstruction. Furthermore, combined with an improved...

[0073] The YOLO-World detection network introduces a cylindrical geometric prior branch into the detection head, which not only achieves high-precision recognition of underground pipeline structural features, but also significantly enhances the geometric representation capability of semantic instances by extracting axial direction vectors and image domain geometric parameters, effectively supporting subsequent geometric consistency judgment and 3D reconstruction.

[0074] The proposed screening mechanism based on pipe segment geometric consistency integrates spatial location, radius constraints, and angular constraints, making the triangulation of feature matching points more stable. Combined with weighting factors constructed from reprojection residuals and diameter consistency residuals, it effectively suppresses anomalous errors during incremental bundle adjustment optimization, improving the accuracy and robustness of the overall sparse semantic point cloud. Through a continuous centerline segment reconstruction method based on RANSAC fitting and segmented clustering, and a graph-based topology mapping process combining semantic annotations of pipeline components, a closed-loop mapping system from image acquisition to structured topology output is realized. This significantly improves the intelligence and automation level of underground pipeline detection and structure reconstruction, demonstrating promising engineering application prospects. Attached Figure Description

[0075] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0076] Figure 1 This is an overall flowchart of an intelligent detection and mapping method for underground pipelines based on image recognition proposed in this invention.

[0077] Figure 2 This is a schematic diagram of the improved YOLO-World detection network structure of an image recognition-based intelligent detection and mapping method for underground pipelines proposed in this invention.

[0078] Figure 3 This is a flowchart of the geometric consistency matching and screening process for an intelligent detection and mapping method for underground pipelines based on image recognition proposed in this invention.

[0079] Figure 4 This is a schematic diagram of the cylindrical geometric prior branch structure of an image recognition-based intelligent detection and mapping method for underground pipelines proposed in this invention. Detailed Implementation

[0080] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0081] refer to Figures 1-4 A method for intelligent detection and mapping of underground pipelines based on image recognition includes the following steps:

[0082] Step 1: Acquire continuous video images, select keyframes, use LightGlue to associate feature matching pairs of adjacent keyframes, establish a pose graph, and form a pose parameter set;

[0083] Step 2: Input the keyframes into the improved YOLO-World detection network, the improved...

[0084] The YOLO-World detection network's detection head includes a cylindrical geometry prior branch, which outputs a set of detection results containing the cylindrical geometry prior parameters.

[0085] Step 3: Based on the detection result set, perform geometric consistency screening on the feature matching pairs to obtain a geometrically consistent matching set;

[0086] Step 4: Perform multi-view triangulation on the geometrically consistent matching set to generate a 3D sparse point set, calculate the reprojection residual, and form the weighting factor;

[0087] Step 5: Introduce weighting factors and perform incremental bundle adjustment optimization on the pose parameter set and the 3D sparse point set to obtain the sparse semantic point cloud;

[0088] Step 6: Group the sparse semantic point cloud, perform cylindrical axis fitting and segmented clustering based on RANSAC, generate a continuous three-dimensional centerline segment set, generate graph edges based on adjacent centerline segments, and output the underground pipeline network topology map.

[0089] This invention proposes an intelligent detection and mapping method for underground pipelines based on image recognition. First, a pose graph is established by keyframe selection and LightGlue feature matching to obtain camera pose information. Second, an improved YOLO-World detection network is used to introduce cylindrical geometric prior parameters to achieve high-precision identification and instance segmentation of underground pipeline components. Then, a three-dimensional sparse point set is generated by geometric consistency screening and multi-view triangulation, and weight factors are calculated. In incremental bundle adjustment optimization, a robust kernel function is introduced to achieve joint optimization of keyframe pose and sparse point coordinates. Finally, a continuous three-dimensional centerline segment set is generated based on RANSAC fitting and piecewise clustering to construct node and edge information and output a clearly structured underground pipeline network topology map.

[0090] In this embodiment, the steps of acquiring continuous video images, selecting keyframes, using LightGlue to associate feature matching pairs of adjacent keyframes, establishing a pose graph, and forming a pose parameter set are as follows:

[0091] Acquire continuous video images and record timestamps, read camera intrinsic parameters, perform distortion correction on continuous video images, and obtain a distortion-free image sequence: perform distortion correction on continuous video images using the inverse mapping distortion correction method based on the Brown-Conrady distortion model, calculate the distortion-free pixel coordinates using the intrinsic parameter matrix K and distortion coefficients, and generate a distortion-free image sequence by bilinear interpolation resampling;

[0092] Keyframes are selected from the distorted image sequence based on time interval thresholds and disparity thresholds to form a keyframe sequence. Specifically, the time interval between the current frame and the nearest keyframe and the median disparity obtained based on the matched inliers are calculated. When the time interval reaches or exceeds the preset time interval threshold and the median disparity reaches or exceeds the preset disparity threshold, the current frame is determined as a keyframe. When the median disparity reaches or exceeds twice the preset disparity threshold, the current frame is determined as a keyframe in advance. When the time interval reaches or exceeds twice the preset time interval threshold, the current frame is determined as a keyframe in a fallback manner. The keyframe sequence is formed by traversing the distorted image sequence according to the above rules.

[0093] The SuperPoint feature extraction network is used to extract local features and calculate descriptor vectors for each keyframe in the keyframe sequence, resulting in a keyframe feature set. Specifically, this involves: performing brightness normalization and denoising on the keyframes to obtain a preprocessed image; inputting the preprocessed image into the SuperPoint feature extraction network to obtain a keypoint confidence heatmap and a dense descriptor feature map; performing threshold filtering and non-maximum suppression on the keypoint confidence heatmap to obtain a candidate local feature point set; selecting local feature points in descending order of confidence, up to a preset limit, and performing sub-pixel refinement on the selected local feature points; performing bilinear interpolation on the dense descriptor feature map using the sub-pixel coordinates of the local feature points to obtain descriptor vectors; performing normalization on the descriptor vectors; and summarizing the pixel coordinates, confidence, optional scale information, and corresponding descriptor vectors of the local feature points to form the keyframe feature set.

[0094] Each pair of adjacent keyframes is selected sequentially from the keyframe sequence, and the keyframe feature set corresponding to the first keyframe and the keyframe feature set corresponding to the second keyframe are respectively used as adjacent keyframe feature sets; the adjacent keyframe pair includes the first keyframe and the second keyframe.

[0095] LightGlue is used to perform feature matching on the feature sets of adjacent keyframes, outputting a set of feature matching pairs. Each pair of adjacent keyframes is selected sequentially from the keyframe sequence, and the keyframe feature sets of the first and second keyframes are read. The pixel coordinates of local feature points in the two keyframes are mapped to a distortion-free pixel coordinate system and coordinate normalization is performed. The descriptor vectors in the two keyframes are also normalized. LightGlue inference is called with the coordinates and descriptors of the two keyframes as input to obtain the matching probability output. Candidate matches are filtered according to a preset matching confidence threshold, and a bidirectional consistency check is performed to preserve the one-to-one correspondence. For each preserved match, a matching vector is calculated in the distortion-free pixel coordinate system, and the matching confidence is recorded. The matching set is pruned according to spatial balanced sampling or a maximum number of samples. The output is a set of feature matching pairs containing the index, coordinates, matching vector, and matching confidence.

[0096] The relative pose between adjacent keyframes is solved using the five-point method based on feature matching pairs.

[0097] Specifically, this includes: mapping the distortion-free pixel coordinates of feature matching pairs to normalized image plane coordinates using camera intrinsic parameters; selecting five pairs of matching points round by round using random consistent sampling and solving for candidate essential matrices using the five-point method; calculating the residuals of all matching pairs for each candidate essential matrix based on epipolar constraints and determining the inlier set according to a preset residual threshold, selecting the candidate with the largest number of inliers; performing nonlinear refinement on the essential matrix on the selected inlier set and applying singular value constraints to the essential matrix; decomposing the refined essential matrix to obtain rotation matrix candidates and translation direction candidates, and selecting a unique rotation and translation direction by combining inlier triangulation and positive depth determination; and normalizing the translation vector to obtain a scale-invariant relative pose.

[0098] Using the first keyframe as a reference coordinate system, the initial pose parameters of each keyframe are obtained by accumulating the pose chain; the pose chain refers to the ordered sequence of relative poses between adjacent keyframes arranged in chronological order in the keyframe sequence.

[0099] A pose graph is constructed using the initial pose parameters of each keyframe as nodes and the relative poses between adjacent keyframes as edges. The initial pose parameters of each keyframe are then aggregated to form a pose parameter set.

[0100] This invention acquires continuous video images and performs distortion correction and keyframe selection, dynamically generating a keyframe sequence by combining time intervals and disparity thresholds. Based on this, the SuperPoint feature extraction network is used to extract local features and descriptor vectors for each keyframe, and the LightGlue algorithm is employed to perform high-confidence feature matching on adjacent keyframes, obtaining a set of feature matching pairs containing coordinates, matching vectors, and confidence scores. Furthermore, the essential matrix between adjacent keyframes is calculated using the five-point method, completing the analysis and decomposition of rotation and translation vectors, constructing scale-invariant relative poses, and using the first keyframe as a reference frame, recovering the initial pose parameters of all keyframes through pose chain multiplication. Finally, a pose graph is established using the initial poses of the keyframes as graph nodes and the relative poses as graph edges, forming a pose parameter set for 3D reconstruction.

[0101] In this embodiment, the keyframe is input into the improved YOLO-World detection network. The detection head of the improved YOLO-World detection network includes a cylindrical geometry prior branch, and the output is a detection result containing cylindrical geometry prior parameters. Specifically:

[0102] Load the improved YOLO-World detection network and set the target category vocabulary, classification confidence threshold, and non-maximum suppression threshold;

[0103] The keyframes are normalized in size and pixels to adjust to a preset input resolution. Multi-scale backbone features are extracted from the processed keyframes and input into the convolutional backbone. Multi-scale fusion, including upsampling, downsampling, and lateral connections, is performed based on a feature pyramid network and a path aggregation network to generate a shared feature map. The convolutional backbone consists of stacked convolutional layers, batch normalization layers, and non-linear activation layers, and outputs the corresponding original... Figure 1 Three sets of backbone features at scales of 8, 1 / 16, and 1 / 32 are used; the backbone features are fused at multiple scales based on the feature pyramid network and the path aggregation network to generate a shared feature map, which can be used in parallel by the category branch, the bounding box branch, and the cylindrical geometry prior branch.

[0104] The detection head branches run in parallel at the same spatial location sharing the feature map:

[0105] The category branch outputs a category probability vector, and the maximum component of the category probability vector is taken as the classification confidence.

[0106] The bounding box branch outputs bounding box parameters, and the bounding box branch regresses the bounding box parameters and generates a set of candidate bounding boxes. The bounding box parameters include the center x-coordinate, the center y-coordinate, the width and height, and the bounding box confidence score.

[0107] The cylinder geometry prior branch outputs the cylinder geometry prior parameters;

[0108] The cylindrical geometry prior branch includes a feature aggregation unit, a depthwise separable convolution stack, and a one-dimensional convolution output unit:

[0109] First, generate a 7-channel parameter map for each shared feature map by sequentially performing a 1×1 convolution, two layers of 3×3 depthwise separable convolutions, and a 1×1 convolution.

[0110] Based on spatial location, establish a correspondence between the 7-channel parametric map and candidate bounding boxes generated at the same scale, and perform average pooling within a fixed window centered on the candidate bounding boxes to read out the original parameter vector:

[0111] Normalize the first and second dimensions to obtain the axial direction vector of the pipe segment;

[0112] The third dimension is constrained by nonnegativity and converted to pixel units based on the preset input resolution of the improved YOLO-World detection network and the downsampling ratio of the shared feature map to obtain the approximate radius of the image domain.

[0113] The fourth and fifth dimensions are used as the major and minor axis parameters of the projected ellipse, respectively, and adjusted in order of magnitude.

[0114] Use the sixth dimension as the orientation parameter of the projected ellipse;

[0115] The seventh dimension is normalized and used as the geometric confidence level;

[0116] When the same candidate bounding box obtains the original parameter vector on the shared feature map at the same time, the original parameter vector is weighted and fused according to the classification confidence to obtain the cylindrical geometric prior parameters of the candidate bounding box;

[0117] The prior geometric parameters of the cylinder include the axial direction vector of the pipe segment, the approximate radius of the image domain, and the parameters of the projected ellipse; the parameters of the projected ellipse include the major axis parameter, the minor axis parameter, and the orientation parameter;

[0118] Candidate detection results are filtered based on classification confidence threshold and redundant suppression is performed based on non-maximum suppression threshold, retaining detection entries that meet the threshold conditions; the candidate detection results include class probability vector, classification confidence, bounding box parameters, cylindrical geometric prior parameters, and geometric confidence.

[0119] The filtering based on classification confidence threshold is as follows: input a set of candidate detection results; read the classification confidence of each result and compare it with a preset classification confidence threshold; delete entries with classification confidence below the threshold to form a first candidate set;

[0120] The redundancy suppression based on the non-maximum suppression threshold is as follows: the first candidate set is grouped by category label; for each category group, it is sorted from high to low classification confidence; the highest-scoring item is selected and added to the retention set, and the intersection-union ratio (IU) of the bounding box of the item and the bounding boxes of the remaining items is calculated; items with IU greater than or equal to the preset non-maximum suppression threshold are deleted; this process is repeated until the category group is processed; the retention sets of each category are merged to obtain the detected items.

[0121] Instance association is performed on the detected items, specifically: the detected items from the previous keyframe and the current keyframe are grouped into a set to be matched according to category; for any detected item from the previous keyframe and any detected item from the current keyframe, the pixel distance between the center points is calculated based on the center x-coordinate and center y-coordinate in the bounding box parameters, and the candidate matching pairs are sorted by combining the angle threshold of the pipe segment axial direction vector and the classification confidence.

[0122] When the distance between the center point pixels is not greater than the preset distance threshold and the angle between the axial direction vectors is not greater than the preset angle threshold, it is determined as a matching pair. A one-to-one greedy strategy is used to select matching pairs from high to low until there are no matching pairs to choose from.

[0123] The same instance identifier is assigned to the successfully matched detection entries, a new instance identifier is assigned to the unmatched detection entries, and the instance identifier is terminated when there are consecutive unmatched entries for a preset number of frames.

[0124] The output detection result set is as follows: the detection entries that have been filtered and retained after redundancy suppression and assigned instance identifiers through instance association are summarized and stored in keyframe time order. Each detection entry includes a category label and classification confidence, bounding box parameters, cylindrical geometric prior parameters and geometric confidence. An index relationship of keyframe identifier-instance identifier-detection entry is established and output.

[0125] This invention introduces an improved YOLO-World detection network to jointly detect semantic instances and geometric priors in keyframe images. The detection head includes a category branch, a bounding box branch, and a cylindrical geometric prior branch. It outputs category labels, bounding box parameters, and cylindrical geometric prior parameters in parallel on a shared feature map. The geometric prior parameters include the axial direction vector, the approximate radius of the image domain, and the projection ellipse parameters. The network achieves efficient detection through multi-scale feature fusion, depthwise separable convolution, and weighted geometric parameter aggregation mechanisms. It combines classification confidence and geometric confidence to perform filtering and non-maximum suppression, and then performs cross-frame instance association based on position and orientation constraints. Finally, it outputs a high-quality detection result set with instance labels, providing accurate geometric priors and semantic labels for subsequent 3D sparse reconstruction and topology mapping.

[0126] In this embodiment, the step of performing geometric consistency screening on feature matching pairs based on the detection result set to obtain a geometrically consistent matching set specifically involves:

[0127] Spatial matching is performed between the coordinates of the first keyframe feature points and the coordinates of the second keyframe feature points of each feature matching pair and the bounding box parameters of the second keyframe, and the matching point pairs that fall inside the corresponding bounding box in the second keyframe are selected.

[0128] Read the pipe segment axial direction vector and the approximate radius of the image domain from the prior cylindrical geometry parameters corresponding to each pair of retained matching points, and calculate the normal distance from the matching point pair to the axial direction line based on the coordinates of the feature points of the second keyframe and the center abscissa and center ordinate.

[0129] Remove matching point pairs whose normal distance is greater than the first distance threshold, and retain matching point pairs that satisfy the position constraints;

[0130] For matching point pairs that satisfy the constraints, calculate the angle between the matching vector and the axial direction vector of the pipe segment in the corresponding bounding box, filter out matching point pairs with an angle greater than the first angle threshold, and retain matching point pairs that satisfy the angle constraints.

[0131] The set of matching point pairs that simultaneously satisfy both position and angle constraints is defined as the geometrically consistent matching set.

[0132] This invention introduces prior cylindrical geometric parameters to perform geometric consistency screening of feature matching pairs between adjacent keyframes. It uses the spatial positional relationship of matching points within the bounding box, the normal distance to the axial direction line of the cylinder, and the angle between the matching points and the axial direction vector as screening criteria. Positional constraints and angular constraints are applied sequentially to effectively eliminate erroneous matching point pairs that do not conform to the target geometric characteristics. Finally, a geometrically consistent matching set with structural consistency is constructed, providing highly reliable input for subsequent 3D sparse point reconstruction and optimization.

[0133] In this embodiment, the step of performing multi-view triangulation on the geometrically consistent matching set to generate a three-dimensional sparse point set, calculating the reprojection residual, and forming a weighting factor specifically involves:

[0134] Map the coordinates of the first keyframe feature points and the second keyframe feature points of any geometrically consistent matching point pair to the camera's normalized image plane coordinates. Combine the pose parameters of the two frames to perform linear triangulation to obtain the corresponding three-dimensional spatial point coordinates. Repeat the above process for all geometrically consistent matching point pairs to generate a three-dimensional sparse point set.

[0135] The linear triangulation performed by combining the pose parameters of the two frames to obtain the corresponding three-dimensional spatial point coordinates is as follows: the feature point coordinates of the first keyframe and the feature point coordinates of the second keyframe of any geometrically consistent matching point pair are mapped to the camera normalized image plane coordinates, which are denoted as the first frame normalized coordinates and the second frame normalized coordinates, respectively; the pose parameters corresponding to the first keyframe and the second keyframe are read, the projection direction line equations from the camera center of the two frames to the three-dimensional point are constructed, and the midpoint of the shortest connection between the two lines is solved using the least squares method as the three-dimensional spatial point coordinates corresponding to the matching point pair;

[0136] The category label and instance identifier recorded in the detection entries of the first keyframe and the second keyframe of the feature matching point pair are respectively assigned to the corresponding three-dimensional sparse points to form a semantic instance sparse point set.

[0137] Read the depth value used in the second keyframe of linear triangulation for each 3D sparse point, as well as the image domain approximate radius in the corresponding cylindrical geometry prior parameters;

[0138] The approximate radius in the image domain is combined with the participating depth value and scaled to obtain the estimated diameter value;

[0139] The estimated diameter value is averaged with the major axis and minor axis parameters of the projected ellipse to obtain the projected diameter value. The difference between the estimated diameter value and the projected diameter value is calculated as the diameter consistency residual.

[0140] Perform forward projection based on the pose parameters of the keyframe corresponding to each 3D sparse point, and calculate the Euclidean distance between the projected pixel position and the original feature point coordinates as the reprojection residual; specifically:

[0141] Read the spatial coordinates of the 3D sparse points and the camera intrinsic parameters and pose parameters of the corresponding keyframes, transform the spatial coordinates of the 3D sparse points into point coordinates in the camera coordinate system of the corresponding keyframes, and calculate the normalized coordinates on the image plane according to the pinhole imaging model.

[0142] The camera intrinsic parameter matrix is ​​used to map the normalized coordinates to the projected pixel coordinates, which are used as the projection positions of the 3D sparse points.

[0143] The Euclidean distance between the projected position of the 3D sparse point and the pixel coordinates of the original feature matching point pair in the corresponding keyframe is calculated and defined as the reprojection residual of the 3D sparse point in the keyframe.

[0144] The above operation is performed sequentially on all 3D sparse points in the semantic instance sparse point set, and the reprojection residual is calculated in their respective keyframes.

[0145] The diameter consistency residual and the reprojection residual are combined in a weighted manner to form a weighting factor.

[0146] After obtaining the geometrically consistent matching set, this invention performs multi-view triangulation on the matching point pairs based on the keyframe pose parameters to construct a 3D sparse point set with semantic labels and instance identifiers. Combining the prior parameters of cylindrical geometry and projection characteristics, it calculates the diameter consistency residual and reprojection residual of the 3D points. By weighted combination, it constructs a weight factor that reflects the geometric fitting accuracy and reconstruction consistency, providing more robust error constraints and feature reliability scores for the subsequent bundle adjustment and optimization process, effectively improving the accuracy and semantic stability of 3D reconstruction.

[0147] In this embodiment, the introduction of a weighting factor to perform incremental bundle adjustment optimization on the pose parameter set and the 3D sparse point set to obtain a sparse semantic point cloud specifically involves:

[0148] The spatial coordinates of 3D sparse points are used as structural variable nodes, the pose parameters of each key frame are used as pose variable nodes, and the relative poses between key frames are used to construct edge constraints, thus establishing variable nodes and edge constraints for the local graph structure.

[0149] Using weighting factors as confidence factors, respectively applied to the reprojection error and diameter consistency error, a weighted minimization objective function is established for the observation error term in the local graph structure:

[0150] 3D sparse point X j The projection residual in keyframe i is e. proj ij, the corresponding diameter consistency residual is e diam ij, with weighting factor w ij The weighted minimization objective function is then expressed as:

[0151]

[0152] Where: R i ,t i X represents the pose variable of keyframe i; j Represents the coordinates of the j-th three-dimensional sparse point in the set of semantic instance sparse points; For reprojection residuals; For diameter consistency residuals; w ij The weighting factor is calculated based on the residual confidence level.

[0153] The observation error terms are the reprojection error term and the diameter consistency error term;

[0154] An incremental optimization method is used to sequentially add keyframes and corresponding 3D sparse points to the local graph structure, iteratively update the pose variable nodes and structure variable nodes, and perform nonlinear least squares optimization.

[0155] The incremental optimization method involves sequentially adding keyframes and corresponding 3D sparse points to the local graph structure, specifically as follows:

[0156] When performing incremental optimization, the newly added keyframe and its corresponding semantic instance sparse points are first read from the pose graph. The pose parameters of the newly added keyframe are used as new pose variable nodes, the spatial coordinates of the newly added semantic instance sparse points are used as new structure variable nodes, and the observation edge constraints between the keyframe and its observable sparse points and the relative pose constraints between the keyframe and the previous keyframe are established in the local graph structure.

[0157] In the local graph structure, the existing optimization results are used as initial values ​​to construct a local subgraph consisting of newly added variable nodes and related observation edges. In this local subgraph, the pose variable nodes of the newly added keyframes and the structure variable nodes of the related sparse points are updated simultaneously, and the weighted residual evaluation is performed on the diameter consistency error and reprojection error.

[0158] During the optimization iteration process, the newly added keyframes and their related sparse points are first initially adjusted to ensure that the new variables maintain continuity and geometric consistency with the existing variables; then, the robust kernel function is used to suppress observations with excessive residuals, gradually reducing the overall residuals.

[0159] Through multiple rounds of iterative updates, until the change in the observed residual is less than a preset threshold or the maximum number of iterations is reached, the optimization results are written into the global pose graph to obtain the converged keyframe pose parameters and sparse point spatial coordinates.

[0160] The nonlinear least squares optimization is performed as follows: In the constructed local graph structure, the keyframe pose variable nodes and 3D sparse point structure variable nodes are used as optimization objects, and the observation error term is used as the constraint target. The local subgraph composed of the newly added keyframe, its observed sparse points, and the edge constraints between adjacent keyframes is solved. During the solution process, the values ​​of pose variables and structure variables are gradually adjusted through iteration to gradually reduce the weighted residual. In each iteration, a robust kernel function is introduced to reduce the impact of abnormal observations on the optimization results, and the calculation is terminated when the iteration convergence condition is met or the maximum number of iterations is reached. Finally, the updated keyframe pose parameters and sparse point spatial coordinates are output, and the optimization results are written into the global graph structure to maintain overall consistency and continuity.

[0161] In each round of incremental optimization, a robust kernel function is introduced to jointly optimize the spatial coordinates of the pose variable nodes and the 3D sparse points of the key frame until the convergence condition is met.

[0162] Convergence conditions include:

[0163] The total change of all observation error terms in the local graph structure is lower than a preset threshold in two consecutive iterations.

[0164] The update magnitudes of the rotation and translation vectors of the keyframe pose variable nodes, as well as the coordinate update magnitudes of the 3D sparse point structure variable nodes, are all lower than the preset thresholds.

[0165] Optimize the number of iterations to reach the preset maximum number of iterations;

[0166] The method involves introducing a robust kernel function in each incremental optimization round to jointly optimize the pose variable nodes of keyframes and the spatial coordinates of 3D sparse points. Specifically, the pose parameters of keyframes and the 3D coordinates of relevant sparse points are read simultaneously in the local graph structure, treating both types of variables as the solution objects of the same optimization problem. During iterative updates, constraints are applied to the pose variable nodes and structure variable nodes through reprojection residuals and diameter consistency residuals, ensuring that the pose parameters of keyframes and the spatial coordinates of sparse points are adjusted synchronously in the same optimization process. In each iteration, the sensitivity of observation errors to pose and structure variables is first calculated, and this sensitivity is used as a directional reference for variable updates. Subsequently, the pose and structure variables are updated simultaneously based on the weighted residuals to ensure consistent convergence of the camera trajectory and the sparse point coordinates.

[0167] The optimized pose parameters of each keyframe and the 3D sparse points of the semantic instance sparse point set are output as a sparse semantic point cloud.

[0168] This invention, based on the construction of a keyframe pose graph and a semantic sparse point set, introduces an observation error term consisting of reprojection residuals and diameter consistency residuals, and constructs a weighted minimization objective function using weighting factors. An incremental graph optimization strategy iteratively updates the pose variable nodes of keyframes and the structural variable nodes of 3D sparse points within the local graph structure. In each optimization round, a robust kernel function is introduced to suppress the influence of anomalous observations. A nonlinear least squares optimization method is used to achieve joint optimization calculations under multi-source observation constraints. The iteration termination time is dynamically determined based on convergence conditions. Finally, continuous and consistent keyframe poses and high-quality sparse point cloud coordinates are output, generating a structurally stable and semantically clear sparse semantic point cloud, providing an accurate foundation for subsequent topology reconstruction and network graph generation.

[0169] In this embodiment, the process of grouping sparse semantic point clouds, using RANSAC-based cylindrical axis fitting and segmented clustering to generate a continuous three-dimensional centerline segment set, generating graph edges based on adjacent centerline segments, and outputting an underground pipeline network topology map specifically involves:

[0170] The 3D sparse points in the sparse semantic point cloud are grouped according to instance identifiers to form a semantic instance sparse point set: the semantic label information and instance identifier carried by each 3D sparse point are read. The instance identifier is a unique mark assigned by comprehensively matching the target category, bounding box parameters and cylindrical geometric prior parameters during the early keyframe detection and instance association process, and is used to indicate the specific target entity to which the sparse point belongs.

[0171] Traverse the entire sparse semantic point cloud dataset, read the structural attributes of each 3D sparse point, including spatial coordinates, category label, corresponding instance identifier, and associated geometric attributes (such as axial direction vector, estimated radius, etc.), and construct a sparse point object containing metadata.

[0172] Clustering is performed on all sparse point objects based on instance identifiers: sparse points with the same instance identifier are grouped into the same set, thus forming multiple independent sparse point sets. Each set represents a target entity with geometric continuity and semantic consistency. In practical applications, these sets typically correspond to specific pipe segments, valves, tees, and other independent components in underground pipeline networks.

[0173] During the grouping process, outlier identifiers need to be removed to ensure that each sparse point belongs to only one instance set. For sparse points with missing or overlapping identifiers, auxiliary merging or removal can be performed based on spatial proximity and semantic similarity rules to enhance the stability and robustness of instance grouping.

[0174] After completing the above grouping, a set of semantic instance sparse points is finally formed with instance identifiers as index keys. Each group contains spatially related three-dimensional sparse points and their attribute information, providing structured input data for subsequent processing such as cylindrical axis fitting, segmented clustering and topology mapping.

[0175] For each set of semantic instance sparse points, the coordinates of the points and the corresponding axial direction vector of the pipe segment are read. The outliers are iteratively eliminated using a RANSAC-based straight line fitting method to obtain candidate cylindrical axes. For the semantic instance sparse point sets that have been grouped, the geometric structure modeling operation is performed group by group. For any set of semantic instance sparse points, all sparse points in the set are traversed first, and the spatial three-dimensional coordinate information of each sparse point and the axial direction vector of the pipe segment in the cylindrical geometric prior parameters obtained in the detection stage are extracted.

[0176] In the data preprocessing stage, the 3D coordinate set of sparse points is centered, that is, the centroid of the point cloud is used as the reference center, and the coordinates of all points are translated and normalized to reduce the interference of overall offset or scale changes on the fitting accuracy. The axial direction vector associated with each point is unit vectorized to ensure that the direction information maintains a consistent scale in the calculation.

[0177] After processing the point and direction data, an iterative modeling method based on the RANSAC (Random Sample Consensus) algorithm was used to fit the sparse point set. The RANSAC algorithm constructs candidate line models by repeatedly sampling the minimum set of point pairs within the point set and calculates the perpendicular distance of all sparse points relative to the model as an error metric. Points with errors below a set tolerance threshold are marked as "inside points," and the rest are marked as "outside points."

[0178] In each iteration, the algorithm updates the current best model and the set of inliers until any of the following convergence conditions are met: the number of inliers reaches a preset proportion threshold, the maximum number of iterations reaches an upper limit, or the current fitting error is lower than the global minimum error threshold. During the iteration process, RANSAC effectively removes noise points, outliers, and mismatched points that deviate from the principal direction, while retaining principal axis points with high consistency.

[0179] The optimal straight line model parameters obtained based on the RANSAC algorithm are used as candidates for the cylindrical axis corresponding to the sparse point set of the semantic instance, which approximately represents the geometric skeleton line of the pipe segment along the main axis direction, providing accurate structural prior information for subsequent segmented clustering and topology graph construction.

[0180] Based on the candidate cylindrical axis, the 3D sparse points in the semantic instance sparse point set are segmented and clustered into approximately straight line segments to generate a continuous set of 3D centerline segments. After obtaining the candidate cylindrical axis corresponding to each set of semantic instance sparse points, the axis is used as the principal direction reference to further perform spatial segmentation clustering operations on the 3D sparse points in the group based on the principal axis direction. Specifically, the spatial coordinates of all 3D sparse points in the current set of semantic instance sparse points are first read and projected onto the candidate cylindrical axis corresponding to the group to obtain the projected coordinate value of each sparse point in the principal axis direction, representing its relative position along the axis direction.

[0181] Then, the sparse points are sorted one-dimensionally according to the order of the projected coordinates. Points whose differences between adjacent positions exceed a set spacing threshold are used as boundary points of different segments, thus initially dividing continuous point segment intervals. Within each interval, density clustering analysis is further performed to determine whether the local distribution density of sparse points within the interval meets the set connectivity threshold. If there are abrupt changes in sparse point density or gap distances greater than the clustering tolerance distance, the segment is further subdivided to ensure that the clustering results can accurately characterize the spatial structural features of the actual pipeline segment.

[0182] For each clustered point set, the least squares method is used to fit its three-dimensional coordinates to a straight line, generating an approximate straight line segment. The starting and ending points of this straight line segment are then used as endpoints to construct a three-dimensional centerline segment. This process is repeated for all clustered segments, ultimately generating a set of continuous three-dimensional centerline segments with spatial order, which fully represents the actual pipe segment structure corresponding to the sparse point set of the semantic instance.

[0183] Read the spatial relationship between adjacent line segments in a continuous three-dimensional centerline segment set, and generate the corresponding graph edge when there are intersections or adjacent endpoints;

[0184] Read the 3D positions of valves, tees, elbows, and flanges marked with detection entries from the sparse semantic point cloud, and generate graph nodes at the corresponding positions;

[0185] The graph structure consisting of graph nodes and graph edges is stored as an underground pipeline network topology map, and the nodes, edges and attribute fields are recorded in the underground pipeline network topology map.

[0186] This invention groups sparse semantic point clouds according to instance identifiers, combines a RANSAC-based cylindrical axis fitting method with a segmented clustering strategy, and constructs a set of continuous three-dimensional centerline segments that conform to geometric features. Furthermore, it constructs graph edges based on the spatial relationship between adjacent centerline segments, and establishes graph nodes by combining the spatial location information of valves, tees, elbows, flanges, and other components already identified in the sparse semantic point cloud. Finally, it forms an underground pipeline network topology map containing nodes, edges, and semantic attribute fields, realizing an efficient conversion from sparse point clouds to structured graph models, and possessing good engineering application scalability and information integrity.

[0187] The specific steps and structure of the improved YOLO-World detection network are as follows:

[0188] Load the improved YOLO-World detection network and complete the parameter settings, including setting the target category vocabulary, preset input resolution, classification confidence threshold and non-maximum suppression threshold, and initializing the category branch, bounding box branch and cylindrical geometry prior branch.

[0189] Size and pixel normalization are performed on the keyframes to adjust them to the preset input resolution. The processed keyframes are input to the convolutional backbone to extract multi-scale backbone features. Multi-scale fusion of upsampling, downsampling and lateral connection is completed based on the feature pyramid network and the path aggregation network to generate a three-scale shared feature map for parallel use by each branch of the detection head.

[0190] The detection head branches run in parallel at the same spatial location on the shared feature map: the category branch outputs the category probability vector, and takes the maximum component of the category probability vector as the classification confidence; the bounding box branch regresses the bounding box parameters, outputs the center x-coordinate, center y-coordinate, width, height and bounding box confidence, and generates a set of candidate bounding boxes.

[0191] The cylindrical geometry prior branch consists of a feature aggregation unit, two layers of 3×3 depthwise separable convolution stacks, and a one-dimensional convolution output unit. For each shared feature map, a 7-channel parameter map is generated by sequentially passing a 1×1 convolution, two layers of 3×3 depthwise separable convolutions, and a 1×1 convolution. The 7-channel parameter map is then correlated with candidate bounding boxes generated at the same scale according to their spatial location. Average pooling is then performed within a fixed window with the center of the candidate bounding box as the center, and the original parameter vector is read out.

[0192] Deterministic analysis is performed on the original parameter vector: the first and second dimensions are normalized to obtain the axial direction vector of the pipe segment; a non-negative constraint is applied to the third dimension, and it is converted to pixel units according to the preset input resolution and the downsampling ratio of the shared feature map to obtain the approximate radius of the image domain; the fourth and fifth dimensions are used as the major and minor axis parameters of the projection ellipse, and are adjusted in order of magnitude; the sixth dimension is used as the orientation parameter of the projection ellipse; the seventh dimension is normalized as the geometric confidence.

[0193] When the same candidate bounding box obtains the original parameter vectors on the multi-scale shared feature map at the same time, the original parameter vectors of multiple sets are weighted and fused according to the classification confidence to obtain the cylindrical geometric prior parameters of the candidate bounding box; the candidate detection results are assembled at the candidate bounding box position, and the candidate detection results include the class probability vector, classification confidence, bounding box parameters, cylindrical geometric prior parameters and geometric confidence.

[0194] The candidate detection results are filtered based on the classification confidence threshold. Entries with classification confidence below the threshold are deleted to form the first candidate set. The first candidate set is grouped by category label and non-maximum suppression is performed. The entries are sorted from high to low classification confidence and the highest-scoring entries are added to the retention set. The intersection-union ratio (IUR) is calculated and entries with IUR greater than or equal to the non-maximum suppression threshold are deleted. The retention sets of each category are merged to obtain the detection entries.

[0195] Instance association is performed on the detection entries of adjacent keyframes, and a set of candidates to be matched is formed according to the category. The pixel distance between the center points of the detection entries in the two frames is calculated, and the candidate matching pairs are sorted by combining the angle threshold of the axial direction vector of the pipe segment and the classification confidence. When the pixel distance between the center points is not greater than the preset distance threshold and the angle between the axial direction vectors is not greater than the preset angle threshold, it is determined as a matching pair. A one-to-one greedy strategy is used to select matching pairs from high to low. The detection entries that are successfully matched are assigned the same instance identifier. The detection entries that are not matched are assigned a new instance identifier, and the instance identifiers that have not been matched for a long time are terminated according to the preset rules.

[0196] The detection entries that have been retained after redundancy suppression and have completed instance association are summarized and stored in keyframe time order, and the detection result set is output. Each detection entry includes a category label and classification confidence, bounding box parameters, cylindrical geometric prior parameters and geometric confidence, and an index relationship of keyframe identifier - instance identifier - detection entry is established.

[0197] Example 1:

[0198] To verify the practical feasibility of the "Intelligent Detection and Mapping Method for Underground Pipelines Based on Image Recognition" described in this invention, the method was applied to a comprehensive underground pipeline survey task in a certain city. The pipeline network in this area has a complex history, with issues such as missing data for some pipelines, significant differences in pipeline burial depth, and some pipelines shifting due to years of disrepair. Traditional manual measurement and electromagnetic detection methods have high errors in complex environments, especially at intersections of various types of pipelines where unclear identification is common, failing to provide complete and reliable image data support for construction.

[0199] In practical application, the project team first used high-resolution camera equipment combined with an inertial measurement unit to continuously acquire images of a 450-meter-long section of the road, obtaining over 42,000 frames of video images. After distortion correction and keyframe selection, 3,200 keyframe frames were ultimately generated, with an average distribution interval of approximately 0.14 meters, covering all area details. The SuperPoint network was used to extract feature points, and the LightGlue algorithm was used to match feature points between adjacent frames, establishing the pose relationship between keyframes and generating a preliminary pose map and pose parameter set.

[0200] After the keyframe data is prepared, the keyframes are input into the improved YOLO-World detection network. Unlike traditional detection methods, this network introduces a cylindrical geometry prior branch into the detection head, enabling pipe detection to output not only the usual class labels and bounding box parameters, but also directly provide the pipe segment's axial direction vector, the approximate radius of the image domain, and the major and minor axes of the projected ellipse. This improvement allows detection entries to simultaneously carry semantic and geometric information, thus providing high-quality input for subsequent 3D modeling. Specific experimental data are shown in Table 1.

[0201] Table 1. Statistical table of experimental data of the improved YOLO-World detection network in underground pipeline detection.

[0202]

[0203] As shown in Table 1, in the experiment, the improved YOLO-World detection network, after normalizing the size and pixels of the keyframes and fusing them with the convolutional backbone and feature pyramid, detected approximately 18,600 candidate bounding boxes. After processing with a classification confidence threshold (0.6) and non-maximum suppression (threshold 0.5), 12,900 high-quality detection entries were retained. Compared to the traditional YOLOv5 model on the same data (approximately 10,200 detection entries), the improved YOLO-World network improved the number of effective detections by approximately 26%.

[0204] Based on the detection results, geometric consistency screening was performed on the feature matching point pairs, eliminating mismatched points whose normal distance exceeded a threshold, ultimately retaining approximately 98,000 geometrically consistent matching points. Based on these matching point pairs, multi-view triangulation was used to generate a sparse point cloud, resulting in over 820,000 3D sparse points. In the point cloud optimization stage, reprojection residuals and diameter consistency residuals were introduced as weighting factors, and incremental bundle adjustment optimization was performed. The final average reprojection error was 1.7 pixels, and the diameter consistency residual was controlled within 16 millimeters. The overall point cloud accuracy was improved by approximately 18% compared to traditional methods.

[0205] In the final topology mapping stage, the sparse semantic point cloud was grouped according to instance identifiers. The RANSAC method was used to fit the axis of each group of point clouds and combined with segmented clustering to generate continuous three-dimensional centerline segments. A total of 305 centerline segments and 272 topology nodes were identified in the 450-meter-long area, including 42 valves, 19 tees, 31 elbows, and 12 flange components. The automatically generated pipeline topology map achieved a completeness of 94%, and the average node position error was ±0.28 meters, which meets the accuracy requirements of municipal design.

[0206] This application demonstrates that the method of this invention, especially the improved YOLO-World detection network, exhibits significant advantages in complex scenarios. While maintaining detection speed, this network improves the detection accuracy of pipelines and their ancillary components, and utilizes prior parameters of cylindrical geometry to make 3D modeling more stable. This solves the problem of insufficient information in the detection and mapping stages of traditional methods, providing a complete technical path for the digital and visual management of underground pipelines.

[0207] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for intelligent detection and mapping of underground pipelines based on image recognition, characterized in that, Includes the following steps: Step 1: Acquire continuous video images, select keyframes, use LightGlue to associate feature matching pairs of adjacent keyframes, establish a pose graph, and form a pose parameter set; Step 2: Input the keyframes into the improved YOLO-World detection network. The detection head of the improved YOLO-World detection network includes a cylindrical geometry prior branch and outputs a set of detection results. Step 3: Based on the detection result set, perform geometric consistency screening on the feature matching pairs to obtain a geometrically consistent matching set; Step 4: Perform multi-view triangulation on the geometrically consistent matching set to generate a 3D sparse point set, calculate the reprojection residual, and form the weighting factor; Step 5: Introduce weighting factors and perform incremental bundle adjustment optimization on the pose parameter set and the 3D sparse point set to obtain the sparse semantic point cloud; Step 6: Group the sparse semantic point cloud, generate a continuous three-dimensional center line segment set based on the RANSAC straight line fitting method and segmented clustering, generate graph edges based on the spatial relationship of adjacent line segments, and output the underground pipeline network topology map.

2. The method for intelligent detection and mapping of underground pipelines based on image recognition according to claim 1, characterized in that, The process of acquiring continuous video images, selecting keyframes, using LightGlue to associate feature matching pairs of adjacent keyframes, establishing a pose graph, and forming a pose parameter set is as follows: Acquire continuous video images and record timestamps, read camera intrinsic parameters, perform distortion correction on the continuous video images, and obtain a distortion-free image sequence; Keyframes are selected from the distortion-reduced image sequence based on time interval thresholds and disparity thresholds to form a keyframe sequence. The SuperPoint feature extraction network is used to extract local features and calculate descriptor vectors for each keyframe in the keyframe sequence to obtain the keyframe feature set. Each pair of adjacent keyframes is selected sequentially from the keyframe sequence, and the keyframe feature set corresponding to the first keyframe and the keyframe feature set corresponding to the second keyframe are respectively used as the adjacent keyframe feature set. LightGlue is used to perform feature matching on the feature sets of adjacent keyframes, and a set of feature matching pairs is output. The relative pose between adjacent keyframes is solved using the five-point method based on feature matching pairs; Using the first keyframe as a reference coordinate system, the initial pose parameters of each keyframe are obtained by accumulating the pose chain; the pose chain refers to the ordered sequence of relative poses between adjacent keyframes arranged in chronological order in the keyframe sequence. A pose graph is constructed using the initial pose parameters of each keyframe as nodes and the relative poses between adjacent keyframes as edges. The initial pose parameters of each keyframe are then aggregated to form a pose parameter set.

3. The method for intelligent detection and mapping of underground pipelines based on image recognition according to claim 1, characterized in that, The keyframes are input into the improved YOLO-World detection network. The detection head of the improved YOLO-World detection network includes a cylindrical geometry prior branch, and the output is a detection result containing the cylindrical geometry prior parameters. Specifically: Load the improved YOLO-World detection network and set the target category, classification confidence threshold, and non-maximum suppression threshold; Size and pixel normalization are performed on keyframes to adjust them to a preset input resolution. The processed keyframes are input into the convolutional backbone to extract multi-scale backbone features. Multi-scale fusion with upsampling, downsampling and lateral connection is performed based on the feature pyramid network and the path aggregation network to generate a shared feature map. The detection head branches run in parallel at the same spatial location sharing the feature map: The detection head branches include a category branch, a bounding box branch, and a cylindrical geometry prior branch; The category branch outputs a category probability vector, and the maximum component of the category probability vector is taken as the classification confidence. The bounding box branch outputs bounding box parameters, and the bounding box branch regresses the bounding box parameters and generates a set of candidate bounding boxes. The cylinder geometry prior branch outputs the cylinder geometry prior parameters; The cylindrical geometry prior branch first generates a 7-channel parameter map by sequentially passing each shared feature map through a 1×1 convolution, two layers of 3×3 depthwise separable convolutions, and a 1×1 convolution. Based on spatial location, establish a correspondence between the 7-channel parametric map and candidate bounding boxes generated at the same scale, and perform average pooling within a fixed window centered on the candidate bounding boxes to read out the original parameter vector: Normalize the first and second dimensions to obtain the axial direction vector of the pipe segment; The third dimension is constrained by nonnegativity and converted to pixel units based on the preset input resolution of the improved YOLO-World detection network and the downsampling ratio of the shared feature map to obtain the approximate radius of the image domain. The fourth and fifth dimensions are used as the major and minor axis parameters of the projected ellipse, respectively, and adjusted in order of magnitude. Use the sixth dimension as the orientation parameter of the projected ellipse; The seventh dimension is normalized and used as the geometric confidence level; When the same candidate bounding box obtains the original parameter vector on the shared feature map at the same time, the original parameter vector is weighted and fused according to the classification confidence to obtain the cylindrical geometric prior parameters of the candidate bounding box; The prior geometric parameters of the cylinder include the axial direction vector of the pipe segment, the approximate radius of the image domain, and the parameters of the projected ellipse. Candidate detection results are filtered based on classification confidence threshold and redundant suppression is performed based on non-maximum suppression threshold, retaining detection entries that meet the threshold conditions; the candidate detection results include class probability vector, classification confidence, bounding box parameters, cylindrical geometric prior parameters, and geometric confidence. Perform instance association on the detection entries; When the distance between the center point pixels is not greater than the preset distance threshold and the angle between the axial direction vectors is not greater than the preset angle threshold, it is determined as a matching pair. A one-to-one greedy strategy is used to select matching pairs from high to low until there are no matching pairs to choose from. The same instance identifier is assigned to the successfully matched detection entries, a new instance identifier is assigned to the unmatched detection entries, and the instance identifier is terminated when there are consecutive unmatched entries for a preset number of frames. The output detection result set is as follows: the detection entries that have been filtered and retained after redundancy suppression and assigned instance identifiers through instance association are summarized and stored in keyframe time order. Each detection entry includes a category label and classification confidence, bounding box parameters, cylindrical geometric prior parameters, and geometric confidence.

4. The method for intelligent detection and mapping of underground pipelines based on image recognition according to claim 1, characterized in that, The geometric consistency screening specifically includes: Spatial matching is performed between the coordinates of the first keyframe feature points and the coordinates of the second keyframe feature points of each feature matching pair and the bounding box parameters of the second keyframe, and the matching point pairs that fall inside the corresponding bounding box in the second keyframe are selected. Read the pipe segment axial direction vector and the approximate radius of the image domain from the prior cylindrical geometry parameters corresponding to each pair of retained matching points, and calculate the normal distance from the matching point pair to the axial direction line based on the coordinates of the feature points of the second keyframe and the center abscissa and center ordinate. Remove matching point pairs whose normal distance is greater than the first distance threshold, and retain matching point pairs that satisfy the position constraints; For matching point pairs that satisfy the constraints, calculate the angle between the matching vector and the axial direction vector of the pipe segment in the corresponding bounding box, filter out matching point pairs with an angle greater than the first angle threshold, and retain matching point pairs that satisfy the angle constraints. The set of matching point pairs that simultaneously satisfy both position and angle constraints is defined as the geometrically consistent matching set.

5. The method for intelligent detection and mapping of underground pipelines based on image recognition according to claim 1, characterized in that, The process of performing multi-view triangulation on the geometrically consistent matching set to generate a 3D sparse point set, calculating the reprojection residual, and forming a weighting factor is as follows: Map the coordinates of the first keyframe feature points and the second keyframe feature points of any geometrically consistent matching point pair to the camera's normalized image plane coordinates. Combine the pose parameters of the two frames to perform linear triangulation to obtain the corresponding three-dimensional spatial point coordinates. Repeat the above process for all geometrically consistent matching point pairs to generate a three-dimensional sparse point set. The category label and instance identifier recorded in the detection entries of the first keyframe and the second keyframe of the feature matching point pair are respectively assigned to the corresponding three-dimensional sparse points to form a semantic instance sparse point set. Read the depth value used in the second keyframe of linear triangulation for each 3D sparse point, as well as the image domain approximate radius in the corresponding cylindrical geometry prior parameters; The approximate radius in the image domain is combined with the participating depth value and scaled to obtain the estimated diameter value; The estimated diameter value is averaged with the major axis and minor axis parameters of the projected ellipse to obtain the projected diameter value. The difference between the estimated diameter value and the projected diameter value is calculated as the diameter consistency residual. Perform forward projection based on the pose parameters of the keyframe corresponding to each 3D sparse point, and calculate the Euclidean distance between the projected pixel position and the original feature point coordinates as the reprojection residual. The diameter consistency residual and the reprojection residual are combined in a weighted manner to form a weighting factor.

6. The method for intelligent detection and mapping of underground pipelines based on image recognition according to claim 1, characterized in that, The introduction of weighting factors and the incremental bundle adjustment optimization of the pose parameter set and the 3D sparse point set yield a sparse semantic point cloud, specifically: The spatial coordinates of 3D sparse points are used as structural variable nodes, the pose parameters of each key frame are used as pose variable nodes, and the relative poses between key frames are used to construct edge constraints, thus establishing variable nodes and edge constraints for the local graph structure. By applying weighting factors as confidence factors to the reprojection error and diameter consistency error respectively, a weighted minimization objective function is established for the observation error term in the local graph structure. An incremental optimization method is used to sequentially add keyframes and corresponding 3D sparse points to the local graph structure, iteratively update the pose variable nodes and structure variable nodes, and perform nonlinear least squares optimization. In each round of incremental optimization, a robust kernel function is introduced to jointly optimize the spatial coordinates of the pose variable nodes and the 3D sparse points of the key frame until the convergence condition is met. The optimized pose parameter set of each keyframe and the 3D sparse point set of the semantic instance sparse point set are output as sparse semantic point cloud.

7. The method for intelligent detection and mapping of underground pipelines based on image recognition according to claim 1, characterized in that, The process involves grouping sparse semantic point clouds, performing cylindrical axis fitting and piecewise clustering based on RANSAC, generating a continuous set of three-dimensional centerline segments, generating graph edges based on adjacent centerline segments, and outputting an underground pipeline network topology map. Specifically: The 3D sparse points in the sparse semantic point cloud are grouped according to the instance identifier to form a semantic instance sparse point set; For each set of sparse points of semantic instances, the coordinates of the points and the corresponding axial direction vector of the pipe segment are read. The outliers are iteratively eliminated using a RANSAC-based straight line fitting method to obtain candidate cylindrical axes. Based on the candidate cylindrical axis, the three-dimensional sparse points in the semantic instance sparse point set are segmented and clustered into approximately straight line segments to generate a continuous three-dimensional centerline segment set. Read the spatial relationship between adjacent line segments in a continuous three-dimensional centerline segment set, and generate the corresponding graph edge when there are intersections or adjacent endpoints; Read the 3D positions of valves, tees, elbows, and flanges marked with detection entries from the sparse semantic point cloud, and generate graph nodes at the corresponding positions; The graph structure consisting of graph nodes and graph edges is stored as an underground pipeline network topology map, and the nodes, edges and attribute fields are recorded in the underground pipeline network topology map.

Images