A tunnel point cloud segmentation processing method, system and platform based on dynamic sparse voxel graph convolution and double-flow decoder and a storage medium

Abstract

The application discloses a tunnel point cloud segmentation processing method, system, platform and storage medium based on dynamic sparse voxel graph convolution and a double-flow decoder. A first data corresponding to original point cloud data is generated and acquired based on a dynamic sparse voxel graph convolution backbone network. A corresponding bolt hole query mechanism is constructed by combining high-curvature area sampling and geometric prior. A double-flow decoder is used to separately segment and process a background branch area and a bolt hole target area, and corresponding second data is generated. A light graph neural network is used to separately perform noise filtering processing and topological optimization processing on the second data, and corresponding third data is generated. The third data is final segmentation mask data. The system, platform and storage medium corresponding to the method significantly improve the accuracy and robustness of bolt hole segmentation by fusing multi-scale feature extraction, geometric prior guidance and double-task decoding, while ensuring high computational efficiency.

Description

Technical Field

[0001] This invention belongs to the field of 3D scanning and processing technology, specifically relating to a method, system, platform, and storage medium for tunnel point cloud segmentation processing based on dynamic sparse voxel map convolution and dual-stream decoder. Background Technology

[0002] With the rapid development of 3D scanning technologies (such as LiDAR and structured light), point cloud data is increasingly being used in industrial inspection, autonomous driving, and robot navigation. In industrial quality control and automated assembly, the accurate detection and segmentation of key components (such as bolt holes) is a crucial step, directly impacting subsequent measurement, analysis, and operational processes. However, due to the small size and complex background of bolt holes, traditional segmentation methods struggle to accurately identify and segment them in point cloud data, especially when point cloud density is uneven and noise levels are high.

[0003] Furthermore, point cloud data is characterized by high sparsity, irregularity, and noise interference, posing significant challenges to bolt hole segmentation. For example, bolt holes are small, and their subtle geometric features are easily obscured by large-scale background point clouds; point clouds generated by scanning devices often have uneven density and contain noise, resulting in blurred and incomplete hole edges; and large-scale point cloud data places high demands on the computational and memory efficiency of processing algorithms. Traditional segmentation methods (such as radius-based clustering and Hough transform for circle detection) are effective in simple scenarios, but suffer from poor generalization ability and low accuracy in complex backgrounds. While deep learning-based point cloud processing methods (such as PointNet++ and KPConv) can automatically learn features, they still have shortcomings: traditional convolutional neural networks struggle to directly process sparse point clouds; graph convolutional networks have high computational overhead; and single-structure networks cannot simultaneously consider the global semantics of large-scale backgrounds and the local fine geometric features of small bolt hole targets.

[0004] Therefore, given the small size of the bolt holes, their subtle geometric features are easily obscured by large-scale background point clouds; and the point clouds generated by scanning equipment often have uneven density and contain noise, resulting in blurred and incomplete hole edges and low computational efficiency, there is an urgent need to design and develop a tunnel point cloud segmentation processing method, system, platform, and storage medium based on dynamic sparse voxel map convolution and dual-stream decoder. Summary of the Invention

[0005] To overcome the shortcomings and difficulties of the existing technology, the purpose of this invention is to provide a method, system, platform and storage medium for tunnel point cloud segmentation based on dynamic sparse voxel graph convolution and dual-stream decoder, which can efficiently, accurately and robustly process bolt hole segmentation tasks in complex point cloud data.

[0006] The first objective of this invention is to provide a tunnel point cloud segmentation processing method based on dynamic sparse voxel map convolution and dual-stream decoder; the second objective of this invention is to provide a tunnel point cloud segmentation processing system based on dynamic sparse voxel map convolution and dual-stream decoder; the third objective of this invention is to provide a tunnel point cloud segmentation processing platform based on dynamic sparse voxel map convolution and dual-stream decoder; and the fourth objective of this invention is to provide a computer-readable storage medium.

[0007] The first objective of this invention is achieved as follows: the method comprises:

[0008] Based on a dynamic sparse voxel graph convolutional backbone network, first data corresponding to the original point cloud data is generated and obtained; wherein, the first data is point cloud multi-scale feature data;

[0009] By combining high curvature region sampling and geometric priors, a corresponding bolt hole query mechanism is constructed.

[0010] The background branch region and the bolt hole target region are segmented and processed separately based on a dual-stream decoder, and corresponding second data is generated; wherein, the second data is the preliminary segmentation result data;

[0011] By combining a lightweight graph neural network, the second data is processed by noise filtering and topology optimization, and corresponding third data is generated; wherein, the third data is the final segmentation mask data.

[0012] The second objective of this invention is achieved as follows: the system is used to implement the tunnel point cloud segmentation processing method based on dynamic sparse voxel map convolution and dual-stream decoder, the system comprising:

[0013] A data generation and acquisition unit is used to generate and acquire first data corresponding to the original point cloud data based on a dynamic sparse voxel graph convolutional backbone network; wherein the first data is multi-scale feature data of the point cloud; a mechanism construction and generation unit is used to construct and generate a corresponding bolt hole query mechanism by combining high curvature region sampling and geometric prior; a first data processing and generation unit is used to segment and process the background branch region and the bolt hole target region respectively based on a dual-stream decoder, and generate corresponding second data; wherein the second data is preliminary segmentation result data; a second data processing and generation unit is used to combine a lightweight graph neural network to perform noise filtering and topology optimization processing on the second data respectively, and generate corresponding third data; wherein the third data is the final segmentation mask data.

[0014] The third objective of this invention is achieved as follows: it includes a processor, a memory, and a control program for a tunnel point cloud segmentation processing platform based on dynamic sparse voxel graph convolution and a dual-stream decoder; wherein the control program is executed on the processor, the control program is stored in the memory, and the control program implements the tunnel point cloud segmentation processing method based on dynamic sparse voxel graph convolution and a dual-stream decoder.

[0015] The fourth objective of this invention is achieved as follows: the computer-readable storage medium stores a control program for a tunnel point cloud segmentation processing platform based on dynamic sparse voxel graph convolution and dual-stream decoder. The control program for the tunnel point cloud segmentation processing platform based on dynamic sparse voxel graph convolution and dual-stream decoder implements the tunnel point cloud segmentation processing method based on dynamic sparse voxel graph convolution and dual-stream decoder.

[0016] This invention utilizes a method based on a dynamic sparse voxel graph convolutional backbone network to generate and acquire first data corresponding to the original point cloud data; wherein, the first data is multi-scale feature data of the point cloud; combining high curvature region sampling and geometric priors, a corresponding bolt hole query mechanism is constructed; based on a dual-stream decoder, the background branch region and the bolt hole target region are segmented and processed respectively, and corresponding second data is generated; wherein, the second data is preliminary segmentation result data; combined with a lightweight graph neural network, the second data is processed by noise filtering and topology optimization respectively, and corresponding third data is generated; wherein, the third data is the final segmentation mask data, along with the corresponding system, platform, and storage medium. By integrating multi-scale feature extraction, geometric prior guidance, and dual-task decoding, the accuracy and robustness of bolt hole segmentation are significantly improved, while ensuring high computational efficiency.

[0017] In other words, the proposed solution efficiently extracts multi-scale features of point clouds through a dynamic sparse voxel graph convolutional backbone network, accurately focuses on bolt hole regions by combining a query generation module based on geometric priors, processes global background and local targets separately using a dual-stream decoder, and intelligently filters noise through lightweight graph neural network post-processing. Ultimately, it achieves high-precision, high-robustness, and high-efficiency segmentation of bolt holes in complex industrial scenarios, significantly improving the reliability of industrial inspection and automated assembly. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the process steps of a tunnel point cloud segmentation method based on dynamic sparse voxel map convolution and dual-stream decoder according to the present invention.

[0020] Figure 2 This is a schematic diagram of the processing flow framework of an embodiment of the tunnel point cloud segmentation processing method based on dynamic sparse voxel map convolution and dual-stream decoder of the present invention.

[0021] Figure 3 This is a schematic diagram of the second embodiment of the tunnel point cloud segmentation processing method based on dynamic sparse voxel map convolution and dual-stream decoder of the present invention.

[0022] Figure 4 This is a schematic diagram of a tunnel point cloud segmentation system architecture based on dynamic sparse voxel graph convolution and dual-stream decoder according to the present invention.

[0023] Figure 5 This is a schematic diagram of a tunnel point cloud segmentation processing platform architecture based on dynamic sparse voxel graph convolution and dual-stream decoder according to the present invention.

[0024] Figure 6 This is a schematic diagram of a computer-readable storage medium architecture in one embodiment of the present invention. Detailed Implementation

[0025] To facilitate a clearer understanding of the objectives, technical solutions, and advantages of this invention, the invention will be further described below in conjunction with the accompanying drawings and specific embodiments. Those skilled in the art can easily understand other advantages and effects of this invention from the content disclosed in this specification.

[0026] This invention can also be implemented or applied through other different specific examples, and various details in this specification can also be modified and changed based on different viewpoints and applications without departing from the spirit of this invention.

[0027] It should be noted that if the embodiments of the present invention involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of the components in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0028] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Secondly, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0029] Preferably, the tunnel point cloud segmentation method based on dynamic sparse voxel map convolution and dual-stream decoder of the present invention is applied in one or more terminals or servers. The terminal is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0030] The terminal can be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal can interact with the customer via a keyboard, mouse, remote control, touchpad, or voice control device.

[0031] This invention provides a method, system, platform, and storage medium for tunnel point cloud segmentation based on dynamic sparse voxel graph convolution and dual-stream decoder.

[0032] like Figure 1 The diagram shown is a flowchart of a tunnel point cloud segmentation method based on dynamic sparse voxel graph convolution and dual-stream decoder provided in an embodiment of the present invention.

[0033] In this embodiment, the tunnel point cloud segmentation processing method based on dynamic sparse voxel map convolution and dual-stream decoder can be applied to terminals with display functions or fixed terminals. The terminals are not limited to personal computers, smartphones, tablets, desktop computers or all-in-one computers with cameras, etc.

[0034] The tunnel point cloud segmentation method based on dynamic sparse voxel graph convolution and dual-stream decoder can also be applied to a hardware environment consisting of a terminal and a server connected to the terminal via a network. The network includes, but is not limited to, wide area networks (WANs), metropolitan area networks (MANs), or local area networks (LANs). The tunnel point cloud segmentation method based on dynamic sparse voxel graph convolution and dual-stream decoder in this embodiment of the invention can be executed by a server, by a terminal, or by both a server and a terminal.

[0035] For example, for terminals requiring tunnel point cloud segmentation processing based on dynamic sparse voxel graph convolution and dual-stream decoder, the tunnel point cloud segmentation processing function based on dynamic sparse voxel graph convolution and dual-stream decoder provided by the method of this invention can be directly integrated onto the terminal, or a client for implementing the method of this invention can be installed. Furthermore, the method provided by this invention can also run on servers or other devices in the form of a Software Development Kit (SDK), providing an interface for the tunnel point cloud segmentation processing function based on dynamic sparse voxel graph convolution and dual-stream decoder in the form of an SDK. Terminals or other devices can then implement the tunnel point cloud segmentation processing function based on dynamic sparse voxel graph convolution and dual-stream decoder through the provided interface. The invention will be further described below with reference to the accompanying drawings.

[0036] like Figures 1-3 As shown, this invention provides a tunnel point cloud segmentation method based on dynamic sparse voxel map convolution and a dual-stream decoder. The method includes the following steps:

[0037] S01. Based on a dynamic sparse voxel graph convolutional backbone network, generate and acquire first data corresponding to the original point cloud data; wherein, the first data is point cloud multi-scale feature data;

[0038] S02. Combining high curvature region sampling and geometric priors, a corresponding bolt hole query mechanism is constructed.

[0039] S03. Based on the dual-stream decoder, the background branch region and the bolt hole target region are segmented and processed respectively, and corresponding second data is generated; wherein, the second data is the preliminary segmentation result data;

[0040] S04. Combining a lightweight graph neural network, the second data is processed by noise filtering and topology optimization respectively, and corresponding third data is generated; wherein, the third data is the final segmentation mask data.

[0041] The process of generating and acquiring first data corresponding to the original point cloud data based on a dynamic sparse voxel graph convolutional backbone network further includes:

[0042] S011. Combining dynamic voxel map construction, hybrid convolutional units, and gating fusion mechanisms, sparse convolution and graph convolution are fused to calculate and generate fourth and fifth data corresponding to the original point cloud data, respectively; wherein, the fourth data is non-empty voxel data; and the fifth data is geometric perception data of high curvature regions.

[0043] The mechanism for generating corresponding bolt hole queries by combining high-curvature region sampling and geometric priors also includes:

[0044] S021. Generate and acquire sixth data corresponding to the first data, and identify and generate seventh data corresponding to the sixth data; wherein, the sixth data is the curvature data of each point in the point cloud; the seventh data is high curvature region data;

[0045] S022. Based on the sixth data, construct a corresponding bolt hole query mechanism, and integrate and process the geometric feature data and semantic feature data corresponding to the bolt hole query mechanism;

[0046] S023. Generate and acquire the eighth data corresponding to the bolt hole, and embed the eighth data through a multilayer perceptron and geometric transformation; wherein the eighth data is the physical prior information data of the bolt hole.

[0047] The mechanism for generating corresponding bolt hole queries by combining high-curvature region sampling and geometric priors also includes:

[0048] S0210. Calculate and generate the sixth data corresponding to the first data; wherein the calculation formula is:

[0049] (4)

[0050] In the formula, Point The curvature characterizes local geometric properties; For point The approximate area of ​​the neighborhood is estimated by the average distance between neighborhood points; For point The set of k nearest neighbors; For point The normal vector; and Points and neighboring points 3D coordinates; Given the vector norm, calculate the projected distance between points; normal vector. The formula, calculated using principal component analysis, is as follows:

[0051] (5)

[0052] In the formula, Indicates the local surface orientation; PCA is principal component analysis, which calculates the principal orientation based on the coordinate difference of neighboring points. For neighborhood points relative to point The coordinate difference vector.

[0053] The process of segmenting and processing the background scene region and the bolt hole target region based on the dual-stream decoder, and generating corresponding second data, also includes:

[0054] S031. Combining self-attention and cross-attention mechanisms, a first model corresponding to the background branch is created; wherein, the first model is a global semantic model;

[0055] S032. Construct a geometric prior-enhanced self-attention mechanism, and based on the geometric prior-enhanced self-attention mechanism, generate the ninth data corresponding to the bolt hole branch; wherein, the ninth data is geometric feature data;

[0056] S033. Sequentially fuse and process the data, and output the second data corresponding to the background branch and the bolt hole branch.

[0057] The method of combining a lightweight graph neural network to perform noise filtering and topology optimization on the second data, and generating corresponding third data, further includes:

[0058] S041. Cluster the second data and generate tenth data corresponding to the second data; wherein, the tenth data is semantic point cluster data;

[0059] S042. Construct a graph structure corresponding to the tenth data, and generate eleventh data corresponding to the graph structure; wherein, the eleventh data is node feature data, including average coordinate data, average normal vector data, point cluster density data, mean curvature data, and semantic confidence data;

[0060] S043. Based on graph convolutional networks, information propagation processing and noise classification processing are performed respectively, and corresponding third data is generated.

[0061] Specifically, in this embodiment of the invention, to further improve the accuracy of bolt hole segmentation, this solution proposes a point cloud feature extraction method based on dynamic sparse voxel map convolution, and introduces a query generation module and a dual-stream decoder structure. The query generation module accurately identifies and focuses on the edges and detail regions of bolt holes by sampling high-curvature regions and introducing geometric priors. Specifically, this module first calculates the curvature of each point in the point cloud and identifies high-curvature regions, which typically correspond to the contours or edges of bolt holes. Then, a query is generated based on the high-curvature points. Combining the point's feature information, geometric transformation, and geometric priors, the network more accurately focuses on the bolt hole region, thereby significantly improving target detection performance in complex backgrounds.

[0062] The dual-stream decoder architecture further optimizes the bolt hole segmentation accuracy. It consists of a background branch and a bolt hole branch, responsible for the semantic segmentation of the large-scale background region and the bolt hole target, respectively. The background branch captures global semantic information of the background region through a semantic query mechanism and a self-attention mechanism, achieving accurate background segmentation. The bolt hole branch employs a geometrically enhanced self-attention mechanism. By explicitly encoding geometric priors such as curvature, depth, and neighborhood distance as positional bias terms and injecting them into attention weight calculations, it enhances the focusing ability on the bolt hole edges, generating a more accurate segmentation mask. The outputs of the two branches are integrated through a fusion mechanism, further improving the overall segmentation performance.

[0063] Although dual-stream decoders significantly improve segmentation accuracy, initial segmentation results may still contain noise and misclassifications, such as isolated point clusters, missegmented regions, or blurred boundaries, especially in complex backgrounds or with uneven point cloud density. To address this, this solution designs a post-processing module based on a lightweight graph neural network. By constructing a graph structure between point clusters, it utilizes a graph convolutional network to learn the topological relationships of these clusters, achieving intelligent noise filtering. Specifically, the post-processing module clusters the segmentation mask to generate semantic point clusters, with each cluster acting as a node in the graph, and edges between nodes representing topological relationships. Information is propagated through a lightweight graph convolutional network, nodes aggregate structural information from their neighbors, learn discriminative features, and finally, a fully connected layer determines whether a point cluster is noise.

[0064] This solution aims to provide an efficient, accurate, and robust approach for bolt hole detection and segmentation in complex point cloud data. By combining the feature extraction capabilities of dynamic sparse voxel graph convolution, the high curvature region focusing ability of the query generation module, the multi-task modeling capabilities of the dual-stream decoder, and the noise filtering capabilities of the lightweight graph neural network post-processing module, the proposed method demonstrates excellent performance in bolt hole detection and segmentation. Experimental results show that this method effectively handles complex backgrounds and noise interference, significantly improving the segmentation accuracy and robustness of bolt holes, and providing important technical support for fields such as industrial inspection and quality control.

[0065] Specifically, for the Dynamic Sparse Voxel Graph Backbone (DSVGB), in point cloud data processing tasks, the core task of the backbone network is to extract high-level features from the original point cloud data, providing crucial support for subsequent tasks such as semantic segmentation, 3D reconstruction, and object detection. In the dual-branch process proposed in this scheme, the backbone network generates feature representations while simultaneously supporting large-scale global semantic modeling of the background branch and accurate segmentation of local geometric features in the bolt hole branch. Traditional convolutional neural networks (CNNs) perform well in image processing, but face significant challenges when processing point cloud data, especially in large-scale point cloud scenarios, where computational efficiency and memory consumption become major bottlenecks. The sparsity and irregular distribution of point cloud data result in most spatial regions being empty, and traditional dense convolutions generate a large amount of invalid computation in these empty regions, causing resource waste. To address this, this scheme proposes an innovative Dynamic Sparse Voxel Graph Backbone (DSVGB), which breaks the traditional two-stage fragmented processing of "voxel-graph" by fusing sparse convolution and graph convolution into a unified dynamic operator. DSVGB significantly improves feature extraction efficiency and geometric perception capabilities through dynamic voxel graph construction, hybrid convolutional units, and gated fusion mechanisms. It meets the background branch's need for global semantic information and the bolt hole branch's requirement for precise local geometric details, laying a solid foundation for subsequent processing. Designed specifically for processing sparse point cloud data, DSVGB's encoder-bottleneck-decoder structure effectively captures and recovers spatial information through multi-scale feature extraction and dynamic graph connections. Traditional sparse convolutions only extract local features based on a fixed voxel grid, ignoring the geometric characteristics of point clouds; traditional graph convolutions (such as DGCNN), while capturing topological relationships between points, lack sparsity optimization, resulting in high computational overhead. DSVGB, by dynamically coupling sparse convolution and graph convolution, integrates them into a single learnable module for the first time, significantly improving computational efficiency and geometric perception capabilities. Its core design includes: dynamic voxel graph construction, dynamically generating K-nearest neighbor graphs at each level of sparse convolution based on the spatial density and geometric features (curvature, normal angle) of voxel center points, rather than relying on fixed grid neighborhoods; enhancing graph connectivity in high-curvature regions (such as bolt hole edges) to capture complex geometric features, and degenerating into efficient sparse convolution in flat background regions. Hybrid convolutional units contain sparse convolutions for efficient extraction of local features within voxels, preserving the spatial structure information of the point cloud; graph convolutions aggregate cross-voxel contextual information through edge weights (based on geometric similarity, such as curvature difference or normal angle), enhancing feature representation in high-curvature regions such as bolt hole edges. The gated fusion mechanism adaptively balances the contributions of voxel features and graph features through a lightweight multilayer perceptron (MLP), with the formula:

[0066] (1)

[0067] in, Voxel features of sparse convolution output The graph features output by graph convolution. Indicates feature splicing, and σ represents the learnable gating parameters, and σ is the sigmoid activation function.

[0068] This gating mechanism allows the network to adaptively adjust feature contributions based on the geometric characteristics of the point cloud region. The overall structure of DSVGB consists of an encoder, a bottleneck layer, and a decoder, forming a symmetrical U-shaped network architecture. The encoder comprises multiple hybrid convolutional units, progressively reducing spatial resolution and extracting high-level features. Each layer generates features through dynamic voxel map construction and hybrid convolutional units. The bottleneck layer further compresses features and enhances global information representation, containing multiple hybrid convolutional blocks. The decoder progressively restores spatial resolution through sparse deconvolution and graph convolution, generating feature representations for each point. Let the input point cloud data be:

[0069] (2).

[0070] In the formula, The three-dimensional coordinates of the point; The color characteristics of the points;

[0071] The feature extraction process of DSVGB can be represented as:

[0072] (3)

[0073] Among them, the encoder generates high-level features. The bottleneck layer is compressed into The decoder recovers the feature F, which has dimensions (N, C), where N is the number of non-empty voxels and C is the number of feature channels.

[0074] Compared to traditional dynamic sparse voxel map convolution, DSVGB enhances feature representation in high-curvature regions such as bolt hole edges through dynamic voxel map and gated fusion, while degenerating into efficient sparse convolution in flat background regions, balancing computational efficiency and geometric perception. DSVGB enhances the capture of complex geometric features through edge weights and gated fusion, while maintaining sparsity optimization and reducing computational load. This design is particularly suitable for two-branch workflows: the background branch uses features generated by DSVGB to capture global semantic patterns, while the bolt hole branch combines feature F and preserved coordinates to focus on local geometric features through high-curvature sampling, supporting accurate bolt hole segmentation.

[0075] This invention also includes a bolt hole query generation module. Achieving accurate segmentation of bolt holes in complex backgrounds is a key challenge in point cloud data processing. This solution proposes a bolt hole query generation module based on geometric priors, which significantly improves segmentation accuracy by locating high-curvature regions in the point cloud. In the dual-branch process, the bolt hole branch utilizes features F and coordinates generated by dynamic sparse voxel map convolution. Through high-curvature sampling and semantic query generation, it focuses on the local geometric features of the bolt holes, complementing the global semantic modeling of the background branch.

[0076] First, high-curvature regions are sampled. To achieve accurate segmentation of bolt hole targets in complex backgrounds, this scheme introduces a query generation module based on geometric priors. In point cloud data, curvature is an important indicator of the local geometric features of a point, especially prominent in the edges and details of the target. The contours and edges of bolt hole targets typically have high curvature, and these high-curvature regions are crucial for subsequent target detection and segmentation. Identification of high-curvature regions is the core step in bolt hole query generation, performed in the preprocessing stage of the bolt hole branch. Curvature in point cloud data is an important indicator of the local geometric characteristics of a point. The edges and contours of bolt holes typically have high curvature, making them suitable as prior information for segmentation targets. Each batch input is a dynamic sparse voxel map convolution, generating features F and coordinates. The bolt hole branch calculates the curvature of each point based on the coordinates: In this module, we first calculate the curvature of each point in the point cloud to identify high-curvature regions. The curvature calculation is based on the neighborhood information of the point, and the specific formula is as follows:

[0077] (4)

[0078] in, The curvature of point p represents the local geometric properties; The area of ​​the neighborhood of point p is an approximation, estimated by the average distance between neighborhood points; Let p be the set of k nearest neighbors. Let be the normal vector of point p; and Let p and q be the three-dimensional coordinates of point p and its neighboring point q, respectively. Given the vector norm, calculate the projected distance between points. Normal vector. The formula, calculated using principal component analysis (PCA), is as follows:

[0079] (5)

[0080] Indicates the local surface orientation; PCA is principal component analysis, which calculates the principal orientation based on the coordinate difference of neighboring points. Let be the coordinate difference vector between the neighboring points and point p.

[0081] Select M high-curvature voxels from N voxels and record their indices. And extract feature subsets from F High-curvature points are concentrated on the outline of the bolt hole, containing a small number of neighboring background points to provide context and enhance segmentation robustness. Focusing attention on these high-curvature regions helps the model to more accurately focus on the bolt hole target, thereby improving segmentation accuracy.

[0082] The next step is to construct a bolt hole query generation mechanism. After identifying high-curvature regions, the next step is to generate queries. The core idea of ​​the query generation module is based on the high-curvature point features of the bolt hole query generation module. Generate semantic queries from geometric information This guides the network to focus on the target region of the bolt holes. The query generation process combines backbone network features, local geometric transformations, and physical priors about the bolt holes to ensure... Simultaneously capturing semantic information and geometric features. The generation process first uses a two-layer multilayer perceptron (MLP) to process the features of high curvature points. A transformation is performed to enhance the expressive power of semantic information. The initial transformation uses the following formula:

[0083] (6)

[0084] in, The intermediate features carry the semantic information after the initial transformation; ReLU is the activation function used to introduce nonlinearity to enhance the expressive power of the features. This is the first-layer weight matrix, which maps the input features to a high-dimensional space; The high curvature point feature subset is extracted by dynamic sparse voxel map convolution based on the high curvature point index; This is the first layer bias vector; adjust the feature offset.

[0085] This transformation enhances the semantic representation of features through linear mapping and nonlinear activation. To prevent overfitting and improve robustness to uneven point cloud density, Dropout regularization is introduced after the first transformation layer to randomly discard some neuron activation values. At the same time, Batch Normalization is applied to standardize the feature distribution and alleviate the scale differences of point cloud data in different scenarios.

[0086] Subsequently, intermediate features The second-layer MLP is input to further refine semantic features, as shown in the formula:

[0087] (7)

[0088] in, This represents the refined semantic features, providing a foundation for subsequent geometric information fusion; This is the second-layer weight matrix, which maps the intermediate features back to the target feature space; For the second layer bias vector, adjust the feature offset. It is an intermediate feature.

[0089] The second-layer transformation aims to compress and optimize semantic information while preserving key features to meet the needs of semantic query generation. Similar to the first layer, the second-layer transformation also applies Batch Normalization and Dropout to ensure the stability of feature distribution and the model's adaptability to noisy point clouds.

[0090] To incorporate the local geometry of the bolt holes, the generation process calculates geometric features G based on the coordinates of high-curvature points, containing three types of information: curvature, depth, and the mean neighborhood distance. These geometric features are then used to generate an embedding through linear transformation:

[0091] (8)

[0092] Among them, GeoTransform is a geometric feature embedding that maps to a space compatible with semantic features; G is the geometric transformation weight matrix; G is the geometric feature matrix, containing curvature, density, and mean distance. The bias vector is used to adjust the embedding offset.

[0093] Geometric features capture the sharp characteristics of bolt hole edges through curvature, and characterize the local shape and scale of the hole through density and distance mean. To improve stability, Layer Normalization is applied after transformation to normalize the geometric embedding and reduce the impact of point cloud scale differences. Furthermore, G is calculated by combining multi-scale neighborhoods (such as point sets with different k values) to capture richer geometric information and enhance stability. Expressive ability.

[0094] Ultimately, the semantic query is generated by fusing semantic features and geometric embeddings, as shown in the formula:

[0095] (9)

[0096] in, For semantic queries, semantic and geometric information are combined to guide subsequent bolt hole segmentation; This is the output of the second-layer MLP; GeoTransform is the geometric feature embedding. The hyperparameter controls the contribution weight of geometric information. The fusion process uses residual connections to preserve original semantic features while enhancing the expressive power of geometric characteristics. By optimizing the validation set, the relative importance of semantic and geometric information is balanced to ensure that the model can adapt to different point cloud scenarios.

[0097] The physical priors of the bolt holes are incorporated into the semantic query generation process by constraining the curvature, depth, and mean neighborhood distance of the geometric features (G), ensuring... This method accurately characterizes the geometric properties of small bolt holes, adapting to segmentation tasks in complex backgrounds. Curvature depicts the high curvature of the bolt hole edges, reflecting the geometric constraints of the hole diameter; depth information indirectly reflects the concavity features of the bolt hole through the spatial distribution of neighboring points; and the mean neighborhood distance characterizes the hole diameter scale and edge morphology. Specific implementations include curvature filtering, retaining only points that conform to the curvature distribution of the bolt hole edges; depth constraints, prioritizing points reflecting concavity features; and a weighted adjustment of the contribution ratios of curvature, depth, and the mean distance. Learnable prior parameters are introduced into GeoTransform to encode typical geometric patterns of bolt holes, and training optimization is used to improve query accuracy. To enhance model robustness, multiple regularization strategies are incorporated into the generation process. Integrating semantic information provides precise guidance for subsequent bolt hole mask generation. Compared to directly using original features, incorporating geometric priors... It can more effectively capture the edge and contour characteristics of bolt holes, and achieve precise segmentation of small bolt hole targets against a large background.

[0098] Ideally, in point cloud segmentation tasks, scenarios where complex backgrounds coexist with small bolt hole targets present challenges to network design in terms of multi-scale feature extraction and accurate segmentation. Traditional single-stream decoders struggle to simultaneously meet the requirements of global semantic modeling of large-scale background regions and local geometric feature extraction of small bolt hole targets. To address this, this solution proposes a dual-stream decoder structure. By constructing background and bolt hole branches in parallel, it achieves semantic segmentation of the background region and accurate segmentation of the bolt hole targets, respectively. The dual-stream decoder shares the global point cloud features F extracted by a dynamic sparse voxel map convolutional backbone network and achieves multi-scale feature modeling through a differentiated query mechanism. The background branch generates queries, keys, and values ​​(QKV) based on the global features F, emphasizing global semantic modeling; the bolt hole branch is based on high-curvature point features. and the semantic queries generated in Chapter 3 The focus is on extracting local geometric features. The outputs of the two branches are fused through a mechanism to generate a unified segmentation result, balancing global consistency and local accuracy. This chapter elaborates on the overall architecture of the dual-stream decoder, the semantic query generation and attention mechanism of the background branch, the local feature extraction and segmentation prediction of the bolt hole branch, and the collaborative fusion mechanism of the two branches.

[0099] In the overall structure of the dual-stream decoder, there are two parallel branches that share the global point cloud features F extracted by the dynamic sparse voxel map convolutional backbone network. Multi-scale feature modeling is achieved through a differentiated query mechanism. The background branch focuses on the semantic segmentation of large-scale background regions, generating query-key-value pairs (QKV) based on the global features F. It models the overall context of the point cloud through self-attention and cross-attention mechanisms to generate segmentation predictions for the background regions. The bolt hole branch targets the precise segmentation of small bolt holes, based on high-curvature point features. and the semantic queries generated in Chapter 3 It extracts local geometric features through a lightweight cross-attention mechanism to achieve accurate localization and segmentation. The two branches process the global feature F and the local feature respectively. To adapt to the characteristics of the background and bolt hole targets, branch outputs are fused through index alignment to form a unified segmentation result. The dual-stream decoder utilizes sparse convolution to optimize computational efficiency, processing only non-empty voxels to reduce the computational load in complex scenes. To enhance robustness, the decoder captures diverse semantic and geometric information through a multi-layer attention mechanism.

[0100] The semantic query mechanism in the background branch achieves global semantic modeling by generating query-key-value pairs (QKV) based on global features F. It enhances the semantic representation and segmentation prediction of the background region through self-attention and cross-attention mechanisms. The background branch does not use the bolt hole branch. Instead, it generates the query matrix directly from F to ensure the integrity of global semantic modeling.

[0101] For semantic query generation, semantic queries in the background branch Generated from global features F, used to characterize the global semantic properties of the background region. The generation process generates queries, keys, and values ​​through learnable linear transformations, as shown in the following formula:

[0102] (10)

[0103] in, The query matrix for the background branches represents global semantic information; This is the key matrix, used to calculate the similarity between points; It is a value matrix that stores feature information; This is the weight matrix; It is the bias vector; The global point cloud features extracted by dynamic sparse voxel map convolution are fused with spatial and semantic information; N is the number of non-empty voxels, C is the number of feature channels output by the backbone network, which depends on the network architecture, and d is the attention embedding dimension, a hyperparameter. This transformation projects F to the QKV space through a linear mapping, generating a representation suitable for global semantic modeling.

[0104] The self-attention mechanism for semantic queries in the solution aims to capture long-range dependencies between background region points. The background branch... The above applies a self-attention mechanism to dynamically adjust the point representation, enhancing global semantic consistency. The formula is as follows:

[0105] (11)

[0106] in, This is a self-attention output; Calculate the similarity matrix; V represents the scaling factor and provides weighted feature information. Layer Normalization standardizes the output distribution to mitigate scale differences. To improve performance, a multi-head self-attention mechanism is employed.

[0107] (12)

[0108] in,

[0109] (13)

[0110] h represents the number of attention heads; This is used to output the weight matrix. The multi-head mechanism enhances the adaptability of the background branch to complex scenes.

[0111] Regarding the cross-attention mechanism for semantic queries in this invention, after self-attention enhancement, the semantic query... By interacting with global features F through a cross-attention mechanism, background-related features are extracted to generate segmentation predictions. The cross-attention formula is as follows:

[0112] (14)

[0113] in, Weighted feature representation of the background region; This is a key-value matrix. Query Semantic representation derived from self-attention enhancement, including global semantic context; key matrix Sum matrix Generates original feature information of the entire point cloud based on global feature F. Calculate the similarity matrix to capture the correlation between semantic queries and global features; scaling factor. To prevent gradient vanishing; softmax normalization is used to generate attention weights, which are then dynamically weighted. The values ​​in the data are used to extract weighted features related to the background. This design allows the background branch to filter noise from global features, focusing on semantically consistent background regions and improving the accuracy of large-scale semantic modeling. Weighted features predict segmentation using a multilayer perceptron (MLP):

[0114] (15)

[0115] in, This is the background segmentation prediction result. It represents the background class probability of each point. The MLP contains multiple linear transformations and a ReLU activation function, mapping weighted features to the semantic class space and activating the output probability distribution through softma.

[0116] The structure and function of bolt hole branches: Bolt hole branches focus on the precise segmentation of small targets such as bolt holes, based on high curvature point features. and the semantic queries generated in Chapter 3 A geometry-prior self-attention (GPSA) mechanism was designed to extract local geometric features. GPSA explicitly encodes the geometric priors of bolt holes (curvature, depth, and neighborhood distance) as position bias terms and injects them into the standard self-attention weight calculation, significantly improving segmentation accuracy in complex backgrounds. The innovation of GPSA lies in combining geometric information with semantic similarity, breaking the limitation of traditional self-attention (such as ViT) that relies solely on semantic similarity. Its core design includes a geometric bias matrix. Adjust attention weights and dynamic lambda learning. Geometric bias matrix. By mapping the curvature, depth, and inter-point distance of point i to bias terms using a multilayer perceptron (MLP), the attention mechanism's ability to focus on high-curvature regions such as bolt hole edges is enhanced. Attention weights are corrected by fusing geometric priors and semantic similarity to improve allocation accuracy. Dynamic λ-learning adaptively adjusts the strength of geometric priors through learnable parameters, avoiding over-constraint. Compared to the original cross-attention mechanism, GPSA suppresses erroneous associations under complex background interference (such as high-reflectivity noise) through geometric priors. Compared to traditional self-attention, GPSA is the first to incorporate geometric priors as learnable biases into weight calculation, achieving collaborative modeling of semantics and geometry. (Direct use...) As a cross-attention query Q, key K and value V are from generate:

[0117] (16)

[0118] in, Features of high curvature points Semantic queries generated by the high curvature point feature and bolt hole query generation module. This is the weight matrix. It is the bias vector;

[0119] , , M represents the number of high curvature points. To ensure Consistent with the dimensions of K and V, The generation process is designed to output GPSA introduces a geometric offset matrix. The self-attention weights are adjusted, and the calculation formula is as follows:

[0120] (17)

[0121] (18)

[0122] in, This is the scaling factor; As a weighting factor; The weighted feature representation of the bolt hole region. Query Generated from semantic queries; key matrix K and value matrix V from Generation. Weighted features are used for segmentation prediction via a multilayer perceptron (MLP). Weighted features are then used to predict a segmentation mask via the MLP.

[0123] (19)

[0124] in, The semantic segmentation prediction results for bolt hole targets represent the bolt hole category probability for each high-curvature point. The MLP includes linear transformation and ReLU activation functions to map weighted features to the semantic category space, and activates the output probability distribution through softmax. Random point cloud perturbation enhances the model's robustness. To improve performance, multi-scale cross-attention can be employed, combining different neighborhood scales. This enhances the ability to represent complex bolt hole morphologies. Bolt hole branches are achieved through... and By combining the GPSA mechanism, it is possible to accurately segment small targets in complex backgrounds.

[0125] A collaborative mechanism for the two-stream branches is constructed. The two-stream decoder generates a unified segmentation mask by fusing the outputs of the background branch and the bolt hole branch. For each non-empty voxel, a probability for the bolt hole class is provided, and the probability for the background class is... The fusion formula is:

[0126] (20)

[0127] Integration is preferred Prediction of bolt holes at high curvature points, due to its utilization and More accurately captures local geometric features; for non-high curvature points, depends on Global semantic prediction. A weighted fusion method with fixed weights is used:

[0128] (twenty one)

[0129] in, To maintain fixed weights, highlight points with high curvature. Dependence on other points Index mask , This indicates that the i-th voxel is a high curvature point. (After fusion) This represents the probability of a bolt hole. During inference, the hard segmentation mask is generated using a threshold:

[0130] (twenty two)

[0131] The hard mask assigns a binary label (1 for bolt holes, 0 for background) to each voxel for visualization or downstream tasks. The fusion mechanism integrates the global semantics of the background branch and the local precision of the bolt hole branch to achieve accurate segmentation of complex point cloud scenes.

[0132] To optimize the network training process and improve segmentation accuracy, we designed various loss functions to guide the network in learning effective object segmentation. In this process, in addition to the standard semantic segmentation loss, we also introduced an edge optimization loss to help address the problem that bolt hole edges are easily misclassified as background.

[0133] Semantic segmentation loss measures the difference between the predicted and ground truth labels. We use the standard cross-entropy loss to calculate the semantic segmentation error. The formula is as follows:

[0134] (twenty three)

[0135] in, It refers to all the points in the point cloud data. It's a real label. It refers to the predicted label. Cross-entropy loss optimizes the network by minimizing the difference between the true label and the predicted label, thereby improving semantic segmentation accuracy.

[0136] Edge optimization loss is introduced to address the issue of bolt hole edges being easily misclassified as background. This loss, based on gradient information, helps the network better recover the details of the target edges. The specific formula is as follows:

[0137] (twenty four)

[0138] in, It is a point Edge gradient at the location, These are the edge gradients of the true labels. By minimizing the differences between gradients, the edge optimization loss enables the network to more accurately recover the target edges, thereby improving the segmentation accuracy of bolt hole targets.

[0139] Ultimately, the network's total loss function is a weighted sum of the semantic segmentation loss and the edge optimization loss. By simultaneously optimizing these two losses, the network can improve overall segmentation accuracy while refining the edges of the target objects. The formula for calculating the total loss function is:

[0140] (25)

[0141] in, This is the total loss function; , These are hyperparameters that control the weights of semantic loss and edge loss. By adjusting these two hyperparameters, the network can achieve a good balance in different tasks and optimize the target segmentation results.

[0142] The dual-stream decoder architecture effectively addresses the challenges of segmenting large backgrounds and bolt hole targets by processing background and bolt hole target branches separately. Through self-attention and cross-attention mechanisms, the background branch captures rich background semantic information, while the bolt hole branch significantly improves the segmentation accuracy of bolt hole targets through query refinement and edge optimization loss. This multi-task joint training method enables the network to accurately identify and segment bolt hole targets in complex backgrounds, improving segmentation performance in point cloud data processing tasks.

[0143] Post-processing and lightweight GNN classifier design: In point cloud segmentation tasks, especially for the detection and segmentation of bolt hole targets, the initial segmentation results output by the network often contain certain noise and misclassification phenomena. These noises mainly manifest as isolated point clusters, missegmented regions, or blurred boundaries, which are more pronounced in complex backgrounds or uneven point cloud density. To further improve the accuracy and robustness of the segmentation results, this scheme designs a post-processing module based on a lightweight Graph Neural Network (GNN) on the basis of the original network structure, which is used to perform topology analysis and noise filtering on the initial segmentation results.

[0144] This solution will detail the construction process of the post-processing module, including the construction of the graph structure, the design of the lightweight GNN classifier, the training strategy, and its performance analysis in practical applications.

[0145] Although dual-stream decoders have significantly improved segmentation accuracy, point cloud data often suffers from the following problems in real-world industrial scenarios: isolated noise clusters: due to sensor errors or environmental interference, small clusters of points unrelated to the real target may appear in the segmentation results; boundary misclassification: the edge points of bolt hole targets are easily misclassified as background, resulting in incomplete target shapes; uneven density distribution: the point cloud density varies greatly in different regions, affecting segmentation consistency.

[0146] Traditional post-processing methods, such as morphological operations and connected component analysis, can alleviate the above problems to some extent, but they struggle to capture high-order topological relationships between point clusters and are sensitive to parameters with limited generalization ability. Therefore, this solution introduces graph neural networks, leveraging their powerful structural modeling capabilities to learn the topological relationships between point clusters, thereby achieving more intelligent noise filtering.

[0147] To construct the graph structure, we first perform clustering on the segmentation mask output by the network to obtain several semantic point clusters. Each point cluster is regarded as a node in the graph, and the edges between nodes represent the topological relationships between clusters, such as spatial adjacency and density similarity.

[0148] The specific steps are as follows: Clustering: The DBSCAN clustering algorithm is used to cluster the segmentation results to obtain several semantic point clusters; Node feature extraction: Features are extracted for each point cluster, including the average coordinates, normal vector, density, and curvature of the points within the cluster; Edge construction: Edges are constructed based on the spatial distance, density difference, connectivity, etc. between clusters to form a graph structure G=(V,E), where V is the set of nodes and E is the set of edges.

[0149] In this way, we transform the original point cloud segmentation results into a structured graph, which facilitates subsequent graph neural network processing.

[0150] To model the topological relationships of point clusters while maintaining efficient inference, we designed a lightweight graph neural network classifier. This classifier takes a graph structure as input and outputs a judgment result indicating whether each point cluster is noise.

[0151] Node feature initialization: The initial features of each node consist of the geometric and semantic attributes of the point cluster, including: the average coordinates of points within the cluster. average normal vector Point cluster density mean curvature Semantic confidence From the segmentation network. The node feature vector is represented as:

[0152] (26)

[0153] In the formula, For node feature vectors, The average coordinates of the points within the cluster. The average normal vector, For point cluster density, The mean curvature and semantic confidence are given. From segmented networks;

[0154] For graph convolutional layer design, we employ a lightweight Graph Convolutional Network (GCN) structure for information propagation. The graph convolutional operations for each layer are as follows:

[0155] (27)

[0156] in, Represent the characteristics of node i in the l-th layer; The set of neighboring nodes of node i, AGG, represents an aggregation function (such as mean, max pooling, etc.). The weight matrix is ​​a learnable matrix; This is a non-linear activation function (such as ReLU). Through multi-layer graph convolution, nodes can aggregate structural information from their neighbors, thereby learning more discriminative feature representations.

[0157] After the graph convolutional layers, we use a fully connected layer to map the node features to a binary classification output (whether it is noise).

[0158] (28)

[0159] in, The node features are those obtained after convolution of the Lth layer graph; For classification weights; For bias terms; This represents the probability that the point cluster is noise.

[0160] Training strategy and loss function: To train this lightweight GNN classifier, we use the cross-entropy loss function:

[0161] (29)

[0162] Where M is the total number of nodes in the graph; The labels are real (0 represents the real target, 1 represents noise). Predict probabilities for the model.

[0163] Training data is generated through manual or automatic annotation to ensure a balance between positive and negative samples. To prevent overfitting, we employ regularization strategies such as Dropout and weight decay, and use an early stopping mechanism to monitor validation set performance.

[0164] First, the GNN classifier effectively captures the topological relationships in point cloud data by constructing a graph structure between point clusters. Bolt holes in point clouds typically appear as clusters of points with high local density and obvious geometric features, while isolated noise clusters often lack obvious geometric structure and adjacency relationships. Through node feature propagation and aggregation, GNN can automatically learn and distinguish between these two types of point clusters, thereby achieving effective identification and filtering of isolated noise and improving the overall consistency of segmentation results.

[0165] Secondly, the GNN model preserves local structural information during information propagation, thus better maintaining the edge features of bolt holes. Traditional morphological methods (such as erosion and dilation) can remove some noise, but they easily lead to blurred edges or shape distortion. In contrast, GNN, through graph convolution operations, can adaptively adjust node features based on adjacency relationships, avoiding over-smoothing of edge points, thereby better preserving the geometric shape and boundary details of bolt holes.

[0166] Furthermore, lightweight GNN designs theoretically possess lower computational complexity. Since the graph structure is built solely from point clusters, the number of nodes is far less than the number of points in the original point cloud, and graph convolution operations can be efficiently implemented using sparse matrices. Therefore, inference overhead is low, making it suitable for deployment on edge devices for real-time processing.

[0167] Finally, the GNN model has strong generalization ability. Because the graph structure can flexibly adapt to different point cloud densities and geometric distributions, and the node features contain a variety of geometric and semantic attributes, the model has good adaptability in different scenarios and different target types, and can effectively handle bolt hole segmentation tasks in complex backgrounds.

[0168] To achieve the above objectives, the present invention also provides a tunnel point cloud segmentation processing system based on dynamic sparse voxel map convolution and a dual-stream decoder, such as... Figure 4 As shown, the system is applied to the tunnel point cloud segmentation processing method based on dynamic sparse voxel map convolution and dual-stream decoder. The system includes:

[0169] A data generation and acquisition unit is used to generate and acquire first data corresponding to the original point cloud data based on a dynamic sparse voxel graph convolutional backbone network; wherein the first data is multi-scale feature data of the point cloud; a mechanism construction and generation unit is used to construct and generate a corresponding bolt hole query mechanism by combining high curvature region sampling and geometric prior; a first data processing and generation unit is used to segment and process the background branch region and the bolt hole target region respectively based on a dual-stream decoder, and generate corresponding second data; wherein the second data is preliminary segmentation result data; a second data processing and generation unit is used to combine a lightweight graph neural network to perform noise filtering and topology optimization processing on the second data respectively, and generate corresponding third data; wherein the third data is the final segmentation mask data.

[0170] The data generation and acquisition unit further includes: a first generation module, used to combine dynamic voxel map construction, hybrid convolutional unit and gated fusion mechanism to fuse sparse convolution and graph convolution, and respectively calculate and generate fourth data and fifth data corresponding to the original point cloud data; wherein, the fourth data is non-empty voxel data; the fifth data is geometric perception data of high curvature region;

[0171] And / or, the mechanism construction generation unit further includes: a second generation module, used to generate and acquire sixth data corresponding to the first data, and identify and generate seventh data corresponding to the sixth data; wherein, the sixth data is curvature data of each point in the point cloud; the seventh data is high curvature region data; a first processing module, used to construct and generate a corresponding bolt hole query mechanism based on the sixth data, and fuse and process geometric feature data and semantic feature data corresponding to the bolt hole query mechanism; a second processing module, used to generate and acquire eighth data corresponding to the bolt hole, and embed the eighth data through a multilayer perceptron and geometric transformation; wherein, the eighth data is physical prior information data of the bolt hole;

[0172] The first calculation module is used to calculate and generate sixth data corresponding to the first data; wherein the calculation formula is:

[0173] (4)

[0174] In the formula, Point The curvature characterizes local geometric properties; For point The approximate area of ​​the neighborhood is estimated by the average distance between neighborhood points; For point The set of k nearest neighbors; For point The normal vector; and Points and neighboring points 3D coordinates; Given the vector norm, calculate the projected distance between points; normal vector. The formula, calculated using principal component analysis, is as follows:

[0175] (5)

[0176] In the formula, Indicates the local surface orientation; PCA is principal component analysis, which calculates the principal orientation based on the coordinate difference of neighboring points. For neighborhood points relative to point The coordinate difference vector;

[0177] And / or, the first data processing and generation unit further includes: a first creation module, configured to combine self-attention and cross-attention mechanisms to create a first model corresponding to the background branch; wherein the first model is a global semantic model; a second creation module, configured to construct a geometrically enhanced self-attention mechanism, and based on the geometrically enhanced self-attention mechanism, generate ninth data corresponding to the bolt hole branch; wherein the ninth data is geometric feature data; and a third processing module, configured to sequentially fuse, process, and integrate the output of second data corresponding to the background branch and the bolt hole branch.

[0178] And / or, the second data processing and generation unit further includes: a first processing and generation module, used for clustering the second data and generating tenth data corresponding to the second data; wherein the tenth data is semantic point cluster data; a second processing and generation module, used for constructing a graph structure corresponding to the tenth data and generating eleventh data corresponding to the graph structure; wherein the eleventh data is node feature data, including average coordinate data, average normal vector data, point cluster density data, mean curvature data, and semantic confidence data; and a third processing and generation module, used for performing information propagation processing and noise classification processing based on a graph convolutional network, and generating corresponding third data.

[0179] In the system solution embodiment of the present invention, the specific details of the method steps involved in the tunnel point cloud segmentation processing based on dynamic sparse voxel map convolution and dual-stream decoder have been described above. That is to say, the functional modules in the system are used to implement the steps or sub-steps in the above method embodiment, and will not be repeated here.

[0180] To achieve the above objectives, the present invention also provides a tunnel point cloud segmentation processing platform based on dynamic sparse voxel map convolution and a dual-stream decoder, such as... Figure 5As shown, the device includes a processor, a memory, and a control program for a tunnel point cloud segmentation processing platform based on dynamic sparse voxel map convolution and a dual-stream decoder. The processor executes the control program, which is stored in the memory. This control program implements the steps of the tunnel point cloud segmentation processing method based on dynamic sparse voxel map convolution and a dual-stream decoder. Furthermore, the device also provides an analysis and calculation method for the safe thickness of pipe roof grouting in the disturbance zone of a tunnel face, including a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the computer program to implement the steps of the pipe roof grouting safe thickness analysis method; the specific details of these steps have been described above and will not be repeated here.

[0181] In this embodiment of the invention, the tunnel point cloud segmentation processing platform based on dynamic sparse voxel graph convolution and dual-stream decoder has a built-in processor that can be composed of integrated circuits. For example, it can be composed of a single packaged integrated circuit, or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor connects to various components using various interfaces and lines, and executes programs or units stored in memory, as well as calling data stored in memory, to perform various functions and process data for tunnel point cloud segmentation based on dynamic sparse voxel graph convolution and dual-stream decoder.

[0182] The memory is used to store program code and various data. It is installed in the tunnel point cloud segmentation processing platform based on dynamic sparse voxel map convolution and dual-stream decoder, and enables high-speed and automatic access to programs or data during operation. The memory includes read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0183] To achieve the above objectives, the present invention also provides a computer-readable storage medium, such as... Figure 6 As shown, the computer-readable storage medium stores a control program for a tunnel point cloud segmentation processing platform based on dynamic sparse voxel map convolution and a dual-stream decoder. This control program implements the steps of the tunnel point cloud segmentation processing method based on dynamic sparse voxel map convolution and a dual-stream decoder. In other words, a computer-readable storage medium is provided, storing a computer program. The computer program, when executed by a processor, implements the step of analyzing the safe thickness of the pipe roof grouting. Specific details of these steps have been described above and will not be repeated here.

[0184] In the description of embodiments of the present invention, it should be noted that any process or method description in the flowcharts or otherwise described herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the present invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order according to the functions involved, as should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0185] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processing module, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, a “computer-readable medium” can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, the computer-readable medium can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0186] This invention utilizes a method based on a dynamic sparse voxel graph convolutional backbone network to generate and acquire first data corresponding to the original point cloud data; wherein, the first data is multi-scale feature data of the point cloud; combining high curvature region sampling and geometric priors, a corresponding bolt hole query mechanism is constructed; based on a dual-stream decoder, the background branch region and the bolt hole target region are segmented and processed respectively, and corresponding second data is generated; wherein, the second data is preliminary segmentation result data; combined with a lightweight graph neural network, the second data is processed by noise filtering and topology optimization respectively, and corresponding third data is generated; wherein, the third data is the final segmentation mask data, along with the corresponding system, platform, and storage medium. By integrating multi-scale feature extraction, geometric prior guidance, and dual-task decoding, the accuracy and robustness of bolt hole segmentation are significantly improved, while ensuring high computational efficiency.

[0187] In other words, the proposed solution efficiently extracts multi-scale features of point clouds through a dynamic sparse voxel graph convolutional backbone network, accurately focuses on bolt hole regions by combining a query generation module based on geometric priors, processes global background and local targets separately using a dual-stream decoder, and intelligently filters noise through lightweight graph neural network post-processing. Ultimately, it achieves high-precision, high-robustness, and high-efficiency segmentation of bolt holes in complex industrial scenarios, significantly improving the reliability of industrial inspection and automated assembly.

[0188] In other words, this proposal presents a complete framework for bolt hole target detection and segmentation in complex point cloud data, including point cloud feature extraction based on dynamic sparse voxel graph convolution, a query generation module, a dual-stream decoder, and a lightweight graph neural network post-processing module. Dynamic sparse voxel graph convolution significantly improves the efficiency and accuracy of point cloud feature extraction by fusing sparse convolution and graph convolution, combined with dynamic voxel graph construction and gating fusion mechanisms, laying a solid foundation for subsequent tasks. The query generation module significantly improves the detection accuracy of bolt hole targets by introducing high-curvature region sampling and geometric priors. The dual-stream decoder processes the background and bolt hole target regions separately, achieving multi-scale, multi-task joint modeling. The background branch captures rich background semantic information through a self-attention mechanism, while the bolt hole branch significantly improves the segmentation accuracy of bolt hole targets through a geometrically enhanced self-attention mechanism and edge optimization loss. The lightweight graph neural network post-processing module further optimizes the segmentation results, effectively filtering noise and preserving the geometric shape and boundary details of the target through graph structure modeling and node feature propagation. Experimental results show that the proposed method performs well in the detection and segmentation of bolt hole targets in complex backgrounds, with high accuracy and robustness. It provides an effective solution for bolt hole target segmentation in the field of point cloud data processing, and has important theoretical significance and practical application value.

[0189] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A method for tunnel point cloud segmentation based on dynamic sparse voxel map convolution and dual-stream decoder, characterized in that, The method includes: Based on a dynamic sparse voxel graph convolutional backbone network, first data corresponding to the original point cloud data is generated and obtained; wherein, the first data is point cloud multi-scale feature data; By combining high curvature region sampling and geometric priors, a corresponding bolt hole query mechanism is constructed. The background branch region and the bolt hole target region are segmented and processed separately based on a dual-stream decoder, and corresponding second data is generated; wherein, the second data is the preliminary segmentation result data; By combining a lightweight graph neural network, the second data is processed by noise filtering and topology optimization, and corresponding third data is generated; wherein, the third data is the final segmentation mask data.

2. The tunnel point cloud segmentation method based on dynamic sparse voxel map convolution and dual-stream decoder according to claim 1, characterized in that, The process of generating and acquiring first data corresponding to the original point cloud data based on a dynamic sparse voxel graph convolutional backbone network further includes: By combining dynamic voxel map construction, hybrid convolutional units, and gating fusion mechanisms, sparse convolution and graph convolution are fused to calculate and generate fourth and fifth data corresponding to the original point cloud data, respectively; wherein, the fourth data is non-empty voxel data; and the fifth data is geometric perception data of high curvature regions.

3. The tunnel point cloud segmentation method based on dynamic sparse voxel map convolution and dual-stream decoder according to claim 1, characterized in that, The mechanism for generating corresponding bolt hole queries by combining high-curvature region sampling and geometric priors also includes: Generate and acquire sixth data corresponding to the first data, and identify and generate seventh data corresponding to the sixth data; wherein, the sixth data is the curvature data of each point in the point cloud; and the seventh data is high curvature region data; Based on the sixth data, a corresponding bolt hole query mechanism is constructed and generated, and geometric feature data and semantic feature data corresponding to the bolt hole query mechanism are fused and processed. The eighth data corresponding to the bolt hole is generated and acquired, and the eighth data is embedded and processed by multilayer perceptron and geometric transformation; wherein the eighth data is the physical prior information data of the bolt hole.

4. The tunnel point cloud segmentation method based on dynamic sparse voxel map convolution and dual-stream decoder according to claim 1 or 3, characterized in that, The mechanism for generating corresponding bolt hole queries by combining high-curvature region sampling and geometric priors also includes: Calculate and generate a sixth data point corresponding to the first data point; wherein the calculation formula is: （4） In the formula, Point The curvature characterizes local geometric properties; For point The approximate area of ​​the neighborhood is estimated by the average distance between neighborhood points; For point The set of k nearest neighbors; For point The normal vector; and Points and neighboring points 3D coordinates; Given the vector norm, calculate the projected distance between points; normal vector. The formula, calculated using principal component analysis, is as follows: （5） In the formula, Indicates the local surface orientation; PCA is principal component analysis, which calculates the principal orientation based on the coordinate difference of neighboring points. For neighborhood points relative to point The coordinate difference vector.

5. The tunnel point cloud segmentation method based on dynamic sparse voxel map convolution and dual-stream decoder according to claim 1, characterized in that, The process of segmenting and processing the background scene region and the bolt hole target region based on the dual-stream decoder, and generating corresponding second data, also includes: By combining self-attention and cross-attention mechanisms, a first model corresponding to the background branch is created; wherein, the first model is a global semantic model; A geometric prior-enhanced self-attention mechanism is constructed, and based on the geometric prior-enhanced self-attention mechanism, a ninth data point corresponding to the bolt hole branch is generated; wherein, the ninth data point is geometric feature data. The data is sequentially fused, processed, and integrated to output second data corresponding to the background branch and the bolt hole branch.

6. The tunnel point cloud segmentation method based on dynamic sparse voxel map convolution and dual-stream decoder according to claim 1, characterized in that, The method of combining a lightweight graph neural network to perform noise filtering and topology optimization on the second data, and generating corresponding third data, further includes: The second data is clustered to generate tenth data corresponding to the second data; wherein, the tenth data is semantic point cluster data; Construct a graph structure corresponding to the tenth data, and generate eleventh data corresponding to the graph structure; wherein, the eleventh data is node feature data, including average coordinate data, average normal vector data, point cluster density data, mean curvature data, and semantic confidence data; Based on graph convolutional networks, information propagation processing and noise classification processing are performed respectively, and corresponding third data is generated.

7. A tunnel point cloud segmentation system based on dynamic sparse voxel map convolution and dual-stream decoder, characterized in that, The system is applied to the tunnel point cloud segmentation processing method based on dynamic sparse voxel map convolution and dual-stream decoder as described in any one of claims 1 to 6, and the system comprises: The data generation and acquisition unit is used to generate and acquire first data corresponding to the original point cloud data based on a dynamic sparse voxel map convolutional backbone network; wherein, the first data is point cloud multi-scale feature data; The mechanism construction generation unit is used to combine high curvature region sampling and geometric priors to construct and generate the corresponding bolt hole query mechanism; The first data processing and generation unit is used to segment and process the background branch region and the bolt hole target region respectively based on the dual-stream decoder, and generate corresponding second data; wherein, the second data is the preliminary segmentation result data; The second data processing and generation unit is used to combine a lightweight graph neural network to perform noise filtering and topology optimization processing on the second data, and generate corresponding third data; wherein the third data is the final segmentation mask data.

8. The tunnel point cloud segmentation system based on dynamic sparse voxel map convolution and dual-stream decoder according to claim 7, characterized in that, The data generation and acquisition unit further includes: The first generation module is used to combine dynamic voxel map construction, hybrid convolutional units and gated fusion mechanism to fuse sparse convolution and graph convolution, and to calculate and generate the fourth data and the fifth data corresponding to the original point cloud data respectively; wherein, the fourth data is non-empty voxel data; the fifth data is geometric perception data of high curvature region; And / or, the mechanism for constructing the generation unit further includes: The second generation module is used to generate and acquire sixth data corresponding to the first data, and identify and generate seventh data corresponding to the sixth data; wherein, the sixth data is the curvature data of each point in the point cloud; and the seventh data is high curvature region data; The first processing module is used to construct and generate a corresponding bolt hole query mechanism based on the sixth data, and to fuse and process the geometric feature data and semantic feature data corresponding to the bolt hole query mechanism. The second processing module is used to generate and acquire the eighth data corresponding to the bolt hole, and to embed the eighth data through a multilayer perceptron and geometric transformation; wherein, the eighth data is the physical prior information data of the bolt hole; The first calculation module is used to calculate and generate sixth data corresponding to the first data; wherein the calculation formula is: （4） In the formula, Point The curvature characterizes local geometric properties; For point The approximate area of ​​the neighborhood is estimated by the average distance between neighborhood points; For point The set of k nearest neighbors; For point The normal vector; and Points and neighboring points 3D coordinates; Given the vector norm, calculate the projected distance between points; normal vector. The formula, calculated using principal component analysis, is as follows: （5） In the formula, Indicates the local surface orientation; PCA is principal component analysis, which calculates the principal orientation based on the coordinate difference of neighboring points. For neighborhood points relative to point The coordinate difference vector; And / or, the first data processing and generation unit further includes: The first creation module is used to combine self-attention and cross-attention mechanisms to create a first model corresponding to the background branch; wherein, the first model is a global semantic model; The second creation module is used to construct a geometrically enhanced self-attention mechanism and, based on the geometrically enhanced self-attention mechanism, generate the ninth data corresponding to the bolt hole branch; wherein the ninth data is geometric feature data. The third processing module is used to sequentially fuse, process, and integrate the output of second data corresponding to the background branch and the bolt hole branch; And / or, the second data processing generation unit further includes: The first processing and generation module is used to perform clustering processing on the second data and generate tenth data corresponding to the second data; wherein, the tenth data is semantic point cluster data; The second processing and generation module is used to construct a graph structure corresponding to the tenth data and generate eleventh data corresponding to the graph structure; wherein, the eleventh data is node feature data, including average coordinate data, average normal vector data, point cluster density data, mean curvature data and semantic confidence data; The third processing and generation module is used to perform information propagation processing and noise classification processing based on graph convolutional networks, and generate corresponding third data.

9. A tunnel point cloud segmentation processing platform based on dynamic sparse voxel map convolution and dual-stream decoder, characterized in that, The system includes a processor, a memory, and a control program for a tunnel point cloud segmentation processing platform based on dynamic sparse voxel graph convolution and a dual-stream decoder. The processor executes the control program, which is stored in the memory. The control program implements the tunnel point cloud segmentation processing method based on dynamic sparse voxel graph convolution and a dual-stream decoder as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a control program for a tunnel point cloud segmentation processing platform based on dynamic sparse voxel graph convolution and dual-stream decoder. The control program for the tunnel point cloud segmentation processing platform based on dynamic sparse voxel graph convolution and dual-stream decoder implements the tunnel point cloud segmentation processing method based on dynamic sparse voxel graph convolution and dual-stream decoder as described in any one of claims 1 to 6.

Images