A steel structure assembly quality detection technology and method based on laser point cloud

Abstract

This invention belongs to the field of steel structure assembly quality inspection and digital construction. It discloses a steel structure assembly quality inspection technology and method based on laser point clouds. The method involves acquiring point clouds, constructing a multi-source point cloud dataset and preprocessing it, and calculating the local normal vectors and geometric edge information of the point clouds. A semantic segmentation model for bolt holes in the steel structure point cloud is constructed. An edge self-attention mechanism is used to construct edge-aware priors by analyzing the point cloud normal vector field, generating a geometric saliency feature map to modulate the deep features of the encoder, and then obtaining edge enhancement features through the self-attention mechanism. The semantic segmentation model is trained using a transfer learning strategy combining pre-training and fine-tuning. An edge-aware focus loss function is introduced to solve the class imbalance problem. The model performance is verified through a semantic segmentation and instance segmentation evaluation mechanism. Based on the semantic segmentation results, the centroids of the bolt hole point clouds are extracted to complete the bolt hole group positioning and matching. Rigid registration is used to achieve virtual pre-assembly of components and detect assembly errors.

Description

Technical Field

[0001] This invention belongs to the field of digital construction and pre-assembly technology of steel structures, specifically involving a steel structure assembly quality inspection technology and method based on laser point cloud, which can be used for manufacturing quality assessment and assembly error analysis of projects such as steel bridges and prefabricated steel structures. Background Technology

[0002] With the continuous development of prefabricated construction technology, precast steel structures have been widely used in modern building engineering, especially steel bridge engineering, due to their ability to effectively reduce on-site work, shorten construction cycles, and improve the stability of component manufacturing quality. Compared with traditional on-site assembly methods, modular and prefabricated construction methods have higher construction efficiency and cost advantages. However, this type of construction method places higher demands on the manufacturing precision of components, especially at connection interfaces. Geometric errors will directly affect the assembly feasibility and structural safety of components.

[0003] In precast steel structures, bolt holes are crucial for reliable connections between components. Their positional accuracy, axial orientation, and relative spatial relationships have a decisive impact on the overall structural stability. Misalignment, deviation, or machining errors in bolt holes can lead to difficulties in on-site assembly, connection failures, and even structural safety hazards. Therefore, effectively verifying the connection accuracy of components before they are transported to the construction site has become a core technical issue in the manufacturing and assembly of steel structures.

[0004] To avoid on-site rework and project delays, physical trial assembly is typically conducted in the manufacturing plant in engineering practice to verify the accuracy of component interfaces through actual alignment. While this method can reduce construction risks to some extent, it is time-consuming and labor-intensive, involving component handling, hoisting, and repetitive organization. Once errors are discovered, the adjustment costs are high, making it difficult to meet the requirements of high-efficiency manufacturing and rapid delivery. To improve efficiency, virtual trial assembly technology has been gradually introduced into BIM platforms, using digital models to simulate and analyze the component assembly process. However, existing virtual trial assembly methods still heavily rely on manual measurement to obtain component geometric information, such as using a total station to collect data on key connection points. This method suffers from low efficiency, susceptibility to environmental influences, and limited measurement accuracy.

[0005] With the development of laser scanning technology, LiDAR-based point cloud acquisition methods have provided a technological foundation for obtaining high-precision three-dimensional geometric information of steel structure components. Dense point cloud data, including the overall shape of the component and details of local connections, can be obtained using handheld or ground-based laser scanning equipment. However, key technical challenges remain in practical applications. On the one hand, connection features such as bolt holes account for a very small proportion of the point cloud, representing a highly unbalanced, small-scale geometric structure. On the other hand, point cloud data is often accompanied by noise, occlusion, and uneven density, making it difficult for traditional rule-based or manual feature engineering methods to achieve stable and automated recognition. Existing point cloud semantic segmentation techniques are effective in recognizing large-scale structural elements, but they often lack accuracy when segmenting small cylindrical bolt holes and have limited adaptability to complex industrial scenarios.

[0006] Therefore, there is an urgent need for a technical solution that can combine multi-source point cloud data, automatically identify key connection features of steel structures, and support virtual pre-assembly and quantitative analysis of assembly errors, so as to realize an integrated digital process from component quality inspection to assembly verification and improve the level of intelligence in steel structure manufacturing and assembly. Summary of the Invention

[0007] To address the problems of low automation, insufficient accuracy in identifying connection features, and difficulty in quantitatively analyzing assembly errors in existing precast steel structure quality inspection and virtual trial assembly processes, this invention proposes a steel structure assembly quality inspection technology, method, and system based on laser point clouds.

[0008] This invention is implemented as follows: a technology and method for inspecting the assembly quality of steel structures based on laser point clouds, the method comprising:

[0009] S1: Theoretical model acquisition and multi-source dataset construction;

[0010] S2: Construction of a semantic segmentation model based on edge self-attention;

[0011] S3: Marginal loss function, model training, and evaluation;

[0012] S4: Pre-assembly and error detection.

[0013] Furthermore, S1 includes:

[0014] (1) Theoretical model acquisition and multi-source dataset construction: By constructing a multi-source data system that combines theoretical point clouds and measured point clouds, a complete and high-fidelity geometric information foundation is provided for the identification of key connection features of steel structures and virtual pre-assembly. Among them, the measured point cloud is acquired by laser scanning equipment, preferably using a handheld laser scanner to scan the steel structure node plates and key connection parts to obtain high-precision three-dimensional data reflecting the actual manufacturing state of the components. The theoretical point cloud is extracted from the steel structure design drawings or three-dimensional CAD models, and its triangular mesh model is uniformly sampled to generate the ideal geometric shape under the design intent. Through the fusion of theoretical point cloud and measured point cloud, a comprehensive characterization of the geometric state of steel structure components is achieved, and a benchmark and comparison are provided for subsequent quality inspection and error analysis.

[0015] (2) Dataset preprocessing and geometric feature construction: To ensure the consistency and reliability of multi-source point cloud data, the theoretical point cloud and the measured point cloud were downsampled and cleaned respectively. While maintaining the key connection feature details such as bolt holes, the spatial resolution was unified and outliers were removed. At the same time, local normal vectors were calculated for each point to characterize the surface direction features, and geometric edge information was extracted based on the change of neighborhood normal vectors to identify potential connection regions. After preprocessing, the theoretical point cloud and the measured point cloud were divided into pre-training data and fine-tuning data according to a preset strategy, so that the model can first learn the ideal geometric structure features and then adapt to the noise and deviation in the real manufacturing environment, thereby improving the robustness and generalization ability in the virtual trial assembly task.

[0016] Furthermore, S2 includes:

[0017] (1) Construction of edge perception prior and self-attention enhancement mechanism;

[0018] (2) Construction of encoder-decoder semantic segmentation network;

[0019] (3) Bolt hole feature refinement and instantiation processing.

[0020] Furthermore, step (1) specifically includes:

[0021] In the point cloud of steel structures, bolt holes are small-scale cylindrical structures with far fewer points than the background planar area, which can easily lead to class imbalance and misidentification during segmentation. To enable the model to focus on geometrically abrupt locations such as bolt hole boundaries, the point cloud normal vector field is analyzed to construct an edge-aware prior. This prior is calculated as follows: for any point in the point cloud... The set of neighboring points is obtained through k-nearest neighbor search, and the degree of difference between the normal vector of the given point and the normal vectors of its neighbors is calculated. The average of these differences is defined as the edge score. As shown in formula (1):

[0022] (1)

[0023] in For point The normal vector, Let the normal vector of the j-th point in the neighborhood be the edge score. As a priori for edge perception, locations with higher edge scores typically correspond to bolt hole boundaries or regions of geometric abrupt changes.

[0024] In addition to explicit priors, this invention also proposes an edge-aware self-attention (ESA) mechanism, which first generates a geometrically saliency feature map through two layers of one-dimensional convolution. And used to modulate deep features of the encoder. To enhance the response in regions of significant geometric change, as shown in Equation (2):

[0025] (2)

[0026] Based on this, the modulated features are input into the self-attention mechanism. The features are mapped to a query vector Q, a key vector K, and a value vector V through three one-dimensional convolutions, and the attention weight matrix is ​​calculated, as shown in equations (3, 4).

[0027] (3)

[0028] (4)

[0029] Then, the enhanced features are obtained by multiplying the attention weight matrix with the value vector, and finally, the enhanced features are obtained through residual connections. As shown in formula (5):

[0030] (5)

[0031] Compared to other mainstream edge feature modeling methods, the ESA mechanism utilizes the coupling of geometric features to achieve a physically-based representation learning process. This is achieved by modulating features from geometric data. The Q and K projections are derived, and the resulting attention weight matrix A no longer depends solely on the similarity of abstract features. Instead, it is intrinsically influenced by the surface geometric priors established during modulation. This ensures that during global context aggregation, the network prioritizes the interactions between structurally significant regions while effectively suppressing redundant information from numerous flat steel surfaces. Therefore, this two-stage refinement enables the ESA module to maintain high sensitivity to micro-geometric details in large-scale structural contexts, resolving the class imbalance problem and ensuring robust identification of bolt holes in subsequent quality inspection tasks.

[0032] Furthermore, step (2) specifically includes:

[0033] After obtaining edge enhancement features, a point cloud semantic segmentation network based on an encoder-decoder structure is constructed to achieve bolt hole recognition. The encoder consists of multiple layers of EdgeConv modules. The input point cloud is first sampled from the farthest point to ensure spatial uniformity. Then, KNN search is used to establish local neighborhood relationships. The coordinate differences of the points in the neighborhood are concatenated with the normal vectors to form edge features. Local geometric relationships are extracted through EdgeConv, so that the features gradually transition from fine-grained geometric details to global structural representation, and finally, low-order features are obtained. Intermediate-order features and higher-order features Advanced features and normal vectors are input together into the ESA mechanism module to obtain edge enhancement features. Decoder , , Features are fused at multiple scales. Features at different spatial scales are extracted and integrated through parallel convolution. At the same time, the global feature extraction module is used to obtain the overall structural semantic information, which is then fused with local features to take into account both the local details of the bolt holes and the overall geometric relationship of the components. The fused features are upsampled layer by layer to restore the original number of points, and the class probability of each point is output through the segmentation head, thereby completing the semantic segmentation of the bolt holes and the background area.

[0034] Furthermore, step (3) specifically includes:

[0035] After obtaining the semantic segmentation results, post-processing is performed on the points predicted to be bolt holes to transform the bolt hole region into independent instances with engineering significance. First, Euclidean clustering is performed based on a spatial distance threshold, aggregating adjacent points into clusters, each cluster corresponding to a candidate bolt hole. Then, a local consistency check is performed on each cluster. Outliers with insufficient neighborhoods are removed by counting the number of neighboring points within a specified radius, and anomalous morphological points are filtered based on the cylindrical geometric features, ensuring the remaining points maintain a continuous and stable spatial structure. The point clusters obtained after clustering and geometric constraint refinement can stably represent the actual bolt hole positions and shapes, thus providing reliable input data for subsequent centroid calculation, hole group matching, and accurate alignment during virtual trial assembly.

[0036] Furthermore, S3 specifically includes:

[0037] (1) Construction of Transfer Learning Training Strategy: To fully utilize both theoretical and measured point cloud data resources, a transfer learning strategy combining pre-training and fine-tuning was adopted to optimize the segmentation network. First, the segmentation network was pre-trained on theoretical point cloud data generated from the design model, allowing the model to learn the general structural features of prefabricated steel structures, including bolt hole morphology, component edges, and surface patterns, in an ideal, noise-free geometric environment, thereby establishing basic geometric cognitive abilities. Subsequently, the pre-trained model was transferred to measured point cloud data for fine-tuning training, allowing the network to gradually adapt to real-world factors such as noise, occlusion, manufacturing deviations, and scanning artifacts present in the laser scanning data. Through sequential optimization on theoretical and measured data, the model maintains ideal geometric recognition capabilities while enhancing its adaptability to real industrial scenarios, effectively bridging the distribution differences between the design state and the actual manufacturing state, and improving overall generalization ability and robustness.

[0038] (2) Construction of Edge-Aware Focus Loss Function: To address the class imbalance problem caused by the extremely small proportion of bolt holes in the point cloud, and to further enhance the learning ability of key boundary regions, the Edge-Aware Focus Loss Function (EAFL) is introduced during the training process. In the forward propagation stage, the segmentation network outputs the class probability of each point. and the edge score calculated by the normal vector gradient module. The standard focus loss function is shown in equation (6):

[0039] (6)

[0040] Where N is the number of points and M is the number of categories. For real labels, To predict the label, representing the point Predicted as The probability of the category. Here are the modulation parameters, where The focusing parameter is a specifyable hyperparameter used to reduce the weight of easily classified samples.

[0041] To enhance the network's focus on regions of geometric abrupt change, an edge weight factor is introduced in addition to the focal loss. ,in This is a hyperparameter used to adjust the importance of points with higher marginal scores. Finally, it is combined with the class balance coefficient. The edge-aware focus loss is constructed as shown in formula (7):

[0042] (7)

[0043] By embedding edge scores into the weights of the loss function, samples in key regions such as bolt hole boundaries generate stronger gradient signals during backpropagation. This dynamically allocates more learning power to sparse but critical structural regions, improving the segmentation ability of small-scale features while maintaining training stability.

[0044] (3) Construction of semantic segmentation and instance segmentation evaluation mechanism; To comprehensively evaluate model performance, precision, recall, F1 score, and intersection-over-union (IoU) are introduced at the semantic segmentation level. For category i, the definitions of the above indicators are shown in formulas (8-11):

[0045] (8)

[0046] (9)

[0047] (10)

[0048] (11)

[0049] TP, FP, and FN represent true positive, false positive, and false negative, respectively. Overall performance is measured by overall accuracy (OA), weighted average (WA), macro average (MA), and mean crossover ratio (mIoU).

[0050] At the instance segmentation level, to evaluate the accuracy of bolt hole instance extraction, the predicted bolt hole point clusters are matched with the ground truth labels. First, Euclidean clustering is used to cluster the predicted points and ground truth points into instances. Then, the Hungarian algorithm is used to establish a one-to-one correspondence between predicted instances and ground truth instances to minimize the total allocation cost. For instance pairs The definition of the intersection-union ratio of its instances is shown in formula (12):

[0051] (12)

[0052] The definition of the scene-level instance average intersection-union ratio is shown in formula (13):

[0053] (13)

[0054] Where N represents the number of matched instance pairs. Ultimately, this instance segmentation metric reflects both the shape accuracy and spatial positioning precision of bolt hole instances, providing a supplementary evaluation of the semantic segmentation results and thus rigorously validating the model's performance in practical applications within industrial point cloud environments.

[0055] Furthermore, S4 specifically includes:

[0056] (1) Precise extraction of the centroid of the bolt hole point cloud:

[0057] After segmenting the bolt hole instances, high-precision positioning of the spatial center of each bolt hole is required to achieve accurate alignment during the virtual pre-assembly process. For a single bolt hole point cloud cluster obtained from the segmentation, the farthest point sampling (FPS) algorithm is first used to select multiple pairs of points with the most dispersed spatial distribution within the cluster. Each pair of points forms a line segment in three-dimensional space. For each line segment, a perpendicular bisector is constructed, and assuming that the principal axis of the bolt hole is aligned with the z-axis, all perpendicular bisectors are orthogonally projected onto a plane perpendicular to the hole axis, i.e., the x-y plane. Let the projection line corresponding to the k-th sampling point be L. k The optimal plane coordinates (x) are solved by least squares estimation. c , y c The goal is to minimize the sum of the squared distances from the point to all projected lines, thereby determining the center position of the bolt hole in the normal plane. Then, the z-axis is determined by statistically analyzing the maximum and minimum values ​​of the point cloud of the hole in the z-direction and taking the average. c Finally, the three-dimensional centroid coordinates (x, y) of the bolt hole were obtained. c , y c , z c The spatial uniformity brought about by sampling a large number of farthest point pairs, along with the mean effect of least squares fitting, can effectively reduce the influence of local noise and outliers, thereby improving the robustness and stability of centroid localization.

[0058] (2) Positioning and matching of bolt hole groups:

[0059] After obtaining the centroid of each bolt hole, the bolt hole groups need to be standardized in positioning and numbering to achieve hole matching between the components to be assembled. First, the orientation of the components is adjusted so that the principal axes of the bolt holes are aligned with the z-axis, resulting in a standard circular distribution of holes in the x-y plane. Then, the bolt hole centers are grouped and analyzed in the x-y plane, and principal component analysis (PCA) is used to extract the principal axes of the hole groups, including the horizontal and vertical principal axes. By projecting each hole center onto this local principal axis coordinate system and numbering them according to the projected coordinates starting from the top left corner in row priority order, a stable one-to-one correspondence can be established between the hole groups of two components even with overall tilt or rotation. This method, through geometric normalization and systematic projection numbering, effectively solves the influence of hole group skew, rotation, or local deformation on matching accuracy, ensuring the accuracy of the subsequent virtual registration process.

[0060] (3) Component assembly standardization and quality assessment:

[0061] After completing the hole group matching, the two components are rigidly registered using the corresponding bolt hole center point set. Specifically, the Iterative Closest Point (ICP) algorithm or Procrustes analysis method is used to solve for the optimal rotation matrix R and translation vector t based on the spatial relationship between the corresponding hole centers, and a 4×4 homogeneous transformation matrix is ​​constructed to rigidly transform one component to the target component coordinate system. After component alignment, the Euclidean distance between the centers of each pair of matching bolt holes is calculated, and the distances of all matching pairs are statistically summarized to obtain the total assembly error index, used to quantitatively evaluate assembly accuracy. In the case of continuous assembly of multiple pipe sections or modules, the registered components participate in the subsequent registration process as a whole rigid body, iteratively executing the above transformation and error calculation steps; for closed-loop assembly scenarios, only the error index is updated without repeating pair registration. Through this virtual pre-assembly and error quantification process, alignment deviations and fitting problems can be identified in advance before physical construction, thereby reducing the risk of on-site rework and improving assembly efficiency and structural reliability.

[0062] Another objective of this invention is to provide a laser point cloud-based steel structure assembly quality inspection system for implementing the aforementioned method, the system comprising:

[0063] The point cloud acquisition and preprocessing module is used for laser point cloud data acquisition, multi-source point cloud fusion, point cloud downsampling, normal vector calculation, and edge feature extraction.

[0064] The feature recognition and model building module, connected to the point cloud acquisition and preprocessing module, is used to build a point cloud semantic segmentation network model based on edge perception mechanism enhancement, so as to realize the automatic recognition and segmentation of key connection features of steel structures.

[0065] The training and evaluation module, connected to the feature recognition and model building module, is used for transfer learning training, edge-weighted loss optimization, and performance evaluation of semantic segmentation and instance segmentation.

[0066] The virtual assembly and error analysis module, connected to the training and evaluation module, is used for bolt hole centroid extraction, hole group matching, rigid registration, and quantitative calculation and quality assessment of assembly errors.

[0067] Another objective of this invention is to provide a computer device, the computer device including a memory and a processor, the memory storing a computer program, the computer program being executed by the processor causing the processor to perform the steps of the steel structure assembly quality inspection technology and method based on laser point clouds, including point cloud preprocessing, key connection feature identification, model training and optimization, virtual assembly registration, and assembly error detection process.

[0068] Based on the above technical solutions and the technical problems solved, please analyze the advantages and positive effects of the technical solution to be protected by this invention from the following aspects:

[0069] To address the problems of reliance on manual measurement and physical trial assembly, low inspection efficiency, high cost, and difficulty in automatically identifying and quantitatively analyzing key bolt hole connection features in existing steel structure assembly processes, this invention proposes a steel structure assembly quality inspection technology and method based on laser point clouds. Through multi-source point cloud fusion, edge enhancement feature recognition, and quantitative analysis of virtual assembly errors, it effectively solves the problem of insufficient automation in assembly quality inspection. The specific description is as follows:

[0070] This invention constructs a multi-source data-driven assembly quality inspection process based on the measured point cloud of the steel structure obtained by laser scanning and the theoretical point cloud data generated by the design drawings. It automatically identifies and segments the key connection areas of the steel structure, and completes the extraction of bolt hole instances and precise positioning of the centroid based on the segmentation results. The results of the verification based on real bridge engineering project data show that: (1) In semantic segmentation and instance segmentation tasks, the proposed method achieves average intersection-union ratios (mIoU) of 0.981 and 0.986 respectively, demonstrating excellent recognition accuracy and stability; (2) Compared with existing mainstream point cloud segmentation models such as PointNet, PointTransformer and DGCNN, this method has higher precision and recall in small-scale cylindrical bolt hole feature recognition, significantly improving the detection capability of key connection features; (3) By introducing an edge enhancement mechanism and a self-attention feature modulation strategy, the class imbalance problem caused by the extremely low proportion of bolt holes in the overall point cloud is effectively alleviated, and the ablation experiment further verifies the effectiveness of each functional module; (4) In the virtual assembly stage, the robust extraction of the bolt hole centroid is achieved by combining the farthest point sampling and least squares estimation, and the optimal spatial transformation is solved by combining the rigid registration algorithm to achieve quantitative analysis of assembly error, which can effectively identify manufacturing deviations and potential alignment problems. Compared to the traditional physical trial assembly process, this method significantly improves the efficiency of assembly and testing, realizes automated and high-precision assembly quality assessment, reduces the risk of rework caused by alignment errors while reducing manual intervention, and provides reliable technical support for digital construction and intelligent quality control of precast steel structures.

[0071] This invention develops a novel intelligent inspection method based on laser point clouds for steel structure assembly quality inspection. By constructing an edge-enhanced point cloud analysis network, it achieves automatic identification of key connection features of the steel structure; through instance-level hole group matching and rigid registration algorithms, it achieves accurate reconstruction of the spatial relationships between components; and through quantitative error calculation methods, it achieves objective evaluation of assembly quality. This invention combines point cloud analysis results with a virtual assembly process to construct an assembly quality inspection framework that can be embedded in digital construction platforms or BIM systems. The proposed method not only improves assembly inspection efficiency but also enhances the quantifiability of assembly accuracy, providing reliable technical support for the digital construction and intelligent management of prefabricated steel structures. This invention helps to form an integrated inspection process of automatic identification, virtual alignment, and error assessment, promoting the transformation of the steel structure assembly process from traditional manual experience-based judgment to a digital and intelligent quality control mode, which is of great significance for improving the construction efficiency and reliability of prefabricated steel structures. Attached Figure Description

[0072] Figure 1 This is an overall framework diagram of steel structure assembly quality inspection provided in an embodiment of the present invention;

[0073] Figure 2 This is a flowchart of obtaining theoretical point clouds from AutoCAD drawings provided in an embodiment of the present invention;

[0074] Figure 3 This is a structural diagram of the bolt hole semantic segmentation network provided in an embodiment of the present invention;

[0075] Figure 4 This is a flowchart illustrating the instantiation and refinement of bolt holes provided in this embodiment of the invention.

[0076] Figure 5 This is a flowchart of the two-dimensional positioning of the bolt hole center point provided in an embodiment of the present invention;

[0077] Figure 6 This is a flowchart of bolt hole positioning and matching provided in an embodiment of the present invention;

[0078] Figure 7 This is a flowchart of the assembly and quality assessment provided in the embodiments of the invention;

[0079] Figure 8 These are the index curves (a) and (b) of the network in the pre-training stage and the fine-tuning stage of transfer learning provided in this embodiment of the invention. The loss value and the learning rate lr are both relative values.

[0080] Figure 9 This is the confusion matrix provided in the embodiments of the present invention;

[0081] Figure 10This is a comparison chart of the inference results of some models provided in the embodiments of the present invention;

[0082] Figure 11 This is a diagram illustrating the three-component assembly process provided in an embodiment of the present invention;

[0083] Figure 12 This is a diagram showing the assembly error detection results provided in an embodiment of the present invention;

[0084] Figure 13 This is a schematic diagram of the overall perspective point cloud of the bulk component (a) and the assembled structure (b) provided in the embodiments of the present invention;

[0085] Figure 14 The embodiments of the present invention provide (a) bulk components of the first and second parts, (b) assembled components of the first and second parts, (c) bulk components of the second and third parts, and (d) schematic diagram of assembled components of the second and third parts;

[0086] Figure 15 This is a schematic diagram of the assembly process provided in the embodiment of the present invention; wherein, (a) point pairs and transformations of part 1 and part 2, (b) error checks of part 1 and part 2, (c) point pairs and transformations of part 2 and part 3, and (d) error checks of part 2 and part 3;

[0087] Figure 16 This is a schematic diagram of segmental virtual assembly scene 2 provided in an embodiment of the present invention; wherein, (a) is a mesh model, and (b) is a point cloud;

[0088] Figure 17 This is a schematic diagram of the local hole-by-hole error distribution of the final assembly step provided in the embodiment of the present invention; (a) point-to-point transformation, (b) error check. Detailed Implementation

[0089] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0090] Reference Figure 1 As shown, this embodiment provides a steel structure assembly quality inspection technology and method based on laser point clouds, including the following steps:

[0091] Theoretical model acquisition and multi-source dataset construction. Specifically, this includes:

[0092] Step 1.1 Acquisition of Theoretical and Measured Point Clouds. In this embodiment, the steel structure point cloud data is collected from precast steel components in a steel bridge under construction. For theoretical data, the 3D CAD design model of the corresponding component is first acquired, and the surface mesh of the model is uniformly sampled to generate theoretical point cloud data. The generated theoretical point cloud has the characteristics of complete geometric structure, no noise interference, and no missing data, and can be used as the benchmark data for subsequent algorithm training and performance verification, while providing stable geometric prior information for the network model. For measured data, the FreeScan Trak Nova laser scanning system manufactured by Shining 3D is used to perform 3D scanning and acquisition of the actual steel structure components. This device can acquire high-density point cloud data with a sampling resolution of about 0.5 mm, thereby accurately recording the fine-grained information of the surface geometry of the component and key structural features such as bolt holes. The measured point cloud acquired by this scanning device can truly reflect the actual situation on the construction site, including complex factors such as scanning noise, local occlusion, and uneven point cloud density. Therefore, this part of the data is mainly used for fine-tuning the model and verifying its performance, so as to improve the robustness and applicability of the method in real engineering environments.

[0093] Step 1.2 Point Cloud Preprocessing and Multi-Source Dataset Construction. After acquiring theoretical and measured point clouds, all point cloud data are imported into CloudCompare software for preprocessing. First, voxel downsampling is performed on each point cloud data to unify the spatial resolution to 0.5 mm. This process effectively reduces point cloud redundancy while maintaining the geometric shape and key structural details of the components, thereby reducing the computational burden and improving data processing efficiency. Subsequently, point-by-point normal vector information is calculated from the downsampled point clouds to obtain local surface geometric features, providing auxiliary information for subsequent feature learning and boundary recognition based on geometric relationships. After preprocessing, manual semantic annotation is performed on the measured point clouds. Specifically, experienced professionals use point cloud annotation tools to classify the point clouds point by point, dividing them into "bolt holes" and "background" categories, thus constructing a high-precision annotation dataset for supervised learning. For theoretical point cloud data, corresponding semantic labels are automatically generated based on the geometric structural information in the CAD design model to reduce the workload of manual annotation and ensure annotation consistency. After annotation, statistical analysis was performed on the preprocessed theoretical point cloud dataset and the measured point cloud dataset. The point count information for each category is summarized in Table 1. Regarding data partitioning, the theoretical point cloud dataset was divided into training and testing sets in an 8:2 ratio, used only for pre-training and initial validation of the network model. Simultaneously, the measured point cloud data was divided into 10 independent scenes for cross-validation during subsequent fine-tuning training, thereby comprehensively evaluating the model's generalization ability under different component scenarios. The point cloud size and category distribution information for each scene are summarized in Table 2.

[0094] Table 1. Categorical statistics of the number of points in theoretical and experimental datasets.

[0095]

[0096] Table 2. Categorical statistics of the number of each scene point in the measured dataset.

[0097]

[0098] Step 2: Constructing a semantic segmentation model based on edge self-attention, specifically including:

[0099] Step 2.1 Construction of Edge-Aware Priors and Edge Self-Attention Module

[0100] In the point cloud of steel structures, bolt holes are small-scale cylindrical structures with far fewer points than the background planar area, which can easily lead to class imbalance and misidentification during segmentation. To enable the model to focus on geometrically abrupt locations such as bolt hole boundaries, the point cloud normal vector field is analyzed to construct an edge-aware prior. This prior is calculated as follows: for any point in the point cloud... The set of neighboring points is obtained through k-nearest neighbor search, and the degree of difference between the normal vector of the given point and the normal vectors of its neighbors is calculated. The average of these differences is defined as the edge score. As shown in formula (1):

[0101] (1)

[0102] in For point The normal vector, Let the normal vector of the j-th point in the neighborhood be the edge score. As a priori for edge perception, locations with higher edge scores typically correspond to bolt hole boundaries or regions of geometric abrupt changes.

[0103] In addition to explicit priors, this invention also proposes an edge-aware self-attention (ESA) mechanism, which first generates a geometrically saliency feature map through two layers of one-dimensional convolution. And used to modulate deep features of the encoder. To enhance the response in regions of significant geometric change, as shown in Equation (2):

[0104] (2)

[0105] Based on this, the modulated features are input into the self-attention mechanism. The features are mapped to a query vector Q, a key vector K, and a value vector V through three one-dimensional convolutions, and the attention weight matrix is ​​calculated, as shown in equations (3, 4).

[0106] (3)

[0107] (4)

[0108] Then, the enhanced features are obtained by multiplying the attention weight matrix with the value vector, and finally, the enhanced features are obtained through residual connections. As shown in formula (5):

[0109] (5)

[0110] Compared to other mainstream edge feature modeling methods, the ESA mechanism utilizes the coupling of geometric features to achieve a physically-based representation learning process. This is achieved by modulating features from geometric data. The Q and K projections are derived, and the resulting attention weight matrix A no longer depends solely on the similarity of abstract features. Instead, it is intrinsically influenced by the surface geometric priors established during modulation. This ensures that during global context aggregation, the network prioritizes the interactions between structurally significant regions while effectively suppressing redundant information from numerous flat steel surfaces. Therefore, this two-stage refinement enables the ESA module to maintain high sensitivity to micro-geometric details in large-scale structural contexts, resolving the class imbalance problem and ensuring robust identification of bolt holes in subsequent quality inspection tasks.

[0111] Step 2.2 Construction of Semantic Segmentation Network for Bolt Holes in Point Cloud of Steel Components Based on Encoder-Decoder

[0112] After obtaining edge enhancement features, a point cloud semantic segmentation network based on an encoder-decoder structure is constructed to achieve bolt hole recognition. The encoder consists of multiple layers of EdgeConv modules. The input point cloud is first sampled from the farthest point to ensure spatial uniformity. Then, KNN search is used to establish local neighborhood relationships. The coordinate differences of the points in the neighborhood are concatenated with the normal vectors to form edge features. Local geometric relationships are extracted through EdgeConv, so that the features gradually transition from fine-grained geometric details to global structural representation, and finally, low-order features are obtained. Intermediate-order features and higher-order features Advanced features and normal vectors are input together into the ESA mechanism module to obtain edge enhancement features. Decoder , , Features are fused at multiple scales. Features at different spatial scales are extracted and integrated through parallel convolution. At the same time, the global feature extraction module is used to obtain the overall structural semantic information, which is then fused with local features to take into account both the local details of the bolt holes and the overall geometric relationship of the components. The fused features are upsampled layer by layer to restore the original number of points, and the class probability of each point is output through the segmentation head, thereby completing the semantic segmentation of the bolt holes and the background area.

[0113] Step 2.3 Instantiation and Refinement of Bolt Hole Semantic Class Point Cloud

[0114] After obtaining the semantic segmentation results, post-processing is performed on the points predicted to be bolt holes to transform the bolt hole region into independent instances with engineering significance. First, Euclidean clustering is performed based on a spatial distance threshold, aggregating adjacent points into clusters, each cluster corresponding to a candidate bolt hole. Then, a local consistency check is performed on each cluster. Outliers with insufficient neighborhoods are removed by counting the number of neighboring points within a specified radius, and anomalous morphological points are filtered based on the cylindrical geometric features, ensuring the remaining points maintain a continuous and stable spatial structure. The point clusters obtained after clustering and geometric constraint refinement can stably represent the actual bolt hole positions and shapes, thus providing reliable input data for subsequent centroid calculation, hole group matching, and accurate alignment during virtual trial assembly.

[0115] Step 3: Phased training and evaluation of the point cloud bolt hole segmentation model of steel components;

[0116] Step 3.1 Determining Training Strategies and Parameters

[0117] After constructing the semantic segmentation network, to ensure the model's generalization ability and recognition stability in complex engineering environments, this embodiment adopts a phased training and evaluation strategy for the bolt hole segmentation model of steel component point cloud. By fusing theoretical point cloud and measured point cloud data, and combining transfer learning methods, the model's adaptability to complex factors such as noise, occlusion, and incomplete point clouds in actual engineering scenarios is gradually improved. In a specific embodiment, the model training platform uses a computing server equipped with a GeForce RTX 4090 graphics card, with Ubuntu 22.04 as the operating system, and a Python implementation program based on the PyTorch deep learning framework is deployed on this platform. To obtain better model performance, this embodiment optimizes the selection of key hyperparameters using the grid search method. The specific values ​​of each hyperparameter and other parameter settings used during training are summarized in Table 3.

[0118] Table 3. Hyperparameter search values ​​and their values

[0119]

[0120] Step 3.2 Phased Training and Verification

[0121] Because the bolt hole region occupies a small proportion in the point cloud of the steel structure, there is a significant class imbalance between it and the background plane region. If the model is trained directly on real data, it is easy for the model to be biased towards the background class, thereby reducing the accuracy of bolt hole recognition. To alleviate this problem, this implementation adopts a phased training strategy based on transfer learning. That is, it is first pre-trained on the theoretical point cloud dataset, and then fine-tuned using measured point cloud data, thereby achieving multi-source data fusion.

[0122] Pre-training Phase. In this phase, the network is initially trained using a theoretical point cloud dataset generated through uniform sampling of a CAD mesh model. This type of theoretical point cloud data is characterized by its complete geometric structure, ideal bolt hole shapes, and accurate annotations. It is unaffected by factors such as scanning equipment noise or occlusion, thus helping the network learn the ideal cylindrical geometric features of bolt holes and the overall structural relationships of components during early training. For data partitioning, the theoretical point cloud dataset is randomly divided into training and test sets in an 8:2 ratio. The segmentation network is trained from scratch on the training set. After training, the model performance is initially evaluated using the test set, and the resulting network weights are used as the initial weights for the subsequent transfer learning phase.

[0123] The fine-tuning phase of transfer learning: After pre-training, the network model is transferred to the measured point cloud dataset for fine-tuning training to further improve the model's adaptability in real engineering environments. Since the measured point cloud data originates from the LiDAR scanning process at prefabricated component production and construction sites, it contains complex factors such as scanning noise, local occlusion, uneven point cloud density, and local missing data. Therefore, transfer learning is needed to adjust the model's parameters so that it can gradually adapt to the data distribution in real-world scenarios while retaining the geometric prior features learned in the theoretical data phase, thereby narrowing the domain gap between theoretical and actual data.

[0124] In this embodiment, to fully utilize the limited measured data and improve the reliability of model evaluation, a cross-validation strategy is adopted for training and validation during the fine-tuning stage. Specifically, the measured point cloud data constructed in step 1 is divided into 10 independent scene subsets according to the component scenes. In each round of training, 9 scenes are selected as the training set for fine-tuning the pre-trained model; the remaining scene is used as the validation set to evaluate the model's recognition performance on data not used in the training. The above process is repeated in a rotation manner, so that each scene can participate in the evaluation as validation data in different rounds, thereby forming a complete cross-validation result. The experimental results of this stage are shown in Table 4.

[0125] Table 4. Experimental results of semantic segmentation cross-validation for bolt hole point categories

[0126]

[0127] As shown in Table 4, the model achieved high segmentation performance in all test scenarios, with an average precision of 0.987, an average recall of 0.994, an average F1 score of 0.990, and an overall classification accuracy of 0.999. Regarding semantic segmentation evaluation metrics, the average mIoU (semantic) reached 0.981, and the average mIoU (instance) reached 0.986. These results demonstrate that the semantic segmentation model constructed in this embodiment can stably identify bolt hole regions in steel structure components and maintains high segmentation accuracy and good generalization ability in different component scenarios.

[0128] Furthermore, to verify the effectiveness of the method of this invention, comparative experiments were conducted between the network model proposed in this invention and several typical point cloud semantic segmentation methods under the same dataset and with the pre-trained weights obtained in the first training stage removed. The comparison methods included representative models such as PointNet, PointNeXt, PointNet++, Point Transformer v1, PointTransformer v3, and DGCNN. All comparison models were trained and validated on the same dataset and with a 10-fold scene-level cross-validation protocol, and the same training parameter settings were used to ensure the fairness of the experimental results. The comparison results of the mIoU index of each method in different scenes are shown in Table 5, and the corresponding segmentation inference results are shown in the table below. Figure 10 As shown.

[0129] Table 5. Comparison of mIoU results with other state-of-the-art methods

[0130]

[0131] As can be seen from Table 5, there are significant differences in performance stability and segmentation error patterns among different networks. Among them, PointNet showed a significant performance decline in multiple scenarios, with an average mIoU of 0.604, and was prone to false detections in the steel plate edge region; PointNeXt showed an overall performance improvement (mIoU of 0.806), but still exhibited undersegmentation of bolt hole regions in some scenarios; PointNet++ achieved good average performance (mIoU of 0.920), but its ability to represent small-scale targets in complex geometric situations was still somewhat limited; Point Transformer v1 improved its context modeling ability (mIoU of 0.954), but still showed slight misclassifications in some steel plate boundary regions; Point Transformer v3 further enhanced its feature representation ability (mIoU of 0.938), but still had some undersegmentation problems at irregular boundaries of bolt holes; DGCNN showed strong and stable segmentation performance (mIoU of 0.970), but still had some confusion in the boundary region between bolt holes and background points.

[0132] In comparison, the semantic segmentation framework proposed in this invention maintains stable performance across all scenarios, achieving an average mIoU of 0.976, the best result among all compared methods. Experimental results show that by introducing edge-aware priors and edge self-attention mechanisms, the proposed model can more accurately characterize the geometric changes in the bolt hole boundary region, while effectively suppressing interference from the background plane region, thus achieving stable and high-precision segmentation results in different scenarios. The above experimental results verify the effectiveness and reliability of the method in actual steel structure engineering scenarios, providing a high-precision data foundation for subsequent bolt hole position calculation, hole group matching, and virtual pre-assembly quality inspection.

[0133] Step 4, Pre-assembly and Error Detection; specifically including:

[0134] Step 4.1 Precise extraction of the centroid of the bolt hole point cloud

[0135] After obtaining the point cloud of bolt hole instances, high-precision positioning of the spatial center of each bolt hole is required to achieve accurate alignment during the virtual pre-assembly process. For the segmented individual bolt hole point cloud clusters, the farthest point sampling method is first used to select multiple pairs of points with the most uniform spatial distribution within the cluster, ensuring good spatial dispersion of the sampling points. Any pair of sampling points forms a line segment in three-dimensional space, and the perpendicular bisector of this line segment can be further constructed.

[0136] In the calculation, it is assumed that the main axis of the bolt hole is basically aligned with the z-axis. All perpendicular bisectors are orthogonally projected onto a plane perpendicular to the hole axis, i.e., the x-y plane. Let L be the projection line corresponding to the k-th pair of sampling points. k The optimal plane coordinates (x, y) are solved using the least squares estimation method. c , y c The goal is to minimize the sum of the squared distances from the point to all projected lines, thus determining the center position of the bolt hole in the normal plane. Subsequently, the maximum and minimum values ​​of the bolt hole point cloud in the z-direction are calculated, and their average value is used to determine the z-axis. c Finally, the three-dimensional centroid coordinates (x, y) of the bolt hole were obtained. c , y c , z c ).

[0137] Since the farthest point sampling can form a uniformly distributed pair of sampling points in the point cloud, and the least squares fitting has good mean noise suppression characteristics, the above method can effectively reduce the influence of local noise and outliers, and improve the stability and robustness of bolt hole center positioning.

[0138] Step 4.2 Positioning and matching of bolt hole groups

[0139] After obtaining the 3D centroids of each bolt hole, it is necessary to establish the hole group correspondence between the components to be assembled to achieve accurate alignment. First, the component point cloud is normalized to ensure the principal axis direction of the bolt holes aligns with the z-axis of the global coordinate system, resulting in a standard circular distribution of bolt holes in the x-y plane. Then, geometric analysis is performed on the center points of the bolt holes in the x-y plane, and Principal Component Analysis (PCA) is used to extract the principal axis directions of the hole group, including the horizontal and vertical principal axes. By establishing a local principal axis coordinate system, the center point of each bolt hole is projected into this coordinate system, and numbered according to row priority starting from the top left corner, thus establishing a unified hole group indexing rule. Through the above geometric normalization and projection numbering method, even if the overall component is tilted or rotated, a stable one-to-one correspondence can still be established between the bolt hole groups of two components. This method effectively reduces the impact of overall hole group skewness, rotation, or local deformation on matching accuracy, providing a reliable basis for subsequent virtual assembly and alignment.

[0140] Step 4.3 Component assembly standardization and quality assessment

[0141] After establishing the bolt hole matching relationship, the corresponding bolt hole center point set is used to rigidly register the components to be assembled. Specifically, the optimal rigid transformation relationship between the two components is solved using the Iterative Closest Point (ICP) algorithm or the Procrustes analysis method. The optimal rotation matrix R and translation vector t are calculated based on the spatial coordinate relationship between the corresponding hole centers, and a corresponding 4×4 homogeneous transformation matrix is ​​constructed to rigidly transform one of the components into the target component coordinate system. After the components are aligned, the Euclidean distance between the centers of each pair of matching bolt holes is calculated, and the distances of all matching hole pairs are statistically analyzed to obtain the component assembly error indicators, including the average error, maximum error, and standard deviation of the error, which are used to quantitatively evaluate the assembly accuracy.

[0142] In multi-component continuous assembly scenarios, once two components are registered, their combined form participates in the registration process of subsequent components as a rigid whole, repeatedly executing the aforementioned transformation solution and error statistics steps. In closed-loop assembly structures, error indices are updated through error accumulation and overall constraints, eliminating the need for repeated pairwise registration calculations. This virtual pre-assembly and error assessment process allows for the prediction of component assembly accuracy before actual construction, thereby identifying potential assembly deviations and geometric mismatches in advance, reducing on-site rework risks, and improving assembly efficiency.

[0143] To verify the effectiveness of the above-described virtual pre-assembly method, this embodiment was experimentally verified in a typical three-component assembly scenario. Point cloud data of three prefabricated steel structural components were selected to simulate the assembly process of structural nodes. Components 1 and 3 used theoretical point cloud data, while component 2 (connecting node plate) used point cloud data obtained from actual LiDAR scanning. During the virtual assembly process, hole group matching and rigid registration were first performed on components 1 and 2 to obtain their assembled combined structure; subsequently, this combined structure was further assembled with component 3. Through the above two-stage assembly process, the construction process of gradually assembling multiple components in a real engineering project can be simulated. The assembly results are shown in Table 6.

[0144] Table 6. Assembly parameters and errors

[0145]

[0146] Experimental results show that during the assembly of component 1 and component 2, the average registration error is approximately 0.157 mm, and the maximum error is approximately 0.337 mm; during the assembly of component 2 and component 3, the average error is approximately 0.149 mm, and the maximum error is approximately 0.335 mm. Furthermore, the high-precision characteristics of the virtual assembly process are further verified by solving the rigid transformation parameters (including translation and rotation angle around the z-axis). The above experimental results demonstrate that the virtual pre-assembly method proposed in this embodiment can achieve sub-millimeter-level registration accuracy in multi-component assembly scenarios, meeting the stringent precision control requirements of industrial-grade prefabricated steel structure assembly. This method can accurately predict component assembly errors before actual construction, providing important technical support for digital construction management, automated quality inspection, and assembly process optimization.

[0147] II. Application Examples. To further illustrate the inventiveness of the technical solution of this invention and its technical value in engineering practice, this example, combined with a specific engineering application scenario, describes the systematic implementation of the steel structure bolt hole identification and virtual pre-assembly quality inspection method based on multi-source point clouds proposed in this invention. This part realizes the application of the technical solution described in the claims in the actual steel structure assembly quality inspection process by constructing a corresponding software system and computer equipment.

[0148] like Figure 12 As shown, the present invention provides a steel structure point cloud quality inspection system for implementing the steel structure assembly quality inspection and virtual pre-assembly analysis method.

[0149] The data acquisition and preprocessing module is used to construct a multi-source point cloud dataset for steel structures. This module acquires point cloud data from CAD theoretical models during the design phase and measured point cloud data obtained through laser scanning equipment during the construction phase. It then performs preprocessing operations such as voxel downsampling, normal vector calculation, and data annotation on the point cloud data to generate standardized point cloud data for model training and inference.

[0150] The model building module, connected to the data acquisition and preprocessing module, is used to construct a point cloud semantic segmentation network based on an edge-aware self-attention mechanism. This module achieves automatic identification and semantic segmentation of bolt hole regions in steel structure point clouds by constructing an encoder-decoder network structure that includes an EdgeConv feature extraction structure and an edge-aware self-attention (ESA) mechanism.

[0151] The training and evaluation module, connected to the model building module, is used to perform phased model training and performance evaluation. This module evaluates the model's recognition performance by pre-training on a theoretical point cloud dataset and fine-tuning it using transfer learning on a measured point cloud dataset, while also incorporating a scene-level cross-validation strategy, and generates bolt hole recognition results and corresponding segmentation accuracy metrics.

[0152] The virtual pre-assembly and error analysis module, connected to the training and evaluation module, is used to realize the virtual pre-assembly and assembly error detection of steel structure components. This module first extracts the centroids of the segmented bolt hole point cloud and establishes the hole position correspondence between components through hole group geometric normalization and number matching. Then, it uses a rigid registration algorithm to calculate the optimal spatial transformation relationship between components, realizing the virtual assembly process, and performs statistical analysis on the assembly error to obtain assembly accuracy evaluation indicators.

[0153] In practical applications, the above modules work together to realize a complete process from steel structure point cloud data acquisition and automatic bolt hole identification to virtual pre-assembly error analysis, providing automated technical means for the assembly quality inspection of prefabricated steel structure components.

[0154] This embodiment selects two typical segmental assembly scenarios from the same steel bridge project as experimental objects. Both scenarios are derived from actual engineering structures, but the experimental dataset is constructed by combining theoretical point clouds and measured point clouds to simulate the impact of component manufacturing errors on the assembly process in real engineering projects.

[0155] Scenario 1: Non-closed segmental structure. This scenario comprises three steel components connected sequentially by bolt holes, without forming a closed loop in the overall structure. In this scenario, component 2 is constructed using measured point cloud data acquired through on-site laser scanning, while the remaining components are generated from theoretical point clouds derived from the design model. This scenario primarily serves to verify whether the proposed stepwise rigid body assembly method can accurately reconstruct the assembly relationships between components under simple chain-like structure conditions, and to assess assembly errors.

[0156] Scenario 2: Closed Segmental Structure. This scenario comprises six steel components connected in a ring, forming a closed structure upon final assembly. In this scenario, component 4 is based on measured point cloud data, while the point clouds of the remaining components are generated from theoretical models. Since a certain degree of closure error inevitably occurs during the virtual assembly process, this scenario primarily serves to evaluate the error accumulation of the proposed method under closed structure conditions and to analyze the assembly accuracy of the final closed node.

[0157] Example 1: Inspection and assembly of non-closed segmental components

[0158] A handheld laser scanner was used to acquire measured point clouds of small steel structure node plates with a resolution of 0.5 mm. Target components were extracted from a 3D CAD model, and their triangular mesh models were uniformly sampled at 1.0 mm intervals to generate theoretical point clouds. Downsampling was performed on both types of point clouds to unify the spatial resolution to 1.0 mm, and outliers were removed using statistical filtering. The local normal vector of each point was calculated, and neighboring points were obtained through k-nearest neighbor search (k = 6). The average difference between the normal vectors was calculated to obtain the edge score. An edge self-attention mechanism semantic segmentation model was constructed. The encoder used a 3-layer EdgeConv module, and the decoder fused low, medium, and high-order features. In transfer learning, the model was first pre-trained on the theoretical point cloud for 100 rounds, and then fine-tuned on the measured point cloud for 250 rounds. The focusing parameter in the edge-aware focus loss function was set to 2. After segmentation, Euclidean clusters of bolt holes were obtained through clustering (distance threshold 2.0 mm). After centroid extraction, hole group matching was completed through attitude adjustment and PCA principal direction extraction. ICP algorithm was used for rigid registration, and two bolt hole offset errors were detected, with error values ​​of 0.337 mm and 0.335 mm, respectively, which effectively avoided misalignment during on-site assembly.

[0159] Example 2: Assembly of multi-module closed segments

[0160] For multi-module closed-loop steel structure assembly, measured and theoretical point clouds of each module were acquired sequentially. After preprocessing, a multi-source dataset was constructed, and local normal vectors and edge scores (k = 6) were calculated. When extracting the centroid of bolt holes, 1000 point pairs were selected, and least squares estimation was used to improve positioning accuracy. Hole group matching employed a combination of principal component analysis and coordinate projection to avoid matching deviations caused by module tilt. During closed-loop assembly, registered modules participated in subsequent registration as a whole, with only error indices updated. Ultimately, a cumulative error of 1.038 mm was detected at the closed-loop splicing point, allowing for timely adjustment of component positions and preventing rework.

[0161] (1) The proposed Virtual Pre-Assembly (VTA) method was validated in the three-component assembly experiment of Scenario 1. This experiment used point cloud data of three steel components to virtually assemble the structural nodes, such as... Figure 13 and Figure 14 As shown. Due to the limited availability of measured data, components 1 and 3 are represented by theoretical point clouds generated from the design model, while component 2 (node ​​connection plate) is represented by measured point clouds obtained from LiDAR scanning. During assembly, the bolt hole groups between components 1 and 2 are first matched, and the optimal spatial transformation between the two components is calculated through rigid body registration, thus completing the first step of assembly. Subsequently, the combined structure formed by components 1 and 2 is treated as a whole and then assembled with component 3 in the second step. Figure 15 (a) Figure 15 (b) demonstrates the assembly process of component 1 and component 2. Figure 15 (c) Figure 15 (d) illustrates the assembly process of component 2 and component 3. The corresponding assembly parameters and error statistics are shown in Table 4-xx. The results show that during the assembly of component 1 and component 2, the average alignment error of the bolt hole centers is 0.157 mm, and the maximum error is 0.337 mm; during the assembly of component 2 and component 3, the average error is 0.149 mm, and the maximum error is 0.335 mm. Simultaneously, the calculated rigid body transformation parameters (including translations in the X and Y directions and rotation angles along the Z axis) further reflect the spatial adjustment between components and verify the stability of the virtual assembly process. Overall, this experiment demonstrates that the virtual pre-assembly method proposed in this paper can achieve sub-millimeter accuracy in multi-component node assembly scenarios, meeting the connection accuracy requirements of prefabricated steel structure engineering, and providing reliable technical support for digital construction, automated quality inspection, and assembly process optimization.

[0162] (2) such as Figure 16 As shown, the proposed Virtual Pre-Assembly (VTA) method was validated in the assembly experiment of a six-component closed-loop structure in Scenario 2. Experimental results show that in this closed-segment assembly scenario, the proposed virtual pre-assembly frame can achieve stable and reliable assembly results, with an average segment assembly error of 0.710 mm and a maximum of 1.038 mm. In the specific assembly process, component 1 and component 2 are first aligned and assembled, followed by the sequential assembly of components 3, 4, 6, and 5 according to a preset procedure. In each assembly step, rigid body registration is achieved only by minimizing the local bolt hole matching error of the current node, without introducing additional global structural constraints, thus maintaining the local optimality of the assembly process. The final assembly accuracy is evaluated by the matching error of the bolt hole group between component 4 and component 5, and the assembly is terminated at this node to ensure that the entire structure remains statically determinate during the calculation process. The relevant assembly parameters and error statistics are shown in Table 6, and the final error distribution is as follows: Figure 17As shown in the figure. Further analysis of the error changes in each assembly step reveals that in the early steps a and b, only theoretical point cloud components are involved, and their average and maximum errors are close to zero. However, micron-level errors appear in steps d and e. This is mainly because the bolt hole axes in these steps are distributed at an angle, while the bolt holes in steps a and b are basically distributed vertically. Therefore, the difference in geometric orientation will have a certain impact on the registration accuracy. This result indicates that the theoretical point cloud data itself has a negligible impact on the assembly accuracy, and the observed errors mainly come from changes in the structural geometric direction rather than data quality issues. In steps c and f, measured component point clouds from LiDAR scanning are introduced, thus becoming the main source of error in the entire assembly process. Since the measured point clouds inevitably contain measurement noise, surface irregularities, and potential manufacturing deviations, the average and maximum errors increase in these steps. Overall, the experimental results show that theoretical point cloud data has a small impact on assembly accuracy, while the geometric errors of the measured components are the main factor leading to the accumulation of assembly errors. This further verifies the effectiveness of the virtual pre-assembly method in identifying and quantifying actual component manufacturing errors.

[0163] Table 7. Detailed parameters and specifications (mm) for segmented assembly

[0164]

[0165] It should be noted that the embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic circuits; the software portion can be stored in memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated computing device. Those skilled in the art will understand that the above systems and methods can be implemented using computer-executable instructions or processor control code and can be stored in a disk, optical disk, read-only memory (ROM), programmable memory, or other data storage media. The systems and modules of the present invention can be implemented using hardware circuits such as large-scale integrated circuits, logic chips, and transistors, or programmable hardware devices such as field-programmable gate arrays (FPGAs) and programmable logic devices, or software programs executed by a processor, or a combination of hardware and software.

[0166] The above description is merely a specific embodiment of the present invention, used to illustrate the technical solution of the present invention, and is not intended to limit the scope of protection of the present invention. Those skilled in the art can make various modifications, substitutions, or improvements to the above embodiments without departing from the spirit and principles of the present invention, and all such modifications, equivalent substitutions, or improvements should be included within the scope of protection of the present invention.

Claims

1. A technology and method for inspecting the assembly quality of steel structures based on laser point clouds, characterized in that, include: S1: Obtain theoretical point clouds and measured point clouds, construct a multi-source point cloud dataset and perform preprocessing, calculate local normal vectors and geometric edge information of the point clouds; S2: Construct a semantic segmentation model for bolt holes in steel structure point cloud based on edge self-attention mechanism. The edge self-attention mechanism constructs edge-aware priors by analyzing the point cloud normal vector field, generates geometric saliency feature maps to modulate deep features of the encoder, and then obtains edge enhancement features through the self-attention mechanism. S3: The semantic segmentation model is trained using a transfer learning strategy that combines pre-training and fine-tuning. An edge-aware focus loss function is introduced to solve the class imbalance problem. The model performance is verified through a semantic segmentation and instance segmentation evaluation mechanism. S4: Extract the centroid of bolt hole point cloud based on semantic segmentation results, complete the bolt hole group positioning and matching, realize virtual pre-assembly of components and detect assembly errors through rigid registration.

2. The method as described in claim 1, characterized in that, In S1, the measured point cloud is acquired by laser scanning equipment, and the theoretical point cloud is extracted from the target component from the steel structure design drawings or 3D CAD model. The triangular mesh model is generated by uniform surface sampling. The preprocessing includes downsampling and data cleaning of the theoretical point cloud and the measured point cloud, unifying the spatial resolution and removing outliers. After preprocessing, the theoretical point cloud and the measured point cloud are divided into pre-training data and fine-tuning data.

3. The method as described in claim 1, characterized in that, In S2, the calculation method of edge-aware prior is as follows: for any point in the point cloud, the set of neighboring points is obtained through k-nearest neighbor search, the difference between the normal vector of the point and the normal vector of the neighboring points is calculated, and the average value is taken as the edge score. The edge score is the edge-aware prior; the position with a larger edge score corresponds to the bolt hole boundary or the geometric change region.

4. The method as described in claim 1, characterized in that, In S2, the semantic segmentation model is an encoder-decoder structure. The encoder consists of multiple layers of EdgeConv modules. The input point cloud is sampled from the farthest point to ensure a uniform spatial distribution. KNN search is used to establish local neighborhood relationships. The coordinate differences of the points in the neighborhood are concatenated with the normal vectors to form edge features. Low-order, mid-order, and high-order features are obtained by extracting local geometric relationships through EdgeConv. The decoder performs multi-scale fusion of the above features, combines global structural semantic information with local features, and outputs the class probability of each point through layer-by-layer upsampling.

5. The method as described in claim 1, characterized in that, In S2, the bolt hole feature refinement and instantiation process includes: performing Euclidean clustering on points predicted as bolt holes based on spatial distance thresholds to obtain candidate bolt hole point clusters; performing local consistency checks on each point cluster, removing outliers with insufficient neighborhoods, and filtering abnormal morphological points based on cylindrical geometric features to obtain point clusters that stably represent the position and shape of bolt holes.

6. The method as described in claim 1, characterized in that, In S4, the method for extracting the centroid of the bolt hole point cloud is as follows: Select multiple pairs of spatially dispersed points within the bolt hole point cloud cluster, each pair of points forms a line segment and constructs a perpendicular bisector, and orthogonally project all perpendicular bisectors onto a plane perpendicular to the hole axis; solve for the optimal plane coordinates by least squares estimation to minimize the sum of the squares of the distances from the point to all projected lines, and determine the center position of the bolt hole in the plane; count the maximum and minimum values ​​of the hole point cloud in the axial direction and take the average value to obtain the three-dimensional centroid coordinates of the bolt hole.

7. The method as described in claim 1, characterized in that, In S4, the positioning and matching of bolt hole groups includes: adjusting the posture of the components to make the main axis of the bolt holes consistent with the preset axis direction; performing group analysis on the bolt hole centers in a plane perpendicular to the axis, and extracting the main direction axis of the hole group using principal component analysis; projecting each hole center onto the local principal axis coordinate system, numbering them according to the row priority order starting from the upper left corner of the projected coordinates, and establishing a one-to-one correspondence between the two component hole groups.

8. A steel structure assembly quality inspection system based on laser point clouds, implementing the method as described in any one of claims 1 to 7, characterized in that, include: The data acquisition and preprocessing module is used to acquire theoretical point clouds and measured point clouds, preprocess the theoretical point clouds and measured point clouds, and calculate the local normal vectors and geometric edge information of the point clouds. The model building module, connected to the data acquisition and preprocessing module, is used to build a semantic segmentation model of bolt holes in steel structure point cloud based on edge self-attention mechanism; The training and evaluation module, connected to the model building module, is used to train the semantic segmentation model using a transfer learning strategy that combines pre-training and fine-tuning, introduces an edge-aware focus loss function, and verifies the model performance through a semantic segmentation and instance segmentation evaluation mechanism. The virtual pre-assembly and error analysis module, connected to the training and evaluation module, is used to extract the centroid of the bolt hole point cloud, complete the positioning and matching of the bolt hole group, and realize the virtual pre-assembly of the component and detect the assembly error through rigid registration.

9. A computer device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the steps of the steel structure assembly quality inspection technology and method based on laser point clouds as described in any one of claims 1 to 7.

10. The computer device as claimed in claim 9, characterized in that, When the computer program is executed by the processor, it is also used to implement the function calls of the data acquisition and preprocessing module, the model building module, the training and evaluation module, and the virtual pre-assembly and error analysis module, so as to complete the entire process of steel structure assembly quality inspection.

Images