Virtual Assembly Error Detection Method Based on Transformer

Abstract

This invention provides a virtual assembly error detection method based on Transformer. The method includes: acquiring source point clouds of each component in a structural member using a scanner; segmenting the BIM model of the structural member and converting it into target point clouds of each component; denoising the source point clouds and then initializing and registering them based on a context-guided module; enhancing the registered source point clouds and then generating features of the source and target point clouds using Transformer based on a self-attention mechanism; determining a similarity matrix of the features of the source and target point clouds; filtering the source point clouds according to the similarity matrix and re-registering the corresponding target point clouds; voxelizing the source and target point clouds; and determining the assembly error of the structural member based on the distance between corresponding voxels in the source and target point clouds. This invention achieves accurate registration of source and target point clouds and high-precision quantification of manufacturing deviations.

Description

Technical Field

[0001] This invention relates to the field of assembly error detection technology, and in particular to a virtual assembly error detection method based on Transformer. Background Technology

[0002] Precast structures are widely used in the construction and infrastructure sectors due to their advantages in improving construction quality, shortening construction periods, saving resources, and addressing labor shortages. They not only represent an important direction for the future development of the industry but also align with the current trends of green building and intelligent construction. With the transformation and upgrading of the construction industry, precast structures play a crucial role. Precast components are manufactured in factories, first using Computer-Aided Design (CAD) and Building Information Modeling (BIM) to complete detailed designs, and then using molds for industrial production. However, manufacturing deviations are an inherent risk in the production of precast components; serious deviations can lead to structural safety hazards, construction delays, and even catastrophic accidents. Therefore, detecting deviations in precast components is crucial for the successful implementation of prefabricated buildings and is a fundamental step in subsequent assembly processes.

[0003] Before prefabricated components are transported to the construction site, potential problems during on-site assembly must be addressed to ensure a smooth assembly process. However, physical pre-assembly has significant limitations, including high labor costs, large space requirements, and the risk of component damage during disassembly. To overcome these limitations, Virtual Trial Assembly (VTA) has been introduced as an innovative solution. VTA utilizes digital technology to simulate component assembly in a computer-generated virtual environment, helping to identify problems early and develop optimal solutions before actual construction activities. Notably, VTA can significantly reduce time and costs while maintaining accuracy.

[0004] Currently, various VTA systems have been applied to the assembly inspection of building structures. Many scholars in academia have also conducted extensive research on VTA methods. Geometric deviation detection of prefabricated components suitable for VTA involves analyzing manufacturing deviations using point cloud data (PCD) and BIM, with the following steps: First, a digital model of each component is established, and point cloud data of the actual component is acquired using 3D laser scanning technology; second, a registration algorithm is used to align the scanned point cloud with the BIM model; finally, after registration, the geometric deviation between the actual component and the BIM model is quantified. In this process, the registration algorithm is a key element, and its accuracy directly affects the accuracy of deviation detection. However, registration algorithms used for VTA mainly rely on manually defined rules, typically following predetermined algorithmic steps. For example, the Iterative Closest Point (ICP) algorithm relies on specific geometric data representations, such as point cloud normals, planes, and edges, as well as manually defined linear optimization and iterative update processes to align the point cloud. This method is highly mechanical and may get stuck in local optima or fail to converge completely. Furthermore, it cannot adjust its registration strategy according to scene complexity or data characteristics. Summary of the Invention

[0005] This invention provides a virtual assembly error detection method based on Transformer to solve the defects of poor assembly error detection accuracy and low adaptability caused by mechanized registration in the prior art, thereby improving the accuracy and adaptability of assembly error detection.

[0006] This invention provides a virtual assembly error detection method based on Transformer, comprising:

[0007] The source point cloud of each component in the component is obtained by using a scanner, and the BIM model of the component is segmented and converted into the target point cloud of each component.

[0008] After denoising the source point clouds of each component, the source point clouds and target point clouds of each component are initialized and registered based on the context guidance module to obtain the registered source point clouds.

[0009] After enhancing the registered source point cloud, the Transformer is used to generate features of the source and target point clouds based on the self-attention mechanism.

[0010] Determine the similarity matrix between the features of the source point cloud and the target point cloud, filter the source point cloud according to the similarity matrix, and re-register the target point cloud corresponding to the filtered source point cloud.

[0011] The source point cloud and target point cloud after re-registration are voxelized, and the assembly error of the component is determined based on the distance between corresponding voxels in the source point cloud and target point cloud.

[0012] According to the present invention, a virtual assembly error detection method based on Transformer is provided, which denoises the source point cloud of each component, including:

[0013] After denoising the source point clouds of each component using a kd-tree, a bilateral filter is used to denoise the source point clouds again.

[0014] According to a Transformer-based virtual assembly error detection method provided by the present invention, the source point cloud is further denoised using a bilateral filter with the following formula:

[0015]

[0016] Where, p i It is the geometric position of the i-th point in the source point cloud, n i It is the normal vector of the i-th point, α i ω is the bilateral filter factor at the i-th point. i It is the curvature of the i-th point. It is the updated geometric position of the i-th point. It is a smoothing weight. It is the feature domain weight, p j It is the distance p i The nearest k1 neighboring points, σ c It is p i The influence factor of the distance to each neighboring point on the point, σ s Is each adjacent point at p i The influence factor of the projected distance on the normal vector at a point on that point.

[0017] According to the present invention, a virtual assembly error detection method based on Transformer is provided, which initializes and registers the source point cloud and target point cloud of each component based on the context guidance module using the following formula to obtain the registered source point cloud, including:

[0018] v = h φ (C(max(f θ (X)),max(f θ (Y))))

[0019] Where X is the source point cloud of each component, Y is the target point cloud of each component, and f θ This is an encoder module shared by the source and target point clouds of all components. `max` represents max pooling, `C` represents the connection operation, and `h`... φIt is a transform decoder. The first four values ​​in v represent quaternion rotation vectors used to calculate the initial transformation matrix R0, and the last three values ​​represent translation vector t0.

[0020] According to the present invention, a virtual assembly error detection method based on Transformer enhances the registered source point cloud using the following formula:

[0021]

[0022] Where x is the geometric coordinate of any point in the registered source point cloud, and m i Let x be the geometric coordinates of the i-th point among the k nearest neighbors. For m i Enhanced features, PPF i For the point-to-point features of the i-th point, For the enhanced features of x, μ θ It contains multiple convolutional layers, with max being the max pooling layer.

[0023] According to a Transformer-based virtual assembly error detection method provided by the present invention, the source point cloud is filtered based on the similarity matrix using the following formula:

[0024]

[0025] in, For the registered source point cloud X' in X' s1 Features of points, features of the registered source point cloud X' Features of the corresponding target point cloud Similarity matrix between M is the number of points in the source point cloud. By deleting non-overlapping points in X', the following condition is met. We obtain, where top-prob represents the index of the highest similarity score obtained in a certain probability proportion of prob, X' s2 From the filtered source point cloud, a similarity matrix of representative overlapping points is obtained. For X' s2 Its characteristics.

[0026] According to the present invention, a virtual assembly error detection method based on Transformer is provided, which determines the target point cloud corresponding to the filtered source point cloud for re-registration using the following formula:

[0027] J = argmax top-k H s2,i

[0028]

[0029] Where J represents the pair of X' in Y. s2 The i-th point x s2,i The index with the top k maximum similarity scores, y i Let Y be the updated geometric coordinates of the target point corresponding to the i-th point, and ω be the target point cloud. ij Let the i-th point and y i The weights between them.

[0030] According to the present invention, a virtual assembly error detection method based on Transformer is provided. Before acquiring the source point cloud of each component in a component using a scanner and converting the BIM model of the component into the target point cloud of each component, the registration model is trained using the following loss function:

[0031]

[0032] Among them, L f To re-register the loss, L init To initialize the registration loss, L ol The overlap score loss is defined as follows: X represents the source point cloud sample; R and R0 are the rotation matrices for re-registration and initial registration, respectively; and t and t0 are the translation vectors for re-registration and initial registration, respectively. and These are the configuration rotation matrix and translation vector for the source point cloud sample annotations, S. X and S Y Let N and M be the overlap scores of the i-th and j-th points in the source point cloud sample and the target point cloud sample, respectively. and Corresponding to S X and S Y The overlapping score of the annotations.

[0033] According to the present invention, a virtual assembly error detection method based on Transformer is provided, S X and S Y The calculation formula is:

[0034]

[0035] Where r represents the operation of expanding the dimension and repeating the feature. For an overlapped decoder, C represents the connection operation, and F represents the overlapped decoder. X and F Y The features are obtained after the source point cloud samples and the corresponding target point cloud samples have been processed by a shared encoder module. and respectively through the corresponding F X and F Y Obtained after pooling.

[0036] The present invention provides a Transformer-based virtual assembly error detection method, which proposes a denoising algorithm to address the noise problem in large-scale scanned point cloud data; develops a deep learning-driven point cloud registration network that achieves high-precision registration while ensuring spatial consistency of point cloud data; and develops a geometric deviation quantification algorithm using the aligned data, which effectively quantifies the deviation value between the actual component and its corresponding theoretical model. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0038] Figure 1 This is a flowchart illustrating the virtual assembly error detection method based on Transformer provided by the present invention.

[0039] Figure 2 This is a schematic diagram of the framework of the Transformer-based virtual assembly error detection method provided by the present invention;

[0040] Figure 3 This is a schematic diagram of the framework of the context guidance module in the Transformer-based virtual assembly error detection method provided by the present invention;

[0041] Figure 4 This is a schematic diagram of the Transformer-based feature matching module structure in the Transformer-based virtual assembly error detection method provided by the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0043] The following is combined Figure 1The present invention describes a Transformer-based virtual assembly error detection method, comprising:

[0044] Step 101: Use a scanner to acquire the source point cloud of each component in the component, and then divide the BIM model of the component into the target point cloud of each component.

[0045] A stand-up or handheld laser scanner can be used to perform a full scan of the component and obtain the source point cloud of the component.

[0046] To ensure the smooth progress of subsequent registration experiments, Revit software can be used to finely process the BIM model, systematically dividing it into multiple independent sub-components. This subdivision process improves the accuracy and efficiency of registration. Subsequently, the segmented BIM model is converted into corresponding point cloud data to ensure consistency in data type between the model data and the measured point cloud.

[0047] Step 102: After denoising the source point cloud of each component, the source point cloud and target point cloud of each component are initialized and registered based on the context guidance module to obtain the registered source point cloud.

[0048] The correspondences generated by repetitive patterns in the environment produce a large number of outliers (e.g., points on the ground, ceiling, or planar walls of a building), which hinder the quality of pose estimation. Therefore, if these points can be detected and filtered out, the registration method can be made more robust. Point cloud denoising not only prevents subsequent steps from converging to poor solutions, but also helps to enhance the saliency of features by discarding redundant and non-salient points in advance.

[0049] The goal of registration is to align two unordered point clouds, namely, source point cloud P and target point cloud Q. To achieve registration, a correspondence needs to be established between the two point clouds, and then robust estimation is used to suppress the adverse effects of outliers. Formally, it is assumed that the Kth pair of point clouds (or the Kth correspondence) obtained through matching is composed of 3D points m. i ∈P and 3D point n j ∈Q. Then, the Kth correspondence can be modeled as:

[0050] n j =Rm i +t+σ i,j

[0051] Where R∈SO(3) is the rotation matrix, It is a translation vector. This refers to measurement noise. If the initial correspondence is represented as τ, then the objective function can be expressed as:

[0052]

[0053] Here, ρ represents a robust loss function designed to suppress errors caused by erroneous correspondences or outliers, and ε represents outliers that have been filtered out before optimization. Therefore, this formula attempts to estimate the relative pose between the source and target point clouds while being robust to outliers.

[0054] Step 103: After enhancing the registered source point cloud, use Transformer to generate features of the source point cloud and target point cloud based on the self-attention mechanism.

[0055] Step 104: Determine the similarity matrix between the features of the source point cloud and the target point cloud, filter the source point cloud according to the similarity matrix, and re-register the target point cloud corresponding to the filtered source point cloud.

[0056] Step 105: Voxelize the re-registered source point cloud and target point cloud, and determine the assembly error of the component based on the distance between corresponding voxels in the source point cloud and target point cloud.

[0057] This study aims to answer three main research questions: (1) how to quickly denoise large-scale point cloud data without affecting accuracy; (2) how to develop a robust learning-based registration algorithm; and (3) how to quantify geometric deviations based on aligned data. To address these questions, this study performed the following: (i) developed a denoising method suitable for large-scale point clouds; (ii) extracted features from the denoised point cloud using machine learning algorithms and trained a point cloud registration neural network; and (iii) proposed a voxel-based high-precision deviation calculation method. The innovation of this study lies in designing a Transformer-enhanced point cloud registration method that effectively combines the global features of the point cloud and proposing a voxel-based geometric deviation quantification method to ensure high-precision quantification of manufacturing deviations.

[0058] This embodiment proposes a method for detecting manufacturing deviations of components through point cloud registration, used to quantify the geometric dimensional deviations between actual components and theoretical BIM models. The method mainly includes four key steps: (1) point cloud acquisition and BIM model segmentation of steel components; (2) noise removal from large-scale scanning point cloud data; (3) a deep learning-based point cloud registration network; and (4) a deviation detection algorithm utilizing aligned data. The overall framework of this method is as follows: Figure 2 As shown.

[0059] Through experiments with actual data, the following key results were obtained: (1) The proposed denoising algorithm effectively reduced noise and smoothed the point cloud while preserving key geometric features; (2) Compared with GeoTransformer, the registration algorithm showed higher accuracy, reducing rotation error (Error(R)) by 14.25% and translation error (Error(t)) by 18.68%. Notably, each pair registration operation consumed only 0.109 seconds per frame; (3) The geometric deviation detection algorithm achieved millimeter-level accuracy, with an average geometric deviation of 2.73 mm observed in the dataset. The innovation of this study lies in proposing a comprehensive point cloud registration network that covers an integrated process from point cloud scanning, BIM segmentation, point cloud denoising, registration to deviation quantization, demonstrating high accuracy, computational efficiency, and versatility in field measurements.

[0060] This embodiment proposes a denoising algorithm to address the noise problem in large-scale scanned point cloud data; it develops a deep learning-driven point cloud registration network that achieves high-precision registration while ensuring spatial consistency of point cloud data; and it develops a geometric deviation quantization algorithm using the aligned data, which effectively quantifies the deviation between the actual component and its corresponding theoretical model.

[0061] Based on the above embodiments, this embodiment denoises the source point clouds of each component, including:

[0062] After denoising the source point clouds of each component using a kd-tree, a bilateral filter is used to denoise the source point clouds again.

[0063] Due to the large number and irregular distribution of point clouds, this embodiment proposes an outlier noise point filtering algorithm that combines kd-trees and bilateral filters. The method first uses a kd-tree algorithm for coarse denoising to filter out isolated outliers in the dataset, and then improves the bilateral filter to smooth the point cloud.

[0064] The original 3D point cloud data obtained from scanning is scattered and disordered. A point cloud topology is established using the kd-tree algorithm. The average distance between any point in the point cloud and all its neighbors follows a Gaussian distribution with a mean of μ and a standard deviation of δ. If the average distance between a point and all its neighbors exceeds a set threshold μ + λδ, the point is considered noise and removed from the point cloud. Before searching for neighboring points using the kd-tree algorithm, initial values ​​for k and the standard deviation factor λ need to be set. The selection of the initial value of k directly affects the noise removal effect of the point cloud model. To determine the values ​​of k and λ, this embodiment added a set of random noise to the Bunny point cloud and conducted experiments.

[0065] While the kd-tree algorithm can remove most outliers, small-scale noise points still exist in point clouds, mixed with the main point cloud data, making them difficult to remove using common methods based on Euclidean distance and point cloud density. This embodiment uses the sampling point curvature ω... i As a parameter, the bilateral filtering factor is improved to filter out small-scale noise points and smooth the entire point cloud.

[0066] Based on the above embodiments, this embodiment uses a bilateral filter to further denoise the source point cloud using the following formula:

[0067]

[0068] Where, p i It is the geometric position of the i-th point in the source point cloud, n i It is the normal vector of the i-th point, α i ω is the bilateral filter factor at the i-th point. i It is the curvature of the i-th point. It is the updated geometric position of the i-th point. and All are Gaussian kernel functions. It is p i Smooth weights (spatial weights) for distances to neighboring points. It is the feature domain weight (influence weight), p j It is the distance p i The nearest k1 neighboring points, σ c It is p i The influence factor of the distance to each neighboring point on the point is generally taken as the radius of the neighborhood, σ. s Is each adjacent point at p i The influence factor of the projected distance on the normal vector at a point on that point is generally taken as the standard deviation of the nearest neighbor points. When σ c Once determined, the smooth distance of the point cloud in the normal direction and σ s They are directly proportional, and <> represents the inner product operation.

[0069] Surface normals are an important property of a geometric surface. By fitting the local surface of the point cloud, the normal vector can be estimated quite well. Principal component analysis can be used to estimate the normal vector of the sampling point p. i Perform a k-nearest neighbor search to obtain k neighborhood points N(p) i To approximate point p i The local neighborhood surface S, i.e., p i ∈S, the covariance matrix C of the neighborhood points is:

[0070]

[0071] The three eigenvalues ​​of the covariance matrix are denoted as λ0, λ1, and λ2. The eigenvector corresponding to the smallest eigenvalue is used as the local surface of the point cloud on p. i The normal vector n at point i The curvature at any sampling point in a point cloud can be characterized by the curvature of the local surface fitted to that point and its neighborhood points. Principal component analysis is used to estimate p. i curvature ω at i It can be represented as:

[0072]

[0073] Bilateral filtering factors rely on local neighborhood features. When local regions of the point cloud model are sharp, the normal directions of sampling points are difficult to distinguish, thus failing to accurately locate noise. To optimize the sampling point neighborhood and improve the accuracy of noise point search, this embodiment adjusts the sampling point curvature ω. i As a parameter, the bilateral filter factor α is improved. i .

[0074] The complete steps of denoising include: (1) initializing the k value, constructing the kd tree, and calculating the point cloud data points p. i (1) Find the k nearest neighbors of the point cloud; (2) Estimate the normal vector n of the point cloud using PCA. i and curvature value ω i (3) Calculate p for each sampling point i (3) Smoothing weight parameter x and feature domain weight parameter y; (4) Calculate smoothing weight and feature domain weights (5) Calculate the bilateral filter factor α i (6) Calculate point p i The geometric position is determined to obtain the denoised point cloud model.

[0075] Based on the above embodiments, this embodiment uses the following formula to initialize and register the source point cloud and target point cloud of each component using the context guidance module, and obtains the registered source point cloud, including:

[0076]

[0077] Where X is the source point cloud of each component, Y is the target point cloud of each component, and f θ An encoder module shared by all components for the source and target point clouds; this module learns high-dimensional features corresponding to X and Y. max represents max pooling, C represents the join operation, and h φ The transform decoder is a simple multilayer perceptron, where the first four values ​​in v represent quaternion rotation vectors. Used to calculate the initial transformation matrix R0, the last three values ​​represent the translation vector. The transformed source point cloud X' can be represented by X' = X·R T +t T Calculations show that the summation of t0 is performed row by row.

[0078] Coarse initial registration is beneficial for point feature learning. Furthermore, overlapping points between point cloud pairs can significantly improve matching accuracy, which is crucial for registration. Therefore, a context-guided module is proposed to initialize registration and predict the scores of overlapping points. Its overall structure is as follows: Figure 3 As shown.

[0079] The modified PointNet++ network is used as the encoder of the context-guided module. The encoder module is used to extract global features and regress a 7-dimensional vector representing rotation and translation transformations.

[0080] Based on the above embodiments, this embodiment enhances the registered source point cloud using the following formula, including:

[0081]

[0082] Where x is the geometric coordinate of any point in the registered source point cloud, and m i Let x be the geometric coordinates of the i-th point among the k nearest neighbors. For m i Enhanced features, PPF i For the point-to-point features of the i-th point, For the enhanced features of x, μ θ It contains several 1-D convolutional layers, with max being a max pooling layer. The point-to-point feature (PPF) describes the relative position and pose of two directed points.

[0083] After coarse registration, fine registration is performed using the point correspondence between the transformed source point cloud X' and the target point cloud Y. In this stage, a Transformer-based feature matching module is proposed to remove non-representative points from the source point cloud X', and its structure is as follows: Figure 4 As shown.

[0084] Prior to the Transformer, point cloud feature enhancement was performed based on a method proposed in RPMNet. For a point x∈X', k nearest neighbors were obtained using a predefined radius. For m i ∈M k The enhanced features are denoted as It connects spatial coordinates and point-to-point features.

[0085] Point features need to consider global information of the point cloud. The Transformer integrates point features based on the global point cloud input, showing an advantage over local features. Given the transformed source point cloud X', the enhanced features are... Point features can be generated based on self-attention, as shown in the following formula:

[0086]

[0087] Where · represents matrix multiplication, and i is the coefficient of self-attention. and These are the learning weights, and C0 is a hyperparameter used to control the computational cost. `softmax` represents row normalization, and `norm` represents column normalization. The output features for X' and Y can be represented as... and

[0088] Based on the above embodiments, this embodiment filters the source point cloud according to the similarity matrix using the following formula:

[0089]

[0090] in, For the registered source point cloud X' in X' s1 Features of points, features of the registered source point cloud X' Features of the corresponding target point cloud Similarity matrix between M is the number of points in the source point cloud, X' s1 By deleting non-overlapping points in X', the following condition is met. We obtain, where top-prob represents the index of the highest similarity score obtained in a certain probability proportion of prob, X' s2 From the filtered source point cloud, a similarity matrix of representative overlapping points is obtained. For X' s2 The feature of this method transforms partial-to-part registration into partial-to-complete registration.

[0091] In the feature matching and removal stage, the similarity matrix is ​​first calculated. However, a point in X' may not have a valid corresponding point in Y. This correspondence matrix satisfies... We can delete the non-overlapping points in X', thus ensuring that the remaining points X' are... s1 satisfy Therefore, it can solve the problem of registration between parts.

[0092] In this way, representative overlap points can be obtained in two steps. First, based on the overlap score S predicted by the context-guided module... X Select the first N points as the overlap points. Secondly, feature matching is performed to further eliminate points that lack descriptive features, resulting in the final representative point X'. s2 .

[0093] Based on the above embodiments, this embodiment uses the following formula to determine the target point cloud corresponding to the filtered source point cloud for re-registration:

[0094] J = argmax top-k H s2,i

[0095]

[0096] Where J represents the pair of X' in Y. s2 The i-th point x s2,i The index with the top k maximum similarity scores, y i Let Y be the updated geometric coordinates of the target point corresponding to the i-th point, and ω be the target point cloud. ij Let the i-th point and y i The weights between them.

[0097] For each point x s2,i ∈X' s2 The corresponding point y is calculated. i To generate the corresponding set, each pair of points is defined as (x... s2,i ,y i ), its weight is Then, weighted singular value decomposition is used to estimate the exact transformation R1∈SO(3) and

[0098] Based on the above embodiments, in this embodiment, before using a scanner to obtain the source point cloud of each component in the component, and before segmenting the BIM model of the component and converting it into the target point cloud of each component, the registration model is trained using the following loss function:

[0099]

[0100] Among them, L f To re-register the loss, L init To initialize the registration loss, L ol The overlap score loss is defined as follows: X represents the source point cloud sample; R and R0 are the rotation matrices for re-registration and initial registration, respectively; and t and t0 are the translation vectors for re-registration and initial registration, respectively. and S represents the configuration rotation matrix and translation vector for the source point cloud sample annotations (ground truth). X and S Y Let N and M be the overlap scores of the i-th and j-th points in the source point cloud sample and the target point cloud sample, respectively. and Corresponding to S X and S Y The overlap score of the ground truth annotations.

[0101] Using rotation error (R) and translation error (t) as evaluation metrics for registration, their calculation formulas can be expressed as follows:

[0102]

[0103] Where R and t represent the rotation matrix and translation vector, and tr represents the matrix trajectory. Furthermore, the mean absolute error of the Euler angles and translation vector is also used as an evaluation metric, calculated as follows:

[0104]

[0105] Where N represents the number of samples.

[0106] Based on the above embodiments, S in this embodiment X and S Y The calculation formula is:

[0107]

[0108] Where r represents the operation of expanding the dimension and repeating the feature. For overlapping decoders, The parameters are shared in the input point cloud, C is for connection operation, F X and F Y The features are obtained after the source point cloud samples and the corresponding target point cloud samples have been processed by a shared encoder module. and respectively through the corresponding F X and F Y Obtained after pooling.

[0109] The shared information between the source and target point clouds is crucial for predicting overlapping points. Therefore, a simple information exchange module is designed to predict overlapping points based on global features, which can be obtained from the initialization step, thus saving computation. Here, the overlapping point prediction problem is formulated as a binary classification problem.

[0110] After registration, the geometric deviation between point cloud pairs cannot be directly calculated. Although the two sets of point clouds are in the same coordinate system, the number of points may not be the same. Furthermore, since the deep neural network used for point cloud registration is a black box model, the indices between points are also unknown. These two reasons make it impractical to calculate the distance between each pair of points using brute force. Voxel-based methods are widely used in point cloud processing. By dividing the space into a regular three-dimensional voxel grid, the problem of uneven point cloud density can be solved, thus unifying the scale of the point cloud.

[0111] To quantify the geometric discrepancies between registered point clouds, a local region definition method based on voxel meshes is adopted. First, the entire point cloud needs to be voxelized, that is, the 3D space is divided into cubic voxel units of uniform size. The size of the voxels is usually set according to the characteristics of the point cloud and the resolution requirements. The basic steps of voxelization can be simply described as follows: given a point cloud dataset P = {p1, p2, ... p...} n Define the side lengths Δx, Δy, Δz of the voxel, and plot each point p in the point cloud. i =(x i ,y i ,z i Mapping its spatial coordinates to the corresponding voxel V k All points belonging to the same voxel are considered as a point set P within that voxel. k ={p j |p j ∈V k For each voxel V k Point set P within k A series of statistical measures are used to describe the geometric properties and distribution of the voxel, and its centroid or mean coordinates are:

[0112]

[0113] Where k is voxel V k The number of midpoints. The standard deviation can be expressed as:

[0114]

[0115] In the source point cloud and the target point cloud, for two corresponding voxels V k and V l The distance difference can be expressed as:

[0116]

[0117] Because the density of points within a voxel varies, each voxel needs to be assigned a certain weight ω. kIt can effectively balance the contribution of different voxels to geometric deviation. The weight values ​​are calculated using the normalized standard deviation (note that the weight calculation is only for the target point cloud voxels), and can be expressed as:

[0118]

[0119] Based on the above analysis, the global geometric deviation of the target point cloud can be obtained by averaging the weighted geometric errors of all voxels. The calculation formula is as follows:

[0120]

[0121] Where, N g It represents the total number of global voxels.

[0122] The above algorithm is used to calculate the global error of the target point cloud, which is used to evaluate the overall manufacturing quality of the component. Local geometric deviations can reveal the manufacturing precision of the component within a specific area. If the local deviation in a certain area is large, it indicates that there is a defect in that area of ​​the component. Local errors can be quantified by analyzing the geometric distribution, centroid, and differences from neighboring points within the local area. The distance values ​​between the current voxel and its surrounding neighboring voxels and the corresponding voxel in the target point cloud are statistically analyzed to define its local region. If the current voxel V... k With corresponding voxel V l The distance value is greater than the global error E global Then, neighboring voxels are clustered using the Adaptive Density-based Spatial Clustering of Applications with Noise (DBSCAN) algorithm. The clustered voxels can be considered as local regions L(V). k The average geometric deviation of a local area can be calculated using the following formula:

[0123]

[0124] Where, N l It represents the number of voxels within a local region.

[0125] To verify the performance of the registration network, we trained it on the collected dataset. All experiments were conducted on a computer equipped with an RTX 4070Ti Super and a 64GB Intel-i9 processor, running Ubuntu 20.04. The algorithms and network models were developed using Python 3.8.0 and PyTorch 1.18.0. The initial learning rate was set to 0.0001, the number of training epochs to 600, the batch size to 8, and the learning rate was varied using a cosine annealing schedule. Non-iterative adaptive learning was performed using the Adam optimizer, and CUDA was employed to accelerate the training process. The nearest neighbor radius used to calculate features in PointNet++ was set to 0.3. For the context-guided module, the number of convolutional filters in the encoding process was [64, 64, 64, 128, 512], and the number of filters in the transform and overlapped decoders were [512, 512, 256, 7] and [512, 512, 256, 2], respectively.

[0126] In this study, to explore in depth the detection method of geometric deviations in virtual assembly, we conducted a systematic experimental verification of the algorithm's effectiveness. Based on the above dataset, we conducted a detailed evaluation of the entire process, including point cloud denoising preprocessing, point cloud BIM registration, and virtual assembly geometric deviation analysis. By removing noise from large-scale scanned point cloud data, we effectively reduced random noise in the data, ensuring a high-quality foundation for subsequent processing. The effectiveness of noise removal provides accurate input data for the point cloud registration step, providing more robust support for the deep learning-driven point cloud registration network. Our proposed bilateral filtering point cloud denoising method, combined with a deep learning network for point cloud registration, can effectively handle complex geometric shapes and high-noise environments. Furthermore, the geometric error detection algorithm based on the registration results can accurately identify geometric deviations in the virtual assembly process with millimeter-level accuracy. To comprehensively evaluate the performance of the proposed method, this study conducted experimental analysis from multiple dimensions, including data processing time, registration accuracy, error accuracy, and its feasibility in practical applications. The comparison between the registration algorithm and state-of-the-art (SOTA) methods further verifies the advantages of the proposed method in several key indicators. The following conclusion analysis section will delve into these experimental results:

[0127] (1) The point cloud bilateral filtering denoising algorithm proposed in this paper can effectively denoise and smooth 3D point cloud data while preserving the geometric features of the model. In this experiment, to test the effect of the point cloud denoising algorithm, two component point clouds were selected, and noise with a variance of 0 and a standard deviation of 0.02m was added to the point cloud models respectively. We compared the proposed algorithm with traditional filtering denoising algorithms (MRPCA, MLS) and deep learning-based point cloud denoising algorithms (PCN, Score). The experimental results are shown in Table 1. The results show that the proposed denoising algorithm significantly reduces the influence of random noise while preserving effective point cloud information. Compared with traditional and learning-based algorithms, P2S is reduced by 13.27% and 10.97% respectively, and CD index is reduced by 20% and 18.28% respectively. By comparing the denoising results of different algorithms, it can be seen that our proposed algorithm effectively smooths the smoothness of the point cloud surface while achieving denoising, reducing surface fluctuations caused by noise. This characteristic can significantly improve the visual quality of the point cloud. The final experiment compared the geometric feature parameters (such as edge sharpness and surface continuity) before and after denoising. The results showed that the proposed algorithm effectively preserved the main geometric features of the model during the denoising process without causing significant loss of key structural details. This advantage makes the algorithm more valuable for application in virtual assembly and precise geometric analysis.

[0128] Table 1

[0129]

[0130] A review of existing literature reveals that most deviation detection methods in prefabricated component manufacturing rely on pre-developed software or lack in-depth research into the overall deviation detection process. These methods typically utilize only partial feature information of the component during registration, such as corner points and holes, while ignoring the overall information of the component. This oversight leads to decreased registration accuracy. To address these issues, this paper proposes a comprehensive and efficient component deviation detection method. First, a fast large-scale point cloud denoising technique is introduced to reduce noise interference. Second, a registration network incorporating global point cloud information is trained to ensure accurate registration between the point cloud data and the BIM model. Finally, a deviation quantification algorithm based on alignment data is developed, which can effectively quantify the manufacturing deviations of the component.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A virtual assembly error detection method based on Transformer, characterized in that, include: The source point cloud of each component in the component is obtained by using a scanner, and the BIM model of the component is segmented and converted into the target point cloud of each component. After denoising the source point clouds of each component, the source point clouds and target point clouds of each component are initialized and registered based on the context guidance module to obtain the registered source point clouds. After enhancing the registered source point cloud, the Transformer is used to generate features of the source and target point clouds based on the self-attention mechanism. Determine the similarity matrix between the features of the source point cloud and the target point cloud, filter the source point cloud according to the similarity matrix, and re-register the target point cloud corresponding to the filtered source point cloud. The source point cloud and target point cloud after re-registration are voxelized, and the assembly error of the component is determined based on the distance between corresponding voxels in the source point cloud and target point cloud.

2. The virtual assembly error detection method based on Transformer according to claim 1, characterized in that, Denoising is performed on the source point clouds of each component, including: After denoising the source point clouds of each component using a kd-tree, a bilateral filter is used to denoise the source point clouds again.

3. The virtual assembly error detection method based on Transformer according to claim 2, characterized in that, The source point cloud is further denoised using a bilateral filter according to the following formula: Where, p i It is the geometric position of the i-th point in the source point cloud, n i It is the normal vector of the i-th point, α i ω is the bilateral filter factor at the i-th point. i It is the curvature of the i-th point. It is the updated geometric position of the i-th point. It is a smoothing weight. It is the feature domain weight, p j It is the distance p i The nearest k1 neighboring points, σ c It is p i The influence factor of the distance to each neighboring point on the point, σ s Is each adjacent point at p i The influence factor of the projected distance on the normal vector at a point on that point.

4. The virtual assembly error detection method based on Transformer according to claim 1, characterized in that, The source and target point clouds of each component are initialized and registered using the context-guided module based on the following formula, resulting in the registered source point cloud, including: v=h φ (C(max(f θ (X)),max(f θ (Y)))) Where X is the source point cloud of each component, Y is the target point cloud of each component, and f θ This is an encoder module shared by the source and target point clouds of all components. `max` represents max pooling, `C` represents the connection operation, and `h`... φ It is a transform decoder. The first four values ​​in v represent quaternion rotation vectors used to calculate the initial transformation matrix R0, and the last three values ​​represent translation vector t0.

5. The virtual assembly error detection method based on Transformer according to claim 1, characterized in that, The registered source point cloud is enhanced using the following formulas, including: Where x is the geometric coordinate of any point in the registered source point cloud, and m i Let x be the geometric coordinates of the i-th point among the k nearest neighbors. For m i Enhanced features, PPF i For the point pair features of the i-th point, For the enhanced features of x, μ θ It contains multiple convolutional layers, with max being the max pooling layer.

6. The virtual assembly error detection method based on Transformer according to claim 1, characterized in that, The source point cloud is filtered based on the similarity matrix using the following formula: in, For the registered source point cloud X' in X' s1 Features of points, features of the registered source point cloud X' Features of the corresponding target point cloud Similarity matrix between M is the number of points in the source point cloud, X' s1 By deleting non-overlapping points in X', the following condition is met. We obtain, where top-prob represents the index of the highest similarity score obtained in a certain probability proportion of prob, X' s2 From the filtered source point cloud, a similarity matrix of representative overlapping points is obtained. For X' s2 Its characteristics.

7. The virtual assembly error detection method based on Transformer according to claim 6, characterized in that, The target point cloud corresponding to the filtered source point cloud is determined using the following formula for re-registration: J=argmax top-k H s2,i Where J represents the pair of X' in Y. s2 The i-th point x s2,i The index with the top k maximum similarity scores, y i Let Y be the updated geometric coordinates of the target point corresponding to the i-th point, and ω be the target point cloud. ij Let the i-th point and y i The weights between them.

8. The virtual assembly error detection method based on Transformer according to any one of claims 1-7, characterized in that, Before using a scanner to acquire the source point clouds of each component in the structure, and before segmenting the BIM model of the structure and converting it into the target point clouds of each component, the registration model is trained using the following loss function: Among them, L f To re-register the loss, L init To initialize the registration loss, L ol The overlap score loss is defined as follows: X represents the source point cloud sample; R and R0 are the rotation matrices for re-registration and initial registration, respectively; and t and t0 are the translation vectors for re-registration and initial registration, respectively. and These are the configuration rotation matrix and translation vector for the source point cloud sample annotations, S. X and S Y Let N and M be the overlap scores of the i-th and j-th points in the source point cloud sample and the target point cloud sample, respectively. and Corresponding to S X and S Y The overlapping score of the annotations.

9. The virtual assembly error detection method based on Transformer according to claim 8, characterized in that, S X and S Y The calculation formula is: Where r represents the operation of expanding the dimension and repeating the feature. For an overlapped decoder, C represents the connection operation, and F represents the overlapped decoder. X and F Y The features are obtained after the source point cloud samples and the corresponding target point cloud samples have been processed by a shared encoder module. and respectively through the corresponding F X and F Y Obtained after pooling.

Images