Steel-concrete frame structure VTA method based on foundation LiDAR point cloud

Abstract

The invention relates to the technical field of virtual assembly of assembly type frame structures, and provides a foundation LiDAR point cloud-based steel-concrete frame structure VTA method, which comprises the following steps: carrying out data acquisition on a target assembly component through a foundation three-dimensional laser radar, and processing the data to make a data set; inputting the data set into the SSBR-Net network model, and outputting semantic information and a 3D bounding box; performing assembly component feature extraction on the 3D bounding box, and calculating to obtain an assembly point location according to the 3D bounding box; registering the assembly component characteristics and the assembly point positions with corresponding data in the assembly component initial standard design model to obtain the assembly component characteristics of the target assembly component and the deviation between the assembly point positions and the assembly component initial standard design model; and analyzing the deviation, and obtaining an optimal virtual assembly scheme according to an analysis result. According to the invention, the redundancy of data is greatly reduced, and the efficiency of deviation analysis and VTA scheme analysis and the VTA precision are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of virtual assembly of prefabricated frame structures, and in particular to a VTA method of a steel-concrete frame structure based on a ground-based LiDAR point cloud. Background Art

[0002] Virtual pre-assembly technology (VTA) is increasingly being used in the assembly process of prefabricated structures as an efficient and low-cost alternative to physical pre-assembly. In traditional VTA methods, assembly components are usually scanned in a targeted manner and the required component point clouds are manually segmented. This method is inefficient in point cloud acquisition and recognition segmentation. Therefore, traditional VTA methods are affected by manual operations when processing large-scale point cloud data, and are limited in both accuracy and efficiency.

[0003] In addition, the traditional solution also has the following technical defects:

[0004] 1. Insufficient automation: Traditional VTA mostly relies on manual segmentation and recognition, which is not only time-consuming, but also increases the complexity of operation and the possibility of errors. The lack of effective automated tools to process and analyze point cloud data leads to a low level of automation in the entire assembly process.

[0005] 2. Accuracy issues: Manual processing of point cloud data is easily affected by the operator's skills and experience, which may lead to inconsistency and accuracy issues in data processing. In addition, the accuracy and reliability of traditional methods may be further reduced when processing complex or obscured components.

[0006] 3. Scalability and flexibility issues: When it comes to large-scale or complex structures, the scalability of traditional VTA is limited. In large projects, manually processing a large amount of point cloud data will significantly increase time costs and resource consumption, and it is difficult to flexibly respond to changes in project scale or demand. Summary of the invention

[0007] The object of the present invention is to solve at least one technical problem in the background technology and to provide a VTA method for steel-concrete frame structure based on ground-based LiDAR point cloud.

[0008] To achieve the above object, the present invention provides a VTA method for a steel-concrete frame structure based on ground-based LiDAR point cloud, comprising:

[0009] The target assembly components are collected through ground-based 3D laser radar, and the data are processed to produce a data set;

[0010] Input the dataset into the SSBR-Net network model and output semantic information and 3D bounding box;

[0011] Extract assembly component features from the 3D bounding box, and then calculate the assembly points based on the 3D bounding box;

[0012] The assembly component features and assembly points are registered with corresponding data in the initial standard design model of the assembly component, and the deviation between the assembly component features and assembly points of the target assembly component and the initial standard design model of the assembly component is obtained;

[0013] The deviation is analyzed, and the best virtual assembly solution is obtained according to the analysis result.

[0014] According to one aspect of the present invention, the data collection of the target assembly component by using a ground-based three-dimensional laser radar includes:

[0015] The ground-based 3D laser radar is used to collect the point cloud of the target assembly components placed on the ground and the overall point cloud of the construction scene.

[0016] According to one aspect of the present invention, the SSBR-Net network model is composed of a RandLaNet semantic segmentation network and a BRNet network with a SAM spatial attention mechanism added.

[0017] According to one aspect of the present invention, the step of extracting assembly component features from a 3D bounding box and calculating assembly points based on the 3D bounding box includes:

[0018] Embedding meshes are performed on each plane of the 3D bounding box of the target assembly component to determine candidate points for plane fitting on each surface of the point cloud of the assembly component;

[0019] Use candidate points to fit the precise projection planes of each surface to determine the precise 3D bounding box, and divide the point clouds of each surface according to the 3D bounding box to perform dimensionality reduction of the bottom surface point cloud;

[0020] The intersection point between the steel bar axis and the top plane of the assembly component is calculated as the top assembly point, and the 3D bounding box boundary is used as the boundary feature of the assembly component;

[0021] The maximum angle method is used to extract the bottom sleeve boundary on the bottom point cloud after dimensionality reduction, and the RANSAC method is used to fit the center of the sleeve boundary circle. The result is used as the assembly point of the bottom surface of the assembly component.

[0022] According to one aspect of the present invention, the RANSAC method is used to fit the center of the sleeve boundary circle, and the result is used as the assembly point of the bottom surface of the assembly component, including:

[0023] First, three points are taken at even intervals on each sleeve boundary to calculate the circle center, and all evenly taken point fitting results are calculated sequentially to obtain the evenly taken point circle center point set Se;

[0024] Use the RANSAC algorithm to calculate the center fitting point of the sleeve boundary point cloud;

[0025] The circle center point set Se is used as the reference point set to screen the RANSAC calculation point set to reduce the dispersion of the circle center fitting points obtained by the RANSAC method;

[0026] The final mean value of the circle center fitting point is used as the assembly point position of the bottom surface of the assembly component.

[0027] According to one aspect of the present invention, the step of registering the assembly component features and assembly points with corresponding data in the initial standard design model of the assembly component to obtain the deviation between the assembly component features and assembly points of the target assembly component and the initial standard design model of the assembly component includes:

[0028] The relationship between the assembly component feature and assembly point reference point X0 (x0, y0, z0) and the corresponding point X (x, y, z) in the point cloud generated by the BIM model is as follows:

[0029] X0―(―λRX i +ΔX)=0;

[0030] Where R is the rotation matrix; X i For each point iterated in the entire point cloud, ΔX is the translation vector and λ is the scale factor with a value of 1;

[0031] When dealing with large-angle rotation in space, the Rodriguez matrix is ​​used to solve the problem. First, according to the rotation vector s=[s x s y s z ] T Construct the antisymmetric matrix S:

[0032]

[0033] The antisymmetric matrix S can be converted to the Rodrigues matrix R using the Rodrigues formula:

[0034] R=I+(sinθ)S+(1―cosθ)S 2 ;

[0035] Where I is the 3rd-order identity matrix and θ is the rotation angle;

[0036] The error equation is constructed through the above parameters, and the correction value of the observation point is solved. The error equation is as follows:

[0037] V1=A1t+BX―L1;

[0038] Among them, V1 is the correction value vector, A1 is the angle coefficient matrix, t is the transformation parameter correction value vector, B is the coefficient matrix of the point to be solved, X is the correction value of the point to be solved, and L1 is the observation value residual vector; the parameters are constructed as follows:

[0039] V1=[V x V y V z ] T ;

[0040]

[0041]

[0042] t=[dΔx dΔy dΔz da db dc] T ;

[0043] B=R ―1 ;

[0044] X=[dx0 dy0 dz0] T ;

[0045]

[0046] According to one aspect of the present invention, the deviation is analyzed, and an optimal virtual assembly solution is obtained according to the analysis result, including:

[0047] Analyze assembly points and component size deviations;

[0048] Analyze the root node position and axial deviation of steel bars;

[0049] Check the assembly point position and component size deviation and the steel bar root node position and axial deviation for assembly feasibility. When assembly feasibility is confirmed, determine the assembly deviation.

[0050] Based on the assembly deviation, the combination with the smallest RMSE of the overall assembly deviation of the whole group of assembly components is selected as the optimal virtual assembly solution.

[0051] To achieve the above object, the present invention also provides a steel-concrete frame structure VTA system based on ground-based LiDAR point cloud, comprising:

[0052] The data acquisition module collects data of the target assembly components through the ground-based three-dimensional laser radar, and processes the data to produce a data set;

[0053] The semantic information and 3D bounding box acquisition module inputs the dataset into the SSBR-Net network model and outputs semantic information and 3D bounding boxes;

[0054] The feature and point acquisition module extracts the assembly component features from the 3D bounding box, and then calculates the assembly points based on the 3D bounding box;

[0055] The deviation calculation module aligns the assembly component features and assembly points with the corresponding data in the initial standard design model of the assembly component, and obtains the deviation between the assembly component features and assembly points of the target assembly component and the initial standard design model of the assembly component;

[0056] The VTA acquisition module analyzes the deviation and acquires the best virtual assembly solution according to the analysis result.

[0057] To achieve the above objectives, the present invention also provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the VTA method for steel-concrete frame structure based on ground-based LiDAR point cloud as described above.

[0058] To achieve the above objectives, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the VTA method for steel-concrete frame structure based on ground-based LiDAR point cloud is implemented as described above.

[0059] According to the solution of the present invention, the present invention first proposes a deep learning method for extracting assembly components in large scenes based on semantic segmentation and voting mechanism, which realizes the accurate and efficient extraction of required component point clouds in large and complex construction scene point clouds, and performs preliminary 3D bounding box estimation.

[0060] Based on the semantic segmentation results of SSBR-Net, the 3D bounding box and direction estimation of the component, a high-precision assembly position positioning method based on the 3D bounding box is proposed. First, the precise 3D bounding box of the component is calculated based on the preliminary estimated 3D bounding box and direction, and a point cloud dimensionality reduction method based on the 3D bounding box is designed. Compared with the dimensionality reduction method based on PCA, this method can provide a more accurate projection plane, and ensure that the density of the point cloud after dimensionality reduction is uniform and the feature points are retained intact, which is more conducive to the accurate calculation of assembly features. Based on the 3D bounding box, various assembly features of the component are calculated, and the component point cloud is abstracted as an assembly feature model representation. This greatly reduces the redundancy of the data, expresses the assembly components in a more intuitive way, and improves the efficiency of subsequent deviation analysis and VTA solution analysis.

[0061] According to the assembly characteristics, various deviation analyses are performed on the components, and the best virtual assembly scheme is obtained based on the analysis results to ensure that the cumulative error of the best virtual assembly scheme is small and the accuracy is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 A flowchart schematically showing a VTA method for a steel-concrete frame structure based on ground-based LiDAR point cloud according to an embodiment of the present invention;

[0063] Figure 2 Schematically showing the precise 3D bounding box calculation principle and component point cloud segmentation result diagram according to an embodiment of the present invention;

[0064] Figure 3 A schematic diagram showing a principle diagram of point cloud dimensionality reduction based on a 3D bounding box according to an embodiment of the present invention;

[0065] Figure 4 A flowchart of assembly point extraction according to an embodiment of the present invention is schematically shown;

[0066] Figure 5 Schematically showing an assembly component deviation analysis diagram according to an embodiment of the present invention;

[0067] Figure 6 Schematically showing the result diagram of semantic segmentation and 3D bounding box estimation according to Embodiment 1 of the present invention;

[0068] Figure 7 Schematically showing the component assembly feature extraction result diagram according to Example 1 of the present invention;

[0069] Figure 8 A diagram schematically showing component deviation analysis results according to Example 1 of the present invention;

[0070] Fig. 9 Schematically showing the CJ1 assembly deviation analysis result diagram according to Example 1 of the present invention;

[0071] Fig.10 Schematic representation of the CJ1 and CJ2 VTA results according to Example 1 of the present invention. DETAILED DESCRIPTION

[0072] The present invention will now be discussed with reference to exemplary embodiments. It should be understood that the embodiments discussed are only to enable those skilled in the art to better understand and thus implement the present invention, rather than implying any limitation on the scope of the present invention.

[0073] As used herein, the term “including” and variations thereof are to be interpreted as open-ended terms meaning “including, but not limited to.” The term “based on” is to be interpreted as “based, at least in part, on.” The terms “one embodiment” and “an embodiment” are to be interpreted as “at least one embodiment.”

[0074] Figure 1The flowchart of the VTA method of steel-concrete frame structure based on ground-based LiDAR point cloud according to one embodiment of the present invention is schematically shown. Figure 1 As shown, in this embodiment, the VTA method of steel-concrete frame structure based on ground-based LiDAR point cloud includes:

[0075] The target assembly components are collected through ground-based three-dimensional laser radar (LiDAR), and the data are processed to produce a data set;

[0076] Input the dataset into the SSBR-Net network model and output semantic information and 3D bounding box;

[0077] Extract assembly component features from the 3D bounding box, and then calculate the assembly points based on the 3D bounding box;

[0078] The assembly component features and assembly points are registered with corresponding data in the initial standard design model of the assembly component, and the deviation between the assembly component features and assembly points of the target assembly component and the initial standard design model of the assembly component is obtained;

[0079] The deviation is analyzed, and the best virtual assembly solution is obtained according to the analysis result.

[0080] Further, according to an embodiment of the present invention, data collection of a target assembly component is performed by using a ground-based three-dimensional laser radar, including:

[0081] The ground-based 3D laser radar is used to collect the point cloud of the target assembly components placed on the ground and the overall point cloud of the construction scene.

[0082] In this embodiment, point cloud data collection requires simultaneous acquisition of point clouds of two different scales: a single target assembly component and the entire construction scene. High-precision assembly component point clouds are used for accurate VTA and deviation analysis, and low-precision construction scene point clouds are used to provide the scene and global coordinates required for VTA. High-precision ground-based three-dimensional laser radar with adjustable acquisition distance and resolution can meet data collection requirements. Assembly components use close-range high-resolution parameters, and construction scenes use long-range lower-resolution parameters, which can meet targeted and rapid point cloud data collection.

[0083] Furthermore, in this embodiment, the initial standard design model (BIM model) of the assembly component is converted into point cloud data, and the conversion process usually adopts a sampling or resampling method. Considering that the subsequent point cloud registration needs to be registered with the assembly component boundary as a feature, in order to ensure the feature accuracy, a curvature-based sampling method is used to obtain the design model point cloud. Curvature-based sampling determines the density of sampling points according to the curvature of the design model surface. The sampling points are denser in areas with larger curvatures, which can maximize the retention of geometric features used for registration. In the processing software, the design model can be resampled and downsampled based on curvature to generate a design model point cloud.

[0084] Further, according to one embodiment of the present invention, the SSBR-Net network model is composed of a RandLaNet semantic segmentation network and a BRNet network with a SAM spatial attention mechanism added.

[0085] In this embodiment, the original method for extracting features in the BRNet network is Pointnet, which is replaced by the RandLaNet method in the present invention. The advantage is that it uses random sampling dimensionality reduction to speed up the speed, and performs LFA module feature aggregation in RandLaNet to ensure the integrity of local features. Through multi-layer MLP dimensionality increase, point features with label properties are finally output, and the sampling point features are enhanced through the SAM spatial attention mechanism and input into the BRNet network. The next step is to follow the process of the BRNet network, vote and sample the point features, generate clusters of voting points and points, backtrack the representative points, generate representative point sets, and finally perform pose estimation and classification. The output result outputs the bounding box through the 3D NMS non-maximum suppression method.

[0086] In this embodiment, the SSBR-Net network model first receives the original three-dimensional point cloud data, and extracts semantic information relative to the entire scene from the complex construction scene through the RandLaNet semantic segmentation network. Based on the obtained semantic information, the model performs voting point generation and clustering operations through the spatial attention module SAM (Spatial Attention module). SAM can enhance the model's perception of spatial details in the input data, thereby improving the overall prediction accuracy and efficiency. When performing voting point generation and clustering operations, the spatial attention module can ensure that the voting points more accurately reflect key structural information. This not only improves the accuracy of representative point and 3D bounding box generation in subsequent steps, but also improves the efficiency and reliability of the overall model.

[0087] In this embodiment, the feature information of the voting point is output, and the sampling voting learning is performed on the semantic information obtained in the area. Semantic seeds are independently converted into voting points. The model uses local features extracted from global semantic data to sample within a specified range. The local point set obtained from the sampling is passed through a multi-layer perceptron (MLP) and a pooling layer to obtain local global features. This step ensures the accurate acquisition of the geometric structure of the feature points. Cluster center points are generated through clustering operations. New representative points are generated around the voting centers, and the accuracy of the seed points is reviewed near them. The backtracking strategy of this step ensures the mutual consistency between the predicted voting point centers and the surface points of the original target point cloud. Finally, by reviewing and refining the voting results again, the generation, refinement and classification of object proposals are performed to generate the final 3D bounding box.

[0088] Further, according to an embodiment of the present invention, the 3D bounding box is subjected to assembly component feature extraction, and the assembly point is obtained by calculation according to the 3D bounding box, including:

[0089] Embedding meshes are performed on each plane of the 3D bounding box of the target assembly component to determine candidate points for plane fitting on each surface of the point cloud of the assembly component;

[0090] Use candidate points to fit the precise projection planes of each surface to determine the precise 3D bounding box, and divide the point clouds of each surface according to the 3D bounding box to perform dimensionality reduction of the bottom surface point cloud;

[0091] The intersection point between the steel bar axis and the top plane of the assembly component is calculated as the top assembly point, and the 3D bounding box boundary is used as the boundary feature of the assembly component;

[0092] The maximum angle method is used to extract the boundary of the bottom sleeve (assembly position) on the bottom point cloud after dimensionality reduction, and the RANSAC method is used to fit the center of the sleeve boundary circle, and the result is used as the assembly point position of the bottom surface of the assembly component.

[0093] In this embodiment, the point cloud dimensionality reduction method based on the 3D bounding box includes:

[0094] 1. Accurate 3D bounding box calculation

[0095] According to the estimation results output by the SSBR-Net network model, a 3D bounding box (3D bounding box) that is roughly consistent with the horizontal angle of the concrete column can be obtained. However, for assembly components that are not placed horizontally, their 3D bounding boxes cannot fit well with the component plane. Therefore, it is necessary to further determine a more accurate 3D bounding box of the assembly component based on the 3D bounding box estimation results.

[0096] Affected by the pitch angle of the assembly component, if the component point cloud is projected according to the six planes of the 3D bounding box, there will be multiple component plane point clouds on a single face of the 3D bounding box, and the purpose of segmenting the component plane point cloud cannot be achieved. Therefore, the sampling plane is embedded in the 3D bounding box plane to narrow the point range to ensure that only the point cloud corresponding to a single plane is taken for projection surface fitting. The size of the sampling plane is mainly affected by the pitch angle of the component. The positive correlation between the size of the 3D bounding box and the pitch angle can be used to calculate the side length l of the sampling plane. m , l n , l o The value of is calculated as follows:

[0097]

[0098] Among them l a , l b , l c The length, width and height of the component design model, l i , l j , l k The length, width and height of the 3D bounding box.

[0099] After determining the sampling plane, select the component point cloud that is closest to the plane range along the plane normal. The fitting plane is the precise projection plane corresponding to each face of the component point cloud. The precise 3D bounding box is determined by calculating the intersection points of the fitting planes of each face as vertices. The segmentation of each face of a single component point cloud can be completed based on this 3D bounding box and the distance threshold. Among them, since a gap is reserved on the bottom surface of the component for pouring cement, its bottom surface is not a complete plane. Therefore, the remaining points after the segmentation of the remaining faces are bottom points. The calculation principle of the precise 3D bounding box and the component point cloud segmentation results are shown in Figure 2. Figure 2 Shown

[0100] 2. Point cloud dimensionality reduction

[0101] In order to achieve efficient and accurate extraction of assembly points, the segmented bottom point cloud with the sleeve is projected onto the bottom surface of the corresponding 3D bounding box. Since the sleeve is embedded in the concrete column, and the assembly effect is affected by the outer surface assembly hole and the inner sleeve, the scanned point cloud of the sleeve is also projected onto the bottom surface. Due to the existence of occlusion, the scanning of the sleeve is uneven, and the sleeve points directly projected onto the bottom surface will cause the point cloud density of the bottom surface assembly hole to have obvious differences locally, which will affect the fitting accuracy of the assembly points. Therefore, the present invention proposes a point cloud dimensionality reduction method based on a 3D bounding box and combined with angle discrimination to solve the problem of uneven point cloud density formed in the dimensionality reduction process.

[0102] The bottom surface point clouds within 2mm of the bottom surface of the 3D bounding box are selected as the bottom surface reference point clouds, and the remaining bottom surface point clouds are the sleeve point clouds. First, the bottom surface reference point cloud is projected onto the bottom surface plane of the 3D bounding box, and the subsequent sleeve point cloud is projected using the projected plane point cloud as the bottom surface reference. For each point in the sleeve point cloud, it is first projected onto the bottom surface along the axial direction obtained by the 3D bounding box, and all bottom surface points in the circular area with a radius of 4mm are extracted with this point as the center. The sleeve projection points and the bottom surface projection points have the following four relationships on the projection surface, such as Figure 3 As shown, a and b are defined as noise points, c is the boundary of the sleeve, and d is a point in the plane. Calculate the vector from the sleeve projection point to each selected bottom projection point, and use a certain vector as a reference to calculate the angle between the remaining vectors and the vector. After distinguishing the directions and sorting them according to the angles, the angle between every two adjacent vectors can be calculated. If all the obtained angles are less than 120 degrees, the point is considered to fall on the repeated area of ​​the plane and no projection is performed. If there is an angle greater than 120 degrees, the point is considered to belong to the assembly hole boundary point cloud and is projected onto the bottom surface. If there are less than three bottom points around the projection point of the sleeve point cloud, the point is considered to be a noise point and no projection is performed. The principle and results of point cloud dimensionality reduction are shown in the figure. Figure 3 Compared with the traditional method, the point cloud dimension reduction according to the above method can ensure the integrity of the internal boundary point cloud information of the sleeve while making the point cloud density around the assembly hole as uniform as possible, and is conducive to the accurate fitting of the assembly points.

[0103] In this embodiment, the improved maximum angle method is used on the projected bottom surface point cloud to extract the boundary point cloud of the sleeve assembly hole. Take any point in the bottom surface point cloud and take the radius r with the point as the center. d The vectors from all points in the scan area to the point are calculated clockwise, and the angle between each two adjacent vectors is calculated. If all angles are greater than 20° and there is an angle greater than 140°, the point is identified as a boundary point. If there is an angle less than 20° and greater than 140° at the same time, the two points with the smaller modulus length of the two vectors that form the minimum angle are selected, and the boundary point is determined respectively, and the point with the maximum angle between the two vectors closest to 180° is identified as the boundary point. In the present invention, r is defined according to the scanning point density. d =3mm.

[0104] For the extracted sleeve boundary points, the present invention proposes a two-step circle center fitting method based on uniform point selection and RANSAC method to ensure the accuracy of the obtained assembly points. First, since the point density of the projected point cloud is guaranteed to be as uniform as possible during dimensionality reduction, three points can be taken at uniform intervals for each sleeve boundary to calculate the circle center, and all uniform point fitting results are calculated in sequence to obtain the uniform point center point set Se. The center of the circle in the circle center point set Se is relatively accurate, but the number of points that can be used for assembly point fitting is relatively small. Therefore, it is necessary to use the RANSAC algorithm to calculate the center of the sleeve boundary point cloud again to obtain a more robust assembly point fitting result. According to the characteristics that the circle center points of the uniform circle center calculation method are relatively accurate but too few in number, and the circle center points calculated by the RANSAC method are numerous but easily dispersed. The RANSAC calculation point set is screened with the circle center point set Se as the reference point set to reduce the dispersion of the fitting points obtained by the RANSAC method, thereby reducing the number of iterations and improving the calculation efficiency. The average of the obtained circle center fitting points is used as the assembly point of the sleeve, and the overall process is as follows: Figure 4 shown.

[0105] Further, according to an embodiment of the present invention, the assembly component features and assembly points are registered with corresponding data in the initial standard design model of the assembly component to obtain the deviation between the assembly component features and assembly points of the target assembly component and the initial standard design model of the assembly component, including:

[0106] The relationship between the assembly component feature and assembly point reference point X0 (x0, y0, z0) and the corresponding point X (x, y, z) in the point cloud generated by the BIM model is as follows:

[0107] X0―(―λRX i +ΔX)=0;

[0108] Where R is the rotation matrix; X i For each point iterated in the entire point cloud, ΔX is the translation vector and λ is the scale factor with a value of 1;

[0109] When dealing with large-angle rotation in space, the Rodriguez matrix is ​​used to solve the problem. First, according to the rotation vector s=[s x s y s z ] T Construct the antisymmetric matrix S:

[0110]

[0111] The antisymmetric matrix S can be converted to the Rodrigues matrix R using the Rodrigues formula:

[0112] R=I+(sinθ)S+(1―cosθ)S 2 ;

[0113] Where I is the 3rd-order identity matrix and θ is the rotation angle;

[0114] The error equation is constructed through the above parameters, and the correction value of the observation point is solved. The error (i.e. deviation) equation is as follows:

[0115] V1=A1t+BX―L1;

[0116] Among them, V1 is the correction value vector, A1 is the angle coefficient matrix, t is the transformation parameter correction value vector, B is the coefficient matrix of the point to be solved, X is the correction value of the point to be solved, and L1 is the observation value residual vector; the parameters are constructed as follows:

[0117] V1=[V x V y V z ] T ;

[0118]

[0119] t=[dΔx dΔy dΔz da db dc] T ;

[0120] B=R ―1 ;

[0121] X=[dx0 dy0 dz0] T ;

[0122]

[0123] Further, according to an embodiment of the present invention, the deviation is analyzed, and the best virtual assembly solution is obtained according to the analysis result, including:

[0124] Analyze assembly points and component size deviations;

[0125] Analyze the root node position and axial deviation of steel bars;

[0126] Check the assembly point position and component size deviation and the steel bar root node position and axial deviation for assembly feasibility. When assembly feasibility is confirmed, determine the assembly deviation.

[0127] Based on the assembly deviation, the combination with the smallest RMSE of the overall assembly deviation of the whole group of assembly components is selected as the optimal virtual assembly solution.

[0128] In this embodiment, the overall process of VTA for the assembled frame structure includes deviation analysis of the assembly points, the root node points and the axial direction of the assembled components to verify the assemblability of the assembled components and calculate the best assembly plan. If the detection is correct, the components can be assembled directly. Otherwise, the components need to be adjusted according to the specific feedback information. The best assembly analysis is performed for each component of the same type, and then the multi-step assembly of the overall structure is performed in the global coordinate system of the overall environment point cloud.

[0129] In this embodiment, analysis of assembly point positions and component size deviations includes:

[0130] The scanned point cloud can be registered with the design model by using the vertices and edges of the precise 3D bounding box of the assembly component as registration features, such as Figure 5 As shown in the figure, the distance difference between the corresponding assembly points extracted from the scanned point cloud and the design model is used as the assembly point deviation of the component. According to the construction standard, the point deviation within 3mm is considered to be a compliant assembly point. After determining that all the assembly points of a single assembly component are compliant, it is necessary to perform a 3D comparison with the design model to confirm its dimensional deviation, and use the root mean square value RMS and root mean square error RMSE to evaluate its accuracy. The calculation formula is as follows:

[0131]

[0132] Among them, in the assembly deviation analysis, X obs,i is the calculated value of assembly deviation, X model,i Calculates values ​​for manufacturing deviations.

[0133] In this embodiment, the root node position and axial deviation of the steel bar are analyzed, including:

[0134] The overall point cloud of a single steel bar is sliced ​​with a thickness of 1 mm at equal intervals. 20 slice point clouds are extracted from each steel bar. The center of a single slice point cloud is fitted, and the axial direction of the steel bar is fitted with the obtained 20 circle centers. Figure 5 The axial direction of each steel bar on the assembled component is compared with the overall axial direction of the component, and the angle between the two is calculated as the axial deviation. It is determined that the axial deviation exceeds the specified threshold c ra For assembly components of different specifications c ra The calculation formula is as follows:

[0135]

[0136] Among them, l r is the length of the steel bar, r s , r e is the radius of the sleeve and the reinforcement, t d For manufacturing tolerances.

[0137] The intersection of the steel bar axis and the top surface of the 3D boundary box of the assembly component is taken as the steel bar root node. The calculated root node is analyzed for deviation from the design model to determine whether it is compliant.

[0138] In this embodiment, the assemblability under the combined effect of the assembly point position and component size deviation and the steel bar root node point position and axial deviation is checked as follows:

[0139] Since the assemblability of the component is affected by the root node position deviation of the steel bar at the assembly position, the axial deviation and the sleeve assembly point deviation of the assembled component. Therefore, after confirming that each single manufacturing deviation of the component is compliant, it is necessary to check the assemblability under the combined effect of the deviation. r (x r ,y r ,z r ) and sleeve assembly point P s (x s ,y s ,z s ) is less than the difference between the sleeve radius and the steel bar radius, then calculate the top node P of the steel bar t (x t ,y t ,z t ) Projection point P on the assembly joint surface m (x m ,y m ,z m ). Calculate P m With P s The distance difference d a , if d a If it is also smaller than the difference between the sleeve radius and the steel bar radius, it is considered that the point can be assembled smoothly, and d a is the assembly deviation, and the calculation formula is as follows:

[0140]

[0141] Among them, l r is the length of the steel bar, θ is the axial deviation of the steel bar, A, B, C, D are the parameters of the plane equation of the assembly joint surface, k1, b1, k2, b2 are the parameters of the axial equation of the steel bar.

[0142] In this embodiment, the optimal virtual assembly solution is:

[0143] Prefabricated frame structures are usually assembled layer by layer, and the error of VTA will gradually accumulate as the number of layers increases. In order to minimize the cumulative error, it is necessary to perform optimal assembly analysis on the components. For each of a group of components of the same type to be assembled, it is necessary to analyze its assembly deviation at each assembly position of the corresponding group, and calculate its RMSE as the accuracy evaluation standard. The assembly deviation of the entire group of assembled components is counted and the combination with the smallest overall RMSE is selected as the best assembly solution. The assembly position, elevation and verticality of the components after VTA are checked to verify the VTA accuracy and guide the final physical assembly.

[0144] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a steel-concrete frame structure VTA system based on ground-based LiDAR point cloud, comprising:

[0145] The data acquisition module collects data of the target assembly components through the ground-based three-dimensional laser radar, and processes the data to produce a data set;

[0146] The semantic information and 3D bounding box acquisition module inputs the dataset into the SSBR-Net network model and outputs semantic information and 3D bounding boxes;

[0147] The feature and point acquisition module extracts the assembly component features from the 3D bounding box, and then calculates the assembly points based on the 3D bounding box;

[0148] The deviation calculation module aligns the assembly component features and assembly points with the corresponding data in the initial standard design model of the assembly component, and obtains the deviation between the assembly component features and assembly points of the target assembly component and the initial standard design model of the assembly component;

[0149] The VTA acquisition module analyzes the deviation and acquires the best virtual assembly solution according to the analysis result.

[0150] The steel-concrete frame structure VTA system based on ground-based LiDAR point cloud of the present invention can realize the above-mentioned steel-concrete frame structure VTA method based on ground-based LiDAR point cloud. The specific steps are as described above and will not be repeated here.

[0151] Furthermore, to achieve the above-mentioned purpose, the present invention also provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the VTA method for steel-concrete frame structure based on ground-based LiDAR point cloud as described above is implemented.

[0152] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the VTA method of steel-concrete frame structure based on ground-based LiDAR point cloud as described above is implemented.

[0153] Furthermore, based on the above-mentioned solution of the present invention, the effectiveness of the above-mentioned solution of the present invention is verified in the form of a specific embodiment in combination with the accompanying drawings.

[0154] Example 1

[0155] The method proposed in this embodiment is actually applied to the construction of a smart logistics port on a certain highway to verify the feasibility of the proposed method. The main building of the logistics port is composed of an assembled prefabricated frame structure. The main building has five floors and contains 184 prefabricated reinforced concrete columns of three different seismic strengths and sizes, all connected by steel bars and sleeves. This embodiment uses two groups of representative assembly positions as examples for result analysis. The selected scene contains a total of 8 assembly components of three different sizes and corresponding assembly positions.

[0156] In this embodiment, the developed model SSBR-Net is trained on a workstation with performance including a memory i9-13900K central processing unit (CPU) and a 24G GPU GeForce RTX4090 graphics processing unit (GPU). For the developed model, to prevent the model from overfitting, the AdamW optimizer with a learning rate of 10-5 is applied as initialization, and the Batchsize is set to 32 to train the model for up to 100 epochs, and the results are evaluated.

[0157] Use high-precision ground-based 3D laser radar to collect point cloud data of the entire construction scene, scan the assembly components with a higher resolution, and scan the entire construction environment with a lower resolution to connect the point cloud of the entire scene. After obtaining the overall point cloud in the global coordinate system in this way, use SSBR-Net to perform semantic segmentation and 3D bounding box estimation of the assembly components. The results are as follows: Figure 6 As shown in the figure. SSBR-Net can realize fast and accurate semantic segmentation of assembly components in large and complex scenes, and can estimate the 3D bounding box and axial direction of the components based on the semantic segmentation results, and divide the three types of assembly components C30, C40, and C50 into instances according to the bounding box size. The intersection over union (IOU) and mean intersection over union (mIOU) are used as the evaluation criteria for segmentation accuracy. The calculation formula is as follows. The segmentation and recognition accuracy of each component is shown in Table 1.

[0158]

[0159] Among them, TP is a true positive example, FP is a false positive example, FN is a false negative example, and TN is a true negative example.

[0160] The segmentation and recognition accuracy analysis of each component is shown in Table 1:

[0161]

[0162] Table 1

[0163] By comparing the design model of the assembly component with the 3D bounding box of the scanned point cloud, the deviation information of the component can be obtained. In order to accurately calculate the manufacturing deviation of the assembly component, it is necessary to further calculate the accurate 3D bounding box. According to the accurate 3D bounding box, the required assembly features can be calculated, including the overall axial direction of the assembly component, the bottom surface assembly point, the axial direction of the steel bar, and the root node point of the steel bar. In this way, the component point cloud can be abstracted into a 3D bounding box and various assembly feature representations to obtain a simplified component assembly feature model, which is convenient for the analysis of various deviations of the assembly component. The results are as follows: Figure 7 shown.

[0164] This embodiment performs deviation analysis on the concrete column size, sleeve assembly point, steel bar length, axial direction and steel bar assembly point according to the construction plan requirements. The deviation analysis of the component is completed by comparing the component scanning point cloud with the assembly feature model extracted from the design model. The results are as follows: Figure 8 As shown in the figure, the concrete column dimensions, sleeve assembly points and steel bar assembly points did not exceed the limit deviation, the steel bar length exceeded the standard value in 2 places, and the steel bar axial length exceeded the specified tolerance in 14 places.

[0165] The components corrected according to the deviation analysis results are assembled at their corresponding assembly positions. For the three components of different models, due to their different sizes and sleeve arrangements, the calculation methods for matching their respective assembly points are slightly different. In order to obtain the result with the smallest assembly deviation, it is necessary to calculate all assembly schemes and determine the best assembly method. The assembly points of the C30 model component sleeve are arranged in a centrally symmetrical shape, so a single component can be assembled in the four positive directions of its corresponding assembly position. However, there is no symmetrical relationship between the C40 and C50 model component sleeves, and there is only a single corresponding relationship between a single component and the assembly position. In this embodiment, two groups of assembly scenarios CJ1 and CJ2 are selected to assemble three types of components respectively. CJ1 can assemble four C30 components, and CJ2 can assemble one C40 and three C50 components. CJ1 is used as an example to perform assembly deviation analysis. For each matching scheme, the deviation of each assembly point is calculated, and its mean, standard deviation, RMS and RMSE are statistically used as the evaluation criteria for assembly accuracy. The results are shown as follows. Fig. 9As shown. Among the 64 matching schemes, 9 were discarded due to excessive assembly deviation. The best assembly analysis was performed on the compliant schemes, and the matching combination with the smallest sum of the evaluation indicators was calculated. The best assembly scheme for CJ1 was C30-1 paired with CJ1-3, C30-2 paired with CJ1-1, C30-3 paired with CJ1-2, and C30-4 paired with CJ1-4. Based on the results of the assembly deviation analysis, the result with the smallest overall assembly deviation was calculated as the best assembly scheme for the component VTA. The VTA results of CJ1 and CJ2 are shown as follows: Fig.10 The assembly position, elevation and verticality of the components after VTA were tested, and the results are shown in Table 2. All the test results meet the standards, proving that the VTA method of this embodiment is accurate and feasible, and can effectively guide the assembly of components.

[0166]

[0167] Table 2

[0168] According to the scheme of this embodiment, this embodiment first proposes a deep learning method for extracting assembly components in large scenes based on semantic segmentation and voting mechanism, which realizes accurate and efficient extraction of required component point clouds in large and complex construction scene point clouds and performs preliminary 3D bounding box estimation. The mIOU of component segmentation is 0.9429, indicating that the accuracy of segmentation is high, and the proposed SSBR-Net can accurately realize the segmentation of assembly components.

[0169] Based on the semantic segmentation results of SSBR-Net, the 3D bounding box and direction estimation of the component, a high-precision assembly position positioning method based on the 3D bounding box is proposed. First, the precise 3D bounding box of the component is calculated based on the preliminary estimated 3D bounding box and direction, and a point cloud dimensionality reduction method based on the 3D bounding box is designed. Compared with the dimensionality reduction method based on PCA, this method can provide a more accurate projection plane, and ensure that the density of the point cloud after dimensionality reduction is uniform and the feature points are retained intact, which is more conducive to the accurate calculation of assembly features. Based on the 3D bounding box, various assembly features of the component are calculated, and the component point cloud is abstracted as an assembly feature model representation. This greatly reduces the redundancy of the data, expresses the assembly components in a more intuitive way, and improves the efficiency of subsequent deviation analysis and VTA solution analysis.

[0170] According to the assembly characteristics, various deviations of the components were analyzed. The results showed that there was no manufacturing deviation in the concrete column part of all components, and the assembly points were relatively accurate. There was a small deviation in the length of the steel bars, and a large deviation in the axial direction of the steel bars. Considering the good overall processing accuracy of the components, the deviation in the axial direction of the steel bars may be due to collisions during transportation. Among them, the No. 6 steel bars of the C30-4 component and the No. 14 steel bars of the C40-1 component, which exceeded the length limit, also had large deviations in the axial direction, indicating that there was a high possibility of manufacturing errors in these two points.

[0171] This embodiment performs a VTA analysis on the qualified components under global coordinates, and calculates all possible assembly schemes for each assembly position to obtain the best assembly scheme. Among the 64 matching schemes of CJ1, 9 are out of limit, and 8 of them appear in the matching with CJ1-4, indicating that the connecting steel bars of CJ1-4 may be hit, and it is recommended to correct them before assembly. The component assembly deviation analysis after VTA did not exceed the tolerance, proving that the VTA method proposed in this embodiment has a small cumulative error and high accuracy. In addition, the VTA method proposed in this embodiment performs assembly under the global coordinate system, and can realize independent VTA analysis of a single group of components without the need to obtain data of all assembled components at one time, saving the space cost of actual assembly. The traditional VTA method is usually performed in a relative coordinate system, and it is necessary to obtain all data at one time for overall VTA to ensure its accuracy.

[0172] Those skilled in the art will appreciate that the modules and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0173] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and equipment can refer to the corresponding processes in the aforementioned method implementation methods, and will not be repeated here.

[0174] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0175] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0176] In addition, each functional module in the embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0177] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the energy-saving signal sending / receiving method of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical disks.

[0178] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other.

[0179] It should be understood that the size of the serial numbers of each step in the content of the invention and the implementation methods of the present invention does not absolutely mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the implementation methods of the present invention.

Claims

1. The VTA method of steel-concrete frame structure based on ground-based LiDAR point cloud is characterized by: include: The target assembly components are collected through ground-based 3D laser radar, and the data are processed to produce a data set; Input the dataset into the SSBR-Net network model and output semantic information and 3D bounding box; Extract assembly component features from the 3D bounding box, and calculate assembly points based on the 3D bounding box; The assembly component features and assembly points are registered with corresponding data in the initial standard design model of the assembly component, and the deviation between the assembly component features and assembly points of the target assembly component and the initial standard design model of the assembly component is obtained; The deviation is analyzed, and the best virtual assembly solution is obtained according to the analysis result.

2. The VTA method for steel-concrete frame structure based on ground-based LiDAR point cloud according to claim 1 is characterized in that: The data collection of the target assembly component by using the ground-based three-dimensional laser radar includes: The ground-based 3D laser radar is used to collect the point cloud of the target assembly components placed on the ground and the overall point cloud of the construction scene.

3. The VTA method for steel-concrete frame structure based on ground-based LiDAR point cloud according to claim 1, characterized in that: The SSBR-Net network model consists of a RandLaNet semantic segmentation network and a BRNet network with a SAM spatial attention mechanism.

4. The VTA method for steel-concrete frame structure based on ground-based LiDAR point cloud according to claim 1, characterized in that: The step of extracting assembly component features from the 3D bounding box and calculating assembly points based on the 3D bounding box includes: Embedding meshes are performed on each plane of the 3D bounding box of the target assembly component to determine candidate points for plane fitting on each surface of the point cloud of the assembly component; Use candidate points to fit the precise projection planes of each surface to determine the precise 3D bounding box, and divide the point clouds of each surface according to the 3D bounding box to perform dimensionality reduction of the bottom surface point cloud; The intersection point between the steel bar axis and the top plane of the assembly component is calculated as the top assembly point, and the 3D bounding box boundary is used as the boundary feature of the assembly component; The maximum angle method is used to extract the sleeve boundary of the bottom surface as the assembly position on the bottom surface point cloud after dimensionality reduction, and the RANSAC method is used to fit the center of the sleeve boundary circle, and the result is used as the assembly point position of the bottom surface of the assembly component.

5. The VTA method for steel-concrete frame structure based on ground-based LiDAR point cloud according to claim 4 is characterized in that: The RANSAC method is used to fit the center of the sleeve boundary circle, and the result is used as the assembly point of the bottom surface of the assembly component, including: First, three points are taken at even intervals on each sleeve boundary to calculate the circle center, and all evenly taken point fitting results are calculated sequentially to obtain the evenly taken point circle center point set Se; Use the RANSAC algorithm to calculate the center fitting point of the sleeve boundary point cloud; The circle center point set Se is used as the reference point set to screen the RANSAC calculation point set to reduce the dispersion of the circle center fitting points obtained by the RANSAC method; The final mean value of the circle center fitting point is used as the assembly point position of the bottom surface of the assembly component.

6. The VTA method for steel-concrete frame structure based on ground-based LiDAR point cloud according to claim 1, characterized in that: The step of registering the assembly component features and assembly points with corresponding data in the initial standard design model of the assembly component to obtain the deviation between the assembly component features and assembly points of the target assembly component and the initial standard design model of the assembly component includes: The relationship between the assembly component features and assembly point reference point X0 (x0, y0, z0) and the corresponding point X (x, y, z) in the point cloud generated by the BIM model is as follows: X0―(―λRX i +ΔX)=0; Where R is the rotation matrix; X i For each point iterated in the entire point cloud, ΔX is the translation vector and λ is the scale factor with a value of 1; When dealing with large-angle rotation in space, the Rodriguez matrix is ​​used for solution. First, according to the rotation vector s=[s x s y s z ] T Construct the antisymmetric matrix S: The antisymmetric matrix S can be converted to the Rodrigues matrix R using the Rodrigues formula: R=I+(sinθ)S+(1―cosθ)S 2 ; Where I is the 3rd-order identity matrix and θ is the rotation angle; The error equation is constructed through the above parameters, and the correction value of the observation point is solved. The error equation is as follows: V1=A1t+BX―L1; Among them, V1 is the correction value vector, A1 is the angle coefficient matrix, t is the transformation parameter correction value vector, B is the coefficient matrix of the point to be solved, X is the correction value of the point to be solved, and L1 is the observation value residual vector; the parameters are constructed as follows: V1=[V x V y V z ] T ; t=[dΔx dΔy dΔz da db dc] T ; B=R ―1 ; X=[dx0 dy0 dz0] T ; 7. The VTA method for steel-concrete frame structure based on ground-based LiDAR point cloud according to any one of claims 1 to 6, characterized in that: The deviation is analyzed, and the best virtual assembly solution is obtained according to the analysis result, including: Analyze assembly points and component size deviations; Analyze the root node position and axial deviation of steel bars; Check the assembly point and component size deviation and the root node point and axial deviation of the steel bar for assembly feasibility. When it is confirmed that assembly is feasible, determine the assembly deviation. Based on the assembly deviation, the combination with the smallest RMSE of the overall assembly deviation of the whole group of assembly components is selected as the optimal virtual assembly solution.

8. The steel-concrete frame structure VTA system based on the ground-based LiDAR point cloud is characterized by: include: The data acquisition module collects data of the target assembly components through the ground-based three-dimensional laser radar, and processes the data to produce a data set; The semantic information and 3D bounding box acquisition module inputs the dataset into the SSBR-Net network model and outputs semantic information and 3D bounding boxes; The feature and point acquisition module extracts the assembly component features from the 3D bounding box, and then calculates the assembly points based on the 3D bounding box; The deviation calculation module aligns the assembly component features and assembly points with the corresponding data in the initial standard design model of the assembly component, and obtains the deviation between the assembly component features and assembly points of the target assembly component and the initial standard design model of the assembly component; The VTA acquisition module analyzes the deviation and acquires the best virtual assembly solution according to the analysis result.

9. An electronic device, characterized in that The invention comprises a processor, a memory and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steel-concrete frame structure VTA method based on ground-based LiDAR point cloud as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steel-concrete frame structure VTA method based on ground-based LiDAR point cloud as described in any one of claims 1 to 7 is implemented.

Images