Steel-concrete frame structure vta method based on ground-based lidar point cloud

Abstract

The present application relates to the technical field of virtual assembly of fabricated frame structure, and provides a steel-concrete frame structure VTA method based on ground LiDAR point cloud, comprising: collecting data of target assembly component by ground three-dimensional laser radar, processing the data to make a data set; inputting the data set into an SSBR-Net network model to output semantic information and a 3D bounding box; extracting assembly component features from the 3D bounding box to calculate assembly points; registering the assembly component features and assembly points with corresponding data in an initial standard design model of assembly component to obtain deviations between the assembly component features and assembly points of target assembly component and the initial standard design model of assembly component; analyzing the deviations to obtain an optimal virtual assembly scheme according to the analysis result. The present application greatly reduces data redundancy, improves the efficiency of deviation analysis and VTA scheme analysis, and improves VTA accuracy.

Description

Technical Field

[0001] This invention relates to the field of virtual assembly technology for prefabricated frame structures, and in particular to a VTA method for steel-concrete frame structures based on foundation LiDAR point clouds. Background Technology

[0002] Virtual pre-assembly (VTA) technology, as an efficient and low-cost alternative to physical pre-assembly, is increasingly being applied in the assembly process of prefabricated structures. Traditional VTA methods typically involve targeted scanning of assembly components and manual segmentation of the required component point clouds. This method is inefficient in point cloud acquisition and segmentation, and therefore, traditional VTA methods are limited in both accuracy and efficiency when processing large-scale point cloud data due to the influence of manual operation.

[0003] Furthermore, traditional solutions also suffer from the following technical drawbacks:

[0004] 1. Insufficient Automation: Traditional VTA largely relies on manual segmentation and identification, which is not only time-consuming but also increases operational complexity and the possibility of errors. The lack of effective automation tools to process and analyze point cloud data results in a low level of automation throughout the assembly process.

[0005] 2. Accuracy Issues: Manual processing of point cloud data is easily affected by the operator's skill and experience, which may lead to inconsistencies and accuracy problems. Furthermore, the accuracy and reliability of traditional methods may be further reduced when processing complex or occluded components.

[0006] 3. Scalability and flexibility issues: When dealing with large-scale or complex structures, the scalability of traditional VTAs is limited. In large projects, manually processing large amounts of point cloud data significantly increases time costs and resource consumption, and it is difficult to flexibly respond to changes in project scale or requirements. Summary of the Invention

[0007] The purpose of this invention is to solve at least one technical problem in the background art and to provide a VTA method for steel-concrete frame structures based on ground-based LiDAR point clouds.

[0008] To achieve the above objectives, this invention provides a VTA method for reinforced concrete frame structures based on ground-based LiDAR point clouds, comprising:

[0009] Data is collected from the target assembly components using a ground-based 3D lidar, and the data is processed to create a dataset.

[0010] Input the dataset into the SSBR-Net network model to output semantic information and 3D bounding boxes;

[0011] The assembly component features are extracted from the 3D bounding box, and then the assembly points are calculated based on the 3D bounding box.

[0012] The assembly component features and assembly points are registered with the corresponding data in the initial standard design model of the assembly component to obtain the deviation between the assembly component features and assembly points of the target assembly component and the initial standard design model of the assembly component.

[0013] The deviations are analyzed, and the optimal virtual assembly scheme is obtained based on the analysis results.

[0014] According to one aspect of the present invention, the data acquisition of the target assembly component by ground-based three-dimensional lidar includes:

[0015] The point cloud of the target assembly components placed on the ground and the overall point cloud of the construction scene are collected by ground-based 3D lidar.

[0016] According to one aspect of the present invention, the SSBR-Net network model consists of a RandLaNet semantic segmentation network and a BRNet network with SAM spatial attention mechanism.

[0017] According to one aspect of the present invention, the step of extracting assembly component features from the 3D bounding box and calculating assembly points based on the 3D bounding box includes:

[0018] The 3D bounding box of the target assembly component is divided into embedded meshes on each plane to determine the candidate points for plane fitting on each face of the assembly component's point cloud.

[0019] The precise 3D bounding box is determined by fitting the accurate projection plane of each face with the candidate points, and the point cloud of each face is divided according to the 3D bounding box, and then the point cloud of the bottom face is reduced in dimensionality.

[0020] The intersection of the axial direction of the reinforcing bars of the assembled component with the top plane is calculated as the top assembly point, and the 3D bounding box boundary is used as the boundary feature of the assembled component.

[0021] The bottom sleeve boundary was extracted from the dimensionality-reduced bottom point cloud using the maximum angle method, and the RANSAC method was used to fit the center of the sleeve boundary circle. The result was used as the assembly point on the bottom surface of the assembly component.

[0022] According to one aspect of the present invention, the method of fitting the center of the sleeve boundary circle using the RANSAC method, the result of which serves as the assembly point location on the bottom surface of the assembly component, includes:

[0023] First, take three points at uniform intervals for each sleeve boundary to calculate the center of the circle. Then, calculate the fitting results of all uniformly selected points in sequence to obtain the set of uniformly selected center points Se.

[0024] The RANSAC algorithm is used to calculate the center fitting points of the sleeve boundary point cloud;

[0025] The RANSAC calculation point set is filtered by using the circle center point set Se as the reference point set to reduce the dispersion of the circle center fitting points obtained by the RANSAC method.

[0026] The average value of the finally obtained center fitting points is used as the assembly point on the bottom surface of the assembly component.

[0027] According to one aspect of the present invention, the step of registering the assembly component features and assembly points with corresponding data in the initial standard design model of the assembly component to obtain the deviation between the assembly component features and assembly points of the target assembly component and the initial standard design model of the assembly component includes:

[0028] The relationship between the assembly component features and the assembly point reference point X0(x0, y0, z0) and the corresponding point X(x, y, z) in the point cloud generated by the BIM model is as follows:

[0029] X0―(―λRX i +ΔX)=0;

[0030] Where R is the rotation matrix; X i For each point in the overall point cloud, ΔX is the translation vector, and λ is a scale factor with a value of 1.

[0031] When dealing with large-angle rotations in space, the Rodrigues matrix is ​​used for solution. First, based on the rotation vector s = [s...] x s y s z ] T Construct the antisymmetric matrix S:

[0032]

[0033] The antisymmetric matrix S can be transformed into a Rodrigues matrix R using the Rodrigues formula:

[0034] R = I + (sinθ)S + (1 - cosθ)S 2 ;

[0035] Where I is a 3rd order identity matrix, and θ is the rotation angle;

[0036] The error equation is constructed using the above parameters, and the correction value for the observation point is solved. The error equation is as follows:

[0037] V1 = A1t + BX - L1;

[0038] Where V1 is the correction vector, A1 is the angle coefficient matrix, t is the transformation parameter correction vector, B is the coefficient matrix of the point to be solved, X is the correction value of the point to be solved, and L1 is the observation residual vector; the parameters are constructed as follows:

[0039] V1 = [V x V y V z ] T ;

[0040]

[0041]

[0042] t=[dΔx dΔy dΔz da db dc] T ;

[0043] B = R ―1 ;

[0044] X = [dx0 dy0 dz0] T ;

[0045]

[0046] According to one aspect of the present invention, analyzing the deviation and obtaining an optimal virtual assembly scheme based on the analysis results includes:

[0047] Analyze assembly points and component dimensional deviations;

[0048] Analyze the location and axial deviation of the rebar root joints;

[0049] The assemblability is checked under the combined effect of the assembly point location and component size deviation, as well as the rebar root node location and axial deviation. If assemblability is confirmed, the assembly deviation is determined.

[0050] Based on assembly deviations, the combination with the smallest overall assembly deviation RMSE of the entire assembly components is selected as the optimal virtual assembly scheme.

[0051] To achieve the above objectives, the present invention also provides a VTA system for reinforced concrete frame structures based on ground-based LiDAR point clouds, comprising:

[0052] The data acquisition module collects data from the target assembly components using a ground-based 3D lidar, and processes the data to create a dataset.

[0053] The semantic information and 3D bounding box acquisition module takes the dataset as input to the SSBR-Net network model and outputs semantic information and 3D bounding boxes.

[0054] The feature and point acquisition module extracts assembly component features from the 3D bounding box and then calculates the assembly points based on the 3D bounding box.

[0055] The deviation calculation module registers the assembly component features and assembly points with the corresponding data in the initial standard design model of the assembly component to obtain the deviation between the assembly component features and assembly points of the target assembly component and the initial standard design model of the assembly component.

[0056] The VTA acquisition module analyzes the deviation and obtains the optimal virtual assembly scheme based on the analysis results.

[0057] To achieve the above objectives, the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the VTA method for steel-concrete frame structures based on ground-based LiDAR point clouds as described above.

[0058] To achieve the above objectives, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the VTA method for steel-concrete frame structures based on ground-based LiDAR point clouds as described above.

[0059] According to the present invention, the present invention first proposes a deep learning method for extracting assembly components in large scenes based on semantic segmentation and voting mechanism, which realizes accurate and efficient extraction of the required component point cloud from large and complex construction scene point cloud and performs preliminary 3D bounding box estimation.

[0060] Based on the semantic segmentation results of SSBR-Net, and the estimation of component 3D bounding boxes and orientations, a high-precision assembly position localization method based on 3D bounding boxes is proposed. First, the precise 3D bounding box of the component is calculated based on the initially estimated 3D bounding box and orientation, and a point cloud dimensionality reduction method based on this 3D bounding box is designed. Compared with PCA-based dimensionality reduction methods, this method can provide a more accurate projection plane and ensure uniform point cloud density and complete feature point retention after dimensionality reduction, which is more conducive to the accurate calculation of assembly features. Various assembly features of the component are calculated based on this 3D bounding box, and the component point cloud is abstracted into an assembly feature model representation. This greatly reduces data redundancy, expresses the assembly component in a more intuitive way, and improves the efficiency of subsequent deviation analysis and VTA scheme analysis.

[0061] Based on the assembly characteristics, various deviation analyses are performed on the components. The optimal virtual assembly scheme is obtained based on the analysis results, ensuring that the cumulative error of the optimal virtual assembly scheme is small and the accuracy is high. Attached Figure Description

[0062] Figure 1 A flowchart illustrating a ground-based LiDAR point cloud-based VTA method for steel-concrete frame structures according to an embodiment of the present invention is shown.

[0063] Figure 2 This diagram illustrates the principle of accurate 3D bounding box calculation and component point cloud segmentation results according to an embodiment of the present invention.

[0064] Figure 3 The diagram illustrates the principle of point cloud dimensionality reduction based on 3D bounding boxes according to one embodiment of the present invention.

[0065] Figure 4 This diagram illustrates an assembly point extraction flowchart according to one embodiment of the present invention.

[0066] Figure 5 A schematic diagram illustrating the deviation analysis of an assembly component according to an embodiment of the present invention;

[0067] Figure 6 This diagram illustrates the semantic segmentation and 3D bounding box estimation results according to Embodiment 1 of the present invention.

[0068] Figure 7 This diagram schematically illustrates the component assembly feature extraction results according to Embodiment 1 of the present invention.

[0069] Figure 8 A schematic diagram illustrating the component deviation analysis results according to Embodiment 1 of the present invention;

[0070] Figure 9 This diagram schematically illustrates the results of CJ1 assembly deviation analysis according to Embodiment 1 of the present invention.

[0071] Figure 10 The diagrams illustrate the VTA results of CJ1 and CJ2 according to Embodiment 1 of the present invention. Detailed Implementation

[0072] The invention will now be discussed with reference to exemplary embodiments. It should be understood that the described embodiments are merely intended to enable those skilled in the art to better understand and thus implement the invention, and are not intended to imply any limitation on the scope of the invention.

[0073] As used herein, the term "comprising" and its variations are to be interpreted as open-ended terms meaning "including but not limited to". The term "based on" is to be interpreted as "at least partially based on". The terms "one embodiment" and "an embodiment" are to be interpreted as "at least one embodiment".

[0074] Figure 1This schematically illustrates a flowchart of a ground-based LiDAR point cloud-based VTA method for reinforced concrete frame structures according to one embodiment of the present invention. Figure 1 As shown, in this embodiment, the VTA method for reinforced concrete frame structures based on ground-based LiDAR point clouds includes:

[0075] Data is collected from the target assembly components using a ground-based three-dimensional lidar (LiDAR), and the data is processed to create a dataset.

[0076] Input the dataset into the SSBR-Net network model to output semantic information and 3D bounding boxes;

[0077] The assembly component features are extracted from the 3D bounding box, and then the assembly points are calculated based on the 3D bounding box.

[0078] The assembly component features and assembly points are registered with the corresponding data in the initial standard design model of the assembly component to obtain the deviation between the assembly component features and assembly points of the target assembly component and the initial standard design model of the assembly component.

[0079] The deviations are analyzed, and the optimal virtual assembly scheme is obtained based on the analysis results.

[0080] Furthermore, according to one embodiment of the present invention, data acquisition of the target assembly component is performed using a ground-based three-dimensional lidar, including:

[0081] The point cloud of the target assembly components placed on the ground and the overall point cloud of the construction scene are collected by ground-based 3D lidar.

[0082] In this embodiment, point cloud data acquisition requires simultaneously acquiring point clouds at two different scales: one for a single target assembly component and the other for the entire construction scene. The high-precision point cloud of the assembly component is used for accurate VTA (Virtual Aspect Ratio) and deviation analysis, while the low-precision point cloud of the construction scene provides the scene and global coordinates required for VTA. A high-precision ground-based 3D LiDAR with adjustable acquisition distance and resolution can meet the data acquisition requirements. The assembly component is acquired using high-resolution parameters at close range, while the construction scene is acquired using lower-resolution parameters at long range, allowing for targeted and rapid point cloud data acquisition.

[0083] Furthermore, in this embodiment, the initial standard design model (BIM model) of the assembly components is converted into point cloud data. This conversion process typically employs sampling or resampling methods. Considering that subsequent point cloud registration requires the assembly component boundaries as features, a curvature-based sampling method is used to obtain the design model point cloud to ensure feature accuracy. Curvature-based sampling determines the density of sampling points based on the curvature of the design model surface; areas with greater curvature have denser sampling points, maximizing the preservation of geometric features for registration. The processing software can then resample and perform curvature-based downsampling on the design model to generate the design model point cloud.

[0084] Furthermore, according to one embodiment of the present invention, the SSBR-Net network model consists of a RandLaNet semantic segmentation network and a BRNet network with SAM spatial attention mechanism.

[0085] In this embodiment, the original method for extracting features in the BRNet network is PointNet. This invention replaces it with the RandLaNet method, which has the advantage of using random sampling dimensionality reduction to speed up the process. In addition, LFA module feature aggregation is performed in RandLaNet to ensure the integrity of local features. Dimensionality is increased through multi-layer MLP, and finally point features with labels are output. The sampled point features are enhanced through the SAM spatial attention mechanism and input into the BRNet network. The next step is to follow the BRNet network process, which performs voting sampling on the point features, generates voting points and point clusters, backtracks representative points to generate representative point sets, and finally performs pose estimation and classification. The output results are output as bounding boxes using the 3D NMS non-maximum suppression method.

[0086] In this implementation, the SSBR-Net network model first receives raw 3D point cloud data and extracts semantic information relative to the entire scene from the complex construction scenario using the RandLaNet semantic segmentation network. Based on the obtained semantic information, the model performs voting point generation and clustering operations through the Spatial Attention module (SAM). SAM enhances the model's ability to perceive spatial details in the input data, thereby improving overall prediction accuracy and efficiency. During voting point generation and clustering operations, the Spatial Attention module ensures that the voting points more accurately reflect key structural information. This not only improves the accuracy of representative point and 3D bounding box generation in subsequent steps but also enhances the overall model's efficiency and reliability.

[0087] In this implementation, the feature information of the voting points is output, and sampling-based voting learning is performed on the semantic information obtained within the region. Semantic seeds are independently converted into voting points. The model utilizes local features extracted from global semantic data to sample within a specified range. The local point set obtained from the sampling is used to acquire local global features through a multilayer perceptron (MLP) and pooling layers. This step ensures the accurate acquisition of the geometric structure of the feature points. Clustering operations are used to generate cluster centers. New representative points are generated around the voting centers, and the accuracy of the seed points in their vicinity is re-examined. This backtracking strategy ensures the consistency between the predicted voting point centers and the original target point cloud surface points. Finally, by reviewing and refining the voting results again, object proposals are generated, refined, and classified to generate the final 3D bounding box.

[0088] Furthermore, according to one embodiment of the present invention, the assembly component features are extracted from the 3D bounding box, and the assembly points are calculated based on the 3D bounding box, including:

[0089] The 3D bounding box of the target assembly component is divided into embedded meshes on each plane to determine the candidate points for plane fitting on each face of the assembly component's point cloud.

[0090] The precise 3D bounding box is determined by fitting the accurate projection plane of each face with the candidate points, and the point cloud of each face is divided according to the 3D bounding box, and then the point cloud of the bottom face is reduced in dimensionality.

[0091] The intersection of the axial direction of the reinforcing bars of the assembled component with the top plane is calculated as the top assembly point, and the 3D bounding box boundary is used as the boundary feature of the assembled component.

[0092] The boundary of the bottom sleeve (assembly position) is extracted from the dimensionality-reduced bottom point cloud using the maximum angle method, and the RANSAC method is used to fit the center of the sleeve boundary circle. The result is used as the assembly point of the bottom surface of the assembly component.

[0093] In this embodiment, the point cloud dimensionality reduction method based on 3D bounding boxes includes:

[0094] 1. Accurate 3D bounding box calculation

[0095] The 3D bounding box (3D bounding box) that is roughly consistent with the horizontal angle of the concrete column can be obtained from the estimation results output by the SSBR-Net network model. However, for non-horizontally placed assembled components, the 3D bounding box cannot fit well with the plane of the component. Therefore, it is necessary to further determine a more accurate 3D bounding box of the assembled component based on the 3D bounding box estimation results.

[0096] Due to the pitch angle of the assembled components, if the component point cloud is projected onto the six planes of the 3D bounding box, multiple component planar point clouds will exist on a single face of the 3D bounding box, failing to achieve the goal of segmenting the component planar point cloud. Therefore, a sampling plane is embedded within the 3D bounding box plane to narrow the sampling range, ensuring that only the point cloud corresponding to a single plane is sampled for projection plane fitting. The size of the sampling plane is mainly affected by the component pitch angle; the side length l of the sampling plane can be calculated using the positive correlation between the size of the 3D bounding box and the pitch angle. m , l n , l o The value of is calculated using the following formula:

[0097]

[0098] Among them l a , l b , l c For the length, width, and height of the component design model, l i , l j , l k The dimensions are the length, width, and height of the 3D bounding box.

[0099] After determining the sampling plane, the nearest component point cloud within the plane's normal range is selected. Its fitting plane is the precise projection plane corresponding to each face of the component point cloud. The intersection points of these fitting planes are calculated as vertices to determine the precise 3D bounding box. Then, based on this 3D bounding box and a distance threshold, the faces of a single component point cloud can be segmented. However, because the bottom surface of the component has a gap reserved for cement pouring, it is not a complete plane; therefore, the remaining points after the other faces are segmented are the bottom surface points. The principle of precise 3D bounding box calculation and the component point cloud segmentation results are as follows: Figure 2 As shown

[0100] 2. Point cloud dimensionality reduction

[0101] To achieve efficient and accurate extraction of assembly points, the segmented point cloud of the bottom surface with sleeves is projected onto the corresponding 3D bounding box bottom surface. Since the sleeves are embedded in the concrete column, and the assembly effect is influenced by both the outer surface assembly holes and the inner sleeve, the scanned point cloud of the sleeves is also projected onto the bottom surface. Due to occlusion, the scanning of the sleeves is uneven, and directly projecting the sleeve points onto the bottom surface leads to significant local differences in the point cloud density of the bottom surface assembly holes, which affects the fitting accuracy of the assembly points. Therefore, this invention proposes a point cloud dimensionality reduction method based on 3D bounding boxes and angle discrimination to solve the problem of uneven point cloud density formed during the dimensionality reduction process.

[0102] Point clouds with a distance of less than 2mm from the bottom surface of the 3D bounding box are selected as the base point clouds. The remaining point clouds are used as sleeve point clouds. First, the base point clouds are projected onto the bottom plane of the 3D bounding box, and then the subsequent sleeve point clouds are projected using this projected plane as the base. For each point in the sleeve point cloud, it is first projected onto the bottom plane along the axial direction obtained from the 3D bounding box, and then all bottom points within a 4mm radius circular area centered on this point are extracted. The sleeve projection points and the bottom projection points have the following four relationships on the projection plane, such as... Figure 3 As shown, a and b are defined as noise points, c as the sleeve boundary, and d as a point in the plane. The vector from the sleeve projection point to each selected bottom surface projection point is calculated. Using a given vector as a reference, the angles between the remaining vectors and that vector are calculated. After distinguishing directions and sorting by angle magnitude, the angle between every two adjacent vectors can be calculated. If all the angles are less than 120 degrees, the point is considered to fall on a repetitive region of the plane and is not projected. If there is an angle greater than 120 degrees, the point is considered to belong to the assembly hole boundary point cloud and is projected onto the bottom surface. If there are fewer than three bottom surface points around the sleeve point cloud projection point, the point is considered noise and is not projected. The principle and results of point cloud dimensionality reduction are as follows: Figure 3 As shown. Compared with traditional methods, the point cloud dimensionality reduction method described above can ensure the integrity of the point cloud information of the inner boundary of the sleeve while making the point cloud density around the assembly hole as uniform as possible, and is conducive to the accurate fitting of the assembly points.

[0103] In this embodiment, the boundary point cloud of the sleeve assembly hole is extracted from the projected bottom point cloud using an improved maximum angle method. Any point in the bottom point cloud is selected, and a radius r is taken centered on that point. d The vectors from all points within the area to the given point are used to calculate the angle between any two adjacent vectors clockwise. If all angles are greater than 20° and there is an angle greater than 140°, then the point is considered a boundary point. If there are angles less than 20° and greater than 140° simultaneously, then the two points of the vector with the smaller magnitude among the two vectors forming the smallest angle are selected, and boundary point determination is performed on each. The point whose maximum angle is closest to 180° is identified as the boundary point. In this invention, r is defined based on the scan point density. d =3mm.

[0104] For the extracted sleeve boundary points, this invention proposes a two-step center-fitting method based on uniform point selection and RANSAC to ensure the accuracy of the obtained assembly points. First, since the point density of the projected point cloud is kept as uniform as possible during dimensionality reduction, three points are taken at uniform intervals for each sleeve boundary to calculate the center. All uniform point-fitting results are calculated sequentially to obtain a uniform point-fitting center point set Se. While the center points in the center point set Se are relatively accurate, the number of points usable for fitting the assembly point is small. Therefore, the RANSAC algorithm is used to recalculate the center points of the sleeve boundary point cloud to obtain a more robust assembly point fitting result. Given that the uniform point-fitting method yields relatively accurate but insufficient center points, while the RANSAC method yields numerous but easily scattered center points, the RANSAC calculation point set is used as a reference point set to filter the RANSAC calculation point set, reducing the dispersion of the fitted points obtained by the RANSAC method, thereby reducing the number of iterations and improving computational efficiency. The average value of the obtained center-fitting points is used as the assembly point of the sleeve. The overall process is as follows: Figure 4 As shown.

[0105] Further, according to one embodiment of the present invention, the features and assembly points of the assembly component are registered with the corresponding data in the initial standard design model of the assembly component to obtain the deviation between the features and assembly points of the target assembly component and the initial standard design model of the assembly component, including:

[0106] The relationship between the assembly component features and the assembly point reference point X0(x0, y0, z0) and the corresponding point X(x, y, z) in the point cloud generated by the BIM model is as follows:

[0107] X0―(―λRX i +ΔX)=0;

[0108] Where R is the rotation matrix; X i For each point in the overall point cloud, ΔX is the translation vector, and λ is a scale factor with a value of 1.

[0109] When dealing with large-angle rotations in space, the Rodrigues matrix is ​​used for solution. First, based on the rotation vector s = [s...] x s y s z ] T Construct the antisymmetric matrix S:

[0110]

[0111] The antisymmetric matrix S can be transformed into a Rodrigues matrix R using the Rodrigues formula:

[0112] R = I + (sinθ)S + (1 - cosθ)S 2 ;

[0113] Where I is a 3rd order identity matrix, and θ is the rotation angle;

[0114] The error equation is constructed using the above parameters, and the correction value for the observation point is solved. The error (i.e., deviation) equation is as follows:

[0115] V1 = A1t + BX - L1;

[0116] Where V1 is the correction vector, A1 is the angle coefficient matrix, t is the transformation parameter correction vector, B is the coefficient matrix of the point to be solved, X is the correction value of the point to be solved, and L1 is the observation residual vector; the parameters are constructed as follows:

[0117] V1 = [V x V y V z ] T ;

[0118]

[0119] t=[dΔx dΔy dΔz da db dc] T ;

[0120] B = R ―1 ;

[0121] X = [dx0 dy0 dz0] T ;

[0122]

[0123] Furthermore, according to one embodiment of the present invention, deviation analysis is performed, and an optimal virtual assembly scheme is obtained based on the analysis results, including:

[0124] Analyze assembly points and component dimensional deviations;

[0125] Analyze the location and axial deviation of the rebar root joints;

[0126] The assemblability is checked under the combined effect of the assembly point location and component size deviation, as well as the rebar root node location and axial deviation. If assemblability is confirmed, the assembly deviation is determined.

[0127] Based on assembly deviations, the combination with the smallest overall assembly deviation RMSE of the entire assembly components is selected as the optimal virtual assembly scheme.

[0128] In this embodiment, the overall process of VTA for prefabricated frame structures includes deviation analysis of the assembly points, rebar root node points, and axial direction of the prefabricated components to verify their assemblability and calculate the optimal assembly scheme. If the detection is error-free, the components can be directly assembled. Otherwise, the components need to be adjusted based on specific feedback information. Optimal assembly analysis is performed for each component of the same type, and then multi-step assembly of the overall structure is carried out in the global coordinate system of the overall environmental point cloud.

[0129] In this embodiment, the analysis of assembly points and component dimensional deviations includes:

[0130] Using the vertices and edges of the precise 3D bounding box of the assembly components as registration features, registration of the scanned point cloud with the design model can be performed, such as... Figure 5 As shown, the distance difference between the corresponding assembly points extracted from the scanned point cloud and the design model is used as the assembly point deviation of the component. According to construction standards, a deviation within 3mm is considered a compliant assembly point. After confirming that all assembly points of a single assembly component are compliant, a 3D comparison with the design model is performed to confirm its dimensional deviation. The accuracy is then evaluated using the root mean square (RMS) value and root mean square error (RMSE), calculated as follows:

[0131]

[0132] In the assembly deviation analysis, X obs,i X is the calculated value for assembly deviation. model,i Calculated values ​​for manufacturing deviations.

[0133] In this embodiment, the analysis of the location and axial deviation of the rebar root node includes:

[0134] The overall point cloud of a single rebar is sliced ​​at equal intervals with a thickness of 1mm. Twenty point clouds are extracted from each rebar. The center of each individual point cloud slice is fitted to a circle, and the axial direction of the rebar is fitted using these 20 centers. Figure 5 As shown. The axial direction of each steel bar on the assembled component is compared with the overall axial direction of the component, and the included angle is calculated as the axial deviation. If the axial deviation exceeds the specified threshold c, it is considered to be axial deviation. ra This is considered non-compliant. This applies to assembly components of different specifications (c). ra The calculation formula is as follows:

[0135]

[0136] Among them, l r r is the length of the reinforcing bar. s r e Let t be the radius of the sleeve and the reinforcing bar. d To manufacture tolerances.

[0137] The intersection of the axial direction of the reinforcing bar and the top surface of the 3D boundary frame of the assembled component is taken as the root node of the reinforcing bar. The deviation analysis between the calculated root node and the design model is used to determine whether it is compliant.

[0138] In this embodiment, the assemblability is tested under the combined effects of assembly point location and component dimensional deviation, and rebar root node location and axial deviation:

[0139] The assemblability of a component is affected by the combined influence of deviations in the root node position of the reinforcing bars at the assembly location, axial deviations, and deviations in the sleeve assembly point of the assembled component. Therefore, after confirming that each individual manufacturing deviation of the component complies with regulations, it is necessary to inspect the assemblability under the combined effect of these deviations. If the root node position of the reinforcing bars after VTA is P r (x r ,y r ,z r ) and sleeve assembly point P s (x s ,y s ,z s If the distance difference between the sleeve radius and the rebar radius is less than the difference between the sleeve radius and the rebar radius, then calculate the rebar top node P. t (x t ,y t ,z t The projection point P on the assembly mating surface m (x m ,y m ,z m ). Calculate P m With P s Distance difference d a If d a If the value is also less than the difference between the sleeve radius and the rebar radius, then the point is considered to be successfully assembled, and d is used as the unit of measurement. a The formula for calculating assembly deviation is as follows:

[0140]

[0141] Among them, l r Let θ be the length of the reinforcing bar, θ be the axial deviation of the reinforcing bar, A, B, C, D be the plane equation parameters of the assembly joint surface, and k1, b1, k2, b2 be the axial equation parameters of the reinforcing bar.

[0142] In this embodiment, the optimal virtual assembly scheme is:

[0143] Prefabricated frame structures are typically assembled layer by layer, and the error of the Variable Assembly Scale (VTA) accumulates gradually with the increase in the number of layers. To minimize this accumulated error, optimal assembly analysis is required for the components. For each component of the same type in a group to be assembled, the assembly deviation at each assembly position within the corresponding group needs to be analyzed, and its RMSE is calculated as the accuracy evaluation standard. The assembly deviations of the entire group of assembled components are statistically analyzed, and the combination with the smallest overall RMSE is selected as the optimal assembly scheme. After VTA, the assembly position, elevation, and verticality of the components are checked to verify the VTA accuracy and guide the final physical assembly.

[0144] Furthermore, to achieve the above objectives, the present invention also provides a VTA system for a steel-concrete frame structure based on ground-based LiDAR point clouds, comprising:

[0145] The data acquisition module collects data from the target assembly components using a ground-based 3D lidar, and processes the data to create a dataset.

[0146] The semantic information and 3D bounding box acquisition module takes the dataset as input to the SSBR-Net network model and outputs semantic information and 3D bounding boxes.

[0147] The feature and point acquisition module extracts assembly component features from the 3D bounding box and then calculates the assembly points based on the 3D bounding box.

[0148] The deviation calculation module registers the assembly component features and assembly points with the corresponding data in the initial standard design model of the assembly component to obtain the deviation between the assembly component features and assembly points of the target assembly component and the initial standard design model of the assembly component.

[0149] The VTA acquisition module analyzes the deviation and obtains the optimal virtual assembly scheme based on the analysis results.

[0150] The ground-based LiDAR point cloud-based VTA system for steel-concrete frame structures of the present invention can realize the above-mentioned ground-based LiDAR point cloud-based VTA method for steel-concrete frame structures. The specific steps are as described above and will not be repeated here.

[0151] Furthermore, to achieve the above objectives, the present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the VTA method for steel-concrete frame structures based on ground-based LiDAR point clouds as described above.

[0152] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the VTA method for steel-concrete frame structures based on ground-based LiDAR point clouds as described above.

[0153] Furthermore, based on the above-described solution of the present invention, the effectiveness of the above-described solution of the present invention will be verified below with reference to the accompanying drawings in a specific embodiment.

[0154] Example 1

[0155] The method proposed in this embodiment was applied to the construction of a smart logistics port on a highway to verify its feasibility. The main building of the logistics port is constructed of a prefabricated frame structure. The main building has five floors and contains 184 prefabricated reinforced concrete columns of three different seismic strengths and sizes, all connected by steel bars and sleeves. This embodiment selects two representative assembly locations as examples for result analysis. The selected scenarios include eight assembly components of three different sizes and their corresponding assembly locations.

[0156] In this embodiment, the developed model SSBR-Net was trained on a workstation with performance including an i9-13900K CPU and a 24GB GeForce RTX4090 GPU. To prevent overfitting, the model was initialized using an AdamW optimizer with a learning rate of 10⁻⁵, and the batch size was set to 32 for training up to 100 epochs. The results were then evaluated.

[0157] A high-precision ground-based 3D LiDAR was used to collect point cloud data of the overall construction scene. High-resolution scanning was used to focus on the assembled components, while a lower-resolution scan of the overall construction environment was used to connect the point clouds of the entire scene. After obtaining the global point cloud in the global coordinate system, SSBR-Net was used for semantic segmentation and 3D bounding box estimation of the assembled components. The results are as follows: Figure 6 As shown in the figure, SSBR-Net can achieve fast and accurate semantic segmentation of assembly components in large and complex scenes. It can also estimate the 3D bounding boxes and axial directions of the components based on the semantic segmentation results, and classify assembly components of three types (C30, C40, and C50) according to the bounding box size. The Intersection over Union (IOU) and Mean Intersection over Union (mIOU) are used as evaluation criteria for segmentation accuracy, and their calculation formulas are as follows. The segmentation and recognition accuracy of each component is shown in Table 1.

[0158]

[0159] In this context, TP represents true positives, FP represents false positives, FN represents false negatives, and TN represents true negatives.

[0160] The accuracy analysis of segmentation and recognition of each component is shown in Table 1:

[0161]

[0162] Table 1

[0163] By comparing the design model of the assembled component with the 3D bounding box of the scanned point cloud, various deviation information of the component can be obtained. To accurately calculate the manufacturing deviations of the assembled component, further calculation of a precise 3D bounding box is required. Based on the precise 3D bounding box, the required assembly features can be calculated, including the overall axial direction of the assembled component, the assembly points on the bottom surface, the axial direction of the reinforcing bars, and the root node points of the reinforcing bars. Therefore, the component point cloud can be abstracted into a 3D bounding box and various assembly feature representations, resulting in a simplified component assembly feature model, which facilitates the analysis of various deviations of the assembled component. The results are as follows: Figure 7 As shown.

[0164] This embodiment performs a deviation analysis on the assembled components according to the construction plan requirements, covering five aspects: concrete column dimensions, sleeve assembly points, rebar length, axial direction, and rebar assembly points. The deviation analysis is completed by comparing the component's scanned point cloud with the assembly feature models extracted from the design model. The results are as follows: Figure 8 As shown, the dimensions of the concrete column, the sleeve assembly points, and the rebar assembly points did not exceed the limits. However, the rebar length exceeded the standard value in 2 places, and the axial length of the rebar exceeded the specified tolerance in 14 places.

[0165] The components, corrected according to the deviation analysis results, are assembled at their corresponding assembly positions. For the three types of components of different models, the calculation methods for matching assembly points differ slightly due to differences in their dimensions and sleeve arrangement. To obtain the result with the minimum assembly deviation, it is necessary to calculate all assembly schemes and determine the optimal assembly method. The assembly points of the sleeves for the C30 model are arranged in a centrally symmetrical shape, so a single component can be assembled in all four positive directions of its corresponding assembly position. However, the sleeves for the C40 and C50 models do not have a symmetrical relationship, and there is only a single correspondence between a single component and its assembly position. In this embodiment, two assembly scenarios, CJ1 and CJ2, are selected to assemble the three types of components respectively. CJ1 can assemble four C30 components, and CJ2 can assemble one C40 and three C50 components. Taking CJ1 as an example, an assembly deviation analysis is performed. For each matching scheme, the deviation of each assembly point is calculated, and its mean, standard deviation, RMS, and RMSE are statistically analyzed as evaluation standards for assembly accuracy. The results are as follows: Figure 9As shown. Of the 64 matching schemes, 9 were discarded due to excessive assembly deviations. Optimal assembly analysis was performed on the compliant schemes, calculating the matching combination with the minimum sum of all evaluation indicators. The optimal assembly schemes for CJ1 were determined to be C30-1 paired with CJ1-3, C30-2 paired with CJ1-1, C30-3 paired with CJ1-2, and C30-4 paired with CJ1-4. Based on the assembly deviation analysis results, the result with the minimum overall assembly deviation was calculated as the optimal assembly scheme for component VTA. The VTA results for CJ1 and CJ2 are shown below. Figure 10 As shown in Table 2, the assembly position, elevation, and verticality of the components after VTA were checked, and the results are shown in Table 2. All inspection results meet the standards, proving that the VTA method in this embodiment is accurate and feasible, and can effectively guide component assembly.

[0166]

[0167] Table 2

[0168] According to the scheme in this embodiment, this embodiment first proposes a deep learning method for extracting assembly components in large scenes based on semantic segmentation and voting mechanism. This method achieves accurate and efficient extraction of the required component point clouds from large and complex construction scene point clouds and performs preliminary 3D bounding box estimation. The mIOU of component segmentation is 0.9429, indicating high segmentation accuracy. The proposed SSBR-Net can accurately segment assembly components.

[0169] Based on the semantic segmentation results of SSBR-Net, and the estimation of component 3D bounding boxes and orientations, a high-precision assembly position localization method based on 3D bounding boxes is proposed. First, the precise 3D bounding box of the component is calculated based on the initially estimated 3D bounding box and orientation, and a point cloud dimensionality reduction method based on this 3D bounding box is designed. Compared with PCA-based dimensionality reduction methods, this method can provide a more accurate projection plane and ensure uniform point cloud density and complete feature point retention after dimensionality reduction, which is more conducive to the accurate calculation of assembly features. Various assembly features of the component are calculated based on this 3D bounding box, and the component point cloud is abstracted into an assembly feature model representation. This greatly reduces data redundancy, expresses the assembly component in a more intuitive way, and improves the efficiency of subsequent deviation analysis and VTA scheme analysis.

[0170] Based on the assembly characteristics, various deviation analyses were performed on the components. The results showed that there were no manufacturing deviations in the concrete column sections of all components, and the assembly points were relatively accurate. There were slight deviations in the length of the reinforcing bars, and larger deviations in the axial direction of the reinforcing bars. Considering the good overall processing accuracy of the components, the deviation in the axial direction of the reinforcing bars was likely due to collisions during transportation. The C30-4 component (exceeding the length limit) also exhibited significant axial deviations in the No. 6 reinforcing bar, and the C40-1 component (No. 14 reinforcing bar), indicating a high probability of manufacturing errors at these two points.

[0171] This embodiment performed a global coordinate VTA analysis on the manufactured qualified components, calculating all possible assembly schemes for each assembly position to determine the optimal assembly scheme. Of the 64 matching schemes for CJ1, 9 exceeded the limits, with 8 of these occurring in the matching with CJ1-4, indicating that the connecting steel bars of CJ1-4 may be subject to collisions. It is recommended to correct these collisions before assembly. The assembly deviation analysis of the components after VTA showed that none exceeded the tolerance, proving that the VTA method proposed in this embodiment has a small cumulative error and high accuracy. Furthermore, the VTA method proposed in this embodiment performs assembly in a global coordinate system, enabling independent VTA analysis of a single group of components without requiring the acquisition of data for all assembled components at once, saving space costs in actual assembly. Traditional VTA methods, on the other hand, are typically performed in a relative coordinate system, requiring the acquisition of all data for a comprehensive VTA to ensure accuracy.

[0172] Those skilled in the art will recognize that the modules and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented using electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0173] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described apparatus and equipment can be referred to the corresponding process in the foregoing method implementation, and will not be repeated here.

[0174] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0175] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the objectives of the embodiments of the present invention, depending on actual needs.

[0176] In addition, the functional modules in the embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0177] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the energy-saving signal transmission / reception methods of various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0178] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in this application.

[0179] It should be understood that the sequence number of each step in the invention and its embodiments does not absolutely imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

Claims

1. A method for VTA of steel-concrete frame structures based on ground-based LiDAR point clouds, characterized in that, The application relates to a method for obtaining a virtual assembly scheme of a target assembly component. Data of the target assembly component is collected by a ground three-dimensional laser radar, and the data is processed to generate a data set; The data set is input into an SSBR-Net network model to output semantic information and a 3D bounding box; Assembly component features are extracted from the 3D bounding box, and assembly points are calculated based on the 3D bounding box; The assembly component features and the assembly points are matched with corresponding data in an initial standard design model of the assembly component to obtain deviations between the assembly component features and the assembly points of the target assembly component and the initial standard design model of the assembly component; The deviations are analyzed, and an optimal virtual assembly scheme is obtained based on the analysis result; The SSBR-Net network model is composed of a RandLaNet semantic segmentation network and a BRNet network with a SAM spatial attention mechanism; The assembly component features are extracted from the 3D bounding box, and the assembly points are calculated based on the 3D bounding box, which comprises the following steps: The planes of the 3D bounding box of the target assembly component are embedded and meshed to determine candidate points for plane fitting of assembly component point clouds; The candidate points are used to fit accurate projection planes of each surface to determine accurate 3D bounding boxes, and the point clouds of each surface are divided according to the 3D bounding boxes, and then the point clouds of the bottom surface are dimensionally reduced; The intersection of the steel bar axial direction and the top surface plane of the assembly component is calculated as the top surface assembly point, and the boundary of the 3D bounding box is taken as the boundary feature of the assembly component; The maximum angle method is used to extract the sleeve boundary of the bottom surface as the assembly position on the dimensionally reduced bottom surface point cloud, and the RANSAC method is used to fit the center of the sleeve boundary, and the result is taken as the bottom surface assembly point of the assembly component.

2. The ground-based LiDAR point cloud based reinforced concrete frame structure VTA method according to claim 1, wherein, The data of the target assembly component is collected by a ground three-dimensional laser radar, and the data is processed to generate a data set; The point clouds of the target assembly component placed on the ground and the overall point clouds of the construction scene are collected by the ground three-dimensional laser radar.

3. The ground-based LiDAR point cloud based reinforced concrete frame structure VTA method of claim 1, wherein, The RANSAC method is used to fit the center of the sleeve boundary, and the result is taken as the bottom surface assembly point of the assembly component, which comprises the following steps: Three points are taken at uniform intervals for each sleeve boundary to calculate the center, and the fitting results of all the uniformly taken points are sequentially calculated to obtain a set of uniformly taken center points Se; The RANSAC algorithm is used to calculate the center fitting point of the sleeve boundary point cloud; The set of center fitting points obtained by the RANSAC method is screened with the set of center points Se as a reference point set to reduce the dispersion of the center fitting points; The mean value of the finally obtained center fitting points is taken as the bottom surface assembly point of the assembly component.

4. The ground-based LiDAR point cloud based reinforced concrete frame structure VTA method of claim 1, wherein, The assembly component features and the assembly points are matched with corresponding data in an initial standard design model of the assembly component to obtain deviations between the assembly component features and the assembly points of the target assembly component and the initial standard design model of the assembly component, which comprises the following steps: Assembly component features and assembly point reference points ( , , (and corresponding points in the point cloud generated from the BIM model) The relationship between them is as follows: wherein, is a rotation matrix; is each point of the iteration in the overall point cloud, is a translation vector, is a scale factor with value 1; Rodrigues matrix is used to solve the large angle rotation in the processing space, first according to the rotation vector Constructing skew-symmetric matrix : Using the Rodrigues' formula, an anti-symmetric matrix can be converted to a Rodrigues' matrix ​ wherein is a 3rd order identity matrix, is a rotation angle; An error equation is constructed by using the above parameters, and an observation point correction value is solved, and the error equation is as follows: wherein is a correction vector of transformation parameters, is an angle coefficient matrix, is a correction vector of transformation parameters, is a coefficient matrix of a point to be solved, is a correction value of a point to be solved, is an observation residual vector; each parameter is constructed as follows: 。 5. The ground-based LiDAR point cloud based reinforced concrete frame structure VTA method according to any one of claims 1-4, wherein, The deviations are analyzed, and an optimal virtual assembly scheme is obtained based on the analysis result, which comprises the following steps: The assembly point and the component size deviations are analyzed; The steel bar root node point and the axial deviations are analyzed; The assembly point and the component size deviations and the steel bar root node point and the axial deviations are tested for assemblability, and the assembly deviations are determined when the assemblability is confirmed; Based on the assembly deviation, the combination of the whole assembly component with the minimum RMSE of the overall assembly deviation is selected as the best virtual assembly scheme.

6. A steel-concrete frame structure VTA system based on ground-based LiDAR point cloud, characterized by, The method comprises the following steps: a data acquisition module acquires data of the target assembly component through a ground-based three-dimensional laser radar, and processes the data to generate a data set; a semantic information and 3D bounding box acquisition module inputs the data set into an SSBR-Net network model, and outputs semantic information and a 3D bounding box; a feature and point position acquisition module extracts assembly component features from the 3D bounding box, and then calculates assembly point positions according to the 3D bounding box; a deviation calculation module registers the assembly component features and the assembly point positions with corresponding data in the initial standard design model of the assembly component, and obtains the deviations between the assembly component features and the assembly point positions of the target assembly component and the initial standard design model of the assembly component; a VTA acquisition module analyzes the deviations, and obtains the best virtual assembly scheme according to the analysis result; the SSBR-Net network model is composed of a RandLaNet semantic segmentation network and a BRNet network with a SAM spatial attention mechanism; the assembly component feature extraction from the 3D bounding box and the calculation of the assembly point positions from the 3D bounding box comprise the following steps: embedded grid division is performed on each plane of the 3D bounding box of the target assembly component to determine candidate points for plane fitting of assembly component point clouds on each plane; an accurate projection plane of each plane is fitted by using the candidate points to determine an accurate 3D bounding box, and the point clouds on each plane are divided according to the 3D bounding box, and then the point clouds on the bottom surface are dimensionally reduced; the intersection of the steel bar axial direction of the assembly component and the top surface plane is calculated as the top surface assembly point, and the boundary of the 3D bounding box is used as the boundary feature of the assembly component; the maximum angle method is used to extract the sleeve boundary of the bottom surface as the assembly position on the dimensionally reduced bottom surface point clouds, and the RANSAC method is used to fit the center of the sleeve boundary, and the result is used as the bottom surface assembly point position of the assembly component.

7. An electronic device, characterized by The computer program is stored on the computer readable storage medium and is executed by the processor to implement the VTA method for a steel-concrete frame structure based on ground-based LiDAR point clouds according to any one of claims 1-5.

8. A computer readable storage medium, characterized in that, The computer program is stored on the computer readable storage medium and is executed by the processor to implement the VTA method for a steel-concrete frame structure based on ground-based LiDAR point clouds according to any one of claims 1-5.

Images