A Method for Evaluating the Paving Precision of Prefabricated Pavements Based on a Depth Camera

Abstract

The present invention relates to a method for evaluating the paving accuracy of prefabricated pavements based on a depth camera, including evaluation in the design stage, prefabrication stage, construction stage, and acceptance stage. Among them, the evaluation in the prefabrication stage includes: obtaining the original point cloud, and through preprocessing, point cloud segmentation, feature extraction, and three-dimensional reconstruction, obtaining the reconstructed model of the prefabricated slab. Combining the feature points of the embedded components with the design scheme, checking the three-dimensional dimensions of the slab and the positions of the embedded components, and performing virtual assembly to detect possible assembly interferences at the joint structures. If there are assembly interferences, the reconstructed model is corrected or the prefabricated slab is remade. Compared with the prior art, the present invention has the advantages of improving the manufacturing efficiency during the construction of prefabricated pavements and reducing design errors, etc.

Description

Technical Field

[0001] The present invention relates to the field of road engineering, and in particular to a method for evaluating the paving accuracy of assembled pavements based on a depth camera. Background Art

[0002] The assembled pavement is an excellent rapid paving structure, which has the characteristics of factory production, standardization, mechanization, etc., and is green, environmentally friendly, fast in construction speed, and high in engineering quality. It is an important paving form for new construction, reconstruction, and expansion. The assembled pavement can effectively alleviate the contradiction between the long setting and hardening time of cement concrete and the short constructible time in pavement repair.

[0003] The three-dimensional dimensional accuracy of the assembled paving slab not only directly affects the engineering quality at each stage, but also determines whether the on-site assembly is feasible and controllable. In the prefabrication stage, due to the influence of factors such as mold deformation, temperature shrinkage, and dry shrinkage, the three-dimensional dimensions of the precast slabs will inevitably deviate, and precise control is required to lay a foundation for subsequent high-quality assembly. In the construction stage, facing the area to be assembled, it is necessary to evaluate its spatial matching degree with the precast slabs to avoid rework caused by geometric conflicts between the slabs and the area to be assembled. Therefore, throughout the construction process of the assembled pavement, checking the deviation of the three-dimensional dimensions of the precast slabs, evaluating the dimensional matching of the area to be assembled and the precast slabs, and ensuring the paving accuracy of the precast slabs are the key links to ensure the project progress and quality of the assembled pavement. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for evaluating the paving accuracy of assembled pavements based on a depth camera, solve the problems of comprehensiveness and poor accuracy of measurement data in the construction process of assembled pavements, and improve the accuracy and efficiency of the construction of assembled pavements.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] A method for evaluating the paving accuracy of assembled pavements based on a depth camera includes evaluations in the design stage, prefabrication stage, construction stage, and acceptance stage. Among them,

[0007] The evaluation in the design stage includes the following steps:

[0008] Step 1-1) Obtain the original point cloud of the existing road surface in the repair area of the assembled pavement collected by the depth camera.

[0009] Step 1-2) Preprocess the original point cloud of the existing road surface in the repair area of the assembled pavement.

[0010] Step 1-3) Based on the point cloud data of the existing road surface in the repair area of the assembled pavement after preprocessing, determine the plane position and elevation data of the repair area through coordinate transformation.

[0011] The prefabrication stage evaluation includes the following steps:

[0012] Step 2-1) Obtain the original point clouds of each surface of the prefabricated slab for the assembled pavement and the embedded components collected successively by the depth camera,

[0013] Step 2-2) Preprocess the original point clouds of each surface of the prefabricated slab for the assembled pavement and the embedded components to obtain the point cloud data of the prefabricated slab;

[0014] Step 2-3) Perform point cloud segmentation, feature extraction, and 3D reconstruction on the point cloud data of the prefabricated slab to obtain the feature points of the embedded components and the reconstructed model of the prefabricated slab,

[0015] Step 2-4) Based on the feature points of the embedded components and the design scheme, check the 3D dimensions of the reconstructed model of the prefabricated slab and the positions of the embedded components,

[0016] Step 2-5) Based on virtual assembly technology, perform virtual assembly on the verified reconstructed model, detect and judge whether there is assembly interference at the joint structure. If there is assembly interference, correct the reconstructed model of the prefabricated slab or remanufacture the prefabricated slab, and re-execute steps 2-1)-2-5). If there is no assembly interference, output the reconstructed model of the prefabricated slab;

[0017] The construction stage evaluation includes the following steps:

[0018] Step 3-1) Obtain the original point cloud of the area to be assembled of the assembled pavement collected by the depth camera,

[0019] Step 3-2) Preprocess the original point cloud of the area to be assembled of the assembled pavement to obtain the point cloud data of the area to be assembled,

[0020] Step 3-3) Perform point cloud segmentation, feature extraction, and 3D reconstruction on the point cloud data of the area to be assembled to obtain the 3D model of the existing slab and the 3D model of the base course,

[0021] Step 3-4) Combine the reconstructed model of the prefabricated slab, and based on virtual assembly technology, detect and judge whether there is interference in the area to be assembled. If there is no interference, perform the installation operation of the slab. If there is interference, remanufacture the prefabricated slab;

[0022] The acceptance stage evaluation includes the following steps:

[0023] Step 4-1) Obtain the original point cloud of the joint of the assembled pavement slab collected by the depth camera,

[0024] Step 4-2) Preprocess the original point cloud of the joint of the assembled pavement slab,

[0025] Step 4-3) Identify the joint position based on the preprocessed point cloud data of the joints of the prefabricated pavement slabs, obtain the joint point cloud and the point clouds of the slabs on both sides of the joint, and complete the evaluation of the paving accuracy of the prefabricated pavement slabs based on the joint point cloud and the point clouds of the slabs on both sides of the joint.

[0026] The process of preprocessing the original point cloud includes denoising, background point removal, point cloud reduction, and point cloud stitching and registration.

[0027] The point cloud segmentation of the point cloud data of the precast slab includes the following steps:

[0028] Step 2-3-1) Based on the input point cloud data P of the precast slab, construct a kd-tree, set a set C to store various point cloud clusters, and use a queue Q to store the points to be classified

[0029] Step 2-3-2) Obtain the points in the point cloud data P of the precast slab and determine whether they have been classified. If not, execute Step 2-3-3). If they have been classified, obtain the next point p i+1 And re-execute Step 2-3-3);

[0030] Step 2-3-3) Add the point p i to the current queue Q of points to be classified;

[0031] Step 2-3-4) For each point p j ∈Q in the queue Q, use the kd-tree to search for the radius neighborhood j with a distance less than r 0 from p And add the unprocessed points in to the queue Q, and classify the point cloud in the queue Q to obtain point cloud clusters;

[0032] Step 2-3-5) Determine whether all points in P have been processed. If so, store the classified point cloud clusters in the queue Q into the set C, clear the queue Q, and complete the point cloud segmentation. If not, return to Step 2-3-2).

[0033] The point cloud segmentation of the point cloud data of the proposed assembly area includes the following steps:

[0034] Step 3-3-1) Obtain the position of the horizontal plane;

[0035] Step 3-3-2) Use the least squares method to fit the plane where the entire point cloud data of the proposed assembly area is located;

[0036] Step 3-3-3) Perform coordinate transformation on the point cloud data of the proposed assembly area based on the plane parameters obtained by fitting, so that the horizontal plane is parallel to the XOY plane of the point cloud coordinate axis, and determine the depth value of the point cloud from the horizontal plane based on the Z-axis coordinate value of the point cloud data;

[0037] Step 3-3-4): Implement point cloud segmentation based on the depth values of the point cloud data to obtain the precast slab point cloud and the base course point cloud.

[0038] Feature extraction of the precast slab point cloud data includes three parts: extraction of plane parameters, extraction of boundaries, and simplification of boundaries.

[0039] The extraction of the plane parameters uses the RANSAC algorithm and includes the following steps:

[0040] Step 2-3-6): Determine the parametric equation of the precast slab plane in three-dimensional space:

[0041] ax + by + cz + d = 0

[0042] where (x, y, z) are the coordinates of the point cloud located in the precast slab plane, and a, b, c, and d are all estimated parameters;

[0043] Step 2-3-7): Randomly select three points (x 1 , x 1 , x 1 ), (x 2 , x 2 , x 2 ), and (x 3 , x 3 , x 3 ) in the point cloud. Substitute the coordinates of the three points into the plane equation in turn, and use Cramer's rule to solve the linear equations to obtain the estimated parameters of the plane where the current point is located as:

[0044]

[0045] Step 2-3-8): Calculate the distance between the points in the point cloud and the estimated plane of the precast slab:

[0046]

[0047] Step 2-3-9): Determine whether the distance d i is less than the pre-configured distance threshold δ 0 . If so, classify the point as an inlier; if not, classify the point as an outlier.

[0048] Step 2-3-10): Repeat steps 2-3-8) - 2-3-9) until all the point cloud is traversed, and count the number of inliers under the current estimated parameters;

[0049] Step 2-3-11) Repeat steps 2-3-7) to 2-3-10) until the iteration termination condition is reached. Record the estimated parameters corresponding to the maximum number of inliers, and obtain the optimal plane parameters. The iteration termination condition is that the number of inliers reaches the preconfigured quantity or the maximum number of iterations is reached. The maximum number of iterations k is:

[0050]

[0051] where t = number of inliers / total number of point clouds, and P is the probability of obtaining the optimal plane parameters.

[0052] The extraction of the boundary includes two parts: the projective transformation of the point cloud and the convex hull calculation.

[0053] The projective transformation of the point cloud includes the following steps:

[0054] Step 2-3-12) Take the normal vector direction of the plane fitted by the precast slab as the Z-axis direction of the new coordinate system, keep the X-axis direction unchanged, determine the basis matrix of the new coordinate system, and transform the coordinates of the precast slab point cloud based on the basis matrix. Let the unit normal vector of the plane fitted by the precast slab be The unit vector in the positive direction of the X-axis of the original coordinate system is Then the basis matrix of the new coordinate system is:

[0055]

[0056] Any point p = (x p , y p , z p ) in the point cloud, and its coordinate value p' = (x' p , y' p , z' p ) in the new coordinate system is:

[0057] (x' p , y' p , z' p ) = (A -1 · (x p , y p , z p ) T )

[0058] Step 2-3-13) Remove the outlier point clouds whose distance from the plane fitted by the precast slab is greater than the preconfigured threshold δ 0 ;

[0059] Step 2-3-14) Project the point cloud data after removing the outlier point clouds onto the XOY plane of the new coordinate system to complete the projective transformation of the point cloud.

[0060] The paving accuracy evaluation of the prefabricated pavement slab includes the evaluation of joint step difference and joint width. Among them, the evaluation of joint step difference is based on the point cloud of the slabs on both sides of the joint, and the evaluation of joint width is based on the point cloud of the joint.

[0061] The evaluation of joint step difference includes the following steps:

[0062] Determine the pixel coordinates of the joint position according to the joint position recognition result, and use the RANSAC algorithm to fit the joint straight line;

[0063] Translate the fitted joint straight line upward and downward along its perpendicular direction by a pre-configured pixel distance to obtain the joint neighborhood, and the two sides of the joint neighborhood are the slab areas;

[0064] According to the pixel indices of the slab areas, extract the point cloud data of the slabs from the structured point cloud, use joint bilateral filtering and radius filtering to denoise the slab point cloud, and use the grid method for thinning to obtain the point cloud data of the slabs on both sides of the joint;

[0065] Calculate the step difference based on the point cloud data of the slabs on both sides of the joint.

[0066] Compared with the prior art, the present invention has the following beneficial effects:

[0067] (1) The present invention solves the technical gap in the prior art, obtains the point cloud data and 3D reconstruction model based on the prefabricated pavement, provides a 3D dimension checking and evaluation technology for each construction stage of the prefabricated pavement, solves the problems of comprehensiveness and poor accuracy of the measurement data in the construction process of the prefabricated pavement, and improves the construction accuracy and efficiency of the prefabricated pavement.

[0068] (2) In the prefabrication stage, the present invention uses the point cloud data of the slabs to realize the digital checking of the prefabrication accuracy of the prefabricated pavement slabs. Compare the boundary point cloud of the prefabricated slab after coordinate transformation with the design dimensions to comprehensively evaluate the prefabrication errors of the slab plane dimensions and thickness; and check the embedded position of the component; through point cloud clustering and projection transformation, complete the identification of the dowel bar groove and embedded dowel bar, and check the layout positions of the two in the prefabricated slab.

[0069] (3) During the construction stage, the present invention clarifies the key steps of the virtual assembly technology for prefabricated pavement and the types of assembly interference. For precast slabs, a method for setting assembly constraints is proposed, and the detection of slab assembly interference is completed in the verification module of FreeCAD. For the assembly area, the establishment process of assembly constraint relationships is simplified through virtual assembly frames and virtual assembly surfaces, and the detection of assembly interference between the existing pavement and the top surface of the base layer and precast slabs is realized. Finally, the virtual assembly process for prefabricated pavement is summarized and divided into two stages: the detection of precast slab assembly interference and the virtual assembly detection of the proposed assembly area.

[0070] (4) During the acceptance stage, the present invention studies the digital detection methods for the slab offset and joint width, realizes the evaluation of the paving accuracy of precast slabs for prefabricated pavement, and based on camera images, proposes an identification method for joint positions based on color distance and gray projection, removing the influence of debris such as slab grooves and gravel on the identification results. Using the plane fitting method, the calculation of the joint offset is realized, and the calculation results can better reflect the spatial position relationship of the slabs. Through coordinate transformation and line fitting, the detection and evaluation of the joint width are completed. Description of the Drawings

[0071] Figure 1 is the flowchart of the method of the present invention. Detailed Embodiments

[0072] The present invention will be described in detail below with reference to the drawings and specific embodiments. This embodiment is implemented on the premise of the technical solution of the present invention, and the detailed implementation manners and specific operation processes are given, but the protection scope of the present invention is not limited to the following embodiments.

[0073] A method for evaluating the paving accuracy of prefabricated pavement based on a depth camera, as Figure 1 shown, includes the evaluation in the design stage, the evaluation in the prefabrication stage, the evaluation in the construction stage, and the evaluation in the acceptance stage.

[0074] ① Evaluation in the design stage

[0075] The evaluation in the design stage includes the following steps:

[0076] Step 1-1) Obtain the original point cloud of the existing pavement within the repair area of the prefabricated pavement collected by the depth camera,

[0077] Step 1-2) Preprocess the original point cloud of the existing pavement within the repair area of the prefabricated pavement,

[0078] Step 1-3) Based on the point cloud data of the existing pavement within the repair area of the prefabricated pavement after preprocessing, determine the plane position and elevation data of the repair area through coordinate transformation.

[0079] The point cloud data of the existing road surface is used as the basis for designing the repair plan, without the need for feature extraction and 3D reconstruction of the point cloud.

[0080] ②Prefabrication stage evaluation

[0081] The prefabrication phase evaluation includes the following steps:

[0082] Step 2-1) obtaining original point clouds of each surface of the prefabricated slab of the assembled pavement and the embedded components collected in sequence by the depth camera;

[0083] Step 2-2) preprocessing the original point clouds of each surface and embedded components of the prefabricated pavement panel to obtain point cloud data of the prefabricated panel;

[0084] The process of preprocessing the original point cloud includes denoising, background point removal, point cloud simplification and point cloud stitching and registration.

[0085] The denoising, background point removal, point cloud simplification and point cloud stitching and registration processes are conventional settings in the field and will not be described in detail here to avoid blurring the purpose of this application.

[0086] Step 2-3) performing point cloud segmentation, feature extraction and three-dimensional reconstruction on the point cloud data of the prefabricated plate to obtain feature points of the embedded components and a reconstructed model of the prefabricated plate;

[0087] Point cloud segmentation of prefabricated plate point cloud data includes the following steps:

[0088] Step 2-3-1) Based on the input prefabricated plate point cloud data P, construct a kd tree, set a set C to store various point cloud clusters, and use a queue Q to store the points to be classified

[0089] Step 2-3-2) Obtain a point in the prefabricated plate point cloud data P and determine whether it has been classified. If it has not been classified, execute step 2-3-3). If it has been classified, obtain the next point p. i+1 And re-execute step 2-3-3);

[0090] Step 2-3-3) Change point p i Add to the current queue Q of points to be classified;

[0091] Step 2-3-4) For each point p in the queue Q j ∈Q, using kd tree search and p j Distance less than r 0 The radius neighborhood and will The unprocessed points are added to the queue Q, and the point cloud in the queue Q is classified to obtain the point cloud cluster;

[0092] Step 2-3-5) Determine whether all points in P have been processed. If so, store the point cloud clusters that have been classified in queue Q into set C, empty queue Q, and complete point cloud segmentation. If not, return to Step 2-3-2).

[0093] Feature extraction of the precast block point cloud data includes three parts: 1° extraction of plane parameters, 2° extraction of boundaries, and 3° simplification of boundaries.

[0094] The extraction of 1° plane parameters uses the RANSAC algorithm, including the following steps:

[0095] Step 2-3-6) Determine the parametric equation of the precast block plane in three-dimensional space:

[0096] ax + by + cz + d = 0

[0097] where (x, y, z) are the coordinates of the point cloud within the precast block plane, and a, b, c, and d are all estimated parameters;

[0098] Step 2-3-7) Randomly select three points (x 1 , x 1 , x 1 )、(x 2 , x 2 , x 2 ) and (x 3 , x 3 , x 3 ) in the point cloud. Substitute the coordinates of the three points into the plane equation in turn, and use Cramer's rule to solve the linear equation system to obtain the estimated parameters of the plane where the current point is located as:

[0099]

[0100] Step 2-3-8) Calculate the distance between the points in the point cloud and the estimated plane of the precast block:

[0101]

[0102] Step 2-3-9) Determine whether the distance d i is less than the pre-configured distance threshold δ 0 . If so, classify the point as an inlier; if not, classify the point as an outlier;

[0103] Step 2-3-10) Repeat Step 2-3-8) - Step 2-3-9) until all the point cloud is traversed, and count the number of inliers under the current estimated parameters;

[0104] Step 2-3-11): Repeat steps 2-3-7) to 2-3-10) until the iteration termination condition is reached. Record the estimated parameters corresponding to the maximum number of inliers, and obtain the optimal plane parameters. Here, the iteration termination condition is that the number of inliers reaches a pre-configured quantity or the maximum number of iterations is reached.

[0105] The maximum number of iterations k of the RANSAC algorithm can be estimated from a probabilistic perspective. Assume that in the optimal estimated plane, the proportion of inliers in the entire point cloud is t = number of inliers / total number of points in the point cloud. At least three independent points are required to estimate the plane parameters. Random sampling ensures that these three points are independently selected. Then the probability that at least one of the three points is an outlier is 1 - t. 3 。

[0106] Therefore, the probability of obtaining the correct model after k iterations is P = 1 - (1 - t 3 ) k . Taking the logarithm of it can obtain the maximum number of iterations k. Considering the possibility that each point may be repeatedly sampled during the iteration, a standard deviation term is added to the result. In the case of having a probability P of obtaining the optimal plane parameters, the maximum number of iterations of the RANSAC algorithm is:

[0107]

[0108] where t = number of inliers / total number of points in the point cloud, and P is the probability of obtaining the optimal plane parameters.

[0109] The proportion t of the number of inliers is a prior value. Since the number of outliers in the point cloud cannot be determined clearly before estimating the plane parameters, the maximum number of iterations cannot be directly determined. The present invention adopts an adaptive number of iterations. At the beginning of the algorithm, the maximum number of iterations k is set to infinity. Each time the plane parameters are updated, the ratio of the number of inliers under the current parameters is used as t and substituted into the above formula to update the maximum number of iterations.

[0110] 2° The extraction of the boundary includes two parts: the projective transformation of the point cloud and the convex hull calculation.

[0111] The projective transformation of the point cloud includes the following steps:

[0112] Step 2-3-12): Take the normal vector direction of the plane fitted by the precast slab as the direction of the Z-axis of the new coordinate system, keep the X-axis direction unchanged, determine the base matrix of the new coordinate system, and transform the coordinates of the precast slab point cloud based on the base matrix. Here, let the unit normal vector of the plane fitted by the precast slab be The unit vector in the positive direction of the X-axis of the original coordinate system is Then the base matrix of the new coordinate system is:

[0113]

[0114] Any point p=(x p , y p , z p ) in the point cloud has the coordinate value p'=(x' p , y' p , z' p ) in the new coordinate system as follows:

[0115] (x' p , y' p , z' p )=(A -1 ·(x p , y p , z p ) T )

[0116] Step 2-3-13) Eliminate the outlier point clouds whose distance from the fitting plane of the prefabricated plate is greater than the preconfigured threshold δ 0 .

[0117] Step 2-3-14) Project the point cloud data after eliminating the outlier point clouds onto the XOY plane of the new coordinate system to complete the projection transformation of the point cloud.

[0118] After the projection transformation, the three-dimensional point cloud is simplified to a two-dimensional point set, greatly reducing the difficulty of point cloud feature extraction.

[0119] The calculation of the convex hull of the point cloud includes the following steps:

[0120] The present invention selects the Graham scan method with relatively high calculation efficiency to extract the convex hull of the plate point cloud. Since there are adjacent points only in some directions for the boundary point cloud, the average distance of its k-nearest neighbors will be slightly greater than that of the point cloud inside the plate. Therefore, the present invention combines the neighborhood features of the point cloud and improves the Graham scan method. First, use the kd-tree to calculate the average distance of the k-nearest neighbors of each point, set the screening prefabrication to obtain the candidate boundary point cloud, and then use the Graham scan method to find the convex hull for the candidate boundary point cloud. After the improvement, the algorithm avoids the participation of the point cloud inside the plate in the operation and improves the efficiency of convex hull extraction.

[0121] The simplification of the 3° boundary includes the following steps:

[0122] During on-site assembly, the precast slabs will be continuously translated within the proposed assembly area to the designed position. Although the convex hull of the slab point cloud can more accurately reflect the actual boundary of the slab, the minimum space required for slab installation is more concerned during the construction process. The planar shape of the precast slab is approximately rectangular, and the minimum area circumscribed rectangle of the convex hull can be used to replace the boundary of the slab. The minimum circumscribed rectangle only needs to record the coordinates of the four vertices of the slab, greatly simplifying the parameters of the slab 3D reconstruction. At the same time, the characteristics of the circumscribed rectangle can meet the dimensional control requirements of the on-site construction of the assembled pavement. The present invention selects the rotating calipers algorithm to calculate the minimum area circumscribed rectangle of the convex hull of the slab.

[0123] The method for 3D reconstruction of the precast slab point cloud data includes the following steps:

[0124] The boundary of the precast slab determines the planar size of the proposed assembly area. According to the corner coordinates, the boundary point coordinates of the precast slab obtained from multiple scans are spliced into the complete boundary of the proposed assembly area. Write a Python script to import the above boundary point coordinates into FreeCAD and automatically generate a boundary curve in the sketch drawing module. Use the RANSAC algorithm to fit the planar parameters of the precast slab boundary and the base point cloud respectively, calculate the distance between the base fitting plane and the precast slab plane, and use this as the height of the side surface of the proposed assembly area. Stretch the above boundary to the side surface height to obtain the 3D reconstruction model of the precast slab boundary. The stretched boundary cannot reflect the surface information of the precast slab. Construct a cube with a planar size slightly larger than the boundary range and a height the same as the boundary stretching height as the existing pavement slab before demolition. Fill the inside of the stretched boundary with a solid and perform a Boolean subtraction operation with the pavement before demolition to further obtain the 3D reconstruction model of the existing pavement.

[0125] Step 2-4) Based on the feature points of the embedded components and the design scheme, check the 3D dimensions and the positions of the embedded components of the precast slab reconstruction model;

[0126] The checking in the precast stage described above includes the checking of the 3D dimensions of the precast slab, the checking of the layout positions of the lifting and leveling components, and the checking of the layout positions of the joint structures.

[0127] Step 2-5) Based on the virtual assembly technology, perform virtual assembly on the checked reconstruction model, detect and judge whether there is assembly interference at the joint structure. If there is assembly interference, correct the precast slab reconstruction model or remanufacture the precast slab, and re-execute Step 2-1)-Step 2-5). If there is no assembly interference, output the precast slab reconstruction model.

[0128] The virtual assembly technology described above includes three-dimensional modeling, assembly constraint setting, and assembly interference detection steps:

[0129] (1) Three-dimensional modeling

[0130] The three-dimensional reconstruction model of the prefabricated pavement is mainly based on the measured point cloud data of the precast slabs or the areas to be assembled.

[0131] (2) Assembly constraint setting

[0132] Assembly constraints refer to the relative positions and their constraint relationships between different three-dimensional models. The purpose of setting the assembly constraint relationship is to reduce the degrees of freedom of each model and integrate them into a complete and accurate assembly. The setting of assembly constraints is based on the feature points, boundaries, central axes, and model surfaces of the three-dimensional models. In FreeCAD, the main assembly constraint relationships between models include parallel constraint, perpendicular constraint, coaxial constraint, angular constraint, coincidence constraint, and fixed constraint, etc.

[0133] (3) Assembly interference detection

[0134] Assembly interference detection is the core issue of the virtual assembly of the prefabricated pavement. The interference detection of virtual assembly is mainly divided into static interference detection and dynamic interference detection. Among them, static interference detection mainly detects the geometric conflicts between models in the state where the assembly is completed, while dynamic interference detection further considers the possible interference situations during the assembly movement process of each component.

[0135] The assembly interference between precast slabs usually occurs at the joint structures of adjacent slabs. Only checking the three-dimensional dimensions of the precast slabs and the layout positions of the joint structures cannot ensure the feasibility of the assembly between the slabs. Because two slabs that meet the precast precision requirements may not match due to the opposite layout errors between the joint structures. It is necessary to detect the interference between the joint structures of the slabs by means of virtual assembly technology. To avoid the situation of mismatched joint structures during the on-site installation of the slabs, the detection of the assembly interference between adjacent slabs should be completed before the slabs are transported to the construction site.

[0136] ③ Construction stage evaluation

[0137] The construction stage evaluation includes the following steps:

[0138] Step 3-1) Obtain the original point cloud of the area to be assembled of the prefabricated pavement collected by the depth camera,

[0139] Step 3-2) Preprocess the original point cloud of the area to be assembled of the prefabricated pavement to obtain the point cloud data of the area to be assembled,

[0140] The process of preprocessing the original point cloud includes denoising, background point removal, point cloud reduction, and point cloud stitching and registration.

[0141] Step 3-3) Perform point cloud segmentation, feature extraction, and three-dimensional reconstruction on the point cloud data of the area to be assembled to obtain the three-dimensional models of the existing slabs and the three-dimensional model of the base course,

[0142] The point cloud segmentation of the to-be-assembled area point cloud data includes the following steps:

[0143] Step 3-3-1): Obtain the position of the horizontal plane;

[0144] Step 3-3-2): Use the least squares method to fit the plane where the entire to-be-assembled area point cloud data is located;

[0145] Step 3-3-3): Perform coordinate transformation on the to-be-assembled area point cloud data based on the plane parameters obtained by fitting, so that the horizontal plane is parallel to the XOY plane of the point cloud coordinate axis, and determine the depth value of the point cloud from the horizontal plane based on the Z-axis coordinate value of the point cloud data;

[0146] Step 3-3-4): Achieve point cloud segmentation based on the depth value of the point cloud data to obtain the existing plate point cloud and the base point cloud.

[0147] The feature extraction includes the feature extraction of the boundary of the existing plate point cloud and the feature extraction of the base point cloud. Among them, the feature extraction of the base point cloud includes the feature extraction of the top surface and the feature extraction of the convex area.

[0148] The steps for feature extraction of the top surface of the base point cloud are as follows:

[0149] The coordinate values of the original base point cloud cannot intuitively reflect the elevation of each point of the base. Before feature extraction, first use the RANSAC algorithm to perform plane fitting and coordinate transformation on the base point cloud, make the XOY plane of the coordinate axis parallel to the horizontal plane, and convert the Z coordinate value of the point cloud into an elevation value. Extract the top surface features based on the elevation value.

[0150] The steps for extracting the convex area of the base point cloud are as follows:

[0151] According to the change of the elevation of the base surface, the base can be divided into a flat area and a convex area with significant elevation change. The present invention considers the neighborhood features of the point cloud and uses the Difference of Normals (DoN) as the discrimination index for the convex area.

[0152] The feature extraction of the boundary of the existing plate point cloud includes the following steps:

[0153] Project the point cloud of the existing plate onto a two-dimensional plane. After the coordinate transformation of the point cloud, the boundary to be extracted is located on the uppermost side of the plane. The present invention uses the grid method to extract the outermost boundary points in the point cloud.

[0154] The selection of boundary points is based on grid division, and each point is approximately evenly distributed along the x-axis. The boundary points obtained by the grid method are often numerous, and further simplify in combination with the boundary characteristics. The judgment of the convex points outside the boundary is added during the simplification process. The main judgment process is as follows:

[0155] S1. Draw a straight line l through the endpoints A and B on both sides of the boundary curve; if the curve only contains points A and B, return l as the result of curve simplification;

[0156] S2. Calculate the distances from each point on the curve to the straight line l in turn, and find the point C that is farthest from the straight line;

[0157] S3. If the distance from point C to l is greater than the threshold, retain point C as a feature point and go to step 5; otherwise go to step 4;

[0158] S4. If point C is outside the straight line l, retain point C as a convex point and go to step S5; otherwise use the straight line l as the curve simplification result;

[0159] S5. Taking point C as the boundary, divide the curve into two segments AC and CB, and repeat the process of steps S1 - S5 for them;

[0160] S6. When all curves are processed, connect the boundary points in turn to obtain the simplified result of the curve.

[0161] The 3D reconstruction of the point cloud in the quasi - assembly area includes the 3D reconstruction of the quasi - assembly area and the 3D reconstruction of the base layer.

[0162] Among them, the 3D reconstruction of the base layer includes the following steps: The present invention uses the Delaunay triangulation growth algorithm to triangulate the base - layer point cloud. Through triangulation, the discrete base - layer point cloud can be transformed into a triangular mesh, thus forming a continuous base - layer surface. In the triangular mesh, each triangle shares a certain side with adjacent triangles, but the triangles do not overlap. The information stored in the triangular mesh mainly includes the vertex coordinates of each triangle (corresponding to the original point cloud), the edges connecting the vertices, and the faces formed by each triangle. Among them, for the triangular mesh obtained by using the Delaunay triangulation growth algorithm for the base - layer point cloud, the triangular mesh of the base layer is only composed of points, lines and curved surfaces, and does not contain parameter information such as mass and volume, and cannot be spliced with entities of 3D reconstruction models such as plates. In the Part module of FreeCAD, the triangular mesh is further transformed into an entity to obtain the final 3D reconstruction model of the base layer.

[0163] According to the spatial position relationship, splice the 3D models of the prefabricated plates and the base layer into a whole to complete the 3D reconstruction of the quasi - assembly area.

[0164] Step 3 - 4) Combine the reconstruction model of the prefabricated plate, and based on virtual assembly technology, detect and judge whether there is interference in the quasi - assembly area. If there is no interference, carry out the installation operation of the plate. If there is interference, remanufacture the prefabricated plate.

[0165] The assembly interference in the area to be assembled includes two types: the interference between the side surface of the plate and the boundary of the existing plate, and the interference between the bottom surface of the plate and the top surface of the base layer. The detection process needs to be carried out after the existing road surface is broken and the three-dimensional reconstruction model of the area to be assembled is obtained. According to the results of the interference detection, the construction personnel can repair the base layer or the existing plate. The repaired area to be assembled can also be scanned three-dimensionally and the assembly interference detected again until there is no interference in the area.

[0166] ④ Acceptance stage evaluation

[0167] The acceptance stage evaluation includes the following steps:

[0168] Step 4-1) Obtain the original point cloud of the joints of the assembled pavement plates collected by the depth camera.

[0169] Step 4-2) Preprocess the original point cloud of the joints of the assembled pavement plates.

[0170] Step 4-3) Identify the joint positions based on the point cloud data of the joints of the assembled pavement plates after preprocessing, obtain the joint point cloud and the point clouds of the plates on both sides of the joints, and complete the evaluation of the paving accuracy of the assembled pavement plates based on the joint point cloud and the point clouds of the plates on both sides of the joints.

[0171] The process of preprocessing the original point cloud includes denoising, background point removal, point cloud reduction, and point cloud stitching and registration.

[0172] The evaluation of the paving accuracy of the assembled pavement plates includes the evaluation of the joint step amount and the evaluation of the joint width. Among them, the evaluation of the joint step amount is realized based on the point clouds of the plates on both sides of the joint, and the evaluation of the joint width is realized based on the joint point cloud.

[0173] The evaluation of the joint step amount includes the following steps:

[0174] Determine the pixel coordinates of the joint position according to the joint position recognition result, and use the RANSAC algorithm to fit the joint straight line.

[0175] Translate the fitted joint straight line 100 pixel distances (about 5 cm) upward and downward along its perpendicular direction respectively to obtain the joint neighborhood, and the two sides of the joint neighborhood are the plate areas.

[0176] According to the pixel indices of the plate areas, extract the point cloud data of the plates from the structured point cloud, use joint bilateral filtering and radius filtering to denoise the point cloud of the plates, and use the grid method for reduction to obtain the point cloud data of the plates on both sides of the joint.

[0177] Calculate the step amount based on the point cloud data of the plates on both sides of the joint. In this embodiment, the plane fitting method is used to calculate the step amount between the plates.

[0178] The evaluation of the seam width includes the following steps:

[0179] Using the boundary tracking algorithm, extract the boundaries on both sides of the seam from the seam recognition result, and set a perimeter threshold to eliminate the shorter boundary to obtain the seam boundary extraction result;

[0180] Fit a straight line according to the seam, and divide the above boundary points into two parts, corresponding to the boundaries of the plates on both sides of the seam;

[0181] Extract the point clouds of the two boundaries from the pixel indices of the boundary points in the image;

[0182] Use the DP algorithm to streamline the boundary point cloud;

[0183] To simplify the calculation process of the seam width, perform a coordinate transformation on the boundary point cloud. After the coordinate transformation, the xoy plane of the coordinate system coincides with the fitting plane of the boundary point cloud, and the seam direction is the same as the x-axis direction of the coordinate system. The seam width can be calculated in the projection plane. Use the RANSAC algorithm to fit the straight lines where the two boundaries are located respectively, and calculate the distance between them to obtain the seam width.

[0184] The preferred specific embodiments of the present invention have been described in detail above. It should be understood that those of ordinary skill in the art can make many modifications and variations based on the concept of the present invention without creative labor. Therefore, all technical solutions that can be obtained by those skilled in the art in the technical field based on the concept of the present invention through logical analysis, reasoning, or limited experiments on the basis of the prior art should be within the protection scope determined by the claims.

Claims

1. A method for evaluating the paving accuracy of prefabricated pavements based on a depth camera, characterized in that, it includes evaluation in the design stage, evaluation in the prefabrication stage, evaluation in the construction stage and evaluation in the acceptance stage. Among them, the evaluation in the design stage includes the following steps: Step 1-1) Obtain the original point cloud of the existing road surface within the repair area of the prefabricated pavement collected by the depth camera, Step 1-2) Preprocess the original point cloud of the existing road surface within the repair area of the prefabricated pavement, Step 1-3) Based on the point cloud data of the existing road surface within the preprocessed repair area of the prefabricated pavement, determine the plane position and elevation data of the repair area through coordinate transformation; The evaluation in the prefabrication stage includes the following steps: Step 2-1) Obtain the original point cloud of each surface of the prefabricated precast slab of the prefabricated pavement and the embedded components in sequence by the depth camera, Step 2-2) Preprocess the original point cloud of each surface of the prefabricated precast slab of the prefabricated pavement and the embedded components to obtain the point cloud data of the precast slab, Step 2-3) Perform point cloud segmentation, feature extraction and 3D reconstruction on the point cloud data of the precast slab to obtain the feature points of the embedded components and the reconstructed model of the precast slab, Step 2-4) Based on the feature points of the embedded components and the design scheme, check the 3D dimensions of the reconstructed model of the precast slab and the positions of the embedded components, Step 2-5) Based on virtual assembly technology, perform virtual assembly on the checked reconstructed model, detect and judge whether there is assembly interference at the joint structure. If there is assembly interference, correct the reconstructed model of the precast slab or remanufacture the precast slab, and re-execute Step 2-1)-Step 2-5). If there is no assembly interference, output the reconstructed model of the precast slab; The evaluation in the construction stage includes the following steps: Step 3-1) Obtain the original point cloud of the proposed assembly area of the prefabricated pavement collected by the depth camera, Step 3-2) Preprocess the original point cloud of the proposed assembly area of the prefabricated pavement to obtain the point cloud data of the proposed assembly area, Step 3-3) Perform point cloud segmentation, feature extraction and 3D reconstruction on the point cloud data of the proposed assembly area to obtain the 3D model of the existing slab and the 3D model of the base course, Step 3-4) Combine the reconstructed model of the precast slab, and based on virtual assembly technology, detect and judge whether there is interference in the proposed assembly area. If there is no interference, perform the installation operation of the slab. If there is interference, remanufacture the precast slab; The evaluation in the acceptance stage includes the following steps: Step 4-1) Obtain the original point cloud of the joints of the prefabricated pavement slabs collected by the depth camera, Step 4-2) Preprocess the original point cloud of the joints of the prefabricated pavement slabs, Step 4-3) Identify the joint positions based on the point cloud data of the joints of the prefabricated pavement slabs after preprocessing to obtain the joint point cloud and the point cloud of the slabs on both sides of the joint, and complete the evaluation of the paving accuracy of the prefabricated pavement slabs based on the joint point cloud and the point cloud of the slabs on both sides of the joint.

2. The method for evaluating the paving accuracy of prefabricated pavements based on a depth camera according to claim 1, characterized in that, the process of preprocessing the original point cloud includes denoising, background point removal, point cloud reduction and point cloud stitching and registration.

3. A method for evaluating the paving accuracy of prefabricated pavements based on a depth camera according to claim 1, characterized in that, the point cloud segmentation of the precast panel point cloud data includes the following steps: Step 2-3-1): Based on the input precast panel point cloud data P, construct a kd tree, set a set C to store various point cloud clusters, and use a queue Q to store the points to be classified Step 2-3-2) Obtain the points in the prefabricated block point cloud data P and determine whether they have been classified. If not, execute Step 2-3-3). If they have been classified, obtain the next point p i+1 and re-execute Step 2-3-3); Step 2-3-3) Add point p i to the current queue Q of points to be classified; Step 2-3-4) For each point p in queue Q j ∈ Q, use the kd-tree to search for the radius neighborhood j with a distance less than r 0 of p and add the unprocessed points in to queue Q, and classify the point cloud in queue Q to obtain point cloud clusters; Step 2-3-5): Determine whether all points in P have been processed. If so, store the classified point cloud clusters in the queue Q into the set C, clear the queue Q, and complete the point cloud segmentation. If not, return to Step 2-3-2).

4. A method for evaluating the paving accuracy of prefabricated pavements based on a depth camera according to claim 1, characterized in that, the point cloud segmentation of the point cloud data of the area to be assembled includes the following steps: Step 3-3-1): Obtain the position of the horizontal plane; Step 3-3-2): Use the least squares method to fit the plane where the entire point cloud data of the area to be assembled is located; Step 3-3-3): Based on the plane parameters obtained by fitting, perform coordinate transformation on the point cloud data of the area to be assembled, so that the horizontal plane is parallel to the XOY plane of the point cloud coordinate axis, and determine the depth value of the point cloud from the horizontal plane based on the Z-axis coordinate value of the point cloud data; Step 3-3-4): Based on the depth value of the point cloud data, perform point cloud segmentation to obtain the existing slab point cloud and the base point cloud.

5. A method for evaluating the paving accuracy of prefabricated pavements based on a depth camera according to claim 1, characterized in that, The feature extraction of the precast panel point cloud data includes three parts: the extraction of plane parameters, the extraction of boundaries, and the simplification of boundaries.

6. A method for evaluating the paving accuracy of prefabricated pavements based on a depth camera according to claim 5, characterized in that, the extraction of the plane parameters uses the RANSAC algorithm and includes the following steps: Step 2-3-6): Determine the parametric equation of the precast panel plane in three-dimensional space: ax + by + cz + d = 0 where (x, y, z) are the coordinates of the point cloud located in the precast panel plane, and a, b, c, and d are all estimated parameters; Step 2-3-7) Randomly select three points in the point cloud (x 1 , x 1 , x 1 ), (x 2 , x 2 , x 2 ), and (x 3 , x 3 , x 3 ). Substitute the coordinates of the three points into the plane equation in turn, and use Cramer's rule to solve the linear equations. The estimated parameters of the plane where the current point is located are: Step 2-3-8): Calculate the distance between the points in the point cloud and the estimated plane of the precast panel; Step 2-3-9) Determine the distance d i whether it is less than a pre-configured distance threshold δ 0 , if so, classify this point as an inlier, if not, classify this point as an outlier; Step 2-3-10): Repeat steps 2-3-8) - 2-3-9) until all the point clouds are traversed, and count the number of inliers under the current estimated parameters; Step 2-3-11): Repeat steps 2-3-7) - 2-3-10) until the iteration termination condition is reached, record the estimated parameters corresponding to the maximum number of inliers, and obtain the optimal plane parameters. The iteration termination condition is that the number of inliers reaches the pre-configured quantity or the maximum number of iterations. The maximum number of iterations k is: where t = number of inliers / total number of point clouds, and P is the probability of obtaining the optimal plane parameters.

7. A method for evaluating the paving accuracy of prefabricated pavements based on a depth camera according to claim 6, characterized in that, the extraction of the boundary includes two parts: the projection transformation of the point cloud and the convex hull calculation.

8. A method for evaluating the paving accuracy of prefabricated pavements based on a depth camera according to claim 7, characterized in that, The projection transformation of the point cloud includes the following steps: Step 2-3-12) Take the direction of the normal vector of the fitting plane of the precast slab as the direction of the Z-axis of the new coordinate system, keep the direction of the X-axis unchanged, determine the base matrix of the new coordinate system, and transform the coordinates of the precast slab point cloud based on the base matrix. Among them, let the unit normal vector of the fitting plane of the precast slab be The unit vector in the positive direction of the X-axis of the original coordinate system is Then the base matrix of the new coordinate system is: Any point p=(x p ,y p ,z p ) in the point cloud, the coordinate value p'=(x' p ,y' p ,z' p ) in the new coordinate system is as follows: (x' p ,y' p ,z' p ) = (A -1 ·(x p ,y p ,z p ) T ) Step 2-3-13) Eliminate the distance between the prefabricated plate fitting plane and the prefabricated plate fitting plane greater than the preconfigured threshold δ 0 Abnormal point cloud; Step 2-3-14) Project the point cloud data after abnormal point cloud removal onto the XOY plane of the new coordinate system to complete the projection transformation of the point cloud.

9. A method for evaluating the paving accuracy of prefabricated pavements based on a depth camera according to claim 1, characterized in that the evaluation of the paving accuracy of the prefabricated pavement slabs includes the evaluation of the joint step amount and the evaluation of the joint width. Among them, the evaluation of the joint step amount is realized based on the point clouds of the slabs on both sides of the joint, and the evaluation of the joint width is realized based on the joint point cloud.

10. A method for evaluating the paving accuracy of prefabricated pavements based on a depth camera according to claim 9, characterized in that the evaluation of the joint step amount includes the following steps: Determine the pixel coordinates of the joint position according to the joint position recognition result, and use the RANSAC algorithm to fit the joint straight line; Translate the fitted joint straight line upward and downward along the perpendicular direction of the straight line by a distance of pre-configured pixels to obtain the joint neighborhood, and both sides of the joint neighborhood are the slab regions; According to the pixel indices of the slab regions, extract the point cloud data of the slabs from the structured point cloud, use joint bilateral filtering and radius filtering to denoise the slab point cloud, and use the grid method to simplify it to obtain the point cloud data of the slabs on both sides of the joint; Calculate the step amount based on the point cloud data of the slabs on both sides of the joint.

Images