Indoor wall size deviation detection method based on point cloud data

Abstract

The invention relates to the technical field of building quality detection, and relates to an indoor wall size deviation detection method based on point cloud data. According to the method, collected and pre-processed indoor scene point cloud data is taken as a data source, point cloud wall surface segmentation is realized by adopting a point cloud neural network PointNext aiming at point cloud, support is provided for size deviation detection, and then indoor wall surface point cloud size deviation detection is carried out, so that the indoor wall surface point cloud size deviation detection accuracy is improved. And finally, carrying out size deviation detection and deviation visualization and accurate positioning on the indoor wall point cloud by combining an image mapping technology. According to the method, segmentation of the point cloud wall surface is realized by adopting the point cloud neural network PointNext with relatively high generalization, the method can adapt to point cloud segmentation in different scenes through training, and the automation degree is relatively high. A point cloud least square method is used for size deviation inspection, and technologies such as point cloud image mapping and the like are combined to realize visualization and automatic analysis of indoor wall surface size deviation, so that the intelligent level of indoor wall surface point cloud size deviation detection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of building quality detection, and in particular to a method for detecting indoor wall size deviation based on point cloud data. Background Art

[0002] Quality inspection is an important means to ensure the quality of projects and promote the healthy development of the construction industry. Most buildings are reinforced concrete structures. In the field of building infrastructure, according to the current quality standards of the construction industry, the types of defects in cast-in-place reinforced concrete structures are mainly divided into two parts: appearance quality defects and position and size deviation defects. According to the recent development of the construction industry, the size deviation defects on the surface of indoor scenes not only affect the appearance quality of the building walls, but also affect the safety performance and service life of the buildings. Surface flatness and verticality are important indicators in the detection of indoor scene position and size deviation. Through field investigations at the construction site and review of relevant materials, it is currently found that surface flatness and verticality detection mostly adopts manual inspection at the construction site.

[0003] Traditional methods for measuring dimensional deviations such as the flatness and verticality of concrete wall surfaces are time-consuming and labor-intensive, especially when conducting large-scale indoor surface flatness inspections, which require high labor costs. The inspection results are easily affected by the subjective cognition and construction experience of construction workers. The inspection results are less objective and more random, and are prone to problems such as missed inspections and false inspections. In addition, it is impossible to accurately locate the specific degree of deviation of the surface flatness and verticality of different positions on the wall. At the same time, at some construction sites, the dangerous inspection environment can easily pose a serious threat to the personal safety of inspectors.

[0004] When using point cloud data to detect indoor wall size deviations, it is necessary to first segment the indoor wall point cloud data to separate the wall from the ceiling and floor. Most indoor building components are horizontal and vertical regular planes, so the point cloud segmentation of indoor building components is mostly plane segmentation. In the field of point cloud segmentation, on the one hand, traditional point cloud segmentation algorithms cannot achieve high-precision point cloud segmentation effects. When the point cloud scale is large, it is impossible to quickly and accurately segment the plane; on the other hand, point cloud deep learning segmentation requires the collection of a large number of suitable data sets for training, which is time-consuming and labor-intensive. For large-scale point cloud data, large-scale point cloud data cannot be directly input into the neural network.

[0005] Therefore, it is particularly important to develop a non-contact, efficient, intelligent method for detecting concrete size deviations that can accurately locate deviations. Summary of the invention

[0006] Based on the above defects or improvement needs of the prior art, the present invention provides a method for detecting indoor wall size deviation based on point cloud data. The collected and pre-processed indoor scene point cloud data is used as the data source, and the point cloud neural network PointNext is used to implement point cloud wall segmentation for point cloud to provide support for size deviation detection, and then the indoor wall point cloud size deviation is detected. Finally, the image mapping technology is combined to realize the size deviation detection and deviation visualization and precise positioning of the indoor wall point cloud.

[0007] To achieve the above object, the technical solution of the present invention is as follows:

[0008] A method for detecting indoor wall size deviation based on point cloud data, the steps are as follows:

[0009] Step (1) Segmentation of indoor wall point cloud data

[0010] A dataset is established based on the collected and acquired indoor wall point cloud data, and the point cloud neural network PointNext is used to achieve effective segmentation of the point cloud wall. The PointNext neural network model adopts the U-Net structure, which is a typical encoder-decoder structure. The encoder uses a series of Set Abstraction (SA) modules to hierarchically abstract the features of the point cloud, while the decoder uses the same number of feature propagation to upsample the features. Among them, the SA module consists of a subsampling layer, a neighbor query layer, a grouping layer, a group of shared multi-layer perceptrons (MLPs) for segmenting features, and a reduction layer for aggregating features within the neighborhood.

[0011] Step (2) Indoor wall point cloud size deviation detection

[0012] Taking the point cloud wall segmentation result of step (1) as input, the size deviation of the indoor wall is calculated and quantified based on the least squares method; it includes two steps: point cloud plane fitting and size deviation calculation. The specific process is as follows:

[0013] Point cloud plane fitting: The least squares method is a mathematical optimization method used to find the best fitting plane of a function so as to minimize the sum of square errors between the plane and the point cloud data. The best fitting parameters can be directly obtained through the least squares method. Therefore, the least squares algorithm is used to fit the plane as the reference plane.

[0014] Calculation of size deviation: Use the least squares method to calculate the verticality and flatness of indoor walls.

[0015] Step (3) Visual analysis of dimensional deviation

[0016] Based on point cloud image mapping and other related technologies, the verticality and flatness detection data can be visualized and the deviation defects of indoor wall surfaces can be automatically analyzed.

[0017] Beneficial effects of the present invention:

[0018] The present invention realizes the segmentation of point cloud walls by adopting the point cloud neural network PointNext with strong generalization. After training, it can adapt to the point cloud segmentation in different scenes and has a high degree of automation.

[0019] In addition, the point cloud least squares method is used to check the size deviation, and the point cloud image mapping and other technologies are combined to realize the visualization and automatic analysis of the indoor wall size deviation, which improves the intelligence level of point cloud size deviation detection of indoor walls of buildings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Flowchart for dimensional deviation detection and visualization analysis.

[0021] Figure 2(a) to Figure 2(d) They are elevation diagrams of the four inspection walls in room 1 in the embodiment. DETAILED DESCRIPTION

[0022] The embodiments of the present invention are described in detail below with reference to the accompanying drawings and examples.

[0023] The indoor wall size deviation detection method based on point cloud data of the present invention is as follows: Figure 1 As shown, the specific steps are as follows:

[0024] Step 1: Create a data set based on the collected indoor wall point cloud data, and use the point cloud neural network PointNext to segment the point cloud wall. The specific steps are as follows:

[0025] Step 1.1 Divide the data into training set and test set according to the ratio.

[0026] Step 1.2: Establish the point cloud neural network PointNext and train it.

[0027] Step 1.3 uses the mean intersection over union (mIoU) and overall accuracy (OA) as evaluation indicators to evaluate the trained model in real time. The intersection over union (IoU) is a commonly used indicator for evaluating the effect of point cloud segmentation. It calculates the ratio of the intersection and union of the true category and the predicted category. It needs to be calculated for each category separately, and the mean intersection over union (mIoU) represents the average value of the IoU of all categories in the segmentation task.

[0028]

[0029]

[0030] Among them, prediction∩ground truth represents the union of the true category and the predicted category; prediction∪ground truth represents the intersection of the true category and the predicted category, k represents the total number of categories in the segmentation task, and IoU represents the IoU of the i-th category.

[0031] Step 2: Calculate and quantify the size deviation of the indoor wall by the least square method. The specific process is as follows:

[0032] Step 2.1 Point cloud plane fitting: Use the least squares method to find the best fitting parameters of the point cloud plane by minimizing the residual square sum between the observed value and the fitting plane. In order to obtain the parameters of the best fitting plane of the point cloud plane, the following three formulas should be satisfied so that the distance from the effective point cloud data of the wall to the plane is as small as possible, that is,

[0033]

[0034] p(x,y,z)=0

[0035]

[0036] Among them, p(x,y,z) is the point cloud p(x i ,y i ,z i ) is the projection point on the fitting plane, the total number of point clouds is n, p(x i ,y i ,z i ) represents the i-th point cloud. When projected to p(x,y,z)=0, then min represents the minimum distance that can be obtained. The horizontal plane equation of the point cloud space is p(x,y,z)=ax-by+cz+1=0, M{(x1,y1,z1),(x2,y2,z2),....(x n ,y n ,z n )} is represented as a wall point set, and a, b, and c are the coefficients of the point cloud fitting plane equation.

[0037] Then find the partial derivatives of a, b, and c respectively, and satisfy After sorting out the above conditions, solve the plane coefficient equation X = Q -1 K, where X is the vector of plane coefficients a, b, c, i.e., the coefficients in the point cloud fitting plane equation, and Q is the matrix containing the point cloud data.

[0038]

[0039] After calculation, the solved a, b, and c are substituted into the wall plane equation z=ax-by+c, thereby calculating the best fitting plane equation for the wall point cloud fitting.

[0040] Step 2.2 Dimension deviation calculation: The purpose of this step is to calculate the verticality and flatness of the indoor construction based on step 2.1.

[0041] When calculating verticality, assume that the wall normal vector is The floor normal vector is The wall normal vector information is obtained based on the fitted plane equation coefficients a, b, and c obtained in step 2.1. The floor is a horizontal structure, and the normal vector is The verticality information of the wall is obtained by calculating the product of the normal vector angle and the corresponding wall height.

[0042]

[0043] Among them, L represents the verticality, and H represents the height of the wall point cloud.

[0044] For the calculation of surface flatness, the point cloud segmentation result is usually used as input, and the best fitting plane is obtained by the least squares method in step 2.1. Then, the surface flatness of the wall is calculated by calculating the distance from the wall point cloud to the fitting plane. The specific calculation steps are as follows:

[0045] (1) Calculate the flatness using the obtained fitting plane equation of the wall to be tested, assuming that the fitting plane is P and the equation is ax-by+cz+1=0;

[0046] (2) Assume that the coordinates of the point cloud in the plane are (x i ,y i ,z i ), the distance d and mean error of each point on the plane to the fitting plane can be calculated according to the following formula

[0047]

[0048] (3) The surface flatness s of the plane to be measured is expressed as follows:

[0049]

[0050] Step 3: Visualize and automatically analyze the deviation data for indoor wall size deviation. The specific steps are as follows:

[0051] Step 3.1 Visual analysis of indoor wall verticality: As can be seen from step 2.2, the verticality information of the calculated wall is obtained by calculating the angle between the normal vector of the wall point cloud fitting plane and the numerical normal vector. In order to visualize and analyze the wall verticality obtained in step 2.2, the wall point cloud is color mapped according to the degree of wall verticality deviation. The point cloud color changes from green to red, corresponding to the maximum and minimum thresholds of the verticality deviation.

[0052] Step 3.2 Visual analysis of indoor wall surface flatness: Based on the surface flatness calculation in step 2.2, the point cloud and image combination technology is used to perform elevation calculation, grid processing, contour line generation and other processing on the indoor wall point cloud, which can more intuitively and conveniently present the degree of wall surface flatness deviation, accurately locate the size deviation position, and lay the foundation for automatic analysis of size deviation. The distance d from the point to the fitting plane in the surface flatness calculation step in step 2.2 is used as the elevation, and the specific implementation methods of the remaining processing are as follows:

[0053] (1) Point cloud mesh processing

[0054] Since point cloud data is disordered, it is necessary to transform the point cloud into a regular matrix for image mapping. First, the point cloud is rotated to a position parallel to the XOY plane using a rotation matrix. The maximum and minimum values ​​of the point cloud on the x-axis and y-axis are represented as X and Y, respectively. max , X min , Y max , Y min , set the interval along the x-axis and y-axis to N, and after meshing the point cloud, the size calculation method of the projection image in the horizontal and vertical directions of the point cloud is as follows:

[0055]

[0056] Among them, D x It is represented by the horizontal dimension of the point cloud grid, D y Indicates the vertical dimension of the point cloud grid, Y max Represented as the maximum Y-axis coordinate value of the point cloud plane data, Y min It is expressed as the minimum Y-axis coordinate value of the point cloud plane data, X max Represented as the maximum coordinate value of the X axis of the point cloud plane data, X min Expressed as the minimum X-axis coordinate value of the point cloud plane.

[0057] (2) Generation of elevation matrix

[0058] Based on the fitting plane of the least squares method, the fitting plane is set to V, and all points P(x i ,y i ,zi ) to the height d of the fitting plane V i It can be expressed as the following formula:

[0059]

[0060] (3) Generation of elevation map contour lines

[0061] After dividing the point cloud plane into grid sizes, the point cloud color changes from blue to red according to the elevation value of the point cloud data in each spatial grid to the fitting plane, corresponding to the maximum and minimum thresholds of the wall flatness deviation, respectively. Image contour lines are generated based on the color mapping of different elevation values.

[0062] This implementation uses the collected indoor point cloud data as the data source and uses PointNext point cloud neural network technology to segment the wall surface of the point cloud data. After completing the point cloud segmentation of the indoor environment, the point cloud data of the indoor wall is further tested for size deviations, and a size deviation management strategy based on the PDCA (Plan-Do-Check-Act) cycle is implemented.

[0063] This embodiment adopts the PointNext model, Ubuntu 20.04.6LTS operating system, Python 3.7 for development, the processor model used is 12th Gen Intel(R)Core(TM)i5-12400F, the memory GPU version is NVIDIA GeForce RTX 4090, and the learning driver is CUDA12.2 and Cudnn8.5.0.

[0064] In terms of data set selection, 39 sets of indoor point cloud data were collected for training and testing, 8 sets of data were collected as training sets, and 31 sets of indoor point cloud data were collected as test sets. For the training data set, in order to obtain the best training effect, this paper used different types of optimizers and training batches (batchsize) to compare the effects on the total accuracy, average accuracy, and average intersection-over-union ratio. The results are shown in Table 1. As can be seen from Table 1, when the training batch (batchsize) is 16 and the optimizer is adamp, according to the comparison of the final training results, the training batch (batchsize) is selected as 16 and the optimizer is adamp.

[0065] The trained model is retested using the 39 sets of room data from room 1 to room 39, and the test results are evaluated by OA, IoU and mIoU. The test results are summarized in Table 2.

[0066] Table 1 Comparison of training results

[0067]

[0068] Table 2 PointNext segmentation effect evaluation (partial)

[0069]

[0070] For the point cloud data of 39 groups of rooms, the average OA is 91.2%. The best result is room 25 with an OA of 95.5%, and the worst result is room 28 with an OA of 89.2%. The OAs are all above 85%, and the model performs well.

[0071] In terms of indoor wall point cloud size deviation detection, the measurement rooms were randomly sampled, representative indoor scenes were selected, and 3-4 walls were randomly selected in each indoor scene for deviation detection. Based on the indoor point cloud segmentation results, the least squares method was used to calculate and quantify the surface flatness and verticality of the indoor walls in the selected area. The flatness and verticality of the selected room were calculated according to the content of step 2, and the results are shown in Table 3.

[0072] Table 3 Dimensional deviation test results

[0073]

[0074]

[0075] The calculation method based on point cloud data can calculate the surface flatness of the entire wall by fitting the plane, which can avoid the influence of chance to a large extent.

[0076] Subsequently, the verticality and flatness of indoor walls are visualized and automatically analyzed based on the dimensional deviation detection results and image mapping technologies.

[0077] In terms of verticality visualization analysis, it can be seen from the current specification "Concrete Structure Engineering Construction Quality Acceptance Code" (GB50204-2015) that the allowable deviation range of verticality is ±10mm. Based on this, the color of the point cloud changes from green to red, and the color bar corresponds to the maximum and minimum thresholds of 10mm and -10mm respectively.

[0078] In terms of flatness visualization analysis, after the point cloud is meshed, a point cloud grid of size Dx×Dy is obtained. Assume that the number of point clouds in each grid is recorded as Pn (n=1...k), and the elevation of the point cloud in each grid to the fitting plane is recorded as dn (n=1...k). All the point cloud elevations in the grid are traversed. In order to avoid the influence of extreme values, the average value of the point cloud elevation in the grid is calculated as the grid elevation value. At the same time, all the formulas in steps 2 and 3 are combined to obtain an elevation matrix of size Dx×Dy, as shown in Figure 2(a) to Figure 2(d)As shown, the elevation value in each grid is the surface flatness condition of the grid.

[0079] Since the visualization effect of the elevation matrix is ​​poor, after the point cloud plane is divided into grid sizes, the elevation value of the point cloud data in each spatial grid to the fitting plane is used as the benchmark, and the allowable deviation value of the surface flatness in the specification "Concrete Structure Engineering Construction Quality Acceptance Code" (GB50204-2015) is used as the reference value for image mapping. The color value changes from blue to red, and the corresponding pixel RGB value ranges from (0,0,255) to (255,0,0). When a spatial grid does not contain a point cloud, the value of the corresponding pixel point is replaced by 0. According to the color mapping of different elevation values, the image contour lines are generated. The elevation map of some rooms is shown in the figure below. Figure 2(a) to Figure 2(d) shown.

[0080] Through visualization, the location of the deviation can be accurately located and the deviation can be displayed in a graphical form, so that the dimensional deviation of the flatness and verticality of the indoor wall can be intuitively understood. Through automatic visualization analysis, problems such as missed detection and false detection caused by human subjective factors can also be avoided, which improves the accuracy and reliability of the results.

[0081] Through the present invention, the indoor wall point cloud size deviation detection can be successfully completed, the point cloud segmentation accuracy of the indoor wall can be improved, the intelligent point cloud segmentation of the indoor scene and the automatic measurement of the size deviation can be realized, and the visualization and automatic analysis of the deviation can be realized by combining the image mapping and other technologies, which makes a certain contribution to the development of a contactless, efficient and intelligent concrete size deviation detection method.

Claims

1. A method for detecting indoor wall size deviation based on point cloud data, characterized in that: Here are the steps: Step 1: Create a data set based on the collected indoor wall point cloud data, and use the point cloud neural network PointNext to segment the point cloud wall. The specific steps are as follows: Step 1.1 Divide the data into training set and test set according to the proportion; Step 1.2 Establish a point cloud neural network PointNext and train it; Step 1.3: Use the mean intersection over union (mIoU) and overall accuracy (OA) as evaluation indicators to evaluate the trained model in real time; the mean intersection over union (mIoU) represents the average value of IoU of all categories in the segmentation task; Among them, prediction∩ground truth represents the union of the true category and the predicted category; prediction∪ground truth represents the intersection of the true category and the predicted category, k represents the total number of categories in the segmentation task, and IoU represents the IoU of the i-th category; Step 2: Calculate and quantify the size deviation of the indoor wall by the least square method. The specific steps are as follows: Step 2.1 Point cloud plane fitting: Use the least squares method to find the best fitting parameters of the point cloud plane by minimizing the residual square sum between the observed value and the fitting plane, and find the best fitting plane; in order to obtain the parameters of the best fitting plane of the point cloud plane, the following three formulas should be satisfied so that the distance from the effective point cloud data of the wall to the plane is as small as possible, that is, p(x,y,z)=0 Among them, p(x,y,z) is the point cloud p(x i ,y i ,z i ) is the projection point on the fitting plane, the total number of point clouds is n, p(x i ,y i ,z i ) represents the i-th point cloud; when projected to p(x,y,z)=0, it satisfies min represents the minimum distance obtained; the horizontal plane equation of the point cloud space is p(x,y,z)=ax-by+cz+1=0, M{(x1,y1,z1),(x2,y2,z2),....(x n ,y n ,z n )} is represented as a set of wall points, a, b, c are the coefficients of the point cloud fitting plane equation; Then find the partial derivatives of a, b, and c respectively, and satisfy After sorting out the above conditions, solve the plane coefficient equation X = Q -1 K, where X is the vector of plane coefficients a, b, c, i.e., the coefficients in the point cloud fitting plane equation, and Q is the matrix containing the point cloud data; After calculation, the solved a, b, and c are substituted into the wall plane equation z=ax-by+c, thereby calculating the best fitting plane equation for the wall point cloud fitting; Step 2.2 Dimension deviation calculation: Calculate the verticality and flatness of the indoor construction based on step 2.1; When calculating verticality, assume that the wall normal vector is The floor normal vector is The wall normal vector information is obtained according to the fitting plane equation coefficients a, b, and c obtained in step 2.

1. The floor is a horizontal structure, and the normal vector is The verticality information of the wall is obtained by calculating the product of the normal vector angle and the corresponding wall height; Among them, L represents verticality, and H represents the height of the wall point cloud; For the calculation of surface flatness, the point cloud segmentation result is used as input, and the best fitting plane is obtained by the least squares method in step 2.

1. Then, the surface flatness of the wall is calculated by calculating the distance from the wall point cloud to the fitting plane. The specific calculation steps are as follows: (1) Calculate the flatness using the obtained fitting plane equation of the wall to be tested, assuming that the fitting plane is P and the equation is ax-by+cz+1=0; (2) Assume that the coordinates of the point cloud in the plane are (x i ,y i ,z i ), calculate the distance d and mean square error of each point on the plane to the fitting plane according to the following formula (3) The surface flatness s of the plane to be measured is expressed as follows: Step 3: Visualize and automatically analyze the deviation data for indoor wall size deviation. The specific steps are as follows: Step 3.1 Visual analysis of indoor wall verticality: To visualize and analyze the wall verticality obtained in step 2.2, color mapping is performed on the wall point cloud according to the degree of wall verticality deviation. The point cloud color changes from green to red, corresponding to the maximum threshold and minimum threshold of the verticality deviation respectively; Step 3.2 Visual analysis of indoor wall surface flatness: Based on the surface flatness calculation in step 2.2, the point cloud and image combination technology is used to perform elevation calculation, grid processing, and contour line generation processing on the indoor wall point cloud; the distance d from the point to the fitting plane in the surface flatness calculation step in step 2.2 is used as the elevation, and the remaining processing is specifically implemented as follows: (1) Point cloud meshing processing For image mapping, the point cloud is transformed into a regular matrix. First, the point cloud is rotated to a position parallel to the XOY plane using a rotation matrix. The maximum and minimum values ​​of the point cloud on the x-axis and y-axis are represented as X and Y, respectively. max , X min , Y max , Y min , set the interval along the x-axis and y-axis to N, and after meshing the point cloud, the size calculation method of the projection image in the horizontal and vertical directions of the point cloud is as follows: Among them, D x It is represented by the horizontal dimension of the point cloud grid, D y Indicates the vertical dimension of the point cloud grid, Y max Represented as the maximum Y-axis coordinate value of the point cloud plane data, Y min It is expressed as the minimum Y-axis coordinate value of the point cloud plane data, X max Represented as the maximum coordinate value of the X axis of the point cloud plane data, X min It is expressed as the minimum coordinate value of the X-axis of the point cloud plane; (2) Generation of elevation matrix Based on the fitting plane of the least squares method, the fitting plane is set to V, and all points P(x i ,y i ,z i ) to the height d of the fitting plane V i It is expressed as the following formula: (3) Generation of elevation map contour lines After dividing the point cloud plane into grid sizes, the point cloud color changes from blue to red according to the elevation value of the point cloud data in each spatial grid to the fitting plane, corresponding to the maximum and minimum thresholds of the wall flatness deviation, respectively. Image contour lines are generated based on the color mapping of different elevation values.

Images