A construction site measurement system based on image processing

Abstract

The present invention belongs to the field of measurement technology. The present invention discloses a construction site measurement system based on image processing, including a data acquisition module for acquiring point cloud data and image data of a construction site; a data processing module for preprocessing and feature extraction of the point cloud data and image data of the construction site to obtain a point cloud feature data set and an image feature data set; a defect detection module for training and obtaining a defect detection model according to the image parameter data set, and obtaining building structure defect data through the defect detection model; a three-dimensional reconstruction module for constructing a three-dimensional model of the construction site according to the point cloud feature data set and the image feature data set; a visualization module for generating a 3D thermal model of the construction site; and visualizing the construction deviation through the 3D thermal model of the construction site, so as to monitor and plan the construction of the construction site, which helps to improve the construction management efficiency.

Description

Technical Field

[0001] The present invention relates to the field of measurement technology. More specifically, the present invention relates to a construction site measurement system based on image processing. Background Art

[0002] The patent with the application publication number CN112149543A discloses a construction dust identification system and method based on computer vision. First, the size and resolution of the acquired single-frame construction site image are initialized, and then the image is preprocessed through Gaussian filtering. The processed image is subjected to color model conversion. Binary image morphological opening operation, first eroding and then dilating, eliminates the influence of small objects and expands the edge area to obtain the background and foreground images, which are merged into a mask image. By calculating the ratio of the pixel area of the extracted construction dust to the total pixel area of the image through the pixel area method, it is judged whether the picture has construction dust. If the image has construction dust, the image is saved and an identification alarm signal is output. The present invention is a system that combines computer vision, image processing, and modern communication technology to automatically complete the automatic identification and alarm of construction dust, solving the drawbacks of the existing dust detectors such as long measurement cycle and poor accuracy, and is helpful for the monitoring work of management personnel.

[0003] The existing construction site measurement systems have the following main problems:

[0004] Failure to combine edge features, corner features, and texture features results in incomplete description of the detailed information and complex structures of the image, affecting the recognition and judgment capabilities of the model; ignoring corner features will cause the model to be unable to accurately locate the key points of the target; the lack of texture affects the recognition of materials and patterns; when the co-occurrence matrix is not normalized, the pixel frequency is affected by the image size, resulting in distortion of the eigenvalue; when the data in the co-occurrence matrix training is not screened using a limiting formula, a large amount of data frequency will increase the storage and calculation burden, affecting the efficiency of the model; the calculation cost is high and the processing time increases, especially in the application of large-scale image datasets, which may lead to a decline in system performance; the frequency of pixel pair occurrences is not controlled, and extreme pixel allocations will dominate the feature data, causing the model to be overly sensitive to these abnormal data, thereby reducing the reliability of the features; in an abnormal environment, the feature data may disrupt the normal distribution, affecting the accuracy and stability of the model;

[0005] The lack of using Alpha Shapes algorithm and Delaunay triangulation will result in a rough 3D mesh structure, which cannot accurately reflect complex building details; the lack of triangle patch optimization and removal of shared patches will lead to waveform data and repeated calculations in the model, not only burdening storage, but also increasing computational complexity, reducing real-time rendering and simulation performance, and affecting the efficiency of construction simulation; the lack of using circumsphere radius screening and limiting factor formula, the generated model will lack control, making the shape loose and affecting the compactness and expressiveness of the 3D model; without accurately mapping image data to the 3D model through camera projection model and bilinear interpolation method, there will be misalignment or distortion between the geometric structure and the texture image; unclear stereo patches and data generation will lead to too high complexity of the 3D model, increasing storage and rendering pressure, making the system unable to respond in real-time scenarios.

[0006] In view of this, the present invention proposes a construction site measurement system based on image processing to solve the above problems. Summary of the Invention

[0007] To overcome the above defects of the prior art and to achieve the above object, the present invention provides the following technical solution: A construction site measurement system based on image processing, comprising:

[0008] A data acquisition module for acquiring construction site point cloud data and image data;

[0009] A data processing module for preprocessing the construction site point cloud data and image data to obtain a point cloud parameter data set and an image parameter data set; extracting features from the point cloud parameter data set and the image parameter data set to obtain a point cloud feature data set and an image feature data set;

[0010] A defect detection module for training to obtain a defect detection model according to the image parameter data set, and obtaining construction structure defect data through the defect detection model;

[0011] A 3D reconstruction module for constructing a 3D model of the construction site according to the point cloud feature data set and the image feature data set;

[0012] A visualization module for mapping the obtained construction structure defect data to the 3D model of the construction site through 3D coordinate transformation to generate a 3D heat model of the construction site; visually displaying construction deviations through the 3D heat model of the construction site; Each module is connected by wired and / or wireless means.

[0013] Further, the construction site point cloud data is point data with attributes; the attributes include timestamp, point coordinates, reflection intensity, construction site elements and environmental status; the image data is multi-view image data of the construction site;

[0014] The method for collecting point cloud data of a construction site is to scan the construction site with a lidar, and then obtain the point cloud data in the three-dimensional space; the method for collecting image data is to take images of the construction site from different angles and positions with a camera device, and then obtain the image data.

[0015] Further, the method for preprocessing the point cloud data of the construction site to obtain a point cloud parameter data set is as follows:

[0016] Adopt the Gaussian filtering algorithm to remove the noise and interference points contained in the original point cloud data; adopt the voxel grid filter to perform downsampling operation to reduce the density of the point cloud data and obtain the downsampled point cloud data; use neighborhood analysis to identify and remove the abnormal points in the downsampled point cloud data; introduce the RANSAC algorithm for plane segmentation to obtain the preprocessed point cloud data; collect the preprocessed point cloud data to obtain the point cloud parameter data set.

[0017] Further, the method for preprocessing the image data to obtain an image parameter data set is as follows:

[0018] Perform smoothing processing, distortion correction, and image enhancement on the image data respectively to obtain the image parameter data set;

[0019] The method for performing smoothing processing on the image data is as follows: perform smoothing processing on the image data using Gaussian filtering; select an appropriate standard deviation of the Gaussian distribution, calculate the Gaussian function; generate a Gaussian kernel according to the Gaussian function and perform normalization processing; use the normalized Gaussian kernel to perform a convolution operation on the image data to obtain the smoothed image data;

[0020] The method for performing distortion correction on the image data is as follows: use a calibration board to take multiple groups of images, estimate the internal parameter matrix and distortion coefficient of the camera; select a corresponding distortion model according to the obtained internal parameter matrix and distortion coefficient to correct the image data; through inverse mapping, map each pixel point in the distorted image to the corrected image coordinate system to obtain the undistorted image data;

[0021] The method for performing image enhancement on the image data is as follows: adopt histogram equalization to adjust the gray distribution of the image data; perform color balance adjustment on the image by modifying the color channels of the image.

[0022] Further, the method for feature extraction from the point cloud parameter data set includes:

[0023] For each point in the point cloud parameter dataset and its neighborhood, use the principal component analysis method to calculate the centroid of the neighborhood points; construct a covariance matrix and perform eigenvalue decomposition on it to obtain eigenvalues, and denote the eigenvector corresponding to the smallest eigenvalue as the normal vector of the point; summarize the normal vectors corresponding to each point to obtain normal vector feature data; according to the eigenvalues obtained from the covariance matrix, use the curvature formula to calculate the curvature feature data;

[0024] Divide the point cloud parameter dataset into voxels of fixed size, and calculate the point density within each voxel, and then obtain the voxelized feature data; based on the normal vector relationship between neighborhood points, construct a fast point feature histogram, and then obtain the FPFH feature data; combine the obtained normal vector feature data, curvature feature data, voxelized feature data and FPFH feature data to form a point cloud feature dataset.

[0025] Furthermore, the method for feature extraction of the image parameter dataset includes:

[0026] Adopt the Canny edge detection algorithm to extract edge feature data, and distinguish strong edges, weak edges and non-edges according to the set high and low thresholds; detect the corner positions through the structure tensor matrix and extract corner feature data; extract texture feature data by calculating the co-occurrence relationship between gray-level pixel pairs; the specific steps for extracting the texture feature data are: construct a gray-level co-occurrence matrix ; traverse the entire image parameter dataset, and count the frequencies of pixel pairs with different gray levels in the specified direction in the image; limit the frequencies of pixel pairs with different gray levels through the pixel occurrence frequency limit formula; the pixel occurrence frequency limit formula is: ; where is the number of times that the pixel with the restricted gray level and the pixel with the gray level appear; is the area of the pixel pair; is the total number of all gray-level combinations; is the total number of pixels in the image; is the number of pixels in the horizontal direction; is the number of pixels in the vertical direction;

[0027] Normalize the gray-level co-occurrence matrix to obtain ; where is the normalized gray-level co-occurrence matrix; is the frequency of the pixel with the gray level and the pixel with the gray level appearing in the specified direction;

[0028] According to Calculate the texture features of the image parameter dataset, extract the contrast feature, entropy feature, energy feature and homogeneity feature, and further integrate them to obtain the texture feature data;

[0029] Summarize the obtained edge feature data, corner feature data and texture feature data to obtain the image feature dataset.

[0030] Furthermore, the training method of the defect detection model includes:

[0031] Divide the dataset into a training set, a validation set and a test set; the sample set is a subset of the dataset, and each sample set includes the historical image parameter dataset and the corresponding building structure defect data;

[0032] Construct a defect detection model and perform multi-classification task processing; the defect detection model includes an input layer, a convolutional layer, a pooling layer, a fully connected layer and an output layer; the input layer is used to input the image parameter dataset, and the output layer is used to generate the corresponding building structure defect data according to different task types; the defect detection model is a 3D convolutional neural network model;

[0033] Use the mean absolute error and cross entropy as the loss function to measure the error between the predicted value and the actual value of the model; use the training set data to train the model, and minimize the loss function through the Adam optimizer; use the validation set to evaluate the performance of the model and tune the hyperparameters of the model until the model performance no longer improves significantly or reaches the preset stop condition, and then stop to obtain the trained defect detection model;

[0034] Use the test set to evaluate the performance of the model in the prediction task, and input the current image parameter dataset into the trained defect detection model to obtain the building structure defect data.

[0035] Furthermore, the building structure defect data is the output of the defect detection model performing different types of tasks, specifically including the defect type, defect area and defect severity.

[0036] Furthermore, the method for constructing the 3D model of the construction site according to the point cloud feature dataset and the image feature dataset includes:

[0037] Use the Alpha Shapes algorithm to construct and optimize the 3D mesh for the point cloud feature dataset to obtain the 3D mesh model; map the image feature dataset onto the 3D mesh model to obtain the 3D model of the construction site; the specific steps are:

[0038] S91. Preset the point cloud feature dataset as ; where is the point cloud feature dataset; is the in the point cloud feature dataset. 3D data points; is the index of the three-dimensional data point;

[0039] S92, construct Delaunay triangulation; decompose the point cloud feature data set into non-overlapping tetrahedron;

[0040] S93, Settings Parameters, screen all tetrahedrons; traverse each tetrahedron in the Delaunay triangulation , calculate the radius of its circumscribed sphere If satisfied , then the tetrahedron is retained, otherwise it is discarded;

[0041] Said The parameters are adjusted by the limiting factor formula; the mathematical expression of the limiting factor formula is: ;in, For the adjusted parameter; is the detail control coefficient, which controls the compactness of the generated shape; is the scale of point cloud feature data; is the distance from each 3D data point to its nearest neighbor in the point cloud feature dataset; is the average distance from the three-dimensional data point to the nearest neighbor; is the overall density of the point cloud feature dataset; is the local change rate of the point cloud feature dataset; and are any two three-dimensional data points in the point cloud feature data set; and is the index of any two three-dimensional data points in the point cloud feature dataset; is the maximum boundary distance; An adjustment factor to control the impact of point cloud feature data scale;

[0042] S94, traversing all retained tetrahedrons, extracting triangular facets corresponding to each tetrahedron; eliminating triangular facets shared by two or more adjacent tetrahedrons, and retaining triangular facets exclusively occupied by only one tetrahedron;

[0043] S95, combining all the retained tetrahedrons and triangular facets to form a final triangular mesh structure, thereby obtaining a three-dimensional mesh model;

[0044] S96, mapping the image feature data set to the three-dimensional mesh model through the camera projection model and bilinear interpolation method; using the LM algorithm to adjust the mapping relationship through iteration to minimize the projection error;

[0045] Through the processing of the above steps, a three-dimensional model of the construction site is finally obtained.

[0046] Furthermore, the method for generating a 3D thermal model of the construction site includes:

[0047] Use the homogeneous transformation matrix to perform coordinate transformation on the defect area data in the building structure defect data, and normalize the defect severity to the interval of 0 to 1; use the RBF interpolation method to construct a continuous defect field, map the values of the defect field to color values using a color mapping function, and obtain a 3D thermal map;

[0048] Use Open3D to superimpose the three-dimensional model of the construction site and the 3D thermal map, apply the color values to the surface of the three-dimensional model of the construction site, and finally obtain a 3D thermal model of the construction site.

[0049] The technical effects and advantages of the present invention are based on a construction site measurement system based on image processing:

[0050] By combining edge features, corner features, and texture features, the details and structure of the image are described multi-dimensionally, improving the expression ability of the feature dataset; the normalized co-occurrence matrix standardizes the frequency of pixel pair occurrences, avoiding eigenvalue distortion due to different image sizes; ensuring the consistency and robustness of the feature extraction results among different images; using the frequency limit formula can screen the data in the co-occurrence matrix, reducing the storage and calculation amount of redundant data, thereby improving the calculation efficiency; the feature dataset after feature extraction is more compact than the original data, facilitating model training and storage management; by controlling the frequency of pixel pairs through the formula, the feature data is not interfered by extreme pixel distributions, improving the reliability of the features;

[0051] Through the Alpha Shapes algorithm and Delaunay triangulation, a three-dimensional mesh with rich details is constructed; the triangular patches are optimized to remove wrinkled patches, making the model structure clearer and more compact, avoiding data redundancy; using the external earth radius screening and limiting factor formula, the compactness of the generated shape can be controlled; mapping the image feature dataset onto the three-dimensional mesh model can enhance the visual expression and detail restoration ability of the model; using Delaunay triangulation to divide the point cloud into non-overlapping tetrahedrons, avoiding repeated calculations and improving the calculation efficiency of three-dimensional reconstruction; the camera projection model and bilinear interpolation method are used to accurately map the image information onto the three-dimensional model, ensuring a high degree of consistency between the geometric structure and the image texture; after combining the three-dimensional model with the image data, a more realistic virtual construction site scene can be generated, thereby monitoring, planning, and simulating the construction site; the carried three-dimensional model can reflect the building structure, material stacking positions, and equipment layouts, helping to improve construction management efficiency. Brief Description of the Drawings

[0052] Figure 1 Schematic structural diagram of a construction site measurement system based on image processing according to the present invention;

[0053] Figure 2 Schematic flow diagram of a construction site measurement method based on image processing according to the present invention. Specific embodiments

[0054] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0055] Embodiment 1

[0056] Please refer to Figure 1 As shown, a construction site measurement system based on image processing in this embodiment includes:

[0057] A data acquisition module for acquiring construction site point cloud data and image data;

[0058] A data processing module for preprocessing the construction site point cloud data and image data to obtain a point cloud parameter data set and an image parameter data set; extracting features from the point cloud parameter data set and the image parameter data set to obtain a point cloud feature data set and an image feature data set;

[0059] A defect detection module for training and obtaining a defect detection model according to the image parameter data set, and obtaining construction structure defect data through the defect detection model;

[0060] A 3D reconstruction module for constructing a 3D model of the construction site according to the point cloud feature data set and the image feature data set;

[0061] A visualization module for mapping the obtained construction structure defect data to the 3D model of the construction site through 3D coordinate transformation to generate a 3D heat model of the construction site; visually displaying the construction deviation through the 3D heat model of the construction site; each module is connected by wired and / or wireless means.

[0062] The construction site point cloud data is point data with attributes; the attributes include timestamp, point coordinates, reflection intensity, construction site elements, and environmental status; the image data is multi-view image data of the construction site;

[0063] The method for collecting point cloud data of a construction site is to scan the construction site with a lidar to obtain point cloud data in a three-dimensional space; the method for collecting image data is to take images of the construction site from different angles and positions with a camera device to obtain image data.

[0064] The method for preprocessing the point cloud data of the construction site to obtain a point cloud parameter dataset is as follows:

[0065] Adopt a Gaussian filtering algorithm to remove the noise and interference points contained in the original point cloud data; adopt a voxel grid filter to perform downsampling operations to reduce the density of the point cloud data and obtain downsampled point cloud data; use neighborhood analysis to identify and remove the abnormal points in the downsampled point cloud data; introduce the RANSAC algorithm for plane segmentation of key structures such as walls and floors to lay a solid foundation for subsequent analysis and obtain the preprocessed point cloud data; collect the preprocessed point cloud data to obtain a point cloud parameter dataset.

[0066] The method for preprocessing the image data to obtain an image parameter dataset is as follows:

[0067] Perform smoothing processing, distortion correction, and image enhancement on the image data respectively to obtain an image parameter dataset;

[0068] The method for performing smoothing processing on the image data is as follows: perform smoothing processing on the image data using Gaussian filtering; select an appropriate standard deviation of the Gaussian distribution, calculate the Gaussian function; generate a Gaussian kernel according to the Gaussian function and perform normalization processing so that the sum of all elements is 1 to maintain the image brightness; use the normalized Gaussian kernel to perform convolution operations on the image data to obtain the smoothed image data;

[0069] The method for performing distortion correction on the image data is as follows: use a calibration board to take multiple groups of images, estimate the internal parameter matrix and distortion coefficients of the camera; select the corresponding distortion model (such as radial distortion and tangential distortion) according to the obtained internal parameter matrix and distortion coefficients to correct the image data; through inverse mapping, map each pixel point in the distorted image to the corrected image coordinate system to obtain undistorted image data;

[0070] The method for performing image enhancement on the image data is as follows: adopt histogram equalization to adjust the gray-scale distribution of the image data and enhance the overall contrast of the image; perform color balance adjustment on the image by modifying the color channels of the image;

[0071] For example: when the green color in the image is too prominent, such as the color of vegetation is too bright, reduce the brightness of the green color channel of the image for color balance adjustment; the specific method is as follows:

[0072] Traverse each pixel of the image, multiply the green channel by a coefficient less than 1, and adjust the red and blue channels simultaneously to maintain color harmony; assume that each pixel value of the green channel is multiplied by 0.8, which means the brightness of the green channel will be reduced by 20% to obtain the adjusted green channel;

[0073] Similarly, multiply each pixel value of the red and blue channels by 1.1, which means the brightness of the red and blue channels will each increase by 10% to obtain the adjusted red and blue channels;

[0074] Merge the adjusted green, red, and blue channels to obtain the image with color balance adjusted.

[0075] The method for feature extraction of the point cloud parameter dataset includes:

[0076] For each point and its neighborhood in the point cloud parameter dataset, use the principal component analysis method to calculate the centroid of the neighborhood points; construct a covariance matrix and perform eigenvalue decomposition on it to obtain eigenvalues, and record the eigenvector corresponding to the smallest eigenvalue as the normal vector of the point; summarize the normal vectors corresponding to each point to obtain the normal vector feature data; according to the eigenvalues obtained from the covariance matrix, use the curvature formula to calculate the curvature feature data;

[0077] Divide the point cloud parameter dataset into voxels of fixed size, calculate the point density within each voxel, and then obtain the voxelization feature data; based on the normal vector relationship between neighborhood points, construct a fast point feature histogram, and then obtain the FPFH feature data; combine the obtained normal vector feature data, curvature feature data, voxelization feature data, and FPFH feature data to form the point cloud feature dataset.

[0078] The method for feature extraction of the image parameter dataset includes:

[0079] Adopt the Canny edge detection algorithm to extract edge feature data, and distinguish strong edges, weak edges, and non-edges according to the set high and low thresholds; detect the corner positions through the structure tensor matrix and extract corner feature data; extract texture feature data by calculating the co-occurrence relationship between gray-level pixel pairs; the specific steps for extracting the texture feature data are: construct a gray-level co-occurrence matrix ; traverse the entire image parameter dataset, and count the frequencies of pixel pairs with different gray levels in the specified directions in the image; the specified directions are defined based on the coordinate system of the image pixels, including horizontal direction, vertical direction, and diagonal direction;

[0080] For example: Suppose there is a simple 4x4 grayscale image with a grayscale range from 0 to 3, i.e., 4 grayscale levels. Select the horizontal direction and count the frequency of occurrence of grayscale combinations of adjacent pixel pairs in the horizontal direction. The specific steps are as follows: Traverse each pixel of the image from left to right and top to bottom. For each pixel, we check the grayscale of its adjacent pixel on the right and count the frequency of occurrence of each grayscale combination.

[0081] If the grayscale of a certain pixel is 1 and the grayscale of its adjacent pixel on the right is 2, then increment the count at the position (1, 2) in the gray-level co-occurrence matrix; repeat the above process until all grayscale combinations of adjacent pixel pairs in the horizontal direction are counted; finally, obtain a 4x4 gray-level co-occurrence matrix, and each element in the matrix represents the frequency of occurrence of grayscale combinations of adjacent pixel pairs in the horizontal direction.

[0082] Limit the frequency of occurrence of pixel pairs with different grayscale levels through the pixel occurrence frequency limit formula; the pixel occurrence frequency limit formula is: ; where is the number of times the pixel with the restricted grayscale level and the pixel with the grayscale level appear; is the area of the pixel pair; is the total number of all grayscale combinations; is the total number of pixels in the image; is the number of pixels in the horizontal direction; is the number of pixels in the vertical direction; Through this formula, the element values in the gray-level co-occurrence matrix can be constrained to avoid the excessive frequency of certain grayscale combinations, thus ensuring that the calculated texture features are not affected by the deviation of a single pixel combination;

[0083] For example: Suppose the image data is a low-resolution image with a size of 100x100, 8 grayscale levels, and a single pixel size of 1; Through the pixel occurrence frequency limit formula, we can obtain , that is, the maximum number of times any grayscale combination appears in this image is 156 times;

[0084] Normalize the gray-level co-occurrence matrix to obtain ; where is the normalized gray-level co-occurrence matrix; is the frequency of occurrence of the pixel with the grayscale level and the pixel with the grayscale level in the specified direction;

[0085] According to Calculate the texture features of the image parameter dataset, extract the contrast feature, entropy feature, energy feature, and homogeneity feature, and further integrate them to obtain texture feature data;

[0086] Summarize the obtained edge feature data, corner feature data, and texture feature data to obtain an image feature dataset.

[0087] The training method of the defect detection model includes:

[0088] Divide the dataset into a training set, a validation set, and a test set; the sample set is a subset of the dataset, and each sample set includes a historical image parameter dataset and corresponding building structure defect data;

[0089] Construct a defect detection model and perform multi-classification task processing; the defect detection model includes an input layer, a convolutional layer, a pooling layer, a fully connected layer, and an output layer; the input layer is used to input the image parameter dataset, and the output layer is used to generate corresponding building structure defect data according to different task types; the defect detection model is a 3D convolutional neural network model;

[0090] Use the mean absolute error and cross entropy as loss functions to measure the error between the predicted value and the actual value of the model; use the training set data to train the model, and minimize the loss function through the Adam optimizer; use the validation set to evaluate the performance of the model, tune the hyperparameters of the model until the model performance no longer improves significantly or reaches the preset stop condition, and then stop to obtain a trained defect detection model;

[0091] Use the test set to evaluate the performance of the model in the prediction task, input the current image parameter dataset into the trained defect detection model, and obtain the building structure defect data.

[0092] The method for constructing a three-dimensional model of a construction site based on the point cloud feature dataset and the image feature dataset includes:

[0093] Use the Alpha Shapes algorithm to construct and optimize a three-dimensional mesh for the point cloud feature dataset to obtain a three-dimensional mesh model; map the image feature dataset onto the three-dimensional mesh model and minimize the reconstruction error to obtain a three-dimensional model of the construction site; the specific steps are:

[0094] S91. Preset the point cloud feature dataset as ; where is the point cloud feature dataset; is the th three-dimensional data point in the point cloud feature dataset; is the index of the three-dimensional data point;

[0095] S92. Construct a Delaunay triangulation; decompose the point cloud feature dataset into non-overlapping tetrahedrons through Delaunay triangulation, and there are no other 3D data points inside the circumscribed sphere of any tetrahedron;

[0096] S93. Set parameters to screen all tetrahedrons; traverse each tetrahedron in the Delaunay triangulation and calculate its circumscribed sphere radius ; if it meets , then retain the tetrahedron, otherwise discard it;

[0097] The parameters are adjusted through the restriction factor formula; the mathematical expression of the restriction factor formula is: ; where is the adjusted parameter; is the detail control coefficient, which controls the compactness of the generated shape; is the scale of the point cloud feature data; is the distance from each 3D data point in the point cloud feature dataset to its nearest neighbor; is the average distance from the 3D data point to its nearest neighbor; is the overall density of the point cloud feature dataset; is the local change rate of the point cloud feature dataset; and are any two 3D data points in the point cloud feature dataset; and are the indices of any two 3D data points in the point cloud feature dataset; is the maximum boundary distance; is the adjustment coefficient that controls the influence of the point cloud feature data scale;

[0098] By adjusting the parameters, the sparsity or compactness of the model can be controlled: a larger parameter generates a geometric shape close to the convex hull of the point cloud, and a smaller parameter generates a more detailed structure, revealing the internal structure of the point cloud;

[0099] For example: Suppose there is a point cloud feature dataset with a scale of 1000, where the overall density of the point cloud feature data is 0.05, the local change rate is 0.1, the maximum boundary distance is 10, the required detail is 0.8, and the adjustment coefficient is ; through the restriction factor formula, it can be obtained that the adjusted parameter is: ;

[0100] S94. Traverse all the remaining tetrahedrons, extract the triangular patches corresponding to each tetrahedron; remove the triangular patches shared by two or more adjacent tetrahedrons, and retain the triangular patches exclusive to only one tetrahedron.

[0101] S95. Combine all the remaining tetrahedrons and triangular patches to form the final triangular mesh structure, and then obtain the three-dimensional mesh model.

[0102] S96. Map the image feature dataset onto the three-dimensional mesh model through the camera projection model and the bilinear interpolation method; use the LM algorithm to iteratively adjust the mapping relationship to minimize the projection error.

[0103] Through the processing of the above steps, finally, we can obtain a complete three-dimensional model of the construction site with image textures. The texture information of the original image is pasted on each triangular patch, making the three-dimensional model more realistic and detailed.

[0104] The method for generating the 3D thermal model of the construction site includes:

[0105] Use the homogeneous transformation matrix to perform coordinate transformation on the defect area data in the building structure defect data, and normalize the defect severity to the interval from 0 to 1; use the RBF interpolation method to construct a continuous defect field, and map the values of the defect field to color values using the color mapping function to obtain a 3D thermal map.

[0106] For example: when the color values are at different numerical values, the represented defect severities are different, and specifically, it can be shown as: : Blue (slight defect); : Green (medium defect); : Red (severe defect);

[0107] Use Open3D to overlay the three-dimensional model of the construction site with the 3D thermal map, apply the color values to the surface of the three-dimensional model of the construction site, and finally obtain the 3D thermal model of the construction site.

[0108] In this embodiment, by combining edge features, corner features, and texture features, the details and structure of the image are described in multiple dimensions, improving the expression ability of the feature dataset; the normalized co-occurrence matrix standardizes the frequency of pixel pair occurrences, avoiding the distortion of feature values caused by different image sizes; ensuring the consistency and robustness of the feature extraction results among different images; using the frequency limit formula can screen the data in the co-occurrence matrix, reducing the storage and calculation amount of redundant data, thereby improving the calculation efficiency; the dataset after feature extraction is more compact than the original data, facilitating model training and storage management; by controlling the frequency of pixel pairs through the formula, the feature data is not interfered by extreme pixel distributions, improving the reliability of the features.

[0109] Through the Alpha Shapes algorithm and Delaunay triangulation, a three-dimensional mesh with rich details is constructed; the triangular patches are optimized to remove wrinkled patches clearly, making the model structure clearer and more compact, and avoiding data redundancy; the external earth radius screening and limiting factor formula are used to control the compactness of the generated shape; mapping the image feature dataset onto the three-dimensional mesh model can enhance the visual expression and detail restoration ability of the model; using Delaunay triangulation to divide the point cloud into non-overlapping tetrahedrons avoids repeated calculations and improves the computational efficiency of three-dimensional reconstruction; the camera projection model and bilinear interpolation method are used to accurately map the image information onto the three-dimensional model, ensuring a high degree of consistency between the geometric structure and the image texture; after combining the three-dimensional model with the image data, a more realistic virtual construction site scene can be generated, thereby monitoring, planning, and simulating the construction site; the carried three-dimensional model can reflect the building structure, the location of material stacking, and the equipment layout, which helps to improve the construction management efficiency.

[0110] Example 2

[0111] Please refer to Figure 2 As shown, a construction site measurement method based on image processing in this embodiment includes:

[0112] S1. Collect the point cloud data and image data of the construction site;

[0113] S2. Preprocess the point cloud data and image data of the construction site to obtain a point cloud parameter dataset and an image parameter dataset; extract features from the point cloud parameter dataset and the image parameter dataset to obtain a point cloud feature dataset and an image feature dataset;

[0114] S3. Train and obtain a defect detection model according to the image parameter dataset, and obtain the building structure defect data through the defect detection model;

[0115] S4. Construct a three-dimensional model of the construction site according to the point cloud feature dataset and the image feature dataset;

[0116] S5. Map the obtained building structure defect data into the three-dimensional model of the construction site through three-dimensional coordinate transformation to generate a 3D thermal model of the construction site; visually display the construction deviation through the 3D thermal model of the construction site; each module is connected by wired and / or wireless means.

[0117] Since the electronic device introduced in this embodiment is the electronic device used in the construction site measurement system based on image processing in the embodiments of the present application, based on the construction site measurement system based on image processing introduced in the embodiments of the present application, those skilled in the art can understand the specific implementation manners and various variations of the electronic device in this embodiment. Therefore, the specific implementation of how this electronic device implements the method in the embodiments of the present application will not be described in detail here. As long as those skilled in the art implement the electronic device used in the construction site measurement system based on image processing in the embodiments of the present application, it falls within the protection scope of the present application.

[0118] The above formulas are all dimensionless and take their numerical calculations. The formula is a formula obtained by collecting a large amount of data for software simulation to approximate the real situation. The preset parameters and threshold selection in the formula are set by those skilled in the art according to the actual situation.

[0119] The above are only the preferred implementation manners of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions within the idea of the present invention belong to the protection scope of the present invention. It should be noted that for ordinary technical users in the technical field, several improvements and refinements made without departing from the principle of the present invention should also be regarded as within the protection scope of the present invention.

Claims

1. A construction site measurement system based on image processing, characterized in that: include: Data acquisition module, used to collect construction site cloud data and image data; A data processing module is used to pre-process the point cloud data and image data of the construction site to obtain a point cloud parameter data set and an image parameter data set; Perform feature extraction on the point cloud parameter data set and the image parameter data set to obtain a point cloud feature data set and an image feature data set; A defect detection module is used to obtain a defect detection model based on image parameter data set training, and obtain building structure defect data through the defect detection model; A 3D reconstruction module, used to construct a 3D model of a construction site based on a point cloud feature dataset and an image feature dataset; A visualization module is used to map the acquired building structure defect data into a three-dimensional model of the construction site through three-dimensional coordinate transformation to generate a 3D thermal model of the construction site; The construction deviations are visualized through the 3D thermal model of the construction site; the modules are connected by wire and / or wireless means; The method for constructing a three-dimensional model of a construction site according to a point cloud feature data set and an image feature data set comprises: The Alpha Shapes algorithm is used to construct and optimize the three-dimensional grid of the point cloud feature data set to obtain a three-dimensional grid model; the image feature data set is mapped to the three-dimensional grid model to obtain a three-dimensional model of the construction site; the specific steps are as follows: S91, preset point cloud feature dataset is ;in, It is a point cloud feature dataset; The point cloud feature dataset 3D data points; is the index of the three-dimensional data point; S92, construct Delaunay triangulation; decompose the point cloud feature data set into non-overlapping tetrahedron; S93, Settings Parameters, screen all tetrahedrons; traverse each tetrahedron in the Delaunay triangulation , calculate the radius of its circumscribed sphere If satisfied , then the tetrahedron is retained, otherwise it is discarded; Said The parameters are adjusted by the limiting factor formula; the mathematical expression of the limiting factor formula is: ;in, For the adjusted parameter; is the detail control coefficient, which controls the compactness of the generated shape; is the scale of point cloud feature data; is the distance from each 3D data point to its nearest neighbor in the point cloud feature dataset; is the average distance from the three-dimensional data point to the nearest neighbor; is the overall density of the point cloud feature dataset; is the local change rate of the point cloud feature dataset; and Any two three-dimensional data points in the point cloud feature data set; and is the index of any two three-dimensional data points in the point cloud feature dataset; is the maximum boundary distance; Adjustment coefficient to control the impact of point cloud feature data scale; S94, traversing all retained tetrahedrons, extracting triangular facets corresponding to each tetrahedron; eliminating triangular facets shared by two or more adjacent tetrahedrons, and retaining triangular facets exclusively occupied by only one tetrahedron; S95, combining all the retained tetrahedrons and triangular facets to form a final triangular mesh structure, thereby obtaining a three-dimensional mesh model; S96, mapping the image feature data set to the three-dimensional mesh model through the camera projection model and bilinear interpolation method; using the LM algorithm to adjust the mapping relationship through iteration to minimize the projection error; Through the processing of the above steps, a three-dimensional model of the construction site is finally obtained.

2. A construction site measurement system based on image processing according to claim 1, characterized in that: The construction site point cloud data is point data with attributes; the attributes include timestamp, point coordinates, reflection intensity, construction site elements and environmental status; the image data is multi-view image data of the construction site; The method for collecting point cloud data of a construction site is to scan the construction site through a laser radar, and then obtain point cloud data in three-dimensional space; the method for collecting image data is to use a camera device to shoot images of the construction site from different angles and positions, and then obtain image data.

3. The construction site measurement system based on image processing according to claim 2, characterized in that: The method for preprocessing the construction site point cloud data to obtain a point cloud parameter data set is: The Gaussian filtering algorithm is used to remove the noise and interference points contained in the original point cloud data; the voxel grid filter is used to implement downsampling operations to reduce the density of point cloud data and obtain downsampled point cloud data; the neighborhood analysis is used to identify and remove abnormal points in the downsampled point cloud data; the RANSAC algorithm is introduced for plane segmentation to obtain preprocessed point cloud data; Collect the preprocessed point cloud data to obtain the point cloud parameter data set.

4. The construction site measurement system based on image processing according to claim 3, characterized in that: The method for preprocessing the image data to obtain the image parameter data set is: The image data is smoothed, the distortion is corrected and the image is enhanced respectively to obtain an image parameter data set; The method for smoothing the image data is as follows: using Gaussian filtering to smooth the image data; selecting a suitable Gaussian distribution standard deviation and calculating the Gaussian function; generating a Gaussian kernel according to the Gaussian function and performing normalization processing; using the normalized Gaussian kernel to perform a convolution operation on the image data to obtain smoothed image data; The method for performing distortion correction on image data is as follows: using a calibration plate to shoot multiple sets of images, estimating the intrinsic parameter matrix and distortion coefficient of the camera; selecting a corresponding distortion model according to the obtained intrinsic parameter matrix and distortion coefficient to correct the image data; Through reverse mapping, each pixel in the distorted image is mapped to the corrected image coordinate system to obtain distortion-free image data; The method for performing image enhancement on image data is: using histogram equalization to adjust the grayscale distribution of image data, and adjusting the color balance of the image by modifying the color channel of the image.

5. The construction site measurement system based on image processing according to claim 4, characterized in that: The method for extracting features from a point cloud parameter data set comprises: The principal component analysis method is used for each point and its neighborhood in the point cloud parameter data set to calculate the centroid of the neighborhood points; the covariance matrix is ​​constructed and the eigenvalues ​​are obtained by eigendecomposition, and the eigenvector corresponding to the minimum eigenvalue is recorded as the normal vector of the point; the normal vectors corresponding to each point are summarized to obtain the normal vector feature data; according to the eigenvalues ​​obtained from the covariance matrix, the curvature feature data is calculated using the curvature formula; The point cloud parameter dataset is divided into voxels of fixed size are generated, and the point density is calculated in each voxel to obtain voxelized feature data; based on the normal vector relationship between neighboring points, a fast point feature histogram is constructed to obtain FPFH feature data; the obtained normal vector feature data, curvature feature data, voxelized feature data and FPFH feature data are combined to form a point cloud feature data set.

6. The construction site measurement system based on image processing according to claim 5, characterized in that: The training method of the defect detection model includes: The data set is divided into a training set, a validation set and a test set; the sample set is a subset of the data set, and each sample set includes a historical image parameter data set and corresponding building structure defect data; Construct a defect detection model to perform multi-classification task processing; the defect detection model includes an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer; the input layer is used to input an image parameter data set, and the output layer is used to generate corresponding building structure defect data according to different task types; the defect detection model is a 3D convolutional neural network model; Use mean absolute error and cross entropy as loss functions to measure the error between the model's predicted value and the actual value; use the training set data to train the model and use the Adam optimizer to minimize the loss function; use the validation set to evaluate the model's performance and tune the model's hyperparameters until the model performance no longer improves significantly or reaches the preset stopping condition, thus obtaining a trained defect detection model; The test set is used to evaluate the performance of the model in the prediction task, and the current image parameter dataset is input into the trained defect detection model to obtain the building structure defect data.

7. The construction site measurement system based on image processing according to claim 6, characterized in that: The building structure defect data is the output of the defect detection model performing different types of tasks, specifically including defect type, defect area and defect severity.

8. The construction site measurement system based on image processing according to claim 7, characterized in that: The method for generating a 3D thermal model of a construction site comprises: The coordinate system of the defect area data in the building structure defect data is transformed using a homogeneous transformation matrix, and the defect severity is normalized to the range of 0 to 1; the continuous defect field is constructed using the RBF interpolation method, and the value of the defect field is mapped to a color value using a color mapping function to obtain a 3D thermal map; Open3D is used to overlay the 3D model of the construction site with the 3D thermal map, so that the color value is applied to the surface of the 3D model of the construction site, and finally the 3D thermal model of the construction site is obtained.

Images