Construction site measuring system based on image processing

Abstract

The invention belongs to the technical field of measurement, and discloses a construction site measurement system based on image processing, and the system comprises a data collection module which is used for collecting point cloud data and image data of a construction site; the data processing module is used for carrying out preprocessing and feature extraction on the point cloud data and the image data of the construction site to obtain a point cloud feature data set and an image feature data set; the defect detection module is used for training according to the image parameter data set to obtain a defect detection model and obtaining building structure defect data through the defect detection model; the three-dimensional reconstruction module is used for constructing a construction site three-dimensional model according to the point cloud feature data set and the image feature data set; the visualization module is used for generating a 3D thermal model of the construction site; the construction deviation is visually displayed through the 3D thermal model of the construction site, so that the construction of the construction site is monitored and planned, and the construction management efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of measurement technology, and more particularly to a construction site measurement system based on image processing. Background Art

[0002] The patent application with the publication number CN112149543A discloses a construction dust recognition system and method based on computer vision. First, the size and resolution of the acquired single-frame construction site image are initialized, and then the image is pre-processed by Gaussian filtering. The processed image is converted to a color model. The morphological opening operation of the binary image is first eroded and then expanded to eliminate the influence of small objects and expand the edge area to obtain the background and foreground images, which are merged into a mask image. The ratio of the pixel area of ​​the extracted construction dust to the total pixel area of ​​the image is calculated by the pixel area method to determine whether the image has construction dust. The image with construction dust is saved and an identification alarm signal is output. The present invention is a system that automatically completes the automatic recognition and alarm of construction dust by combining computer vision, image processing and modern communication technology, which solves the disadvantages of the existing dust detector such as long measurement cycle and poor accuracy, and is helpful for the monitoring work of management personnel.

[0003] The existing construction site measurement system has the following main problems: Failure to combine edge features, corner features, and texture features results in an incomplete description of image details and complex structures, affecting the model's recognition and judgment capabilities; ignoring corner features will cause the model to be unable to accurately locate the key points of the target; missing texture affects the recognition of materials and patterns; when the co-occurrence matrix is ​​not normalized, the pixel frequency is affected by the image size, resulting in eigenvalue distortion; failure to use a restriction formula to screen the data in the co-occurrence matrix training, a large amount of data frequency will increase the storage and computing burden, affecting the efficiency of the model; high computing costs and increased processing time, especially in the application of large-scale image data sets, may lead to a decline in system performance; failure to control the frequency of pixel pairs, extreme pixel allocation will dominate the feature data, causing the model to be overly sensitive to these abnormal data, thereby reducing the reliability of the features; in abnormal environments, feature data may destroy the normal distribution, affecting the accuracy and stability of the model; Failure to use the Alpha Shapes algorithm and Delaunay triangulation will result in a rough three-dimensional mesh structure that cannot accurately reflect complex architectural details; failure to optimize triangular faces and remove shared faces will lead to wavy data and repeated calculations in the model, which will not only load storage burdens, but also increase computational complexity, reduce real-time rendering and simulation performance, and affect the efficiency of construction simulation; failure to use the circumscribed sphere radius screening and restriction factor formula will result in a lack of control over the generated model, making the shape loose and affecting the compactness and expression of the three-dimensional model; failure to accurately map image data to the three-dimensional model through the camera projection model and bilinear interpolation method will result in misalignment or distortion between the geometric structure and the texture image; unclear stereoscopic faces and data generation will lead to excessive complexity of the three-dimensional model, increased storage and rendering pressure, and make the system unable to respond in real-time scenarios.

[0004] In view of this, the present invention proposes a construction site measurement system based on image processing to solve the above problems. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned purpose, the present invention provides the following technical solution: a construction site measurement system based on image processing, comprising: Data acquisition module, used to collect construction site cloud data and image data; The data processing module is used to pre-process the point cloud data and image data of the construction site to obtain a point cloud parameter data set and an image parameter data set; perform feature extraction on the point cloud parameter data set and the image parameter data set to obtain a point cloud feature data set and an image feature data set; A defect detection module is used to obtain a defect detection model based on image parameter data set training, and obtain building structure defect data through the defect detection model; A 3D reconstruction module, used to construct a 3D model of a construction site based on a point cloud feature dataset and an image feature dataset; The visualization module is used to map the acquired building structure defect data to the three-dimensional model of the construction site through three-dimensional coordinate transformation, and generate a 3D thermal model of the construction site; the construction deviation is visualized through the 3D thermal model of the construction site; and each module is connected by wired and / or wireless means.

[0006] Furthermore, the construction site point cloud data is point data with attributes; the attributes include timestamp, point coordinates, reflection intensity, construction site elements and environmental status; the image data is multi-view image data of the construction site; The method for collecting point cloud data of a construction site is to scan the construction site through a laser radar, and then obtain point cloud data in three-dimensional space; the method for collecting image data is to use a camera device to shoot images of the construction site from different angles and positions, and then obtain image data.

[0007] Furthermore, the method of preprocessing the construction site point cloud data to obtain the point cloud parameter data set is: A Gaussian filtering algorithm is used to remove noise and interference points contained in the original point cloud data. A voxel grid filter is used to perform downsampling operations to reduce the density of point cloud data and obtain downsampled point cloud data. Neighborhood analysis is used to identify and remove abnormal points in the downsampled point cloud data. The RANSAC algorithm is introduced for plane segmentation to obtain preprocessed point cloud data. The preprocessed point cloud data is collected to obtain a point cloud parameter data set.

[0008] Furthermore, the method of preprocessing the image data to obtain the image parameter data set is as follows: The image data is smoothed, the distortion is corrected and the image is enhanced respectively to obtain an image parameter data set; The method for smoothing the image data is as follows: using Gaussian filtering to smooth the image data; selecting a suitable Gaussian distribution standard deviation and calculating the Gaussian function; generating a Gaussian kernel according to the Gaussian function and performing normalization processing; using the normalized Gaussian kernel to perform a convolution operation on the image data to obtain smoothed image data; The method for performing distortion correction on image data is as follows: using a calibration plate to shoot multiple sets of images, estimating the intrinsic parameter matrix and distortion coefficient of the camera; selecting a corresponding distortion model to correct the image data according to the obtained intrinsic parameter matrix and distortion coefficient; and mapping each pixel point in the distorted image to the corrected image coordinate system through reverse mapping to obtain distortion-free image data; The method for performing image enhancement on image data is: using histogram equalization to adjust the grayscale distribution of image data; and adjusting the color balance of the image by modifying the color channel of the image.

[0009] Furthermore, the method for extracting features from a point cloud parameter data set includes: The principal component analysis method is used for each point and its neighborhood in the point cloud parameter data set to calculate the centroid of the neighborhood points; the covariance matrix is ​​constructed and the eigenvalues ​​are obtained by eigendecomposition, and the eigenvector corresponding to the minimum eigenvalue is recorded as the normal vector of the point; the normal vectors corresponding to each point are summarized to obtain the normal vector feature data; according to the eigenvalues ​​obtained from the covariance matrix, the curvature feature data is calculated using the curvature formula; The point cloud parameter dataset is divided into voxels of fixed size are generated, and the point density is calculated in each voxel to obtain voxelized feature data; based on the normal vector relationship between neighboring points, a fast point feature histogram is constructed to obtain FPFH feature data; the obtained normal vector feature data, curvature feature data, voxelized feature data and FPFH feature data are combined to form a point cloud feature data set.

[0010] Furthermore, the method for extracting features from the image parameter data set includes: The Canny edge detection algorithm is used to extract edge feature data, and strong edges, weak edges and non-edges are distinguished according to the set high and low thresholds; the corner point positions are detected through the structure tensor matrix to extract the corner point feature data; the texture feature data is extracted by calculating the symbiotic relationship between grayscale pixel pairs; the specific steps of extracting texture feature data are: constructing a grayscale symbiotic matrix ; Traverse the entire image parameter data set, count the frequency of occurrence of pixel pairs of different gray levels in the image in the specified direction; limit the frequency of occurrence of pixel pairs of different gray levels through the pixel occurrence frequency limiting formula; the pixel occurrence frequency limiting formula is: ;in, The gray level after limiting is The pixel and gray level are The number of times the pixel appears; is the area of ​​the pixel pair; is the total number of all gray level combinations; is the total number of pixels in the image; is the number of pixels horizontally; is the number of pixels vertically; The gray-level co-occurrence matrix is ​​normalized to obtain ;in, is the normalized gray-level co-occurrence matrix; In the specified direction, the gray level is The pixel and gray level are The frequency of the pixel appearance; according to The texture features of the image parameter data set are calculated, and contrast features, entropy features, energy features, and homogeneity features are extracted, which are further integrated to obtain texture feature data; The acquired edge feature data, corner feature data and texture feature data are summarized to obtain an image feature data set.

[0011] Furthermore, the training method of the defect detection model includes: The data set is divided into a training set, a validation set and a test set; the sample set is a subset of the data set, and each sample set includes a historical image parameter data set and corresponding building structure defect data; Construct a defect detection model to perform multi-classification task processing; the defect detection model includes an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer; the input layer is used to input an image parameter data set, and the output layer is used to generate corresponding building structure defect data according to different task types; the defect detection model is a 3D convolutional neural network model; Use mean absolute error and cross entropy as loss functions to measure the error between the model's predicted value and the actual value; use the training set data to train the model and use the Adam optimizer to minimize the loss function; use the validation set to evaluate the model's performance and tune the model's hyperparameters until the model performance no longer improves significantly or reaches the preset stopping condition, thus obtaining a trained defect detection model; The test set is used to evaluate the performance of the model in the prediction task, and the current image parameter dataset is input into the trained defect detection model to obtain the building structure defect data.

[0012] Furthermore, the building structure defect data is the output of the defect detection model performing different types of tasks, specifically including defect type, defect area and defect severity.

[0013] Furthermore, the method for constructing a three-dimensional model of a construction site according to a point cloud feature data set and an image feature data set includes: The Alpha Shapes algorithm is used to construct and optimize the three-dimensional grid of the point cloud feature data set to obtain a three-dimensional grid model; the image feature data set is mapped to the three-dimensional grid model to obtain a three-dimensional model of the construction site; the specific steps are as follows: S91, preset point cloud feature dataset is ;in, It is a point cloud feature dataset; The point cloud feature dataset 3D data points; is the index of the three-dimensional data point; S92, construct Delaunay triangulation; decompose the point cloud feature data set into non-overlapping tetrahedron; S93, Settings Parameters, screen all tetrahedrons; traverse each tetrahedron in the Delaunay triangulation , calculate the radius of its circumscribed sphere If satisfied , then the tetrahedron is retained, otherwise it is discarded; Said The parameters are adjusted by the limiting factor formula; the mathematical expression of the limiting factor formula is: ;in, For the adjusted parameter; is the detail control coefficient, which controls the compactness of the generated shape; is the scale of point cloud feature data; is the distance from each 3D data point to its nearest neighbor in the point cloud feature dataset; is the average distance from the three-dimensional data point to the nearest neighbor; is the overall density of the point cloud feature dataset; is the local change rate of the point cloud feature dataset; and are any two three-dimensional data points in the point cloud feature data set; and is the index of any two three-dimensional data points in the point cloud feature dataset; is the maximum boundary distance; An adjustment factor to control the impact of point cloud feature data scale; S94, traversing all retained tetrahedrons, extracting triangular facets corresponding to each tetrahedron; eliminating triangular facets shared by two or more adjacent tetrahedrons, and retaining triangular facets exclusively occupied by only one tetrahedron; S95, combining all the retained tetrahedrons and triangular facets to form a final triangular mesh structure, thereby obtaining a three-dimensional mesh model; S96, mapping the image feature data set to the three-dimensional mesh model through the camera projection model and bilinear interpolation method; using the LM algorithm to adjust the mapping relationship through iteration to minimize the projection error; Through the processing of the above steps, a three-dimensional model of the construction site is finally obtained.

[0014] Furthermore, the method for generating a 3D thermal model of a construction site includes: The coordinate system of the defect area data in the building structure defect data is transformed using a homogeneous transformation matrix, and the defect severity is normalized to the range of 0 to 1; the continuous defect field is constructed using the RBF interpolation method, and the value of the defect field is mapped to a color value using a color mapping function to obtain a 3D thermal map; Open3D is used to overlay the 3D model of the construction site with the 3D thermal map, so that the color value is applied to the surface of the 3D model of the construction site, and finally the 3D thermal model of the construction site is obtained.

[0015] The present invention is based on the technical effects and advantages of a construction site measurement system based on image processing: By combining edge features, corner features and texture features, the details and structure of the image are described in multiple dimensions, improving the expressiveness of the feature data set; the normalized co-occurrence matrix standardizes the frequency of pixel pairs to avoid eigenvalue distortion due to different image sizes; ensuring that the results of feature extraction are consistent and robust between different images; the frequency limit formula can be used to filter the data in the co-occurrence matrix, reducing the storage and calculation of redundant data, thereby improving computing efficiency; the dataset after feature extraction is more compact than the original data, which is conducive to model training and storage management; the frequency of pixel pairs is controlled by the formula, so that the feature data is not disturbed by extreme pixel distribution, improving the reliability of the feature; Through the Alpha Shapes algorithm and Delaunay triangulation, a detailed 3D mesh is constructed; triangular facets are optimized to clarify and remove wrinkles, making the model structure clearer and more compact, avoiding data redundancy; the use of external earth radius screening and limiting factor formulas can control the compactness of the generated shape; mapping the image feature data set to the 3D mesh model can enhance the model's visual expression and detail restoration capabilities; using Delaunay triangulation to divide the point cloud into non-overlapping tetrahedrons to avoid repeated calculations and improve the computational efficiency of 3D reconstruction; the camera projection model and bilinear interpolation method are used to accurately map image information to the 3D model to ensure a high degree of consistency between the geometric structure and the image texture; after combining the 3D model with the image data, a more realistic virtual construction site scene can be generated, thereby monitoring, planning and simulating the construction site; the carried 3D model can reflect the building structure, material stacking location and equipment layout, which helps to improve construction management efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of the structure of a construction site measurement system based on image processing according to the present invention; Figure 2 The figure is a flow chart of a construction site measurement method based on image processing according to the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] Example 1 See also Figure 1 As shown, this embodiment is a construction site measurement system based on image processing, including: Data acquisition module, used to collect construction site cloud data and image data; The data processing module is used to pre-process the point cloud data and image data of the construction site to obtain a point cloud parameter data set and an image parameter data set; perform feature extraction on the point cloud parameter data set and the image parameter data set to obtain a point cloud feature data set and an image feature data set; A defect detection module is used to obtain a defect detection model based on image parameter data set training, and obtain building structure defect data through the defect detection model; A 3D reconstruction module, used to construct a 3D model of a construction site based on a point cloud feature dataset and an image feature dataset; The visualization module is used to map the acquired building structure defect data to the three-dimensional model of the construction site through three-dimensional coordinate transformation, and generate a 3D thermal model of the construction site; the construction deviation is visualized through the 3D thermal model of the construction site; and each module is connected by wired and / or wireless means.

[0019] The construction site point cloud data is point data with attributes; the attributes include timestamp, point coordinates, reflection intensity, construction site elements and environmental status; the image data is multi-view image data of the construction site; The method for collecting point cloud data of a construction site is to scan the construction site through a laser radar, and then obtain point cloud data in three-dimensional space; the method for collecting image data is to use a camera device to shoot images of the construction site from different angles and positions, and then obtain image data.

[0020] The method for preprocessing the construction site point cloud data to obtain a point cloud parameter data set is: A Gaussian filtering algorithm is used to remove noise and interference points contained in the original point cloud data. A voxel grid filter is used to perform downsampling operations to reduce the density of point cloud data and obtain downsampled point cloud data. Neighborhood analysis is used to identify and remove abnormal points in the downsampled point cloud data. The RANSAC algorithm is introduced to perform plane segmentation, such as key structures such as walls and ground, to lay a solid foundation for subsequent analysis and obtain preprocessed point cloud data. The preprocessed point cloud data is collected to obtain a point cloud parameter data set.

[0021] The method for preprocessing the image data to obtain the image parameter data set is: The image data is smoothed, the distortion is corrected and the image is enhanced respectively to obtain an image parameter data set; The method for smoothing the image data is as follows: using Gaussian filtering to smooth the image data; selecting a suitable Gaussian distribution standard deviation and calculating the Gaussian function; generating a Gaussian kernel according to the Gaussian function and normalizing it so that the sum of all elements is 1 to maintain the image brightness; using the normalized Gaussian kernel to perform a convolution operation on the image data to obtain smoothed image data; The method for performing distortion correction on image data is as follows: using a calibration plate to shoot multiple sets of images, estimating the camera's intrinsic parameter matrix and distortion coefficient; selecting a corresponding distortion model (such as radial distortion and tangential distortion) to correct the image data based on the obtained intrinsic parameter matrix and distortion coefficient; and mapping each pixel point in the distorted image to the corrected image coordinate system through reverse mapping to obtain distortion-free image data; The method of image enhancement for image data is: using histogram equalization to adjust the grayscale distribution of image data and enhance the overall contrast of the image; adjusting the color balance of the image by modifying the color channel of the image; For example, when the green in the image is too prominent, such as when the vegetation color is too bright, reduce the brightness of the green channel of the image to adjust the color balance; the specific method is: Traverse each pixel of the image, multiply the green channel by a coefficient less than 1, and adjust the red and blue channels to maintain color harmony; suppose each pixel value of the green channel is multiplied by 0.8, which means that the brightness of the green channel will be reduced by 20%, and the adjusted green channel is obtained; Similarly, multiply each pixel value of the red channel and the blue channel by 1.1, which means that the brightness of the red channel and the blue channel will be increased by 10% respectively, and the adjusted red channel and blue channel are obtained; Merge the adjusted green channel, red channel, and blue channel to obtain an image with color balance adjusted.

[0022] The method for extracting features from a point cloud parameter data set comprises: The principal component analysis method is used for each point and its neighborhood in the point cloud parameter data set to calculate the centroid of the neighborhood points; the covariance matrix is ​​constructed and the eigenvalues ​​are obtained by eigendecomposition, and the eigenvector corresponding to the minimum eigenvalue is recorded as the normal vector of the point; the normal vectors corresponding to each point are summarized to obtain the normal vector feature data; according to the eigenvalues ​​obtained from the covariance matrix, the curvature feature data is calculated using the curvature formula; The point cloud parameter dataset is divided into voxels of fixed size are generated, and the point density is calculated in each voxel to obtain voxelized feature data; based on the normal vector relationship between neighboring points, a fast point feature histogram is constructed to obtain FPFH feature data; the obtained normal vector feature data, curvature feature data, voxelized feature data and FPFH feature data are combined to form a point cloud feature data set.

[0023] Methods for extracting features from image parameter datasets include: The Canny edge detection algorithm is used to extract edge feature data, and strong edges, weak edges and non-edges are distinguished according to the set high and low thresholds; the corner point positions are detected through the structure tensor matrix to extract the corner point feature data; the texture feature data is extracted by calculating the symbiotic relationship between grayscale pixel pairs; the specific steps of extracting texture feature data are: constructing a grayscale symbiotic matrix ; Traverse the entire image parameter data set and count the frequency of pixel pairs of different gray levels in the image appearing in the specified direction; the specified direction is defined based on the coordinate system of the image pixels, including horizontal direction, vertical direction and diagonal direction; For example: suppose there is a simple 4x4 grayscale image with a grayscale range of 0 to 3, that is, 4 grayscale levels; select the horizontal direction and count the frequency of grayscale combinations of adjacent pixel pairs in the horizontal direction. The specific steps are: traverse each pixel of the image from left to right and from top to bottom. For each pixel, we check its grayscale with the adjacent pixel on the right and count the frequency of each grayscale combination.

[0024] If the gray level of a pixel is 1 and the gray level of its adjacent pixel on the right is 2, then the count is increased at the position (1,2) in the gray level co-occurrence matrix; repeat the above process until all gray level combinations of adjacent pixel pairs in the horizontal direction are counted; finally, a 4x4 gray level co-occurrence matrix is ​​obtained, and each element in the matrix represents the frequency of occurrence of gray level combinations of adjacent pixel pairs in the horizontal direction.

[0025] The frequency of occurrence of pixel pairs of different gray levels is limited by a pixel occurrence frequency limiting formula; the pixel occurrence frequency limiting formula is: ;in, The gray level after limiting is The pixel and gray level are The number of times the pixel appears; is the area of ​​the pixel pair; is the total number of all gray level combinations; is the total number of pixels in the image; is the number of pixels horizontally; is the number of pixels in the vertical direction; through this formula, the element values ​​in the gray-level co-occurrence matrix can be constrained to avoid excessive frequencies of certain gray-level combinations, thereby ensuring that the calculated texture features will not be affected by the deviation of a single pixel combination; For example, assuming that the image data is a low-resolution image, the image size is 100x100, the grayscale level is 8, and the size of a single pixel is 1; the pixel frequency limit formula can be used to obtain , that is, any grayscale combination appears in the image at most 156 times; The gray-level co-occurrence matrix is ​​normalized to obtain ;in, is the normalized gray-level co-occurrence matrix; In the specified direction, the gray level is The pixel and gray level are The frequency of the pixel appearance; according to The texture features of the image parameter data set are calculated, and contrast features, entropy features, energy features, and homogeneity features are extracted, which are further integrated to obtain texture feature data; The acquired edge feature data, corner feature data and texture feature data are summarized to obtain an image feature data set.

[0026] The training method of the defect detection model includes: The data set is divided into a training set, a validation set and a test set; the sample set is a subset of the data set, and each sample set includes a historical image parameter data set and corresponding building structure defect data; Construct a defect detection model to perform multi-classification task processing; the defect detection model includes an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer; the input layer is used to input an image parameter data set, and the output layer is used to generate corresponding building structure defect data according to different task types; the defect detection model is a 3D convolutional neural network model; Use mean absolute error and cross entropy as loss functions to measure the error between the model's predicted value and the actual value; use the training set data to train the model and use the Adam optimizer to minimize the loss function; use the validation set to evaluate the model's performance and tune the model's hyperparameters until the model performance no longer improves significantly or reaches the preset stopping condition, thus obtaining a trained defect detection model; The test set is used to evaluate the performance of the model in the prediction task, and the current image parameter dataset is input into the trained defect detection model to obtain the building structure defect data.

[0027] The method for constructing a three-dimensional model of a construction site according to a point cloud feature data set and an image feature data set comprises: The Alpha Shapes algorithm is used to construct and optimize the three-dimensional grid of the point cloud feature data set to obtain a three-dimensional grid model; the image feature data set is mapped to the three-dimensional grid model to minimize the reconstruction error and obtain a three-dimensional model of the construction site; the specific steps are: S91, preset point cloud feature dataset is ;in, It is a point cloud feature dataset; The point cloud feature dataset 3D data points; is the index of the three-dimensional data point; S92, construct Delaunay triangulation; decompose the point cloud feature data set into non-overlapping tetrahedrons, the interior of the circumscribed sphere of any tetrahedron does not contain any other three-dimensional data points; S93, Settings Parameters, screen all tetrahedrons; traverse each tetrahedron in the Delaunay triangulation , calculate the radius of its circumscribed sphere If satisfied , then the tetrahedron is retained, otherwise it is discarded; Said The parameters are adjusted by the limiting factor formula; the mathematical expression of the limiting factor formula is: ;in, For the adjusted parameter; is the detail control coefficient, which controls the compactness of the generated shape; is the scale of point cloud feature data; is the distance from each 3D data point to its nearest neighbor in the point cloud feature dataset; is the average distance from the three-dimensional data point to the nearest neighbor; is the overall density of the point cloud feature dataset; is the local change rate of the point cloud feature dataset; and are any two three-dimensional data points in the point cloud feature data set; and is the index of any two three-dimensional data points in the point cloud feature dataset; is the maximum boundary distance; An adjustment factor to control the impact of point cloud feature data scale; By adjusting Parameter, which controls the sparseness or density of the model: Large The geometry generated by the parameters is close to the convex hull of the point cloud and is smaller The parameters generate more detailed structures, revealing the internal structure of the point cloud; For example, suppose there is a point cloud feature dataset of size 1000, where the overall density of the point cloud feature data is 0.05, the local variation rate is 0.1, the maximum boundary distance is 10, the required detail is 0.8, and the adjustment coefficient is ; Through the limiting factor formula, it can be concluded that the adjusted The parameters are: ; S94, traversing all retained tetrahedrons, extracting triangular facets corresponding to each tetrahedron; eliminating triangular facets shared by two or more adjacent tetrahedrons, and retaining triangular facets exclusively occupied by only one tetrahedron; S95, combining all the retained tetrahedrons and triangular facets to form a final triangular mesh structure, thereby obtaining a three-dimensional mesh model; S96, mapping the image feature data set to the three-dimensional mesh model through the camera projection model and bilinear interpolation method; using the LM algorithm to adjust the mapping relationship through iteration to minimize the projection error; Through the above steps, we can finally get a complete three-dimensional model of the construction site with image texture. Each triangular face is affixed with the texture information of the original image, making the three-dimensional model more realistic and detailed.

[0028] The method for generating a 3D thermal model of a construction site comprises: The coordinate system of the defect area data in the building structure defect data is transformed using a homogeneous transformation matrix, and the defect severity is normalized to the range of 0 to 1; the continuous defect field is constructed using the RBF interpolation method, and the value of the defect field is mapped to a color value using a color mapping function to obtain a 3D thermal map; For example, when the color value is at different values, the severity of the defect represented is different, which can be specifically manifested as: : Blue (slight defect); : Green (medium defect); : Red (serious defect); Open3D is used to overlay the 3D model of the construction site with the 3D thermal map, so that the color value is applied to the surface of the 3D model of the construction site, and finally the 3D thermal model of the construction site is obtained.

[0029] In this embodiment, by combining edge features, corner features and texture features, the details and structure of the image are described in multiple dimensions, thereby improving the expressiveness of the feature data set; the normalized co-occurrence matrix standardizes the frequency of occurrence of pixel pairs to avoid distortion of eigenvalues ​​due to different image sizes; and ensures that the results of feature extraction are consistent and robust between different images; the frequency restriction formula can be used to screen the data in the co-occurrence matrix, reduce the storage and calculation of redundant data, and thus improve the calculation efficiency; the feature-extracted data set is more compact than the original data, which is conducive to model training and storage management; the frequency of pixel pairs is controlled by a formula, so that the feature data is not disturbed by extreme pixel distribution, thereby improving the reliability of the feature; Through the Alpha Shapes algorithm and Delaunay triangulation, a detailed 3D mesh is constructed; triangular facets are optimized to clarify and remove wrinkles, making the model structure clearer and more compact, avoiding data redundancy; the use of external earth radius screening and limiting factor formulas can control the compactness of the generated shape; mapping the image feature data set to the 3D mesh model can enhance the model's visual expression and detail restoration capabilities; using Delaunay triangulation to divide the point cloud into non-overlapping tetrahedrons to avoid repeated calculations and improve the computational efficiency of 3D reconstruction; the camera projection model and bilinear interpolation method are used to accurately map image information to the 3D model to ensure a high degree of consistency between the geometric structure and the image texture; after combining the 3D model with the image data, a more realistic virtual construction site scene can be generated, thereby monitoring, planning and simulating the construction site; the carried 3D model can reflect the building structure, material stacking location and equipment layout, which helps to improve construction management efficiency.

[0030] Example 2 See also Figure 2 As shown, this embodiment provides a construction site measurement method based on image processing, including: S1. Collecting construction site cloud data and image data; S2. Preprocessing the construction site point cloud data and image data to obtain a point cloud parameter data set and an image parameter data set; performing feature extraction on the point cloud parameter data set and the image parameter data set to obtain a point cloud feature data set and an image feature data set; S3. Obtain a defect detection model based on the image parameter data set training, and obtain building structure defect data through the defect detection model; S4. construct a three-dimensional model of the construction site based on the point cloud feature dataset and the image feature dataset; S5. Map the acquired building structure defect data to the three-dimensional model of the construction site through three-dimensional coordinate transformation to generate a 3D thermal model of the construction site; visualize the construction deviation through the 3D thermal model of the construction site; and connect each module through wired and / or wireless means.

[0031] Since the electronic device introduced in this embodiment is an electronic device used to implement a construction site measurement system based on image processing in the embodiment of the present application, based on the construction site measurement system based on image processing introduced in the embodiment of the present application, a person skilled in the art can understand the specific implementation of the electronic device of the present embodiment and its various variations, so how the electronic device implements the method in the embodiment of the present application is not described in detail here. As long as a person skilled in the art implements the electronic device used in the construction site measurement system based on image processing in the embodiment of the present application, it belongs to the scope of protection of this application.

[0032] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters and thresholds in the formula are set by technicians in this field according to actual conditions.

[0033] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technical users in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A construction site measurement system based on image processing, characterized in that: include: Data acquisition module, used to collect construction site cloud data and image data; A data processing module is used to pre-process the point cloud data and image data of the construction site to obtain a point cloud parameter data set and an image parameter data set; Perform feature extraction on the point cloud parameter data set and the image parameter data set to obtain a point cloud feature data set and an image feature data set; A defect detection module is used to obtain a defect detection model according to the image parameter data set training, and obtain building structure defect data through the defect detection model; A 3D reconstruction module, used to construct a 3D model of a construction site based on a point cloud feature dataset and an image feature dataset; A visualization module is used to map the acquired building structure defect data into a three-dimensional model of the construction site through three-dimensional coordinate transformation to generate a 3D thermal model of the construction site; The construction deviations are visualized through the 3D thermal model of the construction site; the modules are connected via wired and / or wireless means.

2. A construction site measurement system based on image processing according to claim 1, characterized in that: The construction site point cloud data is point data with attributes; the attributes include timestamp, point coordinates, reflection intensity, construction site elements and environmental status; the image data is multi-view image data of the construction site; The method for collecting point cloud data of a construction site is to scan the construction site through a laser radar, and then obtain point cloud data in three-dimensional space; the method for collecting image data is to use a camera device to shoot images of the construction site from different angles and positions, and then obtain image data.

3. The construction site measurement system based on image processing according to claim 2, characterized in that: The method for preprocessing the construction site point cloud data to obtain a point cloud parameter data set is: The Gaussian filtering algorithm is used to remove the noise and interference points contained in the original point cloud data; the voxel grid filter is used to implement downsampling operations to reduce the density of point cloud data and obtain downsampled point cloud data; the neighborhood analysis is used to identify and remove abnormal points in the downsampled point cloud data; the RANSAC algorithm is introduced for plane segmentation to obtain preprocessed point cloud data; Collect the preprocessed point cloud data to obtain the point cloud parameter data set.

4. The construction site measurement system based on image processing according to claim 3, characterized in that: The method for preprocessing the image data to obtain the image parameter data set is: The image data is smoothed, distorted and enhanced respectively to obtain an image parameter data set; The method for smoothing the image data is as follows: using Gaussian filtering to smooth the image data; selecting a suitable Gaussian distribution standard deviation and calculating the Gaussian function; generating a Gaussian kernel according to the Gaussian function and performing normalization processing; using the normalized Gaussian kernel to perform a convolution operation on the image data to obtain smoothed image data; The method for performing distortion correction on image data is as follows: using a calibration plate to shoot multiple sets of images, estimating the intrinsic parameter matrix and distortion coefficient of the camera; selecting a corresponding distortion model according to the obtained intrinsic parameter matrix and distortion coefficient to correct the image data; Through reverse mapping, each pixel in the distorted image is mapped to the corrected image coordinate system to obtain distortion-free image data; The method for performing image enhancement on image data is: using histogram equalization to adjust the grayscale distribution of image data, and adjusting the color balance of the image by modifying the color channel of the image.

5. The construction site measurement system based on image processing according to claim 4, characterized in that: The method for extracting features from a point cloud parameter data set comprises: The principal component analysis method is used for each point and its neighborhood in the point cloud parameter data set to calculate the centroid of the neighborhood points; the covariance matrix is ​​constructed and the eigenvalues ​​are obtained by eigendecomposition, and the eigenvector corresponding to the minimum eigenvalue is recorded as the normal vector of the point; the normal vectors corresponding to each point are summarized to obtain the normal vector feature data; according to the eigenvalues ​​obtained from the covariance matrix, the curvature feature data is calculated using the curvature formula; The point cloud parameter dataset is divided into voxels of fixed size are generated, and the point density is calculated in each voxel to obtain voxelized feature data; based on the normal vector relationship between neighboring points, a fast point feature histogram is constructed to obtain FPFH feature data; the obtained normal vector feature data, curvature feature data, voxelized feature data and FPFH feature data are combined to form a point cloud feature data set.

6. The construction site measurement system based on image processing according to claim 5, characterized in that: Methods for extracting features from image parameter datasets include: The Canny edge detection algorithm is used to extract edge feature data, and strong edges, weak edges and non-edges are distinguished according to the set high and low thresholds; the corner point positions are detected through the structure tensor matrix to extract the corner point feature data; the texture feature data is extracted by calculating the symbiotic relationship between grayscale pixel pairs; The specific steps of extracting texture feature data are: constructing a gray level co-occurrence matrix ; Traverse the entire image parameter data set, count the frequency of occurrence of pixel pairs of different gray levels in the image in the specified direction; limit the frequency of occurrence of pixel pairs of different gray levels through the pixel occurrence frequency limiting formula; the pixel occurrence frequency limiting formula is: ;in, The gray level after limiting is The pixel and gray level are The number of times the pixel appears; is the area of ​​the pixel pair; is the total number of all gray level combinations; is the total number of pixels in the image; is the number of pixels horizontally; is the number of pixels vertically; The gray-level co-occurrence matrix is ​​normalized to obtain ;in, is the normalized gray-level co-occurrence matrix; In the specified direction, the gray level is The pixel and gray level are The frequency of the pixel appearance; according to The texture features of the image parameter data set are calculated, and contrast features, entropy features, energy features, and homogeneity features are extracted, which are further integrated to obtain texture feature data; The acquired edge feature data, corner feature data and texture feature data are summarized to obtain an image feature data set.

7. The construction site measurement system based on image processing according to claim 6, characterized in that: The training method of the defect detection model includes: The data set is divided into a training set, a validation set and a test set; the sample set is a subset of the data set, and each sample set includes a historical image parameter data set and corresponding building structure defect data; Construct a defect detection model to perform multi-classification task processing; the defect detection model includes an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer; the input layer is used to input an image parameter data set, and the output layer is used to generate corresponding building structure defect data according to different task types; the defect detection model is a 3D convolutional neural network model; Use mean absolute error and cross entropy as loss functions to measure the error between the model's predicted value and the actual value; use the training set data to train the model and use the Adam optimizer to minimize the loss function; use the validation set to evaluate the model's performance and tune the model's hyperparameters until the model performance no longer improves significantly or reaches the preset stopping condition, thus obtaining a trained defect detection model; The test set is used to evaluate the performance of the model in the prediction task, and the current image parameter dataset is input into the trained defect detection model to obtain the building structure defect data.

8. The construction site measurement system based on image processing according to claim 7, characterized in that: The building structure defect data is the output of the defect detection model performing different types of tasks, specifically including defect type, defect area and defect severity.

9. The construction site measurement system based on image processing according to claim 8, characterized in that: The method for constructing a three-dimensional model of a construction site according to a point cloud feature data set and an image feature data set comprises: The Alpha Shapes algorithm is used to construct and optimize the three-dimensional grid of the point cloud feature data set to obtain a three-dimensional grid model; the image feature data set is mapped to the three-dimensional grid model to obtain a three-dimensional model of the construction site; the specific steps are as follows: S91, preset point cloud feature dataset is ;in, It is a point cloud feature dataset; The point cloud feature dataset 3D data points; is the index of the three-dimensional data point; S92, construct Delaunay triangulation; decompose the point cloud feature data set into non-overlapping tetrahedron; S93, Settings Parameters, screen all tetrahedrons; traverse each tetrahedron in the Delaunay triangulation , calculate the radius of its circumscribed sphere If satisfied , then the tetrahedron is retained, otherwise it is discarded; Said The parameters are adjusted by the limiting factor formula; the mathematical expression of the limiting factor formula is: ;in, For the adjusted parameter; is the detail control coefficient, which controls the compactness of the generated shape; is the scale of point cloud feature data; is the distance from each 3D data point to its nearest neighbor in the point cloud feature dataset; is the average distance from the three-dimensional data point to the nearest neighbor; is the overall density of the point cloud feature dataset; is the local change rate of the point cloud feature dataset; and are any two three-dimensional data points in the point cloud feature data set; and is the index of any two three-dimensional data points in the point cloud feature dataset; is the maximum boundary distance; An adjustment factor to control the impact of point cloud feature data scale; S94, traversing all retained tetrahedrons, extracting triangular facets corresponding to each tetrahedron; eliminating triangular facets shared by two or more adjacent tetrahedrons, and retaining triangular facets exclusively occupied by only one tetrahedron; S95, combining all the retained tetrahedrons and triangular facets to form a final triangular mesh structure, thereby obtaining a three-dimensional mesh model; S96, mapping the image feature data set to the three-dimensional mesh model through the camera projection model and bilinear interpolation method; using the LM algorithm to adjust the mapping relationship through iteration to minimize the projection error; Through the processing of the above steps, a three-dimensional model of the construction site is finally obtained.

10. The construction site measurement system based on image processing according to claim 9, characterized in that: The method for generating a 3D thermal model of a construction site comprises: The coordinate system of the defect area data in the building structure defect data is transformed using a homogeneous transformation matrix, and the defect severity is normalized to the range of 0 to 1; the continuous defect field is constructed using the RBF interpolation method, and the value of the defect field is mapped to a color value using a color mapping function to obtain a 3D thermal map; Open3D is used to overlay the 3D model of the construction site with the 3D thermal map, so that the color value is applied to the surface of the 3D model of the construction site, and finally the 3D thermal model of the construction site is obtained.

Images