Building facade model monitoring method and system based on three-dimensional scanning

Abstract

The invention discloses a building facade model monitoring method and system based on three-dimensional scanning, and relates to the technical field of building monitoring, and the method comprises the steps: collecting the three-dimensional data of a building facade, calculating the minimum sampling spacing between points, and screening a sparse point cloud set; and constructing a point cloud feature vector based on the sparse point cloud set, and constructing a label distribution model. According to the method, the point cloud data set with the label has the characteristics of complete semantic information and high classification accuracy, a reliable data basis is provided for subsequent building facade analysis and decision making, qualitative analysis can be carried out on the crack through point cloud segmentation and crack label identification, and the accuracy of classification is improved. In addition, quantitative evaluation of the crack can be assisted through a segmentation result, subsequent maintenance and decision making are facilitated, introduction of a local detail loss function is combined with a high-curvature point characteristic optimization model, and the feature extraction capacity of the local optimization model for the crack area is remarkably enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of building monitoring, and particularly to a method and system for monitoring a building facade model based on three-dimensional scanning. Background Art

[0002] With the accelerated progress of urbanization and the continuous increase in the number of high-rise buildings, precise and real-time monitoring of the building facade has become an important part of building maintenance and urban management. Traditional building facade monitoring methods usually rely on manual inspections or data processing based on two-dimensional images.

[0003] In the prior art, the acquisition of point cloud data often generates a large amount of redundant data, and traditional methods lack efficient point cloud sparsification and feature extraction mechanisms, resulting in excessive consumption of subsequent computing resources and low processing efficiency. Secondly, in building facade monitoring, the classification and label assignment of point cloud data mostly rely on simple rule-based methods, which are difficult to adapt to the diversity of complex building structures, and the classification accuracy and robustness are limited. In addition, in terms of crack identification and its characteristic analysis, existing methods mainly stay in the two-dimensional plane analysis stage, and it is difficult to effectively integrate three-dimensional spatial features and material information, resulting in inaccurate prediction of key characteristics such as crack width and depth. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a method for monitoring a building facade model based on three-dimensional scanning to solve the problems that traditional methods lack efficient point cloud sparsification and feature extraction mechanisms, resulting in excessive consumption of subsequent computing resources and low processing efficiency. Secondly, in building facade monitoring, the classification and label assignment of point cloud data mostly rely on simple rule-based methods, which are difficult to adapt to the diversity of complex building structures, and the classification accuracy and robustness are limited. In addition, in terms of crack identification and its characteristic analysis, existing methods mainly stay in the two-dimensional plane analysis stage, and it is difficult to effectively integrate three-dimensional spatial features and material information, resulting in inaccurate prediction of key characteristics such as crack width and depth.

[0006] To solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a method for monitoring a building facade model based on three-dimensional scanning, which includes:

[0008] Collect three-dimensional data of the building facade, calculate the minimum sampling distance between points, and screen a sparse point cloud set;

[0009] Construct a point cloud feature vector based on the sparse point cloud set and construct a label assignment model;

[0010] Introduce the global consistency loss calculation based on the adjacent point sets of the sparsified point cloud set, optimize the global optimization parameters and obtain the global optimization model, identify the global label classification. Based on the sparsified point cloud set of the global label classification, calculate the point cloud curvature through principal component analysis and perform screening, introduce the local detail loss function and obtain the local optimization model, identify the crack label classification, and perform point cloud segmentation;

[0011] Based on the point cloud segmentation data of the crack label, construct an identification model based on the convolutional neural network to identify the crack category, width and depth, store the prediction data and generate a log file.

[0012] As a preferred solution of the method for monitoring the building facade model based on three-dimensional scanning according to the present invention, wherein: the three-dimensional data of the building facade is collected, integrating a lidar, a millimeter-wave radar and a multispectral camera to collect the three-dimensional data of the building facade, wherein the lidar obtains point cloud data and the multispectral camera obtains the multispectral reflectivity data of the material;

[0013] Perform timestamp alignment on the collected three-dimensional data and perform three-dimensional data preprocessing;

[0014] Use the lidar point cloud as the reference coordinate system to register the millimeter-wave radar data and the multispectral reflectivity data.

[0015] As a preferred solution of the method for monitoring the building facade model based on three-dimensional scanning according to the present invention, wherein: calculate the minimum sampling distance between points, screen the sparsified point cloud set, and perform spatial normalization on the point cloud data set according to the bounding box Bounding Box of the building facade;

[0016] Define the target point cloud density D as the number of points to be retained per cubic meter. Combine the bounding box volume and use the ratio of the number of points to the bounding box volume as the value of the target point cloud density D;

[0017] At the same time, according to the value of the target point cloud density, calculate the minimum sampling distance between points , expressed as:

[0018] ;

[0019] Adopt the Poisson disk sampling algorithm, set the result set, traverse the point cloud data set, and set the conditions for the points in the point cloud data set to be added to the result set, expressed as:

[0020] ;

[0021] where represents the result set, and represent the i-th and j-th points of the point cloud data set respectively;

[0022] Repeat the sampling until all points are inspected, and the resulting set is used as the sparse point cloud set.

[0023] As a preferred solution of the building facade model monitoring method based on 3D scanning according to the present invention, wherein: the point cloud feature vector is constructed based on the sparse point cloud set, the label assignment model is constructed, the three-dimensional coordinate data of the point cloud is normalized, and at the same time, the multispectral reflectance data registered based on the sparse point cloud set is data-standardized, and the three-dimensional coordinates, curvature and multispectral reflectance data of the points are combined into a point cloud feature vector;

[0024] The label assignment model is constructed based on a sparse convolutional network, including an input layer, a sparse convolutional layer, a feature fusion layer, an activation layer, a fully connected classification layer, an assignment layer and an output layer;

[0025] Among them, the input layer receives the point cloud feature vector as input data, and the sparse convolutional layer performs a convolutional operation on the input feature vector;

[0026] The feature fusion layer splices the outputs of the multi-scale convolutional layers into a comprehensive feature vector. The activation layer applies the ReLU activation function to the comprehensive feature vector of each point to obtain the activated feature vector. The fully connected classification layer maps the activated feature vector to a class probability distribution. The assignment layer determines the label for each point according to the maximum classification probability, and the output layer outputs the point cloud data set with labels.

[0027] As a preferred solution of the building facade model monitoring method based on 3D scanning according to the present invention, wherein: the global consistency loss is calculated by introducing the adjacent point set of the sparse point cloud set, the global optimization parameter is optimized to obtain the global optimization model, the global label classification is identified, and based on the sparse point cloud set of the global label classification, the point cloud curvature is calculated by principal component analysis and screened, the local detail loss function is introduced to obtain the local optimization model, the crack label classification is identified, and the point cloud segmentation is performed. The monitoring items based on the building facade include walls, windows, cracks, balconies, and the model training data set is obtained and the classification labels are calibrated for the monitoring items;

[0028] The monitoring items calibrated in the training data set are used as the training data for global monitoring, including walls, windows, balconies, and the model global optimization training is carried out;

[0029] The adjacent point set of each point in the sparse point cloud set is constructed, where the definition of adjacent points is based on the Euclidean distance between the point and the adjacent point. The sum of the historical mean and standard deviation is used as the distance threshold. If the Euclidean distance between the point and the adjacent point is less than or equal to the distance threshold, it is used as an adjacent point;

[0030] The global consistency loss function is introduced based on the output of the sparse convolutional layer for the sparse point cloud set in the label assignment model, expressed as:

[0031] ;

[0032] where represents the global consistency calculation loss, represents the set of adjacent point pairs, and respectively represent the point cloud feature vectors of the i-th and j-th points;

[0033] Use the Adam optimizer for gradient descent optimization. If the loss calculated during consecutive iterations no longer decreases significantly, stop the iteration, output the global optimization parameters, and use the model with the global optimization parameters as the globally optimized model;

[0034] Input the newly acquired sparsified point cloud set into the globally optimized model for global label classification and recognition;

[0035] For each point in the sparsified point cloud set after global label classification, use principal component analysis to calculate the covariance matrix of the point cloud, expressed as:

[0036] ;

[0037] where C represents the covariance matrix, k represents the number of neighborhood points, determined based on historical data, represents the point 's neighborhood point set, represents the mean coordinate of the neighborhood points, represents the coordinate of the j-th neighborhood point, and the superscript T represents the transpose calculation;

[0038] Perform eigenvalue decomposition on the covariance matrix using the numerical calculation tool NumPy to obtain the decomposed eigenvalues , and , where , and 's sum represents the overall distribution density of the neighborhood points;

[0039] Calculate the curvature of each point based on the decomposed eigenvalues, expressed as:

[0040] ;

[0041] where represents the curvature value of the i-th point;

[0042] Based on the sum of the historical mean and standard deviation of the curvature values as the curvature threshold, if the curvature value is greater than or equal to the curvature threshold, then regard the corresponding point cloud as a high-curvature point and obtain the high-curvature point set;

[0043] Calculate the loss using the database data of calibrated high-curvature regions including cracks;

[0044] Introduce a local detail loss function for the high-curvature point set, expressed as:

[0045] ;

[0046] where represents the local detail loss value, represents the high-curvature point set, represents the output of the sparse convolutional layer of the i-th point cloud, represents the true label of the i-th point cloud;

[0047] Use the Adam optimizer for gradient descent optimization. If the loss calculated during consecutive iterations no longer decreases significantly, stop the iteration, output the local optimization parameters, and use the model with the local optimization parameters as the local optimization model;

[0048] For the newly acquired sparse point cloud set, calculate the curvature of the point cloud, and input the high-curvature points into the local optimization model for crack label recognition;

[0049] Perform point cloud segmentation on the point cloud data according to the crack labels.

[0050] As a preferred solution of the building facade model monitoring method based on 3D scanning described in the present invention, wherein: for the point cloud segmentation data based on crack labels, an identification model is constructed based on a convolutional neural network to identify the crack category, width, and depth. For the point cloud segmentation data based on crack labels, the ratio of the band reflection spectral intensity to the maximum value within the band range in the corresponding multi-spectral reflectance data registered according to the point cloud data is used as the band reflection intensity;

[0051] Use a millimeter-wave radar to measure the internal delamination thickness of the material, expressed as:

[0052] ;

[0053] where represents the delamination thickness, represents the propagation speed of electromagnetic waves in the air, represents the echo time;

[0054] Construct a comprehensive feature matrix based on the curvature value, band reflection intensity, and delamination thickness corresponding to the point cloud of the sparse point cloud set;

[0055] Construct an identification model based on a convolutional neural network, including an input layer, a convolutional layer, and an output layer. The input layer inputs the comprehensive feature matrix, the convolutional layer extracts spatial features, and the output layer outputs the crack classification result;

[0056] The output of the output layer includes crack classification output, width prediction, and depth prediction, expressed as:

[0057] ;

[0058] ;

[0059] ;

[0060] where represents the predicted classification result of crack classification, represents the comprehensive feature matrix, represents the classification label, including linear and punctate, represents the model parameters, represents the probability that a point belongs to the crack classification label given the comprehensive feature matrix X and the model parameters θ, represents the width of the crack, represents the depth of the crack, and respectively represent the weights of width and depth predictions, represents the feature vector after convolution, and respectively represent the bias terms of width and depth predictions;

[0061] Define the multi-objective loss function, expressed as:

[0062] ;

[0063] where represents the multi-objective loss value of the recognition model, represents the classification loss, and respectively represent the prediction losses of width and depth, and respectively represent the loss weights of width and depth;

[0064] Use the training data with labeled crack types, widths, and depths to train the model. Select the cross-entropy loss function to calculate the classification prediction loss, width prediction loss, and depth prediction loss respectively. Use the Adam optimizer for gradient descent optimization to update the weights and biases of the recognition model. Converge when the multi-objective loss value of the model no longer decreases significantly during consecutive iterations;

[0065] Use the newly obtained segmented point cloud data to predict whether it is a crack, as well as the predicted values of the width and depth of the crack.

[0066] As a preferred solution of the 3D-scanning-based building facade model monitoring method of the present invention, wherein: the predicted data is stored and a log file is generated, integrating the results from global, local, and crack predictions, as well as the predicted data of width and depth;

[0067] The data is formatted into a JSON file and transmitted to the cloud for backup using the MQTT protocol;

[0068] The key information of each prediction task is recorded, and a log file is generated using a log generation tool.

[0069] In a second aspect, the present invention provides a system for the 3D-scanning-based building facade model monitoring method, including,

[0070] A data acquisition module for acquiring 3D data of the building facade;

[0071] A data processing module for sparsifying and extracting features from the acquired 3D data;

[0072] A label assignment module for constructing a label assignment model based on the sparse point cloud set, introducing global consistency loss, optimizing global parameters, performing global label classification, and further optimizing and identifying the crack area;

[0073] A crack characteristic prediction module for identifying the category, width, and depth of cracks based on the segmentation data of crack labels;

[0074] A data storage module for storing predicted data and generating a log file to record the key information of each task.

[0075] In a third aspect, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and: when the computer program is executed by the processor, any step of the 3D-scanning-based building facade model monitoring method described in the first aspect of the present invention is implemented.

[0076] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and: when the computer program is executed by the processor, any step of the 3D-scanning-based building facade model monitoring method described in the first aspect of the present invention is implemented.

[0077] The beneficial effects of the present invention are as follows: The point cloud data set with tags has the characteristics of complete semantic information and high classification accuracy, providing a reliable data basis for subsequent building facade analysis and decision-making. Through point cloud segmentation and crack label recognition, not only can qualitative analysis of cracks be carried out, but also quantitative evaluation of cracks can be assisted through the segmentation results, facilitating subsequent maintenance and decision-making. The introduction of the local detail loss function combined with the characteristics of high-curvature points optimizes the model, significantly enhancing the feature extraction ability of the local optimization model for crack regions, so that crack label classification and detail segmentation can be completed more accurately. By constructing an identification model based on the point cloud segmentation data with crack labels, the crack classification can be effectively combined with the tasks of predicting width and depth, realizing accurate identification of cracks and comprehensive estimation of characteristic parameters, meeting the multi-dimensional monitoring requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0079] Figure 1 It is a schematic flow chart of the building facade model monitoring method based on 3D scanning in Embodiment 1.

[0080] Figure 2 It is a schematic structural diagram of the building facade model monitoring system based on 3D scanning in Embodiment 1. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0081] In order to make the above objects, features, and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention will be described in detail below with reference to the drawings in the specification.

[0082] In the following description, many specific details are set forth in order to fully understand the present invention. However, the present invention can also be implemented in other ways different from those described herein. Those skilled in the art can make similar generalizations without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0083] Secondly, the so-called "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that can be included in at least one implementation manner of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that excludes other embodiments.

[0084] Embodiment 1, refer to Figure 1 and Figure 2, which is the first embodiment of the present invention. This embodiment provides a method for monitoring the building facade model based on 3D scanning, including the following steps:

[0085] S1, Collect the 3D data of the building facade, calculate the minimum sampling distance between points, and screen the sparse point cloud set;

[0086] Preferably, integrate lidar, millimeter-wave radar and multispectral camera to collect the 3D data of the building facade. The lidar obtains point cloud data, and the multispectral camera obtains the multispectral reflectance data of the material;

[0087] Align the time stamps of the collected 3D data and perform preprocessing on the 3D data;

[0088] Register the millimeter-wave radar data and the multispectral reflectance data using the lidar point cloud as the reference coordinate system.

[0089] By integrating lidar, millimeter-wave radar and multispectral camera to collect the 3D data of the building facade, data on geometric characteristics, material characteristics and structural characteristics can be obtained simultaneously, forming multi-modal information, improving the comprehensiveness and fineness of building facade monitoring. The point cloud data obtained by lidar provides high-precision 3D geometric information, laying a foundation for the shape description of detail areas such as cracks and edges; by capturing the multispectral reflectance data of materials through the multispectral camera, the optical characteristics and aging status of the materials can be reflected, which is helpful for material identification and status assessment when combined with geometric features; through the millimeter-wave radar, the surface material layer can be penetrated to provide information on the thickness of the structural layer and internal anomalies, making up for the limitations of lidar and multispectral camera that cannot capture internal information; through the data fusion process, the accuracy, reliability and diversity of building facade monitoring can be significantly improved, making complex defect and status analysis feasible and efficient.

[0090] Further, calculate the minimum sampling distance between points, screen the sparse point cloud set, and perform spatial normalization based on the bounding box of the building facade for the point cloud dataset;

[0091] Define the target point cloud density D as the number of points to be retained per cubic meter. Combine the volume of the bounding box and use the ratio of the number of points to the volume of the bounding box as the value of the target point cloud density D;

[0092] At the same time, according to the value of the target point cloud density, calculate the minimum sampling distance between points , expressed as:

[0093] ;

[0094] Using the Poisson disk sampling algorithm, set the result set, traverse the point cloud data set, and set the conditions for the points in the point cloud data set to be added to the result set, which are expressed as:

[0095] ;

[0096] Among them represents the result set, and respectively represent the i-th and j-th points of the point cloud data set;

[0097] Repeat sampling until all points are checked, and the obtained result set is used as the sparse point cloud set.

[0098] By calculating the minimum sampling distance between points and screening the sparse point cloud set, the data volume can be significantly reduced while ensuring data accuracy, improving data processing efficiency, and reducing storage and calculation costs; based on the bounding box of the building facade for spatial normalization, the scale consistency of the point cloud data can be ensured, making the data distribution in different regions uniform, thus optimizing the accuracy of subsequent analysis; by defining the target point cloud density D and combining it with the bounding box volume, the sampling density of the point cloud can be dynamically adjusted to ensure the balanced distribution of the point cloud data in regions of different volumes, while meeting the requirements of the point cloud resolution for specific analysis tasks; by calculating the minimum sampling distance between points and combining the Poisson disk sampling algorithm, redundant sampling in the point cloud can be effectively avoided, key points can be retained, and at the same time, the points in the result set are ensured to have a high degree of uniformity and spatial coverage; the Poisson disk sampling algorithm screens the point cloud through conditional constraints, which can retain the representative point set and remove redundant points, so that the sparse point cloud data reduces the occupancy of processing resources while ensuring the integrity of the geometric structure; the finally obtained sparse point cloud set can significantly improve the processing speed and data operation flexibility, laying a more efficient and accurate data foundation for subsequent tasks such as crack identification and multi-modal feature fusion.

[0099] S2. Based on the sparse point cloud set, construct a point cloud feature vector and build a label assignment model;

[0100] Preferably, based on the sparse point cloud set, construct a point cloud feature vector and build a label assignment model. Based on the three-dimensional coordinate data of the point cloud, perform normalization processing, and at the same time perform data standardization based on the multi-spectral reflectance data registered with the sparse point cloud set. Combine the three-dimensional coordinates, curvature, and multi-spectral reflectance data of the points to form a point cloud feature vector;

[0101] Build a label assignment model based on a sparse convolutional network, including an input layer, a sparse convolutional layer, a feature fusion layer, an activation layer, a fully connected classification layer, an assignment layer, and an output layer;

[0102] Among them, the input layer receives the point cloud feature vector as input data, and the sparse convolutional layer performs convolutional operations on the input feature vector;

[0103] The feature fusion layer concatenates the outputs of the multi-scale convolutional layer into a comprehensive feature vector. The activation layer applies the ReLU activation function to the comprehensive feature vector of each point to obtain the activated feature vector. The fully connected classification layer maps the activated feature vector to a class probability distribution. The assignment layer determines the label for each point according to the maximum classification probability, and the output layer outputs the point cloud data set with labels.

[0104] By constructing a point cloud feature vector based on the sparse point cloud set and performing normalization processing, it can ensure that the input data has a consistent scale and numerical range, thereby improving the stability of model training and the effectiveness of feature extraction. Combining with the standardization of multi-spectral reflectance data, it organically combines geometric information and material information to form a point cloud feature vector with high expression ability, providing richer semantic information for the classification task. Using a sparse convolutional network to construct a label assignment model can make full use of the spatial structure of the sparse point cloud, reduce the consumption of computing resources, and at the same time maintain sensitivity to feature details. By concatenating the multi-scale convolutional outputs through the feature fusion layer, it can comprehensively capture local details and global patterns, improving the expression ability and robustness of the classification model. The application of the ReLU activation function further enhances the non-linear expression ability, effectively filtering out invalid features and making the activated feature vector more compact and efficient. The fully connected classification layer maps the feature vector to a class probability distribution, which can flexibly adapt to multi-class tasks and reflect the uncertainty of classification through probability values for further optimization. By determining the label through the maximum value of the classification probability in the assignment layer, it realizes accurate point cloud semantic classification and assigns clear semantic labels to the point cloud data set. The finally output point cloud data set with labels has the characteristics of complete semantic information and high classification accuracy, providing a reliable data basis for subsequent building facade analysis and decision-making.

[0105] S3. Introduce global consistency loss calculation based on the adjacent point set of the sparse point cloud set, optimize the global optimization parameters and obtain the global optimization model, identify the global label classification. Based on the sparse point cloud set of the global label classification, calculate the point cloud curvature through principal component analysis and perform screening, introduce the local detail loss function and obtain the local optimization model, identify the crack label classification, and perform point cloud segmentation;

[0106] Preferably, based on the adjacent point set of the sparsified point cloud set, the global consistency loss is calculated, the global optimization parameters are optimized to obtain the global optimization model, the global label classification is identified. Based on the sparsified point cloud set of the global label classification, the point cloud curvature is calculated by principal component analysis and screened. The local detail loss function is introduced to obtain the local optimization model, the crack label classification is identified, and the point cloud segmentation is performed. The monitoring items based on the building facade include walls, windows, cracks, and balconies. The model training data set is obtained and the classification labels are calibrated for the monitoring items;

[0107] Use the calibrated monitoring items in the training data set as the training data for global monitoring, including walls, windows, and balconies, and perform global optimization training of the model;

[0108] Construct the adjacent point set of each point in the sparsified point cloud set, where the definition of adjacent points is based on the Euclidean distance between the point and the adjacent point. The sum of the historical mean and standard deviation is used as the distance threshold. If the Euclidean distance between the point and the adjacent point is less than or equal to the distance threshold, it is used as an adjacent point;

[0109] Based on the output of the sparse convolution layer for the sparsified point cloud set in the label assignment model, introduce the global consistency loss function, expressed as:

[0110] ;

[0111] where represents the global consistency calculation loss, represents the adjacent point pair set, and represent the point cloud feature vectors of the i-th and j-th points respectively;

[0112] Use the Adam optimizer to perform gradient descent optimization. If the loss calculated in the continuous iteration process no longer decreases significantly, stop the iteration, output the global optimization parameters, and use the model with the global optimization parameters as the global optimization model;

[0113] Input the newly collected sparsified point cloud set into the global optimization model for global label classification recognition;

[0114] For each point in the sparsified point cloud set after global label classification, use principal component analysis to calculate the covariance matrix of the point cloud, expressed as:

[0115] ;

[0116] where C represents the covariance matrix, k represents the number of neighborhood points, determined based on historical data, represents the point 's neighborhood point set, represents the mean coordinate of the neighborhood points, Denote the coordinates of the j-th domain point, and the superscript T represents the transpose calculation;

[0117] Use the numerical calculation tool NumPy to perform eigenvalue decomposition on the covariance matrix to obtain the decomposed eigenvalues 、 and , where 、 and The sum represents the overall distribution density of the neighborhood points;

[0118] Calculate the curvature of each point based on the decomposed eigenvalues, expressed as:

[0119] ;

[0120] where represents the curvature value of the i-th point;

[0121] Based on the sum of the historical mean and standard deviation of the curvature values as the curvature threshold, if the curvature value is greater than or equal to the curvature threshold, the corresponding point cloud is regarded as a high-curvature point, and a high-curvature point set is obtained;

[0122] Use the database data of the calibrated high-curvature region including cracks to calculate the loss;

[0123] Introduce a local detail loss function for the high-curvature point set, expressed as:

[0124] ;

[0125] where represents the local detail loss value, represents the high-curvature point set, represents the output of the sparse convolutional layer of the i-th point cloud, represents the true label of the i-th point cloud;

[0126] Use the Adam optimizer to perform gradient descent optimization. If the loss calculated during continuous iteration no longer decreases significantly, stop the iteration, output the local optimization parameters, and use the model with the local optimization parameters as the local optimization model;

[0127] For the newly acquired sparse point cloud set, calculate the curvature of the point cloud, and input the high-curvature points into the local optimization model for crack label recognition;

[0128] Segment the point cloud data according to the crack label.

[0129] By introducing the calculation of global consistency loss, the continuity and consistency of the global optimization model for the overall classification of building facades can be enhanced, effectively reducing the occurrence of misclassifications and outliers, improving the overall classification accuracy. Calculating the point cloud curvature based on principal component analysis and screening high-curvature points can accurately capture geometric feature regions with significant features such as cracks and edges, providing an important basis for subsequent local optimization and detail recognition. Global label classification can quickly and effectively divide the main components (such as walls, windows, balconies) of large-area building facades, providing a semantic basis for subsequent high-precision crack recognition. Through curvature calculation and high-curvature point screening, the data processing range can be significantly reduced, concentrating resources on key areas, and improving the efficiency and accuracy of crack recognition. The introduction of the local detail loss function combined with the characteristics of high-curvature points optimizes the model, significantly enhancing the feature extraction ability of the local optimization model for crack regions, and thus enabling more accurate crack label classification and detail segmentation. Combining the division of labor and cooperation between the global optimization model and the local optimization model forms an effective connection between large-scale semantic classification and high-precision crack recognition, making the entire monitoring process have both breadth and depth. Using the calibrated high-curvature region data as the basis for loss calculation ensures a clear goal in the model optimization process and further improves the classification reliability for high-risk regions such as cracks;

[0130] Through point cloud segmentation and crack label recognition, not only can qualitative analysis of cracks be carried out, but also quantitative evaluation of cracks can be assisted through the segmentation results, facilitating subsequent maintenance and decision-making. Using adaptive optimization (such as the Adam optimizer) for iterative updating of model parameters can quickly converge and improve the stability of the model, ensuring that the global and local models perform consistently and efficiently on newly acquired data. Integrating the results of global classification and local recognition can comprehensively cover the monitoring requirements of building facades, achieving intelligent processing from large-scale components to fine cracks, and improving the efficiency and accuracy of the overall building health monitoring. Through the above effects, the system can, on the basis of considering both global and local features, realize the full-process automated analysis of building facades from macroscopic component classification to microscopic crack recognition, providing a reliable guarantee for intelligent monitoring.

[0131] S4. Based on the point cloud segmentation data with crack labels, construct an identification model based on a convolutional neural network to identify the crack category, width, and depth, store the prediction data, and generate a log file;

[0132] Preferably, based on the point cloud segmentation data with crack labels, construct an identification model based on a convolutional neural network to identify the crack category, width, and depth. Based on the point cloud segmentation data with crack labels, use the ratio of the band reflectance spectrum intensity to the maximum value within the band range in the corresponding multi-spectral reflectance data registered with the point cloud data as the band reflectance intensity;

[0133] Measure the internal delamination thickness of the material using a millimeter-wave radar, expressed as:

[0134] ;

[0135] where represents the layer thickness, represents the propagation speed of electromagnetic waves in air, represents the echo time;

[0136] Construct a comprehensive feature matrix based on the curvature values, band reflection intensities, and layer thicknesses corresponding to the point clouds in the sparsified point cloud set;

[0137] Build an identification model based on a convolutional neural network, including an input layer, a convolutional layer, and an output layer, where the input layer inputs the comprehensive feature matrix, the convolutional layer extracts spatial features, and the output layer outputs the crack classification result;

[0138] The output of the output layer includes crack classification output, width prediction, and depth prediction, expressed as:

[0139] ;

[0140] ;

[0141] ;

[0142] where represents the predicted classification result of crack classification, represents the comprehensive feature matrix, represents the classification label, including linear and punctiform, represents the model parameter, represents the probability that a point belongs to the crack classification label given the comprehensive feature matrix X and the model parameter θ, represents the width of the crack, represents the depth of the crack, and respectively represent the weights of width and depth predictions, represents the feature vector after convolution, and respectively represent the bias terms of width and depth predictions;

[0143] Define a multi-objective loss function, expressed as:

[0144] ;

[0145] where represents the multi-objective loss value of the identification model, represents the classification loss, and respectively represent the prediction losses of width and depth, and respectively represent the loss weights of width and depth (determined based on experimental data);

[0146] Use the training data with labeled crack types, widths, and depths to train the model. Select the cross - entropy loss function to calculate the classification prediction loss, width prediction loss, and depth prediction loss respectively. Use the Adam optimizer for gradient descent optimization to update the weights and biases of the recognition model. Converge when the multi - objective loss value of the model no longer decreases significantly during consecutive iteration processes;

[0147] Use the newly obtained segmented point cloud data to predict whether it is a crack, as well as the predicted values of the width and depth of the crack.

[0148] By constructing a recognition model based on the point cloud segmentation data with crack labels, it can effectively combine the crack classification with the width and depth prediction tasks, achieve accurate crack recognition and comprehensive estimation of characteristic parameters, meet the multi - dimensional monitoring requirements. Combining the curvature value, band reflection intensity, and layer thickness of the point cloud data to construct a comprehensive feature matrix can make full use of multi - modal information, enhance the model's understanding of the material characteristics and geometric characteristics of the crack area, thereby improving the classification and prediction accuracy. The standardized processing of the band reflection intensity removes the influence of lighting conditions and material surface characteristic differences on the data, making the role of multi - spectral data in the model more stable and reliable. The internal layer thickness of the material measured by the millimeter - wave radar provides supplementary information on the internal structure characteristics of the material, making up for the deficiencies of point cloud and multi - spectral data in internal monitoring, and making the model's prediction of crack depth more accurate. The application of the convolutional neural network extracts spatial features through convolutional layers, can capture complex local patterns in the crack area, and at the same time reduces redundant features, providing more efficient input features for classification and regression tasks. The output layer simultaneously predicts crack classification, width, and depth, and can complete multi - objective tasks in a single inference, improving the efficiency and application value of the model in crack monitoring tasks. By defining a multi - objective loss function and comprehensively considering the optimization objectives of classification, width, and depth prediction, the model can find a balance point between multiple tasks and improve the overall performance.

[0149] Furthermore, store the prediction data and generate a log file, integrating the results from global, local, and crack predictions, as well as the prediction data of width and depth;

[0150] Format the data into a JSON file and use the MQTT protocol to transmit the data to the cloud for backup;

[0151] Record the key information of each prediction task and use a log generation tool to generate a log file.

[0152] By integrating and storing the results of global, local, and crack predictions, as well as the predicted data of width and depth, it can ensure that all prediction information is centrally stored, facilitating subsequent analysis and query, improving data management efficiency. Formatting the integrated data into a JSON file can provide a unified structured data format, facilitating sharing between different platforms and systems, and ensuring data compatibility and usability. Using the MQTT protocol for data transmission to the cloud can achieve lightweight and highly real-time data upload, meeting the efficient communication requirements of distributed building monitoring systems, while ensuring the security and reliability of cloud backups. Generating a log file to record the key information of each prediction task, such as timestamps, task IDs, classification result statistics, etc., helps with the traceability of the monitoring process, facilitating subsequent performance evaluation and anomaly analysis. Automatically recording prediction task information through a log generation tool can save manual operation time, reduce human omissions, and improve the maintainability of the monitoring system.

[0153] This embodiment also provides a system for the monitoring method of a building facade model based on 3D scanning, including:

[0154] A data acquisition module that acquires 3D data of the building facade;

[0155] A data processing module that sparsifies and extracts features from the acquired 3D data;

[0156] A label assignment module that constructs a label assignment model based on the sparse point cloud set, introduces global consistency loss, optimizes global parameters, performs global label classification, and further optimizes and identifies the crack area;

[0157] A crack characteristic prediction module that identifies the category, width, and depth of cracks based on the segmentation data of crack labels;

[0158] A data storage module that stores prediction data and generates a log file to record the key information of each task.

[0159] This embodiment also provides a computer device applicable to the situation of the monitoring method of a building facade model based on 3D scanning, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the monitoring method of the building facade model based on 3D scanning as proposed in the above embodiment.

[0160] The computer device may be a terminal, which includes a processor, a memory, a communication interface, a display screen, and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a carrier network, NFC (Near Field Communication), or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or buttons, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0161] This embodiment also provides a storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the method for monitoring a building facade model based on three-dimensional scanning as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM for short), Electrically Erasable Programmable Read-Only Memory (EEPROM for short), Erasable Programmable Read Only Memory (EPROM for short), Programmable Red-Only Memory (PROM for short), Read-Only Memory (ROM for short), magnetic memory, flash memory, a magnetic disk, or an optical disc.

[0162] In summary, the point cloud data set with tags in the present invention has the characteristics of complete semantic information and high classification accuracy, providing a reliable data basis for subsequent building facade analysis and decision-making. Through point cloud segmentation and crack label recognition, not only can qualitative analysis of cracks be carried out, but also quantitative evaluation of cracks can be assisted through the segmentation results, facilitating subsequent maintenance and decision-making. The introduction of the local detail loss function combined with the characteristics of high-curvature points optimizes the model, significantly enhancing the feature extraction ability of the local optimization model for crack regions, so that crack label classification and detail segmentation can be completed more accurately. By constructing an identification model based on the point cloud segmentation data with crack labels, the crack classification can be effectively combined with the width and depth prediction tasks, realizing accurate identification of cracks and comprehensive estimation of characteristic parameters, meeting the multi-dimensional monitoring requirements.

[0163] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. A building facade model monitoring method based on three-dimensional scanning, characterized in that: include: Collect 3D data of building facades, calculate the minimum sampling distance between points, and filter the sparse point cloud set; Construct point cloud feature vectors based on the sparse point cloud set and build a label assignment model; Based on the adjacent point set of the sparse point cloud set, the global consistency loss calculation is introduced, the global optimization parameters are optimized and the global optimization model is obtained, the global label classification is identified, the point cloud curvature is calculated and screened by principal component analysis based on the sparse point cloud set of global label classification, the local detail loss function is introduced and the local optimization model is obtained, the crack label classification is identified, and the point cloud is segmented; Based on the point cloud segmentation data of crack labels, a recognition model is built based on a convolutional neural network to identify the crack category, width and depth, store the predicted data and generate a log file.

2. The building facade model monitoring method based on three-dimensional scanning according to claim 1, characterized in that: The three-dimensional data of the building facade is collected by integrating a laser radar, a millimeter-wave radar and a multispectral camera to collect the three-dimensional data of the building facade, wherein the laser radar obtains point cloud data and the multispectral camera obtains multispectral reflectivity data of the material; Perform time stamp alignment on the collected 3D data and perform 3D data preprocessing; The millimeter-wave radar data and multispectral reflectivity data are registered using the lidar point cloud as the reference coordinate system.

3. The building facade model monitoring method based on three-dimensional scanning according to claim 2, characterized in that: The minimum sampling distance between the calculation points is selected to sparse the point cloud set, and the spatial normalization is performed according to the bounding box of the building facade based on the point cloud data set; Define the target point cloud density D as the number of points to be retained per cubic meter. Combined with the bounding box volume, the ratio of the number of points to the bounding box volume is taken as the value of the target point cloud density D. At the same time, according to the value of the target point cloud density, the minimum sampling spacing between points is calculated , expressed as: ； Using the Poisson disk sampling algorithm, set the result set, traverse the point cloud dataset, and set the conditions for adding the points of the point cloud dataset to the result set, which is expressed as: ； in Represents the result set, and Respectively represent the i-th and j-th points of the point cloud dataset; Sampling is repeated until all points have been checked, and the resulting set is used as a sparse point cloud set.

4. The building facade model monitoring method based on three-dimensional scanning as claimed in claim 3, characterized in that: The method comprises constructing a point cloud feature vector based on the sparse point cloud set, constructing a label assignment model, performing normalization processing based on the three-dimensional coordinate data of the point cloud, and performing data standardization based on the multispectral reflectance data registered by the sparse point cloud set, and combining the three-dimensional coordinates, curvature and multispectral reflectance data of the point into a point cloud feature vector; A label assignment model is built based on a sparse convolutional network, including an input layer, a sparse convolutional layer, a feature fusion layer, an activation layer, a fully connected classification layer, an assignment layer, and an output layer; The input layer receives the point cloud feature vector as input data, and the sparse convolution layer performs convolution operation on the input feature vector; The feature fusion layer concatenates the outputs of the multi-scale convolutional layers into a comprehensive feature vector. The activation layer applies the ReLU activation function to the comprehensive feature vector of each point to obtain the activated feature vector. The fully connected classification layer maps the activated feature vector to the category probability distribution. The assignment layer determines the label for each point according to the maximum classification probability. The output layer outputs a labeled point cloud dataset.

5. The building facade model monitoring method based on three-dimensional scanning according to claim 4, characterized in that: The adjacent point set based on the sparse point cloud set introduces global consistency loss calculation, optimizes global optimization parameters and obtains a global optimization model, identifies global label classification, calculates point cloud curvature through principal component analysis and performs screening based on the sparse point cloud set with global label classification, introduces local detail loss function and obtains local optimization model, identifies crack label classification, and performs point cloud segmentation, based on the monitoring items of the building facade including walls, windows, cracks, and balconies, obtains a model training data set and calibrates classification labels for monitoring items; Use the calibrated monitoring items in the training data set as global monitoring training data, including walls, windows, and balconies, to perform global optimization training of the model; Construct a set of neighboring points for each point in the sparse point cloud set, where the definition of neighboring points is based on the Euclidean distance between the point and the neighboring points, and the sum of the historical mean and standard deviation is used as the distance threshold. If the Euclidean distance between the point and the neighboring point is less than or equal to the distance threshold, it is considered as a neighboring point; Based on the sparse convolutional layer output for the sparse point cloud set in the label assignment model, a global consistency loss function is introduced, which is expressed as: ； in represents the global consistency calculation loss, represents the set of adjacent point pairs, and Represent the point cloud feature vectors of the i-th and j-th points respectively; Use the Adam optimizer for gradient descent optimization. If the calculated loss no longer decreases significantly during continuous iterations, stop the iteration, output the global optimization parameters, and use the model with the global optimization parameters as the global optimization model. The newly collected sparse point cloud set is input into the global optimization model for global label classification and recognition; For each point in the sparse point cloud set after global label classification, principal component analysis is used to calculate the covariance matrix of the point cloud, which is expressed as: ； Where C represents the covariance matrix, k represents the number of domain points, which is determined based on historical data. Indicate point The domain point set, represents the mean coordinates of the neighborhood points, represents the coordinates of the jth domain point, and the superscript T indicates the transposed calculation; Use the numerical calculation tool NumPy to perform eigenvalue decomposition on the covariance matrix and obtain the decomposed eigenvalues , and ,in , and The sum represents the overall distribution density of neighborhood points; The curvature of each point is calculated based on the decomposed eigenvalues, expressed as: ； in Represents the curvature value of the i-th point; The sum of the historical mean and standard deviation of the curvature value is used as the curvature threshold. If the curvature value is greater than or equal to the curvature threshold, the corresponding point cloud is regarded as a high curvature point, and a high curvature point set is obtained. Loss calculations were performed using calibrated database data of high curvature areas including cracks; A local detail loss function is introduced for high curvature point sets, expressed as: ； in represents the local detail loss value, represents a set of high curvature points, represents the sparse convolutional layer output of the i-th point cloud, Represents the true label of the i-th point cloud; Use the Adam optimizer for gradient descent optimization. If the calculated loss no longer decreases significantly during continuous iterations, the iteration is stopped and the local optimization parameters are output. The model using the local optimization parameters is used as the local optimization model. The newly collected sparse point cloud set is used to calculate the curvature of the point cloud, and the high curvature points are input into the local optimization model to identify crack labels; The point cloud data is segmented according to the crack labels.

6. The building facade model monitoring method based on three-dimensional scanning according to claim 5, characterized in that: The point cloud segmentation data based on the crack label is used to construct a recognition model based on a convolutional neural network to identify the type, width and depth of the crack. The point cloud segmentation data based on the crack label is used to take the ratio of the band reflection spectrum intensity in the corresponding multispectral reflectance data registered with the point cloud data to the maximum value within the band range as the band reflection intensity; The millimeter wave radar is used to measure the internal layer thickness of the material, which is expressed as: ； in represents the layer thickness, The speed at which electromagnetic waves propagate in the air. Indicates echo time; A comprehensive feature matrix is ​​constructed according to the curvature value, band reflection intensity and layer thickness corresponding to the point cloud of the sparse point cloud set; A recognition model is constructed based on a convolutional neural network, including an input layer, a convolution layer, and an output layer. The input layer inputs a comprehensive feature matrix, the convolution layer extracts spatial features, and the output layer outputs crack classification results. The output of the output layer includes crack classification output, width prediction and depth prediction, which are expressed as: ； ； ； in represents the predicted classification result of crack classification, represents the comprehensive feature matrix, Indicates classification labels, including lines and points. represents the model parameters, It represents the probability that a point belongs to a crack classification label given the comprehensive feature matrix X and model parameters θ. represents the width of the crack, Indicates the depth of the crack, and Represent the weights of width and depth prediction respectively, represents the feature vector after convolution, and Denote the bias terms representing width and depth prediction respectively; Define the multi-objective loss function, expressed as: ； in represents the multi-objective loss value of the recognition model, represents the classification loss, and denote the prediction loss of width and depth respectively, and Represent the loss weights of width and depth respectively; The model is trained using training data with annotated crack types, widths, and depths. The cross entropy loss function is selected to calculate the classification prediction loss, width prediction loss, and depth prediction loss respectively. The Adam optimizer is used for gradient descent optimization to update the weights and biases of the recognition model. The model converges when the multi-objective loss value no longer decreases significantly during continuous iterations. The newly acquired segmented point cloud data is used to predict whether it is a crack, as well as the predicted values ​​of the width and depth of the crack.

7. The building facade model monitoring method based on three-dimensional scanning according to claim 6, characterized in that: The storage of prediction data and generation of log files integrates the results from global, local and crack predictions as well as the prediction data for width and depth; Format the data into a JSON file and use the MQTT protocol to transfer the data to the cloud for backup; Record the key information of each prediction task and use the log generation tool to generate a log file.

8. A system for monitoring a building facade model based on three-dimensional scanning, based on the monitoring method for a building facade model based on three-dimensional scanning according to any one of claims 1 to 7, characterized in that: include, Data acquisition module, collecting three-dimensional data of building facades; The data processing module performs sparseness and feature extraction on the collected three-dimensional data; The label assignment module builds a label assignment model based on sparse point cloud sets, introduces global consistency loss, optimizes global parameters, performs global label classification, and further optimizes and identifies crack areas; The crack characteristic prediction module identifies the type, width and depth of the crack based on the segmented data of the crack label; The data storage module stores the prediction data and generates log files to record the key information of each task.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the building facade model monitoring method based on three-dimensional scanning are implemented as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the building facade model monitoring method based on three-dimensional scanning are implemented as described in any one of claims 1 to 7.

Images