Building component surface defect detection method, device and equipment and storage medium

Abstract

The invention discloses a building component surface defect detection method, device and equipment and a storage medium, and the method comprises the steps: obtaining color point cloud data of a to-be-detected building component through a ground laser scanning technology, and generating a three-dimensional grid model based on the color point cloud data; parameterizing the three-dimensional grid model by adopting a UV expansion mode to generate a two-dimensional UV image, and determining a corresponding texture image based on the two-dimensional UV image; and performing defect detection on the texture image by using a preset semantic segmentation model to obtain a defect detection result of the to-be-detected building component. According to the method, the three-dimensional grid model is generated based on the color point cloud data of the building component to be detected, then the three-dimensional grid model is subjected to dimensionality reduction to form the two-dimensional UV graph, and then defect detection is performed, so that compared with the prior art, the method has the advantages that the detection precision of the surface defects of the building component is effectively improved, and the detection cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection, and in particular to a method, device, equipment and storage medium for detecting surface defects of building components. Background Art

[0002] In recent years, terrestrial laser scanning (TLS) technology has been introduced to assist the inspection process by providing detailed point clouds of building surfaces. This advancement has significantly improved the accuracy of measurements and the speed of data collection. Despite this, point clouds acquired by TLS are still mainly used for manual visual inspection, and directly identifying surface defects in these point clouds remains extremely difficult. Many current methods rely mainly on point cloud object recognition and semantic segmentation algorithms. However, these methods face significant challenges in practical applications, such as the difficulty of collecting datasets and the high cost of model training, which hinder their application and widespread promotion.

[0003] In addition, with the advancement of technology, scanners that can obtain color point clouds have gradually become popular. However, traditional point cloud processing algorithms are mostly designed based on three-dimensional geometric information. For example, in point cloud segmentation, the region growing method usually divides the point cloud region by considering the spatial distance and geometric shape similarity between points. When processing color point clouds, these algorithms find it difficult to directly incorporate RGB information into them because RGB information belongs to the category of color space and has different characteristics from traditional geometric space. Therefore, the RGB information embedded in the point cloud is often underestimated or even ignored. These difficulties have led to the fact that although TLS has been widely used, it is still very difficult to extract and utilize information from color point clouds, which in turn affects the accuracy of defect detection.

[0004] Therefore, there is an urgent need for a method for detecting surface defects of building components that can effectively improve the detection accuracy of surface defects of building components and reduce the detection cost. Summary of the Invention

[0005] The main purpose of the present invention is to provide a method, device, equipment and storage medium for detecting surface defects of building components, aiming to solve the technical problem in the existing technology that it is very difficult to extract and utilize information from color point clouds, resulting in low detection accuracy and high detection cost for surface defects of building components.

[0006] To achieve the above object, the present invention provides a method for detecting surface defects of building components, the method comprising the following steps:

[0007] Acquire color point cloud data of the building components to be inspected by terrestrial laser scanning technology, and generate a three-dimensional mesh model based on the color point cloud data;

[0008] Parameterizing the three-dimensional mesh model using a UV unfolding method to generate a two-dimensional UV map, and determining a corresponding texture image based on the two-dimensional UV map;

[0009] Defect detection is performed on the texture image using a preset semantic segmentation model to obtain a defect detection result of the building component to be detected.

[0010] Optionally, the step of parameterizing the three-dimensional mesh model by UV unfolding to generate a two-dimensional UV map, and determining a corresponding texture image based on the two-dimensional UV map includes:

[0011] The three-dimensional mesh model is parameterized by using a UV unfolding method based on cylindrical projection and combined with a least squares conformal mapping algorithm to generate a two-dimensional UV map;

[0012] Processing the pixel coordinates of the texture image corresponding to the two-dimensional UV map by bilinear interpolation technology to obtain a processing result;

[0013] Texture mapping is performed based on the processing result to update the texture image corresponding to the two-dimensional UV map.

[0014] Optionally, before the step of performing defect detection on the texture image using a preset semantic segmentation model to obtain a defect detection result of the building component to be detected, the step further includes:

[0015] Selecting an initial semantic segmentation model, constructing a sample image dataset, and annotating the sample image dataset to obtain an annotated dataset;

[0016] Performing data augmentation processing on the labeled data set to obtain a training data set;

[0017] Training the initial semantic segmentation model using the training data set to obtain a training result;

[0018] The model parameters of the initial semantic segmentation model are optimized based on the training results to obtain a preset semantic segmentation model.

[0019] Optionally, the step of acquiring color point cloud data of the building component to be inspected by using terrestrial laser scanning technology and generating a three-dimensional mesh model based on the color point cloud data includes:

[0020] Acquire color point cloud data of the building components to be inspected through terrestrial laser scanning technology;

[0021] Performing point cloud segmentation on the color point cloud data to obtain a segmented point cloud subset;

[0022] A three-dimensional mesh model is generated based on the segmented point cloud subset using a Poisson reconstruction method.

[0023] Optionally, the method further includes:

[0024] During the Poisson reconstruction process, the point cloud information and RGB information of the three-dimensional mesh model are recorded;

[0025] During the UV unfolding process, the UV coordinates corresponding to the vertices in the two-dimensional UV map are recorded;

[0026] During the texture mapping process, the pixel coordinates in the texture image corresponding to the UV coordinates are recorded.

[0027] Optionally, the step of performing defect detection on the texture image using a preset semantic segmentation model to obtain a defect detection result of the building component to be detected includes:

[0028] Performing defect detection on the texture image using a preset semantic segmentation model to obtain a label matrix;

[0029] Based on the label matrix, obtaining a set of pixel coordinates corresponding to defects in the texture image;

[0030] A set of pixel coordinates corresponding to defects in the texture image is used as a defect detection result of the building component to be detected.

[0031] Optionally, after the step of performing defect detection on the texture image using a preset semantic segmentation model to obtain a defect detection result of the building component to be detected, the method further includes:

[0032] Determining a UV coordinate set and a point cloud coordinate set corresponding to the defect according to a pixel coordinate set corresponding to the defect in the texture image;

[0033] Based on the UV coordinate set and the point cloud coordinate set corresponding to the defect, a preset quantitative evaluation algorithm is used to quantitatively evaluate the defect of the building component to be inspected, to obtain a defect area ratio and a maximum cross-sectional area loss ratio;

[0034] The damage degree of the building component to be inspected is determined according to the defect area ratio and the maximum cross-sectional area loss ratio.

[0035] In addition, to achieve the above-mentioned purpose, the present invention also provides a device for detecting surface defects of building components, the device comprising:

[0036] A data acquisition module is used to acquire color point cloud data of the building components to be inspected through terrestrial laser scanning technology, and generate a three-dimensional grid model based on the color point cloud data;

[0037] A mesh parameterization module, configured to parameterize the three-dimensional mesh model using a UV unfolding method to generate a two-dimensional UV map, and determine a corresponding texture image based on the two-dimensional UV map;

[0038] The defect detection module is used to perform defect detection on the texture image using a preset semantic segmentation model to obtain a defect detection result of the building component to be detected.

[0039] In addition, to achieve the above-mentioned purpose, the present invention also proposes a building component surface defect detection device, which includes: a memory, a processor, and a building component surface defect detection program stored in the memory and executable on the processor, wherein the building component surface defect detection program is configured to implement the steps of the building component surface defect detection method described above.

[0040] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which a building component surface defect detection program is stored. When the building component surface defect detection program is executed by a processor, the steps of the building component surface defect detection method described above are implemented.

[0041] The present invention discloses a method for acquiring color point cloud data of a building component to be inspected using terrestrial laser scanning technology, generating a three-dimensional mesh model based on the color point cloud data, parameterizing the three-dimensional mesh model using UV expansion to generate a two-dimensional UV map, and determining a corresponding texture image based on the two-dimensional UV map, and performing defect detection on the texture image using a preset semantic segmentation model to obtain defect detection results for the building component to be inspected. Because the present invention generates a three-dimensional mesh model based on the color point cloud data of the building component to be inspected, then reduces the dimensionality of the three-dimensional mesh model to a two-dimensional UV map before performing defect detection, compared to existing technologies, the present invention effectively improves the detection accuracy of building component surface defects and reduces detection costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 Schematic diagram of the process of the first embodiment of the method for detecting surface defects of building components of the present invention;

[0043] Figure 2 This is an example diagram of a scene for point cloud collection in the method for detecting surface defects of building components of the present invention;

[0044] Figure 3 This is an example diagram of mesh parameterization in the building component surface defect detection method of the present invention;

[0045] Figure 4 1. It is an example diagram of defect detection results in the method for detecting surface defects of building components of the present invention;

[0046] Figure 5Schematic diagram of the flow of a second embodiment of a method for detecting surface defects of building components according to the present invention;

[0047] Figure 6 Schematic diagram of the flow of a third embodiment of a method for detecting surface defects of building components according to the present invention;

[0048] Figure 7 This is an example diagram of the cross-sectional area calculation results in the building component surface defect detection method of the present invention;

[0049] Figure 8 Schematic diagram of the overall workflow of the method for detecting surface defects of building components of the present invention;

[0050] Figure 9 This is a structural block diagram of a first embodiment of a device for detecting surface defects of building components according to the present invention;

[0051] Figure 10 It is a structural diagram of a building component surface defect detection device in a hardware operating environment involved in an embodiment of the present invention.

[0052] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0053] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0054] The embodiment of the present invention provides a method for detecting surface defects of building components, referring to Figure 1 , Figure 1 2 is a flow chart of a first embodiment of a method for detecting surface defects of building components according to the present invention.

[0055] In this embodiment, the method for detecting surface defects of building components includes steps S10 to S30:

[0056] Step S10: obtaining color point cloud data of the building component to be inspected by terrestrial laser scanning technology, and generating a three-dimensional grid model based on the color point cloud data.

[0057] It should be noted that the execution subject of this embodiment can be a computer server device with data processing, network communication, and program execution functions used in building component surface defect detection scenarios, such as a server, personal computer, tablet computer, or smartphone, or an electronic device capable of performing the above functions, building component surface defect detection equipment, etc. The following uses building component surface defect detection equipment as an example to illustrate this embodiment and the following embodiments.

[0058] For example, reference Figure 2 , Figure 2This diagram illustrates a point cloud acquisition scenario for the building component surface defect detection method of the present invention. The diagram illustrates how a Leica BLK360 scanner is used to acquire color point cloud data of the building component being inspected. The Leica BLK360 scanner represents the Leica BLK360 scanner, and the upper left corner of the diagram shows the scanner itself. The scanner is a handheld device designed to rapidly capture three-dimensional data of the surrounding environment. Target structural column pointcloud represents the target structural column point cloud (that is, the colored point cloud data of the building component to be detected. The image on the left shows the colored point cloud data of a specific structural column captured by the scanner. A point cloud is a collection of countless points, each of which represents a position in space and usually contains three-dimensional coordinates (X, Y, Z). Overall scene point cloud represents the overall scene point cloud. The image on the right shows the point cloud data of the entire scanned area. These point cloud data are used to create a detailed three-dimensional model that can be used for analyzing building structures, planning and design. In the overall scene point cloud, some green and blue markers can be seen. These markers may represent key points or areas of interest identified during the scanning process, such as structural connection points, damaged areas, or areas that require special attention. The picture also shows some details of the scanning environment, such as exposed concrete structures, scaffolding, scattered building materials, etc. These are common scenes on construction sites or inside buildings undergoing renovation.

[0059] In a specific implementation, the colored point cloud data of the building components to be inspected can be obtained through terrestrial laser scanning technology; the colored point cloud data can be segmented to obtain a segmented point cloud subset; and a three-dimensional mesh model can be generated based on the segmented point cloud subset using the Poisson reconstruction method.

[0060] It should be noted that terrestrial laser scanning technology is a technology that uses laser scanners to obtain target point cloud information from multiple fixed sites. It has high measurement accuracy and is suitable for scenarios with strict precision requirements, such as building quality inspections.

[0061] It should be explained that in this embodiment, the Poisson reconstruction method is used to generate a smooth mesh model from a point cloud. The Poisson reconstruction method first estimates the normal vector of each point in the point cloud, constructs a gradient field, and then uses the Poisson equation to convert the divergence field into an implicit function. Finally, the isosurface is extracted from the implicit function to form a smooth polygonal mesh. The Poisson reconstruction method can effectively smooth noise and fill holes in the point cloud, making the generated three-dimensional surface more complete and accurate. The mesh model generated by point cloud reconstruction can be used for subsequent processing and analysis, which contains the three-dimensional coordinates, RGB information and normal vectors of the vertices, as well as the vertex index of each facet.

[0062] Step S20: parameterizing the three-dimensional mesh model by using a UV unfolding method to generate a two-dimensional UV map, and determining a corresponding texture image based on the two-dimensional UV map.

[0063] It should be understood that UV unfolding is an important step in computer graphics, especially in the texture mapping process, which involves unfolding the surface of a three-dimensional model into a two-dimensional plane so that the texture image can be correctly mapped onto the three-dimensional model.

[0064] In a specific implementation, this embodiment adopts a UV unfolding method based on cylindrical projection, and combines the least squares conformal mapping algorithm to parameterize the three-dimensional mesh model to generate a two-dimensional UV map; through bilinear interpolation technology, the pixel coordinates of the texture image corresponding to the two-dimensional UV map are processed to obtain a processing result; texture mapping is performed based on the processing result to update the texture image corresponding to the two-dimensional UV map.

[0065] UV unwrapping is the key process of mapping the surface of a 3D mesh model to a 2D plane and obtaining UV coordinates. It involves cutting the mesh along certain edges to create a parameterizable map. The challenge is to minimize distortion in terms of angle (conformal mapping) and area (equal-area mapping). A mapping function f can be defined to convert 3D coordinates into UV coordinates, as shown in formula (1).

[0066] (u,v)=f(x,y,z) (1)

[0067] Where (x, y, z) is a point on the surface of the three-dimensional mesh model, and (u, v) is the corresponding point in the two-dimensional UV map.

[0068] It should be explained that in the UV unfolding process of this embodiment, the least squares conformal mapping algorithm (LSCM) is used. This algorithm preserves the local angles of the surface by minimizing angular distortion and reduces deformation as much as possible. This technology uses the least squares method to achieve conformal mapping of the surface. The core of LSCM is to construct an energy function that uses linear algebra and calculus to measure conformal distortion. For LSCM, this energy function is usually based on the angular distortion of each triangle and is described by the following formula:

[0069] E t =∑(||t ij || 2 || ik ‖ 2 +||t ik || 2 ||e ij || 2 -2(t ij ·t ik )(e ij ·e ik )) (2)

[0070]

[0071] Where, E t is the conformal energy of triangle t; e ij and e ik is the edge vector of the triangle in three-dimensional space; t ij and t ik It is the edge vector of the triangle in UV space. (x, y, z) represents the coordinates of a point on the 3D mesh surface; (u, v) is the coordinates of a point on the 2D plane corresponding to the point on the 3D mesh surface. i, j, k are usually used to represent the vertex index of a triangle. When processing a 3D mesh, the mesh is composed of many triangular facets. The vertices of these triangles are identified and referenced by indices. For example, a point in a triangle is (x i ,y i ,z i ), the corresponding point on the two-dimensional plane is (u i ,v i ).

[0072] By minimizing the conformal energy of all triangles in the 3D mesh model, a discrete conformal mapping can be obtained, as shown in formula (5):

[0073]

[0074] Where T is the set of triangles in the mesh, A tis the area of triangle t. This equation yields a linear system that can be solved directly. The solution gives the coordinates (u, v) of each vertex in the UV map.

[0075] It's important to understand that a cylindrical projection-based UV unwrapping method typically produces a rectangular shape after unwrapping, which facilitates texture mapping and is particularly suitable for cylindrical objects. It provides continuity in all directions, particularly along the cylinder's axis, allowing textures to extend seamlessly. Given that most architectural components are essentially rectangular or cylindrical, a cylindrical projection-based UV unwrapping method is highly adaptable.

[0076] After the UV expansion is completed, the coordinates (u, v) of each vertex in the obtained UV map will be mapped to the pixel coordinates (i, j) on the texture image, as shown in the formula.

[0077]

[0078] Where W and H are the width and height of the texture image, respectively.

[0079] Since the UV coordinates are not completely aligned with the pixel coordinates on the texture image, bilinear interpolation technology is used to process the pixel coordinates. Bilinear interpolation calculates the sampling value by taking a weighted average of the four surrounding pixels, as shown in formula (7).

[0080]

[0081] Where i u and i v are the integer parts of the horizontal and vertical image coordinates, respectively. α and β are the fractional parts of the horizontal and vertical image coordinates, respectively. u ,i v )、C(i u +1,i v )、C(i u ,i v +1) and C(i u +1,i v +1) represent the upper left corner, upper right corner, lower left corner, and lower right corner of the four surrounding pixels respectively.)

[0082] The texture image obtained through UV unwrapping is accurate and continuous, closely linked to the 3D mesh model, and editable, accurately reflecting the distribution of the 3D mesh model surface on the 2D plane. Each vertex position on the 3D mesh model has a corresponding pixel coordinate in the texture image, ensuring a precise mapping of the texture image to the 3D mesh model surface. This, in turn, prevents noticeable distortion or misalignment during the texture mapping process, achieving a natural transition.

[0083] For example, reference Figure 3 , Figure 3 This figure illustrates an example of mesh parameterization in the building component surface defect detection method of the present invention. The entire process in the figure demonstrates how, starting from actual point cloud data, a series of processing steps are performed to ultimately achieve high-quality texture mapping. The target structural column point cloud represents the target structural column point cloud. The upper left corner shows the color point cloud data of a specific structural column acquired using a Leica BLK360 scanner. Poisson reconstruction represents Poisson reconstruction. The point cloud data is processed using the Poisson reconstruction algorithm to generate a 3D mesh model of the structural column. Poisson reconstruction is a technique for generating high-quality mesh models based on point cloud data. Mesh model represents a 3D mesh model. The reconstructed 3D mesh model displays the 3D shape of the structural column and provides the basis for subsequent texture mapping. UV unwrapping represents UV unwrapping. The 3D mesh model is subjected to UV unwrapping to map its surface onto a 2D plane, generating a UV map. UV unwrapping is a key step in texture mapping, as it determines the distribution of texture on the 3D model surface. Texture image represents a texture image. The lower left corner of the figure shows the texture image used for texture mapping. This is an actual image taken of the structural column, containing the texture details of the column surface. Texture mapping refers to texture mapping, which uses bilinear interpolation technology to map the texture image to the UV map after UV expansion. Bilinear interpolation is a method for smooth transition between pixels, which is used to improve the quality of texture mapping. Texturemap refers to the texture map. The lower right corner of the figure shows the texture map after texture mapping is completed, which shows how the texture image is applied to the surface of the 3D model.

[0084] Step S30: performing defect detection on the texture image using a preset semantic segmentation model to obtain a defect detection result of the building component to be detected.

[0085] It should be noted that this embodiment uses a preset semantic segmentation model to detect defects in the resulting texture image. Semantic segmentation classifies each pixel in the image and assigns it a specific semantic label to achieve a refined understanding of the image. It can grasp the image's structure and scene as a whole while meticulously processing the features of each local area, thereby achieving precise pixel-level classification based on a global understanding.

[0086] In a specific implementation, a preset semantic segmentation model can be used to perform defect detection on the texture image to obtain a label matrix; based on the label matrix, a set of pixel coordinates corresponding to defects in the texture image is obtained; and the set of pixel coordinates corresponding to defects in the texture image is used as the defect detection result of the building component to be detected.

[0087] The label matrix is shown in formula (8):

[0088] L={L(i,j)∣i∈[0,W-1],j∈[0,H-1]} (8)

[0089] Where L is the label matrix obtained by semantic segmentation. It has the same width W and height H as the texture image. Each element L(i, j) represents the class label of the pixel at position (i, j).

[0090] The pixel coordinate set corresponding to the defects in the texture image is shown in formula (9):

[0091] D c ={(i,j)∣L(i,j)=c,i∈[0,W-1],j∈[0,H-1]} (9)

[0092] Where c is a specific class label. c is the set of pixel coordinates corresponding to the defects in the texture image. Through the above process, the set of pixel coordinates corresponding to the defects D c It can be identified quickly and accurately, thus completing the positioning of defects on the 2D plane.

[0093] For example, reference Figure 4 , Figure 4This is an example diagram of the defect detection results in the surface defect detection method of building components of the present invention. The figure shows the defect detection and marking process of a building structure column. Semantic segmentation result represents the semantic segmentation result, where different colors or areas represent different parts identified by semantic segmentation technology. Semantic segmentation is an image processing technology used to identify and distinguish different objects or materials in an image. In this figure, you can see that the structural column is divided into different areas, each area is represented by a different color or mark (such as Mask 1, Mask 2, etc.). Def ects in pixel coordinates represent defects in pixel coordinates. The green area in the figure may represent the identified defect location. The pixel coordinate system is a commonly used coordinate system in image processing and is used to accurately locate points in the image. Defects in UV coordinates represent defects in UV coordinates. The UV coordinate system is usually used for texture mapping, where U and V represent the width and height directions of the texture image, respectively. The black squares in the figure may represent the defect locations identified in the UV mapping. Defects in point cloud coordinates represent defects marked in the point cloud coordinate system. The point cloud coordinate system is a coordinate system in three-dimensional space that is used to represent the position of each point in the point cloud data. The defects in the figure (Defect 1 to Defect 6) are marked on the 3D model of the structural column. The position of each defect is determined by its coordinates on the X, Y, and Z axes.

[0094] It should be noted that, in a specific implementation, the method for detecting surface defects of building components further includes: recording the point cloud information and RGB information of the three-dimensional mesh model during the Poisson reconstruction process; recording the UV coordinates corresponding to the vertices in the two-dimensional UV map during the UV expansion process; and recording the pixel coordinates in the texture image corresponding to the UV coordinates during the texture mapping process. Through the above steps, the three-dimensional point cloud coordinates are linked to the two-dimensional image coordinates, ensuring that the mapping relationship is bidirectional. Given the coordinates (i, j) of any pixel on the image, the corresponding vertex UV coordinate set Q and point cloud coordinate set P within the pixel range can be found. This is shown in Formulas (11) to (14).

[0095]

[0096] Where, T vertex It is a mapping function that converts 3D point cloud coordinates (x, y, z) to corresponding UV coordinates (u, v). UV It is a mapping function that maps UV coordinates (u, v) to the corresponding pixel coordinates (i, j) in the 2D image.

[0097] This embodiment discloses a method for acquiring color point cloud data of a building component to be inspected using terrestrial laser scanning technology, generating a three-dimensional mesh model based on the color point cloud data, parameterizing the three-dimensional mesh model using UV expansion to generate a two-dimensional UV map, and determining a corresponding texture image based on the two-dimensional UV map, and performing defect detection on the texture image using a preset semantic segmentation model to obtain defect detection results for the building component to be inspected. Because this embodiment generates a three-dimensional mesh model based on the color point cloud data of the building component to be inspected, then reduces the dimensionality of the three-dimensional mesh model to a two-dimensional UV map before performing defect detection, compared to existing technologies, this embodiment effectively improves the detection accuracy of building component surface defects and reduces detection costs.

[0098] refer to Figure 5 , Figure 5 2 is a flow chart of a second embodiment of a method for detecting surface defects of building components according to the present invention.

[0099] Based on the first embodiment described above, in this embodiment, before step S30, steps S211 to S214 are further included:

[0100] Step S211: Select an initial semantic segmentation model, construct a sample image dataset, and annotate the sample image dataset to obtain an annotated dataset.

[0101] Step S212: performing data enhancement processing on the labeled data set to obtain a training data set.

[0102] Step S213: training the initial semantic segmentation model using the training data set to obtain a training result.

[0103] Step S214: Optimizing the model parameters of the initial semantic segmentation model based on the training result to obtain a preset semantic segmentation model.

[0104] It should be noted that the initial semantic segmentation model selected in this embodiment is YOLOV11, which is the most advanced object detection and image segmentation model at present, with extremely high inference speed and accuracy.

[0105] In a specific implementation, a small sample image dataset can be constructed for experimental research. Each image in the dataset is annotated using pre-set annotation software, and data augmentation is used to improve the generalization of the model. During training, this embodiment can utilize pre-trained weights for transfer learning. When the mean Average Precision (mAP) on the validation set reaches a preset threshold, such as 0.95, the preset semantic segmentation model is obtained, thereby ensuring the accuracy of detecting defects of various sizes and shapes.

[0106] This embodiment discloses selecting an initial semantic segmentation model, constructing a sample image dataset, annotating the sample image dataset to obtain an annotated dataset, performing data augmentation processing on the annotated dataset to obtain a training dataset, training the initial semantic segmentation model using the training dataset to obtain training results, and optimizing the model parameters of the initial semantic segmentation model based on the training results to obtain a preset semantic segmentation model. Compared to the prior art, this embodiment optimizes the model parameters of the initial semantic segmentation model based on the training results through data augmentation processing to obtain a preset semantic segmentation model, thereby not only improving the generalization of the preset semantic segmentation model but also further ensuring the accuracy of defect detection.

[0107] refer to Figure 6 , Figure 6 2 is a flow chart of a third embodiment of a method for detecting surface defects of building components according to the present invention.

[0108] Based on the above embodiments, in this embodiment, after step S30, steps S40 to S60 are further included:

[0109] Step S40: determining a UV coordinate set and a point cloud coordinate set corresponding to the defect according to a pixel coordinate set corresponding to the defect in the texture image.

[0110] In a specific implementation, according to the pixel coordinate set corresponding to the defect in the texture image, the UV coordinate set Q corresponding to the defect can be obtained by formula (15) and formula (16): c And the point cloud coordinate set P c , formula (15) and formula (16) are as follows:

[0111]

[0112] Step S50: Based on the UV coordinate set and the point cloud coordinate set corresponding to the defect, a preset quantitative evaluation algorithm is used to perform a quantitative evaluation on the defect of the building component to be inspected, and obtain a defect area ratio and a maximum cross-sectional area loss ratio.

[0113] It should be noted that the preset quantitative evaluation algorithm in this embodiment may be the Alpha Shapes algorithm.

[0114] In the specific implementation, according to Q c For the i-th defect, use the Alpha Shapes algorithm to obtain its boundary points, connect these boundary points to form a closed polygon, and calculate the area A of the closed polygon i According to P cFor the i-th defect, calculate its range in the axial direction and slice the original point cloud within this range. For each slice, calculate the area S of the closed polygon formed by its boundary points using the AlphaShapes algorithm. i .

[0115] It's important to note that the Alpha Shapes algorithm is a key algorithm in computational geometry for describing the shape of a point set. It controls shape generation by defining a parameter α, constructing a disk or sphere centered on a scattered point. The boundary of their intersection forms the AlphaShapes boundary. For parameter α, a greedy strategy based on geometric property derivation is used. By progressively examining the set of triangles, the algorithm determines the minimum set that covers all vertices and automatically calculates the optimal α value based on this.

[0116] For example, when the building component to be inspected is a structural column, a quantitative evaluation can be performed using formulas (17) and (18), which are as follows:

[0117]

[0118] Where R A and R S are the defect area ratio and the maximum cross-sectional area loss ratio respectively. UV is the total area after UV unfolding. S0 is the design area of the cross section.

[0119] Step S60: determining the damage degree of the building component to be inspected according to the defect area ratio and the maximum cross-sectional area loss ratio.

[0120] It should be noted that the provisions on the degree of damage to reinforced concrete components define R A The evaluation criteria is R A ≤10%, indicating that the component is within the general damage range and has not yet reached the level of serious damage; the value of the bearing capacity seismic adjustment coefficient is mostly 0.85, as shown in formula (19):

[0121] N≥0.85N0 (19)

[0122] Where N0 is the design value of the ultimate bearing capacity of the structure, N0 = fcS0. fc is the design value of the compressive strength of the material; N is the actual ultimate bearing capacity of the structure, N0 = fcS0 (1-R S ). Through calculation, it can be concluded that Rs≤15%. Therefore, it can be stipulated that when R A ≤10% and R S ≤15%, the structure can meet the minimum safety requirements and can be further repaired.

[0123] For example, reference Figure 7 , Figure 7 This is an example diagram of the cross-sectional area calculation results in the method for detecting surface defects of building components of the present invention. The coordinate system in the figure is used to indicate the specific location of the defect in the point cloud data. The horizontal axis represents the cross-sectional area (unit: square centimeters), and the vertical axis represents the height (unit: centimeters). The red and blue areas in the chart represent different states of the structural column: the red area (Existing cross-sectional area) represents the existing cross-sectional area of the structural column. The blue area (Lossarea) represents the loss of cross-sectional area due to defects. Defect area: The width of the blue area in the chart represents the loss of cross-sectional area of the defect area. By comparing the red and blue areas, the impact of defects on the strength of the structural column can be quantified. Maximum loss cross-section: A specific point (2933.217, 138.5) is marked in the chart, which may indicate the location with the largest cross-sectional area loss at a height of 138.5 cm. The image on the right shows the point cloud data of the structural column, where the blue dots represent the defective areas. The height of the defective area (138.5 cm) and the location of the maximum loss section are marked in the image. The point cloud data provides three-dimensional information on the surface of the structural column, which helps to more accurately analyze the location and shape of the defect.

[0124] Figure 8This is a schematic diagram of the overall workflow of the surface defect detection method for building components of the present invention. The figure includes five parts: 1. Point Cloud Reconstruction: Terrestrial laser scanning technology is used to obtain point cloud data of buildings or objects; Point cloud segmentation segments the scanned point cloud data for subsequent processing; and the point cloud is converted into a mesh through the Poisson reconstruction algorithm. 2. Mapping Records: Records mesh information, including face index, vertex index, point cloud coordinates (x, y, z), UV coordinates (u, v), RGB color information (R, G, B), and pixel coordinates (i, j). 3. Mesh Parameterization: UV unwrapping maps the 3D mesh to a 2D plane to facilitate texture mapping; texture mapping is the process of mapping a texture image to the surface of a 3D model. 4. Defect Detection: Semantic segmentation is used to identify and classify different areas in the image; the defect detection results are displayed in the form of a colored image, in which the defect areas are marked. 5. Defect Impact Assessment: Defect localization and measurement determine the location and size of the defect; evaluate the defect area, maximum loss cross-section, and location of the defect; and finally output the defect impact assessment results, which may include the severity of the defect (R A and R S ) and other information.

[0125] This embodiment discloses determining a UV coordinate set and a point cloud coordinate set corresponding to the defect in the texture image based on a set of pixel coordinates corresponding to the defect; performing a quantitative assessment of the defect of the building component to be inspected using a preset quantitative assessment algorithm based on the UV coordinate set and the point cloud coordinate set corresponding to the defect, obtaining a defect area ratio and a maximum cross-sectional area loss ratio; and determining the degree of damage to the building component to be inspected based on the defect area ratio and the maximum cross-sectional area loss ratio. Because this embodiment performs a quantitative assessment of the defect of the building component to be inspected using a preset quantitative assessment algorithm based on the UV coordinate set and the point cloud coordinate set corresponding to the defect, and determines the degree of damage to the building component to be inspected based on the defect area ratio and the maximum cross-sectional area loss ratio, compared to the prior art, this embodiment further improves the accuracy of defect detection by performing a quantitative assessment of the defect of the building component to be inspected.

[0126] In addition, an embodiment of the present invention further proposes a storage medium storing a building component surface defect detection program. When the building component surface defect detection program is executed by a processor, the steps of the building component surface defect detection method described above are implemented.

[0127] Reference Figure 9 , Figure 9 This is a structural block diagram of the first embodiment of the building component surface defect detection device of the present invention.

[0128] like Figure 9 As shown, the building component surface defect detection device proposed in the embodiment of the present invention includes: a data acquisition module 901, a grid parameterization module 902 and a defect detection module 903.

[0129] The data acquisition module 901 is used to acquire color point cloud data of the building component to be inspected through terrestrial laser scanning technology, and generate a three-dimensional grid model based on the color point cloud data.

[0130] The mesh parameterization module 902 is configured to parameterize the three-dimensional mesh model using a UV unfolding method to generate a two-dimensional UV map, and determine a corresponding texture image based on the two-dimensional UV map.

[0131] The defect detection module 903 is used to perform defect detection on the texture image using a preset semantic segmentation model to obtain a defect detection result of the building component to be detected.

[0132] The data acquisition module 901 is further used to acquire color point cloud data of the building components to be inspected through terrestrial laser scanning technology; perform point cloud segmentation on the color point cloud data to obtain segmented point cloud subsets; and use the Poisson reconstruction method to generate a three-dimensional mesh model based on the segmented point cloud subsets.

[0133] The mesh parameterization module 902 is also used to parameterize the three-dimensional mesh model using a UV expansion method based on cylindrical projection and combined with a least squares conformal mapping algorithm to generate a two-dimensional UV map; process the pixel coordinates of the texture image corresponding to the two-dimensional UV map through bilinear interpolation technology to obtain a processing result; perform texture mapping based on the processing result to update the texture image corresponding to the two-dimensional UV map.

[0134] The defect detection module 903 is also used to record the point cloud information and RGB information of the three-dimensional mesh model during the Poisson reconstruction process; record the UV coordinates corresponding to the vertices in the two-dimensional UV map during the UV expansion process; and record the pixel coordinates in the texture image corresponding to the UV coordinates during the texture mapping process.

[0135] The defect detection module 903 is further configured to perform defect detection on the texture image using a preset semantic segmentation model to obtain a label matrix; based on the label matrix, obtain a set of pixel coordinates corresponding to defects in the texture image; and use the set of pixel coordinates corresponding to defects in the texture image as a defect detection result for the building component to be detected.

[0136] This device embodiment discloses obtaining color point cloud data of a building component to be inspected using terrestrial laser scanning technology, generating a three-dimensional mesh model based on the color point cloud data, parameterizing the three-dimensional mesh model using UV expansion to generate a two-dimensional UV map, and determining a corresponding texture image based on the two-dimensional UV map, and performing defect detection on the texture image using a preset semantic segmentation model to obtain a defect detection result for the building component to be inspected. Because this device embodiment generates a three-dimensional mesh model based on the color point cloud data of the building component to be inspected, then reduces the dimensionality of the three-dimensional mesh model to a two-dimensional UV map before performing defect detection, compared to existing technologies, this device embodiment effectively improves the detection accuracy of building component surface defects and reduces detection costs.

[0137] Based on the first embodiment of the device for detecting surface defects of building components of the present invention, a second embodiment of the device for detecting surface defects of building components of the present invention is proposed.

[0138] In this embodiment, the defect detection module 903 is also used to select an initial semantic segmentation model, construct a sample image data set, annotate the sample image data set to obtain an annotated data set; perform data enhancement processing on the annotated data set to obtain a training data set; train the initial semantic segmentation model through the training data set to obtain a training result; optimize the model parameters of the initial semantic segmentation model based on the training result to obtain a preset semantic segmentation model.

[0139] Other embodiments or specific implementations of the building component surface defect detection device of the present invention can refer to the above-mentioned method embodiments and will not be repeated here.

[0140] The present application provides a building component surface defect detection device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the building component surface defect detection method in the above-mentioned embodiment 1.

[0141] Reference below Figure 10 , which shows a schematic structural diagram of a building component surface defect detection device suitable for implementing an embodiment of the present application. The building component surface defect detection device in the embodiment of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 10 The building component surface defect detection device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0142] like Figure 10As shown, the building component surface defect detection device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the building component surface defect detection device. Processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, hard disk, etc.; and communication devices 1009. Communication devices 1009 can allow the building component surface defect detection device to communicate with other devices wirelessly or by wire to exchange data. While the figures illustrate a building component surface defect detection device with various systems, it should be understood that implementation or presence of all the illustrated systems is not required. More or fewer systems may alternatively be implemented or present.

[0143] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0144] The building component surface defect detection device provided in this application utilizes the building component surface defect detection method described in the aforementioned embodiment, resolving the technical issues in the prior art where extracting and utilizing information from color point clouds is extremely difficult, resulting in low detection accuracy and high detection costs for building component surface defects. Compared to the prior art, the beneficial effects of the building component surface defect detection device provided in this application are the same as those of the building component surface defect detection method described in the aforementioned embodiment. Other technical features of this building component surface defect detection device are the same as those disclosed in the aforementioned embodiment and are not further elaborated here.

[0145] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0146] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0147] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0148] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0149] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0150] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for detecting surface defects of building components, characterized in that: The method comprises: Acquire color point cloud data of the building components to be inspected by terrestrial laser scanning technology, and generate a three-dimensional mesh model based on the color point cloud data; Parameterizing the three-dimensional mesh model using a UV unfolding method to generate a two-dimensional UV map, and determining a corresponding texture image based on the two-dimensional UV map; Defect detection is performed on the texture image using a preset semantic segmentation model to obtain a defect detection result of the building component to be detected.

2. The method for detecting surface defects of building components according to claim 1, wherein: The step of parameterizing the three-dimensional mesh model by UV unfolding to generate a two-dimensional UV map, and determining a corresponding texture image based on the two-dimensional UV map includes: The three-dimensional mesh model is parameterized by using a UV unfolding method based on cylindrical projection and combined with a least squares conformal mapping algorithm to generate a two-dimensional UV map; Processing the pixel coordinates of the texture image corresponding to the two-dimensional UV map by bilinear interpolation technology to obtain a processing result; Texture mapping is performed based on the processing result to update the texture image corresponding to the two-dimensional UV map.

3. The method for detecting surface defects of building components according to claim 1, wherein: Before the step of performing defect detection on the texture image using a preset semantic segmentation model to obtain a defect detection result of the building component to be detected, the method further includes: Selecting an initial semantic segmentation model, constructing a sample image dataset, and annotating the sample image dataset to obtain an annotated dataset; Performing data augmentation processing on the labeled data set to obtain a training data set; Training the initial semantic segmentation model using the training data set to obtain a training result; The model parameters of the initial semantic segmentation model are optimized based on the training results to obtain a preset semantic segmentation model.

4. The method for detecting surface defects of building components according to claim 1, wherein: The step of acquiring color point cloud data of the building component to be inspected by using terrestrial laser scanning technology and generating a three-dimensional grid model based on the color point cloud data includes: Acquire color point cloud data of the building components to be inspected through terrestrial laser scanning technology; Performing point cloud segmentation on the color point cloud data to obtain a segmented point cloud subset; A three-dimensional mesh model is generated based on the segmented point cloud subset using a Poisson reconstruction method.

5. The method for detecting surface defects of building components according to claim 1, wherein: The method further comprises: During the Poisson reconstruction process, the point cloud information and RGB information of the three-dimensional mesh model are recorded; During the UV unfolding process, the UV coordinates corresponding to the vertices in the two-dimensional UV map are recorded; During the texture mapping process, the pixel coordinates in the texture image corresponding to the UV coordinates are recorded.

6. The method for detecting surface defects of building components according to claim 1, wherein: The step of performing defect detection on the texture image using a preset semantic segmentation model to obtain a defect detection result of the building component to be detected includes: Performing defect detection on the texture image using a preset semantic segmentation model to obtain a label matrix; Based on the label matrix, obtaining a set of pixel coordinates corresponding to defects in the texture image; A set of pixel coordinates corresponding to defects in the texture image is used as a defect detection result of the building component to be detected.

7. The method for detecting surface defects of building components according to claim 6, wherein: After the step of performing defect detection on the texture image using a preset semantic segmentation model to obtain a defect detection result of the building component to be detected, the method further includes: Determining a UV coordinate set and a point cloud coordinate set corresponding to the defect according to a pixel coordinate set corresponding to the defect in the texture image; Based on the UV coordinate set and the point cloud coordinate set corresponding to the defect, a preset quantitative evaluation algorithm is used to quantitatively evaluate the defect of the building component to be inspected, to obtain a defect area ratio and a maximum cross-sectional area loss ratio; The damage degree of the building component to be inspected is determined according to the defect area ratio and the maximum cross-sectional area loss ratio.

8. A device for detecting surface defects of building components, characterized in that: The device comprises: A data acquisition module is used to acquire color point cloud data of the building components to be inspected through terrestrial laser scanning technology, and generate a three-dimensional grid model based on the color point cloud data; A mesh parameterization module, configured to parameterize the three-dimensional mesh model using a UV unfolding method to generate a two-dimensional UV map, and determine a corresponding texture image based on the two-dimensional UV map; The defect detection module is used to perform defect detection on the texture image using a preset semantic segmentation model to obtain a defect detection result of the building component to be detected.

9. A building component surface defect detection device, characterized in that: The device includes: a memory, a processor, and a building component surface defect detection program stored in the memory and executable on the processor, wherein the building component surface defect detection program is configured to implement the steps of the building component surface defect detection method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores a building component surface defect detection program, which, when executed by a processor, implements the steps of the building component surface defect detection method according to any one of claims 1 to 7.

Images