A multi-view based three-dimensional crack detection method, device, apparatus and medium

Abstract

The disclosure provides a multi-view-based three-dimensional crack detection method, device, equipment and medium, wherein the method comprises three-dimensional crack semantic point cloud generation, point cloud processing, crack skeleton point and boundary point determination and crack size determination steps; the multi-view-based three-dimensional crack detection method, device, equipment and medium of the disclosure have the advantages that the deep learning technology is used for accurate pixel-level crack semantic image segmentation, the crack pixel value in the crack semantic image is converted into the color of a specified RGB attribute value, so that there is an obvious color difference between the crack pixel and the background pixel, and a comprehensive three-dimensional crack semantic point cloud is generated and crack skeleton points and boundary points are determined based on the multi-view stereo three-dimensional point cloud reconstruction technology and the point cloud processing technology, complete recording and accurate detection of cross-plane complex cracks are realized.

Description

Technical Field

[0001] This invention relates to the field of image detection technology, and in particular to a method, apparatus, device, and medium for three-dimensional crack detection based on multiple views. Background Technology

[0002] Concrete structures are inevitably subject to external loads and internal physical and chemical changes, leading to the development of cracks during their operational lifespan, which continue to evolve over extended periods of service. If these cracks are not detected and effectively repaired in a timely manner, they will accelerate the degradation of structural performance and may even result in serious engineering accidents. To reduce engineering accidents caused by concrete structural defects and ensure the safety of people's lives and property, it is necessary to regularly inspect the location, length, and width of cracks on the concrete surface to assess the structural safety status.

[0003] In existing technologies, two-dimensional image detection methods based on computer vision are used to improve the efficiency of crack detection and reduce detection costs and risks. These automatic detection methods utilize cameras to acquire images of the concrete surface, and then use image processing techniques to detect crack locations and quantify crack sizes. However, two-dimensional image detection methods are limited by the camera's limited field of view and strict shooting angles, making it impossible to comprehensively record complete cracks on the structural surface, leading to inaccurate crack length measurements. To address this, crack detection methods based on three-dimensional point cloud processing are developed. These methods utilize the intensity and true color information of three-dimensional point clouds. However, not all point cloud data possesses intensity information, and true color information is easily affected by stains, shadows, and handwritten patterns on the structure that are similar in color to cracks, introducing a large number of false candidate points that are not cracks. This results in low crack point cloud detection accuracy and can easily lead to hidden dangers.

[0004] In view of this, there is an urgent need to provide a multi-view-based three-dimensional crack detection method that improves automation and detection accuracy. Summary of the Invention

[0005] To overcome the problems existing in related technologies, this disclosure provides a three-dimensional crack detection method, apparatus, equipment and medium based on multiple views, so as to solve the technical problem that the crack point cloud detection accuracy is low and can easily cause hidden dangers in related technologies.

[0006] This specification provides one or more embodiments of a three-dimensional crack detection method based on multiple views, including the following steps:

[0007] 3D crack semantic point cloud generation: Crack images are acquired and preprocessed to obtain a depth map; crack semantic maps are obtained by recognizing crack images through a semantic segmentation network, and crack pixel values ​​in the crack semantic maps are converted into colors with specified RGB attribute values; the depth map and the processed crack semantic map are fused through a depth map fusion algorithm to obtain a 3D crack semantic point cloud with specified colors.

[0008] Point cloud processing: Based on the RGB color range, point cloud data of 3D crack semantic point cloud is extracted by point cloud extraction method, and the points are divided into several independent point cloud clusters by clustering algorithm. Bounding box fitting is performed on each point cloud cluster to obtain crack point cloud instances containing local background points. Multiple planes where cross-plane cracks are located in the crack point cloud instances are detected, and point clouds near each plane are extracted and fitted to obtain the corresponding crack point cloud plane equation.

[0009] Determination of crack skeleton points and boundary points: 2D crack point clouds are obtained by flattening the crack point cloud instances corresponding to each crack plane equation; the mesh model of the 2D crack point cloud is reconstructed using a mesh model reconstruction algorithm; the skeleton point set of each 2D crack point cloud is extracted using a skeleton extraction method; and the boundary point set of each 2D crack point cloud corresponding to the skeleton point set is extracted using a boundary point extraction algorithm.

[0010] Crack size determination: Based on the obtained set of skeleton points and corresponding boundary point sets, the length and width of each crack are quantitatively calculated.

[0011] This specification provides one or more embodiments of a three-dimensional crack detection device based on multiple views, including:

[0012] 3D crack semantic point cloud generation module: Configured to acquire crack images, preprocess the crack images to obtain a depth map; identify crack images through a semantic segmentation network to obtain a crack semantic map, and convert the crack pixel values ​​in the crack semantic map into colors with specified RGB attribute values; fuse the depth map with the processed crack semantic map through a depth map fusion algorithm to obtain a 3D crack semantic point cloud of specified color;

[0013] Point cloud processing module: Configured to extract point cloud data of 3D crack semantic point cloud based on RGB color range and point cloud extraction method, and divide the points into several independent point cloud clusters through clustering algorithm, and perform bounding box fitting on each point cloud cluster to obtain crack point cloud instance containing local background points. Detect multiple planes in the crack point cloud instance where cross-plane cracks are located, extract point clouds near each plane and fit the corresponding crack point cloud plane equation.

[0014] Crack skeleton point and boundary point determination module: Configured to obtain 2D crack point clouds by flattening the corresponding crack point cloud instances according to the plane equation of each crack; reconstruct the mesh model of the 2D crack point cloud using a mesh model reconstruction algorithm; extract the skeleton point set of each 2D crack point cloud using a skeleton extraction method; and extract the boundary point set of the corresponding skeleton point set of each 2D crack point cloud using a boundary point extraction algorithm.

[0015] Crack size determination module: This module is configured to quantitatively calculate the length and width of each crack based on the acquired set of skeleton points and the corresponding set of boundary points.

[0016] This specification provides one or more embodiments of a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-view-based three-dimensional crack detection method as described above.

[0017] This specification provides one or more embodiments of a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the multi-view-based three-dimensional crack detection method described above.

[0018] This disclosure provides a multi-view-based three-dimensional crack detection method, apparatus, device, and medium. Its advantages lie in using deep learning technology for accurate pixel-level crack semantic image segmentation, converting crack pixel values ​​in the crack semantic image to colors with specified RGB attribute values ​​to create a clear color difference between crack pixels and background pixels. Based on multi-view stereo point cloud reconstruction and point cloud processing techniques, a comprehensive three-dimensional crack semantic point cloud is generated, and crack skeleton points and boundary points are determined. This enables a complete record of complex cross-plane cracks, facilitating subsequent high-precision three-dimensional crack size measurement and achieving rapid structural damage detection and assessment, providing strong support and assurance for concrete structure inspection and maintenance. This method overcomes the limitations of two-dimensional image processing-based detection methods, such as limited field of view and strict angle requirements. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in one or more embodiments of this specification or in the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating a three-dimensional crack detection method based on multiple views, provided for one or more embodiments of this specification;

[0021] Figure 2 A flowchart illustrating the multi-view-based three-dimensional crack detection method provided in one or more embodiments of this specification.

[0022] Figure 3 A schematic diagram illustrating an automatic crack length calculation method based on Euclidean minimum spanning tree provided in one or more embodiments of this specification;

[0023] Figure 4 A schematic diagram of an automatic three-dimensional crack width calculation algorithm based on skeleton point guidance provided for one or more embodiments of this specification;

[0024] Figure 5 A flowchart for calculating crack width provided in one or more embodiments of this specification;

[0025] Figure 6 A schematic diagram of the semantic point cloud of a three-dimensional crack in one or more embodiments provided in this specification;

[0026] Figure 7 Example figures illustrating the verification results of the three-dimensional crack length calculation method provided in one or more embodiments of this specification;

[0027] Figure 8 Example figures illustrating the verification results of the three-dimensional crack width calculation method provided in one or more embodiments of this specification;

[0028] Figure 9 Box plot of absolute error analysis of the three-dimensional crack width calculation method in one or more embodiments provided in this specification;

[0029] Figure 10 A schematic diagram of a three-dimensional visualization model of the crack measurement process provided in one or more embodiments of this specification;

[0030] Figure 11 A block diagram of a three-dimensional crack detection device based on multiple views, provided for one or more embodiments of this specification;

[0031] Figure 12 This is a schematic diagram of the structure of a computer device provided for one or more embodiments of this specification. Detailed Implementation

[0032] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this invention.

[0033] The present invention will now be described in detail with reference to specific embodiments and accompanying drawings.

[0034] Method Implementation Examples

[0035] According to embodiments of the present invention, a three-dimensional crack detection method based on multiple views is provided, such as... Figure 1-2 As shown, Figure 1 This is a flowchart of the multi-view-based three-dimensional crack detection method provided in this embodiment. Figure 2 This is a flowchart of a multi-view-based three-dimensional crack detection method provided in this embodiment. The multi-view-based three-dimensional crack detection method according to this embodiment includes the following steps:

[0036] Step 1: Generation of 3D crack semantic point cloud;

[0037] Crack images are acquired, and depth maps are obtained through preprocessing. Crack semantic maps are obtained by recognizing crack images through a semantic segmentation network, and crack pixel values ​​in the crack semantic maps are converted into colors with specified RGB attribute values.

[0038] The depth map is fused with the processed crack semantic map using a depth map fusion algorithm to obtain a 3D crack semantic point cloud with a specified color.

[0039] Step 2: Point cloud processing;

[0040] Based on the RGB color range, point cloud data of 3D crack semantic point cloud is extracted by point cloud extraction method. The points are divided into several independent point cloud clusters by clustering algorithm. Bounding box fitting is performed on each point cloud cluster to obtain crack point cloud instances containing local background points. Multiple planes where cross-plane cracks are located in the crack point cloud instances are detected. Point clouds near each plane are extracted and fitted to obtain the crack point cloud plane equation of the corresponding plane, denoted as P0.

[0041] Step 3: Determine the crack skeleton points and boundary points;

[0042] 2D crack point clouds are obtained by flattening the crack point cloud instances corresponding to each crack plane equation. Then, the mesh model of the 2D crack point cloud is reconstructed using a mesh model reconstruction algorithm. The skeleton point set of each 2D crack point cloud is extracted using a skeleton extraction method. The boundary point set of each 2D crack point cloud corresponding to the skeleton point set is extracted using a boundary point extraction algorithm.

[0043] Step 4: Determine the crack size;

[0044] Based on the obtained set of skeleton points and corresponding boundary points, the length and width of each crack are quantitatively calculated.

[0045] This embodiment utilizes deep learning technology for accurate pixel-level crack semantic image segmentation. Crack pixel values ​​in the semantic image are converted to colors with specified RGB attribute values ​​to create a clear color difference between crack pixels and background pixels. Based on multi-view stereo point cloud reconstruction and point cloud processing techniques, a comprehensive 3D crack semantic point cloud is generated, and crack skeleton points and boundary points are determined. This enables a complete record of complex cross-plane cracks, facilitating subsequent high-precision 3D crack size measurement and achieving rapid structural damage detection and assessment. This provides strong support and assurance for concrete structure inspection and maintenance. This embodiment overcomes the limitations of 2D image processing-based detection methods, such as limited field of view and strict angle requirements.

[0046] In this embodiment, image data acquisition is performed by using a camera to capture a series of apparent color images of the target structure in a surround shooting manner, and it is necessary to ensure that there are overlapping pixels between the images.

[0047] Use multi-view Figure 3 3D reconstruction algorithms calculate the corresponding depth maps for images acquired by the camera, enabling multi-view reconstruction. Figure 3 3D reconstruction algorithms can be implemented using COLMAP, OpenMVS, OpenMVG, VisualSFM, MVE, PhotoScan, PMVS, and DJI Terra software.

[0048] The trained pixel-level crack semantic segmentation network is then used to identify crack pixels in the acquired image and output crack semantic images. The pixel-level crack semantic segmentation network can be an existing network such as Crackdeep, U-Net, DenseNet, Faster R-CNN, CrackFormer, IBR-Former, or CrackSeg9k. The training and optimization of the network can be achieved through existing technologies, which will not be elaborated on here.

[0049] In this embodiment, in order to make the crack points and background points have obvious color differences, in the crack semantic image output by the pixel-level crack semantic segmentation network, the background pixels that are not cracks retain their original colors, while the crack pixel values ​​in the crack semantic image are converted to the color of a specified RGB attribute value, such as red, but not limited to red, with an RGB value of (255, 0, 0).

[0050] In this embodiment, for depth map fusion algorithms, such as COLMAP's depth map fusion algorithm, the depth map is fused with the processed crack semantic map to generate a three-dimensional crack semantic point cloud. The point cloud contains crack points and background points. The information of each point is represented by six-dimensional data (x, y, z, r, g, b), where (x, y, z) represents the three-dimensional spatial coordinates of the point, and (r, g, b) represents the R, G, and B color attribute values ​​of the point, respectively. For example, crack points in the point cloud are presented as a distinctive red color, and background points are presented as the true colors corresponding to the original image.

[0051] In this embodiment, refer to Figure 2 In step two, the point cloud processing step, the point cloud data for extracting the semantic point cloud of the 3D crack is specifically extracted using the point cloud extraction method as follows:

[0052] A crack point cloud extraction method based on RGB color intervals is designed to extract crack points in 3D crack semantic point clouds. This method determines whether a point in the point cloud is a crack point by judging whether the color of the point falls within a specified RGB color interval. The subset of extracted crack points is denoted as P′, which can be expressed as: P′={p∈P|R1≤R(p)≤R2,G1≤G(p)≤G2,B1≤B(p)≤B2}, where P, R(p), G(p), and B(p) of the 3D crack semantic point cloud represent functions that return the red, green, and blue values ​​of point p, respectively; (R1,R2), (G1,G2), and (B1,B2) are the specific interval thresholds of the red, green, and blue channels, respectively.

[0053] In this embodiment, in order to remove noise points from the extracted crack point cloud data and improve the quality of the crack point cloud, the crack point cloud data is preprocessed to remove noise points. The statistical filtering algorithm can also be replaced by methods such as radius filtering, pass-through filtering, bilateral filtering, voxel filtering, etc., which will not be elaborated on here.

[0054] In this embodiment, the point cloud clustering in step two is processed using OPTICS, K-means, or DBSCAN point cloud clustering algorithms to segment the filtered crack point cloud into independent crack point cloud clusters, so that the size of each individual crack point cloud can be measured individually in subsequent steps.

[0055] This embodiment obtains crack point cloud instances by fitting the point cloud orientation bounding box of each crack point cloud cluster and using it to perform local background point clipping. These background points can provide a reference for determining the plane where the crack is located.

[0056] Finally, crack plane detection is performed on the crack point cloud instances containing local background points to determine the plane where the crack is located. Then, the plane equation of the corresponding crack plane is obtained by fitting, as follows:

[0057] The `segment_plane()` function from the Open3D open-source library is used to process crack point cloud instances containing local background points. Multiple planes containing cross-plane cracks are detected within the point cloud instance, and point clouds near each plane are extracted. The RANSAC algorithm is then used to process the extracted point clouds near each plane, fitting the plane equation P0 = a for the corresponding crack plane. i x+b i y+c i z+d i =0, serving as the planar information of the crack point cloud instance in step three.

[0058] In this embodiment, based on the planar information obtained above, it is first necessary to obtain the crack skeleton points and boundary points through point cloud data as the basis for calculating the crack length and width. Specifically, the determination of crack skeleton points and boundary points in step three includes the following steps.

[0059] Step S31: Flatten the point cloud;

[0060] For each extracted plane crack point cloud instance, the crack points are projected onto the fitted plane where the corresponding crack is located, so that the crack point cloud is flattened to form 2D crack points; in this embodiment, the 2D crack points can not only better represent the edge shape of the corresponding crack on the structural surface, but also reduce the complexity of the subsequent surface model reconstruction.

[0061] Step S32: Mesh model reconstruction;

[0062] The rolling ball algorithm is used to process the 2D crack point cloud in step S31 to generate a mesh model of each crack. In this embodiment, the rolling ball radius of the rolling ball algorithm is determined by the average Euclidean distance of the given 2D crack point cloud and the magnification factor α. In this embodiment, α is preferably taken as 3.

[0063] Step S33: Skeleton point extraction;

[0064] Using skeleton extraction methods, such as the Laplacian shrinkage algorithm or the L1-median skeleton extraction method, 2D crack point clouds are processed to extract a detailed set of crack skeleton points corresponding to each crack. This can more efficiently characterize the accurate topological structure and geometric features of the crack point cloud, providing basic data for subsequent crack length calculation and guiding the subsequent crack width calculation.

[0065] Step S34: Boundary point extraction;

[0066] The compute_boundary_points() function from the Open3D open-source library is used to process the point clouds of each plane crack in step S31, and a fine set of crack boundary points is extracted as the boundary information for subsequent crack width calculation.

[0067] In this embodiment, since algorithms such as Laplacian shrinkage rely on the uniformity and density of the point cloud, only operations on high-quality point clouds can guarantee the output of an ideal skeleton point set. Therefore, in order to obtain a denser and more uniform point cloud, improve the quality of the flattened 2D crack point cloud, and enhance the precision of crack skeleton point and boundary point extraction in subsequent steps S33 and S34, the following steps are also included:

[0068] Point cloud upsampling: In step S32, the 2D crack point cloud is upsampled on the crack mesh model to obtain a denser and more uniform point cloud.

[0069] Based on the extracted skeleton point set and corresponding boundary point set, this embodiment provides an automatic crack size measurement method based on three-dimensional point cloud processing technology to realize crack length and width calculation, as detailed below.

[0070] (1) Crack length calculation;

[0071] This embodiment calculates and determines the length of each crack based on the extracted crack skeleton point set and corresponding boundary point set using the Euclidean minimum spanning tree algorithm.

[0072] refer to Figure 3 The diagram illustrates the automatic crack length calculation method based on the Euclidean minimum spanning tree provided in this embodiment. The Euclidean minimum spanning tree is defined on a complete graph G. In this embodiment, skeleton points are used as vertices, and the Euclidean distance between skeleton points is used as the edge weight. The Euclidean distance matrix of the skeleton points is the edge weight E(G) of graph G, representing the Euclidean distance between skeleton points. The edge set E(T) of the Euclidean minimum spanning tree is a subset of the output of E(G) under the action of the objective function. The objective function returns the edges connecting skeleton points. The result of the objective function represents the sum of all edge lengths in the Euclidean minimum spanning tree, and also the sum of the distance values ​​in the distance matrix of the Euclidean minimum spanning tree, which is the crack length.

[0073] In this embodiment, a skeleton point pair set is formed by determining two connected skeleton points based on the Euclidean minimum spanning tree. Based on the skeleton point set and the obtained skeleton point pair set, the subsequent crack width calculation provides the necessary guiding direction.

[0074] This crack length measurement method enables direct and rapid crack length calculation. Compared with existing technologies, this method does not require manual setting of crack measurement points, endpoints, or directions, greatly improving the automation level of three-dimensional crack length measurement.

[0075] (2) Crack width calculation;

[0076] This embodiment calculates the crack width based on a crack on any plane, using a set of skeleton points as the traversal object. The specific steps include:

[0077] Step A1: Using each skeleton point as the center, generate a sphere with a preset radius, and determine the boundary points within the sphere as boundary candidate points.

[0078] Step A2: Determine the skeleton point pair and connecting line corresponding to the current skeleton point, and determine the equation of the first plane perpendicular to the connecting line and passing through the current skeleton point using the direction of the connecting line as the normal vector.

[0079] Step A3: Based on the current skeleton points, determine a second plane that is perpendicular to both the first plane and the crack point cloud plane, and use this second plane to divide the boundary candidate points into positive and negative sides.

[0080] Step A4: Calculate and determine the pair of candidate boundary points with the shortest distance from each candidate boundary point on both the front and back sides to the first plane, and determine the distance of the candidate boundary point pair by the distance calculation method, which is used as the crack width of the corresponding skeleton point.

[0081] Step A5: Repeat steps A1-A4 to determine the crack width at the next skeleton point.

[0082] In this embodiment, the above process can be implemented through written program code. The algorithm takes the crack plane equation P0 corresponding to the crack, the corresponding boundary point set and skeleton point set, the preset sphere radius, and the skeleton point pair formed by two connected points determined based on the Euclidean minimum spanning tree as input parameters. Each skeleton point is used as a measurement point, and the goal is to quantify the crack width at each measurement point. The basic principle is to identify a suitable pair of boundary points near the skeleton point. Ideally, the connecting line of the boundary point pair should be as perpendicular as possible to the connecting line of the skeleton point pair. Then, the distance between the boundary point pairs is the crack width of the measurement point (corresponding skeleton point). This algorithm can complete the automatic measurement of dense crack widths by using the skeleton points to provide the measurement points and path directions.

[0083] In one specific embodiment, reference is made to Figure 4 and Figure 5 As shown, Figure 4 This is a schematic diagram of the automatic calculation algorithm for three-dimensional crack width based on skeleton points provided in this embodiment. Figure 5 The crack width calculation flowchart provided in this embodiment includes the following steps:

[0084] Step A11: Determine the equation of the crack plane P0, and the set of boundary points P corresponding to this plane. b and skeleton point pair set P s,p ={(p s ′,p s ") s}, and the preset sphere radius R.

[0085] Step A12: Traverse the skeleton point-pair set P s,pEach group (p) s ′,p s ") s Determine if the traversal has ended. If the traversal is complete, proceed to step A; otherwise, proceed to step A13.

[0086] Step A13: Use the center of the sphere as the skeleton point p s For a sphere with radius R, select the skeleton point p. s Candidate boundary points at ′.

[0087] Step A14: Determine the skeleton points (p) s ′,p s The direction of the line connecting "") is represented as by As the normal vector, calculate and determine the perpendicularity to And through the skeleton point p s The first plane P1;

[0088] Let the equation of the first plane be P1 = A1x + B1y + C1z + D1 = 0, written as [A1, B1, C1, D1]. Since the first plane P1 is perpendicular to the line connecting the skeleton point pairs, the normal vector of the first plane P1 [A1, B1, C1] is the skeleton point (p... s ′,p s The direction of the line connecting "") is represented as Then, since the first plane P1 passes through the center of the sphere (i.e., the skeleton point p), s p′), can be p′ s Substituting (x0, y0, z0) into the equation A1x + B1y + C1z + D1 = 0, we can find D1, which can be expressed as d1 = This completes the solution to the first plane equation P1.

[0089] Step A15: Based on the current skeleton point p′ s A second plane P2 is determined that is perpendicular to both the first plane P1 and the crack point cloud plane P0. The boundary candidate points are divided into positive and negative sides by the second plane P2.

[0090] Let the equation of the second plane be P2 = A2x + B2y + C2z + D2 = 0, written as [A2, B2, C2, D2]. Since the second plane P2 is perpendicular to both planes P0 and P1, we can obtain two equations:

[0091]

[0092] Furthermore, because the second plane P2 also passes through the skeleton point (p s ′,p s Therefore, p can be used.s Substituting (x0, y0, z0) into the equation A²x + B²y + C²z + D² = 0;

[0093] Another equation is: x0A² + y0B² + z0C² + D² = 0; finally, D² is given as any real number, such as D² = 1. This yields the final system of equations:

[0094]

[0095] The matrix representation of the system of equations is

[0096]

[0097] Solving the system of equations yields the equation P2 for the second plane.

[0098] Step A16: Based on the second plane, divide the candidate boundary points into two sides, thus obtaining the set of points P on the positive and negative sides. c,N and negative side point set P c,P ,as follows:

[0099]

[0100]

[0101] Step A17: Determine that the point closest to the first plane P1 in the candidate point set of the opposite boundary is p. c The determination formula is: The point closest to the first plane P1 in the candidate point set of the positive and side boundaries is determined to be p. c The determination formula is:

[0102] Step A17: Using the two boundary points on the front and back sides of the first plane P1, a pair of boundary points is formed to calculate the crack width. The width w is calculated using Euclidean distance, as shown in the following formula:

[0103]

[0104] Step A18: Pair the corresponding boundary points (p) c ′,p c The crack width w is saved to the set of boundary point pairs P. b,p ={(p c ′,p c ") i} and the set of crack widths W = {w i}

[0105] In this embodiment, in order to visualize the crack condition of structural components and the corresponding crack parameters, and to quickly complete the detection and assessment of structural damage, the above crack point cloud instance clustering results, the skeleton points of each crack, the boundary point set, the crack width and length calculation results are used to establish a three-dimensional visualization model.

[0106] In one specific embodiment, based on the Open3D open-source library, the obtained 3D crack semantic point cloud, crack point cloud instance clustering results, crack length calculation results, crack width measurement point locations and calculation results are integrated into a 3D visualization model. Each individual crack is accurately segmented and located in the 3D point cloud, and the length, maximum width and location of each crack are annotated, thereby helping to detect severely damaged areas. This embodiment, by integrating into a 3D visualization platform, can integrate accurate location and measurement results of 3D cracks, realize rapid structural damage detection and assessment, and provide strong support and guarantee for concrete structure inspection and maintenance.

[0107] To more clearly describe the technical advantages of the present invention, a detailed description will be provided below with reference to the accompanying drawings and a specific embodiment. It should be noted that this embodiment is only used to illustrate the principle and implementation of the method and should not be considered as a limitation of the present invention.

[0108] Case Study: This embodiment uses a reinforced concrete test beam as an example to illustrate the proposed three-dimensional crack detection method based on multiple views. The beam measures 1400mm × 120mm × 200mm. After a bending failure test, five distinct vertical cracks appeared in the middle of the beam, spanning multiple planes. Therefore, the damage to be detected consists of these five cross-plane cracks on the beam surface.

[0109] The main implementation steps of this embodiment are as follows:

[0110] S01: Data Acquisition: Using a monocular camera with a lens focal length of 26mm, 157 images of the beam surface were acquired through a surround shooting method. The image resolution was 1600×1200, and the working distance of the camera was kept at approximately 0.3m.

[0111] S02: Depth map calculation: Store the images acquired in step S01 in folder A and input them into COLMAP software. Create a new project A, select the pinhole camera model, and calculate the depth map for each image.

[0112] S03: Crack segmentation: Input the image acquired in step S01 into the trained pixel-level crack semantic segmentation network Crackdeep, identify the crack pixels in each image, and output the crack semantic image in which the crack pixels are red and the RGB (Red, Green, Blue) value is (255, 0, 0), while the background pixel color remains the original true color.

[0113] S04: Depth Map Fusion: Replace the image in folder A mentioned in step S02 with the crack semantic image in step S03. Keep the original path of the depth map mentioned in step S02. Restart project A of COLMAP software in step S02 and click the "Fusion" function. The software can then use the crack semantic image and depth map to generate a 3D crack semantic point cloud. The crack points in the point cloud are presented as a distinctive red, and the background points are presented as the true colors corresponding to the original image.

[0114] S05: Restore the True Size of the Point Cloud: Open the 3D crack semantic point cloud generated in step S04 using the "MeshLab" point cloud processing software. Use the "Measurement Tool" in "MeshLab" to measure the size of the beam in the point cloud. The ratio of this measurement to the true size of the beam is the scale factor. Then, use this scale factor to adjust the 3D crack semantic point cloud to the true scale using the "Transform Tool" in "MeshLab". The unit of the restored true size of the 3D crack semantic point cloud is millimeters. Figure 6 As shown.

[0115] S06: Crack Point Cloud Extraction: Using the proposed crack point cloud extraction method based on RGB color range, crack points are extracted from the 3D crack semantic point cloud after restoring the true size as described in step S05. In this case, the specific technical parameters of the method are: the specific interval thresholds of the red, green and blue channels are (R1,R2)=(99,255), (G1,G2)=(0,65), and (B1,B2)=(0,56), respectively.

[0116] S07: Point Cloud Filtering: Use a statistical filtering algorithm to filter noise points in the crack point cloud extracted in step S06 to improve the quality of the crack point cloud. In this case, the specific technical parameters of the statistical filtering algorithm are: neighbor number is 10, and standard deviation ratio is 0.2.

[0117] S08: Point Cloud Clustering: The filtered crack point cloud from step S05 is processed using the OPTICS point cloud clustering algorithm to segment and obtain independent crack point cloud instances. In this case, the specific technical parameters of the OPTICS point cloud clustering algorithm are: maximum radius of 7 and minimum number of samples of 200. The clustering result in this case is 5 independent and complete 3D crack point clouds.

[0118] S09: Point cloud bounding box clipping: For each crack point cloud cluster obtained by clustering in step S08, use the oriented bounding box formed by it to clip the local background points, and obtain a crack point cloud instance containing local background points.

[0119] S10: Plane Detection: Use the segment_plane() function in the Open3D open-source library to process the crack point cloud instance containing local background points in step S09, and detect multiple planes in the point cloud instance where the cross-plane crack is located.

[0120] S11: Plane Fitting: Use the RANSAC algorithm to process the point cloud near each plane extracted in step S09, and fit the plane equation of that plane.

[0121] S12: Point cloud flattening: Project the crack points in the point cloud extracted in step S09 onto the corresponding plane fitted in step S11 to flatten the crack point cloud and form 2D crack points.

[0122] S13: Mesh Model Reconstruction: The rolling sphere radius magnification factor α is set to 3. The 2D crack points in step S12 are processed using the rolling sphere algorithm to generate the crack mesh model.

[0123] S14: Point cloud upsampling: In step S12, the crack mesh model is upsampled to a point cloud with 18,000 points. Compared with the 4,798 points of the 2D crack points mentioned in step S12 in this case, the density and uniformity are greatly improved.

[0124] S15: Skeleton Point Extraction: The Laplacian shrinkage algorithm is used to process the upsampled crack point cloud in step S13 to extract fine crack skeleton points. In this case, there are 427 crack skeleton points, and the distance between any two points is less than 1.5 mm, which is beneficial for improving the accuracy of crack length measurement.

[0125] S16: Boundary point extraction: The compute_boundary_points() function in the Open3D open-source library is used to process the upsampled crack point cloud in step S14 to extract a fine and dense set of crack boundary points.

[0126] S17: Crack Length Calculation: Using the proposed automatic crack length calculation method based on Euclidean minimum spanning tree, the crack skeleton points obtained in step S15 are calculated. The length of the obtained skeleton line is the crack length. The crack length measurement result of one crack in this case is as follows: Figure 7 As shown in the figure; in this case, the actual lengths of the five cross-plane cracks (C-1 to C-5) on the beam surface were manually measured using digital vernier calipers as a reference. The errors, relative errors, and average absolute errors of the method of the present invention in automatically measuring three-dimensional cracks were analyzed. The error analysis results are shown in the table below. The absolute error of the method of the present invention is basically less than 10 mm, and the average relative error is only 1.30%.

[0127] Unit: mm

[0128]

[0129] S18: Crack Width Calculation: The full-segment crack width of each crack is calculated using a skeleton point-guided 3D crack width automatic calculation algorithm developed in this embodiment. Input the crack surface plane equation described in step S11, the boundary point set described in step S16, and the skeleton point pair set described in step S17. In this case, the sphere radius of the 3D crack width automatic calculation algorithm is set to 10. The local width measurement result of a crack in this case is as follows: Figure 8 As shown; for reference Figure 6 The cracks spanned three surfaces of the test beam: the front, bottom, and rear surfaces. Five verification points were randomly selected on each surface of each crack, and their widths were manually measured using a crack detection card and a digital vernier caliper. Therefore, each crack generated 15 verification data points, and the five cracks generated a total of 75 verification data points. The verification results are presented using a box plot, as shown below. Figure 9 As shown, the box plot of the absolute error analysis of the three-dimensional crack width calculation method provided in this case shows that the absolute error distribution of crack width measurement is highly symmetrical with little dispersion, and the median of the absolute error is in the range of 0.10mm to 0.15mm, which meets the requirements of engineering applications.

[0130] S19: Construction of a 3D Visualization Model for Crack Measurement: Based on the Open3D open-source library, the 3D crack semantic point cloud obtained in step S05, the crack point cloud instance clustering results obtained in step S08, the crack length calculation results obtained in step S17, and the crack width measurement point locations and calculation results obtained in step S18 are integrated into a 3D visualization model, such as... Figure 10 The diagram shows a schematic of the 3D visualization model for crack measurement provided in this case. Each individual crack is accurately segmented and located in the 3D point cloud, and the length, maximum width, and location of each crack are annotated. This helps to detect severely damaged areas, effectively display the quantitative results of cracks, and thus quickly complete the detection and assessment of structural damage.

[0131] Device Examples

[0132] According to embodiments of the present invention, a three-dimensional crack detection device based on multiple views is provided, such as... Figure 11 As shown, Figure 11 This is a schematic block diagram of a multi-view-based three-dimensional crack detection device provided in this embodiment. According to an embodiment of the present invention, the multi-view-based three-dimensional crack detection device includes:

[0133] 3D crack semantic point cloud generation module 10: configured to acquire crack images, preprocess crack images to obtain depth maps; identify crack images through a semantic segmentation network to obtain crack semantic maps, and convert crack pixel values ​​in crack semantic maps into colors with specified RGB attribute values.

[0134] The depth map is fused with the processed crack semantic map using a depth map fusion algorithm to obtain a 3D crack semantic point cloud with a specified color.

[0135] Point cloud processing module 20: Configured to extract point cloud data of three-dimensional crack semantic point cloud based on RGB color range using point cloud extraction method, divide the points into several independent point cloud clusters using clustering algorithm, and perform bounding box fitting on each point cloud cluster to obtain crack point cloud instances containing local background points. Detect multiple planes where cross-plane cracks are located in the crack point cloud instances, extract point clouds near each plane and fit the corresponding crack point cloud plane equation, denoted as P0.

[0136] Crack skeleton point and boundary point determination module 30: configured to obtain 2D crack point clouds by flattening the corresponding crack point cloud instances according to the plane equation of each crack, then reconstruct the mesh model of the 2D crack point cloud using a mesh model reconstruction algorithm, extract the skeleton point set of each 2D crack point cloud using a Laplacian shrinkage algorithm, and extract the boundary point set of the corresponding skeleton point set of each 2D crack point cloud using a boundary point extraction algorithm.

[0137] Crack size determination module 40: configured to quantitatively calculate the length and width of each crack based on the acquired set of skeleton points and the corresponding set of boundary points.

[0138] In this embodiment, the 3D crack semantic point cloud generation module 10 uses deep learning technology to perform accurate pixel-level crack semantic image segmentation. It converts the crack pixel values ​​in the crack semantic image into colors with specified RGB attribute values, creating a clear color difference between crack pixels and background pixels. Based on multi-view stereo 3D point cloud reconstruction and point cloud processing technologies, it generates a comprehensive 3D crack semantic point cloud and determines crack skeleton points and boundary points. This enables a complete record of complex cross-plane cracks, facilitating subsequent high-precision 3D crack size measurement and achieving rapid structural damage detection and assessment. This provides strong support and assurance for concrete structure inspection and maintenance. This embodiment overcomes the limitations of 2D image processing-based detection methods, such as limited field of view and strict angle requirements.

[0139] In this embodiment, in order to make the crack points and background points have obvious color differences, in the crack semantic image output by the pixel-level crack semantic segmentation network, the background pixels that are not cracks retain their original colors, while the crack pixel values ​​in the crack semantic image are converted to the color of a specified RGB attribute value, such as red, but not limited to red, with an RGB value of (255, 0, 0).

[0140] In this embodiment, for depth map fusion algorithms, such as COLMAP's depth map fusion algorithm, the depth map is fused with the processed crack semantic map to generate a three-dimensional crack semantic point cloud. The point cloud contains crack points and background points. The information of each point is represented by six-dimensional data (x, y, z, r, g, b), where (x, y, z) represents the three-dimensional spatial coordinates of the point, and (r, g, b) represents the R, G, and B color attribute values ​​of the point, respectively. For example, crack points in the point cloud are presented as a distinctive red color, and background points are presented as the true colors corresponding to the original image.

[0141] In this embodiment, the point cloud processing module 20 extracts the point cloud data of the three-dimensional crack semantic point cloud using the point cloud extraction method as follows:

[0142] A crack point cloud extraction method based on RGB color intervals is designed to extract crack points in 3D crack semantic point clouds. This method determines whether a point in the point cloud is a crack point by judging whether the color of the point falls within a specified RGB color interval. The subset of extracted crack points is denoted as P′, which can be expressed as: P′={p∈P|R1≤R(p)≤R2,G1≤G(p)≤G2,B1≤B(p)≤B2}, where P, R(p), G(p), and B(p) of the 3D crack semantic point cloud represent functions that return the red, green, and blue values ​​of point p, respectively; (R1,R2), (G1,G2), and (B1,B2) are the specific interval thresholds of the red, green, and blue channels, respectively.

[0143] In this embodiment, in order to remove noise points from the extracted crack point cloud and improve the quality of the crack point cloud, a filtering module 201 is also provided, which is configured to use a statistical filtering algorithm to filter noise points in the extracted crack point cloud. The statistical filtering algorithm can also be replaced by methods such as radius filtering, pass-through filtering, bilateral filtering, voxel filtering, etc., which will not be elaborated on here.

[0144] Point cloud processing module 20 is used to perform crack plane detection on crack point cloud instances containing local background points, determine the plane where the crack is located, and then fit the plane equation of the corresponding crack plane, as follows:

[0145] The `segment_plane()` function from the Open3D open-source library is used to process crack point cloud instances containing local background points. Multiple planes containing cross-plane cracks are detected within the point cloud instance, and point clouds near each plane are extracted. The RANSAC algorithm is then used to process the extracted point clouds near each plane, fitting the plane equation P0 = a for the corresponding crack plane. i x+b i y+c i z+d i =0, serving as the planar information of the crack point cloud instance in step three.

[0146] In this embodiment, based on the planar information obtained above, it is first necessary to obtain the crack skeleton points and boundary points through point cloud data as the basis for calculating the crack length and width. Specifically, the crack skeleton point and boundary point determination module 30 is used to determine the crack skeleton points and boundary points through the following modules:

[0147] Point cloud flattening module 31: For each extracted plane crack point cloud instance, the crack points are projected onto the fitted plane where the corresponding crack is located, so that the crack point cloud is flattened and a 2D crack point is formed.

[0148] Mesh model reconstruction module 32: Used to process the 2D crack point cloud obtained by the point cloud flattening module 31 using the rolling ball algorithm, and generate mesh models for each crack;

[0149] Skeleton point extraction module 33: Used to process 2D crack point clouds using skeleton extraction methods, such as the Laplacian shrinkage algorithm or the L1-median skeleton extraction method, to extract a fine set of crack skeleton points corresponding to each crack;

[0150] Boundary point extraction module 34: Used to process the point cloud of each plane crack obtained by the point cloud flattening module 31 using the compute_boundary_points() function in the Open3D open source library, and extract a fine set of crack boundary points as boundary information for subsequent crack width calculation.

[0151] In this embodiment, since algorithms such as Laplacian shrinkage rely on the uniformity and density of the point cloud, only operations on high-quality point clouds can guarantee the output of an ideal skeleton point set. Therefore, in order to obtain a denser and more uniform point cloud, improve the quality of the flattened 2D crack point cloud, and enhance the precision of crack skeleton point and boundary point extraction in subsequent modules, the following measures are also included:

[0152] The point cloud upsampling module 301 is used to upsample the 2D crack point cloud on the generated crack mesh model to obtain a denser and more uniform point cloud.

[0153] Based on the extracted skeleton point set and corresponding boundary point set, this embodiment provides an automatic crack size measurement process based on three-dimensional point cloud processing technology to realize crack length and width calculation, as follows.

[0154] (1) Crack length calculation submodule 302;

[0155] The crack length calculation submodule 302 is configured to calculate and determine the length of each crack based on the extracted crack skeleton point set and the corresponding boundary point set, using the Euclidean minimum spanning tree algorithm.

[0156] refer to Figure 3 The diagram illustrates the automatic crack length calculation method based on the Euclidean minimum spanning tree provided in this embodiment. The Euclidean minimum spanning tree is defined on a complete graph G. In this embodiment, skeleton points are used as vertices, and the Euclidean distance between skeleton points is used as the edge weight. The Euclidean distance matrix of the skeleton points is the edge weight E(G) of graph G, representing the Euclidean distance between skeleton points. The edge set E(T) of the Euclidean minimum spanning tree is a subset of the output of E(G) under the action of the objective function. The objective function returns the edges connecting skeleton points. The result of the objective function represents the sum of all edge lengths in the Euclidean minimum spanning tree, and also the sum of the distance values ​​in the distance matrix of the Euclidean minimum spanning tree, which is the crack length.

[0157] In this embodiment, two connected points are determined to form a skeleton point pair based on the Euclidean minimum spanning tree. Based on the skeleton point set and the obtained skeleton point pair set, the subsequent crack width calculation provides the necessary guiding direction.

[0158] This crack length measurement method enables direct and rapid crack length calculation. Compared with existing technologies, this method does not require manual setting of crack measurement points, endpoints, or directions, greatly improving the automation level of three-dimensional crack length measurement.

[0159] (2) Crack width calculation submodule 303;

[0160] The crack width calculation submodule 303 calculates the crack width based on a crack on any plane, using the skeleton point set as the traversal object. Specifically, it includes the following units to calculate the crack width.

[0161] Unit 1 21: Used to generate a sphere with a preset radius centered on each skeleton point, and to determine the boundary points within the sphere as boundary candidate points.

[0162] Unit 22: Used to determine the skeleton point pair and connection line corresponding to the current skeleton point, and to determine the equation of the first plane perpendicular to the connection line and passing through the current skeleton point using the direction of the connection line as the normal vector.

[0163] Unit 23: Based on the current skeleton point, determine a second plane that is perpendicular to both the first plane and the crack point cloud plane, and use this second plane to divide the boundary candidate points into positive and negative sides.

[0164] Unit 4, 24: Used to calculate and determine the pair of candidate boundary points with the shortest distance from each candidate boundary point on both sides to the first plane, and to determine the distance of the candidate boundary point pair by the distance calculation method, which is used as the crack width of the corresponding skeleton point.

[0165] Unit 5 25: Used to sequentially call Units 21 to 24 to determine the crack width of the next skeleton point.

[0166] The embodiments of the present invention are device embodiments corresponding to the above method embodiments. The specific operations of each module processing step can be understood with reference to the description of the method embodiments, and will not be repeated here.

[0167] like Figure 12 As shown, the present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it implements the identification model training method in the above embodiments, or when the computer program is executed by a processor, it implements the three-dimensional crack detection method based on multiple views in the above embodiments.

[0168] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0169] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for apparatus or system embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The apparatus and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0170] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and the contents not described in detail in the specification of the present invention are well known to those skilled in the art.

Claims

1. A three-dimensional crack detection method based on multiple views, characterized in that, Including the following steps: 3D crack semantic point cloud generation: Crack images are acquired and preprocessed to obtain a depth map; crack semantic maps are obtained by recognizing crack images through a semantic segmentation network, and crack pixel values ​​in the crack semantic maps are converted into colors with specified RGB attribute values; the depth map and the processed crack semantic map are fused through a depth map fusion algorithm to obtain a 3D crack semantic point cloud with specified colors. Point cloud processing: Based on the RGB color range, point cloud data of 3D crack semantic point cloud is extracted by point cloud extraction method, and the points are divided into several independent point cloud clusters by clustering algorithm. Bounding box fitting is performed on each point cloud cluster to obtain crack point cloud instances containing local background points. Multiple planes where cross-plane cracks are located in the crack point cloud instances are detected, and point clouds near each plane are extracted and fitted to obtain the corresponding crack point cloud plane equation. Determination of crack skeleton points and boundary points: 2D crack point clouds are obtained by flattening the crack point cloud instances corresponding to each crack plane equation; the mesh model of the 2D crack point cloud is reconstructed using a mesh model reconstruction algorithm; the skeleton point set of each 2D crack point cloud is extracted using a skeleton extraction method; and the boundary point set of each 2D crack point cloud corresponding to the skeleton point set is extracted using a boundary point extraction algorithm. Crack size determination: Based on the obtained set of skeleton points and corresponding boundary point sets, the length and width of each crack are quantitatively calculated; The calculation of the crack width includes the following steps: Based on a crack in any plane, and using the skeleton point set as the traversal object, perform the following steps: Using each skeleton point as the center, generate a sphere with a preset radius, and determine the boundary points within the sphere as boundary candidate points; Determine the skeleton point pair and the connecting line corresponding to the current skeleton point, and determine the equation of the first plane perpendicular to the connecting line and passing through the current skeleton point, using the direction of the connecting line as the normal vector. Based on the current skeleton points, a second plane is determined that is perpendicular to both the first plane and the crack point cloud plane. The boundary candidate points are then divided into two sides by this second plane. Calculate and determine the pair of candidate boundary points with the shortest distance from each candidate boundary point on both sides to the first plane, and determine the distance of the candidate boundary point pair by the distance calculation method, which is used as the crack width of the corresponding skeleton point; Repeat all the above steps to determine the crack width at the next skeleton point.

2. The three-dimensional crack detection method based on multiple views as described in claim 1, characterized in that, The plane equation of the crack point cloud is obtained through the following steps: The `segment_plane()` function from the Open3D open-source library is used to process crack point cloud instances containing local background points. Multiple planes containing cross-plane cracks are detected within the point cloud instance, and point clouds near each plane are extracted. The RANSAC algorithm is then used to process the extracted point clouds near each plane, fitting the plane equation corresponding to the crack's location. .

3. The three-dimensional crack detection method based on multiple views as described in claim 1, characterized in that, The step of reconstructing the mesh model of the 2D crack point cloud using the mesh model reconstruction algorithm also includes: Point cloud upsampling: Based on the crack mesh model, upsampling processing of 2D crack point clouds is performed.

4. The three-dimensional crack detection method based on multiple views as described in claim 1, characterized in that, The crack length was calculated and determined using the Euclidean minimum spanning tree algorithm. Furthermore, based on the Euclidean minimum spanning tree, it is determined that two connected points form a skeleton point pair set.

5. The three-dimensional crack detection method based on multiple views as described in claim 1, characterized in that, It also includes the following steps: Based on the clustering results of crack point cloud instances, the skeleton points and boundary point sets of each crack are obtained. The calculation results of seam width and length are used to build a three-dimensional visualization model.

6. A three-dimensional crack detection device based on multiple views, characterized in that, include: 3D crack semantic point cloud generation module: configured to acquire crack images and preprocess the crack images to obtain depth maps; A semantic segmentation network is used to identify crack images to obtain a crack semantic map, and the crack pixel values ​​in the crack semantic map are converted into colors with specified RGB attribute values. A depth map fusion algorithm is used to fuse the depth map with the processed crack semantic map to obtain a 3D crack semantic point cloud with specified colors. Point cloud processing module: Configured to extract point cloud data of 3D crack semantic point cloud based on RGB color range and point cloud extraction method, and divide the points into several independent point cloud clusters through clustering algorithm, and perform bounding box fitting on each point cloud cluster to obtain crack point cloud instance containing local background points. Detect multiple planes in the crack point cloud instance where cross-plane cracks are located, extract point clouds near each plane and fit the corresponding crack point cloud plane equation. Crack skeleton point and boundary point determination module: Configured to obtain 2D crack point clouds by flattening the corresponding crack point cloud instances according to the plane equation of each crack; reconstruct the mesh model of the 2D crack point cloud using a mesh model reconstruction algorithm; extract the skeleton point set of each 2D crack point cloud using a skeleton extraction method; and extract the boundary point set of the corresponding skeleton point set of each 2D crack point cloud using a boundary point extraction algorithm. Crack Size Determination Module: Configured to quantitatively calculate the length and width of each crack based on the acquired skeleton point set and corresponding boundary point set; the crack skeleton point and boundary point determination module includes a crack width calculation submodule, which calculates the crack width based on a crack on any plane, using the skeleton point set as the traversal object, through the following units, specifically including: Unit 1: Used to generate a sphere with a preset radius, centered on each skeleton point, and to determine the boundary points within the sphere as candidate boundary points; The second unit is used to determine the skeleton point pair and the connecting line corresponding to the current skeleton point, and to determine the equation of the first plane perpendicular to the connecting line and passing through the current skeleton point using the direction of the connecting line as the normal vector. The third unit is used to determine a second plane that is perpendicular to both the first plane and the crack point cloud plane based on the current skeleton point, and to divide the boundary candidate points into two sides by the second plane. The fourth unit is used to calculate and determine the pair of candidate boundary points with the shortest distance from each candidate boundary point on both sides to the first plane, and to determine the distance of the candidate boundary point pair by the distance calculation method, which is used as the crack width of the corresponding skeleton point. Unit 5: Used to sequentially call Unit 1 through Unit 1 to determine the crack width of the next skeleton point.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the three-dimensional crack detection method based on multiple views as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the multi-view-based three-dimensional crack detection method as described in any one of claims 1 to 5.

Images