A three-dimensional model measurement method and related device for concrete structure cracks

Abstract

The present application discloses a three-dimensional model measurement method for cracks in concrete structures, comprising: training an initial semantic segmentation model based on a labeled concrete crack training dataset to obtain a crack semantic segmentation model; using the crack semantic segmentation model to detect key frames of a captured binocular video to obtain a crack segmentation mask map; performing three-dimensional reconstruction based on the key frames of the binocular video, the crack segmentation mask map, and the camera pose corresponding to the key frames to obtain a three-dimensional point cloud model with crack positioning information; wherein the camera pose is calculated based on the corresponding key frames using a SLAM algorithm based on binocular vision; and performing crack measurement based on the three-dimensional point cloud model to obtain crack measurement data. This improves the efficiency of crack detection and measurement. The present application also discloses a three-dimensional model measurement device, terminal device, and computer-readable storage medium for cracks in concrete structures, which have the above beneficial effects.

Description

Technical Field

[0001] The present application relates to the field of engineering technology, and in particular to a three-dimensional model measurement method, a three-dimensional model measurement device, a terminal device, and a computer-readable storage medium for measuring cracks in a concrete structure. Background Art

[0002] During their service life, concrete structures can develop cracks due to the combined effects of long-term loads, natural disasters, and other adverse factors. These cracks are a sign of deteriorating structural health and pose a serious threat to their safe operation. Therefore, timely and effective detection and assessment of crack damage are essential to ensure the safe operation of concrete structures.

[0003] In related technologies, taking concrete bridges as an example, the detection of apparent cracks in bridges relies heavily on manual observation. Inspectors on bridge inspection vehicles and ships rely on the naked eye to observe apparent cracks. The detection results are affected by the inspectors' technical experience, resulting in low detection efficiency and extremely dangerous inspections in high-risk areas. Furthermore, the determination and recording of crack morphology and location rely on inspectors writing descriptions in inspection reports supplemented by drawings and photographs. This method is cumbersome and inefficient, lacks good three-dimensional visualization, and makes it difficult for bridge engineers to intuitively understand the crack morphology and location and assess the extent of structural damage. Finally, crack measurement relies heavily on traditional contact measurement methods such as rulers and digital calipers, which are difficult to use in areas that are difficult for personnel to reach. Therefore, traditional manual methods for detecting and measuring apparent cracks in structures are cumbersome and inefficient, and the detection results are highly subjective and dangerous.

[0004] Therefore, how to make crack detection and measurement more accurate and efficient is a key issue that those skilled in the art are concerned about. Summary of the Invention

[0005] The purpose of this application is to provide a three-dimensional model measurement method, a three-dimensional model measurement device, a terminal device and a computer-readable storage medium for concrete structure cracks to improve the efficiency of crack detection and measurement.

[0006] To solve the above technical problems, the present application provides a three-dimensional model measurement method for concrete structure cracks, comprising:

[0007] The initial semantic segmentation model is trained based on the labeled concrete crack training dataset to obtain a crack semantic segmentation model.

[0008] Using the crack semantic segmentation model to detect key frames of the collected binocular video to obtain a crack segmentation mask map;

[0009] Performing three-dimensional reconstruction based on the key frames of the binocular video, the crack segmentation mask image, and the camera pose corresponding to the key frames to obtain a three-dimensional point cloud model with crack positioning information; wherein the camera pose is calculated by a binocular vision-based SLAM algorithm on the corresponding key frames;

[0010] Crack measurement is performed based on the three-dimensional point cloud model to obtain crack measurement data.

[0011] Optionally, the initial semantic segmentation model is trained based on the labeled concrete crack training dataset to obtain a crack semantic segmentation model, including:

[0012] Collect pictures of concrete cracks;

[0013] Annotate the concrete crack image using a semantic segmentation and annotation tool to obtain the concrete crack training dataset;

[0014] The initial semantic segmentation model is trained using the concrete crack training dataset to obtain the crack semantic segmentation model.

[0015] Optionally, the crack semantic segmentation model is used to detect key frames of the collected binocular video to obtain a crack segmentation mask map, including:

[0016] Continuously shooting videos using a binocular camera module to obtain the binocular video;

[0017] Performing sparse reconstruction on the video stream data through a binocular vision SLAM algorithm and extracting key frames to obtain the key frames;

[0018] The key frame is detected using the crack semantic segmentation model to obtain the crack segmentation mask image.

[0019] Optionally, performing three-dimensional reconstruction based on the key frames of the binocular video, the crack segmentation mask image, and the camera pose corresponding to the key frames to obtain a three-dimensional point cloud model with crack location information includes:

[0020] Performing dense reconstruction of three-dimensional point clouds based on key frames of the binocular video and camera poses corresponding to the key frames to obtain point cloud data;

[0021] The three-dimensional crack mapping is performed on the point cloud data based on the crack segmentation mask image to obtain the three-dimensional point cloud model with crack positioning information.

[0022] Optionally, performing crack measurement based on the three-dimensional point cloud model to obtain crack measurement data includes:

[0023] performing point cloud segmentation processing on the three-dimensional point cloud model having crack location information based on point cloud categories to obtain a crack point cloud;

[0024] Clustering the crack point cloud based on a Euclidean clustering algorithm to obtain multiple crack clusters;

[0025] Determining the crack direction of each crack cluster based on principal component analysis, and calculating the three-dimensional skeleton points of the cracks;

[0026] Based on the crack direction and the three-dimensional crack skeleton points of each crack cluster, the crack length and crack width are calculated to obtain the crack measurement data.

[0027] The present application also provides a three-dimensional model measuring device for concrete structure cracks, comprising:

[0028] The model training module is used to train the initial semantic segmentation model based on the labeled concrete crack training dataset to obtain a crack semantic segmentation model;

[0029] An image detection module is used to detect key frames of the collected binocular video using the crack semantic segmentation model to obtain a crack segmentation mask image;

[0030] A 3D reconstruction module is configured to perform 3D reconstruction based on the key frames of the binocular video, the crack segmentation mask, and the camera pose corresponding to the key frames, thereby obtaining a 3D point cloud model with crack location information; wherein the camera pose is calculated by a binocular vision-based SLAM algorithm on the corresponding key frames;

[0031] The crack measurement module is used to perform crack measurement based on the three-dimensional point cloud model to obtain crack measurement data.

[0032] Optionally, the model training module is specifically used to collect concrete crack pictures; annotate the concrete crack pictures using a semantic segmentation annotation tool to obtain the concrete crack training data set; and use the concrete crack training data set to train the initial semantic segmentation model to obtain the crack semantic segmentation model.

[0033] Optionally, the image detection module is specifically used to perform continuous video shooting through a binocular camera module to obtain the binocular video; sparsely reconstruct the video stream data through a binocular vision SLAM algorithm, and extract key frames to obtain the key frames; and use the crack semantic segmentation model to detect the key frames to obtain the crack segmentation mask map.

[0034] The present application also provides a terminal device, including:

[0035] Memory for storing computer programs;

[0036] A processor is configured to implement the steps of the three-dimensional model measurement method as described above when executing the computer program.

[0037] The present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the three-dimensional model measurement method described above are implemented.

[0038] The present application provides a three-dimensional model measurement method for concrete structure cracks, comprising: training an initial semantic segmentation model based on a labeled concrete crack training data set to obtain a crack semantic segmentation model; using the crack semantic segmentation model to detect key frames of a captured binocular video to obtain a crack segmentation mask map; performing three-dimensional reconstruction based on the key frames of the binocular video, the crack segmentation mask map, and the camera poses corresponding to the key frames to obtain a three-dimensional point cloud model with crack positioning information; wherein the camera pose is calculated for the corresponding key frames using a SLAM algorithm based on binocular vision; and performing crack measurement based on the three-dimensional point cloud model to obtain crack measurement data.

[0039] A crack semantic segmentation model is obtained by training with a concrete crack training dataset. The image to be detected is then detected based on the model to obtain a crack segmentation mask map. Three-dimensional reconstruction is then performed based on the image to be detected, the crack segmentation mask map, and the camera pose corresponding to the image to be detected to obtain a three-dimensional point cloud model. Finally, measurement is performed based on the three-dimensional point cloud model to realize an automated detection and measurement process, improve the accuracy and efficiency of detection and measurement, and reduce detection costs.

[0040] The present application also provides a three-dimensional model measurement device, terminal equipment and computer-readable storage medium for concrete structure cracks, which have the above beneficial effects and are not described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0042] Figure 1 A flowchart of a three-dimensional model measurement method for concrete structure cracks provided in an embodiment of the present application;

[0043] Figure 2A flowchart of another three-dimensional model measurement method for concrete structure cracks provided in an embodiment of the present application;

[0044] Figure 3 A schematic diagram of semantic segmentation of another three-dimensional model measurement method for concrete structure cracks provided in an embodiment of the present application;

[0045] Figure 4 A schematic diagram of the device structure of another three-dimensional model measurement method for concrete structure cracks provided in an embodiment of the present application;

[0046] Figure 5 A schematic diagram of another three-dimensional model measurement method for concrete structure cracks provided in an embodiment of the present application;

[0047] Figure 6 This is a structural schematic diagram of a three-dimensional model measurement device for concrete structure cracks provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The core of this application is to provide a three-dimensional model measurement method, a three-dimensional model measurement device, a terminal device and a computer-readable storage medium for concrete structure cracks to improve the efficiency of crack detection and measurement.

[0049] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0050] In related technologies, taking concrete bridges as an example, the detection of apparent cracks in bridges relies heavily on manual observation. Inspectors on bridge inspection vehicles and ships rely on the naked eye to observe apparent cracks. The detection results are affected by the inspectors' technical experience, resulting in low detection efficiency and extremely dangerous inspections in high-risk areas. Furthermore, the determination and recording of crack morphology and location rely on inspectors writing descriptions in inspection reports supplemented by drawings and photographs. This method is cumbersome and inefficient, lacks good three-dimensional visualization, and makes it difficult for bridge engineers to intuitively understand the crack morphology and location and assess the extent of structural damage. Finally, crack measurement relies heavily on traditional contact measurement methods such as rulers and digital calipers, which are difficult to use in areas that are difficult for personnel to reach. Therefore, traditional manual methods for detecting and measuring apparent cracks in structures are cumbersome and inefficient, and the detection results are highly subjective and dangerous.

[0051] Therefore, the present application provides a three-dimensional model measurement method for cracks in concrete structures, which obtains a crack semantic segmentation model by training a concrete crack training data set, and then detects the image to be detected based on the model to obtain a crack segmentation mask map, and then performs three-dimensional reconstruction based on the image to be detected, the crack segmentation mask map and the camera pose corresponding to the image to be detected to obtain a three-dimensional point cloud model, and finally performs measurement based on the three-dimensional point cloud model to realize an automated detection and measurement process, improve the accuracy and efficiency of detection and measurement, and reduce detection costs.

[0052] The following describes a three-dimensional model measurement method for concrete structure cracks provided by the present application through an embodiment.

[0053] Please refer to Figure 1 , Figure 1 This is a flow chart of a three-dimensional model measurement method for concrete structure cracks provided in an embodiment of the present application.

[0054] In this embodiment, the method may include:

[0055] S101, training an initial semantic segmentation model based on the labeled concrete crack training dataset to obtain a crack semantic segmentation model;

[0056] This step aims to train the initial semantic segmentation model based on the labeled concrete crack training dataset, obtaining a crack semantic segmentation model. This step aims to train a model for semantic segmentation. The concrete crack training dataset is a training dataset obtained by annotating images containing cracks. The annotation method can be any of the existing methods and will not be detailed here.

[0057] A crack semantic segmentation model is constructed based on a convolutional neural network, and then trained using a crack dataset. The trained crack segmentation model can be used to segment pixels in an image that belong to the crack category, allowing for automatic crack identification in images without manual judgment, improving the accuracy of subsequent automated measurements.

[0058] Furthermore, this step may include:

[0059] Step 1, collect pictures of concrete cracks;

[0060] Step 2: annotate the concrete crack images using semantic segmentation and annotation tools to obtain a concrete crack training dataset;

[0061] Step 3: Use the concrete crack training dataset to train the initial semantic segmentation model to obtain a crack semantic segmentation model.

[0062] As can be seen, this option mainly explains how to train a semantic segmentation model. In this option, the main focus is to collect images of concrete cracks; use semantic segmentation annotation tools to annotate the concrete crack images to obtain a concrete crack training dataset; and use the concrete crack training dataset to train the initial semantic segmentation model to obtain a crack semantic segmentation model.

[0063] S102, using a crack semantic segmentation model to detect key frames of the collected binocular video to obtain a crack segmentation mask image;

[0064] Building on S101, this step uses a crack semantic segmentation model to detect key frames in the captured binocular video and generate a crack segmentation mask. Specifically, inference is performed on the key frames of the binocular video captured on-site to generate a crack segmentation mask. This pixel-level crack segmentation mask accurately determines the geometry of the cracks.

[0065] Furthermore, this step may include:

[0066] Step 1: Shoot continuous video using a binocular camera module to obtain binocular video;

[0067] Step 2: sparsely reconstruct the video stream data through the binocular vision SLAM algorithm and extract key frames to obtain key frames;

[0068] Step 3: Use the crack semantic segmentation model to detect the key frames and obtain the crack segmentation mask map.

[0069] It can be seen that this step is intended to illustrate how to obtain the image to be detected. In this optional solution, the binocular camera module is used to shoot continuous video to obtain binocular video; the binocular vision SLAM algorithm is used to sparsely reconstruct the video stream data and extract key frames to obtain key frames; the crack semantic segmentation model is used to detect the key frames to obtain the crack segmentation mask map.

[0070] The SLAM (simultanous localization and mapping) algorithm, also known as simultaneous positioning and mapping, is the process of real-time positioning (determining the camera's pose) while in motion and constructing a 3D point cloud map of the surrounding environment. The binocular SLAM algorithm can determine the camera pose information corresponding to the camera image and build a sparse 3D model of the bridge surface.

[0071] S103, performing three-dimensional reconstruction based on the key frames of the binocular video, the crack segmentation mask image, and the camera pose corresponding to the key frames to obtain a three-dimensional point cloud model with crack location information; wherein the camera pose is calculated by a binocular vision-based SLAM algorithm for the corresponding key frames;

[0072] Building on S102, this step aims to perform 3D reconstruction based on the binocular video keyframes, the crack segmentation mask, and the camera poses corresponding to the keyframes, generating a 3D point cloud model with crack location information. The camera poses are calculated using a binocular vision-based SLAM algorithm for the corresponding keyframes. Specifically, 3D reconstruction is performed on the portion corresponding to the crack to determine a 3D point cloud model with crack location information, enabling accurate crack measurement based on the 3D data.

[0073] Sparse reconstruction refers to the process of reconstructing a point cloud with low density, while dense reconstruction refers to the process of reconstructing a point cloud with high density. For cracks, a dense point cloud is required to accurately measure them.

[0074] Furthermore, this step may include:

[0075] Step 1: Perform dense reconstruction of 3D point cloud based on the key frames of the binocular video and the camera poses corresponding to the key frames to obtain point cloud data;

[0076] Step 2: Perform three-dimensional crack mapping on the point cloud data based on the crack segmentation mask image to obtain a three-dimensional point cloud model with crack location information.

[0077] This alternative primarily explains how to construct a point cloud model with crack location information. This alternative primarily involves densely reconstructing a 3D point cloud based on the keyframes of the binocular video and the corresponding camera poses to obtain point cloud data. This is then used to perform 3D crack mapping on the point cloud data based on the crack segmentation mask to obtain a 3D point cloud model with crack location information.

[0078] S104: Perform crack measurement based on the three-dimensional point cloud model to obtain crack measurement data.

[0079] Based on S103, this step aims to measure the crack based on the 3D point cloud model to obtain crack measurement data. The point cloud model with crack location information represents the crack portion, which can be accurately measured.

[0080] Furthermore, this step may include:

[0081] Step 1: performing point cloud segmentation processing on the three-dimensional point cloud model with crack location information based on point cloud categories to obtain a crack point cloud;

[0082] Step 2: cluster the crack point cloud based on the Euclidean clustering algorithm to obtain multiple crack clusters;

[0083] Step 3: determine the crack direction of each crack cluster based on principal component analysis and calculate the three-dimensional skeleton points of the cracks;

[0084] Step 4: Based on the crack direction and three-dimensional crack skeleton points of each crack cluster, the crack length and crack width are calculated to obtain crack measurement data.

[0085] As can be seen, this optional solution mainly explains how to measure crack measurement data. In this optional solution, the three-dimensional point cloud model with crack location information is segmented based on point cloud classification to obtain a crack point cloud. The crack point cloud is clustered based on the Euclidean clustering algorithm to obtain multiple crack clusters. The crack direction of each crack cluster is determined based on the principal component analysis method, and the three-dimensional crack skeleton points are calculated. Based on the crack direction and three-dimensional crack skeleton points of each crack cluster, the crack length and width are calculated to obtain crack measurement data.

[0086] Among them, the crack cluster is after Euclidean clustering, and the point clouds belonging to the same crack individual are classified into a point cloud cluster, that is, a crack and a crack unit.

[0087] In summary, this embodiment obtains a crack semantic segmentation model by training the concrete crack training data set, and then detects the image to be detected based on the model to obtain a crack segmentation mask map, and then performs three-dimensional reconstruction based on the image to be detected, the crack segmentation mask map and the camera pose corresponding to the image to be detected to obtain a three-dimensional point cloud model. Finally, measurement is performed based on the three-dimensional point cloud model to realize automated detection and measurement process, improve the accuracy and efficiency of detection and measurement, and reduce detection costs.

[0088] The following is a specific example to further illustrate the three-dimensional model measurement method of concrete structure cracks provided by the present application.

[0089] Please refer to Figure 2 , Figure 2 This is a flow chart of another three-dimensional model measurement method for concrete structure cracks provided in an embodiment of the present application.

[0090] In this embodiment, the method may include:

[0091] Step 1: Data collection and dataset creation. Collect and annotate concrete crack images. Use semantic segmentation and annotation tools to perform pixel-level labeling of structural surface defects. Build a concrete crack dataset for training a deep learning crack semantic segmentation model.

[0092] Step 2: Build a deep learning crack semantic segmentation model. Build a deep neural network with the bridge surface defect image K as input and the pixel-level crack segmentation mask K as output. mask . The crack semantic segmentation model is trained using the constructed dataset.

[0093] Please refer to Figure 3 , Figure 3 A schematic diagram of semantic segmentation of another three-dimensional model measurement method for concrete structure cracks provided in an embodiment of the present application.

[0094] Specifically, the output segmentation mask K mask All pixels in the image can be classified into two categories: {0, 1}, where 0 represents a crack pixel and 1 represents a background pixel. By classifying pixels, the geometric shape of the crack can be represented, achieving pixel-level accurate segmentation of concrete cracks in 2D images.

[0095] Step 3: Deploy the binocular camera module. The binocular camera module consists of three parts: binocular camera 01, computing microcomputer 02 and data transmission module 03.

[0096] Please refer to Figure 4 , Figure 4 A schematic diagram of the device structure of another three-dimensional model measurement method for concrete structure cracks provided in an embodiment of the present application.

[0097] like Figure 4 As shown in , binocular camera 01 needs to be calibrated before shooting video. The camera intrinsic and extrinsic parameters are obtained through the calibration method. The camera intrinsic and extrinsic parameters include: the focal length, image center (fx, fy, cx, cy) and distortion coefficients ([k1, k2, p1, p2, k3]) of the left and right cameras, as well as the translation vector T and rotation matrix R of the right camera relative to the left camera.

[0098] Furthermore, the binocular camera module needs to deploy a computing microcomputer 02 with a CPU (or GPU) to deploy a SLAM algorithm to perform real-time data processing on the binocular video input from the binocular camera 01.

[0099] Furthermore, the binocular camera module needs to deploy a data transmission module 03, which can transmit the data obtained from the computer in real time to the cloud server 04 through the 5G or 4G network for further calculation.

[0100] Step 4: On-site data acquisition and real-time processing. The method used is applicable to any device capable of carrying a binocular camera, such as a drone or the robotic arm of a bridge inspection vehicle. A binocular camera 01 continuously captures video of the bridge inspection area. The video stream data is transmitted to a computer 02, which uses a binocular vision-based SLAM algorithm to perform a sparse reconstruction of the bridge surface in real time. Keyframes are extracted from the input binocular video, and the camera pose {R, t} corresponding to each keyframe is calculated, where R is the rotation matrix relative to the first frame of the video, and t is the translation matrix relative to the first frame of the video.

[0101] Among them, the binocular vision-based SLAM algorithm can adopt the open source algorithm binocular ORB-SLAM2 (a SLAM algorithm). This algorithm can extract key frames from the video of the structure surface, calculate the camera's pose, and generate a sparse point cloud of the structure surface, providing source data for subsequent dense reconstruction in the cloud computing server.

[0102] Step 5: Data is transferred to the cloud computing server 04 for subsequent dense reconstruction and crack measurement. The key frames, camera poses, and sparse point cloud data extracted in step 4 are transferred to the cloud computing server 04 via the data transmission module 03. Subsequent steps will be implemented in the cloud computing server 04.

[0103] Step 6: Use the trained crack semantic segmentation model to perform crack detection on the key frame left eye image extracted from the detection video, and obtain a pixel-level crack segmentation mask K at the output end. mask .

[0104] Step 7: Based on the left eye image K of the key frame of the detection video, the crack segmentation mask image K corresponding to K is obtained. mask And the corresponding camera pose, firstly use the dense reconstruction algorithm to reconstruct the 3D point cloud of the detection area, and then use the crack point cloud 3D mapping to obtain the 3D point cloud model of the detection area with real size information and point cloud classification information.

[0105] Specifically, the key frame camera pose is obtained according to step 4, and the PatchMatch algorithm (a stereo matching algorithm) is used to finely calculate the depth map of the key frame. Then, all the depth maps are fused to achieve dense reconstruction to generate a three-dimensional dense point cloud.

[0106] Specifically, the implementation steps of the crack point cloud 3D mapping are as follows:

[0107] (1) For each 3D point p in the generated 3D dense point cloud i (x i ,y i ,z i ) Find the closest key frame K j ;

[0108] (2) According to the camera pose of the corresponding key frame {R j ,t j}Map the three-dimensional point to the key frame to obtain the mapping point. The position {u, v} of the mapping point on the key frame in the two-dimensional image coordinate system can be calculated by the following formula:

[0109]

[0110] Among them, d i is the depth value of the 3D point on the depth map, and K is the intrinsic parameter matrix of the left eye of the binocular camera;

[0111] (3) The category of the 3D point is marked according to the category {0,1} of the pixel at the mapping position. Based on this, the crack segmented in the key frame can be mapped to the 3D point cloud model.

[0112] Finally, a colored three-dimensional point cloud model with real size information and point cloud classification information is obtained, that is, a three-dimensional structural model with crack location information.

[0113] Among them, the final three-dimensional model of the cracked structure P{p i (x,y,z,r,g,b,c) i＝1,2,3,...,n Each point in} has 7 attributes, including three-dimensional position (x, y, z), color (r, g, b) and category c∈{0,1}.

[0114] Step 8: Perform point cloud processing on the three-dimensional point cloud model obtained in step 7 to segment three-dimensional crack individuals.

[0115] In step 8.1, first, according to the category of the point cloud c∈{0,1}, all point clouds belonging to the background of category 1 are removed, and all crack point clouds belonging to category 0 are retained. Based on this, the crack point clouds can be effectively extracted from the three-dimensional model of the structural surface.

[0116] Step 8.2, the European clustering algorithm in PCL (Point Cloud Library) is used to process the extracted crack point cloud, and the cluster search radius (Cluster Tolerance) C is set. t Parameters are used to cluster adjacent point clouds within this cluster search radius into the same crack cluster. Based on this, the entire extracted crack point cloud can be segmented into multiple crack clusters (i.e., individual cracks). Each crack cluster is assigned an index number to facilitate statistical management of each crack unit. By counting the number of crack clusters, the total number of cracks within the bridge inspection area can be obtained.

[0117] In step 8.3, the SOR (Statistical Outlier Removal) filtering algorithm in PCL is used to remove noise points in each crack cluster point cloud to prevent some discrete noise points from interfering with subsequent measurements.

[0118] Step 9: Measure the length and width of the cracks. Measure each individual crack in a single crack cluster.

[0119] Please refer to Figure 5 , Figure 5 A measurement schematic diagram of another three-dimensional model measurement method for concrete structure cracks provided in an embodiment of the present application.

[0120] In step 9.1, the principal components analysis (PCA) algorithm is used to calculate the principal directions of the 3D crack clusters, followed by point cloud coordinate conversion. Specifically, the direction with the largest variance is used as the principal length direction (X) of the crack, the direction with the second largest variance is used as the principal width direction (Y), and the direction with the smallest variance is used as the principal depth direction (Z).

[0121] Specifically, the PCA method calculates the main direction of the three-dimensional crack cluster: for a single crack cluster P{p i (x i ,y i ,z i ) i＝1,2,3,...,n}, calculate the point cloud center Then calculate the covariance matrix: in Then calculate the eigenvalues ​​and eigenvectors of the covariance matrix Cov, and arrange λ1, λ2, and λ3 in descending order according to the size of the eigenvalues. The corresponding eigenvectors t1, t2, and t3 are obtained to obtain the coordinate transformation matrix R = [t1, t2, t3]. Use P' = PR to perform coordinate transformation to obtain the coordinates of the crack cluster after transformation P' {p i (x i ',y i ',z i ') i＝1,2,3,...,n}, where the x' direction is the main length direction of the crack, the y' direction is the main width direction of the crack, and the z' direction is the main depth direction of the crack.

[0122] Step 9.2, calculate the extreme values ​​of all point clouds in the crack cluster after coordinate transformation in the length main direction x', and use the two points corresponding to the extreme values ​​as the two end points of each crack. Set a division step s, and use the step s to divide the crack from the minimum value x in the length main direction. min (corresponding to the minimum point p min ) to the maximum value x max(corresponding to the maximum point p max ) is divided into intervals, and the total number of intervals G satisfies (G-1)s<x max -x min ≤Gs. Calculate the point cloud center in the gth interval where m g is the total number of three-dimensional points in the g-th interval.

[0123] Step 9.3: Use the two end points of the crack and the center of the point cloud calculated in step 9.2 as the 3D crack skeleton point P min (x',y',z'),P1(x',y',z'),P2(x',y',z'),...,P g (x',y',z'),...,P G (x',y',z'),P max (x', y', z') are sorted and connected in order to obtain the crack skeleton.

[0124] Calculate the Euclidean distance between two adjacent skeleton points The sum of the distances between all adjacent skeleton points D = d(P min ,P1)+d(P1,P2)+d(P2,P3)+...d(P G ,P max ), the D value is the desired crack length.

[0125] Step 9.4, project the point cloud after coordinate transformation onto the XY plane. min ,P1,P2,...,P g ,...,P G ,P max As the segmentation point, the crack is divided into G+1 local crack areas, where G+2 is the total number of crack skeleton points.

[0126] The process of calculating the crack width in each local crack region includes: connecting the crack skeleton points at both ends of the local crack region with a two-dimensional straight line to obtain a skeleton line ax+by+c=0, where the values ​​of a, b, and c can be determined by the coordinates of the skeleton points at both ends. Calculate the distance between the point p(x', y', z') in the local crack region and the skeleton line. Calculate the distance from the skeleton line to all points in the local crack area. Then the crack width W in the local crack area is W = |w max |+|w min |, where |w max The maximum distance between the point and the skeleton line, w min | is the minimum distance from the point to the skeleton line. Calculate the maximum crack width of the local area of ​​the G+1 crack segment, that is, the maximum width of the crack cluster Wmax ; Calculate the average crack width of the local area of ​​the G+1 segment crack, that is, obtain the average width W of the individual crack.

[0127] It can be seen that the crack width includes: the average width W of the crack, the maximum width W max .

[0128] Based on the above description, it can be seen that compared with traditional crack measurement methods based on two-dimensional images, this embodiment is not affected by the limited field of view of a single image, thus solving the problems of difficult crack location and difficulty in obtaining global crack quantitative information. This embodiment can effectively obtain the number of individual cracks in the structure as a whole, and obtain crack length and width in units of individual cracks, which is very practical for crack detection and recording. In addition, this embodiment is more accurate than traditional two-dimensional crack measurement, overcoming the limitations of traditional methods such as the inability to accurately measure non-planar cracks and the accuracy affected by the camera shooting angle.

[0129] It can be seen that this embodiment obtains a crack semantic segmentation model through training of the concrete crack training data set, and then detects the image to be detected based on the model to obtain a crack segmentation mask map, and then performs three-dimensional reconstruction based on the image to be detected, the crack segmentation mask map and the camera pose corresponding to the image to be detected to obtain a three-dimensional point cloud model, and finally performs measurement based on the three-dimensional point cloud model to realize automated detection and measurement process, improve the accuracy and efficiency of detection and measurement, and reduce detection costs.

[0130] The following is an introduction to the three-dimensional model measurement device for concrete structure cracks provided in an embodiment of the present application. The three-dimensional model measurement device for concrete structure cracks described below and the three-dimensional model measurement method for concrete structure cracks described above can be referenced to each other.

[0131] Please refer to Figure 6 , Figure 6 This is a structural schematic diagram of a three-dimensional model measurement device for concrete structure cracks provided in an embodiment of the present application.

[0132] In this embodiment, the device may include:

[0133] A model training module 100 is used to train an initial semantic segmentation model based on the labeled concrete crack training dataset to obtain a crack semantic segmentation model;

[0134] The image detection module 200 is used to detect key frames of the collected binocular video using a crack semantic segmentation model to obtain a crack segmentation mask image;

[0135] The 3D reconstruction module 300 is used to perform 3D reconstruction based on the key frames of the binocular video, the crack segmentation mask, and the camera pose corresponding to the key frames, thereby obtaining a 3D point cloud model with crack location information. The camera pose is calculated by the binocular vision-based SLAM algorithm for the corresponding key frames.

[0136] The crack measurement module 400 is used to perform crack measurement based on the three-dimensional point cloud model to obtain crack measurement data.

[0137] Optionally, the model training module 100 is specifically used to collect concrete crack pictures; annotate the concrete crack pictures using a semantic segmentation annotation tool to obtain a concrete crack training data set; and train an initial semantic segmentation model using the concrete crack training data set to obtain a crack semantic segmentation model.

[0138] Optionally, the image detection module 200 is specifically used to perform continuous video shooting through a binocular camera module to obtain binocular video; sparsely reconstruct the video stream data through a binocular vision SLAM algorithm and extract key frames to obtain key frames; use a crack semantic segmentation model to detect the key frames to obtain a crack segmentation mask map.

[0139] Optionally, the three-dimensional reconstruction module 300 is specifically used to perform dense reconstruction of three-dimensional point clouds based on the key frames of the binocular video and the camera poses corresponding to the key frames to obtain point cloud data; and perform three-dimensional mapping of cracks on the point cloud data based on the crack segmentation mask map to obtain a three-dimensional point cloud model with crack positioning information.

[0140] Optionally, the crack measurement module 400 is specifically used to perform point cloud segmentation processing on a three-dimensional point cloud model with crack positioning information based on point cloud categories to obtain a crack point cloud; cluster the crack point cloud based on a Euclidean clustering algorithm to obtain multiple crack clusters; determine the crack direction of each crack cluster based on principal component analysis, and calculate the three-dimensional skeleton points of the crack; calculate the crack length and crack width based on the crack direction and three-dimensional skeleton points of each crack cluster to obtain crack measurement data.

[0141] The present application also provides a terminal device, including:

[0142] Memory for storing computer programs;

[0143] The processor is configured to implement the steps of the three-dimensional model measurement method as described in the above embodiment when executing the computer program.

[0144] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the three-dimensional model measurement method as described in the above embodiment are implemented.

[0145] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0146] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0147] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0148] The above is a detailed introduction to the three-dimensional model measurement method, three-dimensional model measurement device, terminal equipment and computer-readable storage medium for concrete structure cracks provided by this application. This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method of this application and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to this application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

Claims

1. A three-dimensional model measurement method for concrete structure cracks, characterized in that: include: The initial semantic segmentation model is trained based on the labeled concrete crack training dataset to obtain a crack semantic segmentation model. Continuous video shooting is performed through the binocular camera module to obtain binocular video; Sparsely reconstructing the video stream data through a binocular vision SLAM algorithm and extracting key frames to obtain the key frames; Using the crack semantic segmentation model to detect the key frame to obtain a crack segmentation mask map; Performing dense reconstruction of three-dimensional point clouds based on key frames of the binocular video and camera poses corresponding to the key frames to obtain point cloud data; Based on the crack segmentation mask image, the point cloud data is subjected to three-dimensional crack mapping to obtain a three-dimensional point cloud model with crack positioning information; wherein the camera pose is obtained by calculating the corresponding key frame using a SLAM algorithm based on binocular vision; performing point cloud segmentation processing on the three-dimensional point cloud model having crack location information based on point cloud categories to obtain a crack point cloud; Clustering the crack point cloud based on a Euclidean clustering algorithm to obtain multiple crack clusters; Determining the crack direction of each crack cluster based on principal component analysis, and calculating the three-dimensional skeleton points of the cracks; Based on the crack direction and three-dimensional crack skeleton points of each crack cluster, the crack length and crack width are calculated to obtain crack measurement data.

2. The three-dimensional model measurement method according to claim 1, characterized in that: The initial semantic segmentation model is trained based on the labeled concrete crack training dataset to obtain the crack semantic segmentation model, including: Collect pictures of concrete cracks; Annotate the concrete crack image using a semantic segmentation and annotation tool to obtain the concrete crack training dataset; The initial semantic segmentation model is trained using the concrete crack training dataset to obtain the crack semantic segmentation model.

3. A three-dimensional model measuring device for concrete structure cracks, characterized in that: include: The model training module is used to train the initial semantic segmentation model based on the labeled concrete crack training dataset to obtain a crack semantic segmentation model; An image detection module is used to detect key frames of the collected binocular video using the crack semantic segmentation model to obtain a crack segmentation mask image; A 3D reconstruction module is configured to perform 3D reconstruction based on the key frames of the binocular video, the crack segmentation mask, and the camera pose corresponding to the key frames, thereby obtaining a 3D point cloud model with crack location information; wherein the camera pose is calculated by a binocular vision-based SLAM algorithm on the corresponding key frames; a crack measurement module, configured to perform crack measurement based on the three-dimensional point cloud model to obtain crack measurement data; The image detection module is specifically configured to capture continuous video using a binocular camera module to obtain the binocular video; sparsely reconstruct the video stream data using a binocular vision SLAM algorithm and extract key frames to obtain the key frames; and detect the key frames using the crack semantic segmentation model to obtain the crack segmentation mask map. The 3D reconstruction module is specifically configured to perform dense reconstruction of a 3D point cloud based on the key frames of the binocular video and the camera poses corresponding to the key frames to obtain point cloud data; and perform 3D crack mapping on the point cloud data based on the crack segmentation mask map to obtain a 3D point cloud model with crack location information; The crack measurement module is specifically used to perform point cloud segmentation processing on the three-dimensional point cloud model with crack positioning information based on point cloud categories to obtain crack point clouds; cluster the crack point clouds based on the Euclidean clustering algorithm to obtain multiple crack clusters; determine the crack direction of each crack cluster based on the principal component analysis method and calculate the three-dimensional skeleton points of the cracks; calculate the crack length and crack width based on the crack direction and three-dimensional skeleton points of each crack cluster to obtain crack measurement data.

4. The three-dimensional model measuring device according to claim 3, characterized in that: The model training module is specifically used to collect concrete crack pictures; annotate the concrete crack pictures using a semantic segmentation annotation tool to obtain the concrete crack training data set; and use the concrete crack training data set to train the initial semantic segmentation model to obtain the crack semantic segmentation model.

5. A terminal device, characterized in that: include: memory for storing computer programs; A processor is configured to implement the steps of the three-dimensional model measurement method according to claim 1 or 2 when executing the computer program.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the three-dimensional model measurement method according to claim 1 or 2.

Images