Building facade crack detection method combining two-dimensional image and three-dimensional point cloud

Abstract

The invention relates to the technical field of image data analysis, in particular to a building facade crack detection method combining a two-dimensional image and a three-dimensional point cloud. The method comprises the following steps: acquiring three-dimensional point cloud data and a two-dimensional image of a building facade; converting the three-dimensional point cloud data of each frame into a two-dimensional depth image, and respectively extracting crack areas from the two-dimensional depth image and the two-dimensional image; acquiring a crack region matching pair based on the position overlapping condition of the crack region between the two-dimensional depth image and the two-dimensional image; obtaining the overall matching degree of the crack region matching pair according to the coincidence condition of the pixel point coordinates in the crack region matching pair and the similar condition of the main axis direction and the skeleton length; and determining a real crack point cloud based on the overall matching degree, and obtaining the type, width and depth of the crack according to the three-dimensional coordinate and curvature of the real crack point cloud. According to the method, the real crack point cloud is accurately obtained, so that the accuracy and robustness of building facade crack detection are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of image data analysis technology, specifically to a method for detecting cracks in building facades that combines two-dimensional images with three-dimensional point clouds. Background Technology

[0002] The building facade encompasses all external structural elements of a building except the roof, including walls, windows, balconies, and various decorative components (such as tiles, glass curtain walls, and metal decorative panels). It not only provides insulation, heat insulation, and sound insulation, directly impacting the building's lifespan and safety, but also forms an important part of the urban landscape. However, under long-term environmental erosion and load-bearing conditions, building facades are highly susceptible to surface defects such as cracking and peeling. Cracks, being the most common and potentially dangerous form of damage, can, if not detected and repaired promptly, not only affect aesthetics but also lead to water seepage, detachment of the finishing layer, and even serious safety accidents such as falling objects causing injury. Therefore, accurate detection of cracks in building facades has significant engineering and social value.

[0003] Existing methods typically utilize 3D point cloud data from building facades for crack detection. 3D point cloud data records the three-dimensional spatial coordinates of an object's surface, theoretically enabling the measurement of crack depth and volume. However, in reality, building facades often employ highly reflective materials such as ceramic tiles, glass curtain walls, and metal decorative strips, resulting in strong specular reflection or multipath effects on laser beams. This leads to a large number of false point clouds (i.e., noise points) in the data collected by lidar, which are easily misidentified as crack features. Furthermore, high reflectivity causes point cloud data expansion, blurring the edges of actual cracks. Therefore, relying solely on 3D point cloud data is insufficient for accurately detecting cracks in building facades under complex, highly reflective conditions. Summary of the Invention

[0004] To address the technical problem that relying solely on 3D point cloud data makes it difficult to accurately detect cracks in building facades under complex, highly reflective environments, this invention aims to provide a method for detecting cracks in building facades that combines 2D images with 3D point clouds. The specific technical solution adopted is as follows: This invention provides a method for detecting cracks in building facades that combines two-dimensional images with three-dimensional point clouds. The method includes the following steps: Acquire 3D point cloud data and 2D images of each frame of the building facade; The 3D point cloud data of each frame is converted into a 2D depth image, and the crack region is extracted from the 2D depth image and the 2D image respectively. Based on the positional overlap of the crack region between the 2D depth image and the 2D image in each frame, the crack region matching pair of each frame is obtained. Based on the overlap of pixel coordinates in each crack region matching pair, as well as the similarity of the principal axis direction and skeleton length, the overall matching degree of each crack region matching pair is obtained. The true crack point cloud is determined based on the overall matching degree. The crack type, width and depth are obtained for each frame based on the three-dimensional coordinates and curvature of the true crack point cloud.

[0005] Furthermore, the method for obtaining the crack region matching pairs is as follows: For any frame and any crack region in the two-dimensional depth image of that frame, map the crack region to the coordinate system corresponding to the two-dimensional image of that frame, and obtain the intersection-union ratio of the crack region with each crack region in the two-dimensional image of that frame; The crack region in the 2D image of the frame corresponding to the crack region with the largest cross-union ratio among those with a cross-union ratio greater than the preset overlap threshold is constructed as a crack region matching pair.

[0006] Furthermore, the method for obtaining the overall matching degree is as follows: Based on the overlap of pixel coordinates in each crack region matching pair, the positional consistency of each crack region matching pair is obtained; Based on the similarity of the principal axis direction and skeleton length between each crack region matching pair, the degree of geometric topological consistency of each crack region matching pair is obtained; The average of the positional consistency and geometric topological consistency of each crack region matching pair is taken as the overall matching degree of each crack region matching pair.

[0007] Furthermore, the method for obtaining the degree of positional consistency is as follows: For any crack region matching pair, the crack region in the crack region matching pair that belongs to the two-dimensional depth image is taken as the depth crack region, and the crack region in the crack region matching pair that belongs to the two-dimensional image is taken as the reference crack region. All pixels in the deep crack region are taken as target pixels. For any target pixel, the distance between each pixel in the reference crack region and the corresponding coordinate of the target pixel is obtained and taken as the first distance. When there is a first distance less than a preset distance threshold, the target pixel is marked as a specified pixel. The ratio of the total number of specified pixels to the total number of target pixels is used as the degree of positional consistency of the matching pairs in the crack region.

[0008] Furthermore, the method for obtaining the degree of geometric topological consistency is as follows: The vector formed by normalizing the principal axis direction and skeleton length of the deep crack region is used as the first vector. The vector formed by normalizing the principal axis direction and skeleton length of the reference crack region is used as the second vector. The cosine similarity between the first and second vectors is used as the degree of geometric topological consistency of the matching pair in the crack region.

[0009] Furthermore, the method for obtaining the real crack point cloud is as follows: When the overall matching degree is greater than the preset matching degree threshold, the three-dimensional point cloud corresponding to the pixel points in the two-dimensional depth image of the corresponding crack region will be used as the real crack point cloud.

[0010] Furthermore, the method for obtaining the crack type, width, and depth for each frame based on the three-dimensional coordinates and curvature of the real crack point cloud is as follows: The three-dimensional coordinates and curvature of the real crack point cloud in each frame are input into the pre-trained crack recognition neural network model. The crack recognition neural network model outputs the crack type, width, and depth corresponding to the real crack point cloud for each frame.

[0011] Furthermore, the method for obtaining the crack region is as follows: The suspected crack point cloud of each frame is obtained by the building crack extraction method based on laser point cloud. The region corresponding to the suspected crack point cloud in the two-dimensional depth image is taken as the crack region in the two-dimensional depth image. Crack regions in two-dimensional images were obtained using a YOLO-based method for detecting cracks in building facades.

[0012] Furthermore, the three-dimensional point cloud data of each frame is aligned with the corresponding frame of the two-dimensional image with the smallest time difference.

[0013] Furthermore, the building facade is a building envelope structure that does not bear the load of the main structure, except for the roof.

[0014] The present invention has the following beneficial effects: This invention first obtains crack region matching pairs for each frame based on the positional overlap of crack regions between two-dimensional depth images and two-dimensional images. This effectively eliminates isolated noise and false point clouds that only appear in single-modal data, achieving preliminary screening of potential real crack regions and narrowing the scope of subsequent data analysis. To avoid interference from point cloud dilation caused by high-reflectivity materials and to accurately verify the matching relationship, the overall matching degree of each crack region matching pair is obtained based on the overlap of pixel coordinates, the similarity of principal axis direction and skeleton length, and accurately reflects the three-dimensional geometric topology and two-dimensional optical texture. The consistency in spatial distribution and extension trend helps to eliminate the interference of blocky noise with similar shapes; furthermore, determining the real crack point cloud based on the overall matching degree helps to purify the point cloud dataset and ensure that the point clouds participating in the disease analysis have extremely high confidence in their authenticity; furthermore, based on the three-dimensional coordinates and curvature of the real crack point cloud, the crack type, width and depth of each frame can be obtained, which helps to establish an absolute physical scale benchmark for crack diseases. By capturing the small geometric undulations of the surface through curvature features, a comprehensive and accurate qualitative identification and quantitative assessment of crack diseases can be achieved, effectively solving the problems of high false detection rate and difficulty in parameter extraction in building inspection under complex environments. Attached Figure Description

[0015] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a schematic flowchart illustrating a method for detecting cracks in building facades that combines two-dimensional images and three-dimensional point clouds, provided as an embodiment of the present invention. Figure 2 A flowchart illustrating a method for obtaining overall matching degree according to an embodiment of the present invention; Figure 3 This is a structural diagram of a building facade crack detection system that combines two-dimensional images and three-dimensional point clouds, provided in one embodiment of the present invention. Figure 4 This is a schematic diagram of a computer device provided according to an embodiment of the present invention. Detailed Implementation

[0017] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a method for detecting cracks in building facades that combines two-dimensional images and three-dimensional point clouds, based on the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0019] The following describes in detail, with reference to the accompanying drawings, a specific scheme for a method for detecting cracks in building facades that combines two-dimensional images and three-dimensional point clouds, provided by the present invention.

[0020] Example 1: This invention proposes a method for detecting cracks in building facades that combines two-dimensional images with three-dimensional point clouds. Please refer to [link / reference]. Figure 1 The diagram illustrates a schematic flowchart of a method for detecting cracks in building facades that combines two-dimensional images and three-dimensional point clouds, according to an embodiment of the present invention. The method includes the following steps: Step S1: Obtain the 3D point cloud data and 2D image of each frame of the building facade.

[0021] Specifically, the building facade is the building envelope that does not bear the load of the main structure except for the roof, and includes highly reflective objects such as ceramic tiles, glass curtain walls, and metal decorations. To accurately detect whether cracks exist on the building facade, and addressing the problem that highly reflective objects easily generate false point clouds in 3D point clouds that interfere with detection, this embodiment combines the analysis of 3D point cloud data collected by LiDAR and 2D images collected by a high-definition camera to eliminate false point clouds and achieve accurate detection.

[0022] The specific process is as follows: A LiDAR and a high-definition camera are simultaneously mounted on a drone to acquire 3D point cloud data and 2D images of each frame of the building facade, respectively. The relative poses of the LiDAR and the high-definition camera are calibrated, a unified coordinate system is established, and the frame rates of the LiDAR and the high-definition camera are kept consistent. A hard-triggered synchronization or a timestamp-based nearest neighbor matching strategy is used to align each frame of 3D point cloud data with the corresponding frame of the 2D image with the smallest time difference, ensuring that each set of data reflects the building facade state at the same moment. The hard-triggered synchronization and timestamp-based nearest neighbor matching strategies are well-known technologies and will not be elaborated further.

[0023] Furthermore, Gaussian filtering algorithms are used to denoise each frame of acquired 3D point cloud data and 2D image to eliminate noise interference during acquisition and transmission, thereby improving the robustness of subsequent feature analysis. The Gaussian filtering algorithm is a well-known technique and will not be elaborated further.

[0024] Step S2: Convert the 3D point cloud data of each frame into a 2D depth image, and extract the crack region from the 2D depth image and the 2D image respectively; based on the positional overlap of the crack region between the 2D depth image and the 2D image in each frame, obtain the crack region matching pair for each frame.

[0025] Specifically, to unify data dimensions for feature comparison, this embodiment converts the 3D point cloud data of each frame into a 2D depth image. After the spatiotemporal synchronization processing in step S1, the generated 2D depth image has the same resolution as the corresponding 2D image, and pixels at the same coordinate position correspond to the same physical spatial point on the building facade. The method for converting 3D point cloud data into a 2D depth image is well-known and will not be elaborated further. Considering that real cracks objectively exist in physical space, real cracks should simultaneously exhibit crack characteristics in both the 2D image (optical reflection) and the 3D point cloud (geometric concavity), i.e., have a high degree of consistency in spatial location (i.e., high overlap). Noise caused by highly reflective objects (such as glass and metal) only exists in the 3D point cloud data, and typically appears as a smooth surface at the corresponding position in the 2D image, thus lacking spatial consistency. Therefore, this embodiment first extracts the crack region from the two-dimensional depth image and the two-dimensional image respectively. Then, based on the positional overlap of the crack region between the two-dimensional depth image and the two-dimensional image in each frame, it obtains the crack region matching pair for each frame, preliminarily screens out the potential correspondence of real cracks, lays the foundation for subsequent refined verification based on geometric topological features, and is conducive to accurately obtaining the point cloud corresponding to the real cracks.

[0026] Preferably, in one feasible embodiment, the crack region is obtained as follows: a suspected crack point cloud is obtained for each frame using a laser point cloud-based building crack extraction method; the region corresponding to the suspected crack point cloud in the two-dimensional depth image is taken as the crack region in the two-dimensional depth image; and the crack region in the two-dimensional image is obtained using a YOLO-based building facade crack detection method. Both the laser point cloud-based building crack extraction method and the YOLO-based building facade crack detection method are well-known technologies and will not be described in detail here.

[0027] Preferably, in one feasible embodiment of this invention, the method for obtaining crack region matching pairs is as follows: For any frame and any crack region in the two-dimensional depth image of that frame, the crack region is mapped to the coordinate system corresponding to the two-dimensional image of that frame, and the intersection-union ratio (IU / R) of the crack region with each crack region in the two-dimensional image of that frame is obtained; the larger the IU / R, the more likely the corresponding crack region in the two-dimensional image of that frame is a matching region for the crack region; furthermore, this embodiment sets a preset overlap threshold of 0.6, and the implementer can set the size of the preset overlap threshold according to the actual situation, which is not limited here; then, the crack region in the two-dimensional image of that frame corresponding to the largest IU / R among the IU / R values ​​greater than the preset overlap threshold is constructed as a crack region matching pair with the crack region, ensuring that the constructed crack region matching pair has a very high spatial correspondence confidence, providing high-quality input data for subsequent consistency verification. The method for obtaining the IU / R is a known technique and will not be described in detail here.

[0028] At this point, the crack region matching pairs for each frame are obtained.

[0029] Step S3: Based on the overlap of pixel coordinates in each crack region matching pair, as well as the similarity of the main axis direction and skeleton length, obtain the overall matching degree of each crack region matching pair.

[0030] Specifically, the more overlapping pixels a crack region matching pair has, the more spatially similar the crack region in the 2D depth image is to the crack region in the 2D image, and the higher the probability that they correspond to the same real crack. Therefore, this embodiment first analyzes the correspondence of the crack region matching pair in macroscopic spatial location based on the overlap of pixel coordinates. Considering that relying solely on positional overlap may be affected by point cloud dilation or edge noise, since point cloud dilation caused by highly reflective objects (such as glass and metal) will significantly change the width and area of ​​the crack, but has little effect on the main axis direction (extension trend) and skeleton length (topology) of the crack, these two features have strong anti-interference and stability. To more accurately eliminate false matches that overlap in location but differ significantly in morphology (e.g., incorrect matches between a blocky point cloud formed by noise aggregation and a thin, real crack), this embodiment further analyzes the similarity of each crack region matching pair in the principal axis direction and skeleton length. The more similar a crack region matching pair is in the principal axis direction and skeleton length, the more consistent its geometric morphological features are across both modalities. Therefore, this embodiment obtains the overall matching degree of each crack region matching pair based on the overlap of pixel coordinates and the similarity in the principal axis direction and skeleton length. A higher overall matching degree indicates a high degree of coordination between the 3D point cloud data and 2D image features corresponding to the crack region matching pair in both spatial location and topological morphology, making the 3D point cloud corresponding to the crack region matching pair more likely to be a real crack point cloud.

[0031] Preferably, in one feasible implementation of this embodiment, the method for obtaining the overall matching degree is described in [reference needed]. Figure 2 The document presents a flowchart of a method for obtaining the overall matching degree provided in this embodiment, which includes the following steps: Step S201: Based on the overlap of pixel coordinates in each crack region matching pair, obtain the positional consistency of each crack region matching pair.

[0032] The higher the positional consistency value, the higher the overlap ratio between the deep crack region and the reference crack region in the spatial pixel distribution of the corresponding crack region matching pair, and the stronger the macroscopic positional consistency.

[0033] In one possible implementation of this embodiment, the method for obtaining the degree of positional consistency is as follows: For any crack region matching pair, the crack region belonging to the two-dimensional depth image in the crack region matching pair is taken as the depth crack region, and the crack region belonging to the two-dimensional image in the crack region matching pair is taken as the reference crack region; all pixels in the depth crack region are taken as target pixels; for any target pixel, the Euclidean distance between each pixel in the reference crack region and the corresponding coordinates of the target pixel is obtained, and all are taken as the first distance; considering the image calibration error and the small positional shift that may be caused by discretization sampling, directly requiring the pixel coordinates to be completely equal is too strict, so this embodiment sets the preset distance threshold to 3 pixel units to allow for a certain matching error. The implementer can set the size of the preset distance threshold according to the actual situation, which is not limited here; when there is a first distance less than the preset distance threshold, it means that the target pixel has a corresponding matching point in the reference region, and at this time the target pixel is marked as a specified pixel; in order to quantify this overlap ratio, the ratio of the total number of specified pixels to the total number of target pixels is taken as the degree of positional consistency of the crack region matching pair.

[0034] At this point, the positional consistency of each crack region matching pair is obtained.

[0035] Step S202: Based on the similarity of the principal axis direction and skeleton length between each crack region matching pair, obtain the degree of geometric topological consistency of each crack region matching pair.

[0036] The greater the degree of geometric topological consistency, the more similar the extension direction and topological structure of the corresponding crack region matching pair are under the two modal data. The lower the possibility of deformation due to noise interference such as point cloud expansion, the higher the authenticity.

[0037] In one possible implementation of this embodiment, the method for obtaining the degree of geometric topological consistency is as follows: Taking the crack region matching pair in step S201 as an example, the vector formed by linearly normalizing the principal axis direction and skeleton length of the deep crack region in the crack region matching pair is used as the first vector; similarly, the vector formed by linearly normalizing the principal axis direction and skeleton length of the reference crack region in the crack region matching pair is used as the second vector; the cosine similarity between the first vector and the second vector is obtained as the degree of geometric topological consistency of the crack region matching pair. The linear normalization method and cosine similarity are both well-known techniques and will not be elaborated further.

[0038] At this point, the degree of geometric topological consistency of each crack region matching pair is obtained.

[0039] Step S203: Take the average of the positional consistency and geometric topological consistency of each crack region matching pair as the overall matching degree of each crack region matching pair.

[0040] The degree of positional consistency reflects the confidence level of macroscopic spatial overlap, while the degree of geometrical-topological consistency reflects the confidence level of microscopic morphological similarity. To comprehensively evaluate the reliability of matching pairs, this embodiment uses the average of the positional consistency and geometrical-topological consistency of each crack region matching pair as the overall matching degree of each crack region matching pair, thus achieving multi-dimensional confidence fusion.

[0041] At this point, the overall matching degree of each crack region matching pair is obtained.

[0042] Step S4: Determine the real crack point cloud based on the overall matching degree. Based on the three-dimensional coordinates and curvature of the real crack point cloud, obtain the crack type, width and depth for each frame.

[0043] Specifically, a higher overall matching degree indicates a higher consistency between the matching pairs of the crack region in terms of macroscopic spatial distribution and microscopic geometric topology. This increases the credibility of the crack originating from a real physical crack on the building facade (rather than spurious noise). Therefore, this embodiment determines the real crack point cloud based on the overall matching degree. The acquired real crack point cloud not only confirms the existence of cracks, but its three-dimensional spatial information also supports refined measurement of the cracks. This embodiment further fully utilizes the geometric properties of these point clouds, obtaining the crack type, width, and depth for each frame based on the three-dimensional coordinates and curvature of the real crack point cloud. The three-dimensional coordinates directly reflect the spatial scale and depth changes of the crack, while the curvature features help distinguish the surface texture features of different types of cracks, thereby achieving comprehensive qualitative and quantitative analysis of crack defects.

[0044] Preferably, in one feasible method of this embodiment, the method for obtaining the real crack point cloud is as follows: In this embodiment, a preset matching degree threshold is set to 0.6. The implementer can set the size of the preset matching degree threshold according to the actual situation to ensure that the selected real crack point cloud has high reliability and accuracy, and provides a high-quality data foundation for subsequent refined detection; When the overall matching degree is greater than the preset matching degree threshold, the three-dimensional point cloud corresponding to the pixel points in the crack area of ​​the two-dimensional depth image of the corresponding crack area is taken as the real crack point cloud.

[0045] At this point, the true crack point cloud for each frame is obtained.

[0046] Preferably, in one feasible method of this embodiment, the method for obtaining the crack type, width, and depth of each frame based on the three-dimensional coordinates and curvature of the real crack point cloud is as follows: The three-dimensional coordinates of the real crack point cloud are known to be the absolute physical reference for calculating the crack width and depth. Without three-dimensional coordinates, the model cannot perceive the true physical size of the crack. The curvature of the real crack point cloud reflects the undulations of the point cloud surface. Cracks on walls typically exhibit concave geometric features, and the curvature of their edges and bottoms differs significantly from that of a flat wall surface. Introducing curvature can greatly enhance the model's ability to identify crack types (such as alligator cracks and structural cracks) and to accurately depict crack depth. Therefore, in this embodiment, the three-dimensional coordinates and curvature of the real crack point cloud for each frame are input into a pre-trained crack recognition neural network model. The crack recognition neural network model outputs the crack type, width, and depth corresponding to the real crack point cloud for each frame. The crack recognition neural network model is a well-known technology and will not be described in detail here.

[0047] In summary, this embodiment acquires 3D point cloud data and 2D images of the building facade; converts each frame of 3D point cloud data into a 2D depth image, and extracts crack regions from both the 2D depth image and the 2D image; based on the positional overlap of crack regions between the 2D depth image and the 2D image, obtains crack region matching pairs; based on the overlap of pixel coordinates in the crack region matching pairs, as well as the similarity of principal axis direction and skeleton length, obtains the overall matching degree of the crack region matching pairs; based on the overall matching degree, determines the true crack point cloud, and based on the 3D coordinates and curvature of the true crack point cloud, obtains the crack type, width, and depth. This invention effectively improves the accuracy and robustness of crack detection on building facades by accurately acquiring true crack point clouds.

[0048] Example 2: This invention also proposes a building facade crack detection system that combines two-dimensional images and three-dimensional point clouds. Please refer to [link / reference]. Figure 3 The diagram illustrates a structural diagram of a building facade crack detection system that combines two-dimensional images and three-dimensional point clouds according to an embodiment of the present invention. The system includes: a data acquisition module 10, a crack area matching pair acquisition module 20, an overall matching degree acquisition module 30, and a data processing module 40.

[0049] The data acquisition module 10 is used to acquire the three-dimensional point cloud data and two-dimensional image of each frame of the building facade.

[0050] The crack region matching pair acquisition module 20 is used to convert the three-dimensional point cloud data of each frame into a two-dimensional depth image, extract the crack region from the two-dimensional depth image and the two-dimensional image respectively; and obtain the crack region matching pair for each frame based on the positional overlap of the crack region between the two-dimensional depth image and the two-dimensional image in each frame.

[0051] The overall matching degree acquisition module 30 is used to acquire the overall matching degree of each crack region matching pair based on the overlap of pixel coordinates in each crack region matching pair, as well as the similarity of the main axis direction and skeleton length.

[0052] The data processing module 40 is used to determine the real crack point cloud based on the overall matching degree, and to obtain the crack type, width and depth in each frame according to the three-dimensional coordinates and curvature of the real crack point cloud.

[0053] It should be noted that the system provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the building facade crack detection system combining two-dimensional images and three-dimensional point clouds and the building facade crack detection method combining two-dimensional images and three-dimensional point clouds provided in the above embodiments belong to the same concept. The specific implementation process is detailed in the method embodiments and will not be repeated here.

[0054] Example 3: This invention also proposes a device for detecting cracks in building facades that combines two-dimensional images and three-dimensional point clouds. The device includes a memory and a processor. The memory stores executable program code, and the processor calls and executes this executable program code to perform the method for detecting cracks in building facades that combines two-dimensional images and three-dimensional point clouds provided in the embodiments of this application. Specifically, the device may be a chip, component, or module. The chip may include a connected processor and memory; the memory stores instructions, and when the processor calls and executes the instructions, the chip can perform the method for detecting cracks in building facades that combines two-dimensional images and three-dimensional point clouds provided in the above embodiments.

[0055] Furthermore, this application also protects a computer device; please refer to [link to relevant documentation]. Figure 4 The computer device includes a memory 401, a processor 402, and a computer program 403 stored in the memory 401 and running on the processor 402. When the processor 402 executes the computer program 403, the computer device can execute any of the aforementioned methods for detecting cracks in building facades that combine two-dimensional images and three-dimensional point clouds.

[0056] Example 4: This embodiment also provides a computer-readable storage medium storing computer program code. When the computer program code is run on a computer, the computer executes the above-described method steps to implement the building facade crack detection method combining two-dimensional images and three-dimensional point clouds provided in the above embodiment.

[0057] Example 5: This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the building facade crack detection method that combines two-dimensional images and three-dimensional point clouds provided in the above embodiment.

[0058] In this embodiment, the device, computer-readable storage medium, computer program product, or chip are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0059] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0060] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for detecting cracks in building facades that combines two-dimensional images and three-dimensional point clouds, characterized in that, The method includes the following steps: Acquire 3D point cloud data and 2D images of each frame of the building facade; The 3D point cloud data of each frame is converted into a 2D depth image, and the crack region is extracted from the 2D depth image and the 2D image respectively. Based on the positional overlap of the crack region between the 2D depth image and the 2D image in each frame, the crack region matching pair of each frame is obtained. Based on the overlap of pixel coordinates in each crack region matching pair, as well as the similarity of the principal axis direction and skeleton length, the overall matching degree of each crack region matching pair is obtained. The true crack point cloud is determined based on the overall matching degree. The crack type, width and depth are obtained for each frame based on the three-dimensional coordinates and curvature of the true crack point cloud.

2. The method for detecting cracks in building facades combining two-dimensional images and three-dimensional point clouds as described in claim 1, characterized in that, The method for obtaining the crack region matching pairs is as follows: For any frame and any crack region in the two-dimensional depth image of that frame, map the crack region to the coordinate system corresponding to the two-dimensional image of that frame, and obtain the intersection-union ratio of the crack region with each crack region in the two-dimensional image of that frame; The crack region in the 2D image of the frame corresponding to the crack region with the largest cross-union ratio among those with a cross-union ratio greater than the preset overlap threshold is constructed as a crack region matching pair.

3. The method for detecting cracks in building facades combining two-dimensional images and three-dimensional point clouds as described in claim 1, characterized in that, The method for obtaining the overall matching degree is as follows: Based on the overlap of pixel coordinates in each crack region matching pair, the positional consistency of each crack region matching pair is obtained; Based on the similarity of the principal axis direction and skeleton length between each crack region matching pair, the degree of geometric topological consistency of each crack region matching pair is obtained; The average of the positional consistency and geometric topological consistency of each crack region matching pair is taken as the overall matching degree of each crack region matching pair.

4. The method for detecting cracks in building facades combining two-dimensional images and three-dimensional point clouds as described in claim 3, characterized in that, The method for obtaining the degree of positional consistency is as follows: For any crack region matching pair, the crack region in the crack region matching pair that belongs to the two-dimensional depth image is taken as the depth crack region, and the crack region in the crack region matching pair that belongs to the two-dimensional image is taken as the reference crack region. All pixels in the deep crack region are taken as target pixels. For any target pixel, the distance between each pixel in the reference crack region and the corresponding coordinate of the target pixel is obtained and taken as the first distance. When there is a first distance less than a preset distance threshold, the target pixel is marked as a specified pixel. The ratio of the total number of specified pixels to the total number of target pixels is used as the degree of positional consistency of the matching pairs in the crack region.

5. The method for detecting cracks in building facades combining two-dimensional images and three-dimensional point clouds as described in claim 4, characterized in that, The method for obtaining the degree of geometric topological consistency is as follows: The vector formed by normalizing the principal axis direction and skeleton length of the deep crack region is used as the first vector. The vector formed by normalizing the principal axis direction and skeleton length of the reference crack region is used as the second vector. The cosine similarity between the first and second vectors is used as the degree of geometric topological consistency of the matching pair in the crack region.

6. The method for detecting cracks in building facades combining two-dimensional images and three-dimensional point clouds as described in claim 1, characterized in that, The method for obtaining the real crack point cloud is as follows: When the overall matching degree is greater than the preset matching degree threshold, the three-dimensional point cloud corresponding to the pixel points in the two-dimensional depth image of the corresponding crack region will be used as the real crack point cloud.

7. The method for detecting cracks in building facades combining two-dimensional images and three-dimensional point clouds as described in claim 1, characterized in that, The method for obtaining the crack type, width, and depth for each frame based on the three-dimensional coordinates and curvature of the actual crack point cloud is as follows: The three-dimensional coordinates and curvature of the real crack point cloud in each frame are input into the pre-trained crack recognition neural network model. The crack recognition neural network model outputs the crack type, width, and depth corresponding to the real crack point cloud for each frame.

8. The method for detecting cracks in building facades combining two-dimensional images and three-dimensional point clouds as described in claim 1, characterized in that, The method for obtaining the crack region is as follows: The suspected crack point cloud of each frame is obtained by the building crack extraction method based on laser point cloud. The region corresponding to the suspected crack point cloud in the two-dimensional depth image is taken as the crack region in the two-dimensional depth image. Crack regions in two-dimensional images were obtained using a YOLO-based method for detecting cracks in building facades.

9. The method for detecting cracks in building facades combining two-dimensional images and three-dimensional point clouds as described in claim 1, characterized in that, The 3D point cloud data of each frame is aligned with the corresponding frame of the 2D image with the smallest time difference.

10. The method for detecting cracks in building facades combining two-dimensional images and three-dimensional point clouds as described in claim 1, characterized in that, The building facade is the building envelope that does not bear the load of the main structure, except for the roof.

Images