Method and device for detecting hollowing of external wall based on fusion of thermal infrared and facade point cloud, and storage medium

Abstract

The application discloses a kind of based on thermal infrared and facade point cloud fusion outer wall hollow detection method, equipment and storage medium, it is related to building structure safety detection and intelligent perception technical field, discloses a kind of based on thermal infrared and facade point cloud fusion outer wall hollow detection method, comprising: obtaining thermal infrared camera is collected to the thermal infrared image of the outer wall to be detected by the thermal infrared camera, and the facade laser point cloud that laser radar is collected to the outer wall to be detected;Determine the temperature anomaly region in thermal infrared image and the corresponding outer wall point cloud subset of the temperature anomaly region in facade laser point cloud;Outer wall point cloud subset data is fitted to reference surface, and the outer wall normal deformation model reflecting wall surface concave-convex feature is constructed based on fitting result, and local deformation anomaly region is extracted from outer wall normal deformation model;The detection result of temperature anomaly region and local deformation anomaly region is fused, and target outer wall hollow detection result is judged.The technical effect of improving outer wall hollow detection accuracy is realized.

Description

Technical Field

[0001] This application relates to the field of building structure safety inspection and intelligent sensing technology, and in particular to a method, equipment and storage medium for detecting hollow exterior walls based on the fusion of thermal infrared and facade point cloud. Background Technology

[0002] Existing technologies for detecting hollow areas in exterior walls typically rely on thermal infrared imaging. These technologies utilize the temperature anomalies created by the difference in heat transfer characteristics between hollow and normal areas, employing threshold segmentation or deep learning for identification. However, infrared images contain various false anomalies caused by uneven sunlight, interference from hot and cold sources, reflection interference, and structural differences. These interference factors can create temperature anomalies similar to hollow areas, easily leading to misidentification and increased false alarm rates, thus reducing detection accuracy. Summary of the Invention

[0003] The main purpose of this application is to provide a method, equipment and storage medium for detecting hollow exterior walls based on the fusion of thermal infrared and facade point cloud, aiming to solve the technical problem of inaccurate detection results of hollow exterior walls.

[0004] To achieve the above objectives, this application proposes a method for detecting hollow areas in exterior walls based on the fusion of thermal infrared and facade point cloud data. The method includes:

[0005] Acquire thermal infrared images of the exterior wall to be inspected by a thermal infrared camera, and facade laser point clouds of the exterior wall to be inspected by a lidar.

[0006] Identify temperature anomaly regions in thermal infrared images, and based on the spatial mapping relationship between thermal infrared images and facade laser point clouds, determine the corresponding exterior wall point cloud subsets in facade laser point clouds for temperature anomaly regions in thermal infrared images.

[0007] Reference surface fitting is performed on the subset data of the external wall point cloud. Based on the fitting results, an external wall normal deformation model reflecting the concave and convex characteristics of the wall surface is constructed, and local deformation anomaly areas are extracted from the external wall normal deformation model.

[0008] By integrating the detection results of areas with abnormal temperature and areas with abnormal local deformation, the detection results of hollow areas in the target exterior wall are determined.

[0009] In one embodiment, the step of fitting a reference surface to a subset of external wall point cloud data and constructing an external wall normal deformation model reflecting the concave-convex characteristics of the wall surface based on the fitting result includes:

[0010] The local external wall reference surface is obtained by fitting a subset of the external wall point cloud to a reference surface. The local external wall reference surface is either a planar model or a curved surface model.

[0011] A projection reference plane is constructed based on the geometric features of the local external wall reference plane, and the normal distance residual from each data point in the external wall point cloud subset to the local external wall reference plane is calculated.

[0012] The projection reference plane is meshed to divide it into multiple regular mesh units;

[0013] Based on the projection reference plane and the normal distance residuals in all grid cells, a normal deformation model of the exterior wall distributed along the facade normal is constructed.

[0014] In one embodiment, the step of constructing an exterior wall normal deformation model distributed along the facade normal, based on the projection reference plane and the normal distance residuals within all grid cells, includes:

[0015] Determine the two-dimensional coordinate indices of all regular mesh elements on the projection reference plane;

[0016] The normal distance residuals of the point cloud data mapped to all regular grid cells are statistically analyzed, and the statistical values ​​of the normal distance residuals are calculated. The statistical values ​​are then determined as the normal deformation values ​​of the grid cells, where the statistical values ​​include the mean or median.

[0017] Using two-dimensional coordinate indices as position parameters and the normal deformation values ​​corresponding to all grid cells as deformation parameters, a normal deformation model of the external wall is generated.

[0018] In one embodiment, the step of determining the detection result of hollowness in the target exterior wall by integrating the detection results of the temperature anomaly area and the local deformation anomaly area includes:

[0019] The first detection result identifies the temperature anomaly area, and the second detection result identifies the local deformation anomaly area;

[0020] By combining the first and second test results, the hollowness test results of the target exterior wall are obtained.

[0021] In one embodiment, the steps of determining a first detection result for a temperature anomaly region and a second detection result for a local deformation anomaly region include:

[0022] The contour of the temperature anomaly region in the infrared image is mapped to the local coordinate system of the local deformation anomaly region to obtain the sub-region corresponding to the temperature anomaly region on the normal deformation model of the outer wall.

[0023] Extract thermal features of the temperature anomaly region and determine the first detection result of the temperature anomaly region based on the thermal features. The thermal features include at least the average temperature, the maximum temperature gradient in the region and the temperature difference relative to the background.

[0024] The second detection result is to extract the geometric features of the sub-region and determine the local deformation anomaly region based on the geometric features. The geometric features include at least the average deformation, maximum deformation, maximum deformation gradient, deformation region area, and deformation region contour.

[0025] In one embodiment, the step of determining the temperature anomaly region in the thermal infrared image and, based on the spatial mapping relationship between the thermal infrared image and the facade laser point cloud, determining the corresponding exterior wall point cloud subset in the facade laser point cloud includes:

[0026] Identify the temperature anomaly region in the thermal infrared image and obtain the minimum bounding rectangle of the temperature anomaly region in the thermal infrared image.

[0027] Using the minimum bounding rectangle as a reference, the area within the thermal infrared image is expanded outward to obtain the temperature anomaly expansion area;

[0028] Obtain the intrinsic parameter matrix of the thermal infrared camera and the extrinsic parameter pose during imaging;

[0029] Based on the intrinsic parameter matrix, the pixel coordinates of the contour boundary of the temperature anomaly extension region are converted into imaging ray direction vectors in the camera coordinate system.

[0030] Based on the extrinsic pose, the imaging ray direction vector is converted into a line-of-sight projection ray in the world coordinate system;

[0031] In the laser point cloud of the facade, the outer point cloud data located within the spatial cone formed by the multiple line-of-sight projection rays is retrieved, and the retrieved point cloud data is determined as the subset of the outer wall point cloud.

[0032] In one embodiment, the step of determining temperature anomaly regions in a thermal infrared image includes:

[0033] Radiometric calibration and environmental correction processing are performed on thermal infrared images;

[0034] Based on a preset image segmentation algorithm or deep learning model, extract temperature anomaly regions from thermal infrared images.

[0035] In one embodiment, after determining the target exterior wall hollowness detection result by fusing the detection results of temperature anomaly areas and local deformation anomaly areas, the exterior wall hollowness detection method based on the fusion of thermal infrared and facade point cloud further includes:

[0036] If the target exterior wall hollowness detection result shows that hollowness exists, the target area where hollowness exists is identified;

[0037] Associate the target area with the facade laser point cloud model, oblique photogrammetry model, or the building BIM model of the exterior wall to be inspected;

[0038] Obtain the coordinates, area, range, and risk level of the target area, and generate an external wall hollowing detection report based on the coordinates, area, range, and risk level of the target area.

[0039] In response to the report export command, output the visual export results corresponding to the detection results of hollow areas in the target exterior wall.

[0040] In addition, to achieve the above objectives, this application also proposes an external wall hollow detection device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. The computer program is configured to implement the steps of the external wall hollow detection method based on thermal infrared and facade point cloud fusion as described above.

[0041] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, storing a computer program on the storage medium. When the computer program is executed by a processor, it implements the steps of the external wall hollow detection method based on the fusion of thermal infrared and facade point cloud as described above.

[0042] This application provides a method for detecting hollow exterior walls based on the fusion of thermal infrared and facade point cloud data. It acquires thermal infrared images of the exterior wall to be detected using a thermal infrared camera, and facade laser point clouds collected by a lidar system. Temperature anomaly regions in the thermal infrared images are identified. Based on the spatial mapping relationship between the thermal infrared images and the facade laser point clouds, a subset of the exterior wall point cloud corresponding to the temperature anomaly regions in the thermal infrared images is determined. A reference surface is fitted to the subset of exterior wall point cloud data. Based on the fitting results, a normal deformation model of the exterior wall reflecting the wall's unevenness is constructed, and local deformation anomaly regions are extracted from the normal deformation model. The detection results of the temperature anomaly regions and the local deformation anomaly regions are fused to determine the detection result of the target exterior wall hollowness. By collaboratively acquiring thermal infrared images and facade laser point clouds, constructing a normal deformation model of the exterior wall to measure local deformation, and jointly determining the temperature anomaly regions and local deformation regions, interference from single-dimensional anomalies is eliminated, which has the technical effect of improving the accuracy of exterior wall hollowness detection. Attached Figure Description

[0043] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0044] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 A schematic diagram of the external wall hollowing detection method provided in this application, based on the fusion of thermal infrared and facade point cloud.

[0046] Figure 2 This is a flowchart illustrating an embodiment of the exterior wall hollow detection method based on the fusion of thermal infrared and facade point cloud in this application.

[0047] Figure 3 Infrared images and normal deformation model diagrams of the exterior wall provided for the exterior wall hollow detection method based on the fusion of thermal infrared and facade point cloud in this application;

[0048] Figure 4 This is a schematic diagram of the abnormal area in the infrared image and the normal image of the exterior wall provided by the exterior wall hollow detection method based on the fusion of thermal infrared and facade point cloud in this application.

[0049] Figure 5 This is a schematic diagram of the fusion result provided by the exterior wall hollow detection method based on the fusion of thermal infrared and facade point cloud in this application;

[0050] Figure 6 This is a flowchart illustrating Embodiment 8 of the method for detecting hollow exterior walls based on the fusion of thermal infrared and facade point cloud in this application.

[0051] Figure 7 This is a schematic diagram of the facade laser point cloud model and data acquisition area provided in Embodiment 8 of the exterior wall hollow detection method based on thermal infrared and facade point cloud fusion in this application.

[0052] Figure 8 This is a schematic diagram of the equipment structure of the hardware operating environment involved in the external wall hollow detection method based on the fusion of thermal infrared and facade point cloud in the embodiments of this application.

[0053] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0054] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0055] In this embodiment, for ease of description, the external wall hollow detection device will be used as the subject of the description.

[0056] Current technologies typically rely on thermal infrared image recognition, utilizing the temperature anomalies created by the difference in heat transfer characteristics between hollow and normal areas. Identification is achieved through threshold segmentation or deep learning. However, infrared images are susceptible to various false anomalies, such as uneven sunlight, interference from hot and cold sources, reflection interference, and structural differences. These factors can create temperature anomalies similar to hollow areas, easily leading to misinterpretations and a higher false alarm rate. Furthermore, infrared images are usually two-dimensional projections, lacking the perception of the true three-dimensional geometry of the exterior wall, making it impossible to distinguish between genuine bulges and visual texture changes.

[0057] For example, please refer to Figure 1 , Figure 1 This diagram illustrates the external wall hollowing detection method based on thermal infrared and facade point cloud fusion provided in this application. External wall hollowing is a common quality defect in building exterior walls, specifically referring to gaps and separation between the exterior wall finish layer (e.g., tiles, plaster) and the base wall (e.g., concrete, masonry) due to weak adhesion. Under normal conditions, the finish layer and the base layer should be tightly bonded and evenly stressed. In a hollow state, an air gap forms between them, producing a crisp hollow sound when tapped, unlike the dull sound of a solid area. External wall hollowing reduces the integrity and durability of the exterior wall. Long-term exposure to temperature changes and rain erosion can easily lead to cracking and detachment of the hollow areas. Furthermore, the detached finish material poses a risk of falling objects from heights and can damage the waterproofing performance of the exterior wall.

[0058] This application provides a solution that acquires thermal infrared images of the exterior wall to be inspected using a thermal infrared camera, and facade laser point clouds acquired by a lidar system. Temperature anomaly regions in the thermal infrared images are identified. Based on the spatial mapping relationship between the thermal infrared images and the facade laser point clouds, a subset of the exterior wall point cloud corresponding to the temperature anomaly regions in the thermal infrared images is determined. Reference surface fitting is performed on the subset of exterior wall point cloud data. Based on the fitting results, a normal deformation model of the exterior wall reflecting the wall's unevenness is constructed, and local deformation anomaly regions are extracted from the normal deformation model. The detection results of the temperature anomaly regions and the local deformation anomaly regions are fused to determine the detection result of the target exterior wall hollowness. By collaboratively acquiring thermal infrared images and facade laser point clouds, constructing a normal deformation model of the exterior wall to measure local deformation, and jointly determining the temperature anomaly regions and local deformation regions, interference from single-dimensional anomalies is eliminated, which has the technical effect of improving the accuracy of exterior wall hollowness detection.

[0059] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the above functions, such as an external wall hollow detection device. Preferably, the external wall hollow detection device is a drone, which is equipped with modules such as a visible light camera, a thermal infrared camera, and a lidar. The following description uses a drone as an example to illustrate this embodiment and the subsequent embodiments.

[0060] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0061] Based on this, this application provides a method for detecting hollow areas in exterior walls based on the fusion of thermal infrared and facade point cloud data. Please refer to... Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the exterior wall hollow detection method based on the fusion of thermal infrared and facade point cloud in this application.

[0062] In this embodiment, the method for detecting hollow exterior walls based on the fusion of thermal infrared and facade point cloud includes steps S10~S40:

[0063] Step S10: Acquire the thermal infrared image of the exterior wall to be inspected by the thermal infrared camera, and the facade laser point cloud acquired by the lidar for the exterior wall to be inspected.

[0064] Thermal infrared images are images captured and output by thermal infrared cameras. Their pixel values ​​correspond to the temperature values ​​of the measured object's surface, providing a direct view of the temperature distribution on the object's surface. Different temperature regions will exhibit different colors or grayscale differences in the image, which can be used to identify areas with abnormal temperatures. LiDAR is a sensor that emits a laser beam, receives the laser signal reflected from the target, and measures parameters such as the laser's time of flight and phase difference to calculate the distance between the target and the device. It can quickly and accurately acquire the three-dimensional spatial coordinate information of a target object. Facade laser point clouds are sets of three-dimensional points collected by LiDAR on the exterior walls of buildings. Each point contains information such as three-dimensional spatial coordinates. These points together constitute a three-dimensional contour model of the exterior wall, accurately reflecting the geometric shape, concavity and convexity variations, and other spatial features of the wall surface.

[0065] In this embodiment, the UAV is a quadcopter or multi-rotor UAV. The thermal infrared camera is an uncooled focal plane detector in the 8–14 μm band with a resolution of, for example, 640×512, and has temperature measurement capabilities. The lidar is a multi-line airborne laser scanner with a ranging accuracy better than 3 cm. The GNSS / IMU module (Global Navigation Satellite System / Inertial Measurement Unit) is used to output the attitude and position of the UAV in the world coordinate system. The thermal infrared camera and lidar are set at acquisition positions matching the exterior wall to be inspected. The temperature measurement range and resolution parameters of the thermal infrared camera, and the scanning frequency and point cloud density parameters of the lidar are set respectively. The thermal infrared camera and lidar are then started to acquire thermal infrared image data and facade laser point cloud data of the exterior wall to be inspected. Thermal infrared images and facade laser point clouds characterize the state of the exterior wall to be inspected from the dimensions of temperature features and three-dimensional geometric features, respectively, providing a complementary data foundation for subsequent fusion inspection. This solves the technical defects of single thermal infrared detection being easily affected by ambient temperature and single laser point cloud detection being unable to identify the internal cavities of hollow areas.

[0066] In one feasible implementation, step S10 may include: setting up the thermal infrared camera and lidar at different acquisition points on the exterior wall to be inspected; calibrating the spatial coordinates of the two types of equipment to establish a unified acquisition coordinate system between the equipment; setting the acquisition trigger frequency of the thermal infrared camera and the scanning frequency of the lidar to synchronize the acquisition objects of the two types of equipment; starting the thermal infrared camera and lidar to independently acquire thermal infrared images and facade laser point clouds of the exterior wall to be inspected; recording the acquisition time and spatial coordinate information of each data set; and registering the independently acquired thermal infrared images and facade laser point clouds based on the acquisition time and spatial coordinate information to obtain a matched image-point cloud dataset. Data matching is achieved through spatial coordinate calibration and synchronized acquisition frequencies, adapting to different models and interfaces of thermal infrared cameras and lidars, thus reducing the constraints on equipment selection.

[0067] In another feasible implementation, step S10 may include: dividing the exterior wall to be inspected into several regular detection sub-regions, and determining the acquisition path and equipment setup parameters for each sub-region. A drone equipped with a thermal infrared camera and lidar is used to fly along a preset acquisition path. The drone's positioning module records the latitude, longitude, and altitude information of the acquisition location in real time. The drone is controlled to hover in each detection sub-region for a preset duration, simultaneously acquiring thermal infrared images and facade lidar point clouds of that sub-region. Based on the drone's positioning information, sub-region identifiers are added to each set of thermal infrared images and facade lidar point clouds, and the acquisition data from all sub-regions are integrated into a comprehensive dataset of the exterior wall to be inspected. Leveraging the flight advantages of the drone platform, it can easily reach high-altitude, cantilevered, and irregularly shaped exterior wall areas that are difficult for manual or ground equipment to access, achieving comprehensive, blind-spot-free acquisition of the exterior wall to be inspected.

[0068] Step S20: Determine the temperature anomaly region in the thermal infrared image. Based on the spatial mapping relationship between the thermal infrared image and the facade laser point cloud, determine the corresponding exterior wall point cloud subset in the facade laser point cloud for the temperature anomaly region in the thermal infrared image.

[0069] Temperature anomaly regions refer to localized areas in thermal infrared images where the temperature value significantly deviates from the normal temperature range of the exterior wall surface. Due to the presence of air layers within the exterior wall, the heat conduction efficiency differs from that of undamaged walls, resulting in higher or lower temperatures under the same environmental conditions. This characteristic manifests as distinct color patches in thermal infrared images, and the image areas corresponding to these patches are the temperature anomaly regions. The exterior wall point cloud subset refers to a set of local point cloud data from the facade laser point cloud that spatially corresponds to the temperature anomaly regions in the thermal infrared image. It contains the wall geometry information of these localized areas, used for subsequent deformation analysis.

[0070] In this embodiment, the precise temperature values ​​of each pixel in the image are extracted, and the temperature distribution in areas without hollow risk, such as smooth and defect-free parts of the wall surface, is statistically analyzed in the thermal infrared image to determine the normal temperature range of the exterior wall to be inspected. All pixels in the thermal infrared image are traversed, and the set of pixels with temperature values ​​exceeding the normal temperature range is selected. Adjacent abnormal pixels are analyzed for regional connectivity and merged to form independent temperature anomaly regions, and the image coordinate range of each region is recorded. A pre-established spatial mapping relationship between the thermal infrared image and the facade laser point cloud is invoked, and the image coordinate range of each temperature anomaly region is converted into a three-dimensional coordinate range of the facade laser point cloud through spatial mapping. In the complete facade laser point cloud, all point cloud data within this three-dimensional coordinate range are selected to form a subset of exterior wall point clouds corresponding one-to-one with the temperature anomaly regions. The association between the image and point cloud data is completed through spatial mapping, transforming two-dimensional temperature anomaly information into three-dimensional spatial location information, providing accurate target point cloud data for subsequent deformation analysis.

[0071] In one feasible implementation, step S20 may include: performing Gaussian filtering denoising on the thermal infrared image to extract the temperature value of each pixel in the image. Using a statistical thresholding method, the mean and standard deviation of the temperature of all pixels in the thermal infrared image are calculated. A normal temperature range is defined as the temperature range fluctuating between the mean and the standard deviation above and below the mean. Pixels exceeding this range are then filtered out. Eight-neighbor connectivity analysis is performed on the abnormal pixels, merging adjacent abnormal pixels to form independent temperature anomaly regions. The image pixel coordinates of each region are recorded. A pre-calibrated pixel-to-3D coordinate transformation matrix is ​​used to map the pixel coordinate range of the temperature anomaly region to the 3D spatial coordinate range of the facade laser point cloud. All point cloud data within the aforementioned 3D spatial coordinate range are extracted from the facade laser point cloud to generate a subset of exterior wall point clouds corresponding one-to-one with the temperature anomaly region. By using a statistical thresholding method combined with eight-neighbor connectivity analysis, temperature anomaly region identification can be completed without manual intervention, reducing manual operation costs and subjective judgment errors.

[0072] For example, to accurately eliminate isolated noise points, the thermal infrared image of the exterior wall to be processed is read, the pixel size and temperature value storage format of the image are determined, a Gaussian kernel of size 5×5 is selected, and the standard deviation of the Gaussian function σ=1.0 is set so that the filtering operation suppresses noise while preserving temperature gradient features. A two-dimensional Gaussian convolution operation is performed on the thermal infrared image, and the filtering calculation is completed pixel by pixel, outputting the denoised thermal infrared image. Based on the temperature measurement calibration parameters of the thermal infrared camera, the gray value of each pixel in the denoised image is converted into the corresponding actual temperature value. Gaussian filtering smooths the image through weighted averaging, which can accurately eliminate isolated noise points while preserving the temperature boundary features between hollow areas and normal wall surfaces, avoiding an increase in the false detection rate of subsequent threshold screening due to noise.

[0073] Furthermore, to merge adjacent real anomalous pixels into a complete region, a pixel neighborhood model of the thermal infrared image is constructed, defining the eight-neighbor range of each pixel, i.e., adjacent pixels in the top, bottom, left, right, and four diagonal directions. An anomalous region labeling matrix is ​​initialized, with the same size as the thermal infrared image. All initial anomalous pixels are traversed. For unlabeled anomalous pixels, a region growing algorithm is executed, using the pixel as a seed point. Pixels within its eight-neighbor range are traversed; if a neighboring pixel is an initial anomalous pixel and has not been labeled, it is included in the same region, and growth continues. After all regions have grown, each independent grown region is labeled as a temperature anomalous region. Small regions with fewer than 5 pixels are removed. The minimum bounding rectangle pixel coordinates of each temperature anomalous region are recorded, i.e., the coordinates of the top-left and bottom-right corners. Eight-neighbor connectivity analysis can merge adjacent real anomalous pixels into a complete region while removing discrete residual noise points, ensuring that the generated temperature anomalous region's shape is consistent with the actual void location, providing an accurate two-dimensional region range for subsequent spatial mapping.

[0074] In another feasible implementation, step S20 may include: segmenting the thermal infrared image into several equally sized detection sub-blocks, calculating the average temperature of each detection sub-block, using the isolated forest algorithm to detect anomalies in the average temperature of all sub-blocks, and marking the detection sub-blocks with temperature anomalies. Boundary optimization is performed on the marked abnormal sub-blocks, removing scattered abnormal sub-blocks at the edges to form continuous temperature anomaly regions. Based on the joint calibration parameters of the thermal infrared camera and lidar, a spatial mapping relationship between the thermal infrared image and the facade lidar point cloud is established. Feature points of typical temperature anomaly regions are selected through manual interaction, and the three-dimensional coordinates of the corresponding feature points are matched in the facade lidar point cloud. This serves as a benchmark to delineate the target area range and extract the corresponding subset of the exterior wall point cloud. By establishing a spatial mapping relationship based on the joint calibration parameters, even if there are deviations in the installation positions of the thermal infrared camera and lidar, the mapping error can be corrected through feature point matching, adapting to different equipment installation schemes.

[0075] Step S30: Fit a reference surface to the subset data of the outer wall point cloud, construct a normal deformation model of the outer wall that reflects the concave and convex features of the wall based on the fitting results, and extract local deformation anomaly areas from the normal deformation model of the outer wall.

[0076] Reference plane fitting refers to the process of fitting a reference plane for an ideal, flat wall surface using mathematical algorithms based on the 3D coordinate data of a subset of the wall point cloud. The wall normal deformation model is a mathematical model constructed based on the reference plane fitting results to characterize the offset of the actual wall shape relative to the normal direction of the ideal flat surface, i.e., the direction perpendicular to the wall surface. Local deformation anomaly areas refer to point cloud regions selected from the wall normal deformation model where the normal offset exceeds the normal deformation threshold range.

[0077] In this embodiment, the least squares method is used to perform planar fitting on the preprocessed point cloud subset to obtain an initial reference surface. The normal vector direction of the reference surface is determined, and all points in the point cloud subset are traversed to calculate the normal distance from each point to the reference surface, establishing a normal deformation model for the external wall. The normal distance distribution of the normal wall surface point set is statistically analyzed, a normal deformation threshold range is set, and points whose normal distance exceeds the threshold range are filtered out, ultimately obtaining the local deformation anomaly areas. The constructed external wall normal deformation model can intuitively present the wall surface's unevenness distribution. Combined with threshold filtering to extract local deformation anomaly areas, it can accurately locate the geometric deformation position caused by hollowness, effectively eliminating irrelevant interference such as wall texture and equipment noise, and reducing the misjudgment rate of geometric features.

[0078] In one feasible implementation, step S30 may include: performing plane fitting on the denoised point cloud data using the least squares method to generate an initial reference surface; calculating the normal distance from all points to the initial reference surface; filtering out normal point sets with distances less than, for example, 2 mm; refitting based on this normal point set to obtain an optimized reference surface; determining the normal vector direction of the reference surface, with the outer side of the wall being positive; calculating the normal offset of each point in the point cloud subset to the reference surface; establishing a mapping relationship between three-dimensional coordinates and normal offset; and generating a normal deformation model of the external wall. A normal offset threshold is set, and deformation anomalies with offsets exceeding the threshold are filtered out. Eight-neighbor connectivity analysis is performed on the anomalies, merging adjacent anomalies to form continuous regions, thus obtaining local deformation anomaly regions. Calculating the normal offset on a per-point-cloud basis can accurately capture the minute features caused by wall hollowing; eight-neighbor connectivity analysis can effectively merge adjacent anomalies, ensuring that the extracted local deformation anomaly regions match the boundary height of the actual hollowing locations.

[0079] For example, to reduce the distance error between the fitted plane and the point cloud as a whole, the general form of the plane equation is set as ax + by + cz + d = 0, where (a, b, c) are the plane normal vectors, and constraint a 2 +b 2 +c 2 =1 to avoid multiple solutions for the equation parameters. Construct an error function, calculate the squared normal distance from each point to the plane, and calculate the total error function. Calculate the partial derivatives of the error function with respect to parameters a, b, c, and d, and set the partial derivatives equal to 0 to construct a system of linear equations. Solve the system of linear equations to obtain the initial reference plane. The least squares method can quickly generate an initial reference plane covering the entire subset of point clouds, reflecting the macroscopic flatness trend of the wall surface.

[0080] In another feasible implementation, step S30 may include: using a robust estimation algorithm to perform planar fitting on the external wall point cloud to generate a reference surface; dividing the point cloud subset into rasterized units, such as 3cm×3cm; calculating the average normal offset of all points within each raster; and constructing a rasterized external wall normal deformation model using the raster as the basic unit. Based on the surface finish of the external wall to be detected, such as ceramic tile or plaster, an adaptive threshold for the average raster offset is set; abnormal raster units with offsets exceeding the threshold are filtered out; four-neighborhood region growing is performed on the abnormal raster units, merging them to form continuous local deformation anomaly regions; and the three-dimensional boundary coordinates and maximum deformation of the regions are output. Rasterized modeling transforms discrete point clouds into regular raster units, significantly reducing the complexity of subsequent data processing. Adaptively setting the threshold based on the surface finish type is compatible with the deformation characteristics differences of different external wall materials such as ceramic tile and plaster, improving the versatility of the solution.

[0081] Step S40: Combine the detection results of temperature anomaly areas and local deformation anomaly areas to determine the detection results of hollow areas in the target exterior wall.

[0082] In this embodiment, the detection results of temperature anomaly areas and local deformation anomaly areas are acquired separately. The two detection results are then jointly judged, such as through a simple AND, OR, or NOT judgment. If both results are present, the target result is considered present; if neither is present, it is considered absent; or if one is present, it is considered present. Alternatively, a weighted fusion method can be used to obtain the target external wall hollowness detection result. By weighted fusion of temperature and deformation dual-dimensional detection results, a comprehensive determination of hollowness is achieved, improving the accuracy and reliability of the detection results and effectively reducing the probability of misjudgment and missed detection from single-dimensional detection.

[0083] In one feasible implementation, step S40 may include: acquiring three-dimensional spatial range data of temperature anomaly areas and three-dimensional spatial range data of local deformation anomaly areas, and completing precise coordinate alignment of the two types of areas. A spatial matching degree threshold is set, such as 60%, and the spatial intersection ratio between each temperature anomaly area and each deformation anomaly area is calculated to obtain the area matching degree. Based on the matching degree result, areas with a matching degree ≥ 60% are identified as hollow areas; areas with only temperature anomalies are marked as suspected hollow areas; and areas with only deformation anomalies are identified as non-hollow deformation areas. The number, area, and spatial location information of all hollow areas are statistically analyzed, and an external wall hollow detection report containing area labels is generated. The dual-area fusion judgment based on spatial matching degree directly correlates the temperature anomaly and geometric deformation characteristics of the hollow areas, avoiding the risk of misjudgment from single-dimensional detection, resulting in accurate and reliable detection results.

[0084] In another feasible implementation, step S40 may include: assigning temperature anomaly weight values ​​to temperature anomaly areas, i.e., calculating based on temperature offset, with larger offsets resulting in higher weight values; and assigning deformation anomaly weight values ​​to local deformation anomaly areas, i.e., calculating based on normal offset, with larger offsets resulting in higher weight values. Based on spatial coordinate association, the temperature anomaly weights and deformation anomaly weights at the same physical location are weighted and summed to obtain a comprehensive anomaly score for that location. A comprehensive anomaly score threshold is set; locations with scores ≥ the threshold are identified as hollow areas; locations with scores below the threshold but with a single weight exceeding the threshold are marked as areas awaiting review; and locations with no weight exceeding the threshold are identified as normal areas. Combining the on-site investigation suggestions for areas awaiting review, the information of hollow areas and areas awaiting review is integrated to generate a graded and labeled hollow detection result document. The anomaly level of temperature offset and deformation degree is quantified through weight values ​​to avoid ignoring minor hollow areas or anomaly characteristics under special working conditions.

[0085] This embodiment provides a method for detecting hollow exterior walls based on the fusion of thermal infrared and facade point cloud. By collaboratively acquiring thermal infrared images and facade laser point clouds, a normal deformation model of the exterior wall is constructed to measure local deformation of the exterior wall. Temperature anomaly areas and local deformation areas are jointly judged to eliminate interference from single-dimensional anomalies, which has the technical effect of improving the accuracy of exterior wall hollow detection.

[0086] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment can be referred to the above description, and will not be repeated hereafter. In addition, step 30 further includes steps S31 to S34:

[0087] Step S31: Obtain the local external wall reference surface by fitting the reference surface to the subset of external wall point cloud.

[0088] Among them, the reference surface of the local exterior wall is a planar model or a curved surface model.

[0089] In this embodiment, complete 3D coordinate data of a subset of external wall point clouds is acquired to confirm the actual structural form of the external wall corresponding to the subset, such as a flat wall segment, an arc-shaped wall segment, or an irregularly shaped wall segment. If it is a flat wall segment, the subset is fitted with a plane using algorithms such as least squares or RANSAC to generate a planar local external wall reference surface, outputting the plane equation and normal vector parameters. If it is an arc-shaped or irregularly shaped wall segment, the subset is fitted with a surface using algorithms such as B-spline surface or implicit surface fitting to generate a curved local external wall reference surface, outputting the surface parameter equation and normal vector parameters for each point. The planar or curved surface model is adapted according to the actual structural form of the external wall, ensuring that the reference surface closely matches the real reference form of different types of walls, such as flat, arc-shaped, and irregularly shaped walls, eliminating geometric reference deviations of irregularly shaped walls from the source of the fitting process.

[0090] Step S32: Construct a projection reference plane based on the geometric features of the local external wall reference plane, and calculate the normal distance residual from each data point in the external wall point cloud subset to the local external wall reference plane.

[0091] The projection reference plane is a standard reference plane constructed based on the geometric features of a local exterior wall reference plane. It is highly consistent with the local exterior wall reference plane in terms of spatial orientation and geometric parameters. That is, assuming that the wall surface has no voids, bulges, or depressions, all point cloud data points will be projected onto this plane. The normal distance residual is the vertical distance value from a single data point in the exterior wall point cloud subset to the projection reference plane. It reflects the degree of offset of the actual wall point relative to the ideal flat wall surface. If the value is 0, it means that the point cloud point falls on the projection reference plane, and the wall surface at this position is flat and without deformation. If the value is not 0, it means that there is a normal bulge or depression at that point relative to the flat wall surface. The larger the absolute value, the more obvious the degree of deformation.

[0092] In this embodiment, the geometric features of the local external wall reference surface are analyzed, such as the normal vector of the plane, the normal vectors of each point on the curved surface, the contour direction, and the spatial coordinate range. A projection reference surface that perfectly matches the spatial position is constructed along the geometric contour of the reference surface. The positive direction of the reference surface normal vector is determined to point outward from the wall. For each data point in the external wall point cloud subset, the corresponding calculation formula is called according to the reference surface type. The planar reference surface is directly calculated using the formula for the normal distance from a point to a plane, and the curved surface reference surface is calculated using the formula for the normal distance from a point to the nearest point on the curved surface. The normal distance residual from the data point to the local external wall reference surface is solved point by point. The positive or negative value of the residual represents the direction of offset, i.e., positive indicates outward bulging and negative indicates inward concavity, and the absolute value represents the degree of offset. The mapping relationship between the three-dimensional coordinates of the point cloud data points and the corresponding normal distance residuals is constructed, generating a coordinate-residual value initial dataset, thus completing the deformation of the discrete point cloud. The projection reference plane constructed based on the geometric features of the reference surface achieves the uniqueness and uniformity of the projection reference of point cloud data, eliminates subsequent calculation deviations caused by inconsistent reference standards, and ensures the accuracy of deformation transformation.

[0093] Step S33: The projection reference plane is meshed to divide it into multiple regular mesh units.

[0094] In this embodiment, based on the accuracy requirements for detecting hollow areas in exterior walls, gridding parameters are set, such as a 3cm × 3cm evenly spaced regular grid. These parameters can be adaptively adjusted according to the type of exterior wall finish. Using the projection reference plane as the gridding processing carrier, the projection reference plane is divided at equal intervals according to preset parameters, generating multiple regular grid units, and each grid unit is assigned a unique spatial identifier. This evenly spaced regular gridding achieves spatial normalization and structuring of discrete point cloud residual data, transforming disordered discrete data into ordered grid unit data, and avoiding the interference of random noise from single point clouds with subsequent deformation analysis.

[0095] Step S34: Based on the projection reference plane and the normal distance residuals in all grid cells, construct the external wall normal deformation model distributed along the facade normal.

[0096] In this embodiment, please refer to Figure 3 , Figure 3This application provides infrared images and exterior wall normal maps for the exterior wall hollow detection method based on the fusion of thermal infrared and facade point cloud. For each regular grid cell, statistical calculations are performed on the internal normal distance residuals, such as calculating the mean, maximum, or median, to obtain a representative residual value for each grid cell. Based on the spatial position of the projection reference plane, the spatial identifier of each grid cell is associated with the corresponding representative residual value. Using the grid cell as the basic expression unit, the representative residual value is transformed into a visualized deformation feature along the facade normal direction, such as color depth or height value, generating an exterior wall normal deformation model map distributed along the facade normal. Visualization files of the deformation model are output, such as heat maps or 3D mesh models and structured data files, supporting subsequent abnormal area extraction and analysis. Using the representative residual value of the grid cell to characterize regional deformation preserves the spatial distribution law of wall deformation while weakening the noise interference of single point clouds, allowing the deformation model to accurately reflect the real concave and convex features of the wall surface and improving the reliability of the model.

[0097] In this embodiment, by fitting the local exterior wall reference surface of the model, constructing the projection reference surface and quantifying the normal distance residual, and combining the projection reference surface meshing and mesh residual integration to construct a deformation model distributed along the facade normal, the accurate quantification, structural integration and visualization of the deformation characteristics of exterior walls with different structural forms are realized, providing high-precision and spatial model support for the extraction of local deformation anomaly areas.

[0098] Based on the second embodiment of this application, the third embodiment of this application proposes a method for detecting hollow exterior walls based on the fusion of thermal infrared and facade point cloud, which can be referred to the above description and will not be repeated hereafter. Based on this, the steps for constructing an exterior wall normal deformation model distributed along the facade normal, based on the projection reference plane and the normal distance residuals within all grid cells, include:

[0099] Step S341: Determine the two-dimensional coordinate indices of all regular mesh elements on the projection reference plane.

[0100] In this embodiment, based on the arrangement of grid cells in the projection reference plane coordinate system, a unique two-dimensional coordinate index is assigned to each regular grid cell, with each index value corresponding one-to-one with the actual spatial location of the grid cell. A mapping table is established between the physical boundaries of the grid cells and their two-dimensional coordinate indices, recording the actual spatial range of each grid cell corresponding to its index on the projection reference plane. By assigning a unique two-dimensional coordinate index bound to its spatial location to each grid cell on the projection reference plane, a standardized position identification system for grid cells is established, achieving precise mapping between the spatial location of the wall and its index.

[0101] Step S342: Calculate the normal distance residuals of the point cloud data mapped to all regular grid cells, and determine the statistical value of the normal distance residuals as the normal deformation value of the grid cell. The statistical value includes the mean or median.

[0102] The normal deformation value is a wall deformation quantification index corresponding to a single regular grid cell. It is a unique statistical value obtained by calculating the average or median of the normal distance residuals of all point cloud data mapped to the grid. This value is directly assigned to the corresponding grid cell and represents the average deformation degree of the entire wall area covered by this grid along the normal of the projection reference plane.

[0103] In this embodiment, each point cloud data point is mapped to a corresponding regular grid cell according to its projection position. All regular grid cells are traversed, and the normal distance residuals of all point cloud data mapped within each grid cell are statistically analyzed to form an independent residual dataset for each grid cell. For the residual dataset of each grid cell, a statistical method is selected based on the required detection accuracy; for example, the average residual is calculated to achieve the desired deformation level, while the median residual is calculated to suppress extreme outlier interference. The calculated average or median is determined as the normal deformation value of the corresponding grid cell, completing the deformation quantification at the grid cell level. By using the statistical method of average or median, the discrete residual data is aggregated and quantified, transforming the microscopic deformation information of a single point cloud into the macroscopic deformation characteristics of the grid region, accurately characterizing the overall normal deformation degree of each wall area.

[0104] Step S343: Using the two-dimensional coordinate index as the position parameter and the normal deformation value corresponding to all grid elements as the deformation parameter, generate the external wall normal deformation model.

[0105] In this embodiment, two-dimensional coordinate indices are used as position parameters to locate the actual arrangement position of each grid unit on the projection reference plane. The normal deformation value corresponding to each grid unit is used as a deformation parameter, and a mapping rule between the deformation value and the visualization feature is set, such as the numerical value corresponding to the color depth or normal height value. Based on the planar coordinate system of the projection reference plane, according to the arrangement rule of the two-dimensional coordinate index, the normal deformation value of each grid unit is assigned to the corresponding spatial position, establishing a mapping relationship between the position parameter and the deformation parameter. Based on the above mapping relationship, an exterior wall normal deformation model with grids as basic units and distributed along the facade normal is generated, and visualization files of the model, such as color heatmaps and three-dimensional grid deformation maps, are output, completing the model construction. By constructing a model with two-dimensional coordinate indices as position parameters and normal deformation values ​​as deformation parameters, the digital, structured, and spatial expression of the wall normal deformation features is realized, making the distribution pattern of deformation and the degree of regional deformation intuitively judged, solving the problem that discrete point cloud data is difficult to directly interpret deformation features.

[0106] In one feasible implementation, a RANSAC plane fitting is performed on a subset of the point cloud to obtain the local external wall plane equation and normal vector. nOutliers exceeding a set threshold distance from the plane are removed, retaining only valid external wall points. Two orthogonal direction vectors are selected within the plane. u,v , and normal vector w=n Together they form a three-dimensional orthogonal coordinate system. By transforming the subset point cloud to this local coordinate system, the coordinates of each point can be obtained. u i , v i , w i） ,in w i This represents the normal displacement of a point relative to the fitted plane. In ( u,v) Establish a regular grid in the plane, for example, 3cm × 3cm, and assign points within the grid. w i The values ​​are averaged or median to obtain the result. u, v） For independent variable, w The building facade normal direction DEM (Digital Elevation Model) is a functional value representing the elevation map of local bulges or depressions in the exterior walls. Through RANSAC plane fitting, local coordinate system construction, and regular grid statistics, a discrete subset of exterior wall point clouds is transformed into a quantifiable facade normal direction DEM. This accurately extracts the geometric deformation features of the wall surface, providing intuitive and quantifiable data on the amount of bulges and the depth of depressions for subsequent hollowness determination, effectively supporting the implementation of the two-dimensional joint discrimination rule.

[0107] In this embodiment, the position is accurately calibrated by assigning a unique two-dimensional coordinate index to the regular grid unit of the projection reference plane. The normal deformation value is obtained by combining the average or median statistical normal distance residual within the grid. Then, the model is constructed with the coordinate index and deformation value as the core parameters, realizing the structured quantification, spatial positioning and digital modeling of the normal deformation characteristics of the exterior wall, and accurately representing the overall deformation degree of different areas of the wall.

[0108] Based on any of the above embodiments of this application, Embodiment 4 of this application proposes a method for detecting hollow exterior walls based on the fusion of thermal infrared and facade point cloud, which can be referred to the above description and will not be repeated hereafter. In addition, step S40 further includes steps S41~S42:

[0109] Step S41: Determine the first detection result of the temperature anomaly area and the second detection result of the local deformation anomaly area.

[0110] The first detection result is an independent detection and judgment result for temperature anomaly areas obtained from thermal infrared image analysis, output from the thermal feature dimension. It characterizes the abnormal features of the wall surface temperature distribution. This result is obtained through radiometric calibration, environmental correction, anomaly area extraction, and quantitative analysis of the thermal infrared image, including qualitative judgment of temperature anomaly areas and quantitative indicators and spatial information of thermal features. The second detection result is an independent detection and judgment result for local deformation anomaly areas obtained from point cloud data analysis, output from the geometric deformation dimension. It characterizes the bulging or concave deformation features of the wall surface normal. This result is obtained through planar or curved surface fitting of point cloud subsets, calculation of normal distance residuals, meshing statistics, and deformation anomaly extraction, including qualitative judgment of deformation anomaly areas and quantitative indicators and spatial information of geometric deformation.

[0111] In this embodiment, please refer to Figure 4 , Figure 4 This document presents schematic diagrams of abnormal areas in the infrared and normal maps of the exterior wall, provided by the exterior wall hollowness detection method based on the fusion of thermal infrared and facade point cloud data in this application. The first detection result for temperature anomaly areas is extracted, including the spatial range, quantized temperature offset value, and anomaly level. The second detection result for local deformation anomaly areas is extracted, including the spatial range, quantized normal deformation value, and anomaly level. By extracting the detection results for temperature and deformation and clarifying the characteristics of each dimension, clear decomposition and feature retention of detection information are achieved, avoiding the loss of key anomaly information in a single dimension during the fusion process, and laying a data foundation for subsequent accurate weighted fusion.

[0112] Step S42: Combine the first detection result and the second detection result to obtain the target external wall hollow detection result.

[0113] In this embodiment, appropriate weighting coefficients are assigned to the first and second detection results based on the influence weights of temperature and deformation characteristics on the determination of hollowness in the exterior wall. The quantified values ​​of temperature anomalies and deformation anomalies at the same spatial location are multiplied by their corresponding weighting coefficients and then summed to obtain a comprehensive anomaly evaluation value. A comprehensive anomaly determination threshold is set, and the final detection result of the target exterior wall hollowness is obtained by weighted fusion based on the comparison between the evaluation value and the threshold. Please refer to... Figure 5 , Figure 5This diagram illustrates the fusion result of the exterior wall hollowing detection method based on thermal infrared and facade point cloud fusion proposed in this application. Color differences are used to present the temperature distribution and local deformation of the exterior wall surface. This verifies the high degree of consistency between the detection results and actual tile detachment and depression, while also revealing hidden hollowing defects that are difficult to detect under visible light. By adapting weighting coefficients to the different degrees of influence of temperature and deformation characteristics on hollowing determination, precise weighted fusion of detection results is achieved. The comprehensive anomaly evaluation value is used as the basis for hollowing determination, making the detection results more closely match the actual characteristics of exterior wall hollowing and significantly improving the accuracy and rationality of hollowing determination.

[0114] In one feasible implementation, a joint discrimination rule can be constructed. For example, a region is identified as hollow when the following conditions are met: the temperature difference is greater than a set temperature difference threshold, the maximum deformation is greater than a set deformation threshold, and the overlap rate between the bulging region and the thermal anomaly region is greater than a set threshold. Alternatively, a binary classification model can be trained using thermal and geometric features to classify candidate regions as hollow or non-hollow. By constructing a joint discrimination rule based on thermal and geometric features or training a binary classification model, accurate determination of hollow candidate regions is achieved, improving the accuracy of hollow identification and effectively reducing false positives and false negatives from single-dimensional detection.

[0115] In this embodiment, by extracting the core features of the temperature and deformation dimension detection results and combining them with weighting coefficients, the accurate weighted fusion and comprehensive judgment of the two-dimensional results are achieved, effectively avoiding the limitations of single-dimensional detection and improving the accuracy and reliability of the target exterior wall hollow detection results.

[0116] Based on the fourth embodiment of this application, embodiment five of this application proposes a method for detecting hollow areas in exterior walls based on the fusion of thermal infrared and facade point cloud, which can be referred to the above description and will not be repeated hereafter. Based on this, the steps for determining the first detection result of the temperature anomaly area and the second detection result of the local deformation anomaly area include:

[0117] Step S411: Map the contour of the temperature anomaly region in the infrared image to the local coordinate system of the local deformation anomaly region to obtain the sub-region corresponding to the temperature anomaly region on the normal deformation model of the outer wall.

[0118] In this embodiment, the contour of the candidate temperature anomaly region in the infrared image is mapped to the local coordinate system of the local deformation anomaly region, and the corresponding sub-region is extracted. By mapping the infrared image contour to the local coordinate system, spatial registration between the temperature anomaly region and the normal deformation model of the outer wall is achieved, so that the analysis objects of thermal features and geometric features point to the same physical area of ​​the outer wall, thus eliminating the matching deviation of the two-dimensional detection results at the spatial level.

[0119] Step S412: Extract the thermal features of the temperature anomaly region and determine the first detection result of the temperature anomaly region based on the thermal features.

[0120] The thermal characteristics include at least the average temperature, the maximum temperature gradient within the region, and the temperature difference relative to the background. The average temperature is the arithmetic mean of the apparent temperatures of all pixels within the temperature anomaly region, representing the overall temperature level of the anomaly area. Due to the poor thermal conductivity of the air gap, the average temperature of the hollow area will differ significantly from that of the normal wall surface. The temperature gradient refers to the amount of temperature change per unit distance, reflecting the severity of temperature changes in space. The maximum temperature gradient within the region is the maximum temperature gradient at all locations within the temperature anomaly region, representing the rate of temperature change at the point of most drastic temperature change. At the boundary of the hollow area where it connects to the normal wall surface, the abrupt change in thermal conductivity results in a significant temperature jump, manifested as a significantly increased local temperature gradient. This indicator accurately characterizes the boundary features of the temperature anomaly region and reflects the severity of the thermal anomaly. The temperature difference relative to the background is the difference between the characteristic temperature of the temperature anomaly region and the characteristic temperature of the normal wall background area. The background area is a flat area of ​​the wall surface without hollow areas and with normal thermal conductivity. The larger the absolute value of this temperature difference, the more significant the difference in thermal state between the target area and the normal wall surface, and it also reflects the thickness and extent of the air layer in the hollow area.

[0121] In this embodiment, thermal features such as average temperature, maximum temperature gradient within the region, and temperature difference relative to the surrounding background are extracted from thermal infrared data. Based on the extracted multi-dimensional thermal features, quantitative analysis and anomaly determination are performed to identify the first detection result corresponding to the temperature anomaly region. This comprehensively quantifies the thermal conduction differences caused by hollow areas, avoiding the problem that a single thermal feature cannot distinguish between hollow areas and non-hollow temperature anomalies such as environmental interference and material differences, thus improving the accuracy and objectivity of temperature-dimensional anomaly determination.

[0122] Step S413: Extract the geometric features of the sub-region and determine the second detection result of the local deformation anomaly region based on the geometric features.

[0123] The geometric features include at least the average deformation, maximum deformation, deformation area ratio, and deformation profile. The average deformation refers to the average of the normal deformation values ​​of all regular grid cells within the local deformation anomaly area. It represents the overall normal bulging or depression degree of the deformation area; the larger the absolute value, the more significant the overall deformation of the area. The maximum deformation refers to the maximum absolute value of the normal deformation values ​​within the local deformation anomaly area, representing the extreme degree of deformation in that area. This value corresponds to the location where wall hollowing is most severe. The deformation area ratio refers to the ratio of the actual physical area of ​​the local deformation anomaly area to the total area of ​​the corresponding detection sub-area. The larger the ratio, the wider the coverage of the deformation caused by hollowing in the detection sub-area, reflecting the degree of hollowing diffusion. The deformation profile refers to the boundary shape and spatial coordinate range of the local deformation anomaly area, characterizing the geometric features of the spatial morphology and location of the deformation area.

[0124] In this embodiment, the average deformation is extracted from the sub-region. w 平均 Maximum deformation w max Geometric features such as deformation area ratio and deformation contour are extracted. Based on the extracted multi-dimensional geometric features, quantitative analysis and anomaly judgment are completed to determine the second detection result corresponding to the local deformation anomaly area. The wall normal deformation features caused by hollowing are refined and quantified, and geometric anomalies are characterized from multiple dimensions such as deformation degree, range and shape. This avoids missing the core deformation information of hollowing by a single geometric feature, and improves the comprehensiveness and reliability of deformation dimension anomaly judgment.

[0125] In this embodiment, by accurately mapping the contour of the temperature anomaly region to the local coordinate system to match the corresponding sub-region of the normal deformation model, and then extracting the multi-dimensional thermal features of the temperature anomaly region and the multi-dimensional geometric features of the sub-region to determine the detection results, the accurate matching and multi-dimensional quantitative characterization of temperature and geometric dual-dimensional anomaly features in the same physical region are achieved. This provides spatially unified, feature-rich and quantitatively accurate dual-dimensional data support for the subsequent fusion of hollow detection results.

[0126] Based on any of the above embodiments of this application, Embodiment Six of this application proposes a method for detecting external wall hollowness based on the fusion of thermal infrared and facade point cloud, which can be referred to the above description and will not be repeated hereafter. Based on this, the steps of determining the temperature anomaly region in the thermal infrared image and determining the corresponding external wall point cloud subset in the facade laser point cloud based on the spatial mapping relationship between the thermal infrared image and the facade laser point cloud include:

[0127] Step S21: Determine the temperature anomaly region in the thermal infrared image and obtain the minimum bounding rectangle of the temperature anomaly region in the thermal infrared image.

[0128] Step S22: Using the smallest bounding rectangle as a reference, expand outward within the area of ​​the thermal infrared image to obtain the temperature anomaly expansion area.

[0129] In this embodiment, after identifying the set of pixels in the temperature anomaly region, the maximum value of the horizontal coordinate is extracted by traversing the image coordinates of all pixels in the set. Minimum value and the maximum value of the ordinate Minimum value Based on this, construct the minimum horizontal bounding rectangle for this region, with a width of... ,high In the subsequent step S30, a reference surface needs to be fitted to the subset of the outer wall point cloud to construct a deformation model. If the point cloud corresponding to the original temperature anomaly area is directly used for fitting, since the point cloud in this area may be almost entirely located on the raised surface caused by the hollowness, the fitting algorithm is very likely to misjudge the "hollow raised surface" as the "ideal wall reference surface," resulting in the calculated normal distance residual approaching zero, thus failing to identify the true geometric deformation. Therefore, this embodiment expands the width and height of the minimum bounding rectangle outward (for example, the side length is expanded to 3 times the original size) to ensure that the point cloud subset contained in the obtained temperature anomaly expansion area, after being mapped to the point cloud space, not only covers the deformation area suspected to be hollow, but also includes the surrounding normal wall background points. At the same time, during the expansion process, the system automatically performs boundary verification to ensure that the coordinates of the expanded rectangle do not exceed the original pixel boundary of the thermal infrared image, that is, the temperature anomaly expansion area is located within the thermal infrared image.

[0130] Step S23: Obtain the intrinsic parameter matrix of the thermal infrared camera and the extrinsic parameter pose during imaging.

[0131] The intrinsic parameter matrix is ​​an inherent parameter of the thermal infrared camera, determined by the camera's hardware structure, and is used to describe the projection relationship between the normalized imaging plane and the pixel plane. The extrinsic parameter pose describes the spatial relationship between the thermal infrared camera's coordinate system and the world coordinate system at the moment of image acquisition, including the camera's three-dimensional position and shooting posture in the world coordinate system.

[0132] In this embodiment, the intrinsic parameter matrix of the thermal infrared camera is first obtained through camera calibration. Thermal infrared images and laser point clouds collected for the same building are acquired. Using common observational calibration features (such as calibration board corner points, manually placed feature points, or corresponding geometric corner points), the rotation matrix and translation vector of the thermal infrared camera coordinate system relative to the origin of the point cloud in the lidar coordinate system are solved using least squares optimization to obtain the extrinsic parameter matrix transformation. The carrier pose output in real time by the GNSS / IMU during flight is acquired. Using the installation extrinsic parameters of the radar in the aircraft coordinate system, the point cloud in the radar coordinate system is transformed to the world coordinate system, completing the unified transformation from the radar and camera coordinate systems to the world coordinate system. This ensures the spatial matching consistency between the thermal infrared image and the facade laser point cloud under the same global spatial reference system.

[0133] Step S24: Based on the intrinsic parameter matrix, the pixel coordinates of the contour boundary of the temperature anomaly extension region are converted into the imaging ray direction vector in the camera coordinate system.

[0134] The imaging ray direction vector is a three-dimensional direction vector in the camera coordinate system that points from the camera optical center to the outline boundary of the temperature anomaly region. In machine vision, each pixel in a two-dimensional image corresponds to a ray in three-dimensional space that starts from the camera optical center, passes through the pixel, and extends into the three-dimensional space. The imaging ray direction vector is a vector used to quantify the direction of extension of this ray in the camera coordinate system.

[0135] In this embodiment, the pixel coordinates of the contour boundaries of temperature anomaly regions in the thermal infrared image are extracted. Coordinate transformation is performed based on the intrinsic parameters and distortion parameters of the thermal infrared camera. This step eliminates imaging distortion caused by the physical characteristics of the camera lens and transforms the pixels from a nonlinear pixel plane to an ideal normalized imaging plane, thereby eliminating projection deviations caused by the pixel plane. The imaging ray direction vector in the camera coordinate system corresponding to each boundary pixel coordinate is calculated point-by-point, forming a set of ray direction vectors corresponding to the temperature anomaly expansion region, establishing a ray mapping relationship from the two-dimensional pixel space to the three-dimensional camera coordinate system. Through the inverse operation and geometric transformation of the aforementioned intrinsic parameter matrix, the abstract, discrete pixel contours in the two-dimensional infrared image are transformed into quantifiable, continuously distributed three-dimensional ray directions from the camera's perspective. This transformation achieves a precise mapping from the two-dimensional pixel space to the three-dimensional camera space, providing a high-precision directional basis for subsequently extracting a subset of the target exterior wall point cloud from the global facade laser point cloud using "frustum retrieval."

[0136] Step S25: Based on the extrinsic pose, convert the imaging ray direction vector into a line-of-sight projection ray in the world coordinate system.

[0137] The line-of-sight projection ray is a three-dimensional ray in the world coordinate system that starts from the actual optical center position of the camera and points to the three-dimensional spatial position of the pixel corresponding to the contour boundary of the temperature anomaly expansion area. It is the three-dimensional ray shape after the spatial coordinate system transformation is completed by combining the imaging ray direction vector with the camera extrinsic pose.

[0138] In this embodiment, the imaging pose of the thermal infrared image is obtained, namely the rotation matrix and displacement vector in the world coordinate system. Taking the camera's optical center as the ray origin, the direction vector of the imaging ray in the camera coordinate system is transformed by the rotation matrix, and the origin is spatially located by the displacement vector. Point by point, all imaging ray direction vectors are converted into line-of-sight projection rays in the world coordinate system, forming a set of rays belonging to the world coordinate system. The spatial range of the rays corresponds one-to-one with the contour of the temperature anomaly region. By using the imaging pose to convert the imaging rays in the camera coordinate system into line-of-sight projection rays in the world coordinate system, coordinate unification from the camera's local space to the global space is achieved. This ensures that the temperature anomaly region in the thermal infrared image and the laser point cloud on the facade are in the same spatial reference system, providing a global spatial basis for cross-device point cloud retrieval.

[0139] Step S26: In the facade laser point cloud, retrieve the outer layer point cloud data located within the spatial cone formed by multiple line projection rays, and determine the retrieved point cloud data as a subset of the exterior wall point cloud.

[0140] The spatial view cone is a cone-shaped three-dimensional spatial region formed by extending towards the outer wall of the temperature anomaly expansion region, with multiple line projection rays in the world coordinate system corresponding to the pixels of the outline boundary of the temperature anomaly expansion region as generatrices and the world coordinates of the camera optical center as vertices. It is a dedicated three-dimensional spatial retrieval range formed by mapping the two-dimensional temperature anomaly region in the world coordinate system.

[0141] In this embodiment, a closed spatial view cone is formed by using multiple projection rays in the world coordinate system as boundaries and the position of the optical center of the thermal infrared camera in the world coordinate system as the apex. This view cone represents the three-dimensional spatial retrieval range corresponding to the temperature anomaly region. All point cloud data in the facade laser point cloud are traversed, and it is determined whether the three-dimensional coordinates of each point cloud in the world coordinate system fall within the spatial view cone. All outer point cloud data located within the view cone and closest to the camera's optical center are filtered and retrieved, and integrated into a point cloud set. This set is determined as the subset of outer wall point clouds that precisely matches the temperature anomaly region. By using the spatial view cone as the retrieval range, the subset of outer wall point clouds corresponding to the temperature anomaly region is precisely selected from the facade laser point cloud, achieving precise spatial matching from the two-dimensional temperature anomaly region to the three-dimensional laser point cloud. This ensures that subsequent deformation analysis focuses only on the wall area corresponding to the temperature anomaly.

[0142] In one feasible implementation, based on the actual building parameters of the exterior wall to be detected, such as the vertical distance between the exterior wall and the flight path and the wall thickness, and combined with the ranging accuracy of the lidar, an effective distance threshold is set along the direction of the line-of-sight projection ray. This threshold is a reasonable distance range from the ray's origin to the exterior wall surface. The line-of-sight projection rays in the world coordinate system are traversed, and ray segments within the distance threshold range are intercepted along the ray direction, starting from the ray's origin, to form an effective projection ray with a defined distance. All point cloud data in the facade lidar point cloud are traversed, and it is determined whether the projection points of the point cloud in the ray direction fall within the distance threshold range. Point cloud data with projection distances within the threshold in the ray direction are integrated into a point cloud set, ultimately determining the exterior wall point cloud subset that precisely matches the temperature anomaly area. By combining the actual building parameters of the exterior wall with the lidar ranging accuracy to set the effective distance threshold in the ray direction, intercepting the effective projection rays within the defined distance, and filtering out point cloud data with projection distances within the threshold, redundant point clouds outside the exterior wall within the viewing cone are precisely eliminated, further narrowing down and accurately locking the exterior wall point cloud subset that matches the temperature anomaly area, thus improving the targeting and accuracy of point cloud retrieval.

[0143] Based on any of the above embodiments of this application, Embodiment Seven of this application proposes a method for detecting hollow areas in exterior walls based on the fusion of thermal infrared and facade point cloud data. This can be referred to the above description and will not be repeated hereafter. Based on this, the steps for determining the temperature anomaly area in the thermal infrared image include:

[0144] Step S27: Perform radiometric calibration and environmental correction on the thermal infrared image.

[0145] Radiometric calibration is a quantitative calibration process that converts the raw digital quantization value of each pixel in a thermal infrared image into a radiometric intensity value detectable by the camera, and then further converts it into a true apparent temperature value of the wall surface. This process is used to eliminate the systematic errors inherent in the thermal infrared camera itself. Environmental correction, after obtaining the apparent temperature through radiometric calibration, is a correction process that eliminates interference from various environmental factors at the testing site, restoring the wall surface's true thermal radiation temperature. This process is used to eliminate the influence of the external environment, ensuring that the temperature data only reflects the wall surface's own thermal characteristics.

[0146] In this embodiment, the device calibration curve of the thermal infrared camera is obtained, and the pixel grayscale values ​​of the original thermal infrared image are converted into the actual temperature values ​​of the wall surface, eliminating the temperature measurement deviation caused by the camera's hardware response characteristics. Combining parameters such as the emissivity of the exterior wall material, the reflected background temperature, and the ambient temperature and humidity, the apparent temperature field of the exterior wall is calculated through a temperature inversion model. Simultaneously, the interference of external factors such as ambient temperature and humidity and background reflection on the temperature measurement results is eliminated, ultimately outputting a corrected thermal infrared image and apparent temperature field data that accurately reflects the true temperature distribution of the wall surface. Radiometric calibration achieves accurate conversion from thermal infrared image pixel values ​​to actual temperature values, ensuring the authenticity of the temperature data. Environmental correction eliminates interference from external and internal factors, restoring the true temperature distribution characteristics of the wall surface, thus solving the problems of inaccurate temperature values ​​and susceptibility to environmental interference in the original thermal infrared image.

[0147] Step S28: Extract temperature anomaly regions from thermal infrared images based on preset image segmentation algorithms or deep learning models.

[0148] In this embodiment, single or multiple frames of thermal infrared images undergo temperature normalization. Traditional algorithms involving threshold segmentation and morphological processing are employed, or deep learning models with pre-trained semantic segmentation networks such as U-Net, YOLO-seg, and SAM are loaded. The processed images are then analyzed to identify and segment abnormal regions where the temperature deviates from the background. By unifying the data scale through temperature normalization and adapting traditional algorithms or deep learning models, abnormal temperature region detection is achieved. This ensures the flexibility and adaptability of abnormal region identification, accurately extracting the pixel set and boundary contours of candidate regions, and realizing automated, high-precision segmentation and calibration of abnormal temperature regions.

[0149] In this embodiment, by performing radiometric calibration and environmental correction on thermal infrared images to restore the true apparent temperature field of the exterior wall, and then relying on a preset image segmentation algorithm or deep learning model to accurately extract temperature anomaly areas, the accuracy of temperature detection data and the automation and high precision of anomaly area extraction are realized. This lays a precise and reliable data foundation for temperature anomaly areas for subsequent spatial fusion of thermal infrared and facade point cloud and dual-dimensional determination of hollow areas.

[0150] Based on any of the above embodiments of this application, Embodiment Eight of this application proposes a method for detecting hollow exterior walls based on the fusion of thermal infrared and facade point cloud data. This can be referred to the above description and will not be repeated hereafter. Based on this, please refer to... Figure 6 , Figure 6 This is a flowchart illustrating Embodiment 8 of the method for detecting hollow exterior walls based on the fusion of thermal infrared and facade point clouds in this application. After step S40, the method for detecting hollow exterior walls based on the fusion of thermal infrared and facade point clouds further includes steps S50-S80:

[0151] Step S50: If the target exterior wall hollowness detection result shows that hollowness exists, determine the target area where hollowness exists.

[0152] In this embodiment, the comprehensive judgment results of the target exterior wall hollowness are retrieved, and the areas determined to have hollowness are screened out. Combining the spatial matching relationship of thermal infrared and deformation dual-dimensional detection, the core boundary and spatial range of the hollow area are accurately defined. Based on the hollowness detection results, the accurate screening and morphological definition of the hollow area are completed.

[0153] Step S60: Associate the target area with the facade laser point cloud model, oblique photography model, or building BIM model of the exterior wall to be inspected.

[0154] In this embodiment, please refer to Figure 7 , Figure 7 This is a schematic diagram of the facade laser point cloud model provided in Embodiment 8 of the exterior wall hollow detection method based on the fusion of thermal infrared and facade point cloud in this application. The facade laser point cloud model, oblique photography model, or building BIM model (Building Information Modeling) of the exterior wall to be detected is retrieved, and the coordinates of the target area are unified to the global coordinate system of the model. Using model spatial mapping and geometric overlay technology, the target area is accurately written back and associated with the selected 3D model, realizing the visual annotation and spatial binding of the target area in the model, ensuring that its position and shape in the model are completely consistent with the actual hollow area of ​​the wall. Accurately writing back and associating the target area with the laser point cloud and other 3D models realizes the visual writing back and spatial binding of the hollow detection results to the building 3D model, transforming the hollow area from abstract detection data into a concrete form that can be intuitively viewed in the model.

[0155] Step S70: Obtain the coordinates, area, range, and risk level of the target area, and generate an external wall hollowing detection report based on the coordinates, area, range, and risk level of the target area.

[0156] In this embodiment, based on the 3D model of the target area, the 3D coordinates, actual physical area, bulge range, and spatial range of the target area are accurately extracted. Combining multi-dimensional indicators such as the bulge range, area, importance of the wall structure, and degree of thermal anomaly, the risk level of each hollow area is determined according to a preset risk level assessment system. Core quantitative parameters such as the 3D coordinates, area, bulge range, and risk level of the hollow areas are integrated, along with infrared images and facade normal visualizations collected during the detection process. The above parameters and image data are structured and formatted to generate a standardized external wall hollow detection report containing quantitative indicators, visualization data, and risk assessment. The accurate extraction of core quantitative parameters such as the 3D coordinates and bulge range of the hollow areas and the completion of risk level determination provide precise engineering basis for subsequent repair and reinforcement of external wall hollow areas. Simultaneously, the integration of multiple types of visualization data and tabular data to generate a standardized report ensures the completeness, standardization, and traceability of the detection results.

[0157] Step S80: In response to the report export command, output the visual export result corresponding to the target external wall hollow detection result.

[0158] In this embodiment, after the user presses the export button or sends an export command, the device internally generates a corresponding report export command, identifying the export requirements in the command, including the export format and the content to be exported. For the export requirements, the standardized test report undergoes format adaptation processing, extracting visual image resources such as infrared images and visible light true-color images, as well as tabular data such as the three-dimensional coordinates, area, bulge range, and risk level of the hollow area. Multiple forms of visual export results are generated, supporting export of the entire report in PDF or Word format, batch export of visual images in JPG or PNG format, and export of quantitative data in Excel or CSV table format. Responding to export commands enables the categorized export of standardized reports, multiple types of visual images, and tabular data, meeting the user's needs in different scenarios and improving the practicality and flexibility of the test results.

[0159] In this embodiment, by accurately locating the target area of ​​hollow walls and associating it with various 3D building models, extracting the core quantitative parameters of the area to complete the risk classification, and generating a standardized inspection report, the entire process of detecting hollow walls is realized, from precise positioning and 3D visualization association to quantitative assessment, standardized output, and multi-format export. This provides accurate, intuitive, standardized, and flexibly applicable complete inspection results for subsequent maintenance, safety assessment, and engineering management of hollow walls.

[0160] For example, to help understand the implementation process of the external wall hollow detection method based on the fusion of thermal infrared and facade point cloud obtained in this embodiment combined with the above embodiments, specifically:

[0161] The external wall hollowness detection equipment may include the following modules, the functions of which can be implemented through a combination of hardware and software: A data acquisition module, equipped with a thermal infrared camera and lidar, such as a drone, tracked vehicle, or lifting platform; a GNSS / IMU navigation unit, used to calculate the sensor's pose in the world coordinate system. A calibration and registration module, used to complete the extrinsic parameter calibration between the thermal infrared camera and lidar, calculating the imaging pose of each frame of thermal infrared image in the point cloud coordinate system based on the extrinsic parameters and flight status, achieving unified spatial registration between the infrared image and the point cloud. An infrared image processing module, including a radiometric calibration and environmental correction submodule, which performs temperature inversion on the original thermal infrared grayscale image based on parameters such as emissivity settings, reflected background temperature, and ambient temperature; and a candidate region detection submodule, which uses threshold segmentation, region growing, or deep learning segmentation / detection networks to extract candidate regions with abnormal temperatures. The facade point cloud processing module performs point cloud preprocessing, including denoising, downsampling, and exterior wall point cloud extraction. This includes an exterior wall subset extraction submodule, which extracts corresponding exterior wall point cloud subsets from the point cloud based on infrared anomaly region pixels and imaging pose; and a plane fitting and exterior wall normal deformation model construction submodule, which performs plane fitting on the point cloud subsets to construct a normal direction DEM. The hollow area discrimination module includes a feature extraction submodule, which extracts thermal features such as temperature difference and temperature gradient from infrared data, and geometric features such as average deformation, maximum deformation, deformation area area, and contour from the exterior wall normal deformation model; and a discrimination strategy submodule, which can use rule-based criteria or machine learning classifiers to determine whether each candidate area is hollow or not, and can further provide a risk level. The visualization and reporting module overlays the detection results onto the 3D point cloud or building model for visualization, generating a detection report including hollow location, scale, bulge amount, and risk level, and supports data and map export.

[0162] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the external wall hollow detection method based on the fusion of thermal infrared and facade point cloud. Any simple modifications based on this technical concept are within the protection scope of this application.

[0163] This application provides an external wall hollow detection device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the external wall hollow detection method based on thermal infrared and facade point cloud fusion in the first embodiment described above.

[0164] The following is for reference. Figure 8 It shows a structural schematic diagram of an external wall hollow detection device suitable for implementing the embodiments of this application. Figure 8 The external wall hollow detection device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0165] like Figure 8 As shown, the external wall hollowness detection device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the external wall hollowness detection device. The processing unit 1001, the read-only memory 1002, and the RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the exterior wall hollow detection equipment to communicate wirelessly or wiredly with other devices to exchange data. Although exterior wall hollow detection equipment with various systems is shown in the figure, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0166] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0167] The external wall hollow detection device provided in this application adopts the external wall hollow detection method based on the fusion of thermal infrared and facade point cloud in the above embodiments, which can solve the technical problem of inaccurate external wall hollow detection results. Compared with the prior art, the beneficial effects of the external wall hollow detection device provided in this application are the same as those of the external wall hollow detection method based on the fusion of thermal infrared and facade point cloud provided in the above embodiments, and other technical features of this external wall hollow detection device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0168] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0169] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0170] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the external wall hollow detection method based on thermal infrared and facade point cloud fusion in the above embodiments.

[0171] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory (EPROM, or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, radio frequency (RF), etc., or any suitable combination thereof.

[0172] The aforementioned computer-readable storage medium may be included in the external wall hollow detection equipment; or it may exist independently and not be assembled into the external wall hollow detection equipment.

[0173] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the external wall hollow detection device, the device causes the following: It acquires a thermal infrared image of the external wall to be detected by a thermal infrared camera, and a facade laser point cloud acquired by a lidar for the external wall to be detected. It identifies temperature anomaly regions in the thermal infrared image, and based on the spatial mapping relationship between the thermal infrared image and the facade laser point cloud, determines the corresponding subset of the external wall point cloud in the facade laser point cloud. It performs reference surface fitting on the subset of external wall point cloud data, constructs a normal deformation model of the external wall reflecting the concave-convex characteristics of the wall surface based on the fitting results, and extracts local deformation anomaly regions from the normal deformation model. It fuses the detection results of the temperature anomaly regions and the local deformation anomaly regions to determine the target external wall hollow detection result.

[0174] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0175] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0176] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0177] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described method for detecting hollow exterior walls based on the fusion of thermal infrared and facade point clouds. This method can solve the technical problem of inaccurate detection results for hollow exterior walls. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the method for detecting hollow exterior walls based on the fusion of thermal infrared and facade point clouds provided in the above embodiments, and will not be elaborated upon here.

[0178] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for detecting hollow areas in exterior walls based on the fusion of thermal infrared and facade point cloud, characterized in that, Applied to unmanned aerial vehicles (UAVs) equipped with thermal infrared cameras, lidar, and GNSS / IMU modules, the method includes: The system acquires thermal infrared images of the exterior wall to be inspected captured by the thermal infrared camera when the UAV flies along a preset acquisition route, and facial laser point clouds captured by the lidar of the exterior wall to be inspected. Determining the temperature anomaly region in the thermal infrared image, and based on the spatial mapping relationship between the thermal infrared image and the facade laser point cloud, determining the corresponding exterior wall point cloud subset in the facade laser point cloud for the temperature anomaly region in the thermal infrared image, includes: obtaining the minimum bounding rectangle of the temperature anomaly region in the thermal infrared image, and using the minimum bounding rectangle as a reference, expanding outward within the region of the thermal infrared image to obtain the extended temperature anomaly region. Obtain the intrinsic parameter matrix of the thermal infrared camera and the extrinsic parameter pose during imaging; Based on the intrinsic parameter matrix, the pixel coordinates of the contour boundary of the temperature anomaly extension region are converted into imaging ray direction vectors in the camera coordinate system. Based on the extrinsic pose, the imaging ray direction vector is converted into a line-of-sight projection ray in the world coordinate system; In the laser point cloud of the facade, the outer point cloud data located within the spatial cone formed by multiple line projection rays is retrieved, and the retrieved point cloud data is determined as a subset of the exterior wall point cloud; A reference surface is fitted to the subset of external wall point cloud data, and an external wall normal deformation model reflecting the concave and convex features of the wall surface is constructed based on the fitting result. This includes obtaining a local external wall reference surface after the reference surface is fitted to the subset of external wall point cloud data. The local external wall reference surface is a planar model or a curved surface model. Based on the geometric features of the local external wall reference surface, a projection reference surface is constructed, and the normal distance residual from each data point in the external wall point cloud subset to the local external wall reference surface is calculated. The projection reference plane is meshed to divide it into multiple regular mesh units; Determine the two-dimensional coordinate indices of all the regular mesh elements on the projection reference plane; The normal distance residuals of the point cloud data mapped to all the regular grid cells are statistically analyzed, and the statistical values ​​of the normal distance residuals are calculated. The statistical values ​​are determined as the normal deformation values ​​of the grid cells, wherein the statistical values ​​include the mean or median. Using the two-dimensional coordinate index as the position parameter and the normal deformation value corresponding to all the mesh elements as the deformation parameter, the external wall normal deformation model is generated. Extract local deformation anomaly regions from the external wall normal deformation model; By combining the detection results of the temperature anomaly area and the local deformation anomaly area, the detection result of the target exterior wall hollowing is determined.

2. The method for detecting hollow exterior walls based on the fusion of thermal infrared and facade point cloud as described in claim 1, characterized in that, The step of determining the target exterior wall hollowness detection result by integrating the detection results of the temperature anomaly area and the local deformation anomaly area includes: Determine the first detection result of the temperature anomaly region and the second detection result of the local deformation anomaly region; By combining the first detection result and the second detection result, the hollow detection result of the target exterior wall is obtained.

3. The method for detecting hollow exterior walls based on the fusion of thermal infrared and facade point cloud as described in claim 2, characterized in that, The steps of determining the first detection result of the temperature anomaly region and the second detection result of the local deformation anomaly region include: The contour of the temperature anomaly region in the infrared image is mapped to the local coordinate system of the local deformation anomaly region to obtain the sub-region corresponding to the temperature anomaly region on the normal deformation model of the outer wall. Extract thermal features of the temperature anomaly region and determine a first detection result of the temperature anomaly region based on the thermal features, wherein the thermal features include at least the average temperature, the maximum temperature gradient within the region, and the temperature difference relative to the background. Extract the geometric features of the sub-region and determine the second detection result of the local deformation anomaly region based on the geometric features. The geometric features include at least the average deformation, maximum deformation, maximum deformation gradient, deformation region area, and deformation region contour.

4. The method for detecting hollow exterior walls based on the fusion of thermal infrared and facade point cloud as described in claim 1, characterized in that, The steps for determining the temperature anomaly region in the thermal infrared image include: The thermal infrared image is subjected to radiometric calibration and environmental correction. The temperature anomaly region in the thermal infrared image is extracted based on a preset image segmentation algorithm or deep learning model.

5. The method for detecting hollow exterior walls based on the fusion of thermal infrared and facade point cloud as described in claim 1, characterized in that, After the step of determining the target exterior wall hollowness detection result by fusing the detection results of the temperature anomaly region and the local deformation anomaly region, the exterior wall hollowness detection method based on the fusion of thermal infrared and facade point cloud further includes: If the target exterior wall hollowness detection result indicates the presence of hollowness, the target area containing hollowness is determined. The target area is associated with the facade laser point cloud model, the oblique photography model, or the building BIM model of the exterior wall to be inspected; Obtain the coordinates, area, range, and risk level of the target area, and generate an external wall hollowing detection report based on the coordinates, area, range, and risk level of the target area. In response to the report export command, the visualized export results corresponding to the target external wall hollowness detection results are output.

6. A device for detecting hollow areas in exterior walls, characterized in that, The external wall hollow detection device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the external wall hollow detection method based on thermal infrared and facade point cloud fusion as described in any one of claims 1 to 5.

7. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the external wall hollow detection method based on thermal infrared and facade point cloud fusion as described in any one of claims 1 to 5.

Images