Unmanned aerial vehicle building intelligent inspection method and system

Abstract

The invention provides an unmanned aerial vehicle building intelligent inspection method and system, and relates to the technical field of image processing and computer vision, and the method comprises the step of autonomously planning an inspection path through reinforcement learning based on a BIM model, environmental data and a historical defect thermodynamic diagram. A multi-source sensor is used for collecting data, and local dense point cloud is constructed through space-time registration. A tight coupling graph SLAM is adopted to fuse BIM priori constraints, and a global dense point cloud is generated. A fine-grained three-dimensional grid is obtained through hierarchical Transform, and defect segmentation and classification are realized by using a multi-scale HRNet network. And finally, the defect information is mapped to a BIM model, a semantic enhanced BIM is generated, and accurate data support is provided for building operation and maintenance.

Description

Technical Field

[0001] The present invention relates to the field of image processing and computer vision technology, and in particular to a method and system for intelligent building inspection by unmanned aerial vehicles. Background Art

[0002] The exterior walls of urban buildings are primarily categorized as decorated tiled walls, glass curtain walls, and undecorated cement mortar plaster walls. The exterior walls of ordinary high-rise residential buildings and other low-rise buildings are primarily decorated with tiles or cement mortar plaster, while super-high-rise office buildings are primarily decorated with glass curtain walls. Regardless of the type of exterior wall, it gradually ages. Exterior wall facing tiles can hollow and collapse due to adhesive materials and construction defects, while glass curtain walls and cement mortar exterior walls can crack and peel due to rain, strong winds, and earthquakes. Exterior wall inspections have become a crucial part of daily building maintenance. Hidden dangers such as hollowing and cracking in building exterior walls are a core risk to public safety. To ensure structural safety and extend the life of a building, daily maintenance requires significant manpower and material resources each year.

[0003] To improve inspection efficiency and accuracy, drone technology is gradually being introduced into intelligent building exterior wall inspections. Drone technology significantly enhances inspection efficiency and flexibility, particularly in high-altitude, complex environments, effectively overcoming the limitations of manual inspections.

[0004] Existing technical solutions, such as the authorized Chinese patent application CN202410709447.X, propose a method and system for intelligent inspection of building exterior walls using drones. This solution addresses the challenge of achieving a balance between inspection efficiency and recognition accuracy. However, this solution still presents some challenges. In particular, the multi-source heterogeneous data acquired during the inspection process is difficult to integrate and understand with high precision. This results in insufficient reliability of the final defect identification results, potential distortion of the evaluation results, and the possibility of missed detections and misjudgments. This makes it impossible to guarantee the effectiveness and accuracy of the inspection results, making it impossible to provide effective early warnings and maintenance decisions in a timely manner, ultimately exacerbating public safety risks for building exterior walls. Summary of the Invention

[0005] In order to overcome the shortcomings of the above-mentioned prior art, the main purpose of the present invention is to provide a drone building intelligent inspection method and system, which can provide more reliable and accurate building exterior wall inspection results through precise synchronization of multi-source data, optimized data fusion technology and more detailed analysis.

[0006] To achieve the above objectives, the present invention adopts the following technical solutions, a drone building intelligent inspection method, comprising the following steps: Based on the building BIM model of the building to be inspected, real-time environmental perception data, and historical defect distribution heat map, a reinforcement learning model is used to generate drone inspection routes for the building to be inspected. Synchronously collecting building facade data through multi-source sensors carried by drones, performing spatiotemporal registration on the collected building facade data to obtain synchronized multi-source data, and constructing a local dense point cloud model of the building facade; An improved tightly coupled graph SLAM fusion BIM model with structural prior constraints is introduced to register and fuse the local dense point cloud model of the building facade to obtain a global dense point cloud model of the building exterior wall. Based on the global building exterior wall dense point cloud model, a fine-grained three-dimensional mesh model of the building exterior wall is obtained through a hierarchical Transformer network; Constructing a multi-task deep learning model, using a multi-scale HRNet network and an edge-aware loss function, to segment and classify defect areas on the fine-grained three-dimensional mesh model of the building exterior wall, and obtaining defect classification results; Based on the spatial position of the fine-grained three-dimensional grid model of the building exterior wall, mapping the defect classification results to grid vertices to generate a fine-grained three-dimensional defect model with semantic labels; The semantically labeled fine-grained three-dimensional defect model is semantically aligned and information integrated with the building BIM model to generate a semantically enhanced BIM model containing building exterior wall defect detection information as the final drone building intelligent inspection result.

[0007] The generation of a drone inspection route for a building to be inspected using a reinforcement learning model includes: A state space is constructed using the semantic information of the building BIM model of the building to be inspected, the real-time environmental perception data, and the historical defect distribution heat map; the state space includes a drone state vector and an environmental state vector, the drone state vector includes: position coordinates, attitude angle, battery power, and current sensor mode; the environmental state vector includes: current area material type, wind speed, light intensity, obstacle distance, and historical defect density; The action space is obtained by defining the drone's action set and material-sensitive actions. When a highly reflective material is detected, the trigger action is to lower the flight altitude and activate the polarization camera. When a low-texture area is detected, the trigger action is to increase the lateral flight path and activate the lidar. Design a multi-objective reward function: ; Among them, Rcover is the area coverage reward, which is obtained based on the coverage ratio of the grid cells divided by the BIM model; Rquality is the defect detection reward, which is generated by weighting the historical defect density H; Rsafe is the safety constraint reward, which penalizes actions such as obstacle distance or wind speed; Reff is the efficiency reward, which is negatively correlated with the battery consumption rate; α, β, γ, and δ are dynamic weight coefficients, which are adaptively adjusted according to the mission stage. The policy network is trained through a deep reinforcement learning algorithm to obtain the optimal action sequence, which serves as the drone inspection route for the building to be inspected.

[0008] The multi-source sensor includes a visible light camera, an infrared thermal imager, and a laser radar, and the synchronous collection of building facade data includes: Collecting high-definition images of building facades through the visible light camera; Collecting building facade temperature distribution data by the infrared thermal imager; Collecting three-dimensional point cloud data of the building facade by the laser radar; Using an FPGA module to perform time stamp synchronization and spatial registration on the high-definition image, temperature distribution data, and three-dimensional point cloud data to obtain synchronized multi-source data; Performing spatiotemporal registration on the collected building facade data includes: Performing coarse registration on the multi-source sensor data based on a feature point matching algorithm to obtain coarsely registered multi-source data; Using an iterative closest point algorithm to perform fine registration on the coarsely registered multi-source data to obtain finely registered data; The registered multi-source data are unified into the local coordinate system to generate spatiotemporally aligned synchronized multi-source data.

[0009] The method of obtaining a global building exterior wall dense point cloud model includes: According to the local dense point cloud model, the BIM model structural prior constraints and the obtained initial pose sequence, pose nodes, local point cloud block nodes, point cloud matching edges, BIM constraint edges and motion constraint edges are obtained; Taking pose nodes and local point cloud block nodes as the node layer, and point cloud matching edges, BIM constraint edges, and motion constraint edges as the edge layer, a graph optimization model is constructed to obtain a graph structure that integrates multi-source constraints. The graph optimization model jointly optimizes pose estimation and geometric constraints through a dynamic weight allocation mechanism, where the weights are adaptively adjusted according to the point cloud matching error to obtain an optimized global pose sequence; Performing multi-resolution registration of the optimized global pose sequence and the local dense point cloud model based on the optimized pose on the local point cloud, and generating a global dense point cloud by weighted voxel fusion to obtain a global dense point cloud initial model; The topological relationship between the global dense point cloud initial model and the BIM model is used to structurally complete the occluded area, and semantic labels are given to the geometric components to obtain a global building exterior wall dense point cloud model containing geometric and semantic information.

[0010] The pose node represents the pose parameters of the drone, including position coordinates and attitude quaternion; The local point cloud node is associated with the local point cloud data block with reference to the pose node; The point cloud matching edge calculates the registration error between adjacent point cloud blocks through the ICP algorithm; The BIM constraint edge generates geometric constraints based on plane equations, symmetry rules and component topological relationships extracted from the BIM model; The motion constraint edge constructs relative motion constraints between adjacent poses through IMU pre-integration.

[0011] Obtaining the defect classification result includes: The multi-scale HRNet network is used as the backbone network, and the multi-scale HRNet network includes four sets of parallel resolution branches, a cross-resolution feature fusion module and a dual-task output layer; Input the backbone network of the fine-grained three-dimensional mesh model of the building exterior wall, extract multi-scale geometric and texture features, aggregate features through a cross-resolution feature fusion module, and generate 256-dimensional fused features; Inputting the fused features into a dual-task output layer, wherein the dual-task output layer includes a segmentation task of outputting a vertex-level defect region probability map and a classification task of outputting a vertex-level defect region probability map, wherein the defect types include cracks, spalling, leakage, and hollowing; The total loss is obtained by using the edge-aware loss function, which is a combination of cross entropy loss and edge gradient difference loss with a weight of 7:3. The edge gradient difference loss obtains the gradient difference of the defect boundary through the Sobel operator. Based on the total loss training model, a vertex-level defect type classification result is obtained.

[0012] The defect classification results are mapped to mesh vertices including: According to the patch-level segmentation result, for each patch, obtain all mesh vertices in the coverage area of ​​the patch; The defect type classification result and confidence of each patch are assigned to all mesh vertices in the area covered by the patch as vertex semantic labels; a fine-grained three-dimensional defect model containing vertex coordinates, normal vectors, patch texture mapping coordinates and defect labels is generated.

[0013] The semantic alignment and information integration include: Performing inter-model spatial registration between the fine-grained three-dimensional defect model and the building BIM model based on the ICP algorithm to obtain a spatial transformation matrix; Converting the fine-grained three-dimensional defect model to a BIM model coordinate system using the spatial transformation matrix; Associating the defective vertex with the corresponding building component instance in the BIM model according to the spatial position of the defective vertex converted to the BIM model coordinate system; Defect information associated with building component instances is written into the BIM model in the form of an attribute set to generate a semantically enhanced BIM model.

[0014] An unmanned aerial vehicle (UAV) intelligent building inspection system, comprising: The inspection data preprocessing module is used to generate drone inspection routes for buildings to be inspected using a reinforcement learning model based on the building BIM model, real-time environmental perception data, and historical defect distribution heat maps of the buildings to be inspected. The module also uses multi-source sensors onboard drones to synchronously collect building facade data, performs spatiotemporal registration on the collected building facade data, obtains synchronized multi-source data, and constructs a local dense point cloud model of the building facade. A fusion processing module is used to introduce the structural prior constraints of the improved tightly coupled graph SLAM fusion BIM model, align and fuse the local dense point cloud model of the building facade, and obtain a global building exterior wall dense point cloud model; based on the global building exterior wall dense point cloud model, a hierarchical Transformer network is used to obtain a fine-grained three-dimensional mesh model of the building exterior wall; a multi-task deep learning model is constructed, and a multi-scale HRNet network and an edge-aware loss function are used to segment and classify defect areas on the fine-grained three-dimensional mesh model of the building exterior wall to obtain a defect classification result; The inspection data analysis module is used to map the defect classification results to grid vertices based on the spatial position of the fine-grained three-dimensional grid model of the building exterior wall, and generate a fine-grained three-dimensional defect model with semantic labels; semantically align and integrate the fine-grained three-dimensional defect model with semantic labels with the building BIM model, and generate a semantically enhanced BIM model containing building exterior wall defect detection information as the final drone building intelligent inspection result.

[0015] Compared with the prior art, the present invention has the following beneficial effects: This invention achieves a high degree of automation throughout the entire process, from inspection route planning, data collection, processing and analysis to result generation. This significantly reduces reliance on manual experience and improves the efficiency and consistency of inspection work. Furthermore, through synchronous acquisition by multi-source sensors, tightly coupled SLAM fusion with BIM prior knowledge, and refined recognition based on deep learning, it constructs precise digital models from the macro to the micro level, greatly improving the accuracy and reliability of defect detection. Furthermore, it further breaks down the data barriers from inspection to operation and maintenance. The resulting "semantic-enhanced BIM model" not only contains geometric information but also deeply integrates defect semantic information, achieving seamless integration of inspection results with design and operation and maintenance management, providing a strong data foundation and decision-making support for the intelligent and refined operation and maintenance of buildings.

[0016] The present invention specifically integrates BIM semantics, environmental perception, and historical defect data, dynamically generates optimized inspection paths through reinforcement learning, significantly improves coverage efficiency and defect detection rate, and can adaptively adjust flight and sensing strategies based on material properties. Furthermore, through hardware synchronization and improved registration algorithms, precise spatiotemporal alignment of multi-source heterogeneous data is achieved, providing a high-quality, consistent, synchronized multi-source data foundation for subsequent processing. To obtain more accurate inspection results, a more precise three-dimensional model is required. By introducing BIM structural prior constraints through tightly coupled graph SLAM, pose estimation is effectively optimized, cumulative errors are eliminated, and a high-precision global dense point cloud and fine-grained grid model containing rich semantic information is constructed. Furthermore, to achieve more accurate defect identification, a multi-scale HRNet and edge-aware loss function are used to achieve automated, high-precision segmentation and classification of multiple defects on a three-dimensional grid with high accuracy. Furthermore, the present invention has an efficient closed-loop information integration system. By generating a semantically enhanced BIM model, defect detection results are accurately associated with building components, forming a closed-loop digital asset from detection to operation and maintenance decision-making, greatly improving the intelligence level and efficiency of building operation and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0018] Figure 1 It is a schematic diagram of the overall process framework of the present invention.

[0019] Figure 2 It is a flow chart of inspection route generation in the present invention.

[0020] Figure 3 It is a schematic diagram of the multi-source data synchronous acquisition process in the present invention.

[0021] Figure 4 It is a schematic diagram of the process of global point cloud reconstruction in the present invention.

[0022] Figure 5 It is a flowchart of defect identification in the present invention.

[0023] Figure 6 It is a schematic diagram of the defect result mapping process in the present invention.

[0024] Figure 7 It is a schematic diagram of the semantic BIM integration process in the present invention.

[0025] Figure 8 It is a schematic diagram of the system composition block in the present invention. DETAILED DESCRIPTION

[0026] The application of drone technology in building exterior wall inspections has effectively improved the efficiency and scope of inspections. However, the core bottleneck of existing technical solutions, such as Chinese patent application number CN202410709447.X, lies in the difficulty of high-precision fusion and collaborative understanding of the multi-source heterogeneous data acquired during the inspection process. This results in insufficient reliability of the final defect identification results and an inability to effectively integrate with the digital operation and maintenance system.

[0027] Specifically, due to the vibration of the drone platform and differences in the operating timing of various sensors, there are spatiotemporal synchronization errors between visible light, infrared thermal imaging, and lidar data, resulting in distortion in the registration of three-dimensional point clouds and image data. At the same time, the simultaneous localization and mapping (SLAM) process in large-scale scenarios is prone to cumulative errors and lacks effective utilization of prior building structural information, making it difficult to generate accurate and reliable global three-dimensional models. Defect identification based on this distorted and unreliable model will inevitably reduce its accuracy and reliability. The identification results are mostly in the form of isolated reports, which cannot be automatically semantically integrated with the building information model (BIM). This creates "data islands" from inspection to operation and maintenance, greatly limiting the practical application value of intelligent inspections.

[0028] Therefore, how to fundamentally solve the distortion problem of multi-source data fusion and build an accurate, reliable three-dimensional defect model that can be deeply integrated with BIM has become the most prominent and urgent core technical problem in this field.

[0029] In order to overcome the shortcomings of the above-mentioned prior art, the main purpose of the present invention is to provide a drone building intelligent inspection method and system, which can provide more reliable and accurate building exterior wall inspection results through precise synchronization of multi-source data, optimized data fusion technology and more detailed analysis.

[0030] To achieve the above purpose, the present invention adopts the following technical solutions: UAV building intelligent inspection method, see Figure 1 , including the following steps: Based on the building BIM model of the building to be inspected, real-time environmental perception data, and historical defect distribution heat map, a reinforcement learning model is used to generate drone inspection routes for the building to be inspected. The multi-source sensors carried by drones synchronously collect building facade data, perform spatiotemporal registration on the collected building facade data, obtain synchronized multi-source data, and construct a local dense point cloud model of the building facade; By introducing the structural prior constraints of the improved tightly coupled graph SLAM fusion BIM model, the local dense point cloud model of the building facade is registered and fused to obtain the global dense point cloud model of the building exterior wall. Based on the global building exterior wall dense point cloud model, a fine-grained 3D mesh model of the building exterior wall is obtained through a hierarchical Transformer network. A multi-task deep learning model was constructed, using a multi-scale HRNet network and an edge-aware loss function to segment and classify defect areas in a fine-grained 3D mesh model of a building's exterior wall, obtaining defect classification results. Based on the spatial position of the fine-grained 3D mesh model of the building exterior wall, the defect classification results are mapped to the mesh vertices to generate a fine-grained 3D defect model with semantic labels; The semantically labeled fine-grained three-dimensional defect model is semantically aligned and integrated with the building BIM model to generate a semantically enhanced BIM model containing building exterior wall defect detection information as the final drone building intelligent inspection result.

[0031] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0032] Example 1: This example conducts drone intelligent inspections on the main stadium of a large sports center. Figures 1 to 8 The building is a mixture of reinforced concrete and steel structure, with a large glass curtain wall, metal decorative panels and concrete structure. The building is 45 meters high and has a total facade area of ​​approximately 85,000 square meters. The building is located in a coastal city with a wind speed of 4.5m / s and a temperature of 18 ° C, 75% humidity, and variable light conditions.

[0033] The drone system used for this inspection is equipped with: Canon EOS R5 full-frame visible light camera, with a resolution of 8192×5464 pixels and 3 frames per second; FLIR T1040 infrared thermal imager, resolution 1024×768 pixels, thermal sensitivity ≤0.03 ° C; Velodyne VLS-128 lidar, which emits 1.2 million points per second and has a detection range of 200 meters; Built-in high-precision integrated navigation system, IMU sampling frequency 500Hz, positioning accuracy ±1cm; and Xilinx UltraScale+ FPGA synchronous processing units.

[0034] The system is based on the Revit BIM model of the building to be inspected. The specific file size is 2.1GB, which contains 12 major types of components, 35 material properties, real-time environmental data, and a heat map of the defect distribution in the past five years. It records the locations and types of 327 defects and plans the inspection route through a reinforcement learning model.

[0035] The state space dimension is 28 dimensions, including the drone's position (XYZ), attitude including roll, pitch, yaw, remaining battery power of 85%; sensor operating mode; environmental status including the current facade material type such as glass, metal, concrete, real-time wind speed, light intensity (78,000 lux), the nearest obstacle distance of 12.5m and the historical defect density of 0.38 / m2.

[0036] The action space defines 17 discrete actions, including: encountering a highly reflective glass curtain wall triggers a 2.5m descent and activates the polarizing filter camera; encountering a low-texture concrete area triggers a 3m lateral movement and increases the lidar scanning frequency. The dynamic weight coefficients in the multi-objective reward function are initially set to α=0.35, β=0.25, γ=0.20, and δ=0.20, and are adaptively adjusted as the mission progresses. The policy network is trained using the PPO algorithm, ultimately generating a 38.6km inspection route consisting of 12,450 waypoints and an estimated flight time of 4.2 hours.

[0037] During the data collection phase: The visible light camera collected 43,620 high-definition images, with a raw data volume of 4.7TB; The infrared thermal imager collected 10,240 frames of temperature distribution data, with a data volume of 168GB; The lidar generates 2.15 billion three-dimensional point cloud data, with a data volume of 3.4TB.

[0038] The FPGA hardware synchronization system achieves a multi-source data timestamp synchronization accuracy of ±0.2ms. A SuperPoint feature point algorithm is used for coarse registration, extracting 385,000 matching point pairs with an initial registration error of ±6.8cm. A generalized ICP algorithm is then used for fine registration, ultimately reducing the registration error to ±0.7cm. All data is uniformly converted to a local coordinate system with the northeast corner of the building as the origin, generating a synchronized multi-source dataset that is temporally and spatially aligned.

[0039] The constructed local dense point cloud model of the building facade contains 215 local blocks with an average point density of 28,000 points per square meter. In the tightly coupled graph SLAM optimization model: 12,450 pose nodes were established, each containing a 3D position and quaternion pose; 215 local point cloud block nodes; point cloud matching edges were generated by calculating the constraints of adjacent point cloud blocks through ICP, generating a total of 428 edges; BIM constraint edges were generated based on 86 plane equations, 32 sets of symmetry rules and 572 component topological relationships extracted from the Revit model; motion constraint edges generated 12,449 edges through IMU pre-integration.

[0040] A dynamic weighting mechanism was employed, with an initial point cloud matching error weight of 0.6, a BIM constraint weight of 0.3, and a motion constraint weight of 0.1. After graph optimization, global pose estimation accuracy reached ±1.2 cm, with a pose error of ±0.3°. Through multi-resolution registration and weighted voxel fusion, an initial global dense point cloud with 3.24 billion points was generated. Leveraging the topological relationships of the BIM model, 183 occluded areas were structuredly completed, resulting in a point cloud of approximately 42 million points. Ultimately, a dense point cloud model of the global building exterior wall, encompassing both geometric and semantic information, was obtained.

[0041] A 12-layer hierarchical Transformer network with a feature dimension of 1024 and 16 attention heads was used to process the point cloud data and generate a fine-grained 3D mesh model of the building's exterior wall. The mesh contains 12.3 million triangles, with an average facet area of ​​0.0069 square meters and 6.15 million vertices.

[0042] The multi-task deep learning model uses HRNet-48 as the backbone network and takes as input mesh vertex coordinates, normal vectors, and RGB texture information. The network consists of four parallel resolution branches with resolutions of 1 / 4, 1 / 8, 1 / 16, and 1 / 32, generating a 256-dimensional fused feature through cross-resolution feature fusion. The dual-task output layer simultaneously outputs: Vertex-level defect segmentation probability map with a resolution of 8192×8192; The vertex-level defect classification results include four categories: cracks, spalling, leakage, and hollowing.

[0043] The training data consisted of 25,400 labeled samples, including 8,200 cracks, 7,100 spallings, 6,300 leaks, and 3,800 hollows. An edge-aware loss function was used, with a cross-entropy loss weight of 0.7 and a Sobel operator-calculated edge gradient difference loss weight of 0.3. The final model achieved 94% accuracy and 92% recall on the test set.

[0044] This inspection identified: Cracks: 283, maximum length 3.2m, average width 2.1mm, confidence level 0.96; Spalling: 197 locations, maximum area 0.86 m², average depth 1.5 cm, confidence level 0.93; Leakage: 156 locations, with a maximum area of ​​2.4 m² and a confidence level of 0.91; Hollows: 82, with a maximum area of ​​1.2m² and a confidence level of 0.89.

[0045] The patch-level defect classification results were mapped to mesh vertices, with each patch covering an average of 8.7 vertices, for a total of 10.7 million vertices assigned semantic labels. The resulting fine-grained 3D defect model contained information such as vertex coordinates, normal vectors, RGB texture values, defect type, and confidence level, totaling 8.3GB of data.

[0046] The ICP algorithm is used for model registration, with a registration error of ±0.9cm. After the defect model is converted to the BIM coordinate system through the space transformation matrix, the defect vertices are associated with the component instances in the BIM model: There were 183 defects associated with glass curtain wall components, 297 defects associated with metal decorative panels, and 238 defects associated with concrete structures. All defect information was written into the BIM model as attribute sets, generating a semantically enhanced BIM model in IFC format with a file size of 3.8GB.

[0047] The inspection took 3.5 hours and identified 718 defects, achieving an overall detection accuracy of 94%. Compared to a traditional manual inspection, which is expected to take three weeks, this represents a 40-fold improvement in efficiency. The resulting semantically enhanced BIM model will provide accurate data support for the sports center's subsequent maintenance and renovation, potentially saving 35% in maintenance costs.

[0048] Example 2: This example conducts drone intelligent inspections on a super high-rise building in a financial center. Figures 1 to 8 The building is a steel structure with a reinforced concrete core tube structure. The facade is a combination of unitized glass curtain wall and granite veneer panels. The building is 632 meters high and has a total facade area of ​​approximately 225,000 square meters. The building is located in the urban core area. During the inspection, the wind speed was 3.8m / s and the temperature was 22 ° C, humidity 68%, slight electromagnetic interference.

[0049] The drone system used for this inspection is equipped with: Phase One iXU-RS1000 multispectral camera, with a resolution of 10,100 × 11,320 pixels and 5-channel spectral imaging, including RGB, near-infrared, and red-edge bands; InfraTec ImageIR 8300 infrared thermal imager, resolution 1280 × 1024 pixels, frame rate 500 Hz, temperature measurement accuracy ± 0.02 ° C; Leica BLK2FLY autonomous flying lidar, emitting 360,000 points per second and with a ranging accuracy of ±1mm; Septentrio AsteRx-i3 GNSS / IMU integrated navigation system, supporting multiple frequencies and constellations, with a positioning accuracy of ±0.5cm; And the Xilinx Versal ACAP adaptive computing acceleration platform, which integrates FPGA and AI engine.

[0050] Based on the Bentley BIM model of the building to be inspected, the file size is 4.8GB, containing 23 major component categories, 8,542 component instances, 67 material properties, real-time environmental data, and a heat map of defect distribution over the past eight years. It records the locations and types of 1,285 defects and uses a reinforcement learning model to collaboratively plan the inspection routes of three drones.

[0051] The state space dimension is 42 dimensions, which includes the joint state of the drone swarm, including relative position, power distribution, task load, and environmental dynamic parameters, including real-time wind field model, electromagnetic interference intensity, and light change gradient.

[0052] Sensor operating mode; environmental status includes the current facade material type, such as Low-E glass, granite, and aluminum panel, real-time wind speed, light intensity of 125,000 lux, nearest obstacle distance of 15.8 meters, and historical defect density of 0.42 defects per square meter.

[0053] The action space defines 32 coordinated actions, including: encountering a highly reflective Low-E glass curtain wall triggers a 3.2m descent and activates multispectral imaging mode; encountering complex decorative moldings triggers a three-dimensional orbital flight and increases LiDAR scanning density. The dynamic weight coefficients in the multi-objective reward function are initially set to α=0.38, β=0.28, γ=0.19, and δ=0.15, and are adaptively adjusted as the mission progresses. The policy network is trained using the SAC algorithm, ultimately generating a 126.8km inspection route consisting of 38,450 waypoints and an estimated flight time of 8.5 hours.

[0054] Data collection phase: The multispectral camera collected 58,620 high-definition images, with a raw data volume of 12.8TB; The infrared thermal imager collected 24,580 frames of temperature distribution data, with a data volume of 2.1TB; The lidar generates 5.86 billion three-dimensional point cloud data, with a data volume of 9.2TB.

[0055] The FPGA hardware synchronization system achieves a multi-source data timestamp synchronization accuracy of ±0.1ms. The LoFTR feature matching algorithm is used for coarse registration, extracting 1.268 million matching point pairs with an initial registration error of ±5.2cm. The NDT algorithm is then used for fine registration, ultimately reducing the registration error to ±0.5cm. All data is uniformly converted to a local coordinate system centered on the building center, generating a synchronized, spatiotemporally aligned multi-source dataset.

[0056] The constructed local dense point cloud model of the building facade contains 586 local blocks with an average point density of 36,000 points per square meter. In the tightly coupled graph SLAM optimization model: 38,450 pose nodes were established; 586 local point cloud block nodes; The point cloud matching edges are constrained by calculating adjacent point cloud blocks through GICP, generating a total of 1172 edges; BIM constraint edges are generated based on 156 plane equations, 68 sets of symmetry rules, and 892 component topological relationships extracted from the Bentley model; The motion constraint edges generate 38,449 edges through IMU pre-integration.

[0057] A dynamic weighting mechanism was employed, with an initial point cloud matching error weight of 0.65, a BIM constraint weight of 0.25, and a motion constraint weight of 0.10. After graph optimization, global pose estimation accuracy reached ±0.8cm, with an attitude error of ±0.2°. Through multi-resolution registration and weighted voxel fusion, an initial global dense point cloud with 8.64 billion points was generated. Leveraging the topological relationships of the BIM model, structured completion was performed on 356 occluded areas, resulting in a point cloud of approximately 120 million points. Ultimately, a dense point cloud model of the global building exterior wall, containing both geometric and semantic information, was obtained.

[0058] A 16-layer hierarchical Transformer network with a feature dimension of 2048 and 24 attention heads was used to process the point cloud data and generate a fine-grained 3D mesh model of the building's exterior wall. The mesh contains 38.6 million triangles, with an average facet area of ​​0.0058 square meters and 19.3 million vertices.

[0059] The multi-task deep learning model uses HRNet-64 as its backbone network and takes as input mesh vertex coordinates, normal vectors, and multispectral texture information. The network consists of four parallel branches with resolutions of 1 / 4, 1 / 8, 1 / 16, and 1 / 32, generating a 512-dimensional fused feature map through cross-resolution feature fusion. The dual-task output layer simultaneously outputs: a vertex-level defect segmentation probability map with a resolution of 16384×16384; and vertex-level defect classification results, specifically cracks, spalling, leakage, hollowing, and corrosion.

[0060] The training data contained 68,500 labeled samples, including 18,200 cracks, 16,800 spallings, 15,600 leaks, 9,400 hollows, and 8,500 corrosions. An improved edge-aware loss function was used, with a weighted cross-entropy loss of 0.75 and an adaptive edge gradient difference loss of 0.25. The final model achieved 96.2% accuracy and 94.8% recall on the test set.

[0061] This inspection identified: Cracks: 428, maximum length 4.8m, average width 1.8mm, confidence level 0.97, detection accuracy 97.2%; Spalling: 286 locations, with a maximum area of ​​1.26 m², an average depth of 2.1 cm, a confidence level of 0.95, and a detection accuracy of 96.5%; Leakage: 198 locations, with a maximum area of ​​3.2m², a confidence level of 0.93, and a detection accuracy of 95.8%; Hollows: 125, with a maximum area of ​​1.8m², a confidence level of 0.90, and a detection accuracy of 94.3%; Corrosion: 87 locations, with a maximum area of ​​2.4m², confidence level 0.88, and detection accuracy 93.1%.

[0062] The patch-level defect classification results were mapped to mesh vertices, with each patch covering an average of 9.2 vertices, for a total of 35.5 million vertices assigned semantic labels. The resulting fine-grained 3D defect model contained vertex coordinates, normal vectors, multispectral texture values, defect type, and confidence level, totaling 25.6GB of data.

[0063] An improved ICP algorithm was used for model registration, achieving a registration error of ±0.6 cm. After converting the defect model to the BIM coordinate system using a spatial transformation matrix, defect vertices were associated with component instances in the BIM model: 362 defects were associated with the glass curtain wall components, 418 defects were associated with the granite veneer, and 144 defects were associated with the metal decoration. All defect information was written into the BIM model as an attribute set, generating a semantically enhanced BIM model in IFC format with a file size of 6.4 GB.

[0064] The inspection took a total of 7.8 hours, identified 1,124 defects, and achieved an overall detection accuracy of 96.2%. This represents a 50-fold improvement in efficiency compared to a traditional manual inspection, which would have taken an estimated six weeks. The resulting semantically enhanced BIM model will provide accurate data support for the safe operation and maintenance of the super-high-rise building, potentially saving 42% in maintenance costs and increasing maintenance efficiency by 3.5 times.

[0065] This example demonstrates the high-precision and high-efficiency detection capabilities of the drone intelligent inspection system in complex large-scale public buildings, providing a complete technical solution for the intelligent operation and maintenance of important infrastructure.

[0066] It should be noted that, in the present invention, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0067] The above embodiments are merely examples of the present invention and do not limit the scope of protection of the present invention. Any designs that are identical or similar to the present invention fall within the scope of protection of the present invention.

Claims

1. UAV building intelligent inspection method, characterized by: The following steps are involved: Based on the building BIM model of the building to be inspected, real-time environmental perception data, and historical defect distribution heat map, a reinforcement learning model is used to generate drone inspection routes for the building to be inspected. Synchronously collecting building facade data through multi-source sensors carried by drones, performing spatiotemporal registration on the collected building facade data to obtain synchronized multi-source data, and constructing a local dense point cloud model of the building facade; An improved tightly coupled graph SLAM fusion BIM model with structural prior constraints is introduced to register and fuse the local dense point cloud model of the building facade to obtain a global dense point cloud model of the building exterior wall. Based on the global building exterior wall dense point cloud model, a fine-grained three-dimensional mesh model of the building exterior wall is obtained through a hierarchical Transformer network; Constructing a multi-task deep learning model, using a multi-scale HRNet network and an edge-aware loss function, to segment and classify defect areas on the fine-grained three-dimensional mesh model of the building exterior wall, and obtaining defect classification results; Based on the spatial position of the fine-grained three-dimensional grid model of the building exterior wall, mapping the defect classification results to grid vertices to generate a fine-grained three-dimensional defect model with semantic labels; The semantically labeled fine-grained three-dimensional defect model is semantically aligned and information integrated with the building BIM model to generate a semantically enhanced BIM model containing building exterior wall defect detection information as the final drone building intelligent inspection result.

2. The drone building intelligent inspection method according to claim 1, characterized in that: The generation of a drone inspection route for a building to be inspected using a reinforcement learning model includes: A state space is constructed using the semantic information of the building BIM model of the building to be inspected, the real-time environmental perception data, and the historical defect distribution heat map; the state space includes a drone state vector and an environmental state vector, the drone state vector includes: position coordinates, attitude angle, battery power, and current sensor mode; the environmental state vector includes: current area material type, wind speed, light intensity, obstacle distance, and historical defect density; Define the drone's action set and material-sensitive actions to obtain an action space. When a highly reflective material is detected, the trigger action is to lower the flight altitude and activate the polarization camera. When a low-texture area is detected, the trigger action is to increase the lateral flight path and activate the lidar. Design a multi-objective reward function, expressed as: ； in, Rcover For regional coverage bonus, the coverage ratio of grid cells divided by BIM model is obtained; Rquality Defect detection reward, based on historical defect density H Weighted generation; Rsafe To constrain rewards for safety, penalize actions based on obstacle distance or wind speed; Reff is the efficiency reward, which is negatively correlated with the battery consumption rate; α, β, γ, δ are dynamic weight coefficients, which are adaptively adjusted according to the task stage; The policy network is trained through a deep reinforcement learning algorithm to obtain the optimal action sequence, which serves as the drone inspection route for the building to be inspected.

3. The intelligent inspection method for buildings using drones as claimed in claim 1, wherein: The multi-source sensor includes a visible light camera, an infrared thermal imager, and a laser radar, and the synchronous collection of building facade data includes: Collecting high-definition images of building facades through the visible light camera; Collecting building facade temperature distribution data by the infrared thermal imager; Collecting three-dimensional point cloud data of the building facade by the laser radar; Using an FPGA module to perform time stamp synchronization and spatial registration on the high-definition image, temperature distribution data, and three-dimensional point cloud data to obtain synchronized multi-source data; Performing spatiotemporal registration on the collected building facade data includes: Performing coarse registration on the multi-source sensor data based on a feature point matching algorithm to obtain coarsely registered multi-source data; Using an iterative closest point algorithm to perform fine registration on the coarsely registered multi-source data to obtain finely registered data; The registered multi-source data are unified into the local coordinate system to generate spatiotemporally aligned synchronized multi-source data.

4. The intelligent inspection method for buildings using drones as claimed in claim 1, wherein: The method of obtaining a global building exterior wall dense point cloud model includes: According to the local dense point cloud model, the BIM model structural prior constraints and the obtained initial pose sequence, pose nodes, local point cloud block nodes, point cloud matching edges, BIM constraint edges and motion constraint edges are obtained; Taking pose nodes and local point cloud block nodes as the node layer, and point cloud matching edges, BIM constraint edges, and motion constraint edges as the edge layer, a graph optimization model is constructed to obtain a graph structure that integrates multi-source constraints. The graph optimization model jointly optimizes pose estimation and geometric constraints through a dynamic weight allocation mechanism, where the weights are adaptively adjusted according to the point cloud matching error to obtain an optimized global pose sequence; Performing multi-resolution registration of the optimized global pose sequence and the local dense point cloud model based on the optimized pose on the local point cloud, and generating a global dense point cloud by weighted voxel fusion to obtain a global dense point cloud initial model; The topological relationship between the global dense point cloud initial model and the BIM model is used to structurally complete the occluded area, and semantic labels are given to the geometric components to obtain a global building exterior wall dense point cloud model containing geometric and semantic information.

5. The intelligent inspection method for buildings using a drone as claimed in claim 4, wherein: The pose node represents the pose parameters of the drone, including position coordinates and attitude quaternion; The local point cloud node is associated with the local point cloud data block with reference to the pose node; The point cloud matching edge calculates the registration error between adjacent point cloud blocks through the ICP algorithm; The BIM constraint edge generates geometric constraints based on plane equations, symmetry rules and component topological relationships extracted from the BIM model; The motion constraint edge constructs relative motion constraints between adjacent poses through IMU pre-integration.

6. The intelligent inspection method for buildings using a drone as claimed in claim 5, wherein: Obtaining the defect classification result includes: The multi-scale HRNet network is used as the backbone network, and the multi-scale HRNet network includes four sets of parallel resolution branches, a cross-resolution feature fusion module and a dual-task output layer; Input the backbone network of the fine-grained three-dimensional mesh model of the building exterior wall, extract multi-scale geometric and texture features, aggregate features through a cross-resolution feature fusion module, and generate 256-dimensional fused features; Inputting the fused features into a dual-task output layer, wherein the dual-task output layer includes a segmentation task of outputting a vertex-level defect region probability map and a classification task of outputting a vertex-level defect type label, wherein the defect types include cracks, spalling, leakage, and hollowing; The total loss is obtained by using the edge-aware loss function, which is a combination of cross entropy loss and edge gradient difference loss with a weight of 7:

3. The edge gradient difference loss obtains the gradient difference of the defect boundary through the Sobel operator. Based on the total loss training model, a vertex-level defect type classification result is obtained.

7. The drone building intelligent inspection method according to claim 1, characterized in that: The semantic alignment and information integration include: Performing inter-model spatial registration between the fine-grained three-dimensional defect model and the building BIM model based on the ICP algorithm to obtain a spatial transformation matrix; Converting the fine-grained three-dimensional defect model to a BIM model coordinate system using the spatial transformation matrix; Associating the defective vertex with the corresponding building component instance in the BIM model according to the spatial position of the defective vertex converted to the BIM model coordinate system; Defect information associated with building component instances is written into the BIM model in the form of an attribute set to generate a semantically enhanced BIM model.

8. A system using the drone intelligent building inspection method according to any one of claims 1 to 7, characterized in that: include: The inspection data preprocessing module is used to generate a drone inspection route for the building to be inspected based on the building BIM model, real-time environmental perception data, and historical defect distribution heat map of the building to be inspected through a reinforcement learning model; the module uses the multi-source sensors on the drone to synchronously collect building facade data, performs spatiotemporal registration on the collected building facade data, obtains synchronized multi-source data, and constructs a local dense point cloud model of the building facade; A fusion processing module is used to introduce the structural prior constraints of the improved tightly coupled graph SLAM fusion BIM model, align and fuse the local dense point cloud model of the building facade, and obtain a global building exterior wall dense point cloud model; based on the global building exterior wall dense point cloud model, a hierarchical Transformer network is used to obtain a fine-grained three-dimensional mesh model of the building exterior wall; a multi-task deep learning model is constructed, and a multi-scale HRNet network and an edge-aware loss function are used to segment and classify defect areas on the fine-grained three-dimensional mesh model of the building exterior wall to obtain a defect classification result; The inspection data analysis module is used to map the defect classification results to grid vertices based on the spatial position of the fine-grained three-dimensional grid model of the building exterior wall, and generate a fine-grained three-dimensional defect model with semantic labels; semantically align and integrate the fine-grained three-dimensional defect model with semantic labels with the building BIM model, and generate a semantically enhanced BIM model containing building exterior wall defect detection information as the final drone building intelligent inspection result.

9. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images