External facade detection method and system based on unmanned aerial vehicle

Abstract

The invention relates to the field of building detection, in particular to an outer facade detection method and system based on an unmanned aerial vehicle. Acquiring information of a to-be-detected building, and planning a flight path of the unmanned aerial vehicle; controlling the unmanned aerial vehicle to fly according to the planned flight path, and synchronously triggering a visible light camera and an infrared light camera to collect images in a waypoint suspension stage; determining defect information of the building by using a building defect judgment model based on artificial intelligence and judging the acquired image; and generating a defect information report of the building and generating sample data, and training the building defect judgment model based on artificial intelligence by using the sample data. The method is advantaged in that detection efficiency is improved, a structured report containing maintenance priority ranking is automatically generated, model performance is continuously optimized by employing an incremental learning strategy, good positioning precision and typical defect identification accuracy are acquired, high-altitude operation risk and labor cost are substantially reduced, and reliable technical support is provided for building safety operation and maintenance.

Description

Technical Field

[0001] This invention relates to the field of building inspection, and in particular to a method and system for facade inspection based on unmanned aerial vehicles (UAVs). Background Technology

[0002] The safety performance of buildings is threatened by a variety of surface defects. Influenced by factors such as service life, climate conditions, and animal activity, defects such as leakage, cracks, and damage frequently occur on building surfaces. Building facade defects exhibit a high degree of diversity in shape and scale, significant differences in visibility, complex causes, and dynamic development, posing a severe challenge to their detection. Small, weak targets such as minute cracks are easily obscured by the background, leading to high rates of missed detection. The continuity of narrow cracks is difficult to identify, and the fuzzy boundary segmentation accuracy for large-area leaks is low. Furthermore, irregular peeling and contamination can easily be confused with complex wall textures, causing false detections. This makes accurate, efficient, and comprehensive automated identification of various facade defects a complex task. With the rapid development of UAV hardware and software technology, it has been widely applied in the construction field, such as in bridge inspection, tunnel inspection, and indoor building inspection. Existing technologies for detecting defects in building facades have systemic limitations, primarily manifested in their heavy reliance on manual operation, leading to fragmented processes. From drone flight control and data acquisition to defect identification and report generation, each step is independent, requiring frequent manual intervention, resulting in low efficiency and susceptibility to subjective errors. Defect identification accuracy is limited, with insufficient robustness to detecting minute, narrow, large-area defects and those with complex textures. Spatial positioning and decision support are weak; existing research has failed to establish a precise automated mapping system between defect locations and building facades, and the generation of structured reports is highly dependent on manual intervention, failing to provide immediate and quantitative spatial information support for maintenance decisions. Furthermore, the value of historical data remains untapped, lacking a systematic mechanism for storing and reusing inspection results, constituting a technological barrier to automated, high-precision facade health diagnosis. Summary of the Invention

[0003] The purpose of this invention is to solve the aforementioned problems by designing a method and system for facade inspection based on unmanned aerial vehicles (UAVs). To achieve the above objective, this invention provides the following solution:

[0004] A method for facade inspection based on unmanned aerial vehicles (UAVs) includes the following steps:

[0005] S100. Obtain information about the building to be detected and plan the flight path of the drone;

[0006] S200. Controls the UAV to fly according to the planned flight trajectory, and simultaneously triggers visible light and infrared light cameras to acquire images during the hovering phase of waypoints;

[0007] S300. Using an artificial intelligence-based building defect judgment model, the defect information of the building is determined by judging the acquired images;

[0008] S400. Generate a defect information report of the building and generate sample data, and use the sample data to train the artificial intelligence-based building defect judgment model.

[0009] As a further improvement to this technical solution, the specific method for obtaining information about the building to be detected is as follows: receiving the coordinate file or building information model of the building to be detected, obtaining the vertex coordinates of the polygon on the exterior of the building, the building height value and the spatial coordinate system parameters, and detecting whether the vertex coordinates of the polygon on the exterior of the building to be detected constitute a closed polygon, and forming a configuration file after data verification.

[0010] The specific method for planning the flight trajectory of a drone is as follows: based on the configuration file, the set detection resolution, the camera focal length and sensor size of the drone, the flight altitude and distance of the drone are calculated to obtain the initial flight trajectory of the drone; then, combined with the battery capacity and hovering power consumption of the drone, the power update nodes of the drone are planned, and the initial flight trajectory of the drone is divided into continuous sub-flight trajectories according to the power update nodes, and preset waypoints are determined to obtain the final flight trajectory.

[0011] As a further improvement to this technical solution, the specific method of controlling the UAV flight according to the planned flight trajectory and simultaneously triggering visible light and infrared light cameras to acquire images during the hovering phase of the waypoint is as follows: the preset waypoints are parsed in real time through the MAVLink protocol, driving the UAV to fly according to the planned flight trajectory and set speed, and hovering at the hovering waypoint; when the UAV is hovering, the visible light camera and infrared light camera are simultaneously triggered to acquire images, and the pose data of the images are recorded through the RTK-GPS / IMU fusion positioning system to obtain a timestamp-aligned image-pose data packet.

[0012] As a further improvement to this technical solution, the method for training the building defect judgment model based on artificial intelligence is as follows: based on the image data collected under visible light and infrared light conditions, the image information is labeled according to defect type, lighting conditions, acquisition angle, and wall material. The image data is used for training, and then further training is performed with random rotation angle, random tilt angle, brightness perturbation, and size scaling.

[0013] As a further improvement to this technical solution, the training of the AI-based building defect judgment model includes single-modal data training and dual-modal data training. The single-modal data training uses only image data collected under visible light conditions or only image data collected under infrared light conditions. The dual-modal data training uses image data collected under both visible light and infrared light conditions simultaneously, extracting image features from the image data collected under visible light conditions and the image features collected under infrared light conditions respectively, and fusing them element-wise to obtain a feature map. The Focal Loss loss function is used to suppress class imbalance, the Dice Loss image segmentation loss function is used to optimize the continuity of image segmentation boundaries, and the optimal model is selected based on the validation set index.

[0014] As a further improvement to this technical solution, the defect types include cracks, leaks, and holding defects; the acquisition angles include a frontal angle and a side angle of 30° to 60°; and the wall materials include concrete, plaster, ceramic tiles, and stone.

[0015] The random rotation angle is -15° to 15° along the lens optical axis; the random tilt angle is -10° to 10° around the horizontal axis of the image and -5° to 5° around the vertical axis of the image; the brightness disturbance is a linear transformation of the RGB channels between 0.9 and 1.1; and the size scaling is an image resolution scaling between 0.8 and 1.2.

[0016] As a further improvement to this technical solution, an AI-based building defect judgment model is used to determine the defect information of the building by judging the acquired images. Specifically, based on the image data, the AI-based building defect judgment model is used to determine whether the building has a defect, and outputs the center point coordinates, width, height, defect category, and confidence score of the defect bounding box. Based on the planar equation of the building facade and the pose data of the UAV during image acquisition, an affine transformation model is established to generate a linear mapping relationship between the pixel coordinates of the image and the coordinates of the building facade. The vertex coordinates of the bounding box are transformed, and the corresponding building facade coordinates are found through the linear mapping relationship. The building facade coordinates are then corrected using the pose data of the UAV during image acquisition to obtain the corrected building facade coordinates. The area of ​​the defect is calculated using the corrected building facade coordinates.

[0017] As a further improvement to this technical solution, the specific method for generating a defect information report of the building and generating sample data, and using the sample data to train the AI-based building defect judgment model, is as follows: based on building parameters, collected image data, and the defect information of the building, the defect location of the building is mapped onto the plane of the facade, and the type, location, size, and area of ​​the building defect are associated; defects are classified and severity levels are determined according to area size; maintenance risk priority is calculated according to the defect assessment formula, and a high-risk list is output based on the maintenance risk priority; defect information and maintenance suggestions are marked on the CAD elevation drawing of the building; based on the results of the provided three-view comparison interface and manual correction operations, defects with confidence scores lower than T and areas smaller than S are removed; the data retained after removal is packaged into a training set and stored; when the data exceeds m items, the AI-based building defect judgment model is trained using the training set.

[0018] As a further improvement to this technical solution, the defect evaluation formula is as follows:

[0019] Priority=0.35×Area+0.25×Location+0.2×Material+0.2×Type

[0020] Where: Area represents the defect area;

[0021] Location indicates location risk: load-bearing walls = 1.0, non-load-bearing walls = 0.5, decorative components = 0.3;

[0022] Material indicates the risk level of the material: ceramic tile = 1.0, plaster = 0.8, cement = 0.6, and stone = 0.7.

[0023] Type indicates the defect type: hollow = 1.0, broken = 0.9, leaking = 0.8, crack = 0.7;

[0024] The location risk, material risk, and defect type coefficients can all be adjusted.

[0025] A UAV-based facade inspection system includes:

[0026] The detection preparation module is used to generate a planned flight trajectory for the UAV based on the input information of the building to be detected.

[0027] The data construction and training module is used to train an AI-based building defect judgment model based on image data collected under visible light and infrared light conditions.

[0028] The flight control and image acquisition module is used to control the flight of the UAV according to the flight trajectory planned by the detection preparation module, and to simultaneously trigger the visible light and infrared light cameras to acquire images during the waypoint hovering phase.

[0029] The defect detection module is used to determine the defect information of the building by judging the images acquired by the flight control and image acquisition module using the building defect judgment model.

[0030] The data processing module is used to generate a defect information report and sample data for the building. The sample data is input into the data construction and training module as training data for the artificial intelligence-based building defect judgment model. Attached Figure Description

[0031] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.

[0032] Figure 1 This is a flowchart illustrating a method for facade inspection based on unmanned aerial vehicles (UAVs) according to the present invention.

[0033] Figure 2 This is a schematic diagram of the structure of an unmanned aerial vehicle (UAV)-based facade inspection system according to the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0035] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” and “described” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of the described features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0036] The present invention will now be described in detail with reference to the accompanying drawings.

[0037] Example 1

[0038] like Figure 1 The diagram shows a flowchart of a facade inspection method based on unmanned aerial vehicles (UAVs).

[0039] First, information about the building to be inspected needs to be obtained to plan the drone's flight path. This information can be obtained through a CSV table or BIM model. The main required data includes the vertex coordinates, building height, and spatial coordinate system parameters of the polygons formed by the building's facade. These coordinates should be ordered counter-clockwise and form closed polygons. The formation of closed polygons by all vertices ensures no vertices are missed, and the spatial structure formed by these vertices represents the building. The spatial coordinate system can be WGS-84 latitude and longitude or a local engineering coordinate system. After obtaining this data, topology verification is required to ensure that the polygons on each facade of the building have no self-intersections and no zero-area regions, thus ensuring no facade is missed or duplicated. Subsequently, a standardized JSON configuration file containing the building's geometric features and spatial reference system is generated. This file is also linked to CAD drawings or panoramic images for constructing a 3D visualization base. The advantage of using a JSON configuration file is that it is a lightweight data exchange format, completely independent of language text formats, enabling cross-platform data exchange and facilitating human reading, writing, and machine parsing and generation.

[0040] Based on the dimensions of the building's envelope formed by the vertices of the building to be detected, as parsed from the configuration file, and referencing data such as the preset detection resolution, the focal length of the camera carried by the UAV, and the sensor size, the required flight altitude of the UAV is calculated. The Boustrophedon coverage algorithm is used to generate a UAV flight trajectory parallel to the building facade. The UAV flight trajectory is a parallel line, scanning the interior of the building to be detected in one direction. During the flight trajectory's movement, if it encounters points on the building facade that cause changes in the boundary topology of the interior of the building, such as vertices, the area is decomposed into multiple consecutive small cells. Simultaneously, the forward overlap rate and lateral overlap rate are dynamically adjusted according to the curvature of the building facade to ensure that the UAV's flight trajectory covers the entire building without omissions. The forward overlap rate can be set between 70% and 85%, and the lateral overlap rate can be set between 65% and 75%. During the flight trajectory determination process, the spatial coordinates of the UAV's photo pose points and the gimbal pitch angle of the camera carried by the UAV are also optimized to ensure that the camera lens's time axis is perpendicular to the building facade when capturing images. Finally, considering the drone's battery capacity and hovering power consumption model, a determined drone flight trajectory is obtained. This trajectory divides the facade of the building to be inspected into continuous areas and plans the drone's power update nodes to ensure seamless connection of the flight trajectory and full coverage of the building's facade. The drone's power update can be achieved through charging or battery replacement. Battery replacement is more convenient and faster, effectively shortening the inspection time.

[0041] The system controls the drone's flight according to a planned flight path, using the MAVLink protocol for communication between the control unit and the drone to ensure the control unit can guide the drone to fly along the planned path. The control unit analyzes the preset waypoints on the flight path in real time, controlling the drone to fly at a predetermined speed and trajectory, and hovering at hover points to allow the onboard camera to capture images. The MAVLink protocol is a very lightweight, open-source messaging protocol and communication standard specifically designed for drone communication and control, enabling bidirectional data transmission between the control unit and the drone. The drone carries both visible light and infrared cameras. At the hover point, both cameras are simultaneously triggered to acquire images of the building under inspection under both visible and infrared light conditions. Images acquired by the visible light camera are required to be in 4096×2160 format at 30 frames per second, while images acquired by the infrared camera are required to be in 640×512 format at 25 frames per second. Simultaneously, the RTK-GPS / IMU fusion positioning system records the drone's pose data during image acquisition. RTK-GPS / IMU is a combined navigation system comprising an Inertial Measurement Unit (IMU), a Global Navigation Satellite System (GPS), and a Real-Time Kinematics (RTK) system. The IMU measures the UAV's position and attitude, using inertial sensors such as gyroscopes and accelerometers, along with a navigation calculation system. Gyroscopes and accelerometers are the core components; the gyroscope measures the object's angular velocity, and the accelerometer measures its acceleration. Using three sets of gyroscopes and accelerometers, the angular velocity and acceleration of the three degrees of freedom are measured respectively. Integration yields the object's velocity and trajectory in three-dimensional space. The accelerometer measures acceleration in three directions, and the gyroscope measures angular velocity around three axes. Using this data, the UAV's attitude change—the change in attitude from the previous moment—can be calculated. GPS records the position coordinates. The RTK system uses carrier phase differential technology to calculate the coordinates, further obtaining high-precision position information. Finally, a timestamp-aligned image-pose data packet is generated and transmitted to the control unit. The requirements for pose data are: spatial positioning error ≤ 2cm and attitude angle error ≤ 0.5°.

[0042] An AI-based building defect assessment model is used to determine the building's defect information by analyzing the acquired images. Based on a lightweight architecture, the model employs a two-stage inference process using timestamp-aligned image-pose data packets to determine whether the acquired images of the building contain facade defects. The backbone network of this defect assessment model extracts features and outputs three-layer feature maps (80×80, 40×40, and 20×20), along with axially aligned bounding box parameters. These parameters include the image's center point coordinates (x, y) ∈ [0,1], image dimensions (w, h) ∈ [0,1], the defect type, and a confidence score. Defect types include cracks, leaks, and hollow areas. The confidence score represents the reliability of the defect assessment result, ranging from 0 to 1. Only a confidence score greater than 0.5 indicates a reliable assessment result.

[0043] Defects identified by the defect assessment model are located. First, the image position is located. Based on building facade parameters and UAV pose data, a planar equation is constructed, the camera extrinsic parameter matrix [R|t] is calculated, and an affine transformation model is established.

[0044]

[0045] Based on the affine transformation model, a linear mapping lookup table is generated from pixel coordinates to the local elevation coordinate system, where (u, v) represents the pixel coordinates in the image and (x, y) represents the coordinates in the elevation coordinate system.

[0046] After image localization, defects are then located. Based on the image detection results, spatial mapping is performed on each detection box. First, vertex coordinate transformation is performed, mapping the pixel coordinates (ui, vi) of the four vertices of the i detection bounding boxes to the elevation coordinate system (xi, yi) using a lookup table; then, perspective distortion correction is performed based on the UAV pose angle (pitch). ,yaw Calculate the projection scaling factor:

[0047]

[0048] The corrected coordinates (x'i, y'i) = k·(xi, yi) are obtained. Finally, the area of ​​the target is estimated using the inscribed ellipse of the corrected target bounding box.

[0049]

[0050] The final results are the positioning, correction, and area.

[0051] Artificial intelligence-based building defect assessment models can utilize lightweight architectures from the YOLO series, such as YOLOv5n, v8n, v9s, or v10n. Both unimodal and bimodal branch models are constructed. A unimodal model is trained using images acquired under visible light or infrared light conditions alone. A bimodal branch model, on the other hand, is trained using images acquired under both visible and infrared light conditions simultaneously. Training the bimodal branch model requires a dual-channel input layer to independently extract features from visible and infrared images, respectively. These features are then fused element-wise at the Neck layer. The Focal Loss function is used to suppress class imbalance during training. After each batch of training data is fed into the model, a forward propagation is performed to output a predicted value. The Focal Loss function then calculates the difference between the predicted and true values, i.e., the loss value. After obtaining the loss value, the model updates its parameters through backpropagation to reduce the loss between the true and predicted values, causing the model's predicted values ​​to converge towards the true values, thus achieving the learning objective. After training, the model, through backpropagation, has optimized each parameter. The Focal Loss function can use parameters α=0.25, γ=2. The Dice Loss image segmentation loss function is used to optimize the continuity of the segmentation boundary. The Dice Loss image segmentation loss function is used to measure the similarity between two sample sets. The similarity between the foreground pixel set predicted by the model and the foreground pixel set in the ground truth label is calculated by dividing twice the size of the intersection of these two sets by the sum of the sizes of the two sets. The more similar the predicted segmentation map is to the ground truth segmentation map, the larger their intersection, and the closer the Dice Loss is to 1. The parameters of the Dice Loss image segmentation loss function can be smooth=1e. -5 The optimal model is selected based on the mean average accuracy (mAP) of the validation set.

[0052] Image data acquired under visible and infrared light conditions for training the building defect judgment model are labeled according to defect type, lighting conditions, acquisition angle, and wall material. Defect types include cracks, leaks, and hollow areas; acquisition angles include the front angle of the building and side angles of 30° to 60°; wall materials include concrete, plaster, tiles, and stone. After training, a preliminary building defect judgment model is obtained. Further training with random rotation angles, random tilt angles, brightness perturbations, and size scaling is needed to improve the accuracy of the model in judging images acquired by the UAV under different poses and lighting conditions. For random rotation angle training, the training image data is rotated -15° to 15° along the lens optical axis; for random tilt angle training, the training image data is tilted -10° to 10° around the horizontal axis and -5° to 5° around the vertical axis; for brightness perturbation training, the RGB channels of the training image data are linearly varied between 0.9 and 1.1; for size scaling training, the image resolution of the training image data is scaled between 0.8 and 1.2. The final building defect assessment model was obtained.

[0053] Based on building parameters, raw image data collected by drones, spatial projection coordinates and area data of building defects, a spatial indexing mechanism is used to map the defect locations to the plane of the facade, associating the building defect type, location, size and area, and generating distribution heatmaps and pie charts according to defect category and severity level. The standard for classifying defects by area can be set as follows: area <0.1m² is mild, 0.1-0.5m² is moderate, and >0.5m² is severe.

[0054] The identified defects are graded according to the following defect assessment formula:

[0055] Priority=0.35×Area+0.25×Location+0.2×Material+0.2×Type

[0056] Where: Area represents the defect area;

[0057] Location indicates location risk: load-bearing walls = 1.0, non-load-bearing walls = 0.5, decorative components = 0.3;

[0058] Material indicates the risk level of the material: ceramic tile = 1.0, plaster = 0.8, cement = 0.6, and stone = 0.7.

[0059] Type indicates the defect type: hollow = 1.0, broken = 0.9, leaking = 0.8, crack = 0.7;

[0060] Location risk, material risk, and defect type coefficients can all be adjusted.

[0061] This defect assessment formula comprehensively considers the impact of defect area, location, material, and type when evaluating maintenance risk priority. A larger defect area indicates a more severe defect; defect area is the core risk indicator, reflecting the scope of the defect's impact. Location risk reflects the impact of the defect's location. If it's a load-bearing wall, it has a significant impact on the overall safety of the building, while defects in decorative components have a smaller impact. Material risk reflects the hazards caused by falling debris and the potential for subsequent defect expansion. If the facade is made of tiles, defects could cause them to fall, posing a safety hazard to pedestrians. Plaster also carries the risk of falling. In contrast, cement and stone have a lower risk of falling due to their better material integrity. Defect type reflects the severity and ease of maintenance of different types of defects. Hollow spots and broken surfaces pose a lower risk and are relatively easy to maintain, while leaks and cracks pose a higher risk and are more difficult to maintain. Based on the calculated risk levels, a high-risk list is output, and defect information and maintenance recommendations are marked on the building's CAD elevation drawing.

[0062] Interactive verification of output defect information is provided through a web-based three-view comparison interface. The original images captured by the drone are overlaid with bounding boxes generated by AI detection at the pixel level. Users can adjust the vertex positions of the bounding boxes by dragging. When the vertex displacement exceeds a 5-pixel threshold, an automatic annotation update mechanism is triggered to ensure the traceability of manual corrections. Data cleaning is then performed. A dual filtering rule is applied to remove low-reliability detection results with a confidence score below T, as these may be false positives. Microscopic defects with a projected area smaller than S are also excluded, as these are considered system noise. The confidence score T can be set to 0.6, and the defect projected area S can be set to 0.01 m², retaining high-quality positive samples that have been verified. The verified valid defect data is packaged into a standardized training sample package, containing timestamp-aligned visible-infrared paired image data, normalized center coordinates, and bounding box labels with width and height, and stored in a standardized structure in the database. When the cumulative number of new samples in the database exceeds 1000, incremental training of the building defect judgment model is automatically triggered.

[0063] Example 2

[0064] Figure 2 The diagram shows a structural schematic of an exterior facade inspection system based on an unmanned aerial vehicle (UAV) according to the present invention.

[0065] This drone-based facade inspection system includes:

[0066] The detection preparation module is used to generate a planned flight trajectory for the UAV based on the input information of the building to be detected.

[0067] The data construction and training module is used to train an AI-based building defect judgment model based on image data collected under visible light and infrared light conditions.

[0068] The flight control and image acquisition module is used to control the flight of the UAV according to the flight trajectory planned by the detection preparation module, and to simultaneously trigger the visible light and infrared light cameras to acquire images during the waypoint hovering phase.

[0069] The defect detection module is used to determine the defect information of the building by judging the images acquired by the flight control and image acquisition module using the building defect judgment model.

[0070] The data processing module is used to generate a defect information report and sample data for the building. The sample data is input into the data construction and training module as training data for the artificial intelligence-based building defect judgment model.

[0071] The detection preparation module includes an information preparation submodule and a trajectory planning submodule. The information preparation submodule receives the coordinate file or building information model of the building to be detected and forms a configuration file after data verification. The trajectory planning submodule determines the flight trajectory based on the configuration file, the set detection resolution, the camera focal length and sensor size of the UAV, combined with the battery capacity and hovering power consumption of the UAV.

[0072] The data construction and training module, based on image data acquired under visible and infrared light conditions, labels the image information according to defect type, lighting conditions, acquisition angle, and wall material, and performs preliminary training on the building defect judgment model. Then, it further trains the building defect judgment model by random rotation angle, random tilt angle, brightness perturbation, and size scaling.

[0073] The flight control and image acquisition module includes an automatic flight submodule and an image acquisition submodule. The automatic flight submodule drives the UAV to fly at a set speed by parsing preset waypoints in real time through the MAVLink protocol and hovers at the waypoints. The image acquisition submodule synchronously triggers the visible light camera and infrared light camera to acquire images while the UAV is hovering, and obtains image-pose data packets with timestamp alignment.

[0074] The defect detection module includes an image detection submodule, an image localization submodule, and a defect localization submodule. The image detection submodule uses a building defect judgment model to determine whether a building has a defect based on the image data. The image localization submodule generates a linear mapping relationship between the pixel coordinates of the image and the coordinates of the building facade. The defect localization submodule obtains the corrected coordinates of the building defect through the linear mapping relationship and calculates the area of ​​the defect.

[0075] The data processing module includes a report generation submodule, a result verification submodule, and an incremental training data generation submodule. The report generation submodule annotates defect information and maintenance suggestions on the building's CAD elevation drawing based on building parameters, collected image data, and building defect information. The result verification submodule provides a three-view comparison interface and manual correction tools to remove defects with confidence scores below T and areas smaller than S. The incremental training data generation submodule encapsulates the remaining data after removal into a training set and stores it as incremental training data for the building defect judgment model.

[0076] In summary, this invention proposes a UAV-based facade inspection method and system, constructing a fully automated method for detecting facade defects using UAVs. Through multimodal data fusion and intelligent planning technology, it achieves autonomous flight and precise data acquisition by the UAV. In the defect identification stage, a lightweight single-modal / multimodal detection network architecture is designed. Models are selected optimally based on application scenarios, significantly improving the detection accuracy of typical defects such as cracks and hollow areas under different application scenarios. The system establishes a complete spatial mapping system, realizing the spatial localization of defects. Taking into account factors such as defect area, location criticality, and material properties, it automatically generates a structured report including maintenance priority ranking. An incremental learning strategy is used to continuously optimize model performance, enabling the system's detection capability to continuously improve over time. Compared with traditional detection methods, this invention improves detection efficiency, achieves good positioning accuracy and typical defect identification accuracy, while significantly reducing the risks of high-altitude operations and labor costs, providing reliable technical support for building safety maintenance.

Claims

1. A method for facade inspection based on unmanned aerial vehicles (UAVs), characterized in that, Includes the following steps: S100. Obtain information about the building to be detected and plan the flight path of the drone; S200. Controls the UAV to fly according to the planned flight trajectory, and simultaneously triggers visible light and infrared light cameras to acquire images during the hovering phase of waypoints; S300. Using an artificial intelligence-based building defect judgment model, the defect information of the building is determined by judging the acquired images; S400. Generate a defect information report of the building and generate sample data, and use the sample data to train the artificial intelligence-based building defect judgment model.

2. The facade inspection method based on unmanned aerial vehicles according to claim 1, characterized in that, The specific method for obtaining information about the building to be detected is as follows: receiving the coordinate file or building information model of the building to be detected, obtaining the vertex coordinates of the polygon on the exterior of the building, the building height value and the spatial coordinate system parameters, and detecting whether the vertex coordinates of the polygon on the exterior of the building to be detected constitute a closed polygon. After data verification, a configuration file is formed. The specific method for planning the flight trajectory of a drone is as follows: based on the configuration file, the set detection resolution, the camera focal length and sensor size of the drone, the flight altitude and distance of the drone are calculated to obtain the initial flight trajectory of the drone; then, combined with the battery capacity and hovering power consumption of the drone, the power update nodes of the drone are planned, and the initial flight trajectory of the drone is divided into continuous sub-flight trajectories according to the power update nodes, and preset waypoints are determined to obtain the final flight trajectory.

3. The facade inspection method based on unmanned aerial vehicles according to claim 1, characterized in that, The specific method for controlling the UAV flight according to the planned flight trajectory and simultaneously triggering visible light and infrared light cameras to acquire images during the hovering phase at waypoints is as follows: the preset waypoints are parsed in real time through the MAVLink protocol, driving the UAV to fly according to the planned flight trajectory and set speed, and hovering at the hovering waypoints; when the UAV is hovering, the visible light camera and infrared light camera are simultaneously triggered to acquire images, and the pose data of the images are recorded through the RTK-GPS / IMU fusion positioning system to obtain a timestamp-aligned image-pose data packet.

4. The facade inspection method based on unmanned aerial vehicles according to claim 1, characterized in that, The method for training an AI-based building defect judgment model is as follows: based on image data collected under visible light and infrared light conditions, the image information is labeled according to defect type, lighting conditions, acquisition angle, and wall material. The image data is then used for training, and further training is performed with random rotation angle, random tilt angle, brightness perturbation, and size scaling.

5. The facade inspection method based on unmanned aerial vehicles according to claim 4, characterized in that, The training of the AI-based building defect judgment model includes single-modal data training and dual-modal data training; the single-modal data training uses only image data collected under visible light conditions, or only image data collected under infrared light conditions. The dual-modal data training uses image data acquired under visible light conditions and image data acquired under infrared light conditions for training. Image features of the image data acquired under visible light conditions and the image features of the image data acquired under infrared light conditions are extracted respectively, and feature maps are obtained by adding them element by element. Focal Loss loss function is used to suppress class imbalance, Dice Loss image segmentation loss function is used to optimize the continuity of image segmentation boundaries, and the optimal model is selected based on validation set indicators.

6. The facade inspection method based on unmanned aerial vehicles according to claim 4, characterized in that, The defect types include cracks, leaks, and hollow areas; the acquisition angles include frontal angles and side angles of 30° to 60°; the wall materials include concrete, plaster, ceramic tiles, and stone. The random rotation angle is -15° to 15° along the lens optical axis; the random tilt angle is -10° to 10° around the horizontal axis of the image and -5° to 5° around the vertical axis of the image; the brightness perturbation is a linear transformation of the RGB channels between 0.9 and 1.1; the size scaling is an image resolution scaling between 0.8 and 1.

2.

7. The facade inspection method based on unmanned aerial vehicles according to claim 1, characterized in that, The specific method for determining the defect information of a building using an AI-based building defect judgment model by judging the acquired images is as follows: Based on the image data, the AI-based building defect judgment model is used to determine whether the building has a defect, and outputs the center point coordinates, width, height, defect category, and confidence score of the defect bounding box; Based on the planar equation of the building facade and the pose data of the UAV when acquiring the images, an affine transformation model is established to generate a linear mapping relationship between the pixel coordinates of the image and the coordinates of the building facade; The vertex coordinates of the bounding box are transformed, and the corresponding building facade coordinates are found through the linear mapping relationship; The building facade coordinates are then corrected using the pose data of the UAV when acquiring the images to obtain the corrected building facade coordinates; The area of ​​the defect is calculated using the corrected building facade coordinates.

8. The facade inspection method based on unmanned aerial vehicles according to claim 1, characterized in that, The specific method for generating a defect information report of the building and generating sample data, and using the sample data to train the artificial intelligence-based building defect judgment model, is as follows: based on the building parameters, the collected image data, and the defect information of the building, the defect location of the building is mapped onto the plane of the facade, and the type, location, size and area of ​​the building defect are associated. Defects are classified and their severity levels are determined by their area size; Calculate the maintenance risk priority according to the defect assessment formula, and output a high-risk list based on the maintenance risk priority; mark the defect information and maintenance suggestions on the CAD elevation drawing of the building. Based on the results of the provided three-view comparison interface and manual correction, defects with confidence scores below T and areas less than S are removed. The remaining data after removal is packaged into a training set and stored. When the data exceeds m, the artificial intelligence-based building defect judgment model is trained using the training set.

9. The facade inspection method based on unmanned aerial vehicles according to claim 8, characterized in that, The defect assessment formula is as follows: Priority=0.35×Area+0.25×Location+0.2×Material+0.2×Type Where: Area represents the defect area; Location indicates location risk: load-bearing walls = 1.0, non-load-bearing walls = 0.5, decorative components = 0.3; Material indicates the risk level of the material: ceramic tile = 1.0, plaster = 0.8, cement = 0.6, and stone = 0.

7. Type indicates the defect type: hollow = 1.0, broken = 0.9, leaking = 0.8, crack = 0.7; The location risk, material risk, and defect type coefficients can all be adjusted.

10. A UAV-based facade inspection system, using the UAV-based facade inspection method as described in any one of claims 1 to 9, characterized in that, include: The detection preparation module is used to generate a planned flight trajectory for the UAV based on the input information of the building to be detected. The data construction and training module is used to train an AI-based building defect judgment model based on image data collected under visible light and infrared light conditions. The flight control and image acquisition module is used to control the flight of the UAV according to the flight trajectory planned by the detection preparation module, and to simultaneously trigger the visible light and infrared light cameras to acquire images during the waypoint hovering phase. The defect detection module is used to determine the defect information of the building by judging the images acquired by the flight control and image acquisition module using the building defect judgment model. The data processing module is used to generate a defect information report and sample data for the building. The sample data is input into the data construction and training module as training data for the artificial intelligence-based building defect judgment model.

Images