A bim-based unmanned aerial vehicle perception house safety appraisal management system and method

Abstract

The application discloses a kind of unmanned aerial vehicle based on BIM and perceives house safety appraisal management system and method, it is related to data processing technical field;The application is generated by training U-Net++ model, based on three-dimensional BIM model of house, by space grid sampling algorithm is combined with convex hull optimization algorithm, generates preset track;Unmanned aerial vehicle carries dual-mode sensing pod and flies along preset track, collects image virtual list and metadata;Image sequence is carried out denoising enhancement processing, utilizes BIM feature point projection to guide SIFT feature bidirectional matching to realize precision alignment, after Euclidean distance threshold verification, input pre-training U-Net++ model generates crack binary mask, and is converted into physical crack parameter;Through SfM point cloud and ICP registration technology, the angle of inclination of wall is calculated;Integrate crack parameter, inclination angle and BIM attribute, and automatically trigger graded early warning through risk rule base;Realize the full-process automation from data acquisition, crack identification to risk assessment, significantly improve detection accuracy and efficiency, effectively reduce high-risk missed detection rate.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, and specifically relates to a BIM-based UAV-based building safety assessment and management system and method. Background Technology

[0002] In the field of building operation and maintenance and safety monitoring, problems such as cracks and tilting caused by the aging of existing buildings are becoming increasingly prominent, especially for high-rise buildings and historical and cultural buildings that require regular safety assessments. While the industry has introduced drones and BIM technology, these are mostly used independently: drone-collected image data requires manual comparison with drawings to locate damage, which is inefficient; BIM models serve only as static databases and cannot be linked to on-site inspection results in real time. Furthermore, automatic crack identification algorithms have a high false detection rate on complex wall materials (such as textured stone and soiled surfaces), while structural deformation analysis typically requires the deployment of sensors, which is costly and has limited coverage.

[0003] Current building safety assessments mainly rely on manual visual inspections or basic drone patrols, which suffer from problems such as low efficiency of manual inspections, high risks of high-altitude operations, and difficulty in accurately quantifying crack parameters; data collected by ordinary drones is disconnected from building information models, leading to inaccurate damage location; changes in lighting affect image quality, making it easy to miss cracks; structural tilt measurement relies on equipment such as total stations, which are complex to operate and cannot be linked with crack information for analysis; existing methods lack intelligent fusion mechanisms for multi-source data, making it difficult to achieve automated risk assessment. Summary of the Invention

[0004] (a) Technical problems to be solved

[0005] To address the problems in related technologies, this invention provides a BIM-based UAV-based method for assessing and managing building safety, thereby overcoming the aforementioned technical issues in existing related technologies.

[0006] (II) Technical Solution

[0007] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution:

[0008] S1. Train the U-Net++ model to obtain a pre-trained U-Net++ model;

[0009] S2. Construct a 3D BIM model of the building and plan the flight path of the drone to obtain the preset trajectory.

[0010] S3. The UAV flies along a preset flight path, collects data during the flight, and returns to home and re-flies when an abnormal data is detected, until the flight ends, and obtains metadata and image sequences.

[0011] S4. Denoise and enhance the image sequence to obtain an enhanced image sequence; extract the features of the enhanced image sequence to obtain SIFT image features;

[0012] A bidirectional matching operation is performed based on the SIFT image features to obtain a set of valid matching point pairs; the accuracy is then verified based on the set of valid matching point pairs.

[0013] S5. After the accuracy verification is passed, calculate the parameters of the crack binary mask obtained by combining the enhanced image sequence with the pre-trained U-Net++ model to obtain the crack parameter set.

[0014] S6. Register the high-precision point cloud constructed based on the enhanced image sequence and metadata to obtain the registered point cloud; calculate the house tilt angle based on the registered point cloud.

[0015] S7. Based on the 3D BIM model of the building, the crack parameter set, and the building tilt angle, combined with the risk rule base, a comprehensive risk result is determined.

[0016] This invention deeply integrates BIM models, UAV perception, and artificial intelligence algorithms; improves crack detection rate through pre-trained U-Net++ models; ensures data integrity in complex environments by utilizing BIM-driven autonomous UAV flight paths and dual-mode adaptive acquisition; achieves accurate crack location and tilt detection through bidirectional matching and high-precision point cloud registration; and automatically generates risk results by integrating a risk rule base, improving the detection efficiency of single high-rise buildings, reducing the high-risk missed detection rate, and realizing full-process automation and high-precision quantitative analysis of building safety assessment.

[0017] Preferably, step S1 includes the following steps:

[0018] S11. Construct the U-Net++ model; set the initial kernel size and number of initial convolution kernels for the U-Net++ model;

[0019] S12. Collect crack images containing different lighting and material scenes to obtain historical crack images; perform mask annotation on the images to obtain annotated historical crack images;

[0020] The U-Net++ model is trained using labeled historical crack images. During the training process, the optimal combination of kernel size and number of kernels for the U-Net++ model is found to obtain the optimal combination. The kernel size and number of kernels in the optimal combination are used as the kernel size and number of kernels for the U-Net++ model to obtain the pre-trained U-Net++ model.

[0021] This invention collects and annotates crack images from multiple scenes for training, and finds the optimal combination of convolutional kernel parameters for U-Net++. This significantly improves the model's adaptability to complex materials and lighting changes, increases the accuracy of crack identification, and lays the algorithmic foundation for subsequent precise crack quantification.

[0022] Preferably, step S2 includes the following steps:

[0023] S21. Collect CAD drawings and construction material data of the building to obtain building data; import the building data into BIM parametric modeling software to obtain a 3D BIM model of the building; the 3D BIM model of the building includes a vertex set, an edge set, and an attribute set; the vertex set includes all geometric vertices of the building model, the edge set includes all edges of the building model, and the attribute set includes the attributes of the building model.

[0024] S22. Key points are set on the exterior facade of the building using a spatial grid sampling algorithm to obtain a key point set for the building facade; the key point set for the building facade includes all key points on the building facade.

[0025] Based on the key point set of the building facade, the API interface of the BIM software is called to automatically extract the coordinates of the four corners of the window openings, the intersection points of decorative lines, and the points where pipelines penetrate the wall from the vertex set, edge set, and attribute set of the building facade, thus obtaining the feature point set of the building facade; the feature point set of the building facade contains the feature point data of each key point of the facade in the key point set of the building facade.

[0026] S23. Based on the feature point set of the building's exterior facade, perform UAV flight path planning for the building to obtain the executable UAV trajectory file;

[0027] This invention constructs a BIM model containing geometric attributes and component information using CAD data. Based on spatial grid sampling, it automatically extracts key feature points such as window openings and pipelines, and drives a drone to generate a spiral trajectory that precisely fits the building's outer contour. This eliminates errors from manual measurement, achieves blind-spot scanning, and avoids collision risks.

[0028] Preferably, step S23 includes the following steps:

[0029] S231. Based on the attribute set in the 3D BIM model of the building, set the safety distance; based on the outer envelope cube of the 3D BIM model of the building and the safety distance, obtain the safe flight space range of the UAV;

[0030] S232. Set the layer height interval; based on the safe flight space range of the UAV and the layer height interval, the building is vertically divided into multiple layers to obtain the layer height sequence;

[0031] S233. Set the overlap rate threshold and hovering conditions; use the convex hull optimization algorithm to extract the minimum convex polygon of the outer contour of each layer in the layered height sequence;

[0032] A closed loop path is generated around the building by extending the safety distance along the minimum convex polygon; based on the closed loop path around the building, the UAV camera's field of view is used to calculate the flight strip spacing, ensuring that the overlap rate of adjacent flight strips is greater than the overlap rate threshold, thus obtaining the current layer's horizontal loop track point set;

[0033] S234. Based on the single-layer circular track point set and hovering conditions, obtain the single-layer track function set with timestamps;

[0034] The single-layer track function set with timestamps is connected in order from bottom to top to form a spiral upward path. The transition between layers is achieved by vertical climbing, resulting in a complete three-dimensional track function.

[0035] The complete three-dimensional trajectory function is discretized into a time-position-attitude command sequence, encoded into a format recognizable by the flight control system, and thus a preset trajectory that the UAV can execute is obtained;

[0036] This invention automatically sets safety distances based on BIM model attributes, extracts the minimum outer contour of the building layer by layer through a convex hull optimization algorithm and expands it to generate a circular path, dynamically calculates the flight path spacing based on the camera's field of view, and inserts hovering commands at corners; finally, a timestamped spiral ascent track is formed, and flight control executable commands are output through discretization encoding, realizing centimeter-level seamless scanning of the building facade, avoiding collision risks and ensuring the integrity of image acquisition.

[0037] Preferably, step S3 includes the following steps:

[0038] S31. The UAV is equipped with a multi-sensor pod; the multi-sensor pod includes a visible light camera, an infrared thermal imager, and an RTK positioning module.

[0039] S32. Set the acquisition cycle, light intensity threshold, and offset threshold; the UAV flies along the preset trajectory, and in each acquisition cycle, it collects image data and POS data through the multi-sensor pod; during the flight, it monitors the light intensity and UAV offset in real time, and when the light intensity is lower than the light intensity threshold, it adaptively switches to infrared mode for shooting; when the UAV offset is greater than the offset threshold, it triggers automatic return to home and re-flying until the flight ends, and obtains metadata and image sequences;

[0040] This invention, by equipping a dual-mode camera (visible light / infrared) and an RTK positioning module, monitors the environment and attitude in real time during flight. When the light is insufficient, it automatically switches to infrared mode to ensure data acquisition. If the deviation exceeds a threshold, it triggers a return-to-home and re-flying mechanism, ensuring positioning accuracy, image data and metadata integrity, and eliminating blind spots for manual intervention.

[0041] Preferably, step S4 includes the following steps:

[0042] S41. Median filtering is used to eliminate impulse noise in the image sequence, resulting in a denoised image sequence.

[0043] The denoised image sequence was enhanced using CLAHE enhancement, and contrast adaptive histogram equalization was constrained to improve local image contrast, resulting in an enhanced image sequence.

[0044] S42. Using the improved SIFT algorithm, extract image feature points from the enhanced image sequence to obtain SIFT image features;

[0045] Based on the real-time pose of the UAV and the intrinsic parameter matrix of the camera in the metadata, the set of BIM feature points of the building is projected onto the image plane to obtain the spatial projection of the building.

[0046] Based on SIFT image features, building spatial projection, building 3D BIM model and enhanced image, a bidirectional matching operation is performed to retain bidirectional consistent matching point pairs and obtain a set of effective matching point pairs.

[0047] The bidirectional matching operation includes forward matching and reverse matching; the forward matching is to search for SIFT image features in the neighborhood of the projection point in the spatial projection of the house in the enhanced image, and the reverse matching is to back-project the SIFT image features to the 3D BIM model of the house to verify consistency.

[0048] The set of valid matching point pairs contains all valid matching point pairs; each valid matching point pair contains the coordinates of building BIM feature points and the coordinates of enhanced image pixels.

[0049] S43. Project the coordinates of the building BIM feature points in the effective matching point pair set onto the image plane according to the current pose in the metadata to obtain the desired pixel coordinates.

[0050] Calculate the Euclidean distance between the desired pixel coordinates and the enhanced image pixel coordinates to obtain the Euclidean distance set;

[0051] Calculate the average value of the Euclidean distance set to obtain the average error; extract the maximum value from the Euclidean distance set to obtain the maximum error;

[0052] Set the average error threshold and the maximum error threshold; if the average error is less than the average error threshold and the maximum error is less than the maximum error threshold, the accuracy verification passes; otherwise, the accuracy verification fails and returns to S3.

[0053] This invention eliminates noise and improves local image contrast through median filtering and CLAHE enhancement, and extracts feature points using an improved SIFT algorithm. Based on BIM feature point projection, it guides bidirectional matching (forward neighborhood search and reverse BIM verification) to select consistent point pairs. Finally, it uses dual-threshold Euclidean distance verification (average error < average error threshold, maximum error < maximum error threshold) to ensure precise alignment between the image and BIM, providing a geometric benchmark for accurate crack location. If the verification fails, data resampling is triggered.

[0054] Preferably, step S5 includes the following steps:

[0055] S51. After the accuracy verification is passed, the enhanced image sequence is input into the pre-trained U-Net++ model to obtain the crack binary mask; the coordinate position of the crack binary mask and the coordinates in the effective matching point pair are fused to obtain the location of the house crack;

[0056] S52. Perform closing operations and skeleton extraction operations on the binary mask of the crack to obtain the optimized crack skeleton map; calculate the total pixel length of the skeleton chain in the optimized crack skeleton map; convert the total pixel length of the skeleton chain into physical length.

[0057] S53. Take a normal section along the skeleton line of the optimized crack skeleton map and calculate the number of crack pixels in the section; convert the number of crack pixels in the section into physical width.

[0058] The location, physical length, and width of the cracks in the building together constitute the crack parameter set;

[0059] This invention outputs a binary crack mask using a pre-trained U-Net++ model, and combines it with effective matching point pairs to achieve accurate crack location in the BIM model. It optimizes crack morphology by employing closing operations and skeleton extraction, calculates pixel size along the skeleton line algorithm to the cross section, and converts pixel length and width into physical size using a physical conversion formula based on camera parameters to eliminate image distortion errors. Finally, it outputs a quantized crack parameter set that integrates spatial location, length, and width, providing an accurate basis for safety assessment.

[0060] Preferably, step S6 includes the following steps:

[0061] S61. Based on sequences and metadata, the SfM algorithm is used to generate high-precision point clouds;

[0062] S62. Use ICP registration to register the high-precision point cloud with the building BIM 3D model to obtain the registered point cloud;

[0063] S63. Perform RANSAC plane segmentation on the registered point cloud to obtain the interior point set; calculate the covariance matrix of the interior point set.

[0064] Extract the minimum eigenvalue of the covariance matrix of the interior point set; calculate the eigenvector of the minimum eigenvalue to obtain the actual normal vector;

[0065] The angle between the actual normal vector and the BIM reference normal vector is calculated using the formula for calculating the angle between spatial vectors, thus obtaining the building tilt angle.

[0066] This invention generates a high-precision point cloud of the wall surface using the SfM algorithm, and accurately aligns it with the BIM model through ICP registration. It extracts the internal point set using RANSAC segmentation, calculates the actual normal vector based on the eigenvector decomposition of the covariance matrix, and finally outputs a high-precision tilt angle through the spatial angle formula with the BIM reference normal vector, realizing non-contact quantitative detection of structural deformation and providing key deformation data for building safety assessment.

[0067] Preferably, step S7 includes the following steps:

[0068] S71. Construct a risk rule base; the risk determination rules in the risk rule base are determined based on the building's construction attributes, the building's tilt angle, and the building's tilt angle.

[0069] S72. Based on the attribute set, crack parameter set, and building tilt angle in the 3D BIM model of the building, combined with the risk rule base, a comprehensive risk result is determined.

[0070] This invention constructs a multi-dimensional risk rule base (such as crack width + location + tilt angle), automatically associates BIM attribute sets, crack parameters and tilt angle for intelligent judgment, and outputs graded early warning results (such as high risk / medium risk / low risk), achieving second-level safety assessment and greatly improving the efficiency and objectivity of the assessment.

[0071] A BIM-based UAV-based building safety assessment and management system is used to implement the aforementioned BIM-based UAV-based building safety assessment and management method, including a crack identification model training module, a BIM modeling and trajectory planning module, a UAV multi-source data acquisition module, an image processing and matching module, a crack parameter quantification module, a point cloud registration and deformation analysis module, and a safety risk intelligent assessment module.

[0072] The crack recognition model training module collects and annotates crack images from multiple scenes and dynamically adjusts the convolution kernel parameters of the U-Net++ model to finally generate a deep learning model for crack detection that meets a preset accuracy threshold, providing core algorithm support for subsequent automated crack recognition.

[0073] The BIM modeling and trajectory planning module constructs a 3D BIM model based on building CAD drawings and material data, extracts key feature points of the facade using a spatial grid sampling algorithm, generates a layered circular trajectory through a convex hull optimization algorithm, and forms a precise flight path by combining UAV hovering conditions and overlap rate constraints, and encodes it into an executable trajectory file.

[0074] The UAV multi-source data acquisition module allows the UAV equipped with a visible light / infrared dual-mode camera and an RTK positioning module to fly along a preset path; it automatically switches shooting modes by monitoring light intensity in real time, dynamically detects offset to trigger a return-to-home and re-fly mechanism, and simultaneously acquires image sequences and POS data to ensure data integrity and environmental adaptability.

[0075] The image processing and matching module performs impulse noise filtering and CLAHE contrast enhancement on the original image, and extracts feature points by combining the improved SIFT algorithm. It uses BIM feature point projection to achieve bidirectional matching between image and BIM, and completes sub-pixel level accuracy verification by calculating the Euclidean distance of the matching points, providing a data basis for geometric alignment for subsequent analysis.

[0076] The crack parameter quantization module will input the enhanced image into the pre-trained U-Net++ model to generate a crack binary mask, optimize the crack morphology through closing operation and skeleton extraction; based on the skeleton chain pixel length and normal cross-sectional width, it will be converted into physical size by combining camera parameters, and then fused with BIM coordinates to output the crack spatial location and size parameter set.

[0077] The point cloud registration and deformation analysis module uses the SfM algorithm to generate a high-precision point cloud from the image sequence, and aligns it with the BIM model through ICP registration technology; it uses RANSAC plane segmentation to extract the point set inside the wall, calculates the spatial angle between the actual normal vector and the BIM reference normal vector, and accurately quantifies the tilt angle of the house.

[0078] The intelligent safety risk assessment module integrates crack parameters, tilt angles, and BIM component attributes, and performs multi-dimensional correlation analysis through a predefined risk rule base to automatically output the building safety level determination results, providing a quantitative basis for decision-making.

[0079] (III) Beneficial Effects

[0080] The present invention has the following beneficial effects:

[0081] This invention deeply integrates BIM models, UAV perception, and artificial intelligence algorithms; improves crack detection rate through pre-trained U-Net++ models; ensures data integrity in complex environments by utilizing BIM-driven autonomous UAV flight paths and dual-mode adaptive acquisition; achieves accurate crack location and tilt detection through bidirectional matching and high-precision point cloud registration; and automatically generates risk results by integrating a risk rule base, improving the detection efficiency of single high-rise buildings, reducing the high-risk missed detection rate, and realizing full-process automation and high-precision quantitative analysis of building safety assessment.

[0082] This invention achieves centimeter-level precision flight and multi-angle, blind-spot-free data acquisition through BIM-driven UAV autonomous trajectory planning; it combines an adaptive dual-mode camera and RTK positioning to ensure data integrity in complex environments; and it improves the crack detection rate of multi-material walls by using a pre-trained U-Net++ crack recognition model, overcoming the error of manual measurement by outputting crack parameters through a pixel-to-physical size conversion formula.

[0083] This invention achieves sub-pixel-level image-BIM alignment by using a BIM-image-point cloud bidirectional matching mechanism, guiding SIFT feature matching with BIM feature point projection, and then verifying consistency through back projection. By combining SfM point cloud and ICP registration technology, it effectively controls the residual between the actual point cloud and the BIM model, and accurately calculates the wall tilt angle through a normal vector angle algorithm, providing multi-dimensional basis for structural safety.

[0084] This invention constructs a rule-driven risk assessment engine that integrates crack parameters, tilt angles, and BIM attributes to automatically trigger tiered early warnings. It reduces the inspection time for single high-rise buildings and the rate of missed detection of high-risk cracks, and supports the generation of digital inspection reports bound to BIM spatial locations, significantly improving the efficiency of maintenance decision-making.

[0085] This invention incorporates a built-in offset re-flight mechanism and illumination switching strategy during the flight phase to ensure data acquisition stability; in the image processing phase, CLAHE enhancement and impulse noise filtering are used to effectively improve the visibility of cracks in shadow / strong light areas; in the accuracy verification stage, dual threshold control using Euclidean distance ensures that only qualified data enters subsequent analysis, avoiding error accumulation.

[0086] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0087] To more clearly illustrate the technical solutions of the embodiments of the invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For those skilled in the art, the drawings can be obtained from these drawings without creative effort.

[0088] Figure 1 This is a flowchart illustrating a BIM-based UAV-based building safety assessment and management method according to the present invention.

[0089] Figure 2 This is a schematic diagram of the process of obtaining a preset flight path in the BIM-based UAV-based building safety assessment and management method of the present invention.

[0090] Figure 3 This is a flowchart illustrating the process of obtaining an effective set of matching points in a BIM-based UAV-based building safety assessment and management method according to the present invention.

[0091] Figure 4 This is a schematic diagram of a module of a BIM-based UAV-based building safety assessment and management system according to the present invention. Detailed Implementation

[0092] The technical solutions of the embodiments of the invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the invention, and not all embodiments. Based on the embodiments of the invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the invention.

[0093] In the description of this invention, it should be understood that the terms "opening", "upper", "lower", "top", "middle", "inner", etc., which indicate orientation or positional relationship, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the components or elements referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on the invention.

[0094] Example 1:

[0095] Please see Figure 1 , Figure 2 , Figure 3 This invention discloses a BIM-based UAV-based method for assessing and managing building safety, comprising the following steps:

[0096] S1. Train the U-Net++ model to obtain a pre-trained U-Net++ model;

[0097] S1 includes the following steps:

[0098] S11. Construct the U-Net++ model; set the initial kernel size and number of initial convolution kernels for the U-Net++ model;

[0099] S12. Collect crack images containing different lighting and material scenes to obtain historical crack images; perform mask annotation on the images to obtain annotated historical crack images;

[0100] The U-Net++ model is trained using labeled historical crack images. During the training process, the optimal combination of kernel size and number of kernels for the U-Net++ model is found to obtain the optimal combination. The kernel size and number of kernels in the optimal combination are used as the kernel size and number of kernels for the U-Net++ model to obtain the pre-trained U-Net++ model.

[0101] The optimal combination of kernel size and number of kernels for the U-Net++ model during training can be found using various optimization algorithms, such as the fish swarm algorithm, genetic algorithm, and particle swarm algorithm.

[0102] Taking genetic algorithms as an example:

[0103] S121. Set the training accuracy threshold of the U-Net++ model to u1 and the training accuracy to u2.

[0104] S122. Construct a chromosome population. Let the size of the chromosome population be j. Then, the chromosome population is represented as k = {k1, k2, ..., k}. i ,...,k j}, where k i Let z represent the i-th chromosome in the chromosome population. Each chromosome represents a combination of convolution kernel size and number of convolution kernels. Let z1 be the maximum number of training iterations and z2 be the current number of training iterations.

[0105] S123. Substitute the parameter combination in the chromosome population into the U-Net++ model, and train the U-Net++ model using labeled historical crack images to obtain the training accuracy; based on the training accuracy, perform crossover and mutation operations on the chromosomes in the chromosome population to obtain the chromosome population with crossover and mutation operations.

[0106] S124. Repeat S122 and S123. Stop the iteration when u2≥u1 or z2≥z1 to obtain the optimal combination.

[0107] S2. Construct a 3D BIM model of the building and plan the flight path of the drone to obtain the preset trajectory.

[0108] S2 includes the following steps:

[0109] S21. Collect CAD drawings and construction material data of the building to obtain building data; import the building data into BIM parametric modeling software to obtain a 3D BIM model of the building; the 3D BIM model of the building includes a vertex set, an edge set, and an attribute set; the vertex set includes all the geometric vertices of the building model, which are the most basic units that constitute all the geometric shapes of the model (such as walls, floors, beams, columns, pipes, equipment, etc.). They are the cornerstone of geometric modeling. For example, a cuboid (such as a room or a beam) requires at least 8 vertices to define its 8 corner points; the edge set includes all the edges of the building model, which connect isolated vertices to form the wireframe structure of the model. These are the foundation for the surface contours and internal structural lines of the model; for example, the 12 edges of a cuboid are its 12 sides. The attribute set includes the attributes of the house model, which are a collection of non-geometric information attached to the model's geometric elements (vertices, edges, and more importantly—faces / surfaces and volumes / components composed of vertices and edges); such as component types as walls, doors, windows, beams, columns, floors, pipes, ducts, equipment, and furniture; material information as concrete strength grade C30, steel type Q345B, glass type, and insulation material; and size specifications as thickness 200mm, diameter DN150, length 6000mm, and width 1200mm.

[0110] S22. Key points are set on the exterior of the 3D BIM model of the building using a spatial grid sampling algorithm to obtain a key point set for the exterior of the building; the key point set for the exterior of the building includes all key points on the exterior of the building; the spatial grid sampling algorithm is used to ensure that the key point set for the exterior of the building can cover the entire exterior of the building, such as at least 2 feature points per square meter.

[0111] Based on the key point set of the building facade, the API interface of the BIM software is called to automatically extract the coordinates of the four corners of the window openings (as the main positioning reference), the intersection points of decorative lines (auxiliary positioning), and the points where pipelines penetrate the wall (special area markers) from the vertex set, edge set, and attribute set of the building facade, thus obtaining the feature point set of the building facade; the feature point set of the building facade contains the feature point data of each key point of the facade in the key point set of the building facade.

[0112] S23. Based on the feature point set of the building's exterior facade, perform UAV flight path planning for the building to obtain the executable UAV trajectory file;

[0113] S23 includes the following steps:

[0114] S231. Based on the attribute set in the 3D BIM model of the building, set the safety distance; based on the outer envelope cube of the 3D BIM model of the building and the safety distance, obtain the safe flight space range of the UAV;

[0115] S232. Set the layer height interval; Based on the safe flight space range of the UAV and the layer height interval, the building is vertically divided into multiple layers to obtain the layer height sequence; For example, calculate the number of layers according to the total height of the building (e.g., for a 100m high-rise building, 100 / 3 = 34 layers), generate horizontal cutting surfaces at 3m intervals from bottom to top, and vertically divide the building into 34 layers.

[0116] S233. Set the overlap rate threshold and hovering conditions (such as at corners); use the convex hull optimization algorithm to extract the minimum convex polygon of the outer contour of each layer in the layered height sequence;

[0117] A closed loop path is generated around the building by extending the safety distance along the minimum convex polygon; based on the closed loop path around the building, the UAV camera's field of view is used to calculate the flight strip spacing, ensuring that the overlap rate of adjacent flight strips is greater than the overlap rate threshold, thus obtaining the current layer's horizontal loop track point set;

[0118] S234. Based on the single-layer circular track point set and hovering conditions, insert a hovering command (pause for 1 second) at the vertex of the path corner, calculate the duration of the movement segment based on the UAV's cruising speed (e.g., 2 m / s), and generate a timestamp by combining the hovering time to obtain a single-layer track function set with timestamps.

[0119] The single-layer track function set with timestamps is connected in order from bottom to top to form a spiral upward path. The transition between layers is achieved by vertical climbing, resulting in a complete three-dimensional track function.

[0120] The complete three-dimensional trajectory function is discretized into a time-position-attitude command sequence, encoded into a format recognizable by the flight control system, and thus a preset trajectory that the UAV can execute is obtained;

[0121] S3. The UAV flies along a preset flight path, collects data during the flight, and returns to home and re-flies when an abnormal data is detected, until the flight ends, and obtains metadata and image sequences.

[0122] S3 includes the following steps:

[0123] S31. The UAV is equipped with a multi-sensor pod; the multi-sensor pod includes a visible light camera, an infrared thermal imager, and an RTK positioning module; the multi-sensor pod includes a 20-megapixel visible light camera with a focal length of 35mm for crack detection; a 640×480 infrared thermal imager to identify temperature anomalies caused by water seepage; and an RTK positioning module to output centimeter-level POS data in real time (longitude, latitude, elevation, roll, pitch, and yaw data).

[0124] S32. Set the acquisition cycle, light intensity threshold, and offset threshold; the UAV flies along the preset trajectory, and in each acquisition cycle, it collects image data and POS data through the multi-sensor pod; during the flight, it monitors the light intensity and UAV offset in real time, and when the light intensity is lower than the light intensity threshold, it adaptively switches to infrared mode for shooting; when the UAV offset is greater than the offset threshold, it triggers automatic return to home and re-flying until the flight ends, and obtains metadata and image sequences;

[0125] S4. Denoise and enhance the image sequence to obtain an enhanced image sequence; extract the features of the enhanced image sequence to obtain SIFT image features;

[0126] A bidirectional matching operation is performed based on the SIFT image features to obtain a set of valid matching point pairs; the accuracy is then verified based on the set of valid matching point pairs.

[0127] S4 includes the following steps:

[0128] S41. Use a 5×5 window filter to eliminate impulse noise in the image sequence to obtain a denoised image sequence;

[0129] The denoised image sequence is enhanced using CLAHE enhancement, and contrast adaptive histogram equalization is constrained to improve local image contrast (especially improving shadow / high-light areas), resulting in an enhanced image sequence.

[0130] S42. Using the improved SIFT algorithm, extract image feature points from the enhanced image sequence to obtain SIFT image features;

[0131] Based on the real-time pose of the UAV and the intrinsic parameter matrix of the camera in the metadata, determined by the camera hardware, it is used to establish a BIM mapping between three-dimensional space and image coordinates; the set of building BIM feature points is projected onto the image plane to obtain the building spatial projection.

[0132] Based on SIFT image features, building spatial projection, building 3D BIM model, and augmented image, a bidirectional matching operation is performed to retain bidirectionally consistent matching point pairs, resulting in a set of valid matching point pairs. The bidirectional matching operation includes forward matching and backward matching. Forward matching involves searching for SIFT image features in the neighborhood of projected points in the building spatial projection within the augmented image. Backward matching involves back-projecting the SIFT image features onto the building 3D BIM model to verify consistency. The set of valid matching point pairs contains all valid matching point pairs. Each valid matching point pair includes the coordinates of building BIM feature points and the pixel coordinates of the augmented image.

[0133] S43. Project the coordinates of the building BIM feature points in the effective matching point pair set onto the image plane according to the current pose in the metadata to obtain the desired pixel coordinates.

[0134] Calculate the Euclidean distance between the desired pixel coordinates and the enhanced image pixel coordinates to obtain the Euclidean distance set; the calculation formula is as follows.

[0135]

[0136] Among them, e i p represents the Euclidean distance between the expected pixel coordinates of the i-th valid matching point pair in the set of valid matching point pairs and the pixel coordinates of the enhanced image; 1 i p represents the expected pixel coordinates of the i-th valid matching point pair. i Represents the augmented image pixel coordinates of the i-th valid matching point pair;

[0137] Calculate the average value of the Euclidean distance set to obtain the average error; extract the maximum value from the Euclidean distance set to obtain the maximum error;

[0138] Set the average error threshold and the maximum error threshold; if the average error is less than the average error threshold and the maximum error is less than the maximum error threshold, the accuracy verification passes; otherwise, the accuracy verification fails and returns to S3.

[0139] S5. After the accuracy verification is passed, calculate the parameters of the crack binary mask obtained by combining the enhanced image sequence with the pre-trained U-Net++ model to obtain the crack parameter set.

[0140] S5 includes the following steps:

[0141] S51. Input the enhanced image sequence into the pre-trained U-Net++ model to obtain the binary mask of the crack; fuse the coordinates of the binary mask of the crack with the coordinates of the effective matching point pairs to obtain the location of the crack in the house.

[0142] S52. Perform closing operations and skeleton extraction operations on the binary mask of the crack to obtain the optimized crack skeleton map; calculate the total pixel length of the skeleton chain in the optimized crack skeleton map; convert the total pixel length of the skeleton chain into physical length.

[0143] S53. Take a normal section along the skeleton line of the optimized crack skeleton diagram and calculate the number of crack pixels in the section; convert the number of crack pixels in the section into physical width; the location, physical length and width of the building cracks together constitute the crack parameter set;

[0144] Taking the conversion of the number of pixels in a crack within a cross-section to its physical width as an example, the conversion formula is as follows:

[0145]

[0146] Where w represents the physical width of the crack in the house, h represents the number of pixels in the crack within the pixel cross-section, g represents the physical size of a single pixel on the camera sensor, m represents the actual vertical distance from the camera to the surface of the object where the crack is located, and f represents the focal length of the camera lens.

[0147] S6. Register the high-precision point cloud constructed based on the enhanced image sequence and metadata to obtain the registered point cloud; calculate the house tilt angle based on the registered point cloud.

[0148] S6 includes the following steps:

[0149] S61. Based on the sequence and metadata, a high-precision point cloud is generated using the SfM algorithm. The generation process involves extracting SIFT feature points from each image in the enhanced image and performing cross-frame matching to obtain the matched feature points. Based on the matched feature points, the camera pose and initial coordinates of the spatial points are calculated using the co-view relationship. The reprojection error is minimized through nonlinear optimization (i.e., optimizing the camera pose and 3D point coordinates so that when the 3D points are projected back into the 2D image, the distance between the projected points and the actual detected feature points is minimized), and finally, the point cloud set of the target wall is output.

[0150] S62. Use ICP registration to register the high-precision point cloud with the building BIM 3D model to obtain the registered point cloud; the ICP registration specifically includes establishing the correspondence between the nearest points of the point cloud and the BIM plane, solving the optimal rotation matrix and translation vector to minimize the distance residual from the point cloud to the plane; repeat the above steps until convergence (convergence judgment rule such as residual threshold <0.5mm);

[0151] S63. Perform RANSAC plane segmentation on the registered point cloud to obtain the interior point set; calculate the covariance matrix of the interior point set; extract the minimum eigenvalue of the covariance matrix of the interior point set; calculate the eigenvector of the minimum eigenvalue to obtain the actual normal vector.

[0152] The angle between the actual normal vector and the BIM reference normal vector is calculated using the spatial vector angle calculation formula, thus obtaining the building tilt angle. The spatial vector angle calculation formula is as follows:

[0153]

[0154] Where α represents the tilt angle of the house, cos -1 represents the inverse cosine function, nBIM represents the BIM reference normal vector, and nc represents the actual normal vector;

[0155] S7. Based on the 3D BIM model of the building, the crack parameter set, and the building tilt angle, combined with the risk rule base, a comprehensive risk result is determined.

[0156] The 7th step includes the following steps:

[0157] S71. Construct a risk rule base; the risk judgment rules in the risk rule base are determined based on the building's construction attributes, the building's tilt angle, and the building's tilt angle. For example, if the crack width is ≥0.5mm and it is located in a load-bearing wall, it is judged as high-risk; if the crack width is ≥0.3mm and the building's age is >20 years, it is judged as high-risk; if the building's tilt angle is >2° and it is a cantilever structure, it is judged as high-risk; if the crack width is >0.4mm and the building's tilt angle is >1.5°, it is judged as high-risk.

[0158] S72. Based on the attribute set, crack parameter set, and building tilt angle in the 3D BIM model of the building, combined with the risk rule base, a comprehensive risk result is determined.

[0159] Example 2:

[0160] Please see Figure 4 A BIM-based UAV-based building safety assessment and management system is used to implement the aforementioned BIM-based UAV-based building safety assessment and management method, including a crack identification model training module, a BIM modeling and trajectory planning module, a UAV multi-source data acquisition module, an image processing and matching module, a crack parameter quantification module, a point cloud registration and deformation analysis module, and a safety risk intelligent assessment module.

[0161] The crack recognition model training module collects and annotates crack images from multiple scenes and dynamically adjusts the convolution kernel parameters of the U-Net++ model to finally generate a deep learning model for crack detection that meets a preset accuracy threshold, providing core algorithm support for subsequent automated crack recognition.

[0162] The BIM modeling and trajectory planning module constructs a 3D BIM model based on building CAD drawings and material data, extracts key feature points of the facade using a spatial grid sampling algorithm, generates a layered circular trajectory through a convex hull optimization algorithm, and forms a precise flight path by combining UAV hovering conditions and overlap rate constraints, and encodes it into an executable trajectory file.

[0163] The UAV multi-source data acquisition module allows the UAV equipped with a visible light / infrared dual-mode camera and an RTK positioning module to fly along a preset path; it automatically switches shooting modes by monitoring light intensity in real time, dynamically detects offset to trigger a return-to-home and re-fly mechanism, and simultaneously acquires image sequences and POS data to ensure data integrity and environmental adaptability.

[0164] The image processing and matching module performs impulse noise filtering and CLAHE contrast enhancement on the original image, and extracts feature points by combining the improved SIFT algorithm. It uses BIM feature point projection to achieve bidirectional matching between image and BIM, and completes sub-pixel level accuracy verification by calculating the Euclidean distance of the matching points, providing a data basis for geometric alignment for subsequent analysis.

[0165] The crack parameter quantization module will input the enhanced image into the pre-trained U-Net++ model to generate a crack binary mask, optimize the crack morphology through closing operation and skeleton extraction; based on the skeleton chain pixel length and normal cross-sectional width, it will be converted into physical size by combining camera parameters, and then fused with BIM coordinates to output the crack spatial location and size parameter set.

[0166] The point cloud registration and deformation analysis module uses the SfM algorithm to generate a high-precision point cloud from the image sequence, and aligns it with the BIM model through ICP registration technology; it uses RANSAC plane segmentation to extract the point set inside the wall, calculates the spatial angle between the actual normal vector and the BIM reference normal vector, and accurately quantifies the tilt angle of the house.

[0167] The intelligent safety risk assessment module integrates crack parameters, tilt angles, and BIM component attributes, and performs multi-dimensional correlation analysis through a predefined risk rule base to automatically output the building safety level determination results, providing a quantitative basis for decision-making.

[0168] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0169] The preferred embodiments of the invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention.

Claims

1. A BIM-based UAV-based method for assessing and managing building safety, characterized in that: Includes the following steps: S1. Train the U-Net++ model to obtain a pre-trained U-Net++ model; S2. Construct a 3D BIM model of the building and plan the flight path of the drone to obtain the preset trajectory. S3. The UAV flies along a preset flight path, collects data during the flight, and returns to home and re-flies when an abnormal data is detected, until the flight ends, and obtains metadata and image sequences. S4. Denoise and enhance the image sequence to obtain an enhanced image sequence; extract the features of the enhanced image sequence to obtain SIFT image features; A bidirectional matching operation is performed based on the SIFT image features to obtain a set of valid matching point pairs; the accuracy is then verified based on the set of valid matching point pairs. S4 includes the following steps: S41. Median filtering is used to eliminate noise in the image sequence, resulting in a denoised image sequence. The denoised image sequence was enhanced using CLAHE to obtain the enhanced image sequence. S42. Using the improved SIFT algorithm, extract image feature points from the enhanced image sequence to obtain SIFT image features; Projecting the set of BIM feature points of the building onto the image plane yields the spatial projection of the building; Based on SIFT image features, building spatial projection, building 3D BIM model and enhanced image, a bidirectional matching operation is performed to retain bidirectional consistent matching point pairs and obtain a set of effective matching point pairs. The bidirectional matching operation includes forward matching and reverse matching; the forward matching is to search for SIFT image features in the neighborhood of the projection point in the spatial projection of the house in the enhanced image, and the reverse matching is to back-project the SIFT image features to the 3D BIM model of the house to verify consistency. The set of valid matching point pairs contains all valid matching point pairs; each valid matching point pair contains the coordinates of building BIM feature points and the coordinates of enhanced image pixels. S43. Project the coordinates of the building BIM feature points in the effective matching point pair set onto the image plane according to the current pose in the metadata to obtain the desired pixel coordinates. Calculate the Euclidean distance between the desired pixel coordinates and the enhanced image pixel coordinates to obtain the Euclidean distance set; Calculate the average value of the Euclidean distance set to obtain the average error; extract the maximum value from the Euclidean distance set to obtain the maximum error; Set the average error threshold and the maximum error threshold; if the average error is less than the average error threshold and the maximum error is less than the maximum error threshold, the accuracy verification passes; otherwise, the accuracy verification fails and returns to S3. S5. After the accuracy verification is passed, calculate the parameters of the crack binary mask obtained by combining the enhanced image sequence with the pre-trained U-Net++ model to obtain the crack parameter set. Specifically, it includes: S51. After the accuracy verification is passed, the enhanced image sequence is input into the pre-trained U-Net++ model to obtain the crack binary mask; the coordinate position of the crack binary mask and the coordinates in the effective matching point pair are fused to obtain the location of the house crack; S52. Perform closing operations and skeleton extraction operations on the binary mask of the crack to obtain the optimized crack skeleton map; calculate the total pixel length of the skeleton chain in the optimized crack skeleton map; convert the total pixel length of the skeleton chain into physical length. S53. Take a normal section along the skeleton line of the optimized crack skeleton map and calculate the number of crack pixels in the section; convert the number of crack pixels in the section into physical width. S6. Register the high-precision point cloud constructed based on the enhanced image sequence and metadata to obtain the registered point cloud; calculate the house tilt angle based on the registered point cloud. S6 includes the following steps: S61. Based on sequences and metadata, the SfM algorithm is used to generate high-precision point clouds; S62. Use ICP registration to register the high-precision point cloud with the building BIM 3D model to obtain the registered point cloud; S63. Perform RANSAC plane segmentation on the registered point cloud to obtain the interior point set; calculate the covariance matrix of the interior point set. Extract the minimum eigenvalue of the covariance matrix of the interior point set; calculate the eigenvector of the minimum eigenvalue to obtain the actual normal vector; The angle between the actual normal vector and the BIM reference normal vector is calculated using the formula for calculating the angle between spatial vectors, thus obtaining the building tilt angle. S7. Based on the 3D BIM model of the building, the crack parameter set, and the building tilt angle, combined with the risk rule base, a comprehensive risk result is determined.

2. The method for BIM-based UAV-based building safety assessment and management according to claim 1, characterized in that, S1 includes the following steps: S11. Construct the U-Net++ model; set the initial kernel size and number of initial convolution kernels for the U-Net++ model; S12. Collect crack images containing different lighting and material scenes to obtain historical crack images; perform mask annotation on the images to obtain annotated historical crack images; The U-Net++ model is trained using labeled historical crack images. During training, the optimal combination of kernel size and number of kernels for the U-Net++ model is found, and the optimal combination is obtained. The kernel size and number of kernels in the optimal combination are used as the kernel size and number of kernels for the U-Net++ model, thus obtaining the pre-trained U-Net++ model.

3. The method for BIM-based UAV-based building safety assessment and management according to claim 1, characterized in that, S2 includes the following steps: S21. Collect CAD drawings and construction material data of the building to obtain building data; import the building data into BIM parametric modeling software to obtain a 3D BIM model of the building; the 3D BIM model of the building contains vertex set, edge set, and attribute set; S22. Use the spatial grid sampling algorithm to set key points on the exterior of the 3D BIM model of the building to obtain the key point set of the building exterior. Based on the key point set of the building facade, the API interface of the BIM software is called to extract features from the three-dimensional BIM model of the building to obtain the feature point set of the building facade. S23. Based on the feature point set of the building's exterior facade, plan the flight path of the UAV for the building to obtain the executable flight path file of the UAV.

4. The BIM-based UAV-based building safety assessment and management method according to claim 3, characterized in that, S23 includes the following steps: S231. Based on the attribute set in the 3D BIM model of the building, set the safety distance; based on the outer envelope cube of the 3D BIM model of the building and the safety distance, obtain the safe flight space range of the UAV; S232. Set the layer height interval; based on the safe flight space range of the UAV and the layer height interval, the building is vertically divided into multiple layers to obtain the layer height sequence; S233. Set the overlap rate threshold and hovering conditions; extract the minimum convex polygon of the outer contour of each layer in the layered height sequence using the convex hull optimization algorithm; A closed loop path is generated around the building by extending the safety distance along the minimum convex polygon; based on the closed loop path around the building, the UAV camera's field of view is used to calculate the flight strip spacing, ensuring that the overlap rate of adjacent flight strips is greater than the overlap rate threshold, thus obtaining the current layer's horizontal loop track point set; S234. Based on the single-layer circular track point set and hovering conditions, obtain the single-layer track function set with timestamps; The single-layer track function set with timestamps is connected in order from bottom to top to form a spiral upward path. The transition between layers is achieved by vertical climbing, resulting in a complete three-dimensional track function. The complete 3D trajectory function is discretized into a time-position-attitude command sequence, encoded into a format recognizable by the flight control system, and thus a preset trajectory that the UAV can execute is obtained.

5. A BIM-based UAV-based building safety assessment and management method according to claim 1, characterized in that, S3 includes the following steps: S31. The UAV is equipped with a multi-sensor pod; the multi-sensor pod includes a visible light camera, an infrared thermal imager, and an RTK positioning module. S32. Set the acquisition cycle, light intensity threshold, and offset threshold; the UAV flies along the preset trajectory. In each acquisition cycle, it collects image data and POS data through the multi-sensor pod. During the flight, it monitors the light intensity and UAV offset in real time. When the light intensity is lower than the light intensity threshold, it adaptively switches to infrared mode for shooting. When the UAV offset is greater than the offset threshold, it triggers automatic return and re-flight until the flight ends, and obtains metadata and image sequences.

6. The method for BIM-based UAV-based building safety assessment and management according to claim 1, characterized in that, S7 includes the following steps: S71. Construct a risk rule base; S72. Based on the attribute set, crack parameter set, and building tilt angle in the 3D BIM model of the building, combined with the risk rule base, a comprehensive risk result is determined.

7. A BIM-based UAV-based building safety assessment and management system, characterized in that, The system implements a BIM-based UAV-based building safety assessment and management method as described in any one of claims 1-6, comprising a crack identification model training module, a BIM modeling and trajectory planning module, a UAV multi-source data acquisition module, an image processing and matching module, a crack parameter quantification module, a point cloud registration and deformation analysis module, and a safety risk intelligent assessment module.

Images