Unmanned aerial vehicle sensing house safety appraisal management system and method based on BIM

Abstract

The invention discloses an unmanned aerial vehicle sensing house safety appraisal management system and method based on BIM, and relates to the technical field of data processing. The method comprises the following steps: training a U-Net + + model, and based on a house three-dimensional BIM model, generating a preset track by combining a space grid sampling algorithm with a convex hull optimization algorithm; the unmanned aerial vehicle carries the dual-mode sensing pod to fly along a preset flight path, and image virtual columns and metadata are collected; de-noising enhancement processing is carried out on the image sequence, precision alignment is realized by utilizing BIM feature point projection to guide SIFT feature bidirectional matching, after Euclidean distance threshold verification, a pre-trained U-Net + + model is input to generate a crack binary mask, and the crack binary mask is converted into a physical crack parameter; the inclination angle of the wall is calculated through the SfM point cloud and ICP registration technology; integrating crack parameters, inclination angles and BIM attributes, and automatically triggering graded early warning through a risk rule base; the full-process automation from data acquisition, crack identification to risk assessment is realized, the detection precision and efficiency are remarkably improved, and the high-risk omission ratio is effectively reduced.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of data processing, and in particular relates to a BIM-based unmanned aerial vehicle sensing house safety appraisal management system and method. BACKGROUND

[0002] In the field of building operation and safety monitoring, problems such as cracks and tilting caused by aging of existing buildings are increasingly prominent, especially for high-rise buildings and historical and cultural buildings that need to be regularly appraised for safety. Although the current industry has introduced unmanned aerial vehicles and BIM technology, the two are mostly used independently: image data collected by unmanned aerial vehicles needs to be manually compared with drawings to locate damage, which is inefficient; and BIM models are only used as static databases and cannot be linked to real-time on-site test results. At the same time, automatic crack recognition algorithms have a high false detection rate under complex wall materials (such as textured stone and dirty surfaces), and structural deformation analysis usually requires the deployment of sensors, which is costly and has a limited coverage.

[0003] Existing house safety appraisal mainly relies on manual visual inspection or basic unmanned aerial vehicle inspection, which has the problems of low efficiency of manual detection, high risk of high-altitude operation, and difficulty in accurately quantifying crack parameters; the data collected by ordinary unmanned aerial vehicles is not linked to building information models, resulting in inaccurate damage location; changes in light affect image quality, and cracks are prone to be missed in recognition; structural inclination measurement relies on devices such as total stations, which is complex to operate and cannot be linked to crack information for analysis; and existing methods lack an intelligent fusion mechanism for multi-source data, making it difficult to achieve automated risk assessment. SUMMARY

[0004] (I) Technical problems solved

[0005] To solve the problems in the related art, the present application provides a BIM-based unmanned aerial vehicle sensing house safety appraisal management method to overcome the above technical problems existing in the prior art.

[0006] (II) Technical solutions

[0007] To solve the above technical problems, the present application is implemented by the following technical solutions:

[0008] S1, training a U-Net++ model to obtain a pre-trained U-Net++ model;

[0009] S2, constructing a three-dimensional BIM model of the house and planning an unmanned aerial vehicle flight path for the house to obtain a preset flight path;

[0010] S3, the unmanned aerial vehicle flies along the preset flight path, collects data during the flight, returns to fly again when data anomalies are detected, and continues until the flight ends to obtain metadata and image sequences;

[0011] S4, denoising and enhancing the image sequence to obtain an enhanced image sequence; extracting features of the enhanced image sequence to obtain SIFT image features;

[0012] Performing a bidirectional matching operation according to the SIFT image features to obtain a set of effective matching point pairs; performing precision verification according to the set of effective matching point pairs;

[0013] S5, after the precision verification passes, calculating parameters of a crack binary mask obtained by combining the enhanced image sequence and the pre-trained U-Net++ model to obtain a crack parameter set;

[0014] S6, registering high-precision point clouds constructed based on the enhanced image sequence and the metadata to obtain registered point clouds; and calculating a house tilt angle based on the registered point clouds;

[0015] S7, according to the house three-dimensional BIM model, the crack parameter set and the house tilt angle, combining a risk rule library to perform comprehensive risk result determination;

[0016] The application fuses BIM model, unmanned aerial vehicle perception and artificial intelligence algorithm; the pre-trained U-Net++ model improves the crack detection rate; the BIM-driven unmanned aerial vehicle autonomous track and dual-mode adaptive collection ensure the completeness of the data in complex environments; the bidirectional matching and high-precision point cloud registration realize accurate crack positioning and tilt detection; the integrated risk rule library automatically generates a risk result, improves the detection efficiency of a single high-rise building, reduces the high-risk missed detection rate, and realizes the full-process automation and high-precision quantitative analysis of house safety identification.

[0017] Preferably, the S1 comprises the following steps:

[0018] S11, constructing a U-Net++ model; and setting an initial convolution kernel size and an initial convolution kernel number of the U-Net++ model;

[0019] S12, collecting crack images containing different light and material scenes to obtain historical crack images; and performing mask annotation on the images to obtain annotated historical crack images;

[0020] Training the U-Net++ model using the annotated historical crack images, finding an optimal convolution kernel size and convolution kernel number combination of the U-Net++ model during the training process to obtain an optimal combination; and taking the convolution kernel size and the convolution kernel number in the optimal combination as the convolution kernel size and the convolution kernel number of the U-Net++ model to obtain a pre-trained U-Net++ model;

[0021] The application trains and labels by collecting multi-scene crack images, and finds a convolution kernel parameter combination of U-Net++, so that the adaptability of the model to complex materials and light changes is significantly improved, and the crack recognition accuracy is improved, laying an algorithm foundation for subsequent accurate crack quantification.

[0022] Preferably, the S2 comprises the following steps:

[0023] S21, collect CAD drawings and construction material data of the house to obtain house data; import the house data into a BIM parameterized modeling software to obtain a house three-dimensional BIM model; the house three-dimensional BIM model comprises a vertex set, an edge set and an attribute set; the vertex set comprises all geometric vertices of the house model, the edge set comprises all edges of the house model, and the attribute set comprises attributes of the house model;

[0024] S22, set key points on the facade of the house three-dimensional BIM model through a spatial grid sampling algorithm to obtain a house facade key point set; the house facade key point set comprises all key points on the facade of the house;

[0025] Based on the house facade key point set, an API interface of the BIM software is called to automatically extract window hole four corner coordinates, decoration line intersection points and pipeline wall-penetrating points of the facade of the house from the vertex set, the edge set and the attribute set of the house three-dimensional BIM model to obtain a house facade feature point set; the house facade feature point set comprises feature point data of each facade key point in the house facade key point set;

[0026] S23, plan a flight path of a UAV according to the house facade feature point set to obtain an executable flight path file of the UAV;

[0027] The application constructs a BIM model containing geometric attributes and component information through CAD data, automatically extracts key feature points such as window holes and pipelines based on spatial grid sampling, drives a UAV to generate a spiral flight path that accurately fits the building contour, eliminates manual measurement errors, realizes no-dead-angle scanning and avoids collision risks.

[0028] Preferably, the S23 comprises the following steps:

[0029] S231, set a safety distance according to the attribute set in the house three-dimensional BIM model; obtain a safe flight space range of the UAV based on the outer envelope cube of the house three-dimensional BIM model and the safety distance;

[0030] S232, set a layered height interval; vertically divide the building into multiple layers according to the safe flight space range of the UAV and the layered height interval to obtain a layered height sequence;

[0031] S233, set the overlap rate threshold, hovering condition; the minimum convex polygon of the outer contour of each layer in the hierarchical height sequence is extracted by a convex optimization algorithm;

[0032] A closed annular path surrounding the building is generated by expanding a safety distance outside the minimum convex polygon; based on the closed annular path surrounding the building, the flight strip spacing is calculated according to the field of view angle of the UAV camera, so that the adjacent flight strips have an overlap rate > the overlap rate threshold, and a current layer horizontal annular track point set is obtained;

[0033] S234, according to the single-layer annular track point set and the hovering condition, a single-layer track function set with timestamps is obtained;

[0034] The single-layer track function set with timestamps is connected in order from the bottom layer to the top layer, the tracks of each layer are connected, a spiral ascending path is formed, vertical climbing is adopted for interlayer transition, and a complete three-dimensional track function is obtained;

[0035] The complete three-dimensional track function is discretized into a time-position-pose instruction sequence, encoded into a format recognizable by a flight control system, and a preset track executable by a UAV is obtained;

[0036] The present application automatically sets a safety distance based on the BIM model attribute, extracts the minimum outer contour of the building by layer through a convex optimization algorithm, generates an annular path by expanding outside, dynamically calculates the flight strip spacing in combination with the field of view angle of the camera, and inserts a hovering instruction at the corner; finally, a spiral ascending track with timestamps is formed, the flight control executable instruction is outputted after discretization and encoding, the cm-level close-fitting no-dead-angle scanning of the building facade is realized, the collision risk is avoided, and the image acquisition integrity is guaranteed.

[0037] Preferably, the S3 comprises the following steps:

[0038] S31, the UAV is equipped with a multi-sensor pod; the multi-sensor pod comprises a visible light camera, an infrared thermal imager and an RTK positioning module;

[0039] S32, set the acquisition period, the light intensity threshold and the offset threshold; the UAV flies along the preset track, in each acquisition period, collects image data and POS data through the multi-sensor pod; the light intensity and the offset amount of the UAV are monitored in real time during the flight, when the light intensity is lower than the light intensity threshold, the infrared mode is automatically switched on for shooting; when the offset amount of the UAV is greater than the offset threshold, the automatic return and reflight are triggered again until the flight is completed, and the metadata and the image sequence are obtained;

[0040] The application monitors the environment and attitude in flight in real time through the dual-mode camera (visible light / infrared) and RTK positioning module, automatically switches to the infrared mode to ensure data acquisition when the light is insufficient, triggers the return flight mechanism when the offset exceeds the threshold, ensures the positioning accuracy, the integrity of image data and metadata, and eliminates the blind area of artificial intervention.

[0041] Preferably, the S4 comprises the following steps:

[0042] S41, using median filtering to eliminate impulse noise in the image sequence, obtaining a denoised image sequence;

[0043] Using CLAHE enhancement processing on the denoised image sequence, limiting contrast adaptive histogram equalization, and improving local contrast of the image, obtaining an enhanced image sequence;

[0044] S42, using an improved SIFT algorithm to extract image feature points in the enhanced image sequence, obtaining SIFT image features;

[0045] Based on the real-time pose of the unmanned aerial vehicle in the metadata and the intrinsic matrix of the camera, projecting the house BIM feature point set to the image plane to obtain a house space projection;

[0046] Based on the SIFT image features, the house space projection, the three-dimensional BIM model of the house, and the enhanced image, performing a bidirectional matching operation, retaining matching point pairs consistent in both directions, and obtaining a set of effective matching point pairs;

[0047] The bidirectional matching operation includes forward matching and reverse matching; the forward matching is searching for SIFT image features within the projection point neighborhood in the house space projection in the enhanced image, and the reverse matching is projecting the SIFT image features to the three-dimensional BIM model of the house to verify consistency;

[0048] The set of effective matching point pairs contains all effective matching point pairs; the effective matching point pair contains house BIM feature point coordinates and enhanced image pixel coordinates;

[0049] S43, projecting the house BIM feature point coordinates in the set of effective matching point pairs to the image plane according to the current pose in the metadata to obtain expected pixel coordinates;

[0050] Calculating the Euclidean distance between the expected pixel coordinates and the enhanced image pixel coordinates to obtain a set of Euclidean distances;

[0051] Calculating the average value of the set of Euclidean distances to obtain an average error; extracting the maximum value in the set of Euclidean distances to obtain a maximum error;

[0052] Setting an average error threshold and a maximum error threshold; when the average error is less than the average error threshold and the maximum error is less than the maximum error threshold, the precision verification is passed; otherwise, the precision verification is failed, and returning to S3;

[0053] The application eliminates noise and improves local contrast of images by median filtering and CLAHE enhancement, and extracts feature points by combining an improved SIFT algorithm; consistent point pairs are screened based on BIM feature point projection guided bidirectional matching (forward neighborhood search and reverse BIM verification); finally, the accurate level alignment of image-BIM is ensured by double-threshold Euclidean distance verification (average error < average error threshold, maximum error < maximum error threshold), which provides a geometric reference for accurate crack positioning, and data re-sampling is triggered if the verification fails.

[0054] Preferably, the S5 comprises the following steps:

[0055] S51, after the precision verification is passed, the enhanced image sequence is input into a pre-trained U-Net++ model to obtain a crack binary mask; the coordinate positions of the crack binary mask and the coordinates in the effective matching point pairs are fused to obtain the position of the house crack;

[0056] S52, a closed operation and a skeleton extraction operation are performed on the crack binary mask to obtain an optimized crack skeleton graph; the total pixel length of the skeleton chain in the optimized crack skeleton graph is calculated; and the total pixel length of the skeleton chain is converted into a physical length;

[0057] S53, a normal section is taken along the skeleton line of the optimized crack skeleton graph, and the number of crack pixels in the section is calculated; and the number of crack pixels in the section is converted into a physical width;

[0058] The position, physical length and width of the house crack jointly constitute a crack parameter set;

[0059] The application outputs a crack binary mask by a pre-trained U-Net++ model, and realizes accurate positioning of the crack in the BIM model in combination with effective matching point pairs; the crack morphology is optimized by a closed operation and skeleton extraction, the pixel size is calculated along the skeleton line algorithm, and the pixel length and width are converted into physical sizes according to a camera parameter through a physical conversion formula, so as to eliminate image distortion error, and finally output a quantitative crack parameter set fusing spatial position, length and width, thereby providing an accurate basis for safety evaluation.

[0060] Preferably, the S6 comprises the following steps:

[0061] S61, based on the sequence and the metadata, a high-precision point cloud is generated by using an SfM algorithm;

[0062] S62, the high-precision point cloud is registered with a house BIM three-dimensional model by using ICP registration to obtain a registered point cloud;

[0063] S63, performing RANSAC plane segmentation on the registered point cloud to obtain an inlier point set; and calculating a covariance matrix of the inlier point set;

[0064] extracting a minimum eigenvalue of the covariance matrix of the inlier point set; calculating an eigenvector of the minimum eigenvalue to obtain an actual normal vector;

[0065] calculating an angle between the actual normal vector and a BIM reference normal vector by a spatial vector angle calculation formula to obtain a house tilt angle;

[0066] The application generates a high-precision point cloud of a wall surface through an SfM algorithm, accurately aligns the point cloud with a BIM model through ICP registration, extracts an inlier point set through RANSAC segmentation, calculates an actual normal vector based on eigenvector decomposition of a covariance matrix, and finally outputs a high-precision tilt angle through a spatial angle formula with a BIM reference normal vector, thereby realizing non-contact structural deformation quantification detection and providing key deformation data for house safety evaluation.

[0067] Preferably, the S7 comprises the following steps:

[0068] S71, constructing a risk rule library; and risk determination rules in the risk rule library are determined according to a construction attribute of the house, the house tilt angle and the house tilt angle;

[0069] S72, obtaining a comprehensive risk result determination by combining the risk rule library according to the attribute set, the crack parameter set and the house tilt angle in the three-dimensional BIM model of the house;

[0070] The application automatically associates a BIM attribute set, a crack parameter and a tilt angle for intelligent determination by constructing a multi-dimensional risk rule library (such as a crack width + position + tilt angle), and outputs a graded early warning result (such as high risk / medium risk / low risk), thereby realizing second-level safety evaluation and greatly improving identification efficiency and objectivity.

[0071] A BIM-based unmanned aerial vehicle sensing house safety identification management system for realizing the above-mentioned BIM-based unmanned aerial vehicle sensing house safety identification management method, comprising a crack identification model training module, a BIM modeling and flight path planning module, an unmanned aerial vehicle multi-source data acquisition module, an image processing and matching module, a crack parameter quantification module, a point cloud registration and deformation analysis module and a safety risk intelligent evaluation module;

[0072] The crack identification model training module collects multi-scene crack images for labeling, dynamically adjusts the convolution kernel parameters of the U-Net++ model, and finally generates a deep learning model for crack detection that meets a preset accuracy threshold, thereby providing core algorithm support for subsequent automatic crack identification;

[0073] The BIM modeling and path planning module constructs a three-dimensional BIM model based on house CAD drawings and material data, extracts key feature points of an outer facade by using a spatial grid sampling algorithm, generates a layered annular flight path by a convex hull optimization algorithm, forms an accurate flight path in combination with a hovering condition of the unmanned aerial vehicle and an overlap rate constraint, and encodes the flight path into an executable flight path file;

[0074] The unmanned aerial vehicle multi-source data acquisition module flies along a preset flight path by an unmanned aerial vehicle carrying a visible light / infrared dual-mode camera and an RTK positioning module; automatically switches a shooting mode by real-time monitoring of light intensity, triggers a return flight mechanism by dynamic detection of an offset, synchronously acquires image sequences and POS data, and ensures data integrity and environmental adaptability;

[0075] The image processing and matching module performs pulse noise filtering and CLAHE contrast enhancement processing on original images, extracts feature points in combination with an improved SIFT algorithm, realizes image-BIM bidirectional matching by using BIM feature point projection, completes sub-pixel level accuracy verification by calculating Euclidean distances of matching points, and provides a geometrically aligned data basis for subsequent analysis;

[0076] The crack parameter quantification module inputs an enhanced image into a pre-trained U-Net++ model to generate a crack binary mask, optimizes a crack shape by closed operation and skeleton extraction, converts a physical size based on a skeleton chain pixel length and a normal section width in combination with camera parameters, and outputs a crack spatial position and a size parameter set in combination with BIM coordinates;

[0077] The point cloud registration and deformation analysis module generates high-precision point clouds from image sequences by using an SfM algorithm, aligns the point clouds with the BIM model by using ICP registration technology, extracts wall surface in-point sets by using RANSAC plane segmentation, calculates a spatial included angle between an actual normal vector and a BIM reference normal vector, and accurately quantifies a house inclination angle;

[0078] The safety risk intelligent evaluation module integrates crack parameters, inclination angles and BIM component attributes, performs multi-dimensional correlation analysis by using a pre-defined risk rule library, and automatically outputs a house safety grade determination result, thereby providing a quantitative basis for decision-making.

[0079] (Three) Beneficial Effects

[0080] The present application has the following beneficial effects:

[0081] The application realizes centimeter-level accurate flight and multi-angle dead-angle-free data collection through BIM-driven unmanned aerial vehicle autonomous path planning; ensures data completeness in complex environments by combining light-adaptive dual-mode cameras and RTK positioning; improves the crack detection rate of multi-material walls based on the pre-trained U-Net++ crack identification model, and outputs crack parameters through a pixel-physical size conversion formula, overcoming the error of manual measurement.

[0082] The application realizes centimeter-level accurate flight and multi-angle dead-angle-free data collection through BIM-driven unmanned aerial vehicle autonomous path planning; ensures data completeness in complex environments by combining light-adaptive dual-mode cameras and RTK positioning; improves the crack detection rate of multi-material walls based on the pre-trained U-Net++ crack identification model, and outputs crack parameters through a pixel-physical size conversion formula, overcoming the error of manual measurement.

[0083] The application realizes sub-pixel level image-BIM alignment through BIM-image-point cloud bidirectional matching mechanism, uses BIM feature point projection to guide SIFT feature matching, and then verifies consistency through back projection; combined with SfM point cloud and ICP registration technology, the actual point cloud and BIM model residual error is effectively controlled, and the wall inclination angle is accurately calculated through the normal vector angle algorithm, providing multi-dimensional basis for structure safety.

[0084] The application reduces the time-consuming of single high-rise building detection and the high-risk crack omission rate by constructing a rule-driven risk assessment engine, integrating crack parameters, inclination angles and BIM attributes, and automatically triggering graded early warning; and supports generating a digital detection report bound to the BIM space position, greatly improving the maintenance decision-making efficiency.

[0085] The application has built-in offset reflight mechanism and light switching strategy in the flight phase to ensure data collection stability; CLAHE enhancement and pulse noise filtering are used in the image processing phase to effectively improve the crack visibility in shadow / light area; the Euclidean distance double threshold control is used in the precision verification link to ensure that only qualified data enters the subsequent analysis, avoiding error accumulation.

[0086] Of course, any product implementing the present application does not necessarily need to achieve all the advantages described above. BRIEF DESCRIPTION OF DRAWINGS

[0087] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0088] Figure 1 A flowchart of a BIM-based unmanned aerial vehicle sensing house safety appraisal management method of the present application is shown in the figure.

[0089] Figure 2 A flowchart of obtaining a preset flight path in a BIM-based unmanned aerial vehicle sensing house safety appraisal management method of the present application is shown in the figure.

[0090] Figure 3 A flowchart of obtaining an effective matching point pair set in a BIM-based unmanned aerial vehicle sensing house safety appraisal management method of the present application is shown in the figure.

[0091] Figure 4 A module diagram of a BIM-based unmanned aerial vehicle sensing house safety appraisal management system of the present application is shown in the figure. DETAILED DESCRIPTION

[0092] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the application. Obviously, the described embodiments are only some of the embodiments of the application, but not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0093] In the description of the application, it should be understood that the terms "opening", "upper", "lower", "top", "middle", "inner", etc. indicate the orientation or positional relationship, which are only for the convenience of describing the application and simplifying the description, and do not indicate or imply that the indicated components or elements must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the application.

[0094] Embodiment one:

[0095] Please refer to Figure 1 , Figure 2 , Figure 3 The application discloses a BIM-based unmanned aerial vehicle sensing house safety appraisal management method, comprising the following steps:

[0096] S1, training a U-Net++ model to obtain a pre-trained U-Net++ model;

[0097] The S1 comprises the following steps:

[0098] S11, constructing a U-Net++ model; setting the initial convolution kernel size and the initial convolution kernel quantity of the U-Net++ model;

[0099] S12, collecting crack images containing different light and material scenes to obtain historical crack images; performing mask annotation on the images to obtain annotated historical crack images;

[0100] The U-Net++ model is trained using the labeled historical crack images, and the optimal convolution kernel size and convolution kernel quantity combination of the U-Net++ model is found in the training process to obtain an optimal combination; the convolution kernel size and the convolution kernel quantity in the optimal combination are used as the convolution kernel size and the convolution kernel quantity of the U-Net++ model to obtain a pre-trained U-Net++ model;

[0101] The optimal convolution kernel size and convolution kernel quantity combination of the U-Net++ model in the training process can be found by using various optimization algorithms, such as a fish swarm algorithm, a genetic algorithm, and a particle swarm algorithm;

[0102] Taking the genetic algorithm as an example:

[0103] S121, set the training accuracy threshold of the U-Net++ model as u1, and the training accuracy as u2;

[0104] S122, construct a chromosome population, and set the size of the chromosome population as j, then the chromosome population is expressed as k={k1,k2,...,k i ,...,k j}, wherein k i represents the i-th chromosome in the chromosome population, each chromosome represents a combination of the convolution kernel size and the convolution kernel quantity, and the maximum number of iterations of training is set as z1, and the current number of iterations of training is z2;

[0105] S123, substitute the parameter combination in the chromosome population into the U-Net++ model, and train the U-Net++ model using the labeled historical crack images to obtain a training accuracy; according to the training accuracy, the chromosomes in the chromosome population are subjected to cross and mutation operations to obtain a chromosome population subjected to cross and mutation operations;

[0106] S124, repeat S122 and S123, and when u2≥u1 or z2≥z1, stop iteration to obtain an optimal combination;

[0107] S2, construct a three-dimensional BIM model of the house, and plan a UAV flight path for the house to obtain a preset flight path;

[0108] The S2 includes the following steps:

[0109] S21, collect CAD drawings and construction material data of the house to obtain house data; import the house data into a BIM parametric modeling software to obtain a house three-dimensional BIM model; the house three-dimensional BIM model includes a vertex set, an edge set and an attribute set; the vertex set includes all geometric vertices of the house model, and these points are the most basic units of all geometric shapes (such as walls, floors, beams, columns, pipelines and equipment) constituting the model, and they are the cornerstone of geometric modeling, for example, at least 8 vertices are needed to define 8 corner points of a cuboid (such as a room or a beam)); the edge set includes all edges of the house model, and these edges connect isolated vertices to form the wireframe structure of the model. They are the basis of the surface contour and internal structure line of the model, for example, the 12 edges of a cuboid are 12 edges; the attribute set includes the attributes of the house model, and the attributes are a collection of non-geometric information attached to the geometric elements (vertices, edges, and more importantly, faces / surfaces and bodies / components composed of vertices and edges) of the model; such as component type, wall, door, window, beam, column, floor, pipeline, duct, equipment, furniture, material information, concrete strength grade C30, steel type Q345B, glass type, insulation material, size specification, thickness 200mm, diameter DN150, length 6000mm, width 1200mm;

[0110] S22, set key points on the facade of the house three-dimensional BIM model by a spatial grid sampling algorithm to obtain a house facade key point set; the house facade key point set includes all key points on the facade of the house; the spatial grid sampling algorithm is used to ensure that the house facade key point set can cover the entire facade of the house, such as at least 2 feature points per square meter;

[0111] Based on the house facade key point set, an API interface of the BIM software is called to automatically extract the coordinates of the four corners of the window opening (as the main positioning reference), the intersection points of the decorative lines (auxiliary positioning) and the pipeline wall penetration points (special area identification) of the facade of the house from the vertex set, the edge set and the attribute set of the house three-dimensional BIM model to obtain a house facade feature point set; the house facade feature point set includes feature point data of each facade key point in the house facade key point set;

[0112] S23, according to the house facade feature point set, plan a flight path for the unmanned aerial vehicle to obtain an executable flight path file of the unmanned aerial vehicle;

[0113] The S23 includes the following steps:

[0114] S231, set a safety distance according to the attribute set in the house three-dimensional BIM model; based on the outer envelope cube of the house three-dimensional BIM model and the safety distance, obtain a safe flight space range of the unmanned aerial vehicle;

[0115] S232, set the hierarchical height interval; according to the UAV safe flight space range and the hierarchical height interval, vertically divide the building into multiple layers to obtain a hierarchical height sequence; for example, calculate the number of layers according to the total height of the building (for example, a 100m high building, 100 / 3 = 34 layers), generate horizontal cutting planes with a 3m interval from the bottom to the top, and vertically divide the building into 34 layers;

[0116] S233, set the overlap rate threshold and hovering condition (such as the corner); extract the minimum convex polygon of the outer contour of each layer in the hierarchical height sequence by a convex optimization algorithm;

[0117] Expand the safe distance outside the minimum convex polygon to generate a closed annular path around the building; based on the closed annular path around the building, calculate the flight strip spacing according to the field of view angle of the UAV camera, so that the adjacent flight strips have an overlap rate > the overlap rate threshold, and obtain a single-layer annular flight path point set;

[0118] S234, according to the single-layer annular flight path point set and the hovering condition, insert a hovering instruction (pause for 1 second) at the vertex of the path corner, calculate the moving segment duration according to the UAV cruising speed (such as 2m / s), combine the hovering time to generate a time-stamped single-layer flight path function set;

[0119] Connect the flight paths of each layer in the order from the bottom layer to the top layer to form a spiral ascending path, and use vertical climbing for interlayer transition to obtain a complete three-dimensional flight path function;

[0120] Discretize the complete three-dimensional flight path function into a time-position-pose instruction sequence, encode it into a format recognizable by the flight control system, and obtain a preset flight path executable by the UAV;

[0121] S3, the UAV flies along the preset flight path, collects data during flight, returns to fly when data anomalies are detected, until the flight ends, and obtains metadata and image sequences;

[0122] The S3 includes the following steps:

[0123] S31, the UAV carries a multi-sensor pod; the multi-sensor pod includes a visible light camera, an infrared thermal imager, and an RTK positioning module; the multi-sensor pod includes a 20 million pixel visible light camera with a focal length of 35mm, which is used for crack detection; a 640x480 infrared thermal imager for identifying temperature anomalies caused by water seepage; and an RTK positioning module for outputting real-time centimeter-level POS data (longitude, latitude, elevation, roll, pitch, and yaw data)

[0124] S32, set the acquisition cycle, light intensity threshold and offset threshold; the unmanned aerial vehicle flies along the preset flight path, and in each acquisition cycle, image data and POS data are collected through the multi-sensor pod; the light intensity and the offset of the unmanned aerial vehicle are monitored in real time during the flight, and when the light intensity is lower than the light intensity threshold, the infrared mode shooting is switched on adaptively; when the offset of the unmanned aerial vehicle is greater than the offset threshold, the automatic return and reflight are triggered again until the flight ends, and the metadata and the image sequence are obtained;

[0125] S4, the image sequence is denoised and enhanced to obtain an enhanced image sequence; the features of the enhanced image sequence are extracted to obtain SIFT image features;

[0126] According to the SIFT image features, a bidirectional matching operation is performed to obtain a set of effective matching point pairs; and the accuracy of the set of effective matching point pairs is verified;

[0127] The S4 comprises the following steps:

[0128] S41, a 5*5 window filter is used to eliminate impulse noise in the image sequence to obtain a denoised image sequence;

[0129] The denoised image sequence is enhanced using CLAHE to limit the contrast adaptive histogram equalization and improve the local contrast of the image (especially to improve the shadow / light area) to obtain an enhanced image sequence;

[0130] S42, an improved SIFT algorithm is used to extract image feature points in the enhanced image sequence to obtain SIFT image features;

[0131] Based on the real-time pose of the unmanned aerial vehicle in the metadata and the intrinsic matrix of the camera, the mapping BIM between the three-dimensional space and the image coordinates is established according to the camera hardware; the house space projection is obtained by projecting the house BIM feature point set to the image plane;

[0132] Based on the SIFT image features, the house space projection, the house three-dimensional BIM model and the enhanced image, a bidirectional matching operation is performed to retain matching point pairs consistent in both directions to obtain a set of effective matching point pairs; the bidirectional matching operation comprises forward matching and reverse matching; the forward matching is searching for SIFT image features in the neighborhood of the projected points in the house space projection in the enhanced image, and the reverse matching is projecting the SIFT image features to the house three-dimensional BIM model to verify the consistency; the set of effective matching point pairs contains all effective matching point pairs; the effective matching point pairs contain house BIM feature point coordinates and enhanced image pixel coordinates;

[0133] S43, the house BIM feature point coordinates in the set of effective matching point pairs are projected to the image plane according to the current pose in the metadata to obtain expected pixel coordinates;

[0134] Calculate the Euclidean distance between the expected pixel coordinates and the enhanced image pixel coordinates to obtain a set of Euclidean distances; the calculation formula is as follows,

[0135]

[0136] Wherein, e i represents the Euclidean distance between the expected pixel coordinates and the enhanced image pixel coordinates of the ith effective matching point pair in the set of effective matching point pairs; p 1 i represents the expected pixel coordinates of the ith effective matching point pair, p i represents the enhanced image pixel coordinates of the ith effective matching point pair;

[0137] Calculate the average value of the set of Euclidean distances to obtain the average error; extract the maximum value in the set of Euclidean distances to obtain the maximum error;

[0138] Set the average error threshold and the maximum error threshold; when the average error is less than the average error threshold and the maximum error is less than the maximum error threshold, the precision verification is passed; otherwise, the precision verification is not passed, and return to S3;

[0139] S5, after the precision verification is passed, calculate the parameters of the crack binary mask obtained by combining the enhanced image sequence and the pre-trained U-Net++ model to obtain a set of crack parameters;

[0140] The S5 includes the following steps:

[0141] S51, input the enhanced image sequence into the pre-trained U-Net++ model to obtain a crack binary mask; fuse the coordinate positions of the crack binary mask and the coordinates in the effective matching point pairs to obtain the position of the house crack;

[0142] S52, perform a closing operation and a skeleton extraction operation on the crack binary mask to obtain an optimized crack skeleton graph; calculate the total pixel length of the skeleton chain in the optimized crack skeleton graph; convert the total pixel length of the skeleton chain into a physical length;

[0143] S53, take a normal section along the skeleton line of the optimized crack skeleton graph, and calculate the number of crack pixels in the section; convert the number of crack pixels in the section into a physical width; the position, physical length and width of the house crack together constitute a set of crack parameters;

[0144] Take the number of crack pixels in the section as an example; the conversion formula is as follows,

[0145]

[0146] wherein w represents the physical width of the crack of the house, h represents the number of pixels of the crack in the cross section, g represents the physical size of a single pixel on the camera sensor, m represents the actual vertical distance from the camera to the surface of the object where the crack is located, and f represents the focal length of the camera lens;

[0147] S6, registering the construction of the high-precision point cloud based on the enhanced image sequence and the metadata to obtain a registered point cloud; and calculating a house tilt angle based on the registered point cloud;

[0148] The S6 includes the following steps:

[0149] S61, generating a high-precision point cloud based on the sequence and the metadata using an SfM algorithm; the generation process includes extracting SIFT feature points of each image in the enhanced image sequence and performing cross-frame matching to obtain matched feature points; calculating camera poses and initial coordinates of spatial points based on the matched feature points; and minimizing re-projection errors (i.e., optimizing camera poses and three-dimensional point coordinates to minimize the distance between the projected points and the actual detected feature points when projecting the three-dimensional points back to the two-dimensional image) through non-linear optimization, and finally outputting a point cloud set of the target wall surface;

[0150] S62, registering the high-precision point cloud with a BIM three-dimensional model of the house using ICP registration to obtain a registered point cloud; the ICP registration specifically includes establishing a nearest point correspondence between the point cloud and the BIM plane, solving an optimal rotation matrix and a translation vector to minimize the distance residual of the point cloud to the plane; and repeating the above steps until convergence (convergence determination rules such as a residual threshold <0.5 mm);

[0151] S63, performing RANSAC plane segmentation on the registered point cloud to obtain an inner point set; calculating a covariance matrix of the inner point set; extracting the smallest eigenvalue of the covariance matrix of the inner point set; calculating the eigenvector of the smallest eigenvalue to obtain an actual normal vector;

[0152] An angle between the actual normal vector and a BIM reference normal vector is calculated by a spatial vector angle calculation formula to obtain a house tilt angle; the spatial vector angle calculation formula is as follows,

[0153]

[0154] wherein a represents the house tilt angle, cos -1 denotes an inverse cosine function, and nBIM represents the BIM reference normal vector and nc represents the actual normal vector;

[0155] S7, comprehensively determining a risk result according to the three-dimensional BIM model of the house, the crack parameter set, and the house tilt angle in combination with a risk rule library;

[0156] The S7 includes the following steps:

[0157] S71, a risk rule library is constructed; risk determination rules in the risk rule library are determined according to the construction attribute of the house, the house inclination angle and the house inclination angle; if the crack width is greater than or equal to 0.5 mm and located in a load-bearing wall, it is determined as high risk; if the crack width is greater than or equal to 0.3 mm and the age is greater than 20 years, it is determined as high risk; if the house inclination angle is greater than 2° and is a cantilever structure, it is determined as high risk; if the crack width is greater than 0.4 mm and the house inclination angle is greater than 1.5°, it is determined as high risk;

[0158] S72, according to the attribute set, the crack parameter set and the house inclination angle in the three-dimensional BIM model of the house, the comprehensive risk result determination is obtained by combining the risk rule library.

[0159] Embodiment two:

[0160] Please refer to Figure 4 A BIM-based unmanned aerial vehicle sensing house safety appraisal management system for realizing the above-mentioned BIM-based unmanned aerial vehicle sensing house safety appraisal management method, comprising a crack identification model training module, a BIM modeling and flight path planning module, an unmanned aerial vehicle multi-source data acquisition module, an image processing and matching module, a crack parameter quantification module, a point cloud registration and deformation analysis module and a safety risk intelligent evaluation module.

[0161] The crack identification model training module collects multi-scene crack images for labeling, and dynamically adjusts the convolution kernel parameters of the U-Net++ model, finally generates a deep learning model for crack detection meeting the preset accuracy threshold, and provides core algorithm support for subsequent automatic crack identification.

[0162] The BIM modeling and flight path planning module constructs a three-dimensional BIM model based on house CAD drawings and material data, extracts key feature points of the outer facade using a spatial grid sampling algorithm; generates a layered ring flight path through a convex hull optimization algorithm, combines the unmanned aerial vehicle hovering conditions and the overlap rate constraint to form an accurate flight path, and encodes it into an executable flight path file.

[0163] The unmanned aerial vehicle multi-source data acquisition module flies along the preset flight path by the unmanned aerial vehicle equipped with a visible light / infrared dual-mode camera and an RTK positioning module; automatically switches the shooting mode by real-time monitoring of the light intensity, dynamically detects the offset to trigger the return flight mechanism, synchronously acquires image sequences and POS data, and ensures data integrity and environmental adaptability.

[0164] The image processing and matching module performs pulse noise filtering and CLAHE contrast enhancement processing on the original image, extracts feature points by combining an improved SIFT algorithm, realizes image-BIM bidirectional matching by projecting BIM feature points, and completes sub-pixel level accuracy verification by calculating the Euclidean distance of matching points, thereby providing a geometric alignment data basis for subsequent analysis.

[0165] The crack parameter quantification module inputs the enhanced image into a pre-trained U-Net++ model to generate a crack binary mask, optimizes the crack morphology through closing operation and skeleton extraction, converts the physical size based on the skeleton chain pixel length and normal section width, and combines the camera parameters to output the crack spatial position and size parameter set in combination with the BIM coordinates.

[0166] The point cloud registration and deformation analysis module generates high-precision point cloud from the image sequence by using the SfM algorithm, aligns it with the BIM model by using the ICP registration technology, extracts the wall surface in-point set by using the RANSAC plane segmentation, calculates the spatial angle between the actual normal vector and the BIM reference normal vector, and accurately quantifies the house inclination angle.

[0167] The safety risk intelligent evaluation module integrates the crack parameters, inclination angle and BIM component attributes, performs multi-dimensional correlation analysis through a pre-defined risk rule library, and automatically outputs the house safety grade determination result, thereby providing a quantitative basis for decision-making.

[0168] In the description of the present specification, the description of the terms "one embodiment", "example", "specific example" and the like means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the invention. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0169] The preferred embodiments of the above disclosed invention are only used to help explain the invention. The preferred embodiments do not describe all the details and do not limit the invention to the specific embodiments described. Obviously, many modifications and changes can be made according to the content of the present specification. The present specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the invention, so that those skilled in the art can well understand and utilize the invention.

Claims

1. A BIM-based drone-based house safety assessment and management method, characterized by: The following steps are involved: S1. Train the U-Net++ model to obtain a pre-trained U-Net++ model; S2. Build a 3D BIM model of the house and plan the drone flight path for the house to obtain a preset track; S3: The drone flies along a preset trajectory, collects data during the flight, and returns to the starting point and flies again when an anomaly is detected until the flight ends, obtaining metadata and image sequences. S4, performing denoising and enhancement processing on the image sequence to obtain an enhanced image sequence; extracting features of the enhanced image sequence to obtain SIFT image features; Perform bidirectional matching based on SIFT image features to obtain a valid matching point pair set; perform accuracy verification based on the valid matching point pair set; S5. After the accuracy verification is passed, the parameters of the crack binary mask obtained by combining the enhanced image sequence with the pre-trained U-Net++ model are calculated to obtain the crack parameter set; S6. Registering the high-precision point cloud constructed based on the enhanced image sequence and metadata to obtain a registered point cloud; The tilt angle of the house is obtained based on the registered point cloud computing; S7. Based on the three-dimensional BIM model of the house, the crack parameter set, and the inclination angle of the house, combined with the risk rule library, a comprehensive risk result judgment is made.

2. The BIM-based drone-based house safety assessment and management method according to claim 1 is characterized in that: Said S1 comprises the following steps: S11. Build a U-Net++ model; set the initial convolution kernel size and initial number of convolution kernels of the U-Net++ model; S12, collecting crack images containing different lighting and material scenes to obtain historical crack images; performing mask annotation on the images to obtain annotated historical crack images; The U-Net++ model is trained using annotated historical crack images. During the training process, the optimal combination of convolution kernel size and number of convolution kernels for the U-Net++ model is found and the optimal combination is obtained. The convolution kernel size and number of convolution kernels in the optimal combination are used as the convolution kernel size and number of convolution kernels of the U-Net++ model to obtain a pre-trained U-Net++ model.

3. The BIM-based drone-based house safety assessment and management method according to claim 1 is characterized in that: The S2 comprises the following steps: S21. Collect CAD drawings and construction material data of the house to obtain house data; import the house data into BIM parametric modeling software to obtain a three-dimensional BIM model of the house; and obtain a vertex set, edge set, and attribute set in the three-dimensional BIM model of the house; S22. Setting key points on the facade of the three-dimensional BIM model of the house using a spatial grid sampling algorithm to obtain a key point set of the facade of the house; Based on the key point set of the building facade, calling the API interface of the BIM software, extracting features from the three-dimensional BIM model of the house, and obtaining the feature point set of the building facade; S23. Plan a UAV flight path for the house based on the house facade feature point set to obtain an executable track file for the UAV.

4. The BIM-based drone-based house safety assessment and management method according to claim 3 is characterized in that: The S23 includes the following steps: S231. Setting a safety distance based on a property set in the three-dimensional BIM model of the house; and obtaining a safe flight space range for the UAV based on the outer envelope cube of the three-dimensional BIM model of the house and the safety distance. S232, setting layer height intervals; dividing the building vertically into multiple layers based on the safe flight space range of the drone and the layer height intervals to obtain a layer height sequence; S233, setting an overlap rate threshold and a hovering condition; extracting the minimum convex polygon of the outer contour of each layer in the layered height sequence by using a convex hull optimization algorithm; A closed circular path around the building is generated by expanding the minimum convex polygon by a safe distance. Based on the closed circular path around the building, the drone camera's field of view calculates the flight strip spacing so that the overlap rate of adjacent flight strips exceeds the overlap rate threshold, and the current level circular track point set is obtained. S234. Obtain a single-layer track function set with a timestamp based on the single-layer circular track point set and the hovering condition; The single-layer track function set with timestamps is connected from the bottom layer to the top layer to form a spiral ascending path. The transition between layers adopts vertical climb to obtain a complete three-dimensional track function. The complete three-dimensional trajectory function discretizes the trajectory into a time-position-attitude instruction sequence, which is encoded into a format recognizable by the flight control system to obtain a preset trajectory that can be executed by the UAV.

5. The BIM-based drone-based house safety assessment and management method according to claim 1 is characterized in that: The S3 includes the following steps: S31. The UAV is equipped with a multi-sensor pod, wherein the multi-sensor pod includes a visible light camera, an infrared thermal imager, and an RTK positioning module; S32. Set the acquisition cycle, light intensity threshold, and offset threshold; the UAV flies along the preset track, and in each acquisition cycle, collects image data and POS data through the multi-sensor pod. During the flight, the light intensity and the UAV offset are monitored in real time. When the light intensity is lower than the light intensity threshold, the UAV adaptively switches to the infrared mode for shooting; when the UAV offset is greater than the offset threshold, it triggers an automatic return and re-flight until the end of the flight, and obtains metadata and image sequences.

6. The BIM-based drone-based house safety assessment and management method according to claim 1 is characterized in that: The S4 comprises the following steps: S41, using median filtering to eliminate noise in the image sequence to obtain a denoised image sequence; Use CLAHE to enhance the denoised image sequence to obtain an enhanced image sequence; S42, using an improved SIFT algorithm to extract image feature points in the enhanced image sequence to obtain SIFT image features; Project the house BIM feature point set onto the image plane to obtain the house space projection; Based on SIFT image features, house spatial projection, house 3D BIM model and enhanced image, a bidirectional matching operation is performed, and bidirectional consistent matching point pairs are retained to obtain a valid matching point pair set; The bidirectional matching operation includes forward matching and reverse matching; the forward matching is to search for SIFT image features in the neighborhood of the projection point in the house space projection in the enhanced image, and the reverse matching is to back-project the SIFT image features onto the house 3D BIM model to verify consistency; The effective matching point pair set includes all effective matching point pairs; the effective matching point pairs include the coordinates of the house BIM feature points and the enhanced image pixel coordinates; S43, projecting the coordinates of the building BIM feature points in the valid matching point pair set onto the image plane according to the current posture in the metadata to obtain the expected pixel coordinates; Calculate the Euclidean distance between the expected pixel coordinates and the enhanced image pixel coordinates to obtain the Euclidean distance set; Calculate the average value of the Euclidean distance set to get the average error; extract the maximum value in the Euclidean distance set to get the maximum error; Set the average error threshold and the maximum error threshold. When the average error is less than the average error threshold and the maximum error is less than the maximum error threshold, the accuracy verification passes. Otherwise, the accuracy verification fails and the process returns to S3.

7. The BIM-based drone-based house safety assessment and management method according to claim 1 is characterized in that: The S5 comprises the following steps: S51. After the accuracy verification is passed, the enhanced image sequence is input into the pre-trained U-Net++ model to obtain the crack binary mask; the coordinate position of the crack binary mask is fused with the coordinates of the valid matching point pair to obtain the location of the house crack; S52, performing a closing operation and a skeleton extraction operation on the crack binary mask to obtain an optimized crack skeleton image; calculating the total pixel length of the skeleton chain in the optimized crack skeleton image; and converting the total pixel length of the skeleton chain into a physical length; S53, taking a normal section along the skeleton line of the optimized crack skeleton diagram, calculating the number of crack pixels in the section; and converting the number of crack pixels in the section into a physical width.

8. The BIM-based drone-based house safety assessment and management method according to claim 1 is characterized in that: The S6 comprises the following steps: S61, based on the sequence and metadata, the SfM algorithm is used to generate high-precision point clouds; S62. Use ICP registration to register the high-precision point cloud with the building BIM 3D model to obtain a registered point cloud; S63, performing RANSAC plane segmentation on the registered point cloud to obtain an inlier point set; calculating the covariance matrix of the inlier point set; Extract the minimum eigenvalue of the covariance matrix of the inlier point set; calculate the eigenvector of the minimum eigenvalue to obtain the actual normal vector; The angle between the actual normal vector and the BIM reference normal vector is calculated using the space vector angle calculation formula to obtain the inclination angle of the house.

9. The BIM-based drone-based house safety assessment and management method according to claim 1 is characterized in that: The S7 comprises the following steps: S71. Build a risk rule base; S72. Based on the attribute set, crack parameter set, and building inclination angle in the three-dimensional BIM model of the building, combined with the risk rule library, a comprehensive risk result judgment is obtained.

10. A BIM-based drone-based house safety assessment management system, characterized by: A BIM-based drone-perception house safety appraisal management method as described in any one of claims 1 to 9 is implemented, wherein the system includes a crack recognition model training module, a BIM modeling and trajectory planning module, a drone multi-source data acquisition module, an image processing and matching module, a crack parameter quantification module, a point cloud registration and deformation analysis module, and a safety risk intelligent assessment module.

Images