A concrete crack management method and device combining a UAV and BIM

Abstract

The application provides a concrete crack management method and device combining a UAV and BIM, and relates to the technical field of intelligent detection of civil engineering structures. The method comprises: using a UAV to inspect the surface of concrete, combining fractal characteristics to obtain a crack area through lightweight semantic segmentation; extracting the attributes of each crack to form a crack semantic information set, simultaneously performing multi-view registration and three-dimensional positioning of depth sensing, and outputting the spatial position in the local coordinate system of the UAV and the world coordinate system. Then, the coordinates are converted to the BIM model coordinate system to realize accurate alignment with the BIM. A crack object primitive is created in the BIM and is associated with semantic information to form a crack instance data set, and synchronous updating is realized to achieve life cycle management. The whole method forms a closed-loop process of 'field collection - edge processing - model mapping - information updating - visual management', greatly improves the crack detection efficiency and data consistency, and is convenient for continuous monitoring and maintenance management of structural cracks.

Description

Technical Field

[0001] This invention relates to the field of intelligent detection technology for civil engineering structures, and in particular to a method and device for managing concrete cracks that combines drones and BIM. Background Technology

[0002] During long-term service, concrete bridges, buildings, and other engineering structures often develop defects such as cracks and spalling on their surfaces due to construction techniques, environmental factors, or repeated loads, seriously affecting structural safety and durability. Timely and accurate detection and identification of surface cracks are crucial for developing maintenance measures, eliminating safety hazards, and extending service life.

[0003] Traditional crack inspection relies primarily on manual close-range inspection and photographic recording, which is not only inefficient and inaccessible but also susceptible to human error, making it difficult to manage and store the acquired image data in a standardized manner. With the development of drone technology, using drones equipped with high-definition cameras for crack inspection of bridges or buildings has become a mainstream method. Drones are efficient and flexible, capable of covering areas difficult to reach manually, and quickly acquiring images of large areas of structural surfaces, providing convenient conditions for crack detection. During the operation and maintenance phase of engineering structures, comprehensive and accurate management of crack and other defects is urgently needed. BIM technology can integrate various types of information throughout the entire lifecycle of a structure, providing an effective means to support long-term operation and maintenance management of infrastructure. Linking the crack detection results obtained from drone inspections with the BIM 3D model allows for the location and labeling of cracks in 3D space and information association, providing an intuitive basis for subsequent maintenance decisions.

[0004] However, linking UAV image data with BIM models still faces numerous challenges: traditional image-model registration methods mostly rely on crack texture features or manually marked reference points. The surface texture of concrete structures such as bridges is simple and lacks sufficient features, making automatic registration and positioning based on image features difficult to achieve. Some studies have attempted to use GPS information embedded in UAV images for crack location, but its application in the field and accuracy verification are still limited, and a mature solution has not yet been developed. Furthermore, the large amount of high-definition images acquired by UAVs in existing technologies usually needs to be transmitted back to the ground for offline processing. Data transmission and processing are time-consuming, making it impossible to achieve real-time crack identification and feedback during inspections. Therefore, a new technical solution is urgently needed that can fully utilize the mobility and on-site computing capabilities of UAV platforms to achieve real-time crack detection and precise location, and automatically integrate the detection results into the BIM operation and maintenance model for visualized management. This will greatly improve the efficiency and informatization of crack inspections, providing strong support for structural maintenance decisions. Summary of the Invention

[0005] To address the problem that existing image-model registration methods largely rely on crack texture features or manually marked reference points, which are insufficient due to the limited surface texture and features of concrete structures such as bridges, making automatic registration and positioning based on image features difficult to achieve; furthermore, existing technologies often require offline processing of large amounts of high-definition images acquired by UAVs, resulting in time-consuming data transmission and processing, hindering real-time crack identification and feedback during inspections, and preventing the automatic integration of detection results into the BIM operation and maintenance model for visual management, this invention provides a concrete crack management method and device combining UAVs and BIM. The technical solution is as follows:

[0006] On the one hand, a method for managing concrete cracks combining drones and BIM is provided. This method is implemented by a concrete crack management device that combines drones and BIM, and includes: S1: Use drones to inspect the surface of concrete structures and obtain image data and control point data; S2: At the UAV end, perform lightweight crack semantic segmentation on the image data by combining fractal feature analysis to obtain a segmentation result containing the crack region; S3: Based on the segmentation results of the cracked region, extract the crack mask and attribute information of each crack to obtain a crack semantic information set. The attribute information includes the crack size, direction, component to which it belongs, severity, current state, image acquisition time, and the attitude and positioning coordinate data of the UAV itself. S4: Based on the crack mask and control point data, perform three-dimensional positioning calculations on the crack to obtain the spatial location of the crack and the location of the control points. The three-dimensional positioning calculation includes multi-view image registration and depth sensing methods. The multi-view image registration obtains the crack point cloud through feature matching and spatial reconstruction. The depth sensing method uses lidar point cloud data to project the crack pixels in the image onto real-world three-dimensional points and calculates the depth and location of the crack. The spatial location of the crack is expressed through a spatial coordinate system, which includes the UAV local coordinate system and world coordinates. S5: Perform coordinate system transformation on the spatial location of the crack, transforming the spatial coordinate system to the BIM model coordinate system of the corresponding structure, to obtain the crack coordinates that are precisely aligned with the BIM 3D model. The coordinate system transformation is achieved by a coordinate transformation matrix. The coordinate transformation matrix is ​​calibrated using at least two known control points and then the coordinate transformation matrix parameters are obtained through parameter calculation. The coordinate transformation matrix parameters are homogeneous transformation matrices. The parameter calculation includes SVD, least squares method, and iterative optimization algorithm. The homogeneous transformation matrix is ​​formed by combining the rotation matrix R and the translation vector t. S6: Based on the crack coordinates that are precisely aligned with the BIM 3D model, create a component entity of the crack object in the BIM model, associate the crack semantic information set with the corresponding component entity in the BIM model, and obtain a crack instance dataset in the BIM model. The crack object integrates the crack geometry and semantic information in the form of primitives in the model. S7: Continuously collect crack semantic information and synchronize newly added or updated crack information based on the same crack multi-source data merging strategy to obtain crack instances in the updated BIM model. The crack instances in the updated BIM model are used for visual tracking of cracks and closed-loop management of crack life cycle. The same crack multi-source data merging strategy includes reasonable crack topology assessment and life cycle identification using predetermined symbols or colors. The reasonable crack topology assessment includes extracting a merging threshold based on box counting fractal dimension from historical data.

[0007] Preferably, step S1 utilizes a drone to inspect the surface of the concrete structure, obtaining image data and control point data, including: S11: Using drones, conduct on-site inspections of the concrete structure surface. The on-site inspection includes a method of collecting images of cracks and measuring the actual crack locations by having the drone maintain a constant distance of 2-3 meters and fly parallel across the web surface. The collection method takes the surface of the component under test, with the bridge web as an approximate plane, as the object. S12: After reaching a position near the surface of the component to be tested, dynamically adjust the flight attitude of the UAV to keep the camera optical axis perpendicular to the surface of the component to be tested; S13: Achieve close-range shooting using a gimbal camera, acquire high-resolution images of the surface of the component under test, and obtain image data and control point data; S14: Collect control point data through a gimbal camera. The control point data is obtained by selecting at least two easily identifiable reference points on the surface of the concrete structure and measuring their coordinates in the UAV coordinate system.

[0008] Preferably, in step S3, based on the segmentation results containing the crack region, the crack mask and attribute information of each crack are extracted to obtain a crack semantic information set. The attribute information includes the crack size, direction, component to which it belongs, severity, current state, image acquisition time, and the attitude and positioning coordinates of the UAV itself, including: S31: On the drone side, a preprocessing step is first performed on the image data to obtain preprocessed image data. The preprocessing step includes median filtering noise reduction to reduce noise interference. S32: Input the pre-processed image data into the pre-trained semantic segmentation network, and run it in real time at the video frame rate in the embedded environment to obtain the segmentation result. The pre-trained semantic segmentation network is a crack segmentation model based on full convolution. The crack segmentation model is built based on the improved VGG16 or DeepLab architecture. S33: Extract the attribute information of each crack based on the segmentation results. The attribute information includes the crack shape, acquisition timestamp, UAV pose coordinates, latitude and longitude coordinates of the shooting location, altitude and attitude angle. S34: Perform fractal feature analysis based on the image data to obtain fractal features. The fractal feature analysis includes the generation of prompts based on the fractal dimension of local box counting in the image. The fractal dimension of local box counting in the image includes covering the image with a grid at different scales, counting the number of grids containing crack pixels at each scale, and estimating the local fractal dimension accordingly. S35: The attribute information and the fractal features are transmitted back to the ground station via a wireless network.

[0009] Preferably, in step S4, based on the crack mask and control point data, a three-dimensional localization calculation is performed on the crack to obtain the spatial location of the crack and the location of the control points. The three-dimensional localization calculation includes multi-view image registration and depth sensing methods. The multi-view image registration obtains the crack point cloud through feature matching and spatial reconstruction. The depth sensing method uses lidar point cloud data to project crack pixels in the image onto real-world three-dimensional points and calculates the depth and location of the crack. The spatial location of the crack is expressed through a spatial coordinate system, which includes the UAV local coordinate system and world coordinates, including: S41: Extract location information, IMU attitude angle and UAV position coordinates from the crack semantic information set to obtain the spatial geometric position of the crack on the structural surface. The spatial geometric position is expressed by a pixel coordinate system. S42: Multi-view image registration is used to perform feature matching and spatial reconstruction to obtain crack point cloud. The multi-view image registration includes feature matching and spatial reconstruction of multi-view images. The crack point cloud includes a set of points representing the spatial distribution of cracks. S43: By using a depth sensing method, laser radar point cloud data is collected to obtain the relationship between the point cloud data and the spatial geometric position of the crack on the structural surface, and to obtain the elevation of the plane where the component is located and the position of the crack center in the image. The depth sensing method uses laser radar point cloud data for projection. S44: Project the UAV's position coordinates onto the plane where the component is located, and obtain the coordinates of the image center on the surface of the structure based on the elevation of the plane where the component is located; S45: Based on the pixel offset of the position of the crack center in the image relative to the image center, the actual horizontal distance corresponding to the pixel offset is obtained after conversion of camera intrinsic parameters and height. The pixel offset is with reference to the image center, and the camera intrinsic parameters include parameters for imaging scale conversion. S46: Using the actual horizontal distance, perform superposition calculation with the spatial geometric position of the crack on the structural surface to obtain the spatial position of the crack. The superposition calculation includes adding the actual horizontal distance to the spatial geometric position. The spatial position of the crack is expressed in a spatial coordinate system. The spatial coordinate system expression includes a dual coordinate result for positioning and expression. The spatial coordinate system includes the UAV local coordinate system and world coordinates. The dual coordinate result uses the structural surface as a reference to ensure positional consistency.

[0010] Preferably, in step S5, the spatial location of the crack is transformed to the BIM model coordinate system of the corresponding structure, obtaining crack coordinates precisely aligned with the BIM 3D model. This coordinate transformation is achieved using a coordinate transformation matrix. The coordinate transformation matrix is ​​calibrated using at least two known control points, and then the coordinate transformation matrix parameters are calculated. These parameters are homogeneous transformation matrices, and the parameter calculation includes SVD, least squares method, and iterative optimization algorithms. The homogeneous transformation matrix is ​​formed by combining a rotation matrix R and a translation vector t, and includes: S51: Extract the coordinate values ​​of control points in the UAV coordinate system and BIM model coordinate system based on control point data; S52: Based on the coordinate values ​​of the control points in the UAV coordinate system and the BIM model coordinate system, a coordinate transformation matrix is ​​obtained through rigid body transformation and parameter calculation. The coordinate transformation matrix adopts a homogeneous transformation form and includes transformation parameters. The transformation parameters include a rotation matrix R, a translation vector t, and an optional uniform scale factor. The parameter calculation includes singular value decomposition (SVD), least squares method, and iterative optimization algorithm. S53: Based on the coordinate transformation matrix, apply the two-level transformation to all spatial points or point cloud coordinates of the crack to obtain crack coordinates that are precisely aligned with the BIM 3D model. The two-level transformation includes calling the Gaussian projection algorithm to convert the geographic coordinates obtained by the UAV into a Cartesian coordinate system, and then based on the coordinate transformation matrix, converting the Cartesian coordinate system into the BIM model coordinate system.

[0011] Preferably, in step S6, based on the crack coordinates precisely aligned with the BIM 3D model, a component entity of the crack object is created in the BIM model. The crack semantic information set is associated with the corresponding component entity in the BIM model to obtain a crack instance dataset in the BIM model. The crack object integrates crack geometry and semantic information in the model in the form of primitives, including: S61: Based on the crack coordinates that are precisely aligned with the BIM 3D model, automatically create crack objects to obtain crack instances in the BIM model. The automatic creation of crack objects includes inserting a predefined crack primitive on the surface of the corresponding component at the determined crack spatial location as a geometric representation with dimensions and location consistent with the actual crack. The crack primitive takes the form of a slender polyline segment, curve, or two-dimensional symbol, and is used to store it in the model in the form of parametric primitives and establish data association with the surface of the structural component where it is located. S62: Input the crack semantic information set into the parameter field of the crack object. The parameter field includes crack number, discovery time, spatial location, geometric dimensions, severity and current status. The geometric dimensions include length, average width, maximum width and depth. S63: Using predetermined symbols or colors, establish visual identifiers for crack objects to obtain differentiated displays of different cracks and their states. The predetermined symbols or colors include styles set according to lifecycle identifiers. The lifecycle identifiers include newly discovered, continuously expanding, stable, repaired, and / or severity levels.

[0012] Preferably, in step S7, the continuous acquisition of crack semantic information and the synchronization of newly added or updated crack information based on a multi-source data merging strategy for the same crack result in crack instances in the updated BIM model. These updated crack instances are used for visual tracking and closed-loop management of the crack lifecycle. The multi-source data merging strategy for the same crack includes reasonable crack topology assessment and lifecycle identification using predetermined symbols or colors. The reasonable crack topology assessment includes extracting a merging threshold based on box-count fractal dimension from historical data, including: S71: Continuously collect crack semantic information from the inspection and model maintenance process, and add the newly acquired crack records to the crack instance dataset in the BIM model; S72: Obtain the box count fractal dimension of the newly acquired crack record, and combine it with the center point location, length and orientation information to determine whether there are mergeable objects in the existing crack instance dataset of the newly added or updated crack. The mergeable objects refer to crack instances with similar attributes in the existing crack instance dataset of the BIM model. The box count fractal dimension, length and orientation information are used to calculate the life cycle identifier. S73: If a mergeable object exists, update the parameters of the mergeable object and append a lifecycle identifier to the history property. If no mergeable object exists, create a new crack instance to ensure that the same crack corresponds to only one object.

[0013] On the other hand, a concrete crack management device combining drones and BIM is provided. This device is applied to a concrete crack management method combining drones and BIM, and includes: Inspection module: Used to inspect the surface of concrete structures using drones, and obtain image data and control point data; Semantic segmentation module: used to perform lightweight crack semantic segmentation on the image data by combining fractal feature analysis on the UAV end, and obtain segmentation results containing crack regions; Information extraction module: Based on the segmentation results containing the crack region, extract the crack mask and attribute information of each crack to obtain a crack semantic information set. The attribute information includes the crack size, direction, component to which it belongs, severity, current status, image acquisition time, and the attitude and positioning coordinate data of the UAV itself. The 3D positioning module is used to perform 3D positioning calculations on the cracks based on the crack mask and control point data, to obtain the spatial location of the cracks and the location of the control points. The 3D positioning calculation includes multi-view image registration and depth sensing methods. The multi-view image registration obtains crack point clouds through feature matching and spatial reconstruction. The depth sensing method uses lidar point cloud data to project crack pixels in the image onto real-world 3D points and calculates the depth and location of the cracks. The spatial location of the cracks is expressed through a spatial coordinate system, which includes the UAV local coordinate system and world coordinates. Coordinate Alignment Module: Used to perform coordinate system transformation on the spatial location of the crack, transforming the spatial coordinate system to the BIM model coordinate system of the corresponding structure, and obtaining crack coordinates that are precisely aligned with the BIM 3D model. The coordinate system transformation is achieved by a coordinate transformation matrix. The coordinate transformation matrix is ​​calibrated using at least two known control points and then the coordinate transformation matrix parameters are obtained through parameter calculation. The coordinate transformation matrix parameters are homogeneous transformation matrices. The parameter calculation includes SVD, least squares method and iterative optimization algorithm. The homogeneous transformation matrix is ​​formed by combining the rotation matrix R and the translation vector t. BIM Integration Module: Based on the crack coordinates that are precisely aligned with the BIM 3D model, it creates component entities of crack objects in the BIM model, associates the crack semantic information set with the corresponding component entities in the BIM model, and obtains a crack instance dataset in the BIM model. The crack objects integrate crack geometry and semantic information in the model in the form of primitives. Lifecycle Module: This module continuously collects crack semantic information and, based on a multi-source data merging strategy for the same crack, synchronizes newly added or updated crack information to obtain crack instances in the updated BIM model. These crack instances in the updated BIM model are used for visual tracking of cracks and closed-loop management of the crack lifecycle. The multi-source data merging strategy for the same crack includes reasonable crack topology assessment and lifecycle identification using predetermined symbols or colors. The reasonable crack topology assessment includes extracting a merging threshold based on box-count fractal dimension from historical data.

[0014] On the other hand, a concrete crack management device combining drones and BIM is provided. The concrete crack management device combining drones and BIM includes: a processor; a memory storing computer-readable instructions, which, when executed by the processor, implement the method described in any one of the above-described concrete crack management methods combining drones and BIM.

[0015] On the other hand, a computer-readable storage medium is provided, characterized in that program code is stored in the computer-readable storage medium.

[0016] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: This method enables real-time crack identification and feedback during inspections without relying on crack texture features or manually marked reference points, and automatically integrates the detection results into the BIM operation and maintenance model for visual management. Combining crack semantic information and fractal dimension can compensate for the limitations of simple surface textures and insufficient features in concrete structures such as bridges. The entire method forms a closed-loop process of "on-site acquisition – edge processing – model mapping – information update – visual management," significantly improving crack detection efficiency and data consistency, facilitating continuous monitoring and maintenance management of structural cracks. Specifically:

[0017] (1) Embedded fractal intelligent detection on UAV. Utilizing the embedded computing unit on the UAV platform, a lightweight algorithm integrating fractal features is used to achieve real-time image processing and crack identification on-site. The UAV can complete semantic segmentation and feature extraction of cracks during the inspection process, obtaining detection results without transmitting the original images back to the ground or cloud, thereby significantly reducing communication bandwidth usage and transmission delay. This "front-end intelligence + back-end lightweight" architecture fully leverages the flexibility and maneuverability of UAVs, greatly improving the autonomy and efficiency of inspections.

[0018] (2) Point Cloud Fusion and Precise Coordinate Mapping. This invention significantly improves the accuracy of crack spatial positioning by introducing the fusion of point cloud data and image information, as well as a precise coordinate transformation method. A transformation matrix including translation, rotation, and scaling is established using known calibration points to accurately convert the coordinates of the crack in the measured space to the corresponding position in the BIM model. This process ensures the consistency and high-precision alignment between the UAV measurement coordinate system and the BIM model coordinate system, solving the problem of coordinate docking when fusing UAV image information with the BIM model. At the same time, the use of LiDAR point cloud assistance can compensate for the shortcomings of relying solely on image positioning, making the three-dimensional repositioning results of the crack more reliable.

[0019] (3) Crack information is automatically updated to the BIM operation and maintenance model. The system provided by this invention can automatically trigger the update operation of the BIM model after cracks are identified: creating crack objects and writing their parameterized attributes and status information. By developing plug-in interfaces or Dynamo visualization scripts on BIM platforms such as Autodesk Revit, the entire process from crack location results to model update can be automated. Once the UAV completes the inspection, the identified crack information will be immediately reflected as digital components in the BIM model without manual intervention, realizing efficient linkage between UAV inspection data and the BIM operation and maintenance model. Compared with the traditional method of manually marking the location of cracks and recording information in the model, this invention significantly improves the timeliness and accuracy of information entry.

[0020] (4) Closed-loop visual management and operation and maintenance decision support. This invention constructs a crack management subsystem, which intuitively presents crack detection and location results on the BIM 3D model, and stores the data from each detection for trend analysis, forming a closed-loop process from "on-site collection - edge processing" to "model mapping - information update - visual display". Crack data is always up-to-date and traceable. Operation and maintenance personnel can view key indicators such as the length, width and growth trend of each crack in real time through the platform, remotely monitor the structural condition, and help to discover hidden dangers in a timely manner and formulate maintenance plans. This digital management method improves the consistency and utilization of detection data and promotes the intelligent upgrade of infrastructure operation and maintenance management. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of a concrete crack management method combining drones and BIM provided by an embodiment of the present invention; Figure 2 This is a schematic diagram of a structure provided by an embodiment of the present invention; Figure 3 This is a block diagram of a concrete crack management device combining drones and BIM provided in an embodiment of the present invention; Figure 4 This is a structural schematic diagram of a concrete crack management device combining drones and BIM provided in an embodiment of the present invention. Detailed Implementation

[0023] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0024] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0025] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0026] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0027] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0028] This invention provides a method for managing concrete cracks by combining drones and BIM. This method can be implemented by a concrete crack management device that combines drones and BIM, which can be a terminal or a server. Figure 1 The flowchart shown illustrates a concrete crack management method combining drones and BIM. This method's process may include the following steps:

[0029] S1: Use drones to inspect the surface of concrete structures and obtain image data and control point data; Preferably, S1 includes: S11: Using drones, conduct on-site inspections of the concrete structure surface. The on-site inspection includes a method of collecting images of cracks and measuring the actual crack locations by having the drone maintain a constant distance of 2-3 meters and fly parallel across the web surface. The collection method takes the surface of the component under test, with the bridge web as an approximate plane, as the object. S12: After reaching a position near the surface of the component to be tested, dynamically adjust the flight attitude of the UAV to keep the camera optical axis perpendicular to the surface of the component to be tested; S13: Achieve close-range shooting using a gimbal camera, acquire high-resolution images of the surface of the component under test, and obtain image data and control point data; S14: Collect control point data through a gimbal camera. The control point data is obtained by selecting at least two easily identifiable reference points on the surface of the concrete structure and measuring their coordinates in the UAV coordinate system.

[0030] In some embodiments, a drone equipped with a camera and lidar is used to inspect a target concrete structure, collecting multi-temporal and multi-view images and point cloud data, while simultaneously recording the drone's pose (attitude matrix). , ) and positioning coordinates.

[0031] S2: At the UAV end, perform lightweight crack semantic segmentation on the image data by combining fractal feature analysis to obtain a segmentation result containing the crack region; Preferably, S2 includes: S21: On the drone side, a preprocessing step is first performed on the image data to obtain preprocessed image data. The preprocessing step includes median filtering noise reduction to reduce noise interference. S22: Input the pre-processed image data into the pre-trained semantic segmentation network, and run it in real time at the video frame rate in the embedded environment to obtain the segmentation result. The pre-trained semantic segmentation network is a crack segmentation model based on full convolution. The crack segmentation model is built based on the improved VGG16 or DeepLab architecture. S23: Extract the attribute information of each crack based on the segmentation results. The attribute information includes the crack shape, acquisition timestamp, UAV pose coordinates, latitude and longitude coordinates of the shooting location, altitude and attitude angle. S24: Perform fractal feature analysis based on the image data to obtain fractal features. The fractal feature analysis includes the generation of prompts based on the fractal dimension of local image box counting. The fractal dimension of local image box counting includes covering the image with a grid at different scales, counting the number of grids containing crack pixels at each scale, and estimating the local fractal dimension accordingly. S25: The attribute information and the fractal features are transmitted back to the ground station via a wireless network.

[0032] In some embodiments, a crack segmentation model based on a fully convolutional network can be used to fuse more attributes into the feature map during the encoding stage. After network inference, a crack mask map is output. (The crack pixel is marked as the foreground).

[0033] S3: Based on the segmentation results of the cracked region, extract the crack mask and attribute information of each crack to obtain a crack semantic information set. The attribute information includes the crack size, direction, component to which it belongs, severity, current state, image acquisition time, and the attitude and positioning coordinate data of the UAV itself. In some embodiments, in the mask Extract the connected crack regions and calculate attribute information for each crack contour: the center position of the crack pixel. The projection length (which can be accumulated pixel spacing or based on curve fitting), width (pixel span along the vertical direction), acquisition timestamp, and UAV pose coordinates, etc.

[0034] S4: Based on the crack mask and control point data, perform three-dimensional positioning calculations on the crack to obtain the spatial location of the crack and the location of the control points. The three-dimensional positioning calculation includes multi-view image registration and depth sensing methods. The multi-view image registration obtains the crack point cloud through feature matching and spatial reconstruction. The depth sensing method uses lidar point cloud data to project the crack pixels in the image onto real-world three-dimensional points and calculates the depth and location of the crack. The spatial location of the crack is expressed through a spatial coordinate system, which includes the UAV local coordinate system and world coordinates. Preferably, S4 includes: S41: Extract location information, IMU attitude angle and UAV position coordinates from the crack semantic information set to obtain the spatial geometric position of the crack on the structural surface. The spatial geometric position is expressed by a pixel coordinate system. S42: Multi-view image registration is used to perform feature matching and spatial reconstruction to obtain crack point cloud. The multi-view image registration includes feature matching and spatial reconstruction of multi-view images. The crack point cloud includes a set of points representing the spatial distribution of cracks. S43: By using a depth sensing method, laser radar point cloud data is collected to obtain the relationship between the point cloud data and the spatial geometric position of the crack on the structural surface, and to obtain the elevation of the plane where the component is located and the position of the crack center in the image. The depth sensing method uses laser radar point cloud data for projection. S44: Project the UAV's position coordinates onto the plane where the component is located, and obtain the coordinates of the image center on the surface of the structure based on the elevation of the plane where the component is located; S45: Based on the pixel offset of the position of the crack center in the image relative to the image center, the actual horizontal distance corresponding to the pixel offset is obtained after conversion of camera intrinsic parameters and height. The pixel offset is with reference to the image center, and the camera intrinsic parameters include parameters for imaging scale conversion. S46: Using the actual horizontal distance, perform superposition calculation with the spatial geometric position of the crack on the structural surface to obtain the spatial position of the crack. The superposition calculation includes adding the actual horizontal distance to the spatial geometric position. The spatial position of the crack is expressed in a spatial coordinate system. The spatial coordinate system expression includes a dual coordinate result for positioning and expression. The spatial coordinate system includes the UAV local coordinate system and world coordinates. The dual coordinate result uses the structural surface as a reference to ensure positional consistency.

[0035] In some embodiments, for each pixel in the crack mask If its corresponding depth is known (Can be viewed by multiple views) Figure 3 (If the data is provided by angle measurement or LiDAR point cloud), then first calculate the three-dimensional coordinates of the pixel in the camera coordinate system:

[0036] in, It is the camera intrinsic parameter matrix. That is, the camera coordinates. Then, the drone's attitude transformation is used to convert them to the world coordinate system: i.e. Traversing all crack pixels yields a dense set of 3D crack points. If multiple frames can be processed simultaneously, a multi-view registration technique is employed: registering and fusing crack point clouds from different viewpoints, or directly mapping the point cloud. Specifically, the mask image is traversed, and each crack pixel searches for the nearest point in the point cloud along the ray; if their depths are consistent, they are marked as crack points. This process yields the 3D coordinate point cloud of cracks for each frame, and the data from multiple frames are unified under the same reference frame and fused to form a complete 3D crack point cloud model covering the structural surface. Filtering algorithms can be combined to remove noise points, ensuring the accuracy of the 3D model.

[0037] S5: Perform coordinate system transformation on the spatial location of the crack, transforming the spatial coordinate system to the BIM model coordinate system of the corresponding structure, to obtain the crack coordinates that are precisely aligned with the BIM 3D model. The coordinate system transformation is achieved by a coordinate transformation matrix. The coordinate transformation matrix is ​​calibrated using at least two known control points and then the coordinate transformation matrix parameters are obtained through parameter calculation. The coordinate transformation matrix parameters are homogeneous transformation matrices. The parameter calculation includes SVD, least squares method, and iterative optimization algorithm. The homogeneous transformation matrix is ​​formed by combining the rotation matrix R and the translation vector t. Preferably, S5 includes: S51: Extract the coordinate values ​​of control points in the UAV coordinate system and BIM model coordinate system based on control point data; S52: Based on the coordinate values ​​of the control points in the UAV coordinate system and the BIM model coordinate system, a coordinate transformation matrix is ​​obtained through rigid body transformation and parameter calculation. The coordinate transformation matrix adopts a homogeneous transformation form and includes transformation parameters. The transformation parameters include a rotation matrix R, a translation vector t, and an optional uniform scale factor. The parameter calculation includes singular value decomposition (SVD), least squares method, and iterative optimization algorithm. S53: Based on the coordinate transformation matrix, apply the two-level transformation to all spatial points or point cloud coordinates of the crack to obtain crack coordinates that are precisely aligned with the BIM 3D model. The two-level transformation includes calling the Gaussian projection algorithm to convert the geographic coordinates obtained by the UAV into a Cartesian coordinate system, and then based on the coordinate transformation matrix, converting the Cartesian coordinate system into the BIM model coordinate system.

[0038] In some embodiments, it is assumed that the correspondence between at least three control points in the two coordinate systems is known. Rigid body transformations can be solved: ,in Let be a rotation matrix. This is a translation vector. Specifically, it can be calculated using SVD or least squares methods. , Furthermore, iterative optimization (such as the ICP algorithm) is used to improve registration accuracy. This ensures that the coordinates of all crack points are accurately mapped to the BIM model coordinate system, guaranteeing that the crack locations correspond to and are aligned with the components in the 3D model. The transformation process can be represented as a homogeneous transformation matrix. This method allows for the precise mapping of crack coordinates in the real-world space to the corresponding coordinate system in the BIM model. Through this method, the location data of cracks in the measured space is seamlessly integrated into the BIM model, ensuring that the crack annotations in the 3D model perfectly match the actual structural locations. This lays the foundation for subsequently marking crack locations intuitively in the model.

[0039] S6: Based on the crack coordinates that are precisely aligned with the BIM 3D model, create a component entity of the crack object in the BIM model, associate the crack semantic information set with the corresponding component entity in the BIM model, and obtain a crack instance dataset in the BIM model. The crack object integrates the crack geometry and semantic information in the form of primitives in the model. Preferably, S6 includes: S61: Based on the crack coordinates that are precisely aligned with the BIM 3D model, automatically create crack objects to obtain crack instances in the BIM model. The automatic creation of crack objects includes inserting a predefined crack primitive on the surface of the corresponding component at the determined crack spatial location as a geometric representation with dimensions and location consistent with the actual crack. The crack primitive takes the form of a slender polyline segment, curve, or two-dimensional symbol, and is used to store it in the model in the form of parametric primitives and establish data association with the surface of the structural component where it is located. S62: Input the crack semantic information set into the parameter field of the crack object. The parameter field includes crack number, discovery time, spatial location, geometric dimensions, severity and current status. The geometric dimensions include length, average width, maximum width and depth. S63: Using predetermined symbols or colors, establish visual identifiers for crack objects to obtain differentiated displays of different cracks and their states. The predetermined symbols or colors include styles set according to lifecycle identifiers. The lifecycle identifiers include newly discovered, continuously expanding, stable, repaired, and / or severity levels.

[0040] In some embodiments, after coordinate mapping is completed, crack entities are automatically created or updated in the corresponding BIM model through the development interface of the BIM platform (such as Revit API, IFC interface, Dynamo, etc.). Specifically, the spatial location of the crack is associated with the corresponding structural component object in the BIM model, and the crack's attribute information is written, including but not limited to: crack number (unique identifier), discovery time, spatial location (e.g., coordinates on the component or component ID and relative position), geometric dimensions (length, average width, maximum width, depth, etc.), severity (which can be assessed based on crack width or propagation rate), and current status (e.g., newly discovered / continuously propagating / stable / repaired). The attribute information can be written into the attribute set of the BIM model or stored by attaching custom attribute fields, enabling the BIM model to reflect the latest crack detection results in real time. Simultaneously, the location and shape of the crack are visually presented in the model. For example, the system can insert a predefined crack primitive as the geometric representation of the crack on the surface of the corresponding component at the determined spatial location of the crack. The crack element can take the form of a slender polyline segment, curve, or two-dimensional symbol, with dimensions and location matching the actual crack, used to identify the spatial direction and extent of the crack. Simultaneously with inserting the crack element, the aforementioned crack attributes are written into the element's parameters, associating it with the model in a parametric form. For example, information such as crack number, length, maximum width, direction, discovery date, and inspector can be assigned to a custom parameter field of the element, achieving attribute information embedding. It should be noted that, to visually distinguish different cracks and their severity in the model, the display style of crack objects (e.g., color, line type, or marker symbol) can be automatically set according to crack width or health rating, and highlighted with a preset legend when the model is loaded. The method of this invention supports the use of open formats such as IFC to save and exchange crack-related information, facilitating data interoperability with other operation and maintenance systems.

[0041] S7: Continuously collect crack semantic information and synchronize newly added or updated crack information based on the same crack multi-source data merging strategy to obtain crack instances in the updated BIM model. The crack instances in the updated BIM model are used for visual tracking of cracks and closed-loop management of crack life cycle. The same crack multi-source data merging strategy includes reasonable crack topology assessment and life cycle identification using predetermined symbols or colors. The reasonable crack topology assessment includes extracting a merging threshold based on box counting fractal dimension from historical data.

[0042] Preferably, S7, as Figure 2 As shown, it includes: S71: Continuously collect crack semantic information from the inspection and model maintenance process, and add the newly acquired crack records to the crack instance dataset in the BIM model; S72: Obtain the box count fractal dimension of the newly acquired crack record, and combine it with the center point location, length and orientation information to determine whether there are mergeable objects in the existing crack instance dataset of the newly added or updated crack. The mergeable objects refer to crack instances with similar attributes in the existing crack instance dataset of the BIM model. The box count fractal dimension, length and orientation information are used to calculate the life cycle identifier. S73: If a mergeable object exists, update the parameters of the mergeable object and append a lifecycle identifier to the history property. If no mergeable object exists, create a new crack instance to ensure that the same crack corresponds to only one object.

[0043] In some embodiments, concrete cracks typically exhibit irregular fractal features and self-similarity. This invention utilizes box counting to calculate the fractal dimension of local image regions as a feature cues for identifying mergeable objects. Specifically, different scales are selected... The crack image is covered by a grid, and the number of grids containing crack pixels at each scale is counted. Using relationships Estimating fractal dimension Fractal dimension reflects the complexity of crack texture. Regions whose values ​​reflect changes in the life cycle are more likely to be mergeable objects. The resulting fractal dimension matrix... This serves as a criterion for determining mergingable objects. If the system detects that a mergingable object already exists at that location in the model, it indicates that this is a review of an existing crack. Instead of creating a new object, the system updates the attribute values ​​of the existing object and adds a detection record to its history. In this way, crack information in the BIM model can be automatically and synchronously updated with each inspection, forming a crack archive at the model level, and the entire process requires no manual intervention.

[0044] It should be noted that the crack objects created above are centrally displayed and managed in the BIM visualization platform interface. The specific location of each crack in the 3D model is marked with a special highlighted symbol, and its detailed attribute data, such as number, length, width, and occurrence time, can be viewed interactively. New or updated crack information is simultaneously written to the backend operation and maintenance management database (which can be a relational database or a time-series database) for future querying, statistics, and analysis. This invention constructs a crack management submodule to perform closed-loop tracking management of the crack lifecycle: during subsequent periodic inspections, if new cracks or changes to existing cracks are detected, the system automatically compares the new data with the historical data of that crack in the database to identify the crack's expansion status. For example, it calculates the change in length or width of a crack since the last inspection, obtains its expansion rate, and updates the status attribute of the crack object in the BIM model (e.g., marking the status as "expanding" and recording the expansion value). When the size or expansion rate of a crack exceeds a preset threshold, the system can automatically trigger an alarm and generate maintenance suggestions for operation and maintenance personnel. Through the above process, digital management of concrete structure cracks from discovery, recording, monitoring to early warning is realized, providing a scientific basis for maintenance decisions.

[0045] The above is an introduction to the method embodiments. The following describes the solution described in this application through device embodiments.

[0046] Figure 3 The block diagram of a concrete crack management device combining drones and BIM is shown according to an exemplary embodiment. This device is used for a concrete crack management method combining drones and BIM. (Refer to...) Figure 3 The device for concrete crack management, which combines drones and BIM, includes an inspection module, a semantic segmentation module, an information extraction module, a 3D positioning module, a coordinate alignment module, a BIM integration module, and a lifecycle module.

[0047] Inspection module: Used to inspect the surface of concrete structures using drones, and obtain image data and control point data; Semantic segmentation module: used to perform lightweight crack semantic segmentation on the image data by combining fractal feature analysis on the UAV end, and obtain segmentation results containing crack regions; Information extraction module: Based on the segmentation results containing the crack region, extract the crack mask and attribute information of each crack to obtain a crack semantic information set. The attribute information includes the crack size, direction, component to which it belongs, severity, current status, image acquisition time, and the attitude and positioning coordinate data of the UAV itself. The 3D positioning module is used to perform 3D positioning calculations on the cracks based on the crack mask and control point data, to obtain the spatial location of the cracks and the location of the control points. The 3D positioning calculation includes multi-view image registration and depth sensing methods. The multi-view image registration obtains crack point clouds through feature matching and spatial reconstruction. The depth sensing method uses lidar point cloud data to project crack pixels in the image onto real-world 3D points and calculates the depth and location of the cracks. The spatial location of the cracks is expressed through a spatial coordinate system, which includes the UAV local coordinate system and world coordinates. Coordinate Alignment Module: Used to perform coordinate system transformation on the spatial location of the crack, transforming the spatial coordinate system to the BIM model coordinate system of the corresponding structure, and obtaining crack coordinates that are precisely aligned with the BIM 3D model. The coordinate system transformation is achieved by a coordinate transformation matrix. The coordinate transformation matrix is ​​calibrated using at least two known control points and then the coordinate transformation matrix parameters are obtained through parameter calculation. The coordinate transformation matrix parameters are homogeneous transformation matrices. The parameter calculation includes SVD, least squares method and iterative optimization algorithm. The homogeneous transformation matrix is ​​formed by combining the rotation matrix R and the translation vector t. BIM Integration Module: Based on the crack coordinates that are precisely aligned with the BIM 3D model, it creates component entities of crack objects in the BIM model, associates the crack semantic information set with the corresponding component entities in the BIM model, and obtains a crack instance dataset in the BIM model. The crack objects integrate crack geometry and semantic information in the model in the form of primitives. Lifecycle Module: This module continuously collects crack semantic information and, based on a multi-source data merging strategy for the same crack, synchronizes newly added or updated crack information to obtain crack instances in the updated BIM model. These crack instances in the updated BIM model are used for visual tracking of cracks and closed-loop management of the crack lifecycle. The multi-source data merging strategy for the same crack includes reasonable crack topology assessment and lifecycle identification using predetermined symbols or colors. The reasonable crack topology assessment includes extracting a merging threshold based on box-count fractal dimension from historical data.

[0048] A concrete crack management device combining drones and BIM, the device comprising: a processor; and a memory storing computer-readable instructions, which, when executed by the processor, implement the method described in any of the above-described concrete crack management methods combining drones and BIM.

[0049] A computer-readable storage medium, characterized in that the computer-readable storage medium stores program code, the program code being invoked by a processor to execute the method as described in any one of claims 1 to 7.

[0050] Figure 4 This is a structural schematic diagram of a concrete crack management device combining drones and BIM, provided by an embodiment of the present invention. Figure 4 As shown, concrete crack management equipment combining drones and BIM can include the above-mentioned Figure 3 The illustrated concrete crack management device combines drones and BIM. Optionally, the concrete crack management device 410 combining drones and BIM may include a first processor 2001.

[0051] Optionally, the concrete crack management device 410 combining drones and BIM may also include a memory 2002 and a transceiver 2003.

[0052] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0053] The following is combined Figure 4 A detailed introduction to each component of the concrete crack management equipment 410 that combines drones and BIM: The first processor 2001 is the control center of the concrete crack management device 410 that combines UAVs and BIM. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0054] Optionally, the first processor 2001 can perform various functions of the concrete crack management device 410 that combines UAV and BIM by running or executing software programs stored in memory 2002 and calling data stored in memory 2002.

[0055] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 4 CPU0 and CPU1 are shown in the diagram.

[0056] In a specific implementation, as one example, the concrete crack management device 410 combining drones and BIM can also include multiple processors, for example... Figure 4 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0057] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0058] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be connected via an interface circuit between the UAV and the BIM concrete crack management device 410. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0059] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0060] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 4 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0061] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and can be connected via an interface circuit between the UAV and the BIM concrete crack management device 410. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0062] It should be noted that, Figure 4 The structure of the concrete crack management device 410 combining drones and BIM shown in the diagram does not constitute a limitation on the router. Actual knowledge structure identification devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0063] Furthermore, the technical effects of the concrete crack management equipment 410 combining drones and BIM can be referred to the technical effects of the concrete crack management method combining drones and BIM described in the above method embodiments, and will not be repeated here.

[0064] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0065] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0066] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0067] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0068] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0069] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0070] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0071] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0072] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0073] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0074] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0075] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0076] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for managing concrete cracks combining drones and BIM, characterized in that, The method includes: S1: Use drones to inspect the surface of concrete structures and obtain image data and control point data; S2: At the UAV end, perform lightweight crack semantic segmentation on the image data by combining fractal feature analysis to obtain a segmentation result containing the crack region; S3: Based on the segmentation results of the cracked region, extract the crack mask and attribute information of each crack to obtain a crack semantic information set. The attribute information includes the crack size, direction, component to which it belongs, severity, current state, image acquisition time, and the attitude and positioning coordinate data of the UAV itself. S4: Based on the crack mask and control point data, perform three-dimensional positioning calculations on the crack to obtain the spatial location of the crack and the location of the control points. The three-dimensional positioning calculation includes multi-view image registration and depth sensing methods. The multi-view image registration obtains the crack point cloud through feature matching and spatial reconstruction. The depth sensing method uses lidar point cloud data to project the crack pixels in the image onto real-world three-dimensional points and calculates the depth and location of the crack. The spatial location of the crack is expressed through a spatial coordinate system, which includes the UAV local coordinate system and world coordinates. S5: Perform coordinate system transformation on the spatial location of the crack, transforming the spatial coordinate system to the BIM model coordinate system of the corresponding structure, to obtain the crack coordinates that are precisely aligned with the BIM 3D model. The coordinate system transformation is achieved by a coordinate transformation matrix. The coordinate transformation matrix is ​​calibrated using at least two known control points and then the coordinate transformation matrix parameters are obtained through parameter calculation. The coordinate transformation matrix parameters are homogeneous transformation matrices. The parameter calculation includes SVD, least squares method, and iterative optimization algorithm. The homogeneous transformation matrix is ​​formed by combining the rotation matrix R and the translation vector t. S6: Based on the crack coordinates that are precisely aligned with the BIM 3D model, create a component entity of the crack object in the BIM model, associate the crack semantic information set with the corresponding component entity in the BIM model, and obtain a crack instance dataset in the BIM model. The crack object integrates the crack geometry and semantic information in the form of primitives in the model. S7: Continuously collect crack semantic information and, based on a multi-source data merging strategy for the same crack, synchronize newly added or updated crack information to obtain crack instances in the updated BIM model. These updated BIM model crack instances are used for visual tracking of cracks and closed-loop management of the crack lifecycle. The multi-source data merging strategy for the same crack includes reasonable crack topology assessment and lifecycle identification using predetermined symbols or colors. The reasonable crack topology assessment includes extracting a merging threshold based on box-count fractal dimension from historical data, specifically including: S71: Continuously collect crack semantic information from the inspection and model maintenance process, and add the newly acquired crack records to the crack instance dataset in the BIM model; S72: Obtain the box count fractal dimension of the newly acquired crack record, and combine it with the center point location, length and orientation information to determine whether there are mergeable objects in the existing crack instance dataset of the newly added or updated crack. The mergeable objects refer to crack instances with similar attributes in the existing crack instance dataset of the BIM model. The box count fractal dimension, length and orientation information are used to calculate the life cycle identifier. S73: If a mergeable object exists, update the parameters of the mergeable object and append a lifecycle identifier to the history property. If no mergeable object exists, create a new crack instance to ensure that the same crack corresponds to only one object.

2. The concrete crack management method combining UAVs and BIM according to claim 1, characterized in that, The S1 method utilizes a drone to inspect the surface of the concrete structure, obtaining image data and control point data, including: S11: Using drones, conduct on-site inspections of the concrete structure surface. The on-site inspection includes a method of collecting images of cracks and measuring the actual crack locations by having the drone maintain a constant distance of 2-3 meters and fly parallel across the web surface. The collection method takes the surface of the component under test, with the bridge web as an approximate plane, as the object. S12: After reaching a position near the surface of the component to be tested, dynamically adjust the flight attitude of the UAV to keep the camera optical axis perpendicular to the surface of the component to be tested; S13: Achieve close-range shooting using a gimbal camera, acquire high-resolution images of the surface of the component under test, and obtain image data and control point data; S14: Collect control point data through a gimbal camera. The control point data is obtained by selecting at least two easily identifiable reference points on the surface of the concrete structure and measuring their coordinates in the UAV coordinate system.

3. The concrete crack management method combining UAVs and BIM according to claim 1, characterized in that, Based on the segmentation results of the crack-containing region, step S3 extracts the crack mask and attribute information for each crack to obtain a crack semantic information set. The attribute information includes the crack's size, orientation, associated component, severity, current state, image acquisition time, and the UAV's own attitude and positioning coordinates, including: S31: On the drone side, a preprocessing step is first performed on the image data to obtain preprocessed image data. The preprocessing step includes median filtering noise reduction to reduce noise interference. S32: Input the pre-processed image data into the pre-trained semantic segmentation network, and run it in real time at the video frame rate in the embedded environment to obtain the segmentation result. The pre-trained semantic segmentation network is a crack segmentation model based on full convolution. The crack segmentation model is built based on the improved VGG16 or DeepLab architecture. S33: Extract the attribute information of each crack based on the segmentation results. The attribute information includes the crack shape, acquisition timestamp, UAV pose coordinates, latitude and longitude coordinates of the shooting location, altitude and attitude angle. S34: Perform fractal feature analysis based on the image data to obtain fractal features. The fractal feature analysis includes the generation of prompts based on the fractal dimension of local box counting in the image. The fractal dimension of local box counting in the image includes covering the image with a grid at different scales, counting the number of grids containing crack pixels at each scale, and estimating the local fractal dimension accordingly. S35: The attribute information and the fractal features are transmitted back to the ground station via a wireless network.

4. The concrete crack management method combining UAVs and BIM according to claim 1, characterized in that, S4 performs 3D localization calculations on the crack based on the crack mask and control point data to obtain the spatial location of the crack and the location of the control points. The 3D localization calculation includes multi-view image registration and depth sensing methods. The multi-view image registration obtains the crack point cloud through feature matching and spatial reconstruction. The depth sensing method uses LiDAR point cloud data to project crack pixels in the image onto real-world 3D points and calculates the depth and location of the crack. The spatial location of the crack is expressed using a spatial coordinate system, which includes the UAV local coordinate system and world coordinates, including: S41: Extract location information, IMU attitude angle and UAV position coordinates from the crack semantic information set to obtain the spatial geometric position of the crack on the structural surface. The spatial geometric position is expressed by a pixel coordinate system. S42: Multi-view image registration is used to perform feature matching and spatial reconstruction to obtain crack point cloud. The multi-view image registration includes feature matching and spatial reconstruction of multi-view images. The crack point cloud includes a set of points representing the spatial distribution of cracks. S43: By using a depth sensing method, laser radar point cloud data is collected to obtain the relationship between the point cloud data and the spatial geometric position of the crack on the structural surface, and to obtain the elevation of the plane where the component is located and the position of the crack center in the image. The depth sensing method uses laser radar point cloud data for projection. S44: Project the UAV's position coordinates onto the plane where the component is located, and obtain the coordinates of the image center on the surface of the structure based on the elevation of the plane where the component is located; S45: Based on the pixel offset of the position of the crack center in the image relative to the image center, the actual horizontal distance corresponding to the pixel offset is obtained after conversion of camera intrinsic parameters and height. The pixel offset is with reference to the image center, and the camera intrinsic parameters include parameters for imaging scale conversion. S46: Using the actual horizontal distance, perform superposition calculation with the spatial geometric position of the crack on the structural surface to obtain the spatial position of the crack. The superposition calculation includes adding the actual horizontal distance to the spatial geometric position. The spatial position of the crack is expressed in a spatial coordinate system. The spatial coordinate system expression includes a dual coordinate result for positioning and expression. The spatial coordinate system includes the UAV local coordinate system and world coordinates. The dual coordinate result uses the structural surface as a reference to ensure positional consistency.

5. The concrete crack management method combining UAVs and BIM according to claim 1, characterized in that, S5 performs a coordinate system transformation on the spatial location of the crack, converting the spatial coordinate system to the BIM model coordinate system of the corresponding structure, to obtain crack coordinates precisely aligned with the BIM 3D model. This coordinate system transformation is achieved using a coordinate transformation matrix. The coordinate transformation matrix is ​​calibrated using at least two known control points, and then the coordinate transformation matrix parameters are calculated. The coordinate transformation matrix parameters are homogeneous transformation matrices. The parameter calculation includes SVD, least squares method, and iterative optimization algorithm. The homogeneous transformation matrix is ​​formed by combining a rotation matrix R and a translation vector t, including: S51: Extract the coordinate values ​​of control points in the UAV coordinate system and BIM model coordinate system based on control point data; S52: Based on the coordinate values ​​of the control points in the UAV coordinate system and the BIM model coordinate system, a coordinate transformation matrix is ​​obtained through rigid body transformation and parameter calculation. The coordinate transformation matrix adopts a homogeneous transformation form and includes transformation parameters. The transformation parameters include a rotation matrix R, a translation vector t, and an optional uniform scale factor. The parameter calculation includes singular value decomposition (SVD), least squares method, and iterative optimization algorithm. S53: Based on the coordinate transformation matrix, apply the two-level transformation to all spatial points or point cloud coordinates of the crack to obtain crack coordinates that are precisely aligned with the BIM 3D model. The two-level transformation includes calling the Gaussian projection algorithm to convert the geographic coordinates obtained by the UAV into a Cartesian coordinate system, and then based on the coordinate transformation matrix, converting the Cartesian coordinate system into the BIM model coordinate system.

6. The concrete crack management method combining UAVs and BIM according to claim 1, characterized in that, S6, based on the crack coordinates precisely aligned with the BIM 3D model, creates component entities of crack objects in the BIM model, associates the crack semantic information set with the corresponding component entities in the BIM model, and obtains a crack instance dataset in the BIM model. The crack objects in the model integrate crack geometry and semantic information in the form of primitives, including: S61: Based on the crack coordinates that are precisely aligned with the BIM 3D model, automatically create crack objects to obtain crack instances in the BIM model. The automatic creation of crack objects includes inserting a predefined crack primitive on the surface of the corresponding component at the determined crack spatial location as a geometric representation with dimensions and location consistent with the actual crack. The crack primitive takes the form of a slender polyline segment, curve, or two-dimensional symbol, and is used to store it in the model in the form of parametric primitives and establish data association with the surface of the structural component where it is located. S62: Input the crack semantic information set into the parameter field of the crack object. The parameter field includes crack number, discovery time, spatial location, geometric dimensions, severity and current status. The geometric dimensions include length, average width, maximum width and depth. S63: Using predetermined symbols or colors, establish visual identifiers for crack objects to obtain differentiated displays of different cracks and their states. The predetermined symbols or colors include styles set according to lifecycle identifiers. The lifecycle identifiers include newly discovered, continuously expanding, stable, repaired, and / or severity levels.

7. A concrete crack management device combining drones and BIM, wherein the device is used to implement the concrete crack management method combining drones and BIM as described in any one of claims 1-6, characterized in that, The device includes: Inspection module: Used to inspect the surface of concrete structures using drones, and obtain image data and control point data; Semantic segmentation module: used to perform lightweight crack semantic segmentation on the image data by combining fractal feature analysis on the UAV end, and obtain segmentation results containing crack regions; Information extraction module: Based on the segmentation results containing the crack region, extract the crack mask and attribute information of each crack to obtain a crack semantic information set. The attribute information includes the crack size, direction, component to which it belongs, severity, current status, image acquisition time, and the attitude and positioning coordinate data of the UAV itself. The 3D positioning module is used to perform 3D positioning calculations on the cracks based on the crack mask and control point data, to obtain the spatial location of the cracks and the location of the control points. The 3D positioning calculation includes multi-view image registration and depth sensing methods. The multi-view image registration obtains crack point clouds through feature matching and spatial reconstruction. The depth sensing method uses lidar point cloud data to project crack pixels in the image onto real-world 3D points and calculates the depth and location of the cracks. The spatial location of the cracks is expressed through a spatial coordinate system, which includes the UAV local coordinate system and world coordinates. Coordinate Alignment Module: Used to perform coordinate system transformation on the spatial location of the crack, transforming the spatial coordinate system to the BIM model coordinate system of the corresponding structure, and obtaining crack coordinates that are precisely aligned with the BIM 3D model. The coordinate system transformation is achieved by a coordinate transformation matrix. The coordinate transformation matrix is ​​calibrated using at least two known control points and then the coordinate transformation matrix parameters are obtained through parameter calculation. The coordinate transformation matrix parameters are homogeneous transformation matrices. The parameter calculation includes SVD, least squares method and iterative optimization algorithm. The homogeneous transformation matrix is ​​formed by combining the rotation matrix R and the translation vector t. BIM Integration Module: Based on the crack coordinates that are precisely aligned with the BIM 3D model, it creates component entities of crack objects in the BIM model, associates the crack semantic information set with the corresponding component entities in the BIM model, and obtains a crack instance dataset in the BIM model. The crack objects integrate crack geometry and semantic information in the model in the form of primitives. Lifecycle Module: This module continuously collects crack semantic information and, based on a multi-source data merging strategy for the same crack, synchronizes newly added or updated crack information to obtain crack instances in the updated BIM model. These crack instances in the updated BIM model are used for visual tracking of cracks and closed-loop management of the crack lifecycle. The multi-source data merging strategy for the same crack includes reasonable crack topology assessment and lifecycle identification using predetermined symbols or colors. The reasonable crack topology assessment includes extracting a merging threshold based on box-count fractal dimension from historical data.

8. A concrete crack management device combining drones and BIM, characterized in that, The concrete crack management processor combining UAV and BIM; a memory storing computer-readable instructions, which, when executed by the processor, implement the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 6.

Images