A complex spatial unmanned aerial vehicle multi-modal fusion crack detection and three-dimensional modeling method and system

Abstract

This invention discloses a multimodal fusion crack detection and 3D modeling method and system for complex space unmanned aerial vehicles (UAVs), belonging to the field of intelligent inspection and structural health monitoring technology. The key technical points are: simultaneously acquiring RGB images, depth data, LiDAR point clouds, and GPS / IMU data using a multimodal sensor array mounted on the UAV; identifying and segmenting multiple types of damage, including cracks, based on an improved YOLOv11s-DySnake model; fusing multi-source data to calculate the 3D coordinates of the damage; constructing a structural model using an incremental 3D modeling algorithm and visually annotating the damage; and combining a hybrid A* algorithm with a dynamic obstacle avoidance algorithm to achieve autonomous inspection path planning. This invention achieves full automation from data acquisition to 3D modeling, improving the accuracy, robustness, and positioning accuracy of damage detection.

Description

Technical Field

[0001] This invention belongs to the field of intelligent inspection and structural health monitoring technology, and in particular relates to a method and system for multimodal fusion crack detection and three-dimensional modeling of complex space UAVs. Background Technology

[0002] With the rapid development of infrastructure construction, various engineering structures are prone to damage such as cracks, corrosion, spalling, and exposed steel bars during long-term service due to factors such as load, environmental erosion, and material aging. Timely and accurate detection is the key to ensuring structural safety and preventing accidents.

[0003] Currently, this field mainly relies on manual inspection and traditional single-modal sensor detection technology. Manual inspection is inefficient and risky, making it difficult to achieve full coverage inspection of facilities at high altitudes, with large spans, or with complex spatial structures. Furthermore, the inspection results are easily influenced by the subjective experience of the inspectors, resulting in poor consistency. Traditional automatic detection technologies based on single vision sensors, such as relying solely on RGB cameras for image analysis, lack robustness in complex lighting conditions, background interference, and the presence of minute cracks common in engineering sites. This leads to frequent missed and false detections, failing to meet the demands for high-precision damage identification. In addition, existing technologies mostly focus on two-dimensional plane damage identification, lacking effective means to accurately correlate identified damage with real three-dimensional spatial coordinates. This results in inspection results that do not intuitively reflect the specific spatial location of damage on the structure, leading to vague positioning and difficulty in providing accurate spatial information support for subsequent structural safety assessments, maintenance decisions, and historical data comparison. A more prominent problem is that existing data acquisition, intelligent diagnosis, and 3D modeling processes are often independent, forming data silos and lacking an integrated closed-loop solution from efficient data acquisition and accurate intelligent diagnosis to intuitive 3D visualization. This fragmented process not only reduces overall inspection efficiency but also results in low data utilization, failing to fully leverage the value of multidimensional data in the structural health lifecycle management.

[0004] The root cause of these problems lies in the failure to effectively integrate and synergistically utilize multi-source heterogeneous sensor data, the lack of an algorithmic framework capable of simultaneously processing and deeply fusing image, point cloud, and pose information, and the absence of an automated system integrating high-performance damage identification models and high-precision spatial positioning technology. Therefore, the industry urgently needs an integrated intelligent detection and modeling system capable of achieving full-process automation, possessing high robustness and high-precision positioning capabilities, to overcome the limitations of existing technologies and meet the pressing need for intelligent operation and maintenance of complex engineering structures. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes a method and system for multimodal fusion crack detection and 3D modeling of complex space unmanned aerial vehicles (UAVs), thereby resolving the issues present in the prior art.

[0006] Firstly, to achieve the above objectives, the present invention provides a method for multimodal fusion crack detection and 3D modeling of complex space unmanned aerial vehicles, comprising the following steps: The drone simultaneously collects image data, depth data, lidar point cloud data, and positioning attitude data of the target structure using a multimodal sensor array. The multimodal data is preprocessed to construct a standardized dataset; An improved deep learning model is used to identify and segment multiple types of damage in preprocessed image data at the pixel level, and output damage type and location information. The damage identification results, depth data, lidar point cloud and positioning attitude data are integrated to calculate the three-dimensional spatial coordinates of the damage in the world coordinate system; A three-dimensional model of the target structure is constructed based on the fused point cloud data, and the three-dimensional spatial coordinates and type information of the damage are bound to the three-dimensional model and visualized and annotated. The drone is planned to perform autonomous inspection path and dynamic obstacle avoidance based on real-time perception information.

[0007] Optionally, the process of synchronously collecting data using a multimodal sensor array mounted on a drone includes: The target structure's RGB image, depth image, point cloud data, and positioning attitude data are acquired by an RGB camera, depth camera, LiDAR, and GPS / IMU module, respectively, and a time synchronization algorithm is used to achieve millisecond-level alignment of the multi-source data.

[0008] Optionally, the process of using an improved deep learning model includes: Damage identification is performed using YOLO series models that introduce a cross-scale attention weighting mechanism in the feature fusion layer, and a dynamic serpentine convolutional layer is embedded in its detection head to enhance the extraction of slender crack features.

[0009] Optionally, the training process of the improved deep learning model includes: optimizing bounding box regression using the WIoU loss function, and introducing an L1 regularization term that decays with each training round for the parameters of the batch normalization layer.

[0010] Optionally, the process of fusing the data to calculate three-dimensional spatial coordinates includes: The depth data is registered with the LiDAR point cloud based on the iterative nearest point algorithm. The coordinates of the damaged pixels are then transformed by the camera model and weighted fusion with the registered point cloud and GPS positioning data to calculate the world coordinates of the damage.

[0011] Optionally, the process of constructing the 3D model and binding annotations includes: Voxel filtering downsampling and region growing feature extraction are performed on point cloud data to reconstruct the structural surface. The three-dimensional coordinates and type information of the damage are mapped to the surface of the three-dimensional model and coded and labeled using different colors.

[0012] Optionally, the process of planning the autonomous inspection path and performing dynamic obstacle avoidance includes: The global coverage path is planned based on the hybrid A* algorithm, and the local path is adjusted based on the real-time detection results of the lidar using the artificial potential field method to achieve obstacle avoidance.

[0013] Optionally, the method further includes: extracting the geometric parameters of the damage based on the damage identification result, classifying the damage into levels according to a preset threshold, and generating a structured report containing damage level information.

[0014] Secondly, the present invention also provides a multimodal fusion crack detection and 3D modeling system for complex space unmanned aerial vehicles (UAVs), used to implement a method for multimodal fusion crack detection and 3D modeling of complex space UAVs, the system comprising: The multimodal data acquisition module is used to simultaneously acquire image data, depth data, lidar point cloud, and positioning attitude data of the target structure through the sensor array mounted on the UAV. The data preprocessing module is used to preprocess the multimodal data to construct a standardized dataset; The damage intelligent recognition module is used to identify and segment multiple types of damage in preprocessed image data using an improved deep learning model, and output damage type and location information. The multi-source fusion positioning module is used to fuse the damage identification results, depth data, lidar point cloud and positioning attitude data to calculate the three-dimensional spatial coordinates of the damage in the world coordinate system. The 3D modeling and annotation module is used to construct a 3D model of the target structure based on the fused point cloud data, and to bind and visualize the 3D spatial coordinates and type information of the damage to the 3D model. The autonomous inspection control module is used to plan the autonomous inspection path of the UAV and control the UAV to perform dynamic obstacle avoidance based on real-time perception information.

[0015] Compared with the prior art, the present invention has the following advantages and technical effects: This invention provides a multimodal fusion crack detection and 3D modeling method and system for complex space unmanned aerial vehicles (UAVs). By fusing multimodal sensor data, it effectively overcomes the limitations of single sensors in complex scenarios, significantly improving the detection stability and identification accuracy of various types of damage, including micro-cracks, under different lighting and background conditions. An improved deep learning model combined with multi-source fusion localization technology achieves accurate mapping and association of damage from 2D images to 3D world coordinates, solving the problem of ambiguous spatial damage localization. Incremental 3D modeling and real-time damage annotation technology construct a closed loop linking detection and visualization, allowing damage information to be intuitively displayed in the 3D model. The fully automated design, especially the autonomous inspection based on the hybrid A* algorithm and dynamic obstacle avoidance, significantly improves the efficiency and safety of inspection operations on complex spatial structures, reducing reliance on manual labor and maintenance costs. The system exhibits good generalization ability, adapting to various engineering facilities with different materials and structural types, providing an integrated solution for structural health monitoring from intelligent data acquisition and accurate diagnosis to decision support. Attached Figure Description

[0016] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a flowchart illustrating a method for multimodal fusion crack detection and 3D modeling of complex space unmanned aerial vehicles according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the improved YOLOv11s-DySnake model structure according to an embodiment of the present invention; Figure 3 This is a flowchart of the multi-source data fusion and 3D localization process according to an embodiment of the present invention; Figure 4 This is a flowchart illustrating the autonomous path planning and dynamic obstacle avoidance process according to an embodiment of the present invention. Figure 5 This is a flowchart illustrating the 3D modeling and damage annotation process according to an embodiment of the present invention. Figure 6 This is a diagram showing the crack detection effect of a model according to an embodiment of the present invention; Figure 7 This is the model damage detection confusion matrix in an embodiment of the present invention. Detailed Implementation

[0017] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0018] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0019] Example 1 like Figure 1 As shown, this embodiment provides a method for multimodal fusion crack detection and 3D modeling of complex space UAVs, including: The drone simultaneously collects image data, depth data, lidar point cloud data, and positioning attitude data of the target structure using a multimodal sensor array. The multimodal data is preprocessed to construct a standardized dataset; An improved deep learning model is used to identify and segment multiple types of damage in preprocessed image data at the pixel level, and output damage type and location information. The damage identification results, depth data, lidar point cloud and positioning attitude data are integrated to calculate the three-dimensional spatial coordinates of the damage in the world coordinate system; A three-dimensional model of the target structure is constructed based on the fused point cloud data, and the three-dimensional spatial coordinates and type information of the damage are bound to the three-dimensional model and visualized and annotated. The drone is planned to perform autonomous inspection path and dynamic obstacle avoidance based on real-time perception information.

[0020] Furthermore, the process of synchronously collecting data using a multimodal sensor array mounted on a drone includes: The target structure's RGB image, depth image, point cloud data, and positioning attitude data are acquired by an RGB camera, depth camera, LiDAR, and GPS / IMU module, respectively, and a time synchronization algorithm is used to achieve millisecond-level alignment of the multi-source data.

[0021] Specifically, the implementation process of this embodiment includes: The multimodal sensor array consists of an RGB camera, a D435I depth camera, a 2D LiDAR, and a GPS / IMU module. The parameters of each sensor are strictly matched to the engineering testing requirements: the RGB camera has a resolution of 1920×1080 and a sampling rate of 30fps; the depth camera has a measurement range of 0.1-10m and a depth accuracy of ±2% (@2m); the LiDAR has a detection range of 0-20m, a point cloud density of 1000 points / second, and an angular resolution of 0.1°; the GPS positioning accuracy is 0.1m; the IMU attitude measurement accuracy is ±0.1°; and the data output frequency is 100Hz.

[0022] Furthermore, the preprocessing of the multimodal data includes: RGB image preprocessing employs a combined denoising and enhancement strategy. First, 3×3 Gaussian filtering (σ=0.5) and median filtering remove salt-and-pepper noise and Gaussian noise. Then, histogram equalization enhances image contrast and improves the identification of damage features. For lens distortion, a distortion correction formula is used for geometric correction. ; ; in( , ) represents the corrected pixel coordinates, , ) represents the original pixel coordinates, ( , Let k1 and k2 be the coordinates of the principal point of the lens, and k1 and k2 be the distortion coefficients. .

[0023] The lidar point cloud preprocessing adopts a combination of direct filtering and statistical filtering. Direct filtering removes redundant point clouds from the ground and distant areas according to a preset height range. Statistical filtering sets the number of neighboring points k=10 and the standard deviation threshold α=1.5. When the average distance between the point cloud and its neighboring points exceeds μ+ασ (μ is the average distance and σ is the standard deviation), it is judged as an outlier and removed.

[0024] GPS / IMU data preprocessing is based on Hampel filter to remove outliers. The window size is set to 5 and the standard deviation threshold is 3. When the deviation of a data point from the median in the window exceeds 3σ, it is judged as an outlier and replaced with valid data from the previous moment. Zero bias calibration and integration processing are performed on the IMU data to obtain the attitude angles (roll, pitch, yaw) and velocity information of the UAV.

[0025] Furthermore, the process of using improved deep learning models includes: Damage identification is performed using YOLO series models that introduce a cross-scale attention weighting mechanism in the feature fusion layer, and a dynamic serpentine convolutional layer is embedded in its detection head to enhance the extraction of slender crack features.

[0026] Furthermore, the training process of the improved deep learning model includes: optimizing bounding box regression using the WIoU loss function, and introducing an L1 regularization term that decays with each training round for the parameters of the batch normalization layer.

[0027] Specifically, the implementation process of this embodiment includes: like Figure 2 As shown, the improved YOLOv11s-DySnake model adds a cross-scale feature fusion channel in the neck layer, and uses an attention mechanism to perform weighted fusion of features at different scales. The formula for calculating the attention weight is as follows: ; in The attention weights are for scale s. The importance score of the i-th scale feature is obtained and the feature representation of the microcrack is enhanced by outputting the fully connected layer.

[0028] A DySnakeConv serpentine convolutional layer is embedded in the head layer of the model. A dynamic serpentine scanning path is used to expand the receptive field along the crack length direction, which reduces the amount of computation while improving the feature capture ability of slender cracks. The path parameters of the serpentine convolution are adaptively learned through network training.

[0029] The WIoU loss function is used to optimize bounding box regression. The loss function is defined as follows: ; in is the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box, c is the length of the diagonal of the minimum rectangle enclosing the two boxes, and IoU is the intersection-over-union ratio, which improves the accuracy of bounding box regression.

[0030] Introducing targeted dynamic L1 regularization, the expression is: ; Where λ(t) is the weight coefficient that decays with each training round. For the BatchNorm2d layer parameter set, These are the weight parameters for the BatchNorm2d layer, which suppress overfitting while preserving the core feature extraction capability.

[0031] The pixel-level segmentation module adopts a dual-branch structure, and the segmentation loss is a weighted combination of Dice Loss and cross-entropy loss: ; Where α is the balance coefficient (ranging from 0.3 to 0.7). The cross-entropy loss is used, and Dice is the Dice similarity coefficient, calculated as follows: ; For smoothing terms, To predict the pixel values ​​of the segmentation mask, To accurately segment the pixel values ​​of the mask and balance the imbalance between positive and negative samples.

[0032] Furthermore, the process of fusing the data to calculate three-dimensional spatial coordinates includes: The depth data is registered with the LiDAR point cloud based on the iterative nearest point algorithm. The coordinates of the damaged pixels are then transformed by the camera model and weighted fusion with the registered point cloud and GPS positioning data to calculate the world coordinates of the damage.

[0033] Specifically, the implementation process of this embodiment includes: like Figure 3 As shown, the RANSAC algorithm is used to remove out-point noise in the lidar point cloud, with 1000 iterations and an in-point threshold of 0.05m, thereby improving the purity of the point cloud data.

[0034] Registration of depth images and LiDAR point clouds is achieved based on the ICP algorithm. The registration error function is: ; Where R is the rotation matrix (3×3) and t is the translation vector (3×1). Let i be the i-th point in the lidar point cloud. Let R be the i-th point after the depth image transformation, and N be the total number of point clouds participating in the registration. The Levenberg-Marquardt algorithm is used to iteratively solve for the optimal R and t. The iteration terminates when the registration error E(R,t) < 1e-4 or the number of iterations reaches 100.

[0035] The pixel coordinates (u, v) of the damage detection result are converted into camera coordinate system coordinates using the camera intrinsic parameter matrix K. The conversion formula is as follows: ; in K represents the depth value of the damaged area acquired by the depth camera, and K is the camera intrinsic parameter matrix (obtained from camera calibration).

[0036] A weighted fusion strategy is used to optimize the localization results. The fusion formula is as follows: ; ; ; For weight fusion.

[0037] Furthermore, the process of constructing the 3D model and binding annotations includes: Voxel filtering downsampling and region growing feature extraction are performed on point cloud data to reconstruct the structural surface. The three-dimensional coordinates and type information of the damage are mapped to the surface of the three-dimensional model and coded and labeled using different colors.

[0038] Specifically, the implementation process of this embodiment includes: like Figure 5 As shown, voxel filtering is used to downsample the point cloud data, with the voxel size set to 0.01m×0.01m×0.01m. This reduces the amount of data while preserving key geometric features, thereby improving modeling efficiency.

[0039] Feature points on the structural surface are extracted using a region growing algorithm. Growth thresholds are set: angle threshold ≤ 10° and distance threshold ≤ 0.05m. The topological connection relationship of the structural surface is constructed to provide a basis for surface reconstruction.

[0040] The Moving Cubes algorithm is used to reconstruct the surface of dense point clouds, generating a 3D mesh model of the structure with a mesh resolution of ≤0.02m, ensuring clear model details.

[0041] The system precisely binds the 3D coordinates, type, size, confidence level, and other information of the damage to the 3D model. It uses color coding to distinguish different damage types (cracks: black, corrosion: dark gray, spalling: medium gray, exposed rebar: light gray). The system directly annotates the boundary box, center point coordinates, and key parameters of the damage in the model and supports model rotation, scaling, and translation operations.

[0042] Furthermore, the process of planning the autonomous inspection path and executing dynamic obstacle avoidance includes: The global coverage path is planned based on the hybrid A* algorithm, and the local path is adjusted based on the real-time detection results of the lidar using the artificial potential field method to achieve obstacle avoidance.

[0043] Specifically, the implementation process of this embodiment includes: like Figure 4 As shown, based on the CAD drawings or initial scan data of the target structure, an environmental map is constructed using the grid method, with the grid size set to 0.5m × 0.5m, clearly defining the inspection area and the distribution of obstacles.

[0044] The global inspection path is planned using a hybrid A* algorithm, with the objective function being: ; Where L is the path length weight term, C is the path curvature weight term (to ensure the stability of the drone's flight), and D is the obstacle distance weight term (to ensure obstacle avoidance safety).

[0045] Dynamic obstacle avoidance is achieved based on the artificial potential field method. The lidar detects obstacles in front in real time (detection frequency 10Hz). When the distance between the detected obstacle and the UAV is less than the safety threshold (2m), the repulsive force of the obstacle and the attractive force of the target point are calculated.

[0046] In the model detection effect verification stage, such as Figure 6 The image shown is a diagram illustrating the crack detection effect of a model provided in an embodiment of the present invention. Figure 7 The model damage detection confusion matrix provided in this embodiment of the invention intuitively demonstrates the damage recognition performance of the improved YOLOv11s-DySnake model.

[0047] Furthermore, the method also includes: extracting the geometric parameters of the damage based on the damage identification results, classifying the damage into levels according to a preset threshold, and generating a structured report containing damage level information.

[0048] Example 2 Based on the same general inventive concept, this invention also provides a multimodal fusion crack detection and 3D modeling system for complex space unmanned aerial vehicles (UAVs). The multimodal fusion crack detection and 3D modeling system for complex space UAVs provided by this invention is described below. The multimodal fusion crack detection and 3D modeling system for complex space UAVs described below can be referred to in correspondence with the multimodal fusion crack detection and 3D modeling method for complex space UAVs described above. The system includes: The multimodal data acquisition module is used to simultaneously acquire image data, depth data, lidar point cloud, and positioning attitude data of the target structure through the sensor array mounted on the UAV. The data preprocessing module is used to preprocess the multimodal data to construct a standardized dataset; The damage intelligent recognition module is used to identify and segment multiple types of damage in preprocessed image data using an improved deep learning model, and output damage type and location information. The multi-source fusion positioning module is used to fuse the damage identification results, depth data, lidar point cloud and positioning attitude data to calculate the three-dimensional spatial coordinates of the damage in the world coordinate system. The 3D modeling and annotation module is used to construct a 3D model of the target structure based on the fused point cloud data, and to bind and visualize the 3D spatial coordinates and type information of the damage to the 3D model. The autonomous inspection control module is used to plan the autonomous inspection path of the UAV and control the UAV to perform dynamic obstacle avoidance based on real-time perception information.

[0049] This invention provides a multimodal fusion crack detection and 3D modeling method and system for complex space unmanned aerial vehicles (UAVs). By fusing multimodal sensor data, it effectively overcomes the limitations of single sensors in complex scenarios, significantly improving the detection stability and identification accuracy of various types of damage, including micro-cracks, under different lighting and background conditions. An improved deep learning model combined with multi-source fusion localization technology achieves accurate mapping and association of damage from 2D images to 3D world coordinates, solving the problem of ambiguous spatial damage localization. Incremental 3D modeling and real-time damage annotation technology construct a closed loop linking detection and visualization, allowing damage information to be intuitively displayed in the 3D model. The fully automated design, especially the autonomous inspection based on the hybrid A* algorithm and dynamic obstacle avoidance, significantly improves the efficiency and safety of inspection operations on complex spatial structures, reducing reliance on manual labor and maintenance costs. The system exhibits good generalization ability, adapting to various engineering facilities with different materials and structural types, providing an integrated solution for structural health monitoring from intelligent data acquisition and accurate diagnosis to decision support.

[0050] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for multimodal fusion crack detection and 3D modeling of complex space unmanned aerial vehicles, characterized in that, Includes the following steps: The drone simultaneously collects image data, depth data, lidar point cloud data, and positioning attitude data of the target structure using a multimodal sensor array. The multimodal data is preprocessed to construct a standardized dataset; An improved deep learning model is used to identify and segment multiple types of damage in preprocessed image data at the pixel level, and output damage type and location information. The damage identification results, depth data, lidar point cloud and positioning attitude data are integrated to calculate the three-dimensional spatial coordinates of the damage in the world coordinate system; A three-dimensional model of the target structure is constructed based on the fused point cloud data, and the three-dimensional spatial coordinates and type information of the damage are bound to the three-dimensional model and visualized and annotated. The drone is planned to perform autonomous inspection path and dynamic obstacle avoidance based on real-time perception information.

2. The method according to claim 1, characterized in that, The process of synchronously collecting data using a multimodal sensor array carried by a drone includes: The target structure's RGB image, depth image, point cloud data, and positioning attitude data are acquired by an RGB camera, depth camera, LiDAR, and GPS / IMU module, respectively, and a time synchronization algorithm is used to achieve millisecond-level alignment of the multi-source data.

3. The method according to claim 1, characterized in that, The process of using improved deep learning models includes: Damage identification is performed using YOLO series models that introduce a cross-scale attention weighting mechanism in the feature fusion layer, and a dynamic serpentine convolutional layer is embedded in its detection head to enhance the extraction of slender crack features.

4. The method according to claim 3, characterized in that, The training process of the improved deep learning model includes: using the WIoU loss function to optimize bounding box regression, and introducing an L1 regularization term that decays with each training round for the parameters of the batch normalization layer.

5. The method according to claim 1, characterized in that, The process of fusing the data to calculate three-dimensional spatial coordinates includes: The depth data is registered with the LiDAR point cloud based on the iterative nearest point algorithm. The coordinates of the damaged pixels are then transformed by the camera model and weighted fusion with the registered point cloud and GPS positioning data to calculate the world coordinates of the damage.

6. The method according to claim 1, characterized in that, The process of constructing a 3D model and binding annotations includes: Voxel filtering downsampling and region growing feature extraction are performed on point cloud data to reconstruct the structural surface. The three-dimensional coordinates and type information of the damage are mapped to the surface of the three-dimensional model and coded and labeled using different colors.

7. The method according to claim 1, characterized in that, The process of planning the autonomous inspection path and executing dynamic obstacle avoidance includes: The global coverage path is planned based on the hybrid A* algorithm, and the local path is adjusted based on the real-time detection results of the lidar using the artificial potential field method to achieve obstacle avoidance.

8. The method according to claim 1, characterized in that, The method further includes: extracting the geometric parameters of the damage based on the damage identification results, classifying the damage into levels according to a preset threshold, and generating a structured report containing damage level information.

9. A multimodal fusion crack detection and 3D modeling system for complex space unmanned aerial vehicles, characterized in that, The system for implementing the method of any one of claims 1-8 comprises: The multimodal data acquisition module is used to simultaneously acquire image data, depth data, lidar point cloud, and positioning attitude data of the target structure through the sensor array mounted on the UAV. The data preprocessing module is used to preprocess the multimodal data to construct a standardized dataset; The damage intelligent recognition module is used to identify and segment multiple types of damage in preprocessed image data using an improved deep learning model, and output damage type and location information. The multi-source fusion positioning module is used to fuse the damage identification results, depth data, lidar point cloud and positioning attitude data to calculate the three-dimensional spatial coordinates of the damage in the world coordinate system. The 3D modeling and annotation module is used to construct a 3D model of the target structure based on the fused point cloud data, and to bind and visualize the 3D spatial coordinates and type information of the damage to the 3D model. The autonomous inspection control module is used to plan the autonomous inspection path of the UAV and control the UAV to perform dynamic obstacle avoidance based on real-time perception information.

Images