Bridge disease area 3D inspection intelligent positioning method and system based on unmanned aerial vehicle

Abstract

The invention relates to the technical field of bridge disease detection, in particular to a bridge disease area 3D inspection intelligent positioning method and system based on an unmanned aerial vehicle. Comprising a data acquisition unit; a 3D modeling unit; a disease identification and life cycle tracking unit; a precise positioning fusion unit; and a data storage and output unit. According to the method, the unique persistent identity identification based on the component topological relation, the detection timestamp and the contour check code is allocated for identifying the disease for the first time, and the scale invariance descriptor is generated by combining the multi-scale curvature features extracted by the curvature pyramid with the texture and depth feature weighting, so that cross-period disease accurate matching is realized, and repeated or missing identification is avoided. Meanwhile, the disease size and environment parameters are integrated, an evolution rule is analyzed through an LSTM network and an attention mechanism, trend representation containing the expansion rate and direction is generated, a full-life-cycle data chain is formed, the disease process is continuously tracked, and comprehensive dynamic data support is provided for predictive maintenance of a bridge.

Description

Technical Field

[0001] This invention relates to the field of bridge defect detection technology, and more specifically, to a method and system for intelligent positioning of 3D inspection of bridge defect areas based on unmanned aerial vehicles (UAVs). Background Technology

[0002] With the development of drone, sensor and artificial intelligence technologies, drones equipped with detection equipment for bridge inspection has become the industry mainstream. However, existing technologies still face problems such as insufficient accuracy in locating defects, difficulty in tracking across different periods, and difficulty in predicting evolution trends, which restricts the improvement of preventive maintenance levels.

[0003] In the existing technology, relevant patents have explored the field of bridge defect detection by UAVs. For example, Chinese patent CN202510639575.6 discloses a multi-sensor collaborative detection method for bridge defects by UAVs, which relates to the field of bridge monitoring technology. Its main steps include: generating the corrosion layer thickness distribution through a spectral matching algorithm; collecting three-dimensional point cloud data through lidar to generate deformation distribution data; calculating the local stiffness degradation coefficient matrix and inputting it into a finite element model to obtain predicted deformation data; comparing the measured and predicted deformation data to generate residual data; and then generating a dynamic detection priority map and executing a re-flight plan, thus solving the problem of the one-sidedness of single-sensor detection. Another example is Chinese patent CN202411755388.6, which discloses a bridge defect detection system and method based on UAVs, including a UAV system and a ground station. The UAV system includes a rotor power, navigation control, communication, and intelligent detection unit. The navigation control unit is responsible for environmental perception, positioning, and three-dimensional point cloud model generation. The intelligent detection unit captures bridge images and identifies defect information. The ground station supports flight path planning and defect result display, solving the problem of inspection when UAVs have difficulty obtaining effective GPS positioning information.

[0004] Despite the design advantages of the aforementioned technical solutions, they also suffer from the following technical shortcomings: Firstly, they lack the ability to accurately track diseases across different periods and manage their entire lifecycle. Chinese patent CN202510639575.6 can only generate a retest priority map through multi-sensor data comparison, and CN202411755388.6 can only identify the type and location of diseases in a single inspection. Neither has established a unique association mechanism for a single disease, making it impossible to accurately match the same disease across multiple inspection periods. Furthermore, neither integrates multi-period disease data with environmental parameters to deeply analyze the development and change patterns of diseases, resulting in difficulty in continuously tracking the disease from its emergence to its expansion. The entire process fails to provide dynamic data support for maintenance decisions; secondly, the accuracy of defect location is insufficient, failing to achieve precise mapping from two-dimensional to three-dimensional space: Chinese patent CN202510639575.6 focuses on comparing deformation distribution data with prediction models, while Chinese patent CN202411755388.6 can only mark the approximate location of defects. Neither of these methods considers the distortion effects during defect imaging or combines the inherent geometric features of bridge components with point cloud depth information for coordinate transformation. This results in defect location remaining at the two-dimensional image level or approximate area marking, making it difficult to form precise three-dimensional physical coordinates and failing to meet the high-precision requirements of maintenance construction for defect location. Therefore, we propose a method and system for intelligent 3D inspection and positioning of bridge defect areas based on unmanned aerial vehicles (UAVs). Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for intelligent positioning of bridge defect areas in 3D inspection based on unmanned aerial vehicles (UAVs), in order to solve the problems mentioned in the background art, such as the lack of accurate tracking and full life cycle management capabilities for defects across different periods and insufficient defect positioning accuracy, and the failure to achieve accurate mapping from two-dimensional to three-dimensional space.

[0006] To address the aforementioned technical problems, one objective of this invention is to provide an intelligent positioning system for 3D inspection of bridge defect areas based on unmanned aerial vehicles (UAVs), comprising:

[0007] The data acquisition unit uses a drone equipped with a lidar and a high-definition camera to collect bridge point cloud data and surface image data across different periods, and simultaneously acquires drone attitude data and environmental parameters, which are then transmitted to the 3D modeling unit and the disease identification and life cycle tracking unit.

[0008] The 3D modeling unit receives raw data from the data acquisition unit across different periods, fuses and processes it to generate a high-precision three-dimensional model of the bridge and camera pose parameters, and transmits them to the defect identification and life cycle tracking unit.

[0009] The disease identification and lifecycle tracking unit, upon initial disease identification, assigns a unique and persistent disease identifier based on the topological relationship of bridge components; constructs a curvature pyramid and extracts the contour curvature features of each scale layer, combines surface texture features and three-dimensional spatial depth information to generate a scale-invariant descriptor, and performs cross-period disease feature matching based on the scale-invariant descriptor; based on the three-dimensional point cloud data transmitted by the 3D modeling unit and camera pose parameters, performs spatial integration calculations on the disease contour through a distortion correction model constrained by the geometric features of the bridge structure, converting the pixel coordinates in the two-dimensional image into three-dimensional spatial physical coordinates; based on the multi-period disease matching results and environmental parameters collected by the data acquisition unit, analyzes the disease evolution law through a temporal neural network, generates a disease development trend representation including an expansion rate vector and direction parameters, and outputs the core disease information to the precise positioning fusion unit;

[0010] The precise positioning fusion unit receives the core disease information output by the disease identification and life cycle tracking unit, combines it with UAV positioning data, integrates the high-precision bridge 3D model generated by the 3D modeling unit with the UAV real-time positioning data, and outputs the 3D coordinates and position parameters of the disease area.

[0011] The data storage and output unit receives data processed by the data acquisition unit, 3D modeling unit, disease identification and life cycle tracking unit, and precise positioning fusion unit, stores the data in a distributed encrypted database, and outputs predictive maintenance decision reports and data interfaces.

[0012] As a further improvement to this technical solution, the data acquisition unit includes a bridge data acquisition module, a UAV status acquisition module, and an environmental parameter acquisition module, wherein:

[0013] The bridge data acquisition module is based on the lidar and high-definition camera carried by the UAV, and collects bridge point cloud data and surface image data across periods at preset time intervals.

[0014] The UAV status acquisition module synchronously acquires the attitude data of the UAV during flight, as auxiliary information for data calibration;

[0015] The environmental parameter acquisition module collects environmental data around the bridge.

[0016] The output data from the bridge data acquisition module, the UAV status acquisition module, and the environmental parameter acquisition module are synchronously transmitted to the 3D modeling unit and the disease identification and life cycle tracking unit.

[0017] As a further improvement to this technical solution, the 3D modeling unit includes a point cloud preprocessing module, an image registration module, and a 3D fusion modeling module, wherein:

[0018] The point cloud preprocessing module receives bridge point cloud data from the data acquisition unit, uses a filtering algorithm to remove point cloud noise, and completes cross-period point cloud registration through an iterative nearest point algorithm.

[0019] The image registration module receives bridge surface image data from the data acquisition unit, extracts image feature points through the scale-invariant feature transformation algorithm, and completes the stitching of images across different periods.

[0020] The 3D fusion modeling module integrates the registered point cloud from the point cloud preprocessing module with the stitched image from the image registration module to generate a high-precision 3D bridge model and corresponding camera pose parameters, which are then synchronously transmitted to the defect identification and life cycle tracking unit.

[0021] As a further improvement to this technical solution, the disease identification and life cycle tracking unit includes a disease identification module, a feature description and matching module, a contour distortion correction module, and a disease evolution analysis module, wherein:

[0022] When identifying a defect for the first time, the defect identification module assigns a unique and persistent defect identification based on the topological relationship of the bridge components.

[0023] The feature description and matching module constructs a curvature pyramid and extracts the contour curvature features of each scale layer. It combines surface texture features and three-dimensional spatial depth information to generate a scale-invariant descriptor and completes the feature matching of diseases across different periods.

[0024] The contour distortion correction module performs distortion correction and three-dimensional coordinate transformation on the disease contour based on the three-dimensional point cloud data of the 3D modeling unit and the camera pose parameters.

[0025] The disease evolution analysis module analyzes the disease evolution pattern and generates trend representation based on multi-stage matching results and environmental parameters, and outputs the core disease information to the precise positioning and fusion unit.

[0026] As a further improvement to this technical solution, the process of generating disease identification identifiers by the disease identification module includes the following steps;

[0027] S31.1 Extract the bridge components corresponding to the first identified defects, and generate a 12-bit component topology code based on the component number on the bridge design drawings. The code is mapped one-to-one with the physical location of the component;

[0028] S31.2 Record the time information of the first detection of the disease and generate an 8-digit first detection timestamp. ;

[0029] S31.3 Perform an MD5 hash operation on the initial contour point set of the disease, and take the first 4 bits of the result as the contour feature check code. ;

[0030] S31.4, will and By piecing together the data, we can obtain the disease identification identifier. This establishes a unique identity association for tracking diseases across different periods.

[0031] As a further improvement to this technical solution, the process of generating scale-invariant descriptors and completing cross-period matching by the feature description and matching module includes the following steps;

[0032] S32.1, Based on Scale Factor A multi-scale space is generated, and Gaussian smoothing and Laplacian-Gaussian operation are performed sequentially on the disease contour point set to extract the contour curvature features of each scale layer. ;

[0033] S32.2 Extract surface texture features and three-dimensional spatial depth features of the diseased area, and... After normalization, the components are weighted and fused to generate scale-invariant descriptors. The weights are determined by principal component analysis to assess their contribution to the variance of disease sample characteristics.

[0034] S32.3 Calculate the scale invariance descriptor for diseases at different stages. similarity The system determines whether the diseases belong to the same disease based on a preset threshold, and matches the results with the disease identification identifier in S31.4. Binding forms a feature association chain across different periods.

[0035] As a further improvement to this technical solution, the process of the contour distortion correction module completing the contour distortion correction and three-dimensional coordinate transformation of the defect includes the following steps;

[0036] S33.1. Extract the inherent geometric features of the components where the defects are located from the high-precision 3D bridge model of the 3D modeling unit, and determine the component design benchmark point as a reference point. ;

[0037] S33.2 Call the camera pose parameters output by the 3D modeling unit, and the 3D point cloud depth value corresponding to the disease area;

[0038] S33.3, First perform distortion correction on the two-dimensional pixels of the defect outline, then combine... Camera parameters and point cloud depth values ​​are converted into three-dimensional physical coordinates. ;

[0039] S33.4. Perform interpolation fitting on the coordinate transformation results to obtain the three-dimensional contour of continuous diseases, providing three-dimensional data support for the extraction of three-dimensional spatial depth features, accurate calculation of contour curvature features, and matching of cross-period disease features in the process of generating scale-invariant descriptors.

[0040] As a further improvement to this technical solution, the process of the disease evolution analysis module completing disease evolution analysis and trend characterization includes the following steps;

[0041] S34.1 Integrate the three-dimensional physical size parameters of diseases in multiple phases and the environmental parameters of the data acquisition unit, and normalize them to form a network input matrix;

[0042] S34.2 Input the input matrix into the LSTM network and calculate the weights of the data for each period through the attention mechanism. After weighting the hidden states of the network, they are mapped to disease feature vectors through a fully connected layer. ;

[0043] S34.3 Calculate the disease expansion rate vector based on the differences in disease size parameters across multiple periods and the corresponding inspection time intervals. Combining disease feature vectors Generate a trend vector field containing the expansion rate and direction;

[0044] S34.4 Integrate the three-dimensional contour and trend vector field of the disease into core disease information, and output it to the precise positioning and fusion unit, while simultaneously integrating it with the disease identification information in S31.4. The binding process forms a full lifecycle data chain consisting of "identity-scale invariant descriptor-3D coordinates-development trend".

[0045] As a further improvement to this technical solution, the precise positioning fusion unit includes a positioning data receiving module, a data fusion processing module, and a coordinate parameter output module, wherein:

[0046] The positioning data receiving module receives the core disease information from the disease identification and life cycle tracking unit, the high-precision three-dimensional bridge model from the 3D modeling unit, and the real-time positioning data from the UAV, and performs unified format processing on the received data.

[0047] The data fusion processing module uses real-time UAV positioning data as a basis, combines the spatial benchmark of a high-precision bridge 3D model, and performs multi-source comparison and fusion with the 3D contour data in the core information of the defects to complete data calibration.

[0048] The coordinate parameter output module maps the fused calibration results to the coordinate system of the bridge's three-dimensional model, generates the three-dimensional coordinates of the diseased area and the corresponding position parameters, and transmits them to the data storage and output unit.

[0049] The second objective of this invention is to provide a method for intelligent positioning of 3D bridge defect areas based on unmanned aerial vehicles (UAVs). Based on the aforementioned intelligent positioning system for 3D bridge defect areas based on UAVs, the method includes the following steps:

[0050] S1. Multi-source data acquisition across periods: Using a drone equipped with a lidar and a high-definition camera, bridge point cloud data and surface image data are acquired across periods at preset time intervals. Attitude data of the drone during flight and environmental parameters of the surrounding environment of the bridge are acquired simultaneously. The bridge point cloud data, surface image data, drone attitude data and environmental parameters are then transmitted to S2 and S3.

[0051] S2, High-precision 3D modeling of bridge: Receive the collected raw data from multiple periods, use a filtering algorithm to remove noise from the bridge point cloud data and complete the cross-period registration through the iterative nearest point algorithm, extract feature points from the bridge surface image data through the scale-invariant feature transformation algorithm and complete the cross-period stitching, fuse the registered point cloud and the stitched image to generate a high-precision 3D model of the bridge and the corresponding camera pose parameters, and transmit them to S3.

[0052] S3. Intelligent Disease Identification and Full Lifecycle Tracking: A unique identifier is assigned to each newly identified disease, consisting of a bridge component topological code, a first detection timestamp, and a disease initial contour feature check code. A multi-scale space is generated based on a scale factor, extracting multi-scale contour curvature, surface texture, and 3D depth features of the disease and weighted fusion to generate a scale-invariant descriptor. This completes cross-period disease matching and binds it to the identifier. Combining the geometric features of the bridge's 3D model with camera pose parameters, the 2D pixel coordinates of the disease are corrected and converted into 3D physical coordinates. Multi-period disease parameters and environmental data are integrated, and the disease evolution trend is analyzed using an LSTM network and attention mechanism to form a full lifecycle data chain, which is then output to S4.

[0053] S4. Precise positioning and fusion of diseased areas: Receive core information of diseased areas, high-precision 3D bridge model and real-time positioning data of UAVs. After unifying the data format, based on the real-time positioning data of UAVs, and combined with the spatial reference of the 3D bridge model, complete the multi-source data comparison and calibration, map the fusion results to the 3D bridge coordinate system, generate the 3D coordinates of the diseased areas and corresponding position parameters, and transmit them to S5.

[0054] S5. Data Storage and Decision Output: Receives various types of data processed by S1-S4, stores them in a distributed encrypted database, generates predictive maintenance decision reports based on the stored data, and provides standardized data interfaces for subsequent applications.

[0055] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0056] 1. This invention assigns a unique and persistent identity identifier to each newly identified defect based on the bridge component topology, the first detection timestamp, and the initial contour feature check code. Combined with a scale-invariant descriptor generated by weighted fusion of multi-scale contour curvature features extracted from a curvature pyramid with surface texture and three-dimensional spatial depth features, it achieves accurate matching of defects across different periods, effectively avoiding the problem of repeated or missed identification of the same defect. Simultaneously, it integrates three-dimensional physical size parameters and environmental parameters of defects from multiple periods, analyzes the evolution patterns of defects through an LSTM network and attention mechanism, and generates a development trend representation including an expansion rate vector and direction parameters. This forms a full lifecycle data chain of "identity-scale-invariant descriptor-three-dimensional coordinates-development trend," enabling continuous tracking of the complete process of defects from their emergence to expansion, providing comprehensive and dynamic data support for predictive maintenance decisions for bridges.

[0057] 2. This invention relies on a distortion correction model constrained by the geometric features of the bridge structure. It extracts the inherent geometric features of the components from a high-precision 3D bridge model to determine the design reference points. Combined with camera pose parameters and 3D point cloud depth information, it performs distortion correction on the defect contours and completes the accurate conversion from 2D pixel coordinates to 3D spatial physical coordinates. Simultaneously, through a precise positioning fusion unit, it unifies the format of the core defect information, the high-precision 3D bridge model, and the real-time positioning data from UAVs, and performs multi-source comparison and fusion calibration. Finally, it outputs the 3D coordinates and position parameters corresponding to the defect area, solving the problem that defect positioning in traditional technologies is limited to a 2D level or approximate area marking. This meets the high-precision positioning requirements of bridge maintenance construction for defect locations. Furthermore, all processed data is stored through a distributed encrypted database, which can output predictive maintenance decision reports and provide standardized data interfaces, adapting to the actual application scenarios of routine bridge maintenance. Attached Figure Description

[0058] Figure 1 This is a schematic diagram of the system framework of the present invention;

[0059] Figure 2 This is a schematic diagram of the macroscopic technical process of full life cycle tracking in this invention;

[0060] Figure 3 This is a schematic diagram of the method steps of the present invention;

[0061] The meanings of the labels in the diagram are as follows:

[0062] 1. Data acquisition unit; 11. Bridge data acquisition module; 12. UAV status acquisition module; 13. Environmental parameter acquisition module;

[0063] 2. 3D modeling unit; 21. Point cloud preprocessing module; 22. Image registration module; 23. 3D fusion modeling module;

[0064] 3. Disease identification and life cycle tracking unit; 31. Disease identification module; 32. Feature description and matching module; 33. Contour distortion correction module; 34. Disease evolution analysis module;

[0065] 4. Precise positioning fusion unit; 41. Positioning data receiving module; 42. Data fusion processing module; 43. Coordinate parameter output module;

[0066] 5. Data storage and output unit. Detailed Implementation

[0067] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0068] like Figures 1-2 As shown, this embodiment provides an intelligent positioning system for 3D inspection of bridge defect areas based on unmanned aerial vehicles (UAVs), including:

[0069] Data acquisition unit 1 uses a drone equipped with lidar and high-definition camera to collect bridge point cloud data and surface image data across periods, and simultaneously acquires drone attitude data and environmental parameters, which are then transmitted to 3D modeling unit 2 and disease identification and life cycle tracking unit 3.

[0070] All modules of Data Acquisition Unit 1 are integrated into an industrial-grade multi-rotor UAV platform. This UAV must possess stable hovering capabilities, long endurance, and a payload expansion interface to accommodate the simultaneous mounting of LiDAR, high-definition cameras, and environmental sensors. The UAV's flight path is preset via a ground station, dividing the inspection area (such as key component areas like the main beam, piers, and supports) based on bridge design drawings, ensuring that the collected data completely covers the entire bridge area without any blind spots. All acquisition devices in Data Acquisition Unit 1 are synchronously triggered through the UAV's central controller, ensuring consistency in the timestamps of bridge point cloud data, surface image data, UAV attitude data, and environmental parameters, providing a foundation for subsequent data fusion and calibration.

[0071] In this embodiment, the data acquisition unit 1 includes a bridge data acquisition module 11, a UAV status acquisition module 12, and an environmental parameter acquisition module 13, wherein:

[0072] The bridge data acquisition module 11 uses a lidar and high-definition camera mounted on a drone to collect bridge point cloud data and surface image data across periods at preset time intervals.

[0073] Specifically, the bridge data acquisition module 11 adopts a dual-sensor collaborative acquisition scheme of "LiDAR + HD camera". The LiDAR is a 16-line or 32-line industrial-grade LiDAR (supporting non-contact 3D measurement), and the HD camera is a full-frame RGB camera (resolution not less than 4K). Both are fixed by a drone gimbal, which has 360° rotation and anti-shake function to ensure the stability of the sensor posture during the acquisition process.

[0074] Specifically, the preset time interval is set according to the type of bridge components and the degree of susceptibility to defects. For areas with high incidence of defects, such as the bottom of the main beam and the connection between the piers, the collection time interval is set to 0.5 seconds / frame. For relatively stable areas such as the bridge deck and bearings, the collection time interval is set to 1 second / frame. This ensures that data integrity is guaranteed while avoiding redundant data from occupying storage resources.

[0075] Specifically, the implementation method for cross-period data collection is as follows: the initial inspection is set as the baseline period, collecting point cloud data and surface image data of the bridge in its initial state; subsequent inspections are conducted at regular intervals of 3 months and 6 months (i.e., subsequent periods), with the inspection route and data collection time interval of each period remaining consistent with the baseline period to ensure the comparability of cross-period data. During the data collection process, the lidar mainly acquires the three-dimensional spatial point cloud data of the bridge structure (for subsequent 3D modeling and depth feature extraction), while the high-definition camera simultaneously acquires surface image data of the corresponding area (for disease texture feature identification). The acquisition ranges of the two completely overlap, achieving a one-to-one spatial correspondence.

[0076] The UAV status acquisition module 12 synchronously acquires the attitude data of the UAV during flight, as auxiliary information for data calibration;

[0077] Specifically, the UAV status acquisition module 12 relies on the UAV's built-in inertial measurement unit (IMU), GPS / BeiDou positioning module, and flight control system to synchronously acquire attitude data during the UAV's flight, including pitch angle, roll angle, yaw angle (used to characterize sensor attitude), flight speed, flight altitude, and real-time geographic location coordinates. The acquisition frequency of attitude data is consistent with that of the bridge data acquisition module 11 (0.5 seconds / time or 1 second / time), the data format adopts Euler angle standard format (unit: degree), and the geographic location coordinates adopt the WGS-84 coordinate system.

[0078] Furthermore, the attitude data acquired by the UAV status acquisition module 12 serves as "auxiliary information for data calibration." Specific application scenarios include: when performing point cloud registration in the 3D modeling unit 2, the attitude data is used to compensate for point cloud offsets caused by slight shaking during UAV flight; when performing distortion correction in the disease identification and lifecycle tracking unit 3, the attitude data is combined to correct image distortion caused by the camera's shooting angle, ensuring the accuracy of subsequent data processing.

[0079] Environmental parameter acquisition module 13 collects environmental data around the bridge;

[0080] The output data from the bridge data acquisition module 11, the UAV status acquisition module 12, and the environmental parameter acquisition module 13 are synchronously transmitted to the 3D modeling unit 2 and the disease identification and life cycle tracking unit 3.

[0081] Specifically, the environmental parameter acquisition module 13 adopts an integrated multi-parameter environmental sensor, which is mounted through the UAV load interface and shares a power supply with the bridge data acquisition module 11 and the UAV status acquisition module 12.

[0082] Specifically, the collected "bridge surrounding environment data" includes: ambient temperature (measurement range: -20℃~60℃), relative humidity (measurement range: 0%~100%), near-surface wind speed (measurement range: 0~15m / s), precipitation (qualitative record: no precipitation, light rain, moderate rain), and air quality index (AQI, characterizing the content of dust and corrosive gases in the air). Environmental parameters are collected every 5 minutes, with corresponding timestamps recorded during collection, and are associated and stored with bridge point cloud data and surface image data from the same period. The collected environmental data is primarily used for the disease identification and lifecycle tracking unit 3 to analyze disease evolution, providing data support for determining the impact of environmental factors (such as high humidity and highly corrosive environments) on disease expansion.

[0083] In addition, the output data of the bridge data acquisition module 11, the UAV status acquisition module 12 and the environmental parameter acquisition module 13 are aggregated to the central controller through the UAV’s built-in high-speed data bus (such as CAN bus). The central controller timestamps all the data (with an error of no more than 10ms) and converts it into a unified JSON data format.

[0084] Meanwhile, data transmission employs a dual-mode approach of "real-time transmission + local caching": during drone flight, data is transmitted in real-time to the 3D modeling unit 2 and the disease identification and lifecycle tracking unit 3 at the ground station via a 5G / 4G communication module; additionally, data is synchronously stored on an encrypted storage card (with a capacity of no less than 1TB) on the drone's local machine to prevent data loss due to communication interruptions. After the inspection task is completed, the ground station can read the locally cached data via a wired connection to verify and complete it with the real-time transmitted data, ensuring data integrity.

[0085] 3D modeling unit 2 receives raw data from data acquisition unit 1 across different periods, and after fusion processing, generates a high-precision three-dimensional model of the bridge and camera pose parameters, which are then transmitted to the defect identification and life cycle tracking unit 3.

[0086] The 3D modeling unit 2, as the core of 3D data processing, is deployed on a high-performance ground workstation (adapted to the needs of parallel processing of multi-source data). By receiving the raw data (bridge point cloud data, surface image data, and UAV attitude data) transmitted from the data acquisition unit 1, it performs layered preprocessing, registration and stitching, and fusion modeling to generate a high-precision 3D bridge model and camera pose parameters that meet the needs of defect identification and localization, providing core data support for subsequent defect contour distortion correction and 3D coordinate transformation.

[0087] In this embodiment, the 3D modeling unit 2 includes a point cloud preprocessing module 21, an image registration module 22, and a 3D fusion modeling module 23, wherein:

[0088] The point cloud preprocessing module 21 receives the bridge point cloud data from the data acquisition unit 1, uses a filtering algorithm to remove point cloud noise, and completes the registration of cross-period sub-point clouds through the iterative nearest point algorithm.

[0089] Specifically, the point cloud preprocessing module 21 receives the 3D point cloud data (in LAS standard point cloud format) output by the bridge data acquisition module 11, and completes the preprocessing according to the process of "noise removal - coordinate calibration - cross-period registration" to ensure the integrity and consistency of the point cloud data. The specific operation is as follows:

[0090] Noise Removal:

[0091] A combined filtering algorithm of "Gaussian filtering + statistical filtering" is adopted to achieve accurate removal of point cloud noise in two steps.

[0092] Gaussian filtering: First, determine the point cloud acquisition density based on the lidar parameters (e.g., 16-line lidar) and the UAV flight altitude (50~80m), and set the neighborhood radius to 0.05~0.1m; for each target point cloud data point, traverse all neighboring points within its neighborhood radius, extract the three-dimensional coordinate information of the neighboring points, and calculate the average and variance of the three-dimensional coordinates of all points in the neighborhood; construct a two-dimensional Gaussian kernel function based on the preset standard deviation (0.8~1.2), and use the Euclidean distance between the target point and each neighboring point as the weight coefficient to calculate the weighted average of the coordinates of the points in the neighborhood, so as to obtain the filtered three-dimensional coordinates of the target point. Repeat this process for all point cloud data points to complete the smoothing and denoising of the point cloud and eliminate the slight noise interference caused by flight jitter.

[0093] Statistical filtering: Using the neighborhood radius set by Gaussian filtering, the number of points in the neighborhood of each target point is counted. If the number of neighborhood points is lower than the preset threshold of 15-20, the point is directly identified as an isolated noise point and removed from the point cloud data. For target points with a sufficient number of neighborhood points, the Euclidean distance between the point and all its neighbors is calculated, and the average and standard deviation of all distances are further calculated. The distance threshold is set as "average distance + 1.5 times the standard deviation of distance". If the Euclidean distance between a neighboring point and the target point exceeds this threshold, it is identified as a flying point (such as invalid points caused by dust in the air or reflections from birds) and removed. Finally, valid point cloud data is retained.

[0094] Coordinate calibration:

[0095] The attitude data (pitch angle, roll angle, yaw angle) and GPS / BeiDou positioning data (geodetic coordinates) output by the UAV status acquisition module 12 are used as the calibration basis.

[0096] Initial coordinate system transformation: The original point cloud data is transformed from the UAV's local coordinate system (with the UAV's center of gravity as the origin, the forward direction of the fuselage as the X-axis, and the vertical upward direction as the Z-axis) to the initial values ​​of the geodetic coordinate system. During the transformation process, the initial spatial position is determined by referring to the UAV's GPS positioning data to ensure that the point cloud data is initially associated with the real geographic coordinates.

[0097] Attitude offset compensation: Based on attitude data, coordinate deviation is corrected. The vertical offset of the point cloud is corrected by the pitch angle, the horizontal offset is corrected by the roll angle, and the vertical offset is corrected by the yaw angle. The compensation logic is to use trigonometric functions to convert the attitude angles into the corresponding coordinate axis corrections to offset the point cloud position deviation caused by attitude changes during UAV flight.

[0098] Unified coordinate system: Finally, all point cloud data collected under different flight attitudes are uniformly mapped to the WGS-84 geodetic coordinate system to ensure that subsequent cross-period registration has a unified spatial reference and avoids registration errors caused by coordinate system differences.

[0099] Inter-period point cloud registration:

[0100] Using the point cloud data collected and preprocessed during the initial inspection (baseline period) as a fixed reference benchmark, the point cloud data from subsequent periods, after noise removal and coordinate calibration, are used as the registration data. The registration process is performed in two steps: "coarse registration" and "fine registration".

[0101] The coarse registration process specifically includes: extracting the GPS positioning data (geographic coordinates of the start and end points and key inspection points) of the UAV status acquisition module 12 corresponding to the data to be registered, comparing it with the corresponding position coordinates of the reference point cloud, and calculating the initial translation distance and initial rotation angle of the point cloud to be registered relative to the reference point cloud; based on these initial parameters, performing overall translation and rotation operations on the point cloud to be registered, mapping it to the approximate spatial range of the reference point cloud, so that the overlap area of ​​the two point clouds accounts for no less than 80%, significantly reducing the initial deviation and laying the foundation for fine registration.

[0102] Fine-grained registration (ICP algorithm) specifically includes:

[0103] Sampling: Randomly select sampling points from the point cloud to be registered at a ratio of 5% to 10% (balancing computational efficiency and registration accuracy) to avoid redundant calculations; Nearest point search: For each sampling point, quickly find its corresponding nearest point in the reference point cloud using the kd-tree algorithm to form several pairs of point clouds.

[0104] Error function construction and parameter solving: Using the sum of squared Euclidean distances of all corresponding point pairs as the error function, the optimal rotation matrix and translation vector that minimize the error function are solved by the singular value decomposition method;

[0105] Point cloud transformation and error judgment: Use the rotation matrix and translation vector obtained by solving to perform geometric transformation on all points in the point cloud to be registered, and calculate the mean square error between the point cloud to be registered and the reference point cloud after transformation.

[0106] Iteration termination judgment: If the mean square error is less than the preset convergence threshold of 0.01~0.03m, or the number of iterations reaches the upper limit of 50~100 times, the iteration stops and the final registered point cloud data is output; if the termination condition is not met, the sampling points are re-extracted in the first step and the above process is repeated until the registration accuracy requirement is met, so as to achieve accurate alignment of point clouds across periods.

[0107] The image registration module 22 receives bridge surface image data from the data acquisition unit 1, extracts image feature points through the scale-invariant feature transformation algorithm, and completes the stitching of images across different periods.

[0108] Specifically, the image registration module 22 receives 4K resolution RGB surface image data output by the bridge data acquisition module 11, and completes feature extraction and stitching of images across different time periods using the Scale Invariant Feature Transform (SIFT) algorithm to ensure the texture continuity and spatial consistency of the stitched image. The specific operation is as follows:

[0109] Feature point extraction:

[0110] Multi-scale space construction: The original 4K image is subjected to multi-scale Gaussian blur processing, with the scale factor increasing from 1.6 to 6.4 in steps of 2 to the power of one-third, generating multiple sets of images with different blur levels to form an image scale space; each set of scale space contains several layers of images to ensure that disease feature points of different sizes can be captured.

[0111] Extreme point detection: Subtract the blurred images of two adjacent layers in scale space to generate a Laplacian-Gaussian image; traverse each pixel of the Laplacian-Gaussian image and compare it with the eight neighboring pixels of its own scale layer and adjacent scale layers. If the point is a local extreme point (greater than or less than all the compared pixels), it is determined to be a potential feature point, and candidate points with scale invariance are initially screened out.

[0112] Precise feature point localization: Three-dimensional interpolation calculations are performed on potential feature points to accurately determine their sub-pixel positions, corresponding scales, and orientations in the image; by statistically analyzing the gradient directions and magnitudes of pixels in the neighborhood of the feature point, 16 sets of gradient direction histograms are generated, and the peak direction of the histogram is taken as the main direction of the feature point. If there are multiple peaks (the difference is less than 80% of the peak), an auxiliary direction is added to ensure that the feature point has rotation invariance.

[0113] Feature descriptor generation: Centered on the feature point, a 16×16 neighborhood window is selected in the corresponding scale image and divided into 4×4 sub-windows; 8 sets of gradient direction histograms are calculated for each sub-window, and the histograms of all sub-windows are stitched together to form a 128-dimensional feature descriptor; the feature descriptor is normalized to eliminate the influence of illumination changes, and finally a feature descriptor with scale, rotation and illumination invariance is obtained.

[0114] Feature point matching (K-nearest neighbors + RANSAC algorithm):

[0115] K-Nearest Neighbor Matching (K=2): For each feature descriptor of the image to be matched (subsequent images), calculate its Euclidean distance with all feature descriptors of the reference image (reference image stitched together), and select the two candidate matching pairs with the smallest distance; set a threshold of "second smallest distance / smallest distance ≥ 1.5", if this condition is met, retain the matching pair corresponding to the smallest distance and determine it as a valid candidate matching pair; otherwise, determine it as a fuzzy match (such as a mismatch caused by similar textures) and discard it, thus initially improving the matching accuracy.

[0116] RANSAC Algorithm Refinement: Random Sampling: Randomly select 4 pairs of non-collinear matching points from the valid candidate matching pairs, and calculate the homography matrix (representing the geometric transformation relationship between the two images) based on these 4 pairs of matching points; Reprojection Error Calculation: Map all feature points in the image to be matched to the coordinate system of the reference image through the homography matrix, and calculate the Euclidean distance between the mapped point and the corresponding matching point as the reprojection error; Interior Point Counting: Set a reprojection error threshold of 2~3 pixels, determine the matching pairs with errors less than the threshold as interior points, and count the number of interior points; Optimal Matrix Selection: Repeat the above steps for 1000~2000 iterations, select the homography matrix with the most interior points as the optimal matrix, and retain the corresponding interior point matching pairs as the final valid matching pairs to ensure that the matching accuracy is not less than 95%.

[0117] Image stitching across different periods:

[0118] Geometric transformation alignment: Using the base period image stitching result as the base map, subsequent period images undergo geometric transformations (translation, rotation, scaling) based on the optimal homography matrix, mapping the subsequent period images to the pixel coordinate system of the base image, so that images from different periods and different perspectives are accurately aligned in spatial position, ensuring the continuity of bridge components.

[0119] Overlapping region fusion: For the overlapping regions after image stitching (overlap rate controlled at 15%~25%), a linear weighted fusion algorithm is used for processing; according to the principle of "the closer to the overlap boundary, the smaller the pixel weight", the pixel gray value of the overlapping region of the two images is calculated by weighted average, and the weight transitions smoothly from 0 to 1, eliminating stitching gaps and brightness differences, and ensuring natural texture transition.

[0120] Panoramic image generation: All aligned and fused images are stitched together to generate a panoramic stitched image covering the entire bridge area; the pixel coordinates of the stitched image maintain a one-to-one mapping relationship with the spatial coordinates of the registered point cloud output by the point cloud preprocessing module 21, providing a data foundation for subsequent 3D fusion modeling.

[0121] The 3D fusion modeling module 23 fuses the registered point cloud of the point cloud preprocessing module 21 with the stitched image of the image registration module 22 to generate a high-precision 3D model of the bridge and the corresponding camera pose parameters, which are then synchronously transmitted to the defect identification and life cycle tracking unit 3.

[0122] Specifically, the 3D fusion modeling module 23, based on the registered point cloud (3D geometric information) from the point cloud preprocessing module 21 and the stitched image (2D texture information) from the image registration module 22, generates a high-precision 3D bridge model and corresponding camera pose parameters through the collaborative fusion of geometry and texture. The specific operation is as follows:

[0123] Fusion algorithm implementation (Poisson fusion):

[0124] Point cloud normal vector estimation: Perform neighborhood search on the registered point cloud to obtain the neighboring point set of each point, calculate the normal vector of the point through principal component analysis, and determine the orientation of the point cloud surface; normalize the normal vector and ensure that the normal vectors of all points are in the same orientation (pointing outwards from the bridge), providing a basis for implicit surface construction.

[0125] Implicit Surface Construction: An implicit Poisson equation is constructed based on the point cloud normal vectors, and the point cloud surface is used as the boundary condition of the equation. By solving the Poisson equation, a continuous three-dimensional implicit surface function is obtained. This function can accurately fit the geometric shape of the point cloud and form the three-dimensional geometric contour of the bridge.

[0126] Texture mapping and fusion: The panoramic stitched image generated by the image registration module 22 is used as the texture data source. Based on the spatial coordinates of each vertex on the three-dimensional implicit surface, it is back-mapped to the corresponding pixel position of the stitched image. The texture pixel value at this position is obtained through bilinear interpolation algorithm and mapped to the vertex of the surface to achieve accurate fusion of geometric information and texture information. At the same time, the geometric features of the bridge structure (such as the straight boundary of the beam and the cylindrical surface features of the pier) are combined to constrain and correct the local texture stretching or distortion problem, so as to ensure that the texture and the geometric shape are highly matched.

[0127] High-precision 3D model generation:

[0128] Model parameter settings: The generated 3D model is in PLY (polygon file format), the point cloud density is no less than 500 points / square meter, and the texture resolution is consistent with the original 4K image (3840×2160 pixels); the geometric error of the model is controlled within the allowable range of engineering, and meets the centimeter-level accuracy requirements for subsequent disease location.

[0129] Model layered annotation: The 3D model is divided and annotated in layers according to the type of bridge components (main beam, pier, bearing, bridge deck, etc.). Each component is assigned a unique topology code, which corresponds one-to-one with the component topology code in the defect identification and life cycle tracking unit 3, which facilitates the rapid location of the component where the defect is located.

[0130] Model optimization: The generated 3D model is smoothed and optimized, and local redundant triangular patches are removed to reduce the amount of model data. At the same time, the detailed features of disease-prone areas (such as component connections and corners) are preserved to ensure that the model is both lightweight and meets the needs of disease identification.

[0131] Camera pose parameter calculation:

[0132] Camera pose parameters include intrinsic and extrinsic parameters, which characterize the camera's position, attitude, and imaging characteristics during shooting. The specific calculations are as follows:

[0133] Intrinsic parameter calculation: Camera intrinsic parameters include focal length, principal point coordinates, and distortion coefficients; First, the initial intrinsic parameter data calibrated by the camera at the factory is obtained, and then optimized through on-site calibration: an image of a checkerboard calibration board of known size is captured, and based on the three-dimensional coordinates of the calibration board and the image pixel coordinates, the optimized intrinsic parameter values ​​are solved through a calibration algorithm to eliminate errors caused by lens distortion.

[0134] Extrinsic parameter calculation: The camera extrinsic parameters include the rotation matrix (representing the camera pose) and the translation vector (representing the camera position), which are solved using the PnP (Perspective-n-Point) algorithm; valid feature point pairs are selected from the image registration module 22, where the coordinates of the image feature points come from the stitched image and the coordinates of the three-dimensional space points come from the registration point cloud; an observation equation is constructed based on the pinhole imaging model, and the three-dimensional point coordinates are associated with the two-dimensional image point coordinates. The observation equation is solved using the LM (Levenberg-Marquardt) iterative optimization algorithm to obtain the optimal rotation matrix and translation vector.

[0135] Parameter association storage: All camera pose parameters (intrinsic parameters + extrinsic parameters) are associated and stored with the 3D model, and categorized and indexed by shooting time and inspection area to ensure that the pose parameters of the corresponding shooting angle can be quickly called when the subsequent disease identification and life cycle tracking unit 3 performs distortion correction.

[0136] In addition, the high-precision bridge 3D model (PLY format) and camera pose parameters (JSON format) generated by the 3D fusion modeling module 23 adopt the same "real-time transmission + local caching" dual mode as the data acquisition unit 1, as follows:

[0137] Data transmission: Data is transmitted in real time to the disease identification and life cycle tracking unit 3 via a high-speed network interface (wired gigabit Ethernet or 5G wireless communication). Before transmission, the data is compressed (the model uses the LZ4 compression algorithm, and the image-related data uses the JPEG2000 compression algorithm) to improve transmission efficiency and avoid transmission delays caused by large data volume.

[0138] Local caching: Data is synchronously stored in the distributed storage array of the ground workstation (supporting TB-level data expansion). During storage, data is classified and stored according to the directory structure of "inspection period - bridge component - data type" to facilitate subsequent retrieval and management.

[0139] Data verification: The receiving end (disease identification and life cycle tracking unit 3) generates a data check code using the MD5 algorithm and compares it with the check code of the sending end to verify the data integrity; if there is data loss or damage, the local cache data retransmission mechanism is automatically triggered to ensure that the data received by the disease identification and life cycle tracking unit 3 is accurate and usable.

[0140] The disease identification and life cycle tracking unit 3, upon initial disease identification, assigns a unique and persistent disease identifier based on the topological relationship of bridge components; constructs a curvature pyramid and extracts the contour curvature features of each scale layer, combines surface texture features and 3D spatial depth information to generate a scale-invariant descriptor, and performs cross-period disease feature matching based on the scale-invariant descriptor; based on the 3D point cloud data and camera pose parameters transmitted by the 3D modeling unit 2, it performs spatial integration calculations on the disease contour through a distortion correction model constrained by the geometric features of the bridge structure, converting the pixel coordinates in the 2D image into 3D spatial physical coordinates; based on the multi-period disease matching results and environmental parameters collected by the data acquisition unit 1, it analyzes the disease evolution law through a temporal neural network, generates a disease development trend representation including the expansion rate vector and direction parameters, and outputs the core disease information to the precise positioning fusion unit 4.

[0141] As the core intelligent processing unit, the disease identification and life cycle tracking unit 3 receives environmental parameters from the data acquisition unit 1, the high-precision three-dimensional bridge model from the 3D modeling unit 2, and camera pose parameters. Through identity assignment, multi-feature fusion matching, distortion correction and coordinate transformation, and temporal evolution analysis, it achieves accurate tracking of the entire life cycle of the disease and outputs core information, providing data support for the precise positioning fusion unit 4.

[0142] Disease identification and life cycle tracking unit 3 includes a disease identification module 31, a feature description and matching module 32, a contour distortion correction module 33, and a disease evolution analysis module 34, wherein:

[0143] In this embodiment, when the defect identification module 31 first identifies a defect, it assigns a unique and persistent defect identification based on the topological relationship of the bridge components; the process of generating a defect identification by the defect identification module 31 includes the following steps.

[0144] S31.1 Extract the bridge components corresponding to the first identified defects, and generate a 12-bit component topology code based on the component number on the bridge design drawings. The code is mapped one-to-one with the physical location of the component;

[0145] Specifically, the specific steps for generating 12-bit component topology codes are as follows:

[0146] From the high-precision 3D bridge model output by 3D modeling unit 2, the bridge components where the defects are first identified (such as the left section of the main beam, pier No. 2, etc.) are located, and the components correspond one-to-one with the topological codes of the model layer annotation.

[0147] Referring to the component numbering rules in bridge design drawings, a 12-digit component topology code is generated based on the structure of "bridge segment code + component type code + component serial number code + location subdivision code". The core formula is:

[0148] ;

[0149] in:

[0150] This represents a 12-bit component topology code, which is mapped one-to-one with the physical location of the component.

[0151] This indicates the bridge segment code (2 bits, with a value range of 01 to 99, corresponding to different bridges or bridge segments).

[0152] The component type code is represented by two digits, with the following rules: 01 = main beam, 02 = pier, 03 = support, and 04 = bridge deck.

[0153] This indicates the component serial number code (4 digits, value range 0001~9999, arranged in the order of design drawing serial numbers).

[0154] The location subdivision code (4 bits, value range 0001~9999, representing the specific local location of the component, such as the left segment, the top area, etc.)

[0155] Example as follows: The code corresponding to the left side segment of main beam No. 3 of bridge No. 1 is... .

[0156] S31.2 Record the time information of the first detection of the disease and generate an 8-digit first detection timestamp. ;

[0157] Specifically, the steps for generating the 8-bit initial detection timestamp are as follows:

[0158] The specific time when the disease was first identified is recorded. The time information is taken from the synchronization timestamp of data acquisition unit 1 and kept consistent with the acquisition time of bridge data acquisition module 11 to ensure time synchronization.

[0159] An 8-digit timestamp is generated using a fixed format: "last 4 digits of year + 2 digits of month + 2 digits of day". The core formula is:

[0160] ;

[0161] in:

[0162] This represents an 8-digit timestamp of the first detection.

[0163] Indicates the last 4 digits of the year (e.g., 2025, 2026, etc.);

[0164] Indicates the month (value range 01~12, padded with 0 if less than 2 digits);

[0165] Represents the date (value range 01~31, padded with 0 if less than 2 digits);

[0166] Example: The disease was first detected on June 15, 2025, with the corresponding timestamp being... .

[0167] S31.3 Perform an MD5 hash operation on the initial contour point set of the disease, and take the first 4 bits of the result as the contour feature check code. ;

[0168] Specifically, the steps for generating the 4-bit contour feature check code are as follows:

[0169] Extract the initial contour point set of the disease for the first time. The Canny edge detection algorithm combined with morphological processing is used to extract the set of pixel coordinates of the disease area from the stitched image output by the image registration module 22, and arrange them in clockwise order to ensure consistency.

[0170] Contour point set Perform serialization processing, and convert the two-dimensional coordinates into their corresponding values. The sequence of "" is concatenated into a one-dimensional data string. ;

[0171] For one-dimensional data strings Perform a standard MD5 hash operation to obtain a 64-bit hash result string. The core formula is:

[0172] ;

[0173] in:

[0174] This represents a 4-digit contour feature check code, consisting of a combination of letters and numbers;

[0175] This represents the standard MD5 hash function, which takes a one-dimensional data string as input and outputs a 64-bit string.

[0176] This represents a string truncation function that extracts the first four characters of the result.

[0177] Example as follows: If The calculation result is "7A3F92D1E5C8B064...", then the corresponding contour feature check code is... .

[0178] S31.4, will and By piecing together the data, we can obtain the disease identification identifier. This establishes a unique identity association for tracking diseases across different periods.

[0179] Specifically, the steps for splicing disease identification tags are as follows:

[0180] The strings are concatenated in a fixed order of "component topology code + first detection timestamp + contour feature check code" to form a 24-digit unique defect identifier. The core formula is:

[0181] ;

[0182] Example as follows: When , , At that time, the spliced ​​disease identification mark was ;

[0183] This After being bound to disease information, it is stored in a distributed encrypted database to ensure persistent preservation, establishing a unique identity association for accurate matching of diseases across different periods, and solving the problem of repeated or missed identification of the same disease across different periods in existing technologies.

[0184] In this embodiment, the feature description and matching module 32 constructs a curvature pyramid and extracts the contour curvature features of each scale layer. It then combines surface texture features and three-dimensional spatial depth information to generate a scale-invariant descriptor, thereby completing the cross-period disease feature matching. The process of the feature description and matching module 32 generating a scale-invariant descriptor and completing the cross-period matching includes the following steps.

[0185] S32.1, Based on Scale Factor A multi-scale space is generated, and Gaussian smoothing and Laplacian-Gaussian operation are performed sequentially on the disease contour point set to extract the contour curvature features of each scale layer. ;

[0186] Specifically, the specific operations for extracting contour curvature features at each scale level are as follows:

[0187] First, based on the scale factor A multi-scale space is generated, with the scale factor set according to a geometric progression rule. The core formula is:

[0188] ;

[0189] in:

[0190] Indicates the first Layer scale factor;

[0191] This represents the scale layer index, with a value range of 0 to 5, corresponding to 6 different scales (0.5, 0.65, 0.845, 1.0985, 1.428, 1.856), ensuring coverage of disease features of different sizes;

[0192] Then, the disease outline point set Gaussian smoothing is performed sequentially to eliminate the influence of image noise on the contours. The Gaussian smoothing formula is as follows:

[0193] ;

[0194] in:

[0195] Represents the smoothed first... One outline point;

[0196] express The set of neighborhood points, neighborhood radius It adjusts dynamically with the scale factor;

[0197] express and The Euclidean distance;

[0198] Next, the smoothed contour point set Perform a Laplacian-Gaussian operation to enhance the contour edge features. The formula is as follows:

[0199] ;

[0200] in:

[0201] These are the contour point response values ​​after Laplacian-Gaussian computation. for The number of neighboring points;

[0202] Finally, the contour curvature features of each scale layer are extracted. Based on three adjacent points The curvature is derived from the geometric relationship, and the formula is:

[0203] ;

[0204] in:

[0205] Indicates the first The set of contour curvature features at the scale layer. ;

[0206] Indicates the first Scale layer Curvature values ​​of each contour point;

[0207] This represents the vector cross product operation.

[0208] S32.2 Extract surface texture features and three-dimensional spatial depth features of the diseased area, and... After normalization, the components are weighted and fused to generate scale-invariant descriptors. The weights are determined by principal component analysis to assess their contribution to the variance of disease sample characteristics.

[0209] Specifically, the operations for generating scale-invariant descriptors are as follows:

[0210] First, extract multi-dimensional features:

[0211] Surface texture features The diseased area is extracted from the stitched image of image registration module 22, and three core parameters are calculated using the gray-level co-occurrence matrix. (Energy, entropy, contrast);

[0212] 3D spatial depth features Extract the depth distribution of the diseased area from the registered point cloud of the point cloud preprocessing module 21, and calculate three core parameters, namely... (Mean depth, variance of depth, maximum depth);

[0213] Contour curvature features : Six scale layers Flattened into a one-dimensional vector according to scale index order, i.e. ;

[0214] Then, feature normalization is performed: the min-max normalization method is used to eliminate the dimensional differences between different feature dimensions, and the formula is:

[0215] ;

[0216] in:

[0217] The normalized eigenvalues ​​take values ​​in the range [0,1].

[0218] Represents the original eigenvalues;

[0219] , These represent the minimum and maximum values ​​of the feature dimension in the sample set, respectively; after normalization, we get... ;

[0220] Next, the weights were determined: over 1000 sets of bridge defect samples of different types (cracks, spalling, corrosion, etc.) were collected, and principal component analysis (PCA) was used to calculate the variance contribution of each feature to defect identification, thus obtaining the corresponding weights. ,satisfy ;

[0221] Weighted fusion generates scale-invariant descriptors The core formula is:

[0222] ;

[0223] in:

[0224] The scale-invariant descriptor is a fixed-length vector (in this embodiment, the length is 1). , (Number of contour points);

[0225] The weights for curvature features, texture features, and depth features are respectively (example values: ).

[0226] S32.3 Calculate the scale invariance descriptor for diseases at different stages. similarity The system determines whether the diseases belong to the same disease based on a preset threshold, and matches the results with the disease identification identifier in S31.4. Binding forms a feature association chain across different periods.

[0227] Specifically, the operation for matching disease characteristics across different periods is as follows:

[0228] Calculate the similarity of disease descriptors at different stages The cosine similarity algorithm is used to quantify the similarity between descriptors, and the formula is:

[0229] ;

[0230] in:

[0231] This represents the similarity between scale-invariant descriptors of two disease phases, with a value range of [0,1]. The closer the value is to 1, the higher the similarity.

[0232] The first Period, No. Scale invariance descriptor for periodic diseases;

[0233] Descriptor Length;

[0234] This represents the vector dot product operation;

[0235] Matching determination: Based on the statistical results of the disease sample database, a preset similarity threshold is used. (In this embodiment) ),like Then determine and The corresponding disease is the same disease;

[0236] Association and binding: The matching results are linked to the disease identification generated in S31.4. Binding, forming " The cross-period characteristic association chain is used, and the association chain data is synchronously stored in a distributed encrypted database to ensure that the same disease can be accurately tracked in multiple inspections.

[0237] It should be added that, addressing the limitations of traditional matching methods that rely solely on a single feature or scale, the feature description and matching module 32 constructs a precise matching system based on "multi-scale + multi-dimensional" principles. Compared to traditional matching methods, the feature description and matching module 32 innovatively employs a curvature pyramid to extract contour curvature features at each scale level, while simultaneously integrating surface texture features and three-dimensional spatial depth features. Principal component analysis (PCA) data-driven determination of feature weights, rather than manual fixed assignment, ensures that the generated scale-invariant descriptors possess robustness to scale, pose, and illumination, effectively adapting to potential morphological and perspective changes that may occur during cross-period inspections of bridge defects. The cosine similarity algorithm quantifies descriptor similarity and binds it to the unique identifier of the defect to form a cross-period feature association chain, reducing the probability of duplicate or missed identification of the same defect and improving the accuracy and reliability of cross-period matching.

[0238] In this embodiment, the contour distortion correction module 33 performs distortion correction and three-dimensional coordinate transformation on the disease contour based on the three-dimensional point cloud data of the 3D modeling unit 2 and the camera pose parameters; the process of the contour distortion correction module 33 completing the disease contour distortion correction and three-dimensional coordinate transformation includes the following steps.

[0239] S33.1. Extract the inherent geometric features of the components where the defects are located from the high-precision 3D bridge model of 3D modeling unit 2, and determine the component design benchmark points as reference points. ;

[0240] Specifically, the procedures for determining the design reference points for components are as follows:

[0241] From the high-precision 3D model of the bridge in 3D modeling unit 2, the inherent geometric features of the components where the defects are located are extracted, such as the axis of the beam, the center line of the pier, and the center line of the support.

[0242] Based on the design drawing parameters of the components, key geometric points of the components are selected as design reference points. (Two-dimensional plane reference point, corresponding to the three-dimensional model) (For fixed values), the core formula is:

[0243] ;

[0244] in:

[0245] Two-dimensional plane coordinates representing the design reference points of a component;

[0246] Indicates the geometric feature points of the component (such as the center of symmetry, endpoints, and design reference points);

[0247] This represents a local area of ​​the three-dimensional model of the component where the defect is located.

[0248] This function represents the distance calculation from a point to the model surface.

[0249] Example as follows: If the defect is located in the middle section of the main beam, select the midpoint of the main beam as the reference point, and its two-dimensional coordinates are... The corresponding three-dimensional reference point coordinates are .

[0250] S33.2 Call the camera pose parameters output by 3D modeling unit 2, and the 3D point cloud depth value corresponding to the disease area;

[0251] Specifically, the relevant parameter calls and preprocessing operations are as follows:

[0252] Call the camera pose parameters output by the 3D fusion modeling module 23, including the intrinsic parameter matrix. Rotation matrix Translation vector The parameter format should be consistent with the 3D model data;

[0253] Extract the 3D point cloud depth value corresponding to the diseased area from the registered point cloud of the point cloud preprocessing module 21. Establish a one-to-one mapping relationship between depth values ​​and disease outline pixels, that is, for each pixel... Corresponding unique depth value ;

[0254] The format of the camera pose parameters and depth values ​​is validated to ensure the integrity and validity of the parameters and avoid errors in subsequent calculations.

[0255] S33.3, First perform distortion correction on the two-dimensional pixels of the defect outline, then combine... Camera parameters and point cloud depth values ​​are converted into three-dimensional physical coordinates. ;

[0256] Specifically, the operations for distortion correction and 3D coordinate transformation are as follows:

[0257] First, distortion correction is performed: based on the distortion coefficients in the camera's intrinsic parameters, the two-dimensional pixel points of the defect contour are corrected. Perform radial distortion correction and tangential distortion correction sequentially:

[0258] Radial distortion correction formula:

[0259] ;

[0260] ;

[0261] Tangential distortion correction formula:

[0262] ;

[0263] ;

[0264] in:

[0265] Represents the pixel coordinates after radial distortion correction;

[0266] This represents the pixel coordinates after distortion correction;

[0267] This represents the squared distance from a pixel to the center of the image;

[0268] Radial distortion coefficient;

[0269] The tangential distortion coefficients are derived from the camera intrinsic parameter matrix. ;

[0270] Then, a three-dimensional coordinate transformation is performed: based on the inverse operation logic of the pinhole imaging model, combined with depth values. With component design reference point The core formula for converting two-dimensional pixel coordinates to three-dimensional physical coordinates is:

[0271] ;

[0272] in:

[0273] The three-dimensional physical coordinates of the disease outline points, in meters;

[0274] Representing the rotation matrix The inverse matrix;

[0275] Representing the intrinsic parameter matrix The inverse matrix;

[0276] This represents the point cloud depth value corresponding to the pixel, in meters.

[0277] S33.4. Perform interpolation fitting on the coordinate transformation results to obtain the three-dimensional contour of continuous diseases, providing three-dimensional data support for the extraction of three-dimensional spatial depth features, accurate calculation of contour curvature features, and matching of cross-period disease features in the process of generating scale-invariant descriptors.

[0278] Specifically, the interpolation fitting process for generating continuous 3D contours is as follows:

[0279] For the transformed discrete three-dimensional coordinate points Perform outlier filtering and remove coordinate points that exceed the geometric range of the component (such as points that deviate from the surface of the component by more than 5cm).

[0280] A cubic spline interpolation algorithm is used to fit the selected discrete points to construct a continuous three-dimensional contour curve. The formula is as follows:

[0281] ;

[0282] in:

[0283] Represents a continuous three-dimensional contour curve. , respectively, correspond to continuous functions of three-dimensional coordinates;

[0284] Indicates the first The coefficients of the segment interpolation curve are determined by boundary conditions. and Solve this problem;

[0285] Represents parameterized variables. Corresponding to the discrete points, ;

[0286] The system outputs continuous three-dimensional contour data of diseases and transmits it synchronously to the feature description and matching module 32, providing accurate three-dimensional data support for three-dimensional spatial depth feature extraction, accurate calculation of contour curvature features, and cross-period disease feature matching.

[0287] It should be added that, to address the positioning deviation problem of traditional distortion correction relying solely on camera parameters and being susceptible to the influence of shooting angle, the contour distortion correction module 33 introduces geometric feature constraints of bridge components. The contour distortion correction module 33 does not simply perform distortion correction based on camera intrinsic parameters, but rather extracts the inherent geometric features of the components from the high-precision 3D bridge model, determines the design benchmark point as a coordinate transformation reference, and combines the point cloud depth value output by the 3D modeling unit 2 to achieve a direct conversion from "2D pixel coordinates to 3D physical coordinates," avoiding the accumulation of errors in intermediate steps. Through the process of first correcting distortion (radial + tangential) and then transforming coordinates, coupled with cubic spline interpolation to fit a continuous 3D contour, the accuracy of the defect contour is ensured, and precise 3D data support is provided for subsequent feature extraction and matching, enabling the defect positioning accuracy to meet the actual engineering needs of bridge maintenance.

[0288] In this embodiment, the disease evolution analysis module 34 analyzes the disease evolution pattern and generates trend representation based on multi-stage matching results and environmental parameters, and outputs the core disease information to the precise positioning fusion unit 4. The process of disease evolution analysis module 34 completing disease evolution analysis and trend representation includes the following steps;

[0289] S34.1 Integrate the three-dimensional physical size parameters of diseases in multiple phases and the environmental parameters of data acquisition unit 1, and normalize them to form a network input matrix;

[0290] Specifically, the specific operations for constructing the network input matrix are as follows:

[0291] Integrating data from multiple periods:

[0292] Three-dimensional physical dimensions of the disease Calculated from the three-dimensional profiles of diseases in each stage, including length ,width ,area ,Right now ;

[0293] Environmental parameters Extract the corresponding period data collected by the environmental parameter acquisition module 13 in data acquisition unit 1, including temperature. ,humidity Wind speed Air Quality Index ,Right now ;

[0294] Data normalization: The min-max normalization method is used to process the dimensional parameters and environmental parameters separately. The normalization formula is the same as the normalization formula in S32.2, resulting in... ;

[0295] Constructing the network input matrix Arrange the normalized parameters into a two-dimensional matrix according to the inspection period order. The core formula is:

[0296] ;

[0297] in:

[0298] This represents the input matrix of the LSTM network;

[0299] Indicates the number of inspection periods, and requires... To ensure the extraction of temporal patterns;

[0300] Indicates the first The three-dimensional physical size parameter vector of the disease after period normalization (dimension 3);

[0301] Indicates the first The period-normalized environmental parameter vector (dimension 4).

[0302] S34.2 Input the input matrix into the LSTM network and calculate the weights of the data for each period through the attention mechanism. After weighting the hidden states of the network, they are mapped to disease feature vectors through a fully connected layer. ;

[0303] Specifically, the specific operations for generating disease feature vectors are as follows:

[0304] LSTM network setup: The network structure is set as "input layer → hidden layer (2 layers, 64 neurons per layer) → attention layer → fully connected layer", with an input layer dimension of 7, and the input matrix... The number of columns is the same;

[0305] Attention mechanism calculates the weights of data in each period. The influence of data from each period on disease evolution is learned through the attention layer, and the formula is as follows:

[0306] ;

[0307] ;

[0308] in:

[0309] Indicates the first Attention weights for period data ;

[0310] Indicates the LSTM network's... The hidden state vector for each period (64 dimensions);

[0311] This represents the attention layer weight matrix (32×64 dimensions).

[0312] This represents the attention layer bias vector (32 dimensions).

[0313] Indicates the first Attention score for each period's data;

[0314] Weighted hidden state calculation: The hidden states of each period are summed according to their weights. The formula is as follows:

[0315] ;

[0316] in: The weighted hidden state vector (64 dimensions);

[0317] Fully connected layer mapping: The weighted hidden state is mapped to a fixed-dimensional disease feature vector through a fully connected layer. The formula is:

[0318] ;

[0319] in:

[0320] This represents a disease feature vector with a dimension of 32.

[0321] This represents the weight matrix of the fully connected layer (32×64 dimensions).

[0322] This represents the bias vector of the fully connected layer (32 dimensions).

[0323] This represents the ReLU activation function, used to enhance the nonlinear expressive power of features.

[0324] S34.3 Calculate the disease expansion rate vector based on the differences in disease size parameters across multiple periods and the corresponding inspection time intervals. Combining disease feature vectors Generate a trend vector field containing the expansion rate and direction;

[0325] Specifically, the specific operations for generating the disease development trend vector field are as follows:

[0326] Calculate the disease propagation rate vector The expansion rate of each dimension is obtained by dividing the difference in disease size parameters between the first and last two periods and the intermediate key period by the corresponding inspection time interval. The formula is as follows:

[0327] ;

[0328] in:

[0329] , , These are the differences in length, width, and area between the first and last phases, respectively.

[0330] These are the inspection time intervals (unit: days) for the corresponding size parameters.

[0331] Determine the direction of disease expansion: By comparing the centroid coordinate offset direction of the three-dimensional contours of adjacent disease phases, and combining the geometric characteristics of the components, determine the expansion direction of each dimension (e.g., the positive X-axis direction along the length of the beam and the positive Y-axis direction perpendicular to the pier surface).

[0332] Generate a trend vector field: convert the disease propagation rate vector , Extension direction parameters and disease feature vectors By integrating the data, a trend vector field containing spatial distribution information is constructed. The vector value of each point in this field represents the rate and direction of disease expansion at the corresponding location.

[0333] S34.4 Integrate the three-dimensional contour and trend vector field of the disease into the core information of the disease, and output it to the precise positioning fusion unit 4, while combining it with the disease identification information in S31.4. The binding process forms a full lifecycle data chain consisting of "identity-scale invariant descriptor-3D coordinates-development trend".

[0334] Specifically, the steps for integrating and binding core disease information are as follows:

[0335] Core information integration: Integrating 3D contour data of the disease and scale-invariant descriptors Three-dimensional physical coordinates The trend vector field is integrated into the core information of the disease, and the data format is unified into JSON format;

[0336] Identity binding: Linking core disease information with the disease identity identifier generated in S31.4 Binding is performed to ensure that each piece of core information can be traced back to a unique disease;

[0337] Forming a full lifecycle data chain: Based on the binding results, constructing an "identity identifier". - Scale invariant descriptor - 3D coordinates - A full lifecycle data chain of "development trend vector field", with the data chain synchronously stored in a distributed encrypted database;

[0338] Data output: The integrated disease core information is output to the precise positioning fusion unit 4 according to the preset interface protocol, providing complete data support for the precise positioning of the disease area.

[0339] It is worth noting that, to overcome the limitations of traditional methods that can only identify the current state of disease but cannot predict its development trend, the disease evolution analysis module 34 achieves quantitative characterization and accurate prediction of disease evolution patterns. The disease evolution analysis module 34 integrates LSTM networks and attention mechanisms. Through the attention mechanism, it automatically learns the influence weights of data from each inspection period on disease evolution, prioritizing information from key periods to improve the accuracy of time-series pattern analysis. Simultaneously, it integrates the three-dimensional physical dimensions of the disease with environmental parameters (temperature, humidity, etc.) to construct a multi-dimensional input matrix. This matrix not only calculates the disease expansion rate vector but also combines disease feature vectors to generate a trend vector field containing the expansion rate and direction, achieving visualization and quantitative expression of disease development trends. By binding it with a unique disease identifier, a full lifecycle data chain of "identity-three-dimensional coordinates-development trend" is formed, providing direct data support for subsequent predictive maintenance decisions and adapting to the actual application scenarios of bridge maintenance.

[0340] Precision positioning fusion unit 4 receives the core disease information output by disease identification and life cycle tracking unit 3, combines it with UAV positioning data, integrates the high-precision bridge 3D model generated by 3D modeling unit 2 with UAV real-time positioning data, and outputs the 3D coordinates and position parameters of the disease area.

[0341] In this embodiment, the precise positioning fusion unit 4 includes a positioning data receiving module 41, a data fusion processing module 42, and a coordinate parameter output module 43, wherein:

[0342] The positioning data receiving module 41 receives the core information of the disease from the disease identification and life cycle tracking unit 3, the high-precision three-dimensional model of the bridge from the 3D modeling unit 2, and the real-time positioning data of the UAV, and performs unified format processing on the received data.

[0343] Specifically, the positioning data receiving module 41 receives the core disease information from the disease identification and life cycle tracking unit 3, the high-precision bridge 3D model from the 3D modeling unit 2, and the real-time positioning data from the UAV, and performs format unification processing on the received data. The specific operations are as follows:

[0344] First, clarify the scope of data reception, specifically including:

[0345] Receive core disease information output by the disease identification and lifecycle tracking unit 3, including disease identification identifiers. 3D contour data, scale-invariant descriptor Three-dimensional physical coordinates Trend vector field;

[0346] Receives the high-precision 3D bridge model (PLY format) and corresponding component topology coding system output by 3D modeling unit 2;

[0347] Receive real-time UAV positioning data, which comes from GPS / BeiDou positioning data (WGS-84 geodetic coordinates) and attitude data (pitch angle, roll angle, yaw angle) of UAV status acquisition module 12 in data acquisition unit 1. The acquisition frequency is consistent with that of data acquisition unit 1 (0.5 seconds / time or 1 second / time).

[0348] Then, the data format is standardized:

[0349] All received data is parsed to convert JSON data from the core disease information, PLY data from the 3D model, and coordinate data from UAV positioning into a standardized key-value pair data structure with fields including "data type, association ID, coordinate information, timestamp, and data source".

[0350] Perform data integrity verification by comparing the received data with the sent data based on the MD5 checksum, and remove invalid data with missing fields, incorrect format, or data corruption; perform time stamp synchronization calibration to ensure that the time base of all data is consistent (based on the UAV GPS timestamp).

[0351] Data fusion processing module 42 uses real-time UAV positioning data as a basis, combined with the spatial reference of high-precision bridge 3D model, and performs multi-source comparison and fusion with the 3D contour data in the core information of the defect to complete data calibration.

[0352] Specifically, the data fusion processing module 42 uses real-time UAV positioning data as a basis, combined with the spatial reference of the high-precision bridge 3D model, to perform multi-source comparison and fusion with the 3D contour data in the core information of the defects, and completes data calibration. The specific operation is as follows:

[0353] First, we need to unify the spatial reference:

[0354] Extract the WGS-84 geodetic coordinate system reference parameters from the high-precision bridge 3D model, and align the coordinate system of the UAV real-time positioning data with the coordinate system of the 3D model to ensure that the spatial references of the two are consistent.

[0355] Based on real-time attitude data of the UAV, the positioning deviation of the UAV (such as positioning drift caused by flight jitter) is corrected. The positioning data of 5 consecutive times are smoothed by a moving average algorithm to obtain the stable positioning coordinates of the UAV. .

[0356] Then, multi-source comparison and fusion are performed:

[0357] The first step involves geometrically comparing the 3D contour data from the core information of the bridge defect with a high-precision 3D bridge model. The defect is located in the corresponding area of ​​the model using the topological coding of the affected component, and the fitting error between the 3D contour of the defect and the surface of the model component is calculated. (The average distance from the contour points to the model surface);

[0358] The second step is to combine the stable positioning coordinates of the drone. Calculate the spatial relative distance error between the three-dimensional coordinates of the disease and the UAV positioning point. The core formula is:

[0359] ;

[0360] in:

[0361] The third dimension of the disease outline The coordinates of the points;

[0362] Indicates the number of points on the three-dimensional outline of the disease;

[0363] Finally, there's the data calibration optimization:

[0364] Set error threshold (Fitting error threshold) and (Relative distance error threshold), if or Based on the component geometric features of the 3D model and the stable positioning data of the UAV, the 3D coordinates of the defect are corrected using the following formula:

[0365] ;

[0366] in:

[0367] This indicates the coordinates of the calibrated disease outline points;

[0368] This represents the normal vector of the model component's surface at that point;

[0369] This represents the unit vector pointing from the disease point to the drone's location point.

[0370] This represents the calibration coefficient (determined based on statistical analysis of sample data, with a value range of 0.3 to 0.7).

[0371] After calibration, the error is recalculated until the error meets the threshold requirement, thus completing the multi-source data fusion calibration.

[0372] The coordinate parameter output module 43 maps the fused calibration results to the coordinate system of the bridge's three-dimensional model, generates the three-dimensional coordinates of the diseased area and the corresponding position parameters, and transmits them to the data storage and output unit 5.

[0373] Specifically, the coordinate parameter output module 43 maps the fused calibration results to the bridge's three-dimensional model coordinate system, generates the three-dimensional coordinates of the damaged area and its corresponding position parameters, and transmits them to the data storage and output unit 5. The specific operation is as follows:

[0374] Coordinate system mapping:

[0375] The fused and calibrated three-dimensional coordinates of the disease Based on the component topology encoding of the high-precision 3D bridge model, it is mapped to the local coordinate system inside the model (based on the component design reference point). (with the origin as the coordinate point), to obtain local coordinates The core formula is:

[0376] ;

[0377] Position parameter generation:

[0378] Generate the core 3D coordinates of the diseased area: Calculate the centroid coordinates of the 3D contour of the disease (calculated by equal weighting of the contour point set), and use them as the core positioning coordinates of the diseased area. The formula is:

[0379] ;

[0380] Supplementary location association parameters: including the topology code of the component where the defect is located. The relative distance between the core coordinates of the defect and the reference point of the component, and the hierarchical position of the defect in the model (such as the upper flange of the main beam, the middle of the pier) form a complete set of positional parameters.

[0381] Data transmission:

[0382] The three-dimensional coordinates of the disease core Location parameter set and disease identification The data is bound, encapsulated in a standardized JSON format, and transmitted to the data storage and output unit 5 via a high-speed network interface. During transmission, a data encryption protocol is enabled to ensure data security.

[0383] Data storage and output unit 5 receives data processed by data acquisition unit 1, 3D modeling unit 2, disease identification and life cycle tracking unit 3 and precise positioning fusion unit 4, stores it in a distributed encrypted database, and outputs predictive maintenance decision reports and data interfaces.

[0384] Specifically, the data storage operation is as follows: receiving full data from data acquisition unit 1, 3D modeling unit 2, disease identification and life cycle tracking unit 3, and precise positioning fusion unit 4, and classifying and archiving it according to the hierarchical logic of "data source-inspection period-bridge component-disease ID"; using a distributed storage cluster commonly used in this field to carry the data, coupled with AES encryption technology to ensure the security of core data, while establishing a global index with disease ID as the core, and enabling a regular backup mechanism to support multi-condition data retrieval.

[0385] Based on this, the output process of the predictive maintenance decision report is as follows: integrate the data of the entire life cycle of the disease, the evolution trend across periods and the precise location information, and assess the risk level of the disease in combination with the bridge maintenance industry standards; generate a report that includes basic bridge information, disease distribution overview, details of individual diseases and corresponding maintenance recommendations. The report includes 3D model disease annotation screenshots and visualization charts, and is finally output in commonly used formats such as PDF and Word.

[0386] Meanwhile, the standardized data interface is provided as follows: an open RESTful API interface, using HTTP / HTTPS protocol and JSON data format, covering functions such as data query, batch export, and real-time push, adapting to the access requirements of mainstream bridge maintenance management systems and GIS systems, and providing clear interface field description documents.

[0387] like Figure 3 As shown, this embodiment also provides a method for intelligent positioning of 3D bridge defect areas based on unmanned aerial vehicles (UAVs). Based on the above-mentioned intelligent positioning system for 3D bridge defect areas based on UAVs, the method includes the following steps:

[0388] S1. Multi-source data acquisition across periods: Using a drone equipped with a lidar and a high-definition camera, bridge point cloud data and surface image data are acquired across periods at preset time intervals. Attitude data of the drone during flight and environmental parameters of the surrounding environment of the bridge are acquired simultaneously. The bridge point cloud data, surface image data, drone attitude data and environmental parameters are then transmitted to S2 and S3.

[0389] S2, High-precision 3D modeling of bridge: Receive the collected raw data from multiple periods, use a filtering algorithm to remove noise from the bridge point cloud data and complete the cross-period registration through the iterative nearest point algorithm, extract feature points from the bridge surface image data through the scale-invariant feature transformation algorithm and complete the cross-period stitching, fuse the registered point cloud and the stitched image to generate a high-precision 3D model of the bridge and the corresponding camera pose parameters, and transmit them to S3.

[0390] S3. Intelligent Disease Identification and Full Lifecycle Tracking: A unique identifier is assigned to each newly identified disease, consisting of a bridge component topological code, a first detection timestamp, and a disease initial contour feature check code. A multi-scale space is generated based on a scale factor, extracting multi-scale contour curvature, surface texture, and 3D depth features of the disease and weighted fusion to generate a scale-invariant descriptor. This completes cross-period disease matching and binds it to the identifier. Combining the geometric features of the bridge's 3D model with camera pose parameters, the 2D pixel coordinates of the disease are corrected and converted into 3D physical coordinates. Multi-period disease parameters and environmental data are integrated, and the disease evolution trend is analyzed using an LSTM network and attention mechanism to form a full lifecycle data chain, which is then output to S4.

[0391] S4. Precise positioning and fusion of diseased areas: Receive core information of diseased areas, high-precision 3D bridge model and real-time positioning data of UAVs. After unifying the data format, based on the real-time positioning data of UAVs, and combined with the spatial reference of the 3D bridge model, complete the multi-source data comparison and calibration, map the fusion results to the 3D bridge coordinate system, generate the 3D coordinates of the diseased areas and corresponding position parameters, and transmit them to S5.

[0392] S5. Data Storage and Decision Output: Receives various types of data processed by S1-S4, stores them in a distributed encrypted database, generates predictive maintenance decision reports based on the stored data, and provides standardized data interfaces for subsequent applications.

[0393] Those skilled in the art will understand that the process of implementing all or part of the steps of the above embodiments can be carried out by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0394] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A 3D intelligent positioning system for bridge defect area inspection based on unmanned aerial vehicles (UAVs), characterized in that, include: The data acquisition unit (1) uses a drone equipped with a lidar and a high-definition camera to collect bridge point cloud data and surface image data across periods, and simultaneously acquires drone attitude data and environmental parameters, which are then transmitted to the 3D modeling unit (2) and the disease identification and life cycle tracking unit (3). The 3D modeling unit (2) receives the cross-period raw data from the data acquisition unit (1), and after fusion processing, generates a high-precision three-dimensional bridge model and camera pose parameters, which are then transmitted to the disease identification and life cycle tracking unit (3). The disease identification and life cycle tracking unit (3) assigns a unique and persistent disease identity based on the topological relationship of bridge components when identifying a disease for the first time; constructs a curvature pyramid and extracts the contour curvature features of each scale layer, generates a scale-invariant descriptor by combining surface texture features and three-dimensional spatial depth information, and performs cross-period disease feature matching based on the scale-invariant descriptor; performs spatial integration operation on the disease contour by using the distortion correction model constrained by the geometric features of the bridge structure according to the three-dimensional point cloud data and camera pose parameters transmitted by the 3D modeling unit (2), and converts the pixel coordinates in the two-dimensional image into three-dimensional spatial physical coordinates; and analyzes the disease evolution law through a time-series neural network based on the multi-period disease matching results and the environmental parameters collected by the data acquisition unit (1), generates a disease development trend representation containing the expansion rate vector and direction parameters, and outputs the core disease information to the precise positioning fusion unit (4). Precision positioning fusion unit (4) receives the core information of the disease output by the disease identification and life cycle tracking unit (3), combines the UAV positioning data, integrates the high-precision bridge three-dimensional model generated by the 3D modeling unit (2) with the UAV real-time positioning data, and outputs the three-dimensional coordinates and position parameters of the disease area. The data storage and output unit (5) receives the data processed by the data acquisition unit (1), the 3D modeling unit (2), the disease identification and life cycle tracking unit (3) and the precise positioning fusion unit (4), stores it through a distributed encrypted database, and outputs a predictive maintenance decision report and data interface.

2. The intelligent positioning system for 3D inspection of bridge defect areas based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The data acquisition unit (1) includes a bridge data acquisition module (11), a UAV status acquisition module (12), and an environmental parameter acquisition module (13), wherein: The bridge data acquisition module (11) is based on the lidar and high-definition camera carried by the UAV, and collects bridge point cloud data and surface image data across periods at preset time intervals. The UAV status acquisition module (12) synchronously acquires the attitude data of the UAV during flight, as auxiliary information for data calibration; The environmental parameter acquisition module (13) collects environmental data around the bridge; The output data of the bridge data acquisition module (11), the UAV status acquisition module (12) and the environmental parameter acquisition module (13) are synchronously transmitted to the 3D modeling unit (2) and the disease identification and life cycle tracking unit (3).

3. The intelligent positioning system for 3D inspection of bridge defect areas based on unmanned aerial vehicles (UAVs) according to claim 2, characterized in that, The 3D modeling unit (2) includes a point cloud preprocessing module (21), an image registration module (22), and a 3D fusion modeling module (23), wherein: The point cloud preprocessing module (21) receives bridge point cloud data from the data acquisition unit (1), uses a filtering algorithm to remove point cloud noise, and completes cross-period point cloud registration through the iterative nearest point algorithm. The image registration module (22) receives bridge surface image data from the data acquisition unit (1), extracts image feature points through the scale-invariant feature transformation algorithm, and completes the stitching of images across different periods. The three-dimensional fusion modeling module (23) fuses the registered point cloud of the point cloud preprocessing module (21) with the stitched image of the image registration module (22) to generate a high-precision three-dimensional bridge model and corresponding camera pose parameters, which are then synchronously transmitted to the disease identification and life cycle tracking unit (3).

4. The intelligent positioning system for 3D inspection of bridge defect areas based on unmanned aerial vehicles (UAVs) according to claim 3, characterized in that, The disease identification and life cycle tracking unit (3) includes a disease identification module (31), a feature description and matching module (32), a contour distortion correction module (33), and a disease evolution analysis module (34), wherein: When identifying a defect for the first time, the defect identification module (31) assigns a unique and persistent defect identification based on the topological relationship of the bridge components. The feature description and matching module (32) constructs a curvature pyramid and extracts the contour curvature features of each scale layer. It combines surface texture features and three-dimensional spatial depth information to generate a scale-invariant descriptor and completes the cross-period disease feature matching. The contour distortion correction module (33) performs distortion correction and three-dimensional coordinate transformation on the disease contour based on the three-dimensional point cloud data of the 3D modeling unit (2) and the camera pose parameters. The disease evolution analysis module (34) analyzes the disease evolution pattern and generates trend representation based on the multi-stage matching results and environmental parameters, and outputs the core disease information to the precise positioning fusion unit (4).

5. The intelligent positioning system for 3D inspection of bridge defect areas based on unmanned aerial vehicles (UAVs) according to claim 4, characterized in that, The process of generating disease identification identifiers by the disease identification module (31) includes the following steps: S31.1 Extract the bridge components corresponding to the first identified defects, and generate a 12-bit component topology code based on the component number on the bridge design drawings. The code is mapped one-to-one with the physical location of the component; S31.2 Record the time information of the first detection of the disease and generate an 8-digit first detection timestamp. ; S31.3 Perform an MD5 hash operation on the initial contour point set of the disease, and take the first 4 bits of the result as the contour feature check code. ; S31.4, will and By piecing together the data, we can obtain the disease identification identifier. This establishes a unique identity association for tracking diseases across different periods.

6. The intelligent positioning system for 3D inspection of bridge defect areas based on unmanned aerial vehicles (UAVs) according to claim 5, characterized in that, The feature description and matching module (32) generates scale-invariant descriptors and completes cross-period matching, including the following steps: S32.1, Based on Scale Factor A multi-scale space is generated, and Gaussian smoothing and Laplacian-Gaussian operation are performed sequentially on the disease contour point set to extract the contour curvature features of each scale layer. ; S32.2 Extract surface texture features and three-dimensional spatial depth features of the diseased area, and... After normalization, the components are weighted and fused to generate scale-invariant descriptors. The weights are determined by principal component analysis to assess their contribution to the variance of disease sample characteristics. S32.3 Calculate the scale invariance descriptor for diseases at different stages. similarity The system determines whether the diseases belong to the same disease based on a preset threshold, and matches the results with the disease identification identifier in S31.

4. Binding forms a feature association chain across different periods.

7. The intelligent positioning system for 3D inspection of bridge defect areas based on unmanned aerial vehicles (UAVs) according to claim 6, characterized in that, The contour distortion correction module (33) completes the process of disease contour distortion correction and three-dimensional coordinate transformation, including the following steps: S33.

1. Extract the inherent geometric features of the component where the defect is located from the high-precision three-dimensional bridge model of the 3D modeling unit (2), and determine the component design benchmark point as the reference point. ; S33.2, Call the camera pose parameters output by the 3D modeling unit (2), and the three-dimensional point cloud depth value corresponding to the disease area; S33.3, First perform distortion correction on the two-dimensional pixels of the defect outline, then combine... Camera parameters and point cloud depth values ​​are converted into three-dimensional physical coordinates. ; S33.

4. Perform interpolation fitting on the coordinate transformation results to obtain the three-dimensional contour of continuous diseases, providing three-dimensional data support for the extraction of three-dimensional spatial depth features, accurate calculation of contour curvature features, and matching of cross-period disease features in the process of generating scale-invariant descriptors.

8. The intelligent positioning system for 3D inspection of bridge defect areas based on unmanned aerial vehicles (UAVs) according to claim 7, characterized in that, The disease evolution analysis module (34) completes the disease evolution analysis and trend characterization process, which includes the following steps: S34.1 Integrate the three-dimensional physical size parameters of multiple disease stages and the environmental parameters of the data acquisition unit (1), and normalize them to form a network input matrix; S34.2 Input the input matrix into the LSTM network and calculate the weights of the data for each period through the attention mechanism. After weighting the hidden states of the network, they are mapped to disease feature vectors through a fully connected layer. ; S34.3 Calculate the disease expansion rate vector based on the differences in disease size parameters across multiple periods and the corresponding inspection time intervals. Combining disease feature vectors Generate a trend vector field containing the expansion rate and direction; S34.4 Integrate the three-dimensional contour and trend vector field of the disease into the core information of the disease, and output it to the precise positioning fusion unit (4), while combining it with the disease identification in S31.

4. The binding process forms a full lifecycle data chain consisting of "identity-scale invariant descriptor-3D coordinates-development trend".

9. The intelligent positioning system for 3D inspection of bridge defect areas based on unmanned aerial vehicles (UAVs) according to claim 8, characterized in that, The precise positioning fusion unit (4) includes a positioning data receiving module (41), a data fusion processing module (42), and a coordinate parameter output module (43), wherein: The positioning data receiving module (41) receives the core information of the disease from the disease identification and life cycle tracking unit (3), the high-precision three-dimensional model of the bridge from the 3D modeling unit (2), and the real-time positioning data of the UAV, and performs unified format processing on the received data. The data fusion processing module (42) uses the real-time positioning data of the UAV as a basis, combines the spatial reference of the high-precision bridge three-dimensional model, and performs multi-source comparison and fusion with the three-dimensional contour data in the core information of the defects to complete the data calibration. The coordinate parameter output module (43) maps the fusion calibration results to the bridge three-dimensional model coordinate system, generates the three-dimensional coordinates of the disease area and the corresponding position parameters, and transmits them to the data storage and output unit (5).

10. A method for intelligent positioning of bridge defect areas in 3D inspection based on unmanned aerial vehicles (UAVs), based on the intelligent positioning system for bridge defect areas in 3D inspection based on UAVs as described in any one of claims 1-9, characterized in that, Includes the following steps: S1. Multi-source data acquisition across periods: Using a drone equipped with a lidar and a high-definition camera, bridge point cloud data and surface image data are acquired across periods at preset time intervals. Attitude data of the drone during flight and environmental parameters of the surrounding environment of the bridge are acquired simultaneously. The bridge point cloud data, surface image data, drone attitude data and environmental parameters are then transmitted to S2 and S3. S2, High-precision 3D modeling of bridge: Receive the collected raw data from multiple periods, use a filtering algorithm to remove noise from the bridge point cloud data and complete the cross-period registration through the iterative nearest point algorithm, extract feature points from the bridge surface image data through the scale-invariant feature transformation algorithm and complete the cross-period stitching, fuse the registered point cloud and the stitched image to generate a high-precision 3D model of the bridge and the corresponding camera pose parameters, and transmit them to S3. S3. Intelligent identification and full life cycle tracking of defects: Assign a unique identifier to the first identified defect, consisting of a bridge component topology code, the first detection timestamp, and the initial contour feature check code of the defect; Multi-scale space is generated based on scale factors. Multi-scale contour curvature, surface texture and three-dimensional depth features of diseases are extracted and weighted and fused to generate scale-invariant descriptors. Cross-period disease matching is completed and bound to identity identifiers. Combining the geometric features of the bridge three-dimensional model and camera pose parameters, the two-dimensional pixel coordinates of diseases are corrected and converted into three-dimensional physical coordinates. Multi-period disease parameters and environmental data are integrated. The disease evolution trend is analyzed through LSTM network and attention mechanism to form a full life cycle data chain and output to S4. S4. Precise positioning and fusion of diseased areas: Receive core information of diseased areas, high-precision 3D bridge model and real-time positioning data of UAVs. After unifying the data format, based on the real-time positioning data of UAVs, and combined with the spatial reference of the 3D bridge model, complete the multi-source data comparison and calibration, map the fusion results to the 3D bridge coordinate system, generate the 3D coordinates of the diseased areas and corresponding position parameters, and transmit them to S5. S5. Data Storage and Decision Output: Receives various types of data processed by S1-S4, stores them in a distributed encrypted database, generates predictive maintenance decision reports based on the stored data, and provides standardized data interfaces for subsequent applications.

Images