A method and system for unmanned aerial vehicle (UAV) inspection of bridge force-measuring bearings

Abstract

This invention discloses a UAV inspection method and system for bridge bearings. The method includes: planning a strip-shaped flight path and constructing a coarse 3D point cloud model of the bridge; then planning a scanning flight path around the bridge's outer envelope to collect fine point clouds of bridge components; firstly, using a small amount of labeled data combined with pseudo-label generation and knowledge distillation to complete lightweight training of the PointNet model; then, strengthening structural association features through a contextual attention mechanism and generating virtual points using an occlusion region feature completion method to complete the occluded region and obtain a complete bearing point set; finally, extracting bearing geometric parameters by weighted noise point removal, iteratively fitting bounding boxes, and smoothing; planning a flight path based on the bearing's location to collect the bearing's surface image; then performing image cleaning and enhancement processing, and finally identifying surface defects using image recognition methods to generate an inspection report. Comprehensive technical support is provided.

Description

Technical Field

[0001] This invention belongs to the field of drone and bridge inspection technology, and more specifically, relates to a drone inspection method and system for bridge force-measuring supports. Background Technology

[0002] In bridge engineering, bearings, as key force-transmitting components connecting the superstructure and substructure, bear the important functions of transferring loads, adapting to deformation, and regulating bridge span displacement. Their working condition directly affects the overall stress performance and long-term service safety of the bridge. Force-measuring bearings can monitor the stress state of the bearings in real time, perceive the distribution of internal forces in the structure, and assist maintenance personnel in assessing the bridge's technical condition and formulating targeted repair and maintenance measures. However, due to displacement deviations during construction and installation, as well as frictional deflection during service, force-measuring bearings are prone to deviating from their design positions or experiencing localized voids, affecting their stress state and reducing their service life. Therefore, using drones for regular inspections and condition assessments of force-measuring bearings is an important technical means to ensure bridge structural safety and extend bridge service life.

[0003] However, drones face the following technical challenges when inspecting bridge supports: Supports are located at the junction of the beam bottom and the pier, in a concealed position with low accessibility and signal interference, making it difficult for drones to capture images and resulting in poor positioning and navigation capabilities; obstructions from the support steel plates and nameplates interfere with the identification of internal corrosion, cracks, and other surface damage, and there is a lack of methods for detecting and measuring minute displacements such as support gaps, deformation, and misalignment. To address these challenges, traditional drone bridge inspection methods are no longer sufficient to meet the demands of efficient, intelligent, and safe bridge maintenance. There is an urgent need to develop specialized bridge inspection drones integrating high-precision positioning and navigation technologies to achieve automatic identification and quantitative analysis of support conditions, thereby comprehensively improving the accuracy of bridge support inspection and maintenance efficiency, and promoting the transformation of infrastructure management towards informatization and digitalization. Summary of the Invention

[0004] To address the aforementioned technical challenges, this invention provides a safe, environmentally friendly, efficient, accurate, interference-resistant, economical, and reusable UAV inspection method and system for bridge force-measuring bearings. It solves the problems of traditional UAV bearing inspection technologies, such as reliance on pilot manual control, poor positioning and navigation capabilities, insufficient accuracy in surface damage identification, and weak ability to measure minute displacements.

[0005] To address the aforementioned deficiencies or improvement needs of existing technologies, as a first aspect of this invention, the present invention provides a method for unmanned aerial vehicle (UAV) inspection of bridge force-measuring bearings, comprising: S1. Plan the ribbon-shaped route and complete the rough model construction of the bridge's 3D point cloud; then plan the scanning route around the bridge's outer envelope surface, and use a lidar to collect the fine point cloud of bridge components, including supports and piers. S2. First, use a small amount of labeled data combined with pseudo-label generation and knowledge distillation to complete the lightweight training of the PointNet model; then, use the occlusion region feature completion method to generate virtual points to complete the occlusion region and obtain the complete support point set; finally, after removing noise points, extract the support geometric parameters by iteratively fitting the bounding box and smoothing the data. S3. Plan the flight path according to the location of the support, calculate the shooting distance based on the ground sampling distance, and collect the appearance image of the support by zooming or getting close. S4. After data acquisition, image cleaning and enhancement processing are performed; then, surface defects are identified and bearing edge lines are extracted using image recognition methods; simultaneously, bearing displacement is measured and bearing service status is determined; for bearings in service with defects, the bearing position is located in the bridge 3D model and an inspection report is generated.

[0006] Furthermore, in S1, the strip route task uses the center line of the bridge road as the baseline and extends symmetrically to both sides to form multiple parallel flight strips, ensuring that the main structure of the bridge is completely within the coverage area of ​​the flight strips; wherein, the collection range of adjacent flight strips is set with a lateral overlap rate, which is between 40% and 50%.

[0007] Furthermore, the process of setting up the scanning route task around the outer envelope of the bridge in S1 is as follows: Based on the acquired 3D point cloud rough model of the bridge, scanning routes were set for different parts of the bridge. The top route flew parallel to the bridge deck, scanning the top surface of the beam and the superstructure of the bridge; the side route flew parallel to the two sides of the bridge, scanning the sides of the beam and the piers; the bottom route passed through the area under the beam and used an inclined angle to scan the bottom of the beam and the support area. Based on the required point cloud density and lidar performance, set flight parameters such as relative distance, overlap rate, and flight speed; among them, set the relative distance between the flight path and the outer envelope surface to be between 10m and 20m, the overlap rate between the flight path and the heading to be no less than 60%, and the flight speed to be between 3m / s and 5m / s.

[0008] Furthermore, the lightweight training process of the PointNet model in S2 is specifically as follows: In the semi-supervised pre-training stage, the first step is to construct a labeled point cloud feature set. With unlabeled point cloud feature sets Define feature contrast loss To achieve structured learning of the feature space: , in, A single feature element in the feature set of a labeled point cloud represents the feature representation of a certain labeled point cloud; A single feature element in the unlabeled point cloud feature set represents the feature representation of a certain unlabeled point cloud; calculate and The cosine similarity between these two feature vectors is used to measure their similarity in the feature space. For indicator functions; when and If they belong to the same type of components, then ,otherwise By using the nonlinear mapping of feature cosine similarity, features of similar components are automatically aggregated and features of dissimilar components are naturally separated, providing a discriminative feature basis for subsequent identification. Based on the learned feature representations, predict the probability for unlabeled samples. Construct a dynamic threshold function to filter high-confidence samples: , in, For the model, the labeled point cloud features The predicted probability of belonging to the category; For labeling point cloud feature sets The average predicted probability of all elements in the set; The degree of dispersion and overall confidence level of the predicted probabilities for unlabeled data; when At that time, Included in the expanded training set threshold It is determined by the mean of the predicted probabilities of labeled data and the root mean square of the predicted probabilities of unlabeled data; Entering the self-distillation fine-tuning stage, using a mixed training set Given the input, the teacher model output probability distribution is as follows: The output of the lightweight student model is Define distillation loss: , By using the Euclidean distance between the probability distributions of the two models and combining the logarithmic ratio of the sum of probabilities, the student model can automatically learn the decision-making logic of the teacher model to achieve knowledge transfer.

[0009] Furthermore, the method for completing the occluded region features in S2 is specifically as follows: First, define the local density change rate: , in, It is one of the points in the cloud; For point The set of neighborhood points; For point Density function value; when hour, The global average number of neighborhood points is used to determine... The area in question is a candidate area for occlusion. Next, the complete region point set Extracting geometric features Construct the feature distribution function: , in, A set of points for the complete region; For point set The number of midpoints; Let be the geometric feature vector to be evaluated; For the Dirac delta function, when The function value is 1 when the time is right and 0 otherwise, used to capture points. The case where the geometric features are completely identical; For point Distance to the center of the region; For point The angle of inclination of the normal vector; For point Curvature at that point; for The average distance of features is used to capture the feature distribution patterns of the entire region. Finally, virtual points are generated in the occlusion candidate area. ,satisfy: , in, For virtual points ; geometric eigenvectors; For point The angle of inclination of the normal vector; To shield the area, To form a complete set of support points, the virtual points are matched with the real points in terms of feature distribution through integral conservation constraints.

[0010] Furthermore, the iterative fitting of the bounding box and the smoothing process in S2 specifically involve: For the support point set Calculate the range of each coordinate dimension: , , , in, For the midpoint of the point cloud of Coordinate values; For the midpoint of the point cloud of Coordinate values; For the midpoint of the point cloud of Coordinate values; For point clouds The range along the axial direction; For point clouds The range along the axial direction; For point clouds The range along the axial direction; Construct the initial bounding box using the extreme coordinates of the vertices. Next, define the distance function from the point to each face of the bounding box:

[0011] in, For a point in a point cloud, Let these be its coordinates; bounding box exist Two boundary values ​​along the axis; bounding box exist Two boundary values ​​along the axis; bounding box exist Two boundary values ​​along the axis; For point To the The bounding box of the next iteration The minimum distance; Calculate the median distance of all points in each iteration. Remove distance greater than For outliers, generate new bounding boxes based on the remaining point set. Repeat the iteration until the difference between adjacent bounding boxes is found. , The iteration count is determined by a dynamic convergence condition for automatic optimization. Finally, for the edges of the converged bounding box, the average coordinates of the two endpoints and three neighboring points of the edge are used as control points. Construct the parametric equations of the Bézier curve: , in, The parameter is Points on the Bézier curve; For parameter variables; These are the control points of the Bézier curve; these points determine the shape of the curve. ; By relying on the geometric features to naturally smooth the boundary, a regular boundary frame that fits the actual shape of the support is finally obtained.

[0012] Furthermore, the method for calculating the shooting distance in S3 is as follows: , In the formula, Ground sampling distance, The relative distance between the aircraft and the surface being photographed. The pixel size of the sensor can be determined based on the sensor size and the number of pixels in the sensor direction. This refers to the focal length of the camera lens.

[0013] Furthermore, the specific method for extracting the support edge line in S4 is as follows: For the grayscale matrix of the support image The gradient response of edge pixels is enhanced by calculating the local gradient magnitude using second-order difference: , in, The grayscale value is the pixel coordinate. Next, define with The neighborhood centered Calculate the gray-level dispersion of the region : , in, The neighborhood mean The number of neighboring pixels; when Select edge candidate point set in time Finally, for Calculate the Euclidean distance between each point in the middle. Difference with gradient direction The cost of building a connection : , in, These are two points in the set of edge candidate points; For point With point The difference in gradient direction between them; The cost of connecting using the minimum spanning tree method is less than , The average distance between the points forms a complete edge line that fits the actual contour of the support.

[0014] As a second aspect of the present invention, the present invention provides an unmanned aerial vehicle (UAV) inspection system for bridge force-measuring bearings, comprising: The bridge point cloud acquisition unit is used to plan the strip-shaped route task and complete the rough model construction of the bridge's 3D point cloud; then it plans the scanning route task around the outer envelope of the bridge and carries a lidar to acquire the fine point cloud of bridge components including supports and piers. The support point cloud processing and modeling unit is used to first complete the lightweight training of the PointNet model with a small amount of labeled data combined with pseudo-label generation and knowledge distillation; then, virtual points are generated by using the occlusion region feature completion method to complete the occlusion region to obtain the complete support point set; finally, after removing noise points, the support geometric parameters are extracted by iteratively fitting the bounding box and smoothing the data. The bearing appearance image acquisition unit is used to plan the flight path according to the location of the bearing, calculate the shooting distance based on the ground sampling distance, and acquire the bearing appearance image by zooming or getting close. The bearing inspection and evaluation report unit is used to perform image cleaning and enhancement processing after data acquisition; then, surface defects are identified and bearing edge lines are extracted using image recognition methods; at the same time, bearing displacement is measured and the service condition of the bearing is judged; for bearings in service with defects, the bearing position is located in the bridge 3D model and an inspection report is generated.

[0015] As a third aspect of the invention, the invention provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor of any step of the aforementioned unmanned aerial vehicle (UAV) inspection method for bridge force-measuring bearings.

[0016] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: 1. The UAV inspection method for bridge load-bearing bearings of this invention utilizes a layered route planning approach of "coarse scanning along a strip-shaped route + fine scanning along the outer envelope surface," combined with LiDAR, to complete the coarse construction of a 3D point cloud model of the bridge and the acquisition of fine point clouds of components. First, a strip-shaped route is used to quickly acquire overall spatial information of the bridge, laying the foundation for 3D modeling. Then, a scanning route is planned around the bridge's outer envelope surface to accurately acquire fine point clouds of key components such as bearings and piers, ensuring that the point cloud data covers both the overall bridge structure and clearly presents the detailed morphology of the bearings. This technology achieves efficient and comprehensive acquisition of point cloud data for bridge load-bearing bearings, providing high-precision 3D data support for subsequent extraction of bearing geometric parameters and state analysis, avoiding the problems of incomplete or insufficient data acquisition in traditional inspections.

[0017] 2. The UAV inspection method for bridge bearing stress measurement of this invention improves the utilization efficiency and analysis accuracy of bearing point cloud data through lightweight PointNet model training and point cloud optimization processing technology. First, lightweight model training is completed using a small amount of labeled data combined with pseudo-label generation and knowledge distillation, reducing computational costs while retaining core recognition capabilities. Then, a contextual attention mechanism is used to strengthen structural association features, and virtual points are generated to complete occluded areas, thus obtaining a complete bearing point set. Finally, through weighted noise point removal, iterative bounding box fitting, and smoothing, the bearing geometric parameters are accurately extracted. This technology solves the problems of occlusion, noise interference, and complex model calculations in point cloud data, achieving accurate extraction of bearing geometric parameters and providing reliable geometric data for assessing the bearing stress state.

[0018] 3. The UAV inspection method for bridge load-bearing bearings of this invention achieves accurate detection and evaluation of bearing service status through ground sampling distance (GSD)-based image acquisition and multi-dimensional data fusion analysis. The method plans the flight path based on the bearing location, calculates the shooting distance using the GSD calculation formula, and acquires surface images of the bearings that meet clarity requirements using zoom or close-up methods. After image cleaning and enhancement processing, surface defects are identified through image recognition, the bearing edge lines are extracted, and displacement is measured to determine the service status. Simultaneously, bearings with defects are located in the bridge's 3D model, and an inspection report is generated. This technology achieves comprehensive detection of bearing surface defects and displacement status, and combines 3D modeling to complete defect location and report generation, making inspection results more intuitive and providing comprehensive and accurate technical support for maintenance decisions regarding bridge load-bearing bearings. Attached Figure Description

[0019] Figure 1 This is a flowchart of a method for unmanned aerial vehicle (UAV) inspection of bridge force-measuring bearings according to an embodiment of the present invention. Figure 2 This is a schematic diagram of a three-dimensional bridge model according to an embodiment of the present invention; Figure 3 This is a diagram illustrating the effect of bridge bearing defect identification according to an embodiment of the present invention. Figure 4 This is a contour view of the bridge support edge according to an embodiment of the present invention; Figure 5 This is a system unit diagram of an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0021] Example 1 Please refer to Figure 1 Example 1 provides a method for unmanned aerial vehicle (UAV) inspection of bridge force-measuring bearings, including: S1. Plan the ribbon-shaped route and complete the rough model construction of the bridge's 3D point cloud; then plan the scanning route around the bridge's outer envelope surface, and use a lidar to collect the fine point cloud of bridge components, including supports and piers. S2. First, use a small amount of labeled data combined with pseudo-label generation and knowledge distillation to complete the lightweight training of the PointNet model; then, use the occlusion region feature completion method to generate virtual points to complete the occlusion region and obtain the complete support point set; finally, after removing noise points, extract the support geometric parameters by iteratively fitting the bounding box and smoothing the data. S3. Plan the flight path according to the location of the support, calculate the shooting distance based on the ground sampling distance, and collect the appearance image of the support by zooming or getting close. S4. After data acquisition, image cleaning and enhancement processing are performed; then, surface defects are identified and bearing edge lines are extracted using image recognition methods; simultaneously, bearing displacement is measured and bearing service status is determined; for bearings in service with defects, the bearing position is located in the bridge 3D model and an inspection report is generated.

[0022] This embodiment 1 further elaborates on the above steps.

[0023] (1) Bridge point cloud collection In this embodiment 1, the drone pilot first imports a satellite 2D map of the bridge area into the drone flight control software DJITerra, and manually draws a strip area covering the entire length and width of the bridge section.

[0024] Based on the required point cloud density and LiDAR performance, flight parameters such as flight altitude, flight strip overlap rate, and flight speed are set. Higher flight altitudes result in higher data acquisition efficiency and sparser point cloud density. A certain lateral overlap rate needs to be set for adjacent flight strip acquisition ranges to improve model integrity. Slower flight speeds result in lower data acquisition efficiency and denser point cloud density. In a preferred embodiment, the ribbon-like flight path task uses the bridge / road centerline as a baseline, symmetrically extending to both sides to form multiple parallel flight strips, ensuring that the main structure of the bridge is completely within the coverage area of ​​the flight strips. The lateral overlap rate for adjacent flight strip acquisition ranges is set between 40% and 50%. A flight altitude of 50m to 100m and a flight speed of 5m / s to 10m / s are recommended.

[0025] Using a drone platform equipped with a lidar sensor and a high-precision POS system, the drone flies automatically along a preset "strip" route, emits laser pulses onto the bridge surface, and receives the returned signals to obtain raw bridge point cloud data.

[0026] After the flight, the collected raw data is imported into professional processing software. The software uses direct georeferencing technology to fuse the relative distance of each laser point with the instantaneous absolute position and attitude of the UAV. The software algorithm automatically analyzes the geometry of the rough model and generates a simplified triangular mesh rough model that encloses the bridge, providing spatial reference and planning basis for subsequent fine scanning.

[0027] Please refer to Figure 2 In this embodiment 1, the rough model of the bridge's 3D point cloud obtained in the previous stage is imported into professional flight path planning software, and scanning flight paths are set for different parts of the bridge. Among them, the top flight path flies parallel to the bridge deck, scanning the top surface of the beam and the superstructure of the bridge; the side flight path flies parallel to the two sides of the bridge, scanning the sides of the beam and the piers; the bottom flight path passes through the area under the beam, using an inclined angle, scanning the bottom of the beam and the support area.

[0028] Based on the required point cloud density and lidar performance, set flight parameters such as relative distance, overlap rate, and flight speed. Specifically, set the relative distance between the flight path and the outer envelope surface to be between 10m and 20m, the overlap rate between the flight path and the heading to be no less than 60%, and the flight speed to be between 3m / s and 5m / s.

[0029] After the drone automatically executes a preset flight path to perform a detailed scan of the bridge surface, point cloud processing software is used to solve the data using direct georeferencing technology. First, point cloud registration and denoising are performed. Point cloud data collected from multiple flight paths and angles are automatically stitched together based on their absolute coordinates and overlapping areas to form a complete, seamless bridge point cloud model. Noise from moving objects such as birds and vehicles is also filtered out.

[0030] (2) Support point cloud processing and modeling In this embodiment 1, through data processing, the massive and disordered point cloud of the entire bridge is transformed into structured information with engineering significance that can be managed in parts, ultimately achieving geometric quantification of the bearing technical condition.

[0031] First, the PointNet algorithm is used to automatically identify and separate point sets belonging to different components such as "piers," "cap beams," and "supports" in the point cloud. The technical process of PointNet in processing bridge point cloud data is as follows: Input transformation and feature transformation: The network first learns a transformation matrix to align the point cloud (e.g., rotate the support point cloud to the standard orientation) to enhance its robustness to spatial rotation.

[0032] Shared Multilayer Perceptron (MLP): Features are extracted independently for each point, and local features of each point are calculated.

[0033] Symmetric max-pooling function: PointNet aggregates the features of all points into a single global feature vector through a max-pooling operation. This operation is symmetric; the output global features are the same regardless of the input point order.

[0034] Classification / Segmentation Output: PointNet's output vector includes a global feature vector and local feature vectors. The global feature vector is used to output the label for the entire point cloud. The global features are concatenated with the local features of each point, and then a semantic label is predicted for each point using an MLP (e.g., point A belongs to "support", point B belongs to "pier").

[0035] Before using the PointNet model, a large amount of labeled bridge point cloud data is required as a training set. Each point in the point cloud is manually labeled beforehand (indicating which component it belongs to), and then fed into the network for training, allowing it to learn the spatial shape and distribution characteristics of each component. When the trained model receives new bridge point clouds, it automatically labels each point (e.g., label:4, where 4 represents "support"). Finally, by classifying all points according to the labels, a clean set of support point clouds can be separated.

[0036] In a preferred embodiment, lightweight training was performed on the PointNet model, specifically as follows: In the semi-supervised pre-training stage, the first step is to construct a labeled point cloud feature set. With unlabeled point cloud feature sets Define feature contrast loss To achieve structured learning of the feature space: , in, A single feature element in the feature set of a labeled point cloud represents the feature representation of a certain labeled point cloud; A single feature element in the unlabeled point cloud feature set represents the feature representation of a certain unlabeled point cloud; calculate and The cosine similarity between these two feature vectors is used to measure their similarity in the feature space. For indicator functions; when and If they belong to the same type of components, then ,otherwise By using the nonlinear mapping of feature cosine similarity, features of similar components are automatically aggregated and features of dissimilar components are naturally separated, providing a discriminative feature basis for subsequent identification. Based on the learned feature representations, predict the probability for unlabeled samples. Construct a dynamic threshold function to filter high-confidence samples: , in, For the model, the labeled point cloud features The predicted probability of belonging to the category; For labeling point cloud feature sets The average predicted probability of all elements in the set; The degree of dispersion and overall confidence level of the predicted probabilities for unlabeled data; when At that time, Included in the expanded training set threshold It is determined by the mean of the predicted probabilities of labeled data and the root mean square of the predicted probabilities of unlabeled data; Entering the self-distillation fine-tuning stage, using a mixed training set Given the input, the teacher model output probability distribution is as follows: The output of the lightweight student model is Define distillation loss: , By using the Euclidean distance between the probability distributions of the two models and combining the logarithmic ratio of the sum of probabilities, the student model can automatically learn the decision-making logic of the teacher model to achieve knowledge transfer.

[0037] After the support point cloud is separated, the geometric contour, spatial position and attitude of the support need to be accurately calculated.

[0038] First, the directed bounding boxes of the supports are extracted using principal component analysis. The technical process is as follows: Calculate the center point and covariance matrix: Calculate the average three-dimensional coordinates of all points in the support point cloud to obtain the geometric center point O. Calculate the covariance matrix of all points relative to the center point O. The covariance matrix expresses the dispersion and correlation of the point cloud distribution in the X, Y, and Z directions.

[0039] Eigenvalue and eigenvector decomposition: Mathematically decompose the covariance matrix to obtain three eigenvalues. and the corresponding three feature vectors These three eigenvectors are orthogonal to each other, forming a completely new local coordinate system that best represents the geometric distribution of the point cloud itself. Among them, the eigenvector corresponding to the largest eigenvalue... Eigenvectors corresponding to the second largest eigenvalue , pointing to the extension directions of the point cloud, i.e., the length and width directions of the support. The eigenvector corresponding to the smallest eigenvalue. Pointing towards the thinnest point cloud, i.e., the direction of the normal to the support.

[0040] Constructing a directed bounding box: Projecting all points onto the bounding box. Under the newly constructed coordinate system, it is easy to find the maximum and minimum values ​​of the point cloud along the three principal axes ([min_x, max_x], [min_y, max_y], [min_z, max_z]). These six values ​​define a three-dimensional cuboid that tightly encloses the support point cloud, i.e., a directed bounding box, which represents the main contour of the support.

[0041] Top Boundary Fitting: The bottom surface of the bounding box should be a rectangle, but its edges may not be smooth enough due to point cloud noise. Using the least squares method, a plane fit is performed on the point cloud of the plane containing the bottom surface of the bounding box to obtain an accurate plane equation. The bottom surface point cloud is projected onto this fitted plane, and then the convex hull algorithm is used to calculate the accurate outer boundary polygon of the projected point set. For a rectangular support, this polygon is the four boundary lines of its top surface, and its corner coordinates can be accurately extracted.

[0042] In a preferred embodiment, feature completion is performed on the occluded area, specifically as follows: First, define the local density change rate: , in, It is one of the points in the cloud; For point The set of neighborhood points; For point Density function value; when hour, The global average number of neighborhood points is used to determine... The area in question is a candidate area for occlusion. Next, the complete region point set Extracting geometric features Construct the feature distribution function: , in, A set of points for the complete region; For point set The number of midpoints; Let be the geometric feature vector to be evaluated; For the Dirac delta function, when The function value is 1 when the time is right and 0 otherwise, used to capture points. The case where the geometric features are completely identical; For point Distance to the center of the region; For point The angle of inclination of the normal vector; For point Curvature at that point; for The average distance of features is used to capture the feature distribution patterns of the entire region. Finally, virtual points are generated in the occlusion candidate area. ,satisfy: , in, For virtual points ; geometric eigenvectors; For point The angle of inclination of the normal vector; To shield the area, To form a complete set of support points, the virtual points are matched with the real points in terms of feature distribution through integral conservation constraints.

[0043] Meanwhile, in a preferred embodiment, the bounding box is iteratively fitted and smoothed, specifically as follows: For the support point set Calculate the range of each coordinate dimension: , , , in, For the midpoint of the point cloud of Coordinate values; For the midpoint of the point cloud of Coordinate values; For the midpoint of the point cloud of Coordinate values; For point clouds The range along the axial direction; For point clouds The range along the axial direction; For point clouds The range along the axial direction; Construct the initial bounding box using the extreme coordinates of the vertices. Next, define the distance function from the point to each face of the bounding box: , in, For a point in a point cloud, Let these be its coordinates; bounding box exist Two boundary values ​​along the axis; bounding box exist Two boundary values ​​along the axis; bounding box exist Two boundary values ​​along the axis; For point To the The bounding box of the next iteration The minimum distance; Calculate the median distance of all points in each iteration. Remove distance greater than For outliers, generate new bounding boxes based on the remaining point set. Repeat the iteration until the difference between adjacent bounding boxes is found. , The iteration count is determined by a dynamic convergence condition for automatic optimization. Finally, for the edges of the converged bounding box, the average coordinates of the two endpoints and three neighboring points of the edge are used as control points. Construct the parametric equations of the Bézier curve: , in, The parameter is Points on the Bézier curve; For parameter variables; These are the control points of the Bézier curve; these points determine the shape of the curve. ; By relying on the geometric features to naturally smooth the boundary, a regular boundary frame that fits the actual shape of the support is finally obtained.

[0044] By following the above process, the absolute coordinates of the support's center point, as well as the support's length, width, height, inclination angle, and orientation angle, can be obtained. This information is crucial for confirming the accuracy of the support's construction and installation, and for assessing its service condition.

[0045] (3) Acquisition of apparent images of the support In this embodiment 1, the core element lies in the calculation of the relative shooting distance, which is determined based on the shooting requirements and the ground sampling distance (GSD). The ground sampling distance refers to the size of the actual object represented by one pixel in the image (unit: cm / pixel). Generally, the smaller the GSD, the higher the image resolution and the clearer the details. The calculation formula is as follows: , In the formula, H is the relative distance (in meters) between the aircraft and the imaging surface, s is the pixel size of the sensor, which can be determined according to the sensor size and the number of pixels in the sensor direction, and f is the focal length of the camera lens.

[0046] Based on the sharpness requirements of the testing specifications, determine the maximum required GSD. For example, to identify a 0.5mm crack on the support surface, a GSD of no more than 0.5mm / pixel is typically required. Substituting the target GSD, camera focal length, and sensor size into the formula, the maximum permissible shooting distance that the drone needs to maintain can be derived. Based on the field of view of a single camera and the calculated shooting distance, the shooting range that a single image can cover can be calculated.

[0047] During actual shooting, it is recommended to use a "bow" or "zigzag" shaped support for the drone. Starting from the starting point, the drone flies parallel to the width of the bridge lanes, completing image acquisition for the first strip-shaped area. Upon reaching the end of this strip, the drone does not return directly, but instead moves to the starting point of the adjacent second strip-shaped area via direct translation or a semi-circular curve. It then flies in the opposite direction to the first strip, repeating this process until all strip-shaped areas are covered. Maintain a constant speed and shooting interval throughout the flight to ensure uniform image quality and complete scanning of all designated areas.

[0048] (4) Bearing inspection and evaluation report In this embodiment 1, this step transforms the collected bridge bearing image data into a structured report that can directly guide maintenance decisions. This involves four steps: data cleaning and enhancement, automatic defect identification, geometric condition assessment, and diagnostic report generation.

[0049] 4.1 Enhanced Data Cleaning: When using drones for aerial photography of bridge supports, the images are prone to overexposure, underexposure, out-of-focus, and blurring due to variations in light levels under the bridge and the drone's own vibration. To ensure the accuracy of subsequent analysis, it is necessary to guarantee clear details and stable image quality. Therefore, an image quality assessment standard based on "information entropy—sharpness—grayscale" was established.

[0050] Information entropy: Measures the amount of information and texture richness contained in an image. It is calculated by taking the entropy value of the image's grayscale histogram. A low entropy value indicates that the image texture is simple, and details may be lost due to overexposure, underexposure, or blurring.

[0051] Sharpness: Typically assessed by calculating the gradient magnitude of an image. Low sharpness indicates a blurry image, which may be caused by drone shake, focus failure, or motion blur. Used to filter blurry images.

[0052] Average gray level: Calculates the average gray level of all pixels in an image. A high average gray level may indicate overexposure, with details appearing as "white"; a low average gray level may indicate underexposure, with details buried in darkness.

[0053] Threshold ranges were set for these three indicators, and an algorithm was written to perform batch calculations and automatic filtering on all acquired images. Overexposed or underexposed images were removed, and out-of-focus or blurry images were sent to the enhancement stage.

[0054] Image super-resolution reconstruction can improve image resolution and sharpness, restoring image details. This invention uses the Real-ESRGan algorithm, based on a generative adversarial network, to perform super-resolution reconstruction on blurred images obtained from UAV aerial photography. The Real-ESRGan algorithm is an adversarial network trained using purely synthetic data. Its principle is to simulate the real degradation process of high-resolution images decreasing in resolution, analyzing low-resolution images to infer high-resolution images. In this embodiment, several UAV aerial images of bridge supports were captured and collected. The Real-ESRGan algorithm was used to simulate the out-of-focus blurring characteristics of the images, training a restoration model suitable for blurred UAV aerial images. This model was then used to reconstruct the selected blurred aerial images, enriching image details and enhancing image sharpness.

[0055] 4.2 Automatic Defect Identification: Please refer to Figure 3 For aerial images of UAV support structures, the YOLOv8 target detection algorithm was used to identify support defects. YOLO is an advanced single-stage target detection algorithm. YOLOv8 further improves accuracy and speed on this basis. It meshes the image, and each grid directly predicts the bounding box and class probability, completing localization and recognition in one step with extremely high efficiency.

[0056] Aerial images of drone supports were collected and labeled to create a training sample dataset. Image enhancement methods (such as random rotation, cropping, adjusting brightness and contrast, and adding noise) were used to expand the limited original defect images, increasing the diversity and scale of the data, thereby improving the model's generalization ability and avoiding overfitting. The essence of model training is minimizing the loss function. YOLOv8's loss function includes classification loss and regression loss, representing the difference between the predicted and true categories and the overlap between the predicted and true bounding boxes, respectively. Through the backpropagation algorithm, the model continuously adjusts its internal parameters to reduce the total loss, meaning the prediction results become increasingly closer to the true annotations. After training, given a support image, the model outputs the support defect identification result in the form of a bounding box and a confidence score. In this embodiment, by appropriately setting the confidence threshold according to the type of support defect, automated and accurate identification of surface defects on the support can be achieved.

[0057] 4.3 Geometric State Assessment and Identification: For the corrected front-view image of the support, edge detection algorithms such as Canny are used to extract its contour edges. The Hough transform algorithm is then used to fit the four edge lines of the support: top, bottom, left, and right.

[0058] Please refer to Figure 4 The extracted edge lines are compared with a standard rectangle. The straightness of the edges and the perpendicularity of the corners are calculated, which can quantitatively assess the degree of deformation such as warping and skewing of the support.

[0059] Analyze the relationship between the bottom edge line of the support and the image of the pad stone surface. If an obvious gap appears between the support surface edge line and the adjacent layer, it can be preliminarily determined that a void exists. By combining the 3D point cloud data, distance can be measured to calculate the size of the void.

[0060] In a preferred embodiment, the specific method for extracting the support edge line is as follows: For the grayscale matrix of the support image The gradient response of edge pixels is enhanced by calculating the local gradient magnitude using second-order difference: , in, The grayscale value is the pixel coordinate. Next, define with The neighborhood centered Calculate the gray-level dispersion of the region : , in, The neighborhood mean The number of neighboring pixels; when Select edge candidate point set in time Finally, for Calculate the Euclidean distance between each point in the middle. Difference with gradient direction The cost of building a connection : , in, These are two points in the set of edge candidate points; For point With point The difference in gradient direction between them; The cost of connecting using the minimum spanning tree method is less than , The average distance between the points forms a complete edge line that fits the actual contour of the support.

[0061] 4.4 Diagnostic Report Generation: Based on standards such as the "Technical Condition Assessment Standard for Highway Bridges" (JTG / TH21-2011), a combination of qualitative assessment (e.g., slight corrosion) and quantitative measurement (e.g., crack length 50mm, width 0.3mm, gap 2mm) was used to classify and grade the bearings and defects. All data was compiled by the inspection personnel to generate a structured inspection report. The report includes: project overview, inspection methods, inspection basis, detailed defect record table for each bearing, pictures with defect annotations, summary of technical condition scores, and maintenance recommendations.

[0062] This invention proposes a UAV-based detection method for aerial inspection of bridge bearings. This method rapidly and efficiently collects bridge point cloud data, confirms the spatial location of the bearings, plans inspection routes, and acquires high-resolution images of the bearings. By combining various deep learning algorithms with image processing algorithms, the method ensures high-quality aerial images, automatically identifies bearing defects, measures minute displacements, and generates inspection reports. Furthermore, the deep learning model training method enables zero-shot learning for new datasets and task requirements, allowing for rapid training on large-scale, wide-ranging datasets and generalization to downstream tasks. This improves the model's generalization ability. Even in scenarios where the type of bearing defect is unclear or multiple defect types may be involved, the method can accurately detect objects containing at least one reasonable mask (defect). It can be applied to detect defects in bridge bearings of various configurations and spans without requiring the construction of multiple complex detection models.

[0063] Example 2 Please refer to Figure 5 This embodiment 2 provides a method for unmanned aerial vehicle (UAV) inspection of bridge force-measuring bearings, including: The bridge point cloud acquisition unit is used to plan the strip-shaped route task and complete the rough model construction of the bridge's 3D point cloud; then it plans the scanning route task around the outer envelope of the bridge and carries a lidar to acquire the fine point cloud of bridge components including supports and piers. The support point cloud processing and modeling unit is used to first complete the lightweight training of the PointNet model with a small amount of labeled data combined with pseudo-label generation and knowledge distillation; then, virtual points are generated by using the occlusion region feature completion method to complete the occlusion region to obtain the complete support point set; finally, after removing noise points, the support geometric parameters are extracted by iteratively fitting the bounding box and smoothing the data. The bearing appearance image acquisition unit is used to plan the flight path according to the location of the bearing, calculate the shooting distance based on the ground sampling distance, and acquire the bearing appearance image by zooming or getting close. The bearing inspection and evaluation report unit is used to perform image cleaning and enhancement processing after data acquisition; then, surface defects are identified and bearing edge lines are extracted using image recognition methods; at the same time, bearing displacement is measured and the service condition of the bearing is judged; for bearings in service with defects, the bearing position is located in the bridge 3D model and an inspection report is generated.

[0064] Example 3 This embodiment 3 also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement any step of a UAV inspection method for bridge force-measuring supports.

[0065] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0066] For a description of the computer-readable storage medium provided in this application, please refer to the above method embodiments; further details will not be repeated here.

[0067] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for unmanned aerial vehicle (UAV) inspection of bridge force-measuring bearings, characterized in that, include: S1. Plan the ribbon-shaped route and complete the rough model construction of the bridge's 3D point cloud; then plan the scanning route around the bridge's outer envelope surface, and use a lidar to collect the fine point cloud of bridge components, including supports and piers. S2. First, use a small amount of labeled data combined with pseudo-label generation and knowledge distillation to complete the lightweight training of the PointNet model; Then, virtual points are generated by using the occlusion region feature completion method to complete the occlusion region and obtain the complete set of support points; Finally, after removing noise points, the support geometric parameters are extracted by iteratively fitting the bounding box and smoothing the process. S3. Plan the flight path according to the location of the support, calculate the shooting distance based on the ground sampling distance, and collect the appearance image of the support by zooming or getting close. S4. After data acquisition, image cleaning and enhancement processing are performed; then, surface defects are identified and the support edge lines are extracted using image recognition methods. Simultaneously, the bearing displacement is measured and the bearing service condition is determined; for bearings in service with defects, the bearing position is located in the bridge 3D model and an inspection report is generated; The specific method for feature completion of the occluded region in S2 is as follows: First, define the local density change rate: ， in, It is one of the points in the cloud; For point The set of neighborhood points; For point Density function value; when hour, The global average number of neighborhood points is used to determine... The area in question is a candidate area for occlusion. Next, the complete region point set Extracting geometric features Construct the feature distribution function: ， in, A set of points for the entire region; For point set The number of midpoints; Let be the geometric feature vector to be evaluated; For the Dirac delta function, when The function value is 1 when the time is right and 0 otherwise, used to capture points. The case where the geometric features are completely identical; For point Distance to the center of the region; For point The angle of inclination of the normal vector; For point Curvature at that point; for The average distance of features is used to capture the feature distribution patterns of the entire region. Finally, virtual points are generated in the occlusion candidate area. ,satisfy: ， in, For virtual points ; geometric eigenvectors; For point The angle of inclination of the normal vector; To shield the area, To form a complete region corresponding to the space, integral conservation constraints are used to ensure that the feature distribution of virtual points is consistent with that of real points, thus forming a complete set of support points. The specific process of iterative fitting of the bounding box and smoothing in S2 is as follows: For the support point set Calculate the range of each coordinate dimension: ， ， ， in, For the midpoint of the point cloud of Coordinate values; For the midpoint of the point cloud of Coordinate values; For the midpoint of the point cloud of Coordinate values; For point clouds The range along the axial direction; For point clouds The range along the axial direction; For point clouds The range along the axial direction; Construct the initial bounding box using the extreme coordinates of the vertices. Next, define the distance function from the point to each face of the bounding box: ， in, For a point in a point cloud, Let these be its coordinates; bounding box exist Two boundary values ​​along the axis; bounding box exist Two boundary values ​​along the axis; bounding box exist Two boundary values ​​along the axis; For point To the The bounding box of the next iteration The minimum distance; Calculate the median distance of all points in each iteration. Remove distance greater than For outliers, generate new bounding boxes based on the remaining point set. Repeat the iteration until the difference between adjacent bounding boxes is found. , The iteration count is determined by a dynamic convergence condition for automatic optimization. Finally, for the edges of the converged bounding box, the average coordinates of the two endpoints and three neighboring points of the edge are used as control points. Construct the parametric equations of the Bézier curve: ， in, The parameter is Points on the Bézier curve; For parameter variables; These are the control points of the Bézier curve; these points determine the shape of the curve. ; By relying on the geometric features to naturally smooth the boundary, a regular boundary frame that fits the actual shape of the support is finally obtained.

2. The method for unmanned aerial vehicle (UAV) inspection of bridge force-measuring bearings according to claim 1, characterized in that, The S1 route task uses the centerline of the bridge and road as the baseline and extends symmetrically to both sides to form multiple parallel routes, ensuring that the main structure of the bridge is completely covered by the routes; the lateral overlap rate is set for the collection range of adjacent routes, and the overlap rate is between 40% and 50%.

3. The method for unmanned aerial vehicle (UAV) inspection of bridge force-measuring bearings according to claim 1, characterized in that, The process of setting up the scanning route task around the outer envelope of the bridge in S1 is as follows: Based on the acquired 3D point cloud rough model of the bridge, scanning routes were set for different parts of the bridge. The top route flew parallel to the bridge deck, scanning the top surface of the beam and the superstructure of the bridge; the side route flew parallel to the two sides of the bridge, scanning the sides of the beam and the piers; the bottom route passed through the area under the beam and used an inclined angle to scan the bottom of the beam and the support area. Based on the required point cloud density and lidar performance, set the relative distance, overlap rate, and flight speed parameters; among them, set the relative distance between the flight path and the outer envelope surface to be between 10m and 20m, the overlap rate between the flight path and the heading to be no less than 60%, and the flight speed to be between 3m / s and 5m / s.

4. The UAV inspection method for bridge force-measuring bearings according to claim 1, characterized in that, The lightweight training process of the PointNet model in S2 is as follows: In the semi-supervised pre-training stage, the first step is to construct a labeled point cloud feature set. With unlabeled point cloud feature sets Define feature contrast loss To achieve structured learning of the feature space: ， in, A single feature element in the feature set of a labeled point cloud represents the feature representation of a certain labeled point cloud; A single feature element in the unlabeled point cloud feature set represents the feature representation of a certain unlabeled point cloud; calculate and The cosine similarity between these two feature vectors is used to measure their similarity in the feature space. For indicator functions; when and If they belong to the same type of components, then ,otherwise By using the nonlinear mapping of feature cosine similarity, features of similar components are automatically aggregated and features of dissimilar components are naturally separated, providing a discriminative feature basis for subsequent identification. Based on the learned feature representations, predict the probability for unlabeled samples. Construct a dynamic threshold function to filter high-confidence samples: ， in, For the model, the labeled point cloud features The predicted probability of belonging to the category; For labeling point cloud feature sets The average predicted probability of all elements in the set; The degree of dispersion and overall confidence level of the predicted probabilities for unlabeled data; when At that time, Included in the expanded training set threshold It is determined by the mean of the predicted probabilities of labeled data and the root mean square of the predicted probabilities of unlabeled data; Entering the self-distillation fine-tuning stage, using a mixed training set Given the input, the teacher model output probability distribution is as follows: The output of the lightweight student model is Define distillation loss: ， By using the Euclidean distance between the probability distributions of the two models and combining the logarithmic ratio of the sum of probabilities, the student model can automatically learn the decision-making logic of the teacher model to achieve knowledge transfer.

5. The method for unmanned aerial vehicle (UAV) inspection of bridge force-measuring bearings according to claim 1, characterized in that, The method for calculating the shooting distance in S3 is as follows: ， In the formula, Ground sampling distance, The relative distance between the aircraft and the surface being photographed. The pixel size of the sensor is determined based on the sensor size and the number of pixels in the sensor direction. This refers to the focal length of the camera lens.

6. The method for unmanned aerial vehicle (UAV) inspection of bridge force-measuring bearings according to claim 1, characterized in that, The specific method for extracting the support edge line in S4 is as follows: For the grayscale matrix of the support image The gradient response of edge pixels is enhanced by calculating the local gradient magnitude using second-order difference: ， in, The grayscale value is the pixel coordinate. Next, define with The neighborhood centered Calculate the gray-level dispersion of the region : ， in, The neighborhood mean The number of neighboring pixels; when Select edge candidate point set in time Finally, for Calculate the Euclidean distance between each point in the middle. Difference with gradient direction The cost of building a connection : ， in, These are two points in the set of edge candidate points; For point With point The difference in gradient direction between them; The cost of connecting using the minimum spanning tree method is less than , The average distance between the points forms a complete edge line that fits the actual contour of the support.

7. A drone inspection system for bridge force-measuring bearings, used to implement the drone inspection method for bridge force-measuring bearings as described in claim 1, characterized in that, include: The bridge point cloud acquisition unit is used to plan the strip-shaped route task and complete the rough model construction of the bridge's 3D point cloud; then it plans the scanning route task around the outer envelope of the bridge and carries a lidar to acquire the fine point cloud of bridge components including supports and piers. Support point cloud processing modeling unit, used to first complete lightweight training of PointNet model with a small amount of labeled data combined with pseudo-label generation and knowledge distillation; Then, virtual points are generated by using the occlusion region feature completion method to complete the occlusion region and obtain the complete set of support points; Finally, after removing noise points, the support geometric parameters are extracted by iteratively fitting the bounding box and smoothing the process. The bearing appearance image acquisition unit is used to plan the flight path according to the location of the bearing, calculate the shooting distance based on the ground sampling distance, and acquire the bearing appearance image by zooming or getting close. The bearing inspection and evaluation report unit is used to perform image cleaning and enhancement processing after data acquisition; then, surface defects are identified and bearing edge lines are extracted using image recognition methods. At the same time, the bearing displacement is measured and the service condition of the bearing is determined; For bearings that are in service with defects, locate the bearing position in the bridge's 3D model and generate an inspection report.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program is executed by a processor according to any one of claims 1-6, which describes a method for unmanned aerial vehicle (UAV) inspection of bridge force-measuring bearings.

Images