Bridge member identification method based on unmanned aerial vehicle point cloud reconstruction and three-dimensional synthetic data

Abstract

A bridge component identification method based on unmanned aerial vehicle point cloud reconstruction and three-dimensional synthetic data is characterized in that a point cloud is generated through video acquisition of an unmanned aerial vehicle, accurate identification of bridge components is realized by combining with an improved PointNet + + model, and a technical process of data acquisition-point cloud generation-model training-component identification is constructed. The method comprises the following specific implementation steps: (1) acquiring a bridge member field video by using an unmanned aerial vehicle; (2) building a three-dimensional reconstruction framework to generate dense point cloud of bridge components; and (3) generating diversified bridge point cloud synthetic data in batches. (4) constructing a point cloud semantic segmentation data set with labels, and preprocessing and enhancing the point cloud semantic segmentation data set for model training; (5) realizing point cloud identification based on an improved PointNet + + model; and (6) post-processing an identification result, and extracting geometric parameters of the component for bridge structure state evaluation. According to the invention, efficient generation and high-precision identification of the point cloud of the bridge member can be realized, and reliable data support is provided for quality evaluation of the bridge structure.

Description

TECHNICAL FIELD

[0001] The application relates to a bridge structure quality evaluation method, in particular to a method for reconstructing and identifying three-dimensional point clouds of bridge components based on unmanned aerial vehicle (UAV) collection and computer vision technology, and belongs to the field of structural engineering. BACKGROUND

[0002] As an important part of the modern transportation network, bridges play an irreplaceable role in ensuring the integrity and convenience of the transportation network. As of 2023, the number of in-service highway bridges in China has reached 1.07 million, and their safe operation is directly related to social public safety. However, as the service life of bridges increases, natural aging, cracking and damage inevitably occur, leading to a decrease in the structural damage and carrying capacity of bridges. In response to the problem of bridge structure aging, structural health detection has become an effective means to ensure the safe service of bridges.

[0003] Traditional detection methods such as manual visual inspection and telescope detection have low efficiency and poor accuracy, which cannot meet the actual application requirements. In recent years, with the rise of machine vision technology, two-dimensional close-up image analysis has been widely used in bridge structure detection. However, due to the lack of three-dimensional spatial context information, this method cannot be associated with specific structural components, resulting in inaccurate damage positioning and hindering the evaluation of the overall health status of bridges. Three-dimensional point cloud technology can clearly present the three-dimensional spatial information, surface condition and structural parameters of components, providing a reliable basis for accurate classification and feature extraction of components. However, the current point cloud dataset for bridges has the problems of small quantity, small coverage of bridge types, high data acquisition cost and insufficient annotation accuracy, which leads to low precision of the trained point cloud semantic recognition model. Therefore, it is urgent to develop a high-precision and low-cost point cloud semantic recognition method to realize the spatial segmentation and category determination of bridge components and achieve feature extraction of components, thereby providing technical support for long-term maintenance and operation safety of bridges. SUMMARY

[0004] In view of the deficiencies of the prior art, the application provides a bridge component three-dimensional point cloud generation and recognition method based on an unmanned aerial vehicle (UAV) to improve the feasibility of bridge structure detection based on point cloud technology in actual application. The specific content includes:

[0005] A bridge component recognition method based on UAV point cloud reconstruction and three-dimensional synthetic data, characterized in that it comprises the following steps:

[0006] A. collecting videos of on-site bridge components using a UAV;

[0007] B. building a bridge component three-dimensional reconstruction method framework, and reconstructing a bridge component three-dimensional point cloud model using the bridge component videos;

[0008] B1. Extract the key frames of the bridge component video based on FFmpeg software;

[0009] B2. Process the key frame images based on COLMAP software, and output the three-dimensional point cloud of the bridge component;

[0010] C. Bridge large-scale three-dimensional point cloud synthesis data generation;

[0011] C1. Set the bridge component parameters (such as piers, beams, etc.), perform parameter random disturbance, generate diversified models, and convert them into mesh models. Uniform sampling method is used to generate triangular face sampling points; Finally, add noise simulation (Gaussian noise, block occlusion, missing, etc.) to enhance data diversity;

[0012] C2. Simulate laser scanning of the triangulated model by unmanned aerial vehicle, calculate the flight time (ToF) to restore the point cloud space position by planning the virtual flight route, simulating the laser emission angle, position, and receiving time, and generate the bridge three-dimensional point cloud;

[0013] C3. Repeat the above operation to obtain a large amount of diversified bridge point cloud synthesis data;

[0014] D. Construct a bridge component point cloud semantic recognition dataset;

[0015] D1. Point cloud data preprocessing: perform three-dimensional normalization processing on the bridge point cloud synthesis data, calculate the point cloud centroid to realize centering, and then scale the point cloud as a whole to the unit sphere according to the maximum radius to ensure the geometric consistency of the data; Construct a nine-dimensional input feature vector, including normalized XYZ coordinates (core geometric features), normalized RGB channel values (color features), and normal vector features (key spatial features), and reorganize the feature matrix into the required "9xN" format (N is the number of points);

[0016] D2. Data annotation and division: For the main components of the bridge, use professional three-dimensional labeling tools for manual fine labeling to clearly determine their class attribution; For small sample components with few quantities, first use their inherent shape features for algorithmic automatic pre-labeling, and then strictly check and correct by manual, to maximize the accuracy of the labeling results; Divide the point cloud dataset into training set and test set according to the preset proportion, and abide by the basic principle that the point number proportion of each component class in the training set and test set remains consistent, wherein the training set accounts for 80% of the total data, and the test set accounts for 20% of the total data;

[0017] D3. Data augmentation: Various data augmentation operations are performed on the training set to expand the sample diversity. The basic augmentation methods include main rotation (to simulate different observation angles), XY plane rotation (to simulate horizontal direction changes), coordinate jitter (to introduce slight position perturbations), random rotation (all-around view angle changes), scale scaling (to simulate distance changes), and local translation (to simulate local position shifts). For small sample component classes with few points, additional and more intensive targeted augmentations are implemented to significantly increase the diversity of their features.

[0018] E. Building an improved PointNet++ bridge component point cloud recognition model;

[0019] E1. Network structure design and optimization: Based on the PointNet++ architecture, the classification, segmentation, and utils modules are integrated to build the model. The multi-dimensional feature fusion strengthens the geometric relationship learning ability and improves the adaptability of the model to bridge scenes.

[0020] E2. Training strategy and execution process: During training, a balanced sampler is used to dynamically adjust sample weights, and an enhanced weighted loss function is used to apply additional penalties to small sample error predictions and low IoU classes. The initial learning rate is set, and a linear warm-up combined with cosine annealing scheduling is used to ensure training stability with gradient clipping strategy. The test set mIoU is used as the core indicator, and when there is no improvement for several consecutive rounds, the early stopping mechanism is triggered, and the final output of the optimal model's class prediction results is completed. Bridge component point cloud recognition;

[0021] F. Post-processing and application of recognition results: Calculate geometric parameters to provide data support for bridge structure health state evaluation.

[0022] Further, the bridge component video acquisition method in step A is: using a drone device to surround the target component 360°, collecting the target component video, the video acquisition requirements include the information of the component from top to bottom at different heights, and finally collecting the bridge component video.

[0023] Further, in step B1, the bridge component video key frame image is extracted using FFmpeg, and by selecting 0.25 as the time scaling coefficient to ensure that the output key frame contains as much feature information of the bridge component as possible.

[0024] Further, in step B2, the output of the bridge component three-dimensional point cloud is realized by COLMAP software: COLMAP software realizes image feature extraction through SIFT algorithm, realizes feature matching by KNN, LoweRatio, RANSAC algorithm, and combines SfM algorithm to realize sparse point cloud reconstruction of the target component, and finally realizes dense point cloud reconstruction through MVS algorithm.

[0025] Further, the parameter random disturbance in step C1 generates diversified models: by extracting the basic geometric parameters of the bridge components (such as the diameter / height of the pier, the cross-sectional size of the box girder), and applying random disturbance to the parameters to realize model diversification; by controlling the disturbance intensity (size ± 5% tolerance, position offset <10 cm, angle deflection <3°), the morphological variation degree of the generated model can be adjusted; by configuring a reasonable disturbance threshold range, the generated component model can cover the common deviation form in actual engineering while maintaining structural rationality, so as to output training level point cloud data with strong generalization ability.

[0026] Further, the data enhancement operation in step D3 includes basic enhancement and small sample targeted enhancement; wherein the basic enhancement is performed for all samples, including rotation, scale, Gaussian noise, etc.; and the small sample exclusive enhancement doubles the enhancement intensity for small sample categories (such as cables, anchors), including:

[0027] Δ ~ U (-0.05λ, 0.05λ) (2)

[0028] Wherein, xyz' is the 3D coordinate after enhancement, xyz i is the original 3D coordinate, m i is the local deformation mask (randomly select 40% of the points), s is the principal axis direction scaling factor (0.8-1.2, calculated by principal component analysis PCA), Δ is the translation amount, and the enhancement small sample form diversity.

[0029] Further, the multi-dimensional feature fusion mechanism in step E1 includes local coordinate normalization, RGB feature standardization and coordinate gradient feature extraction, wherein the local coordinate normalization enhances the local geometric feature expression by decentralization and scale reduction, which can be expressed as:

[0030]

[0031] Wherein, xyz i is the original 3D coordinate, xyz center is the point cloud center point coordinate, max dist is the distance from the farthest point in the point cloud to the center point (used for scale normalization), xyz local is the normalized local coordinate (mapped into a unit sphere); RGB feature standardization can normalize the color feature to the interval of 0-1, and process the influence of light change, which can be expressed as:

[0032]

[0033] Wherein, rgb iFor the original RGB value, rgb min , rgb max are the minimum and maximum values of the RGB channel in the point cloud, respectively, rgb norm is the normalized RGB feature; the coordinate gradient feature extraction calculates the local continuity feature of the point cloud through neighborhood difference, simulates the normal vector information, and can be expressed as follows:

[0034]

[0035] Where grad i is the coordinate gradient of point i, N(i) is the neighborhood point set of point i (obtained through the K-neighbor algorithm, K = 16), K is the number of neighborhood points, and is used to balance the gradient amplitude.

[0036] Further, the balanced sampler (BalancedSampler) in the step E2 solves the class imbalance problem through a process of first determining small sample classes, then dynamically adjusting weights, and finally stratified sampling, wherein the small sample class determination dynamically determines a threshold value based on the class distribution of the training set, and the formula is:

[0037] T = max (1000, min (median (C valid ), 1.5 * median (C valid ))) (7)

[0038] Where C valid ={c k |c k > 0} is the point set of valid classes (sample number > 0), median () is the median function, and T is the small sample threshold value (ensuring no less than 1000 points, adapting to small components such as cables and connectors); the dynamic weight calculation assigns a sampling weight according to the number of dominant classes of the sample, and the weight is higher when the sample number is smaller, and the formula is:

[0039]

[0040] Where w i is the sampling weight of the i-th sample, c dom(i) is the total number of dominant classes of the sample (the dominant class is defined as the class with the most occurrences in the sample), and the weight array is normalized to ensure that the probability sum is 1; the stratified sampling strategy allocates sampling quotas according to classes for multi-class point clouds in each sample, the maximum proportion of the dominant class (non-small sample) is ≤30%, and the minimum proportion of the small sample class is ≥20%, the small sample class uses sampling with replacement, ensuring that each small sample class contains at least 200 points in the training batch; the insufficient number is supplemented by repeated sampling, but the repeated number is limited to ≤3 times (to avoid feature redundancy).

[0041] Further, the enhanced version of the loss in step E2 is constructed by fusing cross-entropy loss, IoU loss and dynamic weight mechanism to build a total loss function, wherein the cross-entropy loss improves the prediction accuracy of small classes by introducing a small sample error penalty, and the formula is:

[0042]

[0043] wherein, is the predicted probability, y i is the true label, s(y i ) is a small sample mask (y i 1 when the small sample class is taken, otherwise 0); the IoU loss can dynamically adjust the class weight, focus on low IoU class optimization, and the formula is:

[0044]

[0045] wherein, C present is the set of classes appearing in the current batch, IoU k is the intersection over union of the kth class, w k = 2.0 (small sample class) or 1.0 (non-small sample class), and when IoU k <0.3 w k is additionally multiplied by 1.5; the total loss function calculation formula is:

[0046] L total = alpha * L CE + (1-alpha) * L IoU (11)

[0047] wherein, alpha is the cross-entropy loss weight, and 1-alpha is the IoU loss weight.

[0048] The advantages of the present application are:

[0049] (1) Compared with artificially shooting multiple view images of bridge components, using FFmpeg software to extract key frames of bridge component videos can provide more feature information of bridge components, and realize high-precision three-dimensional point cloud reconstruction of bridge components.

[0050] (2) A large number of synthetic point clouds are generated by parameter random disturbance, noise simulation and virtual laser scanning, which greatly improves the diversity of training data and alleviates the problem of insufficient point cloud training data, so that the model generalization ability is significantly better than the model trained only with real shot data.

[0051] (3) Compared with the data set construction mode which simply depends on manual annotation, the mode of "manual fine annotation of main components-algorithm pre-annotation-manual verification of small sample components" is adopted, so as to reduce the annotation cost of small sample components while ensuring the annotation accuracy, and solve the problems of low annotation efficiency and difficult small sample annotation of the traditional annotation.

[0052] (4) Compared with the ordinary point cloud recognition model, the improved PointNet++ model strengthens geometric learning through multi-dimensional feature fusion, and the balance sampler and enhanced weighted loss are combined, so that the small sample component recognition accuracy is much higher than that of the traditional model, and the overall mIoU index is better. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 is a flowchart of the method of the present application;

[0054] Figure 2 is a sample drawing of the digital model of the highway bridge in the embodiment of the present application;

[0055] Figure 3 is the key frame of the target component extracted by FFmpeg in the present application;

[0056] Figure 4 is the reconstructed bridge component three-dimensional point cloud diagram of the present application;

[0057] Figures 5(a)-5(d) is a schematic diagram of the synthesized bridge three-dimensional point cloud data of the present application, wherein Fig. 5(a) is a schematic diagram of a bridge deck point cloud synthesis model, Fig. 5(b) is a schematic diagram of a bridge pier point cloud synthesis model, Fig. 5(c) is a schematic diagram of a bridge pier combination point cloud synthesis model, and Fig. 5(d) is a schematic diagram of an entire bridge point cloud synthesis model. DETAILED DESCRIPTION

[0058] The present application will be further described in detail below with reference to the accompanying drawings.

[0059] The implementation process of the bridge component recognition method based on the point cloud reconstruction and three-dimensional synthesis data of the unmanned aerial vehicle of the present application is shown in Figure 1 , and specifically includes the following steps:

[0060] A. Select a digital model of a certain highway bridge as an example object, and the model structure is shown in Figure 2 , and the bridge pier is photographed in 360° full omnidirectional surround, the collection process covers the information of different heights of the bridge pier from top to bottom, and the target component time length 6s of the field video is obtained.

[0061] B. Build a bridge component three-dimensional reconstruction framework:

[0062] B1. Extract key frames from the collected video based on FFmpeg software, select 0.25 as the time scaling coefficient, and finally obtain 93 key frame images of the target components, part of the key frame images are as shown in Figure 3

[0063] B2. Import the key frame images into COLMAP software, extract image features through SIFT algorithm, use KNN, LoweRatio and RANSAC algorithms for feature matching, combine SfM algorithm to reconstruct sparse point cloud, and then generate dense three-dimensional point cloud through MVS algorithm; the reconstructed point cloud is as shown in Figure 4

[0064] C. Generate bridge large-scale three-dimensional point cloud synthesis data:

[0065] C1. Set the parameters of bridge piers, beams and other bridge components, randomly perturb the parameters (size ±5% tolerance, position offset <10 cm, angle deflection <3°), generate diversified models and convert them into mesh models, use uniform sampling method to generate triangular face sampling points, and finally add Gaussian noise, block occlusion and other enhanced data diversity;

[0066] C2. Simulate the laser scanning of the triangulated model by the unmanned aerial vehicle, plan the virtual flight route, simulate the laser emission angle and position, calculate the flight time (ToF) to restore the point cloud space position, and generate the bridge three-dimensional point cloud, part of the point cloud data is as shown in FIG. 5;

[0067] C3. Repeat the above operation, after obtaining a large amount of diversified bridge point cloud synthesis data, perform data set labeling and division. Adopt the "manual leading + automatic auxiliary" mode for labeling: manually label the main components by using professional three-dimensional labeling tools, and first automatically pre-label the small sample components according to the shape features and then manually check them;

[0068] C4. The final data includes 845 npy files, which are divided into training set and test set according to the ratio of 8:2;

[0069] D. Improved PointNet++ model building

[0070] D1. Point cloud data preprocessing and feature construction. Perform three-dimensional normalization processing on the synthesized point cloud data: calculate the centroid to realize centering, and scale to the unit sphere according to the maximum radius; construct a nine-dimensional input feature vector and reorganize it into "9xN" format, which contains normalized XYZ coordinates, normalized RGB channel values, and normal vector features. This feature vector will be used as the input of the improved PointNet++ model, and in the model training stage, the geometric relationship learning will be further strengthened through multi-dimensional feature fusion;

[0071] ​​D2. Training set data augmentation. Data augmentation is performed on the training set, including main rotation, XY plane rotation, coordinate jitter, etc. For small sample components, more intensive targeted augmentation is performed to increase their feature diversity, to alleviate the class imbalance problem in model training.

[0072] D3. Model structure optimization and training strategy adaptation. Based on the PointNet++ architecture, the bridge scene is optimized, including a multi-scale feature fusion network: a local-global interaction module is added, the local branch extracts neighborhood details, the global branch obtains contour features, and bidirectional fusion is achieved through cross attention. At the same time, a dynamic balance training mechanism is designed: a balance sampler is used to improve the sampling probability of small samples (3-5 times), coupled with an enhanced weighted loss, combined with a "linear preheating + cosine annealing" learning rate and gradient clipping, to ensure stable convergence of the model and focus on difficult-to-identify areas.

[0073] E. Post-processing and application of recognition results: extract the point cloud of each component by category, calculate the length, volume, axis deviation and other geometric parameters, and provide data support for bridge structure health assessment.

[0074] This embodiment is verified by a bridge digital model, which proves that the method can realize efficient generation and high-precision recognition of bridge component point clouds, and meet the engineering needs of bridge structure detection. The protection scope of the present application is not limited to the above embodiments, and any equivalent modifications made according to the principles of the present application shall be included in the protection scope.

Claims

1. A method for identifying bridge components based on UAV point cloud reconstruction and 3D synthetic data, characterized in that, Includes the following steps: A. Use drones to collect on-site videos of bridge components; B. Establish a framework for a three-dimensional reconstruction method of bridge components, and use the video of the bridge components to reconstruct a three-dimensional point cloud model of the bridge components; B1. Extract keyframes from the video of the bridge components using FFmpeg software; B2. Process the keyframe images using COLMAP software to output a three-dimensional point cloud of the bridge components; C. Generation of large-scale 3D point cloud synthetic data for bridges; C1. Set bridge component parameters (such as piers, beams, etc.), perform random parameter perturbation, generate diverse models, and convert them into mesh models. Use uniform sampling to generate triangular facet sampling points. Finally, add noise simulation (Gaussian noise, block occlusion, missing data, etc.) to enhance data diversity. C2. Perform UAV-simulated laser scanning on the triangulated model, calculate the flight time (ToF) by planning virtual flight routes, simulating laser emission angle, position, and reception time, and reconstructing the spatial position of the point cloud to generate a 3D point cloud of the bridge. C3. Repeat the above operations to obtain a large amount of diverse bridge point cloud composite data; D. Construct a semantic segmentation dataset of point clouds of bridge components; D1. Point Cloud Data Preprocessing: The bridge point cloud synthetic data is subjected to three-dimensional normalization processing, the centroid of the point cloud is calculated to achieve centering, and then the point cloud is scaled to a unit sphere according to the maximum radius to ensure the geometric consistency of the data; a nine-dimensional input feature vector is constructed, which includes normalized XYZ coordinates (core geometric features), normalized RGB channel values ​​(color features), and normal vector features (key spatial features), and the feature matrix is ​​reorganized into the "9×N" format required by the model (N is the number of points); D2. Data Labeling and Classification: For the main components of the bridge, professional 3D labeling tools are used for manual and detailed labeling to clarify their category classification; for a small number of components, their inherent shape features are first used for automatic pre-labeling by algorithms, and then they are strictly verified and corrected manually to ensure the accuracy of the labeling results to the greatest extent; the point cloud dataset is divided into training set and test set according to a preset ratio, and the basic principle of keeping the proportion of points of each component category in the training set and test set consistent is followed. D3. Data Augmentation: Various data augmentation operations are performed on the training set to expand the sample diversity. Basic augmentation methods include simulating master rotation from different observation angles, simulating XY plane rotation with horizontal changes, introducing coordinate jitter with slight positional perturbations, random rotation with all-around viewpoint changes, scaling with distance changes, and local translation with local positional shifts. For small sample component categories with few points, additional and more targeted augmentations are performed to significantly increase the diversity of their features. E. Build an improved PointNet++ point cloud recognition model for bridge components; E1. Network Structure Design and Optimization: Based on the PointNet++ architecture, the model is built by integrating the classification, segmentation and utils modules. The geometric relationship learning ability is enhanced by multi-dimensional feature fusion, which improves the model's adaptability to bridge scenarios. E2. Training Strategy and Execution Process: During training, a balanced sampler is used to dynamically adjust sample weights, and an enhanced weighted loss function is used to impose additional penalties on small sample misprediction and low IoU categories. Set an initial learning rate, adopt a scheduling method that combines linear warm-up and cosine annealing, and use gradient pruning strategy to ensure training stability; Using the test set mIoU as the core indicator, an early stopping mechanism is triggered when there is no improvement for several consecutive rounds, and finally the category prediction result of the optimal model is output to complete the point cloud recognition of bridge components. F. Post-processing and application of identification results: Calculate geometric parameters to provide data support for the assessment of the health status of bridge structures.

2. The bridge component identification method based on UAV point cloud reconstruction and 3D synthetic data according to claim 1, characterized in that: The method for acquiring bridge component video in step A is as follows: using a drone to surround the target component in a 360° omnidirectional manner to acquire video of the target component. The video acquisition requirements include information of the component at different heights from top to bottom, and finally, the bridge component video is collected.

3. The bridge component identification method based on UAV point cloud reconstruction and 3D synthetic data according to claim 1, characterized in that: In step B1, FFmpeg will be used to extract key frame images of the bridge components from the video. By selecting 0.25 as the time scaling factor, it will be ensured that the output key frames contain as much feature information of the bridge components as possible.

4. The method for identifying bridge components based on UAV point cloud reconstruction and 3D synthetic data according to claim 1, characterized in that: In step B2, the output of the three-dimensional point cloud of the bridge component is achieved using COLMAP software. COLMAP software uses the SIFT algorithm to extract image features, KNN, LoweRatio, and RANSAC algorithms to perform feature matching, and combines the SfM algorithm to reconstruct the sparse point cloud of the target component. Finally, the MVS algorithm is used to reconstruct the dense point cloud.

5. The bridge component identification method based on UAV point cloud reconstruction and 3D synthetic data according to claim 1, characterized in that: The random perturbation of parameters in step C1 to generate diverse models includes: extracting the basic geometric parameters of bridge components and applying random perturbations to the parameters to achieve model diversification; adjusting the degree of morphological variation of the generated model by controlling the perturbation intensity; and configuring a reasonable perturbation threshold range so that the generated component model can cover common deviation morphologies in actual engineering while maintaining structural rationality, in order to output training-level point cloud data with strong generalization ability.

6. The method for identifying bridge components based on UAV point cloud reconstruction and 3D synthetic data according to claim 1, characterized in that: The data augmentation operations in step D3 include basic augmentation and small-sample targeted augmentation. The basic augmentation is performed on all samples, including rotation, scaling, and Gaussian noise operations; Small sample-specific augmentation doubles the augmentation intensity for small sample categories, including: Δ~U(-0.05λ,0.05λ)(2) Where xyz′ represents the enhanced 3D coordinates, xyz i For the original 3D coordinates, m i For local deformation masking, s is the scaling factor along the main axis, and Δ is the translation amount, which enhances the morphological diversity of small samples.

7. The bridge component identification method based on UAV point cloud reconstruction and 3D synthetic data as described in claim 1, characterized in that: The multi-dimensional feature fusion mechanism in step E1 includes local coordinate normalization, RGB feature standardization, and coordinate gradient feature extraction. Local coordinate normalization enhances the expression of local geometric features through decentralization and scale scaling, and can be expressed as: Among them, xyz i The original 3D coordinates, xyz center Let max be the coordinates of the center point of the point cloud. dist The distance from the farthest point in the point cloud to the center point is used for scale normalization, xyz local This maps the normalized local coordinates to the unit sphere; RGB feature normalization normalizes color features, expressed as: Among them, rgb i The original RGB values, rgb min , rgb max These represent the minimum and maximum values ​​of the RGB channels in the point cloud, respectively. norm The standardized RGB features; coordinate gradient feature extraction calculates the local continuity features of the point cloud through neighborhood difference, simulating normal vector information, and can be expressed as: Among them, grad i Let N(i) be the coordinate gradient of point i, N(i) be the set of neighboring points of point i, and K be the number of neighboring points, used to balance the gradient magnitude.

8. The method for identifying bridge components based on UAV point cloud reconstruction and 3D synthetic data according to claim 1, characterized in that: The BalancedSampler in step E2 solves the class imbalance problem by first determining the class of small samples, then dynamically adjusting the weights, and finally performing stratified sampling. The determination of the small sample class is based on a dynamically determined threshold based on the distribution of the training set classes, using the following formula: T=max(1000,min(median(C valid ),1.5×median(C valid ))) (7) Among them, C valid ={c k |c k >0} represents the set of points for the effective category (number of samples > 0), median() is the median function, and T is the small sample threshold; dynamic weight calculation allocates sampling weights based on the number of samples belonging to the dominant category, with higher weights for smaller sample sizes, and the formula is: Among them, w i c is the sampling weight for the i-th sample. dom(i) The total number of points in the dominant class of the sample is defined as the class that appears most frequently in the sample. The weight array is normalized to ensure that the sum of probabilities is 1. The stratified sampling strategy allocates sampling quotas according to the class for the multi-class point cloud in each sample. The maximum proportion of non-small sample classes is ≤30%, and the minimum proportion of small sample classes is ≥20%. Small sample classes are sampled with replacement to ensure that each small sample class contains at least 200 points in the training batch.

9. A method for identifying bridge components based on UAV point cloud reconstruction and 3D synthetic data according to claim 1, characterized in that: The enhanced weighted loss in step E2 constructs a total loss function by fusing cross-entropy loss, IoU loss, and a dynamic weighting mechanism. The cross-entropy loss improves the accuracy of small-class predictions by introducing a few-sample error penalty, and its formula is as follows: in, To predict the probability, y i For real labels, s(y) i ) is a small sample mask, y i The IoU loss is set to 1 for small sample classes and 0 otherwise; the IoU loss can dynamically adjust the class weights, focusing on optimizing low IoU classes, and the formula is: Among them, C present IoU is the set of categories appearing in the current batch. k For the intersection-union ratio of the k-th class, for a small sample class w k =2.0, for non-small sample classes w k =1.0, and when IoU k When w < 0.3 k Multiply by an additional 1.5; the total loss function is calculated as follows: L total =α·L CE +(1-a)·L IoU (11) Where α is the cross-entropy loss weight and 1-α is the IoU loss weight.

Images