A cable-stayed bridge apparent damage detection method based on three-dimensional reconstruction and digital twinning

Abstract

The application provides a cable-stayed bridge apparent damage detection method based on three-dimensional reconstruction and digital twinning, relates to the technical field of cable-stayed bridge damage detection, and comprises the following steps: controlling a unmanned aerial vehicle and a cable-sticking aircraft to respectively collect global image pose data and single-cable refined image data sets; adopting a topological constraint reconstruction method to combine with cylindrical texture mapping to construct a full-bridge three-dimensional real scene model; adopting an improved YOLOv8 to detect disease pixel coordinates and categories after single-cable image enhancement; adopting a pixel-level segmentation network to extract an outline and calculate quantitative parameters of each disease; establishing a five-level coordinate conversion model to convert the disease pixel coordinates into a single-cable local three-dimensional coordinate point set; after the point set and the full-bridge model are registered through point cloud registration with a curved surface curvature constraint, each information detected is mounted to the three-dimensional real scene model to construct a cable-stayed bridge digital twinning model. The application can realize automatic identification, accurate quantification and three-dimensional space accurate mounting of apparent diseases of a cable-stayed cable.

Description

Technical Field

[0001] This invention relates to the field of cable-stayed bridge damage detection technology, and in particular to a method for detecting apparent damage to cable-stayed bridges based on three-dimensional reconstruction and digital twins. Background Technology

[0002] Cable-stayed bridges are a widely used long-span bridge type in modern highways and municipal infrastructure. As the core load-bearing component, the stay cables are exposed to complex natural environments for extended periods, making them prone to surface defects such as sheath cracks, bulging, and breakage, directly impacting the bridge's structural safety. Traditional inspection methods typically rely on manual visual inspection or high-altitude work in suspended baskets, which not only pose significant safety risks but also have limited detection rates for subtle defects on the stay cable sheath surface, making it difficult to generate systematic and quantifiable inspection results. Furthermore, traditional two-dimensional image acquisition and recording methods cannot effectively correlate defect location information with the actual three-dimensional space of the bridge, posing considerable challenges to subsequent maintenance decisions and defect evolution tracking.

[0003] Patent application CN118758267A discloses a bridge crack detection method based on close-up photogrammetry. This method utilizes an airborne LiDAR system to construct a coarse 3D point cloud model of the bridge. Based on this, it plans close-up photogrammetry flight paths for each bridge structural surface. A rotary-wing UAV equipped with a five-lens tilting camera acquires detailed images of the bridge, which are then reconstructed into a 3D tilting model. The method combines Faster R-CNN deep learning and image processing algorithms to identify cracks and measure their width, ultimately marking the detection results in the 3D model. While this method has some practical effect on general crack detection of bridge structural surfaces (piers, beams, towers, etc.), it only detects the entire bridge structure and does not specifically address the geometric characteristics and spatial distribution of stay cables, which are slender cylindrical components. It lacks a mechanism for distinguishing the attribution of defects in densely intersecting areas with multiple cables, and it does not establish a precise spatial registration method between 2D defect detection results and single-cable 3D real-world models. Therefore, it is difficult to support the needs of refined maintenance throughout the entire lifecycle of stay cables. Summary of the Invention

[0004] In view of this, the present invention provides a method for detecting apparent damage to cable-stayed bridges based on 3D reconstruction and digital twins. By establishing a two-layer UAV acquisition architecture oriented towards the geometric features of the cable-stayed cables, a 3D reconstruction and texture correction method for slender cylinders, a deep learning-based method for defect detection and pixel-level segmentation adapted to the background features of the cable-stayed cables, and a multi-level attitude-related coordinate transformation and single-cable surface adaptive registration mechanism, the present invention achieves automatic identification, accurate quantification, and precise 3D spatial mounting of apparent defects in cable-stayed cables, and constructs a digital twin model of cable-stayed bridges that supports full life-cycle maintenance.

[0005] The technical solution of this invention is implemented as follows: This invention provides a method for detecting apparent damage to cable-stayed bridges based on three-dimensional reconstruction and digital twins, comprising: S1. Control the UAV to perform global oblique photography of the cable-stayed bridge, and collect a global image dataset and corresponding pose data; control the cable-attaching aircraft to fly spirally along the surface of each cable, and collect a refined image dataset of a single cable. S2. Using topological constraints and multi-view stereo vision reconstruction methods, the full-domain image dataset and pose data are used to perform 3D reconstruction. Combined with cylindrical adaptive texture mapping, a 3D real-scene model of the entire cable-stayed bridge is constructed. S3. Perform local gain texture enhancement processing on each frame of the single-crystal refined image dataset, and use the improved YOLOv8 network for disease detection to obtain the pixel coordinates and disease categories of each disease area. S4. For the pixel coordinates of each disease area, a pixel-level segmentation network is used to extract the disease outline, and the quantitative parameters of each disease are calculated by combining the cylindrical distortion correction algorithm. S5. Establish a five-level attitude-related coordinate transformation model between the pixel coordinate system, image coordinate system, camera coordinate system, world coordinate system and single-cable local coordinate system, and transform the pixel coordinates of each disease area into a single-cable local three-dimensional coordinate point set step by step. S6. The local three-dimensional coordinate point set of a single cable is registered with the corresponding cable surface in the three-dimensional real scene model of the whole bridge using a point cloud registration algorithm with surface curvature constraints. The disease category, quantification parameters, spatial location, detection time sequence and damage level of each disease are loaded into the three-dimensional real scene model of the whole bridge to construct a digital twin model of the cable-stayed bridge.

[0006] Preferably, step S1 includes: S11. Calibrate the camera intrinsic parameters of the UAV and the cable-attached aircraft, determine the focal length, pixel size and photosensitive size, and set the overall aerial survey heading overlap to be no less than 80%, the lateral overlap to be no less than 70% and the corresponding flight path spacing. S12. Control the drone to plan a full-coverage grid flight path, collect multi-angle oblique photographic images of the entire cable-stayed bridge, and simultaneously record the pose vector corresponding to each frame of image in real time by the POS positioning and attitude determination system. Where B is the latitude of the data collection point, L is the longitude of the data collection point, and H is the absolute elevation of the data collection point. For camera tilt angle, For camera roll angle, This refers to the camera yaw angle; S13. The aircraft is controlled to plan a vertical spiral following route based on the central axis of a single cable. The aircraft maintains a constant distance of 0.5m to 1m from the cable and collects detailed images of each cable from the anchorage end of the bridge tower to the anchorage end of the main beam. S14. Perform quality screening on the two types of acquired images, removing invalid images caused by flight shaking, strong light overexposure, backlight shadows, obstruction by debris, and blurring or defocusing. Filter and denoise the original POS pose data and remove abnormal jump values. Complete the matching of timestamps and spatial coordinates between the images and pose data, and establish a global image dataset and a single-index refined image dataset.

[0007] Preferably, step S2 includes: S21. Based on the global image dataset and corresponding pose data, perform motion reconstruction structure calculation and optimize the exterior orientation elements of each frame image. S22, using vertically adjacent three-dimensional feature points , elevation value , Calculate the elevation constraint function: ; in To preset the vertical elevation continuous threshold, the subscripts a and a+1 represent the three-dimensional feature point index; when When determining two points as feature points of a continuous cable body; S23. Generate a full-bridge dense point cloud using the MVS multi-view dense matching algorithm, and generate a continuous grid model using the triangular mesh reconstruction algorithm. S24. Use the cylindrical texture mapping distortion correction formula. The texture coordinates are corrected, where u and v are the horizontal and vertical coordinates of the original texture pixels in the planar mapping. R is the texture pixel coordinate after cylindrical correction, R is the radius of the cable stay, and y is the texture pixel ordinate. The texture of each cable body surface is baked to construct a three-dimensional real scene model of the entire bridge.

[0008] Preferably, in step S3, when applying local gain texture enhancement processing to each frame of the single-frame refined image dataset, the enhanced grayscale value of each pixel is calculated according to the following formula: ; in , Image coordinates Gray values ​​before and after pixel enhancement The average grayscale value of pixels in the ROI region of the object. Let be the grayscale variance of the cable body region, and d be the vertical distance from the current pixel to the central axis of the cable body. This is the gain attenuation coefficient.

[0009] Preferably, in step S3, the improved YOLOv8 network adds a shallow fine-grained feature weighting and fusion module to the original YOLOv8 network, which performs fine-grained feature mapping on the output of k shallow feature layers. Weighted fusion is performed according to the following formula: ; in This is the feature map after multi-scale feature fusion. represents the adaptive weight coefficients for each feature layer, and j is the index of the shallow feature layer.

[0010] Preferably, in step S4, extracting the disease contour using a pixel-level segmentation network for the pixel coordinates of each disease area includes: The image regions corresponding to the pixel coordinates of each diseased area are input into the Dense-DeepLabv3+ network, and features are extracted using a multi-scale dilated convolutional spatial pyramid pooling structure. The output is a binary crack segmentation map: ; in , , These are dilated convolutions with dilation rates of 1, 3, and 5, respectively. This is a global pooling feature. This is a channel concatenation operation, where F is the input feature map; Skeletonization is performed on the crack segmentation binary image output by the network, and the skeleton pixels at the center of the crack are extracted. According to the coordinate mapping formula for cylindrical distortion correction: ; Image coordinates Mapped to three-dimensional coordinates of the cable cylinder Where R is the radius of the stay cable. The central angle of the cylinder corresponding to the pixel; The quantitative parameters of each disease are calculated in three-dimensional cylindrical coordinates.

[0011] Preferably, the crack quantification parameters are calculated according to the following formula: Total area of ​​cracks The actual total length of the crack Average crack width Maximum crack width Where N is the total number of valid pixels in the binary image of the crack segmentation. Here, GSD represents the total number of pixels in the crack skeleton, and GSD represents the ground resolution during single-cable fine image acquisition. Let s = 1, 2, ..., m, where m is the number of crack segments.

[0012] Preferably, step S5 includes: S51, according to pixel row and column numbers Convert to image coordinates ,in The row and column numbers of the principal pixel in the image. The physical dimensions of a single pixel are defined horizontally and vertically. S52. Convert the image coordinates to three-dimensional coordinates in the camera coordinate system according to the camera intrinsic parameter matrix. The camera's focal length is f; S53, with pitch angle Roll angle Yaw angle Construct a rotation matrix and a translation matrix using latitude B, longitude L, and absolute elevation H to convert camera coordinates to world coordinates; S54. Establish a local coordinate system for each cable-stayed cable, convert the world coordinates to local three-dimensional coordinates for each cable, complete the coordinate system alignment, and obtain the local three-dimensional coordinate point set of each cable in each diseased area; and synchronously transform the feature point set of each cable surface from the full bridge three-dimensional real scene model to the corresponding local coordinate system of each cable.

[0013] Preferably, step S6 includes: S61. For the local three-dimensional coordinate point set of each cable and the corresponding cable surface feature point set in the three-dimensional real-world model of the entire bridge, iterative registration is performed using the following formula as the objective function: ; in For the total registration error, For the feature point set of the cable body surface, Let R and T be the local 3D coordinate point set of a single cable, and R and T be the rotation and translation matrices to be optimized. For the curvature constraint term of the cable surface, To balance the weighting coefficients, n is the number of points involved in the registration; S62. Based on the optimal rotation matrix R and translation matrix T, the three-dimensional spatial position of each disease after registration is transformed back to the world coordinate system through the inverse transformation from the local coordinate system of the single cable to the world coordinate system. The disease category, quantitative parameters, spatial position, detection time sequence and damage level of each disease are then mounted to the corresponding cable surface position in the three-dimensional real scene model of the whole bridge. S63. Verify the mounting position of each cable-stayed bridge defect one by one, correct the defect attribution deviation in the multi-cable intersection area, and construct a digital twin model of the cable-stayed bridge.

[0014] Preferably, in step S1, when the cable-attaching aircraft spirals along the surface of each cable, the ground resolution is based on the fine-grained imagery of a single cable. To control the acquisition accuracy, where H is the vertical flight altitude of the cable-attached aircraft relative to the cable surface. f is the physical size of a single pixel in the camera, and f is the focal length of the camera lens.

[0015] The present invention has the following advantages over the prior art: (1) A dual-layer acquisition architecture combining global oblique photography and cable-attached drones was adopted to resolve the contradiction between the integrity of the cable-stayed bridge point cloud reconstruction and the local detail resolution. Conventional single-altitude UAV aerial surveys are difficult to achieve complete cable-stayed bridge point cloud coverage in the tower area. This invention complements the macro-level full-bridge modeling with the micro-level single-cable fine-grained acquisition, ensuring both the integrity of the full-bridge 3D model and the fineness of the single-cable texture, thus avoiding the problems of cable point cloud breakage and voids in traditional methods.

[0016] (2) The introduction of a vertical continuous topological constraint mechanism and a cylindrical adaptive texture mapping method effectively solves the model quality problem in the 3D reconstruction of cable stays. Cable stays are weakly textured slender cylindrical components, and conventional SfM-MVS reconstruction is prone to multi-cable adhesion and texture stretching distortion. This invention maintains the geometric continuity of a single cable through topological constraints and eliminates texture distortion when the cylindrical surface is unfolded through cylindrical adaptive mapping, so that the fine texture defects on the surface of the cable sheath can be completely preserved.

[0017] (3) The YOLOv8 detection network was improved to address the background features of the cable sheath, enhancing its ability to detect minor defects. The cylindrical surface of the cable exhibits a unique gradient distribution of light and dark in the image, and traditional detection networks have a high rate of missed detection for ultra-small and sparse defects in such backgrounds. This invention suppresses background interference through a strip-shaped ROI adaptive local gain enhancement algorithm, and combines shallow fine-grained feature weighted fusion with an optimized small target localization loss function, enabling the detection network to maintain relatively stable recognition performance for minor defects under complex cable background conditions.

[0018] (4) The Dense-DeepLabv3+ pixel-level segmentation network combined with the cylindrical distortion correction algorithm was used to accurately calculate the crack quantization parameters. Cracks in the cable sheath are usually long and thin, and ordinary segmentation networks have difficulty maintaining the continuity of the crack skeleton. This invention enhances long-distance pixel dependencies through a densely cascaded ASPP structure, reduces crack segmentation breakpoints, and uses cylindrical distortion correction to convert pixel distances on the cylindrical surface into real physical dimensions, thereby obtaining geometrically meaningful crack area, length, and width quantization parameters.

[0019] (5) A five-level attitude-related coordinate transformation model was established, which fundamentally solved the problem of spatial attribution of defects in multi-cable intersection areas. In the anchorage area of ​​cable-stayed bridges, multiple cable stays intersect and overlap, and it is difficult to accurately determine the cable to which the defect belongs by simply relying on image coordinates. This invention constructs a complete transformation link from pixel coordinates to the local coordinate system of a single cable, and combines the POS pose data of each frame of image to accurately limit the spatial location of each defect to the coordinate system of the corresponding cable, effectively eliminating the ambiguity of defect attribution.

[0020] (6) The information on the defects of each cable-stayed bridge is registered to the three-dimensional real-world model through surface adaptive ICP, thus constructing a digital twin model with spatial information on the defects. Traditional methods annotate defect information on two-dimensional images, which lack spatial correspondence with the physical structure. This invention uses the curvature of a single cable surface as a constraint for iterative registration, so that the defect contour and quantitative parameters are accurately fitted to the three-dimensional surface of the cable body, realizing the visualization of the defects in three-dimensional roaming, the temporal comparison of the results of previous tests, and the quantitative assessment of the damage level. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a schematic diagram of the UAV oblique photography of the present invention; Figure 3 This is a flowchart of the three-dimensional reconstruction process of the present invention; Figure 4 This is a flowchart illustrating the disease classification and quantification process of the present invention; Figure 5 This is a schematic diagram of the five-polar coordinate transformation and three-dimensional attachment of the present invention; Figure 6 This is a schematic diagram of the digital twin of the present invention. Detailed Implementation

[0023] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0024] like Figure 1 As shown, this invention provides a method for detecting apparent damage to cable-stayed bridges based on three-dimensional reconstruction and digital twins, comprising the following steps: S1. Control the UAV to perform global oblique photography of the cable-stayed bridge, and collect a global image dataset and corresponding pose data; control the cable-attaching aircraft to fly spirally along the surface of each cable, and collect a refined image dataset of a single cable. S2. Using topological constraints and multi-view stereo vision reconstruction methods, the full-domain image dataset and pose data are used to perform 3D reconstruction. Combined with cylindrical adaptive texture mapping, a 3D real-scene model of the entire cable-stayed bridge is constructed. S3. Perform local gain texture enhancement processing on each frame of the single-crystal refined image dataset, and use the improved YOLOv8 network for disease detection to obtain the pixel coordinates and disease categories of each disease area. S4. For the pixel coordinates of each disease area, a pixel-level segmentation network is used to extract the disease outline, and the quantitative parameters of each disease are calculated by combining the cylindrical distortion correction algorithm. S5. Establish a five-level attitude-related coordinate transformation model between the pixel coordinate system, image coordinate system, camera coordinate system, world coordinate system and single-cable local coordinate system, and transform the pixel coordinates of each disease area into a single-cable local three-dimensional coordinate point set step by step. S6. The local three-dimensional coordinate point set of a single cable is registered with the corresponding cable surface in the three-dimensional real scene model of the whole bridge using a point cloud registration algorithm with surface curvature constraints. The disease category, quantification parameters, spatial location, detection time sequence and damage level of each disease are loaded into the three-dimensional real scene model of the whole bridge to construct a digital twin model of the cable-stayed bridge.

[0025] like Figure 2 As shown, in one embodiment of the present invention, step S1 includes: S11. Before the data collection operation, the internal parameters of the cameras mounted on the UAV and the cable-attached aircraft were calibrated. Multi-angle and multi-distance calibration shooting was completed using a checkerboard target. The focal length f and principal point coordinates of each camera were solved using the Zhang Zhengyou calibration method. Pixel physical size The radial and tangential distortion coefficients are calculated, and the calibration results are written into the camera configuration file.

[0026] Based on this, the ground physical size (GSD) of a single pixel corresponding to different flight altitudes H is calculated using the ground resolution formula: ; in is the physical size of a single pixel of the camera (mm / pixel), and f is the focal length of the camera lens (mm). Based on this, the heading overlap of the whole-area aerial survey is set to be no less than 80%, the lateral overlap to be no less than 70%, and the corresponding route spacing is calculated.

[0027] S12. Control the drone to plan a full-coverage grid flight path, collect multi-angle oblique photographic images of the entire cable-stayed bridge, and simultaneously record the pose vector corresponding to each frame of image in real time by the POS positioning and attitude determination system. Where B is the latitude of the data collection point, L is the longitude of the data collection point, and H is the absolute elevation of the data collection point. For camera tilt angle, For camera roll angle, The above six parameters constitute the original pose reference for subsequent aerial triangulation and coordinate transformation, with the camera yaw angle being the yaw angle.

[0028] S13. The aircraft is controlled to plan a vertical spiral following route based on the central axis of a single cable. The aircraft maintains a constant distance of 0.5m to 1m from the cable body and collects detailed images of each cable from the anchorage end of the bridge tower to the anchorage end of the main beam to ensure that the cable body sheath is fully covered without blind spots.

[0029] S14. Perform quality screening on the two types of acquired images, check and remove invalid images frame by frame due to flight jitter, strong light overexposure, backlight shadow, clutter obstruction, and blurring / out-of-focus. Perform median filtering noise reduction on the original POS pose data, remove abnormal jump values ​​in elevation and attitude angle caused by flight turbulence and signal interruption, complete the accurate matching of timestamps and spatial coordinates between the images and POS pose data, and establish a global image dataset and a single-cable refined image dataset respectively.

[0030] like Figure 3 As shown, in one embodiment of the present invention, step S2 includes: S21. Import the global image dataset and corresponding pose data into ContextCapture software, set a unified geographic coordinate system, and use the POS pose data as the initial exterior orientation element.

[0031] Image feature point extraction and cross-image feature matching based on the SfM motion reconstruction algorithm: First, the SIFT algorithm is used to extract scale- and rotation-invariant feature points for each frame of the image, resulting in the feature point set and corresponding descriptor for each frame of the image. Then, cross-image feature descriptor matching is performed using the nearest neighbor Euclidean distance ratio method to select initial matching point pairs; Subsequently, the RANSAC random sampling consensus algorithm was introduced to remove incorrect matching point pairs. The fundamental matrix was estimated based on the retained interior point set between image pairs to complete the epipolar constraint verification and retain the correct matching point pairs that conform to the geometric relationship.

[0032] To address the weak texture and slender features of the cable-stayed bridge, a slender feature filtering threshold is added during the matching stage. This requires that the distribution of matching point pairs in the image conforms to the shape of vertical slender strips, retaining effective feature point pairs of slender strips of the cable body. At the same time, a spatial neighborhood distance threshold is set to distinguish feature points of different cable bodies, eliminating cross-cable erroneous matches in multi-cable intersection areas and reducing aerial triangulation errors.

[0033] After feature matching is completed, the matched point pairs are triangulated using multi-view geometric constraints to obtain the initial sparse 3D point cloud and the initial exterior orientation elements of each frame image. Then, the global camera pose and 3D point cloud spatial coordinates are jointly optimized using the bundle adjustment algorithm. The optimization objective function is: ; Where F is the sum of global reprojection errors. Describes the Euclidean norm. Let be the measured pixel coordinates of the i-th feature point on the j-th image. For three-dimensional space points Image exterior orientation elements The corresponding theoretical projection pixel coordinates are used to minimize the global reprojection error through least squares iteration, thereby accurately optimizing the exterior orientation elements and 3D point cloud spatial coordinates of each frame of image.

[0034] S22. In the aerial triangulation optimization process, a vertical elevation continuity constraint is introduced for the longitudinally adjacent three-dimensional feature points of the cable stays. elevation value Calculate the elevation constraint function: ; in To establish a preset vertical elevation continuity threshold based on the cable slope and flight accuracy, the subscripts a and a+1 represent the three-dimensional feature point indices. Describe the Euclidean norm when When two points are identified as continuous cable feature points, their matching relationship is retained; otherwise, they are identified as fracture or abrupt points and are removed to eliminate vertical point cloud fractures and void defects in the cable.

[0035] S23. Based on the optimized exterior orientation elements, generate a dense point cloud of the entire bridge using the MVS multi-view stereo vision dense matching algorithm.

[0036] The specific process is as follows: using the sparse point cloud obtained from aerial triangulation as the initial depth estimation reference, a PatchMatch stereo matching strategy is used for each image pair to perform depth hypothesis search and random propagation for each pixel. Under the criterion of minimizing the neighborhood matching cost, the depth value of each pixel is optimized through multiple rounds of iterative propagation to generate a dense depth map for each frame of image; multi-view analysis is then performed on all depth maps. Figure 1 Consistency fusion verification removes noise points with inconsistent depth values ​​across images. Depth pixels that pass the consistency verification are back-projected into 3D space and fused to obtain a high-density 3D point cloud of the entire bridge. In the dense matching stage, an additional cylindrical geometric constraint is added to the cable body region, requiring the point distribution of the dense point cloud in the radial direction to match the actual radius of the cable body. Abnormal points that deviate from the cylindrical constraint are removed to avoid multiple cables from sticking together.

[0037] S24. Based on dense point clouds, a Poisson surface reconstruction algorithm is used to reconstruct triangular meshes from the point clouds. An implicit function is constructed to describe the cylindrical surface of the cable-stayed structure, adaptively filling local mesh holes to generate a continuous and complete bridge and cable mesh model. To address the stretching and compression distortions caused by traditional planar texture mapping during the baking of the cable-stayed cylindrical surface, an axial cylindrical adaptive texture mapping technique is adopted, using the cylindrical texture mapping distortion correction formula: ; The texture coordinates are corrected, where u and v are the horizontal and vertical coordinates of the original texture pixels in the planar mapping. Here, R represents the actual radius of the cable stay, and y represents the ordinate of the texture pixels after cylindrical surface correction. This formula utilizes the geometric properties of cylindrical surfaces to compress and correct the lateral stretched texture while keeping the ordinate unchanged, eliminating surface texture distortion and fully preserving the fine texture defects of the cable sheath. Finally, combined with global geographic coordinate control points, the model coordinate deviation is calibrated, overall translation and rotation errors are corrected, and a full-bridge 3D realistic model with unified geographic coordinates, faithful texture, and complete structure is output.

[0038] The above reconstruction process uses narrow feature screening and longitudinal elevation continuity constraint to jointly optimize feature matching, eliminating point cloud fractures and multi-cable adhesion defects of the weak texture of the slender column of the cable from the source. Combined with column surface texture mapping correction, the fine texture of the cable sheath surface can be baked into the 3D model completely and faithfully.

[0039] like Figure 4 As shown, in one embodiment of the present invention, step S3 includes: S31. For each frame of the single-cable refined image dataset, based on the vertical strip-shaped morphology of the stay cable, adaptive threshold segmentation and connected component analysis are used to extract the bounding rectangle of the vertical cable body region in the image as a strip-shaped Region of Interest (ROI). Invalid background regions such as the sky, ground, and main beam are cropped and removed. Local gain texture enhancement processing is applied to the retained cable body ROI region, and the grayscale value of each pixel after enhancement is calculated according to the following formula: ; in , Image coordinates Gray values ​​before and after pixel enhancement The average grayscale value of pixels in the ROI region of the object. Let be the grayscale variance of the cable body region, and d be the vertical distance from the current pixel to the central axis of the cable body. is the gain attenuation coefficient; this formula achieves high-intensity texture gain in the central region of the cable body and adaptive suppression in the edge background region, accurately preserving the fine texture features of hair-level defects.

[0040] S32. Construct a dedicated disease perception dataset for cable-stayed bridges. Select samples from real-world images of cable-stayed bridges, covering various small-target diseases such as micro-cracks, surface peeling, exposed reinforcement, and corrosion. Expand sample diversity through methods such as horizontal flipping, random cropping, brightness and contrast perturbation, Gaussian noise addition, and Mosaic stitching enhancement. Divide the dataset into training, validation, and test sets in a 7:2:1 ratio, and annotate the category and pixel coordinates of each disease region using YOLO annotation format.

[0041] S33. Targeted improvements to the native YOLOv8 network. The native YOLOv8 network uses CSPDarkNet as its backbone, with the C2f module (containing a cross-stage feature fusion module with multiple Bottleneck units) forming the main body. The network contains five downsampling stages, outputting multi-scale feature maps with strides of 8, 16, and 32. The PAN-FPN neck network enables bidirectional feature fusion from top to bottom and bottom to top. Finally, three detector heads are responsible for the classification and regression prediction of small, medium, and large targets, respectively.

[0042] To address the issue of extremely small target size and easy feature loss in deep layers of the cable-stayed bridge sheath due to ultra-small sparse defects, a shallow fine-grained feature weighting and fusion module is added after the second and fourth stages of convolution in the shallow layers of the backbone network. This module performs fine-grained feature mapping on the outputs of k shallow feature layers. Weighted fusion is performed according to the following formula: ; in This is the feature map after multi-scale feature fusion. The adaptive weight coefficients for each shallow feature layer are automatically updated via backpropagation, and j is the index of the shallow feature layer. The fused feature map and deep semantic features are concatenated in the neck PAN-FPN structure to transmit high-resolution, fine-grained semantic information to the small target detection head. Simultaneously, the original bounding box regression loss in the detection head is replaced with an optimized small target CIoU loss function. ; in These are the predicted bounding box and the actual disease bounding box, respectively. Let c be the square of the Euclidean distance between the center points of the two frames, c be the length of the diagonal of the smallest bounding rectangle, and v be the aspect ratio difference coefficient. To balance the weights, this loss function takes into account overlap, center distance, and aspect ratio, significantly improving the regression accuracy of ultra-small target bounding boxes.

[0043] S34. Use transfer learning to train the model.

[0044] In the first stage, YOLOv8 weights pre-trained on the COCO large-scale dataset are loaded, all convolutional layer parameters of the first three stages of the backbone network are frozen, and only the parameters of the neck network, the detection head and the newly added shallow feature fusion module are opened for training. The initial learning rate is set to 0.01 and the cosine annealing scheduling strategy is used. After 50 iterations, the new module can achieve initial convergence at a low learning rate to prevent the pre-trained features from being destructively forgotten. In the second stage, all network layers are unfrozen, and the complete network is fine-tuned globally with a small learning rate of 0.001. Training continues on the cable-stayed bridge-specific dataset until the validation set accuracy (mAP@0.5) no longer improves. The network weights with the best performance on the validation set are saved as the final model through an early stopping strategy.

[0045] S35. Input the preprocessed enhanced high-definition image into the optimal model, perform inference detection frame by frame, output the category label, confidence score and pixel coordinate bounding box of each disease, generate the region of interest (ROI) of each disease, and accurately locate all minor diseases.

[0046] The above improvements integrate shallow high-resolution fine-grained features with deep semantic features in the neck structure. At the same time, the training is guided by a CIoU loss function adapted to the geometric characteristics of small targets. Without significantly increasing the number of network parameters, the improvements make up for the gradual loss of ultra-small sparse disease features in the deep layers of the network, effectively improving the detection rate and positioning accuracy of sparse and small diseases on the surface of the cable sheath.

[0047] In one embodiment of the present invention, step S4 includes: The crack ROI images locked by S3 detection are preprocessed with grayscale normalization and mean filtering to reduce noise, and the image size is unified to the network standard input resolution to eliminate the impact of uneven illumination and noise on segmentation accuracy.

[0048] The preprocessed cracked ROI image is input into the Dense-DeepLabv3+ network. This network uses ResNet-50 as the backbone feature extraction network: the input image first passes through a 7×7 convolutional layer (stride 2, output channels 64) and a 3×3 max pooling layer (stride 2), and then sequentially passes through four residual blocks of ResNet-50 (Layer 1~Layer 4, output channels 256, 512, 1024, and 2048 respectively). The convolutional layers of Layer 3 and Layer 4 use dilated convolutions instead of downsampling to maintain the spatial resolution of the feature map. The feature map output from Layer 4 enters the improved ASPP module, which introduces dense cascaded connections on the basis of the original DeepLabv3+ ASPP structure. The outputs of the three dilated convolutions with dilation rates of 1, 3, and 5 are concatenated with the global average pooling result, and the output of the previous channel is superimposed as a residual connection to the input of the next channel, forming a densely cascaded multi-scale dilated convolutional spatial pyramid pooling structure. The output fused global feature map is as follows: ; in These are dilated convolutions with dilation rates of 1, 3, and 5, respectively. This is a global average pooling feature. This is a channel-dimensional concatenation operation, where F is the input feature map; Dense cascaded connections enable ASPP branches at each scale to reuse features from the previous branch, effectively establishing long-distance continuous pixel dependencies of ultra-long strip cracks along the longitudinal direction of the cable body, thus solving the problem of breakpoints and edge blurring when traditional segmentation algorithms process ultra-long cracks.

[0049] In the decoding stage, the ASPP output is concatenated with the shallow features of ResNet-50 Layer 1 (reduced to 48 channels by 1×1 convolution), the boundaries are refined by two layers of 3×3 convolution, and finally upsampled to the original input resolution by bilinear interpolation to output a pixel-by-pixel binary segmentation map that completely distinguishes the crack area from the intact area of ​​the cable.

[0050] Morphological refinement is performed on the binary crack segmentation map output by the network to extract the crack center skeleton.

[0051] The specific process is as follows: First, morphological erosion (3×3 rectangular structuring element) is applied to the binary image to gradually reduce the edge pixels of the crack region; then, the Zhang-Suen thinning algorithm is used to iteratively skeletonize the eroded binary image. Each iteration consists of two sub-steps: determining whether to mark and delete a foreground pixel by judging whether the connectivity of its 8-neighborhood and the number of its foreground neighbors meet the deletion condition. This process is repeated until the image no longer changes, and finally, a crack center skeleton line with a width of one pixel is output, and the total number of skeleton pixels is counted. .

[0052] Connectivity analysis is performed on the binary image segmented by cracks. An eight-connectivity labeling algorithm is used to label all foreground pixels as connected components. The number of pixels in each independent connected component is counted. Noise and pseudo-disease connected components with an area below a set threshold are filtered out, and valid crack connected components are retained. The total number of valid crack pixels N is obtained by summing the counts of all valid pixels in each connected component.

[0053] To address the perspective distortion introduced by the cylindrical surface of the cable-stayed bridge, the coordinate mapping formula for cylindrical distortion correction is applied as follows: ; Image coordinates Mapped to three-dimensional coordinates of the cable cylinder Where R is the actual radius of the stay cable. The central angle of the cylinder corresponding to the pixel; this formula is a small-angle engineering approximation formula, taken under the premise of close-range shooting with the aircraft close to the cable, extremely small imaging field of view, and weak surface distortion. This is approximately true. In cylindrical 3D coordinates, for each skeleton pixel in the skeleton, calculate its local tangent direction vector. Draw a perpendicular line along this tangent direction (i.e., the skeleton normal direction), and the distance between the intersection points of this perpendicular line and the boundary contours on both sides of the crack segmentation binary image is the measured width of that segment. By iterating through all skeleton pixels, the width of each segment is measured sequentially to obtain the width sequence of each segment. m is the number of crack segments.

[0054] Crack quantification parameters are calculated using the following formula: Total crack area The actual total length of the crack Average crack width Maximum crack width Where N is the total number of valid pixels in the binary image of the crack segmentation. Here, GSD represents the total number of pixels in the crack skeleton, and GSD represents the ground resolution during single-cable fine image acquisition. The measured widths of each segment of the crack are as follows: m represents the number of crack segments; all parameters are converted to millimeter-level physical dimensions to achieve millimeter-level precise quantification of crack defects.

[0055] The above steps combine the long-distance pixel dependency capability of the densely cascaded ASPP structure with the skeleton vertical line point-by-point ranging algorithm under cylindrical coordinate mapping, which improves the segmentation integrity and width quantization accuracy of ultra-long strip cracks, and solves the problem of crack measurement deviation caused by perspective distortion when the traditional segmentation algorithm processes the cylindrical curved surface image of the cable-stayed bridge.

[0056] like Figure 5 As shown, in one embodiment of the present invention, step S5 includes: S51. Calibrate the origin, coordinate axis direction, and scale parameters of each coordinate system one by one to complete the parameter calibration from the pixel coordinate system to the image coordinate system. Use the following formula to convert the pixel row and column numbers... Convert to two-dimensional coordinates in the image coordinate system : ; in The row and column numbers of the principal pixel in the image. These are the horizontal and vertical physical dimensions of a single pixel, respectively; this transformation restores the dimensionless pixel row and column numbers to image coordinates in units of physical length.

[0057] S52. Rearrange the image coordinates according to the camera intrinsic parameter matrix. Convert to 3D coordinates in camera coordinate system The transformation relationship is as follows: ; Where f is the camera focal length. The depth value is generated from the dense depth map of the corresponding frame image by MVS dense matching in S23, according to the disease pixel coordinates. The floating-point depth value at that location is read directly by index; the dense depth map is stored in a single-channel floating-point image format with the same resolution as the original image. Each pixel location stores the distance from the corresponding 3D scene point along the camera's optical axis to the camera's optical center. This depth value is correlated with the image coordinates. Substitute all the values ​​into the intrinsic parameter matrix formula. This step expands the two-dimensional image coordinates into three-dimensional spatial coordinates with the camera optical center as the origin.

[0058] S53, using pose vector pitch angle Roll angle Yaw angle Rotation matrices around the X, Y, and Z axes are constructed sequentially, and multiplied together to obtain a 3×3 camera extrinsic rotation matrix R; a 3×1 translation matrix is ​​obtained by transforming latitude B, longitude L, and absolute elevation H into a geographic coordinate system. Convert camera coordinates to three-dimensional coordinates in the world coordinate system using the following formula. : ; Where R is a 3×3 camera extrinsic rotation matrix and T is a 3×1 translation matrix, both of which are solved by the POS pose parameters to achieve a unified mapping from local camera coordinates to global geographic coordinates.

[0059] S54. For each cable, based on its actual tilt attitude and spatial orientation, taking the center of the anchorage end closest to the bridge tower as the origin and the direction of the cable's central axis as the main Z-axis, measure and establish a dedicated local coordinate system for each cable in the three-dimensional model of the entire bridge; solve the dedicated rotation matrix for each cable. (3×3) and translation matrix (3×1), adapting to the differentiated spatial attitude of each cable, the three-dimensional points in the world coordinate system are converted into three-dimensional coordinates in the local coordinate system of a single cable using the following formula. : ; in For the coordinates of the local model of a single cable, For each cable, a dedicated rotation and translation matrix is ​​used; simultaneously, the corresponding feature point set of the cable surface in the 3D reality model of the entire bridge is generated. Through the same Transforming to a single-cable local coordinate system provides a unified coordinate benchmark for subsequent ICP registration, enabling precise spatial binding of defects to each cable and effectively distinguishing the attribution of defects in multi-cable intersection areas.

[0060] like Figure 6 As shown, in one embodiment of the present invention, step S6 includes: S61. Perform the following adaptive iterative registration process for each cable: use the alignment of the cable's center of gravity as the initial registration pose, initialize the rotation matrix R=I (identity matrix) and translation matrix T=0; enter the ICP iteration loop, and in each iteration, first process the local three-dimensional coordinate point set of a single cable. Each point in the feature point set of the cable body surface The KD-tree data structure is used to quickly search for the nearest neighbor and establish the correspondence between point pairs in the current round. Then, the optimal rigid body transformation is calculated for the established point pairs. The following formula is used as the objective function to solve for the optimal rotation matrix R and translation matrix T in the current round: ; Where E(R,T) is the total registration error. For the feature point set of the cable body surface, Let R and T be the local 3D coordinate point set of a single cable, and R and T be the rotation and translation matrices to be optimized. This is a curvature constraint term for the cable body surface, limiting the deviation of the point set from the cylindrical surface shape during registration. To balance the weighting coefficients, n is the number of points participating in the registration. Denotes the Euclidean norm; This round of solution uses the Singular Value Decomposition (SVD) method to calculate R and T that minimize the objective function. Update the position based on the obtained R and T, calculate the registration error E for this round, and determine whether the change of E relative to the previous round is lower than the preset convergence threshold. If convergence is achieved, stop the iteration and output the optimal R and T. If convergence is not achieved, enter the next round of iteration, continue to search for point pairs and solve the transformation, until the maximum number of iterations is reached or the error converges.

[0061] curvature constraint Quantification is achieved by calculating the normal distance from each registration point to the fitted cylinder. Points whose normal distances exceed the cylinder radius tolerance range have larger corresponding normal distances. The value is increased, thereby penalizing registration schemes that deviate from the cylindrical shape in the objective function, preventing non-physical geometric offsets in the registration results. This registration process is performed independently for each cable body, abandoning the traditional globally unified registration mode and adapting to the differentiated postures of each cable body in space.

[0062] S62. Based on the optimal rotation matrix R and translation matrix T, the local three-dimensional coordinates of each defect after registration are transformed back to the world coordinate system through the inverse transformation from the local coordinate system to the world coordinate system. The defect category, quantification parameters, spatial location, detection sequence and damage level of each defect are then mounted to the corresponding cable surface position in the three-dimensional real scene model of the whole bridge.

[0063] The specific attributes of the mounting include: the type of damage (micro-cracks, surface peeling, exposed reinforcement, corrosion, etc.) and quantitative parameters (crack area S, actual total length L, average width). Maximum width The five attributes together constitute the complete attribute node of each disease in the three-dimensional digital twin model: spatial location (three-dimensional coordinates in the world coordinate system), detection sequence (detection date and equipment number), and damage level (three levels: minor / moderate / severe, classified according to the comprehensive damage index D assessment results).

[0064] S63. After completing the installation of all cable-stayed bridge defects, perform attribution verification and deviation correction for each densely intersecting area of ​​multiple cables. Extract the world coordinates of each installed defect within that area. Calculate the perpendicular distance from the node to the central axis of each adjacent cable within the region, and determine the cable to which the defect belongs based on the smallest distance. If the verification result is inconsistent with the initially mounted cable, remove the defect attribute node from the original cable surface and re-attach it to the corresponding cable in S5. Transform the projection to the correct cable surface to complete attribute reloading. For cases where the defect mounting position has a slight offset along the cable axis due to accumulated coordinate transformation errors, use the image capture position and elevation angle corresponding to the defect frame. As a constraint, the theoretical offset of the defects along the cable axis is calculated in reverse, and the mounting position is axially corrected along the cable axis to eliminate the cumulative axial deviation. After all verification and correction are completed, all defects in the three-dimensional digital twin model are accurately located in the correct spatial position of their actual cable surface, thus completing the construction of the digital twin model of the cable-stayed bridge.

[0065] The aforementioned single-cable surface adaptive ICP registration scheme introduces a cylindrical curvature constraint term into the objective function, which constrains the solution space of the general ICP registration within the range that conforms to the geometric constraints of the cable-stayed cable cylinder. This effectively prevents registration offset caused by mutual interference of point sets in dense multi-cable areas and ensures the accuracy of the three-dimensional spatial attribution of the disease.

[0066] After the above steps are completed, intelligent applications are implemented on the dedicated digital twin smart management platform for cable-stayed bridges. The platform uses Web3D technology as its underlying 3D rendering engine and the Three.js framework to load a full-bridge 3D real-world model, supporting interactive operations such as free-roaming of the entire bridge, zooming in on individual cables, and rotation / scaling. The backend uses a relational database to structurally store all defect attributes, with the cable number as the primary key. Each defect record includes the defect number, the corresponding cable number, the defect category, the crack area S, the actual total length L, and the average width. Maximum width Fields such as 3D spatial coordinates, image acquisition path, detection time sequence, and damage level are displayed. When any disease annotation point is selected on the 3D model at the front end, the system automatically retrieves the complete attributes of the disease from the database and displays them in a pop-up window, while simultaneously displaying the corresponding high-definition disease image.

[0067] Based on this, a comprehensive assessment of the damage level of each disease is conducted, and the comprehensive damage index of a single disease is calculated using the following formula: ; Where D is the comprehensive damage index of the disease, and S and L are the area and length of a single disease site. , To correspond to the total exposed area and total length of a single cable sheath, K is the hazard coefficient of the type of defect (the hazard coefficient of cracks and exposed reinforcement is higher than that of surface peeling and corrosion). The normalized weighting coefficients are calibrated according to bridge maintenance specifications and meet the following requirements. The damage level is divided into three levels based on the D value: a low D value indicates a minor level, where the disease is in its nascent or early development stage, and routine regular monitoring is recommended; a D value in the middle range indicates a moderate level, where the disease has reached a certain scale, and it is recommended to include it in the recent maintenance plan; a D value exceeding the upper threshold indicates a severe level, where the disease has expanded significantly or has a high hazard factor, and emergency response and key early warning should be initiated immediately.

[0068] The platform integrates the following functional modules: a 3D walkthrough and disease spatial measurement module, allowing operators to freely walk in the 3D model and directly measure the spatial dimensions of diseases; a disease quantitative grading and retrieval module, which can filter and query by combination of multi-dimensional conditions such as disease category, damage level, cable number, and detection time sequence; a multi-cycle time-series evolution comparison module, which overlays and compares the results of multiple detection periods in the same 3D coordinate system, automatically calculates the area and length evolution increments of diseases at each location, triggers intelligent early warnings for diseases exceeding the preset expansion rate threshold, and pushes them to the management end; a single cable full life cycle traceability module, which takes a single cable as the object and completely traces its detection records and disease development process; and a standardized detection report intelligent output module, which automatically generates standardized detection reports based on the full disease information and assessment results in the database, including detection overview, list of diseases by cable, distribution of damage levels, typical disease images and 3D spatial screenshots, and maintenance suggestions, supporting export in PDF and Word formats, realizing intelligent full life cycle management of cable-stayed cable surface damage that is visualized, quantifiable, and traceable.

[0069] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting apparent damage to cable-stayed bridges based on three-dimensional reconstruction and digital twins, characterized in that, include: S1. Control the UAV to perform global oblique photography of the cable-stayed bridge, and collect a global image dataset and corresponding pose data; control the cable-attaching aircraft to fly spirally along the surface of each cable, and collect a refined image dataset of a single cable. S2. Using topological constraints and multi-view stereo vision reconstruction methods, the full-domain image dataset and pose data are used to perform 3D reconstruction. Combined with cylindrical adaptive texture mapping, a 3D real-scene model of the entire cable-stayed bridge is constructed. S3. Perform local gain texture enhancement processing on each frame of the single-crystal refined image dataset, and use the improved YOLOv8 network for disease detection to obtain the pixel coordinates and disease categories of each disease area. S4. For the pixel coordinates of each disease area, a pixel-level segmentation network is used to extract the disease outline, and the quantitative parameters of each disease are calculated by combining the cylindrical distortion correction algorithm. S5. Establish a five-level attitude-related coordinate transformation model between the pixel coordinate system, image coordinate system, camera coordinate system, world coordinate system and single-cable local coordinate system, and transform the pixel coordinates of each disease area into a single-cable local three-dimensional coordinate point set step by step. S6. The local three-dimensional coordinate point set of a single cable is registered with the corresponding cable surface in the three-dimensional real scene model of the whole bridge using a point cloud registration algorithm with surface curvature constraints. The disease category, quantification parameters, spatial location, detection time sequence and damage level of each disease are loaded into the three-dimensional real scene model of the whole bridge to construct a digital twin model of the cable-stayed bridge.

2. The method according to claim 1, characterized in that, Step S1 includes: S11. Calibrate the camera intrinsic parameters of the UAV and the cable-attached aircraft, determine the focal length, pixel size and photosensitive size, and set the overall aerial survey heading overlap to be no less than 80%, the lateral overlap to be no less than 70% and the corresponding flight path spacing. S12. Control the drone to plan a full-coverage grid flight path, collect multi-angle oblique photographic images of the entire cable-stayed bridge, and simultaneously record the pose vector corresponding to each frame of image in real time by the POS positioning and attitude determination system. Where B is the latitude of the data collection point, L is the longitude of the data collection point, and H is the absolute elevation of the data collection point. For camera tilt angle, For camera roll angle, This refers to the camera yaw angle; S13. The aircraft is controlled to plan a vertical spiral following route based on the central axis of a single cable. The aircraft maintains a constant distance of 0.5m to 1m from the cable and collects detailed images of each cable from the anchorage end of the bridge tower to the anchorage end of the main beam. S14. Perform quality screening on the two types of acquired images, removing invalid images caused by flight shaking, strong light overexposure, backlight shadows, obstruction by debris, and blurring or defocusing. Filter and denoise the original POS pose data and remove abnormal jump values. Complete the matching of timestamps and spatial coordinates between the images and pose data, and establish a global image dataset and a single-index refined image dataset.

3. The method according to claim 2, characterized in that, Step S2 includes: S21. Based on the global image dataset and corresponding pose data, perform motion reconstruction structure calculation and optimize the exterior orientation elements of each frame image. S22, using vertically adjacent three-dimensional feature points , elevation value , Calculate the elevation constraint function: ； in To preset the vertical elevation continuous threshold, the subscripts a and a+1 represent the three-dimensional feature point index; when When determining two points as feature points of a continuous cable body; S23. Generate a full-bridge dense point cloud using the MVS multi-view dense matching algorithm, and generate a continuous grid model using the triangular mesh reconstruction algorithm. S24. Use the cylindrical texture mapping distortion correction formula. The texture coordinates are corrected, where u and v are the horizontal and vertical coordinates of the original texture pixels in the planar mapping. R is the texture pixel coordinate after cylindrical correction, R is the radius of the cable stay, and y is the texture pixel ordinate. The texture of each cable body surface is baked to construct a three-dimensional real scene model of the entire bridge.

4. The method according to claim 1, characterized in that, In step S3, when applying local gain texture enhancement processing to each frame of the single-frame refined image dataset, the enhanced grayscale value of each pixel is calculated according to the following formula: ； in , Image coordinates Gray values ​​before and after pixel enhancement The average grayscale value of pixels in the ROI region of the object. Let be the grayscale variance of the cable body region, and d be the vertical distance from the current pixel to the central axis of the cable body. This is the gain attenuation coefficient.

5. The method according to claim 4, characterized in that, In step S3, the improved YOLOv8 network adds a shallow fine-grained feature weighting and fusion module to the original YOLOv8 network, which performs fine-grained feature mapping on the output of k shallow feature layers. Weighted fusion is performed using the following formula: ； in This is the feature map after multi-scale feature fusion. represents the adaptive weight coefficients for each feature layer, and j is the index of the shallow feature layer.

6. The method according to claim 1, characterized in that, In step S4, for the pixel coordinates of each diseased area, a pixel-level segmentation network is used to extract the disease contour, including: The image regions corresponding to the pixel coordinates of each diseased area are input into the Dense-DeepLabv3+ network, and features are extracted using a multi-scale dilated convolutional spatial pyramid pooling structure. The output is a binary crack segmentation map: ； in , , These are dilated convolutions with dilation rates of 1, 3, and 5, respectively. For global pooling features, This is a channel concatenation operation, where F is the input feature map; Skeletonization is performed on the crack segmentation binary image output by the network, and the skeleton pixels at the center of the crack are extracted. According to the coordinate mapping formula for cylindrical distortion correction: ； Image coordinates Mapped to three-dimensional coordinates of the cable cylinder Where R is the radius of the stay cable. The central angle of the cylinder corresponding to the pixel; The quantitative parameters of each disease are calculated in three-dimensional cylindrical coordinates.

7. The method according to claim 6, characterized in that, Crack quantification parameters are calculated using the following formula: Total area of ​​cracks The actual total length of the crack Average crack width Maximum crack width Where N is the total number of valid pixels in the binary image of the crack segmentation. Here, GSD represents the total number of pixels in the crack skeleton, and GSD represents the ground resolution during single-cable fine image acquisition. Let s = 1, 2, ..., m, where m is the number of crack segments.

8. The method according to claim 2, characterized in that, Step S5 includes: S51, according to pixel row and column numbers Convert to image coordinates ,in The row and column numbers of the principal pixel in the image. The physical dimensions of a single pixel are defined horizontally and vertically. S52. Convert the image coordinates to three-dimensional coordinates in the camera coordinate system according to the camera intrinsic parameter matrix. The camera's focal length is f; S53, with pitch angle Roll angle Yaw angle Construct a rotation matrix, and a translation matrix using latitude B, longitude L, and absolute elevation H to convert camera coordinates to world coordinates; S54. Establish a local coordinate system for each cable-stayed cable, convert the world coordinates to local three-dimensional coordinates for each cable, complete the coordinate system alignment, and obtain the local three-dimensional coordinate point set of each cable in each diseased area; and synchronously transform the feature point set of each cable surface from the full bridge three-dimensional real scene model to the corresponding local coordinate system of each cable.

9. The method according to claim 8, characterized in that, Step S6 includes: S61. For the local three-dimensional coordinate point set of each cable and the corresponding cable surface feature point set in the three-dimensional real-world model of the entire bridge, iterative registration is performed using the following formula as the objective function: ； in For the total registration error, For the feature point set of the cable body surface, Let R and T be the local 3D coordinate points of a single-line system, and R and T be the rotation and translation matrices to be optimized. For the curvature constraint term of the cable surface, To balance the weighting coefficients, n is the number of points involved in the registration; S62. Based on the optimal rotation matrix R and translation matrix T, the three-dimensional spatial position of each disease after registration is transformed back to the world coordinate system through the inverse transformation from the local coordinate system of the single cable to the world coordinate system. The disease category, quantitative parameters, spatial position, detection time sequence and damage level of each disease are then mounted to the corresponding cable surface position in the three-dimensional real scene model of the whole bridge. S63. Verify the mounting position of each cable-stayed bridge defect one by one, correct the defect attribution deviation in the multi-cable intersection area, and construct a digital twin model of the cable-stayed bridge.

10. The method according to claim 2, characterized in that, In step S1, when the cable-attaching aircraft spirals along the surface of each cable, it operates according to the ground resolution of the refined single-cable image acquisition. To control the acquisition accuracy, where H is the vertical flight altitude of the cable-attached aircraft relative to the cable surface. f is the physical size of a single pixel in the camera, and f is the focal length of the camera lens.

Images