Image recognition-based cable defect positioning method and system

Abstract

A method and system for locating defects in cable-stayed bridges based on image recognition is disclosed, relating to general image data processing or generation. In this method, the spiral features of the cable-stayed bridge in consecutive video frames are extracted, and a sequence of equally spaced virtual nodes is generated by combining these features with a reference pixel spacing. The cumulative pixel displacement of the current frame is obtained by matching the translation differences between virtual nodes in adjacent frames. When a defect is identified, the intra-frame pixel offset of the defect relative to the first virtual node is calculated. Finally, the cumulative displacement and the intra-frame offset are added to obtain the total pixel coordinates of the defect, and calibration parameters are used to convert them into absolute positioning coordinates in physical space. This application aims to suppress missed and false detections during image feature extraction, reduce accumulated errors in visual distance calculation, and thus improve the accuracy of locating cable-stayed bridge defects in physical space.

Description

Technical Field

[0001] This application belongs to the general field of image data processing or generation, and in particular relates to a method and system for locating defects in cable-stayed bridges based on image recognition. Background Technology

[0002] As the core load-bearing component of cable-stayed bridges, stay cables are exposed to the natural environment for a long time. The PE protective sleeves on their surfaces are prone to defects such as scratches, cracks or peeling due to wind and rain erosion, ultraviolet radiation and alternating stress. Therefore, it is necessary to conduct surface defect inspections on stay cables regularly.

[0003] A related technology proposes a defect localization technique for stay cables based on visual counting of surface periodic textures. Modern stay cables typically have spiral grooves with fixed physical spacing pressed into the PE protective sleeve surface during manufacturing to suppress rain and wind vibrations. This technology utilizes a drone or cable-climbing robot equipped with a high-definition camera that moves at a constant speed along the stay cable and continuously captures surface images. In the image processing stage, an edge detection algorithm identifies the continuously appearing spiral groove features in the image. Since the physical spacing between two adjacent spirals is a known fixed constant, the system calculates the actual physical distance the camera moves along the stay cable by accumulating the number of spirals passing the frame baseline in consecutive video frames and multiplying this number by the fixed physical spacing. When the image recognition model detects a surface defect, it directly uses the currently accumulated physical distance as the location coordinates of the defect.

[0004] Airborne dust and vehicle exhaust particles adhere to the surface of stay cables over long periods. Because the cables are inclined, rainwater flowing down the cable body washes away dirt from raised surfaces, but also carries away large amounts of silt and particles, depositing them in the recessed spiral grooves. Over time, this uneven deposition causes some sections of the shallow spiral grooves to be completely filled with hardened dirt, losing their original shadow contrast. Simultaneously, irregular water stains and dark dirt streaks left by rainwater create noticeable oblique pseudo-edges on the cable surface. When using related technologies to extract image features from these stay cables, edge detection algorithms may miss detections due to the decay of the true visual features of the spiral grooves. Furthermore, they are prone to misidentifying high-contrast dirt streaks as spiral lines, reducing the continuity and accuracy of visual counting and thus decreasing the accuracy of locating stay cable defects in physical space. Summary of the Invention

[0005] This application provides a method and system for locating cable defects based on image recognition, which is used to suppress missed detections and false detections in the image feature extraction process, reduce the cumulative error in visual distance accumulation calculation, and thus improve the accuracy of locating cable defects in physical space.

[0006] In the first aspect, this application provides a method for locating defects in cable-stayed bridges based on image recognition, which extracts linear features that match the inclination angle of the cable-stayed bridge spiral in continuous video frames captured by a camera moving along the cable-stayed bridge and performs one-dimensional axial projection to obtain a set of one-dimensional candidate positions.

[0007] Obtain the physical spacing of the cable-stayed cable spirals and map it to the baseline pixel spacing in the image coordinate system;

[0008] Traverse the one-dimensional candidate location set and extract the target subset containing the most pixel coordinates as the consensus anchor point set. The difference between the ratio of the pixel distance between any two pixel coordinates in the consensus anchor point set and the benchmark pixel distance and the positive integer closest to the ratio is less than the preset pixel error tolerance.

[0009] Using the pixel coordinates in the consensus anchor point set as the reference point and the reference pixel spacing as the step size, interpolate to generate an equally spaced virtual node sequence that coincides with the consensus anchor point set in the one-dimensional image space of the current video frame;

[0010] Match the translation pixel difference between the virtual node sequences of two adjacent video frames, and accumulate all translation pixel differences from the initial video frame to the current video frame in a continuous video frame to obtain the cumulative pixel displacement of the current video frame;

[0011] In response to the identification of surface defects on the cable-stayed bridge in the current video frame, the intra-frame pixel offset of the defect between the one-dimensional projected coordinates of the surface defect and the first virtual node in the virtual node sequence of the current video frame is calculated.

[0012] The total pixel coordinates of the defect are obtained by adding the cumulative pixel displacement to the pixel offset within the defect frame.

[0013] Based on the calibration parameters of the camera equipment, the total pixel coordinates of the defects are converted into the absolute positioning coordinates of the surface defects in the physical space of the cable-stayed bridge.

[0014] By employing the above technical solution, linear features matching the inclination angle of the cable-stayed bridge spiral are extracted and subjected to one-dimensional axial projection, reducing the dimensionality of two-dimensional image information to one-dimensional space for processing, thus reducing computational complexity. A mapping relationship between physical space and image space is established by obtaining the baseline pixel spacing of the spiral's physical spacing in the image coordinate system. The set of one-dimensional candidate positions is traversed, and a subset of targets conforming to integer multiples of the baseline pixel spacing is extracted as a consensus anchor point set. The periodicity of the spiral is utilized to improve the accuracy of anchor point identification. An equally spaced virtual node sequence is generated by interpolation using the consensus anchor point set as a reference, constructing a unified measurement reference coordinate system. Continuous displacement tracking during camera movement is achieved by matching and accumulating the translation differences between virtual node sequences of adjacent video frames, reducing the cumulative error of single-frame positioning. The total pixel coordinates of the defect are obtained by adding the accumulated pixel displacement to the pixel offset within the defect frame, completing the global positioning of the defect in the continuous video sequence. This approach suppresses missed and false detections during image feature extraction and reduces the cumulative error in visual distance accumulation calculation, thereby improving the accuracy of locating cable-stayed bridge defects in physical space.

[0015] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes, prior to identifying surface defects in the cable-stayed bridge in the current video frame:

[0016] Based on the known manufacturing inclination angle of the cable-stayed bridge spiral, the virtual node sequence of the current video frame is inversely mapped to the two-dimensional image space to generate a structure suppression mask;

[0017] The grayscale image of the current video frame is subjected to pixel-level difference operation with the structure suppression mask to obtain the structure residual image;

[0018] Calculate the gradient magnitude of each pixel in the structural residual image in multiple preset directions, and construct the directional gradient histogram of each pixel;

[0019] The local gradient direction entropy of each pixel is calculated based on the histogram of directional gradients. The local gradient direction entropy is the information entropy that characterizes the uniformity of gradient direction distribution in the neighborhood of a pixel. The value of the local gradient direction entropy is positively correlated with the degree of dispersion of the gradient direction.

[0020] Extract the set of pixels whose local gradient direction entropy is greater than a preset entropy threshold and perform connected component analysis. The connected component with the largest area is taken as the surface defect of the cable-stayed bridge.

[0021] By employing the above technical solution, a structure suppression mask is generated by inversely mapping the virtual node sequence to a two-dimensional image space based on the known manufacturing inclination angle of the cable-stayed bridge spiral, utilizing the geometric regularity of the spiral. A structural residual image is obtained by performing pixel-level difference operations between the grayscale image and the structure suppression mask, suppressing the interference of regular spiral textures and enhancing the contrast between defects and the background. The gradient magnitude of each pixel in the structural residual image in multiple preset directions is calculated, and an directional gradient histogram is constructed to extract the gradient direction distribution information of the pixel's neighborhood. Local gradient direction entropy is calculated based on the directional gradient histogram. This entropy value reflects the dispersion of the gradient direction; regular spiral textures exhibit low entropy values, while defect areas exhibit high entropy values ​​due to the diversity of edge directions, improving the distinguishability between defects and normal structures. The set of pixels with local gradient direction entropy greater than a preset threshold is extracted and connected component analysis is performed. The connected component with the largest area is identified as a surface defect, reducing the false detection rate.

[0022] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes, prior to identifying surface defects in the cable-stayed bridge in the current video frame:

[0023] Along the axis of the cable-stayed bridge, the current video frame is divided into multiple periodic image blocks with a length equal to the reference pixel spacing;

[0024] Median filtering is applied to the pixel grayscale values ​​of all periodic image blocks at the same relative coordinate position to reconstruct the baseline background template.

[0025] Calculate the pixel-level absolute difference between each periodic image patch and the baseline background template, and stitch them together to generate an anomaly saliency map with the same size as the current video frame.

[0026] An adaptive threshold segmentation is performed on the anomaly saliency map to obtain a binarized image, and candidate connected components are extracted from the binarized image;

[0027] Calculate the geometric compactness of each candidate connected region, and identify the candidate connected regions whose geometric compactness is within a preset range as surface defects of the cable-stayed bridge. The geometric compactness is the ratio of the area of ​​the candidate connected region to the area of ​​the smallest circumcircle surrounding the candidate connected region.

[0028] By employing the above technical solution, the video frame is divided into multiple periodic image blocks with a length equal to the reference pixel spacing along the cable-stayed bridge axis, utilizing the periodic structural features of the spiral. Median filtering is applied to the pixel grayscale values ​​at the same relative coordinate positions of all periodic image blocks to reconstruct the reference background template. Multi-period sampling statistics suppress the influence of random noise and isolated outliers, improving the representativeness of the background template. The pixel-level absolute difference between each periodic image block and the reference background template is calculated and stitched together to generate an anomaly saliency map, enhancing the saliency of defect regions deviating from the periodic pattern. Adaptive threshold segmentation is performed on the anomaly saliency map to obtain a binarized image and extract candidate connected components, adapting to the influence of different regional illumination variations. The geometric compactness of each candidate connected component is calculated. This index reflects the regularity of the connected component's shape; defects typically exhibit relatively compact morphological features, while noise or false defects often exhibit scattered or elongated shapes. Candidate connected components with geometric compactness within a preset range are identified as surface defects, further reducing the false detection rate and improving the accuracy of defect identification.

[0029] In conjunction with some embodiments of the first aspect, in some embodiments, after converting the total pixel coordinates of the defects into the absolute positioning coordinates of the surface defects in the physical space of the cable-stayed bridge, the method further includes:

[0030] Obtain the physical installation offset angle of the target camera device in the circumferential detection array; the target camera device is the camera device that captures surface defects.

[0031] Based on the image features of the surface defect in the current video frame and the physical installation offset, calculate the global central angle of the surface defect on the circular section of the cable-stayed bridge.

[0032] Using the absolute positioning coordinates as the axial coordinates, and combining the global central angle with the radius of the cable, a three-dimensional cylindrical coordinate system for surface defects is constructed.

[0033] Obtain the three-dimensional cylindrical coordinates output by different camera devices in the circumferential detection array within the same detection batch;

[0034] Based on the spatial distribution relationship between the three-dimensional cylindrical coordinates, the three-dimensional cylindrical coordinates belonging to the same physical defect are spatially fused to output the unique three-dimensional spatial coordinates of the surface defect after deduplication.

[0035] By employing the above technical solution, the physical installation offset angle of the target camera equipment in the circumferential detection array was obtained, establishing the correspondence between the camera equipment's observation angle and the circumferential position of the cable-stayed bridge. Based on the image features of the surface defect in the video frame and the physical installation offset angle, the global central angle of the defect on the circular cross-section of the cable-stayed bridge was calculated, realizing the conversion from two-dimensional image coordinates to a three-dimensional cylindrical coordinate system for circumferential angles. Using the absolute positioning coordinates as axial coordinates, combined with the global central angle and the cable-stayed bridge radius, a three-dimensional cylindrical coordinate system was constructed, completing the three-dimensional spatial positioning of the defect. The three-dimensional cylindrical coordinates output by different camera equipment within the same inspection batch were obtained, utilizing the redundant information from multi-view observations. Based on the spatial distribution relationship between the three-dimensional cylindrical coordinates, the coordinates belonging to the same physical defect were spatially fused. Cross-validation of multi-source data improved the positioning accuracy, and the output of unique, deduplicated three-dimensional spatial coordinates avoided duplicate counting, reduced defect statistical errors, and improved the reliability and practicality of the detection system.

[0036] In conjunction with some implementations of the first aspect, in some implementations, the global central angle of the surface defect on the circular section of the cable-stayed bridge is calculated based on the image features of the surface defect in the current video frame and the physical installation offset angle, specifically including:

[0037] Determine the lateral feature reference points of surface defects in the current video frame;

[0038] Based on the imaging projection relationship of the target camera device, the lateral feature reference points are converted into local deflection angles of surface defects relative to the optical axis of the target camera device;

[0039] By superimposing the physical installation deviation angle and the local deviation angle, the global central angle of the surface defect on the circular section of the cable is obtained.

[0040] By employing the aforementioned technical solution, a mapping starting point for the defect image features and their circumferential position on the cable-stayed bridge's circular cross-section was established by determining the lateral feature reference point of the surface defect in the current video frame. Based on the imaging projection relationship of the target camera, the lateral feature reference point was converted into a local deflection angle of the surface defect relative to the camera's optical axis. Utilizing the geometric perspective principle of camera imaging, the two-dimensional image coordinates were converted into angular information with the camera as the observation reference. The physical installation deflection angle and the local deflection angle were superimposed to obtain the global central angle, completing the coordinate transformation from the local observation coordinate system of the camera to the global cylindrical coordinate system of the cable-stayed bridge. This step-by-step transformation method decomposes the complex three-dimensional spatial positioning problem into two relatively independent sub-problems: local deflection angle calculation and global coordinate transformation, reducing computational complexity. Through angle superposition operations, the observation data from different camera devices were represented in a unified global coordinate system, improving the comparability and fusion accuracy of multi-view data and enhancing the accuracy of three-dimensional positioning.

[0041] In conjunction with some implementations of the first aspect, in some implementations, after outputting the unique three-dimensional spatial coordinates of the surface defects after deduplication, the method further includes:

[0042] Obtain the two-dimensional defect contour features output by each target camera device associated with a unique three-dimensional spatial coordinate;

[0043] Using a unique three-dimensional spatial coordinate as a spatial anchor point, the two-dimensional defect contour features are projected onto a preset three-dimensional surface model of the cable-stayed bridge to obtain the corresponding three-dimensional contour fragment.

[0044] Based on the topological overlap relationship of the three-dimensional contour segments on the three-dimensional surface model of the cable-stayed bridge, the boundary of the three-dimensional contour segments is fused to obtain the fused complete three-dimensional contour.

[0045] Extract and fuse the complete 3D contour to generate the physical size parameters of surface defects.

[0046] By employing the aforementioned technical solution, two-dimensional defect contour features output by each target camera device associated with a unique three-dimensional spatial coordinate are obtained, fully utilizing the complementary information provided by multi-view observation. Using the unique three-dimensional spatial coordinate as a spatial anchor point, the two-dimensional defect contour features are projected onto a pre-defined three-dimensional surface model of the cable-stayed bridge to obtain three-dimensional contour fragments. This achieves contour reconstruction from two-dimensional image space to three-dimensional physical space, allowing the local contours of defects observed from different perspectives to be represented on a unified three-dimensional model. Based on the topological overlap relationship of the three-dimensional contour fragments on the three-dimensional surface model of the cable-stayed bridge, boundary fusion of the contour fragments is performed to obtain a fused complete three-dimensional contour. Spatial stitching of multi-view contours compensates for the lack of contour information caused by occlusion or observation angle limitations from a single perspective, improving the completeness of contour reconstruction. The physical size parameters of the surface defects generated by the fused complete three-dimensional contour are extracted, enabling quantitative measurement of defect geometric features. Compared to single-view measurement, multi-view fusion reduces the size measurement deviation caused by the observation angle, improving the measurement accuracy and reliability of defect size parameters.

[0047] In conjunction with some implementation methods of the first aspect, in some implementation methods, based on the topological overlap relationship of the three-dimensional contour segments on the three-dimensional surface model of the cable-stayed bridge, boundary fusion is performed on the three-dimensional contour segments to obtain a fused complete three-dimensional contour, specifically including:

[0048] Analyze the topological overlap relationship to determine the intersection nodes of the three-dimensional contour segments on the three-dimensional surface model of the cable-stayed bridge, as well as the overlapping internal line segments and non-overlapping external line segments divided by the intersection nodes;

[0049] Eliminate overlapping internal line segments and splice non-overlapping external line segments at intersection nodes to generate the initial envelope boundary.

[0050] Based on a preset curve smoothing algorithm, the splicing segments of the initial envelope boundary at the intersection nodes are smoothed to obtain a fused complete 3D contour.

[0051] By employing the aforementioned technical solution, the intersecting nodes of the 3D contour segments on the 3D surface model of the cable-stayed bridge are determined by analyzing the topological overlap relationship. This, along with the overlapping internal line segments and non-overlapping external line segments divided by these intersection nodes, enables precise identification of the spatial relationships between multiple contour segments. Overlapping internal line segments are removed, and non-overlapping external line segments are spliced ​​end-to-end at the intersection nodes to generate the initial envelope boundary. This preserves the non-redundant and effective information of each contour segment, avoids the influence of repeated regions on the fusion result, and improves the accuracy of contour fusion. Based on a preset curve smoothing algorithm, the spliced ​​segments of the initial envelope boundary at the intersection nodes are smoothly transitioned to obtain a fused complete 3D contour. This eliminates the discontinuities and sharp transitions caused by direct splicing, making the fused contour more consistent with the continuity characteristics of real defect boundaries. The smoothing process reduces the interference of splicing marks on subsequent dimensional parameter extraction, improves the accuracy of physical dimensional parameters calculated based on the fused contour, and enhances the reliability and practicality of the measurement results.

[0052] In a second aspect, embodiments of this application provide an image recognition-based cable-stayed bridge defect location system, which includes: one or more processors and a memory; the memory is coupled to one or more processors, the memory is used to store computer program code, the computer program code includes computer instructions, and one or more processors call the computer instructions to cause the system to perform the method described in the first aspect and any possible implementation thereof.

[0053] Thirdly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a system, cause the system to perform the method described in the first aspect and any possible implementation thereof.

[0054] Fourthly, embodiments of this application provide a computer program product that, when run on a system, causes the system to execute the method described in any possible implementation of the first aspect.

[0055] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0056] 1. This application provides an image recognition-based method for locating defects in cable-stayed bridges. By extracting linear features matching the inclination angle of the cable-stayed bridge spiral and performing one-dimensional axial projection, the two-dimensional image information is reduced to a one-dimensional space for processing, thus reducing computational complexity. A mapping relationship between physical space and image space is established by obtaining the baseline pixel spacing of the spiral's physical spacing in the image coordinate system. The one-dimensional candidate position set is traversed, and a subset of targets conforming to integer multiples of the baseline pixel spacing is extracted as a consensus anchor point set. The periodicity of the spiral is utilized to improve the accuracy of anchor point identification. An equally spaced virtual node sequence is generated by interpolation using the consensus anchor point set as a reference, constructing a unified measurement reference coordinate system. By matching and accumulating the translation differences of virtual node sequences in adjacent video frames, continuous displacement tracking during camera movement is achieved, reducing the cumulative error of single-frame positioning. The total pixel coordinates of the defect are obtained by adding the accumulated pixel displacement to the pixel offset within the defect frame, completing the global positioning of the defect in the continuous video sequence. It suppresses the phenomena of missed detection and false detection in the image feature extraction process, reduces the cumulative error in visual distance accumulation calculation, and thus improves the accuracy of locating cable defects in physical space.

[0057] 2. This application provides an image recognition-based method for locating defects in cable-stayed bridges. It obtains the physical installation offset angle of the target camera device in a circumferential detection array and establishes the correspondence between the camera's viewing angle and the circumferential position of the cable-stayed bridge. Based on the image features of the surface defect in the video frame and the physical installation offset angle, the global central angle of the defect on the circular cross-section of the cable-stayed bridge is calculated, realizing the conversion from two-dimensional image coordinates to a three-dimensional cylindrical coordinate system for circumferential angles. The absolute positioning coordinates are used as axial coordinates, combined with the global central angle and the cable-stayed bridge radius to construct a three-dimensional cylindrical coordinate system, completing the three-dimensional spatial location of the defect. The three-dimensional cylindrical coordinates output by different camera devices within the same inspection batch are obtained, utilizing the redundant information from multi-view observations. Based on the spatial distribution relationship between the three-dimensional cylindrical coordinates, the coordinates belonging to the same physical defect are spatially fused. Cross-validation of multi-source data improves the positioning accuracy. Outputting unique, deduplicated three-dimensional spatial coordinates avoids duplicate counting, reduces defect statistical errors, and improves the reliability and practicality of the detection system.

[0058] 3. This application provides an image recognition-based method for locating defects in cable-stayed bridges. It acquires the two-dimensional defect contour features output by various target cameras associated with unique three-dimensional spatial coordinates, fully utilizing the complementary information provided by multi-view observations. Using the unique three-dimensional spatial coordinates as spatial anchor points, the two-dimensional defect contour features are projected onto a pre-defined three-dimensional surface model of the cable-stayed bridge to obtain three-dimensional contour segments. This achieves contour reconstruction from two-dimensional image space to three-dimensional physical space, allowing the local contours of defects observed from different perspectives to be represented on a unified three-dimensional model. Based on the topological overlap relationship of the three-dimensional contour segments on the three-dimensional surface model of the cable-stayed bridge, boundary fusion is performed on the contour segments to obtain a fused complete three-dimensional contour. Spatial stitching of multi-view contours compensates for the lack of contour information caused by occlusion or observation angle limitations from a single perspective, improving the completeness of contour reconstruction. The physical size parameters of the surface defects generated by the fused complete three-dimensional contour are extracted, enabling quantitative measurement of the defect's geometric features. Compared to single-view measurement, multi-view fusion reduces the size measurement deviation caused by the observation angle, improving the measurement accuracy and reliability of defect size parameters. Attached Figure Description

[0059] Figure 1 This is a flowchart illustrating a method for locating defects in a cable-stayed bridge based on image recognition, as described in an embodiment of this application.

[0060] Figure 2 This is another flowchart illustrating a method for locating defects in cable-stayed bridges based on image recognition, as described in this application.

[0061] Figure 3 This is another flowchart illustrating a method for locating defects in cable-stayed bridges based on image recognition, as described in this application.

[0062] Figure 4 This is a schematic diagram of the physical device structure of a cable-stayed bridge defect location system based on image recognition, provided in an embodiment of this application. Detailed Implementation

[0063] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to any or all possible combinations including one or more of the listed items.

[0064] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0065] The following example is used in conjunction with Figure 1 The present application describes a method for locating defects in cable stays based on image recognition.

[0066] Please see Figure 1 This is a flowchart illustrating a method for locating defects in a cable-stayed bridge based on image recognition, as described in an embodiment of this application.

[0067] S101. Extract the linear features that match the inclination angle of the cable spiral in the continuous video frames captured by the camera moving along the cable and perform one-dimensional axial projection to obtain a one-dimensional candidate position set.

[0068] Camera equipment refers to a visual sensor used to acquire image information of the surface of a cable-stayed bridge, and is not limited to high-definition cameras mounted on drones, cable-climbing robots, or handheld devices. A continuous video frame refers to a sequence of two-dimensional images continuously acquired by the camera equipment at a certain frame rate during movement. The helix angle of a cable-stayed bridge refers to the fixed angle formed between the helical texture formed on the surface of the cable-stayed bridge due to the manufacturing process and the central axis of the cable-stayed bridge. Linear features refer to the set of pixels that appear as edges or lines with a certain length and direction in an image. One-dimensional axial projection refers to compressing and mapping feature points on a two-dimensional image plane onto a straight line parallel to the axis in a direction perpendicular to the central axis of the cable-stayed bridge. The one-dimensional candidate position set refers to the group of coordinate points that, after projection, form a one-dimensional straight line and can all possibly represent the position of the helix. The system processes the input video stream frame by frame, identifies texture lines that conform to specific angles, and reduces the dimensionality of these two-dimensional line information, projecting them onto a one-dimensional coordinate axis, thereby transforming complex two-dimensional image features into a one-dimensional position distribution sequence that is easy to calculate.

[0069] This step can be implemented using a combined algorithm based on edge detection and Hough transform. First, the system uses an edge detection operator to perform convolution on the current video frame, extracting all edge pixels with significant grayscale changes. These edge pixels are then input into the Hough transform algorithm to find straight line models in the parameter space. The system sets an angular constraint range corresponding to the inclination angle of the cable-stayed bridge spiral, retaining only straight line segments falling within this range as extracted linear features. Finally, the pixels on these straight line segments undergo coordinate transformation along a direction perpendicular to the cable-stayed bridge's central axis, accumulating them onto a one-dimensional coordinate axis to form a one-dimensional candidate position set. Alternatively, the system can also achieve this by constructing a two-dimensional weighted filter with a specific direction. The system designs a direction-selective filter kernel based on the known inclination angle of the cable-stayed bridge spiral, and uses this filter kernel to perform sliding window convolution on consecutive video frames. Since the direction of the filter kernel is consistent with the spiral inclination angle, texture regions in the image matching this inclination angle will produce extremely high response values, while background or defect regions will have lower response values. The system extracts pixel regions whose response values ​​exceed a set threshold as linear features, and calculates the centroid or peak points of these regions in the same direction perpendicular to the axis, mapping them to a one-dimensional space to generate a set of one-dimensional candidate locations.

[0070] S102. Obtain the reference pixel spacing in the image coordinate system by mapping the physical spacing of the cable-stayed spiral.

[0071] Physical spacing refers to the actual distance between two adjacent spiral lines on the surface of a stay cable along its axis in real three-dimensional physical space, typically determined by the cable's manufacturing specifications. An image coordinate system is a two-dimensional Cartesian coordinate system established with a vertex of a video frame as the origin and pixels as the unit. Reference pixel spacing refers to the number of pixels occupied by physical distances in the real world in a two-dimensional digital image after being imaged through the optical lens of a camera. The system establishes a scale mapping relationship between the real physical world and the digital image world, transforming known physical dimensions into pixel scale reference values ​​that can be directly used in computer vision processing, providing a fundamental metric for subsequent distance calculations, feature matching, and the final physical coordinate system transformation.

[0072] This step can be achieved through geometric projection calculation based on the calibration of the camera's intrinsic and extrinsic parameters. The system pre-obtains the camera's internal parameters, such as focal length, principal point coordinates, and distortion coefficients, using a calibration board. It then combines this with a ranging sensor on the camera's platform to obtain the vertical physical distance from the camera's optical center to the cable surface. Utilizing a pinhole imaging model, the system combines the known spiral physical spacing with the vertical physical distance and focal length, directly calculating the pixel length corresponding to this physical spacing on the image sensor plane using the principle of similar triangles. This pixel length is then used as the baseline pixel spacing. Alternatively, the system can achieve this through proportional conversion based on a known-sized reference object within the image. The system extracts the upper and lower edge contours of the cable in the video frame and calculates the cable's pixel diameter in the image. Since the cable's physical diameter is known prior information, the system divides the cable's pixel diameter by its physical diameter to obtain the global pixel equivalent in the current video frame, i.e., the number of pixels per physical unit. Subsequently, the system multiplies the known spiral physical spacing by this global pixel equivalent to obtain the baseline pixel spacing mapped to the image coordinate system.

[0073] S103. Traverse the one-dimensional candidate location set and extract the target subset containing the most pixel coordinates as the consensus anchor point set.

[0074] The system traverses the one-dimensional candidate location set and extracts the target subset containing the most pixel coordinates as the consensus anchor point set. The ratio of the pixel distance between any two pixel coordinates in the consensus anchor point set to the baseline pixel spacing, and the difference between this ratio and the nearest positive integer, is less than a preset pixel error tolerance. The target subset refers to the combination of coordinate points selected from the one-dimensional candidate location set that satisfies specific distance constraints. The consensus anchor point set refers to the sequence of coordinate points with high periodic consistency, which, after rigorous screening, is deemed by the system to best represent the true spiral position. The preset pixel error tolerance refers to the maximum allowed pixel deviation when determining whether the distance between two coordinate points conforms to an integer multiple of the baseline pixel spacing. The system performs a comprehensive search within the one-dimensional candidate location set to find the largest point set such that the distance between any two points in this set is approximately equal to an integer multiple of the baseline pixel spacing. In this way, the system can effectively eliminate false feature points caused by image noise, surface stains, or uneven local illumination, retaining the most reliable spiral feature points as a solid foundation for subsequent calculations.

[0075] This step can be implemented using a longest effective sequence search algorithm based on dynamic programming. The system sorts the coordinates of the one-dimensional candidate location set in ascending order of position size, constructing a state transition array. The system iterates through each coordinate point, calculates its distance to all previous coordinate points, and checks if the remainder after dividing this distance by the baseline pixel spacing is less than a preset pixel error tolerance. If the condition is met, the state array is updated, recording the length of the longest effective sequence that the current point can form and its predecessor node. After traversal, the system backtracks to the path corresponding to the longest length recorded in the state array, extracts these coordinate points to form the target subset containing the largest number of pixel coordinates, and uses this subset as the consensus anchor point set. Alternatively, the system can also be implemented using a periodic model fitting algorithm based on the idea of ​​random sampling consensus. The system randomly selects two coordinate points from the candidate set, calculates the distance between them, and derives an initial phase offset to construct an ideal one-dimensional mesh model with a period of the baseline pixel spacing. Subsequently, the system calculates the distance from all remaining points in the candidate set to the nearest node of this ideal mesh model, and includes points whose distance is less than the preset pixel error tolerance in the inlier set. The system repeats the above random sampling and inlier counting process multiple times, and finally selects the inlier set corresponding to the iteration with the largest number of inliers as the target subset, i.e., the consensus anchor set.

[0076] S104. Using the pixel coordinates in the consensus anchor point set as the reference point and the reference pixel spacing as the step size, interpolate to generate an equally spaced virtual node sequence that coincides with the consensus anchor point set in the one-dimensional image space of the current video frame.

[0077] A reference point refers to a reliable coordinate point in the consensus anchor point set used to determine the overall spatial phase and starting position. Step size refers to the fixed pixel distance increment between two adjacent nodes during sequence generation. One-dimensional image space refers to a single-dimensional coordinate system established along the axis of a cable-stayed bridge. Interpolation is the mathematical process of generating new data points based on known discrete data points according to specific rules. An equally spaced virtual node sequence refers to a perfect set of periodic coordinate points constructed by the system, without any missing points or spacing fluctuations. Using the selected highly reliable anchor points as references, combined with known physical mapping spacing, the system reconstructs a uniformly distributed scale across the effective length of the entire video frame. This scale not only covers the original anchor point positions but also fills in blank areas where no features were extracted, providing a unified and standardized measurement benchmark for subsequent inter-frame displacement calculations.

[0078] This step can be implemented using a linear sequence generation algorithm based on global phase alignment. The system first calculates the remainders of all anchor point coordinates in the consensus anchor point set divided by the reference pixel spacing, and then averages these remainders to obtain the globally optimal phase offset. Subsequently, starting from the initial boundary of the current video frame's one-dimensional image space, the system, combining the calculated globally optimal phase offset and the reference pixel spacing, directly calculates the coordinates of all theoretical nodes covering the entire one-dimensional image space using an arithmetic sequence generation formula, forming a sequence of equally spaced virtual nodes. Because global phase is used, the generated sequence naturally maintains maximum overlap with the consensus anchor point set. Alternatively, the system can also be implemented using a piecewise interpolation extrapolation algorithm based on local anchor point constraints. The system divides the one-dimensional image space into an internal region and two external regions, using the leftmost and rightmost anchor points in the consensus anchor point set as boundaries. In the internal region, the system uses two adjacent real anchor points as endpoints, calculates the number of nodes to be inserted based on their distance and the reference pixel spacing, and performs uniform piecewise interpolation. In the outer regions on both sides, the system takes the leftmost and rightmost anchor points as the starting points and performs addition or subtraction extrapolation outward with a fixed step size of the reference pixel spacing until the image boundary is reached. Finally, the internal interpolation nodes and the external extrapolation nodes are spliced ​​together to form a complete sequence of equally spaced virtual nodes.

[0079] S105. Match the translation pixel difference between the virtual node sequences of two adjacent video frames, and accumulate all translation pixel differences from the initial video frame to the current video frame in the continuous video frames to obtain the cumulative pixel displacement of the current video frame.

[0080] Two adjacent video frames refer to the preceding and following frames that are closely connected in time sequence. The translation pixel difference refers to the coordinate offset of the same physical location in the coordinate system of two adjacent frames caused by the movement of the camera device along the cable-stayed bridge. The initial video frame is the first frame in the video sequence used to establish the origin of the global absolute coordinate system. The current video frame is the latest frame being processed by the system. The cumulative pixel displacement refers to the total distance traveled by the camera device at the image pixel scale from the initial state to the current state. By comparing the relative positional changes of the standardized virtual node sequences constructed in the preceding and following frames, the system accurately calculates the movement distance of the camera device within this extremely short time interval. These minute movement distances are then accumulated frame by frame, thereby achieving continuous tracking of the global motion trajectory of the camera device and establishing a macroscopic spatial connection between the current frame and the initial frame.

[0081] This step can be implemented using a signal matching algorithm based on a one-dimensional cross-correlation function. The system converts the virtual node sequences of two adjacent video frames into one-dimensional discrete pulse signals, where the signal amplitude at the node location is at its maximum, and zero at other locations. The system performs a sliding cross-correlation operation on these two one-dimensional discrete signals in the spatial domain, calculating the cross-correlation coefficient under different relative displacements. The system finds the spatial displacement corresponding to the maximum peak value of the cross-correlation coefficient and uses this as the translation pixel difference between the two adjacent frames. Subsequently, the system maintains a global accumulator variable, sums the calculated translation pixel difference for each frame, and updates the value of the accumulator variable to obtain the cumulative pixel displacement of the current video frame. Alternatively, the system can also be implemented using a fast estimation algorithm based on frequency domain phase correlation. The system performs a one-dimensional fast Fourier transform on the virtual node sequences of two adjacent frames, transforming them from the spatial domain to the frequency domain. The system calculates the cross-power spectrum of the two spectra in the frequency domain and performs an inverse Fourier transform on it to obtain the phase correlation function in the spatial domain. The system extracts the pulse peak position in this function, which directly corresponds to the translation pixel difference between the two frames. Similarly, the system adds this difference to the total historical displacement to calculate the cumulative pixel displacement since the initial video frame.

[0082] S106. In response to identifying surface defects of the cable-stayed bridge in the current video frame, calculate the intra-frame pixel offset of the surface defect between the one-dimensional projected coordinates of the surface defect and the first virtual node in the virtual node sequence of the current video frame.

[0083] Surface defects refer to abnormal structural states such as cracks, scratches, damage, or bulges on the outer protective layer of the cable-stayed bridge. One-dimensional projected coordinates refer to the position of the center or edge point of the detected two-dimensional defect area compressed and mapped onto a one-dimensional image space along a direction perpendicular to the axis. The first virtual node refers to the virtual node with the smallest coordinate value in the one-dimensional image space of the current video frame, usually serving as the reference zero point of the local coordinate system for that frame. The pixel offset within the defect frame refers to the pixel distance of the defect's position in one-dimensional space relative to the local reference zero point of that frame. While continuously tracking the displacement of the camera equipment, the system runs the defect detection module in parallel. Once an abnormal target is detected, its two-dimensional position information is immediately converted into one-dimensional coordinates of the same dimension as the virtual node sequence, and its relatively precise position in the current image's local coordinate system is calculated, providing local positioning data for subsequent global coordinate transformation.

[0084] The system retrieves the first virtual node with the smallest coordinate value from the virtual node sequence of the current frame, subtracts the coordinate value of the first virtual node from the one-dimensional projected coordinates, and obtains the pixel offset within the defect frame. Alternatively, the system can be implemented using algorithms based on traditional image morphological analysis and connected component extraction. The system performs grayscale conversion and adaptive threshold segmentation on the current video frame to obtain a binary image, then uses morphological opening and closing operations to eliminate isolated noise points and fill internal holes in the defect. The system labels the connected components of the processed binary image and calculates the centroid coordinates of connected components whose areas match the defect characteristics. The system projects these centroid coordinates onto a one-dimensional axis to obtain one-dimensional projected coordinates, and similarly calculates the pixel offset within the defect frame by subtracting the coordinates of the first virtual node.

[0085] S107. Add the cumulative pixel displacement to the pixel offset within the defect frame to obtain the total pixel coordinates of the defect;

[0086] The total pixel coordinates of a defect refer to the precise pixel measurement of its location within the global pixel space encompassed by the initial video frame, with the starting position as the absolute zero point. The system mathematically fuses global displacement data, representing the macroscopic motion trajectory of the camera equipment, with local offset data, representing the microscopic position of the defect in the current frame. Through simple addition, the system eliminates the differences in local coordinate systems between different video frames, uniformly mapping the defect position isolated in a single frame onto a global one-dimensional pixel coordinate axis that runs through the entire cable-stayed bridge, achieving absolute pixel-level positioning of the defect.

[0087] This step can be implemented using a direct addition technique based on one-dimensional scalars of high-precision floating-point variables. The system allocates double-precision floating-point data structures in memory to store the accumulated pixel displacement and the intra-frame pixel offset of the defect, ensuring that the precision of the decimal part is not lost during long-term accumulation and sub-pixel-level feature extraction. The system calls the arithmetic logic unit of the central processing unit to directly execute addition instructions on these two one-dimensional scalar values. After calculation, the system stores the added floating-point result as the total pixel coordinates of the defect in the system's global positioning result database for subsequent use by the physical space transformation module. Alternatively, the system can also achieve this using algebraic operations based on homogeneous coordinate transformation matrices. The system constructs the intra-frame pixel offset of the defect as a one-dimensional local position vector and the accumulated pixel displacement as a one-dimensional translation transformation matrix. The system uses matrix multiplication rules to multiply the translation transformation matrix by the local position vector on the left. This matrix-based operation method can not only achieve the summation of values ​​to obtain the total pixel coordinates of the defect, but also can be easily extended to scenarios with more complex coordinate system transformations such as scaling or rotation, thus improving the versatility of the system architecture.

[0088] S108. Based on the calibration parameters of the camera equipment, convert the total pixel coordinates of the defect into the absolute positioning coordinates of the surface defect in the physical space of the cable-stayed bridge.

[0089] Calibration parameters refer to a set of mathematical variables describing the internal optical characteristics and external spatial orientation of the camera equipment, primarily representing the conversion ratio between pixel scale and physical scale. Physical space refers to the three-dimensional objective space in the real world, measured in meters or millimeters. Absolute positioning coordinates refer to the specific distance value of the defect on the actual surface of the cable-stayed bridge, with a fixed endpoint as the physical origin. The system utilizes pre-established or real-time updated scale mapping relationships to translate the virtual pixel coordinates obtained through computer processing into real physical dimensions that engineers can directly understand and use. This conversion process is the ultimate goal of the entire defect localization method; it imbues the purely image processing results with practical engineering guidance, enabling maintenance personnel to directly locate the defect on the actual cable-stayed bridge based on these coordinates.

[0090] This step can be implemented using a linear mapping algorithm based on a globally constant scaling factor. The system extracts the baseline pixel spacing calculated in step S102 and, combined with the known physical spacing of the cable-stayed bridge spiral, calculates a globally constant scaling factor by dividing the physical spacing by the baseline pixel spacing. This factor represents the actual physical millimeter number corresponding to each pixel. Subsequently, the system directly multiplies the calculated total pixel coordinates of the defect by this globally constant scaling factor, linearly scaling the pixel-dimensional coordinates to the physical dimension, obtaining the absolute positioning coordinates of the surface defect in the physical space of the cable-stayed bridge. Alternatively, the system can also achieve this using a 3D spatial back-projection algorithm based on depth information fusion. The system reads the depth sensor data synchronously recorded by the camera device when capturing the current video frame to obtain the true physical depth of the defect area from the optical center of the camera device. Combining the intrinsic parameter matrix of the camera device, the system uses the total pixel coordinates of the defect as the image plane coordinates, and, using the inverse transformation formula of pinhole imaging and the true physical depth, reversely calculates the 3D physical coordinates of the defect point in the camera coordinate system with the camera device as the origin. Finally, the system uses the external attitude transformation matrix of the camera device relative to the starting point of the cable to transform the three-dimensional physical coordinates into the physical space of the cable and extracts its axial component as the absolute positioning coordinates.

[0091] In the above embodiments, by extracting linear features matching the inclination angle of the cable-stayed bridge spiral and performing one-dimensional axial projection, the two-dimensional image information is reduced to a one-dimensional space for processing, thus reducing computational complexity. By obtaining the reference pixel spacing of the spiral's physical spacing in the image coordinate system, a mapping relationship between physical space and image space is established. Traversing the one-dimensional candidate position set and extracting a subset of targets that conform to integer multiples of the reference pixel spacing as a consensus anchor point set utilizes the periodicity of the spiral, improving the accuracy of anchor point identification. Using the consensus anchor point set as a reference, an equally spaced virtual node sequence is generated through interpolation, constructing a unified measurement reference coordinate system. By matching and accumulating the translation differences of virtual node sequences in adjacent video frames, continuous displacement tracking during camera movement is achieved, reducing the cumulative error of single-frame positioning. The cumulative pixel displacement is added to the pixel offset within the defect frame to obtain the total pixel coordinates of the defect, completing the global positioning of the defect in the continuous video sequence. This suppresses missed and false detections during image feature extraction and reduces the cumulative error in visual distance accumulation calculation, thereby improving the accuracy of locating cable-stayed bridge defects in physical space.

[0092] In step S106 of the above embodiment, after the system identifies a surface defect in the cable-stayed bridge in the current video frame, it calculates the intra-frame pixel offset between the one-dimensional projected coordinates of the surface defect and the first virtual node in the virtual node sequence of the current video frame. Further, the system can identify surface defects in the cable-stayed bridge in at least the following two ways:

[0093] First, based on the known manufacturing tilt angle of the stay cable helix, the system inversely maps the virtual node sequence of the current video frame to a two-dimensional image space, generating a structure suppression mask. The known manufacturing tilt angle refers to the fixed standard physical angle between the PE (polyethylene) sheath helix on the surface of the stay cable and the central axis of the cable during manufacturing. The virtual node sequence contains one-dimensional coordinate information representing the periodic position of the helix. Using this tilt angle and node positions, the system reconstructs the ideal normal helix texture distribution on a two-dimensional blank matrix of the same size as the current video frame through geometric projection, thus generating the structure suppression mask. This mask is essentially a two-dimensional weight matrix used to cover or cancel out normal periodic structural features in the image.

[0094] Secondly, the system converts the current video frame into a grayscale image and performs pixel-level difference operations on this grayscale image and the aforementioned structure suppression mask to obtain a structural residual image. Pixel-level difference operations refer to subtracting or weighted subtracting the grayscale values ​​of pixels whose spatial coordinates completely correspond in the two images. Through this step, normal background textures in the grayscale image that coincide with the theoretical spiral are significantly weakened or eliminated, while abnormal regions that do not conform to normal periodicity (i.e., potential defect regions) are significantly preserved and highlighted in the structural residual image.

[0095] Subsequently, the system calculates the gradient magnitude of each pixel in the structural residual image in multiple preset directions and constructs a histogram of directional gradients for each pixel. The system can employ edge detection operators such as Sobel and Scharr to calculate the rate of grayscale change, i.e., the gradient magnitude, of each pixel in multiple preset directions (e.g., divided into 8 or 9 directional intervals), including horizontal, vertical, and diagonal directions. Next, a local neighborhood window is defined centered on this pixel, and the sum of the gradient magnitudes in each directional interval within this window is calculated, thereby constructing a histogram of directional gradients (HOG) reflecting the texture direction distribution characteristics of this local region.

[0096] Furthermore, the system calculates the local gradient direction entropy for each pixel based on the histogram of oriented gradients. Local gradient direction entropy is a numerical index calculated based on the principle of information entropy (such as the Shannon entropy formula), used to characterize the uniformity of gradient direction distribution within the neighborhood of a pixel. In normal smooth or regularly textured areas of a cable-stayed bridge surface, the gradient directions tend to concentrate in a few specific directions, resulting in a relatively uniform directional distribution and thus a low calculated information entropy. Conversely, when the surface has defects such as damage, cracks, or scratches, the surface becomes extremely irregular, and reflections and textures exhibit high randomness, leading to extremely dispersed and chaotic gradient directions in the local neighborhood. The more uniform the gradient distribution in each direction, the larger the calculated local gradient direction entropy. That is, the value of the local gradient direction entropy is positively correlated with the degree of dispersion of the gradient directions.

[0097] Finally, the system extracts the set of pixels whose local gradient direction entropy is greater than a preset entropy threshold and performs connected component analysis, identifying the connected component with the largest area as the surface defect of the cable-stayed bridge. The preset entropy threshold is a critical value pre-calibrated or dynamically calculated by the system to distinguish between normal roughness and real defects. The system traverses the entire image, filtering out all pixels with high entropy values; these pixels typically correspond to the most chaotic abnormal regions. The system performs a connected component labeling algorithm on these discrete pixel sets, combining spatially adjacent and connected pixels into independent blocks. Considering the possibility of small dust or noise interference in actual detection, the system calculates the pixel area of ​​all independent connected components, filters out smaller interference terms, and directly locks down the connected component with the largest area as the final surface defect of the cable-stayed bridge.

[0098] The system can also first divide the current video frame into multiple periodic image blocks with a length equal to the reference pixel spacing along the axis of the cable-stayed bridge. Because the spiral on the surface of the cable-stayed bridge has strict physical periodicity, its projection onto the two-dimensional image also exhibits corresponding spatial repeatability. The reference pixel spacing corresponds to the pixel length spanned by a complete physical cycle of the spiral on the image. Using this reference pixel spacing as the cutting step size, the system uniformly crops the current video frame along the central axis of the cable-stayed bridge, thereby obtaining several rectangular sub-images of the same size and containing similar background textures, i.e., periodic image blocks.

[0099] Secondly, the system performs median filtering on the pixel grayscale values ​​of all periodic image blocks at the same relative coordinate positions to reconstruct a baseline background template. The system spatially stacks the aforementioned multiple periodic image blocks in memory, and for each relative coordinate position in the stacked matrix, extracts the pixel grayscale value sequence of all periodic image blocks at that position. Since surface defects (such as damage and stains) on the cable-stayed bridge are random and sporadic local anomalies, they usually only appear in a very small number of periodic image blocks. Therefore, the system sorts this set of grayscale value sequences and extracts the median (i.e., median filtering), which effectively filters out these occasional extreme values ​​(abnormally dark or bright defect pixels), thus using the median of all coordinate positions to reconstruct an ideal periodic image block representing the flawless state of the cable-stayed bridge surface, i.e., the baseline background template.

[0100] Subsequently, the system calculates the pixel-level absolute difference between each periodic image patch and the baseline background template, and stitches them together to generate an anomaly saliency map with the same size as the current video frame. The system compares each original periodic image patch with the reconstructed clean baseline background template, calculating the absolute difference in grayscale values ​​of the corresponding pixels. The larger the difference, the greater the deviation from the normal spiral background. Next, the system reassembles these calculated difference image patches according to their original spatial order and position to generate a mapping image reflecting the global anomaly degree, i.e., the anomaly saliency map. In this map, the normal periodic background is effectively eliminated and presented as a low grayscale value, while potential defective areas are significantly enhanced and presented as high grayscale values.

[0101] Furthermore, the system performs adaptive thresholding segmentation on the anomaly saliency map to obtain a binarized image, and extracts candidate connected components from the binarized image. The system employs Otsu's method or a local adaptive thresholding algorithm to dynamically calculate the optimal segmentation threshold based on the grayscale distribution characteristics of the anomaly saliency map, transforming the image into a binarized image containing only the foreground (potential defects, usually marked as white) and the background (normal areas, usually marked as black). Subsequently, the system uses a connected component labeling algorithm to extract all independent white pixel blocks from the binarized image; these blocks are the candidate connected components that may represent defects.

[0102] Finally, the system calculates the geometric compactness of each candidate connected component and identifies those with geometric compactness within a preset range as surface defects of the cable-stayed bridge. Geometric compactness is defined as the ratio of the actual pixel area of ​​a candidate connected component to the area of ​​the smallest circumcircle surrounding that component. This metric measures the regularity of the connected component's shape. Real surface defects of cable-stayed bridges (such as scratches, cracks, or blocky peeling) typically exhibit irregular shapes, and their geometric compactness falls within a specific empirical range. By setting a preset range (including explicit upper and lower limits), the system can effectively eliminate false targets with excessively low compactness (e.g., elongated free pixels caused by image noise) or excessively high compactness (e.g., interference caused by water droplets or perfectly circular reflective spots), thus ultimately confirming candidate connected components that meet the morphological requirements as real surface defects of the cable-stayed bridge.

[0103] In the above embodiments, the system has successfully achieved high-precision one-dimensional absolute positioning of surface defects along the axial direction of the stay cable by using one-dimensional axial projection and pixel displacement accumulation technology. However, in practical engineering applications, the stay cable is a three-dimensional cylindrical structure with a certain spatial volume, and obtaining only the axial distance (i.e., the coordinate in the length direction) is insufficient to completely determine the exact spatial location of the defect on the stay cable surface. In order to further obtain the distribution of defects along the circumference of the stay cable and achieve the leap from one-dimensional axial positioning to three-dimensional spatial positioning, the following combines... Figure 2 Another method for locating cable defects based on image recognition in the embodiments of this application is described below:

[0104] Please see Figure 2 This is another flowchart illustrating a cable-stayed bridge defect localization method based on image recognition in an embodiment of this application.

[0105] S201. Obtain the physical installation offset angle of the target camera device in the circumferential detection array;

[0106] The system acquires the physical installation offset angle of the target camera device within the circumferential inspection array. The target camera device is the one that captures images of surface defects. The target camera device refers to specific image acquisition hardware that successfully captures abnormal conditions such as damage, cracks, or scratches on the surface of the cable-stayed bridge during visual inspection. Such devices include, but are not limited to, charge-coupled device (CCD) cameras or complementary metal-oxide-semiconductor (CMOS) cameras. The circumferential inspection array is a panoramic image acquisition system composed of multiple cameras arranged uniformly or non-uniformly around the circumference of the cable-stayed bridge according to a specific spatial geometric topology, aiming to achieve 360-degree visual coverage of the cable-stayed bridge surface without blind spots. The physical installation offset angle refers to the fixed angle formed between the radial plane containing the optical axis of the target camera device's lens and the system's preset zero-degree reference plane when the target camera device is fixedly installed in the circumferential inspection array. This angle reflects the absolute circumferential position of the camera device in the real three-dimensional physical world. Surface defects refer to any damaged areas on the surface of the outer protective sheath or inner steel wire of the stay cable that do not conform to normal physical morphology, including but not limited to cracks, scratches, dents, or corrosion of the internal steel wire. The system acquires the physical installation offset angle of the target camera device in the circumferential detection array. Essentially, after identifying an image containing surface defects, the system retrieves and extracts the spatial attitude parameters bound to that specific camera device from the system's underlying hardware configuration information database by parsing the image's source channel or device unique identifier. This establishes a circumferential spatial reference benchmark for subsequently mapping the two-dimensional image data to three-dimensional physical space.

[0107] The first specific method is parameter reading technology based on a static configuration database. During system initialization, the spatial position of each camera in the circumferential detection array is pre-calibrated using a high-precision theodolite or total station. The angle between the optical axis of each camera and the zero-degree reference plane is measured, and these angle values, along with the camera's media access control address or network protocol address, are persistently stored as key-value pairs in the system's internal relational database or extensible markup language configuration file. When the system receives an abnormal video frame transmitted by the target camera, it extracts the device identification information from the packet header and performs a precise match in the database using a query statement, thereby directly reading the corresponding physical installation offset angle value. The second specific method is dynamic sensing technology based on inertial measurement units (IMUs). A microelectromechanical system (MEMS) IMU containing a three-axis gyroscope and a three-axis accelerometer is rigidly connected to the internal circuit board or external housing of each camera. While acquiring video frames, the system simultaneously receives the attitude calculation data output by this IMU. The system uses attitude calculation algorithms, such as Kalman filtering, to fuse the angular velocity data from the gyroscope and the gravity vector data from the accelerometer. It calculates the Euler angle of the camera relative to the direction of gravity and the geomagnetic north pole in real time. Then, through a coordinate transformation matrix, it converts the Euler angle into the real-time physical installation angle of the camera in the cylindrical coordinate system of the circumferential detection array.

[0108] S202. Based on the image features of the surface defect in the current video frame and the physical installation offset, calculate the global central angle of the surface defect on the circular section of the cable-stayed bridge.

[0109] The system calculates the global central angle of the surface defect on the circular cross-section of the cable-stayed bridge based on the image features of the surface defect in the current video frame and the physical installation offset angle. Specifically, this includes: determining the lateral feature reference point of the surface defect in the current video frame; converting the lateral feature reference point into a local offset angle of the surface defect relative to the optical axis of the target camera based on the imaging projection relationship of the target camera; and superimposing the physical installation offset angle and the local offset angle to obtain the global central angle of the surface defect on the circular cross-section of the cable-stayed bridge. A video frame refers to a discrete static digital image acquired by the camera within a continuous time series, serving as the basic data unit for constructing a dynamic video stream. Image features refer to a set of highly recognizable visual attributes of the surface defect in the video frame, including but not limited to pixel grayscale gradient changes, edge contour shapes, texture distribution patterns, and color contrast. The global central angle is the absolute angle formed between the radial line where the surface defect is located and the polar axis, with the geometric center of the circular cross-section of the cable-stayed bridge as the pole and a preset zero-degree reference radial line as the polar axis. It uniquely characterizes the position of the defect in the circumferential direction of the cable-stayed bridge. A circular cross-section refers to an ideal two-dimensional geometric plane perpendicular to the central axis of the cable-stayed bridge. A lateral feature reference point refers to a specific pixel coordinate point in the image coordinate system of a two-dimensional video frame, representing the core location of a surface defect; it is typically the geometric center of the defect bounding box or the centroid of the defect profile. The optical axis refers to an imaginary straight line passing through the optical center of the camera lens and perpendicular to the imaging sensor. The local deflection angle refers to the relative angle between the position of the surface defect in real physical space and the plane containing the optical axis of the target camera. The system first accurately locates the pixel-level reference point of the surface defect in the lateral direction of the two-dimensional image using image processing technology. Then, based on the pinhole imaging principle and the intrinsic parameter matrix of the camera, it maps this pixel-level lateral coordinate to a relative angular offset in physical space relative to the center of the camera's line of sight. Finally, it algebraically sums this relative offset with the absolute installation angle of the camera itself, thereby achieving a transition from a local relative observation perspective to a global absolute physical perspective, accurately calculating the absolute azimuth angle of the defect on the circumference of the cable-stayed bridge.

[0110] The system can implement this refinement technology in two ways: geometric mapping based on traditional computer vision and end-to-end regression based on deep learning. The first method is geometric mapping based on traditional computer vision. The system first performs grayscale conversion and Gaussian filtering on the current video frame for noise reduction. It then uses an edge detection operator to extract the pixel set of the defect's edge contour, calculates the spatial moments of this contour set, and obtains the lateral pixel coordinates of the contour centroid as the lateral feature reference point. Subsequently, the system calls a pre-calibrated camera intrinsic parameter matrix to calculate the pixel difference between the lateral feature reference point pixel coordinates and the principal point pixel coordinates of the image center. This pixel difference is multiplied by the physical size of a single pixel, and combined with the lens's physical focal length, the system uses the arctangent trigonometric function to calculate the local deflection angle of the surface defect relative to the optical axis. Finally, based on the clockwise or counterclockwise arrangement of the target camera in the array, the system performs a signed addition operation on the physical installation deflection angle and the local deflection angle to obtain the global central angle. The second method is end-to-end regression based on deep learning. The system inputs the current video frame into a pre-trained convolutional neural network object detection model. This model achieves its functionality by incorporating multiple convolutional and pooling layers. Through backpropagation training on a large amount of labeled data, it automatically learns and extracts high-dimensional abstract semantic features of surface defects under different lighting and morphologies. Using the bounding box regression branch at the network's end, it directly predicts the center coordinates of the rectangle surrounding the defect, using the x-coordinate of this center coordinate as the lateral feature reference point. Next, the system uses a polynomial distortion correction model to correct radial and tangential distortion at this lateral feature reference point. Then, through a lookup table or polynomial curve fitting, the corrected lateral pixel coordinates are directly mapped to a local skew angle. Finally, the system performs angle superposition to obtain the global central angle.

[0111] S203. Using the absolute positioning coordinates as the axial coordinates, and combining the global central angle with the radius of the cable, construct the three-dimensional cylindrical coordinates of the surface defects.

[0112] Absolute positioning coordinates refer to the precise linear distance values ​​of surface defects along the length of the cable-stayed bridge, obtained through one-dimensional axial projection and pixel displacement accumulation techniques in the pre-processing workflow. They represent the absolute position of the defect in the longitudinal dimension. Axial coordinates refer to the coordinate components along the central axis of a cylinder in a standard three-dimensional cylindrical coordinate system, usually represented by the letter Z. The radius is the straight-line distance from the geometric center of the cable-stayed bridge's circular cross-section to its outermost surface; it is a key geometric parameter describing the dimensions of the cable-stayed bridge's cross-section. Three-dimensional cylindrical coordinates are a method of representing three-dimensional spatial coordinates, composed of three independent components: radial distance, azimuth, and height, used to accurately describe the position of any point in three-dimensional space. The system directly assigns the absolute positioning coordinates representing the longitudinal position to the height component in the three-dimensional cylindrical coordinate system, assigns the global central angle representing the circumferential position to the azimuth component, and assigns the physical radius of the cable to the radial distance component. Through this structured combination of multi-dimensional data, the system successfully integrates the originally scattered one-dimensional length information, two-dimensional angle information, and prior geometric dimension information, and constructs a three-dimensional spatial coordinate point in the virtual digital space that can completely map the exact location of surface defects in the real physical world.

[0113] The system can construct the 3D cylindrical coordinates of surface defects using two methods: object-oriented programming-based structure instantiation and matrix conversion based on point cloud data formats. The first method is object-oriented programming-based structure instantiation. The system predefines a data structure or class named "cylindrical coordinate point" in memory. This structure contains three double-precision floating-point member variables, corresponding to radial distance, azimuth angle, and axial height, respectively. After obtaining the absolute positioning coordinates, global central angle, and the radius of the cable, the system dynamically allocates storage space in the memory heap to instantiate this structure object. The system directly writes the radius of the cable into the radial distance variable of the structure, converts the global central angle to radians and writes it into the azimuth angle variable, and writes the absolute positioning coordinates into the axial height variable, thus completing the data encapsulation and construction of the 3D cylindrical coordinates of a single surface defect. This allows subsequent modules to directly access the structure data through pointers or references. The second method is matrix conversion based on point cloud data formats. To ensure compatibility with subsequent 3D visualization rendering engines, the system first organizes the absolute positioning coordinates, global central angle, and radius into a one-dimensional row vector. Then, utilizing the mathematical transformation logic from cylindrical coordinates to Cartesian coordinates, the system constructs a non-linear transformation matrix containing trigonometric functions. This one-dimensional row vector is input into the transformation matrix for calculation, determining the X, Y, and Z axis coordinates of the surface defects in a 3D Cartesian coordinate system. Finally, these Cartesian coordinate values, along with the original cylindrical coordinate components, are packaged into a data record containing multi-dimensional spatial attributes according to the specifications of standard polygon file formats or point cloud data formats, thus completing the deep construction and formatted output of the 3D coordinates.

[0114] S204. Obtain the three-dimensional cylindrical coordinates output by different camera devices in the circumferential detection array within the same detection batch;

[0115] The same inspection batch refers to a complete, end-to-end automated inspection cycle performed by the system on the same specific cable-stayed bridge within a continuous and uninterrupted working time. All data generated within this cycle belongs to the same logical group. Different camera devices refer to multiple image sensor nodes in the circumferential inspection array that are physically adjacent or opposite each other and each has an independent data acquisition and processing channel. Since the circumferential inspection array is designed to ensure panoramic coverage, there will inevitably be a certain proportion of physical overlap between the fields of view of adjacent camera devices. When a large surface defect or one located in the overlapping area passes through the inspection array, it will be captured simultaneously or sequentially by multiple different camera devices. The system uses an internal high-speed data bus or network communication protocol to overcome the physical isolation of different hardware nodes, centrally collecting and caching the three-dimensional cylindrical coordinate data describing the same physical area, which are independently calculated by different camera devices. This provides a complete multi-source heterogeneous data set for subsequent global data deduplication and fusion processing.

[0116] The system can acquire 3D cylindrical coordinates from different camera devices using two main methods: asynchronous subscription based on message queues and synchronous polling based on shared memory. The first method is asynchronous subscription based on message queues. The system deploys a distributed message publish-subscribe middleware, such as a server based on an advanced message queue protocol, in the central processing unit. Each camera node in the ring-shaped detection array is configured as a message publisher. When a camera completes the calculation of its 3D cylindrical coordinates, it serializes the coordinate data along with the device identifier, timestamp, and batch number into a generic data format string and publishes it to a specific topic channel in the message queue. The system's main control program, acting as a message subscriber, continuously listens to this topic channel. Once a new message arrives, it asynchronously triggers a callback function, pushing the parsed 3D cylindrical coordinate data into the batch data buffer pool in the system's main memory, thus achieving decoupling and efficient aggregation of data from multiple devices. The second method is synchronous polling based on shared memory. The system allocates a sufficiently large physical memory region at the operating system kernel level as a shared memory segment, accessible to multiple processes. This segment is configured with read / write mutexes or semaphores. Each camera device's underlying driver, after generating 3D cylindrical coordinates, requests a mutex, writes the coordinate data to a specific block within the shared memory segment at a predetermined memory offset, and then releases the mutex. The system's main control thread, operating on a fixed clock cycle, periodically requests read access, scans all blocks within the shared memory segment, and batches newly added 3D cylindrical coordinate data, marked with the current detection batch, to its local workspace, thus completing centralized data acquisition.

[0117] S205. Based on the spatial distribution relationship between the three-dimensional cylindrical coordinates, perform spatial position fusion on the three-dimensional cylindrical coordinates belonging to the same physical defect, and output the unique three-dimensional spatial coordinates of the surface defect after deduplication.

[0118] Spatial distribution relationships refer to the topological characteristics of multiple 3D cylindrical coordinate points in a virtual 3D space, such as their geometric distances, azimuth differences, and relative clustering of axial heights. A single physical defect refers to a unique, objectively existing damaged entity on the actual cable-stayed bridge surface, even though it may be mapped to multiple different coordinate data records in the system. Spatial location fusion refers to the process by which the system uses specific mathematical algorithms to merge multiple coordinate points representing the same physical entity with slight numerical differences into a single, more confident composite coordinate point. Deduplication refers to the operation of identifying and removing redundant coordinate records that repeatedly describe the same object from the dataset. Unique 3D spatial coordinates refer to the final coordinate data output after fusion and deduplication, which accurately and uniquely represents the true physical location of the surface defect within the system. The system analyzes the clustering patterns of massive amounts of coordinate points in three-dimensional space, identifies coordinate clusters generated by repeated shooting from multiple perspectives, and uses mathematical statistics or machine learning algorithms to reduce the dimensionality and center these coordinate clusters, eliminating redundancy and noise from multi-source data. Finally, it extracts the unique spatial location identifier of each real surface defect, thereby significantly improving the accuracy of defect statistics and the reliability of the system's detection.

[0119] The system can achieve spatial location fusion and output unique coordinates through two specific methods: density-based spatial clustering and confidence-weighted graph matching. The first method is density-based spatial clustering. The system inputs all acquired 3D cylindrical coordinates into a density-based clustering algorithm engine. The system pre-sets a spatial neighborhood radius threshold and a minimum number of points. The algorithm engine traverses each coordinate point, calculates its Euclidean spatial distance to other surrounding coordinate points, and groups points with a distance less than the spatial neighborhood radius threshold into a core cluster. Since coordinate points of the same physical defect captured by different cameras will inevitably be highly clustered in space, these points will be automatically assigned to the same cluster by the algorithm. Subsequently, the system calculates the arithmetic mean of the radial distance, azimuth angle, and axial height of all coordinate points within each cluster, and uses the calculated average coordinates as the unique 3D spatial coordinates after fusion for that physical defect, discarding the original redundant coordinates within the cluster to complete deduplication. The second method is confidence-weighted graph matching. The system constructs an undirected weighted graph from all 3D cylindrical coordinates. Nodes in the graph represent coordinate points, and edges between nodes represent the spatial Euclidean distance between those points. Using a connected component extraction algorithm, the system extracts the set of nodes with distances less than a preset merging threshold into an independent connected subgraph, representing the observation set of the same physical defect. Next, the system assigns a confidence weight to each coordinate point based on its local skew angle. A smaller local skew angle indicates that the defect is closer to the optical axis center of the camera, resulting in higher image quality, and thus a larger weight is assigned. The system uses these weights to perform a weighted average calculation on all coordinate points within the connected subgraph, obtaining a unique fused 3D spatial coordinate, which is then output to the final defect detection report database.

[0120] In the above embodiments, the physical installation offset angle of the target camera device in the circumferential detection array is obtained, establishing a correspondence between the camera device's observation angle and the circumferential position of the cable-stayed bridge. Based on the image features of the surface defect in the video frame and the physical installation offset angle, the global central angle of the defect on the circular cross-section of the cable-stayed bridge is calculated, realizing the conversion from two-dimensional image coordinates to circumferential angles in a three-dimensional cylindrical coordinate system. The absolute positioning coordinates are used as axial coordinates, combined with the global central angle and the cable-stayed bridge radius to construct three-dimensional cylindrical coordinates, completing the three-dimensional spatial positioning of the defect. The three-dimensional cylindrical coordinates output by different camera devices within the same detection batch are obtained, utilizing the redundant information from multi-view observations. Based on the spatial distribution relationship between the three-dimensional cylindrical coordinates, the coordinates belonging to the same physical defect are spatially fused. Cross-validation of multi-source data improves positioning accuracy, and the output of unique, deduplicated three-dimensional spatial coordinates avoids duplicate counting, reduces defect statistical errors, and improves the reliability and practicality of the detection system.

[0121] Through the above embodiments, the system utilizes a circumferential detection array composed of multiple cameras to successfully acquire the three-dimensional cylindrical coordinates of the defect on the cable-stayed bridge surface. Furthermore, by fusing spatial positions, it effectively eliminates the problem of duplicate recording caused by multi-view observations, achieving precise spatial "point" location of the defect. However, for the structural safety assessment and subsequent maintenance of the cable-stayed bridge, relying solely on a single coordinate point is insufficient; engineers also need to accurately grasp the actual shape, coverage area, and specific physical dimensions of the defect. The following section combines... Figure 3 This application describes another method for locating defects in stay cables based on image recognition:

[0122] Please see Figure 3 This is another flowchart illustrating a method for locating defects in cable stays based on image recognition, as described in this application.

[0123] S301. Obtain the two-dimensional defect contour features output by each target camera device associated with a unique three-dimensional spatial coordinate.

[0124] Two-dimensional defect contour features refer to the geometric morphological description data used to define the visual boundary of surface defects, extracted in a two-dimensional image coordinate system through pixel-level edge detection or region segmentation techniques. These features can take various forms, including but not limited to ordered sets of edge pixel coordinates, polygon vertex sequences, or spline curve control points. Association refers to the mapping link established between macroscopic three-dimensional spatial locations and microscopic two-dimensional image data through the system's internal data indexing mechanism or temporal-space matching algorithm. The system acquires the two-dimensional defect contour features output by each target camera device associated with a unique three-dimensional spatial coordinate. Essentially, after establishing the globally unique physical location of the defect, the system uses this location coordinate as a search keyword to perform a reverse search in the system's distributed relational database or high-speed memory cache, extracting the original two-dimensional edge morphological data generated by all camera devices that successfully captured the physical defect from different observation perspectives within the same detection batch. This process not only achieves accurate alignment of heterogeneous data across devices but also collects sufficient multi-view basic materials for subsequent high-precision morphological reconstruction and dimensional measurement in three-dimensional space, ensuring the comprehensiveness of the defect morphological description.

[0125] The system can acquire two specific features of two-dimensional defect contours: edge tracking based on traditional image processing operators and instance segmentation based on deep learning. The first method is edge tracking based on traditional image processing operators. The system first extracts the region of interest (ROI) image containing surface defects from the target camera output, performs grayscale conversion and Gaussian filtering for noise reduction, then calls an edge detection operator to calculate the gradient magnitude and direction of image pixels, and extracts continuous edge pixel sets through non-maximum suppression and double thresholding. Next, the system uses an eight-neighbor contour tracking algorithm to connect the edge pixels point by point, ultimately generating closed two-dimensional defect contour feature data. The second method is instance segmentation based on deep learning. The system inputs video frames containing surface defects into a pre-trained fully convolutional instance segmentation neural network model. This model achieves its function because its network architecture not only includes a deep convolutional feature pyramid for extracting multi-scale, high-dimensional abstract visual features, but also integrates a mask prediction branch specifically for pixel-level classification. The model is trained through iterative backpropagation on a massive dataset of defect images with precise manual annotations. It can automatically learn and recognize the complex texture and shape features of different types of defects, thereby directly generating a pixel-level binary mask image that highly matches the defect region at the output. The system then extracts the outer boundary pixels of the binary mask image and transforms them into vectorized two-dimensional defect contour features.

[0126] S302. Using the unique three-dimensional spatial coordinates as spatial anchor points, project the two-dimensional defect contour features onto the preset three-dimensional surface model of the cable-stayed bridge to obtain the corresponding three-dimensional contour segment.

[0127] A spatial anchor point is a fixed reference point with absolute positional reference value established in a virtual three-dimensional spatial coordinate system, used to unify and align relative geometric data from different sources to the same global coordinate framework. A preset three-dimensional surface model of the cable-stayed bridge refers to a three-dimensional geometric mesh or parametric surface model pre-constructed within the system for digitally simulating the external physical morphology of a real cable-stayed bridge. A three-dimensional contour segment refers to an incomplete three-dimensional spatial curve segment formed after mapping defect boundary data from a two-dimensional image plane to a three-dimensional spatial surface. Projection refers to the mathematical mapping process of converting low-dimensional image coordinates into high-dimensional spatial coordinates using spatial transformation matrices in computer graphics. The system uses a unique three-dimensional spatial coordinate as a spatial anchor point to project the two-dimensional defect contour features onto the preset three-dimensional surface model of the cable-stayed bridge, obtaining the corresponding three-dimensional contour segment. Essentially, the system uses the determined defect center position as a reference, combined with the internal and external parameter matrices of the camera equipment, and utilizes ray tracing or perspective transformation principles to "project" and "fit" the two-dimensional planar contour points extracted from various perspectives onto the virtual cylindrical surface model, thus completing the leapfrog reconstruction of the defect morphology from two-dimensional image space to three-dimensional physical space.

[0128] The system can project two-dimensional defect contour features onto a three-dimensional surface model using two specific methods: mesh intersection based on ray projection and inverse calculation based on perspective projection matrix. The first method is mesh intersection based on ray projection. Starting from the optical center of the target camera, the system constructs a series of spatial rays in virtual three-dimensional space, using each pixel of the two-dimensional defect contour feature as a direction vector. Then, the system discretizes the preset cable-stayed bridge three-dimensional surface model into a three-dimensional polygonal mesh composed of numerous tiny triangular facets. The system calls a ray tracing algorithm to calculate the coordinates of the intersection points between these spatial rays and the three-dimensional polygonal mesh. The system connects these calculated three-dimensional intersection coordinates according to the topological order of the original two-dimensional contour, thereby generating a three-dimensional contour fragment that fits onto the model surface. The second method is inverse calculation based on perspective projection matrix. The system first constructs a complete perspective projection matrix containing focal length, principal point coordinates, and rotation and translation vectors based on the calibration data of the target camera. Since the preset 3D surface model of the cable-stayed bridge is a standard cylindrical surface equation, the system substitutes the homogeneous coordinates of the 2D defect contour features into the inverse matrix of the perspective projection matrix, and combines this with the geometric constraints of the cylindrical surface equation to construct a nonlinear system of equations. The system uses Newton's iteration method or nonlinear least squares method to solve this system of equations, directly calculating the precise 3D coordinates of the 2D contour points on the 3D cylindrical surface, and then generating the corresponding 3D contour fragments.

[0129] S303. Analyze the topological overlap relationship, determine the intersection nodes of the three-dimensional contour segments on the three-dimensional surface model of the cable-stayed bridge, and the overlapping internal line segments and non-overlapping external line segments divided by the intersection nodes.

[0130] The system can implement the analysis of topological overlap relationships and the determination of intersecting nodes in two ways: line segment intersection based on bounding box collision detection and graph theory analysis based on half-edge data structures. The first method is line segment intersection based on bounding box collision detection. The system first constructs an axial bounding box for each 3D contour segment. By calculating the intersection of these bounding boxes in 3D space, the system quickly eliminates obviously non-intersecting contour segment pairs, thus narrowing the scope of precise calculation. For contour segment pairs where bounding boxes collide, the system discretizes them into a series of tiny spatial line segments and uses a spatial line segment intersection algorithm to calculate the shortest distance between these line segments one by one. When the shortest distance is less than a preset tolerance threshold, the system determines that an intersection has occurred and calculates the 3D coordinates of the intersecting node. Subsequently, the system breaks the original contour segment using the intersecting node as the boundary and, based on the normal vector direction of each segment relative to the geometric center of the defect, determines and marks overlapping internal line segments and non-overlapping external line segments. The second method is graph theory analysis based on half-edge data structures. The system maps all 3D contour fragments onto a 2D parametric plane, constructing a planar graph data structure. The system uses a half-edge data structure to store the topological relationships between vertices, edges, and faces in the graph. The system utilizes a scanline algorithm to efficiently find the intersections of all line segments on the parametric plane, mapping them back to 3D space as intersection nodes. Next, by traversing the face information in the half-edge data structure, the system identifies internal polygonal regions surrounded by multiple edges. The system marks the edges constituting these internal polygonal regions as overlapping internal line segments, while edges belonging to only one external infinite face are marked as non-overlapping external line segments, thus completing a deep analysis of topological relationships.

[0131] S304. Remove overlapping internal line segments and splice the non-overlapping external line segments at the intersection nodes to generate the initial envelope boundary.

[0132] Elimination refers to the permanent deletion of data records marked as redundant or invalid from the system's data structure or memory space. Head-to-tail splicing refers to the process of connecting multiple discrete spatial curves sequentially into a whole based on the spatial proximity of their endpoints and the continuity of their tangent directions. The initial envelope boundary refers to the set of closed or semi-closed spatial curves formed after redundancy elimination and preliminary splicing operations, which can roughly outline the overall contour of the surface defect. The system eliminates overlapping internal line segments and splices non-overlapping external line segments at intersection nodes to generate the initial envelope boundary. Essentially, this is a substantial data cleaning and reorganization operation performed after the system has completed the topological relationship analysis. The system discards the messy internal lines caused by repeated shooting from multiple perspectives, retaining only the outermost valid boundary, and uses intersection nodes as bridges to stitch together these scattered external line segments that originally belonged to different perspectives, thus initially piecing together the complete outer contour skeleton of the physical defect in three-dimensional space.

[0133] The system can implement the removal of overlapping internal line segments and the end-to-end splicing in two ways: splicing based on directed graph Eulerian path traversal and endpoint nearest neighbor matching based on a greedy strategy. The first method is splicing based on directed graph Eulerian path traversal. The system first traverses the line segment attribute list in memory, using a memory release function to completely remove all data nodes marked as overlapping internal line segments from the linked list. Then, the system constructs a directed graph model from the remaining non-overlapping external line segments and their intersecting nodes, where the intersecting nodes are the vertices and the non-overlapping external line segments are the directed edges. The system calculates the in-degree and out-degree of each vertex to determine if the directed graph contains an Eulerian circuit or path. The system calls a depth-first search algorithm to traverse the directed graph, extracting directed edges sequentially according to the traversal order, thus achieving seamless end-to-end splicing of non-overlapping external line segments at intersecting nodes at the logical level, ultimately outputting a continuous sequence of vertex coordinates as the initial envelope boundary. The second method is endpoint nearest neighbor matching based on a greedy strategy. After cleaning up overlapping internal line segments, the system extracts the start and end coordinates of all non-overlapping external line segments and places them into a set of unmatched endpoints. The system randomly selects a non-overlapping external line segment as the starting segment, extracts its end coordinates, and then calculates the Euclidean distance to all other endpoints in the set of unmatched endpoints. Using a greedy strategy, the system selects the endpoint that is closest to it and satisfies the tangent vector angle constraint as the splicing object, connects the two line segments, and removes the matched endpoint from the set. The system repeatedly performs the above nearest neighbor search and connection operations until the set of unmatched endpoints is empty or no endpoint satisfying the constraints can be found, thus completing the splicing of all external line segments and generating the initial envelope boundary.

[0134] S305. Based on the preset curve smoothing algorithm, the splicing segments of the initial envelope boundary at the intersection nodes are smoothly transitioned to obtain a fused complete three-dimensional contour.

[0135] Curve smoothing algorithms refer to a class of numerical processing methods used to eliminate high-frequency noise, jagged lines, or unnatural abrupt changes in discrete data point sequences, enabling curves to possess higher-order continuity in mathematical expression. Smoothing transition processing refers to applying smoothing algorithms to curve seams or areas of abrupt curvature changes, making the transition of geometric shapes more natural and consistent with real physical laws. A fused complete 3D contour refers to the final spatial curve generated after splicing and smoothing processes, which has no data redundancy, possesses good geometric continuity, and can realistically reproduce the true 3D boundary shape of surface defects. Based on a preset curve smoothing algorithm, the system performs smoothing transition processing on the splicing segments at intersection nodes of the initial envelope boundary to obtain a fused complete 3D contour. Essentially, to eliminate the abrupt "C0 continuity" or even discontinuous angles caused by direct splicing of multiple curve segments, the system uses mathematical fitting tools to redistribute and fine-tune the coordinate points of the splicing area, making the entire defect boundary present a natural and smooth transition shape in 3D space, just like real physical damage, thereby significantly improving the visual quality and geometric accuracy of the 3D reconstruction results.

[0136] The system can achieve smooth transition processing of the initial envelope boundary in two ways: curve fitting based on non-uniform rational B-splines and discrete vertex smoothing based on the Laplacian operator. The first method is curve fitting based on non-uniform rational B-splines. The system uses all 3D coordinate points on the initial envelope boundary as a set of control points. Based on the spatial distribution density and curvature variation of the control points, the system adaptively generates node vectors and weight coefficients. Subsequently, the system calls the non-uniform rational B-spline basis function calculation formula to construct a high-order continuous parameterized spatial curve equation. The system uses this equation to resample the local region near the intersection nodes, generating new smooth vertex coordinates to replace the original harsh inflections, thus achieving C2-level curvature continuity in the spliced ​​segment and outputting a fused, complete 3D contour. The second method is discrete vertex smoothing based on the Laplacian operator. The system treats the initial envelope boundary as a closed one-dimensional discrete mesh. The system traverses every vertex on the boundary, especially vertices near intersection nodes, extracting several neighboring vertices. The system calculates the spatial deviation vector between the vertex and the centroid of its neighboring vertices, and multiplies this deviation vector by a preset smoothing convergence coefficient as the displacement update amount for the vertex. By repeatedly performing the above Laplacian displacement update operation, the sharp abrupt changes at the intersecting nodes are gradually smoothed out, the curve as a whole shrinks inward and tends to become smooth, and finally converges to obtain a geometrically natural and fully integrated 3D contour.

[0137] S306. Extract and fuse the complete three-dimensional contour to generate the physical size parameters of surface defects.

[0138] Physical dimensional parameters refer to specific numerical indicators used to quantitatively describe the geometric scale of surface defects in three-dimensional physical space. These typically include, but are not limited to, the surface area, maximum axial length, maximum circumferential width, and equivalent circle diameter of the defect. The system extracts and fuses the complete three-dimensional contour to generate the physical dimensional parameters of the surface defects. Essentially, after completing high-precision three-dimensional morphological reconstruction, the system uses computer graphics and calculus principles to mathematically measure the spatial curved surface region enclosed by the closed three-dimensional contour. This transforms the intuitive geometric shape into objective quantitative data that engineers can directly use for safety assessments, life predictions, or maintenance decisions, thus completing the final closed loop from image recognition to engineering measurement.

[0139] The system can extract and fuse a complete 3D contour and generate physical dimension parameters using two specific methods: area calculation based on discrete triangular mesh integration and bounding box size calculation based on principal component analysis. The first method involves area calculation based on discrete triangular mesh integration. The system first uses the restricted Delaunay triangulation algorithm to divide the cylindrical surface region bounded by the fused complete 3D contour into a series of non-overlapping micro-triangular patches. Then, the system iterates through all the triangular patches, using Heron's formula or the geometric interpretation of the cross product to calculate the absolute area of ​​each micro-triangle in 3D space. Finally, the system sums the areas of all the triangular patches, and the sum is the precise physical surface area parameter of the surface defect. The second method involves bounding box size calculation based on principal component analysis. The system extracts the coordinates of all discrete vertices on the fused complete 3D contour and calculates the covariance matrix of these vertices. The system performs eigenvalue decomposition on this covariance matrix to obtain three mutually orthogonal eigenvectors. These eigenvectors represent the principal directions in which the defect is most dispersed in 3D space. The system projects the contour vertices onto these three main directions and calculates the difference between the maximum and minimum projected coordinates to construct a directed bounding box that tightly encloses the defect. The system directly extracts the side length of this directed bounding box along the cable axis as the maximum axial length parameter of the defect, and extracts the side length along the circumferential tangent as the maximum circumferential width parameter, thus completing the generation of multi-dimensional physical dimensions.

[0140] In the above embodiments, the two-dimensional defect contour features output by each target camera device associated with a unique three-dimensional spatial coordinate are obtained, making full use of the complementary information provided by multi-view observation. Using the unique three-dimensional spatial coordinate as a spatial anchor point, the two-dimensional defect contour features are projected onto a preset three-dimensional surface model of the cable-stayed bridge to obtain three-dimensional contour segments, realizing contour reconstruction from two-dimensional image space to three-dimensional physical space. This allows the local contours of defects observed from different perspectives to be represented on a unified three-dimensional model. Based on the topological overlap relationship of the three-dimensional contour segments on the three-dimensional surface model of the cable-stayed bridge, boundary fusion of the contour segments is performed to obtain a fused complete three-dimensional contour. Spatial stitching of multi-view contours compensates for the lack of contour information caused by occlusion or observation angle limitations from a single perspective, improving the completeness of contour reconstruction. The physical size parameters of the surface defects generated by the fused complete three-dimensional contour are extracted, realizing the quantitative measurement of defect geometric features. Compared with single-view measurement, multi-view fusion reduces the size measurement deviation caused by the observation angle, improving the measurement accuracy and reliability of defect size parameters.

[0141] The system in the embodiments of this invention is described below from the perspective of hardware processing. Please refer to [link / reference needed]. Figure 4 This is a schematic diagram of the physical device structure of a cable-stayed bridge defect location system based on image recognition, provided in an embodiment of this application.

[0142] It should be noted that, Figure 4 The structure of the system shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0143] like Figure 4 As shown, the system includes a Central Processing Unit (CPU) 401, which can perform various appropriate actions and processes based on a program stored in Read-Only Memory (ROM) 402 or a program loaded from storage portion 408 into Random Access Memory (RAM) 403, such as executing the methods described in the above embodiments. The RAM 403 also stores various programs and data required for system operation. The CPU 401, ROM 402, and RAM 403 are interconnected via a bus 404. An Input / Output (I / O) interface 405 is also connected to the bus 404.

[0144] The following components are connected to I / O interface 405: input section 406 including a camera, infrared sensor, etc.; output section 407 including a liquid crystal display (LCD) and speakers, etc.; storage section 408 including a hard disk, etc.; and communication section 409 including a network interface card such as a LAN (Local Area Network) card and a modem, etc. Communication section 409 performs communication processing via a network such as the Internet. Drive 410 is also connected to I / O interface 405 as needed. Removable media 411, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 410 as needed so that computer programs read from it can be installed into storage section 408 as needed.

[0145] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 409, and / or installed from removable medium 411. When the computer program is executed by central processing unit (CPU) 401, it performs the various functions defined in the present invention.

[0146] It should be noted that the computer-readable medium shown in the embodiments of the present invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. The transmitted data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof.

[0147] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0148] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in the system described in the above embodiments; or it may exist independently and not assembled into the system. The storage medium carries one or more computer programs that, when executed by a processor of a system, cause the system to implement the methods provided in the above embodiments.

[0149] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0150] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0151] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0152] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for locating defects in stay-stayed cables based on image recognition, characterized in that, include: Linear features matching the inclination angle of the cable spiral are extracted from continuous video frames captured by the camera moving along the cable and then projected in one dimension to obtain a set of one-dimensional candidate positions. Obtain the physical spacing of the cable-stayed cable spirals and map it to the reference pixel spacing in the image coordinate system; Traverse the one-dimensional candidate position set and extract the target subset containing the most pixel coordinates as the consensus anchor point set. The difference between the ratio of the pixel distance between any two pixel coordinates in the consensus anchor point set and the reference pixel distance and the positive integer closest to the ratio is less than the preset pixel error tolerance. Using the pixel coordinates in the consensus anchor point set as reference points and the reference pixel spacing as the step size, interpolate to generate an equally spaced virtual node sequence that coincides with the consensus anchor point set in the one-dimensional image space of the current video frame; The translation pixel difference between the virtual node sequences of two adjacent video frames is matched, and all translation pixel differences from the initial video frame to the current video frame in the consecutive video frames are accumulated to obtain the cumulative pixel displacement of the current video frame. In response to identifying a cable-stayed bridge surface defect in the current video frame, the intra-frame pixel offset of the defect between the one-dimensional projected coordinates of the surface defect and the first virtual node in the virtual node sequence of the current video frame is calculated. The total pixel coordinates of the defect are obtained by adding the cumulative pixel displacement to the pixel offset within the defect frame. Based on the calibration parameters of the camera device, the total pixel coordinates of the defect are converted into the absolute positioning coordinates of the surface defect in the physical space of the cable-stayed bridge.

2. The method according to claim 1, characterized in that, Prior to identifying a cable-stayed bridge surface defect in the current video frame, the method further includes: Based on the known manufacturing inclination angle of the cable-stayed spiral, the virtual node sequence of the current video frame is reverse-mapped to a two-dimensional image space to generate a structure suppression mask; Perform pixel-level difference operations between the grayscale image of the current video frame and the structure suppression mask to obtain the structure residual image; Calculate the gradient magnitude of each pixel in the structural residual image in multiple preset directions, and construct the directional gradient histogram of each pixel; The local gradient direction entropy of each pixel is calculated based on the directional gradient histogram. The local gradient direction entropy is an information entropy that characterizes the uniformity of gradient direction distribution in the neighborhood of the pixel. The value of the local gradient direction entropy is positively correlated with the degree of dispersion of the gradient direction. Extract the set of pixels whose local gradient direction entropy is greater than a preset entropy threshold and perform connected component analysis. The connected component with the largest area is taken as the surface defect of the cable-stayed bridge.

3. The method according to claim 1, characterized in that, Prior to identifying a cable-stayed bridge surface defect in the current video frame, the method further includes: Along the axial direction of the cable, the current video frame is divided into multiple periodic image blocks with a length equal to the reference pixel spacing; Median filtering is applied to the pixel grayscale values ​​of all the periodic image blocks at the same relative coordinate position to reconstruct the reference background template. Calculate the pixel-level absolute difference between each periodic image block and the reference background template, and stitch them together to generate an anomaly saliency map with the same size as the current video frame; An adaptive threshold segmentation is performed on the anomaly saliency map to obtain a binarized image, and candidate connected components are extracted from the binarized image; Calculate the geometric compactness of each candidate connected region, and determine the candidate connected regions whose geometric compactness is within a preset range as surface defects of the cable-stayed bridge. The geometric compactness is the ratio of the area of ​​the candidate connected region to the area of ​​the smallest circumcircle surrounding the candidate connected region.

4. The method according to claim 1, characterized in that, After converting the total pixel coordinates of the defect into the absolute positioning coordinates of the surface defect in the physical space of the cable-stayed bridge, the method further includes: The physical installation offset angle of the target camera device in the circumferential detection array is obtained, wherein the target camera device is the camera device that captures the surface defect; Based on the image features of the surface defect in the current video frame and the physical installation offset angle, calculate the global central angle of the surface defect on the circular section of the cable-stayed bridge. Using the absolute positioning coordinates as axial coordinates, and combining the global central angle with the radius of the cable, a three-dimensional cylindrical coordinate system for the surface defect is constructed. Obtain the three-dimensional cylindrical coordinates output by different camera devices in the circumferential detection array within the same detection batch; Based on the spatial distribution relationship between the three-dimensional cylindrical coordinates, the spatial positions of the three-dimensional cylindrical coordinates belonging to the same physical defect are fused, and the unique three-dimensional spatial coordinates of the surface defect after deduplication are output.

5. The method according to claim 4, characterized in that, The step of calculating the global central angle of the surface defect on the circular cross-section of the cable-stayed bridge based on the image features of the surface defect in the current video frame and the physical installation offset angle specifically includes: Determine the lateral feature reference point of the surface defect in the current video frame; Based on the imaging projection relationship of the target camera device, the lateral feature reference point is converted into a local deflection angle of the surface defect relative to the optical axis of the target camera device; The physical installation deviation angle and the local deviation angle are superimposed to obtain the global central angle of the surface defect on the circular section of the cable.

6. The method according to claim 4, characterized in that, After outputting the unique three-dimensional spatial coordinates of the surface defects after deduplication, the method further includes: Obtain the two-dimensional defect contour features output by each target camera device associated with the unique three-dimensional spatial coordinates; Using the unique three-dimensional spatial coordinates as spatial anchor points, the two-dimensional defect contour features are projected onto the preset three-dimensional surface model of the cable-stayed bridge to obtain the corresponding three-dimensional contour fragments. Based on the topological overlap relationship of the three-dimensional contour segments on the three-dimensional surface model of the cable-stayed bridge, the boundary of the three-dimensional contour segments is fused to obtain a fused complete three-dimensional contour. Extract the fused complete 3D contour to generate the physical size parameters of the surface defect.

7. The method according to claim 6, characterized in that, The step of fusing the three-dimensional contour segments according to their topological overlap relationship on the three-dimensional surface model of the cable-stayed bridge to obtain a fused complete three-dimensional contour specifically includes: The topological overlap relationship is analyzed to determine the intersection nodes of the three-dimensional contour segments on the three-dimensional surface model of the cable-stayed bridge, as well as the overlapping internal line segments and non-overlapping external line segments divided by the intersection nodes; Remove the overlapping internal line segments and splice the non-overlapping external line segments end to end at the intersection nodes to generate an initial envelope boundary. Based on a preset curve smoothing algorithm, the splicing segment of the initial envelope boundary at the intersection node is smoothed to obtain the fused complete three-dimensional contour.

8. A cable-stayed bridge defect localization system based on image recognition, characterized in that, The system includes: One or more processors and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the system to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on the system, the system performs the method as described in any one of claims 1-7.

10. A computer program product, characterized in that, When the computer program product is run on the system, the system performs the method as described in any one of claims 1-7.

Images