Cable-stayed bridge cable detection robot based on visual recognition and system thereof

Abstract

The application discloses a cable-stayed bridge cable detection robot based on visual identification and a system thereof, and belongs to the technical field of detection robots, and comprises visual identification, multi-sensor space-time registration, phased array ultrasonic three-dimensional imaging, fiber grating cable force acquisition, residual strength evaluation, decision output, intelligent navigation and obstacle avoidance modules, a full-chain detection system is constructed through multi-module cooperation, and full-automatic and high-precision comprehensive evaluation of the cable state is realized. The application integrates three types of detection technologies of vision, ultrasonic and cable force, breaks through the limitation of traditional single-dimensional detection, realizes accurate correlation of surface and internal damage through multi-sensor space-time registration, provides multi-dimensional and traceable complete data support for cable health state evaluation, simultaneously constructs a full-automatic detection closed loop, adaptively clamps and adapts to various special-shaped cables, realizes blind area-free detection through intelligent navigation, greatly reduces the risk of high-altitude operation, and significantly improves the detection efficiency and the standardization degree.

Description

Technical Field

[0001] This invention relates to the field of inspection robot technology, and in particular to a cable inspection robot and system for cable-stayed bridges based on vision recognition. Background Technology

[0002] Cable-stayed bridges are the core structural form of long-span bridges, possessing significant technical advantages in complex traffic scenarios such as crossing rivers, seas, and canyons. They transfer the load of the main beam to the bridge towers through cables, combining economy with good seismic performance, and have become the mainstream choice for extra-large span bridges. As the core load-bearing component of cable-stayed bridges, the health of the cables directly determines the overall operational safety of the bridge. Cables exposed to the natural environment for a long time are subject to the coupled effects of wind vibration, rain vibration, and corrosion. Their polyethylene sheaths are prone to aging, cracking, and other defects, which in turn lead to internal steel wire corrosion and wire breakage, seriously threatening the structural safety of the bridge. To reduce the wind vibration effect, cables with irregular cross-sections such as spiral patterns and indentations have been widely used in newly built bridges. However, these special surface structures also bring new technical challenges to cable inspection.

[0003] Currently, cable inspection of cable-stayed bridges is still mainly done manually. Typically, a winch is used to pull a basket carrying inspectors to work along the cable. This method has problems such as low work efficiency, high safety risks, and easy secondary damage to the cable sheath.

[0004] While existing wheeled cable-climbing robots have achieved a certain degree of automated inspection, they still have several technical shortcomings. Most are only equipped with a single vision or ultrasonic inspection module, making it difficult to simultaneously correlate surface defects, internal damage, and cable stress status. The inspection results lack overall consistency. Furthermore, during the robot's crawling along the cable, slippage and vibration can cause significant spatial positional deviations in the data collected by different sensors, making it difficult to accurately match multi-source data at the same location. This can easily lead to missed defects or misaligned correlations. Secondly, there are also shortcomings in adaptability to irregularly shaped cables. For the widely used low-wind-vibration cables with spiral protrusions, traditional fixed wheel gauge clamping structures are prone to scratching the polyethylene sheath during crawling, and body posture fluctuations can easily lead to a decrease in the stability of inspection data. As a result, existing cable inspection technologies are difficult to adapt to the maintenance needs of large-scale bridges. Summary of the Invention

[0005] To address the shortcomings mentioned in the background technology, we propose a vision-based cable inspection robot and system for cable-stayed bridges.

[0006] The technical solution mainly consists of a vision-based cable-stayed bridge cable inspection system, including a vision recognition module, a multi-sensor spatiotemporal registration module, a phased array ultrasonic 3D imaging module, a fiber optic cable force acquisition module, a residual strength assessment module, and a decision output module.

[0007] The visual recognition module is configured to acquire multispectral images and infrared thermal imaging data of the cable surface of the cable-stayed bridge to identify the location, type and severity of surface defects.

[0008] The multi-sensor spatiotemporal registration module is configured to align the surface defect data of the visual recognition module with the internal damage data of the phased array ultrasonic three-dimensional imaging module, and correct the position deviation during the robot's crawling process through visual feature anchor point recognition and position calibration logic.

[0009] The phased array ultrasonic 3D imaging module is configured to acquire ultrasonic data inside the cable, generate a 3D image of the internal damage through a fully focused phased array ultrasonic module and full-focus imaging technology, and output the geometric feature parameters of the damage.

[0010] The fiber optic cable force acquisition module is configured to acquire real-time cable force data through a fiber optic grating sensor, and use the wavelength shift of reflected light caused by the periodic change of the grating during cable deformation to infer the tension magnitude, and output a high-precision cable force value to the remaining strength assessment module.

[0011] The remaining strength assessment module is configured to combine the damage parameters from the phased array ultrasonic three-dimensional imaging module with real-time cable force data to calculate the cable's remaining strength and safety factor.

[0012] The decision output module is configured to correlate surface defect and internal damage data, combine them with safety factor assessment results to generate maintenance recommendations, and output an inspection report.

[0013] Preferably, the visual recognition module includes a multispectral adaptive illumination unit, a 4K high-definition image acquisition unit, an infrared thermal imaging unit, and an AI defect recognition unit.

[0014] The multispectral adaptive supplementary lighting unit is configured to adaptively adjust the supplementary lighting spectrum according to ambient light and visibility parameters, covering the visible light, near-infrared and mid-infrared bands, and enhancing the contrast of cable surface defect features.

[0015] The 4K high-definition image acquisition unit is configured to acquire high-resolution visible light images of the cable surface and identify the morphological features of surface cracks, scratches, and pits.

[0016] The infrared thermal imaging unit is configured to collect infrared thermal imaging data of the cable surface, extract temperature anomaly features at defects, and assist in identifying potential internal corrosion and cracks.

[0017] The AI ​​defect recognition unit is configured to input multispectral images and infrared thermal imaging data into a pre-trained deep learning model and output the type, pixel-level coordinates, and severity rating of surface defects.

[0018] Preferably, the multi-sensor spatiotemporal registration module includes a visual feature anchor point recognition unit, a multi-source data synchronous acquisition unit, and a position calibration unit.

[0019] The visual feature anchor point recognition unit is configured to identify cable clamps and pre-made marks on the cable surface as feature anchor points, and convert the anchor point pixel coordinates obtained by the 4K high-definition image acquisition unit into three-dimensional positions in the robot's local coordinate system.

[0020] The multi-source data synchronous acquisition unit is configured to synchronously acquire ultrasonic data from the phased array ultrasonic three-dimensional imaging module, cable force data from the fiber optic cable force acquisition module, and visual image data, and mark the acquisition timestamps of all data.

[0021] The position calibration unit is configured to combine the robot's roller circumference, encoder pulse count, and cable tilt angle parameters to calculate the robot's actual axial displacement relative to the feature anchor point, calibrate the acquisition position of ultrasonic and cable force data, achieve spatial coordinate unification of multi-source data, and map the spatial coordinates of the calibrated ultrasonic data and cable force data with the pixel-level coordinates of surface defects output by the visual recognition module to the same global coordinate system, so as to achieve a one-to-one correspondence between surface and internal damage data at the same location.

[0022] Preferably, the phased array ultrasound three-dimensional imaging module includes a fully focusing phased array ultrasound module, a three-dimensional imaging processing unit, and a damage feature extraction unit.

[0023] The fully focused phased array ultrasonic module is configured to emit multi-angle ultrasonic beams and receive reflected echo signals from inside the cable, enabling full-section coverage detection inside the cable.

[0024] The three-dimensional imaging processing unit is configured to perform full matrix capture processing on the echo signals acquired by the fully focused phased array ultrasonic module to reconstruct a three-dimensional image of the internal damage of the cable.

[0025] The damage feature extraction unit is configured to segment the damage area from the three-dimensional image, extract the depth, area, and volume geometric feature parameters of the damage, and distinguish between two types of damage: broken wire and corrosion.

[0026] Preferably, the residual strength assessment module includes a damage volume quantification unit, a residual effective area calculation unit, a temperature correction calculation unit, and a safety factor calculation unit.

[0027] The damage volume quantization unit is configured to calculate the total damage volume based on the three-dimensional geometric feature parameters of the damage, and quantify the severity of internal wire breakage and corrosion through a voxel integral algorithm.

[0028] The remaining effective area calculation unit is configured to combine the volume and area conversion factors corresponding to the two types of damage, wire breakage and corrosion, to convert the total damage volume into the cross-sectional area loss, and to obtain the remaining effective cross-sectional area by subtracting the cross-sectional area loss from the original cross-sectional area of ​​the cable.

[0029] The temperature correction calculation unit is configured to collect current ambient temperature data, calculate the difference between the measured temperature and the design reference temperature, and obtain the temperature correction coefficient by combining the elastic modulus and linear expansion coefficient parameters of the steel wire.

[0030] The safety factor calculation unit is configured to combine the remaining effective cross-sectional area, the ultimate tensile strength of the steel wire, the real-time cable force output by the fiber optic grating cable force acquisition module, and the temperature correction coefficient parameters, multiply the remaining effective cross-sectional area of ​​the cable by the ultimate tensile strength of the steel wire, and then divide by the real-time cable force after temperature correction to obtain the safety factor.

[0031] Preferably, the decision output module includes a damage correlation analysis unit, a maintenance suggestion matching unit, and a test report generation unit.

[0032] The damage correlation analysis unit is configured to correlate surface defect data and internal damage data at the same spatial location to determine the causal correspondence between surface defects and internal damage.

[0033] The maintenance suggestion matching unit is configured to preset three threshold intervals based on the bridge cable design specifications. According to the threshold interval where the safety factor is located, it is divided into normal interval, warning interval and dangerous interval from high to low, and corresponding to different priority maintenance treatment plans, including three treatment levels: sheath repair, local reinforcement and cable replacement. The interval division is based on the corresponding cable design safety margin level.

[0034] The inspection report generation unit is configured to integrate defect location, type, severity, safety factor, and maintenance recommendation data to generate a structured inspection report and store it synchronously in the bridge operation and maintenance management platform.

[0035] Preferably, it also includes an intelligent navigation and obstacle avoidance module, which comprises an environmental perception unit, a path planning unit, and a high-precision positioning unit.

[0036] The environmental perception unit is configured to carry a lidar and a binocular vision camera to construct a three-dimensional environmental map of the cable surface and the area required for radial operation detection, and to identify the location, size and type of obstacles in real time.

[0037] The path planning unit is configured to dynamically adjust the robot's crawling speed along the cable axis and the circumferential clamping offset angle based on the 3D environment map and obstacle recognition results, so as to avoid obstacles and ensure that the detection path covers the entire circumference of the cable surface.

[0038] The high-precision positioning unit is configured to use a combination of global positioning system and inertial measurement unit for navigation. It combines the preset coordinate points of the bridge tower with RFID tags at key locations of the cable for position correction, achieving centimeter-level calibration of the damage location. The calibration results are output to the multi-sensor spatiotemporal registration module for data position calibration.

[0039] The RFID tags are pre-installed at key reference positions on the bridge tower and cables, storing global coordinate information of the corresponding positions. When the robot identifies the RFID tag, it performs a difference calibration between the current positioning coordinates and the coordinates stored in the tag.

[0040] The present invention also provides a visual recognition-based cable inspection robot for cable-stayed bridges, comprising a robot body, wherein an adaptive flexible gripping device is integrated on the robot body, and the adaptive flexible gripping device comprises a guide wheel assembly, a driven wheel assembly, a pneumatic flexible gripper, and a dynamic balance control unit.

[0041] The driven wheel assembly is equipped with a drive cylinder assembly, and the robot body is equipped with multiple sets of sensors. The robot body detects the spacing and height of the spiral protrusions on the cable surface through multiple sets of sensors, and automatically adjusts the extension and retraction of the driven wheel assembly through the drive cylinder assembly to adapt to the smooth climbing of the spiral cable structure.

[0042] The guide wheel assembly is equipped with a gear drive mechanism, which drives the guide wheel assembly to move the robot body on the cable.

[0043] The pneumatic flexible gripper uses a bellows-type pneumatic muscle as the drive unit, and the gripping force is controlled by an air pressure regulating valve. The gripper surface is covered with a flexible silicone layer.

[0044] The dynamic balance control unit has a built-in three-axis accelerometer and PID controller to monitor the robot's posture data in real time, adjust the gripper angle, and maintain the robot's horizontal posture during the climbing process.

[0045] Preferably, the robot body also integrates a data acquisition unit, a control and processing unit, and an instruction execution unit.

[0046] The data acquisition unit is configured to acquire all detection data from the visual recognition module, the phased array ultrasonic three-dimensional imaging module, the fiber optic cable force acquisition module, and the intelligent navigation and obstacle avoidance module in real time, and simultaneously mark the timestamps.

[0047] The control and processing unit is configured to process the collected detection data and generate clamping force adjustment, ultrasonic acquisition, and robot movement control commands.

[0048] The instruction execution unit is configured to send control instructions to the corresponding actuators to achieve fully automatic closed-loop control of the detection process.

[0049] Compared with the prior art, the beneficial effects of the present invention are:

[0050] This invention integrates three detection technologies: visual recognition, phased array ultrasound, and fiber optic cable force monitoring. It breaks through the limitations of traditional detection schemes that can only cover single-dimensional defects, achieving full-dimensional coverage from surface defects to internal wire damage, and from static structural state to dynamic stress characteristics. Through multi-sensor spatiotemporal registration technology, it solves the problem of multi-source data position deviation caused by slippage and vibration during robot crawling, and realizes accurate correlation between surface defects and internal damage at the same location. It provides multi-dimensional and traceable complete data support for cable health status assessment, while greatly improving the comprehensiveness and accuracy of the assessment results.

[0051] This invention constructs a fully automated detection closed loop, encompassing path planning, autonomous cable climbing, multi-source data acquisition, intelligent analysis and evaluation, and maintenance decision output. The adaptive flexible clamping device can adapt to cables of different diameters and surface structures, ensuring stable climbing while avoiding secondary damage to the cable sheath. The intelligent navigation and obstacle avoidance module can adapt to complex working environments, achieving blind-spot-free full-coverage detection. Compared to traditional manual suspended platforms and high-altitude operations, this invention significantly reduces the safety risks of high-altitude operations, improving detection efficiency while ensuring the safety and standardization of the operation process. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating the operation of the vision-based cable detection system for cable-stayed bridges in this invention.

[0053] Figure 2 This is a flowchart illustrating the operation of residual strength assessment and safety decision-making in this invention;

[0054] Figure 3 This is a first-view schematic diagram of the cable-stayed bridge cable inspection robot based on vision recognition in this invention.

[0055] Figure 4 This is a second-view schematic diagram of the cable inspection robot for cable-stayed bridges based on vision recognition in this invention. Detailed Implementation

[0056] The present invention will now be described in further detail with reference to the accompanying drawings and preferred embodiments.

[0057] Example 1, refer to Figure 1-4As shown, the cable-stayed bridge cable inspection system based on vision recognition includes a vision recognition module, a multi-sensor spatiotemporal registration module, a phased array ultrasonic three-dimensional imaging module, a fiber optic cable force acquisition module, a residual strength assessment module, a decision output module, and an intelligent navigation and obstacle avoidance module. Through the coordinated linkage of these modules, a complete chain inspection system is constructed, from surface defect identification, internal damage detection, real-time cable force acquisition, multi-source data registration, residual strength assessment to maintenance decision output, realizing fully automatic and high-precision comprehensive assessment of cable condition.

[0058] In this implementation scheme, when the system is running, the intelligent navigation and obstacle avoidance module first plans the detection path and locates the robot's position, drives the robot to crawl smoothly along the cable, and simultaneously triggers the visual recognition module to collect multispectral images and infrared thermal imaging data of the cable surface to identify the location, type and severity of surface defects.

[0059] The phased array ultrasonic 3D imaging module synchronously acquires ultrasonic data inside the cable, generates a 3D image of the internal damage, and outputs the geometric feature parameters of the damage. The fiber optic cable force acquisition module synchronously acquires real-time cable force data and outputs it to the residual strength assessment module.

[0060] The multi-sensor spatiotemporal registration module aligns surface defect data with internal damage data, corrects positional deviations during robot crawling, and achieves spatiotemporal coordinate unification of multi-source data. The remaining strength assessment module combines damage parameters with real-time cable force data to calculate the cable's remaining strength and safety factor.

[0061] Finally, the decision output module correlates surface defect and internal damage data, combines the safety factor assessment results to generate maintenance recommendations, and outputs an inspection report.

[0062] Through the orderly collaboration of multiple modules, the integration and unified analysis of three types of detection data—surface defects, internal damage, and cable stress status—is achieved. This breaks through the limitation of traditional detection technologies that can only cover a single detection dimension, forming a complete closed-loop detection process from data acquisition and status assessment to decision output. Automated detection of the entire cable can be completed without human intervention, greatly improving detection efficiency and the comprehensiveness of assessment.

[0063] In this embodiment, the positional error caused by slippage and vibration during robot crawling is eliminated by the multi-source data registration mechanism, and the accurate correlation between surface defects and internal damage at the same location is achieved, providing accurate and reliable data support for the source analysis of cable damage and the full life cycle management.

[0064] Example 2, refer to Figure 1-4As shown, the multi-sensor spatiotemporal registration module includes a visual feature anchor point recognition unit, a multi-source data synchronous acquisition unit, and a position calibration unit. The phased array ultrasonic three-dimensional imaging module includes a fully focused phased array ultrasonic module, a three-dimensional imaging processing unit, and a damage feature extraction unit. The multi-sensor spatiotemporal registration module is linked with the phased array ultrasonic three-dimensional imaging module and the residual intensity assessment module to achieve accurate spatiotemporal alignment of multi-source detection data and quantification of damage status.

[0065] During system operation, the visual feature anchor point recognition unit of the multi-sensor spatiotemporal registration module first identifies the cable clamps and pre-made marks on the cable surface as feature anchor points, converts the anchor point pixel coordinates obtained by the 4K high-definition image acquisition unit into three-dimensional positions in the robot's local coordinate system, and obtains the axial position of the feature anchor points. The multi-source data synchronous acquisition unit synchronously acquires ultrasonic data from the phased array ultrasonic three-dimensional imaging module, cable force data from the fiber optic cable force acquisition module, and visual image data, and marks all data with a unified acquisition timestamp.

[0066] The position calibration unit combines the robot roller circumference, the number of pulses the roller rotates after the feature anchor point is identified, the total number of encoder pulses corresponding to one rotation of the roller, and the cable tilt angle parameter to calculate the axial position after ultrasonic-cable force data calibration. The specific formula is as follows:

[0067] ;

[0068] In the formula, P s The axial position after calibration of the calculated ultrasonic-cable force data represents the actual axial distance between the detection point and the lower end anchor point of the cable.

[0069] P c The axial position of the surface feature anchor point identified by vision is the reference coordinate for multi-source data calibration. The 4K high-definition image acquisition unit identifies feature anchor points such as cable clamps and pre-made marks on the cable surface and converts the pixel coordinates of the anchor points into the axial position in the robot's local coordinate system.

[0070] L v The circumference of the robot's roller is the basic hardware parameter for calculating the robot's crawling distance;

[0071] N r The number of pulses that the roller rotates after the feature anchor point is identified reflects the encoder count corresponding to the robot's crawling distance after the feature anchor point is identified. It is collected in real time by the encoder built into the robot's roller and the number of pulses is accumulated from the moment the feature anchor point is identified until the target detection point is reached.

[0072] N t This represents the total number of encoder pulses corresponding to one revolution of the roller, and is an inherent parameter of the encoder used to convert the number of pulses into the actual rotation distance.

[0073] α is the tilt angle of the cable, that is, the angle between the cable axis and the horizontal plane, which is used to convert the crawling distance of the robot along the cable surface into axial displacement. cosα is the cosine of the tilt angle, which is the distance conversion coefficient, and projects the crawling distance along the cable surface into the axial displacement of the cable.

[0074] In the formula, The crawling distance of the robot after the feature anchor point is identified is multiplied by cosα to convert the crawling distance into the axial displacement of the cable, and then compared with the position P of the feature anchor point. c The axial position after calibration is obtained by adding the data from other sensors. After calibration, the spatial coordinates of the ultrasonic data and cable force data are mapped to the same global coordinate system with the pixel-level coordinates of the surface defects output by the visual recognition module, so as to achieve a one-to-one correspondence between the surface and internal damage data at the same location.

[0075] Simultaneously, the fully focused phased array ultrasonic module of the phased array ultrasonic three-dimensional imaging module emits multi-angle ultrasonic beams and receives the reflected echo signals inside the cable, realizing full cross-sectional coverage detection inside the cable. The three-dimensional imaging processing unit performs full matrix capture processing on the echo signals to reconstruct a three-dimensional image of the damage inside the cable. The damage feature extraction unit segments the damage area from the three-dimensional image, extracts the equivalent diameter and type coefficient of all damage points, and outputs them to the residual strength assessment module.

[0076] The damage volume quantification unit of the remaining strength assessment module counts the total number of internal damage points detected at this location. The remaining effective area calculation unit, combined with the original total cross-sectional area of ​​all wires in the cable, calculates the remaining effective cross-sectional area of ​​the wire at the damage location. The specific calculation formula is as follows:

[0077] ;

[0078] In the formula, A e The remaining effective cross-sectional area of ​​the steel wire at the calculated damage location represents the effective cross-sectional area of ​​the steel wire in the cable that can still withstand tensile force after deducting all internal damage losses. It is the core parameter for evaluating the cable's load-bearing capacity. The closer the value is to A0, the less severe the damage.

[0079] A0 is the original total cross-sectional area of ​​all steel wires in the cable, which is the theoretical total cross-sectional area when the cable has not suffered any damage. It is the benchmark value for calculating the cross-sectional area.

[0080] n represents the total number of internal damage points detected at this location. The total number of all independent damage regions segmented in the 3D reconstructed image is counted, including two types of damage: broken wires and corrosion.

[0081] d i The equivalent diameter of the i-th damage point is used to represent the irregularly shaped damage region as the diameter of a circular region, and is used to calculate the cross-sectional area loss of a single damage.

[0082] k i The type coefficient for the i-th damage point distinguishes the effects of two types of damage: wire breakage and corrosion.

[0083] When the damage is a broken filament, k i =1 indicates that the steel wire in this area has completely lost its load-bearing capacity and the entire cross-sectional area has been lost.

[0084] When the damage is corrosion, k i The percentage of the corroded volume to the total volume of the steel wire section (range 0-1) is obtained by measuring the volume ratio of the corroded area using ultrasonic three-dimensional imaging, reflecting the partial cross-sectional area loss caused by corrosion.

[0085] In the formula, The theoretical cross-sectional area loss for a single damage point is the circular cross-sectional area calculated based on the equivalent diameter, reflecting the maximum theoretical area loss of the damage.

[0086] The total cross-sectional area loss at this location is calculated by summing the corrected cross-sectional area loss values ​​for all damage points, resulting in the total load-bearing area loss of the entire tested section.

[0087] By using the position calibration logic of feature anchor points, the problem of multi-source data position deviation caused by robot crawling slippage and vibration is fundamentally solved, achieving accurate and unified spatial coordinates of detection data from different sensors, avoiding the problem of misalignment between surface defects and internal damage, and providing an accurate positional basis for subsequent damage quantification and intensity assessment.

[0088] In this embodiment, the damage quantification logic based on ultrasonic three-dimensional imaging can distinguish the different effects of two types of damage: wire breakage and corrosion. It can accurately calculate the remaining effective cross-sectional area, greatly improving the accuracy of damage quantification and providing reliable input parameters for subsequent safety factor calculation.

[0089] Example 3, referring to Figure 1-4 As shown, the remaining strength assessment module includes a damage volume quantification unit, a remaining effective area calculation unit, a temperature correction calculation unit, and a safety factor calculation unit. The decision output module includes a damage correlation analysis unit, a maintenance suggestion matching unit, and a test report generation unit. The remaining strength assessment module and the decision output module work together to achieve accurate assessment of the cable's safety status and automatic generation of maintenance decisions.

[0090] During system operation, the temperature correction calculation unit of the residual strength assessment module collects the current ambient temperature data, calculates the difference between the measured temperature and the design reference temperature, and calculates the temperature correction coefficient by combining the parameters of the steel wire's elastic modulus and linear expansion coefficient. The specific calculation formula is as follows:

[0091] ;

[0092] In the formula, γ T The calculated temperature correction factor is used to eliminate the influence of ambient temperature changes on the actual stress calculation of the cable. After correction, the equivalent stress of the cable at the design reference temperature can be obtained, ensuring that the safety factor calculation is not affected by the ambient temperature during the test, and the value is always greater than 0.

[0093] When the temperature is higher than the reference temperature, γ T >1, when the temperature is lower than the reference temperature, γ T <1;

[0094] E is the elastic modulus of steel wire, which reflects the ratio of stress to strain in the steel wire material within the elastic range. It is an inherent physical parameter of high-strength steel wire used in bridge cables.

[0095] α s The coefficient of linear expansion of steel wire reflects the relative elongation of the steel wire when the temperature changes by 1°C. It is an inherent thermophysical parameter of high-strength steel wire.

[0096] △T is the difference between the measured temperature and the design reference temperature, △T = T 实测 -T 基准 , among which, T 实测 The ambient temperature during testing is collected in real time by a temperature sensor mounted on the robot. 基准 This is the reference temperature for cable design, usually taken as 20℃, and is a known design parameter.

[0097] Furthermore, in the formula, α s ×△T×E represents the correction factor for the elastic modulus of the steel wire caused by temperature changes, reflecting the degree of influence of temperature changes on the load-bearing capacity of the steel wire. The load-bearing capacity of the steel wire decreases slightly with increasing temperature. This parameter is used to quantify this effect. E-α s ×△T×E is the equivalent elastic modulus of the steel wire after temperature correction, which is the actual elastic bearing capacity parameter of the steel wire after considering the effect of temperature.

[0098] In this embodiment, the safety factor calculation unit calculates the safety factor at the damaged location by combining the remaining effective cross-sectional area of ​​the steel wire at the damaged location, the ultimate tensile strength of the steel wire, the real-time cable force measured by the fiber optic grating sensor, and the temperature correction factor. The specific calculation formula is as follows:

[0099] ;

[0100] In the formula, S f The safety factor for the calculated damage location is the ratio of the cable's ultimate bearing capacity to the actual force, reflecting the cable's current safety margin. It is a core indicator for assessing the cable's health status and making maintenance decisions.

[0101] When Sf ≥ First safety threshold: Within the normal range, no additional action is required, just periodic re-inspection;

[0102] When the second security threshold ≤ S f <First safety threshold: The area is in the warning zone and requires treatment such as sheath repair and local reinforcement;

[0103] When S f <Second safety threshold: The area is in danger and the cable needs to be replaced;>

[0104] Among them, the first safety threshold and the second safety threshold are pre-calibrated according to the design specifications and safety redundancy requirements of the corresponding cables. Cables of different grades and application scenarios can be matched with different threshold settings to adapt to the operation and maintenance management standards of various bridges.

[0105] A e γ represents the remaining effective cross-sectional area of ​​the wire at the location of the damage. T This is a temperature correction factor;

[0106] f u The ultimate tensile strength of steel wire reflects the maximum stress that the steel wire material can withstand before it breaks. It is an inherent mechanical property parameter of high-strength steel wire used in bridge cables.

[0107] F t The real-time cable force is collected by a fiber optic grating sensor and obtained by inversely calculating the wavelength shift of reflected light caused by the periodic change of the grating during cable deformation, reflecting the actual stress state of the cable at the time of detection.

[0108] In the formula, A e ×f u The ultimate bearing capacity of the cable at the damaged location represents the maximum tensile force that the cable at that damaged location can withstand, and is a direct quantitative result of the cable's remaining strength.

[0109] F t ×γ T To correct for the actual cable stress caused by temperature, the measured cable force at the test temperature is converted into the equivalent force at the design reference temperature, eliminating the interference of temperature fluctuations on the cable force and making the test results at different temperatures comparable.

[0110] Simultaneously, the damage correlation analysis unit of the decision output module associates surface defect data and internal damage data at the same spatial location. When the coordinates of the surface defect location and the internal damage location match, and the surface defect type and the internal damage type have a corresponding correlation, it is determined that there is a causal correspondence between the two.

[0111] The maintenance suggestion matching unit is based on three preset threshold ranges. According to the range where the safety factor is located, it matches different priority maintenance and treatment plans. The normal range corresponds to no additional treatment and periodic re-inspection. The warning range corresponds to the treatment level of sheath repair and local reinforcement. The dangerous range corresponds to the treatment level of cable replacement. The range division is based on the corresponding cable design safety margin level.

[0112] The final inspection report generation unit integrates data on defect location, type, severity, safety factor, and maintenance recommendations to generate a structured inspection report, which is then synchronously stored in the bridge operation and maintenance management platform.

[0113] By using temperature-corrected safety factor calculation logic, the influence of ambient temperature changes on cable force and cable bearing capacity is eliminated, so that the safety factor assessment results are not affected by environmental conditions during testing. The accuracy and applicability of the assessment results are greatly improved, and they can be directly used as the basis for operation and maintenance decisions.

[0114] In this embodiment, by using the preset mapping relationship between the safety factor threshold and the maintenance plan, the automatic conversion from detection results to maintenance decisions is realized, avoiding the subjectivity and experience differences of manual assessment. The output maintenance suggestions and handling levels are highly standardized and can be directly matched with different levels of operation and maintenance response processes, greatly improving the efficiency of operation and maintenance decision-making.

[0115] Example 4, refer to Figure 1-4 As shown, the environmental perception unit of the intelligent navigation and obstacle avoidance module is equipped with a lidar and a binocular vision camera to construct a three-dimensional environmental map of the cable surface and the radial operation detection range, and to identify the location, size and type of obstacles in real time.

[0116] Before the robot starts climbing, the environmental perception unit of the intelligent navigation and obstacle avoidance module first completes an initial environmental scan using LiDAR and binocular vision camera. It integrates the depth information of the LiDAR point cloud with the texture information of the binocular camera to construct a three-dimensional point cloud map of the cable surface and the radial working range. Through a pre-trained instance segmentation model, it identifies obstacle categories in real time, distinguishes three common obstacles: cable entanglement, attachments, and structural protrusions, and outputs the minimum bounding box size and relative coordinates of each obstacle simultaneously.

[0117] The path planning unit inputs the 3D environment map and obstacle information into the adaptive path planning algorithm, dynamically allocates weights according to the obstacle type, and outputs the path planning result to the robot controller. The controller dynamically adjusts the clamping offset angle and crawling speed of the adaptive flexible gripping device according to the path adjustment command to achieve a balance between obstacle avoidance and full-circumference detection: for rigid structural protrusions, the circumferential clamping offset angle is adjusted first to avoid them, and for soft attachments, the crawling speed is reduced first to pass through them. While avoiding obstacles, the pitch of the detection path is kept constant, ensuring that there are no blind spots in the detection of the entire circumference of the cable.

[0118] The high-precision positioning unit outputs real-time pose based on GPS (Global Positioning System) + IMU (Inertial Measurement Unit) combined navigation. When the robot travels to key reference positions such as bridge towers and cable clamps, it automatically reads the global coordinate information stored in the pre-installed RFID tags, calculates the difference between the current positioning coordinates and the tag coordinates, corrects the accumulated error of the IMU caused by long-distance crawling, and finally outputs the damage location calibration result with centimeter-level accuracy. It is then pushed to the multi-sensor spatiotemporal registration module as a global reference for multi-source data alignment.

[0119] In this embodiment, the obstacle recognition accuracy is greatly improved by using a multimodal environment perception scheme of "LiDAR + binocular vision", which can adapt to the obstacle avoidance needs of complex working environments such as strong light, backlight, rain and fog.

[0120] In this embodiment, the cumulative positioning error of long-distance crawling is controlled within the centimeter range by intermittent position correction of RFID tags. The damage location can be accurately calibrated without the need for additional ground positioning base stations, and the detected defect location can be directly matched with the coordinate system of the bridge operation and maintenance management system.

[0121] Example 5, refer to Figure 1-4 As shown, the present invention also provides a cable-stayed bridge cable inspection robot based on vision recognition, equipped with a cable-stayed bridge cable inspection system based on vision recognition, including a robot body 1, on which an adaptive flexible gripping device is integrated, the adaptive flexible gripping device including a guide wheel group 2, a driven wheel group 3, a pneumatic flexible gripper 4, and a dynamic balance control unit 7.

[0122] Driven wheel assembly 2 is equipped with drive cylinder assembly 6. The robot body 1 detects the spacing and height of the spiral protrusions on the cable surface through sensors, and automatically adjusts the extension and retraction of driven wheel assembly 3 through drive cylinder assembly 6 to adapt to the smooth climbing of the spiral structure cable.

[0123] The guide wheel assembly 2 is equipped with a gear drive mechanism 5, which drives the guide wheel assembly 2 to move the robot body 1 on the cable;

[0124] The pneumatic flexible gripper 4 uses a bellows-type pneumatic muscle as the drive unit, and controls the gripping force through a pneumatic pressure regulating valve. The gripper surface is covered with a flexible silicone layer.

[0125] The dynamic balance control unit 7 has a built-in three-axis accelerometer and PID controller to monitor the robot's posture data in real time, adjust the gripper angle, and maintain the robot's horizontal posture during the climbing process.

[0126] Specifically, the robot body 1 adopts a modular splicing ring frame structure, which can quickly open and close to hold cables of different diameters. The driven wheel group 3 is linked with the drive cylinder group 6. During the robot's climbing process, the sensors on the robot body 1 collect the height and spacing pulse signals of the spiral protrusions on the cable surface in real time. The controller automatically outputs air pressure commands according to the pulse characteristics to adjust the output pressure of the drive cylinder group 6, which drives the driven wheel group 3 to make periodic extension and retraction movements that match the spiral protrusions, thus counteracting the climbing fluctuations caused by the spiral structure. The guide wheel group 2 outputs power through the planetary gear drive mechanism to drive the whole machine to move along the cable axis. The pneumatic flexible gripper 4 uses a bellows-type pneumatic muscle as a power source and dynamically adjusts the gripping air pressure according to the cable diameter and surface friction coefficient. The surface of the gripper in contact with the cable is covered with a flexible silicone layer to avoid damage to the sheath. The dynamic balance control unit collects the roll and pitch angle data of the robot body in real time through a three-axis accelerometer and automatically corrects the robot body attitude to a horizontal state to prevent the whole machine from tipping over.

[0127] The robot body 1 also integrates a data acquisition unit, a control and calculation unit, and an instruction execution unit: the data acquisition unit collects all detection data from the vision recognition module, the phased array ultrasonic 3D imaging module, the fiber optic cable force acquisition module, and the intelligent navigation and obstacle avoidance module in real time, and simultaneously marks the timestamps; the control and calculation unit processes the collected detection data to generate clamping force adjustment, ultrasonic acquisition, and robot movement control instructions; and the instruction execution unit sends the control instructions to the corresponding actuators to realize fully automatic closed-loop control of the detection process.

[0128] In this embodiment, the multiple sets of wheels of the adaptive flexible gripping device work together with the pneumatic gripper to adapt to cables of different diameters and surface structures. This ensures stable crawling while avoiding secondary damage to the cable sheath. At the same time, the dynamic balance control unit can eliminate posture deviation during the climbing process, greatly improving the robot's operational stability and adaptability on complex cable structures.

[0129] By coordinating the various units, the detection data acquisition, processing, and execution are fully automated and closed-loop controlled. The entire cable can be autonomously inspected without the need for remote real-time manual operation, effectively reducing the risk of human intervention in high-altitude inspections, while improving the standardization and efficiency of the inspection process.

[0130] It should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this invention should also be within the scope of protection of this invention.

Claims

1. A cable inspection system for cable-stayed bridges based on visual recognition, characterized in that: It includes a visual recognition module, a multi-sensor spatiotemporal registration module, a phased array ultrasonic 3D imaging module, a fiber optic cable force acquisition module, a residual strength assessment module, and a decision output module. The visual recognition module is configured to acquire multispectral images and infrared thermal imaging data of the cable surface of the cable-stayed bridge to identify the location, type and severity of surface defects. The multi-sensor spatiotemporal registration module is configured to align the surface defect data of the visual recognition module with the internal damage data of the phased array ultrasonic three-dimensional imaging module, and correct the position deviation during the robot's crawling process through visual feature anchor point recognition and position calibration logic. The phased array ultrasonic 3D imaging module is configured to acquire ultrasonic data inside the cable, generate a 3D image of the internal damage through a fully focused phased array ultrasonic module and full-focus imaging technology, and output the geometric feature parameters of the damage. The fiber optic cable force acquisition module is configured to acquire real-time cable force data through a fiber optic grating sensor, and use the wavelength shift of reflected light caused by the periodic change of the grating during cable deformation to infer the tension magnitude, and output a high-precision cable force value to the remaining strength assessment module. The remaining strength assessment module is configured to combine the damage parameters from the phased array ultrasonic three-dimensional imaging module with real-time cable force data to calculate the cable's remaining strength and safety factor. The decision output module is configured to correlate surface defect and internal damage data, combine them with safety factor assessment results to generate maintenance recommendations, and output an inspection report.

2. The cable detection system for cable-stayed bridges based on visual recognition according to claim 1, characterized in that: The visual recognition module includes a multispectral adaptive illumination unit, a 4K high-definition image acquisition unit, an infrared thermal imaging unit, and an AI defect recognition unit. The multispectral adaptive supplementary lighting unit is configured to adaptively adjust the supplementary lighting spectrum according to ambient light and visibility parameters, covering the visible light, near-infrared and mid-infrared bands, and enhancing the contrast of cable surface defect features. The 4K high-definition image acquisition unit is configured to acquire high-resolution visible light images of the cable surface and identify the morphological features of surface cracks, scratches, and pits. The infrared thermal imaging unit is configured to collect infrared thermal imaging data of the cable surface, extract temperature anomaly features at defects, and assist in identifying potential internal corrosion and cracks. The AI ​​defect recognition unit is configured to input multispectral images and infrared thermal imaging data into a pre-trained deep learning model and output the type, pixel-level coordinates, and severity rating of surface defects.

3. The cable detection system for cable-stayed bridges based on visual recognition according to claim 1, characterized in that: The multi-sensor spatiotemporal registration module includes a visual feature anchor point recognition unit, a multi-source data synchronous acquisition unit, and a position calibration unit. The visual feature anchor point recognition unit is configured to identify cable clamps and pre-made marks on the cable surface as feature anchor points, and convert the anchor point pixel coordinates obtained by the 4K high-definition image acquisition unit into three-dimensional positions in the robot's local coordinate system. The multi-source data synchronous acquisition unit is configured to synchronously acquire ultrasonic data from the phased array ultrasonic three-dimensional imaging module, cable force data from the fiber optic cable force acquisition module, and visual image data, and mark the acquisition timestamps of all data. The position calibration unit is configured to combine the robot's roller circumference, encoder pulse count, and cable tilt angle parameters to calculate the robot's actual axial displacement relative to the feature anchor point, calibrate the acquisition position of ultrasonic and cable force data, achieve spatial coordinate unification of multi-source data, and map the spatial coordinates of the calibrated ultrasonic data and cable force data with the pixel-level coordinates of surface defects output by the visual recognition module to the same global coordinate system, so as to achieve a one-to-one correspondence between surface and internal damage data at the same location.

4. The cable detection system for cable-stayed bridges based on visual recognition according to claim 1, characterized in that: The phased array ultrasound three-dimensional imaging module includes a fully focusing phased array ultrasound module, a three-dimensional imaging processing unit, and a damage feature extraction unit. The fully focused phased array ultrasonic module is configured to emit multi-angle ultrasonic beams and receive reflected echo signals from inside the cable, enabling full-section coverage detection inside the cable. The three-dimensional imaging processing unit is configured to perform full matrix capture processing on the echo signals acquired by the fully focused phased array ultrasonic module to reconstruct a three-dimensional image of the internal damage of the cable. The damage feature extraction unit is configured to segment the damage area from the three-dimensional image, extract the depth, area, and volume geometric feature parameters of the damage, and distinguish between two types of damage: broken wire and corrosion.

5. The cable detection system for cable-stayed bridges based on visual recognition according to claim 4, characterized in that: The residual strength assessment module includes a damage volume quantification unit, a residual effective area calculation unit, a temperature correction calculation unit, and a safety factor calculation unit. The damage volume quantization unit is configured to calculate the total damage volume based on the three-dimensional geometric feature parameters of the damage, and quantify the severity of internal wire breakage and corrosion through a voxel integral algorithm. The remaining effective area calculation unit is configured to combine the volume and area conversion factors corresponding to the two types of damage, wire breakage and corrosion, to convert the total damage volume into the cross-sectional area loss, and to obtain the remaining effective cross-sectional area by subtracting the cross-sectional area loss from the original cross-sectional area of ​​the cable. The temperature correction calculation unit is configured to collect current ambient temperature data, calculate the difference between the measured temperature and the design reference temperature, and obtain the temperature correction coefficient by combining the elastic modulus and linear expansion coefficient parameters of the steel wire. The safety factor calculation unit is configured to combine the remaining effective cross-sectional area, the ultimate tensile strength of the steel wire, the real-time cable force output by the fiber optic grating cable force acquisition module, and the temperature correction coefficient parameters, multiply the remaining effective cross-sectional area of ​​the cable by the ultimate tensile strength of the steel wire, and then divide by the real-time cable force after temperature correction to obtain the safety factor.

6. The cable detection system for cable-stayed bridges based on vision recognition according to claim 5, characterized in that: The decision output module includes a damage correlation analysis unit, a maintenance suggestion matching unit, and a test report generation unit. The damage correlation analysis unit is configured to correlate surface defect data and internal damage data at the same spatial location to determine the causal correspondence between surface defects and internal damage. The maintenance suggestion matching unit is configured to preset three threshold intervals based on the bridge cable design specifications. According to the threshold interval where the safety factor is located, it is divided into normal interval, warning interval and dangerous interval from high to low, and corresponding to different priority maintenance treatment plans, including three treatment levels: sheath repair, local reinforcement and cable replacement. The interval division is based on the corresponding cable design safety margin level. The inspection report generation unit is configured to integrate defect location, type, severity, safety factor, and maintenance recommendation data to generate a structured inspection report and store it synchronously in the bridge operation and maintenance management platform.

7. The cable detection system for cable-stayed bridges based on visual recognition according to claim 1, characterized in that: It also includes an intelligent navigation and obstacle avoidance module, which comprises an environmental perception unit, a path planning unit, and a high-precision positioning unit. The environmental perception unit is configured to carry a lidar and a binocular vision camera to construct a three-dimensional environmental map of the cable surface and the area required for radial operation detection, and to identify the location, size and type of obstacles in real time. The path planning unit is configured to dynamically adjust the robot's crawling speed along the cable axis and the circumferential clamping offset angle based on the 3D environment map and obstacle recognition results, so as to avoid obstacles and ensure that the detection path covers the entire circumference of the cable surface. The high-precision positioning unit is configured to use a combination of global positioning system and inertial measurement unit for navigation. It combines the preset coordinate points of the bridge tower with RFID tags at key locations of the cable for position correction, achieving centimeter-level calibration of the damage location. The calibration results are output to the multi-sensor spatiotemporal registration module for data position calibration. The RFID tags are pre-installed at key reference positions on the bridge tower and cables, storing global coordinate information of the corresponding positions. When the robot identifies the RFID tag, it performs a difference calibration between the current positioning coordinates and the coordinates stored in the tag.

8. A vision-based cable-stayed bridge cable inspection robot, equipped with the vision-based cable inspection system for cable-stayed bridges as described in any one of claims 1-7, characterized in that: The robot body (1) is integrated with an adaptive flexible gripping device, which includes a guide wheel assembly (2), a driven wheel assembly (3), a pneumatic flexible gripper (4), and a dynamic balance control unit (7). The driven wheel assembly (3) is equipped with a drive cylinder assembly (6), and the robot body (1) is equipped with multiple sets of sensors. The robot body (1) detects the spacing and height of the spiral protrusions on the cable surface through multiple sets of sensors, and automatically adjusts the extension and retraction of the driven wheel assembly (3) through the drive cylinder assembly (6) to adapt to the smooth climbing of the spiral structure cable. The guide wheel assembly (2) is provided with a gear drive mechanism (5), which drives the guide wheel assembly (2) to move the robot body (1) on the cable; The pneumatic flexible gripper (4) uses a bellows-type pneumatic muscle as the driving unit, and the gripping force is controlled by a pneumatic pressure regulating valve. The gripper surface is covered with a flexible silicone layer. The dynamic balance control unit (7) has a built-in three-axis acceleration sensor and PID controller to monitor the robot's posture data in real time, adjust the gripper angle, and maintain the horizontal posture of the robot body (1) during the climbing process.

9. The cable-stayed bridge cable inspection robot based on vision recognition according to claim 8, characterized in that: The robot body (1) also integrates a data acquisition unit, a control and calculation unit, and an instruction execution unit: The data acquisition unit is configured to acquire all detection data from the visual recognition module, the phased array ultrasonic three-dimensional imaging module, the fiber optic cable force acquisition module, and the intelligent navigation and obstacle avoidance module in real time, and simultaneously mark the timestamps. The control and processing unit is configured to process the collected detection data and generate clamping force adjustment, ultrasonic acquisition, and robot movement control commands. The instruction execution unit is configured to send control instructions to the corresponding actuators to achieve fully automatic closed-loop control of the detection process.

Images