A visual detection and location method for bridge cable defects

Abstract

The present invention discloses a method for visual detection and positioning of bridge cable defects. Through video data and known cable diameters, the camera's field of view size and position changes are calculated to achieve real-time detection and positioning of defects. The specific steps include: Camera field of view calculation: Obtain the camera's internal parameter matrix, and calculate the vertical distance from the camera to the cable and the axial length of the cable in the field of view by identifying key points in the cable image. Real-time camera positioning: Continuously process image frames in the video, calculate the vertical pixel change between adjacent frames through template matching, and calculate the actual camera position change according to the formula to accumulate and obtain the current camera position. Defect detection and positioning: Use the YOLOV8 algorithm to perform defect detection on the initial frame, and set the trigger condition. When the camera's cumulative movement exceeds 90% of the axial length of the field of view, perform defect detection again. Record the type of defect and its location to ensure the efficiency and accuracy of detection.

Description

Technical Field

[0001] The invention belongs to the field of cable bridge maintenance, and in particular relates to a visual detection and positioning method for bridge cable diseases. Background Art

[0002] As a new form of modern bridge, cable-stayed bridges have been widely used around the world. However, as one of the main load-bearing components of cable-stayed bridges, the cables are exposed to the air for a long time, and are subjected to wind, rain and sun, which causes the polyethylene sheath on the outer surface of the cables to harden and crack to varying degrees, causing serious problems such as rust and broken wires to the steel wire bundles in the sheath.

[0003] In the past, telescope observation and manual hanging basket observation were mainly used to detect defects on the outer surface of cables. However, although the telescope is a commonly used auxiliary inspection tool, it is not completely suitable for cable inspection. The manual hanging basket method is very dangerous and may affect traffic during the inspection process, especially on bridges, where gusts are common and the wind is strong, and there are dangerous situations such as the hanging basket shaking. Especially when inspecting the top of the cable at a height of 100 meters, the manual hanging basket method is less efficient. Therefore, these manual inspection methods have obvious defects and safety hazards, and require a lot of manpower and material resources.

[0004] The current mainstream method for detecting surface defects of cables is to use a cable-climbing robot. The robot is equipped with a camera and can perform a comprehensive inspection of defects on the cable surface. The encoder on the robot determines the location of the defect on the cable for subsequent repair work. Compared with manual inspection, this method avoids the risk of personnel safety, shortens the inspection cycle, and improves inspection efficiency.

[0005] One implementation of the cable-climbing robot detection is that the cable-climbing robot transmits the collected image data and encoder position data to the ground station in real time. The operator of the ground station immediately pauses or retracts the robot according to the defects found in the image, then captures the screen image at the defect location and records the encoder position data. This detection method has high requirements for operators and takes a long time on site, which is not conducive to rapid operation.

[0006] Another more mainstream cable-climbing robot detection solution is that the cable-climbing robot uses a camera to directly collect and save the video data of the entire cable, and then uses an image recognition algorithm to identify defects in the cable video frame by frame. Although this processing method is more efficient, there is a problem that it is difficult to obtain the location data of the defect because the location data is stored in the encoder rather than in the video. A method to roughly estimate the position data is to calculate the location of the defect based on the average speed of the cable-climbing robot, the initial frame time of the cable-climbing robot's operation, and the time when the defect frame appears. However, in actual data collection, it is difficult for the cable-climbing robot to maintain a constant speed and uniform linear motion. During movement, it may encounter obstacles and slip, or require manual intervention to pause or retract the robot. These factors will lead to large deviations in position estimation. Summary of the invention

[0007] The present invention aims to solve the technical problems existing in the background technology and to provide a visual detection and positioning method for bridge cable defects.

[0008] In order to solve the technical problem, the technical solution of the present invention is:

[0009] A visual detection and positioning method for bridge cable defects, the method comprising:

[0010] Obtain the camera's internal parameter data and calculate the vertical distance d from the camera to the cable based on the cable radius;

[0011] Based on the vertical distance d, the axial length of the cable in the field of view along the cable direction and the axial distance L corresponding to each pixel are calculated;

[0012] Calculate the vertical pixel change between adjacent frames, calculate the actual position change of the camera between two adjacent frames based on the vertical pixel change and the axial distance corresponding to each pixel, and accumulate the position changes of two adjacent frames to obtain the position of the current camera relative to the starting point;

[0013] After real-time positioning of the camera relative to the starting point, the cable disease recognition algorithm is used to detect cable diseases in the cable image, thus achieving real-time detection and positioning of cable diseases.

[0014] Furthermore, the camera's internal parameter data (fx, fy, cx, cy) is obtained, including:

[0015] Obtain the camera's intrinsic parameter matrix M through calibration methods or using the data provided by the manufacturer: [fx, 0, cx; 0, fy, cy; 0, 0, 1];

[0016] The pixel point P'=[u; v; 1] in the image is converted to the point P=[x; y; z] in the camera coordinate system using the intrinsic parameter matrix, and the formula is P'=M*P / z.

[0017] Furthermore, by obtaining the radius R of the cable, the vertical distance d from the camera to the cable is calculated:

[0018] After identifying the pixel coordinates of the edge points on the left and right sides of the cable in the image, the intrinsic parameter matrix is ​​used to project them into rays in the camera coordinate system, the angle α between the two rays is calculated, and the vertical distance d is calculated according to the formula d=[R / sin(α / 2)]-R.

[0019] Furthermore, the axial length L of the cable in the field of view along the cable direction and the axial distance corresponding to each pixel are calculated:

[0020] Determine the perpendicular direction n from the camera to the cable surface and the rays n1 and n2 at the upper and lower edges of the cable in the field of view, calculate the angles β1 and β2 between the perpendicular direction n and the rays n1 and n2, and calculate the axial length of the field of view according to the formula L=L1+L2=d*tan(β1)+d*tan(β2);

[0021] Then calculate the axial distance corresponding to each pixel: the distance corresponding to each pixel is L / h, where h is the image pixel height.

[0022] Further, the vertical pixel change dv between adjacent frames is calculated, including:

[0023] In the previous frame image, a plurality of pixel templates of a certain size are intercepted in the central axis area of ​​the cable, and their coordinate positions are recorded;

[0024] Perform template matching in the specified area of ​​the next frame image, and for each template, perform template matching in the specified area of ​​the current frame; record the templates whose matching scores are greater than a preset threshold and their vertical changes as good matches;

[0025] The average value of the vertical pixel changes of all well-matched templates is calculated as dv.

[0026] Furthermore, the size of the template is 50*50 pixels.

[0027] Furthermore, the specified area is an image area covered by the template position moving 10 pixels to the left and 40 pixels to the top and bottom.

[0028] Furthermore, the preset threshold is set to 0.8.

[0029] Furthermore, the formula ds=dv*L / h is used to calculate the camera position change ds between two adjacent frames; starting from the initial frame, the value of ds is accumulated to obtain the current camera position relative to the starting point.

[0030] Furthermore, the real-time detection and positioning of cable defects includes performing defect detection at a certain interval; and the defect detection algorithm is YOLOV8.

[0031] Compared with the prior art, the advantages of the present invention are:

[0032] An efficient pure visual cable disease detection and positioning method based on a monocular camera is proposed. It does not require additional hardware support and simplifies the complexity and cost of the system. By calculating the vertical distance between the camera and the cable and the axial length of the cable in the field of view, accurate geometric relationship measurement is achieved, providing a reliable basis for disease detection and positioning. Template matching technology is used to calculate the position change of the camera in the video in real time, and the camera is accurately positioned relative to the starting point. A method is designed to perform disease detection only when the camera moves more than 0.9 times the longitudinal field of view, which effectively saves computing resources and reduces computing power consumption. This method not only ensures the comprehensiveness of disease detection, but also avoids the problem of multiple detections of the same disease, reducing the workload of subsequent manual processing. By calculating the update degree of the cable image in the camera field of view, it is ensured that the cable image is fully updated for more than 90% before disease detection is performed, ensuring high reliability and accuracy in the detection process. The YOLOV8 algorithm is used for disease detection, which ensures the accuracy and real-time performance of the detection and can efficiently identify various types of cable diseases. The detected disease type and its position relative to the starting point are recorded in real time to facilitate subsequent maintenance and management. It is suitable for cable disease detection in various complex environments, does not need to rely on external environment or other auxiliary equipment, and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 ,Keypoints marking map in the cable image;

[0034] Figure 2 , geometric relationship diagram for calculating the vertical distance from camera to cable;

[0035] Figure 3 : Calculation diagram of the axial length of the cable within the camera field of view;

[0036] Figure 4 , Typical result diagram of template matching between adjacent frames. DETAILED DESCRIPTION

[0037] The specific implementation mode of the present invention is described below in conjunction with embodiments:

[0038] It should be noted that the structures, proportions, sizes, etc. shown in this specification are only used to match the contents disclosed in the specification so that people familiar with this technology can understand and read them, and are not used to limit the conditions under which the present invention can be implemented. Any structural modification, change in proportional relationship or adjustment of size should still fall within the scope of the technical content disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0039] At the same time, the terms such as "upper", "lower", "left", "right", "middle" and "one" cited in this specification are only for the convenience of description and are not used to limit the scope of implementation of the present invention. Changes or adjustments to their relative relationships should be regarded as the scope of implementation of the present invention without substantially changing the technical content.

[0040] Embodiment 1:

[0041] The present invention provides a visual detection and location method for cable defects, which can not only identify defects but also locate the locations of defects based on cable video data and known cable diameters. Specifically, the method includes the following steps:

[0042] 1. Use a picture to calculate the camera's field of view

[0043] Step 101: Get the camera's internal reference data (fx, fy, cx, cy). You can calibrate it yourself (such as Zhang Zhengyou's calibration method) or directly use the data provided by the manufacturer.

[0044] The camera's intrinsic parameter matrix is ​​a 3*3 matrix M, which is [fx, 0, cx; 0, fy, cy; 0, 0, 1]. A point P = [x; y; z] in the camera coordinate system (a three-dimensional space coordinate system established with the camera as the origin) corresponds to a point P' = [u; v; 1] in the camera image pixel, where u and v are the horizontal and vertical coordinates in the image. Using the camera's intrinsic parameter matrix, the pixel point and the space point can be converted using the formula P' = M*P / z.

[0045] Step 102: Using the known radius R of the cable, calculate the vertical distance d from the camera to the cable.

[0046] Figure 1 is the image of the cable actually captured by the camera. When the longitudinal position in the image is determined to be cy, the left and right edge points of the cable are as follows: Figure 1 Marked. Use the internal parameter matrix in step 1 to project these two key points into two rays in the camera coordinate system, and then calculate the angle between the two rays as α. Figure 2 The geometric relationship between the camera and the cable can be calculated as d = [R / sin(α / 2)]-R

[0047] Step 103: Calculate the length L of the cable axial direction covered by the camera field of view along the cable axial direction (axial direction).

[0048] First calculate the direction of the perpendicular line from the camera to the cable surface, that is Figure 2 The direction n of the bisector of the median angle α.

[0049] as follows Figure 3As shown, two pixel points at the upper and lower edges of the cable in the camera field of view are selected, and the pixel coordinates of these two points are converted into two rays n1 and n2 in the camera coordinate system using the camera intrinsic parameter matrix in step 1. The angles between the vertical direction n from the camera to the cable surface and the rays n1 and n2 are calculated as β1 and β2 respectively.

[0050] Then according to the geometric relationship, we can get the length of the camera field of view along the axial direction of the cable: L=L1+L2=d*tan(β1)+d*tan(β2).

[0051] Step 4: For the middle area of ​​the cable, calculate the distance corresponding to each pixel along the axial direction as L / h, where h is the height of the image (measured in pixels).

[0052] 2. Real-time positioning of the current camera in the video relative to the starting point.

[0053] Step 201: Calculate the vertical pixel change dv of the pixels in the middle area of ​​the cable between two adjacent image frames. The specific method is to intercept a lot of 50 pixel * 50 pixel square images as templates for the previous frame image, and record their coordinate positions in the image. For each 50 pixel * 50 pixel template, perform template matching in a specified area in the next frame image (if the position of the upper left corner point of the template in the previous frame is u1, v1, when the point moves 10 pixels to the left and right and 40 pixels up and down, the area covered by the template is taken as the specified area). For templates with a matching score greater than 0.8, it is regarded as a good match, and the pixels that change in the vertical direction are recorded. Typical matching results are as follows. Figure 4 The average value of all well-matched vertical pixel changes is taken as the pixel value direction change dv of the two frames of images.

[0054] Step 202: Calculate the change in camera position between two adjacent frames ds=dv*L / h.

[0055] Step 203: Starting from the initial frame, the changes in the camera positions of two adjacent frames are accumulated to obtain the position of the current camera relative to the starting point along the axial direction of the cable.

[0056] There is a close relationship between the real-time positioning of the camera relative to the starting point and the disease detection step. Specifically:

[0057] Position calculation as a trigger for disease detection:

[0058] The camera position change ds and cumulative position change are calculated in real time to determine whether the camera moves more than 0.9L.

[0059] The next disease detection is triggered only when the camera moves more than 0.9 L. This trigger condition relies on the real-time calculation of the camera position.

[0060] The effectiveness of disease detection relies on accurate camera positioning:

[0061] Accurate camera position calculation ensures that each disease detection is performed in a new area within the camera's field of view, avoiding repeated detection of the same disease and improving detection efficiency.

[0062] Accumulated location updates ensure continuous monitoring:

[0063] Accumulate camera position changes to ensure that disease detection can cover the entire cable area and achieve continuous monitoring and positioning.

[0064] In summary, real-time positioning of the camera relative to the starting point is a key step in the disease detection process, ensuring effective triggering and efficient positioning of the detection.

[0065] 3. Detect and locate cable defects in the video in real time.

[0066] According to the above scheme, the current camera position is calculated in real time; a defect detection is performed in the initial frame, and if the moving distance exceeds 0.9 times the axial length of the cable in the field of view, that is, exceeds 0.9*L, another detection is performed, and this is repeated. The defect detection algorithm is implemented using YOLOV8 with self-collected data and training. If a defect is detected, the type of the defect and its position relative to the starting point are recorded.

[0067] Initialization phase:

[0068] Initialize the starting point of the camera and set the initial frame as the reference frame.

[0069] Use the YOLOV8 algorithm to detect defects on the initial frame and record the defect type and its position relative to the starting point.

[0070] Continuous frame processing:

[0071] Process each frame of the video, set the frame as the current frame (the current frame number is n), and the previous frame as n-1.

[0072] In the previous frame image, multiple 50 pixel * 50 pixel square image blocks are intercepted as templates, and the coordinate position of each template is recorded.

[0073] Template Matching:

[0074] For each template, template matching is performed in the specified area in the current frame. The specified area is the area covered by the template when the upper left corner point in the previous frame is (u1, v1), and the left and right sides are moved 10 pixels and the upper and lower sides are moved 40 pixels.

[0075] Calculate the template matching score, record the templates with matching scores greater than 0.8, and record their vertical pixel change dv.

[0076] Calculate the camera position change:

[0077] The average of the vertical pixel changes of all well-matched templates is taken as the pixel value direction change dv of the two frames of images.

[0078] The change in the camera position between two adjacent frames is calculated according to the formula ds=dv*L / h, where L is the actual length of the cable axis in the field of view, and h is the image height.

[0079] Starting from the initial frame, the changes in the camera positions of two adjacent frames are accumulated to obtain the position of the camera in the current frame relative to the starting point.

[0080] Disease detection trigger conditions:

[0081] Determine whether the cumulative change in the camera position between the current frame and the last frame for disease detection exceeds 0.9 times the axial length L of the cable in the field of view.

[0082] If the cumulative change exceeds 0.9*L, a disease detection is performed and the type of disease detected and its position relative to the starting point are recorded.

[0083] Update the frame of the last disease detection to the current frame.

[0084] Disease detection and recording:

[0085] Use the YOLOV8 algorithm to detect defects in the current frame that meets the conditions.

[0086] If a disease is detected, the type of disease and its position relative to the starting point are recorded, including the frame number and corresponding position of the disease.

[0087] Loop processing:

[0088] Continue to process each frame of the video and repeat the above steps to update the camera position and detect diseases in real time.

[0089] The preferred embodiments of the present invention are described in detail above, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

[0090] Many other changes and modifications may be made without departing from the concept and scope of the present invention.It should be understood that the present invention is not limited to the specific embodiments, and the scope of the present invention is defined by the appended claims.

Claims

1. A visual detection and positioning method for bridge cable defects, characterized in that: The method comprises: Obtain the camera's internal parameter data and calculate the vertical distance d from the camera to the cable based on the cable radius; Based on the vertical distance d, the axial length of the cable in the field of view along the cable direction and the axial distance L corresponding to each pixel are calculated; Calculate the vertical pixel change between adjacent frames, calculate the actual position change of the camera between two adjacent frames based on the vertical pixel change and the axial distance corresponding to each pixel, and accumulate the position changes of two adjacent frames to obtain the position of the current camera relative to the starting point; After real-time positioning of the camera relative to the starting point, the cable disease recognition algorithm is used to detect cable diseases in the cable image, thus achieving real-time detection and positioning of cable diseases.

2. A visual detection and positioning method for bridge cable defects according to claim 1, characterized in that: Get the camera's internal parameter data (fx, fy, cx, cy), including: Obtain the camera's intrinsic parameter matrix M through calibration methods or using the data provided by the manufacturer: [fx, 0, cx; 0, fy, cy; 0, 0, 1]; The pixel point P'=[u; v; 1] in the image is converted to the point P=[x; y; z] in the camera coordinate system using the intrinsic parameter matrix, and the formula is P'=M*P / z.

3. A visual detection and positioning method for bridge cable defects according to claim 1, characterized in that: By obtaining the radius R of the cable, calculate the vertical distance d from the camera to the cable: After identifying the pixel coordinates of the edge points on the left and right sides of the cable in the image, the intrinsic parameter matrix is ​​used to project them into rays in the camera coordinate system, the angle α between the two rays is calculated, and the vertical distance d is calculated according to the formula d=[R / sin(α / 2)]-R.

4. A visual detection and positioning method for bridge cable defects according to claim 1, characterized in that: Calculate the axial length L of the cable in the field of view along the cable direction and calculate the axial distance corresponding to each pixel: Determine the perpendicular direction n from the camera to the cable surface and the rays n1 and n2 at the upper and lower edges of the cable in the field of view, calculate the angles β1 and β2 between the perpendicular direction n and the rays n1 and n2, and calculate the axial length of the field of view according to the formula L=L1+L2=d*tan(β1)+d*tan(β2); Then calculate the axial distance corresponding to each pixel: the distance corresponding to each pixel is L / h, where h is the image pixel height.

5. A visual detection and positioning method for bridge cable defects according to claim 4, characterized in that: Calculate the vertical pixel change dv between adjacent frames, including: In the previous frame image, a plurality of pixel templates of a certain size are intercepted in the central axis area of ​​the cable, and their coordinate positions are recorded; Perform template matching in a specified area of ​​the next frame image; record the templates whose matching scores are greater than a preset threshold and their vertical changes as good matches; The average value of the vertical pixel changes of all well-matched templates is calculated as dv.

6. A visual detection and positioning method for bridge cable defects according to claim 5, characterized in that: The size of the template is 50*50 pixels.

7. A visual detection and positioning method for bridge cable defects according to claim 5, characterized in that: The specified area is an image area covered by the template position moving 10 pixels to the left and 40 pixels to the top and bottom.

8. The method for visually detecting and locating bridge cable defects according to claim 5, characterized in that: The preset threshold is set to 0.

8.

9. A visual detection and positioning method for bridge cable defects according to claim 5, characterized in that: Use the formula ds=dv*L / h to calculate the camera position change ds between two adjacent frames; Starting from the initial frame, the value of ds is accumulated to obtain the current position of the camera relative to the starting point.

10. A visual detection and positioning method for bridge cable defects according to claim 9, characterized in that: The real-time detection and positioning of cable diseases includes performing disease detection at a certain interval, and the disease detection algorithm is YOLOV8.

Images