Real-time cable surface defect detection method for bridge inspection robot

By installing defect detection cameras and image processing algorithms on bridge inspection robots, the problems of low efficiency and low accuracy in bridge cable inspection have been solved, enabling real-time and accurate defect detection and reducing the safety hazards of manual inspection.

CN117974541BActive Publication Date: 2024-11-12南京乐道机器人科技有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310837564.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-07-10
Publication Date
2024-11-12
Estimated Expiration
2043-07-10

AI Technical Summary

Technical Problem

Existing technologies for bridge cable-stayed cable inspection suffer from low efficiency, low accuracy, and poor real-time performance, and manual inspection also poses safety hazards.

Method used

Four defect detection cameras are installed at four circumferentially divided positions on the cable inspection robot. Combined with a long-distance image transmission and reception module and a remote recognition host, real-time detection of surface defects in the cable is achieved through image processing algorithms, including steps such as grayscale conversion, boundary extraction, filtering, image enhancement, region growing, and differential operation.

Benefits of technology

It enables rapid and accurate identification of surface defects in cables, reduces the workload of manual inspection, improves inspection efficiency and accuracy, and provides accurate location information for subsequent maintenance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117974541B_ABST
    Figure CN117974541B_ABST
Patent Text Reader

Abstract

The present invention provides a method for real-time cable surface defect detection for a bridge inspection robot, comprising the steps of (1): the bridge inspection robot collects an original image of the cable surface, (2): a host receiving module receives the original image and converts the image into a one-channel grayscale image, (3): determining the rectangular area to which the cable belongs in the image by column grayscale values ​​and extracting the image, (4): filtering the image extracted in step (3) by an adaptive median filtering algorithm, (5): using a contrast-limited adaptive histogram equalization (CLAHE) algorithm to enhance the image filtered in step (4), (6): extracting and segmenting the defective area on the cable surface, and (7): using a cable surface image in a defect-free state as a reference model, performing a differential operation with the cable image segmented in step (6), generating a binary image, and saving the defective image in step (8).
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of bridge inspection robots, and in particular, relates to a real-time cable surface defect detection method for a bridge inspection robot. Background Art

[0002] The cable-stayed bridge is composed of three basic structures: tower, beam and cable. The cable is an extremely important load-bearing component. The dead load and load of the bridge are transmitted to the tower column through the cable, and its construction cost accounts for about 25%-30% of the whole bridge. Since the cable is subjected to alternating stress caused by the changing load on the bridge deck for a long time and is directly exposed to the air and the adverse effects of the environment, it will suffer damage of different types and degrees. When the PE sheath is severely damaged and the internal steel wire bundle is directly exposed to the air, it will rust or even break due to moisture and acidic substances in the air, seriously endangering the safety of the bridge.

[0003] In order to prevent the cable from being damaged to the point where it must be replaced, which will cause greater losses, it is necessary to detect and repair the defects of the cable in the early stage. Many cables still use traditional manual inspection methods, which are inefficient, have low inspection quality, and have certain safety hazards. At the same time, the operation of setting up the winch equipment on the top of the bridge is complicated, and the inspection basket is heavy, which may cause additional damage to the PE sheath during the rolling process along the cable.

[0004] The bridge cable inspection robot is an aerial work robot that effectively replaces manual inspection. At present, the robot's automated cable defect recognition and inspection is still a difficulty and research focus in its on-site operation and application process. The surface defect detection of bridge cables still has the problems of inaccuracy, low precision and poor real-time performance. The fast and stable defect detection method based on machine vision is the key means to realize the robot's automated inspection operation, reduce inspection costs and improve efficiency. Summary of the invention

[0005] To solve the above problems, the present invention discloses a real-time cable surface defect detection method for a bridge inspection robot, which solves the problem of inaccurate and low-precision identification of cable surface defects, and at the same time combines with the inspection robot to realize real-time defect detection of cable defects.

[0006] A method for real-time cable surface defect detection by a bridge inspection robot, the method is implemented by the following device, the device includes four defect detection cameras, a cable inspection robot, a long-distance image sending module, a long-distance image receiving module, and a remote identification host, the four defect detection cameras are installed at the cable inspection robot at four equal circumferential positions to capture cable surface images, the detection cameras transmit the long-distance image to the module via a signal interface, and the remote identification host is connected to the long-distance image receiving module via a USB, the method realizes real-time detection of cable surface defects by acquiring real-time cable surface images and executing a defect detection algorithm, the method includes the following steps;

[0007] Step (1): The bridge inspection robot collects 360-degree original images of the cable surface through four defect detection cameras, and synthesizes them into one image and sends it through a long-distance image sending module;

[0008] Step (2): The remote recognition host receives the original image collected by the detection robot through the long-distance image receiving module, and converts the color RGB three-channel image into a one-channel grayscale image. The grayscale conversion process is based on:

[0009] G r =0.321·R t +0.602 G t +0.121 B t

[0010] Among them, G r Represents the gray value of the processed image, R t , G t , B t Represent the RGB values ​​of the pixels in the original image respectively;

[0011] Step (3): Based on the large grayscale value of the cable image, the rectangular area to which the cable belongs in the image is determined by listing the grayscale values ​​to reduce the interference of irrelevant areas on subsequent image processing, and the Grab Cut algorithm is combined to extract the final boundary area of ​​the cable. The boundary area is extracted based on the mixed Gaussian model, and the energy formula used is:

[0012]

[0013] in, The regional data term representing the energy function, represents the boundary term of the energy function, and α is the correlation coefficient; Step (4): filter the image extracted in step (3) by using an adaptive median filter algorithm or a recursive average filter algorithm to remove the salt and pepper noise caused by high-altitude water mist and the Gaussian noise generated by signal transmission, thereby reducing the impact of noise on subsequent surface defect recognition. The filtering formula is:

[0014] G (x,y) =median{f(xk,yl),(k,l)∈W},(x,y)∈D;

[0015] Among them, G (x,y) represents the filtered image, f(x, y) represents the image pixel value obtained in the previous step, median{} represents the median of the pixel, W is the filter window size range and (k, l)∈W; Step (5): According to the uneven illumination characteristics of the cable image, the contrast-limited adaptive histogram equalization CLAHE algorithm is used to enhance the image filtered in step (4), highlight the surface defects of the cable, and weaken the irrelevant areas;

[0016] Step (6): according to the range of gray values ​​of defective pixels in the cable image, multiple groups of gray values ​​of defective pixels are set as seed points, and the defective area on the cable surface is extracted and segmented using the region growing method;

[0017] Step (7): The cable surface image without defects is used as a reference model, and a differential operation is performed with the cable image segmented in step (6) to weaken the influence of the image pixels of the normal cable surface, and generate a binary image to highlight the pixels belonging to the defects. The differential operation formula is:

[0018]

[0019] in, Represents the final cable defect differential image, is the segmented image generated in step (6), A pre-set image of the cable surface in a defect-free state;

[0020] Step (8): According to the ratio between the actual size of the defect and the image pixel, the size of the connected range of the defective pixel is determined. When the size of the defect range in the differential image generated in step (7) is greater than 8 mm, the pixel area is considered to be a surface defect of the cable. Otherwise, it is considered that there is no defect.

[0021] Step (9): If the remote host recognizes that there is a defect on the cable surface, save the defect image of step (8) and the corresponding original image, and mark the position information on the image according to the cable number and running distance of the robot. If no cable defect is recognized, do not save it. Return to step (1) to perform cable defect detection at the next position.

[0022] Furthermore, the specific method for determining the rectangular area in step (3) is to first set the corresponding pixel width to s columns according to the cable diameter, and then calculate the sum of multiple groups of grayscale values ​​according to the formula:

[0023] i is the pixel column value of the image, w t is the image pixel width value;

[0024] When Sum i When is the maximum, the pixels in the i-th column are the left boundary of the rectangular area, and the pixels in the i+s-1-th column are the right boundary of the rectangular area.

[0025] Furthermore, in the step (6), the region growing method uses the grayscale values ​​of multiple groups of defective pixels that are preset as initial seed points, and performs four region growing operations, with the region growing thresholds of 10, 6, 5, and 2 respectively for each operation.

[0026] Beneficial effects of the present invention:

[0027] (1) By determining the rectangular area to which the cable belongs in the image through the column grayscale value and using the Grab Cu algorithm to extract the final boundary area of ​​the cable, the interference of irrelevant areas on subsequent image processing can be effectively reduced, and the algorithm processing time can be reduced.

[0028] (2) The processing method that combines the adaptive median filtering algorithm, CLAHE algorithm, region growing method and differential operation method performs image processing at multiple levels, effectively reducing the interference and noise in the image and improving the accuracy of cable surface defect segmentation and identification.

[0029] (3) The cable surface defect detection method is used to realize the rapid detection of cable surface defects by the cable detection robot, which effectively replaces human eye recognition and reduces the workload of cable detection.

[0030] (4) Using this method, the inspection robot can quickly complete the bridge cable inspection operation, identify and save the location image of the bridge cable defects, and provide a basis for subsequent maintenance and inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 , a flow chart of the method of the present invention;

[0032] Figure 2 , the effect diagram of different stages of processing of a single camera image of the cable surface.

[0033] Figure 3 , the effect diagram of defect segmentation and recognition of a single camera cable image. DETAILED DESCRIPTION

[0034] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. It should be noted that the words "front", "rear", "left", "right", "upper" and "lower" used in the following description refer to directions in the accompanying drawings, and the words "inner" and "outer" refer to directions toward or away from the geometric center of a specific component, respectively.

[0035] like Figure 1 As shown, the method for real-time cable surface defect detection of a bridge inspection robot of this embodiment includes the following steps:

[0036] Step (1): The bridge inspection robot collects 360-degree original images of the cable surface through four defect detection cameras, and synthesizes them into one image and sends it through a long-distance image sending module;

[0037] Step (2): The remote recognition host receives the original image collected by the detection robot through the long-distance image receiving module, and converts the color RGB three-channel image into a one-channel grayscale image. The grayscale conversion process is based on:

[0038] G r =0.321·R t +0.602 G t +0.121 B t

[0039] Among them, G r Represents the gray value of the processed image, R t , G t , B t Represent the RGB values ​​of the pixels in the original image respectively;

[0040] Step (3): Based on the large grayscale value of the cable image, the rectangular area to which the cable belongs in the image is determined by listing the grayscale values ​​to reduce the interference of irrelevant areas on subsequent image processing, and the Grab Cut algorithm is combined to extract the final boundary area of ​​the cable. The boundary area is extracted based on the mixed Gaussian model, and the energy formula used is:

[0041]

[0042] in, The regional data term representing the energy function, represents the boundary term of the energy function, α is the correlation coefficient;

[0043] like Figure 2 As shown,

[0044] The specific method for determining the rectangular area in step (3) is to first set the corresponding pixel width to s columns according to the cable diameter, and then calculate the sum of multiple groups of gray values ​​according to the formula:

[0045] i is the pixel column value of the image, w t is the image pixel width value;

[0046] When Sum i When is the maximum, the pixels in the i-th column are the left boundary of the rectangular area, and the pixels in the i+s-1-th column are the right boundary of the rectangular area.

[0047] Step (4): Filter the image extracted in step (3) by using an adaptive median filter algorithm or a recursive average filter algorithm to remove the salt and pepper noise caused by high-altitude water mist and the Gaussian noise generated by signal transmission, thereby reducing the impact of noise on subsequent surface defect recognition. The filtering formula is:

[0048] G (x,y) =median{f(xk,yl),(k,l)∈W},(x,y)∈D;

[0049] Among them, G (x,y) represents the filtered image, f(x, y) represents the image pixel value obtained in the previous step, median{} represents the median of the pixel, W is the filter window size range and (k, l)∈W; Step (5): According to the uneven illumination characteristics of the cable image, the contrast-limited adaptive histogram equalization CLAHE algorithm is used to enhance the image filtered in step (4), highlight the surface defects of the cable, and weaken the irrelevant areas;

[0050] like Figure 3 As shown, step (6): according to the range shown by the grayscale values ​​of the defective pixels in the cable image, multiple groups of grayscale values ​​of the defective pixels are set as seed points, and the defective area on the cable surface is extracted and segmented using the region growing method;

[0051] The region growing method in step (6) is to use the grayscale values ​​of multiple groups of defective pixels that are preset as initial seed points, and perform four region growing operations, with the region growing thresholds of 10, 6, 5, and 2 respectively for each operation.

[0052] Step (7): The cable surface image without defects is used as a reference model, and a differential operation is performed with the cable image segmented in step (6) to weaken the influence of the image pixels of the normal cable surface, and generate a binary image to highlight the pixels belonging to the defects. The differential operation formula is:

[0053]

[0054] in, Represents the final cable defect differential image, is the segmented image generated in step (6), A pre-set image of the cable surface in a defect-free state;

[0055] Step (8): According to the ratio between the actual size of the defect and the image pixel, the size of the connected range of the defective pixel is determined. When the size of the defect range in the differential image generated in step (7) is greater than 8 mm, the pixel area is considered to be a surface defect of the cable. Otherwise, it is considered that there is no defect.

[0056] Step (9): If the remote host recognizes that there is a defect on the cable surface, save the defect image of step (8) and the corresponding original image, and mark the position information on the image according to the cable number and running distance of the robot. If no cable defect is recognized, do not save it. Return to step (1) to perform cable defect detection at the next position.

[0057] The technical means disclosed in the scheme of the present invention are not limited to the technical means disclosed in the above-mentioned implementation mode, but also include technical schemes composed of any combination of the above technical features.

Claims

1. A method for real-time cable surface defect detection by a bridge inspection robot, the method being implemented by the following device, the device comprising four defect detection cameras, a cable inspection robot, a long-distance image sending module, a long-distance image receiving module, and a remote identification host. The four defect detection cameras are installed at four equal circumferential positions of the cable inspection robot to capture cable surface images. The detection cameras transmit the long-distance image to the module via a signal interface, and the remote identification host is connected to the long-distance image receiving module via a USB. The method realizes real-time detection of cable surface defects by acquiring real-time cable surface images and executing defect detection algorithms. Features ,The method includes the following steps; Step (1): The bridge inspection robot collects 360-degree original images of the cable surface through four defect detection cameras, and synthesizes them into one image and sends it through the long-distance image sending module; Step (2): The remote recognition host receives the original image collected by the inspection robot through the long-distance image receiving module, and converts the color RGB three-channel image into a one-channel grayscale image. The grayscale conversion processing is based on: G r =0.321·R t +0.602·G t +0.121·B t Among them, G r Represents the gray value of the processed image, R t , G t , B t Represent the RGB values ​​of the pixels in the original image respectively; Step (3): Based on the large grayscale value of the cable image, the rectangular area to which the cable belongs in the image is determined by listing the grayscale values ​​to reduce the interference of irrelevant areas on subsequent image processing, and the Grab Cut algorithm is combined to extract the final boundary area of ​​the cable. The boundary area is extracted based on the mixed Gaussian model, and the energy formula used is: in, The regional data term representing the energy function, represents the boundary term of the energy function, and α is the correlation coefficient; the specific method for determining the rectangular area in step (3) is to first set the corresponding pixel width to s columns according to the cable diameter, and then calculate the sum of multiple groups of gray values ​​according to the formula: i is the pixel column value of the image, w t is the image pixel width value; when Sum i When it is the largest, the pixels in the i-th column are the left boundary of the rectangular area, and the pixels in the i+s-1-th column are the right boundary of the rectangular area; Step (4): Filter the image extracted in step (3) by using an adaptive median filter algorithm or a recursive average filter algorithm to remove the salt and pepper noise caused by high-altitude water mist and the Gaussian noise generated by signal transmission, thereby reducing the impact of noise on subsequent surface defect recognition. The filtering formula is: G (x,y) =median{f(x-k,y-l),(k,l)∈W},(x,y)∈D Among them, G (x,y) represents the filtered image, f(x, y) represents the image pixel value obtained in the previous step, median{} represents the median of the pixel, W is the filter window size range and (k, l)∈W; Step (5): According to the uneven illumination characteristics of the cable image, the contrast-limited adaptive histogram equalization CLAHE algorithm is used to enhance the image filtered in step (4), highlight the surface defects of the cable, and weaken the irrelevant areas; Step (6): according to the range shown by the grayscale values ​​of the defective pixels in the cable image, multiple groups of grayscale values ​​of the defective pixels are set as seed points, and the defective area on the cable surface is extracted and segmented by using the region growing method; the region growing method in step (6) is to use the grayscale values ​​of the multiple groups of defective pixels set in advance as the initial seed points, and perform four region growing operations, and the thresholds of each region growing operation are 10, 6, 5, and 2 respectively; Step (7): The cable surface image without defects is used as a reference model, and a differential operation is performed with the cable image segmented in step (6) to weaken the influence of the image pixels of the normal cable surface, and generate a binary image to highlight the pixels belonging to the defects. The differential operation formula is: in, Represents the final cable defect differential image, is the segmented image generated in step (6), A pre-set image of the cable surface in a defect-free state; Step (8): Based on the proportional relationship between the actual size of the defect and the image pixels, determine the size of the connected range of the defective pixels. When the defect range size in the differential image generated in step (7) is greater than 8 mm, the pixel area is considered to be a cable surface defect. Otherwise, it is considered that there is no defect. Step (9): If the remote host recognizes that there is a defect on the cable surface, save the defect image of step (8) and the corresponding original image, and mark the position information on the image according to the cable number and running distance of the robot. If no cable defect is identified, do not save it; return to step (1) to perform cable defect detection at the next position.