Main cable erection quality defect online monitoring method and system based on image analysis

Abstract

The invention provides a main cable erection quality defect online monitoring method and system based on image analysis, and belongs to the technical field of suspension bridge construction quality control. Carrying out defect target positioning on the image with the defect in the preprocessed image by utilizing a YOLO deep learning model, then determining a twisting angle in the image with the defect by utilizing Hough transform, determining a surface damage area in the image with the defect by utilizing a region growing algorithm, judging a twisting condition according to the twisting angle, and determining a surface damage area in the image with the defect. The surface damage condition is judged according to the surface damage area, so that online quality monitoring is carried out in the cable strand erecting process; the technical problems that in the prior art, quality inspection is conducted in a manual inspection mode, the defect recognition error is large, and efficiency is low are solved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of construction quality control of suspension bridges, and relates to a quality monitoring technology during construction of a suspension bridge, in particular to a main cable erection quality defect online monitoring method and system based on image analysis. BACKGROUND

[0002] The main cable is the most important and core load-bearing component of a suspension bridge, has a huge tension, and is used to bear all vertical loads generated by the entire bridge deck and loads on the bridge (vehicles, pedestrians, wind, etc.), and transmit the loads to the cable tower and anchorage, and finally to the foundation. Therefore, the construction quality of the main cable is crucial, and the construction quality directly relates to the load-bearing performance and service life of the entire bridge.

[0003] The main cable is composed of a large number of strands, and each strand needs to be pulled along the length of the catwalk by a traction device during construction, hoisted to the position of the main cable tower and / or auxiliary cable tower, and grouped and fixed to form the main cable. During construction, the strands are prone to defects such as twisting (filament dispersion or filament drumming) and surface scratches due to tension, wind and other factors. Therefore, the quality of each strand needs to be checked during traction to ensure that each strand is uniformly stressed on the main cable and to ensure the safety of the suspension bridge.

[0004] In the prior art, the quality detection of each strand during traction is mainly performed by manual inspection. Manual inspection is performed by a patrol personnel observing the appearance of the strand by visual observation or through a telescope. For the preliminary judgment of defective parts, a caliper is used to measure the cross-sectional size of the strand, and the cross-sectional size is used to determine whether the strand has defects such as twisting and surface scratches. For a long-span suspension bridge, the length of the strand reaches thousands of meters, and manual inspection takes several hours, which is low in construction efficiency. In addition, manual observation is difficult to capture small-area surface damage or twisting, and there is a risk of missed detection or false detection. Therefore, the current manual inspection method has the technical problems of large defect identification error and low efficiency. SUMMARY

[0005] In view of the technical problems of large defect identification error and low efficiency in the process of strand traction by manual inspection, the present application provides a main cable erection quality defect online monitoring method and system based on image analysis.

[0006] The application utilizes an image acquisition device to acquire corresponding strand images, after preprocessing, utilizes a YOLO deep learning model to locate defects in the preprocessed images, and then utilizes Hough transformation to determine the kinking angle in the images with defects, and utilizes a region growing algorithm to determine the surface damage area in the images with defects; according to the kinking angle, the kinking condition is determined, and according to the surface damage area, the surface damage condition is determined, so as to perform online quality monitoring during the strand erection process; the technical problems of large defect identification error and low efficiency in the prior art through manual inspection for quality inspection are solved.

[0007] To solve the above technical problems, the application adopts the following technical solutions: The main cable erection quality defect online monitoring method based on image analysis comprises the following steps: S1: Distribute image acquisition devices along the traction direction of the strand, and acquire corresponding strand images through the image acquisition devices during the strand traction process; S2: Preprocess the strand images to obtain preprocessed images; S3: Utilize a YOLO deep learning model to locate defects in the preprocessed images to obtain reprocessed images, wherein the reprocessed images contain images with defects and images without defects; S4: Identify the images with defects in the reprocessed images, utilize Hough transformation to determine the kinking angle in the images with defects, and utilize a region growing algorithm to determine the surface damage area in the images with defects; S5: According to the kinking angle, the kinking condition is determined, and according to the surface damage area, the surface damage condition is determined, so as to perform online quality monitoring during the strand erection process.

[0008] Further limited, step S1 further comprises: dividing the strand into multiple segments along the traction direction of the strand and numbering; the strand images acquired by the image acquisition device contain the number of each segment of the strand.

[0009] Further limited, in step S2, the preprocessing of the strand images comprises: sequentially performing grayscale processing, Gaussian filter processing and histogram equalization processing on the strand images.

[0010] Further limited, in step S4, the utilization of Hough transformation to determine the kinking angle in the images with defects specifically comprises: Performing strand edge extraction on the images with defects; Based on the principle of Hough transformation, establishing a rectangular coordinate system with the center of the image with defects as the origin, and converting the strand edge in the image with defects from the rectangular coordinate space to the polar coordinate space; In the range of 0°~180°, all pixel points of the one or more identified steel wires in the strand edge are traversed, and a corresponding ρ value is calculated, the ρ value being a vertical distance from the origin to the pixel point; Based on the ρ value, an optimal fitting angle of the strand edge is determined according to a mapping relationship between a rectangular coordinate space and a polar coordinate space, as the twisting angle.

[0011] Further limitation, in step S4, the determination of the surface damage area in the image with defects by using the region growing algorithm specifically includes: Taking a seed pixel in the damage area in the image with defects as a starting point, merging adjacent pixels in the image with defects whose gray value difference with the seed pixel is less than 10, determining a damage edge contour, and determining the damage area in the image with defects based on the damage edge contour; Calculating a damage area of the damage area as the surface damage area.

[0012] Further limitation, the surface damage area A s is calculated by the following formula: In the formula, A s is the surface damage area, unit: cm 2 ; B is the number of pixel points in the damage area, unit: ppi, and d is the number of pixel points per square centimeter of the area, unit: cm 2 / ppi.

[0013] Further limitation, in step S5, the determination of the twisting condition according to the twisting angle specifically includes: if the difference between the twisting angle and the reference angle of the strand is ≥3°, there is twisting deformation. In step S5, the determination of the surface damage condition according to the surface damage area specifically includes: if the surface damage area is ≥0.5 cm 2 , it is considered that there is a surface damage defect.

[0014] Further limitation, the image analysis-based main cable erection quality defect online monitoring method further includes S6: based on the numbering of the strands, associating a plurality of continuous strands at the same time, determining whether there is twisting deformation and / or surface damage defect between the two adjacent strands, if so, it indicates that the strand has a defect deterioration trend; otherwise, it indicates that the strand does not have a defect deterioration trend.

[0015] The image analysis-based main cable erection quality defect online monitoring system formed based on the above image analysis-based main cable erection quality defect online monitoring method includes: An image acquisition module: configured to acquire corresponding strand images by image acquisition devices during the strand traction process, wherein the image acquisition devices are distributed along the traction direction of the strand; a pretreatment module configured to perform pretreatment on the strand image to obtain a pretreated image; a defect positioning module configured to perform defect target positioning on the image with defects in the pretreated image by using a YOLO deep learning model to obtain a reprocessed image, wherein the reprocessed image comprises the image with defects and the image without defects; a defect determination module configured to identify the image with defects in the reprocessed image, determine the kink angle in the image with defects by using a Hough transform, and determine the surface damage area in the image with defects by using a region growing algorithm; and a monitoring module configured to judge the kink condition according to the kink angle and judge the surface damage condition according to the surface damage area, so as to perform online quality monitoring during the strand erection process.

[0016] A computer storage medium stores a program file, and the program file is executed to realize the program instructions of the above-mentioned image analysis-based main cable erection quality defect online monitoring method.

[0017] Compared with the prior art, the present application has the following advantages: 1. The image analysis-based main cable erection quality defect online monitoring method of the present application distributes image acquisition devices along the traction direction of the strand, acquires corresponding strand images by the image acquisition devices, performs pretreatment, performs defect target positioning on the image with defects in the pretreated image by using a YOLO deep learning model, determines the kink angle in the image with defects by using a Hough transform, and determines the surface damage area in the image with defects by using a region growing algorithm; judges the kink condition according to the kink angle and judges the surface damage condition according to the surface damage area, so as to perform online quality monitoring during the strand erection process; and solves the technical problems of large defect identification error and low efficiency in the quality inspection by the manual inspection method in the prior art.

[0018] 2. The image analysis-based main cable erection quality defect online monitoring method of the present application divides the strand into multiple segments along the traction direction of the strand and numbers them, which facilitates the analysis of the defect deterioration trend of the strand in the later stage.

[0019] 3. The pretreatment of the strand image of the present application comprises grayscale processing, Gaussian filtering processing and histogram equalization processing on the strand image in sequence, which can realize image denoising on the one hand and better highlight the defects of the strand on the other hand, facilitating the defect target positioning in the later stage. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 FIG. 1 is a schematic diagram of the image analysis-based main cable erection quality defect online monitoring method of the present application; Figure 2 A schematic diagram of the main cable erection quality defect online monitoring system based on image analysis of the present application; Figure 3 A schematic diagram of the image acquisition device layout; Figure 4 A schematic diagram of the cable strand and image acquisition device layout position; Figure 5 A schematic diagram of the main cable erection; Explanation of reference signs: 1-industrial camera, 2-light supplement module, 3-catwalk portal, 4-roller, 5-cable strand, 6-cable tower, 7-main cable, 8-catwalk. DETAILED DESCRIPTION

[0021] The technical solutions of the present application will be further explained and described below in combination with the drawings and examples, but the present application is not limited to the following described embodiments.

[0022] Referring to Figure 5 , the main cable 7 is erected between two cable towers 6, wherein the main cable 7 is composed of a large number of cable strands 5, and each cable strand 5 needs to be pulled along the length of the catwalk 8 by a pulling device during construction. Referring to Figure 3 and Figure 4 , a plurality of catwalk portals 3 are arranged along the length direction of the catwalk 8, i.e. along the pulling direction of the cable strand 5, and specifically, the spacing between adjacent two catwalk portals 3 belongs to the common sense known to those skilled in the art. Image acquisition devices are arranged at both ends of the top of each catwalk portal 3, wherein the image acquisition device is an industrial camera 1, and the camera head of the industrial camera 1 is arranged towards the direction of the cable strand 5. As a preferred embodiment of the present application, there is a 60° angle between the camera head of the industrial camera 1 and the top crossbeam of the catwalk portal 3, and the industrial camera 1 is fixedly connected with the catwalk portal 3 through bolts and ear plates. The present application also provides a light supplement module 2 on the top crossbeam of the catwalk portal 3, which is used for supplementing light to the industrial camera 1, solving the problem of uneven illumination of the industrial camera 1, and ensuring the quality of the cable strand image collected by the industrial camera 1.

[0023] As a preferred embodiment of the present application, two image acquisition devices are arranged at each end of the top of each catwalk portal 3.

[0024] The present application fixedly provides a roller 4 on the catwalk portal 3, which is used to assist the forward sliding of the cable strand 5 during the pulling process of the cable strand 5, and reduce the friction between the cable strand 5 and the roller 4.

[0025] Referring to Figure 1 , the present application proposes a main cable erection quality defect online monitoring method based on image analysis, comprising the following steps: S1: distribute the image acquisition device along the traction direction of the strand 5, and acquire the corresponding strand image through the image acquisition device during the traction of the strand 5; wherein the strand 5 is divided into multiple sections along the traction direction of the strand 5 and numbered; the strand image acquired by the image acquisition device contains the number of each section of the strand 5. Preferably, the strand 5 can be divided into 2 sections, 3 sections, 4 sections, 5 sections, or even more sections. As a preferred embodiment of the present application, the number of sections of the strand 5 is the same as the number of image acquisition devices.

[0026] S2: pre-process the strand image to obtain a pre-processed image; wherein the pre-processing of the strand image includes: sequentially performing grayscale processing, Gaussian filtering processing and histogram equalization processing on the strand image. The grayscale processing is to convert the strand image into a grayscale image, and the conversion formula is: Gray = 0.395R + 0.210G + 0.395B, Gray is the converted grayscale value, R is the red channel, G is the green channel, and B is the blue channel. The grayscale processing preserves the brightness information in the strand image and reduces the color data of the strand image; the Gaussian filtering processing is to denoise the image noise in the strand image caused by dust, vibration and other factors, and highlight the defect features; the histogram equalization processing is to enhance the feature points of the strand image and adjust the grayscale values in the strand image to be uniformly distributed in the 0-255 interval, improve the contrast between the strand 5 and the background, and make the defect features clearer.

[0027] S3: using a YOLO deep learning model to locate the defects in the pre-processed image to obtain a re-processed image, wherein the re-processed image contains images with defects and images without defects; the YOLO deep learning model mainly includes model structure initial construction and model training, the model structure initial construction adopts CSPDarknet-53 as the backbone network, extracts multi-scale features of the image (from 16x16 feature map to 64x64 feature map) through residual connection, the neck of the backbone network adopts PANet network (path aggregation network) to realize high-low layer feature fusion and enhance the recognition ability of small defects (such as 2mm broken wire in the strand 5), and the head of the backbone network adopts decoupled detection head to respectively output class probability, bounding box coordinates and confidence, and improve the positioning accuracy. The training data set contains at least 10,000 strand images during model training, including 50% of the strand images without defects, 25% of the strand images with twisted deformation, and 25% of the strand images with surface damage area. The YOLO deep learning model can accurately locate the defect features in the pre-processed image.

[0028] S4: identify the images with defects in the re-processed image, determine the twist angle in the images with defects by using Hough transform, and determine the surface damage area in the images with defects by using region growing algorithm; The image acquisition device determines the kink angle in the image with defects by using a Hough transform, specifically including: performing strand edge extraction on the image with defects; based on the Hough transform principle, establishing a rectangular coordinate system with the center of the image with defects as the origin, and converting the strand edge in the image with defects from the rectangular coordinate space to the polar coordinate space; in the range of 0°~180°, traversing all pixel points of the strand edge that identify one or more steel wires, calculating the corresponding p value, which is the vertical distance from the origin to the pixel point; based on the p value, determining the optimal fitting angle of the strand edge as the kink angle according to the mapping relationship between the rectangular coordinate space and the polar coordinate space. Wherein, the identified steel wire is one or two, three steel wires in the same strand, which are color identified (such as color, red, etc.) as a reference for identification and calculation.

[0029] The mapping relationship between the rectangular coordinate space and the polar coordinate space is p=xcosθ+ysinθ, θ is the optimal fitting angle, unit: degree; x and y are coordinate values in the rectangular coordinate system, unit: cm; p is the vertical distance from the origin to the pixel point, unit: cm; since the identified steel wire is a continuous straight line, the same value θ fitted by the pixel points corresponding to each identified steel wire is the optimal fitting angle.

[0030] The image acquisition device determines the surface damage area in the image with defects by using a region growing algorithm, specifically including: taking a seed pixel in the damage area in the image with defects as a starting point, merging adjacent pixels with a gray value difference of less than 10 from the seed pixel, determining a damage edge contour, and determining a damage area in the image with defects based on the damage edge contour; calculating the damage area of the damage area as the surface damage area.

[0031] The surface damage area A s The calculation formula is: In the formula, A s is the surface damage area, unit: cm 2 ; B is the number of pixel points in the damage area, unit: ppi, and d is the number of pixel points per square centimeter of the area, unit: cm 2 / ppi.

[0032] S5: judging the kink condition according to the kink angle and judging the surface damage condition according to the surface damage area, thereby performing online quality monitoring during the erection of the strand 5.

[0033] In step S5, the image acquisition device determines the twisting condition according to the twisting angle, specifically: if the difference between the twisting angle and the reference angle of the strand 5 is greater than or equal to 3°, there is a twisting deformation. In step S5, the image acquisition device determines the surface damage condition according to the surface damage area, specifically: if the surface damage area is greater than or equal to 0.5 cm 2 , it is considered that there is a surface damage defect. Wherein, the reference angle is calculated from the strand image collected by the industrial camera 1 on the first catwalk gantry, denoted as θ0. The twisting angle is calculated from the strand image collected by the industrial camera 1 on the catwalk gantry other than the first catwalk gantry, denoted as θ i-1 , i is the serial number of the catwalk gantry 3; that is, when |θ i-1 -θ0|≥3°, there is a twisting deformation.

[0034] Preferably, the image acquisition device based on image analysis in the present application The main cable erection quality defect online monitoring method further comprises S6: based on the number of the strand 5, the continuous multiple strands 5 at the same time are associated, it is judged whether there is a twisting deformation and / or surface damage defect between the adjacent two strands 5, if there is, it indicates that the strand 5 has a defect deterioration trend; otherwise, it indicates that the strand 5 does not have a defect deterioration trend.

[0035] The main cable erection quality defect online monitoring method based on image analysis of the present application is to distribute the image acquisition device along the traction direction of the strand 5, collect the corresponding strand image through the image acquisition device, and after pretreatment, use the YOLO deep learning model to locate the defect target in the pretreated image, and then use the Hough transform to determine the twisting angle in the image with defects. The area of the surface damage in the image with defects is determined by using the region growing algorithm; the twisting condition is determined according to the twisting angle, and the surface damage condition is determined according to the surface damage area, so as to perform online quality monitoring during the erection process of the strand 5; The technical problems of large defect recognition error and low efficiency in the prior art are solved by manual inspection.

[0036] Referring to Figure 2 , the present application also provides a main cable erection quality defect online monitoring system based on image analysis, which is formed based on the above-mentioned main cable erection quality defect online monitoring method based on image analysis, comprising an image acquisition module, a pretreatment module, a defect positioning module, a defect determination module and a monitoring module. The image acquisition module is used to collect the corresponding strand image through the image acquisition device during the traction process of the strand 5, wherein the image acquisition device is distributed along the traction direction of the strand 5; The pretreatment module is used for pretreating the strand image to obtain a pretreated image; The defect positioning module is configured to locate a defect target in the preprocessed image by using a YOLO deep learning model, and obtain a reprocessed image, wherein the reprocessed image comprises images with defects and images without defects; The defect determination module is configured to identify the images with defects in the reprocessed image, determine a kink angle in the images with defects by using a Hough transform, and determine a surface damage area in the images with defects by using a region growing algorithm. The monitoring module is configured to judge a kink condition according to the kink angle, judge a surface damage condition according to the surface damage area, and thus perform online quality monitoring during the cable strand erection.

[0037] The image acquisition module, the preprocessing module, the defect positioning module, the defect determination module and the monitoring module of the main cable erection quality defect online monitoring system based on image analysis correspond to the main cable erection quality defect online monitoring method based on image analysis.

[0038] The application further provides a computer storage medium storing a program file, wherein the program file is executed to realize the program instructions of the main cable erection quality defect online monitoring method based on image analysis.

[0039] The memory in the application specifically can include a random access memory (RAM), an internal memory, a read-only memory (ROM), a programmable ROM, an erasable programmable ROM, a register, a hard disk, a removable disk or a CD-ROM. It should be noted that a person skilled in the art can select the form and type of the storage medium according to actual use requirements, and the application does not make further specific limitations.

[0040] The above content is only used to illustrate the technical solutions of the application, and is not a limitation on the application; although the application has been described in detail above, a person skilled in the art should understand that the technical solutions described above can still be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the application.

Claims

1. A method for online monitoring of main cable erection quality defects based on image analysis, characterized in that, Includes the following steps: S1: Image acquisition devices are distributed along the traction direction of the cable strand (5). During the traction process of the cable strand (5), the corresponding cable strand images are acquired through the image acquisition devices. S2: Preprocess the strand image to obtain a preprocessed image; S3: Use the YOLO deep learning model to locate the defect target in the preprocessed image, and obtain the reprocessed image, which contains the image with defects and the image without defects; S4: Identify images with defects in the reprocessed images, use Hough transform to determine the twisting angle in the defective images, and use the region growing algorithm to determine the surface damage area in the defective images. S5: The twisting condition is judged based on the twisting angle, and the surface damage condition is judged based on the surface damage area, so as to carry out online quality monitoring during the erection of the cable strand (5).

2. The online monitoring method for main cable erection quality defects based on image analysis according to claim 1, characterized in that, Step S1 further includes: dividing the cable strand (5) into multiple segments along the traction direction of the cable strand (5) and numbering them; the cable strand image acquired by the image acquisition device contains the number of each segment of the cable strand (5).

3. The online monitoring method for main cable erection quality defects based on image analysis according to claim 1, characterized in that, In step S2, the preprocessing of the strand image includes: performing grayscale processing, Gaussian filtering processing, and histogram equalization processing on the strand image in sequence.

4. The online monitoring method for main cable erection quality defects based on image analysis according to claim 1, characterized in that, In step S4, determining the twisting angle in the defective image using the Hough transform specifically includes: Extracting the edges of defective images; Based on the Hough transform principle, a rectangular coordinate system is established with the center of the defective image as the origin, and the strand edges in the defective image are transformed from the rectangular coordinate space to the polar coordinate space. Within the range of 0° to 180°, traverse all pixels of one or more marked steel wires in the edge of the cable strand and calculate the corresponding ρ value, where ρ is the vertical distance from the origin to the pixel. Based on the ρ value, the optimal fitting angle of the strand edge is determined according to the mapping relationship between rectangular coordinate space and polar coordinate space, and is used as the twisting angle.

5. The online monitoring method for main cable erection quality defects based on image analysis according to claim 1, characterized in that, In step S4, determining the surface damage area in the defective image using the region growing algorithm specifically includes: Starting from the seed pixel in the damaged area of ​​the defective image, adjacent pixels in the defective image whose gray value difference with the seed pixel is less than 10 are merged to determine the damage edge contour, and the damaged area in the defective image is determined based on the damage edge contour. Calculate the damage area of ​​the damaged region as the surface damage area.

6. The online monitoring method for main cable erection quality defects based on image analysis according to claim 5, characterized in that, Surface damage area A s The calculation formula is: In the formula, A s Surface damage area, unit: cm 2 ; B represents the number of pixels in the damaged area, in ppi; d represents the number of pixels per square centimeter, in cm. 2 / ppi.

7. The online monitoring method for main cable erection quality defects based on image analysis according to claim 1, characterized in that, In step S5, the determination of the twisting situation based on the twisting angle specifically includes: if the difference between the twisting angle and the reference angle of the strand (5) is ≥3°, then there is twisting deformation; In step S5, determining the surface damage based on the surface damage area specifically includes: if the surface damage area is ≥0.5 cm... 2 If so, it is considered that there is a surface damage defect.

8. The online monitoring method for main cable erection quality defects based on image analysis according to claim 2, characterized in that, The online monitoring method for main cable erection quality defects based on image analysis also includes S6: based on the number of the cable strand (5), multiple consecutive cable strands (5) at the same time are associated to determine whether there are twisting deformation and / or surface damage defects between two adjacent cable strands (5). If there are, it indicates that the cable strand (5) has a defect deterioration trend. Otherwise, it indicates that the defect (5) does not have a deterioration trend.

9. An online monitoring system for main cable erection quality defects based on image analysis, formed by the online monitoring method for main cable erection quality defects based on image analysis as described in claim 1, is characterized in that, include: Image acquisition module: used to acquire corresponding images of the cable strand (5) during the traction process of the cable strand (5), wherein the image acquisition devices are distributed along the traction direction of the cable strand (5); Preprocessing module: Used to preprocess the strand image to obtain a preprocessed image; Defect localization module: Used to locate defects in images with defects in preprocessed images using the YOLO deep learning model, resulting in a reprocessed image, which contains images with defects and images without defects; Defect determination module: used to identify images with defects in the reprocessed images, use Hough transform to determine the twist angle in the defective images, and use region growing algorithm to determine the surface damage area in the defective images; And a monitoring module: used to determine the twisting situation based on the twisting angle and the surface damage situation based on the surface damage area, so as to carry out online quality monitoring during the erection of the cable strand (5).

10. A computer storage medium, characterized in that, The system stores a program file, which is executed to implement the program instructions formed by the image analysis-based online monitoring method for main cable erection quality defects as described in any one of claims 1-8.

Images