Suspension bridge main cable form identification method based on unmanned aerial vehicle image

Abstract

The invention belongs to the technical field of bridge detection, and particularly relates to an unmanned aerial vehicle image-based suspension bridge main cable form recognition method, which comprises the following steps of: performing image calibration on an unmanned aerial vehicle, acquiring a first image by the unmanned aerial vehicle, and segmenting the first image through colors to obtain a second image comprising a main cable and a cable clamp; converting the second image into a grayscale image, and fitting to obtain a main cable axis equation; converting the first image into a three-dimensional point cloud, and fitting to obtain a suspender axis equation; the main cable axis equation and the suspender axis equation are combined to solve pixel coordinates of main cable intersection points; calculating the horizontal distance and the elevation difference between the intersection points of the adjacent main cables; and taking the midspan main cable intersection point as a starting point coordinate, and obtaining the coordinates of the main cable intersection points on the two sides according to the horizontal distance and the elevation difference between the adjacent main cable intersection points. The unmanned aerial vehicle is adopted to obtain the form of the main cable, the measurement precision is high, the measurement efficiency is greatly improved, and the danger of measurement is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of bridge detection, and in particular to a method for identifying the morphology of a main cable of a suspension bridge based on unmanned aerial vehicle images. Background Art

[0002] In order to ensure the structural safety and stability of the suspension bridge, it is necessary to calculate the axial forces of the main cables and hangers. Obtaining the main cable shape of the suspension bridge is the basis for calculating the axial forces of the main cables and hangers.

[0003] The traditional method of obtaining the main cable shape of a suspension bridge generally uses a total station to measure the main cable shape, finds the location of the main cable intersection on the cable clamp, and manually places prisms to measure the intersection coordinates. However, this method requires interrupting traffic, and for suspension bridges, there are many intersections, and placing prisms one by one at the main cable intersections is extremely labor-intensive. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a method for identifying the main cable morphology of a suspension bridge based on drone images. The main cable morphology is obtained by using a drone, which has high measurement accuracy, improves measurement efficiency and reduces workload.

[0005] The technical solution adopted by the present invention is as follows: A method for identifying the main cable morphology of a suspension bridge based on drone images, characterized in that it comprises the following steps: S1: Calibrate the image of the drone to obtain the relationship between the three-dimensional geometric position and pixels; S2: The drone collects the first image and segments the first image by color to obtain a second image including the main cable and the cable clamp; S3: converting the second image into a grayscale image, identifying the upper and lower edges of the main cable, and fitting to obtain the axis equation of the main cable; S4: converting the first image into a three-dimensional point cloud, extracting a slice of each boom according to the gray value, and fitting to obtain the boom axis equation; S5: Solve the pixel coordinates of the main cable intersection point by combining the main cable axis equation with the boom axis equation; S6: Calculate the horizontal distance and elevation difference between adjacent main cable intersections based on the pixel coordinates of the main cable intersections, the relationship between the three-dimensional geometric positions and the pixels; S7: Taking the main cable intersection point in the middle of the span as the starting point coordinate, the coordinates of the main cable intersection points on both sides are obtained according to the horizontal distance and elevation difference between adjacent main cable intersection points to form a main cable shape diagram.

[0006] As a preferred embodiment of the present invention, in step S1: Laser scanning is used to obtain the point cloud data of the three reference objects. The dimensions of the three reference objects in width and height are extracted from the point cloud data. A drone is used to collect image data of the three reference objects, and the vertical distance between the drone and the three reference objects is equal. By comparing the point cloud data, the actual distance represented by a single pixel in the horizontal direction x and the vertical direction y is obtained. ; , Among them, B and H represent the actual size of the object, the unit is m, and B' and H' represent the size of the object in the image, the unit is pixel, and the drone image calibration is completed.

[0007] As a preferred embodiment of the present invention, step S3 specifically includes: S31: converting the second image into a grayscale image; S32: Take the bottom point of the image As the starting point, search for the first point in the negative direction of the y-axis where the starting point is located, and then use the first point as the starting point to search for the second point along the negative direction of the y-axis. When the distance between the second point and the first point meets the constraint range When The other pixels are removed, and so on. All pixels of Where d is the distance between the upper and lower edges at the same x-coordinate. is the error value affected by noise and shooting angle during edge recognition; S33: The main cable edge line of the same segment passes through two pixel points on the same x coordinate of the upper edge line and the lower edge line, and the median of the y pixel coordinate is taken as the main cable axis coordinate. The main cable axis coordinate is fitted with a quadratic parabola to obtain the axis equation of the main cable segment: .

[0008] As a preferred embodiment of the present invention, step S4 specifically includes: S41: Using the pixel coordinates of the first image as the (x, y) coordinates of the point cloud, and using the image grayscale value as the z coordinate of the point cloud, to obtain a point cloud model of the image; S42: Slice the converted point cloud along the x-axis direction, calculate the weighted average of the z values ​​of the point cloud in the slice, and calculate the mean of all slices Sort and extract the first n slices as the first slice; Where n is the number of booms extracted; S43: taking each first slice as the center, the point clouds within the width of m pixels on both sides of the x-axis are saved as the boom point cloud data. m The value range is ; S44: Slice the point cloud data of a single boom along the y-axis direction to obtain the second slice, project each second slice point cloud on the y-axis, and divide the projected point cloud into two parts on the left and right sides in the x-axis direction from the point with the largest z value in the second slice point cloud. Use MATLAB to perform 4th-order polynomial fitting on the point clouds on both sides. The first derivative of the polynomial represents the slope of a point, and the second derivative represents the first-order derivative of the slope, that is, the rate of change of the slope. In the cross section of the boom, the slopes of the horizontal section are not much different, and the slopes of the inclined section are almost the same. When the slope changes from a horizontal section to an inclined section, its slope changes dramatically, and the point where the dramatic change occurs is the point where the second-order derivative is zero, that is, the slope foot point P. Therefore, this paper uses the maximum point of the second-order derivative of the polynomial as the criterion for determining the edge point of the boom. The maximum point of the second-order derivative of the fitting equation is the boundary point of the boom; S45: The midpoint of the x values ​​of the two extreme points of each second slice is the point on the boom axis. All second slices are traversed in sequence, and linear fitting is performed on the midline to obtain the boom axis equation.

[0009] As a preferred embodiment of the present invention, in step S42, the mean The calculation formula is as follows: In the formula, p represents the number of points in the slice, Represents the z coordinate of each point cloud. Represents the weight, which is obtained by taking the minimum value of the elevation coordinates of all slice point clouds and maximum elevation ,Will It is divided into 10 segments, with weights from small to large ranging from 0.1, 0.2, 0.3 to 1, and then the elevation in each segment is multiplied by the corresponding weight.

[0010] As a preferred embodiment of the present invention, in step S42, When the suspension bridge is a single hanger, the hanger axis equation is as follows: When the suspension bridge has double hangers, the axis equation of the left hanger is , the equation of the right boom axis is The equation of the boom axis is as follows: .

[0011] As a preferred embodiment of the present invention, in step S5, The pixel coordinates of the main cable intersection point are solved by combining the main cable axis equation and the boom axis equation. ; When the suspension bridge is a single hanger, the calculation formula is as follows: When the suspension bridge has double hangers, the calculation formula is as follows: .

[0012] As a preferred embodiment of the present invention, in step S6: According to the main cable intersection , calculate the horizontal pixel distance between adjacent main cable intersections and elevation difference Then, based on the relationship between pixel accuracy and actual distance, the horizontal distance between adjacent main cable intersections is calculated. and elevation difference , the calculation formula is: , . .

[0013] As a preferred embodiment of the present invention, in step S7, Horizontal distance between adjacent main cable intersections Elevation difference The calculation formula is as follows: .

[0014] Compared with the prior art, the beneficial effects are as follows: 1. Since the colors of the main cable and the cable clamp are generally yellow or orange, which are quite different from the ambient color, the corresponding image can be directly cut. By limiting the constraint range, the upper and lower edges of the main cable can be identified, and the remaining pixels can be removed to eliminate the influence of the cable clamp. Since the centerline of the main cable is a curve with a lower mid-span and higher ends, it can be fitted by a quadratic parabola to ensure the accuracy of the main cable axis equation.

[0015] 2. For the white boom, it is not easy to segment directly because it is overall slender and has a similar color to the river surface, railings, etc. Therefore, a point cloud model is established, slices are extracted using grayscale values, and then a fourth-order polynomial fitting is performed to solve the second-order derivative maximum point of the fitting equation to obtain the left and right boundary points of the boom. After taking the midpoint, a linear fitting is performed to obtain the boom axis equation, thereby effectively eliminating the influence of other similar objects in the background and ensuring the accuracy of the boom axis equation.

[0016] 3. By using the relationship between pixels and three-dimensional space points in the captured image, combined with the main cable axis equation and the boom axis equation, the horizontal distance and elevation difference between adjacent main cable intersections can be calculated, thereby obtaining a graphic reflecting the main cable shape; the main cable shape can be obtained by drone, without blocking traffic, and with high measurement accuracy, replacing long-term manual work, greatly improving measurement efficiency, and avoiding danger to surveyors during the measurement process. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a schematic diagram of a laser scanning reference object in an embodiment of a method for identifying the main cable morphology of a suspension bridge based on drone images of the present invention; Figure 2 is a schematic diagram of a reference object photographed by a drone in an embodiment of a method for identifying the main cable morphology of a suspension bridge based on drone images of the present invention; Figure 3 is a schematic diagram of a first image and a second image in an embodiment of a method for identifying the main cable morphology of a suspension bridge based on drone images of the present invention; Figure 4 is a schematic diagram of the upper and lower edges of the main cable in an embodiment of the method for identifying the shape of the main cable of a suspension bridge based on drone images of the present invention; Figure 5 It is a schematic diagram of a point cloud converted from a first image in an embodiment of a method for identifying the main cable morphology of a suspension bridge based on drone images of the present invention; Figure 6 It is a schematic diagram of extracting a hanger point cloud in an embodiment of a method for identifying the main cable shape of a suspension bridge based on drone images of the present invention; Figure 7 is a schematic diagram of extracting the sideline of the suspender rod in an embodiment of the method for identifying the main cable shape of a suspension bridge based on drone images of the present invention; Figure 8 is a schematic diagram of the main cable shape in an embodiment of the method for identifying the main cable shape of a suspension bridge based on drone images of the present invention; Fig. 9 It is a flowchart of the steps of an embodiment of a method for identifying the main cable morphology of a suspension bridge based on drone images of the present invention. DETAILED DESCRIPTION

[0018] Typical embodiments that embody the features and advantages of the present invention will be described in detail in the following description. It should be understood that the present invention can have various changes in different embodiments without departing from the scope of the present invention, and the descriptions and illustrations therein are essentially used for illustration purposes rather than for limiting the present invention.

[0019] In the description of the present application, the terms "first", "second", etc. are only used to facilitate the description of the present application and simplify the description, and do not indicate or imply that the structure referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0020] See also Fig. 9 As shown, the method for identifying the main cable morphology of a suspension bridge based on drone images in this embodiment includes the following steps: S1: Calibrate the image of the drone to obtain the relationship between the three-dimensional geometric position and pixels; S2: The drone collects the first image and segments the first image by color to obtain a second image including the main cable and the cable clamp; S3: converting the second image into a grayscale image, identifying the upper and lower edges of the main cable, and fitting to obtain the axis equation of the main cable; S4: converting the first image into a three-dimensional point cloud, extracting a slice of each boom according to the gray value, and fitting to obtain the boom axis equation; S5: Solve the pixel coordinates of the main cable intersection point by combining the main cable axis equation with the boom axis equation; S6: Calculate the horizontal distance and elevation difference between adjacent main cable intersections based on the pixel coordinates of the main cable intersections, the relationship between the three-dimensional geometric positions and the pixels; S7: Take the main cable intersection point in the middle of the span as the starting point coordinate, and obtain the coordinates of the main cable intersection points on both sides according to the horizontal distance and elevation difference between adjacent main cable intersection points.

[0021] In step S1, refer to Figure 1 and Figure 2 , laser scanning is used to obtain the point cloud data of the three reference objects, and the dimensions of the three reference objects in the width and height directions are extracted from the point cloud data, distributed as H1, H2, H3 and G1, G2, G3. The image data of the three reference objects are collected by drones, and the vertical distances between the drones and the three reference objects are equal. By comparing the point cloud data, the actual distance represented by a single pixel in the horizontal direction x and the vertical direction y is obtained. , , Among them, B and H represent the actual size of the object, the unit is m, and B' and H' represent the size of the object in the image, the unit is pixel, and the drone image calibration is completed.

[0022] In step S2, refer to Figure 3 ,Since the colors of the main cables and cable clamps in the suspension bridge are obviously different from the ,suspenders and background, color clustering is used to segment the main cables and cable clamps and ,separate the main cables and cable clamps from the image.

[0023] In step S3, refer to Figure 4 , specifically including the following steps: S31: converting the second image into a grayscale image; S32: Take the bottom point of the image As the starting point, search for the first point in the negative direction of the y-axis where the starting point is located, and then use the first point as the starting point to search for the second point along the negative direction of the y-axis. When the distance between the second point and the first point meets the constraint range When The other pixels are removed, and so on. All pixels of Where d is the distance between the upper and lower edges at the same x-coordinate. is the error value affected by noise and shooting angle during edge recognition; S33: The main cable edge line of the same segment passes through two pixel points on the same x coordinate of the upper edge line and the lower edge line, and the median of the y pixel coordinate is taken as the main cable axis coordinate. The main cable axis coordinate is fitted with a quadratic parabola to obtain the axis equation of the main cable segment: .

[0024] In step S4, refer to Figures 5 to 7 , specifically including the following steps: S41: Figure 5 The pixel coordinates of the first image shown in (a) are used as the (x, y) coordinates of the point cloud, and the image grayscale value is used as the z coordinate of the point cloud to obtain Figure 5 (b) The point cloud model shown; S42: Slice the converted point cloud along the x-axis direction. Figure 6 As shown in (a), the weighted average of the z values ​​of the point cloud within the slice is calculated, and the mean of all slices is calculated. Sort and extract the first n slices, where n is the number of extracted booms, and the calculation formula is as follows: In the formula, p represents the number of points in the slice, Represents the z coordinate of each point cloud. Represents the weight, which is obtained by taking the minimum value of the elevation coordinates of all slice point clouds and maximum elevation ,Will It is divided into 10 segments, with weights from small to large ranging from 0.1, 0.2, 0.3 to 1, and then the elevation in each segment is multiplied by the corresponding weight.

[0025] S43: Figure 6 As shown in (b), the point cloud data of the boom is saved with each first slice as the center and the point cloud within the width of m pixels on both sides of the x-axis. m The value range is ; S44: Figure 7 As shown in (a), the point cloud data of a single boom is sliced ​​along the y-axis direction to obtain the second slice, and each second slice point cloud is projected on the y-axis. The projected point cloud is divided into two parts in the x-axis direction from the point with the largest z value in the second slice point cloud, as shown in Figure 7 (c) and Figure 7As shown in (d), MATLAB is used to fit the point clouds on both sides with a fourth-order polynomial. The first derivative of the polynomial represents the slope of a certain point, and the second derivative represents the first-order derivative of the slope, that is, the rate of change of the slope. In the cross section of the boom, the slopes of the horizontal section are not much different, and the slopes of the inclined section are almost the same. When the slope changes from a horizontal section to an inclined section, its slope changes dramatically, and the point where the dramatic change occurs is the point where the second-order derivative is zero, which is the slope foot point P. Therefore, this paper uses the maximum point of the second-order derivative of the polynomial as the criterion for determining the edge point of the boom. The maximum point of the second-order derivative of the fitting equation is the boundary point of the boom; S45: The midpoint of the x values ​​of the two extreme points of each second slice is the point on the hanger axis. All second slices are traversed in sequence, and the midline is linearly fitted to obtain the hanger axis equation. When the suspension bridge is a single hanger, the hanger axis equation is as follows: When the suspension bridge has double hangers, the hanger axis equation is as follows: .

[0026] In step S5, the main cable axis equation and the boom axis equation are combined to solve the pixel coordinates of the main cable intersection point. ; When the suspension bridge is a single hanger, the calculation formula is as follows: When the suspension bridge has double hangers, the calculation formula is as follows: .

[0027] In step S6, according to the main cable intersection point , calculate the horizontal pixel distance between adjacent main cable intersections and elevation difference Then, based on the relationship between pixel accuracy and actual distance, the horizontal distance between adjacent main cable intersections is calculated. and elevation difference , the calculation formula is: , .

[0028] In step S7, the horizontal distance between adjacent main cable intersections is Elevation difference The calculation formula is as follows: .

[0029] The above-mentioned embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and substitutions made by technicians in this field on the basis of the present invention shall fall within the scope of protection required by the present invention.

Claims

1. A method for identifying the main cable morphology of a suspension bridge based on drone images, characterized in that: The steps include: S1: Calibrate the image of the drone to obtain the relationship between the three-dimensional geometric position and pixels; S2: The drone collects the first image and segments the first image by color to obtain a second image including the main cable and the cable clamp; S3: converting the second image into a grayscale image, identifying the upper and lower edges of the main cable, and fitting to obtain the axis equation of the main cable; S4: converting the first image into a three-dimensional point cloud, extracting a slice of each boom according to the gray value, and fitting to obtain the boom axis equation; S5: Solve the pixel coordinates of the main cable intersection point by combining the main cable axis equation with the boom axis equation; S6: Calculate the horizontal distance and elevation difference between adjacent main cable intersections based on the pixel coordinates of the main cable intersections, the relationship between the three-dimensional geometric positions and the pixels; S7: Taking the main cable intersection point in the middle of the span as the starting point coordinate, the coordinates of the main cable intersection points on both sides are obtained according to the horizontal distance and elevation difference between adjacent main cable intersection points to form a main cable shape diagram.

2. The method for identifying the main cable morphology of a suspension bridge based on drone images according to claim 1 is characterized in that: In step S1: Laser scanning is used to obtain the point cloud data of the three reference objects. The dimensions of the three reference objects in width and height are extracted from the point cloud data. A drone is used to collect image data of the three reference objects, and the vertical distance between the drone and the three reference objects is equal. By comparing the point cloud data, the actual distance represented by a single pixel in the horizontal direction x and the vertical direction y is obtained. ; Among them, B and H represent the actual size of the object, the unit is m, and B' and H' represent the size of the object in the image, the unit is pixel, and the drone image calibration is completed.

3. The method for identifying the main cable morphology of a suspension bridge based on drone images according to claim 1 is characterized in that: The step S3 specifically includes: S31: converting the second image into a grayscale image; S32: Take the bottom point of the image As the starting point, search for the first point in the negative direction of the y-axis where the starting point is located, and then use the first point as the starting point to search for the second point along the negative direction of the y-axis. When the distance between the second point and the first point meets the constraint range When The other pixels are removed, and so on. All pixels of Where d is the distance between the upper and lower edges at the same x-coordinate. is the error value affected by noise and shooting angle during edge recognition; S33: The main cable edge line of the same segment passes through two pixel points on the same x coordinate of the upper edge line and the lower edge line, and the median of the y pixel coordinate is taken as the main cable axis coordinate. The main cable axis coordinate is fitted with a quadratic parabola to obtain the axis equation of the main cable segment: .

4. The method for identifying the main cable morphology of a suspension bridge based on drone images according to claim 1 is characterized in that: The step S4 specifically includes: S41: Using the pixel coordinates of the first image as the (x, y) coordinates of the point cloud, and using the image grayscale value as the z coordinate of the point cloud, to obtain a point cloud model of the image; S42: Slice the converted point cloud along the x-axis direction, calculate the weighted average of the z values ​​of the point cloud in the slice, and calculate the mean of all slices Sort and extract the first n slices as the first slice; Where n is the number of booms extracted; S43: taking each first slice as the center, the point clouds within the width of m pixels on both sides of the x-axis are saved as the boom point cloud data. m The value range is ; S44: Slice the point cloud data of a single boom along the y-axis direction to obtain a second slice, project each second slice point cloud onto the y-axis, and divide the projected point cloud into two parts, left and right, from the point with the largest z value in the second slice point cloud in the x-axis direction. Use MATLAB to fit the point clouds on both sides with a 4th-order polynomial respectively; the first derivative of the polynomial represents the slope of a certain point, and the second derivative represents the first-order derivative of the slope, that is, the rate of change of the slope; the maximum point of the second-order derivative of the fitting equation is the boundary point of the boom; S45: The midpoint of the x values ​​of the two extreme points of each second slice is the point on the boom axis. All second slices are traversed in sequence, and linear fitting is performed on the midline to obtain the boom axis equation.

5. The method for identifying the main cable morphology of a suspension bridge based on drone images according to claim 4 is characterized in that: In step S42, the mean The calculation formula is as follows: In the formula, p represents the number of points in the slice, Represents the z coordinate of each point cloud; Represents the weight, which is obtained by taking the minimum value of the elevation coordinates of all slice point clouds and maximum elevation ,Will It is divided into 10 segments, with weights from small to large ranging from 0.1, 0.2, 0.3 to 1, and then the elevation in each segment is multiplied by the corresponding weight.

6. The method for identifying the main cable morphology of a suspension bridge based on drone images according to claim 4 is characterized in that: In step S45, in step S42, When the suspension bridge is a single hanger, the hanger axis equation is as follows: When the suspension bridge has double hangers, the hanger axis equation is as follows: 。 7. The method for identifying the main cable morphology of a suspension bridge based on drone images according to claim 1 is characterized in that: In step S5, The pixel coordinates of the main cable intersection point are solved by combining the main cable axis equation and the boom axis equation. ; When the suspension bridge is a single hanger, the calculation formula is as follows: When the suspension bridge has double hangers, the calculation formula is as follows: 。 8. The method for identifying the main cable morphology of a suspension bridge based on drone images according to claim 7 is characterized in that: In step S6: According to the main cable intersection , calculate the horizontal pixel distance between adjacent main cable intersections and elevation difference Then, based on the relationship between pixel accuracy and actual distance, the horizontal distance between adjacent main cable intersections is calculated. and elevation difference , the calculation formula is: , .

9. The method for identifying the main cable morphology of a suspension bridge based on drone images according to claim 8 is characterized in that: In step S7, Horizontal distance between adjacent main cable intersections Elevation difference The calculation formula is as follows: 。

Images