A batch detection method for bridge pier inclination based on UAV image recognition

Abstract

This invention discloses a method for batch detecting bridge pier inclination based on drone image recognition. The method involves five steps: batch acquisition of bridge pier images by drone, extraction of the left and right edges of the pier outlines, calculation of the inclination angles of the left and right edges, calculation of the pier inclination, return of a list of inclinations, and issuance of an over-limit warning. The method utilizes image acquisition, image processing, image information recognition, extraction and calculation of key parameters to generate a list of pier inclinations and return the numbers and total number of piers whose inclinations exceed the limit. The overall process is high-volume, fast, and applicable to a wide range of terrains, eliminating the need for a total station or GPS receiver.

Description

Technical Field

[0001] The invention relates to a method for batch detecting the inclination of bridge piers based on unmanned aerial vehicle (UAV) image recognition. Background Art

[0002] With the large-scale construction of transportation infrastructure, the number of existing bridges on highways, railways, and urban expressways has gradually expanded, and the number of aging bridges has also increased. Bridge piers are the key load-bearing structures of all types of bridges. Pier tilt and deformation have a significant adverse impact on the load-bearing capacity of bridge structures. Superstructure collapses caused by pier tilt are common, seriously endangering the lives and property of bridge users. To prevent excessive tilt in existing bridge piers and to identify and control dangerous bridges, large-scale pier tilt monitoring and screening are crucial. However, on major transportation routes such as highways and national roads, bridges are distributed over long distances and have a large number of piers, making monitoring a significant workload. Furthermore, many piers are located in flowing water and rugged terrain, making conventional measurement equipment inconvenient. Currently, the tilt inspection of many bridge piers often relies on visual inspection by inspectors. Visual inspection can only provide a qualitative estimate of pier tilt, not a quantitative measurement indicator. For older bridge piers located in complex terrain and with poor viewing conditions, visual inspection is even impossible, resulting in significant safety hazards. Summary of the Invention

[0003] The present invention aims to solve the problems existing in the above-mentioned prior art and provides a batch detection method for bridge pier inclination based on drone image recognition.

[0004] The technical solutions adopted in the present invention are:

[0005] A method for batch detection of bridge pier inclination based on drone image recognition includes the following steps:

[0006] S1) Use drones to batch collect bridge pier images. For the kth bridge pier, collect bridge pier images P from four directions. k,1 、P k,2 、P k,3 、P k,4 , where P k,1 、P k,3 P is the front and rear elevation images collected parallel to the bridge direction, k,2 、P k,4 The left and right facade images are collected perpendicular to the bridge direction; for all N bridge piers, a total of 4N bridge pier images are collected;

[0007] S2) For each bridge pier image P k,i , extract the left and right boundary lines of the pier outline in the pier image;

[0008] S3) Calculate each pier image P separately k,iThe inclination angle of the two boundary lines in the corresponding AL k,i and AR k,i ;

[0009] S4) Based on the front and rear elevation images P of each pier k,1 、P k,3 The vertical inclination of the pier is calculated based on the inclination angles of the four boundary lines. k,s ; According to the left and right facade images P of each pier k,2 、P k,4 The inclination angle of the four boundary lines is used to calculate the parallel inclination of the pier I k,f ;

[0010] S5) Create a list of the inclinations of all N bridge piers in the vertical and parallel directions. If any inclination exceeding the limit value appears in the list, return the number and total number of the bridge piers whose inclination exceeds the limit value.

[0011] Furthermore, in S1), when the drone is used to capture images of the inside of the four sides of the bridge pier, the height of the bridge pier in the captured image is greater than 1 / 2 of the total height of the image.

[0012] Furthermore, in S2), the image shooting direction number i∈{1,2,3,4} is recorded. For any image file P k,i , convert it into a grayscale image G k,i , using the Canny algorithm to k,i Perform edge detection and get G k,i The edge set E in k,i , use Hough transform to detect edge set E k,i The straight lines in the image are obtained by k,i ;

[0013] For set L k,i Filter the elements in the image, remove the straight line elements whose vertical height difference is less than 1 / 2 of the image height, and record the filtered straight line set as V k,i ;

[0014] Compare the set of lines V k,i The average value of the horizontal coordinates of the endpoints of the middle element. The straight line element with the smallest average horizontal coordinate of the endpoints is the left boundary line EL of the pier. k,i The straight line element with the largest mean of the abscissa of the endpoints is the right boundary line ER of the pier. k,i .

[0015] Furthermore, in S3),

[0016] For any image file P k,i , get its left boundary line EL k,iThe endpoint coordinates are recorded as (XL1, YL1) and (XL2, YL2), and the inverse trigonometric tangent function is used to calculate the left boundary line EL of the pier. k,i The tilt angle AL k,i ,

[0017] (1);

[0018] For any image file P k,i , get its right boundary line ER k,i The endpoint coordinates are recorded as (XR1, YR1) and (XR2, YR2), and the inverse trigonometric tangent function is used to calculate the right boundary line ER of the pier. k,i The tilt angle AR k,i ,

[0019] (2);

[0020] When AL obtained by formula (1) and (2) k,i or AR k,i If the calculated value is negative, increase the result by 180°.

[0021] Furthermore, in S4), for any bridge pier number k, the following formula is used to calculate the inclination perpendicular to the bridge direction I k,s ,

[0022] (3);

[0023] The following formula is used to calculate the inclination I parallel to the bridge direction k,f ,

[0024] (4).

[0025] Furthermore, in S5), for all N bridge piers, return the bridge pier inclination set I={I k,s , I k,f , k∈{1,2,3,…,N}}, which is the quantitative detection result of the inclination of each pier. For the permissible value δ of the pier inclination, the set of pier numbers J whose inclination exceeds the permissible value is returned. That is, for any j∈J, |I j,s |> δ or |I j,f |> δ, and returns the total number of elements in set J.

[0026] The present invention has the following beneficial effects:

[0027] 1) Utilizing drones to capture high-resolution images, combined with advanced edge detection algorithms and Hough transforms, the system can precisely extract the left and right edges of the pier's outline and accurately calculate its inclination angle, providing reliable data support for the quantitative assessment of pier inclination. Without requiring direct contact with the piers, and unrestricted by their geographical location, inspections can be easily completed even in inaccessible areas such as rivers and mountainous areas, ensuring the safety of inspectors while avoiding interference with the pier structure.

[0028] 2) The bridge pier is photographed from four different angles, encompassing front, back, left, and right perspectives, comprehensively capturing information from all pier facades. This effectively avoids blind spots, resulting in more comprehensive and accurate inspection results that more accurately reflect the actual tilt of the pier. Data processing using two pairs of oppositely oriented facade images eliminates roll deviation from the drone's gimbal and determines the pier's true tilt. The system can inspect multiple piers simultaneously, collecting large amounts of image data at once. Algorithmic batch processing and analysis significantly improves inspection efficiency, making it suitable for regular inspections and safety screenings of large-scale bridge facilities, saving significant time and labor costs.

[0029] 3) Compared with traditional bridge pier tilt detection methods, this method eliminates the need for expensive measuring instruments such as total stations and GPS receivers, reducing equipment procurement and maintenance costs. Furthermore, the relatively low operating costs of drones further reduce overall inspection costs. By promptly identifying potential pier tilt hazards, proactive measures can be taken for repair and reinforcement, avoiding major accidents such as bridge collapses caused by excessive pier tilt. This extends the service life of bridges and reduces maintenance and replacement costs, resulting in significant long-term economic benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Schematic diagram of the location for drone imagery;

[0031] Figure 1 The X direction is perpendicular to the bridge direction, and the Y direction is parallel to the bridge direction.

[0032] Figure 2 This is a schematic diagram of the left and right edge angles of the bridge pier image;

[0033] Figure 2 Middle dotted line L B1 is the horizontal reference line in the real physical world, the solid line L B2 It is the horizontal reference line in the photographic image.

[0034] Figure 3 Measured from four photos of the bridge pier with k=2 in the embodiment. DETAILED DESCRIPTION

[0035] The present invention will be further described below with reference to the accompanying drawings.

[0036] Step 1: The drone collects bridge pier images in batches.

[0037] like Figure 1 , record the bridge pier number k∈{1,2,3,…,N}, start from bridge pier No. 1, for any bridge pier No. k, use a drone to take the front elevation image P of the bridge pier k,1 , left facade image P k,2 , rear facade image P k,3 , right elevation image P k,4 The height of the bridge pier in the captured image is greater than 1 / 2 of the total image height. When capturing the image, the pitch, roll, and yaw parameters of the drone's three-axis gimbal must be set to zero. Repeat the above image capture for bridge pier number k+1 until image capture of bridge pier number N is complete, obtaining a total of 4N image files.

[0038] Step 2: Extract the left and right boundary lines of the pier outline.

[0039] The image shooting direction number i∈{1,2,3,4}, for any image file P k,i , convert it into a grayscale image G k,i , using the Canny algorithm to k,i Perform edge detection and get G k,i The edge set E in k,i .

[0040] Hough transform is used to detect the set E k,i The straight lines in the image are obtained by k,i .

[0041] For set L k,i Filter the elements in the image, remove the straight line elements whose vertical height difference is less than 1 / 2 of the image height, and record the filtered straight line set as V k,i , compare the straight line set V k,i The average value of the horizontal coordinates of the endpoints of the middle element. The straight line element with the smaller average horizontal coordinates of the endpoints is the left boundary line EL of the pier. k,i The straight line element with the larger mean of the horizontal coordinates of the endpoints is the right boundary line ER of the pier k,i .

[0042] Step 3: Calculate the inclination angles of the left and right boundary lines of the pier.

[0043] like Figure 2 , for any image file P k,i , get its left boundary line EL k,iThe endpoint coordinates are recorded as (XL1, YL1) and (XL2, YL2), and the inverse trigonometric tangent function is used to calculate the left boundary line EL of the pier. k,i The tilt angle AL k,i :

[0044] (1);

[0045] For any image file P k,i , get its right boundary line ER k,i The endpoint coordinates are recorded as (XR1, YR1) and (XR2, YR2), and the inverse trigonometric tangent function is used to calculate the right boundary line ER of the pier. k,i The tilt angle AR k,i :

[0046] (2);

[0047] When AL obtained by formula (1) and (2) k,i or AR k,i When the calculated value is negative, increase the result by 180° to make the angle range uniform between 0 and 180°.

[0048] Step 4: Calculate the inclination of the bridge pier.

[0049] like Figure 2 For any k-th bridge pier, the inclination angles of the left and right boundary lines of the eight bridge piers can be obtained according to step 3 (AL k,1 , AL k,2 , AL k,3 , AL k,4 , AR k,1 , AR k,2 , AR k,3 , AR k,4 ), where each boundary line inclination angle includes the roll angle deviation D of the drone gimbal. To eliminate the roll angle deviation D in the final measurement result, two pier facade images taken in opposite directions can be combined. By calculating the difference in the inclination angles of the pier in these two images, the influence of the roll angle deviation D is eliminated, and the true angle between the pier and the physical plumb line is obtained. The inclination is then calculated based on the tangent value of this angle.

[0050] Specifically, combined with the front and rear elevation images of Pier K (P k,1 、P k,3 ) of the four boundary lines (AL k,1 ,AL k,3 , AR k,1 , AR k,3 ), the following formula is used to calculate the inclination of the pier perpendicular to the bridge direction Ik,s :

[0051] (3);

[0052] Combined with the left and right elevation images of Pier K (P k,2 、P k,4 ) of the four boundary lines (AL k,2 , AL k,4 ,AR k,2 , AR k,4 ), the following formula is used to calculate the inclination of the pier parallel to the bridge direction I k,f :

[0053] (4).

[0054] As shown in formulas (3) and (4), the roll angle deviation D in the measured value has been automatically offset and does not affect the tilt detection result.

[0055] Step 5: Return to the tilt list and over-allocation warning.

[0056] For all N bridge piers, return the bridge pier inclination set I={I k,s , I k,f , k∈{1,2,3,…,N}}, which is the quantitative detection result of the inclination of each pier,

[0057] For the permissible value of the pier inclination δ, return the set of pier numbers J whose inclination exceeds the permissible value, that is, for any j∈J, |I j,s |> δ or |I j,f |> δ, and returns the total number of elements in set J.

[0058] like Figure 3 Case: Assume that the bridge under inspection has 5 piers and the inclination angle of the left boundary line is a 5 × 4 matrix [AL k,i ]As shown in formula (5):

[0059] (5);

[0060] The right boundary line inclination angle is 5 × 4 matrix [AR k,i ]As shown in formula (6):

[0061] (6);

[0062] Then the pier inclination set I={I k,s , I k,f , k∈{1,2,3,4,5}}, where the vertical inclination vector I k,s Calculated by formula (3), the result is shown in formula (7):

[0063] (7);

[0064] Parallel direction inclination vector I k,f Calculated by formula (4), the result is shown in formula (8):

[0065] (8);

[0066] Assume that the permissible value of the pier inclination is δ = 0.002, and compare δ with the absolute value of each element in the set I. Only |I 2,f |=0.00326> δ, then the bridge pier number with excessive inclination is J={2}, the inclination of pier No. 2 parallel to the bridge direction exceeds the allowable value, and the number of excessive piers is 1.

[0067] The above description is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements can be made without departing from the principles of the present invention. These improvements should also be regarded as the scope of protection of the present invention.

Claims

1. A method for batch detection of bridge pier inclination based on drone image recognition, characterized by: The steps include: S1) Use drones to batch collect bridge pier images. For the kth bridge pier, collect bridge pier images P from four directions. k,1 、P k,2 、P k,3 、P k,4 , where P k,1 、P k,3 P is the front and rear elevation images collected parallel to the bridge direction, k,2 、P k,4 Left and right facade images collected perpendicular to the bridge direction; S2) For each bridge pier image P k,i , extract the left and right boundary lines of the pier outline in the pier image; S3) Calculate each pier image P separately k,i The inclination angle of the two boundary lines in the corresponding AL k,i and AR k,i ; Obtain the endpoint coordinates of the left and right boundary lines, and use the inverse trigonometric tangent function to calculate the inclination angles of the left and right boundary lines of the pier. If the result is negative, increase it by 180°. S4) Based on the front and rear elevation images P of each pier k,1 、P k,3 The vertical inclination of the pier is calculated based on the inclination angles of the four boundary lines. k,s ; According to the left and right facade images P of each pier k,2 、P k,4 The inclination angle of the four boundary lines is used to calculate the parallel inclination of the pier I k,f ; S5) creating a list of the inclinations of all N bridge piers in the vertical and parallel directions. If any inclination exceeds the limit value, the number and total number of bridge piers with inclination exceeding the limit value are returned. In S4), for any pier k, the following formula is used to calculate the inclination perpendicular to the bridge direction I k,s , ； Combined with the left and right elevation images of Pier K (P k,2 、P k,4 ) of the four boundary lines (AL k,2 , AL k,4 , AR k,2 ,AR k,4 ), the following formula is used to calculate the inclination of the pier parallel to the bridge direction I k,f : ； Through the above two formulas, the roll angle deviation D in the measured value has been automatically offset and does not affect the tilt detection result.

2. The method for batch detecting bridge pier inclination based on drone image recognition according to claim 1, characterized in that: In S1), when the drone is used to capture images of the inside of the four sides of the bridge pier, the height of the bridge pier in the captured image is greater than 1 / 2 of the total height of the image.

3. The method for batch detection of bridge pier inclination based on drone image recognition according to claim 1, characterized in that: In S2), the image shooting direction number i∈{1,2,3,4} is recorded. For any image file P k,i , convert it into a grayscale image G k,i , using the Canny algorithm to k,i Perform edge detection and get G k,i The edge set E in k,i , use Hough transform to detect edge set E k,i The straight lines in the image are obtained by k,i ; For set L k,i Filter the elements in the image, remove the straight line elements whose vertical height difference is less than 1 / 2 of the image height, and record the filtered straight line set as V k,i ; Compare the set of lines V k,i The average value of the horizontal coordinates of the endpoints of the middle element. The straight line element with the smallest average horizontal coordinate of the endpoints is the left boundary line EL of the pier. k,i The straight line element with the largest mean of the abscissa of the endpoints is the right boundary line ER of the pier. k,i .

4. The method for batch detecting bridge pier inclination based on drone image recognition according to claim 1, characterized in that: In S3), for any pier image file P k,i , get its left boundary line EL k,i The endpoint coordinates are recorded as (XL1, YL1) and (XL2, YL2), and the inverse trigonometric tangent function is used to calculate the left boundary line EL of the pier. k,i The tilt angle AL k,i , （1）； For any pier image file P k,i , get its right boundary line ER k,i The endpoint coordinates are recorded as (XR1, YR1) and (XR2, YR2), and the inverse trigonometric tangent function is used to calculate the right boundary line ER of the pier. k,i The tilt angle AR k,i , （2）； When AL obtained by formula (1) and (2) k,i or AR k,i If the calculated value is negative, increase the result by 180°.

5. The method for batch detection of bridge pier inclination based on drone image recognition according to claim 1, characterized in that: By combining two pier facade images in opposite directions and calculating the difference in the pier's inclination angles in the two images, the influence of the roll angle deviation of the drone gimbal is eliminated, and the true angle between the pier and the physical plumb line is obtained. The inclination is then calculated based on the tangent value of the angle.

6. The method for batch detecting bridge pier inclination based on drone image recognition according to claim 1, characterized in that: In S5), for all N bridge piers, return the bridge pier inclination set I={I k,s , I k,f , k∈{1,2,3,…,N}}, which is the quantitative detection result of the inclination of each pier. For the permissible value δ of the pier inclination, the set of pier numbers J whose inclination exceeds the permissible value is returned. That is, for any j∈J, |I j,s |> δ or |I j,f |> δ, and returns the total number of elements in set J.

Images