Batch detection method for gradient of bridge pier based on image recognition of unmanned aerial vehicle

Abstract

The invention discloses a pier inclination batch detection method based on unmanned aerial vehicle image recognition, and the method comprises five steps: collecting pier images in batches by an unmanned aerial vehicle, extracting left and right side lines of a pier contour, calculating the inclination angles of the left and right side lines of the pier, calculating the inclination of the pier, returning an inclination list, and carrying out excess early warning. Starting from the aspects of image acquisition, image processing, image information identification, key parameter extraction and calculation and the like, a bridge pier inclination list is obtained, and the serial numbers and the total number of bridge piers with the inclination exceeding a limit value are returned. The whole process has the advantages of being large in batch, rapid and wide in terrain application range, and a total station and GPS receiving equipment do not need to be additionally used.

Description

Technical Field

[0001] The present invention relates to a method for batch detection of pier inclination based on UAV image recognition. Background Art

[0002] With the large-scale construction of transportation infrastructure, the scale of existing bridges on highways, railways, and urban expressways is gradually expanding, and the number of old bridges is also increasing. Piers are the key load-bearing structures of various bridges, and the inclined deformation of piers has a significant adverse impact on the stress of bridge structures. The collapse of the superstructure caused by pier inclination occurs frequently, seriously affecting the life and property safety of bridge users. To avoid excessive inclination of existing piers and to identify and control dangerous bridges, large-scale monitoring and screening of pier inclination are crucial. However, for transportation arteries such as highways and national roads, the bridges are distributed over a long distance and the number of piers is numerous, resulting in a huge monitoring workload. Moreover, a large number of piers are located in flowing water and rugged terrains, making it inconvenient to use conventional measuring equipment. Currently, the inclination inspection of a large number of piers often relies on visual inspection by inspectors. Visual inspection can only provide a qualitative estimate of pier inclination and cannot obtain quantitative inclination measurement indicators. For old piers with complex terrain and poor observation conditions, visual inspection cannot even be carried out, resulting in a large safety hazard for the piers. Summary of the Invention

[0003] The present invention provides a method for batch detection of pier inclination based on UAV image recognition to solve the problems existing in the above-mentioned prior art.

[0004] The technical solution adopted by the present invention is as follows:

[0005] A method for batch detection of pier inclination based on UAV image recognition, comprising the following steps:

[0006] S1) Batch collect pier images by means of a UAV. For the k-th pier, collect pier images P k,1 , P k,2 , P k,3 , P k,4 from 4 directions respectively, where P k,1 , P k,3 are the front and rear elevation images collected parallel to the bridge direction, and P k,2 , P k,4 are the left and right elevation images collected perpendicular to the bridge direction; for all N piers, a total of 4N pier images are collected;

[0007] S2) For each pier image P k,i , extract the left and right boundary lines of the pier contour in the pier image;

[0008] S3) Calculate each pier image P k,iThe inclination angles of the two boundary lines in it correspond to AL k,i and AR k,i ;

[0009] S4) Calculate the vertical inclination I k,1 of the pier according to the inclination angles of the 4 boundary lines of the front and rear elevation images P k,3 of each pier; Calculate the parallel inclination I k,s of the pier according to the inclination angles of the 4 boundary lines of the left and right elevation images P k,2 of each pier; k,4 ; k,f ;

[0010] S5) Establish a list of pier inclinations in the vertical and parallel directions for all N piers. If there are inclinations exceeding the limit value in the list, return the pier numbers and the total number of piers with inclinations exceeding the limit value.

[0011] Furthermore, in S1), when using a drone to take the inner images of the four sides of the pier, the height of the pier in the captured image is greater than 1 / 2 of the total height of the image.

[0012] Furthermore, in S2), denote the image shooting direction number i ∈ {1, 2, 3, 4}. For any image file P k,i , convert it into a grayscale image G k,i , perform edge detection on G k,i using the Canny algorithm to obtain the edge set E k,i in G k,i , detect the lines in the edge set E k,i using the Hough transform to obtain the line set L k,i in the image;

[0013] Filter the elements in the set L k,i , remove the line elements with a vertical height difference less than 1 / 2 of the image height, and denote the filtered line set as V k,i ;

[0014] Compare the average abscissa of the endpoints of the line elements in the line set V k,i . The line element with a smaller average abscissa of the endpoints is the left boundary line EL k,i of the pier, and the line element with a larger average abscissa of the endpoints is the right boundary line ER k,i of the pier.

[0015] Furthermore, in S3),

[0016] For any image file P k,i , obtain its left boundary line EL k,iThe endpoint coordinates are denoted as (XL1, YL1) and (XL2, YL2) respectively, and the left boundary line EL of the pier is calculated using the arctangent function. k,i The inclination angle AL k,i ,

[0017] (1);

[0018] For any image file P k,i , obtain its right boundary line ER k,i The endpoint coordinates are denoted as (XR1, YR1) and (XR2, YR2) respectively, and the right boundary line ER of the pier is calculated using the arctangent function. k,i The inclination angle AR k,i ,

[0019] (2);

[0020] When the calculated value of AL k,i or AR k,i obtained from formulas (1) and (2) is negative, add 180° to the calculation result.

[0021] Furthermore, in S4), for any k-th pier, the following formula is used to calculate the inclination I perpendicular to the bridge direction k,s ,

[0022] (3);

[0023] The following formula is used to calculate the inclination I parallel to the bridge direction k,f ,

[0024] (4).

[0025] Furthermore, in S5), for all N piers, return the set of pier inclinations I = {I k,s , I k,f , k ∈ {1, 2, 3, …, N}}, which is the quantitative detection result of the inclinations of each pier. For the allowable value δ of the pier inclination, return the set J of the numbers of the piers whose inclinations exceed the allowable value, that is, for any j ∈ J, |I j,s | > δ or |I j,f | > δ, and return the total number of elements in the set J.

[0026] The present invention has the following beneficial effects:

[0027] 1) By using drones to collect high - resolution images and combining advanced edge - detection algorithms and Hough transform, the left and right edges of the pier contour can be accurately extracted, and its inclination angle can be accurately calculated, providing reliable data support for the quantitative evaluation of the pier inclination. Without direct contact with the pier and being unrestricted by the geographical environment where the pier is located, even if the pier is in difficult - to - reach areas such as rivers or mountains, the detection can be easily completed, ensuring the safety of the detection personnel and avoiding interference with the pier structure.

[0028] 2) The pier is photographed from 4 directions, covering different perspectives of front, back, left, and right, comprehensively obtaining information on all elevations of the pier, effectively avoiding detection blind spots, making the detection results more comprehensive and accurate, and being able to more truly reflect the actual inclination of the pier. By using two pairs of opposite - facing elevation images for data processing, the rolling - angle deviation of the drone gimbal can be eliminated, and the true inclination of the pier can be obtained. Multiple piers can be detected simultaneously, a large amount of image data of piers can be collected at one time, and through algorithmic batch processing and analysis, the detection efficiency is greatly improved, which is applicable to the regular detection and safety screening of large - scale bridge facilities, saving a large amount of time and labor costs.

[0029] 3) Compared with traditional pier - inclination detection methods, there is no need to use expensive measuring instruments such as total stations and GPS receiving devices, reducing equipment procurement and maintenance costs. At the same time, the operating cost of the drone is relatively low, further reducing the overall detection cost. By timely discovering potential pier - inclination hazards, measures can be taken in advance for repair and reinforcement, avoiding major accidents such as bridge collapses caused by excessive pier inclination, thus extending the service life of the bridge and reducing the costs of bridge maintenance and replacement, with significant long - term economic benefits. Brief Description of the Drawings

[0030] Figure 1 is a schematic diagram of the drone image - shooting position;

[0031] Figure 1 In it, the X - direction is perpendicular to the bridge direction, and the Y - direction is parallel to the bridge direction.

[0032] Figure 2 is a schematic diagram of the left - and - right edge angles of the pier image;

[0033] Figure 2 In it, the dashed line L B1 is the horizontal reference line in the real physical world, and the solid line L B2 is the horizontal reference line in the photographic image.

[0034] Figure 3 is the measurement of four photos of the pier with k = 2 in the embodiment. Detailed Implementation Manner

[0035] The present invention will be further described below with reference to the accompanying drawings.

[0036] Step 1: The unmanned aerial vehicle (UAV) batch-collects pier images.

[0037] As Figure 1 , denote the pier number as k ∈ {1, 2, 3, …, N}. Starting from pier No. 1, for any k-th pier, use the UAV to take a front elevation image P k,1 , a left elevation image P k,2 , a rear elevation image P k,3 , and a right elevation image P k,4 . The height of the pier in the captured image is greater than 1 / 2 of the total height of the image. When taking the image, the pitch angle, roll angle, and yaw angle parameters of the UAV's three-axis gimbal should be set to zero. Repeat the above image capture for the (k + 1)-th pier until the image capture of pier No. N is completed, and a total of 4N image files are obtained.

[0038] Step 2: Extract the left and right boundary lines of the pier contour.

[0039] Denote the image capture direction number as i ∈ {1, 2, 3, 4}. For any image file P k,i , convert it into a grayscale image G k,i , and use the Canny algorithm to perform edge detection on G k,i to obtain the edge set E k,i in G k,i .

[0040] Use the Hough transform to detect the lines in the set E k,i to obtain the line set L k,i in the image.

[0041] Filter the elements in the set L k,i , remove the line elements with a vertical height difference less than 1 / 2 of the image height, and denote the filtered line set as V k,i . Compare the average abscissa of the endpoints of the line elements in the line set V k,i . The line element with a smaller average abscissa of the endpoints is the left boundary line EL k,i of the pier, and the line element with a larger average abscissa of the endpoints is the right boundary line ER k,i .

[0042] Step 3: Calculate the inclination angles of the left and right boundary lines of the pier.

[0043] As Figure 2 , for any image file P k,i , obtain its left boundary line EL k,iThe endpoint coordinates are denoted as (XL1, YL1) and (XL2, YL2) respectively, and the left boundary line EL of the pier is calculated using the inverse tangent function. k,i The inclination angle AL k,i :

[0044] (1);

[0045] For any image file P k,i , obtain its right boundary line ER k,i The endpoint coordinates are denoted as (XR1, YR1) and (XR2, YR2) respectively, and the right boundary line ER of the pier is calculated using the inverse tangent function. k,i The inclination angle AR k,i :

[0046] (2);

[0047] When the calculated value of AL k,i or AR k,i obtained from formulas (1) and (2) is negative, add 180° to the calculation result to unify the angle range to 0~180°.

[0048] Step 4: Calculate the inclination of the pier.

[0049] For example Figure 2 , for any k-th pier, according to Step 3, the inclination angles of the left and right boundary lines of 8 piers can be obtained (AL k,1 , AL k,2 , AL k,3 , AL k,4 , AR k,1 , AR k,2 , AR k,3 , AR k,4 ), where the inclination angle of each boundary line includes the roll angle deviation D of the drone gimbal. In order to eliminate the roll angle deviation D in the final measurement result, the elevation images of the piers with opposite shooting directions can be combined, and by calculating the difference in the inclination angles of the piers in these two images, the influence of the roll angle deviation D can be eliminated, and the true angle between the pier and the physical plumb line can be obtained. Then, the inclination is calculated based on the tangent value of this angle.

[0050] Specifically, combining the inclination angles of the 4 boundary lines (AL k,1 , AL k,3 ) of the front and rear elevation images (P k,1 , AL k,3 , AR k,1 , AR k,3 ) of the k-th pier, the inclination I of the pier perpendicular to the bridge direction is calculated using the following formulak,s :

[0051] (3);

[0052] Combined with the inclination angles (AL k,2 , P k,4 ) of the 4 boundary lines of the left and right elevation images of Pier k (P k,2 , AL k,4 , AR k,2 , AR k,4 ), the inclination I k,f of the pier parallel to the bridge direction is calculated using the following formula:

[0053] (4).

[0054] As shown in formulas (3) and (4), the roll angle deviation D in the measured values has been automatically cancelled out and does not affect the inclination detection result.

[0055] Step Five: Return the inclination list and over-limit warning.

[0056] For all N piers, return the pier inclination set I = {I k,s , I k,f , k ∈ {1, 2, 3, …, N}}, which is the quantitative detection result of the inclination of each pier.

[0057] For the allowable value δ of the pier inclination, return the set J of the pier numbers with inclination exceeding the allowable value, that is, for any j ∈ J, |I j,s | > δ or |I j,f | > δ, and return the total number of elements in set J.

[0058] For example Figure 3 Case: Suppose there are 5 piers in the inspected bridge, and the inclination angles of the left boundary lines are a 5 × 4 matrix [AL k,i as shown in formula (5):

[0059] (5);

[0060] The inclination angles of the right boundary lines are a 5 × 4 matrix [AR k,i as shown in formula (6):

[0061] (6);

[0062] Then the pier inclination set I = {I k,s , I k,f , k ∈ {1, 2, 3, 4, 5}}, where the vertical inclination vector I k,s is calculated by formula (3), and its result is as shown in formula (7):

[0063] (7);

[0064] Parallel direction inclination vector I k,f Calculated by Equation (4), and the result is shown in Equation (8):

[0065] (8);

[0066] Let the allowable value of the pier inclination be δ = 0.002. Compare the absolute values of δ and each element in set I. Only |I 2,f | = 0.00326 > δ. Then the pier number J with over-limit inclination = {2}. The inclination of Pier No. 2 parallel to the bridge direction exceeds the allowable value, and the number of piers with over-limit inclination is 1.

[0067] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, several improvements can be made without departing from the principle of the present invention, and these improvements should also be regarded as the protection scope of the present invention.

Claims

1. A batch detection method for the inclination of bridge piers based on UAV image recognition, characterized in that: Including the following steps: S1) Batch collect pier images by drone. For the k-th pier, collect pier images P k,1 , P k,2 , P k,3 , P k,4 from 4 directions respectively, where P k,1 , P k,3 are the front and rear elevation images collected parallel to the bridge direction, and P k,2 , P k,4 are the left and right elevation images collected perpendicular to the bridge direction; S2) For each pier image P k,i , extract the left and right boundary lines of the pier contour in the pier image; S3) Calculate the inclination angles of the two boundary lines in each pier image P k,i respectively, corresponding to AL k,i and AR k,i ; S4) Calculate the vertical inclination I of the pier according to the inclination angles of the 4 boundary lines of the front and rear elevation images P k,1 and P k,3 of each pier; Calculate the parallel inclination I of the pier according to the inclination angles of the 4 boundary lines of the left and right elevation images P k,s and P k,2 of each pier; k,4 Calculate the parallel inclination I of the pier according to the inclination angles of the 4 boundary lines of the left and right elevation images P k,f ; S5) Establish a list of the inclination degrees of all N piers in the vertical and parallel directions. If there are inclination degrees exceeding the limit value in the list, return the pier numbers and the total number of piers with inclination degrees exceeding the limit value.

2. The method for batch detection of pier inclination based on UAV image recognition according to claim 1, characterized in that: In S1), when using a drone to take images of the four sides of the pier, the height of the pier in the captured image is greater than 1 / 2 of the total height of the image.

3. The method for batch detection of pier inclination based on UAV image recognition according to claim 1, characterized in that: In S2), let the image shooting direction number \(i\in\{1,2,3,4\}\), for any image file P k,i , convert it into a grayscale image G k,i , use the Canny algorithm to perform edge detection on G k,i , and obtain the edge set E k,i in G k,i , use the Hough transform to detect the straight lines in the edge set E k,i , and obtain the straight line set L k,i in the image; Filter the elements in set L k,i and remove the straight-line elements with a vertical height difference less than half of the image height. Denote the set of filtered straight lines as V k,i ; Compare the average abscissa of the endpoints of the line elements in the set V k,i The line element with a smaller average abscissa of the endpoints is the left boundary line EL of the pier k,i The line element with a larger average abscissa of the endpoints is the right boundary line ER of the pier k,i .

4. The method for batch detection of pier inclination based on UAV image recognition according to claim 1, characterized in that: In S3), for any pier image file P k,i , obtain the endpoint coordinates of its left boundary line EL k,i , denoted as (XL1, YL1) and (XL2, YL2) respectively, and use the arctangent function to calculate the inclination angle AL k,i of the left boundary line EL of the pier k,i , （1）； For any pier image file P k,i , obtain the endpoint coordinates of its right boundary line ER k,i , denoted as (XR1, YR1) and (XR2, YR2) respectively, and use the inverse tangent function to calculate the inclination angle AR k,i of the right boundary line ER k,i , （2）； When the calculated value of AL k,i or AR k,i obtained from formulas (1) and (2) is negative, increase the calculation result by 180°.

5. The method for batch detection of pier inclination based on UAV image recognition according to claim 1, wherein: Combine the pier elevation images in two opposite directions. By calculating the difference in the inclination angles of the pier in these two images, eliminate the influence of the rolling angle deviation of the drone's gimbal, obtain the true angle between the pier and the physical plumb line, and then calculate the inclination degree based on the tangent value of this angle.

6. The method for batch detection of pier inclination based on UAV image recognition according to claim 5, characterized in that: In S4), for any pier numbered k, the inclination I perpendicular to the bridge direction is calculated using the following formula k,s , （3）； The inclination I parallel to the bridge direction is calculated using the following formula k,f , （4）。 7. The method for batch detection of pier inclination based on UAV image recognition according to claim 1, wherein: In S5), for all N piers, return the set of pier inclinations I = {I k,s , I k,f , k ∈ {1, 2, 3, …, N}}, which is the quantitative detection result of the inclination of each pier. For the allowable value δ of the pier inclination, return the set J of the pier numbers with inclinations exceeding the allowable value, that is, for any j ∈ J, |I j,s | > δ or |I j,f | > δ, and return the total number of elements in the set J.

Images