An intelligent perception method for plane displacement field of bridge structure

Abstract

The present invention proposes an intelligent perception method for the plane displacement field of a bridge structure, which belongs to the field of monitoring, including: setting a target at a key position of the bridge to be tested, scanning the bridge using a camera fixed on an automatic rotating pan-tilt head, projecting the scanned image to the same plane through perspective transformation and splicing to obtain a two-dimensional panoramic image; eliminating camera disturbance, using the GrabCut algorithm to extract the image of the bridge structure to be tested, and performing grid discretization; calibrating the camera using the target to eliminate distortion, and calculating the pixel-displacement conversion coefficient at the same time, using a template matching algorithm improved by quadratic surface fitting to obtain the displacement of key nodes; using triangle shape functions and rectangle shape functions to transmit the displacement information of key nodes, so as to obtain the plane displacement field of the bridge structure to be tested. This method realizes accurate, fast and non-contact acquisition of the plane displacement field of the bridge structure. The present invention has the advantages of convenience and accuracy.

Description

Technical Field

[0001] The present invention belongs to the field of bridge engineering detection and health monitoring, and specifically relates to an intelligent perception method for a plane displacement field of a bridge structure. Background Art

[0002] Deformation information is crucial for assessing the integrity and status of bridge structures, as it can reflect abnormalities or defects in the structure. In structural inspection or health monitoring systems, displacement measurement is the main way to obtain structural deformation information. Therefore, displacement measurement is closely related to modal identification, structural damage identification, and structural safety assessment, and is an indispensable part. The displacement measurement method based on computer vision mainly uses a camera to track feature areas and analyze their movement to achieve structural displacement measurement. This method can achieve fast and low-cost non-contact measurement, and has received widespread attention from researchers in recent years.

[0003] However, the focus of these latest methods is on the displacement measurement of a single point or multiple points on the structure. For large-scale bridge structures, the current methods can only obtain the displacement information of a certain point or several points on the structure. Due to the limited field of view of the camera, it is impossible to image and analyze large-scale bridge structures at the same time. The current visual measurement method cannot measure the plane displacement field of the bridge structure. Therefore, it is urgent to develop a set of vision-based intelligent perception methods for the plane displacement field of bridge structures based on the latest computer vision technology. Summary of the invention

[0004] Based on the above shortcomings, the purpose of the present invention is to provide an intelligent perception method for the plane displacement field of a bridge structure, which can more intuitively reflect the state of the entire structure and provide a richer data basis for subsequent disease identification and safety assessment of the bridge structure.

[0005] The technical solution adopted by the present invention is as follows: a method for intelligently sensing the plane displacement field of a bridge structure, which is implemented by the following steps:

[0006] Step 1: Set a target with obvious features at the key position of the bridge structure to be tested. The key position is determined to ensure that two adjacent images obtained by subsequent scanning imaging contain at least one target;

[0007] Step 2: Use a visible light camera fixed on an automatic rotating pan-tilt head to perform plane scanning imaging of the bridge structure to be measured within a certain distance;

[0008] Step 3: Project the scanned images onto the same plane through perspective transformation and perform image stitching to obtain a two-dimensional panoramic image of the bridge structure to be tested;

[0009] Step 4: Select the background area of ​​the non-bridge part in the image as the static reference point, eliminate the disturbance of the camera, use the GrabCut algorithm to extract the image of the bridge structure to be tested, and discretize it into a grid;

[0010] Step 5: Use the target to calibrate the camera to eliminate distortion, calculate the pixel-displacement conversion coefficient, and use the template matching algorithm improved by quadratic surface fitting to obtain the displacement of key nodes;

[0011] Step 6: Use triangular shape functions and rectangular shape functions to transmit the displacement information of key nodes, so as to obtain the plane displacement field of the bridge structure to be tested.

[0012] Furthermore, the process of scanning the bridge structure plane in step 2 includes:

[0013] The visible light camera fixed on the automatic rotating pan-tilt head is set at the mid-span position of the bridge structure to be measured, and the vertical distance from the plane of the bridge structure to be measured is determined based on the focal length of the camera and the accuracy requirements of the displacement measurement; the initial position of the automatic rotating pan-tilt head needs to ensure that the vertical plane where the lens of the visible light camera is located is parallel to the plane of the bridge structure to be measured; the rotation angle of the automatic rotating pan-tilt head needs to be determined by the forward overlap rate and lateral overlap rate of adjacent images required for image stitching, and the determination of the rotation speed should not produce motion blur.

[0014] Furthermore, the process of mesh discretizing the two-dimensional image of the bridge structure plane in step 4 includes:

[0015] After camera disturbance correction and re-cropping, the grid size is determined, and the horizontal and vertical lines of the grid are drawn on the panoramic image of the same size. At this time, the grid has not yet discarded the background part, and is a grid of the full image size, which is a regular rectangular grid discretization of the full image; the image extracted from the bridge structure to be tested in the two-dimensional panoramic image is subjected to binary threshold processing, the background is set to 0, and the bridge structure position is set to 1, forming a binary image, called a mask; each pixel of the mask is bitwise ANDed with the corresponding pixel of the full-size grid image, that is, 1&1=1, 0&1= 0, thereby retaining the grid within the range of the bridge structure; connecting the end points of the straight lines within the range of the bridge structure, and confining the vertical and horizontal grid lines in it, forming a grid with a boundary. At this time, the grid division has been initially formed, and the grid that does not touch the boundary is a regular rectangular grid; the grid that touches the boundary has different rules and needs to be further refined. The grid shapes at the edge are trapezoids, pentagons and triangles. In order to unify the shape and refine it, the trapezoids and pentagons are divided into several triangles, and the discrete division of the entire displacement field grid is completed.

[0016] Furthermore, the process of using triangle and rectangle shape functions to transmit the displacement information of key nodes in step 6 includes:

[0017] (1) The displacements at the four key nodes of the known rectangular grid are obtained as p i , then the displacement p(a,b) of all non-nodes in the grid is:

[0018] p(a,b)=∑Y i p i (1)

[0019] Where a and b are the regularized coordinates of the rectangular grid, i is the shape function of node i, which satisfies the following formula:

[0020] Y i =0.25v(a+m)(b+n) (2)

[0021] Among them, v, m and n are adjustment coefficients. When i is an odd number, v is 1, otherwise it is -1. When i is 2 or 3, m is 1, otherwise it is -1. When i is greater than 2, n is 1, otherwise it is -1.

[0022] (2) The displacements at the three key nodes of the known triangular mesh are obtained as q i , then the displacement q(x,y) of all non-nodes in the grid is:

[0023] q(x, y) = ∑X i q i (3)

[0024] Where x and y are the coordinates of the triangular mesh, Xi is the shape function of node i, which satisfies the following formula:

[0025] X i (x, y)=[(x1y2+x2y3+x3y1)-(x2y1+x3y2+x1y3)](h i +j i x+k i y) (4)

[0026] in:

[0027]

[0028] h, j, k are adjustment coefficients, (x1, y1), (x2, y2), (x3, y3) are the coordinates of the 1st, 2nd, and 3rd nodes of the triangle mesh respectively.

[0029] The beneficial effects and advantages of the present invention are as follows: the present invention realizes the intelligent perception of the plane displacement field of a large-scale bridge structure based on computer vision technology; the present invention is convenient and accurate, and improves the intelligent level of bridge structure displacement measurement. The present invention can also meet the displacement measurement needs of bridge structure construction monitoring and completion acceptance, and improves the informationization and automation level of bridge construction and management and maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 2D panoramic image of the bridge structure.

[0031] Figure 2 Image mesh discretization results of the plane part of the bridge structure.

[0032] Figure 3 Perception results of the plane displacement field of the bridge structure. DETAILED DESCRIPTION

[0033] The present invention aims at the limitation that the existing visual displacement measurement method can only perceive the displacement of a single point or multiple points. After obtaining a large-scale image of the plane of the bridge structure by using the image stitching method, the image of the structural parts is segmented and then grid discretized. The displacement information of the key nodes is transmitted to the non-nodes through the shape function concept with physical meaning, thereby realizing the accurate, fast and non-contact acquisition of the plane displacement field of the bridge structure considering the physical limitations. The present invention is further explained below by taking an example and using a large-scale bridge model test set up in the laboratory. The experimental object is the cantilever span of a three-span scaled plexiglass bridge.

[0034] (2644mm).

[0035] Embodiment 1:

[0036] An intelligent perception of the plane displacement field of a bridge structure is implemented according to the following steps:

[0037] Step 1: Set targets with obvious features at key positions of the bridge structure to be tested. The key positions are determined to ensure that two adjacent images obtained by subsequent scanning imaging contain at least one target. Targets are scattered on the surface in a checkerboard pattern, starting from the free end of the bridge and marked every 100 mm.

[0038] Step 2: Use a visible light camera fixed on an automatic rotating pan-tilt head to perform scanning imaging of the bridge structure to be measured within a certain distance; set up the pan-tilt head at a vertical distance of 3.5m from the half span of the bridge, and set the camera focal length to a maximum value of 50mm to obtain as many web details as possible. Due to the small height of the bridge, the pan-tilt head's rotation angle is set to 2°, and 20 partial images of the entire bridge can be collected by rotating the pan-tilt head left and right only in the direction of the bridge span. This step addresses the problems of limited field of view and poor synchronization of multiple cameras shooting simultaneously in current visual displacement measurement methods. It is proposed to apply an automatic rotating pan-tilt head to visual displacement measurement in combination with subsequent image stitching technology to obtain a larger field of view;

[0039] Step 3: Project the scanned images onto the same plane through perspective transformation and perform image stitching to obtain a two-dimensional panoramic image of the bridge structure to be measured; combine the tilt angle of each image when it is taken, and unify the projection plane of each local image through perspective transformation; finally, realize the bridge panoramic stitching image based on SIFT feature points; in the stitching image, the scattered checkerboard pattern is not deformed, and the conversion coefficient obtained is 0.073734mm / pixel; the relative error of the visual measurement distance of 27 100mm spacing marking points is within 1%; the visual measurement displacement error of the two measuring points where sensors are installed is within 2%, indicating that the image stitching effect is good;

[0040] Step 4: Select the background area of ​​the non-bridge part in the image as the static reference point, eliminate the disturbance of the camera, use the GrabCut algorithm to extract the image of the bridge structure to be measured, and discretize it into a grid; This step aims at the technical gap in the current visual displacement measurement research in the displacement field of the structural plane. It is proposed to use the grid discretization method to describe and solve the displacement field after the two-dimensional panoramic imaging of the structural plane to be measured, which not only speeds up the calculation process, but also can adjust the grid size autonomously according to the accuracy requirements;

[0041] Step 5: Use the target to calibrate the camera to eliminate distortion, and calculate the pixel-displacement conversion coefficient. Use the template matching algorithm improved by quadratic surface fitting to obtain the displacement of key nodes. Use Zhang's calibration method to obtain the camera internal parameters and complete the camera calibration. The calibration error is only 0.178 pixels, and the distortion correction is completed accordingly. Use the Harris corner point detection method to obtain the conversion coefficient corresponding to different shooting distances according to the actual distance of the corner point. Use the zero-mean normalized correlation coefficient and use template matching to obtain the whole pixel displacement. Then, based on the discrete correlation coefficient matrix, use the quadratic surface fitting sub-pixel algorithm to obtain the sub-pixel displacement, and the measurement error is within 1 pixel.

[0042] Step 6: Use triangle shape function and rectangle shape function to transmit the displacement information of key nodes, so as to obtain the plane displacement field of the bridge structure to be tested;

[0043] (1) The displacements at the four key nodes of the known rectangular grid are obtained as p i , then the displacement p(a,b) of all non-nodes in the grid is:

[0044] p(a,b)=∑Y i p i (1)

[0045] Where a and b are the regularized coordinates of the rectangular grid, i is the shape function of node i, which satisfies the following formula:

[0046] Y i =0.25v(a+m)(b+n) (2)

[0047] Among them, v, m and n are adjustment coefficients. When i is an odd number, v is 1, otherwise it is -1. When i is 2 or 3, m is 1, otherwise it is -1. When i is greater than 2, n is 1, otherwise it is -1.

[0048] (2) The displacements at the three key nodes of the known triangular mesh are obtained as q i , then the displacement q(x, y) of all non-nodes in the grid is:

[0049] q(x, y) = ∑X i q i (3)

[0050] Where (x, y) are the coordinates of the triangular mesh, Xi is the shape function of node i, which satisfies the following formula:

[0051] X i (x, y)=[(x1y2+x2y3+x3y1)-(x2y1+x3y2+x1y3)](h i +j i x+k i y) (4)

[0052] in:

[0053]

[0054] h, j, k are adjustment coefficients, (x1, y1), (x2, y2), (x3, y3) are the coordinates of the 1st, 2nd, and 3rd nodes of the triangle mesh respectively;

[0055] This step aims to solve the problem that after the discretization of the panoramic plane image of the bridge structure, only the displacement at the nodes can be solved through the above process, but the displacement at the non-nodes cannot be quickly solved. It creatively draws on the idea of ​​shape function in the finite element method, and uses triangular shape functions and rectangular shape functions to model the node-non-node information transmission, thereby achieving a fast solution of non-node displacements that meets physical limitations.

[0056] Example 2

[0057] In order to verify the accuracy of the algorithm proposed in the present invention, a finite element model was established for the bridge model used in the experiment using Abaqus, and the data of the vertical displacement field were extracted and compared with the displacement field data calculated by the present invention. The normalized correlation coefficient (Formula 6) was used to describe the similarity of the two displacement fields.

[0058]

[0059] Finally, the normalized correlation coefficient was obtained to reach 0.9995, which verified the accuracy of the algorithm proposed in the present invention.

Claims

1. An intelligent perception method for plane displacement field of a bridge structure, characterized in that: The method is implemented by the following steps: Step 1: Set a target with obvious features at the key position of the bridge structure to be tested. The key position is determined to ensure that two adjacent images obtained by subsequent scanning imaging contain at least one target; Step 2: Use a visible light camera fixed on an automatic rotating pan-tilt head to perform plane scanning imaging of the bridge structure to be measured within a certain distance; Step 3: Project the scanned images onto the same plane through perspective transformation and perform image stitching to obtain a two-dimensional panoramic image of the bridge structure to be tested; Step 4: Select the background area of ​​the non-bridge part in the image as the static reference point, eliminate the disturbance of the camera, use the GrabCut algorithm to extract the image of the bridge structure to be tested, and discretize it into a grid; Step 5: Use the target to calibrate the camera to eliminate distortion, calculate the pixel-displacement conversion coefficient, and use the template matching algorithm improved by quadratic surface fitting to obtain the displacement of key nodes; Step 6: Use triangular shape functions and rectangular shape functions to transmit the displacement information of key nodes, so as to obtain the plane displacement field of the bridge structure to be tested.

2. According to claim 1, a method for intelligently sensing a plane displacement field of a bridge structure is characterized by: Step 2: The process of scanning the bridge structure plane includes: The visible light camera fixed on the automatic rotating pan-tilt head is set at the mid-span position of the bridge structure to be measured, and the vertical distance from the plane of the bridge structure to be measured is determined based on the focal length of the camera and the accuracy requirement of displacement measurement; The initial position of the automatic pan-tilt head must ensure that the vertical plane where the lens of the visible light camera is located is parallel to the plane of the bridge structure to be measured; The rotation angle of the automatic pan-tilt head needs to be determined by the forward overlap rate and lateral overlap rate of adjacent images required for image stitching, and the rotation speed should not produce motion blur.

3. The method for intelligently sensing the plane displacement field of a bridge structure according to claim 2 is characterized by: Step 4: The process of meshing and discretizing the two-dimensional image of the bridge structure plane includes: After camera disturbance correction and re-cropping, the grid size is determined, and the horizontal and vertical lines of the grid are drawn on the panoramic image of the same size. At this time, the grid has not yet discarded the background part, and is a grid of the full image size, which is a regular rectangular grid discretization of the full image; the image extracted from the bridge structure to be tested in the two-dimensional panoramic image is subjected to binary threshold processing, the background is set to 0, and the bridge structure position is set to 1, forming a binary image, called a mask; each pixel of the mask is bitwise ANDed with the corresponding pixel of the full-size grid image, that is, 1&1=1, 0&1= 0, thereby retaining the grid within the range of the bridge structure; connecting the end points of the straight lines within the range of the bridge structure, and confining the vertical and horizontal grid lines in it, forming a grid with a boundary. At this time, the grid division has been initially formed, and the grid that does not touch the boundary is a regular rectangular grid; the grid that touches the boundary has different rules and needs to be further refined. The grid shapes at the edge are trapezoids, pentagons and triangles. In order to unify the shape and refine it, the trapezoids and pentagons are divided into several triangles, and the discrete division of the entire displacement field grid is completed.

4. The method for intelligently sensing the plane displacement field of a bridge structure according to claim 3 is characterized by: Step 6 The process of using triangle and rectangle shape functions to transmit the displacement information of key nodes includes: (1) The displacements at the four key nodes of the known rectangular grid are obtained as p i , then the displacement p(a,b) of all non-nodes in the grid is: p(a,b)=∑Y i p i (1) Where a and b are the regularized coordinates of the rectangular grid, i is the shape function of node i, which satisfies the following formula: Y i =0.25v(a+m)(b+n) (2) Among them, v, m and n are adjustment coefficients. When i is an odd number, v is 1, otherwise it is -1. When i is 2 or 3, m is 1, otherwise it is -1. When i is greater than 2, n is 1, otherwise it is -1. (2) The displacements at the three key nodes of the known triangular mesh are obtained as q i , then the displacement q(x,y) of all non-nodes in the grid is: q(x,y)=ΣX i q i (3) Where (x, y) is the coordinate of the triangle mesh, X i is the shape function of node i, which satisfies the following formula: X i (x,y)=[(x1y2+x2y3+x3y1)-(x2y1+x3y2+x1y3)](h i +j i x+k i y) (4) in: h, j, k are adjustment coefficients, (x1, y1), (x2, y2), (x3, y3) are the coordinates of the 1st, 2nd, and 3rd nodes of the triangle mesh respectively.

Images