Bridge displacement measurement method and device based on unmanned aerial vehicle vision

Abstract

The invention relates to the technical field of bridge deformation monitoring, and particularly provides a bridge displacement measurement method and device based on unmanned aerial vehicle vision, and the method comprises the steps: S1, arranging a measurement target and a reference target below a to-be-measured bridge; s2, controlling the unmanned aerial vehicle loaded with the double-camera device to fly to a position between the measurement target and the reference target below the bridge for image acquisition; s3, calculating the pose change of the unmanned aerial vehicle in a hovering state according to the reference target image data collected by the auxiliary camera, and further obtaining the pose change of an imaging plane of the auxiliary camera; s4, correcting the pose of the imaging plane of the main camera according to the pose change of the imaging plane of the auxiliary camera; and S5, calculating the continuous displacement of the measurement target according to the measurement target image data acquired by the main camera, thereby realizing bridge displacement measurement. According to the method, flexible and easy-to-operate bridge displacement high-precision measurement can be realized, and the measurement process is simple, efficient and high in safety.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of bridge deformation monitoring, in particular to a bridge displacement measurement method and device based on unmanned aerial vehicle vision. BACKGROUND

[0002] For bridge facilities crossing water areas and gorges, the dependence of the ground station visual method on the fixed platform limits its applicability when deployed on site. Unmanned aerial vehicles are very suitable as a low-cost mobile platform due to their small size and high mobility, effectively solving the bottleneck problems of installation inconvenience and insufficient imaging resolution caused by fixed platforms. The existing technology uses unmanned aerial vehicles for bridge measurement, including: (1) The patent application publication number CN 109813510 A, entitled "High-speed rail bridge vertical disturbance measurement method based on unmanned aerial vehicle", uses a camera on the unmanned aerial vehicle to take images of the measurement points on the high-speed rail bridge at a set time interval, while a fixed total station instrument at a distance measures the position coordinates of the markers on the unmanned aerial vehicle at the same set time interval as the camera on the unmanned aerial vehicle. The data output by the total station instrument and the inertial navigation system of the unmanned aerial vehicle are used to correct the image coordinates of the measurement points, and the vertical disturbance of the high-speed rail bridge is obtained based on the corrected image coordinates of the measurement points. The main disadvantage is that the total station instrument is far from the unmanned aerial vehicle and the unmanned aerial vehicle is constantly changing, resulting in low accuracy of the total station instrument measurement coordinates; the unmanned aerial vehicle carries additional markers, increasing the load of the unmanned aerial vehicle and reducing the measurement endurance time of the unmanned aerial vehicle; the measurement frequency of the total station instrument is limited, which prevents the unmanned aerial vehicle from conducting high-frequency measurement.

[0003] (2) The patent application publication number CN 109855822 A, entitled "High-speed rail bridge vertical disturbance measurement method based on unmanned aerial vehicle", arranges measurement points and two reference points on the high-speed rail bridge to be measured, and installs three cameras with the same imaging center and imaging resolution on the unmanned aerial vehicle. The three cameras can capture the measurement points and the two reference points, respectively. The three cameras on the unmanned aerial vehicle are used to take images of the high-speed rail bridge to be measured at a set time interval, and the vertical disturbance of the high-speed rail bridge to be measured is calculated based on the images taken by the three cameras. However, the three-camera device relies heavily on the camera calibration in the early stage, and the independent imaging method of the three non-parallel cameras is not very convenient to use, requiring high control and hovering stability of the unmanned aerial vehicle, which restricts the popularization and application of the technology.

[0004] (3) The patent application publication number CN 110470226 A, the patent name is "Bridge structure displacement measurement method based on unmanned aerial vehicle system", which utilizes the unmanned aerial vehicle to hover at the surface of the bridge structure to be measured, and collects images of the bridge vibration process; the collected images are tracked and analyzed frame by frame through digital image correlation technology, and the displacement of the bridge structure is extracted; the collected images are calibrated through Zhang Zhengyou calibration method, and the motion displacement of the unmanned aerial vehicle is solved; the displacement of the bridge structure is subtracted from the motion displacement of the unmanned aerial vehicle, and the real displacement of the corrected bridge structure is obtained. The disadvantage of this method is that the checkerboard target needs to be placed on the ground under the bridge, so it is not suitable for bridges across valleys or rivers.

[0005] In addition, in the above technical solutions, the unmanned aerial vehicle flies on the side or above the bridge, which has certain safety hazards, such as the unmanned aerial vehicle falling due to electromagnetic interference, losing control and colliding with the train or car in motion, touching the overhead contact system and causing power failure, etc. In order to ensure the safety of bridge transportation and the safety of life and property, the transportation department prohibits unmanned aerial vehicles from flying within 50 meters of the railway and provincial highway and its two sides, and within 100 meters of the high-speed railway and its two sides; under the limitation of the above flight rules, the unmanned aerial vehicle can choose to observe the bridge at a long distance, but long distance observation will result in low image resolution and affect the measurement accuracy.

[0006] In summary, there is an urgent need to provide a bridge displacement measurement method and device based on unmanned aerial vehicle vision to solve the technical problems existing in the prior art. SUMMARY

[0007] The purpose of the present application is to provide a bridge displacement measurement method and device based on unmanned aerial vehicle vision to solve the technical problems existing in the prior art, and the specific technical solutions are as follows: A bridge displacement measurement method based on unmanned aerial vehicle vision, comprising the following steps: Step S1, arranging a measurement target and a reference target under the bridge to be measured; Step S2, controlling the unmanned aerial vehicle loaded with a dual-camera device to fly to the measurement target and the reference target under the bridge for image collection; wherein the dual-camera device includes a main camera and a sub-camera arranged in a back-to-back manner, the main camera has a line of sight towards the measurement target, and the sub-camera has a line of sight towards the reference target; Step S3, calculating the pose change of the unmanned aerial vehicle in the hovering state based on the reference target image data collected by the sub-camera, and then obtaining the pose change of the imaging plane of the sub-camera; Step S4, correcting the pose of the imaging plane of the main camera according to the pose change of the imaging plane of the sub-camera; Step S5: Calculate the continuous displacement of the target based on the target image data acquired by the main camera, thus realizing bridge displacement measurement.

[0008] Furthermore, in step S1, the bridge to be tested includes two piers and a beam erected between the two piers; the measuring target is set below the beam, and the reference target is set at the pier, with the measuring target and the reference target positioned opposite each other and at the same height.

[0009] Furthermore, in step S2, the drone equipped with the dual-camera device flies to the middle position between the measurement target and the reference target, with the line of sight of the dual-camera device parallel to the line of sight in the direction of the bridge and road.

[0010] Furthermore, step S3 specifically involves: Using the first frame of the reference target image data acquired by the secondary camera as the reference frame, a homography matrix is ​​used. H Describe the pose change between the imaging planes of the secondary camera before and after the UAV pose change: 1); in, This indicates the image coordinates of the point corresponding to the first frame of the reference target image data. This indicates the point corresponding to the image coordinates of the current frame of the reference target image data.

[0011] Furthermore, step S4 specifically involves: Homography matrix H Perform an inverse transformation to correct the pose of the main camera's imaging plane: 2); in, This represents the image coordinates of the current frame of the target image acquired by the main camera before pose correction. This indicates the point corresponding to the pose-corrected image coordinates of the current frame of the target image acquired by the main camera.

[0012] Furthermore, step S5 specifically involves: Using the first frame of the target image data acquired by the main camera as the reference frame, the displacement of the current frame relative to the reference frame after pose correction is calculated; and so on, the displacement of each subsequent frame of the image data acquired by the main camera relative to the first frame is calculated, thereby obtaining the continuous displacement data of the target, that is, the continuous displacement data of the bridge beam, and realizing bridge displacement measurement.

[0013] Furthermore, taking the first frame of the target image data acquired by the main camera as the reference frame, the displacement of the current frame relative to the reference frame after pose correction is calculated as follows: 3); wherein, represents the horizontal and vertical displacement of the current frame of the measurement target image acquired by the main camera after pose correction; represents the image coordinates of the reference frame of the measurement target image; represents the image scale factor.

[0014] A bridge displacement measurement device based on unmanned aerial vehicle vision is applied in the bridge displacement measurement method as described above, comprising a double camera device, a cloud platform adapter plate and an unmanned aerial vehicle, the double camera device is connected with the unmanned aerial vehicle platform through the cloud platform adapter plate, and is used for realizing image acquisition of the measurement target and the reference target.

[0015] Further, the double camera device comprises a main camera, a secondary camera and a mounting seat, the main camera and the secondary camera are oppositely arranged on the two mounting surfaces of the mounting seat, and the lines of sight of the main camera and the secondary camera are parallel and opposite.

[0016] Further, a camera interface is mounted on the side surface adjacent to the main camera or the secondary camera of the mounting seat, the camera interface is connected with the main camera and the secondary camera, and is used for power supply and data transmission of the main camera and the secondary camera.

[0017] The technical scheme of the application has the following beneficial effects: (1) The bridge displacement measurement method based on unmanned aerial vehicle vision provided by the application arranges measurement targets and reference targets below the bridge to be measured, and an unmanned aerial vehicle flies to the position below the bridge to collect images between the measurement targets and the reference targets. This way of applying the unmanned aerial vehicle to the displacement measurement under the bridge can ensure the measurement accuracy while avoiding safety accidents. In addition, the measurement method of the application uses an unmanned aerial vehicle to carry a double camera device, and does not use a total station. The method has the advantages of simple use, flexibility and high efficiency. The data acquisition frequency does not need to consider the measurement frequency of the total station, and can be flexibly set according to the measurement requirements. At the same time, a single pier is used as a reference datum, and the displacement of a bridge with a larger span can be measured. The method is suitable for bridges crossing valleys or rivers, and has better applicability to different types of bridges.

[0018] (2) The application also provides a bridge displacement measurement device based on unmanned aerial vehicle vision, comprising a double camera device, a cloud platform adapter plate and an unmanned aerial vehicle, the double camera device is mounted on the unmanned aerial vehicle platform in a line-of-sight horizontal manner through the cloud platform adapter plate, and is used for realizing image acquisition of the measurement target and the reference target. The device is simple to operate, flexible and efficient. Only two cameras are used, and the structure is simpler and the cost is lower.

[0019] In addition to the purposes, features and advantages described above, the application has other purposes, features and advantages. The application will be further described in detail below with reference to the drawings. Attached Figure Description

[0020] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a flowchart of the bridge displacement measurement method based on UAV vision in this invention; Figure 2 This is a schematic diagram showing the target placement and the drone's flight position; Figure 3 This is a schematic diagram of the reference target; Figure 4 This is a schematic diagram of the measurement target; Figure 5 This is a schematic diagram of the bridge displacement measurement device based on UAV vision in this invention (UAV is not shown). Among them, 1. Dual camera device, 1.1. Main camera, 1.2. Secondary camera, 1.3. Mounting base, 1.4. Camera interface, 2. Gimbal adapter plate, A. Pier, B. Beam, C. Reference target, D. Measurement target. Detailed Implementation

[0021] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, the present invention can be implemented in many different ways as defined and covered.

[0022] In the description of this invention, it should be noted that the terms "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", "front", "back", "lateral", "longitudinal", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0023] Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0024] Example: See Figure 1 This embodiment provides a bridge displacement measurement method based on UAV vision, including the following steps: Step S1: Arrange the measurement target D and the reference target C under the bridge to be measured; see [link / reference]. Figure 2The bridge to be measured includes two piers A and a beam B erected between the two piers; the measuring target D is set below the beam B, preferably at the mid-span of the beam; the reference target C is set at pier A. Both the measuring target D and the reference target C are arranged vertically, and the measuring target D and the reference target C are arranged opposite each other at the same height. The plane of the target graphic faces the camera.

[0025] Step S2: Control the drone equipped with a dual-camera system to fly between the measurement target D and the reference target C below beam B (preferably at an approximately midpoint). At this position, adjust the drone's attitude so that the line of sight of the dual-camera system is parallel to the line of sight in the direction of the bridge road, and the height of the line of sight is consistent with the height of the target. The dual-camera system includes a main camera and a secondary camera positioned back-to-back. The main camera's line of sight is towards the measurement target D, and the secondary camera's line of sight is towards the reference target C. After the measurement begins, the main and secondary cameras will synchronously acquire target image data.

[0026] Step S3: The object of measurement in this invention is the displacement of the bridge beam. Since the position of pier A can be approximately considered stable relative to the beam B, that is, the reference target C arranged at pier A is relatively stable, the pose change of the UAV in the hovering state can be calculated from the reference target image data collected by the secondary camera, and then the pose change of the secondary camera imaging plane can be obtained; specifically: Using the first frame of the target image data acquired by the secondary camera as the reference frame, changes in the UAV's pose will cause changes in the pose of the camera's imaging plane relative to the first frame. Therefore, a homography matrix can be used. H Describe the pose change between the imaging planes of the secondary camera before and after the UAV pose change: 1); in, This represents the image coordinates of the first frame of the reference target image data (before the UAV pose changes). This indicates the image coordinates corresponding to the current frame of the reference target image data (after the UAV pose has changed).

[0027] Homography matrix H It is a 3×3 matrix with 8 degrees of freedom. Solving the homography matrix requires at least 4 pairs of corresponding points. Therefore, in this embodiment, the graphic design of the reference target C is as follows: Figure 3 The form shown is given. The reference target C consists of 5×5 alternating black and white squares, containing a total of 4×4=16 cross-shaped corner points, satisfying the homography matrix. H The solution requirement.

[0028] Step S4: Correct the pose of the main camera's imaging plane based on the pose change of the secondary camera's imaging plane, i.e., use the homography matrix calculated in step S3. HThe pose of the main camera imaging plane is corrected, and specifically: The homography matrix H is inversely transformed, and the pose of the main camera imaging plane is corrected: 2) ; Wherein, represents the image coordinate corresponding point of the current frame of the measurement target image collected by the main camera before the pose correction, represents the image coordinate corresponding point of the current frame of the measurement target image collected by the main camera after the pose correction. It should be noted that the lines of sight of the main camera and the auxiliary camera are parallel and opposite in direction, so the inverse transformation of the homography matrix H should also be processed in reverse, and a negative sign is added in front of the matrix .

[0029] Step S5, calculate the continuous displacement of the measurement target D according to the measurement target image data collected by the main camera, that is, realize the measurement of the displacement of the bridge; Specifically: Take the first frame of the measurement target image data collected by the main camera as the reference frame, and calculate the displacement of the current frame relative to the reference frame after the pose correction; Specifically: 3) ; Wherein, represents the horizontal and vertical displacement of the current frame of the measurement target image obtained by the main camera after the pose correction; represents the image coordinates of the reference frame of the measurement target image; represents the image scale factor, which can convert pixel units to actual physical units. The pattern of the measurement target D is shown in Figure 4 , and the scale factor can be calculated by the following formula: 4) ; In formula 4), represents the actual physical distance between the two cross-shaped corner points in the pattern of the measurement target D, for example, 200 mm; represents the pixel distance of the two cross-shaped corner points in the pattern, which can be calculated after the corner point coordinates are obtained by the corner point detection algorithm.

[0030] The displacement of the current frame of the main camera image data relative to the first frame can be calculated by formula 3), and similarly, the displacement of each subsequent frame of the image data collected by the main camera relative to the first frame can be calculated, and then the continuous displacement data of the measurement target D, that is, the continuous displacement data of the bridge girder, is obtained, and the measurement of the displacement of the bridge is realized.

[0031] The bridge displacement measurement method based on unmanned aerial vehicle vision provided by the application arranges a measurement target and a reference target below a bridge to be measured, and an unmanned aerial vehicle flies to the measurement target and the reference target below the bridge to collect images, so that the unmanned aerial vehicle is applied to the displacement measurement of the position below the bridge, the measurement accuracy is ensured, and safety accidents are avoided. In addition, the measurement method of the application uses an unmanned aerial vehicle to carry a double-camera device, does not use a total station, has the advantages of simple use, flexibility and high efficiency, and the data acquisition frequency does not need to consider the measurement frequency of the total station, and can be flexibly set according to the measurement requirement. Meanwhile, a single pier is used as a reference datum, the displacement of a bridge with a larger span can be measured, the bridge measurement method is suitable for bridges crossing valleys or rivers, and has better applicability to different types of bridges.

[0032] The application further provides a bridge displacement measurement device based on unmanned aerial vehicle vision, which is applied to the bridge displacement measurement method. Figure 5 The device comprises a double-camera device 1, a holder adapter plate 2 and an unmanned aerial vehicle (not shown in the figure), the double-camera device 1 is mounted on the unmanned aerial vehicle platform in a line-of-sight horizontal manner through the holder adapter plate 2, and is used for collecting images of the measurement target D and the reference target C.

[0033] In the embodiment, the double-camera device 1 comprises a main camera 1.1, an auxiliary camera 1.2 and a mounting seat 1.3, the main camera 1.1 and the auxiliary camera 1.2 are oppositely arranged on two mounting surfaces of the mounting seat 1.3, and the line-of-sight directions of the main camera 1.1 and the auxiliary camera 1.2 are opposite and parallel. In the embodiment, the sensors of the two cameras have the same parameters, the maximum pixel number is 1280*1080, and the pixel resolution is 4.8

[0034] In the embodiment, a camera interface 1.4 is mounted on the side surface adjacent to the main camera 1.1 or the auxiliary camera 1.2 of the mounting seat 1.3, the camera interface 1.4 is connected with the main camera 1.1 and the auxiliary camera 1.2, the two cameras share one camera interface, and the camera interface is used for power supply and image data transmission of the main camera 1.1 and the auxiliary camera 1.2.

[0035] The bridge displacement measurement device based on unmanned aerial vehicle vision adopted by the application is simple to operate, flexible and efficient, has a simpler structure and lower cost due to the use of only two cameras.

[0036] The above only describes the preferred embodiments of the application and is not used to limit the application, and the application can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the application shall be included in the protection scope of the application.

Claims

1. A bridge displacement measurement method based on UAV vision, characterized in that, Includes the following steps: Step S1: Place measurement targets and reference targets under the bridge to be measured; Step S2: Control the drone equipped with a dual-camera device to fly between the measurement target and the reference target under the bridge to collect images; wherein, the dual-camera device includes a main camera and a secondary camera set back from each other, the main camera's line of sight is towards the measurement target, and the secondary camera's line of sight is towards the reference target; Step S3: Calculate the pose change of the UAV in hovering state using the reference target image data acquired by the secondary camera, and then obtain the pose change of the secondary camera imaging plane. Step S4: Correct the pose of the main camera's imaging plane based on the pose change of the secondary camera's imaging plane. Step S5: Calculate the continuous displacement of the target based on the target image data acquired by the main camera, thus realizing bridge displacement measurement.

2. The bridge displacement measurement method based on UAV vision according to claim 1, characterized in that, In step S1, the bridge to be tested includes two piers and a beam erected between the two piers; the measuring target is set below the beam, and the reference target is set at the pier, with the measuring target and the reference target positioned opposite each other and at the same height.

3. The bridge displacement measurement method based on UAV vision according to claim 2, characterized in that, In step S2, the drone equipped with dual cameras flies to the middle position between the measurement target and the reference target, with the line of sight of the dual cameras parallel to the line of sight in the direction of the bridge and road.

4. The bridge displacement measurement method based on UAV vision according to claim 1, characterized in that, Step S3 is as follows: Using the first frame of the reference target image data acquired by the secondary camera as the reference frame, a homography matrix is ​​used. H Describe the pose change between the imaging planes of the secondary camera before and after the UAV pose change: 1）； in, This indicates the image coordinates of the point corresponding to the first frame of the reference target image data. This indicates the point corresponding to the image coordinates of the current frame of the reference target image data.

5. The bridge displacement measurement method based on UAV vision according to claim 4, characterized in that, Step S4 is as follows: Homography matrix H Perform an inverse transformation to correct the pose of the main camera's imaging plane: 2）； in, This represents the image coordinates of the current frame of the target image acquired by the main camera before pose correction. This indicates the point corresponding to the pose-corrected image coordinates of the current frame of the target image acquired by the main camera.

6. The bridge displacement measurement method based on UAV vision according to claim 5, characterized in that, Step S5 is as follows: Using the first frame of the target image data acquired by the main camera as the reference frame, the displacement of the current frame relative to the reference frame after pose correction is calculated; and so on, the displacement of each subsequent frame of the image data acquired by the main camera relative to the first frame is calculated, thereby obtaining the continuous displacement data of the target, that is, the continuous displacement data of the bridge beam, and realizing bridge displacement measurement.

7. The bridge displacement measurement method based on UAV vision according to claim 6, characterized in that, Using the first frame of the target image data captured by the main camera as the reference frame, the displacement of the current frame relative to the reference frame after pose correction is calculated as follows: 3）； in, This indicates the horizontal and vertical displacement of the target image acquired by the main camera in the current frame after pose correction. This represents the image coordinates of the reference frame for measuring the target image; This represents the image scale factor.

8. A bridge displacement measurement device based on UAV vision, applied in the bridge displacement measurement method as described in any one of claims 1-7, characterized in that, It includes a dual-camera device (1), a gimbal adapter plate (2), and a drone. The dual-camera device (1) is connected to the drone platform through the gimbal adapter plate (2) to realize image acquisition of the measurement target and the reference target.

9. A bridge displacement measurement device based on UAV vision according to claim 8, characterized in that, The dual-camera device (1) includes a main camera (1.1), a secondary camera (1.2) and a mounting base (1.3). The main camera (1.1) and the secondary camera (1.2) are arranged opposite to each other on two mounting surfaces of the mounting base (1.3). The lines of sight of the main camera (1.1) and the secondary camera (1.2) are parallel and their lines of sight are opposite.

10. A bridge displacement measurement device based on UAV vision according to claim 9, characterized in that, The mounting base (1.3) has a camera interface (1.4) installed on the side adjacent to the main camera (1.1) or the secondary camera (1.2). The camera interface (1.4) is connected to the main camera (1.1) and the secondary camera (1.2) and is used to supply power to the main camera (1.1) and the secondary camera (1.2) and transmit data.

Images