Unmanned aerial vehicle monocular vision bridge multi-point vibration displacement synchronous monitoring method

Abstract

The invention relates to the technical field of unmanned aerial vehicle displacement monitoring, in particular to an unmanned aerial vehicle monocular vision bridge multi-point vibration displacement synchronous monitoring method, which comprises the following steps of: arranging a plurality of targets along the length direction of a bridge; collecting target image data by using an unmanned aerial vehicle on which a tilt-shift camera is mounted; camera displacement compensation is introduced to process target image data, and target actual displacement is obtained; and monitoring bridge vibration displacement based on the actual displacement of the target. The method has the advantages that all targets are located in the limited view field range of the camera by adopting the monocular tilt-shift imaging principle of the unmanned aerial vehicle, and compared with a multi-camera system, the use cost is lower; the camera displacement compensation calculation disclosed based on unmanned aerial vehicle monocular vision has the advantages of high precision, multi-point synchronous monitoring, simple parameter setting and no need of camera calibration.

Description

Technical Field

[0001] The present invention relates to the technical field of UAV displacement monitoring, and specifically to a method for synchronously monitoring multi-point vibration displacement of a bridge by using UAV monocular vision. Background Art

[0002] Bridges are an important part of transportation infrastructure, and the aging and performance degradation of bridge facilities will have various impacts. Therefore, it is crucial to develop a low-cost, high-precision, non-contact and non-destructive bridge monitoring solution.

[0003] Currently, a common solution is to use a fixed target as a reference to calculate the movement of the UAV. For example, the two ends of the beam body or the piers of the bridge are used as fixed reference targets; however, suitable fixed targets cannot be found for all applications. Therefore, some studies have turned to using signal filtering methods to eliminate the influence of UAV fluctuations, such as high-pass filtering and differential filtering. However, these methods require no overlap between the UAV fluctuation frequency and the bridge structure vibration frequency, so their applicability is greatly limited. Some scholars have also proposed projecting a stable laser beam onto the bridge and resolving the movement of the UAV by means of the light spot formed by the laser beam on the bridge surface, and the effectiveness of this solution has been verified through experiments. However, in practical applications, the implementation of this solution is difficult, and it is also difficult to guarantee the accuracy of light spot detection and positioning.

[0004] Based on the above analysis, UAV vision displacement measurement has the advantages of high efficiency, flexibility, and high degree of autonomy, and shows great application potential in the field of bridge displacement monitoring. However, the problem of UAV body jitter seriously affects the accuracy of its displacement measurement.

[0005] In summary, there is an urgent need for a method for synchronously monitoring multi-point vibration displacement of a bridge by using UAV monocular vision to solve the problems in the prior art. Summary of the Invention

[0006] The purpose of the present invention is to provide a method for synchronously monitoring multi-point vibration displacement of a bridge by using UAV monocular vision. The specific technical solution is as follows:

[0007] A method for synchronously monitoring multi-point vibration displacement of a bridge by using UAV monocular vision includes the following steps:

[0008] Set multiple target marks along the length direction of the bridge;

[0009] Use a UAV equipped with a shift camera to collect target mark image data;

[0010] Introduce camera displacement compensation to process the target mark image data to obtain the actual displacement of the target mark;

[0011] Monitor the vibration displacement of the bridge based on the actual displacement of the target mark.

[0012] Preferably, the targets include a measurement target and a reference target, and each target image data includes two reference targets and one measurement target.

[0013] Preferably, the process of using a drone to measure target image data is as follows:

[0014] The drone hovers at the front end of the beam of the bridge to be measured, so that the line of sight of the on-board camera remains horizontal and the height of the on-board camera is the same as that of the beam of the bridge to be measured. The drone takes target images through a shift camera and collects target image data.

[0015] Preferably, the process of introducing camera displacement compensation to process target image data is as follows:

[0016] Process the target image data to obtain the original image coordinates of the target;

[0017] Calculate the original image displacement of the target based on the image coordinates of the target;

[0018] Solve the camera motion components of the drone;

[0019] Calculate the target image displacement based on the camera motion components and the original image displacement;

[0020] Calculate the actual displacement of the target based on the target image displacement.

[0021] Preferably, during the process of calculating the original image displacement of the target, the scale conversion factor of the plane where the target is located is calculated simultaneously.

[0022] Preferably, the process of solving the camera motion components of the drone is as follows:

[0023] Establish a coordinate system, with the X-axis direction being the horizontal direction, the Y-axis direction being the vertical direction, and the Z-axis direction being along the bridge length direction;

[0024] Calculate the change in the position of the first target image caused by the rotation of the camera around the Z-axis direction;

[0025] Calculate the change in the position of the second target image caused by the displacement of the camera in the X-axis and Y-axis directions based on the original image displacement of the target, the change in the position of the first target image, and the scale transformation factor;

[0026] Calculate the change in the position of the third target image caused by the rotation of the camera around the X-axis and Y-axis directions based on the change in the position of the first target image and the change in the position of the second target image;

[0027] The camera motion components include the change in the position of the first target image, the change in the position of the second target image, and the change in the position of the third target image.

[0028] Preferably, calculating the target image displacement specifically includes: adding the camera motion component to the original image displacement to obtain the target image displacement.

[0029] Preferably, calculating the actual displacement of the target specifically includes: introducing a scale transformation factor, and calculating the actual displacement of the target according to the displacement of the target image.

[0030] The application of the technical solution of the present invention has the following beneficial effects:

[0031] The present invention discloses a high-precision UAV monocular vision bridge multi-point vibration displacement synchronous monitoring method. The method introduces the camera axis shift principle to expand the depth of field of airborne monocular vision imaging, and achieves high resolution and large depth of field clear imaging of airborne monocular vision by means of measurement targets installed and arranged in a linear array; and based on this, an airborne monocular vision bridge vibration displacement multi-point measurement method is proposed. The method of the present invention can effectively extract the actual displacement of the bridge from the original displacement of the UAV, and has the advantages of high precision, multi-point synchronous monitoring, simple parameter setting, and no need for camera calibration.

[0032] In addition to the above-described purposes, features and advantages, the present invention has other purposes, features and advantages. The present invention will be further described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0034] Figure 1 It is a flowchart of the steps of the method for synchronously monitoring multi-point vibration displacement of a bridge using monocular vision of an unmanned aerial vehicle in a preferred embodiment of the present invention;

[0035] Figure 2 It is an overview diagram of the target, bridge and drone in the preferred embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0037] like Figure 1As shown, this embodiment discloses a method for synchronously monitoring the multi-point vibration displacement of a bridge using a single monocular vision of an unmanned aerial vehicle (UAV), specifically a high-precision method for synchronously monitoring the multi-point vibration displacement of a bridge using a single monocular vision of an unmanned aerial vehicle, including the following steps:

[0038] S100: Set multiple target markers along the length direction of the bridge;

[0039] S200: Use a UAV equipped with a shift camera to collect the target marker image data;

[0040] S300: Introduce camera displacement compensation to process the target marker image data to obtain the actual displacement of the target marker;

[0041] S400: Monitor the vibration displacement of the bridge based on the actual displacement of the target marker.

[0042] In this embodiment, the target marker includes a measurement target marker and a reference target marker. Each target marker image data includes two reference target markers and one measurement target marker. In this embodiment, as Figure 2 shown, taking three target markers Ma, Mb, and Mc in one target marker image data as an example, where Ma and Mc are reference target markers, Mb is the measurement target marker, Ma and Mc are located at both ends of the beam body of the bridge, Mb is located between the reference target markers Ma and Mc, and the cross-shaped corner points corresponding to the three target markers are marked as M a1 、M a2 、M b1 、M b2 、M c1 、M c2 in the UAV image, and the subscripts 1 and 2 respectively represent the two corner points of the target marker.

[0043] In this embodiment, the process of using the UAV to measure the target marker image data is as follows:

[0044] The UAV hovers at the front end of the beam body of the bridge to be measured, so that the line of sight of the on-board camera remains horizontal and the height of the on-board camera is the same as that of the beam body of the bridge to be measured. The UAV takes pictures of the target markers through the shift camera to collect the target marker image data.

[0045] In this embodiment, the process of introducing camera displacement compensation to process the target marker image data is as follows:

[0046] S310: Process the target marker image data to obtain the original image coordinates of the target marker. In the process of obtaining the original image displacement of the target marker, the scale conversion factor of the plane where the target marker is located is calculated at the same time. Calculating the scale conversion factor is a conventional technical means and will not be elaborated here.

[0047] S320: Calculate the original image displacement of the target marker based on the image coordinates of the target marker. In this embodiment, and respectively represent the two corner points M of the target marker Maa1 , M a2 Original corner displacements in the X-axis and Y-axis directions respectively represent the original corner displacements of two corner points of target Mb and Mc in the X-axis and Y-axis directions. The above-mentioned various displacements constitute the original image displacement of the target.

[0048] S330. Solve the camera motion components of the UAV. The process is as follows:

[0049] S331. Establish a coordinate system, as Figure 2 shown. The X-axis direction is the horizontal direction, the Y-axis direction is the vertical direction, and the Z-axis direction is along the bridge length direction.

[0050] S332. Calculate the change in the first target image position caused by the rotation of the camera around the Z-axis. The expression is as follows:

[0051]

[0052] where y a1 and y a2 represent the corner image coordinates of target Ma, and θ represents the angle of rotation of the camera around the Z-axis. Based on the angle of rotation of the camera around the Z-axis, the change in the first target image position of target Ma can be obtained and The change in the first target image position of target Mb and Mc can also be calculated through the above calculation method (Change in the first target image position of target Mb) and (Change in the first target image position of target Mc). The calculation process is not elaborated here.

[0053] S333. Calculate the change in the second target image position caused by the rotation of the camera around the Y-axis according to the original image displacement of the target, the change in the first target image position, and the scale transformation factor. The expression is as follows:

[0054]

[0055] where s a represents the scale transformation factor of target Ma, s c represents the scale transformation factor of target Mc, and represent the change in the second target image of target Ma. The change in the second target image position of the other two targets can also be calculated through the above calculation method (Change in the second target image of target Mb) and (Change in the second target image of target Mc). The calculation process is not elaborated here.

[0056] S334. Calculate the change in the position of the third target image caused by the rotation of the camera around the X-axis based on the changes in the positions of the first and second target images. The calculation formula is as follows:

[0057]

[0058] Among them, and represent the changes in the second target image of two corner points in the target Ma. Through the above calculation method, the changes in the positions of the third target images of the other two targets can be obtained (the change in the third target image of the target Mb) and (the change in the third target image of the target Mc). The calculation process will not be elaborated here.

[0059] S335. The camera motion components include the change in the position of the first target image, the change in the position of the second target image, and the change in the position of the third target image.

[0060] S340. Calculate the displacement of the target image based on the camera motion components and the displacement of the original image. Specifically, add the camera motion components to the displacement of the original image to obtain the displacement of the target image. The calculation formula is as follows:

[0061]

[0062] Among them, represents the actual displacement of the bridge of two corner points of the measured target Mb relative to the reference target Ma, represents the actual displacement of the bridge of two corner points of the measured target Mb relative to the reference target Mc.

[0063] S350. Calculate the actual displacement of the target based on the displacement of the target image. Specifically, introduce a scale transformation factor and calculate the actual displacement of the target based on the displacement of the target image.

[0064]

[0065] Among them, CMD y represents the actual displacement of the target in the vertical direction, and s b represents the scale transformation factor.

[0066] It should be noted that in this embodiment, the displacement calculation in the vertical direction (Y-axis direction) of the target is taken as an example, and the calculation method in the horizontal direction (X-axis direction) is exactly the same as that in the Y-axis direction, which will not be elaborated here. At the same time, in this embodiment, the displacement calculation of a single target is taken as an example. For the calculation process of measuring multiple targets, the displacement calculation of a single target can be repeated.

[0067] An embodiment of the present invention discloses a high-precision multi-point vibration displacement synchronous monitoring method for a bridge using an unmanned aerial vehicle (UAV) monocular vision. This method introduces the camera shift principle to expand the depth of field of the airborne monocular vision imaging, and with the help of measurement targets arranged in a linear array, achieves high-resolution and large-depth-of-field clear imaging of the airborne monocular vision; and based on this, a multi-point measurement method for the vibration displacement of the bridge using the airborne monocular vision is proposed. The method of the present invention can effectively extract the actual displacement of the bridge from the original displacement of the UAV, and has the advantages of high precision, multi-point synchronous monitoring, simple parameter setting, and no need for camera calibration.

[0068] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for synchronously monitoring multi-point vibration displacement of bridges using UAV monocular vision, characterized in that: The steps include: Multiple targets were set up along the length of the bridge; Use a drone equipped with a tilt-shift camera to collect target image data; The camera displacement compensation is introduced to process the target image data to obtain the actual displacement of the target; Monitor bridge vibration displacement based on actual target displacement.

2. The method for synchronously monitoring multi-point vibration displacement of a bridge using a monocular vision of an unmanned aerial vehicle according to claim 1 is characterized in that: The targets include measurement targets and reference targets, and each target image data includes two reference targets and one measurement target.

3. The method for synchronously monitoring multi-point vibration displacement of bridges using unmanned aerial vehicle monocular vision according to claim 1 is characterized in that: The process of using drones to measure target image data is as follows: The UAV hovers at the front end of the beam of the bridge to be tested, so that the line of sight of the airborne camera remains horizontal, and the height of the airborne camera is consistent with the beam of the bridge to be tested. The UAV captures the target image through the shift camera and collects the target image data.

4. The method for synchronously monitoring multi-point vibration displacement of bridges using unmanned aerial vehicle monocular vision according to claim 1 is characterized in that: The process of introducing camera displacement compensation to process target image data is as follows: Process the target image data and obtain the original image coordinates of the target; Calculating a raw image displacement of the target based on the image coordinates of the target; Solve for the camera motion components of the drone; The target image displacement is calculated based on the camera motion component and the original image displacement; The actual displacement of the target is calculated based on the displacement of the target image.

5. The method for synchronously monitoring multi-point vibration displacement of bridges using unmanned aerial vehicle monocular vision according to claim 4 is characterized in that: In the process of calculating the displacement of the original image of the target, the scale conversion factor of the plane where the target is located is also calculated.

6. The method for synchronously monitoring multi-point vibration displacement of bridges using unmanned aerial vehicle monocular vision according to claim 5 is characterized in that: Solving the camera motion components of the drone, the process is as follows: Establish a coordinate system, with the X-axis direction being the horizontal direction, the Y-axis direction being the vertical direction, and the Z-axis direction being along the length of the bridge; Calculate the position change of the first target image caused by the rotation of the camera around the Z axis; The position change of the second target image caused by the displacement of the camera in the X-axis direction and the Y-axis direction is calculated according to the original image displacement of the target, the position change of the first target image and the scale transformation factor; The position change of the third target image caused by the rotation of the camera around the X-axis direction and the Y-axis direction is calculated according to the position change of the first target image and the position change of the second target image; The camera motion component includes a first target image position change, a second target image position change, and a third target image position change.

7. The method for synchronously monitoring multi-point vibration displacement of a bridge using a monocular vision of an unmanned aerial vehicle according to claim 6 is characterized in that: The target image displacement is calculated specifically by adding the camera motion component to the original image displacement to obtain the target image displacement.

8. The method for synchronously monitoring multi-point vibration displacement of bridges using unmanned aerial vehicle monocular vision according to claim 7 is characterized in that: The actual displacement of the target is calculated by introducing a scale transformation factor and calculating the actual displacement of the target according to the displacement of the target image.

Images