Method for correcting motion of onboard camera in unmanned aerial vehicle visual displacement monitoring

Abstract

The present application relates to unmanned aerial vehicle vision displacement monitoring technical field, specifically provide a kind of unmanned aerial vehicle vision displacement monitoring on-board camera motion correction method, comprising the following steps: installation monitoring target and two reference targets, acquisition unmanned aerial vehicle vision image data;Calculate the image point coordinates of two corner points on all targets, the image point displacement of two corner points midpoint and the scale factor of target;Calculate the z-axis rotation motion error of monitoring target;Calculate the z-axis translation motion error of monitoring target;Calculate the x and y axis rotation and translation motion error of monitoring target;Correct the rotation and translation motion error of z axis, x axis and y axis, obtain the actual displacement result of monitoring target.The method provided by the present application does not need to rely on the rotation center when correcting the z-axis rotation motion of the camera, and is suitable for the actual situation where the rotation center deviates from the image center, thereby improving the applicability of the method while effectively improving the accuracy of unmanned aerial vehicle vision displacement monitoring.

Description

Technical Field

[0001] This invention relates to the field of UAV visual displacement monitoring technology, and specifically to a method for correcting the motion of an airborne camera in UAV visual displacement monitoring. Background Technology

[0002] Structural health monitoring is a crucial stage in the entire life cycle of engineering structures and is of great significance for ensuring the safe operation of structures. In recent years, with the rapid development of visual displacement monitoring technology and drone technology, drone-based visual displacement monitoring technology has emerged and is widely used in the field of structural health monitoring of civil infrastructure such as bridge vibration measurement and slope deformation monitoring.

[0003] However, applying UAV visual displacement monitoring technology to high-precision displacement monitoring of engineering structures still faces many challenges. The most critical issue is that changes in attitude and displacement during UAV hovering cause instability in the onboard camera, thus affecting the accuracy of displacement measurements. To address this, scholars have proposed various solutions, including filtering techniques, background compensation methods, and UAV attitude compensation methods. However, these methods have certain limitations in practical applications: filtering techniques require a significant difference between the UAV's vibration frequency and the structure's vibration frequency, limiting their applicability; UAV attitude compensation methods rely on high-precision attitude data acquisition, but this condition is difficult to meet in complex environments; background compensation methods require the monitoring point to be as close as possible to the reference background, and the choice of background and its distance from the monitoring point have a significant impact on the results. In recent research, the paper "UAV Visual Displacement Measurement Method with Camera Motion Error Compensation" summarizes the motion laws of UAVs and conducts work on compensating for camera motion errors based on this. However, this method still has two limitations: (1) the solution of z-axis rotation error depends on the rotation center coinciding with the image center, so it is difficult to apply to actual working conditions where the rotation center deviates from the image center; (2) the correction process of camera motion error lacks a detailed description, the method is not very operable, and it is difficult to apply it directly to actual engineering.

[0004] In summary, this invention provides a method for correcting the motion of an airborne camera in UAV visual displacement monitoring, which is more efficient, more universal, and easier to implement for correcting camera motion errors. Summary of the Invention

[0005] The purpose of this invention is to provide a method for motion correction of an airborne camera in visual displacement monitoring of unmanned aerial vehicles (UAVs), in order to solve the technical problems existing in the prior art. The specific technical solution is as follows:

[0006] The motion correction method for airborne cameras in UAV visual displacement monitoring includes the following steps:

[0007] Step S1: Install a monitoring target and two reference targets on the monitoring target, at least one of the reference targets being vertically mounted; collect visual image data from the UAV using its onboard camera.

[0008] Step S2: Calculate the image point coordinates of the two corner points of all targets in the UAV visual image data, the image point displacement of the midpoint between the two corner points, and the scale factor of the target.

[0009] Step S3: Calculate the z-axis rotational motion error of the monitoring target based on the vertically positioned reference target. Specifically:

[0010] Step S3.1, Definition These represent the x-direction motion error components of the z-axis rotational motion errors of points N, M, and P, respectively. These are the motion error components in the y-direction of the z-axis rotational motion errors of points N, M, and P, respectively. Point N is the midpoint of the vertically placed reference target corner point, point M is the midpoint of another reference target corner point, and point P is the midpoint of the monitoring target corner point.

[0011] Step S3.2: Calculate the rotation angle θ z ;

[0012] Step S3.3: Based on the rotation angle θ z Calculate the z-axis rotational motion error of point P on the monitoring target;

[0013] Step S4: Calculate the z-axis translational motion error of point P on the monitoring target based on the unit pixel displacement change;

[0014] Step S5: Calculate the x-axis rotation and translation error and the y-axis rotation and translation error of the monitoring target based on the two reference targets;

[0015] Step S6: Correct the rotation and translation errors of the z-axis, x-axis and y-axis to obtain the actual displacement results of the monitoring target.

[0016] Furthermore, step S2 specifically involves:

[0017] Step S2.1: Obtain the image point coordinates of two corner points on the monitoring target and the reference target according to the corner detection and localization method;

[0018] Step S2.2: Based on the image point coordinates of the two corner points, calculate the pixel distance between the two corner points of the same target according to the distance formula; based on the image point coordinates of the two corner points, obtain the midpoint coordinates of the target corner points;

[0019] Step S2.3: Calculate the scale factor SF for each target. The calculation formula is as follows:

[0020]

[0021] In the above formula, d pixel denoted as , where is the pixel distance between two corner points of the same target, and D is the actual distance between the two corner points of the same target;

[0022] Step S2.4: Perform the calculations from steps S2.1 to S2.3 on each image in a set of image sequences. The image point displacement can be obtained by subtracting the midpoint coordinates of the two corner points of the target in each image from the midpoint coordinates of the two corner points of the target in the first image.

[0023] Furthermore, in step S3.2, the rotation angle θ is calculated. z Specifically:

[0024] (1) Calculate the x-direction image displacement of the two corner points based on the x-direction components of the image point coordinates of the two corner points of the reference target where point N is located;

[0025] (2) Establish a z-axis rotational motion model; define points N, M, and P as rotating about the z-axis by θ. z The points after rotation are N′, M′, and P′, and their corresponding image points before and after rotation are n(x) and n(x) respectively. n ,y n ), m(x m ,y m ), p(x p ,y p ) and n′(x′ n ,y′ n ), m′(x′) m ,y′ m ), p′(x′) p ,y′ p ); C(x) in the image plane c ,y c () is the center of rotation;

[0026] When points N, M, and P rotate counterclockwise around the z-axis by θ z At that time, the image point coordinates of N′, M′, and P′ are:

[0027]

[0028] Then the displacement of the image point after rotating points N, M, and P is:

[0029]

[0030] Perform differential processing on the image point displacement:

[0031]

[0032] Similarly, when points N, M, and P rotate clockwise by θ around the z-axis... z At that time, the image point displacement difference processing result is:

[0033]

[0034] (3) Calculate the rotation angle θ z Since the reference target at point N is vertically positioned, the x-coordinates of the two corner points n1 and n2 on this target are equal, and due to the rotation angle θ along the z-axis... z For a small angle, then (cosθ) z -1)< <sinθ z Simplifying equation 4), we get equation 8:

[0035]

[0036] In the formula, and This represents the x-direction motion error component in the z-axis rotational motion error of the two corner points on the reference target. and The y-axis components of the image coordinates of the two corner points on the reference target; Solve using the image point displacement difference in the x direction of the two corner points obtained in (1);

[0037] Solve for the z-axis rotation angle θ according to equation 8). z .

[0038] Furthermore, in step S3.3, the formula for calculating the z-axis rotational motion error between point M on the reference target and point P on the monitoring target is as follows:

[0039]

[0040] The z-axis rotational motion errors of points M and P relative to point N are obtained by solving equations 9 and 10.

[0041] Furthermore, in step S4, the calculation of the z-axis translational motion error of the monitored target based on the unit pixel displacement change is specifically as follows:

[0042] Establish a z-axis translational motion model;

[0043] Calculate the unit pixel displacement change, and based on the unit pixel displacement change and the distances from the image points of points N, M, and P to the midpoint of the image, solve for the z-axis translation error of point P.

[0044] Furthermore, the specific steps to solve for the z-axis translation error of point P are:

[0045] The camera translates along the z-axis. Taking the change in the y-coordinate of point N as an example, the distance of point N from the z-axis is N. y The point N after translation is N′, o is the image center, f is the focal length, and the corresponding image points of point N before and after translation are n(x) and n′, respectively.n ,y n ) and n′(x′ n ,y′ n If the coordinate change error of point N is... for:

[0046]

[0047] Unit pixel displacement change for:

[0048]

[0049] Equation 12) is transformed to obtain Equation 13):

[0050]

[0051] Let n be the scale factor of image point n. Let be the scale factor of image point on′;

[0052] According to Equation 13), the formula for calculating the translational error of point N along the z-axis is:

[0053]

[0054] Similarly, the formula for the motion error in the y-direction of the z-axis translational motion error of points N, M, and P is:

[0055]

[0056] The formula for the motion error in the x-direction of the z-axis translational motion error of points N, M, and P is:

[0057]

[0058] In the above formula, |on| y ,、|om| y |op| y Let |on| be the distance between the y-coordinates of points n, m, and p and H / 2. x 、|om| x |op| x H represents the distance between the x-coordinates of image points n, m, and p and W / 2, where H represents the image height and W represents the image width. These are the x-direction motion error components of the z-axis translational motion errors of points N, M, and P, respectively. These are the y-direction motion error components of the z-axis translational motion errors of points N, M, and P, respectively.

[0059] Furthermore, step S5 specifically involves combining the two reference targets to solve for the rotational and translational motion errors of point P on the monitoring target along the x-axis and the y-axis.

[0060] Furthermore, step S6 specifically involves subtracting the rotation and translation errors of point P along the x, y, and z axes from the actual displacement of point P to obtain the actual displacement result of point P, which is the actual displacement result of the monitoring target.

[0061] The application of the technical solution of the present invention has the following beneficial effects:

[0062] (1) This invention provides a method for correcting the motion of an airborne camera in UAV visual displacement monitoring, specifically including the following steps: installing a monitoring target and a reference target on the monitoring target; acquiring UAV visual image data through the UAV's airborne camera; calculating the image point coordinates of two corner points on all targets in the UAV visual image data, the image point displacement of the midpoint between the two corner points, and the scale factor of the target; calculating the z-axis rotational motion error of the monitoring target based on the vertically placed reference target; calculating the z-axis translational motion error of the monitoring target based on the unit pixel displacement change; calculating the x-axis rotational and translational motion errors and the y-axis rotational and translational motion errors of the monitoring target based on the two reference targets; correcting the rotational and translational motion errors of the z-axis, x-axis, and y-axis to obtain the actual displacement result of the monitoring target. The method provided by this invention does not rely on the rotation center when correcting the camera's z-axis rotational motion, and is applicable to situations where the rotation center deviates from the image center, thus improving the applicability of the method while effectively improving the accuracy of UAV visual displacement monitoring.

[0063] (2) The present invention provides detailed steps for camera motion correction, which improves the operability of the method; in addition, the method of the present invention can calculate the z-axis rotational motion error by using a single reference target, which improves the efficiency of camera motion error correction.

[0064] (3) The method provided by the present invention calculates the z-axis translational motion error based on the unit pixel displacement change. Since the scale change is the same at different points (the numerator of the unit pixel displacement change calculation formula), the scale change can be calculated based on the reference target closest to the camera to improve the accuracy of the unit pixel displacement change calculation, thereby improving the correction accuracy of the z-axis translational motion error.

[0065] In addition to the objectives, features, and advantages described above, the present invention has other objectives, features, and advantages. The invention will now be described in further detail with reference to the figures. Attached Figure Description

[0066] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0067] Figure 1 This is a flowchart of the airborne camera motion correction method in UAV visual displacement monitoring in this invention;

[0068] Figure 2 This is the measurement model in the embodiments of the present invention;

[0069] Figure 3 The directions of rotation and translation of the airborne camera in this embodiment of the invention;

[0070] Figure 4 This is a schematic diagram of the rotational motion of an airborne camera about the z-axis according to an embodiment of the present invention;

[0071] Figure 5 This is a schematic diagram of the translational motion of the airborne camera along the z-axis according to an embodiment of the present invention;

[0072] Figure 6 This is a schematic diagram of an experimental scenario for UAV visual displacement monitoring;

[0073] Figure 7 It is a diagram showing the on-site layout of the target, translation slide, and fixed camera;

[0074] Figure 8 These are images taken by a camera mounted on a drone.

[0075] Figure 9 It is the horizontal (x-direction) displacement curve of the monitored target T2 before and after the motion error correction of the UAV's airborne camera;

[0076] Figure 10 It is the vertical (y-direction) displacement curve of the monitored target T2 before and after the motion error correction of the UAV's airborne camera;

[0077] Figure 11 It is the horizontal (x-direction) displacement curve of the monitored target T3 before and after the motion error correction of the UAV's airborne camera;

[0078] Figure 12 It is the vertical (y-direction) displacement curve of the monitored target T3 before and after the motion error correction of the UAV's airborne camera;

[0079] Figure 13 It shows the horizontal (x-direction) displacement curve of the target T4 before and after the motion error correction of the UAV's onboard camera;

[0080] Figure 14 It shows the vertical (y-direction) displacement curve of the monitored target T4 before and after the motion error correction of the UAV's airborne camera;

[0081] Figure 15 This is a comparison chart of the displacement curves of the monitoring target T2 measured by the UAV's airborne camera and the fixed camera. Detailed Implementation

[0082] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, the present invention can be implemented in many different ways as defined and covered.

[0083] In the description of this invention, it should be noted that the terms "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", "front", "back", "lateral", "longitudinal", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0084] Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0085] Example:

[0086] See Figure 1 This embodiment provides a method for motion correction of an airborne camera in visual displacement monitoring of an unmanned aerial vehicle (UAV), including the following steps:

[0087] Step S1, see Figure 2 A monitoring target and a reference target are installed on the monitoring target. There is one monitoring target and two reference targets. At least one reference target is vertically placed. In this embodiment, preferably, the reference target closer to the camera is vertically placed. Visual image data of the UAV is collected by the UAV's onboard camera.

[0088] Step S2: Calculate the image point coordinates of the two corner points of all targets in the UAV visual image data, the image point displacement of the midpoint between the two corner points, and the scale factor of the target; specifically:

[0089] Step S2.1: Obtain the image point coordinates of two corner points on the monitoring target and the reference target according to the corner detection and localization method in Reference 1;

[0090] Step S2.2: Based on the image point coordinates of the two corner points, calculate the pixel distance between the two corner points of the same target according to the distance formula; based on the image point coordinates of the two corner points, obtain the midpoint coordinates of the target corner points;

[0091] Step S2.3: Calculate the scale factor SF for each target. The calculation formula is as follows:

[0092]

[0093] In the above formula, d pixel is the pixel distance between two corner points of the same target, in pixels; D is the actual distance between the two corner points of the same target (obtained by measuring the distance between the two corner points), in mm.

[0094] Step S2.4: Perform the calculations from steps S2.1 to S2.3 on each image in a set of image sequences. The image point displacement can be obtained by subtracting the midpoint coordinates of the two corner points of the target in each image from the midpoint coordinates of the two corner points of the target in the first image.

[0095] Step S3: Calculate the z-axis rotational motion error of the monitoring target based on the vertically positioned reference target. Specifically:

[0096] Step S3.1: The motion of the airborne camera includes rotational and translational motion, occurring in the x, y, and z directions respectively, such as... Figure 3 As shown. The z-axis rotational motion error of the monitoring target caused by the camera's rotation around the z-axis includes motion error components in the x-direction and y-direction; defined... These represent the x-direction motion error components of the z-axis rotational motion errors of points N, M, and P, respectively. These are the motion error components in the y-direction of the z-axis rotational motion errors of points N, M, and P, respectively. Point N is the midpoint of the vertically placed reference target corner point, point M is the midpoint of another reference target corner point, and point P is the midpoint of the monitoring target corner point.

[0097] Step S3.2: Calculate the rotation angle θ z Specifically:

[0098] (1) Calculate the x-direction image displacement of the two corner points based on the x-direction components of the image point coordinates of the two corner points of the reference target where point N is located;

[0099] (2) Establish the z-axis rotational motion model, see Figure 4 Define points N, M, and P as rotating θ around the z-axis. z The points after rotation are N′, M′, and P′, and their corresponding image points before and after rotation are n(x) and n(x) respectively. n ,y n ), m(x m ,y m ), p(x p ,y p ) and n′(x′ n,y′ n ), m′(x′) m ,y′ m ), p′(x′) p ,y′ p ); forming C(x) in the image plane c ,y c () is the center of rotation;

[0100] When points N, M, and P rotate counterclockwise around the z-axis by θ z At that time, the image point coordinates of N′, M′, and P′ are:

[0101]

[0102] Then the displacement of the image point after rotating points N, M, and P is:

[0103]

[0104] Perform differential processing on the image point displacement:

[0105]

[0106] Similarly, when points N, M, and P rotate clockwise by θ around the z-axis... z At that time, the image point displacement difference processing result is:

[0107]

[0108] According to equations 4)-7), the multi-point displacement difference caused by the rotation of points N, M, and P around the z-axis is independent of the rotation center. That is, this method is applicable to z-axis rotational motion with the rotation center at any position in the image plane.

[0109] (3) Calculate the rotation angle θ z Since the reference target at point N is vertically positioned, the x-coordinates of the two corner points n1 and n2 on this target are equal, and due to the rotation angle θ along the z-axis... z For small angles (less than 0.1°), then (cosθ) z -1)< <sinθ z Simplifying equation 4), we get equation 8:

[0110]

[0111] In the formula, and This represents the x-direction motion error component in the z-axis rotational motion error of the two corner points on the reference target. and The y-axis components of the image coordinates of the two corner points on the reference target.

[0112] The image point displacement difference in the x direction of the two corner points obtained in (1) is used to solve the problem: Since the two corner points on the reference target are located on the same target, their rotation and translation motion error components in the x and y directions, as well as the translation motion error component in the z axis (the same target scale factor) are the same. That is, after the image point displacement difference in the x direction of the two corner points is obtained, it is the difference of the motion error component in the x direction in the z axis rotation motion error of the two corner points.

[0113] Solve for the z-axis rotation angle θ according to equation 8). z .

[0114] Step S3.3: Based on the rotation angle θ z The z-axis rotational motion error between point M on the reference target and point P on the monitoring target is calculated using the following formula:

[0115]

[0116] The z-axis rotational motion errors of points M and P relative to point N are obtained by solving equations 9 and 10.

[0117] Step S4: Calculate the z-axis translational motion error of point P on the monitoring target based on the unit pixel displacement change; specifically:

[0118] Establish a z-axis translational motion model, see [link / reference]. Figure 5 Calculate the unit pixel displacement change, and based on the unit pixel displacement change and the distances from the image points N, M, and P to the midpoint of the image, solve for the z-axis translational motion error of point P. When the camera translates along the z-axis, it results in image point displacement in both the x and y directions on the image plane. The z-axis translational motion error caused by the camera's z-axis translation includes motion error components in the x and y directions.

[0119] The camera translates along the z-axis. Taking the change in the y-coordinate of point N as an example, such as... Figure 5 As shown, let the camera's translation along the z-axis be Δ. s The distance from point N to the z-axis is N. y The point N after translation is N′, o is the image center, f is the focal length, and the corresponding image points of point N before and after translation are n(x) and n′, respectively. n ,y n ) and n′(x′ n ,y′ n If the coordinate change error of point N is... for:

[0120]

[0121] Unit pixel displacement change for:

[0122]

[0123] Equation 12) is transformed to obtain Equation 13):

[0124]

[0125] Let n be the scale factor of image point n. Let be the scale factor of image point on′; For scale changes, by formula The calculated value is the z-axis translation Δ. s Furthermore, since the focal length *f* is independent of the measurement point's position, it can be concluded that the scale change is consistent across different points. To improve the unit pixel displacement variation... To improve the accuracy of z-axis translation error correction, the scale change is calculated using the nearest reference target (the one closest to the camera) at point N. Then, based on the scale factor of different points, the unit pixel displacement change at each point is calculated. This method effectively improves the accuracy of z-axis translation error correction, ensuring accuracy in practical applications.

[0126] According to Equation 13), the formula for calculating the translational error of point N along the z-axis is:

[0127]

[0128] Similarly, the formula for the motion error in the y-direction of the z-axis translational motion error of points N, M, and P is:

[0129]

[0130] The formula for the motion error in the x-direction of the z-axis translational motion error of points N, M, and P is:

[0131]

[0132] In the above formula, |on| y ,、|om| y |op| y Let |on| be the distance between the y-coordinates of points n, m, and p and H / 2. x 、|om| x |op| x H represents the distance between the x-coordinates of image points n, m, and p and W / 2, where H represents the image height and W represents the image width. These are the x-direction motion error components of the z-axis translational motion errors of points N, M, and P, respectively. These are the y-direction motion error components of the z-axis translational motion errors of points N, M, and P, respectively.

[0133] Step S5: Calculate the x-axis rotation and translation error and the y-axis rotation and translation error of the monitoring target based on the two reference targets;

[0134] Specifically, according to the method in Reference 2, the axis shift angle in the method is set to 0° (at this time, the method can be used to correct the rotation and translation motion errors of the x-axis and y-axis of a conventional camera), and the rotation and translation motion errors of the x-axis and y-axis of point P on the monitoring target are solved by combining two reference targets.

[0135] Step S6: Subtract the rotation and translation errors of point P on the x, y and z axes from the actual displacement of point P to obtain the actual displacement result of point P, which is the actual displacement result of the monitoring target.

[0136] Reference 1: LIU J, DAI W, ZHANGY, et al. AnAdaptive Radon-Transform-BasedMarker Detection and Localization Method for Displacement Measurements Using Unmanned Aerial Vehicles[J]. Sensors, 2024, 24(6):1930.

[0137] Reference 2: XING L, DAI W, ZHANG Y. Scheimpflug Camera-Based Technique for Multi-Point Displacement Monitoring of Bridges[J]. Sensors, 2022, 22(11):4093.

[0138] An experiment on visual displacement monitoring using unmanned aerial vehicles (UAVs) was conducted. The experimental equipment is shown in Table 1.

[0139] Table 1 Experimental Equipment for Visual Displacement Monitoring of Unmanned Aerial Vehicles

[0140]

[0141] Experimental scenarios such as Figure 6 As shown in the diagram, reference targets, numbered T1 and T5, were positioned at both ends of the survey area, spaced 28m apart. T1 was 15m from the airborne camera, and T5 was 43m from the airborne camera. Between the two reference targets, three monitoring targets, numbered T2, T3, and T4, were fixed at equal intervals of 7m. Target T2 had a translational slide mounted on its bottom to simulate vertical displacement, and a fixed camera was positioned 3m in front of it to measure the true displacement of the slide.

[0142] Figure 7 This is a diagram showing the on-site layout of the target, translation slide, and fixed camera. Figure 8 Images captured by the drone's onboard camera. Both the onboard and fixed cameras had a sampling rate of 100 frames per second, and the experiment lasted 20 seconds. Due to significant vibrations in the drone system during the experiment, some targets were lost from the camera's field of view; ultimately, a total of 1695 valid images were acquired. Figures 9-14 The displacement curves of the monitoring targets T2, T3, and T4 before and after correction using the method of this invention are shown.

[0143] To further verify the measurement accuracy of the method, the corrected T2 target displacement monitoring results were compared and analyzed with reference data acquired by a fixed camera, using a total of 1500 images. Figure 15 The displacement results of the monitoring target T2 measured by the airborne camera and the fixed camera were compared. The comparison results show that the corrected displacement curve is highly consistent with the monitoring results of the fixed camera, which intuitively verifies the measurement accuracy of this method.

[0144] To quantitatively assess the measurement error, the root mean square error (RMSE) statistical analysis was performed on the above results. The specific statistical results are shown in Table 2.

[0145] Table 2. RMSE Statistics of UAV Visual Displacement Monitoring Experiment Results (mm)

[0146]

[0147] As shown in Table 2, the root mean square error (RMSE) of the displacement measurement results of the monitoring targets in the vertical direction (y-axis) and horizontal direction (x-axis) after correction by the method of this invention is significantly reduced. Taking the measurement results of the fixed camera as the true value, for target T2, the RMSE of the airborne camera measurement results after error correction is only 0.11 mm. After motion error correction, the RMSE of the measurement results of all monitoring targets is less than 0.3 mm, indicating that the correction method effectively improves the measurement accuracy.

[0148] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for motion correction of an airborne camera in visual displacement monitoring of unmanned aerial vehicles (UAVs), characterized in that, Includes the following steps: Step S1: Install a monitoring target and two reference targets on the monitoring target, at least one of the reference targets being vertically mounted; collect visual image data from the UAV using its onboard camera. Step S2: Calculate the image point coordinates of the two corner points of all targets in the UAV visual image data, the image point displacement of the midpoint between the two corner points, and the scale factor of the target. Step S3: Calculate the z-axis rotational motion error of the monitoring target based on the vertically positioned reference target. Specifically: Step S3.1, Definition , , Points N, M , P The motion error component in the x-direction of the z-axis rotational motion error. , , Points N, M , P The motion error component in the y-direction of the z-axis rotational motion error, where point N The midpoint of the vertically positioned reference target corner point, point M The midpoint of another reference target corner. P To monitor the midpoint of the target corner; Step S3.2: Calculate the z-axis rotation angle. ; Step S3.3: Rotate according to the z-axis angle Calculate the monitoring target P z-axis rotational motion error of the point; Step S4: Calculate the value of the monitored target based on the unit pixel displacement change. P The z-axis translational error of the point; Step S5: Calculate the x-axis rotation and translation error and the y-axis rotation and translation error of the monitoring target based on the two reference targets; Step S6: Correct the rotation and translation errors of the z-axis, x-axis, and y-axis to obtain the actual displacement results of the monitored target; the z-axis, x-axis, and y-axis are the coordinate system of the airborne camera; In step S3.2, the rotation angle along the z-axis is calculated. Specifically: (1) Based on the point N The x-direction displacements of the image points at the two corner points of the reference target are calculated using the x-direction components of the image point coordinates. (2) Establish the z-axis rotational motion model; define the point N, M , P Rotation about the z-axis The point after is , , The corresponding image points before and after the rotation are respectively , , and , , In the image plane Center of rotation; On point N, M , P Rotate counterclockwise around the z-axis hour, , , The image point coordinates are: Equation 2); Then point N, M , P The displacement of the image point after rotation is: Equation 3); Perform differential processing on the image point displacement: Equation 4); Equation 5); Similarly, when point N , M , P Rotate clockwise around the z-axis At that time, the image point displacement difference processing result is: Formula 6); Equation 7); (3) Calculate the rotation angle along the z-axis ; due to point N If the reference target is vertically positioned, then for the two corner points on the target... and The x-coordinates of the two are equal, and due to the rotation angle along the z-axis... For small angles, then << Simplifying equation 4), we get equation 8. Equation 8); In the formula, and This represents the x-direction motion error component in the z-axis rotational motion error of the two corner points on the reference target. and The y-axis components of the image coordinates of the two corner points on the reference target; Solve using the image point displacement difference in the x direction of the two corner points obtained in (1); Solve for the z-axis rotation angle according to equation 8). .

2. The method for correcting the motion of an airborne camera in UAV visual displacement monitoring according to claim 1, characterized in that, Step S2 is as follows: Step S2.1: Obtain the image point coordinates of two corner points on the monitoring target and the reference target according to the corner detection and localization method; Step S2.2: Based on the image point coordinates of the two corner points, calculate the pixel distance between the two corner points of the same target according to the distance formula; based on the image point coordinates of the two corner points, obtain the midpoint coordinates of the target corner points; Step S2.3: Calculate the scale factor for each target. The calculation formula is as follows: Formula 1); In the above formula, The pixel distance between two corner points of the same target. D The actual distance between two corner points of the same target; Step S2.4: Perform the calculations from steps S2.1 to S2.3 on each image in a set of image sequences. The image point displacement can be obtained by subtracting the midpoint coordinates of the two corner points of the target in each image from the midpoint coordinates of the two corner points of the target in the first image.

3. The method for correcting the motion of an airborne camera in UAV visual displacement monitoring according to claim 2, characterized in that, In step S3.3, refer to the point on the target. M With the monitoring target point P The formula for calculating the z-axis rotational motion error is as follows: Equation 9); Formula 10); The point is obtained by solving equations 9 and 10. M With point P Relative to point N The error of rotational motion along the z-axis.

4. The method for correcting the motion of an airborne camera in UAV visual displacement monitoring according to claim 2, characterized in that, In step S4, the calculation of the z-axis translational motion error of the monitored target based on the unit pixel displacement change is specifically as follows: Establish a z-axis translational motion model; Calculate the unit pixel displacement change, based on the unit pixel displacement change and the point N, M, P Solving for the distance from the image point to the midpoint of the image. P The error of translational motion along the z-axis.

5. The method for motion correction of airborne cameras in UAV visual displacement monitoring according to claim 4, characterized in that, Solution point P The specific error of the z-axis translational motion is: The camera translates along the z-axis, with point N Taking the resulting y-coordinate change as an example, point N The distance from the z-axis is ,point N The point after translation is , Center of the image For focal length, point N The corresponding image points before and after the translation are respectively and Then point N coordinate transformation error for: Equation 11). Unit pixel displacement change for: Equation 12); Equation 12) is transformed to obtain Equation 13): Equation 13); For image points n The scale factor, For image points Scale factor; According to equation 13), point... N The formula for calculating the z-axis translation error is: Equation 14). Similarly, for point N, M , P The formula for the motion error in the y-direction of the z-axis translational motion error is: Equation 15); For point N, M , P The formula for the motion error in the x-direction of the z-axis translational motion error is: Equation 16). In the above formula, , , For image points n, m, p The distance between the y-coordinate and H / 2, , , For image points n, m, p The distance between the x-coordinate and W / 2, where H represents the image height and W represents the image width; , , Points N, M , P The motion error component in the y-direction of the z-axis translational motion error; , , Points N, M , P The x-direction motion error component of the z-axis translational motion error.

6. The method for motion correction of airborne cameras in UAV visual displacement monitoring according to claim 1, characterized in that, Step S5 specifically involves combining the two reference targets to solve for the points on the monitoring target. P The rotational and translational motion errors along the x-axis and the y-axis.

7. The method for motion correction of airborne cameras in UAV visual displacement monitoring according to claim 1, characterized in that, Step S6 specifically involves placing the point P The rotational and translational motion errors along the x, y, and z axes originate from point... P Subtracting from the actual displacement, we get the point. P The actual displacement result is obtained, that is, the actual displacement result of the monitoring target.

Images