Airborne camera motion correction method in unmanned aerial vehicle visual displacement monitoring

Abstract

The invention relates to the technical field of unmanned aerial vehicle visual displacement monitoring, and particularly provides an airborne camera motion correction method in unmanned aerial vehicle visual displacement monitoring, comprising the following steps: installing a monitoring target and two reference targets, and collecting unmanned aerial vehicle visual image data; calculating image point coordinates of two angular points on all targets, image point displacement of a midpoint of the two angular points and scale factors of the targets; the z-axis rotation motion error of the monitoring target is calculated; the z-axis translational motion error of the monitoring target is calculated; calculating x-axis and y-axis rotation and translation motion errors of the monitoring target; and correcting rotation and translation motion errors of the z-axis, the x-axis and the y-axis to obtain an actual displacement result of the monitoring target. The method provided by the invention does not depend on the rotation center when correcting the z-axis rotation motion of the camera, is suitable for the actual situation that the rotation center deviates from the image center, and improves the applicability on the premise of effectively improving the visual displacement monitoring precision of the unmanned aerial vehicle.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) visual displacement monitoring, and in particular to an airborne camera motion correction method in UAV visual displacement monitoring. Background Art

[0002] Structural health monitoring is a crucial component of the entire lifecycle of engineering structures and is crucial for ensuring their safe operation. In recent years, with the rapid development of visual displacement monitoring and drone technology, drone-based visual displacement monitoring has emerged and is widely used in structural health monitoring applications for civil infrastructure, such as bridge vibration measurement and slope deformation monitoring.

[0003] However, applying UAV visual displacement monitoring technology to high-precision displacement monitoring of engineering structures still faces numerous challenges. The most critical issue is that changes in the UAV's attitude and displacement during hovering can cause instability in the onboard camera, thereby affecting the accuracy of displacement measurements. To address this issue, researchers have proposed various solutions, including filtering techniques, background compensation, and UAV attitude compensation. However, these methods have limitations in practical applications: filtering techniques require a significant difference between the UAV's vibration frequency and the structure's vibration frequency, limiting their applicability; UAV attitude compensation relies on high-precision attitude data acquisition, a requirement that is difficult to meet in complex environments; and background compensation requires that the monitoring point be as close as possible to a reference background. The choice of background and its distance from the monitoring point significantly influence the results. In a recent study, the paper "UAV Visual Displacement Measurement Method with Camera Motion Error Compensation" summarized the motion patterns of UAVs and used this information to compensate for camera motion errors. However, this method still has two limitations: (1) the solution of the z-axis rotation error depends on the coincidence of the rotation center and the image center, and is therefore difficult to apply to actual working conditions where the rotation center deviates from the image center; (2) the correction process of the camera motion error lacks a detailed description, the method is less operable, and is difficult to be directly applied in actual engineering.

[0004] In summary, the present invention provides a method for correcting camera motion in UAV visual displacement monitoring, which is more efficient, more universal, and easier to implement to correct camera motion errors. Summary of the Invention

[0005] The present invention aims to provide a method for correcting the motion of an onboard camera in visual displacement monitoring of an unmanned aerial vehicle (UAV) to solve the technical problems existing in the prior art. The specific technical solution is as follows:

[0006] The method for correcting the motion of an onboard camera in UAV visual displacement monitoring includes the following steps:

[0007] Step S1: Install a monitoring target and two reference targets on the monitoring target, with at least one reference target being placed vertically; and collect UAV visual image data using an onboard camera of the UAV;

[0008] Step S2, calculating the image point coordinates of two corner points on all targets in the UAV visual image data, the image point displacement of the midpoint between the two corner points, and the scale factor of the target;

[0009] Step S3: Calculate the z-axis rotational motion error of the monitoring target based on the vertically placed reference target. Specifically:

[0010] Step S3.1. Definition are the motion error components in the x-direction of the z-axis rotational motion errors of points N, M, and P, respectively. are the y-direction motion error components of the z-axis rotational motion errors of points N, M, and P, respectively, where point N is the midpoint of a vertically placed reference target corner, point M is the midpoint of another reference target corner, and P is the midpoint of a monitoring target corner;

[0011] Step S3.2: Calculate the rotation angle θ z ;

[0012] Step S3.3, according to the rotation angle θ z , calculate the z-axis rotational motion error of point P on the monitoring target;

[0013] Step S4, calculating the z-axis translation motion error of point P on the monitoring target based on the unit pixel displacement change;

[0014] Step S5: Calculate the x-axis rotation and translation motion errors and the y-axis rotation and translation motion errors of the monitoring target based on the two reference targets;

[0015] Step S6: Correct the rotation and translation errors of the z-axis, x-axis, and y-axis to obtain the actual displacement result of the monitoring target.

[0016] Furthermore, step S2 specifically includes:

[0017] Step S2.1, obtaining the image coordinates of two corner points on the monitoring target and the reference target according to the corner point detection and positioning method;

[0018] Step S2.2, based on the image point coordinates of the two corner points, calculate the pixel distance between the two corner points of the same target according to the distance formula; based on the image point coordinates of the two corner points, obtain the midpoint coordinates of the target corner point;

[0019] Step S2.3: Calculate the scale factor SF of each target using the following formula:

[0020]

[0021] In the above formula, d pixel is the pixel distance between two corner points of the same target, and D is the actual distance between two corner points of the same target;

[0022] Step S2.4: Perform the calculations of steps S2.1 to S2.3 on each image in a set of image sequences. The image point displacement can be obtained by subtracting the midpoint coordinates of the two corner points of the target on each image in the image sequence from the midpoint coordinates of the two corner points of the target corresponding to the first image.

[0023] Furthermore, in step S3.2, the rotation angle θ is calculated z Specifically:

[0024] (1) Calculate the x-direction image point displacement of the two corner points of the reference target where point N is located according to the x-direction components of the image point coordinates of the two corner points;

[0025] (2) Establish a z-axis rotational motion model; define points N, M, and P to rotate around the z-axis by θ z The points after rotation are N′, M′, and P′, and the corresponding image points before and after rotation are n(x n ,y n )、m(x m ,y m )、p(x p ,y p ) and n′(x′ n ,y′ n )、m′(x′ m ,y′ m ),p′(x′ p ,y′ p ); in the image plane C(x c ,y c ) is the centre of rotation;

[0026] When points N, M, and P rotate counterclockwise around the z axis by θ z When , the image point coordinates of N′, M′, and P′ are:

[0027]

[0028] Then the image point displacement of points N, M, and P after rotation is:

[0029]

[0030] Perform differential processing on the image point displacement:

[0031]

[0032] Similarly, when points N, M, and P rotate clockwise about the z axis by θ z When , the result of image point displacement difference processing is:

[0033]

[0034] (3) Calculate the rotation angle θ z Since the reference target where point N is located is placed vertically, the x coordinates of the two corner points n1 and n2 on the target are equal, and due to the z-axis rotation angle θ z For a small angle, then (cosθ z -1)< <sinθ z , simplifying Equation 4) to obtain Equation 8):

[0035]

[0036] Where, and is the motion error component in the x-direction of the z-axis rotational motion error of the two corner points on the reference target; and The y-direction components of the coordinates of the two corner image points on the reference target; Solve the problem by differentially shifting the image points in the x direction of the two corner points obtained in (1);

[0037] According to formula 8) solve the z-axis rotation angle θ z .

[0038] Furthermore, in step S3.3, the z-axis rotational motion error between point M on the reference target and point P on the monitoring target is calculated as follows:

[0039]

[0040] The z-axis rotational motion errors of points M and P relative to point N are obtained by solving equations 9 and 10.

[0041] Furthermore, in step S4, the z-axis translation motion error of the monitoring target is calculated based on the unit pixel displacement change, specifically:

[0042] Establish a z-axis translation motion model;

[0043] Calculate the displacement change per unit pixel, and solve the z-axis translation motion error of point P based on the displacement change per unit pixel and the distance from the image points of points N, M, and P to the midpoint of the image.

[0044] Furthermore, the z-axis translation error of point P is solved as follows:

[0045] The camera moves along the z-axis. Taking the y-coordinate change of point N as an example, the distance between point N and the z-axis is N. y , the point after point N is translated is N′, o is the center of the image, f is the focal length, and the corresponding image points of point N before and after translation are n(xn ,y n ) and n′(x′ n ,y′ n ), then the coordinate change error of point N is for:

[0046]

[0047] Unit pixel displacement change for:

[0048]

[0049] By changing equation 12) to equation 13)

[0050]

[0051] is the scale factor of image point n, is the scale factor of the image point on′;

[0052] According to formula 13), the calculation formula for the z-axis translation motion error of point N is:

[0053]

[0054] Similarly, the motion error formula for the z-axis translation motion error of points N, M, and P in the y direction is:

[0055]

[0056] The motion error formula for the z-axis translation motion error of points N, M, and P in the x-direction is:

[0057]

[0058] In the above formula, |on| y ,、|om| y 、|op| y is the distance between the y-coordinate of image points n, m, and p and H / 2, |on| x 、|om| x 、|op| x is the distance between the x-coordinate of image points n, m, and p and W / 2, H represents the image height, and W represents the image width; are the x-direction motion error components of the z-axis translation motion errors of points N, M, and P, respectively; are the y-direction motion error components of the z-axis translation motion errors of points N, M, and P respectively.

[0059] Furthermore, step S5 specifically involves solving the x-axis rotation and translation motion errors and the y-axis rotation and translation motion errors of the point P on the monitoring target in combination with the two reference targets.

[0060] Furthermore, step S6 specifically subtracts the rotational and translational motion errors of the x, y and z axes of point P from the actual displacement of point P to obtain the actual displacement result of point P, that is, the actual displacement result of the monitoring target.

[0061] The application of the technical solution of the present invention has the following beneficial effects:

[0062] (1) The present invention provides an airborne camera motion correction method for unmanned aerial vehicle (UAV) visual displacement monitoring, specifically comprising the following steps: installing a monitoring target and a reference target on the monitoring target, collecting UAV visual image data through the UAV airborne camera; calculating the image point coordinates of two corner points on all targets in the UAV visual image data, the image point displacement of the midpoint of the two corner points, and the scale factor of the target; calculating the z-axis rotational motion error of the monitoring target based on the vertically placed reference target; calculating the z-axis translational motion error of the monitoring target based on the unit pixel displacement change; calculating the x-axis rotational and translational motion errors and the y-axis rotational and translational motion errors of the monitoring target based on the two reference targets; correcting the rotational and translational motion errors of the z-axis, x-axis, and y-axis to obtain the actual displacement result of the monitoring target. The method provided by the present invention does not need to rely on the rotation center when correcting the camera z-axis rotational motion, and is applicable to the actual situation where the rotation center deviates from the image center. The applicability of the method is improved while effectively improving the accuracy of UAV visual displacement monitoring.

[0063] (2) The present invention provides detailed steps for the camera motion correction method to improve the operability of the method; in addition, the method of the present invention can use a single reference target to calculate the z-axis rotational motion error, thereby improving the efficiency of camera motion error correction.

[0064] (3) The method provided by the present invention calculates the z-axis translation motion error based on the unit pixel displacement change. Since the scale changes at different points are the same (the numerator of the unit pixel displacement change calculation formula), the scale change can be calculated based on the reference target closest to the camera to improve the accuracy of the unit pixel displacement change calculation, thereby improving the correction accuracy of the z-axis translation motion error.

[0065] In addition to the above-described objects, features and advantages, the present invention has other objects, features and advantages. The present invention will be further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The accompanying drawings, which constitute part of the present invention, are provided to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are provided to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0067] Figure 1 It is a flow chart of the method for correcting the motion of an onboard camera in visual displacement monitoring of a UAV in the present invention;

[0068] Figure 2 is a measurement model in an embodiment of the present invention;

[0069] Figure 3 is the rotation and translation movement direction of the onboard camera of the embodiment of the present invention;

[0070] Figure 4 is a schematic diagram of the rotational motion of an airborne camera around the z-axis according to an embodiment of the present invention;

[0071] Figure 5 is a schematic diagram of the translational motion of an airborne camera along the z-axis according to an embodiment of the present invention;

[0072] Figure 6 This is a schematic diagram of the UAV visual displacement monitoring experimental scene;

[0073] Figure 7 It is the on-site layout diagram of the target, translation slide and fixed camera;

[0074] Figure 8 It is a picture taken by the drone's onboard camera;

[0075] Figure 9 It is the T2 horizontal (x-direction) displacement curve of the monitoring target before and after the motion error correction of the UAV airborne camera;

[0076] Figure 10 is the vertical (y-direction) displacement curve of the monitoring target T2 before and after the motion error correction of the UAV airborne camera;

[0077] Figure 11 It is the horizontal (x-direction) displacement curve of the monitoring target T3 before and after the motion error correction of the UAV airborne camera;

[0078] Figure 12 is the vertical (y-direction) displacement curve of the monitoring target T3 before and after the motion error correction of the UAV airborne camera;

[0079] Figure 13 It is the horizontal (x-direction) displacement curve of the monitoring target T4 before and after the motion error correction of the UAV airborne camera;

[0080] Figure 14 is the vertical (y-direction) displacement curve of the monitoring target T4 before and after the motion error correction of the UAV airborne camera;

[0081] Figure 15 This is a comparison chart of the displacement curves of the monitoring target T2 measured by the drone-mounted camera and the fixed camera. DETAILED DESCRIPTION

[0082] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, the present invention can be implemented in many different ways as defined and covered.

[0083] In the description of the present invention, it should be noted that the terms "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", "front", "back", "lateral", "longitudinal", etc., indicating directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, they should not be understood as limiting the present invention.

[0084] Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature identified with "first," "second," etc., may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0085] Example:

[0086] See also Figure 1 This embodiment provides a method for correcting the motion of an onboard camera in visual displacement monitoring of a UAV, comprising the following steps:

[0087] Step S1, see Figure 2 , installing a monitoring target and a reference target on the monitoring target, with one monitoring target and two reference targets, at least one of which is vertically arranged. In this embodiment, preferably, the reference target closer to the camera is vertically arranged; collecting drone visual image data through the drone's onboard camera;

[0088] Step S2: Calculate the image point coordinates of two corner points on all targets in the UAV visual image data, the image point displacement of the midpoint between the two corner points, and the scale factor of the target; specifically:

[0089] Step S2.1, obtain the image coordinates of two corner points on the monitoring target and the reference target according to the corner point detection and positioning method in Reference 1;

[0090] Step S2.2, based on the image point coordinates of the two corner points, calculate the pixel distance between the two corner points of the same target according to the distance formula; based on the image point coordinates of the two corner points, obtain the midpoint coordinates of the target corner point;

[0091] Step S2.3: Calculate the scale factor SF of each target using the following formula:

[0092]

[0093] In the above formula, d pixel is the pixel distance between two corner points of the same target, in pixels; D is the actual distance between two corner points of the same target (obtained by measuring the distance between the two corner points), in mm;

[0094] Step S2.4: Perform the calculations of steps S2.1 to S2.3 on each image in a set of image sequences. The image point displacement can be obtained by subtracting the midpoint coordinates of the two corner points of the target on each image in the image sequence from the midpoint coordinates of the two corner points of the target corresponding to the first image.

[0095] Step S3: Calculate the z-axis rotational motion error of the monitoring target based on the vertically placed reference target. Specifically:

[0096] Step S3.1, the onboard camera motion includes rotation and translation, which occur in the x, y and z directions respectively, as shown in Figure 3 The z-axis rotation motion error of the monitoring target caused by the camera rotating around the z-axis includes the motion error component in the x-direction and the motion error component in the y-direction; are the motion error components in the x-direction of the z-axis rotational motion errors of points N, M, and P, respectively. are the y-direction motion error components of the z-axis rotational motion errors of points N, M, and P, respectively, where point N is the midpoint of a vertically placed reference target corner, point M is the midpoint of another reference target corner, and P is the midpoint of a monitoring target corner;

[0097] Step S3.2: Calculate the rotation angle θ z Specifically:

[0098] (1) Calculate the x-direction image point displacement of the two corner points of the reference target where point N is located according to the x-direction components of the image point coordinates of the two corner points;

[0099] (2) Establish a z-axis rotational motion model, see Figure 4 ; Define points N, M, P to rotate around the z axis by θ z The points after rotation are N′, M′, and P′, and the corresponding image points before and after rotation are n(x n ,y n )、m(x m ,y m )、p(x p ,y p ) and n′(x′ n,y′ n )、m′(x′ m ,y′ m ),p′(x′ p ,y′ p ); into the image plane C(x c ,y c ) is the centre of rotation;

[0100] When points N, M, and P rotate counterclockwise around the z axis by θ z When , the image point coordinates of N′, M′, and P′ are:

[0101]

[0102] Then the image point displacement of points N, M, and P after rotation is:

[0103]

[0104] Perform differential processing on the image point displacement:

[0105]

[0106] Similarly, when points N, M, and P rotate clockwise about the z axis by θ z When , the result of image point displacement difference processing is:

[0107]

[0108] According to Equations 4) to 7), the multi-point displacement differential caused by the rotation of points N, M, and P around the z-axis is independent of the rotation center. That is, this method is applicable to z-axis rotation motion with the rotation center at any position in the image plane.

[0109] (3) Calculate the rotation angle θ z Since the reference target where point N is located is placed vertically, the x coordinates of the two corner points n1 and n2 on the target are equal, and due to the z-axis rotation angle θ z For a small angle (less than 0.1°), then (cosθ z -1)< <sinθ z , simplifying Equation 4) to obtain Equation 8):

[0110]

[0111] Where, and is the motion error component in the x-direction of the z-axis rotational motion error of the two corner points on the reference target; and are the y-direction components of the coordinates of the two corner image points on the reference target.

[0112] The solution is obtained by the difference of the image point displacement in the x direction of the two corner points obtained in (1): Since the two corner points on the reference target are located on the same target, the rotation and translation motion error components in the x and y directions, as well as the z-axis translation motion error component (the scale factor of the same target is the same), that is, the difference of the image point displacement in the x direction of the two corner points is the difference of the motion error component in the x direction of the z-axis rotation motion error of the two corner points.

[0113] According to formula 8) solve the z-axis rotation angle θ z .

[0114] Step S3.3, according to the rotation angle θ z , calculate the z-axis rotational motion error between point M on the reference target and point P on the monitoring target. The calculation formula is as follows:

[0115]

[0116] The z-axis rotational motion errors of points M and P relative to point N are obtained according to equations 9) and 10).

[0117] Step S4: Calculate the z-axis translation motion error of point P on the monitoring target based on the unit pixel displacement change; specifically:

[0118] To build a z-axis translation motion model, see Figure 5 Calculate the change in unit pixel displacement and determine the z-axis translation error for point P based on the change in unit pixel displacement and the distances from the image points of points N, M, and P to the image midpoint. Camera translation along the z-axis manifests as image point displacement in both the x- and y-axis directions on the image plane. The z-axis translation error caused by camera translation along the z-axis includes both x- and y-axis motion error components.

[0119] The camera translates along the z-axis, taking the y-coordinate change of point N as an example, Figure 5 As shown, let the camera translation along the z axis be Δ s , the distance between point N and the z-axis is N y , the point after point N is translated is N′, o is the center of the image, f is the focal length, and the corresponding image points of point N before and after translation are n(x n ,y n ) and n′(x′ n ,y′ n ), then the coordinate change error of point N is for:

[0120]

[0121] Unit pixel displacement change for:

[0122]

[0123] By changing equation 12) to equation 13)

[0124]

[0125] is the scale factor of image point n, is the scale factor of the image point on′; is the scale change, according to the formula Calculated, the z-axis translation Δ s And the focal length f is independent of the position of the measurement point, so it can be concluded that the scale changes at different points are consistent. To improve the accuracy of calculations, the nearest reference target (the one closest to the camera) at point N is selected to calculate the scale change. The unit pixel displacement change at each point is then calculated based on the scale factor at each point. This method can effectively improve the accuracy of z-axis translation error correction, ensuring accuracy in practical applications.

[0126] According to formula 13), the calculation formula for the z-axis translation motion error of point N is:

[0127]

[0128] Similarly, the motion error formula for the z-axis translation motion error of points N, M, and P in the y direction is:

[0129]

[0130] The motion error formula for the z-axis translation motion error of points N, M, and P in the x-direction is:

[0131]

[0132] In the above formula, |on| y ,、|om| y 、|op| y is the distance between the y-coordinate of image points n, m, and p and H / 2, |on| x 、|om| x 、|op| x is the distance between the x-coordinate of image points n, m, and p and W / 2, H represents the image height, and W represents the image width; are the x-direction motion error components of the z-axis translation motion errors of points N, M, and P, respectively; are the y-direction motion error components of the z-axis translation motion errors of points N, M, and P respectively.

[0133] Step S5: Calculate the x-axis rotation and translation motion errors and the y-axis rotation and translation motion errors of the monitoring target based on the two reference targets;

[0134] Specifically, according to the method in Reference 2, the shift angle in the method is set to 0° (in this case, the method can be used to correct the rotation and translation motion errors of the x-axis and y-axis of a conventional camera), and the rotation and translation motion errors of the x-axis and the y-axis of the point P on the monitoring target are solved in combination with two reference targets.

[0135] Step S6: Subtract the rotational and translational motion errors of point P on the monitoring target about the x, y and z axes from the actual displacement of point P to obtain the actual displacement result of point P, that is, the actual displacement result of the monitoring target.

[0136] Reference 1: LIU J, DAI W, ZHANGY, et al. AnAdaptive Radon-Transform-BasedMarker Detection and Localization Method for Displacement Measurements Using Unmanned Aerial Vehicles[J]. Sensors, 2024, 24(6):1930.

[0137] Reference 2: XING L, DAI W, ZHANG Y. Scheimpflug Camera-Based Technique for Multi-Point Displacement Monitoring of Bridges[J]. Sensors, 2022, 22(11):4093.

[0138] The UAV visual displacement monitoring experiment was carried out. The experimental equipment is shown in Table 1.

[0139] Table 1 UAV visual displacement monitoring experimental equipment

[0140]

[0141] Experimental scenario such as Figure 6 As shown in the figure, reference targets, numbered T1 and T5, are placed at the front and rear ends of the measurement area, 28 meters apart. T1 is 15 meters from the airborne camera, and T5 is 43 meters away. Between the two reference targets, three monitoring targets, numbered T2, T3, and T4, are fixed at 7-meter intervals. A translation slide is installed at the bottom of target T2 to simulate vertical displacement, and a fixed camera is placed 3 meters in front of it to measure the true displacement of the slide.

[0142] Figure 7 This is the on-site layout diagram of the target, translation slide and fixed camera. Figure 8 Images captured by the drone's onboard camera. Both the onboard and fixed cameras had a sampling rate of 100 frames per second, and the experiment lasted 20 seconds. Due to significant vibration of the drone system during the experiment, some targets were lost from the camera's field of view. A total of 1,695 valid images were captured. Figures 9-14 These are the displacement curves of monitoring targets T2, T3, and T4 before and after correction using the method of the present invention.

[0143] To further verify the measurement accuracy of the method, the corrected T2 target displacement monitoring results were compared with the reference data obtained by the fixed camera, and a total of 1500 images were matched. Figure 15 The displacement of the monitoring target T2 measured by the airborne camera was compared with that measured by a fixed camera. The comparison results show that the corrected displacement curve is highly consistent with the monitoring results of the fixed camera, which directly verifies the measurement accuracy of this method.

[0144] In order to quantitatively evaluate the measurement error, the root mean square error (RMSE) statistical analysis was performed on the above results. The specific statistical results are shown in Table 2.

[0145] Table 2 RMSE statistics of UAV visual displacement monitoring experimental results (mm)

[0146]

[0147] Table 2 shows that the root mean square error (RMSE) of the displacement measurements of the monitoring targets in the vertical (y-axis) and horizontal (x-axis) directions significantly decreased after correction using the proposed method. Taking the fixed camera's measurement results as the true value, the RMSE of the airborne camera's measurement results for target T2 after error correction was only 0.11 mm. After correcting for kinematic errors, the RMSE of the measurement results for all monitoring targets was less than 0.3 mm, demonstrating that the correction method effectively improved measurement accuracy.

[0148] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A method for correcting the motion of an onboard camera in UAV visual displacement monitoring, characterized in that: The following steps are involved: Step S1: Install a monitoring target and two reference targets on the monitoring target, with at least one reference target being placed vertically; and collect UAV visual image data using an onboard camera of the UAV; Step S2, calculating the image point coordinates of two corner points on all targets in the UAV visual image data, the image point displacement of the midpoint between the two corner points, and the scale factor of the target; Step S3: Calculate the z-axis rotational motion error of the monitoring target based on the vertically placed reference target. Specifically: Step S3.

1. Definition are the motion error components in the x-direction of the z-axis rotational motion errors of points N, M, and P, respectively. are the y-direction motion error components of the z-axis rotational motion errors of points N, M, and P, respectively, where point N is the midpoint of a vertically placed reference target corner, point M is the midpoint of another reference target corner, and P is the midpoint of a monitoring target corner; Step S3.2: Calculate the rotation angle θ z ; Step S3.3, according to the rotation angle θ z , calculate the z-axis rotational motion error of point P on the monitoring target; Step S4, calculating the z-axis translation motion error of point P on the monitoring target based on the unit pixel displacement change; Step S5: Calculate the x-axis rotation and translation motion errors and the y-axis rotation and translation motion errors of the monitoring target based on the two reference targets; Step S6: Correct the rotation and translation errors of the z-axis, x-axis, and y-axis to obtain the actual displacement result of the monitoring target.

2. The method for correcting the motion of an onboard camera in UAV visual displacement monitoring according to claim 1, characterized in that: Step S2 specifically includes: Step S2.1, obtaining the image coordinates of two corner points on the monitoring target and the reference target according to the corner point detection and positioning method; Step S2.2, based on the image point coordinates of the two corner points, calculate the pixel distance between the two corner points of the same target according to the distance formula; based on the image point coordinates of the two corner points, obtain the midpoint coordinates of the target corner point; Step S2.3: Calculate the scale factor SF of each target using the following formula: In the above formula, d pixel is the pixel distance between two corner points of the same target, and D is the actual distance between two corner points of the same target; Step S2.4: Perform the calculations of steps S2.1 to S2.3 on each image in a set of image sequences. The image point displacement can be obtained by subtracting the midpoint coordinates of the two corner points of the target on each image in the image sequence from the midpoint coordinates of the two corner points of the target corresponding to the first image.

3. The method for correcting the motion of an onboard camera in UAV visual displacement monitoring according to claim 2, characterized in that: In step S3.2, the rotation angle θ is calculated z Specifically: (1) Calculate the x-direction image point displacement of the two corner points of the reference target where point N is located according to the x-direction components of the image point coordinates of the two corner points; (2) Establish a z-axis rotational motion model; define points N, M, and P to rotate around the z-axis by θ z The points after rotation are N′, M′, and P′, and the corresponding image points before and after rotation are n(x n ,y n )、m(x m ,y m )、p(x p ,y p ) and n′(x′ n ,y′ n )、m′(x′ m ,y′ m ),p′(x′ p ,y′ p ); in the image plane C(x c ,y c ) is the centre of rotation; When points N, M, and P rotate counterclockwise around the z axis by θ z When , the image point coordinates of N′, M′, and P′ are: Then the image point displacement after the rotation of points N, M, and P is: Perform differential processing on the image point displacement: Similarly, when points N, M, and P rotate clockwise about the z axis by θ z When , the result of image point displacement difference processing is: (3) Calculate the rotation angle θ z Since the reference target where point N is located is placed vertically, the x coordinates of the two corner points n1 and n2 on the target are equal, and due to the z-axis rotation angle θ z For a small angle, then (cosθ z -1)< <sinθ z , simplifying Equation 4) to obtain Equation 8): Where, and is the motion error component in the x-direction of the z-axis rotational motion error of the two corner points on the reference target; and The y-direction components of the coordinates of the two corner image points on the reference target; Solve the problem by differentially shifting the image points in the x direction of the two corner points obtained in (1); According to formula 8) solve the z-axis rotation angle θ z .

4. The method for correcting camera motion in UAV visual displacement monitoring according to claim 3, characterized in that: In step S3.3, the z-axis rotational motion error between point M on the reference target and point P on the monitoring target is calculated as follows: The z-axis rotational motion errors of points M and P relative to point N are obtained by solving equations 9 and 10.

5. The method for correcting the motion of an onboard camera in visual displacement monitoring of an unmanned aerial vehicle according to claim 2, wherein: In step S4, the z-axis translation motion error of the monitoring target is calculated based on the unit pixel displacement change, specifically: Establish a z-axis translation motion model; Calculate the displacement change per unit pixel, and solve the z-axis translation motion error of point P based on the displacement change per unit pixel and the distance from the image points of points N, M, and P to the midpoint of the image.

6. The method for correcting camera motion in UAV visual displacement monitoring according to claim 5, characterized in that: The specific solution for the z-axis translation motion error of point P is: The camera moves along the z-axis. Taking the y-coordinate change of point N as an example, the distance between point N and the z-axis is N. y , the point after point N is translated is N′, o is the center of the image, f is the focal length, and the corresponding image points of point N before and after translation are n(x n ,y n ) and n′(x′ n ,y′ n ), then the coordinate change error of point N is for: Unit pixel displacement change for: By changing equation 12) to equation 13) is the scale factor of image point n, is the scale factor of the image point on′; According to formula 13), the calculation formula for the z-axis translation motion error of point N is: Similarly, the motion error formula for the z-axis translation motion error of points N, M, and P in the y direction is: The motion error formula for the z-axis translation motion error of points N, M, and P in the x-direction is: In the above formula, |on| y ,、|om| y 、|op| y is the distance between the y-coordinate of image points n, m, and p and H / 2, |on| x 、|om| x 、|op| x is the distance between the x-coordinate of image points n, m, and p and W / 2, H represents the image height, and W represents the image width; are the y-direction motion error components of the z-axis translation motion errors of points N, M, and P, respectively; are the x-direction motion error components of the z-axis translation motion errors of points N, M, and P, respectively.

7. The method for correcting camera motion in UAV visual displacement monitoring according to claim 1, characterized in that: Specifically, step S5 is to solve the x-axis rotation and translation motion errors and the y-axis rotation and translation motion errors of the point P on the monitoring target in combination with the two reference targets.

8. The method for correcting camera motion in UAV visual displacement monitoring according to claim 1, characterized in that: Specifically, step S6 subtracts the rotational and translational motion errors of the x, y, and z axes of point P from the actual displacement of point P to obtain the actual displacement result of point P, that is, the actual displacement result of the monitoring target.

Images