Image distortion correction method for multi-pose shooting of UAV

Abstract

The present invention relates to image distortion correction technology and provides an image distortion correction method for multi-pose photography using unmanned aerial vehicles (UAVs), comprising the following steps: calibrating camera intrinsic parameter coefficients to obtain a camera intrinsic parameter matrix in a camera coordinate system; calculating a rotation matrix from the camera coordinate system to a carrier coordinate system using an inertial sensor (IMU); constructing a pinhole camera imaging model to project points in the camera coordinate system to an image coordinate system; calculating a rotation matrix from the carrier coordinate system to a geographic coordinate system using the inertial sensor (IMU); and obtaining a transformation formula and a scaling factor based on the relationship between perspective projection and weak perspective projection to perform image distortion correction. Based on the coordinate transformation relationship and the weak perspective transformation relationship, the present invention achieves scale and rotation consistency between an aerial image and a reference image, improves the accuracy of image geometric distortion correction, and effectively improves the accuracy and reliability of scene matching navigation systems in practical applications.

Description

Technical Field

[0001] The present invention belongs to the technical field of image distortion correction, and in particular to an image distortion correction method for multi-posture shooting of unmanned aerial vehicles. Background Art

[0002] High-precision navigation and positioning are fundamental to autonomous drone control. In GNSS-denied environments, positioning errors from inertial navigation systems accumulate over time. To overcome this problem, scene matching navigation uses image sensors to capture images of the flight or target area and matches them against stored reference images to obtain the aircraft's position data. Unlike traditional navigation systems, scene matching navigation does not rely on external information, boasts a simple device structure, and offers high positioning accuracy. This enables more reliable autonomous drone control and navigation in complex or GNSS-denied environments.

[0003] The core of scene matching navigation lies in matching aerial images with reference images. However, during UAV flight, factors such as flight altitude and attitude angle cause the camera's imaging perspective to constantly change with the ground, leading to differences in the spatial resolution of image points. This causes severe trapezoidal distortion in the target image, reduces the success rate of image matching, and results in low scene matching accuracy. Therefore, to improve the accuracy and reliability of scene matching navigation systems in practical applications, it is urgent and necessary to find an image distortion correction method for multi-pose UAV photography, perform comprehensive geometric correction on real-time aerial images, and optimize and improve the matching algorithm. Summary of the Invention

[0004] In response to the above-mentioned deficiencies in the prior art, the present invention proposes a method for image distortion correction for multi-pose photography using unmanned aerial vehicles. The method comprises calibrating the camera's intrinsic parameter coefficients to obtain the camera's intrinsic parameter matrix in the camera coordinate system; calculating the rotation matrix from the camera coordinate system to the carrier coordinate system using an inertial sensor (IMU); constructing a pinhole camera imaging model to project points in the camera coordinate system to the image coordinate system; calculating the rotation matrix from the carrier coordinate system to the geographic coordinate system using the inertial sensor (IMU); and obtaining a transformation formula and scaling coefficient based on the relationship between perspective projection and weak perspective projection to perform image distortion correction. Based on the coordinate transformation relationship and the weak perspective transformation relationship, the present invention achieves scale and rotation consistency between the aerial image and the reference image, thereby improving the accuracy of image geometric distortion correction and effectively enhancing the accuracy and reliability of the scene matching navigation system in practical applications.

[0005] The present invention provides an image distortion correction method for multi-posture shooting of an unmanned aerial vehicle, which comprises the following steps:

[0006] S1. Obtain camera intrinsic parameter matrix: Calibrate the camera intrinsic parameter coefficients based on Zhang Zhengyou method to obtain the camera intrinsic parameter matrix in the camera coordinate system; the camera coordinate system is based on the camera optical center O c is the origin, O c x c Axis, O c y c Axis and O c z c The axes are the first, second and third axes that are perpendicular to each other and conform to the right-hand rule;

[0007] S2. Calculate the rotation matrix from the camera coordinate system to the carrier coordinate system using the inertial sensor IMU The carrier coordinate system is centered at the camera support O b is the origin, O b x b Axis, O b y b Axis and O b z b The axes are the first, second and third axes that are perpendicular to each other and conform to the right-hand rule;

[0008] S3, constructing a pinhole camera imaging model and projecting points in the camera coordinate system to the image coordinate system;

[0009] S4. Calculate the rotation matrix from the carrier coordinate system to the geographic coordinate system using the inertial sensor IMU The geographic coordinate system is based on g is the origin, O g x g Axis, O g y g Axis and O g z g The axes are the first, second and third axes that are perpendicular to each other and conform to the right-hand rule;

[0010] S5. Perform image distortion correction based on the relationship between perspective projection and weak perspective projection;

[0011] S51, by means of perspective projection, obtain the perspective projection point P of the target point P on the ground I0 on the imaging plane I i ' and the approximate perspective projection point P' on the target approximate plane I'; the perspective projection point P i 'The pixel coordinates in the image coordinate system are (u' i ,v' i );

[0012] S52, using weak perspective projection, obtain the weak perspective projection point P of the target point P on the ground I0 on the imaging plane I i" and the approximate weak perspective projection point P on the target approximate plane I'; the weak perspective projection point P i "The pixel coordinates in the image coordinate system are (u" i ,v” i );

[0013] S53, according to the perspective projection point P i ' and weak perspective projection point P i ", the perspective projection point P i The pixel coordinates of ' are transformed into the weak perspective projection point P by using the transformation formula i "The pixel coordinates of the image are used to complete the image distortion correction. The transformation formula is:

[0014]

[0015] Among them, u0 and v0 represent the first axis offset value and the second axis offset value of the pixel center respectively; k represents the perspective projection point P i 'Transform to weak perspective projection point P i "The scaling factor satisfies:

[0016]

[0017] in, Represents the origin F of the imaging plane I to the perspective projection point P i 'direction vector; Represents the origin F of the imaging plane I to the weak perspective projection point P i ” direction vector.

[0018] Furthermore, the calculation of the conversion formula and the scaling factor k in step S53 specifically includes the following steps:

[0019] S531. Calculate the direction vector of the origin T of the target approximate plane I′ in the camera coordinate system

[0020] S532, calculate the direction vector from the origin T of the target approximate plane I' to the approximate perspective projection point P'

[0021] S533, calculate the camera optical center O c Direction vector of the approximate perspective projection point P' to the target approximate plane I' for:

[0022]

[0023] Among them, e c Represents along the camera coordinate system O c z c The unit vector in the direction of the axis and ec =[0 0 1] T ; H represents the height from the optical center of the camera to the ground I0; f represents the focal length of the camera; subscript 3 represents the third element;

[0024] S534. According to the scaling factor expression (6), k satisfies:

[0025]

[0026] in, Indicates the camera optical center O c Direction vector to the target point P; Represents the direction vector from the origin T of the target approximate plane I′ to the approximate weak perspective projection point P”; |||| represents the length of the obtained vector;

[0027] Then it satisfies:

[0028]

[0029] S535. Based on the conversion relationship among the carrier coordinate system, the geographic coordinate system, and the camera coordinate system, obtain the representation of the scaling factor k:

[0030]

[0031] S536, calculating the transformation formula and the scaling factor k;

[0032] S5361. According to the conversion relationship between the imaging plane coordinate system and the image coordinate system, the perspective projection point P i ' and weak perspective projection point P i "satisfy:

[0033]

[0034] Wherein, dx represents the length of a single pixel on the first axis of the image coordinate system corresponding to the first axis of the imaging plane coordinate system; dy represents the length of a single pixel on the second axis of the image coordinate system corresponding to the second axis of the imaging plane coordinate system; and Represent the perspective projection point P i 'Coordinate values ​​on the first and second axes of the imaging plane coordinate system; and Represents the weak perspective projection point P i ”Coordinate values ​​on the first and second axes of the imaging plane coordinate system;

[0035] S5362, in the imaging plane coordinate system, Represented as perspective projection point P i ''s coordinates, i.e. Represented as weak perspective projection point P i "The coordinates of Based on formula (6), the scaling factor k is expressed as:

[0036]

[0037] S5363. Based on formula (22) and formula (23), obtain the transformation formula:

[0038]

[0039] S5364. Based on formula (21) and formula (22), obtain the scaling factor k:

[0040]

[0041] Preferably, step S531 specifically includes the following steps:

[0042] S5311. In the camera coordinate system, Expressed as:

[0043]

[0044] Where λ is a constant;

[0045] S5312, rotation matrix from carrier coordinate system to geographic coordinate system And the rotation matrix from the camera coordinate system to the carrier coordinate system The representation in the geographic coordinate system is:

[0046]

[0047] in, Represents the rotation matrix from the camera coordinate system to the geographic coordinate system;

[0048] S5313. Combining formula (7) and formula (8), we get:

[0049]

[0050] S5314, based on Projected to O in geographic coordinate system g Z g The coordinate value of the axis, that is, the height H from the optical center of the camera to the ground I0, is obtained:

[0051]

[0052] The constant λ is obtained as:

[0053]

[0054] S5315, combined with formula (7) and formula (11), we get for:

[0055]

[0056] Step S532 specifically includes the following steps:

[0057] S5321. According to the perspective projection relationship, Expressed as:

[0058]

[0059] Where D represents the optical center of the camera O c The distance to the origin T of the target approximation plane I′;

[0060] S5322, combined with formula (12) and formula (13), we get for:

[0061]

[0062] Step S535 specifically includes the following steps:

[0063] S5351, Rotation matrix from carrier coordinate system to geographic coordinate system And the rotation matrix from the camera coordinate system to the carrier coordinate system The representation in the geographic coordinate system is:

[0064]

[0065] S5352, based on Projected to O in geographic coordinate system g Z g The coordinate value of the axis, that is, the height H from the optical center of the camera to the ground I0, is obtained:

[0066]

[0067] Then k satisfies:

[0068]

[0069] S5353, combining equations (15) and (20), obtains the representation of the scaling factor k.

[0070] Preferably, step S21 specifically includes the following steps:

[0071] S21, installing an inertial sensor IMU on the camera bracket and the camera respectively, and measuring the camera installation error angle; the camera installation error angle includes the camera installation error yaw angle γ', the camera installation error pitch angle α' and the camera installation error roll angle β';

[0072] S22. Calculate the rotation matrix from the camera coordinate system to the carrier coordinate system based on the camera installation error angle

[0073]

[0074] Here, cos and sin represent cosine and sine functions respectively.

[0075] Step S41 specifically includes the following steps:

[0076] S41, using the inertial sensor IMU on the camera bracket to output the carrier attitude angle; the carrier attitude angle includes the carrier attitude yaw angle γ, the carrier attitude pitch angle α and the carrier attitude roll angle β;

[0077] S42. Calculate the rotation matrix from the carrier coordinate system to the geographic coordinate system based on the carrier attitude angle

[0078]

[0079] Preferably, the camera intrinsic parameter matrix K in step S1 is:

[0080]

[0081] Among them, f x Indicates the focal length f of the camera at O c x c The projection of the axis; f y Indicates the focal length f of the camera at O c y c Projection of axis; c x Indicates O c x c The pixel offset corresponding to the axis; c y Indicates O c y c The pixel offset corresponding to the axis.

[0082] Preferably, the pinhole camera imaging model in step S3 is:

[0083]

[0084] Among them, x, y and z represent the point in the camera coordinate system at O c x c Axis, O c yc Axis and O c z c Coordinate values ​​on the axis; u and v represent the pixel coordinates of the point in the camera coordinate system on the first axis and the second axis of the image coordinate system, respectively.

[0085] Preferably, the ground I0 in step S51 is x in the geographic coordinate system. g O g y g Plane; the target approximate plane I' is the origin O of the camera coordinate system c The point T where the extended line of the line connecting the origin F of the imaging plane I intersects with the ground I0 is the origin, and the plane is parallel to the imaging plane I.

[0086] Compared with the prior art, the technical effects of the present invention are:

[0087] 1. The present invention designs an image distortion correction method for multi-pose photography using drones. Based on the transformation relationship between the geographic coordinate system, the carrier coordinate system, the camera coordinate system, and the image coordinate system, and the transformation relationship of the weak perspective transformation, the method corrects the spatial resolution in real time, solves the serious problem of image geometric deformation caused by different depth information at each point when the drone takes tilted photos of the ground, and achieves scale and rotation consistency between the aerial image and the reference image.

[0088] 2. The present invention designs an image distortion correction method for multi-posture shooting of unmanned aerial vehicles, which uses an inertial sensor IMU to perform three-dimensional calibration of the camera installation error and the carrier posture, obtains the camera installation error angle and the carrier posture angle, reduces the influence of the initial camera installation error and the initial carrier posture on the image distortion, and improves the accuracy of image geometric distortion correction. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] Other features, objects and advantages of the present application will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings.

[0090] Figure 1 This is a flow chart of the image distortion correction method for multi-pose shooting of unmanned aerial vehicles of the present invention;

[0091] Figure 2 is a schematic diagram of the weak perspective transformation of the present invention;

[0092] Figure 3 This is a diagram showing the distortion angle error results of the sampling area before and after the image distortion correction of the present invention. DETAILED DESCRIPTION

[0093] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0094] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0095] Figure 1 The present invention shows an image distortion correction method for multi-pose shooting of a drone, which includes the following steps:

[0096] S1. Obtain camera intrinsic parameter matrix: Calibrate the camera intrinsic parameter coefficient based on Zhang Zhengyou method to obtain the camera intrinsic parameter matrix in the camera coordinate system; the camera coordinate system is based on the camera optical center O c is the origin, O c x c Axis, O c y c Axis and O c z c The axes are the first, second, and third axes that are perpendicular to each other and conform to the right-hand rule.

[0097] The camera intrinsic parameter matrix K is:

[0098]

[0099] Among them, f x Indicates the focal length f of the camera at O c x c The projection of the axis; f y Indicates the focal length f of the camera at O c y c Projection of axis; c x Indicates O c x c The pixel offset corresponding to the axis; c y Indicates O c y c The pixel offset corresponding to the axis.

[0100] S2. Calculate the rotation matrix from the camera coordinate system to the carrier coordinate system using the inertial sensor IMU The carrier coordinate system is centered on the camera bracket O b is the origin, O b x b Axis, O b y b Axis and O b z bThe axes are the first, second, and third axes that are perpendicular to each other and conform to the right-hand rule.

[0101] S21. Install inertial sensors (IMUs) on the camera bracket and the camera, respectively, to measure the camera installation error angles. The camera installation error angles include the camera installation error yaw angle γ', the camera installation error pitch angle α', and the camera installation error roll angle β'.

[0102] S22. Calculate the rotation matrix from the camera coordinate system to the carrier coordinate system based on the camera installation error angle

[0103]

[0104] Here, cos and sin represent cosine and sine functions respectively.

[0105] S3, build a pinhole camera imaging model, project the points in the camera coordinate system to the image coordinate system,

[0106] The pinhole camera imaging model is:

[0107]

[0108] Among them, x, y and z represent the point in the camera coordinate system at O c x c Axis, O c y c Axis and O c z c Coordinate values ​​on the axis; u and v represent the pixel coordinates of the point in the camera coordinate system on the first axis and the second axis of the image coordinate system, respectively.

[0109] S4. Calculate the rotation matrix from the carrier coordinate system to the geographic coordinate system using the inertial sensor IMU The geographic coordinate system is O g is the origin, O g x g Axis, O g y g Axis and O g z g The axes are the first, second, and third axes that are perpendicular to each other and conform to the right-hand rule.

[0110] S41. Output the carrier attitude angle using the inertial sensor IMU on the camera bracket. The carrier attitude angle includes the carrier attitude yaw angle γ, the carrier attitude pitch angle α, and the carrier attitude roll angle β.

[0111] S42. Calculate the rotation matrix from the carrier coordinate system to the geographic coordinate system based on the carrier attitude angle

[0112]

[0113] S5, such as Figure 2 As shown, image distortion correction is performed based on the relationship between perspective projection and weak perspective projection.

[0114] S51, by means of perspective projection, obtain the perspective projection point P of the target point P on the ground I0 on the imaging plane I i ' and the approximate perspective projection point P' on the target approximate plane I'; the perspective projection point P i 'The pixel coordinates in the image coordinate system are (u' i ,v' i ).

[0115] Ground I0 is the x in the geographic coordinate system g O g y g Plane; the target approximate plane I′ is the origin O of the camera coordinate system c The point T where the extended line of the line connecting the origin F of the imaging plane I intersects with the ground I0 is the origin, and the plane is parallel to the imaging plane I.

[0116] S52, using weak perspective projection, obtain the weak perspective projection point P of the target point P on the ground I0 on the imaging plane I i " and the approximate weak perspective projection point P on the target approximate plane I'; the weak perspective projection point P i "The pixel coordinates in the image coordinate system are (u" i ,v” i ).

[0117] S53, according to the perspective projection point P i ' and weak perspective projection point P i ", the perspective projection point P i The pixel coordinates of ' are transformed into the weak perspective projection point P by using the transformation formula i "The pixel coordinates of the image are used to complete the image distortion correction. The transformation formula is:

[0118]

[0119] Among them, u0 and v0 represent the first axis offset value and the second axis offset value of the pixel center respectively; k represents the perspective projection point P i 'Transform to weak perspective projection point P i "The scaling factor satisfies:

[0120]

[0121] in, Represents the origin F of the imaging plane I to the perspective projection point Pi 'direction vector; Represents the origin F of the imaging plane I to the weak perspective projection point P i ” direction vector.

[0122] The conversion formula and the calculation of the scaling factor k specifically include the following steps:

[0123] S531. Calculate the direction vector of the origin T of the target approximate plane I′ in the camera coordinate system

[0124] S5311. In the camera coordinate system, Expressed as:

[0125]

[0126] Where λ is a constant.

[0127] S5312, rotation matrix from carrier coordinate system to geographic coordinate system And the rotation matrix from the camera coordinate system to the carrier coordinate system The representation in the geographic coordinate system is:

[0128]

[0129] in, Represents the rotation matrix from the camera coordinate system to the geographic coordinate system.

[0130] S5313. Combining formula (7) and formula (8), we get:

[0131]

[0132] S5314, based on Projected to O in geographic coordinate system g Z g The coordinate value of the axis, that is, the height H from the optical center of the camera to the ground I0, is obtained:

[0133]

[0134] The constant λ is obtained as:

[0135]

[0136] S5315, combined with formula (7) and formula (11), we get for:

[0137]

[0138] S532, calculate the direction vector from the origin T of the target approximate plane I' to the approximate perspective projection point P'

[0139] S5321. According to the perspective projection relationship, Expressed as:

[0140]

[0141] Where D represents the optical center of the camera O c The distance to the origin T of the target approximation plane I′.

[0142] S5322, combined with formula (12) and formula (13), we get for:

[0143]

[0144] S533, calculate the camera optical center O c Direction vector of the approximate perspective projection point P' to the target approximate plane I' for:

[0145]

[0146] Among them, e c Represents along the camera coordinate system O c z c The unit vector in the direction of the axis and e c =[0 0 1] T ; H represents the height from the optical center of the camera to the ground I0; f represents the focal length of the camera; the subscript 3 represents the third element.

[0147] S534. According to the scaling factor expression (6), k satisfies:

[0148]

[0149] in, Indicates the camera optical center O c Direction vector to the target point P; Represents the direction vector from the origin T of the target approximate plane I′ to the approximate weak perspective projection point P”; |||| indicates the length of the obtained vector.

[0150] Then it satisfies:

[0151]

[0152] S535 . Obtain a representation of the scaling factor k based on the conversion relationship among the carrier coordinate system, the geographic coordinate system, and the camera coordinate system.

[0153] S5351, Rotation matrix from carrier coordinate system to geographic coordinate system And the rotation matrix from the camera coordinate system to the carrier coordinate system The representation in the geographic coordinate system is:

[0154]

[0155] S5352, based on Projected to O in geographic coordinate system g Z g The coordinate value of the axis, that is, the height H from the optical center of the camera to the ground I0, is obtained:

[0156]

[0157] Then k satisfies:

[0158]

[0159] S5353, combined with equation (15) and equation (20), obtain the representation of the scaling factor k:

[0160]

[0161] S536: Calculate the transformation formula and scaling factor k.

[0162] S5361. According to the conversion relationship between the imaging plane coordinate system and the image coordinate system, the perspective projection point P i ' and weak perspective projection point P i "satisfy:

[0163]

[0164] Wherein, dx represents the length of a single pixel on the first axis of the image coordinate system corresponding to the first axis of the imaging plane coordinate system; dy represents the length of a single pixel on the second axis of the image coordinate system corresponding to the second axis of the imaging plane coordinate system; and Represent the perspective projection point P i 'Coordinate values ​​on the first and second axes of the imaging plane coordinate system; and Represents the weak perspective projection point P i ” The coordinate values ​​on the first axis and the second axis of the imaging plane coordinate system.

[0165] S5362, in the imaging plane coordinate system, Represented as perspective projection point P i ''s coordinates, i.e. Represented as weak perspective projection point Pi "The coordinates of Based on formula (6), the scaling factor k is expressed as:

[0166]

[0167] S5363. Based on formula (22) and formula (23), the transformation formula is obtained:

[0168]

[0169] S5364. Based on formula (21) and formula (22), obtain the scaling factor k:

[0170]

[0171] In a specific embodiment, the proposed method is used to obtain the distortion angle error result graph of the sampling area before and after image distortion correction, as shown in FIG. Figure 3 As shown in the figure, after using the proposed method to correct image distortion, the maximum value of the distortion correction error is reduced from 9.6° before correction to 3.9°, and the minimum value of the distortion correction error is reduced from 2.6° before correction to 0.5°, which shows the effectiveness and excellence of the method proposed in this invention.

[0172] The present invention designs an image distortion correction method for multi-pose photography using a drone. Based on the transformation relationship among the geographic coordinate system, the carrier coordinate system, the camera coordinate system, and the image coordinate system, and the transformation relationship of the weak perspective transformation, the method corrects the spatial resolution in real time, solves the serious problem of image geometric deformation caused by different depth information at each point when the drone photographs the ground at an angle, and achieves scale and rotation consistency between the aerial image and the reference image. Furthermore, the method uses an inertial sensor (IMU) to perform three-dimensional calibration on the camera installation error and the carrier attitude, obtains the camera installation error angle and the carrier attitude angle, reduces the influence of the initial camera installation error and the initial carrier attitude on the image distortion, and improves the accuracy of image geometric distortion correction.

[0173] Finally, it should be noted that the above embodiments are only intended to illustrate rather than limit the technical solutions of the present invention. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the present invention can still be modified or replaced by equivalents. Any modification or partial replacement that does not depart from the spirit and scope of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A method for correcting image distortion in multi-pose photography of unmanned aerial vehicles, characterized in that: It includes the following steps: S1. Calibrate the camera intrinsic parameter coefficients to obtain the camera intrinsic parameter matrix in the camera coordinate system; the camera coordinate system is based on the camera optical center O c is the origin, O c x c Axis, O c y c Axis and O c z c The axes are the first, second and third axes that are perpendicular to each other and conform to the right-hand rule; S2. Calculate the rotation matrix from the camera coordinate system to the carrier coordinate system using the inertial sensor IMU The carrier coordinate system is centered at the camera support O b is the origin, O b x b Axis, O b y b Axis and O b z b The axes are the first, second and third axes that are perpendicular to each other and conform to the right-hand rule; S3, constructing a pinhole camera imaging model and projecting points in the camera coordinate system to the image coordinate system; S4. Calculate the rotation matrix from the carrier coordinate system to the geographic coordinate system using the inertial sensor IMU The geographic coordinate system is based on g is the origin, O g x g Axis, O g y g Axis and O g z g The axes are the first, second and third axes that are perpendicular to each other and conform to the right-hand rule; S5. Perform image distortion correction based on the relationship between perspective projection and weak perspective projection; S51, by means of perspective projection, obtain the perspective projection point P of the target point P on the ground I0 on the imaging plane I i ' and the approximate perspective projection point P' on the target approximate plane I'; the perspective projection point P i 'The pixel coordinates in the image coordinate system are (u′ i ,v′ i ); S52, using weak perspective projection, obtain the weak perspective projection point P of the target point P on the ground I0 on the imaging plane I i " and the approximate weak perspective projection point P on the target approximate plane I'; the weak perspective projection point P i The pixel coordinates in the image coordinate system are (u i ”,v i ”); S53, according to the perspective projection point P i ' and weak perspective projection point P i ", the perspective projection point P i The pixel coordinates of ' are transformed into the weak perspective projection point P by using the transformation formula i "The pixel coordinates of the image are used to complete the image distortion correction. The transformation formula is: Among them, u0 and v0 represent the first axis offset value and the second axis offset value of the pixel center respectively; k represents the perspective projection point P i 'Transform to weak perspective projection point P i "The scaling factor, the scaling factor expression satisfies: in, Represents the origin F of the imaging plane I to the perspective projection point P i 'direction vector; Represents the origin F of the imaging plane I to the weak perspective projection point P i " direction vector; In step S53, the transformation formula and scaling factor k are obtained by the following specific steps: S531. Calculate the direction vector of the origin T of the target approximate plane I′ in the camera coordinate system S532, calculate the direction vector from the origin T of the target approximate plane I' to the approximate perspective projection point P' S533, calculate the camera optical center O c Direction vector of the approximate perspective projection point P' to the target approximate plane I' for: Among them, e c Represents along the camera coordinate system O c z c The unit vector in the direction of the axis and e c =[001] T ; H represents the height from the optical center of the camera to the ground I0; f represents the focal length of the camera; subscript 3 represents the third element; S534. According to the scaling factor expression (6), k satisfies: in, Indicates the camera optical center O c Direction vector to the target point P; Represents the direction vector from the origin T of the target approximate plane I′ to the approximate weak perspective projection point P”; || || represents the length of the obtained vector; Then it satisfies: S535. Based on the conversion relationship among the carrier coordinate system, the geographic coordinate system, and the camera coordinate system, obtain the representation of the scaling factor k: S536, calculating the transformation formula and the scaling factor k; S5361. According to the conversion relationship between the imaging plane coordinate system and the image coordinate system, the perspective projection point P i ' and weak perspective projection point P i "satisfy: Wherein, dx represents the length of a single pixel on the first axis of the image coordinate system corresponding to the first axis of the imaging plane coordinate system; dy represents the length of a single pixel on the second axis of the image coordinate system corresponding to the second axis of the imaging plane coordinate system; and Represent the perspective projection point P i 'Coordinate values ​​on the first and second axes of the imaging plane coordinate system; and Represents the weak perspective projection point P i ”Coordinate values ​​on the first and second axes of the imaging plane coordinate system; S5362, in the imaging plane coordinate system, Represented as perspective projection point P i ''s coordinates, i.e. Represented as weak perspective projection point P i "The coordinates of Based on formula (6), the scaling factor k is expressed as: S5363. Based on formula (22) and formula (23), obtain the transformation formula: S5364. Based on formula (21) and formula (22), obtain the scaling factor k:

2. The image distortion correction method for multi-pose shooting of unmanned aerial vehicle according to claim 1, characterized in that: Step S531 specifically includes the following steps: S5311. In the camera coordinate system, Expressed as: Where λ is a constant; S5312, rotation matrix from carrier coordinate system to geographic coordinate system And the rotation matrix from the camera coordinate system to the carrier coordinate system The representation in the geographic coordinate system is: in, Represents the rotation matrix from the camera coordinate system to the geographic coordinate system; S5313. Combining formula (7) and formula (8), we get: S5314, based on Projected to O in geographic coordinate system g Z g The coordinate value of the axis, that is, the height H from the optical center of the camera to the ground I0, is obtained: The constant λ is obtained as: S5315, combined with formula (7) and formula (11), we get for: Step S532 specifically includes the following steps: S5321. According to the perspective projection relationship, Expressed as: Where D represents the optical center of the camera O c The distance to the origin T of the target approximation plane I′; S5322, combined with formula (12) and formula (13), we get for: Step S535 specifically includes the following steps: S5351, Rotation matrix from carrier coordinate system to geographic coordinate system And the rotation matrix from the camera coordinate system to the carrier coordinate system The representation in the geographic coordinate system is: S5352, based on Projected to O in geographic coordinate system g Z g The coordinate value of the axis, that is, the height H from the optical center of the camera to the ground I0, is obtained: Then k satisfies: S5353, combining equations (15) and (20), obtains the representation of the scaling factor k.

3. The image distortion correction method for multi-pose shooting of unmanned aerial vehicle according to claim 1, characterized in that: Step S2 specifically includes the following steps: S21, installing an inertial sensor IMU on the camera bracket and the camera respectively, and measuring the camera installation error angle; the camera installation error angle includes the camera installation error yaw angle γ', the camera installation error pitch angle α' and the camera installation error roll angle β'; S22. Calculate the rotation matrix from the camera coordinate system to the carrier coordinate system based on the camera installation error angle Among them, cos and sin represent cosine and sine functions respectively; Step S4 specifically includes the following steps: S41, using the inertial sensor IMU on the camera bracket to output the carrier attitude angle; the carrier attitude angle includes the carrier attitude yaw angle γ, the carrier attitude pitch angle α and the carrier attitude roll angle β; S42. Calculate the rotation matrix from the carrier coordinate system to the geographic coordinate system based on the carrier attitude angle 4. The image distortion correction method for multi-pose shooting of unmanned aerial vehicles according to claim 1, characterized in that: The camera intrinsic parameter matrix K in step S1 is: Among them, f x Indicates the focal length f of the camera at O c x c The projection of the axis; f y Indicates the focal length f of the camera at O c y c Projection of axis; c x Indicates O c x c The pixel offset corresponding to the axis; c y Indicates O c y c The pixel offset corresponding to the axis.

5. The image distortion correction method for multi-pose shooting of unmanned aerial vehicle according to claim 4, characterized in that: The pinhole camera imaging model in step S3 is: Among them, x, y and z represent the point in the camera coordinate system at O c x c Axis, O c y c Axis and O c z c Coordinate values ​​on the axis; u and v represent the pixel coordinates of the point in the camera coordinate system on the first axis and the second axis of the image coordinate system, respectively.

6. The image distortion correction method for multi-pose shooting of unmanned aerial vehicle according to claim 1, characterized in that: In step S51, the ground I0 is x in the geographic coordinate system. g O g y g Plane; the target approximate plane I' is the origin O of the camera coordinate system c The point T where the extended line of the line connecting the origin F of the imaging plane I intersects with the ground I0 is the origin, and the plane is parallel to the imaging plane I.

Images