Image processing method for aerial photography of unmanned aerial vehicle

Abstract

The invention relates to the technical field of unmanned aerial vehicle aerial photography, and discloses an image processing method for unmanned aerial vehicle aerial photography. According to the image processing method for unmanned aerial vehicle aerial photography, shooting position coordinates of an unmanned aerial vehicle aerial photography image and unmanned aerial vehicle attitude angle data during image shooting are obtained, an external parameter matrix is constructed through a picture offset vector and an unmanned aerial vehicle attitude rotation matrix, and focal length data and image center point coordinate data of an unmanned aerial vehicle aerial photography camera are obtained; constructing an internal parameter matrix according to the data, establishing a projection matrix in combination with the external parameter matrix and the internal parameter matrix, projecting each coordinate point of the image to an image plane, realizing geometric correction of the image, outputting the corrected image, performing malformation correction on the image by adopting a linear approximation method, reducing the overhead of iterative calculation, and improving the image distortion correction efficiency. The method improves the calculation speed, facilitates the real-time processing of the image, is simple in algorithm formula, improves the image correction precision, and improves the calculation efficiency of the geometric correction of the image.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) aerial photography image processing, and in particular to an image processing method for UAV aerial photography. Background Art

[0002] Drone aerial photography refers to the use of unmanned aerial vehicles equipped with cameras to capture aerial footage. Its flexibility, cost-effectiveness, and high-quality images have led to its widespread application in various fields. Drones can fly over complex terrain and environments, capturing high-altitude perspectives difficult to reach with traditional equipment, such as mountains, forests, and urban high-rise buildings. Drones are also less expensive than helicopters or manned aircraft and do not require professional pilots. Modern drones are equipped with high-definition cameras and sensors, capable of capturing high-quality images and videos. They also have real-time transmission capabilities, allowing operators to monitor the footage in real time at a ground station and easily adjust shooting angles and parameters. Many drones also have automated flight modes and advanced features, such as automatic obstacle avoidance and automatic return to home, which improve the reliability and safety of filming. Drone aerial photography provides unique aerial footage in film and television production, enhancing the visual effects of film and television works; it can quickly reach the scene of news reports and provide a real-time, unique perspective; in environmental monitoring, it helps monitor natural environments such as forests, wetlands, and rivers and conduct ecological research; in agriculture, it is used to monitor crop growth and conduct precision agricultural management; in construction and real estate, it provides panoramic views of construction sites and three-dimensional perspectives of real estate; in tourism and advertising, it is used to shoot promotional videos of tourist attractions, providing a unique aerial perspective and enhancing the travel experience;

[0003] Currently, the image processing process of drones after aerial photography is complicated, and it is impossible to perform geometric correction on images efficiently and concisely, which increases the time of image processing and reduces the efficiency of image processing. Summary of the Invention

[0004] (1) Technical problems solved

[0005] In response to the deficiencies in the prior art, the present invention provides an image processing method for drone aerial photography, which has the characteristics of obtaining the shooting position coordinates of the drone aerial image and the drone attitude angle data when the image is shot, converting the drone attitude angle data into radian values, and establishing the drone attitude rotation matrix when the current image is shot according to the radian system. At the same time, the image offset vector is calculated according to the image shooting position coordinates, and the image offset vector and the drone attitude rotation matrix are used to construct an external parameter matrix, obtain the focal length data of the drone aerial camera and the image center point coordinate data, and construct an internal parameter matrix based on these data, and combine the external parameter matrix and the internal parameter matrix to establish a projection matrix, and project each coordinate point of the image onto the image plane to achieve geometric correction of the image, and output the corrected image. The linear approximation method is used to correct the deformity of the image, which reduces the overhead of iterative calculation, improves the calculation speed, and facilitates real-time image processing. At the same time, the algorithm formula is simple, and while improving the image correction accuracy, it also improves the calculation efficiency of image geometric correction, thereby solving the above-mentioned problems.

[0006] (2) Technical solution

[0007] To achieve the above object, the present invention provides the following technical solution: an image processing method for drone aerial photography, comprising the following steps:

[0008] S1, UAV aerial image, and obtain the shooting position of the current image and the UAV attitude angle data when the current image was taken;

[0009] S2. Convert the drone attitude angle data when the current image is taken into the corresponding radian value, and calculate the rotation matrix of the drone attitude when the current image is taken according to the radian setting;

[0010] S3, calculating the image translation vector according to the shooting position of the current image;

[0011] S4, constructing an external parameter matrix based on the rotation matrix of the drone's posture when the current image is taken and the image translation vector;

[0012] S5. Obtain focal length data, center point coordinate data, and lens distortion parameters of the drone aerial camera, and construct an internal parameter matrix based on these data and parameters;

[0013] S6. Establish a projection matrix based on the external parameter matrix and the internal parameter matrix;

[0014] S7. Reproject each coordinate point onto the image plane according to the projection matrix to complete the geometric correction, obtain the corrected image, and output it.

[0015] Preferably, the shooting position of the current image is expressed as: (x, y, z), and the drone attitude angle data when the current image is shot includes the drone roll angle, the drone pitch angle and the drone yaw angle, the drone roll angle is expressed as G, the drone pitch angle is expressed as F, and the drone yaw angle is expressed as P.

[0016] Preferably, the numerical calculation expression for converting the roll angle of the drone into the roll angle radian is as follows:

[0017]

[0018] In the formula, Gz represents the roll angle converted to the roll angle radian value, which is obtained by rotating around the X-axis. 1 means that the rotation of the drone's roll angle G has no effect on the X-axis component, and the rotation operation does not change the coordinate of the X-axis. 0 means that the rotation of the drone's roll angle G around the X-axis does not affect the zero position of the Y-axis or the rotation of the drone's roll angle G around the X-axis does not affect the zero position of the Z-axis. cos(G) represents the cosine coefficient of the Z-axis component after the drone's roll angle G rotates, and sin(G) represents the sine coefficient of the Y-axis component after the drone's roll angle G rotates.

[0019] Preferably, the numerical calculation expression for converting the pitch angle of the drone into the roll angle radian is as follows:

[0020]

[0021] In the formula, Fz represents the roll angle converted from the pitch angle of the drone, which is obtained by rotating around the Y-axis. Cos(F) represents the X-axis component of the pitch angle of the drone after the X-axis rotation. Sin(F) represents the X-axis component of the pitch angle of the drone after the Z-axis rotation. 0 represents the Y-axis component of the pitch angle of the drone after the Z-axis rotation. 1 represents that the pitch angle rotation of the drone has no effect on the Y-axis component.

[0022] Preferably, the numerical calculation expression for converting the yaw angle of the drone into the roll angle radian is as follows:

[0023]

[0024] In the formula, Pz represents the radian value of the drone's yaw angle converted to the roll angle, sin(P) represents the X-axis component of the drone's yaw angle after the Y-axis is rotated, cos(P) represents the Y-axis component of the drone's yaw angle after the Y-axis is rotated, 0 represents the Z-axis component of the drone's yaw angle after the Y-axis is rotated, and 1 represents the Z-axis component of the drone's yaw angle after the Z-axis is rotated.

[0025] Preferably, the rotation matrix expression of the UAV attitude is:

[0026] ZTxz=Gz·Fz·Pz

[0027] In the formula, ZTxz represents the rotation matrix of the drone's attitude, and Gz·Fz·Pz represents the product of the radian values ​​of the drone's three attitudes in the rotation order of the Euler angle, which is the rotation matrix of the drone's attitude.

[0028] Preferably, the image translation vector calculation expression is as follows:

[0029]

[0030] In the formula, PY represents the image translation vector.

[0031] Preferably, the external parameter matrix expression is as follows:

[0032] WB=[ZTxz|PY]

[0033] In the formula, WB represents the external parameter matrix, and | represents the horizontal concatenation of the rotation matrix of the drone attitude and the external parameter matrix.

[0034] Preferably, the internal parameter matrix expression is as follows:

[0035]

[0036] In the formula, NC represents the internal parameter matrix, J x Indicates the focal length of the drone camera in the X-axis direction, J y Represents the focal length of the drone's aerial camera in the Y-axis direction, zx represents the X-axis coordinate of the image center point, zy represents the Y-axis coordinate of the image center point, in the matrix, 0 in the first row represents no scaling or translation in the X-axis direction, 0 in the second row represents no scaling or translation in the X-axis direction, 0 in the third row represents ensuring additional scaling and translation in the X-axis and Y-axis, and 1 represents the homogeneous coordinate normalization factor.

[0037] Preferably, the projection matrix is ​​expressed as:

[0038] TY=WB·NC

[0039] In the formula, TY represents the projection matrix, and WB·NC represents the combination of the external parameter matrix and the internal parameter matrix to form the projection matrix.

[0040] Compared with the prior art, the present invention provides an image processing method for drone aerial photography, which has the following beneficial effects:

[0041] The present invention obtains the shooting position coordinates of the drone aerial image and the drone attitude angle data when the image is shot, converts the drone attitude angle data into radian values, establishes the drone attitude rotation matrix when the current image is shot according to the radian system, and at the same time, calculates the picture offset vector according to the image shooting position coordinates, constructs an external parameter matrix with the picture offset vector and the drone attitude rotation matrix, obtains the focal length data of the drone aerial camera and the image center point coordinate data, and constructs an internal parameter matrix based on these data, establishes a projection matrix in combination with the external parameter matrix and the internal parameter matrix, and projects each coordinate point of the image onto the image plane to achieve geometric correction of the image, and outputs the corrected image. The linear approximation method is used to correct the deformity of the image, which reduces the overhead of iterative calculation, improves the calculation speed, and facilitates real-time image processing. At the same time, the algorithm formula is simple, which improves the accuracy of picture correction while improving the calculation efficiency of image geometric correction. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 Schematic diagram of the steps of the method of the present invention. DETAILED DESCRIPTION

[0043] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0044] To improve the efficiency of image processing during drone aerial photography, reduce the computing power overhead of drone processor iterative calculations, increase the speed of image processing while ensuring the accuracy of image processing, please refer to Figure 1 , an image processing method for UAV aerial photography is proposed, comprising the following steps:

[0045] S1. Take an aerial image from a drone and obtain the shooting position of the current image and the drone attitude angle data when the current image is shot. The shooting position of the current image is represented by (x, y, z). The drone attitude angle data when the current image is shot include the drone roll angle, drone pitch angle, and drone yaw angle. The drone roll angle is represented by G, the drone pitch angle is represented by F, and the drone yaw angle is represented by P.

[0046] S2. Convert the drone attitude angle data when the current image is taken into corresponding radian values, and calculate the rotation matrix of the drone attitude when the current image is taken based on the radian values.

[0047] The numerical calculation expression for converting the drone's roll angle into roll angle radians is as follows:

[0048]

[0049] In the formula, Gz represents the roll angle converted to the roll angle radian value, which is obtained by rotating around the X-axis. 1 means that the rotation of the drone's roll angle G has no effect on the X-axis component, and the rotation operation does not change the coordinate of the X-axis. 0 means that the rotation of the drone's roll angle G around the X-axis does not affect the zero position of the Y-axis or the rotation of the drone's roll angle G around the X-axis does not affect the zero position of the Z-axis. cos(G) represents the cosine coefficient of the Z-axis component after the drone's roll angle G rotates, and sin(G) represents the sine coefficient of the Y-axis component after the drone's roll angle G rotates.

[0050] The numerical calculation expression for converting the pitch angle of the drone into the roll angle radian is as follows:

[0051]

[0052] In the formula, Fz represents the roll angle converted from the pitch angle of the drone, which is obtained by rotating around the Y axis. Cos(F) represents the X-axis component of the pitch angle of the drone after the X axis is rotated. Sin(F) represents the X-axis component of the pitch angle of the drone after the Z axis is rotated. 0 represents the Y-axis component of the pitch angle of the drone after the Z axis is rotated. 1 represents that the pitch angle rotation of the drone has no effect on the Y-axis component.

[0053] The numerical calculation expression for converting the yaw angle of the drone into the roll angle radian is as follows:

[0054]

[0055] In the formula, Pz represents the yaw angle of the drone converted to the roll angle radian value, sin(P) represents the X-axis component of the drone's yaw angle Y-axis after rotation, cos(P) represents the Y-axis component of the drone's yaw angle Y-axis after rotation, 0 represents the Z-axis component of the drone's yaw angle Y-axis after rotation, and 1 represents the Z-axis component of the drone's yaw angle Z-axis after rotation;

[0056] The rotation matrix expression of the drone attitude is:

[0057] ZTxz=Gz·Fz·Pz

[0058] In the formula, ZTxz represents the rotation matrix of the drone's attitude, and Gz·Fz·Pz represents the product of the radian values ​​of the drone's three attitudes in the rotation order of the Euler angle, which is the rotation matrix of the drone's attitude.

[0059] S3. Calculate the image translation vector based on the shooting position of the current image. The specific calculation expression is as follows:

[0060]

[0061] In the formula, PY represents the image translation vector.

[0062] S4. Construct an external parameter matrix based on the rotation matrix of the drone's posture when the current image was taken and the image translation vector. The external parameter matrix expression is as follows:

[0063] WB=[ZTxz|PY]

[0064] In the formula, WB represents the external parameter matrix, and | represents the horizontal concatenation of the rotation matrix of the drone attitude and the external parameter matrix.

[0065] S5. Obtain the focal length data and center point coordinate data of the drone aerial camera, and construct an internal parameter matrix based on these data. The internal parameter matrix expression is as follows:

[0066]

[0067] In the formula, NC represents the internal parameter matrix, J x Indicates the focal length of the drone camera in the X-axis direction, J y Represents the focal length of the drone's aerial camera in the Y-axis direction, zx represents the X-axis coordinate of the image center point, zy represents the Y-axis coordinate of the image center point, in the matrix, 0 in the first row represents no scaling or translation in the X-axis direction, 0 in the second row represents no scaling or translation in the X-axis direction, 0 in the third row represents ensuring additional scaling and translation in the X-axis and Y-axis, and 1 represents the homogeneous coordinate normalization factor.

[0068] S6. Establish a projection matrix based on the external parameter matrix and the internal parameter matrix, wherein the projection matrix is ​​specifically as follows:

[0069] TY=WB·NC

[0070] In the formula, TY represents the projection matrix, and WB·NC represents the combination of the external parameter matrix and the internal parameter matrix to form the projection matrix.

[0071] S7. Reproject each coordinate point onto the image plane according to the projection matrix to complete the geometric correction, obtain the corrected image, and output it.

[0072] The specific implementation is as follows:

[0073] The internal parameter matrix is ​​used to project the three-dimensional points of the image onto the image plane, and the projection is changed. The homogeneous coordinates generated after the projection are normalized, that is, the homogeneous coordinates are converted into ordinary two-dimensional coordinates, and the corrected image is obtained.

[0074] By obtaining the shooting position coordinates of the drone aerial image and the drone attitude angle data when the image is taken, the drone attitude angle data is converted into radian values, and the drone attitude rotation matrix when the current image is taken is established according to the radian system. At the same time, the image offset vector is calculated according to the image shooting position coordinates, and the image offset vector and the drone attitude rotation matrix are used to construct an external parameter matrix. The focal length data of the drone aerial camera and the image center point coordinate data are obtained, and an internal parameter matrix is ​​constructed based on these data. The projection matrix is ​​established by combining the external parameter matrix and the internal parameter matrix, and each coordinate point of the image is projected onto the image plane to achieve geometric correction of the image, and the corrected image is output. The linear approximation method is used to correct the image deformity, which reduces the overhead of iterative calculation, improves the calculation speed, and facilitates real-time image processing. At the same time, the algorithm formula is simple, which improves the accuracy of image correction while improving the calculation efficiency of image geometric correction.

[0075] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. An image processing method for drone aerial photography, characterized by: The following steps are involved: S1, UAV aerial image, and obtain the shooting position of the current image and the UAV attitude angle data when the current image was taken; S2. Convert the drone attitude angle data when the current image is taken into corresponding radian values, and calculate the rotation matrix of the drone attitude when the current image is taken based on the radian values; S3, calculating the image translation vector according to the shooting position of the current image; S4, constructing an external parameter matrix based on the rotation matrix of the drone's posture when the current image is taken and the image translation vector; S5. Obtain the focal length data and center point coordinate data of the drone aerial camera, and construct an internal parameter matrix based on these data; S6. Establish a projection matrix based on the external parameter matrix and the internal parameter matrix; S7. Reproject each coordinate point onto the image plane according to the projection matrix to complete the geometric correction, obtain the corrected image, and output it.

2. The image processing method for drone aerial photography according to claim 1, characterized in that: The shooting position of the current image is expressed as: (x, y, z), and the drone attitude angle data when the current image is shot includes the drone roll angle, the drone pitch angle and the drone yaw angle. The drone roll angle is expressed as G, the drone pitch angle is expressed as F, and the drone yaw angle is expressed as P.

3. The image processing method for drone aerial photography according to claim 2, characterized in that: The numerical calculation expression for converting the roll angle of the drone into the roll angle radian is as follows: In the formula, Gz represents the roll angle converted to the roll angle radian value, which is obtained by rotating around the X-axis. 1 means that the rotation of the drone's roll angle G has no effect on the X-axis component, and the rotation operation does not change the coordinate of the X-axis. 0 means that the rotation of the drone's roll angle G around the X-axis does not affect the zero position of the Y-axis or the rotation of the drone's roll angle G around the X-axis does not affect the zero position of the Z-axis. cos(G) represents the cosine coefficient of the Z-axis component after the drone's roll angle G rotates, and sin(G) represents the sine coefficient of the Y-axis component after the drone's roll angle G rotates.

4. The image processing method for drone aerial photography according to claim 1, characterized in that: The numerical calculation expression for converting the pitch angle of the drone into the roll angle radians is as follows: In the formula, Fz represents the roll angle converted from the pitch angle of the drone, which is obtained by rotating around the Y-axis. Cos(F) represents the X-axis component of the pitch angle of the drone after the X-axis rotation. Sin(F) represents the X-axis component of the pitch angle of the drone after the Z-axis rotation. 0 represents the Y-axis component of the pitch angle of the drone after the Z-axis rotation. 1 represents that the pitch angle rotation of the drone has no effect on the Y-axis component.

5. The image processing method for drone aerial photography according to claim 4, characterized in that: The numerical calculation expression for converting the yaw angle of the drone into the roll angle radians is as follows: In the formula, Pz represents the radian value of the drone's yaw angle converted to the roll angle, sin(P) represents the X-axis component of the drone's yaw angle after the Y-axis is rotated, cos(P) represents the Y-axis component of the drone's yaw angle after the Y-axis is rotated, 0 represents the Z-axis component of the drone's yaw angle after the Y-axis is rotated, and 1 represents the Z-axis component of the drone's yaw angle after the Z-axis is rotated.

6. The image processing method for drone aerial photography according to claim 5, characterized in that: The rotation matrix expression of the UAV attitude is: ZTxz=Gz·Fz·Pz In the formula, ZTxz represents the rotation matrix of the drone's attitude, and Gz·Fz·Pz represents the product of the radian values ​​of the drone's three attitudes in the rotation order of the Euler angle, which is the rotation matrix of the drone's attitude.

7. The image processing method for drone aerial photography according to claim 1, characterized in that: The image translation vector calculation expression is as follows: In the formula, PY represents the image translation vector.

8. The image processing method for drone aerial photography according to claim 7, characterized in that: The external parameter matrix expression is as follows: WB=[ZTxz|PY] In the formula, WB represents the external parameter matrix, and || represents the horizontal concatenation of the rotation matrix of the drone attitude and the external parameter matrix.

9. The image processing method for drone aerial photography according to claim 8, characterized in that: The internal parameter matrix expression is as follows: In the formula, NC represents the internal parameter matrix, J x Indicates the focal length of the drone camera in the X-axis direction, J y Represents the focal length of the drone's aerial camera in the Y-axis direction, zx represents the X-axis coordinate of the image center point, zy represents the Y-axis coordinate of the image center point, in the matrix, 0 in the first row represents no scaling or translation in the X-axis direction, 0 in the second row represents no scaling or translation in the X-axis direction, 0 in the third row represents ensuring additional scaling and translation in the X-axis and Y-axis, and 1 represents the homogeneous coordinate normalization factor.

10. The image processing method for drone aerial photography according to claim 9, characterized in that: The projection matrix is ​​expressed as: TY=WB·NC In the formula, TY represents the projection matrix, and WB·NC represents the combination of the external parameter matrix and the internal parameter matrix to form the projection matrix.

Images