A geometric correction method for aerial images based on UAV attitude information

Abstract

The present invention discloses a method for geometric correction of aerial images based on attitude information of unmanned aerial vehicles. The present invention utilizes the internal orientation elements of the aerial images, and directly uses the POS parameters of the aerial images as the external orientation elements of the images, and realizes geometric correction of the pre-processed unmanned aerial vehicle remote sensing images through three-dimensional reconstruction space coordinate transformation and indirect image correction. The present invention makes full use of the pixel information of the original image, and uses multiple attitude parameters to participate in the correction at the same time, avoiding the correction error caused by the incoherent participation of multiple parameters in the correction of the traditional method, and performs geometric correction on the unmanned aerial image when it is impossible to select the artificial control point pair, so that the image is more in line with the standard, and obtains a corrected image with regularly arranged image points, which is convenient for subsequent aerial image registration and splicing and other processing.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method for geometric correction of aerial images based on unmanned aerial vehicle (UAV) attitude information. Background Art

[0002] With the rapid development of drones, drone low-altitude remote sensing technology has gradually become an important means of obtaining spatial information. By using a specific processing system, a series of operations are performed on the pixels in the drone aerial images, ultimately achieving the purpose of information processing and information extraction. So far, drones have been well applied in various projects such as building measurement, fault detection, and wind power inspection.

[0003] However, when a drone is performing an aerial photography mission, its flight motion may be affected by some external factors such as wind and airflow, which may change the posture of the drone imaging sensor, and the drone shooting platform may pitch, roll and yaw, which will cause the acquired aerial images to be pitched, rolled and yawed. Because of the existence of these geometric deformations, the image cannot directly represent the shape and plane position of the object, resulting in geometric distortion of the acquired aerial image. For the same object area, there will be relatively large differences in the aerial images obtained by different flight postures. Therefore, it is necessary to perform geometric correction on the acquired images to obtain aerial images based on the same reference projection surface, thereby improving the speed of drone remote sensing image data processing and the quality of quantitative information extraction. The existing UAV aerial image geometric correction usually considers manual arrangement of control points for correction. However, although this method has high correction accuracy, it requires a lot of manpower, material and financial resources. It is particularly difficult to arrange control points in mountainous areas, water surfaces, deserts and other places, and it is not suitable for implementation. In addition, considering that some aerial photography staff do not have relevant knowledge, the relevant distortion in the data set after aerial photography cannot be corrected by the control point method. The quality and speed of the geometric correction of UAV aerial images will directly affect the subsequent data processing and analysis decisions. Therefore, how to quickly and accurately eliminate the geometric distortion of the original image is a key technology for the preprocessing and application of UAV aerial images.

[0004] Considering that the UAV is equipped with an inertial navigation unit (INU) and GPS, the attitude angle and coordinates of the UAV during navigation can be saved in real time. According to the imaging sensor attitude, aerial photography time, auxiliary navigation positioning data and data obtained by various aerial sensors during the flight of the UAV, image correction without control points or orthophoto references can be completed, which is easy to implement in practical applications. In order to enable the UAV remote sensing data to be quickly and effectively applied to information extraction and analysis, it is necessary to establish a corresponding processing model to complete the geometric correction of UAV remote sensing images in the absence of ground control points. The essence of image correction is to achieve the geometric transformation between two two-dimensional images. Therefore, before image correction, the geometric relationship between the original image and the corrected image must be established. Usually, two methods are used for correction, namely direct method and indirect method. Li Zheng proposed a method for image geometric correction using only aerial flight information (Li Zheng, "Research on Geometric Correction Technology of UAV Remote Sensing Images Lacking Control Points", Master's thesis of University of Electronic Science and Technology of China, 2010). The direct method uses the image plane coordinates of each image point on the original image and the geometric relationship established through the attitude information to solve the image plane coordinates of the corresponding image points on the corrected image one by one. However, the image points on the image corrected by this method are irregularly arranged, and blank or repeated pixels may appear, which causes a certain degree of difficulty in the interpolation of pixel values. Therefore, it is difficult to obtain a corrected image with regularly arranged image points. Summary of the invention

[0005] Due to the above-mentioned shortcomings of the direct method and taking into account the authenticity of the original information of the aerial image, the present invention provides an approach in which, in the absence of ground control points, the internal orientation elements of the image, namely the coordinates of the principal point in the frame coordinate system and the focal length of the camera, are known according to the camera calibration report. At the same time, the POS parameters of the aerial image are directly used as the external orientation elements of the image. Through three-dimensional reconstruction space coordinate transformation and indirect image correction, the geometric correction of the pre-processed UAV remote sensing image is achieved, and the plane of the UAV remote sensing image with geometric deformation is transformed to the geoid plane.

[0006] During the implementation of the method of the present invention, the scene is imaged on the camera sensor to form a two-dimensional image. The image plane coordinates of a certain point in space on the corrected image are (u', v'), and the image plane coordinates of a certain point on the original image are (u, v).

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] Step 1: Find the physical equivalent coordinates corresponding to the corrected image coordinates

[0009] Calculate the physical length of a unit pixel (d x ,d y), according to the physical length and the camera focal length f, the internal parameter matrix K1 is calculated. There is no rotational or translational distortion between the image point and the coordinates of the world coordinate system after correction. The physical equivalent coordinates corresponding to the image point (u', v') after correction are (X, Y, Z). The physical equivalent coordinates refer to the coordinate values ​​of the physical coordinate system divided by the coordinates of the aircraft's flight altitude, which can be expressed as:

[0010]

[0011] Step 2: Find the plane coordinates of the image before correction corresponding to the physical equivalent coordinates

[0012] According to the physical equivalent coordinates and flight attitude parameters obtained in step 1, the image plane coordinates before correction are calculated according to the following formula;

[0013]

[0014] Where R is the rotation matrix calculated based on the flight attitude parameters, and T is the translation vector;

[0015] Step 3: Calculate the corrected image

[0016] For each corrected image point coordinate (u', v'), the coordinate transformation relationship obtained in steps 1 and 2 is used to obtain the image point coordinate (u, v) of the image before correction; by interpolating the pixels of the image before correction, the pixel value with the coordinate (u', v') in the corrected image is obtained;

[0017] Step 4: Select an appropriate resampling method to assign grayscale values ​​to the output image pixels

[0018] For each corrected image point coordinate (u', v'), the coordinate transformation relationship obtained in steps 1 and 2 is used to obtain the pixel coordinates (u, v) of the image before correction, and the corrected image is resampled using an interpolation method to obtain the pixel value with coordinates (u', v') in the corrected image. The interpolation method includes nearest neighbor interpolation, bilinear interpolation, and cubic convolution interpolation.

[0019] Furthermore, the calculation formula of the internal parameter matrix K1 in steps 1 and 2 is:

[0020]

[0021] Among them (C x ,C y ) represents the coordinates of the optical center of the image in the image pixel coordinates, dx and dy represent the size of the unit pixel on the x-axis and y-axis of the sensor respectively, and f is the focal length of the camera.

[0022] Furthermore, the calculation formula of the rotation matrix R in step 2 is:

[0023]

[0024] in, θ and ω are the pitch angle, roll angle and yaw angle of the image attitude parameters respectively.

[0025] Beneficial effects of the present invention:

[0026] (1) The mapping relationship before and after correction is constructed using the geographic posture information of remote sensing images. When manual control point selection is not possible, the UAV aerial images are geometrically corrected to make the images more consistent with the standards and obtain corrected images with regularly arranged image points, which is convenient for subsequent aerial image registration and stitching.

[0027] (2) Multiple posture parameters are corrected simultaneously, which avoids the correction error caused by the incoherent correction of multiple parameters in the traditional method. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 Schematic diagram comparing direct and indirect correction methods

[0029] Figure 2 Indirect image correction diagram

[0030] Figure 3(a) is an aerial image of the original component;

[0031] Figure 3(b) shows the component image after correction

[0032] Figure 4(a) is an aerial image of the original component;

[0033] Figure 4(b) shows the component detail image after correction DETAILED DESCRIPTION

[0034] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is described in detail below in conjunction with specific embodiments and drawings, and specific embodiments are described to simplify the present invention. However, it should be recognized that the present invention is not limited to the illustrated embodiments, and various modifications of the present invention are possible without departing from the basic principles, and these equivalent forms also fall within the scope defined by the claims attached to this application.

[0035] like Figure 1 and Figure 2 As shown in the figure, a geometric correction method based on the attitude information of UAV aerial images is specifically implemented as follows:

[0036] Step 1: Find the physical equivalent coordinates corresponding to the corrected image coordinates

[0037] Calculate the physical length of a unit pixel (d x ,d y), according to the physical length and the camera focal length f, the internal parameter matrix K1 is calculated. There is no rotational or translational distortion between the corrected image point and the coordinates of the world coordinate system. The physical equivalent coordinates corresponding to the corrected image point (u', v') are (X, Y, Z). The physical equivalent coordinates refer to the coordinate values ​​of the physical coordinate system divided by the coordinates of the aircraft's flight altitude. The coordinate conversion formula is:

[0038]

[0039] Step 2: Find the plane coordinates of the image before correction corresponding to the physical equivalent coordinates

[0040] According to the physical equivalent coordinates and flight attitude parameters obtained in step 1, the image plane coordinates before correction are calculated according to the following formula;

[0041]

[0042] Where R is the rotation matrix calculated based on the flight attitude parameters, and T is the translation vector;

[0043] Step 3: Select a suitable resampling method to assign grayscale values ​​to the output image pixels;

[0044] For each corrected coordinate (u', v'), the coordinate transformation relationship obtained in steps 1 and 2 is used to obtain the pixel coordinates (u, v) of the image before correction, and the mapping from the pixel coordinates of the corrected image to the pixel coordinates of the image before correction is completed. After the coordinate transformation is completed, the corrected image is resampled using an interpolation method to obtain the pixel value of the coordinate (u', v') in the corrected image, and the interpolation method includes the nearest neighbor interpolation method, the bilinear interpolation method, and the cubic convolution interpolation method.

[0045] The implementation of the interpolation method is to take out the grayscale value of the original image point calculated by the above steps and fill it back to the corresponding pixel point in the blank output image dot matrix. Since it is not necessarily located exactly at the center of a certain pixel in the original image, the grayscale value at that location must be determined through grayscale interpolation. A suitable resampling method can be selected to assign grayscale values ​​to the output image pixels of the distorted image. Among them, there are many resampling methods, and the most commonly used ones are the nearest neighbor interpolation method, the bilinear interpolation method, and the cubic convolution interpolation method. By interpolating the pixels of the image before correction, the pixel values ​​of the corresponding coordinates of the corrected image are obtained, and the image correction can be completed.

[0046] Furthermore, the calculation formula of the internal parameter matrix K1 in steps 1 and 2 is:

[0047]

[0048] Among them (C x ,C y) represents the coordinates of the optical center of the image in the image pixel coordinates, dx and dy represent the size of the unit pixel on the x-axis and y-axis of the sensor respectively, and f is the focal length of the camera.

[0049] Furthermore, the calculation formula of the rotation matrix R in step 2 is:

[0050]

[0051] in, θ and ω are the pitch angle, roll angle and yaw angle of the image attitude parameters respectively.

[0052] The present invention utilizes the internal orientation elements of the aerial image, and directly uses the POS parameters of the aerial image as the external orientation elements of the image, and realizes the geometric correction of the pre-processed UAV remote sensing image through three-dimensional reconstruction space coordinate transformation and indirect image correction. The present invention makes full use of the pixel information of the original image, and uses multiple attitude parameters to participate in the correction at the same time, avoiding the correction error caused by the incoherent participation of multiple parameters in the traditional method. When it is impossible to select the artificial control point pair, the UAV aerial image is geometrically corrected, so that the image is more in line with the standard, and a corrected image with regularly arranged image points is obtained, which is convenient for subsequent aerial image registration and splicing and other processing.

[0053] The specific implementation process of the present invention is as follows:

[0054] The aerial photos of photovoltaic modules taken by UAV are selected, and the images with slightly larger angle offset are selected, as shown in Figure 3(a), and geometric correction is performed on them. The relevant image acquisition parameters are shown in Table 1.1:

[0055] Table 1.1 Image acquisition parameters

[0056]

[0057] The image correction process is as follows:

[0058] Step 1: Find the physical equivalent coordinates corresponding to the corrected image coordinates

[0059] The intersection of the camera optical axis and the image plane is usually located at the center of the image. The coordinates of the optical center of the image C are calculated as follows: x and C y :

[0060]

[0061] According to the size of the sensor, its width F can be found by looking up the table w 7.4mm, high F h is 5.6 mm, calculate the size d of the unit pixel on the x-axis and y-axis of the sensor x =Fw / W,d y =F h / H.

[0062] The intrinsic parameter matrix K1 is calculated according to formula (3), and the physically equivalent coordinates are calculated pixel by pixel according to formula (1).

[0063] Step 2: Find the image plane coordinates before correction corresponding to the physically equivalent coordinates

[0064] The pitch angle in the image attitude parameter Substituting the roll angle θ and the yaw angle ω into formula (4), we can calculate the rotation matrix R and the translation vector T = [0 0 0] -1 , the image plane coordinates before correction are calculated pixel by pixel according to formula (2).

[0065] Step 3: Calculate the corrected image

[0066] For each corrected coordinate (u', v'), the coordinate transformation relationship obtained in steps 1 and 2 is used to obtain the pixel coordinates (u, v) of the image before correction. The corrected image is resampled using bilinear interpolation to obtain the pixel value of the coordinate (u', v') in the corrected image. The result is shown in Figure 3(b). The image details are enlarged to obtain the detail comparison before and after correction, as shown in Figures 4(a) and 4(b). The shadow part of the component is covered by a mask of the same size. It can be seen from Figure 4(b) that the overlap rate of the shadow part of the corrected component and the mask is higher.

Claims

1. A geometric correction method based on the attitude information of UAV aerial images. It is characterized in that The steps include: Step 1: Find the physical equivalent coordinates corresponding to the corrected image coordinates Calculate the physical length of a unit pixel (d x ,d y ), according to the physical length and the camera focal length f, find the intrinsic parameter matrix K 1 There is no rotational or translational distortion between the corrected image point and the coordinates of the world coordinate system. The physical equivalent coordinates corresponding to the corrected image point (u', v') are (X, Y, Z). The physical equivalent coordinates refer to the coordinate values ​​of the physical coordinate system divided by the coordinates of the aircraft's flight altitude, which can be expressed as: Step 2: Find the plane coordinates of the image before correction corresponding to the physical equivalent coordinates According to the physical equivalent coordinates and flight attitude parameters obtained in step 1, the image plane coordinates before correction are calculated according to the following formula; Where R is the rotation matrix calculated based on the flight attitude parameters, and T is the translation vector; Step 3: Select a suitable resampling method to assign grayscale values ​​to the output image pixels; For each corrected coordinate (u', v'), the coordinate transformation relationship obtained in steps 1 and 2 is used to obtain the pixel coordinates (u, v) of the image before correction, and the mapping from the pixel coordinates of the corrected image to the pixel coordinates of the image before correction is completed; after the coordinate transformation is completed, the corrected image is resampled by interpolation method to obtain the pixel value of the coordinate (u', v') in the corrected image; The internal parameter matrix K described in steps 1 and 2 1 The calculation formula is Among them (C x ,C y ) represents the coordinates of the optical center of the image in the image pixel coordinates, dx and dy represent the size of the unit pixel on the x-axis and y-axis of the sensor respectively, and f is the focal length of the camera; The calculation formula of the rotation matrix R in step 2 is: in, θ and ω are the pitch angle, roll angle, and yaw angle of the image attitude parameters, respectively; The interpolation method includes nearest neighbor interpolation method, bilinear interpolation method and cubic convolution interpolation method; The implementation of the interpolation method is to take out the grayscale value of the original image point calculated by the above steps and fill it back to the corresponding pixel point in the blank output image dot matrix; because it is not necessarily located exactly at the center of a certain pixel in the original image, the grayscale value at the location must be determined through grayscale interpolation, and the resampling method is selected to assign grayscale to the output image pixels of the distorted image. By interpolating the pixels of the image before correction, the pixel values ​​of the corresponding coordinates of the corrected image are obtained, and the image correction can be completed.

Images