A method for geometric correction of hyperspectral pushbroom imaging of unmanned aerial vehicles

Abstract

The application discloses a kind of unmanned aerial vehicle hyperspectral push-broom imaging geometric correction methods, comprising the following steps: step 1, according to POS data, DEM data, using collinear condition equation obtains orthophoto, using Canny / Devernay subpixel edge detector, extract the edge point of corrected orthophoto;Step 2, using BM3D algorithm and median filter smooth orthophoto obtained in step 1, the image that will eliminate sawtooth distortion is used as reference image, using Canny / Devernay subpixel edge detector, extract the edge point of this image;Step 3, to the orthophoto obtained in step 1, select the area needing further correction, using the edge point extracted in step 1, 2 calculates the roll angle correction number, and corrects roll angle error;Step 4, the roll angle in POS data is replaced by the roll angle corrected in step 3, repeat step 1, and obtain the orthophoto that eliminates the distortion of roll angle error.The application can effectively correct the distortion of unmanned aerial vehicle line scanning hyperspectral image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a UAV hyperspectral push-broom imaging geometric correction method. BACKGROUND

[0002] The UAV hyperspectral image has the characteristics of high spatial resolution and high spectral resolution, and has high ground feature recognition and classification capability. The UAV has the advantages of flexibility, low cost, small interference from clouds, etc., and has been widely used in resource investigation and monitoring fields such as mineral resources, forestry, water quality, and agriculture.

[0003] Due to the lightness of the UAV platform and the low flight height, the UAV is easily affected by airflow and terrain, resulting in serious geometric distortion of the image, which adversely affects subsequent operations and applications. Therefore, geometric correction of the UAV hyperspectral image is one of the basic processes of application and processing. The traditional correction method relies on navigation data, but it is difficult to obtain accurate navigation data due to the measurement accuracy and frequency of the instrument itself. This results in that although the overall ground shape can be restored by using the traditional method, local distortion cannot be eliminated, and there is a "burr" phenomenon, which affects the application of the UAV hyperspectral image. How to realize high-precision geometric correction of the UAV hyperspectral scanning image is a problem to be solved at present. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a UAV hyperspectral push-broom imaging geometric correction method, which can better correct image distortion, especially the sawtooth distortion problem.

[0005] To solve the above technical problems, the present application provides a UAV hyperspectral push-broom imaging geometric correction method, comprising the following steps:

[0006] Step 1: According to the POS data and the DEM data, the orthographic image is obtained by using the collinear condition equation, and the Canny / Devernay sub-pixel edge detector is used to extract the edge points of the corrected orthographic image;

[0007] Step 2: The orthographic image obtained in step 1 is smoothed using the BM3D algorithm and the median filter, and the image without sawtooth distortion is used as a reference image, and the Canny / Devernay sub-pixel edge detector is used to extract the edge points of the image;

[0008] Step 3: The orthographic image obtained in step 1 is selected to further correct the area, and the edge points extracted in steps 1 and 2 are used to calculate the roll angle correction number to correct the roll angle error;

[0009] Step 4: The roll angle in the POS data is replaced by the roll angle corrected in step 3, and step 1 is repeated to obtain the orthographic image without roll angle error distortion.

[0010] Preferably, the step 1 of extracting edge points from the corrected orthographic image specifically comprises the following steps:

[0011] (1) Eliminate the non-uniform noise of the original image by using the single variational method, and then obtain the orthographic image according to the POS data and DEM data by using the collinearity condition equation;

[0012] (2) Calculate the mean and variance of different wavebands of the original image, and select three wavebands with the largest mean and variance for image fusion;

[0013] (3) Extract the sub-pixel edge points from the fused image by using the Canny / Devernay sub-pixel edge detector.

[0014] Preferably, in the step 1, the Canny / Devernay sub-pixel edge detector is used to extract the edge points of the corrected orthographic image, specifically: the Canny edge detection algorithm performs non-maximum suppression on the gradient amplitude, that is, finds the local gradient maximum value, so that the detected edge points are pixel-level accuracy, but the resolution of the orthographic image is within 1m, and sub-pixel level edge points are needed; the Devernay sub-pixel correction algorithm is used to obtain sub-pixel level edge points by calculating the gradient modulus of the adjacent three points in the gradient direction, and the calculation formula is as follows:

[0015]

[0016] Wherein, B is the edge point obtained by the non-maximum suppression of Canny, A and C are two adjacent points along the gradient direction of B. ‖g(A)‖, ‖g(B)‖, ‖g(C)‖ are the gradient amplitudes of A, B and C pixel points respectively, and η represents the maximum gradient amplitude between A, B and C, that is, the sub-pixel edge point.

[0017] Preferably, in the step 2, the specific steps of extracting edge points from the reference image are as follows:

[0018] (1) Perform BM3D smoothing on the fused image;

[0019] (2) Further smooth the smoothed image by using median filtering to eliminate the sawtooth, and obtain the reference image;

[0020] (3) Extract the sub-pixel edge points from the reference image by using the Canny / Devernay sub-pixel edge detector.

[0021] Preferably, in step 3, the orthographic image obtained in step 1 is selected for the area needing further correction, the rolling angle correction number is calculated by using the edge points extracted in steps 1 and 2 to correct the rolling angle error, and the specific correction is as follows: a local distortion area of the orthographic image is selected for further correction, the reference image edge point closest to the edge point of the orthographic image is searched according to the pixel points of the selected area, and the reference point is used to calculate the correction number of the rolling angle of the scanning line, and the calculation formula is as follows:

[0022]

[0023] Wherein, △Y represents the deviation of the ground projection coordinate in the Y direction caused by the rolling angle error, ω represents the rolling angle of the corresponding original scanning line, Ψ represents the field angle of the camera probe element, H represents the flight height, and Δω represents the rolling angle correction number of the scanning line.

[0024] The beneficial effects of the present application are as follows: (1) the rolling angle error correction method based on linear features proposed in the present application analyzes the image distortion caused by the attitude angle, corrects the rolling angle error, effectively corrects the image sawtooth distortion problem, restores the true shape of the target, has good robustness and practicability, and can be widely applied to the geometric correction of aerial hyperspectral push-broom images; (2) the geometric correction is performed according to the physical characteristics of the aerial hyperspectral push-broom image, so that the adaptability of the method of the present application to different aerial hyperspectral push-broom images is ensured. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 It is a method flow diagram of the present application.

[0026] Figure 2 It is a hyperspectral original image.

[0027] Figure 3 It is a geometric correction result diagram of the original geometric correction method of the present application.

[0028] Figure 4 It is a geometric correction result diagram of the present application.

[0029] Fig. 5(a) is a local result zoomed-in diagram of the geometric correction of the original geometric correction method of the present application.

[0030] Fig. 5(b) is a local result zoomed-in diagram of the geometric correction of the original geometric correction method of the present application.

[0031] Fig. 6(a) is a local result zoomed-in diagram of the geometric correction of the present application.

[0032] Fig. 6(b) is a local result zoomed-in diagram of the geometric correction of the present application. DETAILED DESCRIPTION

[0033] As Figure 1As shown, a UAV hyperspectral push-broom imaging geometric correction method comprises the following steps:

[0034] Step 1: According to the POS data and DEM data, the orthographic image is obtained by using the collinearity condition equation, and the Canny / Devernay sub-pixel edge detector is used to extract the edge points of the corrected orthographic image.

[0035] The image exterior orientation elements are calculated by using the POS data, and the calculation formula is as follows:

[0036]

[0037]

[0038] Wherein, m represents the geodetic coordinate system, E represents the geocentric coordinate system, g represents the navigation coordinate system, b represents the IMU coordinate system, c represents the sensor coordinate system, and i represents the image space coordinate system. Ψ represents the rotation matrix from the IMU coordinate system (b) to the navigation coordinate system (g), and Ψ, Θ and Φ represent the UAV attitude angles measured by the POS data. ω represents the rotation matrix from the image space coordinate system (i) to the geodetic coordinate system (m), and ω, κ represents the angular element of the image exterior orientation elements. represents the POS data measured at the imaging time of the UAV imaging center coordinates, represents the geocentric rectangular coordinates of the origin of the geodetic coordinate system, represents the linear element of the image exterior orientation elements.

[0039] The relationship between the original image point and the ground point is established by using the exterior orientation elements and the collinearity condition equation, and the corresponding ground point coordinates of the original image point are obtained by using the DEM data iteration, and the calculation formula is as follows:

[0040]

[0041] Wherein, represents the coordinates of the image point p in the image space coordinate system, and λ represents the scale factor, represents the rectangular coordinates of the image point p in the geodetic coordinate system.

[0042] Traverse each pixel point (i, j) of the original image, and the corresponding ground coordinates are (X, Y, Z), and the corresponding orthographic image point position is (x, y). Since (x, y) is generally not an integer, the gray value of the point is distributed to the surrounding four pixel points according to the weight.

[0043] The calculation formula of the corresponding orthographic image point position (x, y) is as follows:

[0044] x = (X-minX) / GSd

[0045] y = (Y - minY) / G = D

[0046] Wherein, X, Y are the corresponding ground coordinates (X, Y, ()) of the image point, minx, minY are the minimum values of X, Y, GSD is the ground resolution corresponding to the original image.

[0047] The pixel value calculation formula of the corresponding pixel point is:

[0048]

[0049]

[0050] Wherein, C(x,y) is the gray value of orthophoto point (x,y), P n is the original image pixel gray value of image (x,y) point, d n is the weight value of the corresponding original image point (x,y) pixel, sum is the weight sum.

[0051] Orthophoto sub-pixel edge points are extracted by Canny / Devernay.

[0052] The Canny edge detection algorithm performs non-maximum suppression on the gradient amplitude, that is, finds the local gradient maximum value, so that the detected edge points are pixel-level accuracy, but the orthophoto resolution is within 1m, which requires sub-pixel level edge points. Using Devernay sub-pixel correction algorithm, through calculating the gradient modulus of the adjacent three points in the gradient direction to perform quadratic function interpolation, sub-pixel level edge points are obtained, and the calculation formula is as follows:

[0053]

[0054] Wherein, B is the edge point obtained by non-maximum suppression of Canny, A and C are two points adjacent to B along the gradient direction.‖g(A)‖,‖g(B)‖,‖g(C)‖ are the gradient amplitudes of A, B and C pixel points respectively. η represents the maximum gradient amplitude among A, B and C, that is, the sub-pixel edge point.

[0055] Step 2: Smooth the orthophoto obtained in step 1 using BM3D algorithm and median filter, take the image without sawtooth distortion as the reference image, use Canny / Devernay sub-pixel edge detector to extract the edge points of the image.

[0056] Step 3: Select the area that needs to be further corrected for the orthophoto obtained in step 1, calculate the roll angle correction number using the edge points extracted in steps 1 and 2, and correct the roll angle error.

[0057] Select the local distortion area of ​​the orthophoto for further correction. Search the reference image edge point closest to the orthophoto edge point based on the pixel points in the selected area, and use it as the reference point to calculate the correction value of the roll angle of the scan line. The calculation formula is as follows:

[0058]

[0059] Where △Y represents the offset of the ground projection coordinate along the Y direction caused by the roll angle error, ω represents the roll angle corresponding to the original scan line, Ψ is the field of view of the camera detector, H is the flight altitude, and Δω is the roll angle correction value for the scan line.

[0060] Step 4: Replace the roll angle in the POS data with the roll angle corrected in step 3, and repeat step 1 to obtain an orthophoto with the roll angle error eliminated.

[0061] like Figure 2 As shown in the figure, the original hyperspectral push-broom image obtained by the UAV is severely distorted. Figure 3 The following figure shows the correction effect using the original geometric correction algorithm. Although the original geometric correction method restores the shape of the ground object to a large extent, due to the influence of the external environment during flight and the measurement error of the instrument itself, the POS data cannot accurately measure the flight attitude and position of the drone at the time of shooting. This results in local distortion after correction. As can be seen from Figure 5(a)-(b), the local aliasing distortion greatly affects the quality of geometric correction. Figure 4 Figures 6(a)-(b) are the processing results of the method of the present invention. The improved geometric correction method corrects the roll angle error through the extracted line feature information. The roll angle error is the main reason why the orthophoto still has local distortion after geometric correction. Compared with the original image, it can be found that the jagged distortion phenomenon of houses, rivers, etc. is significantly improved. The geometric correction effect is good, and the shape of the ground objects is restored, which facilitates the subsequent processing and application of hyperspectral images.

Claims

1. A method for geometric correction of hyperspectral pushbroom imaging of unmanned aerial vehicles, characterized in that, Comprising the following steps: Step 1, according to the POS data, DEM data, using the collinearity condition equation to obtain the orthographic image, using Canny / Devernay sub-pixel edge detector, extracting the edge points of the corrected orthographic image; Canny edge detection algorithm for non-maximum suppression of gradient amplitude, that is, finding the local gradient maximum, so that the detected edge points are pixel-level accuracy, but the orthographic image resolution is within 1m, which needs sub-pixel level edge points; Using Devernay sub-pixel correction algorithm, through calculating the gradient modulus of the adjacent three points in the gradient direction to carry out quadratic function interpolation, obtaining sub-pixel level edge points, the calculation formula is as follows: Wherein, B is the edge point obtained by non-maximum suppression of Canny, A and C are two points adjacent to B along the gradient direction, ||g(A)||, ||g(B)||, ||g(C)|| are the gradient amplitudes of A, B and C pixel points respectively, and η represents the maximum gradient amplitude among A, B and C, that is, the sub-pixel edge point; Step 2, using BM3D algorithm and median filter to smooth the orthographic image obtained in step 1, taking the image eliminating sawtooth distortion as a reference image, using Canny / Devernay sub-pixel edge detector to extract the edge points of the reference image; Step 3, selecting the area needing further correction from the orthographic image obtained in step 1, calculating the roll angle correction number by using the edge points extracted in steps 1 and 2, and correcting the roll angle error; Step 4, replacing the roll angle in the POS data with the corrected roll angle in step 3, repeating step 1 to obtain the orthographic image eliminating the distortion of roll angle error.

2. The UAV hyperspectral pushbroom geometric correction method of claim 1, wherein, In step 1, the edge points of the corrected orthographic image include the following steps: (1) using single variational method to eliminate the non-uniformity noise of the original image, and then obtaining the orthographic image according to the POS data and DEM data by using the collinearity condition equation; (2) calculating the mean and variance of different bands of the original image, and selecting three bands with the largest mean and variance for image fusion; (3) using Canny / Devernay sub-pixel edge detector to extract sub-pixel edge points from the fused image.

3. The UAV hyperspectral pushbroom geometric correction method of claim 1, wherein, In step 2, the specific steps of extracting the edge points of the reference image are as follows: (1) performing BM3D smoothing on the fused image; (2) further smoothing the smoothed image by using median filter to eliminate sawtooth, obtaining the reference image; (3) using Canny / Devernay sub-pixel edge detector to extract sub-pixel edge points from the reference image.

4. The UAV hyperspectral pushbroom geometric correction method of claim 1, wherein, In step 3, selecting the area needing further correction from the orthographic image obtained in step 1, calculating the roll angle correction number by using the edge points extracted in steps 1 and 2, and correcting the roll angle error, which is specifically: selecting the local distortion area of the orthographic image for further correction, searching for the nearest reference image edge point from the selected area pixel point to the edge point of the orthographic image as the reference point, and calculating the correction number of the scan line roll angle where the selected area pixel point is located, the calculation formula is as follows: Wherein, △Y represents the deviation of ground projection coordinate along Y direction caused by roll angle error, ω represents the roll angle of the original scan line, Ψ is the field angle of the camera probe element, H is the flight height, and Δω is the roll angle correction number of the scan line.

Images