A method for correcting distortion of push-broom images of light and small UAVs

Abstract

The present invention provides a method for correcting distortion of push-broom images of light and small UAVs, which relates to the technical field of UAV image correction, and includes the following steps: setting up an RTK base station at a known point and building a mobile workstation; acquiring the push-broom images of the UAV through a hyperspectral imager carried by the UAV gimbal; the UAV sends the recorded data of the carried POS and the push-broom image data to the mobile workstation; performing preliminary geometric correction through the collinear equation method; performing orthorectification in combination with DEM terrain data; and performing sub-pixel correction in combination with the SIFT algorithm and the FFT method. The present invention is highly efficient and robust to medium noise and lens optical blur, and can handle image distortion problems caused by rotation effects such as tilt, pitch and roll during flight.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle image correction, and in particular to a method for correcting distortion of push-broom images of a light and small unmanned aerial vehicle. Background Art

[0002] The hyperspectral push-broom imaging method is extremely sensitive to the attitude and position changes of the UAV. During the imaging process, due to the influence of aircraft movement and airflow, the acquired hyperspectral data has large geometric distortion. This distortion will seriously affect the quality of the image, and sometimes even make the image impossible to visually interpret, seriously affecting the normal application of the image.

[0003] In addition, during the acquisition process of drone images, changes in altitude and attitude angle, atmospheric refraction, earth curvature, terrain undulations, earth rotation, and the structural performance of the remote sensor itself will cause geometric deformation of the image. Geometric deformation causes the image to be distorted in geometric shape or position, which is mainly manifested in displacement, rotation, scaling, affine, bending and higher-order bending. The combination of the two may cause complex rotation and distortion problems in the images of light and small drones when performing hyperspectral push-broom imaging tasks.

[0004] In the existing technology, radiation correction and geometric correction are more commonly used for the correction of drone images. However, traditional radiation correction must clearly define the real-time atmospheric state parameters (such as water vapor and aerosol characteristics) at the moment of imaging and the appropriate radiation transfer equation to simulate the interaction between the atmosphere and electromagnetic radiation. These data are not easy to obtain, especially for many historical remote sensing images, which cannot be effectively implemented. Although geometric correction is more convenient to use, it cannot achieve a good correction effect for complex rotation and distortion problems. For applications that require high-precision push-broom imaging, such as high-precision positioning, image stitching, and multi-spectral band registration, it is difficult to achieve the required correction accuracy requirements. Summary of the invention

[0005] The present invention is made in view of the above problems, and its purpose is to provide a method for correcting the distortion and deformation of push-broom images of light and small unmanned aerial vehicles. The RTK base station is used to ensure that the unmanned aerial vehicle flies accurately along a predetermined route. The mobile workstation performs preliminary geometric correction of the push-broom image based on the recorded data of the POS on the unmanned aerial vehicle through the collinear equation method to obtain a high-spectral image with low geometric accuracy; then, the high-spectral image with low geometric accuracy is corrected through the orthorectification technology in combination with the DEM terrain data to obtain an orthophoto image; finally, the orthophoto image is corrected at the sub-pixel level in combination with the SIFT algorithm and the FFT method to obtain a corrected sub-pixel image, which is efficient and robust to medium noise and lens optical blur, and can handle image distortion problems caused by rotation effects such as tilt, pitch and roll during flight. It is particularly suitable for scenes that need to handle complex rotations and distortions, and can provide efficient sub-pixel accuracy.

[0006] Specifically, a first aspect of the present invention provides a method for correcting distortion of push-broom images of a light and small UAV, comprising the following steps:

[0007] Step 1: Set up an RTK base station at a known point, build a mobile workstation, set the working mode and RTK status of the RTK base station, and establish connections between the drone and the RTK base station and the mobile workstation respectively;

[0008] Step 2: After the connection is successful, the drone starts the push-sweep mission along the predetermined route based on RTK technology, and obtains the push-sweep image of the drone through the hyperspectral imager carried by the drone gimbal;

[0009] Step 3: The drone sends the recorded data and push-scan image data of the POS to the mobile workstation;

[0010] Step 4: Based on the recorded data of POS, the mobile workstation performs preliminary geometric correction of the push-scan image by the collinear equation method to obtain a hyperspectral image with low geometric accuracy;

[0011] Step 5: Correct the low geometric accuracy hyperspectral image by orthorectification technology in combination with DEM terrain data to obtain an orthorectified image;

[0012] Step 6: Combine the SIFT algorithm and FFT method to perform sub-pixel correction on the orthophoto image to obtain a corrected sub-pixel image.

[0013] Further, the setting of the working mode and RTK state of the RTK base station specifically includes: setting the working mode of the RTK base station to a reference value mode and the RTK state to a fixed solution state.

[0014] Furthermore, the drone establishes connections with the RTK base station and the mobile workstation respectively, specifically including: the drone is connected to the RTK base station through a custom network, and the drone and the mobile workstation are communicated through a radio and image transmission.

[0015] Furthermore, the UAV starts a push-broom mission along a predetermined route based on RTK technology, which specifically includes: the RTK base station sends positioning correction data to the GNSS receiver on the UAV in real time through a custom network, thereby eliminating satellite signal propagation errors, achieving high-precision positioning of the UAV in three-dimensional space, and ensuring that the UAV performs push-broom missions along the predetermined route.

[0016] Furthermore, the step 4 specifically includes:

[0017] The mobile workstation obtains the exterior orientation elements of the image based on the RTK-corrected position parameters and the UAV attitude data provided by the POS, obtains the transformation matrix of the image point and the object point based on the collinear equation method, and then performs preliminary correction of the image through several ground control points. The formula is as follows:

[0018] ;

[0019] ;

[0020] in:( , ) is the image point coordinates of the image point corresponding to the ground control point in the image;

[0021] ( , ) is the coordinate of the image principal point in the photo frame coordinate system;

[0022] is the vertical distance from the photographic center S to the image;

[0023] ( , , ) is the coordinate value of the ground control point in the ground space rectangular coordinate system;

[0024] ( , , ) is the coordinate value of the photography center S in the rectangular coordinate system of the ground space;

[0025] The 9 direction cosines are composed of the 3 exterior azimuth elements of the image.

[0026] Furthermore, the step five specifically includes:

[0027] Step 5.1: Select several ground control points on the image to obtain DEM terrain data that covers the image range and has a resolution matching the image;

[0028] Step 5.2: Use the camera model, reference markers, and image registration points to construct the interior orientation.

[0029] Step 5.3: Select ground control points and enter the corresponding map coordinates to relate points in the image to known ground locations to construct an external orientation;

[0030] Step 5.4: Combine the internal orientation, external orientation, RPC model and DEM file to correct the geometric distortion of the image using the orthorectification algorithm;

[0031] Step 5.5: Resample the image to meet the required spatial accuracy requirements.

[0032] Furthermore, the step 5.2 specifically includes:

[0033] Before using the camera, the focal length of the camera is calibrated, and the position of the principal point of the captured image is confirmed through the reference mark inside the camera. Based on the specific model of the camera, the internal orientation of the camera is completed through the pinhole camera model and the distortion model of the camera lens.

[0034] The camera model needs to be determined, which is the basis of interior orientation. The camera model includes the pinhole camera model and the distortion model. The pinhole camera model describes how light passes through the camera's lens system and forms an image on the sensor. The distortion model is used to correct lens distortion, including radial distortion and decentering distortion.

[0035] Reference markers are an important tool for determining the internal orientation of a camera. They are usually fixed inside the camera and used to measure the position of the principal point. The principal point is the origin of the image coordinate system and its position is determined by measuring the pixel coordinates of the calibration markers. The position and coordinates of these reference markers need to be accurately measured and recorded during the camera calibration process.

[0036] Determine the position of the principal point: The position of the principal point is one of the key parameters for interior orientation. By measuring the pixel coordinates of the calibration markers, the position of the principal point can be determined.

[0037] Corrected focal length: Focal length is another important internal orientation parameter that determines the scale of the image. Focal length is usually derived through the camera calibration process and stored in the camera information file.

[0038] The purpose of internal orientation is to eliminate the effects of lens distortion, sensor imperfections and other factors on the image through a correction process to ensure the accuracy and clarity of the image.

[0039] Furthermore, the step 5.3 specifically includes:

[0040] Select the target ground control point, measure the image coordinates corresponding to the target control point, associate the points in the image with the known ground control point positions by clicking pixels on the image and entering the corresponding map coordinates, and then use the collinearity equation to perform external orientation calculations to complete external orientation.

[0041] Select target control points: First, select the target ground control points and determine the rough image positions corresponding to the target control points so that the sub-image can be displayed in the measurement window. Measure image coordinates: Measure the image coordinates corresponding to the target control points. Enter map coordinates: Select ground control points and enter the corresponding map coordinates to associate points in the image with known ground locations. This step is done by clicking on pixels on the aerial image and entering the corresponding map coordinates. Calculate extrinsic orientation parameters: Extrinsic orientation calculations are performed using collinearity equations. The extrinsic orientation parameters include the six extrinsic elements of the camera, which are determined using a mathematical model.

[0042] Furthermore, the step 5.4 specifically includes:

[0043] First, the size and geographic range of the output orthophoto are determined according to the RPC model, DEM data, target projection and output resolution, and an empty orthophoto is created; then, the image plane coordinates of each pixel in the orthophoto corresponding to the image to be corrected are calculated according to the RPC model and DEM data, and the grayscale values ​​are interpolated from the image to be corrected using the bilinear interpolation method based on the coordinates and assigned to the orthophoto pixels.

[0044] Furthermore, the step six specifically includes:

[0045] The SIFT algorithm is used to extract key points in the image and describe them to obtain a descriptor containing size and main gradient direction information, thereby achieving image matching. After the SIFT algorithm completes the image correction, the FFT method is used to reduce the dimension of the cross-correlation Fourier matrix and use forward and backward search to obtain accurate sub-pixel displacement estimation, thereby improving the correction accuracy of the image and ultimately obtaining a sub-pixel corrected image.

[0046] First, the images taken by the drone are corrected using the SIFT algorithm. The SIFT algorithm can effectively extract and match feature points in images, and maintain good performance even in the presence of geometric and photometric distortions. The SIFT algorithm achieves image matching by extracting key points in the image and describing them to obtain descriptors containing size and main gradient direction information. This step aims to correct image distortions caused by rotational effects such as tilt, pitch, and roll during flight. Sub-pixel correction using the FFT method: After the SIFT algorithm completes the image correction, the FFT method is used to further correct the images that do not achieve sub-pixel accuracy. The FFT method reduces the dimension of the Fourier matrix and uses forward and backward searches to improve efficiency and robustness, thereby accurately estimating sub-pixel displacements.

[0047] In a second aspect, the present invention further provides a computing device, which has the function of implementing the method described in the first aspect above. The beneficial effects can be found in the description of the first aspect and will not be repeated here. The functions can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0048] In a third aspect, the present invention further provides a computing device, which is used to implement the functions of the method described in the first aspect above. The beneficial effects can be found in the description of the first aspect and will not be repeated here. The structure of the computing device includes a processor and a memory, and the memory is used to store instructions and / or data. The memory is coupled to the processor, and the processor executes the program instructions stored in the memory. The structure of the computing device also includes a communication interface for communicating with other devices.

[0049] In a fourth aspect, the present invention further provides a computer-readable storage medium, in which instructions are stored, and when the computer-readable storage medium is run on a computer, the computer executes the method in the above-mentioned first aspect and various possible designs of the first aspect.

[0050] In a fifth aspect, the present invention further provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method in the above-mentioned first aspect and various possible designs of the first aspect.

[0051] In a sixth aspect, the present invention further provides a computing chip, which is connected to a memory, and is used to read and execute a software program stored in the memory, and to execute the methods in the above-mentioned first aspect and various possible implementations of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present drawings or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present drawings. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0053] Figure 1 is a flow chart of the steps of the present invention;

[0054] Figure 2 This is a comparison of push-broom images before and after orthorectification;

[0055] Figure 3 This is a magnified comparison of push-broom image details before and after orthorectification;

[0056] Figure 4 Comparison of push-broom images before and after sub-pixel correction.

[0057] The purpose, features and advantages of this figure will be further described in conjunction with the embodiments with reference to the accompanying drawings. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments provided by the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0059] Obviously, the drawings described below are only some examples or embodiments of the present invention. For ordinary technicians in this field, the present invention can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed by the present invention, some changes in design, manufacturing or production based on the technical content disclosed by the present invention are just conventional technical means, and should not be understood as insufficient content disclosed by the present invention.

[0060] If not otherwise specified, all embodiments and optional embodiments of the present invention can be combined with each other to form a new technical solution.

[0061] Unless otherwise specified, all technical features and optional technical features of the present invention can be combined with each other to form a new technical solution.

[0062] If not otherwise specified, all steps of the present invention may be performed sequentially or randomly, preferably sequentially. For example, the method includes steps (a) and (b), which means that the method may include steps (a) and (b) performed sequentially, or may include steps (b) and (a) performed sequentially. For example, the method may further include step (c), which means that step (c) may be added to the method in any order, for example, the method may include steps (a), (b) and (c), or may include steps (a), (c) and (b), or may include steps (c), (a) and (b), etc.

[0063] If there is no special explanation, the "include" and "comprising" mentioned in the present invention represent open-ended or closed-ended expressions. For example, the "include" and "comprising" may represent that other components not listed may also be included or only the listed components may be included or only the listed components may be included.

[0064] If not specifically stated, in the present invention, the term "or" is inclusive. For example, the phrase "A or B" means "A, B, or both A and B". More specifically, any of the following conditions satisfies the condition "A or B": A is true (or exists) and B is false (or does not exist); A is false (or does not exist) and B is true (or exists); or both A and B are true (or exist).

[0065] In order to better understand the solutions of the embodiments of the present invention, some relevant terms and concepts that may be involved in the embodiments of the present invention are first introduced below.

[0066] (1) RTK (Real-Time Kinematic): RTK is a high-precision satellite navigation technology that improves positioning accuracy by using a fixed base station and a mobile receiver (rover) to transmit correction data in real time. The base station receives and calculates the correction parameters of the satellite signal, and then transmits these parameters to the mobile receiver through a data link, thereby achieving centimeter-level positioning accuracy.

[0067] (2) POS (Positioning and Orientation System): POS system is usually used to measure the position and orientation of a device. It combines multiple sensor technologies, such as inertial measurement unit (IMU), GPS and other sensors to provide accurate position and orientation information. POS system is widely used in fields such as autonomous vehicles, robot navigation and aerial photography.

[0068] (3) DEM (Digital Elevation Model): DEM is a digital elevation model used to represent the height information of terrain. It is usually stored in raster form, with each pixel representing the height value of the terrain. DEM is very important in geographic information systems (GIS), remote sensing image processing and terrain analysis, and is often used for orthorectification and terrain modeling.

[0069] (4) Ortho-rectification: Ortho-rectification is the process of correcting the geometric distortion in remote sensing images to map projection. This process uses DEM data to eliminate image distortion caused by camera geometry, elevation angle and terrain to generate a geometrically correct image. Ortho-rectification is widely used in aerial photography, satellite image processing and geographic information systems.

[0070] (5) SIFT (Scale-Invariant Feature Transform): SIFT is a feature detection algorithm used in image processing that can identify and match image features at different scales. The SIFT algorithm detects key points in an image and extracts descriptors to achieve feature matching. It is widely used in image recognition, 3D reconstruction, and robot vision.

[0071] (6) FFT (Fast Fourier Transform): FFT is a fast Fourier transform algorithm used to efficiently calculate discrete Fourier transform (DFT). FFT converts time domain signals into frequency domain signals and is widely used in signal processing, image processing, audio analysis and communication systems. FFT can significantly improve the calculation speed of Fourier transform and is an important tool in modern digital signal processing.

[0072] In this embodiment, Figure 1 As shown, a method for correcting distortion of push-broom images of a small light UAV comprises the following steps:

[0073] Step 1: Set up an RTK base station at a known point, build a mobile workstation, set the working mode and RTK status of the RTK base station, and establish connections between the drone and the RTK base station and the mobile workstation respectively;

[0074] Step 2: After the connection is successful, the drone starts the push-sweep mission along the predetermined route based on RTK technology, and obtains the push-sweep image of the drone through the hyperspectral imager carried by the drone gimbal;

[0075] Step 3: The drone sends the recorded data and push-scan image data of the POS to the mobile workstation;

[0076] Step 4: Based on the recorded data of POS, the mobile workstation performs preliminary geometric correction of the push-scan image by the collinear equation method to obtain a hyperspectral image with low geometric accuracy;

[0077] Step 5: Combine the DEM terrain data and use the orthorectification technology to correct the low geometric accuracy hyperspectral image to obtain the orthophoto image;

[0078] Step 6: Combine the SIFT algorithm and FFT method to perform sub-pixel correction on the orthophoto image to obtain a corrected sub-pixel image.

[0079] RTK technology can provide accurate location information in real time, and the drone can automatically adjust its route during flight to ensure that the captured images have high overlap and high-quality geographic information metadata. This enables drone photogrammetry to complete large-area surveying tasks in a shorter time, improving work efficiency.

[0080] Use POS data to perform preliminary geometric correction to eliminate image distortion caused by changes in the UAV's flight attitude. This step is usually achieved by using the collinear equation method by calculating the transformation matrix between image points and object points. In addition, the row correlation matching and POS correction methods can be combined to further reduce geometric distortion.

[0081] Orthorectification is the process of resampling an image into an orthophoto. This process usually requires the use of digital elevation model (DEM) data to correct the image for tilt and projection errors, thereby eliminating the impact of terrain undulations and making the image proportions uniform. The specific operation includes selecting ground control points and using these control points to geometrically align and resample the image.

[0082] Furthermore, setting the working mode and RTK state of the RTK base station specifically includes: setting the working mode of the RTK base station to a reference value mode and the RTK state to a fixed solution state.

[0083] Furthermore, the UAV establishes connections with the RTK base station and the mobile workstation respectively, specifically including: the UAV and the RTK base station are connected through a custom network, and the UAV and the mobile workstation are communicated through a radio and image transmission.

[0084] Furthermore, the UAV starts the push-broom mission along a predetermined route based on RTK technology, which includes: the RTK base station sends positioning correction data to the GNSS receiver on the UAV in real time through a custom network, thereby eliminating satellite signal propagation errors, achieving high-precision positioning of the UAV in three-dimensional space, and ensuring that the UAV performs push-broom missions along the predetermined route.

[0085] Furthermore, step four specifically includes:

[0086] The mobile workstation obtains the exterior orientation elements of the image based on the RTK-corrected position parameters and the UAV attitude data provided by the POS, obtains the transformation matrix of the image point and the object point based on the collinear equation method, and then performs preliminary correction of the image through several ground control points. The formula is as follows:

[0087] ;

[0088] ;

[0089] Furthermore, step five specifically includes:

[0090] Step 5.1: Select several ground control points on the image to obtain DEM terrain data that covers the image range and has a resolution matching the image;

[0091] Step 5.2: Use the camera model, reference markers, and image registration points to construct the interior orientation.

[0092] Step 5.3: Select ground control points and enter the corresponding map coordinates to relate points in the image to known ground locations to construct an external orientation;

[0093] Step 5.4: Combine the internal orientation, external orientation, RPC model and DEM file to correct the geometric distortion of the image using the orthorectification algorithm;

[0094] Step 5.5: Resample the image to meet the required spatial accuracy requirements.

[0095] The image correction results of step 5 are compared as follows: Figure 2 As shown, the left side is the image before correction, and the right side is the image after correction. The details in the picture are enlarged and compared as shown in Figure 3 As shown in the figure, it can be seen that the obvious distortion and deformation in the corrected image are basically corrected.

[0096] Further, step 5.2 specifically includes:

[0097] Before using the camera, the focal length of the camera is calibrated, and the position of the principal point of the captured image is confirmed through the reference mark inside the camera. Based on the specific model of the camera, the internal orientation of the camera is completed through the pinhole camera model and the distortion model of the camera lens.

[0098] The camera model needs to be determined, which is the basis of interior orientation. The camera model includes the pinhole camera model and the distortion model. The pinhole camera model describes how light passes through the camera's lens system and forms an image on the sensor. The distortion model is used to correct lens distortion, including radial distortion and decentering distortion.

[0099] Reference markers are an important tool for determining the internal orientation of a camera. They are usually fixed inside the camera and used to measure the position of the principal point. The principal point is the origin of the image coordinate system and its position is determined by measuring the pixel coordinates of the calibration markers. The position and coordinates of these reference markers need to be accurately measured and recorded during the camera calibration process.

[0100] Determine the position of the principal point: The position of the principal point is one of the key parameters for interior orientation. By measuring the pixel coordinates of the calibration markers, the position of the principal point can be determined.

[0101] Corrected focal length: Focal length is another important internal orientation parameter that determines the scale of the image. Focal length is usually derived through the camera calibration process and stored in the camera information file.

[0102] The purpose of internal orientation is to eliminate the effects of lens distortion, sensor imperfections and other factors on the image through a correction process to ensure the accuracy and clarity of the image.

[0103] Furthermore, step 5.3 specifically includes:

[0104] Select the target ground control point, measure the image coordinates corresponding to the target control point, associate the points in the image with the known ground control point positions by clicking pixels on the image and entering the corresponding map coordinates, and then use the collinearity equation to perform external orientation calculations to complete external orientation.

[0105] Select the target control point: First, select the target ground control point and determine the rough image position corresponding to the target control point so that the sub-image can be displayed in the measurement window. Measure image coordinates: Measure the image coordinates corresponding to the target control point. Enter map coordinates: Select the ground control point and enter the corresponding map coordinates to associate the point in the image with the known ground position. This step is done by clicking on the pixel on the aerial image and entering the corresponding map coordinates. Calculate external orientation parameters: Use the collinearity equation to perform external orientation calculations. The external orientation parameters include the six external elements of the camera. These parameters can be determined by a mathematical model. The standard central perspective model is used in this embodiment.

[0106] Furthermore, step 5.4 specifically includes:

[0107] First, the size and geographic range of the output orthophoto are determined according to the RPC model, DEM data, target projection and output resolution, and an empty orthophoto is created; then, the image plane coordinates of each pixel in the orthophoto corresponding to the image to be corrected are calculated according to the RPC model and DEM data, and the grayscale values ​​are interpolated from the image to be corrected using the bilinear interpolation method based on the coordinates and assigned to the orthophoto pixels.

[0108] In this embodiment, bilinear interpolation is used for interpolation to ensure that the grayscale value of the orthophoto does not undergo nonlinear changes. The use of bilinear interpolation can effectively ensure image quality and avoid errors caused by nonlinear changes caused by interpolation, so as to facilitate subsequent sub-pixel correction processing.

[0109] Internal Orientation: First, the internal orientation of the camera needs to be established. This includes determining the camera's principal point position, focal length, and fiducial markers. These parameters are the basis for describing the camera's internal geometry and ensure that pixels in the image can be accurately mapped to the camera's imaging plane.

[0110] External Orientation: Next, external orientation is required, which determines the relationship between the image and the earth's surface by selecting ground control points and entering their corresponding map coordinates. These control points are used to relate points in the image to known ground locations, ensuring that the image can be correctly mapped into the geographic coordinate system.

[0111] Application of DEM files: After completing the internal and external orientation, use the DEM file for orthorectification. DEM provides terrain height information, which can be used to correct the projection difference and tilt correction caused by terrain undulations. Specifically, through DEM data, the actual height of each pixel on the terrain can be calculated, and the position of the pixel can be adjusted accordingly to meet the requirements of ortho projection.

[0112] Furthermore, step six specifically includes:

[0113] The SIFT algorithm is used to extract key points in the image and describe them to obtain a descriptor containing size and main gradient direction information, thereby achieving image matching. After the SIFT algorithm completes the image correction, the FFT method is used to reduce the dimension of the cross-correlation Fourier matrix and use forward and backward search to obtain accurate sub-pixel displacement estimation, thereby improving the correction accuracy of the image and ultimately obtaining a sub-pixel corrected image.

[0114] First, the images taken by the drone are corrected using the SIFT algorithm. The SIFT algorithm can effectively extract and match feature points in images, and maintain good performance even in the presence of geometric and photometric distortions. The SIFT algorithm achieves image matching by extracting key points in the image and describing them to obtain descriptors containing size and main gradient direction information. This step aims to correct image distortions caused by rotational effects such as tilt, pitch, and roll during flight. Sub-pixel correction using the FFT method: After the SIFT algorithm completes the image correction, the FFT method is used to further correct the images that do not achieve sub-pixel accuracy. The FFT method reduces the dimension of the Fourier matrix and uses forward and backward searches to improve efficiency and robustness, thereby accurately estimating sub-pixel displacements.

[0115] The image correction results of step six are compared as follows Figure 4 As shown, the left side is the image before correction, and the right side is the image after correction. It can be seen that there are still distortion and blurring problems in the lower half of the image before correction. The correction accuracy of the image after correction is greatly improved, meeting the required push-scan image accuracy requirements.

[0116] It should be noted that the present invention is not limited to the above-mentioned embodiments. The above-mentioned embodiments are only examples, and the embodiments having the same structure as the technical idea and exerting the same effect within the scope of the technical solution of the present invention are all included in the technical scope of the present invention. In addition, without departing from the scope of the main purpose of the present invention, various modifications that can be thought of by those skilled in the art to the embodiments and other methods of combining some of the constituent elements in the embodiments are also included in the scope of the present invention.

Claims

1. A method for correcting distortion of push-broom images of a small light UAV, characterized in that: The following steps are involved: Step 1: Set up an RTK base station at a known point, build a mobile workstation, set the working mode and RTK status of the RTK base station, and establish connections between the drone and the RTK base station and the mobile workstation respectively; Step 2: After the connection is successful, the drone starts the push-sweep mission along the predetermined route based on RTK technology, and obtains the push-sweep image of the drone through the hyperspectral imager carried by the drone gimbal; Step 3: The drone sends the recorded data and push-scan image data of the POS to the mobile workstation; Step 4: Based on the recorded data of POS, the mobile workstation performs preliminary geometric correction of the push-scan image by the collinear equation method to obtain a hyperspectral image with low geometric accuracy; Step 5: Correct the low geometric accuracy hyperspectral image by orthorectification technology in combination with DEM terrain data to obtain an orthorectified image; Step 6: Combine the SIFT algorithm and FFT method to perform sub-pixel correction on the orthophoto image to obtain a corrected sub-pixel image.

2. According to claim 1, a method for correcting distortion of push-broom images of a small light UAV is characterized in that: The setting of the working mode and RTK state of the RTK base station specifically includes: setting the working mode of the RTK base station to a reference value mode and the RTK state to a fixed solution state.

3. The method for correcting distortion of push-broom images of a small light UAV according to claim 1 is characterized in that: The drone establishes connections with the RTK base station and the mobile workstation respectively, specifically including: the drone is connected to the RTK base station through a custom network, and the drone and the mobile workstation are communicated through a radio and image transmission.

4. The method for correcting distortion of push-broom images of a small light UAV according to claim 1, characterized in that: The UAV starts a push-broom mission along a predetermined route based on RTK technology, which specifically includes: the RTK base station sends positioning correction data to the GNSS receiver on the UAV in real time through a custom network, thereby eliminating satellite signal propagation errors, achieving high-precision positioning of the UAV in three-dimensional space, and ensuring that the UAV performs push-broom missions along the predetermined route.

5. The method for correcting distortion of push-broom images of a small light UAV according to claim 1, characterized in that: The step 4 specifically includes: The mobile workstation obtains the exterior orientation elements of the image based on the RTK-corrected position parameters and the UAV attitude data provided by the POS, obtains the transformation matrix of the image point and the object point based on the collinear equation method, and then performs preliminary correction of the image through several ground control points. The formula is as follows: ； ； in:( , ) is the image point coordinates of the image point corresponding to the ground control point in the image; ( , ) is the coordinate of the image principal point in the photo frame coordinate system; is the vertical distance from the photographic center S to the image; ( , , ) is the coordinate value of the ground control point in the ground space rectangular coordinate system; ( , , ) is the coordinate value of the photography center S in the rectangular coordinate system of the ground space; The 9 direction cosines are composed of the 3 exterior azimuth elements of the image.

6. The method for correcting distortion of push-broom images of a small light UAV according to claim 1, characterized in that: The step five specifically includes: Step 5.1: Select several ground control points on the image to obtain DEM terrain data that covers the image range and has a resolution matching the image; Step 5.2: Use the camera model, reference markers, and image registration points to construct the interior orientation. Step 5.3: Select ground control points and enter the corresponding map coordinates to relate points in the image to known ground locations to construct an external orientation; Step 5.4: Combine the internal orientation, external orientation, RPC model and DEM file to correct the geometric distortion of the image using the orthorectification algorithm; Step 5.5: Resample the image to meet the required spatial accuracy requirements.

7. The method for correcting distortion of push-broom images of a small light UAV according to claim 6, characterized in that: The step 5.2 specifically includes: Before using the camera, the focal length of the camera is calibrated, and the position of the principal point of the captured image is confirmed through the reference mark inside the camera. Based on the specific model of the camera, the internal orientation of the camera is completed through the pinhole camera model and the distortion model of the camera lens.

8. The method for correcting distortion of push-broom images of a small light UAV according to claim 6, characterized in that: The step 5.3 specifically includes: Select the target ground control point, measure the image coordinates corresponding to the target control point, associate the points in the image with the known ground control point positions by clicking pixels on the image and entering the corresponding map coordinates, and then use the collinearity equation to perform external orientation calculations to complete external orientation.

9. The method for correcting distortion of push-broom images of a small light UAV according to claim 6, characterized in that: The step 5.4 specifically includes: First, the size and geographic range of the output orthophoto are determined according to the RPC model, DEM data, target projection and output resolution, and an empty orthophoto is created; then, the image plane coordinates of each pixel in the orthophoto corresponding to the image to be corrected are calculated according to the RPC model and DEM data, and the grayscale values ​​are interpolated from the image to be corrected using the bilinear interpolation method based on the coordinates and assigned to the orthophoto pixels.

10. The method for correcting distortion of push-broom images of a small light UAV according to claim 1, characterized in that: The step six specifically includes: The SIFT algorithm is used to extract key points in the image and describe them to obtain a descriptor containing size and main gradient direction information, thereby achieving image matching. After the SIFT algorithm completes the image correction, the FFT method is used to reduce the dimension of the cross-correlation Fourier matrix and use forward and backward search to obtain accurate sub-pixel displacement estimation, thereby improving the correction accuracy of the image and ultimately obtaining a sub-pixel corrected image.

Images