A geocoding method for UAV SAR images using optical orthoimages

By replacing the manual acquisition control points, the geographic coding problem of low geocoding efficiency of the drone SAR image and optical orthophotographs, the geographic coordinates of DEM data interpolation are solved, and efficient and automated Geocode process.

CN114004870BActive Publication Date: 2024-12-03Chinese People's Liberation Army Cyberspace Force Information Engineering University
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202111183465.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-11
Publication Date
2024-12-03
Estimated Expiration
2041-10-11

AI Technical Summary

Technical Problem

The low geocoding efficiency of drone SAR images is mainly due to the time-consuming and labor-intensive collection of control points in manual acquisition, which affects efficient and automated image production.

Method used

By acquiring the matching point pairs of the drone SAR image and the optical orthophoto image, using DEM data to interpolate geographical coordinates as control points, refine the geometric conformation model of the SAR image and perform geocoding, replacing the method of manually obtaining control points .

Benefits of technology

It significantly improves the efficiency of SAR image geocoding of drones, reduces the time to manually acquire control points, and improves the accuracy and automation of images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114004870B_ABST
    Figure CN114004870B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for geocoding unmanned aerial vehicle SAR images using optical orthophoto images, and belongs to the technical field of remote sensing image processing. The present invention firstly adopts a method based on rank self-similar features to realize accurate extraction of matching point pairs of optical orthophoto images and unmanned aerial vehicle SAR images; secondly, the geographic coordinates of the matching points are interpolated on the DEM corresponding to the optical orthophoto image, and the matching points of the geographic coordinates are obtained as control points; then the extracted control points are used to refine the SAR image range Doppler geometric image model; finally, the refined range Doppler model and the corresponding DEM data are used to geocode the unmanned aerial vehicle SAR images. The present invention uses the existing optical orthophoto images to extract a large number of control points required for refining the SAR image geometric image model, avoiding the time-consuming and labor-intensive manual collection process of the control points, and greatly improving the efficiency of the geocoding of unmanned aerial vehicle SAR images.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a geocoding method for unmanned aerial vehicle SAR images using optical orthophoto images, and belongs to the technical field of remote sensing image processing. Background Art

[0002] Synthetic Aperture Radar (SAR) is an active imaging sensor that can obtain high-resolution images all day and all weather. The penetration and polarization characteristics of its microwaves make SAR images contain richer target information, making SAR images widely used in remote sensing mapping fields such as terrain surveying, geological exploration, ocean monitoring, and military target identification. With the increasing application of SAR, the performance and functions of SAR systems are also constantly improving and the size is developing towards miniaturization, which makes it possible to use unmanned aerial vehicle platforms to carry SAR systems. The unmanned aerial vehicle SAR system combines the advantages of unmanned aerial vehicles and SAR, making the acquisition of microwave remote sensing information more flexible and cheaper.

[0003] Different from optical images, SAR images are acquired through side-view imaging of range projection. Affected by the undulating terrain, SAR images will have geometric distortions such as overlapping, shadowing, perspective contraction and vertex displacement, which are not conducive to the interpretation and interpretation of images. Therefore, SAR images need to be geocoded. SAR image geocoding is the process of correcting SAR images in the image coordinate system to the geographic coordinate system. In order to realize the geocoding of UAV SAR images, it is first necessary to collect control points to refine the geometric image model of SAR images, and then use the refined geometric image model and external digital elevation model (DEM) to correct the SAR image to the geographic coordinate system. Collecting control points usually requires manual setting of corner reflectors and field measurement of geographic coordinates. This process is time-consuming and labor-intensive, and is not conducive to the efficient and automated production of UAV SAR orthophoto images. Therefore, how to improve the efficiency of control point collection or how to collect high-precision control points by other means is the key to solving this problem. Summary of the invention

[0004] The purpose of the present invention is to provide a method for geocoding UAV SAR images using optical orthoimages, so as to solve the problem of low efficiency of geocoding UAV SAR images caused by manually collecting control points.

[0005] The present invention provides a method for geocoding unmanned aerial vehicle SAR images using optical orthophoto images, the method comprising the following steps:

[0006] 1) Obtain the UAV SAR image and the corresponding optical orthophoto image, extract feature points from the optical orthophoto image, and use the extracted feature points to search for matching points on the UAV SAR image to obtain matching point pairs;

[0007] 2) obtaining DEM data corresponding to the optical orthoimage, interpolating the geographic coordinates of the matching points on the DEM, and using the matching points with the obtained geographic coordinates as control points;

[0008] 3) Refining the SAR image range Doppler geometric imaging model using the extracted control points to obtain a refined SAR image geometric imaging model;

[0009] 4) Geocoding the UAV SAR image using the geometric configuration model and the DEM data.

[0010] The present invention obtains matching point pairs by matching UAV SAR images and corresponding optical orthoimages, and uses the matching points with geographic coordinates interpolated on the DEM as control points for refining the geometric image model, effectively replacing the method of relying on manual acquisition of control points and greatly improving the efficiency of UAV SAR image geocoding.

[0011] Furthermore, in order to accurately extract the feature points of the optical orthophoto image, in the step 1), when extracting the feature points of the optical orthophoto image, a Harris corner point detector is used to detect and screen the feature points, the optical orthophoto image is divided into blocks, the feature response values ​​of the Harris feature points extracted from each image block are sorted, and k points with the largest feature response in each image block are selected as feature points.

[0012] Furthermore, the step 1) uses a method based on rank self-similarity features to achieve matching between optical orthoimages and UAV SAR images.

[0013] Furthermore, the image matching process in step 1) is as follows:

[0014] a. For each extracted feature point, a dense rank-based self-similarity (DRSS) descriptor is extracted within the template window centered on it;

[0015] b. Roughly geocode the UAV SAR image based on the known navigation information to determine the estimated location of the feature points in the SAR image;

[0016] c. Calculate the DRSS descriptor pixel by pixel within the local search window at the estimated position, and calculate the similarity measure between the DRSS descriptor and the feature point;

[0017] d. Find the maximum similarity measure in the local search window and determine the matching point according to the position of the maximum value.

[0018] Furthermore, the calculation formula of the DRSS descriptor similarity measure in step c of the image matching process is:

[0019]

[0020] Among them, V1 and V2 are the DRSS descriptor vector values ​​on the reference image and the image to be registered. and are the average values ​​of V1 and V2 respectively.

[0021] Furthermore, after obtaining the matching point pairs in step 1), the obtained matching point pairs are screened to eliminate the incorrect matching point pairs. The process is as follows: all matching point pairs are used to estimate the parameters of the projection transformation model between the two images; the residual of each matching point pair and the RMSE of all matching point pairs are calculated according to the estimated model, and the points with the largest residuals are eliminated until the RMSE of all matching point pairs is less than a given threshold.

[0022] Furthermore, in the step 2), when the geographic coordinates of the matching points are interpolated on the DEM corresponding to the optical orthoimage, it is necessary to select the matching points on flat ground as control points. The process is as follows: the elevation values ​​corresponding to all image points in the local neighborhood of the matching point are obtained, and the standard deviation of these elevation values ​​is calculated. If the standard deviation is less than a certain threshold, the matching point is considered to be a control point.

[0023] Furthermore, the refinement process of the distance Doppler geometric imaging model in step 3) is as follows: set the initial value of the directional parameter, substitute the control point data into the linearized RD model to obtain the error equation, solve the directional parameter correction number by the error equation method, and then use the solved correction number to correct the initial value of the directional parameter until the given convergence condition is met to output the directional parameter value.

[0024] Furthermore, the RD geometric model formula in step 3) is as follows:

[0025]

[0026] Among them, (X, Y, Z) are the object space coordinates of the ground point, x and y are the distance coordinates and azimuth coordinates of the image point corresponding to the ground point, respectively, M slant is the slant range sampling interval, R0 is the close range delay, f dc is the Doppler center frequency, λ is the radar wavelength, P S =[X S Y S Z S ] Tis the instantaneous position vector of the antenna phase center S, V = [V X V Y V Z ] T is the instantaneous velocity vector of the antenna phase center S. If the platform moves in a uniform straight line, then:

[0027]

[0028] in, is the initial position vector. t is P S relatively Imaging time, t = K y, K is the time interval between two adjacent lines of image sampling.

[0029] Furthermore, in the step 4), the refined RD geometric imaging model and the corresponding DEM are used to perform geocoding of the SAR image, and the process is as follows: the object area corresponding to the UAV SAR image is gridded according to the encoded sampling interval, and the elevation value is interpolated in the DEM to obtain the three-dimensional coordinates of each grid point, and the refined RD geometric imaging model is used to calculate the image point coordinates corresponding to each grid point, so as to realize the geocoding of the UAV SAR image. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a flow chart of a method for geocoding UAV SAR images using optical orthoimages of the present invention;

[0031] Figure 2 It is a flow chart of DRSS descriptor construction of the present invention;

[0032] Figure 3 is the original UAV SAR image in the experiment of the present invention;

[0033] FIG4( a ) is a diagram showing matching points on an optical orthoimage in an experiment of the present invention;

[0034] FIG4( b ) is a diagram showing the matching points on the UAV SAR image in the experiment of the present invention;

[0035] Figure 5 It is the UAV SAR image after geocoding in the experiment of the present invention. DETAILED DESCRIPTION

[0036] The specific implementation of the present invention will be further described below in conjunction with the accompanying drawings.

[0037] The present invention provides a method for geocoding unmanned aerial vehicle SAR images using optical orthophoto images, such as Figure 1As shown in the figure, the method mainly includes the following four steps: extraction of precise matching point pairs between optical orthoimage and UAV SAR image, extraction of control points, refinement of geometric image model and geocoding of UAV SAR image.

[0038] Step 1. Extract point pairs by accurately matching the optical orthophoto image with the UAV SAR image

[0039] The present invention adopts an accurate matching method for optical and SAR images based on rank self-similarity features. Firstly, feature points of the optical orthophoto image (reference image) are detected by Harris corner detector. Secondly, matching point pairs of the optical orthophoto image and the UAV SAR image are extracted based on DRSS descriptor. Finally, mismatched point pairs are eliminated to obtain accurate matching point pairs.

[0040] (1) Image feature point detection

[0041] Harris corner detector is used to obtain the feature points of optical orthophoto images. In order to obtain evenly distributed matching pairs, a feature screening strategy is introduced, that is, the image is first divided into m×m image blocks during feature extraction. Harris feature points are extracted from each image block and the feature response values ​​are sorted. The k points with the largest feature response are selected as feature points. The total number of feature points extracted is km. 2 .

[0042] (2) Matching point pair extraction

[0043] After determining the feature points of the optical orthophoto image, first, for each feature point, a DRSS descriptor is constructed in a template window centered on it, that is, by introducing the rank correlation theory, the rank value is used to replace the feature correlation value of the central image block in the relevant surface in the image, so as to avoid the problem of partial structural information loss in the local self-similar descriptor; then, the UAV SAR image (image to be registered) is roughly geocoded according to the known information, so as to eliminate the rotation and scale differences between the two images and obtain a rough geometric correspondence, and the position of the feature point of the optical orthophoto image on the UAV SAR image is estimated according to the geometric correspondence; then, in the local search window of the estimated position, the DRSS descriptor is calculated pixel by pixel as the center, and the similarity measure DRSS is calculated with the DRSS descriptor of the feature point. NCC (Such as formula (1)); Finally, find the DRSS in the search window NCC The maximum value of is found and the matching points are obtained by fitting near the maximum value.

[0044]

[0045] Among them, V1 and V2 are the DRSS descriptor vector values ​​on the reference image and the image to be registered. and are the average values ​​of V1 and V2 respectively.

[0046] Among them, Figure 2 As shown, the specific construction process of DRSS descriptor is as follows:

[0047] a) Extract the self-similar features of the image, in which the feature neighborhood radius is r;

[0048] b) sorting the channel dimension data of each point in the feature map and extracting rank self-similar features;

[0049] c) Select a template window of a certain size on the image, and divide the template window into a plurality of image blocks with half overlap, and the size of the image blocks is (2r+1)×(2r+1) pixels;

[0050] d) extracting RSS descriptors based on the rank self-similarity features in each image block and performing L2 norm normalization processing;

[0051] e) Concatenate the RSS descriptors of all image patches to form the final DRSS descriptor vector.

[0052] When matching the optical orthophoto image and the UAV SAR image, the UAV's flight altitude is low, the terrain undulations cause the SAR image to have large geometric distortion, the image resolution is very high, and the image structure features are sparse. The matching template can be set with a larger template size and search window. The present invention sets the template window size to 200×200 pixels and the search window size to 40×40 pixels. As other implementations, the template window size and the search window size can be determined according to specific circumstances.

[0053] (3) Elimination of incorrect matching points

[0054] Since there may be occlusions and shadows in the image and the time difference of the image may cause the target to change, the existence of mismatches in the matching point pairs is inevitable. Therefore, the transformation model consistency constraint is used to eliminate the mismatched points, and the transformation model uses the projection transformation. The specific process is: first, all matching point pairs are used to estimate the parameters of the projection transformation model by least squares; then, the residual of each matching point pair and the RMSE of all matching point pairs are calculated according to the estimated model, and the points with the largest residuals are eliminated; and the above process is repeated until the RMSE is less than the given threshold (1 pixel), thereby realizing the elimination of mismatched points in optical orthophotos and UAV SAR images. As other implementation methods, the setting of the RMSE threshold can be determined according to the specific situation.

[0055] Step 2. Control point extraction

[0056] The geographic coordinates of the matching points are interpolated on the DEM corresponding to the optical orthoimage, and the matching points with the obtained geographic coordinates are used as control points. When interpolating geographic coordinates, it is necessary to screen matching points on flat ground as control points, as such control points are more reliable. For each matching point, the geographic coordinates can be interpolated on the corresponding DEM according to the matching image point coordinates of the optical orthoimage, and the elevation values ​​corresponding to all pixels in the local neighborhood (9×9 pixels) of the matching image point can be obtained, and the standard deviation of these elevation values ​​is calculated. If the standard deviation is less than a certain threshold (0.2 meters), the matching point is considered to be a valid control point. As another implementation method, the setting of the standard deviation threshold can be determined according to the specific situation.

[0057] Step 3. Geometry model refinement

[0058] The range Doppler geometric imaging model is adopted, and the parameters of the model are refined using the extracted control points. The process of model refinement is as follows: given the initial value of the directional parameter, the control point data is substituted into the linearized range Doppler geometric imaging model to obtain the error equation, the error equation is normalized to solve the directional parameter correction number, and then the solved correction number is used to correct the initial value of the directional parameter. The above calculation is iterated until the given convergence condition is met and the directional parameter value is output.

[0059] The range Doppler geometry model is:

[0060]

[0061] Among them, (X, Y, Z) are the object space coordinates of the ground point, x and y are the distance coordinates and azimuth coordinates of the image point corresponding to the ground point, respectively, M slant is the slant range sampling interval, R0 is the close range delay, f dc is the Doppler center frequency, λ is the radar wavelength, P S =[X S Y S Z S ] T is the instantaneous position vector of the antenna phase center S, V = [V X V Y V Z ] T is the instantaneous velocity vector of the antenna phase center S. Assuming the platform moves in a uniform linear motion, we have:

[0062]

[0063] in, is the initial position vector. t is P S relatively The imaging time is t = K·y, where K is the time interval between two adjacent lines of image sampling. Formula (2) is about the orientation parameter The nonlinear equation needs to be linearized when solving the directional parameters.

[0064] Step 4. Geocoding of UAV SAR images

[0065] The UAV SAR images are geocoded using the refined range-Doppler geometric model and the corresponding DEM data.

[0066] (1) According to the average elevation value of the survey area and the image point coordinates of the four corner points of the image, the refined RD model is used to solve the object space coordinates corresponding to the four corner points, and the circumscribed rectangle of the solution result is calculated and used as the object space area range corresponding to the UAV SAR image.

[0067] (2) The object space area is divided into regular grids with equal intervals. The grid interval is set as the sampling interval of the encoded image. The image points of the encoded image correspond one-to-one to the object space grid points.

[0068] (3) The elevation values ​​are interpolated on the DEM according to the object coordinates of the grid points to obtain the three-dimensional coordinates of the grid points, and the image point coordinates of the grid points are solved by the refined range-Doppler geometric imaging model.

[0069] (4) The obtained image point coordinates are generally not integers. It is necessary to interpolate the grayscale value of the point on the original image and assign the grayscale value to the corresponding image point in the encoded image.

[0070] The present invention further verifies the above by conducting experiments on a UAV SAR image of a certain area in Henan and its optical orthophoto image. The experiments are as follows:

[0071] Among them, the optical image is obtained by a multi-rotor drone and processed to obtain an orthophoto; the SAR image is obtained by a fixed-wing drone. After the data is acquired, it is processed from slant range to ground range to eliminate some geometric differences and reduce the difficulty of image matching. The original drone SAR image is as follows Figure 3 As shown in Figure 4, the results of matching points between the optical orthophoto image and the UAV SAR image are shown in Figure 4(a) and Figure 4(b), and the geocoded UAV SAR image is shown in Figure 4(a). Figure 5 As shown in Figures 4(a) and 4(b), the present invention can automatically extract a large number of control points, avoiding the time-consuming and laborious manual collection of control points. Figure 5 , Figure 3 Compared with Figure 4(a), it can be clearly seen that Figure 5The geocoded SAR images are closer to optical orthophotos. The experimental results prove the practicability and effectiveness of geocoding SAR images using matching points of UAV SAR images and optical orthophotos as control points. It also verifies that compared with manually collected control points, the algorithm of the present invention can greatly improve the efficiency of geocoding SAR images.

[0072] The computer configuration is as follows: operating system: Windows 10 64-bit, processor: Intel(R) Core(TM) i7-8750H 2.20GHz, memory: 32GB, graphics card: NVIDI-AQuadro P1000, algorithm implementation platform: MATLABR2018a software.

Claims

1. A method for geocoding UAV SAR images using optical orthophotos, characterized in that: The method comprises the following steps: 1) Obtain a UAV SAR image and a corresponding optical orthoimage, extract feature points from the optical orthoimage, use a rank-based self-similar feature method to match the optical orthoimage and the UAV SAR image, and use the extracted feature points to search for matching points on the UAV SAR image to obtain matching point pairs; the image matching process is as follows: a. For each extracted feature point, extract the DRSS descriptor within the template window centered on it; b. Roughly geocode the UAV SAR image based on known navigation information to determine the estimated location of feature points in the SAR image; c. Calculate the DRSS descriptor pixel by pixel within the local search window at the estimated position, and calculate the similarity measure between it and the DRSS descriptor of the feature point; the calculation formula for the similarity measure is: Where V_1 and V_2 are the DRSS descriptor vector values ​​on the reference image and the image to be registered, (V_1)^ and (V_2)^ are the average values ​​of V_1 and V_2 respectively; d. Find the maximum similarity measure within the local search window and determine the matching point based on the position of the maximum value; 2) obtaining DEM data corresponding to the optical orthoimage, interpolating the geographic coordinates of the matching points on the DEM, and using the matching points with the obtained geographic coordinates as control points; 3) Refining the SAR image range-Doppler geometric model using the extracted control points to obtain a refined SAR image geometric model; 4) Geocoding the UAV SAR image using the geometric configuration model and the DEM data.

2. The method for geocoding UAV SAR images using optical orthoimages according to claim 1, wherein: In step 1), when extracting feature points of the optical orthophoto image, a Harris corner detector is used to detect and screen the feature points, the optical orthophoto image is divided into blocks, the feature response values ​​of the Harris feature points extracted from each image block are sorted, and k points with the largest feature response in each image block are selected as feature points.

3. The method for geocoding UAV SAR images using optical orthoimages according to claim 1, wherein: The method also includes screening the obtained matching point pairs to eliminate incorrect matching point pairs. The process is as follows: using all matching point pairs to estimate the parameters of the projective transformation model between the two images; calculating the residual of each matching point pair and the RMSE of all matching point pairs based on the estimated model, and eliminating the points with the largest residuals until the RMSE of all matching point pairs is less than a given threshold.

4. The method for geocoding UAV SAR images using optical orthoimages according to claim 1, wherein: The step 2) requires screening matching points on flat ground as control points. The process is as follows: obtain the elevation values ​​corresponding to all image points in the local neighborhood of the matching point, and calculate the standard deviation of these elevation values. If the standard deviation is less than a certain threshold, the matching point is considered to be a control point.

5. The method for geocoding UAV SAR images using optical orthoimages according to claim 1, wherein: The refinement process of the RD geometric configuration model in step 3) is as follows: set the initial value of the orientation parameter, substitute the control point data into the linearized RD geometric configuration model to obtain the error equation, calculate the orientation parameter correction number by normalizing the error equation, and then use the calculated correction number to correct the initial value of the orientation parameter until the given convergence condition is met and the orientation parameter value is output.

6. The method for geocoding UAV SAR images using optical orthoimages according to claim 5, characterized in that: The RD geometric model is: Among them, (X, Y, Z) is the object space coordinate of the ground point, x and y are the distance coordinate and azimuth coordinate of the image point corresponding to the ground point, respectively, M slant is the slant range sampling interval, R0 is the close range delay, f dc is the Doppler center frequency, λ is the radar wavelength, P S =[X S Y S Z S ] T is the instantaneous position vector of the antenna phase center S, V=[V X V Y V Z ] T is the instantaneous velocity vector of the antenna phase center S. If the platform moves in a uniform straight line, then: in, is the initial position vector, t is P S relatively The imaging time is t=K·y, where K is the time interval between two adjacent rows of image sampling.

7. The method for geocoding UAV SAR images using optical orthoimages according to claim 1, wherein: The object area corresponding to the UAV SAR image is gridded according to the encoded sampling interval, and the elevation value is interpolated in the DEM to obtain the three-dimensional coordinates of each grid point. The refined RD geometric imaging model is used to calculate the image point coordinates corresponding to each grid point, thereby realizing the geocoding of the UAV SAR image.

Citation Information

Patent Citations

  • Image match navigation method for double-area image formation synthetic aperture radars

    CN103245948A

  • Multi-source remote sensing image feature matching method based on directional phase consistency

    CN109523585A

  • SAR image quality detection method

    CN110111294A

  • Aerial photography integrated method

    CN110986888A

  • Method for extracting elevation control point with assistance of satellite laser altimetry data

    US20200103530A1