An unmanned aerial vehicle image upscaling matching method based on local self-convolution

Abstract

The application discloses a local self-convolution image upscaling matching method suitable for visual navigation of a UAV, and comprises the following steps: S1, performing self-convolution on each pixel point in a training image to realize upscaling operation of the training image; S2, extracting features by using a SIFT method on the training image after self-convolution to obtain descriptors; S3, matching the training image and a query image based on the extracted SIFT descriptors; and S4, eliminating error matching generated in the matching process. The image upscaling matching method provided by the application can effectively realize removal of unnecessary details of an image, and meanwhile, important edge information can be reserved or highlighted.

Description

Technical Field

[0001] This invention relates to image matching, and in particular to a method for upscaling UAV images based on local self-convolution. Background Technology

[0002] Visual navigation in environments without GPS or other positioning systems requires feature matching between UAV visual images and reference images (remote sensing satellite images with geographic location information) to determine the UAV's specific location. The key issue to be addressed is feature matching between images with different ground resolutions.

[0003] 1. Without upscaling, the significant difference in ground resolution between the image to be matched and the reference image leads to inaccurate matching of local and global feature points, resulting in too many mismatched or missed feature points. Figure 6 As shown.

[0004] 2. Upscaling based on downsampling or Gaussian filtering may miss matching feature points, such as... Figure 7 As shown.

[0005] 3. Self-convolutional upscaling can more effectively preserve matching feature points, such as... Figure 8 As shown.

[0006] In image feature point matching, the query image is typically an image taken by a satellite. The training image is an image acquired by a drone.

[0007] Image feature point matching primarily involves two tasks: feature point detection and descriptor extraction. Feature point detection typically involves corner detectors, such as the Harris and FAST corner detectors. The goal of feature point detection is to find salient points that are easily detected in two completely independent detection iterations on different images of the same region, even though these images may differ significantly in terms of illumination, scale, rotation, and viewpoint. Descriptor extraction is the step of extracting feature vectors from the region surrounding the feature point. The goal is to construct a descriptor that can be used to match the feature point using metrics such as Euclidean distance or Hamming distance.

[0008] Reference images are typically large-scale images taken by satellite. When using methods such as SIFT for detection, the training images contain many unnecessary details compared to the query images. These details increase matching time and affect matching accuracy. Therefore, it is necessary to upscale the training images to remove unnecessary details. Current image upscaling methods mainly use Gaussian kernels to filter the images, but a problem with this is that it blurs all the information in the image, which may result in some important image parts not being well preserved. Summary of the Invention

[0009] The purpose of this invention is to overcome the shortcomings of the prior art and provide a UAV image upscaling matching method based on local self-convolution, which can effectively remove unnecessary details of the image while preserving or highlighting important edge information.

[0010] The objective of this invention is achieved through the following technical solution: a method for upscaling UAV images based on local self-convolution, comprising the following steps:

[0011] S1. By performing self-convolution on each pixel in the training image, the training image is upscaled.

[0012] In step S1, when performing self-convolution on each pixel in the training image, a matrix of size (2a+1)×(2b+1) centered on that pixel is used as the convolution kernel for that pixel.

[0013] The pixel values ​​of different pixels contained in a matrix of size (2a+1)×(2b+1) are used as matrix elements.

[0014] Step S1 includes:

[0015] S101. For any point (x0, y0) in the training image, the value of the resulting self-convolutional image g(x, y) at (x0, y0) is g(x0, y0):

[0016]

[0017] in This represents a matrix of size (2a+1)×(2b+1) centered at (x0, y0) representing the original image. It is the convolution kernel at (x0, y0), with a size of (2a+1)×(2b+1), and f(x0-dx, y0-dy) represents the pixel value at pixel (x0-dx, y0-dy);

[0018] S102. Repeat step S101 for each point of the training image to obtain the value of the self-convolution image g(x,y) at each point, thereby obtaining the self-convolution image g(x,y) as an upscaled image;

[0019] For pixels in the image boundary where a convolution kernel of size (2a+1)×(2b+1) cannot be obtained, the pixel value of that pixel is directly used as the self-convolution result.

[0020] S2. Use the SIFT method to extract features from the self-convolutioned training images to obtain descriptors;

[0021] Step S2 includes the following sub-steps:

[0022] S201. Construct the DoG pyramid for upscaling the image g(x,y):

[0023] A1. Constructing a Gaussian scale space as the result of Gaussian blurring:

[0024] The Gaussian scale space of an image is defined as a function L(x,y,σ), which is generated by the convolution of a Gaussian kernel function G(x,y,σ) with the input image I(x,y):

[0025] L(x,y,σ)=G(x,y,σ)*I(x,y)

[0026] Where * denotes convolution.

[0027]

[0028] σ is called the scale space factor, which is the standard deviation of the Gaussian normal distribution. It reflects the degree of blurring of the image. The larger the value, the more blurred the image, and the larger the corresponding scale.

[0029] A2. First, Gaussian blur the obtained upscaled image g(x,y) to obtain the Gaussian blurred image g(x,y) as the first layer of the Gaussian pyramid. Then, based on the Gaussian blurred image g(x,y), continuous downsampling is performed to obtain a series of continuously reduced images. Each downsampled image is used as a layer, and the image pyramid g(x,y) is formed according to the downsampling order.

[0030] A3. For any layer of the image pyramid, Gaussian blur is performed using n sequentially arranged scale spatial factors to obtain n Gaussian blurred images with different scale spatial factors; for any two adjacent scale spatial factors, the ratio of the first scale spatial factor to the second scale spatial factor is k.

[0031] For n Gaussian blurred images with different scale spatial factors, calculate the DoG space:

[0032] For Gaussian blurred images corresponding to two adjacent scale spatial factors, the DoG (Difference of Gaussian) is calculated as follows:

[0033] D(x,y,σ)=[G(x,y,kσ)-G(x,y,σ)]*I(x,y)=L(x,y,kσ)-L(x,y,σ)

[0034] Where L(x,y,σ) is the Gaussian scale space of the image;

[0035] Since there are n Gaussian blurred images with different spatial factors at different scales, a total of n-1 DoG spaces are calculated.

[0036] A4. For each layer of the image pyramid, repeat step A3 to obtain the DoG pyramid of g(x,y);

[0037] S202. Detect the extreme values ​​in the DoG pyramid and remove the extreme value points that do not meet the conditions to obtain the feature points:

[0038] B1. For each layer of the image pyramid, the resulting n-1 DoG spaces are processed as follows:

[0039] To find the extreme points in the scale space, for each DoG space, each pixel is compared with all its neighboring pixels in the same scale space and in adjacent scale spaces. When its pixel value is greater than or less than the pixel values ​​of all its neighboring pixels, the current pixel is the extreme point.

[0040] B2. Next, the extreme points obtained are used as candidate feature points, and extreme points that do not meet the conditions are removed:

[0041] For any candidate feature point x, its offset is defined as Δx, and its contrast is the absolute value of D(x)|D(x)|, where the Taylor expansion is applied to D(x).

[0042]

[0043] Since x is an extreme point of D(x), taking the derivative of the above equation and setting it to 0, we get...

[0044]

[0045] Then substitute the obtained Δx into the Taylor expansion of D(x).

[0046]

[0047] Let the contrast threshold be T, if If the feature point is not specified, it is retained; otherwise, it is discarded.

[0048] S203. Determine the principal direction of the feature points:

[0049] Image at the scale of the feature points

[0050] L(x,y)=G(x,y,σ)*I(x,y)

[0051] The amplitude and magnitude of the gradient of the region image centered at the feature point and with a radius of 3 × 1.5σ are calculated. The magnitude m(x,y) and direction θ(x,y) of the gradient at each point L(x,y) are obtained using the following formula.

[0052]

[0053]

[0054] After calculating the gradient direction, a histogram is used to statistically analyze the gradient direction and magnitude of pixels in the neighborhood of the feature point. The horizontal axis of the gradient direction histogram is the angle of the gradient direction, and the vertical axis is the sum of the gradient magnitudes corresponding to the gradient direction. The peak value in the histogram is the main direction of the feature point.

[0055] S204. Generate feature descriptors:

[0056] For each feature point, in order to ensure the rotation invariance of the vector, the coordinate axes should be rotated by an angle θ in the nearby coordinate system with the feature point as the center, where θ is the principal direction angle of the feature point.

[0057] After rotation, a 16×16 window is taken centered on the main direction. The gradient magnitude and gradient direction of each pixel within the window are calculated. Then, a Gaussian function G(x,y,σ) with σ=4 is used to assign weights to the magnitude of each sampling point.

[0058] in:

[0059] Finally, the weighted summation of the magnitude in each of the eight directions is calculated on each 4×4 block to form a seed point; that is, each keypoint is described by 16 seed points, so that a keypoint generates a 128-dimensional SIFT feature vector.

[0060] Finally, normalization is performed to normalize the length of the obtained feature vectors and further remove the influence of illumination to obtain SIFT features, which are the descriptors.

[0061] S3. Match the training image with the query image based on the extracted SIFT descriptors;

[0062] First, the query image is substituted into step S1 to obtain the self-convolution result of the query image. Then, the self-convolution image of the query image is substituted into step S2 as the training image to obtain the feature points and descriptors of the query image. After that, for each feature point in the training image, the Euclidean distance between the descriptor and all feature points in the query image is calculated. If the Euclidean distance between a descriptor in the training image and a descriptor in the query image is less than a given threshold, then the feature points corresponding to the two descriptors are considered to be successfully matched.

[0063] S4. Eliminate erroneous matches generated during the matching process.

[0064] In step S4, when eliminating incorrect matches, any of the following methods may be used:

[0065] I. Cross-filtering:

[0066] If the Euclidean distance between a descriptor in the training image and a descriptor in the query image is less than a given threshold, the feature points corresponding to the two descriptors are considered to have successfully matched. When a feature point in the training image matches a feature point in the query image, a reverse check is performed, that is, the feature point in the query image is matched with the feature point in the training image. If the match is successful, it is considered correct; if the match is unsuccessful, it is considered an incorrect match and is removed.

[0067] II. Ratio Test:

[0068] For each match, return the two nearest neighbor descriptors, that is, the two descriptors with the smallest Euclidean distance on the query image to the descriptor corresponding to the match on the training image. A match is considered correct only if the Euclidean distance between the descriptor corresponding to the first match on the query image and the descriptor corresponding to the second match on the query image is less than a set threshold. If the Euclidean distance is greater than the set threshold, it is considered an incorrect match and is removed.

[0069] The beneficial effects of this invention are: this invention matches images captured in real time by a drone with images previously captured by a satellite, and through the self-convolution method, it can effectively remove unnecessary details from the image while preserving or highlighting important edge information. Attached Figure Description

[0070] Figure 1 This is a flowchart of the method of the present invention;

[0071] Figure 2 This is the original image used in the embodiment;

[0072] Figure 3 This is a self-convolutional image obtained when using a 5x5 convolutional kernel in the example;

[0073] Figure 4 This is the image obtained by using the Prewitt algorithm to extract edges from the original image in this embodiment;

[0074] Figure 5 The image shown is obtained by using the Prewitt algorithm to extract edges from a self-convolutional image in this embodiment.

[0075] Figure 6The result obtained by matching the unprocessed training image with the query image in the example is shown below.

[0076] Figure 7 The result obtained by matching the training image (obtained through Gaussian blurring) with the query image in this embodiment is shown below.

[0077] Figure 8 The result obtained by matching the training image obtained using the self-convolution method with the query image in this embodiment.

[0078] Figure 9 The training image shown is the third layer of the Gaussian pyramid in this example.

[0079] Figure 10 The training image shown is the third layer of the DoG pyramid in this example.

[0080] Figure 11 The descriptor shown is obtained from the training images in this example (only the top left corner is displayed). Detailed Implementation

[0081] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings, but the scope of protection of the present invention is not limited to the following description.

[0082] like Figure 1 As shown, a UAV image upscaling matching method based on local self-convolution includes the following steps: S1. Upscaling the training image using local self-convolution; S2. Extracting features from the self-convolved training image using the SIFT method; S3. Matching the training image with the query image; S4. Eliminating erroneous matches generated during the matching process.

[0083] In step S1, when performing self-convolution on each pixel in the training image, a matrix of size (2a+1)×(2b+1) centered on that pixel is used as the convolution kernel for that pixel.

[0084] The pixel values ​​of different pixels contained in a matrix of size (2a+1)×(2b+1) are used as matrix elements.

[0085] Step S1 includes:

[0086] S101. For any point (x0, y0) in the training image, the value of the resulting self-convolutional image g(x, y) at (x0, y0) is g(x0, y0):

[0087]

[0088] in This represents a matrix of size (2a+1)×(2b+1) centered at (x0, y0) representing the original image. It is the convolution kernel at (x0, y0), with a size of (2a+1)×(2b+1), and f(x0-dx, y0-dy) represents the pixel value at pixel (x0-dx, y0-dy);

[0089] S102. Repeat step S101 for each point of the training image to obtain the value of the self-convolution image g(x,y) at each point, thereby obtaining the self-convolution image g(x,y) as an upscaled image;

[0090] For pixels in the image boundary where a convolution kernel of size (2a+1)×(2b+1) cannot be obtained, the pixel value of that pixel is directly used as the self-convolution result.

[0091] For an image with a (5,5) region, perform a self-convolution on it using a (3,3) kernel.

[0092]

[0093] The size of the nucleus is (3,3).

[0094] Therefore, only in the positions (2,2), (2,3), (2,4), (3,2), (3,3), (3,4), (4,2), (4,3), (4,4) will the position values ​​change, and the changed results are...

[0095]

[0096] This yields the self-convolution result of the image.

[0097] When using the Prewitt operator to compute edges (only the x-direction is shown here).

[0098]

[0099] Here we focus on the position (3,3), where the value is...

[0100] x24(x24+x35)-x22(x22-x33)+x34(x34+x23)-x32(x32-x43)+x44(x44+x33) -x42(x42-x31)+(x14*x25+x34*x45+x43*x54)-(x41*x52+x12*x23+x21*x32)

[0101] This yields the Prewitt operator value at position (3,3) after the image undergoes self-convolution. This is compared to the Prewitt value extracted from an image without self-convolution.

[0102] x24-x22+x34-x32+x44-x42

[0103] It is evident that self-convolution affects edges not only by considering the pixel itself but also by taking into account surrounding pixels. It weights pixels further away by assigning them different weights, and finally adds two symmetrical terms to correct the value.

[0104] for Figure 2 When using a 5x5 convolution kernel, the resulting self-convolutional image is: Figure 3 .

[0105] It can be observed that self-convolution eliminates lines with smaller widths. The Prewitt algorithm is used to further analyze this process. Figure 2 , Figure 3 By extracting the edges, we can obtain the following: Figure 4 , Figure 5 .

[0106] It is evident that image self-convolution effectively removes unnecessary details while preserving, and even enhancing, important edge information.

[0107] In step S2, SIFT is used to extract image features:

[0108] The SIFT algorithm was proposed by Lowe in 2004. It is invariant to rotation, scaling, and brightness changes, making it a very stable local feature.

[0109] To extract SIFT features, the main steps include:

[0110] First, construct the image pyramid.

[0111] Using different Gaussian kernels yields images with varying degrees of blur. The Gaussian scale space of an image can be obtained by convolving it with different Gaussian kernels:

[0112] L(x,y,σ)=G(x,y,σ)*I(x,y)

[0113] Where G(x,y,σ) is the Gaussian kernel function.

[0114]

[0115] σ is called the scale space factor, which is the standard deviation of the Gaussian normal distribution. It reflects the degree of blurring of the image. The larger the value, the more blurred the image, and the larger the corresponding scale.

[0116] The purpose of constructing a scale space is to detect feature points that exist at different scales.

[0117] Let k be the scaling factor between two adjacent Gaussian scale spaces, then the definition of DoG is:

[0118] D(x,y,σ)=[G(x,y,kσ)-G(x,y,σ)]*I(x,y)=L(x,y,kσ)-L(x,y,σ)

[0119] Where L(x,y,σ) is the Gaussian scale space of the image;

[0120] An image pyramid is a set of results obtained from the same image at different resolutions. To generate an image pyramid, the original image is first Gaussian blurred, and then the Gaussian blurred image is downsampled to obtain a series of images with continuously reduced size.

[0121] A Gaussian pyramid is obtained by applying Gaussian blur to each layer of the image pyramid using different parameters σ.

[0122] The third layer of the Gaussian pyramid obtained is as follows: Figure 9 As shown. During downsampling, the first image in the upper group of images of the pyramid is obtained by downsampling the third-to-last image in the group below it. After the Gaussian pyramid is constructed, subtracting adjacent Gaussian pyramids yields the DoG pyramid. The resulting DoG pyramid is shown in the image below. Figure 10 As shown.

[0123] The purpose of constructing a scale space is to detect feature points that exist at different scales, and the operator that is better at detecting feature points is Δ. 2 G (Laplace of Gauss, LoG)

[0124]

[0125] Directly using LoG computation is computationally expensive; therefore, DoG is typically used to approximate LoG. Let k be the scaling factor between two adjacent Gaussian scale spaces, then the definition of DoG is:

[0126] D(x,y,σ)=[G(x,y,kσ)-G(x,y,σ)]*I(x,y)=L(x,y,kσ)-L(x,y,σ)

[0127] Where L(x,y,σ) is the Gaussian scale space of the image.

[0128] Next, extreme values ​​in the DoG space are detected, and undesirable extreme values ​​are removed:

[0129] A candidate feature point x has an offset defined as Δx and a contrast defined as the absolute value of D(x) |D(x)|. Applying the Taylor expansion to D(x)...

[0130]

[0131] Since x is an extreme point of D(x), taking the derivative of the above equation and setting it to 0, we get...

[0132]

[0133] Then substitute the obtained Δx into the Taylor expansion of D(x).

[0134]

[0135] Let the contrast threshold be T, if If the feature point is selected, it is retained; otherwise, it is discarded.

[0136] Next, the principal direction of the feature points is determined:

[0137] Image at the scale of the feature points

[0138] L(x,y)=G(x,y,σ)*I(x,y)

[0139] The argument and magnitude of the region image centered on the feature point and with a radius of 3 × 1.5σ are calculated. The magnitude m(x,y) and direction θ(x,y) of the gradient at each point L(x,y) can be obtained using the following formula.

[0140]

[0141]

[0142] After calculating the gradient direction, a histogram is used to statistically analyze the gradient direction and magnitude of pixels within the neighborhood of the feature point. The horizontal axis of the gradient direction histogram represents the angle of the gradient direction (the gradient direction ranges from 0 to 360 degrees; the histogram has 10 bars, each representing 36 degrees, or 8 bars, each representing 45 degrees). The vertical axis represents the cumulative gradient magnitude corresponding to the gradient direction. The peak value in the histogram represents the principal direction of the feature point.

[0143] Finally, feature descriptions are generated.

[0144] 1. Correct the principal direction of rotation to ensure rotational invariance.

[0145] 2. Generate descriptors, ultimately forming a 128-dimensional feature vector.

[0146] 3. Normalization processing: The feature vector length is normalized to further remove the influence of illumination.

[0147] Obtain SIFT features.

[0148] To ensure the rotation invariance of the vector, the coordinate axes should be rotated by an angle θ (the principal direction of the feature point) in the nearby coordinate system, with the feature point as the center.

[0149] After rotation, a 16×16 window is taken centered on the main direction. The gradient magnitude and direction of each pixel within the window are calculated, and then weighted using a Gaussian window. Finally, the cumulative values ​​in each of the eight directions are calculated for each 4×4 block, forming a seed point. That is, each keypoint is described using 16 seed points, resulting in a 128-dimensional SIFT feature vector for each keypoint. The resulting descriptor is as follows: Figure 11 As shown.

[0150] In step S4, a ratio test is used to eliminate mismatches.

[0151] I. Cross-filtering:

[0152] If the Euclidean distance between a descriptor in the training image and a descriptor in the query image is less than a certain threshold, the feature points corresponding to the two descriptors are considered to have successfully matched. When a feature point in the training image matches a feature point in the query image, a reverse check is performed: the feature point in the query image is matched with the feature point in the training image. If the match is successful, it is considered correct; otherwise, it is considered an incorrect match and is removed.

[0153] II. Ratio Test:

[0154] For each match, the two nearest-neighbor descriptors are returned; that is, the two descriptors in the training image with the smallest Euclidean distance to the descriptor corresponding to the match in the query image. A match is considered correct only if the Euclidean distance between the descriptors in the query image corresponding to the first match and the descriptors in the query image corresponding to the second match is less than a certain threshold. If the Euclidean distance is greater than the set threshold, it is considered an incorrect match and is removed.

[0155] The foregoing description illustrates and describes a preferred embodiment of the present invention. However, as previously stated, it should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the inventive concept described herein through the foregoing teachings or techniques or knowledge in related fields. Any modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A method for upscaling UAV images based on local self-convolution, characterized in that: Includes the following steps: S1. By performing self-convolution on each pixel in the training image, the training image is upscaled. When performing self-convolution on each pixel in the training image, a matrix of size (2a+1)×(2b+1) centered on that pixel is used as the convolution kernel for that pixel. The pixel values ​​of different pixels contained in a matrix of size (2a+1)×(2b+1) are used as matrix elements; Step S1 includes: S101. For any point (x0, y0) in the training image, the value of the resulting self-convolutional image g(x, y) at (x0, y0) is g(x0, y0): in This represents a matrix of size (2a+1)×(2b+1) centered at (x0, y0) representing the original image. It is the convolution kernel at (x0, y0), with a size of (2a+1)×(2b+1), and f(x0-dx, y0-dy) represents the pixel value at pixel (x0-dx, y0-dy); S102. Repeat step S101 for each point of the training image to obtain the value of the self-convolution image g(x,y) at each point, thereby obtaining the self-convolution image g(x,y) as an upscaled image; For pixels in the image boundary where a convolution kernel of size (2a+1)×(2b+1) cannot be obtained, the pixel value of that pixel is directly used as the self-convolution result. S2. Use the SIFT method to extract features from the self-convolutioned training images to obtain descriptors; S3. Match the training image with the query image based on the extracted SIFT descriptors; S4. Eliminate erroneous matches generated during the matching process.

2. The UAV image upscaling matching method based on local self-convolution according to claim 1, characterized in that: Step S2 includes the following sub-steps: S201. Construct the DoG pyramid for upscaling the image g(x,y): A1. Constructing a Gaussian scale space as the result of Gaussian blurring: The Gaussian scale space of an image is defined as a function L(x,y,σ), which is generated by the convolution of a Gaussian kernel function G(x,y,σ) with the input image I(x,y): L(x,y,σ)=G(x,y,σ)*I(x,y) Where * denotes convolution. σ is called the scale space factor, which is the standard deviation of the Gaussian normal distribution. It reflects the degree of blurring of the image. The larger the value, the more blurred the image, and the larger the corresponding scale. A2. First, Gaussian blur the obtained upscaled image g(x,y) to obtain the Gaussian blurred image g(x,y) as the first layer of the Gaussian pyramid. Then, based on the Gaussian blurred image g(x,y), continuous downsampling is performed to obtain a series of continuously reduced images. Each downsampled image is used as a layer, and the image pyramid g(x,y) is formed according to the downsampling order. A3. For any layer of the image pyramid, Gaussian blur is performed using n sequentially arranged scale spatial factors to obtain n Gaussian blurred images with different scale spatial factors; for any two adjacent scale spatial factors, the ratio of the first scale spatial factor to the second scale spatial factor is k. For n Gaussian blurred images with different scale spatial factors, calculate the DoG space: For Gaussian blurred images corresponding to two adjacent scale spatial factors, the DoG is calculated as follows: D(x,y,σ)=[G(x,y,kσ)-G(x,y,σ)]*I(x,y)=L(x,y,kσ)-L(x,y,σ) Where L(x,y,σ) is the Gaussian scale space of the image; Since there are n Gaussian blurred images with different spatial factors at different scales, a total of n-1 DoG spaces are calculated. A4. For each layer of the image pyramid, repeat step A3 to obtain the DoG pyramid of g(x,y); S202. Detect the extreme values ​​in the DoG pyramid and remove the extreme value points that do not meet the conditions to obtain the feature points: For each layer of the image pyramid, the resulting n-1 DoG spaces are processed as follows: To find the extreme points in the scale space, for each DoG space, each pixel is compared with all its neighboring pixels in the same scale space and in adjacent scale spaces. When its pixel value is greater than or less than the pixel values ​​of all its neighboring pixels, the current pixel is the extreme point. Next, the obtained extreme points are used as candidate feature points, and extreme points that do not meet the conditions are removed: For any candidate feature point x, its offset is defined as Δx, and its contrast is the absolute value of D(x)|D(x)|, where the Taylor expansion is applied to D(x). Since x is an extreme point of D(x), taking the derivative of the above equation and setting it to 0, we get... Then substitute the obtained Δx into the Taylor expansion of D(x). Let the contrast threshold be T, if If the feature point is not specified, it is retained; otherwise, it is discarded. S203. Determine the principal direction of the feature points: Image at the scale of the feature points L(x,y)=G(x,y,σ)*I(x,y) The amplitude and magnitude of the gradient of the region image centered at the feature point and with a radius of 3 × 1.5σ are calculated. The magnitude m(x,y) and direction θ(x,y) of the gradient at each point L(x,y) are obtained using the following formula. After calculating the gradient direction, a histogram is used to statistically analyze the gradient direction and magnitude of pixels in the neighborhood of the feature point. The horizontal axis of the gradient direction histogram is the angle of the gradient direction, and the vertical axis is the sum of the gradient magnitudes corresponding to the gradient direction. The peak value in the histogram is the main direction of the feature point. S204. Generate feature descriptors: For each feature point, in order to ensure the rotation invariance of the vector, the coordinate axes should be rotated by an angle θ in the nearby coordinate system with the feature point as the center, where θ is the principal direction angle of the feature point. After rotation, a 16×16 window is taken centered on the main direction. The gradient magnitude and gradient direction of each pixel within the window are calculated. Then, a Gaussian function G(x,y,σ) with σ=4 is used to assign weights to the magnitude of each sampling point. in: Finally, the weighted summation of the magnitude in each of the eight directions is calculated on each 4×4 block to form a seed point; that is, each keypoint is described by 16 seed points, so that a keypoint generates a 128-dimensional SIFT feature vector. Finally, normalization is performed to normalize the length of the obtained feature vectors and further remove the influence of illumination to obtain SIFT features, which are the descriptors.

3. The UAV image upscaling matching method based on local self-convolution according to claim 1, characterized in that: Step S3 includes: First, the query image is substituted into step S1 to obtain the self-convolution result of the query image. Then, the self-convolution image of the query image is substituted into step S2 as the training image to obtain the feature points and descriptors of the query image. After that, for each feature point in the training image, the Euclidean distance between the descriptor and all feature points in the query image is calculated. If the Euclidean distance between a descriptor in the training image and a descriptor in the query image is less than a given threshold, then the feature points corresponding to the two descriptors are considered to be successfully matched.

4. The UAV image upscaling matching method based on local self-convolution according to claim 1, characterized in that: In step S4, when eliminating incorrect matches, any of the following methods may be used: I. Cross-filtering: If the Euclidean distance between a descriptor in the training image and a descriptor in the query image is less than a given threshold, the feature points corresponding to the two descriptors are considered to have successfully matched. When a feature point in the training image matches a feature point in the query image, a reverse check is performed, that is, the feature point in the query image is matched with the feature point in the training image. If the match is successful, it is considered correct; if the match is unsuccessful, it is considered an incorrect match and is removed. II. Ratio Test: For each match, return the two nearest neighbor descriptors, that is, the two descriptors with the smallest Euclidean distance on the query image to the descriptor corresponding to the match on the training image. A match is considered correct only if the Euclidean distance between the descriptor corresponding to the first match on the query image and the descriptor corresponding to the second match on the query image is less than a set threshold. If the Euclidean distance is greater than the set threshold, it is considered an incorrect match and is removed.

Images