Satellite-unmanned aerial vehicle cross-view fixed facility matching algorithm combining full convolution feature extraction and attention feature matching

Abstract

The invention relates to a satellite-unmanned aerial vehicle cross-view-angle fixed facility matching algorithm combining full convolution feature extraction and attention feature matching. The algorithm comprises the following steps: preprocessing the consistency of different-source images; establishing a full convolution feature extraction network based on sub-pixel convolution and score screening; establishing a feature point matching network based on a bidirectional mixed attention and pruning screening strategy; homography post-processing is carried out, and an output result is calculated. Through the heterogeneous data consistency processing method, the difference between the satellite image and the unmanned aerial vehicle image is reduced, and the matching difficulty is reduced; extracting feature information in the image through a full convolution feature extraction network based on sub-pixel convolution and score screening; matching the extracted feature information through a feature point matching network based on a bidirectional mixed attention and pruning screening strategy; and calculating the position of the selected target in the unmanned aerial vehicle image in the satellite image according to the successfully matched feature point pairs.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and particularly to a satellite - UAV cross - view fixed facility matching algorithm that combines full - convolutional feature extraction and attention feature matching. Background Art

[0002] With the rapid development of remote sensing technology and UAV technology, it has become increasingly easy to obtain surface image data at different heights, different angles, and different time periods. As a result, the application prospects of remote sensing image data in fields such as disaster emergency, urban planning, environmental monitoring, agricultural management, and military security are becoming increasingly broad. Traditional remote sensing image analysis methods mainly rely on image data from a single perspective and are difficult to comprehensively reflect the actual situation of surface targets. Therefore, integrating remote sensing data from different perspectives and different platforms, especially satellite images and UAV images, effectively combining the macroscopic detection ability of satellite images with the microscopic detail ability of UAV images, and constructing a multi - source cross - view remote sensing data processing method have become an innovative image analysis strategy with significant application value.

[0003] In existing research, satellite - UAV cross - view image matching mainly focuses on image - level matching tasks, that is, given a picture to be retrieved, find the picture of the same location from a candidate dataset of another perspective. However, image - level matching can only provide the approximate location of the target and cannot meet the more refined target recognition requirements, especially in dynamic and complex surface environments. The object - level matching task aims to select an area of interest in a satellite image, then use a UAV to take images or videos near the target area, and finally accurately capture the selected target in the images and videos from the UAV perspective. This cross - view matching method can combine the large - range scanning ability of satellite images with the high - resolution advantage of UAV images and has very broad potential and value in practical applications. In the field of emergency rescue, satellite images provide large - scale coverage of the disaster area, and UAVs take close - range photos in specific target areas, match and locate the disaster site, and accurately deliver relief supplies to improve rescue efficiency; in the field of agricultural management, satellite images detect the health status of crops and identify abnormal areas, and UAVs obtain specific crop details through more refined shooting or directly perform pesticide spraying to optimize farm management decisions and reduce production costs; in the field of infrastructure, satellite images discover abnormal areas of infrastructure such as power grids, roads, and railways, and UAVs finely confirm the damaged parts or potential hazards, saving the time of manual inspection and reducing the inspection cost.

[0004] To build a cross - perspective matching algorithm from satellites to drones, it is necessary to solve not only the technical problems of image feature extraction and matching under different perspectives, resolutions, and lighting conditions, but also to conduct in - depth exploration in terms of algorithm efficiency, real - time performance, and robustness. With the development of technologies such as deep learning and multi - perspective geometry, this research field is expected to make breakthrough progress and promote the intelligent processing and application of cross - platform and cross - perspective remote sensing data. In the future, through automated cross - perspective target matching methods, the recognition and tracking speed of specific targets can be significantly improved, especially in scenarios that require rapid response, where it will demonstrate great application potential.

[0005] However, in practical applications, the satellite - drone cross - perspective matching algorithm has the following technical difficulties:

[0006] (1) Perspective difference of heterogeneous data: Remote sensing satellites usually take pictures vertically above the earth's surface, and the obtained satellite images are usually top - view images. In contrast, drone cameras generally take pictures obliquely downward to obtain corresponding side - view images. The side information of buildings and other objects captured in drone images has no corresponding features in satellite images. Additionally, the operating trajectory of remote sensing satellites is relatively fixed. After orthorectification of satellite images, the geographical due - north direction in the image can be aligned with the vertical - upward direction of the image. However, due to the high mobility of drones, the heading angle can vary arbitrarily, and the geographical due - north direction can point to any direction in the image.

[0007] (2) Pixel difference of heterogeneous data: The resolution of remote sensing satellite images is relatively low. Currently, the pixel resolution of relatively advanced high - resolution satellites is between 0.3 meters and 0.5 meters. In contrast, the resolution of drone images is relatively high. When shooting 4K images or videos, the central pixel resolution can reach 0.15 meters or less. In addition, due to the differences in imaging principles and image scales between satellite images and drone images, there are significant differences in attributes such as spectral bands, colors, brightness, and contrast between satellite images and drone aerial images. Therefore, the features extracted at the same coordinate position may vary greatly. Summary of the Invention

[0008] The present invention aims to solve the technical problems in the prior art and provides a satellite - drone cross - perspective fixed - facility matching algorithm that combines full - convolutional feature extraction and attention - based feature matching.

[0009] To solve the above - mentioned technical problems, the technical solution of the present invention is as follows:

[0010] A satellite - drone cross - perspective fixed - facility matching algorithm that combines full - convolutional feature extraction and attention - based feature matching, comprising the following steps:

[0011] Step 1: Pre - processing the consistency of heterogeneous images;

[0012] Step 2: Establish a fully convolutional feature extraction network based on sub-pixel convolution and score screening;

[0013] Step 3: Establish a feature point matching network based on bidirectional hybrid attention and pruning screening strategy;

[0014] Step 4: Perform homography post-processing and calculate the output result.

[0015] In the above technical solution, Step 1 is specifically as follows:

[0016] For the panchromatic band satellite image, convert the UAV image into monochrome. For the RGB three-band satellite image, modify the brightness and contrast of the satellite image to reduce the difference in color space;

[0017] Obtain the heading angle of the UAV image, rotate the satellite image so that the viewing angles of the satellite image and the UAV image tend to be the same, reducing the robustness requirements for the rotational invariance of the subsequent matching algorithm.

[0018] In the above technical solution, Step 2 is specifically as follows:

[0019] After obtaining the basic feature map, perform feature compression to reduce the dimension of the feature vector and thus reduce the computational amount; then input it into the key point description branch and the feature operator branch respectively, which are used to calculate the key point description and the feature operator respectively;

[0020] In the key point description branch, input the tensor F Pos After SoftMax normalization, the obtained normalized map contains the feature information of each position in the image; perform feature response on the normalized map, use sub-pixel convolution for upsampling, and output the heat map H Pos ∈H×W×1, and the output heat map is the same size as the original input image; finally, obtain the positions of the k feature points with the highest scores through feature screening and perform coordinate transformation, which is used as the output of the key point description branch, and output the tensor T Pos ∈k×2, where k is the number of output feature points, and 2 represents the two-dimensional coordinates of the feature point positions;

[0021] In the feature operator branch, first pass through a 1×1 convolutional layer to increase the number of channels of the feature tensor to 256; at this time, the obtained F Des Each pixel position actually represents the feature information of an 8×8 pixel size area of the original input image; perform feature alignment; feature alignment is decoded using a fixed parameter-free method; for F Des Use bicubic interpolation for upsampling to output the feature representation of the image The size is the same as the input image size; normalize the obtained feature representation, and finally obtain the feature operator T of the image Des∈ k × 256; k is the number of output feature points, which is consistent with the key point description branch, and 256 represents the feature vector of each output feature point;

[0022] After calculating the key point description and the feature operator in the key point description branch and the feature operator branch respectively, the feature point T output by the fully convolutional feature extraction network based on sub-pixel convolution and score screening Feat ∈ k × 2 × 256.

[0023] In the above technical solution, step 3 is specifically as follows:

[0024] The feature extraction network extracts the feature information in the satellite image and the UAV image, outputs the corresponding feature vectors, and the feature matching network performs similarity measurement on the feature vectors to screen out the homologous feature points;

[0025] In the self-attention module, for the feature vector f at each position i , the correlation relationship sa with the feature vector f at another position in the image j is expressed as: ij It is expressed as:

[0026]

[0027] Among them, q and k are the query vector and the key vector obtained by performing different linear transformations on the input vector f in imitation of the Transformer structure, and R is the rotational position encoding, which stores the relative position relationship between two feature points;

[0028] In the cross-attention module, for each feature vector and any feature vector in another image is expressed as:

[0029]

[0030] Among them, k is also the key vector obtained by performing a linear transformation on the input vector p;

[0031] After passing through the bidirectional hybrid attention network, calculate the confidence of each feature vector; judge whether the output condition is satisfied according to the confidence of all feature points; if the output condition is not satisfied, perform point pruning to delete the points with lower confidence, and these feature points are considered isolated feature points and there are no feature points in another image that can match them; loop multiple times until the exit condition is satisfied, and output enough feature point pairs with high enough confidence.

[0032] ​In the above technical solution, step 4 is specifically as follows: Calculate the homography matrix from the satellite image to the UAV image according to the matching point pairs, and apply the homography matrix to the target position in the satellite image to obtain the target position in the UAV image.

[0033] The present invention has the following beneficial effects:

[0034] The satellite-UAV cross-view fixed facility matching algorithm of the present invention combining full convolutional feature extraction and attention feature matching reduces the difference between satellite images and UAV images and lowers the matching difficulty through a heterogeneous data consistency processing method; extracts the feature information in the images through a full convolutional feature extraction network based on sub-pixel convolution and score screening; matches the extracted feature information through a feature point matching network based on bidirectional hybrid attention and pruning screening strategy; calculates the position of the selected target in the satellite image in the UAV image according to the successfully matched feature point pairs.

[0035] The satellite-UAV cross-view fixed facility matching algorithm of the present invention combining full convolutional feature extraction and attention feature matching has an average center point error smaller than other methods in the prior art. The full convolutional feature extraction network based on sub-pixel convolution and score screening proposed in the method of the present invention can accurately extract more representative feature points, and the feature point matching network based on bidirectional hybrid attention and pruning screening strategy can effectively match the extracted feature points correctly, so as to generate accurate target matching results.

[0036] The satellite-UAV cross-view fixed facility matching algorithm of the present invention combining full convolutional feature extraction and attention feature matching can achieve relatively accurate matching for targets of different sizes and aspect ratios, and has strong robustness. Brief Description of the Drawings

[0037] The present invention will be further described in detail below with reference to the drawings and specific embodiments.

[0038] Figure 1 It is a schematic diagram of the overall process of the satellite-UAV cross-view fixed facility matching algorithm of the present invention combining full convolutional feature extraction and attention feature matching.

[0039] Figure 2 It is a schematic diagram of the full convolutional feature extraction network based on sub-pixel convolution and score screening.

[0040] Figure 3 It is a schematic diagram of the feature points extracted from the satellite image and the UAV image.

[0041] Figure 4 It is a schematic diagram of the feature point matching network based on bidirectional hybrid attention and pruning screening strategy.

[0042] Figure 5 Schematic diagram of pairs of matching feature points in satellite images and UAV images.

[0043] Figure 6 Schematic diagram of the targets selected in the satellite image and the targets matched in the UAV image. Specific implementation manners

[0044] The present invention will be described in detail below with reference to the accompanying drawings.

[0045] The satellite-UAV cross-view fixed facility matching algorithm of the present invention that combines full convolutional feature extraction and attention feature matching performs feature extraction and feature matching on the satellite image and the UAV image after the consistency processing of heterogeneous images, and converts the result of feature matching into the output in the UAV image to achieve the matching result at the target level. The flow of this method is as Figure 1 shown.

[0046] First, in order to alleviate the differences between the satellite image and the UAV image and reduce the matching difficulty of subsequent steps, a heterogeneous image consistency preprocessing method is first performed on the input satellite image and UAV image. Secondly, the present invention proposes a full convolutional feature extraction network based on sub-pixel convolution and score screening, and inputs the preprocessed image into this network to extract the feature information in the image. Then the present invention proposes a feature point matching network based on bidirectional hybrid attention and pruning screening strategy to screen and match the extracted feature vectors, and output pairs of matching points with a sufficiently high confidence level. Finally, calculate the position of the target in the UAV image according to the pairs of matching points.

[0047] The satellite-UAV cross-view fixed facility matching algorithm of the present invention that combines full convolutional feature extraction and attention feature matching includes the following steps:

[0048] Step 1: Heterogeneous image consistency preprocessing.

[0049] The heterogeneous image preprocessing method aims to reduce the differences such as different bands and different heading angles between the satellite image and the UAV image due to different image sources, and effectively reduce the difficulty of subsequent feature extraction algorithms and feature matching algorithms. For the panchromatic band satellite image, convert the UAV image into monochrome. For the RGB three-band satellite image, modify the brightness and contrast of the satellite image to reduce the difference in color space. In addition, obtain the heading angle of the UAV image, and rotate the satellite image so that the viewing angles of the satellite image and the UAV image tend to be consistent, reducing the robustness requirements for the rotational invariance of subsequent matching algorithms.

[0050] Step 2: Establish a full convolutional feature extraction network based on sub-pixel convolution and score screening.

[0051] Based on the excellent performance and potential demonstrated by convolutional neural networks in computer vision in recent years, the feature extraction network adopts a fully convolutional network structure. The feature extraction network receives preprocessed satellite images and drone images as inputs, and uses a convolutional neural network in the form of VGG to extract the basic feature information of the images. The dimension of the input image is H×W×3, and the size of the output basic feature map is 1 / 8 of the original image. After obtaining the basic feature map, feature compression is performed to reduce the dimension of the feature vector and thus reduce the computational amount. Subsequently, it is input into two branches, which are used to calculate the key point description and the feature operator respectively.

[0052] For the key point description branch, the input tensor F Pos After SoftMax normalization, the obtained normalized map contains the feature information at each position in the image. To calculate the accurate position of the feature points, feature response needs to be performed on the normalized map, and sub-pixel convolution is used for upsampling to output the heat map H Pos ∈H×W×1, and the size of the output heat map is the same as that of the original input image. Finally, the positions of the k feature points with the highest scores are obtained through feature screening, and coordinate conversion is performed as the output of the key point description branch, and the output tensor T Pos ∈k×2, where k is the number of output feature points, and 2 represents the two-dimensional coordinates of the feature point positions.

[0053] For the feature operator branch, first pass through a 1×1 convolutional layer to increase the number of channels of the feature tensor to 256. At this time, the obtained F Des Each pixel position actually represents the feature information of an 8×8 pixel area in the original input image. To ensure that each feature point position output by the key point description branch has a corresponding feature representation, feature alignment is required. To reduce the computational complexity, feature alignment is decoded using a fixed parameter-free method. For F Des Bicubic interpolation is used for upsampling to output the feature representation of the image The size is the same as that of the input image. To make the feature length consistent, the obtained feature representation is normalized, and finally the feature operator T of the image is obtained Des ∈k×256. k is the number of output feature points, which is the same as the key point description branch, and 256 represents the feature vector of each output feature point.

[0054] After the two branches calculate the key point description and the feature operator respectively, the feature points T output by the fully convolutional feature extraction network based on sub-pixel convolution and score screening Feat ∈k×2×256.

[0055] The positions of the feature points extracted from the satellite images and drone images after feature extraction are as Figure 3 shown.

[0056] Step 3: Establish a feature point matching network based on bidirectional hybrid attention and pruning screening strategy.

[0057] The feature extraction network extracts the feature information in the satellite image and the UAV image and outputs the corresponding feature vectors. The feature matching network performs similarity measurement on the feature vectors and screens out the homologous feature points. In recent years, with the excellent performance of the Transformer structure in the field of natural language processing, many excellent algorithms based on the Transformer structure have emerged in the field of computer vision. Introducing the idea of the Transformer architecture into the feature matching task, the self-attention module adaptively adjusts the weight of each feature according to the different input data, and can more flexibly capture the global context information of the feature sequence; the mutual-attention module calculates the correlation relationship between heterogeneous features, better captures the similarity and correlation between the matching points, and at the same time weakens the influence of irrelevant or noisy parts, thereby improving the accuracy of feature matching. Therefore, the feature matching network introduces the self-attention and mutual-attention mechanisms in the Transformer architecture to construct a feature point matching network based on bidirectional hybrid attention and pruning screening strategy.

[0058] In the self-attention module, for the feature vector f at each position i , and the feature vector f at another position in the image j the correlation relationship sa ij is expressed as:

[0059]

[0060] Among them, q and k are the query vector and key vector obtained by performing different linear transformations on the input vector f in the Transformer structure, and R is the rotation position encoding, which stores the relative position relationship between two feature points. In the mutual-attention module, for each feature vector and any feature vector in another image the correlation relationship

[0061]

[0062] Among them, k is also the key vector obtained by performing a linear transformation on the input vector p.

[0063] After passing through the bidirectional hybrid attention network, the confidence of each feature vector is calculated. A high confidence indicates that the feature points at this position can be successfully matched with the corresponding feature points. Whether the output conditions are met is judged based on the confidence of all feature points. If the output conditions are not met, point pruning is performed to delete the feature points with lower confidence. These feature points are considered isolated feature points and there are no feature points in the other image that can match them. After multiple loops until the exit condition is met, a sufficient number of feature point pairs with sufficient confidence are output.

[0064] For the features extracted from the satellite image and the UAV image, after feature matching, the positional relationship of the corresponding feature point pairs is as Figure 5 shown.

[0065] Step 4: Homography post-processing to calculate the output result.

[0066] The method proposed by the present invention is used to find the specified fixed facility target in the satellite image in the UAV image. After the previous steps of feature extraction and feature matching, a series of matching point pairs are obtained. These matching point pairs can be regarded as the projection points of the same position in the actual scene in the satellite image and the UAV image respectively. Therefore, the homography matrix from the satellite image to the UAV image can be calculated based on these matching point pairs, and the target position in the UAV image can be obtained by applying the homography matrix to the target position in the satellite image.

[0067] The satellite-UAV cross-view fixed facility matching algorithm of the present invention that combines full convolutional feature extraction and attention feature matching reduces the difference between the satellite image and the UAV image and the matching difficulty through the heterologous data consistency processing method; extracts the feature information in the image through the full convolutional feature extraction network based on sub-pixel convolution and score screening; matches the extracted feature information through the feature point matching network based on bidirectional hybrid attention and pruning screening strategy; finally calculates the position of the selected target in the satellite image in the UAV image according to the successfully matched feature point pairs.

[0068] In terms of experiments, since there has been no in-depth research on this task in the academic community, we actually selected several scenarios, used the "Jilin-1" satellite to take satellite images of the selected scenarios, and used a DJI UAV to take on-site photos of the selected scenarios to construct a test data set.

[0069] The parameters of the satellite images and UAV images in the test data set are shown in Table 1:

[0070] Table 1 Parameters of satellite images and UAV images in the test data set

[0071] Satellite image UAV image Image band Panchromatic single band, RGB multi - band RGB multi - band Image input size 1280×1280 1920×1080 Pixel resolution 0.75 m 0.2 m to 0.4 m

[0072] To comprehensively evaluate the performance of the matching algorithm, the following two evaluation metrics are adopted in the experimental part:

[0073] (1) Matching success rate:

[0074] The calculation method of the matching success rate refers to the accuracy calculation method of the object detection task. After the algorithm generates the predicted object bounding box, the intersection over union (IoU) of the predicted bounding box and the actual bounding box of the object is calculated. We select the results with an IoU greater than 50%, and consider the current match as successful. The calculation method of the matching success rate is the ratio of the number of successful matches (true positives, TP) to the total number of all matching results (the sum of true positives and false positives, TP + FP). The higher the matching success rate, the higher the accuracy of the algorithm is considered to be.

[0075] (2) Average center point error:

[0076] On the basis of successful matching, the closer the predicted object box is to the true object box, the higher the accuracy of the algorithm is represented. However, calculating the matching accuracy through IoU cannot accurately reflect the degree of overlap between the two boxes. In practical applications, for a specified object, people are more concerned about its center position. Therefore, the average center point error is adopted in the experiment to comprehensively evaluate the performance of the algorithm. Considering the different scales of different objects, the calculation method of the center point error is: on the basis of successful matching, the ratio of the distance between the center points of the predicted object box and the true object box to the length of the true object box in the direction of the line connecting the two center points. During testing, the successful center point errors in each group of experiments are taken, and their average value is calculated to obtain the average center point error.

[0077] In the experiment, several methods of image matching and correlation filtering are selected for comparison, and the test results are shown in Table 2:

[0078] Table 2 Performance comparison of various matching methods

[0079] Matching success rate Average center point error Template matching method 25％ 30％ SIFT + RANSAC method 35％ 30％ NanoTrack algorithm 50％ 20％ DFM algorithm 70％ 20％ The method of the present invention 85％ 15％

[0080] As can be seen from Table 2, the matching success rate of the method of the present invention is significantly better than that of other algorithms. In addition, the average center point error of the method of the present invention is also smaller than that of other methods. This shows that the fully convolutional feature extraction network based on sub-pixel convolution and score screening proposed in the method of the present invention can accurately extract more representative feature points, and the feature point matching network based on bidirectional hybrid attention and pruning screening strategy can effectively match the extracted feature points correctly, so as to generate accurate object matching results.

[0081] Figure 6Specifically, it shows the results of selecting different targets in satellite images and performing target matching in UAV images. It can be seen that the method of the present invention can achieve relatively accurate matching for targets of different sizes and aspect ratios, proving that the method of the present invention has strong robustness.

[0082] The following combines specific examples to introduce in detail the satellite-UAV cross-view fixed facility matching algorithm of the present invention that combines full convolutional feature extraction and attention feature matching.

[0083] 1. Implementation scheme of heterologous image consistency preprocessing technology.

[0084] For the collected satellite images, UAV images and corresponding UAV flight data, first perform orthorectification and downsampling on the satellite images to convert the satellite images into a format that is easy for algorithm automation processing. Then, according to the heading angle of the UAV flight data, rotate the satellite images in the same direction to reduce the robustness requirements for subsequent algorithm steps. Finally, according to the bands of the satellite images, process the two images to make the color spaces of the two images tend to be consistent.

[0085] 2. Implementation scheme of full convolutional feature extraction algorithm based on sub-pixel convolution and score screening.

[0086] Send the input image into a convolutional neural network in the form of VGG to extract features layer by layer and output the basic feature map. Input the basic feature map into two branches respectively to calculate the feature point positions and feature descriptors. The feature point position branch restores the feature map size to the original image size through sub-pixel convolution operation, and screens out the positions of the k feature points with the highest scores through the score screening strategy. The feature descriptor branch first generates a semi-dense feature representation, uses bicubic interpolation to keep the size of the feature representation consistent with the input image, and then uses the L2 regularization method to align the lengths of the feature descriptors. Finally, output the positions of the k feature points and the corresponding feature descriptors.

[0087] 3. Implementation scheme of feature point matching algorithm based on bidirectional hybrid attention and pruning screening strategy.

[0088] Input the two groups of feature vectors into the bidirectional hybrid attention network. The self-attention module calculates the correlation between each position and the features of other positions in the image, and the mutual-attention module calculates the feature correlation between the two groups of feature tensors. After passing through the attention network, calculate the confidence for each feature point. A high confidence represents that the feature can be correctly matched. If the exit condition is not met, perform point pruning to delete the feature points that cannot be matched. After multiple loops until the exit condition is reached, output enough and correct matching point pairs.

[0089] 4. Implementation scheme of homography post-processing.

[0090] Process the matching point pairs in the two acquired images into a format that can be automatically processed by the algorithm. Calculate the homography matrix of the change from the satellite image to the UAV image through the matching point pairs, and transform the target in the satellite image into the UAV image through the homography matrix to obtain the output result. Perform anomaly detection on the result in the UAV image. If the result in the UAV image is an abnormal result, output the information of failed matching. If the result is not an abnormal result, output the information of successful matching.

[0091] Obviously, the above embodiments are only examples for clear illustration and not limitations on the implementation manners. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to enumerate all the implementation manners here. And the obvious changes or modifications derived therefrom are still within the protection scope of the present invention.

Claims

1. A satellite-UAV cross-view fixed facility matching algorithm combining full convolution feature extraction and attention feature matching, characterized in that: The following steps are involved: Step 1: Preprocessing of heterogeneous images for consistency; Step 2: Establish a fully convolutional feature extraction network based on sub-pixel convolution and score screening; Step 3: Establish a feature point matching network based on bidirectional hybrid attention and pruning screening strategy; Step 4: Homography post-processing, calculate the output results.

2. According to claim 1, the satellite-UAV cross-view fixed facility matching algorithm combining full convolution feature extraction and attention feature matching is characterized in that: Step 1 is as follows: For panchromatic satellite images, convert the drone images to monochrome; for RGB three-band satellite images, modify the brightness and contrast of satellite images to reduce the difference in color space; The heading angle of the drone image is obtained and the satellite image is rotated to make the viewing angle of the satellite image consistent with that of the drone image, thus reducing the robustness requirements for the rotation invariance of the subsequent matching algorithm.

3. According to claim 1, the satellite-UAV cross-view fixed facility matching algorithm combining full convolution feature extraction and attention feature matching is characterized in that: Step 2 is as follows: After obtaining the basic feature map, feature compression is performed to reduce the dimension of the feature vector and thus reduce the amount of calculation; it is then input into the key point description branch and the feature operator branch respectively to calculate the key point description and feature operator respectively; In the key point description branch, the input tensor F Pos After SoftMax normalization, the obtained normalized image contains the feature information of each position in the image; the normalized image is subjected to feature response, up-sampled using sub-pixel convolution, and the output heat map H is obtained. Pos ∈H×W×1, the output heat map is consistent with the size of the original input image; finally, the k feature points with the highest scores are obtained through feature screening, and the coordinates are transformed as the output of the key point description branch, and the output tensor T Pos ∈k×2, k is the number of feature points output, and 2 represents the two-dimensional coordinates of the feature point position; In the feature operator branch, a 1×1 convolution layer is first used to increase the number of channels of the feature tensor to 256. Des Each pixel position actually represents the feature information of an 8×8 pixel area of ​​the original input image; Perform feature alignment; feature Alignment uses a fixed parameter-free method for decoding; Des Use bicubic interpolation to upsample and output the feature representation of the image The size is consistent with the input image size; the obtained feature representation is normalized, and finally the feature operator T of the image is obtained Des ∈k×256; k is the number of output feature points, which is consistent with the key point description branch, and 256 represents the feature vector of each output feature point; After the key point description branch and the feature operator branch are calculated to obtain the key point description and feature operator respectively, the feature point T output by the full convolution feature extraction network based on sub-pixel convolution and score screening is Feat ∈k×2×256.

4. The satellite-UAV cross-view fixed facility matching algorithm combining full convolution feature extraction and attention feature matching according to claim 1 is characterized in that: Step 3 is as follows: The feature extraction network extracts the feature information from satellite images and drone images and outputs the corresponding feature vectors. The feature matching network measures the similarity of the feature vectors and selects the feature points with the same name. In the self-attention module, for each position feature vector f i , and the feature vector f at another location in the image j The relationship between ij It is expressed as: Among them, q and k are the query vector and key vector obtained by performing different linear transformations on the input vector f in the Transformer structure, and R is the rotation position encoding, which stores the relative position relationship between two feature points; In the mutual attention module, for each feature vector with any feature vector in another image Correlation It is expressed as: Among them, k is also the key vector obtained by linear transformation of the input vector p; After passing through the bidirectional hybrid attention network, the confidence of each feature vector is calculated; the confidence of all feature points is used to determine whether the output conditions are met; if the output conditions are not met, point pruning is performed to delete points with lower confidence. These feature points are considered to be isolated feature points, and there are no feature points that can match them in the other image; multiple cycles are repeated until the exit conditions are met, and enough feature point pairs with high enough confidence are output.

5. The satellite-UAV cross-view fixed facility matching algorithm combining full convolution feature extraction and attention feature matching according to claim 1 is characterized in that: Step 4 is specifically as follows: a homography matrix from the satellite image to the UAV image is calculated based on the matching point pairs, and the homography matrix is ​​applied to the target position in the satellite image to obtain the target position in the UAV image.

Images