Registration method of infrared and visible light remote sensing images

Through semi-dense end-to-end matching algorithm and deep learning feature extraction, combined with HNSW and soft constraint method, the data set shortage and radiation distortion problems in infrared and visible light remote sensing image registration are solved, and efficient and robust matching effect is achieved.

CN117557605BActive Publication Date: 2024-11-19CHANGCHUN INST OF OPTICS FINE MECHANICS & PHYSICS CHINESE ACAD OF SCI
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311848593.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-29
Publication Date
2024-11-19
Estimated Expiration
2043-12-29

AI Technical Summary

Technical Problem

The prior art has insufficient data sets and weak characteristic similarity caused by radiation distortion in infrared and visible remote sensing image registration. The cumulative error of multi-stage deep learning algorithms is large, and it is highly dependent on edges and corners, making it difficult to use in challenging scenarios. Get good matching results.

Method used

Using a semi-dense end-to-end matching algorithm without detectors, multi-level features are extracted through Resnet, combined with Transformer network for coarse-grained and fine-grained feature extraction, HNSW algorithm is used for nearest neighbor search, soft constraint method eliminates pseudo-match points, and multi-constraint The incremental matching method builds a spatial transformation model, and the outer point removal method obtains the final matching result.

Benefits of technology

The matching of texture and corners is achieved without dependence, which improves matching robustness and computing efficiency, reduces dependence on specific data sets, and improves the sensitivity and final matching effect of radiation distortion.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117557605B_ABST
    Figure CN117557605B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of image processing technology, and in particular to a registration method for infrared and visible light remote sensing images. The method comprises: S1: inputting the infrared and visible light remote sensing image pairs to be registered into the Resnet extraction network to obtain 1 / 8 and 1 / 2 resolution feature maps; S2: inputting the 1 / 8 resolution feature map into the Transformer network for coarse-grained feature extraction to obtain a coarse-grained feature map; S3: using the HNSW algorithm to perform nearest neighbor search and matching on the coarse-grained feature map to obtain a preliminary matching point set; S4: eliminating pseudo matching points in the preliminary matching point set to obtain a control point set; S5: solving the control point set to obtain a spatial transformation model matrix; S6: constructing a coarse-grained matching point set; S7: using an external point elimination method to eliminate external points of the fine-grained feature map to obtain the final matching result. The present invention solves the problems of sparse texture and radiation distortion existing in heterogeneous images.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a registration method for infrared and visible light remote sensing images. Background Art

[0002] Before the emergence of deep learning algorithms, methods based on handcrafted descriptors were widely used in various registration tasks. However, due to the design limitations of existing knowledge, traditional matching algorithms often perform poorly when dealing with more complex matching tasks due to handcrafted descriptors. With the vigorous development of deep learning algorithms, their deep feature extraction and expression capabilities can make up for the defects of shallow feature instability and improve the robustness of registration algorithms. Therefore, deep learning algorithms are applied to feature-based registration problems, while multi-stage deep learning algorithms cannot avoid the problem of cumulative errors in traditional multi-stage registration. In addition, the feature detection method used relies on edges and corners. For challenging scenes with blurred edges or repeated textures, it is difficult to extract enough points of interest to obtain good descriptors, resulting in poor matching results. The cost of achieving global attention in the Transformer model is that it requires tens of thousands or even more data samples to train a better model. At the same time, it consumes a lot of computing resources and requires the use of large-scale GPU clusters for training. Summary of the invention

[0003] In order to solve the problems in the prior art of less annotated data sets of heterogeneous remote sensing images and weak feature similarity due to radiation distortion, the present invention provides a registration method for infrared and visible light remote sensing images, adopts a detector-free semi-dense end-to-end matching algorithm, and solves the problem of the existing algorithm's dependence on corner points and edge points.

[0004] The infrared and visible light remote sensing image registration method proposed in the present invention specifically comprises the following steps:

[0005] S1: Input the infrared and visible remote sensing images to be registered into the Resnet extraction network for multi-level feature extraction to obtain 1 / 8 resolution feature maps and 1 / 2 resolution feature maps;

[0006] S2: Input the 1 / 8 resolution feature map into the Transformer network for coarse-grained feature extraction, and obtain the coarse-grained feature maps of the infrared remote sensing image and the visible light remote sensing image;

[0007] S3: Use the HNSW algorithm to perform nearest neighbor search and bidirectional matching on the coarse-grained feature maps of infrared remote sensing images and visible light remote sensing images to obtain a preliminary matching point set;

[0008] S4: Use the soft constraint method to establish a geometric consistency error judgment formula, and use the geometric consistency error judgment formula to eliminate false matching points in the preliminary matching point set to obtain a control point set;

[0009] S5: Solve the control point set using the least squares fitting method to obtain the spatial transformation model matrix;

[0010] S6: Based on the spatial transformation model matrix, a coarse-grained matching point set is constructed using a multi-constrained incremental matching method;

[0011] S7: Establish a local feature window, input the local feature window into the Transformer network for feature fusion, obtain a fine-grained feature map, use the outlier removal method to remove the outliers of the fine-grained feature map, and obtain the final matching result.

[0012] Preferably, step S3 specifically includes the following steps:

[0013] S31: Establish a to-be-matched index and a template index, and input the coarse-grained feature maps of the infrared remote sensing image and the visible light remote sensing image as the to-be-matched image and the template image respectively to the to-be-matched index and the template index for k-neighborhood search, and obtain the k-nearest neighbor sets of the to-be-matched image and the template image accordingly:

[0014] (1);

[0015] (2);

[0016] in, is the k nearest neighbor set of the image to be matched, is the k nearest neighbor set of the template image, is the image to be matched Points Neighbor distance, The template image Points Neighbor distance, is the point of the template image corresponding to the image to be matched, is the point in the template image corresponding to the image to be matched;

[0017] S32: Filter the external points of the image to be matched and the template image by the following formula, match the nearest neighbors of the filtered image to be matched and the template image, and obtain a preliminary matching point set, including the template point set and the point set to be matched:

[0018] (3);

[0019] (4);

[0020] in, is the threshold value, .

[0021] Preferably, step S4 specifically includes the following steps:

[0022] S41: Determine a target matching point pair in the preliminary matching point set, and randomly select five matching point pairs other than the target matching point pair in the preliminary matching point set to establish a local random matrix of the image to be matched and the template image:

[0023] (5);

[0024] (6);

[0025] in, is the local random matrix of the image to be matched, is the local random matrix of the template image, is the target matching point in the image to be matched The coordinates of the center point , , To remove the The coordinates of the randomly selected matching points among the remaining matching points outside the point , , is the first target matching point in the template image. The coordinates of the center point , , To remove the The coordinates of the randomly selected matching points among the remaining matching points outside the point , , is the total number of preliminary matching point sets;

[0026] S42: According to the calculation result of step S41, the Euclidean distance between the target matching point and the remaining matching points in the image to be matched and the template image is calculated by the following formula:

[0027] (7);

[0028] (8);

[0029] in, is the Euclidean distance of the image to be matched, is the Euclidean distance of the template image;

[0030] S43: In the image to be matched and the template image, connect the target matching points with the remaining matching points one by one, and calculate the angle between the adjacent connecting lines of the target matching points and the remaining matching points by the following formula:

[0031] (9);

[0032] (10);

[0033] in, is the angle between the target matching point of the image to be matched and the adjacent connecting lines of the remaining matching points, The angle between the target matching point of the template image and the adjacent connecting lines of the remaining matching points;

[0034] S44: According to equations (5) to (10), a geometric consistency error determination formula is established:

[0035] (11);

[0036] (12)

[0037] (13);

[0038] (14);

[0039] in, is the angle error, is the length error; The average value of the length ratio between the line segment formed by the two matching points of the template image and the line segment of the two matching points of the image to be matched;

[0040] S45: inputting all matching point pairs in the preliminary matching point set into the geometric consistency error determination formula in sequence to eliminate false matching point pairs, and obtaining a control point set.

[0041] Preferably, step S5 specifically includes the following steps:

[0042] S51: Randomly select control point pairs from the control point set to perform polynomial fitting:

[0043] (15);

[0044] in, and is the control point pair, a, b, c, d, e, f, g and h are the parameters to be fitted;

[0045] S52: Transform equation (15) into the least squares matrix form:

[0046] (16);

[0047] S53: Substitute all control point pairs in the control point set into equation (16) in sequence:

[0048] (17);

[0049] (18);

[0050] in, and is the control point set pair, W is the space transformation model matrix;

[0051] S54: Solve the transformation space transformation model matrix W according to formula (18):

[0052] (19);

[0053] in, The control point set The transposed matrix of .

[0054] Preferably, step S6 specifically includes the following steps:

[0055] S61: After removing the preliminary matching point set from all the point sets included in the 1 / 2 resolution feature map, a remaining matching point set is obtained, and after multiplying the remaining matching point set with the spatial transformation model matrix, a desired matching point set is obtained;

[0056] S62: saving all the matching point pairs that are two-way three nearest neighbors to each other in the expected matching point set after removing the error points as internal points, and obtaining a two-way three nearest neighbor matching point set;

[0057] S63: taking all the matching point pairs that are not two-way three nearest neighbors in the expected matching point set after removing the error points as the first column of point set pairs of the spatial similarity matching point set, and randomly selecting control point pairs in the control point set as the last five columns of point set pairs of the spatial similarity matching point set, thereby completing the construction of the spatial similarity matching point set;

[0058] S64: Construct a coarse-grained matching point set using the control point set, the bidirectional three-nearest neighbor matching point set and the spatial similarity matching point set.

[0059] Preferably, step S7 specifically includes the following steps:

[0060] S71: Map the coarse-grained matching point set to the point set of the 1 / 2 resolution feature map, and crop the mapped 1 / 2 resolution feature map into a pair of local feature windows of size 5*5;

[0061] S72: Input the local feature window into the Transformer network for feature fusion to obtain a fine-grained feature map;

[0062] S73: After similarity calculation is performed on the fine-grained feature map and desired fine-tuning is performed, external points are eliminated from the fine-grained feature map after desired fine-tuning by the following formula to obtain the final matching result:

[0063] (20);

[0064] in, is the homography matrix, T is the threshold constant, T=2, It is the L2 paradigm.

[0065] Preferably, the Resnet extraction network includes a feature pyramid extraction network.

[0066] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0067] (1) The present invention adopts an end-to-end registration framework from coarse to fine, so that the matching no longer depends on texture and corner points. That is, the Transformer architecture based on zero-shot learning is adopted, which proves the effectiveness of the pre-trained model and makes data-driven no longer limited by data sets in specific fields.

[0068] (2) The present invention adopts a knowledge-driven approach in the coarse-grained matching stage, that is, the HNSW algorithm is introduced in the field of deep learning image registration to efficiently and accurately obtain a wide range of correspondences, and a soft constraint method is used to establish a geometric consistency error judgment formula to obtain a control point set.

[0069] (3) The present invention adopts a multi-constrained incremental matching method to screen among the predicted matching points, making the image registration insensitive to the overall radiation distortion. In addition, the present invention proposes a simple and effective external point elimination method that does not require iterative calculation, thereby effectively improving the final matching effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 It is a schematic diagram of the process of a registration method of infrared and visible light remote sensing images provided according to an embodiment of the present invention;

[0071] Figure 2 is a structural block diagram of a registration method for infrared and visible light remote sensing images provided according to an embodiment of the present invention;

[0072] Figure 3 is a structural block diagram of a HMSW algorithm provided according to an embodiment of the present invention;

[0073] Figure 4 is a structural diagram of angle calculation between a template image and an image to be matched according to an embodiment of the present invention;

[0074] Figure 5It is a schematic diagram of the effect of the infrared and visible light remote sensing image registration method provided by an embodiment of the present invention in an urban environment;

[0075] Figure 6 It is a schematic diagram of the effect of the registration method of infrared and visible light remote sensing images provided by an embodiment of the present invention in a desert environment;

[0076] Figure 7 It is a schematic diagram of the effect of the registration method of infrared and visible light remote sensing images provided by an embodiment of the present invention in a mountain environment;

[0077] Figure 8 It is a schematic diagram of the effect of the infrared and visible light remote sensing image registration method provided by an embodiment of the present invention in a coastal environment. DETAILED DESCRIPTION

[0078] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, the same modules are represented by the same reference numerals. In the case of the same reference numerals, their names and functions are also the same. Therefore, the detailed description thereof will not be repeated.

[0079] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation of the present invention.

[0080] Figure 1 The flowchart of the registration method of infrared and visible light remote sensing images provided in accordance with an embodiment of the present invention is shown. Figure 2 The structure of the registration method of infrared and visible light remote sensing images provided according to an embodiment of the present invention is shown.

[0081] like Figure 1-Figure 2 As shown, the registration method of infrared and visible light remote sensing images proposed in the present invention specifically includes the following steps:

[0082] S1: Input the infrared and visible light remote sensing image pairs to be registered into the Resnet extraction network for multi-level feature extraction to obtain 1 / 8 resolution feature maps and 1 / 2 resolution feature maps.

[0083] The Resnet extraction network includes a feature pyramid extraction network.

[0084] S2: Input the 1 / 8 resolution feature map into the Transformer network for coarse-grained feature extraction, and obtain the coarse-grained feature maps of the infrared remote sensing image and the visible light remote sensing image.

[0085] S3: Use the HNSW algorithm to perform nearest neighbor search and bidirectional matching on the coarse-grained feature maps of infrared remote sensing images and visible light remote sensing images to obtain a preliminary matching point set.

[0086] Step S3 specifically includes the following steps:

[0087] S31: Establish a to-be-matched index and a template index, and input the coarse-grained feature maps of the infrared remote sensing image and the visible light remote sensing image as the to-be-matched image and the template image respectively to the to-be-matched index and the template index for k-neighborhood search, and obtain the k-nearest neighbor sets of the to-be-matched image and the template image accordingly:

[0088] (1);

[0089] (2);

[0090] in, is the k nearest neighbor set of the image to be matched, is the k nearest neighbor set of the template image, is the image to be matched Points Neighbor distance, The template image Points Neighbor distance, is the point of the template image corresponding to the image to be matched, is the point in the template image corresponding to the image to be matched;

[0091] S32: Filter the external points of the image to be matched and the template image by the following formula, match the nearest neighbors of the filtered image to be matched and the template image, and obtain a preliminary matching point set, including the template point set and the point set to be matched:

[0092] (3);

[0093] (4);

[0094] in, is the threshold value, .

[0095] S4: Use the soft constraint method to establish a geometric consistency error judgment formula, and use the geometric consistency error judgment formula to eliminate pseudo matching points in the preliminary matching point set to obtain a control point set.

[0096] Step S4 specifically includes the following steps:

[0097] S41: Determine a target matching point pair in the preliminary matching point set, and randomly select five matching point pairs other than the target matching point pair in the preliminary matching point set to establish a local random matrix of the image to be matched and the template image:

[0098] (5);

[0099] (6);

[0100] in, is the local random matrix of the image to be matched, is the local random matrix of the template image, is the target matching point in the image to be matched The coordinates of the center point , , To remove the The coordinates of the randomly selected matching points among the remaining matching points outside the point , , is the first target matching point in the template image. The coordinates of the center point , , To remove the The coordinates of the randomly selected matching points among the remaining matching points outside the point , , is the total number of preliminary matching point sets;

[0101] S42: According to the calculation result of step S41, the Euclidean distance between the target matching point and the remaining matching points in the image to be matched and the template image is calculated by the following formula:

[0102] (7);

[0103] (8);

[0104] in, is the Euclidean distance of the image to be matched, is the Euclidean distance of the template image;

[0105] S43: In the image to be matched and the template image, connect the target matching points with the remaining matching points one by one, and calculate the angle between the adjacent connecting lines of the target matching points and the remaining matching points by the following formula:

[0106] (9);

[0107] (10);

[0108] in, is the angle between the target matching point of the image to be matched and the adjacent connecting lines of the remaining matching points, The angle between the target matching point of the template image and the adjacent connecting lines of the remaining matching points;

[0109] S44: According to equations (5) to (10), a geometric consistency error determination formula is established:

[0110] (11);

[0111] (12)

[0112] (13);

[0113] (14);

[0114] in, is the angle error, is the length error; It is the average value of the length ratio between the line segment formed by the two matching points of the template image and the line segment of the two matching points of the image to be matched.

[0115] S45: inputting all matching point pairs in the preliminary matching point set into the geometric consistency error determination formula in sequence to eliminate false matching point pairs, and obtaining a control point set.

[0116] S5: Use the least squares fitting method to solve the control point set and obtain the spatial transformation model matrix.

[0117] Step S5 specifically includes the following steps:

[0118] S51: Randomly select control point pairs from the control point set to perform polynomial fitting:

[0119] (15);

[0120] in, and is the control point pair, a, b, c, d, e, f, g and h are the parameters to be fitted;

[0121] S52: Transform equation (15) into the least squares matrix form:

[0122] (16);

[0123] S53: Substitute all control point pairs in the control point set into equation (16) in sequence:

[0124] (17);

[0125] (18);

[0126] in, and is the control point set pair, W is the space transformation model matrix;

[0127] S54: Solve the transformation space transformation model matrix W according to formula (18):

[0128] (19);

[0129] in, The control point set The transposed matrix of .

[0130] S6: Based on the spatial transformation model matrix, a coarse-grained matching point set is constructed using a multi-constrained incremental matching method.

[0131] Step S6 specifically includes the following steps:

[0132] S61: After removing the preliminary matching point set from all the point sets included in the 1 / 2 resolution feature map, a remaining matching point set is obtained, and after multiplying the remaining matching point set with the spatial transformation model matrix, a desired matching point set is obtained;

[0133] S62: saving all the matching point pairs that are two-way three nearest neighbors to each other in the expected matching point set after removing the error points as internal points, and obtaining a two-way three nearest neighbor matching point set;

[0134] S63: taking all the matching point pairs that are not two-way three nearest neighbors in the expected matching point set after removing the error points as the first column of point set pairs of the spatial similarity matching point set, and randomly selecting control point pairs in the control point set as the last five columns of point set pairs of the spatial similarity matching point set, thereby completing the construction of the spatial similarity matching point set;

[0135] S64: Construct a coarse-grained matching point set using the control point set, the bidirectional three-nearest neighbor matching point set and the spatial similarity matching point set.

[0136] S7: Establish a local feature window, input the local feature window into the Transformer network for feature fusion, obtain a fine-grained feature map, use the outlier removal method to remove the outliers of the fine-grained feature map, and obtain the final matching result.

[0137] Step S7 specifically includes the following steps:

[0138] S71: Map the coarse-grained matching point set to the point set of the 1 / 2 resolution feature map, and crop the mapped 1 / 2 resolution feature map into a pair of local feature windows of size 5*5;

[0139] S72: Input the local feature window into the Transformer network for feature fusion to obtain a fine-grained feature map;

[0140] S73: After similarity calculation is performed on the fine-grained feature map and desired fine-tuning is performed, external points are eliminated from the fine-grained feature map after desired fine-tuning by the following formula to obtain the final matching result:

[0141] (20);

[0142] in, is the homography matrix, T is the threshold constant, T=2, It is the L2 paradigm.

[0143] Before the 1 / 8 resolution feature map is input into the Transformer network, sine and cosine position encoding is required to make the features output by the Transformer network relevant to the position, thus improving the model's ability to express features of sparse textures and repeated areas in the 1 / 8 resolution feature map after position encoding. The Transformer network is composed of multiple alternating self-attention layers and cross-attention layers. The self-attention layer enables each point to pay attention to all other points around it in an image to capture the relationship between them, while the cross-attention layer enables each point to pay attention to all points in another image. By utilizing the excellent characteristics of the global attention of the Transformer network, the features of each point are integrated with the contextual information of other related points, obtaining a richer expression and a coarse-grained feature map.

[0144] In practical applications, it is difficult to match infrared and visible light remote sensing images from different sources, mainly due to different imaging mechanisms, large nonlinear spectral radiation distortion, and obvious grayscale gradient differences. In response to the above problems, the embodiment of the present invention proposes to use the HNSW algorithm to obtain a preliminary matching point set and use the idea of ​​geometric consistency to construct a geometric consistency error judgment formula to eliminate false matching points.

[0145] Figure 3 The structure of the HMSW algorithm provided according to an embodiment of the present invention is shown.

[0146] The HNSW algorithm is an efficient approximate nearest neighbor search algorithm, which is not widely used in the field of image registration. It builds a hierarchical graph index structure and implements jump search between layers and within layers to greatly reduce the amount of distance calculation, thereby accelerating the nearest neighbor search process. Figure 3As shown in the figure, an index to be matched and a template index are established, and the coarse-grained feature maps of the infrared remote sensing image and the visible light remote sensing image are respectively input as the image to be matched and the template image to the index to be matched and the template index to perform a k-neighbor search, and the k-nearest neighbor sets of the image to be matched and the template image are obtained accordingly, the nearest neighbors and the next nearest neighbors are filtered out by using the proportional filtering in the SIFT algorithm, and the nearest neighbors of the image to be matched and the template image are matched to obtain a preliminary matching point set.

[0147] Since the geometric structure properties are less affected by radiation differences, and the heterogeneous remote sensing images taken by satellites usually do not have a large viewing angle deviation in a small area, and the viewing angle of satellite remote sensing images is relatively high, the ground can be regarded as an approximate plane, and the scene depth is relatively small, which makes the remote sensing image can be regarded as a two-dimensional plane projection, satisfying the condition of similar spatial structure. When the image does not contain serious affine transformation or projective transformation, the spatial structure between the correct matching points is similar. According to this feature, the mismatching points can be eliminated by calculating the Euclidean distance and direction angle between its adjacent matching points. The core idea of ​​geometric consistency is: for all correct matching point pairs, the distance ratio of any two matching point pairs should be equal or approximately equal, and the distance ratio should also be approximately equal to the true scale ratio of the two images, and the angle formed by any three matching point pairs should be approximately equal. Therefore, the distance ratio of all correct matching point pairs can form a class centered on the true scale ratio. The matching point pairs far from the class center are mismatching points, and the angles between matching point pairs should also meet the corresponding approximately equal constraints. Therefore, based on the above idea, the present invention uses the soft constraint method to establish a geometric consistency error judgment formula to eliminate false matching points.

[0148] In order to test the registration effect of the infrared and visible light remote sensing image registration method proposed in the embodiment of the present invention in different environments, the present invention has carried out image registration in four environments: city, desert, mountain and coastal. The results are as follows: Figure 5-Figure 8 As shown, the registration method of infrared and visible light remote sensing images proposed in the embodiment of the present invention can be applied to different registration environments.

[0149] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the disclosure of the present invention can be performed in parallel, sequentially or in different orders, as long as the desired results of the technical solution disclosed in the present invention can be achieved, and this document does not limit this.

[0150] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A registration method for infrared and visible light remote sensing images, characterized in that: The specific steps include: S1: Input the infrared and visible remote sensing images to be registered into the Resnet extraction network for multi-level feature extraction to obtain 1 / 8 resolution feature maps and 1 / 2 resolution feature maps; S2: Input the 1 / 8 resolution feature map into the Transformer network for coarse-grained feature extraction, and obtain the coarse-grained feature maps of the infrared remote sensing image and the visible light remote sensing image; S3: using the HNSW algorithm to perform nearest neighbor search and bidirectional matching on the coarse-grained feature maps of the infrared remote sensing image and the visible light remote sensing image to obtain a preliminary matching point set; The step S3 specifically includes the following steps: S31: Establishing a to-be-matched index and a template index, inputting the coarse-grained feature maps of the infrared remote sensing image and the visible light remote sensing image as the to-be-matched image and the template image respectively into the to-be-matched index and the template index to perform a k-neighborhood search, and obtaining a k-nearest neighbor set of the to-be-matched image and the template image accordingly: (1); (2); in, is the k nearest neighbor set of the image to be matched, is the k nearest neighbor set of the template image, is the image to be matched Points Neighbor distance, The template image Points Neighbor distance, is the point of the template image corresponding to the image to be matched, is the point in the template image corresponding to the image to be matched; S32: Filter the external points of the image to be matched and the template image by the following formula, match the nearest neighbors of the image to be matched and the template image after filtering, and obtain a preliminary matching point set, including the template point set and the point set to be matched: (3); (4); in, is the threshold value, ; S4: using a soft constraint method to establish a geometric consistency error determination formula, and using the geometric consistency error determination formula to eliminate pseudo matching points in the preliminary matching point set to obtain a control point set; The step S4 specifically includes the following steps: S41: Determine a target matching point pair in the preliminary matching point set, and randomly select five matching point pairs other than the target matching point pair in the preliminary matching point set to establish a local random matrix of the image to be matched and the template image: (5); (6); in, is the local random matrix of the image to be matched, is the local random matrix of the template image, is the target matching point in the image to be matched The coordinates of the center point , , To remove the The coordinates of the randomly selected matching points among the remaining matching points outside the point , , is the first target matching point in the template image. The coordinates of the center point , , To remove the The coordinates of the randomly selected matching points among the remaining matching points outside the point , , is the total number of preliminary matching point sets; S42: According to the calculation result of step S41, the Euclidean distance between the target matching point and the remaining matching points in the image to be matched and the template image is calculated by the following formula: (7); (8); in, is the Euclidean distance of the image to be matched, is the Euclidean distance of the template image; S43: In the image to be matched and the template image, the target matching points are connected with the remaining matching points one by one, and the angle between the adjacent connecting lines of the target matching points and the remaining matching points is calculated by the following formula: (9); (10); in, is the angle between the adjacent lines of the target matching point and the remaining matching points of the image to be matched, An angle between adjacent lines between the target matching point of the template image and the remaining matching points; S44: According to formula (5)-formula (10), a geometric consistency error determination formula is established: (11); (12) (13); (14); in, is the angle error, is the length error; The average value of the length ratio between the line segment formed by the two matching points of the template image and the line segment of the two matching points of the image to be matched; S45: inputting all the matching point pairs in the preliminary matching point set into the geometric consistency error determination formula in sequence to eliminate false matching point pairs, and obtaining a control point set; S5: solving the control point set using the least squares fitting method to obtain a spatial transformation model matrix; S6: constructing a coarse-grained matching point set using a multi-constraint incremental matching method according to the spatial transformation model matrix; S7: Establish a local feature window, input the local feature window into the Transformer network for feature fusion, obtain a fine-grained feature map, use an outlier elimination method to eliminate outliers in the fine-grained feature map, and obtain a final matching result.

2. The registration method of infrared and visible light remote sensing images according to claim 1, characterized in that: The step S5 specifically includes the following steps: S51: Randomly select a control point pair from the control point set to perform polynomial fitting: (15); in, and are the control point pairs, a, b, c, d, e, f, g and h are all parameters to be fitted; S52: Transform equation (15) into the least squares matrix form: (16); S53: Substitute all control point pairs in the control point set into equation (16) in sequence: (17); (18); in, and is the control point set pair, W is the space transformation model matrix; S54: Solve the transformation space transformation model matrix W according to formula (18): (19); in, The control point set The transposed matrix of .

3. The registration method of infrared and visible light remote sensing images according to claim 1, characterized in that: The step S6 specifically includes the following steps: S61: After removing the preliminary matching point set from all point sets included in the 1 / 2 resolution feature map, a residual matching point set is obtained, and after multiplying the residual matching point set with the spatial transformation model matrix, a desired matching point set is obtained; S62: saving all the matching point pairs that are two-way three nearest neighbors to each other in the expected matching point set after removing the error points as internal points, and obtaining a two-way three nearest neighbor matching point set; S63: taking all the matching point pairs that are not two-way three-nearest neighbors in the expected matching point set after removing the error points as the first column of point set pairs of the spatial similarity matching point set, and randomly selecting control point pairs in the control point set as the last five columns of point set pairs of the spatial similarity matching point set, to complete the construction of the spatial similarity matching point set; S64: constructing the coarse-grained matching point set using the control point set, the bidirectional three-nearest-neighbor matching point set and the spatial similarity matching point set.

4. The registration method of infrared and visible light remote sensing images according to claim 2, characterized in that: The step S7 specifically includes the following steps: S71: mapping the coarse-grained matching point set to the point set of the 1 / 2 resolution feature map, and cropping the mapped 1 / 2 resolution feature map into a pair of local feature windows of size 5*5; S72: Inputting the local feature window into the Transformer network for feature fusion to obtain a fine-grained feature map; S73: Calculate the similarity of the fine-grained feature map, perform desired fine-tuning, and then remove outliers from the fine-grained feature map after desired fine-tuning using the following formula to obtain a final matching result: (20); in, is the homography matrix, T is the threshold constant, T=2, It is the L2 paradigm.

5. The registration method of infrared and visible light remote sensing images according to claim 1, characterized in that: The Resnet extraction network includes a feature pyramid extraction network.