A method for registering visible and infrared images based on region similarity

Abstract

The application provides a visible light and infrared image registration method based on region similarity, and the method comprises the following steps: rendering a synthetic dataset with a graph with obvious corner features, training a key point detector on the synthetic dataset; performing scaling and grayscale processing on a registration image pair of visible light and infrared, inputting the key point detector to generate corresponding key point coordinates, and performing random homography transformation on the infrared and visible light images with key points to simulate the transformation relationship between the image pair, generating a grid through Delaunay triangulation, and processing the topological relationship of the point set and the triangular mesh generation; inputting the transformed image pair into a descriptor generation network, designing a region-based loss function, and training the registration relationship between the grids. The method effectively solves the problem of inconsistent corner points in cross-modal image matching, realizes high-precision and high-anti-interference-capability image registration, and can be applied to the fields of remote sensing image processing and target tracking.

Description

Technical Field

[0001] This invention belongs to the fields of nighttime surveillance and reconnaissance, medical diagnosis, and planetary surface monitoring, and specifically relates to a visible light and infrared image registration method based on regional similarity. Background Technology

[0002] Image matching technology has wide applications in computer vision, remote sensing, robot navigation, and medical image processing. With the development of technology, the research and application of image matching technology in various fields are becoming increasingly in-depth, and the requirements for accuracy and speed in image matching are also increasing. Traditional image matching methods are mainly based on the grayscale information of images, such as template matching and feature point matching. However, these methods have certain limitations when processing different types of images. For example, in matching visible light images and infrared images, due to differences in imaging mechanisms and lighting conditions, directly using traditional image matching methods often fails to achieve satisfactory matching results. To solve the problem of matching visible light and infrared images, researchers have proposed several image matching methods for specific applications in recent years. These methods are mainly divided into two categories: feature-based matching methods and model-based matching methods. Feature-based matching methods first extract representative feature points, such as corner points and edge points, from visible light and infrared images. Then, they determine the matching relationship by calculating similarity metrics between feature points (such as Euclidean distance and mutual information). This type of method can improve the robustness of matching to a certain extent, but it is relatively sensitive to geometric transformations and changes in lighting between images. Model-based matching methods establish a transformation model between visible light and infrared images, mapping the visible light image to the coordinate system of the infrared image. The key to these methods lies in model establishment and parameter solving. Common transformation models include linear models, affine models, and projection models. Model-based matching methods have certain advantages in handling geometric transformations and illumination changes between images, but the complexity of the model and the solution process can lead to a large computational burden.

[0003] Although existing image matching methods can solve the problem of matching visible light and infrared images to some extent, challenges remain in matching accuracy, computational efficiency, and robustness. In particular, most existing methods have not solved the problem of inconsistency in corner points when matching visible light and infrared images. Summary of the Invention

[0004] The technical problem solved by this invention is to address the issue of inconsistency in corner points between visible light and infrared image pairs. It provides a visible light and infrared image registration method based on region similarity, which uses region matching instead of feature point pair matching. Combined with the Lagrange multiplier method, a global optimization objective function is constructed, and appropriate constraints are applied to seek the optimal region matching relationship between the visible and infrared domains. This overcomes the corner point offset problem and improves the robustness and reliability of image registration.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A visible light and infrared image registration method based on region similarity is described below:

[0007] Step 1: Render the basic pattern to obtain a composite image and record the corner coordinates in the composite image to obtain a specified number of composite images to form a synthetic dataset;

[0008] Step 2: Train the keypoint detector on the synthetic dataset obtained in Step 1;

[0009] Step 3: After scaling and grayscale processing of the visible light image and infrared image, input them into the key point detector to generate the corresponding key point coordinates;

[0010] Step 4: Randomly select infrared and visible light images with key point coordinates and perform random homography transformation;

[0011] Step 5: Perform Delaunay triangulation on the infrared image and visible light image with key point coordinates after random homography transformation according to the key point coordinates.

[0012] Step 6: Input the infrared and visible light images with meshes obtained after Delaunay triangulation into the descriptor generation network model for model training;

[0013] Step 7: Use the trained descriptor generation network model for inference testing to complete the registration task of unregistered visible light images and infrared images.

[0014] Furthermore, in step 1, the synthetic dataset is rendered using basic patterns with distinct corner features, including triangles, quadrilaterals, lines, cubes, checkerboards, and stars. The specific steps for obtaining the synthetic dataset are as follows:

[0015] Step 1.1: Obtain the corner coordinates of each basic pattern. The basic patterns include triangles, quadrilaterals, lines, cubes, checkerboards, and stars.

[0016] Step 1.2: Perform random homography transformation on each basic pattern to obtain new corner coordinates. ;

[0017] Step 1.3: Randomly combine the homography-transformed basic pattern onto an image of a specified resolution, and obtain the coordinates of each corner point on the combined image. ;

[0018] Step 1.4: Randomly select the coordinates of several corner points on each combined image as labels for key point locations;

[0019] Step 1.5: Repeat steps 1.2 to 1.4 until a specified number of combined images N are rendered, forming a synthetic dataset.

[0020] Furthermore, in step 2, the keypoint detector is a small convolutional neural network similar to the VGG style, containing eight... The convolutional layers have channel sizes of 64-64-64-64-128-128-128-128, with one channel between every two layers. The max pooling layer is used. Since the pooling layer reduces the image resolution, bicubic interpolation is used to restore the feature map resolution to the original image. The keypoint detector is trained using the synthetic dataset obtained in step 1, as follows:

[0021] Step 2.1: Divide the combined images in the synthetic dataset into 192 cells according to the range covered by the labels, ensuring that each cell has at least one label;

[0022] Step 2.2: For each cell, predict the coordinates of a keypoint using a keypoint detector. ;

[0023] Step 2.3: Based on the predicted key point coordinates The loss value is calculated to optimize the network parameters. If a cell has multiple labels, one label is randomly selected as the key point coordinate of that cell.

[0024] Furthermore, in step 3, in order to adapt to the trained keypoint detector, the images to be registered, one in the visible light image and one in the infrared image, are scaled down to a resolution of [resolution missing]. The two images were then processed to grayscale.

[0025] Furthermore, in step 4, the random homography transformation is achieved by adding random perturbations to the four vertices of the rectangular block. Since each perturbation randomly selects either a visible light image or an infrared image, the resulting image pair is... and or and ,in, These represent the transformed visible light image, the untransformed infrared image, the untransformed visible light image, and the transformed infrared image, respectively.

[0026] Furthermore, in step 5, the transformed image is... and or and Generate a mesh by performing Delaunay triangulation based on the key point locations.

[0027] Furthermore, in step 6, the descriptor generation network used retains the structure of the keypoint detector, that is, it contains eight... The convolutional layers have sizes of 64-64-64-64-128-128-128-128, with a space between each pair of layers. The max-pooling layer ultimately contains a decoder with 256 channels. Convolutional layers with 256 channels Convolutional layers constitute the structure. Specifically:

[0028] Step 6 includes:

[0029] Step 6.1: Establish the correspondence between the grids in the infrared image and the visible light image;

[0030] Step 6.2: Use the Lagrange multiplier method to determine the descriptor vectors of corresponding points inside the corresponding grids of the infrared image and the visible light image, and make the two descriptor vectors tend to be consistent;

[0031] Step 6.3: Traverse each grid and each pair of infrared and visible light images, and generate network parameters by optimizing the descriptor through gradient backpropagation;

[0032] Step 6.4: Stop after reaching the maximum number of training rounds, and obtain the trained descriptor generation network model.

[0033] Furthermore, in step 7, for unregistered visible light and infrared image pairs, the trained keypoint detector and descriptor generation network are used to predict keypoint locations and descriptor vectors, respectively. Specifically:

[0034] Step 7 includes:

[0035] Step 7.1: Perform scaling and grayscale processing on the unregistered image pairs of the visible light image and the infrared image;

[0036] Step 7.2: Use a keypoint detector to detect keypoints in unregistered image pairs to ensure that the number of keypoints is consistent;

[0037] Step 7.3: Generate a mesh by performing Delaunay triangulation based on the key point coordinates;

[0038] Step 7.4: Predict the correspondence between grids of unregistered image pairs using the trained descriptor generation network model, and regress the coordinates of the key point pairs matched by the image pairs to be registered and the confidence score of the matching.

[0039] Step 7.5: Use a filtering algorithm to filter out products with confidence scores higher than the threshold. Based on the reliable point pairs, the homography matrix between the image pairs to be registered is calculated according to the matching relationship of the point pairs, and the homography transformation from infrared to visible light image or from visible light to infrared image is completed.

[0040] The advantages of this invention compared to the prior art are:

[0041] (1) This invention uses a synthetic dataset to train the keypoint detector, eliminating the need for manual annotation of keypoint locations. The keypoint detector can be pre-trained using a synthetic dataset. To further improve the detection performance, multiple synthetic datasets are typically rendered to train multiple keypoint detectors, and the best-performing keypoint detector is selected for subsequent tasks. In actual training, only the registered visible light and infrared image pairs are needed to train the descriptor generation network, eliminating the need to train the keypoint detector again, thus greatly saving training costs and the cost of manually annotating the dataset.

[0042] (2) This invention uses region matching instead of traditional point-to-point matching, thus solving the problem of corner inconsistency that cannot be avoided in traditional cross-domain point matching. In traditional point-to-point matching, due to the difference in imaging mechanisms between cross-domain image pairs, even registered point pairs will have a certain offset in coordinate position, which brings great difficulties to point-to-point matching. The region similarity matching of this invention solves the corner offset problem by using the Lagrange multiplier method to search for precise matching positions in two regions, improving matching accuracy and robustness, and providing a new idea for cross-domain image matching research.

[0043] (3) This invention uses Delaunay triangulation as the region generation method. Compared with other methods where Delaunay triangulation is the only unique region generation method given a set of points, this method is an optimal triangulation method with the smallest maximum angle. The generated triangles are more uniform in shape and closer to equilateral triangles. Given a set of points, Delaunay triangulation is unique, thus ensuring the uniqueness of the generated regions, which makes network training easier. Attached Figure Description

[0044] Figure 1 This is a flowchart of a visible light and infrared image registration method based on region similarity according to the present invention;

[0045] Figure 2 This is an example diagram of the synthetic dataset generated in an embodiment of the present invention;

[0046] Figure 3 This is a flowchart of the inference process for unregistered image pairs in an embodiment of the present invention.

[0047] Figure 4 The image shows the experimental results of a visible light and infrared image registration method based on region similarity according to the present invention, where: a is the input visible light image, b is the input infrared image, and c is the result image of the visible light image transformed to the infrared image view. Detailed Implementation

[0048] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. In this embodiment, the infrared image in the visible light and infrared image registration dataset is a near-infrared image, with a wavelength range of approximately 0.75 micrometers to 1.4 micrometers. Near-infrared images within this wavelength range contain rich detail information. Both images used in this embodiment have a resolution of [missing information]. The method of this invention has broad applicability and can be adapted to the registration of visible light and infrared images with different resolutions and infrared wavelength ranges in real-world scenarios, and is not limited to the experimental conditions given in the specific embodiments.

[0049] like Figure 1 As shown, the specific steps for training the keypoint detector and descriptor generation network are as follows:

[0050] Step 1: Render the composite image using the basic pattern and record the corner coordinates in the composite image to obtain a specified number of composite images to form a synthetic dataset;

[0051] Step 2: Train a keypoint detector on the synthetic dataset obtained in Step 1;

[0052] Step 3: After scaling and grayscale processing of the image pairs to be registered, the visible light image and the infrared image are input into the key point detector to generate the corresponding key point coordinates;

[0053] Step 4: Randomly select infrared or visible light images with key point coordinates and perform random homography transformation;

[0054] Step 5: Perform Delaunay triangulation on the infrared and visible light images with key point coordinates after random homography transformation in Step 4 to generate a mesh according to the key point positions.

[0055] Step 6: Input the infrared and visible light images with grids from Step 5 into the descriptor generation network model for training;

[0056] Step 7: Use the model trained in Step 6 for inference testing to complete the registration task of unregistered visible light images and infrared images;

[0057] In step 1, the synthetic dataset is rendered using basic patterns with distinct corner features, such as... Figure 2 As shown, these patterns include triangles, quadrilaterals, lines, cubes, chessboards, and stars. Before rendering the dataset, the key points of these basic patterns are labeled. Because their corner features are obvious, labeling the key points is relatively easy; for example, the three vertices of a triangle, the four vertices of a quadrilateral, and the two endpoints of a line. After labeling the key points of the basic patterns, these patterns are subjected to random homography transformation and random combination to obtain a synthetic image. The specific steps for obtaining the synthetic dataset are as follows:

[0058] Step 1.1: Obtain the corner coordinates of each basic pattern. ;

[0059] Step 1.2: Perform random homography transformation on each basic pattern to obtain new corner coordinates. ;

[0060] Step 1.3: Randomly combine the homography-transformed base pattern onto a sheet with a specified resolution. On the image, obtain the coordinates of each corner point on the combined image. ;

[0061] Step 1.4: Randomly select the coordinates of 192 corner points on each image as labels for key point locations;

[0062] Step 1.5: Repeat steps 1.2 to 1.4 until the specified number of renders are obtained. The combined images form a synthetic dataset.

[0063] In step 2, the keypoint detector is a small convolutional neural network similar to VGG, containing eight... The convolutional layers have channel sizes of 64-64-64-64-128-128-128-128, with one channel between every two layers. The maximum pooling layer is used. Since pooling layers reduce image resolution, bicubic interpolation is used after the convolutional layer to restore the feature map resolution to the original image size. The detector is trained using the synthetic dataset obtained in step 1, as follows:

[0064] Step 2.1: Divide the combined images in the synthetic dataset into individual categories according to the range covered by the labels. Each cell must have at least one label.

[0065] Step 2.2: Predict the coordinates of a keypoint in each cell using a keypoint detector. ;

[0066] Step 2.3: Based on the predicted key point coordinates The network parameters are optimized by calculating the loss value. If a cell contains multiple labels, one label is randomly selected as the keypoint coordinates for that cell, and the loss value is calculated. As shown in the following formula:

[0067] ,

[0068] in, These represent the number of cells along the height direction and the number of cells along the width direction, respectively. , These are the first along the height direction The and along the width direction The x and y coordinates of the key points predicted for each cell. , These are the first along the height direction The and along the width direction In this embodiment, the actual x and y coordinates of the key points in each cell are... .

[0069] In step 3, in order to adapt to the trained keypoint detector, the following will be performed: Figure 4 a, Figure 4 The registered image of visible light and infrared light shown in b is scaled down to a resolution of The two images are then subjected to grayscale processing using the following formula. Figure 4 In the image, c represents the result of transforming the visible light image to the infrared image viewpoint.

[0070] ,

[0071] in, It is the processed single-channel grayscale value. These represent the pixel values ​​for the red, green, and blue channels, respectively. This formula is based on the differences in human eye sensitivity to different colors, with the green channel having the highest weight, followed by the red channel, and the blue channel having the lowest weight.

[0072] In step 4, the random homography transformation is achieved by adding random perturbations to the four vertices of the rectangular block. The specific steps are as follows:

[0073] Step 4.1: Select a rectangular block inside the image and record the coordinates of its four vertices. , , , ;

[0074] Step 4.2: Add a specified range to each of the four vertices. [ − ε , ε ] The random perturbation yields the coordinates of four new vertices. , , , ;

[0075] Step 4.3: Using the coordinates of the four new vertices and linear algebra tools, solve for the homography transformation matrix. ;

[0076] Step 4.4: Based on the obtained homography transformation matrix For the original image The transformed image is obtained by performing homography. The image transformation randomly selects either a visible light image or an infrared image. If the visible light image is transformed, the resulting image pair is... and ,in, These are the transformed visible light image and the untransformed infrared image, respectively; conversely, if the infrared image is transformed, the resulting image pair is... and ,in, These are the untransformed visible light image and the transformed infrared image, respectively. Regardless of the transformation chosen, the homography transformation matrix must be saved. .

[0077] In step 5, the transformed image is... and or and The mesh is generated by performing Delaunay triangulation based on the key point locations. The specific steps are as follows:

[0078] Step 5.1: Select a sufficiently large super triangle that contains all the key points to be triangulated, and add this super triangle to the triangle list;

[0079] Step 5.2: Insert each key point to be triangulated into the super triangle;

[0080] Step 5.3: For each key point to be inserted, find the target super triangle containing that point;

[0081] Step 5.4: For the target super triangle found, check the properties of the circle formed by it and the newly inserted point to ensure that the conditions of Delaunay triangulation are met.

[0082] Step 5.5: If the circumcircle of the target super triangle contains the newly inserted point, then an edge flipping operation needs to be performed, that is, the edge is replaced with its diagonal, thereby updating the triangular mesh structure.

[0083] Step 5.6: After performing edge flipping, update the edge list of adjacent supertriangles to reflect the new triangle;

[0084] Step 5.7: Repeat the above process until all points are inserted into the triangle mesh and there are no edges that need to be flipped, ensuring that all newly formed triangles satisfy the properties of Delaunay triangulation.

[0085] Step 5.8: Delete the super triangle and its associated edges to obtain the final Delaunay triangulation.

[0086] In step 6, the descriptor generation network used retains the structure of the keypoint detector, that is, it contains eight... The convolutional layers have sizes of 64-64-64-64-128-128-128-128, with a space between each pair of layers. The max-pooling layer ultimately contains a decoder with 256 channels. Convolutional layers with 256 channels Convolutional layers are constructed. The specific steps are as follows:

[0087] Step 6.1: Use the image pairs as input descriptor generation network, and set the convolutional layer parameters so that the output dimensions of each image are [dimensions to be filled in]. The tensor of, where, It is the number of key points. It is the length of the descriptor vector, that is, each keypoint has a length of... The descriptor vector;

[0088] Step 6.2: Based on the original registration relationship between the visible light and infrared images and the homography transformation matrix... Whether it is an image pair and or and For any triangular mesh of a visible light image The corresponding triangular mesh can definitely be found in the infrared image. Correspondingly, among them, It refers to the triangular mesh set of a visible light image. It refers to the triangular mesh set of an infrared image. It is the descriptor vector of each of the three vertices of a triangular mesh in a visible light image. It is the descriptor vector of each of the three vertices of a triangular mesh in an infrared image. It refers to three different vertices. (Establishment) and Correspondence ;

[0089] Step 6.3: Store all items according to the one-to-one correspondence. The final number of stored correspondences is equal to the total number of triangular grids;

[0090] Step 6.4: For each correspondence... Optimize the descriptor vectors extracted from visible light and infrared images:

[0091] ,

[0092] in, It is an interpolation function. These are the parameters of the Lagrange multiplier method, and also the parameters of the interpolation function. The interpolation function interpolates the descriptor vectors of all points within the triangular region based on the descriptor vectors of the three vertices of the triangular mesh, as shown in the following formula:

[0093] ,

[0094] ,

[0095] Therefore, the loss function for predicting the descriptor vector can be obtained as follows:

[0096] ,

[0097] in, This represents the number of triangular mesh correspondences. The network parameters are generated by optimizing the descriptor through gradient backpropagation after calculating the loss value.

[0098] In step 7, for unregistered visible light and infrared image pairs, the trained keypoint detector and descriptor generation network are used to predict keypoint locations and descriptor vectors, respectively. The prediction process is as follows: Figure 3 As shown, the specific steps are as follows:

[0099] Step 7.1: Scale the unregistered visible light and infrared images to a size of [size missing]. The scaled image is then processed into grayscale.

[0100] Step 7.2: Divide the unregistered visible light and infrared images after grayscale processing into 192 cells each. Predict the coordinates of a key point in each cell using a trained key point detector.

[0101] Step 7.3: Perform Delaunay triangulation based on the keypoint coordinates and generate a mesh. Since the keypoint positions of the two images are not exactly the same, the final number of meshes will also be different. Set the triangular mesh sets as follows: and ;

[0102] Step 7.4: Input the unregistered visible light and infrared images after grayscale processing into the trained descriptor generation network model to obtain the descriptor vector for each key point;

[0103] Step 7.5, for the set For any triangular mesh in the set, calculate its relationship with the set using the Lagrange multiplier method. Dissimilarity of each triangular mesh The specific formula is as follows:

[0104] ,

[0105] in, It is a set The triangular grid index in the middle, It is a set The triangular grid index in the middle, It is a set The Middle Triangular grid and set The Middle Dissimilarity of triangular meshes;

[0106] Step 7.6: Obtain the smallest value by sorting. ,if Less than or equal to the threshold If the registration relationship between the two triangular meshes is considered reliable, the parameters can be solved using the Lagrange multiplier method. The matching relationship can be accurate to point pairs based on the interpolation function; if Greater than the threshold Then the set is considered The index is The key points of the triangular mesh were extracted incorrectly; delete the mesh.

[0107] Step 7.7: Repeat steps 7.5 to 7.6 until all matching point pairs are obtained;

[0108] Step 7.8: By matching point pairs Value selection confidence level is higher than Reliable point pair, The lower the value, the higher the confidence level; some high-confidence values ​​are eliminated. The matching point pairs of values ​​are used to solve for the homography transformation matrix. .

[0109] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A visible light and infrared image registration method based on region similarity, characterized in that, The method includes the following steps: Step 1: Render the basic pattern to obtain a composite image and record the corner coordinates in the composite image to obtain a specified number of composite images to form a synthetic dataset; Step 2: Train the keypoint detector on the synthetic dataset obtained in Step 1; Step 3: After scaling and grayscale processing of the visible light image and infrared image, input them into the key point detector to generate the corresponding key point coordinates; Step 4: Randomly select infrared and visible light images with key point coordinates and perform random homography transformation; Step 5: Perform Delaunay triangulation on the infrared image and visible light image with key point coordinates after random homography transformation according to the key point coordinates. Step 6: Input the infrared and visible light images with meshes obtained after Delaunay triangulation into the descriptor generation network model for model training; Step 7: Use the trained descriptor generation network model for inference testing to complete the registration task of unregistered visible light images and infrared images; among which, Step 4 includes: Step 4.1: Randomly crop a rectangular block from the randomly selected infrared or visible light image; Step 4.2: Obtain the coordinates of the four vertices of the rectangular block; Step 4.3: Add random perturbations within a specified range to the coordinates of the four vertices to obtain four new vertex coordinates; Step 4.4: Calculate the homography matrix based on the coordinates of the four new vertices. ; Step 4.5: Based on the homography matrix A new image is obtained by performing a homography transformation on a randomly selected infrared or visible light image; Step 6 includes: Step 6.1: Establish the correspondence between the grids in the infrared image and the visible light image; Step 6.2: Use the Lagrange multiplier method to determine the descriptor vectors of corresponding points inside the corresponding grids of the infrared image and the visible light image, and make the two descriptor vectors tend to be consistent; Step 6.3: Traverse each grid and each pair of infrared and visible light images, and generate network parameters by optimizing the descriptor through gradient backpropagation; Step 6.4: Stop after reaching the maximum number of training rounds, and obtain the trained descriptor generation network model; Step 7 includes: Step 7.1: Perform scaling and grayscale processing on the unregistered image pairs of the visible light image and the infrared image; Step 7.2: Use a keypoint detector to detect keypoints in unregistered image pairs to ensure that the number of keypoints is consistent; Step 7.3: Generate a mesh by performing Delaunay triangulation based on the key point coordinates; Step 7.4: Predict the correspondence between grids of unregistered image pairs using the trained descriptor generation network model, and regress the coordinates of the keypoint pairs matched by the unregistered image pairs and the confidence scores of the matches; Step 7.5: Use a filtering algorithm to filter out products with confidence scores higher than the threshold. Based on the reliable point pairs, the homography matrix between unregistered image pairs is calculated according to the matching relationship of the point pairs, and the homography transformation from infrared to visible light image or from visible light to infrared image is completed.

2. The visible light and infrared image registration method based on region similarity according to claim 1, characterized in that, Step 1 includes: Step 1.1: Obtain the corner coordinates of each basic pattern. The basic patterns include triangles, quadrilaterals, lines, cubes, checkerboards, and stars. Step 1.2: Perform random homography transformation on each basic pattern to obtain new corner coordinates. ; Step 1.3: Randomly combine the homography-transformed basic pattern onto an image of a specified resolution, and obtain the coordinates of each corner point on the combined image. ; Step 1.4: Randomly select the coordinates of several corner points on each combined image as labels for key point locations; Step 1.5: Repeat steps 1.2 to 1.4 until a specified number of combined images N are rendered, forming a synthetic dataset.

3. The visible light and infrared image registration method based on region similarity according to claim 1, characterized in that, Step 2 includes: Step 2.1: Divide the combined images in the synthetic dataset into multiple cells according to the range covered by the labels, ensuring that each cell has at least one label; Step 2.2: For each cell, predict the coordinates of a keypoint using a keypoint detector. ; Step 2.3: Based on the predicted key point coordinates The loss value is calculated to optimize the network parameters. If a cell has multiple labels, one label is randomly selected as the key point coordinate of that cell.

Images