A method for bidirectional registration of infrared and visible light images

Abstract

This invention belongs to the field of computer vision and image processing, specifically relating to a bidirectional registration method for infrared and visible light images. It uses source image pairs and deformed image pairs as input images, and employs a bidirectional registration network to predict the deformation fields from infrared to visible light and from visible light to infrared images, thereby achieving registration between the infrared and visible light images. During the registration process, semantic features of the input images are extracted, and some features from these semantic features are used as latent features. Common features of the input images are extracted to obtain cross-modal and same-modal features for learning, and a multi-scale similarity matrix is ​​constructed based on this matrix. This matrix is ​​used to query the common features of infrared and visible light images in both forward and inverse deformation field predictions, achieving multi-modal collaborative supervision and enhancing the stability and accuracy of infrared and visible light image registration.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and image processing, and specifically relates to a method for bidirectional registration of infrared and visible light images. Background Technology

[0002] With the rapid development of computer vision and image processing technologies, infrared and visible light registration technology has attracted much attention. The purpose of this technology is to construct a precise pixel mapping relationship between the source and target images, fully utilizing the complementary information of the two modalities to improve the performance of downstream tasks. Visible light images are widely used due to their rich texture information; however, their imaging quality is largely constrained by ambient lighting conditions. In contrast, infrared images can provide thermal radiation information unaffected by changes in lighting, revealing the essential characteristics of a scene. The primary challenge in effectively combining these two types of information is accurately registering images of different modalities.

[0003] While significant progress has been made in single-modal image registration, registration accuracy drops drastically when applied to multimodal images such as infrared and visible light. This is primarily due to the inherent differences between different image modalities and the unidirectional mapping method employed in single-modal registration techniques. This strategy is effective in single-modal images, but effectively utilizing the common information across different modalities becomes a challenge when processing multimodal images, which in turn affects the stability of the registration. Summary of the Invention

[0004] The purpose of this invention is to provide a bidirectional registration method for infrared and visible light images to solve the problems of low registration accuracy and poor stability of multimodal images.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A method for bidirectional registration of infrared and visible light images includes the following steps:

[0007] Step 1: Acquire and process training samples. The training samples include source image pairs, deformed image pairs, and real deformed fields. Both deformed image pairs and real deformed fields are obtained by transforming source image pairs. The source image pairs consist of visible light images and infrared light images from the same scene. The real deformed fields include the forward deformed field and the inverse deformed field generated during the transformation of the source image pairs.

[0008] Step 2: Construct a bidirectional registration network for infrared and visible light images based on source image pairs and deformed image pairs, for registering infrared and visible light images. The bidirectional registration network includes a common feature encoder, a semantic encoder, and a bidirectional iterative module. Its registration implementation method includes:

[0009] 2.1 Based on infrared and visible light images, a common feature encoder is used to extract features F1 and F2 for learning cross-modal and intramodal modes. A semantic encoder is used to extract semantic features, which are divided into two parts: one part remains the semantic features H1 and H2, and the other part serves as the initial latent state. and

[0010] 2.2 Calculate the cosine similarity between features F1 and F2 using the inner product to obtain the similarity matrix between F1 and F2; perform average pooling on the similarity matrix between F1 and F2 to obtain the multi-scale similarity matrix for bidirectional query;

[0011] 2.3. Assume the predicted deformation field The initial value is 0. The predicted deformation field is used to query the common features related to the multi-scale pixels in the multi-scale similarity matrix. The common features, semantic features, predicted deformation field and latent features are used as inputs. The bidirectional iterative module is used to perform forward deformation field prediction and inverse deformation field prediction between infrared light image and visible light image N times to complete the registration of infrared light image and visible light image.

[0012] Step 3: Define the loss function A loss function is used to train and optimize a bidirectional registration network for infrared and visible light images; loss function The calculation formula is as follows:

[0013]

[0014]

[0015] in, Indicates endpoint loss. This represents the loss of bidirectional consistency. This represents the feature similarity loss, where γ and β both represent weights. Represents the actual forward deformation field. Represents the true inverse deformation field. This represents the predicted forward deformation field of the i-th time. This represents the predicted i-th inverse deformation field. Represents two-dimensional grid coordinates. This indicates interpolation sampling at the corresponding coordinates, and the symbol ‖·‖1 indicates the calculation of the average absolute value error;

[0016] Step 4: Input the infrared image and visible light image to be registered into the trained infrared and visible light image bidirectional registration network to complete the registration of the visible light image and the infrared light image.

[0017] Furthermore, the method for generating the forward deformation field and the inverse deformation field during the transformation of the source image pair in step 1 includes:

[0018] 1.1. Perform a rigid transformation on the source image pair, and calculate the affine transformation matrix of the source image based on the transformation parameters. The formula for calculating the affine transformation matrix is ​​as follows:

[0019]

[0020] Where M represents the affine transformation matrix, θ represents the rotation angle, s represents the scaling factor, and t x and t y These represent the pixel translation distances in the x and y directions of the image, respectively.

[0021] 1.2. By introducing a random deformation field into the affine transformation matrix, the forward deformation field is obtained. ,

[0022]

[0023] in, Indicates the forward deformation field. Represents two-dimensional grid coordinates, φ rand Indicates based on Δ x ,Δ y A random deformation field generated by ∈[-1,1] and standard deviation σ∈[16,32];

[0024] 1.3. Based on the forward deformation field, the inverse deformation field is calculated using the interpolation sampling method.

[0025] Furthermore, the deformed image pair is generated by sampling the source image pair based on the forward deformed field obtained in step 1.2.

[0026] Furthermore, the implementation method of step 2.3 includes:

[0027] 2.3.1 Define the query range Δφ, and add the defined query range to the forward deformation field through a broadcast mechanism. and reverse deformation field

[0028] 2.3.2. Assume the predicted deformation field The initial value is 0. The predicted deformation field is used to interpolate and search in the multi-scale similarity matrix to obtain the common features related to pixels at multiple scales.

[0029] 2.3.3. Take the common features, semantic features, predicted deformation fields and latent features related to multi-scale pixels as input, and concatenate the input features of the forward deformation field and the input features of the inverse deformation field. Use a bidirectional iterative algorithm to perform N iterations to optimize the forward deformation field and the inverse deformation field, and obtain N predicted forward deformation fields and predicted inverse deformation fields.

[0030] 2.3.4. The N predicted forward deformation fields are spliced ​​together to form a forward deformation field sequence. The N predicted inverse deformation fields are spliced ​​together to form an inverse deformation field sequence.

[0031] 2.3.5. Sequence of forward deformation fields With the reverse deformation field sequence By applying the technology to the infrared and visible light images to be registered, the registration of the infrared and visible light images can be achieved.

[0032] Furthermore, step 2.3.3 uses parallel computing of the GPU to predict N forward deformation fields and inverse deformation fields.

[0033] This invention provides a bidirectional registration method for infrared and visible light images. By constructing a bidirectional registration network for infrared and visible light images, it achieves bidirectional registration from infrared to visible light images and from visible light to infrared images. By extracting semantic features from the input images and using some features as latent features, the expressive power of the features is optimized. Simultaneously, by extracting common features from the input images, cross-modal and same-modal features for learning are obtained, and a multi-scale similarity matrix is ​​constructed based on this. Based on the multi-scale similarity matrix, the method enables effective querying of common features in forward and inverse deformation field prediction, thereby achieving multi-modal collaborative supervision and significantly improving the stability and accuracy of infrared and visible light image registration. Attached Figure Description

[0034] Figure 1 This is a flowchart of the infrared and visible light image bidirectional registration method in an embodiment;

[0035] Figure 2 This is a schematic diagram illustrating the transformation between infrared and visible light images in an embodiment.

[0036] Figure 3 A diagram of the infrared and visible light image bidirectional registration network structure constructed for this embodiment;

[0037] Figure 4 This is a diagram of a multimodal collaborative supervision framework for an embodiment.

[0038] Figure 5 This is a schematic diagram of the similarity matrix results for an example.

[0039] Figure 6 This is a schematic diagram illustrating the changes in registration accuracy during iterative optimization in an example. Detailed Implementation

[0040] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0041] like Figure 1 As shown in the figure, this embodiment provides a method for bidirectional registration of infrared and visible light images, including the following steps:

[0042] Step 1: Acquire and process training samples. The training samples include source image pairs, deformed image pairs, and real deformed fields. Both deformed image pairs and real deformed fields are obtained by transforming source image pairs. The source image pairs consist of visible light images and infrared light images from the same scene. The real deformed fields include the forward deformed field and the inverse deformed field generated during the transformation of the source image pairs.

[0043] In this embodiment, the training and test samples were obtained from the FLIR, MSRS, and RoadScene datasets. The training set contains 5911 image pairs, with 4627 pairs from the FLIR dataset, 1103 pairs from the MSRS dataset, and 181 pairs from the RoadScene dataset. The test set contains 896 image pairs, with 515 pairs from the FLIR dataset, 361 pairs from the MSRS test set, and 20 pairs from the RoadScene test set.

[0044] The method for calculating the forward and inverse deformation fields generated during the transformation of the source image is as follows: Figure 2 Shown, including:

[0045] 1.1 Perform rigid transformations on the infrared and visible light source images respectively. Calculate the affine transformation matrices for the infrared and visible light source images based on the transformation parameters. The formula for calculating the affine transformation matrix is:

[0046]

[0047] Where M represents the affine transformation matrix, θ represents the rotation angle, s represents the scaling factor, and t x and t y These represent the pixel translation distances in the x and y directions of the image, respectively. In this embodiment, the rotation angle θ∈[-10°, 10°], the scaling factor s∈[0.8, 1.2], and the pixel translation distance t x ,t y ∈[-0.05,0.5]

[0048] 1.2. By introducing a random deformation field into the affine transformation matrix, the forward deformation field is obtained. ,

[0049]

[0050] in, Indicates the forward deformation field. Represents two-dimensional grid coordinates, φ rand Indicates based on Δ x ,Δ y A random deformation field generated by ∈[-1,1] and standard deviation σ∈[16,32];

[0051] 1.3. Based on the forward deformation field, the inverse deformation field is calculated using the interpolation sampling method.

[0052]

[0053] in, Indicates the forward deformation field; Represents the two-dimensional grid coordinates of any image.

[0054] The method for generating the transformed image is as follows:

[0055] Using forward to become field Infrared light source image I ir Image I with visible light source vi Sampling is performed to generate the deformed image I. i ′ r and I v ′ i , where I ir ,I vi ,I i ′ r ,I v ′ i ∈R 3×H×W .

[0056] Step 2: Construct a bidirectional registration network for infrared and visible light images based on source image pairs and deformed image pairs. In this embodiment, the bidirectional registration network for infrared and visible light images employs a multimodal collaborative supervision approach to achieve registration between infrared and visible light images. The structure of the bidirectional registration network for infrared and visible light images is as follows: Figure 3 As shown, it includes two common feature encoders, two semantic encoders, a multi-scale similarity matrix, and a bidirectional iterative module. Its registration process is as follows: Figure 4 As shown below:

[0057] 2.1 Based on infrared and visible light images, a common feature encoder is used to extract features F1 and F2 for learning cross-modal and same-modal features, and a semantic encoder is used to extract semantic features.

[0058] Both common feature encoders in this embodiment use the first three blocks of a ResNet18 structure. The first block of the common feature encoder is a convolutional layer with a kernel size of 7×7, a stride of 2, 3 input channels, and 64 output channels. The second block consists of two 3×3 residual blocks with 64 output channels. The third block also consists of two 3×3 residual blocks with 128 output channels. The two semantic feature encoders also use the first three blocks of a ResNet18 structure; the difference from the common feature encoder is that the second block of the semantic feature encoder has 128 output channels, and the third block has 256 output channels.

[0059] The extraction methods for features F1 and F2 are as follows:

[0060] Using a common feature encoder Extract the common features F of the source image pair and the deformed image pair using the following formula. ir F′ ir F vi F′ vi :

[0061]

[0062] Among them, I ir Indicates an infrared image, I vi Represents a visible light image, I′ ir Represents an infrared distorted image, I′ vi Represents a visible light distortion image, I ir ,I vi ,i i ′ r ,I v ′ i ∈R 3×H×W ;

[0063] For common features F ir ,F i ′ r ,F vi ,F v ′ i By concatenating the features, we obtain learning features F1 and F2, where F1, F2 ∈ R, for learning cross-modal and intra-modal features. 4×D×H / 8×W / 8 D = 128 is the feature encoding dimension, and the features within F1 and F2 correspond one-to-one.

[0064] F1 = cat(F ir ,F vi ,F ir F vi )

[0065] F2 = cat(F′) vi ,F′ir F′ ir ,F′ vi )

[0066] Among them, F ir F represents the shared features of infrared light source images. vi F represents the shared features of images from visible light sources. i ′ r F represents the shared features of infrared distorted images. v ′ i This represents the shared features of visible light distorted images.

[0067] The method for extracting the semantic features is as follows:

[0068] Using a semantic feature encoder Extract semantic features from source image pairs and deformed image pairs using the following formula:

[0069]

[0070] Among them, H ir For the semantic features of infrared light source images, H i ′ r For semantic features of infrared light deformation images, H vi For semantic features of visible light source images, H v ′ i For semantic features of visible light distorted images; here, H ir H′ ir H vi H′ vi ∈R 2D ×H / 8×Wf8 The semantic features are divided into two parts: one part remains the semantic features H1 and H2, and the other part serves as the initial latent state. and

[0071] The semantic features H1 and H2 are expressed as follows:

[0072] H1 = cat(H ir [:D],H vi [:D],H ir [:D],H vi [:D])

[0073] H2 = cat(H′) vi [:D],H′ ir [:D],H′ ir [:D],H′ vi [:D])

[0074] Initial hidden state and The expression is as follows:

[0075]

[0076] Therefore, the dimensions of the semantic features and the initial latent features are 4×D×H / 8×W / 8.

[0077] 2.2. The cosine similarity between features F1 and F2 is calculated using the inner product, yielding a similarity matrix between F1 and F2. This similarity matrix is ​​then subjected to average pooling to obtain a multi-scale similarity matrix for bidirectional queries. The expression for the multi-scale similarity matrix is:

[0078]

[0079] Among them, C 0 Represents the similarity matrix. This represents the similarity matrix between infrared light source images and visible light distorted images. This represents a similarity matrix between a visible light source image and an infrared light distortion image. This represents the similarity matrix between an infrared light source image and an infrared light distortion image. This represents the similarity matrix between a visible light source image and a visible light distortion image.

[0080] This embodiment uses the common feature F ir With F′ vi For example, construct a similarity matrix Taking this as an example, the process of constructing a multi-scale similarity matrix is ​​described in detail: Similarity Matrix The calculation formula for coordinates [i,j,k,l] is as follows:

[0081]

[0082] Among them, (F ir ) dij This represents the characteristics of an infrared image at coordinates [d,i,j], (F′) vi ) dkl This represents the characteristics of a visible light deformed image in coordinates [d,k,l].

[0083] eigenvectors of similarity matrices The first two dimensions represent the feature dimensions of the infrared image, and the last two dimensions represent the feature dimensions of the visible image. In the similarity matrix... Repeated average pooling yields a multi-scale similarity matrix. Average pooling is performed using an average pooling layer with a kernel size and stride of 2. When average pooling is applied to the last two dimensions, the feature size of the infrared image remains unchanged, while the feature size of the visible light image decreases, achieving I-based similarity. irQuery I′ vi The similarity matrix C ir→vi′ Construction; when average down-pooling is applied to the first two dimensions, the feature size of the infrared image is small, while the feature size of the visible light image remains unchanged, realizing I′-based vi Query I ir The similarity matrix C vi′→ir Construct, thereby obtaining such Figure 5 The multi-scale similarity matrix shown is used for bidirectional queries, and its expression is as follows:

[0084]

[0085] Similarly, the same method is used to calculate the multi-scale similarity matrix for bidirectional queries for other feature pairs. Finally, the multi-scale similarity matrix between features F1 and F2 is constructed.

[0086] 2.3. Obtain common features related to pixels at multiple scales. Using these common features, semantic features, predicted deformation fields, and latent features as input, a bidirectional iterative module is used to perform N forward and inverse deformation field predictions between the infrared and visible light images to complete the registration of the infrared and visible light images. Specifically, this includes:

[0087] Step 2.3.1: Obtain common features related to pixels at multiple scales. To ensure that the query yields common features at both the local fine-grained level and the global maximum range, this embodiment uses the following method to obtain common features:

[0088] The defined query range Δφ is added to the forward deformation field via a broadcast mechanism. and reverse deformation field

[0089] Δφ={(u,v)|u,v∈{-r,-r-1,…0,…,r-1,r}}

[0090] Where Δφ represents the query range, u represents the horizontal query coordinate, v represents the vertical query coordinate, and r represents the query radius. In this embodiment, the query radius r = 4.

[0091] Interpolation queries are performed in a multi-scale similarity matrix to obtain common features related to pixels in infrared and visible light images;

[0092]

[0093] 2.3.3. Taking the common features, semantic features, predicted deformation field, and latent features related to multi-scale pixels as input, and concatenating the input features of the forward deformation field and the input features of the inverse deformation field, the concatenated features are as follows:

[0094]

[0095] The forward and inverse deformation fields are optimized N times using a bidirectional iterative module to obtain N predicted forward and inverse deformation fields. To improve computational speed, this embodiment uses GPU parallel computing to predict the N forward and inverse deformation fields.

[0096] The bidirectional iterative module in this embodiment consists of a gated recurrent unit and a convolutional prediction module, wherein the gated recurrent unit is represented by the following formula:

[0097]

[0098] Where Conv 3×3 (·) indicates a convolution operation, with a kernel size of 3×3, W z W r W h This represents the parameters of the convolution kernel. ⊙ represents the element-wise multiplication operation. Using gated recurrent units combined with the hidden state, the differential deformation field is calculated and the predicted deformation field is updated according to the following formula:

[0099]

[0100] Repeat the iterations to output N predicted deformation fields. Similarly, the process is the same for other registered image pairs.

[0101] 2.3.3. The predicted pre-deformation fields of N different registered image pairs are stitched together to form a forward deformation field sequence. The predicted inverse deformation fields of N different registered image pairs are stitched together to form an inverse deformation field sequence.

[0102] 2.3.4. Sequence of forward deformation fields With the reverse deformation field sequence By applying the technology to the infrared and visible light images to be registered, the registration of the infrared and visible light images can be achieved.

[0103] Step 3: To improve the registration accuracy and robustness of infrared and visible light images, this embodiment designs a loss function. A loss function is used to train and optimize a bidirectional registration network for infrared and visible light images. The loss function in this embodiment... The calculation formula is as follows:

[0104]

[0105] in, To predict the endpoint loss between the deformed field and the actual deformed field, To predict the bidirectional consistency loss between the variable field and the actual deformable field, The loss is based on feature similarity.

[0106] Endpoint loss Used to regress the flow in the estimator over the entire prediction sequence. and Supervision was conducted, in conjunction with real deformation fields. and Endpoint loss The calculation formula is as follows:

[0107]

[0108] Where γ represents the weight, γ = 0.8, Represents the actual forward deformation field. Represents the true inverse deformation field. This represents the predicted forward deformation field of the i-th time. This represents the predicted i-th inverse deformation field, and the symbol ‖·‖1 indicates the error in calculating the average absolute value.

[0109] Bidirectional consistency loss Used to ensure the consistency and robustness of predictions. Bidirectional consistency loss. The calculation formula is as follows:

[0110]

[0111] Where β represents the weight, β = 0.65, Represents the two-dimensional grid coordinates of the image. This indicates interpolation sampling at the corresponding coordinates.

[0112] Similar losses This method is used to find common features between two images, improving the accuracy of multimodal image registration, and employs feature similarity loss. The formula is shown below:

[0113]

[0114] Weights γ = 0.8, β = 0.65, and temperature coefficient τ = 0.07.

[0115] Step 4: Input the infrared image and visible light image to be registered into the trained infrared and visible light image bidirectional registration network to complete the registration of the visible light image and the infrared light image.

[0116] like Figure 6As shown, this embodiment tested the above-mentioned two-way registration method for infrared and visible light images on the FLIR, MSRS, and RoadScene test sets. The registration endpoint errors obtained from the tests were 2.680, 2.557, and 1.510, respectively. This indicates that the method has stronger registration accuracy.

[0117] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A method for bidirectional registration of infrared and visible light images, characterized in that, Includes the following steps: Step 1: Acquire and process training samples. The training samples include source image pairs, deformed image pairs, and real deformed fields. Both deformed image pairs and real deformed fields are obtained by transforming source image pairs. The source image pairs consist of visible light images and infrared light images from the same scene. The real deformed fields include the forward deformed field and the inverse deformed field generated during the transformation of the source image pairs. Step 2: Construct a bidirectional registration network for infrared and visible light images based on source image pairs and deformed image pairs, so as to register infrared and visible light images; The bidirectional registration network for infrared and visible light images includes a common feature encoder, a semantic encoder, and a bidirectional iterative module. Its registration implementation methods include: 2.1 Based on infrared and visible light images, a common feature encoder is used to extract features F1 and F2 for learning cross-modal and intramodal modes. A semantic encoder is used to extract semantic features, which are divided into two parts: one part remains the semantic features H1 and H2, and the other part serves as the initial latent state. and 2.2 Calculate the cosine similarity between features F1 and F2 using the inner product to obtain the similarity matrix between F1 and F2; perform average pooling on the similarity matrix between F1 and F2 to obtain the multi-scale similarity matrix for bidirectional query; 2.

3. Assume the predicted deformation field The initial value is 0. The predicted deformation field is used to query the common features related to the multi-scale pixels in the multi-scale similarity matrix. The common features, semantic features, predicted deformation field and latent features are used as inputs. The bidirectional iteration module is used to perform forward deformation field prediction and inverse deformation field prediction between infrared light image and visible light image N times to complete the registration of infrared light image and visible light image. Step 3: Define the loss function A loss function is used to train and optimize a bidirectional registration network for infrared and visible light images; loss function The calculation formula is as follows: in, Indicates endpoint loss. This represents the loss of bidirectional consistency. This represents the feature similarity loss, where γ and β both represent weights. Represents the actual forward deformation field. Represents the true inverse deformation field. This represents the predicted forward deformation field of the i-th time. This represents the predicted i-th inverse deformation field. Represents two-dimensional grid coordinates. This indicates interpolation sampling at the corresponding coordinates, and the symbol ‖·‖1 indicates the calculation of the average absolute value error; Step 4: Input the infrared image and visible light image to be registered into the trained infrared and visible light image bidirectional registration network to complete the registration of the visible light image and the infrared light image.

2. The method for bidirectional registration of infrared and visible light images according to claim 1, characterized in that, The method for generating the forward deformation field and the inverse deformation field in the source image pair transformation process in step 1 includes: 1.

1. Perform a rigid transformation on the source image pair, and calculate the affine transformation matrix of the source image based on the transformation parameters. The formula for calculating the affine transformation matrix is ​​as follows: Where M represents the affine transformation matrix, θ represents the rotation angle, s represents the scaling factor, and t x and t y These represent the pixel translation distances in the x and y directions of the image, respectively. 1.

2. By introducing a random deformation field into the affine transformation matrix, the forward deformation field is obtained. in, Indicates the forward deformation field. Represents two-dimensional grid coordinates, φ rand Indicates based on Δ x ,Δ y A random deformation field generated by ∈[-1,1] and standard deviation σ∈[16,32]; 1.

3. Based on the forward deformation field, the inverse deformation field is calculated using the interpolation sampling method.

3. The method for bidirectional registration of infrared and visible light images according to claim 2, characterized in that, The deformed image pairs are generated by sampling the source image pairs based on the forward deformed field obtained in step 1.

2.

4. The method for bidirectional registration of infrared and visible light images according to claim 2, characterized in that, The implementation method of step 2.3 includes: 2.3.1 Define the query range Δφ, and add the defined query range to the forward deformation field through a broadcast mechanism. and reverse deformation field 2.3.

2. Assume the predicted deformation field The initial value is 0. The predicted deformation field is used to interpolate and search in the multi-scale similarity matrix to obtain the common features related to pixels at multiple scales. 2.3.

3. Take the common features, semantic features, predicted deformation fields and latent features related to multi-scale pixels as input, and concatenate the input features of the forward deformation field and the input features of the inverse deformation field. Use a bidirectional iterative algorithm to perform N iterations to optimize the forward deformation field and the inverse deformation field, and obtain N predicted forward deformation fields and predicted inverse deformation fields. 2.3.

4. The N predicted forward deformation fields are spliced ​​together to form a forward deformation field sequence. The N predicted inverse deformation fields are spliced ​​together to form an inverse deformation field sequence. 2.3.

5. Sequence of forward deformation fields With the reverse deformation field sequence By applying the technology to the infrared and visible light images to be registered, the registration of the infrared and visible light images can be achieved.

5. The method for bidirectional registration of infrared and visible light images according to claim 4, characterized in that, Step 2.3.3 uses the parallel computing method of GPU to predict the forward deformation field and the inverse deformation field.

Images