A registration method for large-deformation images combined with super-resolution reconstruction

Abstract

This invention discloses a registration method for large-deformation images combining super-resolution reconstruction, belonging to the field of image registration technology. The method first estimates the motion matrix using optical flow and then performs image similarity segmentation using side-window filtering. Next, a non-parametric local adaptive method is used for super-resolution reconstruction of the image. Further steps include solving for the image registration model parameters, image resampling, and finally outputting the registered image. This method effectively improves the registration accuracy of large-deformation images by combining super-resolution reconstruction and registration.

Description

Technical Field

[0001] This invention relates to the technical field of large deformation image registration, and more particularly to a large deformation image registration method that combines super-resolution reconstruction. Background Technology

[0002] Image registration is the process of matching and overlaying two or more images acquired at different times, using different sensors (imaging devices), or under different conditions (weather, illumination, camera position and angle, etc.). It has been widely applied in remote sensing data analysis, computer vision, and image processing. However, research on registration of images with large deformations is currently limited. In reality, due to changes in the captured scene, differences in imaging angles, and different shooting times, images often exhibit significant deformation differences. Achieving high-precision registration of such images will effectively facilitate related subsequent applications. When two images to be registered exhibit large deformations, they can be divided into two main parts based on their content: one part consists of objects with similar names and minimal deformation, where pixels and geometric shapes are similar; the other part consists of objects representing the same ground feature but with significant deformation, where pixel grayscale differences are large and geometric distortion of the objects within the image is significant. For images with inconsistent deformation and significant deformation in local areas, traditional global transformation registration methods cannot accurately obtain registration parameters, leading to spatial misalignment and insufficient registration accuracy, which affects further applications. Therefore, there is a need to invent a registration method for large-deformation images with good registration stability, high spatial consistency, and the ability to handle local deformation.

[0003] In the current problem of large-deformation image registration, the main solutions to improve the stability and spatial consistency of image registration are: ① Non-rigid registration methods based on image features (reference: DGLowe, Object recognition from local scale-invariant features, Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, Greece, 1999, pp. 1150-1157 vol. 2, doi: 10.1109 / ICCV. 1999. 790410.), which uses square descriptors, but requires the calculation of principal directions to ensure rotation invariance, thus increasing the computational cost; ② Supervised learning-based deformable image registration (reference: SOKOOTlI H, DE VOS B, BERENDSEN F, et al. 3D convolutional neural networks image registration based on efficient supervised learning from artificial deformations[J]. arXiv PreprintarXiv, 2019:1908.10235.), but this method heavily relies on labels with real transformation parameters; ③ Image registration method based on grayscale information (Reference: Wells W MI, Viola P, Atsumi H, et al. Multi-modal volume registration by maximization of mutual information[J]. Medical image Analysis, 1996, 1(1):35-51.), this method is simple to implement, only using the correspondence between individual pixels, but it cannot obtain the spatial information of the image; ④ Image registration method based on the transform domain (Reference: Kugin CD, Hines D C. The phasecorrelation image registration method[C]. Proceedings of LEEE International Conference on Cybernetics and Society, 1975:163-165.), by converting two images to be registered to the frequency domain and using the cross power spectrum to calculate their translation vector, but this method has limited effect on image registration with local deformation.

[0004] In summary, current methods for registering images with large deformations often struggle with feature extraction and maintaining spatial consistency when dealing with images containing complex textures and structures. Therefore, there is an urgent need to develop a registration method for large deformation images that offers good registration stability, high spatial consistency, and the ability to handle localized deformations. Summary of the Invention

[0005] To address the aforementioned technical problems, the present invention aims to provide a registration method for large-deformation images that combines super-resolution reconstruction. Based on local super-resolution reconstruction, this method achieves high-precision registration even when there is inconsistent deformation within the image or large local deformation.

[0006] To achieve the above objectives, the present invention provides a method for registering large-deformation images combined with super-resolution reconstruction, comprising the following steps:

[0007] S1: Input reference image A and image B to be registered;

[0008] S2: The motion matrix M between image A and image B is estimated using the pyramid-layered optical flow algorithm;

[0009] S3: The side-window filtering method is used to calculate the similarity between images A and B;

[0010] S4: Determine the similarity regions of the images based on a threshold. Regions with similarity greater than the threshold are considered high-similarity regions, uniformly denoted as W. Regions with similarity less than the threshold are considered low-similarity regions, uniformly denoted as S. The similarity patches in S are denoted as Q. i i = 1, 2, 3...;

[0011] S5: For the highly similar region W in images A and B, optical flow is directly used for registration to obtain the registration parameters R for this region. W ;

[0012] S6: For the parts of images A and B that belong to the low similarity region S, each similarity block in image A is represented as Q. iA Let represent the i-th similar region in image A. Using the displacement information in matrix M and the super-resolution reconstruction factor r, similar blocks Q in B are combined with image A as the reference image. iB Let represent the i-th similarity region in image B, and HQ is obtained by reconstructing it using a MAP-based super-resolution reconstruction algorithm. iA The expression for the MAP-based super-resolution reconstruction algorithm is:

[0013]

[0014] Among them, the first term ||Y k -Dk B k F k X|| p This is the data fidelity term, p is the order of the norm, k is the image frame sequence (k = 1, 2, ..., N), and Y... k It is a low-resolution sequence of images, D is the motion matrix, B is the blur matrix, F is the downsampling matrix, X is the high-resolution image, and the second term λR(X) is the regularization term, where λ is the regularization parameter.

[0015] S7: For the parts of images A and B that belong to the low similarity region S, for each similarity block Q in image B... iB Using the displacement information in matrix M and the super-resolution reconstruction factor r, the image is combined with similar blocks Q in image A, with B as the reference image. iA HQ values ​​were obtained by reconstructing using a MAP-based super-resolution reconstruction algorithm. iB ;

[0016] S8: HQ of the super-resolution image block iA and image block HQ iB Registration was performed using optical flow, with the registration parameters downsampled by a factor of r to obtain Q. iA And Image Q iB Registration parameter set R of the block S ;

[0017] S9: Based on registration parameter R W and R S Resample image B;

[0018] S10: Output the registration result. Attached Figure Description

[0019] The description of the present invention will become apparent and readily understood in conjunction with the following drawings, wherein:

[0020] Figure 1 This is a flowchart of a large-deformation image registration method combining super-resolution reconstruction according to the present invention. Detailed Implementation

[0021] To register images with large deformations, this invention provides a method for registering images with large deformations that combines super-resolution reconstruction, comprising the following steps:

[0022] Step S1: Input the reference image A and the image to be registered B.

[0023] Step S2: Estimate the motion matrix M between image A and image B using the pyramid-layered optical flow algorithm.

[0024] Step S3: Calculate the similarity of images A and B using the side-window filtering method.

[0025] Step S4: Determine the similarity regions of the image based on the threshold. Regions with similarity greater than the threshold are considered high-similarity regions, uniformly denoted as W. Regions with similarity less than the threshold are considered low-similarity regions, uniformly denoted as S. The similarity blocks in S are denoted as Q. i , i = 1, 2, 3...

[0026] Step S5: For the highly similar region W in images A and B, optical flow is directly used for registration to obtain the registration parameters R of the region. W .

[0027] Step S6: For the parts of images A and B that belong to the low similarity region S, each similarity block in image A is represented as Q. iA Let represent the i-th similar region in image A. Using the displacement information in matrix M and the super-resolution reconstruction factor r, similar blocks Q in B are combined with image A as the reference image. iB Let represent the i-th similar region in image B, and HQ is obtained by reconstructing it using a super-resolution reconstruction algorithm. iA The expression for the MAP-based super-resolution reconstruction algorithm is:

[0028]

[0029] Among them, the first term ||Y k -D k B k F k X|| p This is the data fidelity term, p is the order of the norm, k is the image frame sequence (k = 1, 2, ..., N), and Y... k It is a low-resolution sequence of images, D is the motion matrix, B is the blur matrix, F is the downsampling matrix, X is the high-resolution image, and the second term λR(X) is the regularization term, where λ is the regularization parameter.

[0030] Step S7: For the parts of images A and B that belong to the low similarity region S, for each similarity block Q in image B... iB Using the displacement information in matrix M and the super-resolution reconstruction factor r, the image is combined with similar blocks Q in image A, with B as the reference image. iA HQ values ​​were obtained by reconstructing using the MAP-based super-resolution reconstruction algorithm. iB .

[0031] Step S8: Perform HQ processing on the super-resolution image blocks. iA HQ Images iB Registration was performed using optical flow, with the registration parameters downsampled by a factor of r to obtain Q. iA And Image Q iB Registration parameter set R of the block S .

[0032] Step S9: Based on the registration parameter R W and R S Resample image B.

[0033] Step S10: Output the registration result.

Claims

1. A method of large deformation image registration combined with super-resolution reconstruction, characterized in that, The steps include: S1: inputting a reference image A and a to-be-registered image B; S2: estimating a motion matrix M between the image A and the image B by using a pyramid layering optical flow algorithm; S3: calculating the similarity of the images by using a side window filtering method for the images A and B; S4: judging the similarity region of the image according to the threshold value, the high similarity region is greater than the threshold value, which is uniformly represented as W, and the low similarity region is less than the threshold value, which is uniformly represented as S, and the similarity block image in S is represented as Q i , i = 1, 2, 3...; S5: directly adopt optical flow method to register the high similarity region W in A and B images, and obtain the registration parameter R of the region W ; S6: For the parts of A, B images belonging to the low similarity region S, each similarity block in image A is represented as Q iA , represents the i-th similarity region in image A, using the displacement information in matrix M and the super-resolution reconstruction multiple r, combines the similarity block Q iB , represents the i-th similarity region in image B, reconstructs to obtain HQ iA using the MAP-based super-resolution reconstruction algorithm; S7: for the parts of A, B images belonging to the low similarity region S, for each similarity block Q in image B iB , using the displacement information in matrix M and the super-resolution reconstruction multiple r, combining the similarity block Q in image A with image B as the reference image iA , using the MAP-based super-resolution reconstruction algorithm to reconstruct to obtain HQ iB ; S8: super-resolution of the image block HQ iA and the image block HQ iB Using optical flow registration, the registration parameters are down-sampled by r times, obtaining Q iA and the image Q iB The registration parameter set R S ; S9: resample the B image according to the registration parameters R W and R S , resample the B image; S10: outputting a registration result.

Images