A heterogeneous image registration method and device based on phase correlation calculation

Abstract

The present invention discloses a heterogeneous image registration method and device based on phase correlation calculation, which respectively extract a first feature map S1 of a template image and a second feature map S2 of a reference image; determine channel information KP1 according to S1; determine channel information KP2 according to S2; generate a phase correlation map RS according to KP1 and KP2; determine a rotation parameter r and a scale parameter s between the template image and the reference image according to RS; calculate the translation parameters (x, y) between the template image and the reference image based on r, s, residual channel information L1 of S1 and residual channel information L2 of S2; the present invention extracts feature maps of the template image and the reference image, and then processes the feature maps by channel, so as to calculate the rotation parameters, scale parameters and translation parameters of the template image and the reference image at one time, thereby reducing the amount of calculation and the redundancy of the calculation, being applicable to devices with small video memory space, reducing the requirements for device performance and improving real-time performance.

Description

Technical Field

[0001] The present invention belongs to the technical field of heterogeneous image registration, and in particular relates to a heterogeneous image registration method and device based on phase correlation calculation. Background Art

[0002] Image registration is a fundamental problem in the fields of computer vision, remote sensing data analysis, and image processing. Its purpose is to match and superimpose two or more images acquired at different times, using different sensors (imaging devices), or under different conditions (weather, illumination, camera position and angle, etc.).

[0003] Compared to homologous image registration, heterogeneous image registration is more difficult because heterogeneous images have different imaging conditions and different image features, such as texture edges. Heterogeneous image registration can be applied to many scenarios, such as the registration of synthetic aperture radar and optical images, and the positioning of ground images on maps. Even when GPS, BeiDou, and other positioning systems provide positioning information during the acquisition process, some critical applications with extremely high precision and real-time requirements cannot tolerate the delays and excessive accuracy deviations generated by these systems. Therefore, these positioning systems can only serve as auxiliary methods in the registration process. In contrast, methods for directly obtaining registration results from heterogeneous images are extremely important.

[0004] Currently, a deep phase correlation method uses a neural network to calculate feature maps of multiple modalities. These feature maps are then Fourier transformed and logarithmically transformed. Phase correlation metrics are then used to determine the optimal rotation and scale (i.e., scale change). The original image is then calibrated using the rotation and scale data. The calibrated image is then used to determine the optimal translation, ultimately yielding the final calibrated image.

[0005] Although the above method is highly accurate, the fixed search space limits the solution accuracy. In addition, the current solution uses different neural networks for rotation, scale, and translation, requiring two-stage training. It does not recognize the equivariance of neural networks, has redundant parameters, and requires a large amount of computation, making it unsuitable for resource-constrained devices. Summary of the Invention

[0006] The purpose of the present invention is to provide a heterogeneous image registration method and device based on phase correlation calculation, which integrates rotation, scale and translation in the same neural network, reduces the amount of calculation and improves real-time performance.

[0007] The present invention adopts the following technical solution: a heterogeneous image registration method based on phase correlation calculation, comprising the following steps:

[0008] Extracting a first feature map S1 of the template image and a second feature map S2 of the reference image respectively;

[0009] Determine channel information KP1 according to S1; determine channel information KP2 according to S2;

[0010] Generate a phase correlation diagram RS based on KP1 and KP2;

[0011] Determine the rotation parameter r and scale parameter s between the template image and the reference image according to RS;

[0012] The translation parameters (x, y) between the template image and the reference image are calculated based on r, s, the residual channel information L1 of S1 and the residual channel information L2 of S2.

[0013] Furthermore, determining the channel information KP1 according to S1 includes:

[0014] Extract a channel information K1 of S1;

[0015] Perform Fourier transform on K1 and normalize it to obtain the transformed channel information KF1;

[0016] Perform log-polar coordinate transformation on KF1 to obtain channel information KP1.

[0017] Furthermore, the method for determining KP2 is the same as the method for determining KP1, and the channel ordinal numbers corresponding to KP1 and KP2 are the same.

[0018] Furthermore, generating a phase correlation graph RS according to KP1 and KP2 includes:

[0019] Fourier convolution and correlation calculation are performed in sequence with KP1 as the two-dimensional convolution kernel and KP2 as the two-dimensional input signal to obtain the phase correlation graph RS.

[0020] Furthermore, determining a rotation parameter r and a scale parameter s between the template image and the reference image according to the RS includes:

[0021] Calculate the highest point of the response value in RS and obtain the coordinate value (r, log a s);

[0022] Based on (r,log a s) determining a rotation parameter r and a scale parameter s between the template image and the reference image; wherein a is the base of the log-polar coordinate transformation.

[0023] Furthermore, based on (r,log a s) Determining the scale parameter s between the template image and the reference image includes:

[0024] According to (r,log a s) and the base a in the log-polar transformation determine s.

[0025] Furthermore, calculating the translation parameters (x, y) between the template image and the reference image based on r, s, the residual channel information L1 of S1, and the residual channel information L2 of S2 includes:

[0026] Generate a phase correlation diagram XY based on r, s, residual channel information L1 of S1 and residual channel information L2 of S2;

[0027] Calculate the highest point of the response value in XY and obtain the coordinate value (x0, y0);

[0028] Determine the translation parameters (x,y) based on (x0,y0) and s.

[0029] Furthermore, generating a phase correlation diagram XY based on r, s, the residual channel information L1 of S1, and the residual channel information L2 of S2 includes:

[0030] Transform L1 based on r and s to obtain the transformed residual channel information L1;

[0031] Upsample L1 to obtain sampling information L1;

[0032] Upsample L2 to obtain sampling information L2;

[0033] With L1 as the two-dimensional convolution kernel and L2 as the two-dimensional input signal, Fourier convolution and correlation calculation are performed in sequence to obtain the phase correlation graph XY.

[0034] Furthermore, determining the translation parameters (x, y) according to (x0, y0) and s includes:

[0035]

[0036] Another technical solution of the present invention: a heterogeneous image registration device based on phase correlation calculation, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned heterogeneous image registration method based on phase correlation calculation is implemented.

[0037] The beneficial effects of the present invention are as follows: by extracting feature maps of the template image and the reference image, and then processing the feature maps by channel, the present invention can calculate the rotation parameters, scale parameters and translation parameters of the template image and the reference image at one time, thereby reducing the amount of calculation and reducing the redundancy of the calculation. It can be applied to devices with small video memory space, reduces the requirements for device performance, and improves real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a schematic diagram of a heterogeneous image registration method based on phase correlation calculation according to the present invention;

[0039] Figure 2 Schematic diagram of a template image and a reference image in an embodiment of the present invention;

[0040] Figure 3 Schematic diagram of a template image feature map and a reference image feature map in an embodiment of the present invention;

[0041] Figure 4 Schematic diagram of KP1 and KP2 after logarithmic polar coordinate transformation in an embodiment of the present invention;

[0042] Figure 5 is a phase correlation diagram RS in an embodiment of the present invention;

[0043] Figure 6 Schematic diagram of L1 and L2 in an embodiment of the present invention;

[0044] Figure 7 Schematic diagram of a phase correlation diagram XY in an embodiment of the present invention;

[0045] Figure 8 is a schematic diagram after registration obtained according to prediction parameters in an embodiment of the present invention;

[0046] Figure 9 Schematic diagram of other groups of reference images, template images, and registered images in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0048] Traditional phase correlation-based methods use hand-designed feature descriptors or grayscale information, which rely on expert knowledge and have poor generalization. However, the current deep learning-based phase correlation method has better accuracy for affine transformation than feature point-based methods.

[0049] Because existing methods slow down computations as resolution increases, they split an entire image into multiple blocks and execute the method on a per-block basis. However, blocks contain only local information, resulting in a fixed search space and limited solution accuracy. Furthermore, most current methods use two separate neural networks to predict rotation scale and translation parameters, respectively. This results in computational redundancy, significant memory usage, and high device requirements. Furthermore, two-stage training fails to account for the equivariance of neural networks, resulting in slower inference speeds.

[0050] The present invention discloses a heterogeneous image registration method based on phase correlation calculation, comprising the following steps: respectively extracting a first feature map S1 of a template image and a second feature map S2 of a reference image; determining channel information KP1 according to S1; determining channel information KP2 according to S2; generating a phase correlation map RS according to KP1 and KP2; determining a rotation parameter r and a scale parameter s between the template image and the reference image according to RS; and calculating a translation parameter (x, y) between the template image and the reference image based on r, s, residual channel information L1 of S1, and residual channel information L2 of S2.

[0051] The present invention extracts feature maps of the template image and the reference image, and then processes the feature maps by channel. It can calculate the rotation parameters, scale parameters and translation parameters of the template image and the reference image at one time, reducing the amount of calculation and the redundancy of the calculation. It is suitable for devices with small video memory space, reduces the requirements for device performance, and improves real-time performance.

[0052] Specifically, in an embodiment of the present invention, a lightweight twin network structure composed of an invert block (InvertBlock) is used to extract a low-resolution feature map. After the feature map is logarithmically transformed, a deep phase correlation operation and interpolation of rotation and scale are performed to obtain the optimal rotation and scale. The original feature map is then adjusted according to the rotation and scaling scale, and a deep phase correlation operation and interpolation are performed on the original feature map to obtain the optimal translation information and complete the registration. Among them, the deep phase correlation operation uses Fourier convolution, which can greatly improve the computational efficiency compared to traditional convolution, making the algorithm method suitable for real-time reasoning in embedded devices.

[0053] More specifically, the lightweight Siamese network consists of a 3×3 convolution and four InvertBlocks. The InvertBlock consists of a normal convolution, a depthwise separable convolution, and a skip-chain structure. The first convolution increases the dimension of the input features, followed by a depthwise separable convolution. This decouples the depthwise convolution from the pointwise convolution. Depthwise convolution refers to convolution on each channel, while batch normalization and nonlinear mapping of the activation function are performed. This structure achieves lower computational complexity, making it suitable for inference on edge devices. Finally, the pointwise convolution only performs point-by-point feature fusion and does not perform nonlinear mapping of the activation function.

[0054] First, the lightweight twin network needs to be trained. The training process in this embodiment of the present invention is as follows:

[0055] S01, input an optical image with a resolution of 256*256 as the reference image, and use a remote sensing image with a resolution of 192*192 as the template image (as the image to be registered). The template image is input into a lightweight Siamese network to obtain a feature map S1 of size [1, c+1, 48, 48] (c represents the number of channels minus 1). The reference image is input into another lightweight Siamese network to obtain a feature map S2 of size [1, c+1, 64, 64].

[0056] S02. Take out one channel of the template image feature map S1 (since the feature maps are all multi-channel feature maps, multiple channels can be selected in this embodiment according to the training situation), mark it as K1, and mark the remaining channel information as L1. Take out the first channel of the reference image feature map S2, mark it as K2, and mark the rest as L2. In this step, it should be noted that the channel ordinal numbers of K1 and K2 should be the same to ensure higher accuracy. The above content can be described in mathematical language as follows:

[0057] S1=encoder A (I 模板 ) (1)

[0058] S2=encoder B (I 参考 )

[0059] K1,L1=S1[:,0,...],S1[:,1:,...] (2)

[0060] K2,L2=S2[:,0,...],S2[:,1:,...]

[0061] Among them, I 模板 represents the template image, I 参考 represents the reference image, encoder A A lightweight twin network for representing template images, encoder B Represents a lightweight twin network for the reference image. In fact, encoder A and encoder B The network structures are the same and the network parameters are updated simultaneously.

[0062] S03. Perform Fourier transform on K1 and normalize it at the same time to obtain KF1 with a resolution of [48,48]. Perform Fourier transform on K2 and normalize it at the same time to obtain KF2 with a resolution of [64,64].

[0063] S04. Perform a log-polar coordinate transformation on KF1, where the vertical axis is the rotation angle and the horizontal axis is the logarithmic modulus. The modulus represents the scale, and the base of the logarithm is determined according to the actual situation. This yields KP1. Similarly, perform a log-polar coordinate transformation on KF2 to yield KP2. Mathematically, this is expressed as:

[0064]

[0065] Wherein, FFT represents fast Fourier transform, log-polar represents log-polar coordinate transform, r represents rotation parameter, a is a constant, and s represents scale parameter.

[0066] S05. Use KP1 as the two-dimensional convolution kernel and KP2 as the two-dimensional input signal, perform Fourier convolution and correlation calculation, and obtain the phase correlation graph RS. Mathematically expressed as:

[0067] RS=phase_correlation(KP1,KP2) (4)

[0068] The cross entropy loss function is used to supervise the phase correlation graph RS. The optimization target is the true value of the rotation parameter and scale parameter (tr, ts), and the error is LossRS:

[0069]

[0070] S06. Use the true value (tr, ts) to rotate L1 to obtain the rotated L1, upsample the L1 to obtain the rotated L1 as the two-dimensional convolution kernel, and upsample the L2 to obtain the L2 to obtain the input signal. Perform Fourier convolution and correlation calculation to obtain the phase correlation graph XY.

[0071] S08. Use the cross entropy loss function to supervise the phase correlation graph XY. The optimization target is the true value of the translation parameter (tx, ty), and the loss is LossXY. Mathematically expressed as:

[0072]

[0073] L1, L2 = up(L1), up(L2) (6)

[0074] XY=phase-correlation(L1,L2)

[0075] LossXY=exp(XY tx,ty )+log∑ x＝0,y＝0 ((exp(XY tx,ty )))

[0076] Here, up means upsampling.

[0077] Total back-propagation error:

[0078] Loss=LossRS+LossXY (7)

[0079] During the training process, the AdamW optimizer is used to update the network parameters. The above process is repeated until the loss function converges. At this point, the training of the lightweight twin network is completed. In this embodiment of the present invention, in order to reduce the computing time, a lightweight twin network is used to train the template image and the reference image synchronously. In other cases, a pseudo twin network can also be used, that is, the two network parameters are updated separately.

[0080] In the present invention, the actual application steps and some of the training steps are actually the same steps. The application steps of the present invention are described in detail below:

[0081] In one embodiment, determining the channel information KP1 according to S1 includes: extracting a piece of channel information K1 from S1; performing Fourier transform and normalizing K1 to obtain transformed channel information KF1; performing log-polar transform on KF1 to obtain channel information KP1.

[0082] More specifically, the method for determining KP2 is the same as the method for determining KP1, and the channel ordinal numbers corresponding to KP1 and KP2 are the same.

[0083] Preferably, generating the phase correlation graph RS according to KP1 and KP2 includes: performing Fourier convolution and correlation calculation in sequence with KP1 as a two-dimensional convolution kernel and KP2 as a two-dimensional input signal to obtain the phase correlation graph RS.

[0084] At the same time, the rotation parameter r and scale parameter s between the template image and the reference image are determined according to RS, including: calculating the highest point of the response value in RS, and obtaining the coordinate value (r, log a s); based on (r,log a s) Determine the rotation parameter r and scale parameter s between the template image and the reference image; where a is the base of the logarithmic polar coordinate transformation. a s) and the base a in the log-polar transformation determine s.

[0085] In one embodiment, calculating the translation parameters (x, y) between the template image and the reference image based on the residual channel information L1 of r, s, S1 and the residual channel information L2 of S2 includes: generating a phase correlation diagram XY based on the residual channel information L1 of r, s, S1 and the residual channel information L2 of S2; calculating the highest point of the response value in XY to obtain the coordinate value (x0, y0); and determining the translation parameters (x, y) based on (x0, y0) and s.

[0086] Specifically, generating a phase correlation diagram XY based on the residual channel information L1 of r, s, S1 and the residual channel information L2 of S2 includes: transforming L1 based on r and s to obtain the transformed residual channel information L1; upsampling L1 to obtain sampling information L1; upsampling L2 to obtain sampling information L2; and performing Fourier convolution and correlation calculation in sequence with L1 as a two-dimensional convolution kernel and L2 as a two-dimensional input signal to obtain the phase correlation diagram XY.

[0087] Finally, the translation parameters (x, y) are determined based on (x0, y0) and s, including:

[0088]

[0089] At this point, the final rotation, scale, and translation parameters (r, s, x, y) are obtained.

[0090] In order to describe the present invention more clearly, the method of the present invention is illustrated below using specific examples. The training set and test set used are from the SEN12 public dataset, with a total of 83,670 image pairs, where the ratio of the training set to the test set is 9:1, the optical image resolution is 256*256, and the SAR image resolution is 192*192. The original SEN12 dataset provides translation labels between each image pair, and there is no rotation and scale transformation between images. Based on the SEN12 data, image pairs with rotation and scale transformation are constructed by ourselves, and rotation and scale labels are generated. The specific steps are as follows:

[0091] S11, input an optical image with a size of 256*256 as a reference image and a remote sensing image with a resolution of 192*192 as a template image. Figure 2 As shown, Figure 2 (a) is the template image, Figure 2 (b) is the reference image. The template image is rotated by 10 degrees, the real rotation angle is 45 degrees, the scale change is 0.8, and the real scale change is 1.0. The template image is input into the lightweight twin network A to obtain a feature map S1 of size [64, 48, 48]. The reference image is input into the lightweight twin network B to obtain a feature map S2 of size [64, 64, 64]. Figure 3 As shown, Figure 3 (a) is the template image feature map, Figure 3 (b) is the reference image feature map.

[0092] S12: Extract the first channel of the template image feature map S1, mark it as K1, set the resolution to [48, 48], and mark the remaining channel information as L1. Extract the first channel of the reference image feature map S2, mark it as K2, set the resolution to [64, 64], and mark the remaining channel information as L2.

[0093] S13. Perform Fourier transform on K1 and normalize it at the same time to obtain KF1 with a resolution of [48,48]. Perform Fourier transform on K2 and normalize it at the same time to obtain KF2 with a resolution of [64,64].

[0094] S14. Perform a log-polar coordinate transformation on KF1, where the vertical axis is the rotation angle, the horizontal axis is the logarithmic modulus, and the base of the logarithm is determined during training, to obtain KP1. Perform a log-polar coordinate transformation on KF2, where the vertical axis is the rotation angle, the horizontal axis is the logarithmic modulus, and the base of the logarithm is determined during training, to obtain KP2. Figure 4 As shown, Figure 4 (a) is KP1, Figure 4 (b) is KP2.

[0095] S15, using KP1 as a two-dimensional convolution kernel and KP2 as a two-dimensional input signal, perform Fourier convolution and correlation calculation, such as Figure 5 As shown, a phase correlation plot RS with a resolution of [16,16] is obtained. The point with the highest response value is calculated and the coordinates are obtained as (45°, log1.006). Based on the base, the predicted rotation and scale parameters (45°, 1.006) are restored.

[0096] S16. Use the predicted (45°, 1.006) to rotate L1, as shown in Figure 6 As shown, the rotated L1 (i.e. Figure 6 (a)), the L1 obtained by upsampling is used as the two-dimensional convolution kernel, and the L2 obtained by upsampling L2 (i.e. Figure 6 (b)) is used as the input signal, and Fourier convolution and correlation calculation are performed to obtain the phase correlation diagram XY with a resolution of [16,16], as shown in Figure 7 shown.

[0097] S17. Calculate the highest point on the XY axis and obtain the coordinates (13, 15). Based on the scale ratio of the XY axis relative to the original input image, restore the high-precision translation parameters (52, 60). The final rotation, scale, and translation parameters are (45°, 1.006, 52, 60).

[0098] In addition, if Figure 9As shown, there are schematic diagrams of reference images (such as Figures a1, b1 and c1) of other groups of the present invention, template images (such as Figures a2, b2 and c2), template images transformed according to the predicted rotation and scale parameters (such as Figures a3, b3 and c3)) and images after registration (such as Figures a4, b4 and c4). From these result figures, it can be seen that the method of the present invention is applicable to a variety of remote sensing scenes, among which Figures a1-a4 correspond to urban and mountainous environments, Figures b1-b4 and Figures c1-c4 correspond to mountainous and river environments, and Figures c1-c4 correspond to cultivated land environments. The optical image is used as the reference image, and the SAR with random affine transformation is used as the template image. The transformed image can be obtained by performing affine transformation on the SAR template image in terms of rotation and scale parameters using the method of the present invention, and finally the SAR template image is registered to the corresponding position of the optical image.

[0099] In summary, in the present invention, the template image and the reference image are input into the feature extraction network in full size, and downsampling is introduced in the process of extracting the feature map. On the one hand, the full-size image input ensures a larger search space for the reference image, while being able to fully utilize the contextual information of the full image. On the other hand, the introduction of downsampling can effectively control the model size and reduce the burden on the operating equipment. Moreover, the entire process only has one feature extraction network, which can be trained end-to-end, without the need for two-stage training and manual feature design, and does not rely on expert knowledge. The equivariance of neural networks is utilized, which saves training time, improves the inference speed, and has good generalization.

[0100] Furthermore, after downsampling the feature map during feature extraction, upsampling it later when solving for the affine transformation parameters can avoid the loss of accuracy caused by ambiguity in matching positions due to downsampling the feature map during the feature extraction phase. Furthermore, during training, the rotation and scaling coefficients are first estimated, using a cross-entropy loss to supervise network training. The translation coefficients are then estimated, also using a cross-entropy loss to supervise network training. The network is jointly supervised and optimized using these two loss functions, resulting in excellent robustness.

[0101] The present invention achieves rotation scaling and translation of a shared neural network by coupling the feature extraction portion of the neural network. Downsampling is performed during feature extraction, while upsampling is added during transformation. This effectively suppresses computational redundancy, saves computing resources, and improves accuracy to a certain extent.

[0102] The present invention provides an end-to-end heterogeneous image registration solution, whose main operations are matrix multiplication and Fourier transform. Therefore, the method is platform-independent and can be run on hardware platforms such as GPU, CPU, and FPGA. End-to-end training can be achieved, and the deep phase correlation method used has good generalization compared to the solution based on feature points. At the same time, compared with the existing deep phase correlation method, the method of the present invention utilizes the equivariance of neural networks, does not require two-stage training and expert experience intervention, has fewer parameters and calculations, and has higher efficiency. The method of the present invention is computationally efficient, introduces downsampling in the feature extraction stage, and uses upsampling in the stage of solving the registration parameters to compensate for the quantization error of downsampling. The phase correlation calculation in the stage of solving the registration parameters adopts Fourier convolution, which greatly reduces the amount of calculation. The metric based on phase correlation has good interpretability, and the phase correlation heat map can be visualized for rotation, scaling, and translation. Furthermore, the method employs a data-driven approach. Its feature extraction network and phase correlation method are capable of processing diverse features generated by different sensors. This makes the network highly portable and applicable to a variety of multimodal image registrations, including but not limited to bird's-eye view and drone image registration, multimodal medical images, and multimodal natural images. The loss function exhibits excellent trainability and generalizability for this problem. The cross-entropy loss function for rotation and scaling and the cross-entropy loss function for translation parameters jointly supervise the network, making it more robust.

[0103] The present invention also discloses a heterogeneous image registration device based on phase correlation calculation, which includes a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, it implements the above-mentioned heterogeneous image registration method based on phase correlation calculation.

[0104] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices are based on the same concept as the embodiment of the method of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0105] The device can be a computing device such as a desktop computer, laptop, PDA, radar, or cloud server. The device may include, but is not limited to, a processor and memory. Those skilled in the art will appreciate that the device may include more or fewer components, or a combination of certain components, or different components, such as input / output devices, network access devices, etc.

[0106] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0107] In some embodiments, the memory may be an internal storage unit of the extraction device, such as a hard disk or memory of the extraction device. In other embodiments, the memory may also be an external storage device of the extraction device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, etc. equipped on the extraction device. Furthermore, the memory may also include both an internal storage unit of the extraction device and an external storage device. The memory is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory may also be used to temporarily store data that has been output or is to be output.

Claims

1. A heterogeneous image registration method based on phase correlation calculation, characterized in that: The following steps are involved: Extracting a first feature map S1 of the template image and a second feature map S2 of the reference image respectively; Determine channel information KP1 according to S1; determine channel information KP2 according to S2; Generate a phase correlation graph RS according to KP1 and KP2; Determine a rotation parameter r and a scale parameter s between the template image and the reference image according to the RS; Calculating a translation parameter (x, y) between the template image and the reference image based on r, s, the residual channel information L1 of S1, and the residual channel information L2 of S2; Determining a rotation parameter r and a scale parameter s between the template image and the reference image according to the RS includes: Calculate the highest point of the response value in the RS and obtain the coordinate value (r, log a s); Based on the (r,log a s) determining a rotation parameter r and a scale parameter s between the template image and the reference image; wherein a is the base number in the logarithmic polar coordinate transformation; based on the (r, log a s) determining a scale parameter s between the template image and the reference image includes: According to (r,log a s) and the base a in the log-polar coordinate transformation to determine the s; Calculating the translation parameters (x, y) between the template image and the reference image based on r, s, the residual channel information L1 of S1, and the residual channel information L2 of S2 includes: Generate a phase correlation diagram XY based on r, s, the residual channel information L1 of S1 and the residual channel information L2 of S2; Calculate the highest point of the response value in XY and obtain the coordinate value (x0, y0); The translation parameters (x, y) are determined based on (x0, y0) and s.

2. The heterogeneous image registration method based on phase correlation calculation according to claim 1, characterized in that: Determining the channel information KP1 according to S1 includes: Extracting a channel information K1 of said S1; Performing Fourier transform and normalizing the K1 to obtain transformed channel information KF1; Perform log-polar coordinate transformation on the KF1 to obtain channel information KP1.

3. The heterogeneous image registration method based on phase correlation calculation according to claim 2, characterized in that: The method for determining KP2 is the same as the method for determining KP1, and the channel ordinal numbers corresponding to KP1 and KP2 are the same.

4. A heterogeneous image registration method based on phase correlation calculation according to claim 2 or 3, characterized in that: Generating a phase correlation diagram RS according to KP1 and KP2 includes: Fourier convolution and correlation calculation are performed in sequence with KP1 as the two-dimensional convolution kernel and KP2 as the two-dimensional input signal to obtain a phase correlation graph RS.

5. The heterogeneous image registration method based on phase correlation calculation according to claim 4, characterized in that: Generating a phase correlation diagram XY based on r, s, the remaining channel information L1 of S1, and the remaining channel information L2 of S2 includes: Transform the L1 based on the r and s to obtain transformed residual channel information L1; Upsampling the L1 to obtain sampling information L1; Upsampling the L2 to obtain sampling information L2; Fourier convolution and correlation calculation are performed in sequence with L1 as the two-dimensional convolution kernel and L2 as the two-dimensional input signal to obtain a phase correlation graph XY.

6. A heterogeneous image registration method based on phase correlation calculation according to claim 4 or 5, characterized in that: Determining the translation parameters (x, y) according to (x0, y0) and s includes:

7. A heterogeneous image registration device based on phase correlation calculation, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the heterogeneous image registration method based on phase correlation calculation described in any one of claims 1 to 6 is implemented.

Images