Multi-path fusion image registration method

Abstract

The invention relates to a multi-path fusion image registration method. The method comprises the following steps of: firstly, combining images of all channels in pairs, calculating a transformation matrix, a root mean square error and coordinate transformation of image combination, and realizing direct registration of the image combination; then, multi-path indirect registration is carried out on the image combination, and conversion matrixes, root-mean-square errors and coordinate transformation of different paths are calculated; and finally, carrying out weighted fusion on coordinate transformation of direct registration and indirect registration by adopting an optimal linear unbiased estimation method to realize multi-path fusion registration of the multispectral image. By adopting the method provided by the invention, the consistency of image registration can be effectively improved, the random error of image registration is reduced, and the precision of image registration is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and analysis, in particular to image registration of multispectral cameras, and further relates to a multi-path fusion image registration method. Background Art

[0002] When a target is imaged by a multispectral camera, multispectral framed images can be obtained. The multispectral camera decomposes the incident light into multiple narrow-band optical signals through a spectroscopic technique and separately images them on an image detector, thereby obtaining images of different spectral bands. Currently, common framing numbers are four-frame (four channels), six-frame (six channels), eight-frame (eight channels), etc.

[0003] Image registration is an important technique in image processing and analysis. Its basic principle is to align multi-channel images to the same coordinate system for subsequent analysis and processing. The core objective of image registration is to find the geometric transformation relationship between images so that the images can be accurately matched in space.

[0004] In the document with the patent number "CN201510047728", "A High-Precision Multispectral Image Registration Method and Device for a Large Format with a Small Overlap Region" is given. It performs feature point matching on a reference image and an image to be registered through the SIFT method to obtain a preliminary registered image, uses the random sample consensus method to screen inliers, combines it with the least squares method to fit a curve, estimates a preliminary transformation matrix, and uses the root mean square error to screen out mis-matching points causing errors to obtain a transformation matrix. It still adopts a conventional registration method, taking one of the images as a reference image and the other images as images to be registered, and mapping the images to be registered into the coordinate system of the reference image respectively. The existing problems are: since different sub-images are selected as the reference image, the obtained affine transformation equations and registration errors are different, so the registration consistency is poor and the image registration error is large. Summary of the Invention

[0005] The present invention provides a multi-path fusion image registration method, which overcomes the problems of poor registration consistency and low image registration accuracy in the prior art, and realizes multi-path fusion registration of multispectral images.

[0006] To achieve the object of the present invention, the technical solution provided by the present invention is: a multi-path fusion image registration method, characterized by including the following steps:

[0007] Step 1: Combine the images of different channels of the multispectral pairwise, calculate the transformation matrix, standard deviation and coordinate transformation of the image combination, and realize the direct registration of the image combination;

[0008] Step 2: Perform multi-path indirect registration on the directly registered image combination. Utilize the result of direct registration to perform indirect registration on the multi-channel images, calculate the transformation matrices, standard deviations, and coordinate transformations for different paths, and obtain the coordinate transformation results;

[0009] Step 3: Adopt the optimal linear unbiased estimation method to perform weighted fusion on the multi-path coordinate transformation results, and calculate the fused coordinate transformation and registration error.

[0010] Further, in the above Step 2, the multi-path indirect registration refers to the direct registration of two-channel images or the indirect registration through images of different channels.

[0011] Further, in the above Step 2, for the indirect registration between multi-channel images, first register from channel i to channel k, and then register from channel k to channel j, which is expressed as:

[0012]

[0013] In the formula, H ik is the transformation matrix for directly transforming from channel i to channel k, and H kj is the transformation matrix for directly transforming from channel k to channel j;

[0014] Define H ikj as the transformation matrix for indirect registration, then: H ikj = H kj H ik

[0015] The standard deviation of indirect registration is:

[0016]

[0017] where σ ik is the registration error for directly transforming from channel i to channel k, and σ kj is the registration error for directly transforming from channel k to channel j.

[0018] Further, in the above Step 3, adopt the optimal linear unbiased estimation method to fuse the multi-path registration results, and the weight values are the reciprocals of the registration variances of each path. The fused coordinate transformation is expressed as:

[0019]

[0020] where n is the number of registration paths, i is the serial number of the registration path, X i is the transformation coordinates of different registration paths, and σ i is the standard deviation of different registration paths

[0021] The fused standard deviation is:

[0022]

[0023] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0024] 1. In step one of the present invention, pairwise combinations are randomly made between images in different channels without the need to select a reference image, and different registration results will not occur due to changes in the reference image, improving the consistency of image registration. The present invention performs data fusion through multi-path image registration, adopts the principle of the best linear unbiased estimation, and assigns weights to different registration paths, reducing the random error of image registration.

[0025] 2. After pairwise combining the images of different channels and directly registering them, the transformation matrix and root mean square error of the image combination can be obtained. By selecting the images of other channels as intermediate images and using the calculation results of direct registration, the indirect registration of two images can be achieved. Selecting an intermediate image for the registration of two images can utilize the registration results of different paths, reduce the error introduced by a single path, and improve the accuracy of image registration. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is the feature point pairing diagram of the four-frame image.

[0027] Figure 2 is the schematic diagram of the registration path of the four-frame image from channel 1 to channel 2. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] The present invention will be further described in detail below in conjunction with the drawings.

[0029] A multi-path fusion image registration method provided by the present invention first makes pairwise combinations of the images of all channels, performs direct registration of the image combinations, then performs multi-path indirect registration on the image combinations, calculates the transformation matrix, root mean square error, and coordinate transformation of each indirect registration. Finally, the optimal linear unbiased estimation method is used to perform weighted fusion on the coordinate transformations of direct registration and indirect registration to achieve high-precision image registration.

[0030] Embodiment: For the four-frame image, the present invention provides a multi-path fusion image registration method, see Figure 1 and Figure 2 , which specifically includes the following steps:

[0031] Step 1, make pairwise combinations of the images of different channels, calculate the transformation matrix, standard deviation (root mean square error), and coordinate transformation of the image combinations, and achieve direct registration of the image combinations. The specific steps are as follows:

[0032] In the images of different channels, after the feature point (x0, y0) undergoes affine transformation, the coordinates are (x'0, y'0), which is expressed as:

[0033]

[0034] An affine transformation has 6 parameters. At least 3 pairs of feature points are required to solve the affine transformation matrix.

[0035] The affine transformation equation is a system of linear equations, which is expressed as AX = B, where A is the coefficient matrix, X is the solution vector, and B is the constant term vector. For the affine transformation equation, when the number of feature point pairs is n, the coefficient matrix, the solution vector, and the constant term vector are respectively:

[0036]

[0037]

[0038] The coefficient matrix has 6 columns and 2n rows. One pair of feature points can list two equations. When n > 3, the equation is an overdetermined equation. Multiply both sides of the equation by the transpose of the coefficient matrix A, and the overdetermined equation can be converted into a positive definite equation, so as to obtain the solution of the equation (the least squares solution), which is expressed as:

[0039] A T AX = B,

[0040] The least squares solution is:

[0041] X = (A T A) -1 B,

[0042] After obtaining the geometric transformation parameters, the registration of the reference image and the image to be registered can be realized.

[0043] Using the least squares method to obtain 6 parameters of the affine transformation, and then substituting the 6 parameters into the 2n equations listed, calculate the residual ε of each equation i , and the residual of the equation is the registration error of each pair of feature points. The standard deviation (root mean square error) of image registration is:

[0044]

[0045] Among them, ε i is the residual of the affine transformation equation, n is the number of feature point pairs, and i is the equation number.

[0046] Step 2: Perform multi-path indirect registration on the directly registered image combination. Using the result of direct registration, perform indirect registration on the multi-channel image to calculate the transformation matrix, standard deviation, and coordinate transformation of different paths, and obtain the coordinate transformation result. The specific steps are as follows;

[0047] In this embodiment, indirect registration is performed using images of different channels, that is, different intermediate images are selected for indirect registration. If the number of channels is n, there are n - 2 indirect registration paths in total.

[0048] Taking four-frame imaging as an example, the four sub-images are respectively called ①, ②, ③, and ④. The affine transformation between any two channels is defined as:

[0049] X j = H ij X i (i, j = 1, 2, 3, 4, i ≠ j),

[0050] Image registration is essentially a coordinate transformation, and there are multiple paths for coordinate transformation between images. For example, there are 3 paths for coordinate transformation from channel 1 to channel 2.

[0051] The calculated values and errors of the 3 paths are respectively:

[0052] (1) ① Direct coordinate transformation to ②, the transformation matrix is H 12 , and the coordinate transformation is The registration error is σ0 = σ 12 .

[0053] (2) ① First, coordinate transformation to ③, and then coordinate transformation from ③ to ②, the transformation matrix is H 132 = H 32 H 13 , and the coordinate transformation is The registration error is

[0054] (3) ① First, coordinate transformation to ④, and then coordinate transformation from ④ to ②, the transformation matrix is H 142 = H 42 H 14 , and the coordinate transformation is The registration error is

[0055] Step 3, using the optimal linear unbiased estimation method, perform weighted fusion on the coordinate transformation results of multiple paths, calculate the coordinate transformation and registration error after fusion, and the weight is the reciprocal of the error (variance) of each measurement:

[0056] Taking the weighted average of the calculated values of these 3 paths, the optimal estimate of the registration from ① to ② can be obtained, and the weight values are the reciprocals of the variances of each path. That is, the coordinate transformation after fusion is expressed as:

[0057]

[0058] The registration error after fusion is:

[0059]

[0060] Assume that the pairwise registration errors of four images are the same, denoted as σ0. Then the registration errors of the three paths are respectively: σ0, The registration error of the multi-path image registration method is:

[0061]

[0062] That is, under the condition of the same direct registration error, for the registration of four images, by adopting the multi-path image registration method and fusing the coordinate transformation results of direct registration and indirect registration, the image registration error can be reduced by 29%.

[0063] The more channels the multi-spectral image has, the higher the accuracy of multi-path fusion registration. When there are 6 channels, the registration error is reduced by 42%. When there are 8 channels, the registration error is reduced by 50%.

[0064] Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Claims

1. A multi-path fusion image registration method, characterized in that Including the following steps: Step 1: Combine the images of different channels of the multi-spectral pairwise, calculate the transformation matrix, standard deviation and coordinate transformation of the image combination, and realize the direct registration of the image combination; Step 2: Perform multi-path indirect registration on the directly registered image combination. Utilize the result of the direct registration to perform indirect registration on the multi-channel images, and calculate the transformation matrix, standard deviation and coordinate transformation of different paths to obtain the coordinate transformation result; Step 3: Adopt the optimal linear unbiased estimation method to perform weighted fusion on the coordinate transformation results of multiple paths, and calculate the fused coordinate transformation and registration error.

2. The multi-path fusion image registration method according to claim 1, wherein: In the said Step 2, the multi-path indirect registration refers to the direct registration of the images of two channels, or the indirect registration through the images of different channels.

3. The multi-path fusion image registration method according to claim 1, characterized in that: In the said Step 2, for the indirect registration between multi-channel images, first register from channel i to channel k, and then register from channel k to channel j, which is expressed as: where H ik is the conversion matrix for directly converting channel i to channel k, and H kj is the conversion matrix for directly converting channel k to channel j; Define H ikj as the transformation matrix for indirect registration, then: H ikj = H kj H ik The standard deviation of the indirect registration is: where σ ik is the registration error of directly transforming channel i to channel k, and σ kj is the registration error of directly transforming channel k to channel j.

4. The multi-path fusion image registration method according to claim 1, wherein: In the said Step 3, adopt the optimal linear unbiased estimation method to fuse the multi-path registration results, and the weight values are the reciprocals of the registration variances of each path respectively. The fused coordinate transformation is expressed as: where n is the number of registration paths, i is the serial number of the registration path, and X i is the transformation coordinates of different registration paths, and σ i is the standard deviation of different registration paths The fused standard deviation is:

Images