Similarity optimal on-board image registration method based on inertial navigation data fast convergence

Abstract

The application discloses a similarity optimal on-board image registration method based on inertial navigation data fast convergence, and comprises the following steps: constructing a projection transformation model between an image transformation matrix, a to-be-registered image and a reference image; obtaining the projection transformation matrix according to the field of view optical axis position and three-angle offset of the satellite at the imaging moment and coarse registration; performing global coarse matching by using the projection transformation model and the projection transformation matrix to obtain a coarse matching image; decoupling the image transformation matrix according to a mathematical model to obtain a to-be-solved transformation parameter, substituting the to-be-solved transformation parameter into the image transformation matrix to obtain an optimal transformation matrix, and applying the optimal transformation matrix to the to-be-registered image to obtain a fine registration image; and calculating the similarity of the fine registration image and the reference image according to a local normalized image similarity measurement algorithm to obtain an optimal matching result. The application has the advantages of meeting the on-board calculation capacity requirement, meeting the image processing precision requirement and meeting the on-board storage resource requirement.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of on-board image registration, and particularly relates to a similarity-optimal on-board image registration method based on fast convergence of inertial navigation data. BACKGROUND

[0002] Currently, the research on image registration technology mainly focuses on three directions: image registration based on projection transformation matrix, image registration based on gray information, and image registration based on feature points. For a satellite platform, due to limited storage resources, most ground algorithms have some problems that cannot be directly transplanted.

[0003] The image registration technology based on projection transformation is based on the principle of camera projection imaging. Through the coordinate conversion among the camera coordinate system, the world coordinate system and the imaging coordinate system, the six-degree-of-freedom parameters of the camera are measured by using the inertial navigation device of the system, the parameters of the image conversion matrix are calculated, and the image registration is completed. However, this method has the following problems: (1) the attitude solution of the inertial navigation system is an integral process, which has cumulative error by itself, and the cumulative error increases with time; (2) the acquisition, storage and deployment of the parameters of the inertial navigation system; (3) the vibration of the satellite platform, which may cause a deviation of 3 pixels at the maximum, and the inertial navigation system parameters cannot measure this disturbance.

[0004] The image registration based on gray information and the image registration based on feature points both belong to the feature-based method. (1) The remote sensing image data is large, the natural scene imaging has only 8bit width, the remote sensing image has a maximum of 16bit width, the width of the remote sensing image is much larger than that of the natural scene, the resolution is much higher than that of the natural scene, and the effective pixels contained are several tens or even several hundreds of times of those of the natural scene; (2) the calculation amount is large, the calculation of the most commonly used corner points, surf feature points or sift points still requires a large amount of calculation resources for on-orbit calculation of the satellite platform, and it is a major challenge for the implementation on the satellite. SUMMARY

[0005] The technical problem solved by the application is to overcome the shortcomings of the prior art and provide a similarity-optimal on-board image registration method based on fast convergence of inertial navigation data. The method solves the problems of low calculation accuracy and inability to overcome system errors in image registration by simply using inertial navigation data, and the problem of large resource occupation of the starboard hardware platform in the implementation of the image information calculation. A similarity-optimal global search algorithm is proposed to simplify the calculation process. However, this global search has a serious speed deficiency, so the inertial navigation data is used to quickly converge the global search algorithm. Finally, an on-board image registration algorithm is proposed, which meets the requirements of on-board calculation capability, image processing accuracy and on-board storage resources.

[0006] The application is achieved by the following technical scheme: a similarity optimal on-orbit image registration method based on inertial navigation data fast convergence, comprising the following steps: step one: according to the related basic theory of image projection transformation, a projection transformation model between an image transformation matrix, a to-be-registered image and a reference image is constructed; step two: the field of view optical axis position and three-angle offset of a satellite at an imaging time are obtained by using the parameters of an inertial navigation system at a current time, and the projection transformation matrix of the coarse registration is obtained according to the field of view optical axis position and three-angle offset of the satellite at the imaging time; step three: the projection transformation model in step one and the projection transformation matrix calculated in step two are used to perform global coarse matching to obtain a coarse matching image; step four: the image transformation matrix is decoupled to obtain the transformation parameters to be solved according to the mathematical model in step one, the optimal transformation matrix is obtained by substituting the transformation parameters to be solved into the image transformation matrix, and the fine registration image is obtained by applying the optimal transformation matrix to the to-be-registered image; and step five: the similarity of the fine registration image and the reference image is calculated to obtain the optimal matching result according to the local normalized image similarity measurement algorithm.

[0007] The similarity optimal on-orbit image registration method based on inertial navigation data fast convergence further comprises: step six: the inertial navigation parameter error in step two is positively corrected by using the optimal transformation matrix calculated at the current time.

[0008] In the similarity optimal on-orbit image registration method based on inertial navigation data fast convergence, in step four, the optimal value is searched in the coarse matching image during decoupling.

[0009] In the similarity optimal on-orbit image registration method based on inertial navigation data fast convergence, in step five, the local normalized image similarity measurement algorithm comprises: the angle similarity, Mahalanobis distance and intensity similarity are coupled, and the coupling result is normalized to calculate the optimal matching result.

[0010] The application discloses a similarity optimal on-orbit image registration system based on inertial navigation data fast convergence, which comprises the following modules: a first module is used for constructing a projection transformation model among an image transformation matrix, a to-be-registered image and a reference image according to an image projection transformation correlation basic theory; a second module is used for obtaining a field of view optical axis position and a three-angle offset of a satellite at an imaging time according to parameters of an inertial navigation system at a current time, and obtaining a projection transformation matrix from the field of view optical axis position and the three-angle offset of the satellite at the imaging time to coarse registration; a third module is used for performing global coarse matching by using the projection transformation model and the projection transformation matrix to obtain a coarse matching image; a fourth module is used for decoupling the image transformation matrix according to a mathematical model to obtain to-be-solved transformation parameters, substituting the to-be-solved transformation parameters into the image transformation matrix to obtain an optimal transformation matrix, and applying the optimal transformation matrix to the to-be-registered image to obtain a fine registration image; and a fifth module is used for calculating the similarity of the fine registration image and the reference image according to a local normalized image similarity measurement algorithm to obtain an optimal matching result.

[0011] In the similarity optimal on-orbit image registration system based on inertial navigation data fast convergence, a sixth module is further used for performing positive correction on inertial navigation parameter errors by using an optimal transformation matrix obtained at the current time.

[0012] In the similarity optimal on-orbit image registration system based on inertial navigation data fast convergence, when decoupling, the optimal value is searched in the coarse matching image.

[0013] In the similarity optimal on-orbit image registration system based on inertial navigation data fast convergence, the local normalized image similarity measurement algorithm comprises the following steps: coupling is performed by using angle similarity, Mahalanobis distance and intensity similarity, and the coupling result is normalized to calculate the optimal matching result.

[0014] Compared with the prior art, the application has the following beneficial effects:

[0015] (1) The application fuses feature-based image registration and projection transformation-based image registration algorithms on the basis of using image gray feature information, reduces the calculation amount as much as possible under the condition of meeting the calculation precision, and meets the requirement on hardware;

[0016] (2) The application accelerates and optimizes the registration process by using inertial navigation parameters, meets the requirement on operation speed, and is a real-time image registration algorithm which can be realized on an on-orbit satellite;

[0017] (3) The application corrects errors by using the accurate registration result and the inertial navigation system, and improves the calculation precision of coarse registration. BRIEF DESCRIPTION OF DRAWINGS

[0018] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0019] Figure 1 This is a flowchart of the similarity-optimal on-board image registration method based on fast convergence of inertial navigation data provided in an embodiment of the present invention;

[0020] Figure 2 This is a schematic diagram of the vertical optical axis offset model;

[0021] Figure 3 This is a schematic diagram of the search algorithm model;

[0022] Figure 4 A schematic diagram of the search algorithm process for optimizing inertial navigation parameters. Detailed Implementation

[0023] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] Current algorithm research mainly focuses on feature extraction, description, and feature matching based on ground-based computer simulations. Therefore, this invention addresses the limitations of spaceborne platform data processing capabilities and proposes an on-board image registration technique based on fast convergence of inertial navigation data, aiming to provide an image registration algorithm suitable for on-board platform applications.

[0025] like Figure 1 As shown, this on-board image registration method based on fast convergence of inertial navigation data and optimal similarity includes the following steps:

[0026] Step 1: Based on the fundamental theories of image projection transformation, construct a mathematical model between the image transformation matrix, the image to be registered, and the reference image;

[0027] The image transformation matrix is

[0028] Step two: using the parameters of the inertial navigation system at the current time, calculate the position of the field of view optical axis and the three-angle offset of the satellite at the imaging time, and obtain the projection transformation matrix of the coarse registration.

[0029] Step three: using the projection transformation model in step one and the projection transformation matrix calculated in step two, perform global coarse matching to obtain a coarse matching image, which is also used for fast convergence in the optimization process in step three and four.

[0030] Step four: decouple the image transformation matrix according to the mathematical model in step one, perform fine registration, search for the optimal value in a certain range, obtain the transformation parameters to be solved, and substitute the image transformation matrix with the optimal transformation matrix to obtain the optimal transformation matrix. Apply the transformation matrix to the image to be registered to calculate the fine registration image.

[0031] Step five: calculate the similarity of the fine registration image and the reference image, propose a local normalized image similarity measurement algorithm, and use angle similarity, Mahalanobis distance, and intensity similarity for coupling. Normalize the coupling result to calculate the optimal matching result.

[0032] Step six: use the optimal transformation matrix calculated at the current time to accurately register the inertial navigation parameter error in step two.

[0033] Image registration based on similarity measurement:

[0034] The angular displacement measurement system measures the offset of the camera optical axis during the imaging of multiple images, which is converted into image deviation for image registration. In order to calculate the offset between different image frames, the gyroscope can measure the attitude change of different frames of images. Through a strict imaging geometric model, the optical axis offset between adjacent frames can be calculated, and the image pixel offset between adjacent frames can be calculated by the optical axis offset, as shown in Figure 2 The rotational displacement in the roll axis and pitch axis directions can be collectively referred to as the rotational displacement in the vertical optical axis direction, and they have similar effects on image motion, so they are placed together. The general projection transformation formula is:

[0035]

[0036] For a two-dimensional image, w is 0 in the above formula, u and v are the original image coordinates, and x and y are the coordinates of the transformed image. The transformation matrix can be divided into four parts: represents linear transformation, corresponding to scaling, rotation, [a 31 a 32 ] represents translation transformation, [a 13 a 23 ] T corresponding to perspective transformation.

[0037] For geostationary satellites with an orbit of 36,000 kilometers and an Earth observation swath of 300 kilometers, calculations and analysis show that the impact of satellite rotation around the optical axis on image shift is small for their remote sensing satellite imaging platform, and the resulting image scaling is almost zero. Therefore, these two types of transformations are no longer considered during image registration.

[0038] Therefore, after simplifying the transformation matrix:

[0039]

[0040] For satellite platforms, geostationary orbit satellites, the impact [a] 31 a 32 The main factor is satellite vibration, which has a maximum envelope value, typically ±3 to 5 pixels, and this value can be determined in orbit. Therefore, image registration can be transformed into an optimization problem.

[0041] For the x-direction and y-direction respectively in [-a 31 +a 31 ]、[-a 32 +a 32 Find the optimal solution within the range [a] x a y The transformation matrix is ​​used to obtain the image with the best similarity to the original image, as shown below:

[0042] a x ∈[-a 31 +a 31 ]a y ∈[-a 32 +a 32 (3)

[0043]

[0044] The original image has coordinates [x, y, z]. Similarity calculation:

[0045]

[0046] In the above formula: a is vector [x,y], representing the coordinate system of the reference image; b is vector [x′,y′], representing the coordinate system of the finely registered image; Δθ is the angular similarity between vectors a and b; x is the abscissa of the reference image; x′ is the abscissa of the finely registered image; y is the ordinate of the reference image; y′ is the ordinate of the finely registered image; ΔD is the Mahalanobis distance between the reference image and the finely registered image; Δρ is the correlation measure between the reference image and the finely registered image; D(x′) is the variance of x′; DIS is the final local normalized similarity measure, representing the similarity between the reference image and the finely registered image; Cov(x,x′) is the covariance of x and x′; D(x) is the variance of x; ΔD 0～1The value of AD is normalized [0-1], and the calculation process is as follows:

[0047]

[0048] The final formula is that the similarity measure DIS is in (0, 1), and the closer to 1, the higher the similarity.

[0049] The above formula can represent the rotation-invariant similarity, scaling similarity and pixel similarity of two images. Compared with a single similarity measure function, the method effectively improves the stability, and has better effect on the bottom critical point problem.

[0050] Fast convergence optimization model based on inertial navigation parameters:

[0051] For the above method, the optimal matching translation needs to be optimized in a certain range, such as Figure 3 The rough target movement direction can be obtained by using the inertial navigation system. First, the target displacement image is obtained, and the data of the previous frame is used to roughly determine the quadrant, and one-step coarse matching is performed. This process searches for compensation and can obtain a larger value. The coarse matching result and the current time parameter are used for fine matching. This process realizes 0.5 pixel accuracy registration, and the selected step is accurate to 0.2.

[0052] Finally, the fine matching result of the current frame is applied to the inertial navigation system parameter of the next time to filter and eliminate the cumulative error of the inertial navigation system. The flow chart is shown in Figure 4

[0053] In the simulation experiment, a random vibration model is added to the remote sensing image, and the optimal simulation of random disturbance is performed by using the method. Before registration, the same pixel offset of the first two images is 0.8 pixels in the x direction, 3.8 pixels in the y direction, and 4.3 pixels in the distance. After registration, the image offset is 0.3 pixels in the x direction, 0.3 pixels in the y direction, and 0.5 pixels in the distance. Therefore, the invention can realize 0.5 pixel image registration accuracy, and the calculation amount is simple, which can realize the requirements of on-board platform processing.

[0054] ​The embodiment also provides a similarity-optimal on-satellite image registration system based on quick convergence of inertial navigation data, comprising: a first module configured to construct a projection transformation model among an image transformation matrix, a to-be-registered image and a reference image according to a related basic theory of image projection transformation; a second module configured to obtain a field-of-view optical axis position and a three-angle offset of a satellite at an imaging time by using parameters of an inertial navigation system at a current time, and obtain a projection transformation matrix for coarse registration according to the field-of-view optical axis position and the three-angle offset of the satellite at the imaging time; a third module configured to obtain a coarse matching image by using the projection transformation model and the projection transformation matrix for global coarse matching; a fourth module configured to decouple the image transformation matrix according to a mathematical model to obtain to-be-solved transformation parameters, substitute the to-be-solved transformation parameters into the image transformation matrix to obtain an optimal transformation matrix, and apply the optimal transformation matrix to the to-be-registered image to obtain a fine registration image; a fifth module configured to obtain an optimal matching result by calculating similarity of the fine registration image and the reference image according to a local normalized image similarity measurement algorithm; and a sixth module configured to perform positive correction on inertial navigation parameter errors by using the optimal transformation matrix obtained at the current time for accurate registration.

[0055] The application fuses feature-based image registration and projection transformation-based image registration algorithms on the basis of image gray feature information, reduces the amount of calculation as much as possible under the condition of meeting the calculation precision, and meets the requirement on hardware; the application accelerates and optimizes the registration process by using inertial navigation parameters, meets the requirement on operation speed, and is a real-time image registration algorithm that can be implemented on a satellite; and the application corrects errors by using the accurate registration result and the inertial navigation system, and improves the precision of coarse registration calculation.

[0056] Although the application has been disclosed as above with reference to the preferred embodiments, it is not intended to limit the application, and any person skilled in the art can make possible changes and modifications to the technical solutions of the application by using the disclosed methods and technical contents without departing from the spirit and scope of the application, therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the application without departing from the technical solutions of the application all belong to the protection scope of the technical solutions of the application.

Claims

1. A similarity-optimal on-board image registration method based on fast convergence of inertial navigation data, characterized in that, The method includes the following steps: Step 1: Construct the image transformation matrix and the projection transformation model between the image to be registered and the reference image; Step 2: Use the parameters of the inertial navigation system at the current moment to obtain the optical axis position and three-angle offset of the satellite at the imaging moment, and then use the projection transformation matrix of the satellite's optical axis position and three-angle offset at the imaging moment to perform coarse registration. Step 3: Use the projection transformation model from Step 1 and the projection transformation matrix calculated in Step 2 to perform global coarse matching to obtain the coarse matching image; Step 4: Based on the projection transformation model in Step 1, decouple the image transformation matrix to obtain the transformation parameters to be solved. Substitute the transformation parameters to be solved into the image transformation matrix to obtain the optimal transformation matrix. Apply the optimal transformation matrix to the image to be registered to obtain the finely registered image. Step 5: Calculate the similarity between the finely registered image and the reference image using the local normalized image similarity measurement algorithm to obtain the optimal matching result; The offset of the camera optical axis during the imaging of multiple frames is measured using an angular displacement measurement system and converted into image deviation for image registration; the general transformation formula for projection transformation is as follows: For two-dimensional images, w is 0 in the above formula, u and v are the coordinates of the original image, and x and y are the coordinates of the transformed image; Simplify the transformation matrix: Image registration is transformed into an optimization problem; For the x-direction and y-direction respectively in [-a 31 +a 31 ]、[-a 32 +a 32 Find the optimal solution within the range [a] x a y The transformation matrix is ​​used to obtain the image with the best similarity to the original image, as shown below: a x ∈[-a 31 +a 31 ]a y ∈[-a 32 +a 32 ](3) The original image has coordinates [x, y, z]. Similarity calculation: In the above formula: a is vector [x,y], representing the coordinate system of the reference image; b is vector [x′,y′], representing the coordinate system of the finely registered image; Δθ is the angular similarity between vectors a and b; x is the abscissa of the reference image, x′ is the abscissa of the finely registered image, y is the ordinate of the reference image, y′ is the ordinate of the finely registered image, ΔD is the Mahalanobis distance between the reference image and the finely registered image, Δρ is the correlation measure between the reference image and the finely registered image, D(x′) is the variance of x′; DIS is the final local normalized similarity measure, representing the similarity between the reference image and the finely registered image; Cov(x,x′) is the covariance of x and x′, D(x) is the variance of x, ΔD 0～1 The normalized [0-1] value of ΔD is calculated as follows: In the final expression above, the similarity measure DIS∈(0,1) is closer to 1, indicating a higher degree of similarity.

2. The similarity-optimal on-board image registration method based on fast convergence of inertial navigation data according to claim 1, characterized in that... It also includes: Step 6: Using the optimal transformation matrix calculated at the current time to accurately register the matrix to positively correct the inertial navigation parameter error in Step 2.

3. The similarity-optimal on-board image registration method based on fast convergence of inertial navigation data according to claim 1, characterized in that: In step four, during decoupling, the optimal value is searched in the coarse matching image.

4. A similarity-optimal on-board image registration system based on fast convergence of inertial navigation data, characterized in that... include: The first module is used to construct the image transformation matrix, the projection transformation model between the image to be registered and the reference image, based on the fundamental theories related to image projection transformation. The second module is used to obtain the optical axis position and three-angle offset of the satellite at the imaging time using the parameters of the inertial navigation system at the current time, and to obtain the projection transformation matrix of coarse registration based on the optical axis position and three-angle offset of the satellite at the imaging time. The third module is used to perform global coarse matching using the projection transformation model and the projection transformation matrix to obtain a coarse matching image. The fourth module is used to decouple the image transformation matrix according to the mathematical model to obtain the transformation parameters to be solved, substitute the transformation parameters to be solved into the image transformation matrix to obtain the optimal transformation matrix, and apply the optimal transformation matrix to the image to be registered to obtain the finely registered image. The fifth module is used to calculate the similarity between the finely registered image and the reference image based on the local normalized image similarity measurement algorithm to obtain the optimal matching result; The offset of the camera optical axis during the imaging of multiple frames is measured using an angular displacement measurement system and converted into image deviation for image registration; the general transformation formula for projection transformation is as follows: For two-dimensional images, w is 0 in the above formula, u and v are the coordinates of the original image, and x and y are the coordinates of the transformed image; Therefore, after simplifying the transformation matrix: Image registration can be transformed into an optimization problem; For the x-direction and y-direction respectively in [-a 31 +a 31 ]、[-a 32 +a 32 Find the optimal solution within the range [a] x a y The transformation matrix is ​​used to obtain the image with the best similarity to the original image, as shown below: a x ∈[-a 31 +a 31 ]a y ∈[-a 32 +a 32 ](3) The original image has coordinates [x, y, z]. Similarity calculation: In the above formula: a is vector [x,y], representing the coordinate system of the reference image; b is vector [x′,y′], representing the coordinate system of the finely registered image; Δθ is the angular similarity between vectors a and b; x is the abscissa of the reference image, x′ is the abscissa of the finely registered image, y is the ordinate of the reference image, y′ is the ordinate of the finely registered image, ΔD is the Mahalanobis distance between the reference image and the finely registered image, Δρ is the correlation measure between the reference image and the finely registered image, D(x′) is the variance of x′; DIS is the final local normalized similarity measure, representing the similarity between the reference image and the finely registered image; Cov(x,x′) is the covariance of x and x′, D(x) is the variance of x, ΔD 0～1 The normalized [0-1] value of ΔD is calculated as follows: In the final expression above, the similarity measure DIS∈(0,1) is closer to 1, indicating a higher degree of similarity.

5. The similarity-optimal on-board image registration system based on fast convergence of inertial navigation data according to claim 4, characterized in that... It also includes a sixth module, which is used to perform positive correction of inertial navigation parameter errors by using the optimal transformation matrix calculated at the current time to accurately register the matrix.

6. The similarity-optimal on-board image registration system based on fast convergence of inertial navigation data according to claim 4, characterized in that: During decoupling, the optimal value is searched in the coarse matching image.

Images