A method for improving the quality of high-resolution satellite fusion images by compensating for mapping errors

Abstract

The present invention discloses a method for improving the quality of high-resolution satellite fusion images, which is divided into four parts: high-precision dense matching, solving the parameters of the linear fitting model, solving the parameters of the piecewise model, and solving the parameters of the trigonometric series model. First, high-precision matching of panchromatic and multispectral images is completed to obtain dense matching points, and the fusion registration error is obtained through the "back-projection - reprojection" process. Secondly, a linear model is used to fit the fusion registration error to solve the parameters of the linear model. Then, a piecewise model is used to fit the remaining error to solve the parameters of the piecewise model. Then, a trigonometric series model is used to fit the finally remaining error to solve the parameters of the trigonometric series model. Finally, the coordinate correction value is calculated for each point of the original multispectral image using the sum of the above three models, and the image point position is corrected to obtain a multispectral image without error, completing the improvement of the quality of the fusion image.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing image processing methods, and particularly relates to a method for improving the quality of high-resolution satellite fusion images by compensating mapping errors. Background Art

[0002] Image fusion is an important process in optical remote sensing image processing. Optical remote sensing satellites generally obtain data in the form of a combination of high-resolution panchromatic images and low-resolution multispectral images, and perform image fusion during ground processing, thus obtaining high-resolution multi-band images while saving valuable satellite communication bandwidth, on-board storage space, and signal processing power consumption. Before image fusion, it is generally necessary to first perform registration of the panchromatic image and the multispectral image, which is called "fusion registration". This is because affected by factors such as the CCD placement accuracy of the panchromatic optical sensor and the multispectral optical sensor, lens distortion, exterior orientation element errors, and satellite platform flutter, there are often non-linear and discontinuous geometric misalignments between the panchromatic image and the multispectral image. If effective geometric compensation cannot be performed, the fused image often exhibits a halo phenomenon, resulting in a decrease in image sharpness and seriously affecting the detectability, resolvability, and measurability of the fused image.

[0003] Image fusion is an important research direction in the field of remote sensing. Currently, commonly used fusion methods include principal component transformation, high-pass filtering, wavelet transformation, and the Pansharp method, etc. These methods are all used under the condition that the panchromatic image and the multispectral image are accurately registered. Therefore, the registration problem needs to be solved first in actual production. Currently, when commonly used commercial software performs fusion error compensation on first-level image products (i.e., satellite images expressing geographic information through Rational Polynomial Coefficient (RPC)), the small facet method is generally adopted. However, the fusion registration error compensation model based on small facet correction has too many parameters and requires uniformly distributed matching points to be stably solved, which not only limits the operation efficiency but also is difficult to handle situations such as misalignment of the panchromatic and multispectral camera sub-array CCDs in the rear view of satellite sensors. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, the present invention proposes a method for improving the quality of high-resolution satellite fusion images by compensating panchromatic-multispectral mapping errors to solve the problem of improving the quality of high-resolution satellite fusion.

[0005] Based on the imaging geometric laws of multi-spectral images and panchromatic images, this invention analyzes the generation mechanism of fusion registration errors, proposes a panchromatic-multi-spectral mapping error model applicable to the entire image, conducts quantitative calculation and precise compensation of fusion registration errors, and maximally improves the quality of fused image products. Meanwhile, a method of "segmented term multiplexing" is proposed to provide a reference for high-efficiency and high-precision production of fused images. The technical solution adopted by this invention is: a method for improving the quality of high-resolution satellite fused images by compensating panchromatic-multi-spectral mapping errors, including the following steps:

[0006] Step 1: Obtain high-precision dense matching points of panchromatic-multi-spectral images. Define the matching points between panchromatic / multi-spectral as {(p, p′)}. Project the image point p = (s, l) with column number s and row number l on the panchromatic image to the multi-spectral image plane through the "back-projection - reprojection" process (i.e., with the assistance of elevation data, project it to the object space using the RPC parameter model of the panchromatic image, and then project it to its multi-spectral image space using the RPC parameter model of the multi-spectral image) to obtain the image point p″ = (s″, l″). Solve the coordinate difference between p′ = (s′, l′) and p″ = (s″, l″) (where as the error for solution.

[0007] Step 2: Fit the error with (′, l′) as the independent variable and as the dependent variable using a linear model, solve for the linear model parameters, and simultaneously record the obtained linear error magnitude as

[0008] Step 3: Subtract the linear error from the original error Fit the error with s′ as the independent variable and the remaining part as the dependent variable using a segmented model, i.e., a segmented polynomial, solve for the segmented model parameters, and simultaneously record the obtained segmented error magnitude as

[0009] Step 4: Subtract the linear error part and the segmented term error part from the original error Fit the error with (s ′ , ′) as the independent variable and the remaining part as the dependent variable using a periodic trigonometric series model, solve for the trigonometric series model parameters, and simultaneously record the obtained trigonometric series error magnitude as

[0010] Step 5: Calculate the coordinate correction value for each point of the original multi-spectral image using the sum of the errors of the above linear model, segmented model, and trigonometric series model, and correct the position of the image point to obtain a multi-spectral image without errors.

[0011] Step 6: Since the CCD placement error and lens distortion error can remain unchanged for a long time, the piecewise term parameters are theoretically reusable for a period of time. Optionally, for satellite images of the same orbit, there is no need to re-solve the model parameters. The piecewise model parameters of the image with dense matching points can be directly used to preliminarily correct the images of the same orbit, improving the image fusion quality in cases where clouds, water, etc. are not conducive to matching.

[0012] Furthermore, the linear model in Step 2 is solved as follows: The coordinate differences between p′=(s′, l′) and p″=(s″, l″) are used as errors to solve the linear term, and the calculation formula is as follows:

[0013]

[0014] where is the linear correction value to be fitted in the column direction of the image, is the linear correction value to be fitted in the row direction of the image. The linear model parameters are solved according to the following formula:

[0015]

[0016] where A1, A2, …, A6 are the linear model parameters.

[0017] Furthermore, the piecewise model in Step 3 is solved as follows: The magnitude of the linear error obtained in Step 2 is Subtract the linear term model part from the original error The remaining part is used to solve the piecewise term parameters:

[0018]

[0019] where is the piecewise term correction value to be fitted in the column direction of the image, is the piecewise term correction value to be fitted in the row direction of the image. The piecewise model parameters are solved according to the following formula:

[0020]

[0021] where CCD1, CCD2, …, CCD n is the CCD segmentation interval of the image, and a0, a1, ……, f3, f4 are the piecewise model parameters.

[0022] Furthermore, the trigonometric series model in Step 4 is solved as follows: The magnitude of the piecewise error solved in Step 3 is Then subtract the piecewise term model part from the above error from the above error The remaining part is used for the analytical solution of the periodic trigonometric series:

[0023]

[0024] Where J s (s′, l′) is the correction value of the segmented term to be fitted in the image column direction, and J l (s′, l′) is the correction value of the segmented term to be fitted in the row direction. Solve the trigonometric series model parameters according to the following formula:

[0025]

[0026] Where B1 and B2 are the coefficients of the image column number s′, and T i 、S i are the amplitudes of the i-th trigonometric function model, p i 、q i are the frequency values of the i-th trigonometric function model, ψ i is the phase value of the i-th trigonometric function model. The error size of the trigonometric series obtained by solving in step 4 is

[0027] Furthermore, calculate the coordinate correction value for each point of the original multi-spectral image using the sum of the errors of the above linear model, segmented model, and trigonometric series model And correct the position of the image point to obtain a multi-spectral image without errors.

[0028] Optionally, for satellite images in the same orbit, there is no need to re-solve the model parameters. The segmented term parameters of the image with dense matching points can be directly used to preliminarily correct the images in the same orbit, improving the image fusion quality in cases where clouds, water, etc. are not conducive to matching.

[0029] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0030] ①At present, when commonly used commercial software performs fusion error compensation on first-level image products (i.e., satellite images expressing geographic information through Rational Polynomial Coefficient (RPC)), the small facet method is generally adopted. However, the fusion registration error compensation model based on small facet correction has too many parameters and requires uniformly distributed matching points to be stably solved, which not only limits the operation efficiency but also makes it difficult to handle situations such as the misalignment of the panchromatic and multispectral camera sub-array CCDs of Gaofen-7 in the rear view. The present invention adopts a panchromatic-multispectral mapping error compensation model, which includes a linear term for compensating linear errors, a piecewise quartic polynomial for compensating the placement error of the sub-array CCD and lens distortion, and a trigonometric series term for compensating periodic errors, and can compensate high-resolution fusion registration errors with high precision.

[0031] ②Since the placement error of the CCD and the lens distortion error can remain unchanged for a long time, the piecewise term parameters theoretically have reusability within a certain period. The present invention proposes a method of "reusing piecewise terms" to provide a reference for the efficient and high-precision production of fused images. Brief Description of the Drawings

[0032] Figure 1 : Flow chart of the method of the present invention; Detailed Embodiments

[0033] To facilitate the understanding and implementation of the present invention by those of ordinary skill in the art, the present invention will be further described in detail below with reference to the drawings and embodiments. It should be understood that the embodiments described herein are only for the purpose of illustrating and explaining the present invention and are not intended to limit the present invention.

[0034] Step 1: Obtain the high-resolution image fusion registration error

[0035] High-precision dense matching points of panchromatic-multispectral images are obtained through a high-precision dense matching method. Define the matching points between panchromatic / multispectral as {(p, p′)}. The image point p = (s, l) on the panchromatic image is projected onto the multispectral image plane to obtain the image point p″ = (s″, l″) through the "back-projection - reprojection" process (i.e., with the assistance of elevation data, the panchromatic image RPC parameter model is used to back-project to the object space, and then projected to its multispectral image space through the multispectral image RPC parameter model), and the coordinate difference between p′ = (s′, l′) and p″ = (s″, l″) is solved as the error.

[0036] Step 2: Solve the parameters of the linear fitting model

[0037] Use a linear model to fit the error with (′, l′) as the independent variable and as the dependent variable. The calculation formula for the dependent variable error is shown in formula (1):

[0038]

[0039]

[0040] where is the linear correction value to be fitted in the image column direction, is the linear correction value to be fitted in the image row direction. Solve for the linear model parameters according to the following formula:

[0041]

[0042] where A1, A2, …, A6 are the linear model parameters.

[0043] Step 3: Solve for the piecewise fitting model parameters

[0044] Define the magnitude of the linear error obtained in Step 2 as Subtract the linear term model part from the original error, and the remaining part is used to solve for the piecewise term parameters. The calculation formula for the remaining part of the error is shown in Formula (3):

[0045]

[0046] where is the piecewise term correction value to be fitted in the image column direction, is the piecewise term correction value to be fitted in the row direction. Solve for the piecewise model parameters according to the following formula:

[0047]

[0048] where CCD1, CCD2, …, CCD n is the CCD piecewise interval of the image, and a0, a1, ……, f3, f4 are the piecewise model parameters.

[0049] Step 4: Solve for the trigonometric series model parameters

[0050] Define the magnitude of the piecewise error obtained in Step 3 as Then subtract the piecewise error part from the error in Step 3, and the remaining part is used for the analysis and solution of the periodic trigonometric series. The calculation formula for the remaining part of the error is shown in Formula (5):

[0051]

[0052] where J s (s′, l′) is the piecewise term correction value to be fitted in the image column direction, J l (s′, l′) is the piecewise term correction value to be fitted in the row direction. Solve for the trigonometric series parameters according to the following formula:

[0053]

[0054] where B1 and B2 are the coefficients of the image column number s′, and T i , S i are the amplitudes of the i-th trigonometric function model, and p i , q i are the frequency values of the i-th trigonometric function model, ψ i is the phase value of the i-th trigonometric function model.

[0055] Step 5: Multispectral image resampling and correction

[0056] Calculate the coordinate correction value for each point of the original multispectral image using the sum of the errors of the above linear model, piecewise model, and trigonometric series model and correct the position of the image point to obtain a multispectral image without errors.

[0057] Step 6: Reuse of piecewise term parameters for images in the same orbit

[0058] For satellite images in the same orbit, there is no need to re-solve the model parameters. The piecewise term parameters of the image with dense matching points can be directly used to preliminarily correct the images in the same orbit, thereby improving the image fusion quality in cases where clouds, water, etc. are not conducive to matching.

[0059] It should be understood that the parts not detailed in this specification all belong to the prior art.

[0060] It should be understood that the above description of the preferred embodiment is relatively detailed, and it should not be considered as a limitation to the protection scope of the present invention. Under the inspiration of the present invention, those of ordinary skill in the art can also make substitutions or deformations without departing from the protection scope defined by the claims of the present invention, and all fall within the protection scope of the present invention. The scope of protection requested by the present invention shall be subject to the appended claims.

Claims

1. A method for improving the quality of high-resolution satellite fusion images by compensating mapping errors, characterized in that It includes the following steps: Step 1: Obtain high-precision dense matching points for panchromatic-multispectral images. Define the matching points between the panchromatic and multispectral images as \(\{(p, p')\}\). Project the image point \(p=(s, l)\) with column number \(s\) and row number \(l\) on the panchromatic image to the image plane of the multispectral image through the "back-projection - reprojection" process to obtain the image point \(p''=(s'', l'')\). Solve the coordinate differences between \(p'=(s', l')\) and \(p''=(s'', l'')\) as errors, where Step 2: Fit a linear model to the error with (s′, l′) as the independent variables and as the dependent variable, solve for the linear model parameters, and at the same time denote the obtained linear error as Step 3, subtract the linear model error from the original error, and use a piecewise model, i.e., piecewise polynomial fitting, to fit the error with s′ as the independent variable and the remaining part as the dependent variable, solve for the piecewise model parameters, and simultaneously denote the obtained piecewise error as ​ Step 4, subtract the linear error and the piecewise error from the original error, and fit the remaining part with a periodic trigonometric series model, where (s′, l′) is the independent variable and the remaining part is the dependent variable of the error, solve for the trigonometric series model parameters, and at the same time record the obtained trigonometric series error as ​ Step 5: Calculate the coordinate correction value for each point of the original multispectral image by summing the errors of the above linear model, piecewise model, and trigonometric series model And correct the position of the image points to obtain a multispectral image without errors.

2. A method for improving the quality of high-resolution satellite fusion images by compensating for mapping errors according to claim 1, characterized in that: The linear model in Step 2 is solved as follows: The coordinate differences between p′ = (s′, l′) and p″ = (s″, l″) are used as errors to solve the linear terms: where is the linear correction value to be fitted in the image column direction, the linear correction value to be fitted in the image row direction, and the linear model parameters are calculated according to the following formula: where A1, A2, …, A6 are the linear model parameters.

3. A method for improving the quality of high - resolution satellite fusion images compensating for mapping errors, according to claim 1, characterized in that: The piecewise model in Step 3 is solved as follows: The linear error obtained in Step 2 is Subtract the linear error part from the original error, and the remaining part is used to solve the piecewise term parameters: Among them is the correction value of the segmented item to be fitted in the image column direction, is the correction value of the segmented item to be fitted in the row direction. The parameters of the segmented model are solved according to the following formula: where CCD1, CCD2, …, CCD n are the CCD segmentation intervals of the image, and a0, a1, ……, f3, f4 are the segmentation model parameters.

4. A method for improving the quality of high - resolution satellite fusion images compensating for mapping errors according to claim 1, characterized in that: The trigonometric series model in Step 4 is solved as follows: The piecewise error solved in Step 3 is Then subtract the piecewise error part from the above error The remaining part is used for the analytical solution of the periodic trigonometric series: where J s (s′, l′) is the correction value of the segmented term to be fitted in the image column direction, J l (s′, l′) is the correction value of the segmented term to be fitted in the row direction, and the trigonometric series model parameters are solved according to the following formula: where B1 and B2 are the coefficients of the image column number s′, T i , S i is the amplitude of the i-th trigonometric function model, p i , q i is the frequency value of the i-th trigonometric function model, ψ i is the phase value of the i-th trigonometric function model, and the trigonometric series error obtained by solving in step 4 is 5. A method for improving the quality of high-resolution satellite fusion images compensating for mapping errors, as described in claim 1, characterized in that: It also includes Step 6. For satellite images of the same orbit, without re-solving the model parameters, the piecewise term parameters of the image with dense matching points are directly used to preliminarily correct the images of the same orbit, thereby improving the image fusion quality in the cases where clouds and water are not conducive to matching.

6. A method for improving the quality of high-resolution satellite fusion images by compensating for mapping errors according to claim 1, characterized in that: The back-projection and re-projection in Step 1 refer to, with the assistance of elevation data, using the panchromatic image RPC parameter model to back-project to the object space and then projecting to its multispectral image space through the multispectral image RPC parameter model.

Images