A color preservation method for high-resolution optical satellite panchromatic and multispectral image fusion

Abstract

The present invention provides a color preservation method for high-resolution optical satellite panchromatic multispectral image fusion, firstly, based on the panchromatic image, the full spectrum image is coarsely virtualized and registered, then the image after the coarse virtualization registration is finely registered, then the panchromatic image is blue-suppressed, and then the panchromatic image after blue-suppressed is fused with the multispectral image after fine registration, so that the fused image maintains the original spectral information. Since the method of the present invention suppresses the blue-suppressed problem during the fusion process, the fused image can more effectively maintain the original spectral information, which is conducive to the improvement of the final fusion effect and the maintenance of the spectral information.

Description

Technical Field

[0001] The invention relates to the field of remote sensing image processing, and in particular to a color preservation method for high-resolution optical satellite panchromatic and multispectral image fusion. Background Art

[0002] As one of the basic products in the ground processing of remote sensing images, the panchromatic multispectral image fusion product has important practical significance. It can retain both high spatial resolution and high spectral resolution. It has been used as one of the standard products in many land, resources, agriculture and other application departments. In recent years, with the advancement of the national high-resolution project, more and more satellites have been put into practical application, which has greatly promoted the fusion application of panchromatic multispectral images.

[0003] However, in actual applications, it is found that some fused images may appear blue (see Figure 2 ), especially in areas with dense vegetation. This is because the current panchromatic (PAN) spectrum of optical satellites is mostly in the range of 0.45-0.90μm. In multispectral imaging (MS), the blue (B, Blue) spectrum range is: 0.45-0.52μm, the green (G, Green) spectrum range is: 0.52-0.59μm, the red (R, Red) spectrum range is: 0.63-0.69μm, and the near-infrared (NIR, Near Infreared) spectrum range is: 0.76-0.89μm. This is the case for our country's Gaofen-2, Gaofen-6, Gaofen-7, Gaojing-1 and other satellites.

[0004] For some reasons, high-resolution optical satellites set the full color spectrum range to 0.45-0.90μm, which has a certain impact on our fusion. This is because the full color spectrum contains too many near-infrared components. The spectral response characteristics of the near-infrared component are significantly different from those of the visible light part, the most typical of which is the response to green vegetation. The near-infrared reflectivity of the green vegetation part will be significantly increased, which is reflected in the remote sensing image as the DN value of the near-infrared of green vegetation is significantly higher than that of the visible light part. Similarly, in high-resolution optical satellites, the DN value of the near-infrared of green vegetation in the panchromatic part is also significantly higher than that of the visible light part, which will result in a lower correlation coefficient between the full color spectrum segment and the visible spectrum segment of this type of image, and a phenomenon of blue light right shift during fusion, which ultimately causes the fusion result to be blue, especially in images of large areas of vegetation. This phenomenon is particularly obvious (see Figure 2 ). Summary of the invention

[0005] The present invention proposes a color preservation method for high-resolution optical satellite panchromatic multispectral image fusion, which suppresses the blue bias problem during the fusion process, so that the fused image can more effectively maintain the original spectral information, solving the technical problem of blue bias in the fusion result in the prior art.

[0006] In order to solve the above technical problems, the present invention provides a color preservation method for high-resolution optical satellite panchromatic multispectral image fusion, comprising:

[0007] S1: Perform virtual coarse registration of the full-spectrum image based on the panchromatic image to obtain a virtual full-spectrum image that is coarsely registered with the panchromatic image;

[0008] S2: Perform fine registration on the images after virtual rough registration;

[0009] S3: Suppress the blue bias of the panchromatic image by removing the NIR component in the panchromatic image;

[0010] S4: Fusing the panchromatic image after blue bias suppression with the multispectral image after precise registration, so that the fused image retains the original spectral information.

[0011] In one embodiment, step S1 includes:

[0012] S1.1: The object coordinates of the panchromatic image and the full-spectrum image are calculated using a rational function model. The object coordinates of the overlapping area are calculated based on the object coordinates of the respective image coordinates. The image coordinates of the respective image coordinates are calculated based on the object coordinates of the overlapping area. The image coordinates of the overlapping area are calculated based on the respective image coordinates.

[0013] S1.2: The pixel mapping relationship between the panchromatic image and the full-spectrum image is constructed based on the image coordinates of the overlapping area. Based on the pixel mapping relationship, the full-spectrum image is mapped to the coordinate system of the panchromatic image to obtain a virtual full-spectrum image that is roughly aligned with the panchromatic image.

[0014] In one embodiment, step S2 includes:

[0015] The small-surface differential correction method is adopted to complete the small-surface differential correction by matching the same-name points of the panchromatic image and the virtual multispectral image, and the precisely registered panchromatic image and multispectral image are obtained.

[0016] In one embodiment, step S3 includes removing the NIR component in the panchromatic image according to the matching of the multispectral image RGB and NIR, and the formula is as follows:

[0017]

[0018] In the above formula, Pan is the DN value of the original full-color image, Pan pseudoBand is the DN value of the full-color image after removing the NIR component. B , Band G , Band R , Band Nm They are the DN values ​​of the B band, G band, R band and NIR band of the multispectral image.

[0019] In one embodiment, step S4 uses a panchromatic spectrum decomposition method to fuse the panchromatic image after blue bias suppression and the multispectral image after precise registration.

[0020] In one embodiment, a panchromatic spectral decomposition method is used to fuse the panchromatic image after blue bias suppression and the multispectral image after precise registration, including:

[0021] Obtaining a linear relationship between the panchromatic image and the multispectral image after blue bias suppression;

[0022] According to the panchromatic image and multispectral image after blue bias suppression, the proportional coefficient, bias coefficient and residual matrix included in the linear relationship are fitted;

[0023] The fused image is obtained based on the proportional coefficient, bias coefficient and residual matrix obtained by fitting.

[0024] The above one or more technical solutions in the embodiments of the present application have at least one or more of the following technical effects:

[0025] The present invention provides a color preservation method for high-resolution optical satellite panchromatic multispectral image fusion, firstly, based on the panchromatic image, the full spectrum image is coarsely virtualized and registered, then the image after the coarse virtualization registration is finely registered, then the panchromatic image is blue-biased suppressed, and then the panchromatic image after blue-biasing suppression and the multispectral image after fine registration are fused, so that the fused image maintains the original spectral information. Since the method of the present invention suppresses the blue-biasing problem during the fusion process, the fused image can more effectively maintain the original spectral information, which is conducive to the improvement of the final fusion effect and the maintenance of the spectral information. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 The present invention is a flow chart of a color preservation method for high-resolution optical satellite panchromatic multispectral image fusion.

[0028] Figure 2 It is a comparison chart of the effects of the fusion product of the present invention before and after blue bias inhibition. DETAILED DESCRIPTION

[0029] In order to solve the problem of blue bias in high-resolution optical satellite panchromatic multispectral image fusion products, the present invention suppresses this blue bias phenomenon during the fusion process, which is beneficial to the improvement of the final fusion effect and the preservation of spectral information. The present invention proposes a color preservation method for high-resolution optical satellite panchromatic multispectral image fusion.

[0030] The technical scheme of the present invention includes: virtualized coarse registration of panchromatic images and full-spectral images, high-precision and accurate registration of panchromatic and multispectral images, suppression of blue outliers in panchromatic images, and rapid fusion of panchromatic and multispectral images.

[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] The embodiment of the present invention provides a color preservation method for high-resolution optical satellite panchromatic multispectral image fusion, comprising:

[0033] S1: Perform virtual coarse registration of the full-spectrum image based on the panchromatic image to obtain a virtual full-spectrum image that is coarsely registered with the panchromatic image;

[0034] S2: Perform fine registration on the images after virtual rough registration;

[0035] S3: Suppress the blue bias of the panchromatic image by removing the NIR component in the panchromatic image;

[0036] S4: Fusing the panchromatic image after blue bias suppression with the multispectral image after precise registration, so that the fused image retains the original spectral information.

[0037] Specifically, the resolution of panchromatic multispectral images is generally 4 times worse, so they need to be registered before fusion operations can be performed. The present invention first performs virtualized coarse registration of full-spectral images based on panchromatic images, which can be achieved based on rational function models, and performs fine registration on the obtained virtualized coarsely registered images. Due to satellite platform tremors, relative calibration errors of panchromatic multispectral images, etc., there are still certain deviations in the images after coarse registration, which may cause image distortion in subsequent fusion, so further fine registration is required.

[0038] The inventors of this application have learned through extensive research and practice that the main problem that causes the fusion of panchromatic and multispectral images to have a blue bias is that too many NIR components are mixed into the panchromatic image. Therefore, it is necessary to remove these NIR components from the panchromatic image to suppress the blue bias of the fused image. Rapidly fusing the panchromatic and multispectral images that have been suppressed for a blue bias can suppress the blue bias of the fused image, thereby maintaining the original spectral information, that is, maintaining the color. The specific fusion method can be selected according to the actual situation.

[0039] See also Figure 1 , which is an overall flow chart of a color preservation method for high-resolution optical satellite panchromatic and multispectral image fusion.

[0040] In one embodiment, step S1 includes:

[0041] S1.1: The object coordinates of the panchromatic image and the full-spectrum image are calculated using a rational function model. The object coordinates of the overlapping area are calculated based on the object coordinates of the respective image coordinates. The image coordinates of the respective image coordinates are calculated based on the object coordinates of the overlapping area. The image coordinates of the overlapping area are calculated based on the respective image coordinates.

[0042] S1.2: The pixel mapping relationship between the panchromatic image and the full-spectrum image is constructed based on the image coordinates of the overlapping area. Based on the pixel mapping relationship, the full-spectrum image is mapped to the coordinate system of the panchromatic image to obtain a virtual full-spectrum image that is roughly aligned with the panchromatic image.

[0043] Specifically, since there may be inconsistencies when panchromatic and multispectral images are cut, the overlapping area needs to be cropped out first. The calculation of the overlapping area is performed through the rational function model (RFM). RFM is essentially a rational polynomial model that can fit a strict geometric imaging model with high precision and complete the conversion between image coordinates and object coordinates.

[0044] In the specific implementation process, the object coordinates of the panchromatic image and the full-spectrum image are first calculated (the object coordinates of the panchromatic image and the object coordinates of the full-spectrum image). The given object coordinates include the geodetic latitude B, the geodetic longitude L, and the geodetic height H, which are expressed as (B, L, H), where the geodetic height is the average elevation. After obtaining the respective object coordinates, the object coordinates of the overlapping area can be calculated, and then the object coordinates of the overlapping area are calculated back to the respective image coordinates (the image coordinates of the panchromatic image and the image coordinates of the full-spectrum image), and finally the image coordinates of the overlapping area are obtained.

[0045] After calculating the image coordinates of the overlapping area, the pixel mapping relationship between the panchromatic image and the full-spectrum image can be constructed, and then the full-spectrum image can be mapped to the panchromatic image coordinate system to achieve coarse registration.

[0046] In one embodiment, step S2 includes:

[0047] The small-surface differential correction method is adopted to complete the small-surface differential correction by matching the same-name points of the panchromatic image and the virtual multispectral image, and the precisely registered panchromatic image and multispectral image are obtained.

[0048] Among them, after correction, accurately aligned panchromatic and virtual multispectral images can be obtained.

[0049] In one embodiment, step S3 includes removing the NIR component in the panchromatic image according to the matching of the multispectral image RGB and NIR, and the formula is as follows:

[0050]

[0051] In the above formula, Pan is the DN value of the original full-color image, Pan pseudo Band is the DN value of the full-color image after removing the NIR component. B , Band G , Band R , Band NIR They are the DN values ​​of the B band, G band, R band and NIR band of the multispectral image.

[0052] Specifically, it can be seen from the panchromatic and multispectral spectral ranges of high-resolution satellites (background technology) that the panchromatic image just basically covers the multispectral spectral range. The blue bias suppression method proposed in this embodiment is to remove the NIR component in the panchromatic image based on the matching of the multispectral image RGB and NIR. According to the energy ratio of each band of the multispectral image, the near-infrared component in the panchromatic image can be removed to obtain a panchromatic image with a spectral range consistent with RGB.

[0053] In one embodiment, step S4 uses a panchromatic spectrum decomposition method to fuse the panchromatic image after blue bias suppression and the multispectral image after precise registration.

[0054] Specifically, since the fusion algorithm with relatively high correlation requirement has a more obvious effect, the PanSharpern, full-color spectral decomposition (PSD) method is adopted in this embodiment.

[0055] In one embodiment, a panchromatic spectral decomposition method is used to fuse the panchromatic image after blue bias suppression and the multispectral image after precise registration, including:

[0056] Obtaining a linear relationship between the panchromatic image and the multispectral image after blue bias suppression;

[0057] According to the panchromatic image and multispectral image after blue bias suppression, the proportional coefficient, bias coefficient and residual matrix included in the linear relationship are fitted;

[0058] The fused image is obtained based on the proportional coefficient, bias coefficient and residual matrix obtained by fitting.

[0059] Specifically, this embodiment adopts a PSD-based image fusion method, which utilizes the strong correlation between the panchromatic image with blue bias suppression and the multispectral image, decomposes the panchromatic band DN value into the multispectral band DN value by linear transformation plus residual subtraction, and then uses the ratio transformation method to perform image fusion. In the ratio transformation method, the DN value ratio of the panchromatic image with blue bias suppression and the panchromatic image with blue bias suppression after degradation is equal to the ratio of the fused image to the multispectral upsampled image, that is:

[0060]

[0061] Among them, Pan HQ High-quality full-color images with blue suppression; Pan LQ It is a low-quality blue-suppressed panchromatic image after the high-quality panchromatic image is degraded; MS HQ It is an ideal high-quality multispectral image, that is, a fused image; MS LQ is a low-quality multispectral image, which can be obtained by sampling the original multispectral image; i, j, g are the serial numbers of row, column, and band respectively. The ratio transformation method repeatedly adds the same high-frequency information to different bands, which is an important reason for the spectral distortion of the fused image. The basis for the validity of formula (2) is that each pixel Pan(i, j) of the blue-suppressed panchromatic image is equal to the corresponding quality multispectral image pixel MS. (g) (i, j) have an unbiased linear relationship, namely:

[0062] Pan(i,j)=k (g) *MS (g) (i, j) (3)

[0063] Among them, k (g) is the proportional coefficient corresponding to the g-th band. Formula (3) is not completely consistent in practice. The blue-suppressed panchromatic image Pan and the multispectral image have a certain divergence. That is, the blue-suppressed panchromatic image Pan and the multispectral image MS (g) The mathematical relationship between them is described as:

[0064] Pan=k (g) *MS (g) +b (g) T+E (g) (4)

[0065] Among them, b (g)is the bias coefficient corresponding to the g-th band of the multispectral spectrum, k (g) is the proportional coefficient corresponding to the g-th band, T is a matrix with the same dimension as Pan and all its elements are 1, E (g) is the residual matrix corresponding to the g-th band of the multispectral image. Formula (4) is the linear relationship between the panchromatic image and the multispectral image after blue bias suppression.

[0066] Then, the low-resolution blue-suppressed panchromatic image (i.e., the panchromatic image after blue-suppressed) is fitted with the multispectral image to obtain the values ​​of the scale coefficient, bias coefficient, and residual matrix, and these coefficients are used to obtain the fused high-resolution image, so that a better fused image can be obtained, that is, a fused image with good color preservation. The calculation formula is as follows:

[0067] fus (g) =(Pan-E (g) -b (g) T) / k (g) (5)

[0068] fus (g) is a fused image, and Pan is a full-color image after blue suppression.

[0069] Figure 2 This is a comparison chart of the effects before and after color preservation using the fusion method of the present invention, where part A is the original image, part B is the image after color preservation, A1 and B1 are the full images before and after color preservation respectively, and A2, B2, A3, B3 are the partial enlarged images before and after color preservation respectively.

[0070] from Figure 2 It can be clearly seen that the original blue bias problem of large vegetation areas has been significantly corrected, which is closer to the real spectral performance, and can provide strong support for subsequent interpretation and processing. In order to better demonstrate the effect of the present invention, please refer to the attached drawings of the actual examination reference materials.

[0071] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A color preservation method for high-resolution optical satellite panchromatic multispectral image fusion, characterized in that: include: S1: Perform virtual coarse registration of the full-spectrum image based on the panchromatic image to obtain a virtual full-spectrum image that is coarsely registered with the panchromatic image; S2: Perform fine registration on the images after virtual rough registration; S3: Suppress the blue bias of the panchromatic image by removing the NIR component in the panchromatic image; S4: Fusing the panchromatic image after blue bias suppression with the multispectral image after precise registration, so that the fused image maintains the original spectral information; Among them, step S4 adopts the full-color spectrum decomposition method to fuse the full-color image after blue suppression and the multi-spectral image after precise registration, which specifically includes: Obtaining a linear relationship between the panchromatic image and the multispectral image after blue bias suppression; According to the panchromatic image and multispectral image after blue bias suppression, the proportional coefficient, bias coefficient and residual matrix included in the linear relationship are fitted; The fused image is obtained based on the proportional coefficient, bias coefficient and residual matrix obtained by fitting.

2. The color preservation method for high-resolution optical satellite panchromatic and multispectral image fusion according to claim 1, characterized in that: Step S1 includes: S1.1: The object coordinates of the panchromatic image and the full-spectrum image are calculated using a rational function model. The object coordinates of the overlapping area are calculated based on the object coordinates of the respective image coordinates. The image coordinates of the respective image coordinates are calculated based on the object coordinates of the overlapping area. The image coordinates of the overlapping area are calculated based on the respective image coordinates. S1.2: The pixel mapping relationship between the panchromatic image and the full-spectrum image is constructed based on the image coordinates of the overlapping area. Based on the pixel mapping relationship, the full-spectrum image is mapped to the coordinate system of the panchromatic image to obtain a virtual full-spectrum image that is roughly aligned with the panchromatic image.

3. The color preservation method for high-resolution optical satellite panchromatic and multispectral image fusion according to claim 1, characterized in that: Step S2 includes: The small-surface differential correction method is adopted to complete the small-surface differential correction by matching the same-name points of the panchromatic image and the virtual multispectral image, and the precisely registered panchromatic image and multispectral image are obtained.

4. The color preservation method for high-resolution optical satellite panchromatic and multispectral image fusion according to claim 1, characterized in that: Step S3 includes removing the NIR component in the panchromatic image according to the matching of the multispectral image RGB and NIR, and the formula is as follows: In the above formula, Pan is the DN value of the original full-color image, Pan pseudo Band is the DN value of the full-color image after removing the NIR component. B ,Band G ,Band R ,Band NIR They are the DN values ​​of the B band, G band, R band and NIR band of the multispectral image.

Images