A fast fusion method for GF-6 high-resolution camera images and wide-format camera images

Abstract

The present invention provides a method for fast fusion of Gaofen-6 high-resolution camera images and wide-format camera images. First, virtual coarse registration of full-spectrum images is performed based on panchromatic images, and then the panchromatic and full-spectrum images after coarse registration are finely registered by using a small facet differential correction method to obtain finely registered panchromatic images and virtual full-spectrum images. Finally, the panchromatic bands are decomposed based on smoothing filtering to fuse the Gaofen-6 panchromatic and full-spectrum images. The method of the present invention can effectively fuse the full-spectrum band of Gaofen-6 with the visible light band, retain the spectral information of the full-spectrum band, and improve its spatial resolution.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing image processing, and in particular to a method for quickly fusing images of a GF-6 high-resolution camera with images of a wide-format camera. Background Art

[0002] In 2006, my country included the High-Resolution Earth Observation System Major Project (hereinafter referred to as GF Project) in the Outline of the Medium- and Long-Term Science and Technology Development Plan (2006-2020); in 2009, the implementation plan was reviewed and approved by the group meeting; in May 2010, after review and approval, the GF Project was fully launched. The implementation of the GF Project has greatly promoted the development of my country's high-resolution earth observation satellites and effectively guaranteed the major strategic needs of modern agriculture, disaster prevention and mitigation, resource investigation, environmental protection and security. GF-6 is a high-resolution wide-field imaging optical satellite launched with the support of the GF Project.

[0003] Gaofen-6 satellite adopts a sun-synchronous near-polar circular orbit with an orbital altitude of about 644.5km. Gaofen-6 satellite carries two cameras, a 2m / 8m high-resolution camera (referred to as "Gaofen Camera") and a 16m wide-field imaging camera (referred to as "Wide Camera"). The imaging width of the Gaofen camera at the sub-satellite point is better than 90km, and the imaging width of the wide-format camera at the sub-satellite point is better than 800km. The panchromatic (PAN) spectrum of the Gaofen camera is mostly in the range of 0.45-0.90μm, and the multispectral image (MS) has four bands, the blue (B, Blue) spectrum range is: 0.45-0.52μm, the green (G, Green) spectrum range is: 0.52-0.59μm, the red (R, Red) spectrum range is: 0.63-0.69μm, and the near infrared (NIR, Near Infreared) spectrum range is: 0.76-0.90μm. The wide-format camera has the ability to image in the entire spectrum, with a total of eight bands: the blue light (B, Blue) band range is: 0.45-0.52μm, the green light (G, Green) band range is: 0.52-0.59μm, the red light (R, Red) band range is: 0.63-0.69μm, the near infrared (NIR, Near Infreared) band range is: 0.76-0.89μm, the purple light (P, Purple) band range is: 0.40-0.45μm, the yellow light (Y, Yellow) band range is: 0.59-0.63μm, the red edge 1 (NR1, NearRed 1) band range is: 0.69-0.73μm, and the red edge 2 (NR2, NearRed 2) band range is: 0.73-0.77μm.

[0004] From the above parameters of the Gaofen-6 satellite, we can see that the wide-band camera of Gaofen-6 has the ability of full-spectrum imaging, which has 4 more bands than ordinary multispectral images. Its spectral resolution is also stronger than that of ordinary multispectral images. It is of great significance for the classification of land objects, especially agriculture, land and resources surveys, etc. However, its resolution is lower than that of the high-resolution camera. Therefore, it can be integrated with the imaging results of the high-resolution camera to improve its spatial resolution while maintaining the spectral information.

[0005] Traditional panchromatic multispectral image fusion methods are mainly used for panchromatic multispectral image fusion with a resolution difference of 4 times, and the fused bands are only 4. The spatial resolution of the GF-6 wide-format camera is 8 times different from that of the GF camera, and the fused spectral bands are also 8. Traditional fusion methods are difficult to adapt to the fusion requirements of the GF-6 high-resolution camera and the wide-format camera. Summary of the invention

[0006] The present invention proposes a method for quickly fusing images of the GF-6 high-resolution camera and images of a wide-format camera, which solves the technical problem of poor fusion effect in the prior art, and improves the spatial resolution of the full spectrum of the GF-6 while maintaining the spectral information of the full spectrum of the GF-6.

[0007] In order to solve the above technical problems, the present invention provides a method for quickly fusing images of a GF-6 high-resolution camera and images of a wide-format camera, comprising:

[0008] S1: Perform virtual coarse registration of the full-spectrum image based on the panchromatic image to obtain a virtual full-spectrum image that is coarsely registered with the panchromatic image, where the high-resolution camera image is the panchromatic image and the wide-format camera image is the full-spectrum image;

[0009] S2: Small-surface differential correction is completed by matching the same-name points of the panchromatic image and the virtual full-spectrum image to obtain the precisely aligned panchromatic image and the virtual full-spectrum image.

[0010] S3: Based on the panchromatic band decomposition with smoothing filtering, the precisely registered panchromatic image is fused with the virtual full-spectrum image.

[0011] In one embodiment, step S1 includes:

[0012] S1.1: The object coordinates of the panchromatic image and the full-spectrum image are calculated using a rational function model. The object coordinates of the overlapping area are calculated based on the object coordinates of the respective image coordinates. The image coordinates of the respective image coordinates are calculated based on the object coordinates of the overlapping area. The image coordinates of the overlapping area are calculated based on the respective image coordinates.

[0013] S1.2: The pixel mapping relationship between the panchromatic image and the full-spectrum image is constructed based on the image coordinates of the overlapping area. Based on the pixel mapping relationship, the full-spectrum image is mapped to the coordinate system of the panchromatic image to obtain a virtual full-spectrum image that is roughly aligned with the panchromatic image.

[0014] In one embodiment, step S3 includes:

[0015] S3.1: Equalize the precisely registered panchromatic image to make the distribution of the panchromatic image and the full-spectrum image consistent;

[0016] S3.2: Smoothing the equalized panchromatic image to make the equalized panchromatic image have the same clarity as the full-spectrum image at the scale of the full-spectrum image;

[0017] S3.3: Calculate spectral decomposition coefficients;

[0018] S3.4: Decompose the bands of the panchromatic image according to the spectral decomposition coefficients to obtain the final fused image.

[0019] In one embodiment, the equalization process in step S3.1 is implemented by the following formula:

[0020]

[0021] In the above formula, Pan Enchance represents the equalized panchromatic image, Pan represents the original panchromatic image, Mean pan Represents the mean of the panchromatic image, Std Pan represents the standard deviation of the panchromatic image, represents the mean value of the entire spectrum of band b, Represents the standard deviation of the entire spectrum of band b.

[0022] In one embodiment, step S3.3 calculates the spectral decomposition coefficient by the following formula:

[0023]

[0024] In the above formula, is the b-band decomposition coefficient on the full-spectrum image scale, MS (b) is the full spectrum image of band b, Pan EnchanceLR It is a balanced full-color image.

[0025] In one embodiment, step S3.4 is fused by the following formula:

[0026]

[0027] In the above formula, fus (b)For the fused image, According to the spectral decomposition coefficient Coefficients for upsampling to full color resolution.

[0028] The above one or more technical solutions in the embodiments of the present application have at least one or more of the following technical effects:

[0029] The present invention provides a method for quickly fusing images of a GF-6 high-resolution camera and an image of a wide-format camera. Firstly, a virtual rough registration is performed on a full-spectrum image based on a full-color image in an overlapping area. Then, a quasi-surface differential correction method is adopted to finely register the obtained virtual rough registration image. Then, according to the low-frequency information of the full-spectrum, the band of the full-color image is decomposed into a fused image whose spectral characteristics are consistent with the full-spectrum according to the low-frequency information of the full-spectrum. In this way, the high-frequency spatial information of the full-color band can be maintained, and the low-frequency spectral information of the full-spectrum can also be retained. Therefore, the full-spectrum of GF-6 is effectively fused with the visible light band. While retaining the spectral information of the full-spectrum, its spatial resolution is improved, thereby improving the effect of image fusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 The invention discloses a method for quickly fusing images of a GF-6 high-resolution camera and a wide-format camera.

[0032] Figure 2 This is the full-color and full-spectrum fusion effect diagram of the Gaofen-6 satellite of the present invention. DETAILED DESCRIPTION

[0033] In order to improve the spatial resolution of the full spectrum of GF-6 while maintaining the spectral information of the full spectrum of GF-6, the present invention proposes a method for fast fusion of images of GF-6 high-resolution camera and wide-format camera.

[0034] The technical scheme of the present invention includes: virtualization coarse registration of full-spectrum images based on panchromatic images, precise registration of GF-6 satellite panchromatic images and full-spectrum images; and fusion of GF-6 panchromatic and full-spectrum images based on smoothing filter panchromatic band decomposition.

[0035] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] The embodiment of the present invention provides a method for quickly fusing images of a GF-6 high-resolution camera and images of a wide-format camera, comprising:

[0037] S1: Perform virtual coarse registration of the full-spectrum image based on the panchromatic image to obtain a virtual full-spectrum image that is coarsely registered with the panchromatic image, where the high-resolution camera image is the panchromatic image and the wide-format camera image is the full-spectrum image;

[0038] S2: Small-surface differential correction is completed by matching the same-name points of the panchromatic image and the virtual full-spectrum image to obtain the precisely aligned panchromatic image and the virtual full-spectrum image.

[0039] S3: Based on the panchromatic band decomposition with smoothing filtering, the precisely registered panchromatic image is fused with the virtual full-spectrum image.

[0040] In the specific implementation process, step S1 can use the advantageous function model to achieve virtual coarse registration of the panchromatic image and the full-spectrum image.

[0041] Due to satellite platform tremors, relative calibration errors of panchromatic and full-spectrum images, etc., there are still certain deviations in the images after coarse registration, which may affect the subsequent fusion, so further fine registration is required. The fine registration in step S2 adopts the method of small-surface differential correction, and completes the small-surface differential correction by matching the same-name points of the panchromatic image and the virtual full-spectrum image. After correction, high-precision registered panchromatic and virtual full-spectrum images can be obtained.

[0042] Remote sensing images are generally composed of high-frequency spatial information and low-frequency spectral information. In GF-6, the high-frequency space of the panchromatic band is relatively rich, while the low-frequency information of the full spectrum is relatively rich. Therefore, step S3 decomposes the panchromatic band into a fused image with spectral characteristics consistent with the full spectrum according to the low-frequency information of the full spectrum. In this way, the high-frequency spatial information of the panchromatic band can be maintained, and the low-frequency spectral information of the full spectrum can also be retained.

[0043] See also Figure 1 , which is the overall idea of ​​the method for fast fusion of images from the GF-6 high-resolution camera and the wide-format camera.

[0044] In one embodiment, step S1 includes:

[0045] S1.1: The object coordinates of the panchromatic image and the full-spectrum image are calculated using a rational function model. The object coordinates of the overlapping area are calculated based on the object coordinates of the respective image coordinates. The image coordinates of the respective image coordinates are calculated based on the object coordinates of the overlapping area. The image coordinates of the overlapping area are calculated based on the respective image coordinates.

[0046] S1.2: The pixel mapping relationship between the panchromatic image and the full-spectrum image is constructed based on the image coordinates of the overlapping area. Based on the pixel mapping relationship, the full-spectrum image is mapped to the coordinate system of the panchromatic image to obtain a virtual full-spectrum image that is roughly aligned with the panchromatic image.

[0047] Specifically, since the imaging ranges of panchromatic and full-spectrum images are inconsistent, the imaging range of panchromatic images is 90 km, while the imaging range of full-spectrum images is 800 km. In general, the imaging range of full-spectrum images will include the imaging range of panchromatic images, so the overlapping area needs to be cropped out first. The calculation of the overlapping area is performed through the rational function model (RFM). RFM is essentially a rational polynomial model that can fit a rigorous geometric imaging model with high precision and complete the conversion between image coordinates and object coordinates.

[0048] In the specific implementation process, the object coordinates of the panchromatic image and the full-spectrum image are first calculated (the object coordinates of the panchromatic image and the object coordinates of the full-spectrum image). The given object coordinates include the geodetic latitude B, the geodetic longitude L, and the geodetic height H, which are expressed as (B, L, H), where the geodetic height is the average elevation. After obtaining their respective object coordinates, the object coordinates of the overlapping area can be calculated, and then the object coordinates of the overlapping area are calculated back to their respective image coordinates (the image coordinates of the panchromatic image and the image coordinates of the full-spectrum image), and finally the image coordinates of the overlapping area are obtained.

[0049] After calculating the image coordinates of the overlapping area, the pixel mapping relationship between the panchromatic image and the full-spectrum image can be constructed, and then the full-spectrum image can be mapped to the panchromatic image coordinate system to achieve coarse registration.

[0050] In one embodiment, step S3 includes:

[0051] S3.1: Equalize the precisely registered panchromatic image to make the distribution of the panchromatic image and the full-spectrum image consistent;

[0052] S3.2: Smoothing the equalized panchromatic image to make the equalized panchromatic image have the same clarity as the full-spectrum image at the scale of the full-spectrum image;

[0053] S3.3: Calculate spectral decomposition coefficients;

[0054] S3.4: Decompose the bands of the panchromatic image according to the spectral decomposition coefficients to obtain the final fused image.

[0055] Specifically, since the distribution of panchromatic and full-spectrum images is often not balanced, it is necessary to keep the distribution of panchromatic images consistent with that of full-spectrum images, and then perform equalization processing (enhancement processing) on ​​the precisely registered panchromatic images.

[0056] Step S3.2 removes high-frequency information of the panchromatic image based on smoothing filtering, so that the equalized panchromatic image can maintain the same clarity as the full-spectrum image in terms of scale. Specifically, the equalized full-spectrum image is smoothed and filtered to keep it consistent with the scale of the full-spectrum image. There are two ways to implement smoothing filtering:

[0057] The first method is spatial domain filtering based on Gaussian filter

[0058] The Gaussian filter has been proven in the prior art to be a filter that can effectively maintain clarity when the scale is reduced. The two-dimensional convolution function used for remote sensing image filtering is:

[0059]

[0060] In the above formula, G(x,y) is the corresponding value in the convolution template, σ is the Gaussian smoothing factor, and (x,y) is the pixel position corresponding to the convolution template. Referring to the relevant technology in the existing scale-invariant feature transform (SIFT), σ is set to 1.6, and Gaussian smoothing filtering is performed every time the scale is doubled.

[0061] In the specific implementation process, assuming that the size of the panchromatic band is 8k*8k and the size of the full spectrum band is 1k*1k, the panchromatic band is first subjected to a Gaussian filter, and then its scale is reduced to 4k*4k, a smoothing filter is performed, the scale is reduced to 2k*2k, a smoothing filter is performed, and finally the scale is reduced to 1k*1k to keep the same scale as the full spectrum image.

[0062] The second method is frequency domain smoothing filtering based on wavelet transform

[0063] Wavelet transform can transform the panchromatic image into the frequency domain, filter the high-frequency information in the frequency domain, and then inversely transform it into the spatial domain to obtain a smoothed image, eliminating the high-frequency information. At this time, the panchromatic image after smoothing is downsampled by 8 times to obtain a panchromatic image with the same size and clarity as the full spectrum. The formula of wavelet transform is as follows:

[0064]

[0065] Where a is the scale parameter and b is the translation parameter. In image processing, because the signal is discrete, it is necessary to discretize the scale parameter a and the translation parameter b, that is, the wavelet basis and f(t) is the image signal distribution function.

[0066] In one embodiment, the equalization process in step S3.1 is implemented by the following formula:

[0067]

[0068] In the above formula, Pan Enchance represents the equalized panchromatic image, Pan represents the original panchromatic image, Mean pan Represents the mean of the panchromatic image, Std Pan represents the standard deviation of the panchromatic image, represents the mean value of the entire spectrum of band b, Represents the standard deviation of the entire spectrum of band b.

[0069] It should be noted that Pan represents the original full-color image, which refers to the full-color image obtained after the equalization process. The original full-color image Pan refers to the full-color image obtained after the precise registration in step S2.

[0070] In one embodiment, step S3.3 calculates the spectral decomposition coefficient by the following formula:

[0071]

[0072] In the above formula, is the b-band decomposition coefficient on the full-spectrum image scale, MS (b) is the full spectrum image of band b, Pan EnchanceLR It is a balanced full-color image.

[0073] Specifically, the spectral decomposition coefficient is the panchromatic band decomposition coefficient. Through the calculation of steps S3.1-S3.2, a balanced panchromatic image with the same clarity as the full-spectrum image has been obtained on the scale of the full-spectrum image. At this time, the high-frequency spatial information of the panchromatic image has been filtered out, so the spectral decomposition coefficient can be calculated.

[0074] In one embodiment, step S3.4 is fused by the following formula:

[0075]

[0076] In the above formula, fus (b) For the fused image, is the spectral decomposition coefficient of band b Coefficients for upsampling to full color resolution.

[0077] Specifically, after obtaining the equalized panchromatic band and the panchromatic band decomposition coefficient Afterwards, it is upsampled to full color resolution to obtain At this point, the panchromatic band can be decomposed to obtain the final fused image.

[0078] The effect of the present invention on the fusion of the Gaofen-6 high-resolution camera and the wide-format camera can be seen in Figure 2 , where the A part of the coordinates is the original image of the wide camera, and the B part is the image after the high-resolution camera and the wide camera images are fused. A1 and B1 are the full images, and A2, B2, A3, and B3 are the local enlarged images.

[0079] from Figure 2 It can be clearly seen that the fusion effect of the full spectrum image of Gaofen-6 is good. Not only the spatial resolution is maintained well, but also the spectral resolution has a good fidelity effect. In order to better demonstrate the effect of the present invention, please refer to the attached drawings of the actual examination reference materials.

[0080] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for fast fusion of Gaofen-6 high-resolution camera images and wide-format camera images, characterized in that: include: S1: Perform virtual coarse registration of the full-spectrum image based on the panchromatic image to obtain a virtual full-spectrum image that is coarsely registered with the panchromatic image, where the high-resolution camera image is the panchromatic image and the wide-format camera image is the full-spectrum image; S2: Small-surface differential correction is completed by matching the same-name points of the panchromatic image and the virtual full-spectrum image to obtain the precisely aligned panchromatic image and the virtual full-spectrum image. S3: Based on the panchromatic band decomposition with smoothing filtering, the precisely registered panchromatic image is fused with the virtual full-spectrum image; Wherein, step S1 comprises: S1.1: The object coordinates of the panchromatic image and the full-spectrum image are calculated using a rational function model. The object coordinates of the overlapping area are calculated based on the object coordinates of the respective image coordinates. The image coordinates of the respective image coordinates are calculated based on the object coordinates of the overlapping area. The image coordinates of the overlapping area are calculated based on the respective image coordinates. S1.2: Construct the pixel mapping relationship between the panchromatic image and the full-spectrum image based on the image coordinates of the overlapping area, map the full-spectrum image to the coordinate system of the panchromatic image based on the pixel mapping relationship, and obtain a virtual full-spectrum image roughly aligned with the panchromatic image; Step S3 includes: S3.1: Equalize the precisely registered panchromatic image to make the distribution of the panchromatic image and the full-spectrum image consistent; S3.2: Smoothing the equalized panchromatic image to make the equalized panchromatic image have the same clarity as the full-spectrum image at the scale of the full-spectrum image; S3.3: Calculate spectral decomposition coefficients; S3.4: Decompose the bands of the panchromatic image according to the spectral decomposition coefficients to obtain the final fused image.

2. The method for rapid fusion of high-resolution camera images and wide-format camera images as claimed in claim 1, characterized in that: The equalization process in step S3.1 is implemented by the following formula: In the above formula, Pan EnchanceHR represents the equalized panchromatic image, Pan represents the original panchromatic image, and Mean pan Represents the mean of the panchromatic image, Std Pan represents the standard deviation of the panchromatic image, represents the mean value of the entire spectrum of band b, Represents the standard deviation of the entire spectrum of band b.

3. The method for rapid fusion of high-resolution camera images and wide-format camera images as claimed in claim 1, characterized in that: Step S3.3 calculates the spectral decomposition coefficient by the following formula: In the above formula, is the b-band decomposition coefficient on the full-spectrum image scale, MS (b) is the full spectrum image of the b-band, Pan EnchanceLR A balanced full-color image.

4. The method for rapid fusion of high-resolution camera images and wide-format camera images as claimed in claim 2, characterized in that: Step S3.4 is fused using the following formula: In the above formula, fus (b) is the fused image, is the spectral decomposition coefficient of band b Coefficients for upsampling to full color resolution.

Images