Remote sensing image fusion method and system for improving image color naturalness

Abstract

The invention provides a remote sensing image fusion method and system for improving image color naturalness, and the method comprises the steps: carrying out the geometric registration of a multispectral image and a panchromatic image, and resampling the multispectral image to the same size as the panchromatic image; averaging B, G, R and NIR wave bands of the multispectral image to obtain a simulated low-resolution panchromatic image; enhancing the green wave band of the multispectral image by adopting a vegetation enhancement method, and replacing the original green wave band; taking the simulated low-resolution panchromatic image as a first component of GS transformation, and carrying out GS positive transformation on each wave band of the multispectral image in sequence; performing band matching on the panchromatic image to obtain an adjusted panchromatic image; and replacing the GS transformation first component with the adjusted panchromatic image, and carrying out GS inverse transformation to obtain a high-resolution multispectral image. According to the method, the vegetation is enhanced by using the near-infrared band, the color naturalness of the image is effectively improved while the spatial resolution of the image is improved, and the visual effect of the image is improved.

Description

Technical Field

[0001] The present invention belongs to the field of natural resource information science, and in particular relates to a remote sensing image fusion method and system for improving the naturalness of image colors. Background Art

[0002] In recent years, with the continuous development of remote sensing technology, especially the continuous strengthening of domestic high-resolution satellite remote sensing data collection capabilities, the demand for remote sensing data in various industries has been increasing. How to use various types of data efficiently has always been a major issue in remote sensing applications. As an effective means of image processing, remote sensing image fusion technology can obtain fused images with both high geometric resolution and rich multispectral information, which can greatly improve the spatial resolution characteristics of multispectral images and has been continuously developed in recent years. So far, common remote sensing image fusion methods include: Intensity Hue Saturation (HIS) transformation method, Brovey transformation method, Principal Component Analysis (PCA) method, Hyperspherical Color Space (HCS) transformation method, High Pass Filter (HPF) method, GS (Gram-Schmidt) transformation method, Pansharp method, Wavelet Transform (WT) method and Contourlet transformation method. These fusion methods have achieved good results in improving the spatial resolution of multispectral images, but they differ in preserving spectral information. Among them, IHS transform, Brovey transform, PCA method, etc. have certain spectral distortion; the combination of color information and spatial features of wavelet transform and Contourlet transform methods is not natural; the HCS transform method, HPF method, GS transform method and Pansharp method have better fusion effects.

[0003] Although some existing fusion methods can better maintain the original spectral characteristics of remote sensing images, due to the influence of the atmosphere on the multispectral imaging detector and its own spectral response function, the "true color" image synthesized by the R, G, and B bands has poor visual effect, especially it cannot reflect vegetation information well. Many remote sensing images cannot obtain the ideal true color effect after atmospheric correction or other correction methods, and often need to be enhanced in the later stage to improve the visual effect and enhance the recognition rate of information. Summary of the invention

[0004] In order to improve the color naturalness of images during image fusion, the present invention proposes a remote sensing image fusion method and system for improving the color naturalness of images, which not only ensures that the fusion result has high clarity, but also improves the color naturalness of the image and improves the visual effect of the image. The remote sensing image fusion method comprises: A multispectral image and a panchromatic image are acquired, the multispectral image is geometrically registered with the panchromatic image, and the multispectral image is resampled to the same size as the panchromatic image.

[0005] The four bands of blue, green, red, and near infrared (B, G, R, NIR) of the multispectral image are averaged to obtain a simulated low-resolution full-color image.

[0006] The green band of the multispectral image is enhanced by using a vegetation enhancement method to obtain an enhanced green band, and the enhanced green band is used to replace the green band. The vegetation enhancement method is: , In the formula, G' is the enhanced green band, G is the green band, and N' is the near-infrared band matched with the green band. a is the vegetation enhancement coefficient, 0≤ a ≤1.

[0007] The simulated low-resolution panchromatic image is used as the first component of the GS transformation, and the GS (Gram-Schmidt) forward transformation is performed on each band of the multispectral image in turn.

[0008] The panchromatic image is subjected to band matching with reference to the first component of the GS transformation to obtain an adjusted panchromatic image.

[0009] The adjusted full-color image is used to replace the first component of the GS transformation, and then the GSGS (Gram-Schmidt) transformation method is inversely transformed to obtain a high-resolution multispectral image.

[0010] Preferably, the step of using the simulated low-resolution panchromatic image as the first component of the GS transformation and sequentially performing a GS forward transformation on each band of the multispectral image specifically includes: According to the following formula, the GS positive transformation is performed on each band of the multispectral image in turn, where the Tth GS component is constructed by the previous T-1 GS components: , In the formula, GS T is the Tth band of the current GS transformation, is the multispectral band involved in the transformation, is the mean value of the T band, parameter for: , in, Band B T and Band GS i The covariance of For band GS i The variance of .

[0011] Preferably, the GS inverse transformation is specifically: , In the formula, GS T is the Tth band of the current GS transformation, is the multispectral band involved in the transformation, is the mean value of the T band, parameter for: , in, Band B T and Band GS i The covariance of For band GS i The variance of .

[0012] Preferably, the a is the vegetation enhancement coefficient, and its value is determined by a fixed coefficient method.

[0013] Preferably, a is the vegetation enhancement coefficient, and its value is obtained by using the dynamic coefficient method. a The value of varies according to the change of the pixel, and the normalized vegetation index of the pixel is assigned in sections according to the following formula: , Among them, NDVI is the normalized difference vegetation index.

[0014] The present invention also provides a remote sensing image fusion system for improving the naturalness of image color, the system comprising: The panchromatic multispectral registration module is used to obtain a multispectral image and a panchromatic image, geometrically register the multispectral image with the panchromatic image, and resample the multispectral image to the same size as the panchromatic image.

[0015] The green band processing module averages the four bands of B, G, R, and NIR of the multispectral image to obtain a simulated low-resolution full-color image, and uses a vegetation enhancement method to enhance the green band of the multispectral image to obtain an enhanced green band, and replaces the green band with the enhanced green band. The vegetation enhancement method is: , In the formula, G' is the enhanced green band, G is the green band, and N' is the near-infrared band matched with the green band. ais the vegetation enhancement coefficient, 0≤ a ≤1.

[0016] The GS forward transformation module uses the simulated low-resolution full-color image as the first component of the GS transformation and sequentially performs GS forward transformation on each band of the multispectral image.

[0017] The band matching module performs band matching on the panchromatic image with reference to the first component of the GS transformation to obtain an adjusted panchromatic image.

[0018] The GS inverse transformation module uses the adjusted full-color image to replace the first component of the GS transformation, and then performs a GS inverse transformation to obtain a high-resolution multispectral image.

[0019] The present invention also provides a computer device, comprising a processor and a memory for storing a program executable by the processor. When the processor executes the program stored in the memory, the above-mentioned remote sensing image fusion method can be implemented.

[0020] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the remote sensing image fusion method described above can be implemented.

[0021] Starting from improving the color naturalness of images, the present invention proposes a remote sensing image fusion method and system for improving the color naturalness of images. The method uses the near-infrared band to enhance vegetation, effectively improving the color naturalness of images while improving the spatial resolution of images, and improving the visual effects of images. The specific effects are: (1) Improving color naturalness: By improving the color performance of vegetation areas, it makes them more consistent with the visual perception of the human eye.

[0022] (2) Improve spatial resolution: Combined with the GS transform method, the spatial details of multispectral images can be significantly enhanced.

[0023] (3) Optimizing visual effects: The generated fused image is more realistic and easier to identify ground object information.

[0024] (4) Strong applicability: It is suitable for post-processing of high-resolution satellite images and can be widely used in environmental monitoring, agricultural remote sensing and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 Flow chart of the remote sensing image fusion method of the embodiment.

[0026] Figure 2 This is a vegetation fusion result diagram in the embodiment.

[0027] Figure 3 This is a diagram of the water body fusion result in the embodiment.

[0028] Figure 4 This is a diagram of the building fusion result in the embodiment.

[0029] Figure 5 This is a bare ground fusion result diagram in the embodiment.

[0030] Figure 6 FIG. 4 is a block diagram of a remote sensing image fusion system according to an embodiment. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0032] This embodiment provides a remote sensing image fusion method for improving the naturalness of image color. The method flow chart is as follows: Figure 1 As shown, the method comprises the following steps: The first step is to obtain a multispectral image and a panchromatic image, geometrically align the multispectral image with the panchromatic image, and resample the multispectral image to the same size as the panchromatic image.

[0033] In the second step, the four bands of B, G, R, and NIR of the multispectral image are averaged to obtain a simulated low-resolution full-color image.

[0034] The third step is to enhance the green band of the multispectral image using a vegetation enhancement method to obtain an enhanced green band, and replace the green band with the enhanced green band. The vegetation enhancement method is: , In the formula, G' is the enhanced green band, G is the green band, and N' is the near-infrared band matched with the green band. a is the vegetation enhancement coefficient, 0≤ a ≤1, there are two ways to determine its value: (1) Fixed coefficient method, that is, a is a constant value; (2) Dynamic coefficient method, that is, the value of a changes according to the change of the pixel. This paper assigns values ​​in sections according to the NDVI value of the pixel as follows: .

[0035] Due to the influence of the atmosphere and the sensor itself, visible light remote sensing images often cannot reflect vegetation information well. Vegetation-covered areas are generally dark green, which is not conducive to the expression of natural colors in the image. Considering that vegetation has strong reflective characteristics in the near-infrared band, it is an ideal method to enhance the vegetation area using the near-infrared band in this step.

[0036] The fourth step is to use the simulated low-resolution full-color image as the first component of the GS transformation, and perform GS positive transformation on each band of the multispectral image in turn. Specifically, GS positive transformation is performed on each band of the multispectral image in turn according to the following formula, wherein the Tth GS component is constructed from the previous T-1 GS components: , In the formula, GS T is the Tth band of the current GS transformation, is the multispectral band involved in the transformation, is the mean value of the T band, parameter for: , in, Band B T and Band GS i The covariance of For band GS i The variance of .

[0037] The fifth step is to perform band matching on the panchromatic image with reference to the first component of GS transformation to obtain an adjusted panchromatic image.

[0038] The band matching mentioned in this step and the vegetation enhancement method in the third step above is calculated as follows (assuming the band and Matched ): , In the formula, is the mean value of the T band, calculated as follows: , is the standard deviation of the T band, calculated as follows: , in is the pixel value of the T band, C and R are the number of columns and rows of the image respectively.

[0039] Step 6: Use the adjusted full-color image to replace the first component of the GS transformation, and then perform an inverse GS transformation to obtain a high-resolution multispectral image. The inverse transformation formula is as follows: .

[0040] In order to verify the above method, this embodiment uses different vegetation enhancement methods to conduct comparative experiments, and the experimental results and analysis are as follows: A GF-6 satellite remote sensing image of Shaoyang, Hunan Province was selected as experimental data. The image covers various typical landforms such as vegetation, water bodies, buildings, and bare land. The GF-6 satellite is equipped with a 2-meter panchromatic and 8-meter multispectral camera, with four bands of blue, green, red, and near-infrared. The data quantization level is 12 bits, which can obtain good spectral information of landforms. This embodiment conducts conventional GS fusion, fixed coefficient vegetation enhancement GS fusion (taking a=0.3), and dynamic coefficient vegetation enhancement GS fusion experiments respectively. The fusion result is displayed by 1% linear stretching by selecting the true color band combination in the ENVI software, and the fusion results of typical landforms (vegetation, water bodies, buildings, and bare land) are intercepted for comparison, as shown in the figure. Figure 2-Figure 5 As shown, Figure 2-Figure 5 (a) is the original multispectral image, (b) is the conventional GS fusion image, (c) is the fixed coefficient method vegetation enhanced GS fusion image, and (d) is the dynamic coefficient method vegetation enhanced GS fusion image.

[0041] from Figure 2-Figure 5 From the visual effect, the images of the three fusion methods have significantly improved in spatial resolution, and the texture information of the images has become richer. In terms of color, the image color of the conventional GS fusion method is basically consistent with the original multispectral image in various types of objects, showing obvious dark green characteristics in the vegetation area; the images of the two vegetation enhancement GS fusion methods effectively improve the display characteristics of vegetation, making the vegetation appear more emerald, the water color deepens, and the color of the building area is reddish, which is more in line with the color perception of the human body. There is no obvious difference in the image color of the three fusion methods in the bare land area. Carefully comparing the vegetation enhancement GS fusion images of the fixed coefficient method and the dynamic coefficient method, it can be found that the layering of the vegetation color of the fixed coefficient method fusion image is not as good as that of the dynamic coefficient method, and its water color is blue-red, which is not as natural as the dynamic coefficient method.

[0042] In order to further quantitatively evaluate the impact of different fusion methods on spatial detail information and spectral information, this embodiment selects four indicators, namely information entropy, average gradient, deviation and distortion, for quantitative evaluation, and the results are shown in Table 1. Among them, information entropy reflects the spatial detail information of the image. The larger the information entropy value, the richer the information contained in the image and the better the fusion quality; the average gradient measures the clarity of the image and reflects the texture change of the image before and after fusion. The larger the average gradient, the higher the clarity of the fusion result and the better the visual effect; the deviation reflects the relative deviation difference between the fusion result and the original image. The larger the deviation, the lower the information matching degree between the fusion result and the original image; the distortion reflects the spectral distortion degree of the fusion result. The smaller the distortion value, the smaller the spectral distortion degree of the fusion result compared with the original image.

[0043] Table 1 The quantitative evaluation results are as follows:

[0044] Note: The values ​​in the table are the average values ​​calculated for the three bands R, G and B respectively.

[0045] From the quantitative evaluation results, it can be seen that the information entropy and average gradient of the fused image are higher than those of the original multispectral image, which proves that the information content and clarity of the fused image have increased, and the information entropy and average gradient values ​​of the three fusion methods are not much different, indicating that the effects of the three fusion methods in spatial information enhancement are similar. In terms of deviation and distortion, the conventional GS fusion image is the smallest, the dynamic coefficient method vegetation enhanced GS fusion image is second, and the fixed coefficient method vegetation enhanced GS fusion image is the largest, indicating that vegetation enhancement during the fusion process will lead to an increase in spectral distortion, but it is precisely because of the change in the original spectral characteristics of the image that the color naturalness of the image is improved, which is more conducive to the expression of image information.

[0046] The above experimental results show that the remote sensing image fusion method proposed in this embodiment improves the spatial resolution of the image while effectively improving the color naturalness of the image and improving the visual effect of the image. It is a remote sensing image fusion method that can be used to improve image quality.

[0047] Another embodiment also provides a remote sensing image fusion system for improving the naturalness of image colors. The system block diagram is as follows: Figure 6 As shown, the system comprises: The panchromatic multispectral registration module is used to obtain a multispectral image and a panchromatic image, geometrically register the multispectral image with the panchromatic image, and resample the multispectral image to the same size as the panchromatic image.

[0048] The green band processing module averages the four bands of B, G, R, and NIR of the multispectral image to obtain a simulated low-resolution full-color image, and uses a vegetation enhancement method to enhance the green band of the multispectral image to obtain an enhanced green band, and replaces the green band with the enhanced green band. The vegetation enhancement method is: , In the formula, G' is the enhanced green band, G is the green band, and N' is the near-infrared band matched with the green band. a is the vegetation enhancement coefficient, 0≤ a ≤1.

[0049] The GS forward transformation module uses the simulated low-resolution full-color image as the first component of the GS transformation and sequentially performs GS forward transformation on each band of the multispectral image.

[0050] The band matching module performs band matching on the panchromatic image with reference to the first component of GS transformation to obtain an adjusted panchromatic image.

[0051] The GS inverse transformation module uses the adjusted full-color image to replace the first component of the GS transformation, and then performs a GS inverse transformation to obtain a high-resolution multispectral image.

[0052] Another embodiment further provides a computer device, including a processor and a memory for storing a program executable by the processor. When the processor executes the program stored in the memory, the remote sensing image fusion method in the above embodiment can be implemented.

[0053] Another embodiment further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the remote sensing image fusion method in the above embodiment can be implemented.

[0054] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described with reference to the preferred embodiments of the present invention, it should be understood by those skilled in the art that various changes may be made in form and details without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A remote sensing image fusion method for improving the naturalness of image color, characterized in that: The method comprises: Acquire a multispectral image and a panchromatic image, geometrically register the multispectral image with the panchromatic image, and resample the multispectral image to the same size as the panchromatic image; The blue, green, red and near-infrared bands of the multispectral image are averaged to obtain a simulated low-resolution full-color image; The green band of the multispectral image is enhanced by using a vegetation enhancement method to obtain an enhanced green band, and the enhanced green band is used to replace the green band. The vegetation enhancement method is: ， In the formula, G' is the enhanced green band, G is the green band, and N' is the near-infrared band matched with the green band. a is the vegetation enhancement coefficient, 0≤ a ≤1; Using the simulated low-resolution full-color image as the first component of the GS transformation, and sequentially performing a GS forward transformation on each band of the multispectral image; Perform band matching on the panchromatic image with reference to the first component of GS transformation to obtain an adjusted panchromatic image; The adjusted full-color image is used to replace the first component of the GS transformation, and then an inverse GS transformation is performed to obtain a high-resolution multispectral image.

2. The remote sensing image fusion method according to claim 1, characterized in that: The method of using the simulated low-resolution panchromatic image as the first component of the GS transformation and sequentially performing the GS forward transformation on each band of the multispectral image specifically includes: According to the following formula, the GS positive transformation is performed on each band of the multispectral image in turn, where the Tth GS component is constructed by the previous T-1 GS components: ， In the formula, GS T is the Tth band of the current GS transformation, is the multispectral band involved in the transformation, is the mean value of the T band, parameter for: ， in, Band B T and Band GS i The covariance of For band GS i The variance of .

3. The remote sensing image fusion method according to claim 1, characterized in that: The GS inverse transformation is specifically: ， in, Band B T and Band GS i The covariance of For band GS i The variance of .

4. The remote sensing image fusion method according to any one of claims 1 to 3, characterized in that: The a is the vegetation enhancement coefficient, and its value is obtained by using the fixed coefficient method.

5. The remote sensing image fusion method according to any one of claims 1 to 3, characterized in that: The a is the vegetation enhancement coefficient, and its value is obtained by the dynamic coefficient method. a The value of varies according to the change of the pixel, and the normalized vegetation index of the pixel is assigned in sections according to the following formula: ， Among them, NDVI is the normalized difference vegetation index.

6. A remote sensing image fusion system for improving the naturalness of image color, characterized in that: The system comprises: A panchromatic multispectral registration module, used for acquiring a multispectral image and a panchromatic image, geometrically registering the multispectral image with the panchromatic image, and resampling the multispectral image to the same size as the panchromatic image; The green band processing module averages the blue, green, red and near-infrared bands of the multispectral image to obtain a simulated low-resolution full-color image, and uses a vegetation enhancement method to enhance the green band of the multispectral image to obtain an enhanced green band, and replaces the green band with the enhanced green band. The vegetation enhancement method is: , In the formula, G' is the enhanced green band, G is the green band, and N' is the near-infrared band matched with the green band. a is the vegetation enhancement coefficient, 0≤ a ≤1; A GS forward transformation module uses the simulated low-resolution full-color image as the first component of the GS transformation and sequentially performs GS forward transformation on each band of the multispectral image; A band matching module performs band matching on the panchromatic image with reference to the first component of GS transformation to obtain an adjusted panchromatic image; The GS inverse transformation module uses the adjusted full-color image to replace the first component of the GS transformation, and then performs a GS inverse transformation to obtain a high-resolution multispectral image.

7. A computer device comprising a processor and a memory for storing a program executable by the processor, characterized in that: When the processor executes the program stored in the memory, the remote sensing image fusion method described in any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the remote sensing image fusion method described in any one of claims 1 to 5 is implemented.

Images