A method and system for image product fusion based on scale invariance principle

Abstract

The present invention provides an image product fusion method and system based on the scale invariance principle. Based on the scale invariance assumption of remote sensing images, the present invention obtains an image product fusion formula based on mean filtering according to the spatial detail differences and spectral detail differences between the image's low spatial resolution multispectral band M, high spatial resolution panchromatic band P, low spatial resolution panchromatic band I, and high spatial resolution multispectral band Mf. Aiming at the defect that the traditional Brovey fusion method is difficult to predict the fusion result, the present invention uses the panchromatic band P, the multispectral band M, and the intermediate band I to construct a pair of product fusion operators, transform the product fusion into a linear fusion form, and realize the predictability of the fusion result, so as to facilitate the selection of fusion parameters. The method of the present invention provides two modes of image product fusion: panchromatic band injected into multispectral band product fusion and multispectral band injected into panchromatic band.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image fusion enhancement, and more specifically, to an image product fusion method and system based on the scale invariance principle. Background Art

[0002] With the development of multi-platform, multi-sensor, multi-weather, multi-temporal and multi-resolution remote sensing technology, the types of remote sensing images available for use in various industries are becoming increasingly diverse. These rich and colorful remote sensing images not only provide a flexible object selection space for remote sensing image application research, but also pose challenges to image pre-application processing research such as remote sensing image selection, synthesis, correction and enhancement.

[0003] Different remote sensing image data have different basic image features such as spatial resolution, temporal resolution, and spectral resolution, and they have different application performance and potential in different application fields. Traditional remote sensing image processing focuses on enhancing the general image features of a single image type, such as color, texture, and hierarchy; remote sensing image fusion processing focuses on integrating the basic features of different images into a new remote sensing image, fully tapping its comprehensive application potential and improving its application performance. In the past two decades, as a new direction in remote sensing image processing, remote sensing image fusion technology has made great progress and achieved a series of new results.

[0004] From the perspective of fusion principle, remote sensing image fusion methods can be divided into three categories: fusion methods based on color space theory, information analysis principle and mathematical operation. Fusion methods based on color space theory apply the main color models RGB, CMYK, Lab, IHS, HSV, etc. to image fusion. Among them, the IHS model is a classic image fusion method (Te-ming TU et al., A new look at IHS-look image fusion methods, Information Fusion 2, 2001, 177-18). The color model components of this type of method are relatively independent and can be controlled separately, which can accurately describe the color characteristics (Li Lin et al., Comparison of ZY-3 satellite panchromatic and multispectral image fusion methods, Transactions of the Chinese Society of Agricultural Engineering, 30(16), 2014, 157-165). The advantage is that it can quickly and easily perform image fusion. The disadvantage is that although the fusion result obtained by simple component replacement well maintains the details of the panchromatic band image, it usually changes the spectral characteristics of the satellite remote sensing image before and after fusion. The fusion method based on the principle of information analysis is to apply information analysis techniques such as spatial filtering, principal component analysis (PCA), Fourier transform (FFT), wavelet transform, Gram-Schimdt transform, curvelet transform, contourlet transform, ridgelet transform, bandlet transform, wedgelet transform, beamlet transform to image fusion. This method injects the detail information of the panchromatic band into the multispectral image (Xiao Liang et al., Progress and Challenges of Multi-source Spatial-spectral Remote Sensing Image Fusion Methods, Journal of Image and Graphics, 25(05), 2020). The fused image has good spectral continuity, but the calculation is complex and usually loses some high-frequency information of the image. The fusion methods based on mathematical operations mainly include ratio method, difference method, weighted superposition, multiple amplification and four arithmetic mixed operation method. Among these methods, the classic methods are Brovey fusion method and CN fusion method.

[0005] Brovey fusion is a fusion method based on product transformation proposed by American scholar Brovey, also known as color normalization transformation fusion method (Vrabel J. Multispectral Imagery Band Sharpening Study. Photogrammetric Engineering & Remote Sensing, 1996, 62 (9): 1075-1083). It performs product operation on three multispectral bands and high spatial resolution panchromatic image, simplifies the image fusion operation coefficients, and retains the spectral information of the source image to a large extent. However, there are also cases where only three bands can be fused. CN fusion (color normalized transformation) is also called energy subdivision transformation. It uses high spatial resolution panchromatic band to enhance the low spatial resolution multispectral band of the input image. This method only fuses the corresponding input bands contained in the spectral range of the fused image band, and other input bands are directly output without fusion processing. Brovey fusion method and CN fusion method require that the spectral response range of the panchromatic band should be consistent or similar to that of the multispectral image, otherwise it will cause significant deviations in the local brightness of the fused image, change the spectral characteristics of the multispectral image before and after fusion, and cause spectral distortion. To this end, scholars have made various optimizations and improvements to Brovey, CN and other fusion methods. Some scholars have introduced adaptive optimization methods to adjust the weight parameters of each multispectral band and panchromatic band, and improved the Brovey fusion algorithm by weighted averaging. While retaining the spectral physical characteristics of the multispectral source image, it can also improve the spatial resolution characteristics of the image (Lin Zhilei et al., Research on ALI Image Fusion Algorithm Based on Improved Brovey Transform. Remote Sensing Technology and Applications, 2020, 35(4): 893-900). Some scholars have used the nonlinear spectral data mining characteristics of the kernel principal component transform (KPCA) algorithm to extract the three principal components KPC1, KPC2 and KPC3 with the largest amount of information in multispectral images; then the Brovey algorithm is used to perform normalized fusion operations on the three principal components and the panchromatic band, so that the spatial and spectral information in the fusion results are richer (Ke Hongxia et al., A remote sensing image fusion method based on KPCA and Brovey transform, Journal of Chongqing Jiaotong University (Natural Science Edition), 2019, 38(2)). Some scholars have introduced wavelet analysis methods to improve the shortcomings of Brovey transform, such as the fusion image being greatly affected by noise and the excessive retention of sporadic details in high-resolution images (Chen Sijin et al., Using wavelet analysis to improve Brovey remote sensing image fusion method, Journal of College of Surveying and Mapping, 2004, 21(2)).

[0006] In summary, the traditional Brovey spatial-spectral fusion method of remote sensing images has the disadvantages of large changes in color information, limited number of fused image bands, and difficult to predict fusion results. Summary of the invention

[0007] In view of the above problems, the purpose of the present invention is to provide an image product fusion method and system based on the principle of image scale invariance, aiming to break through the band limitation of traditional Brovey fusion and the unpredictable defects of fusion results, and significantly improve the image fusion result's ability to preserve the spectral information of the original multispectral image and the spatial detail information of the original panchromatic band image.

[0008] A first aspect of the present invention provides an image product fusion method based on the principle of image scale invariance, the method comprising:

[0009] Obtain satellite remote sensing images, which include high spatial resolution panchromatic band images P and low spatial resolution multispectral images, which include images of n bands, denoted as M 1 、M 2 ,……,M n ;

[0010] All images are spatially registered to make the geometrical spatial positions of the same ground object in the panchromatic band image P and the multispectral image consistent, and the multispectral image to be fused is resampled as a high spatial resolution image, and the multispectral image to be fused is recorded as M;

[0011] Count the mean, mean square error and correlation coefficient matrix of all multispectral images and panchromatic band images P;

[0012] Construct intermediate band I using low-resolution multispectral imagery:

[0013]

[0014] Wherein, l is the preset number of multispectral bands involved in constructing the intermediate band I, l≤n;

[0015] Select the panchromatic band injected into the multispectral band product fusion mode or the multispectral band injected into the panchromatic band product fusion mode, and construct the corresponding product fusion operator;

[0016] The space-spectrum fusion calculation is performed according to the product fusion operator, fusion mode and fusion parameters, and the fusion parameters include k I , k E , k I is the feature extraction coefficient, k E is the feature fusion coefficient.

[0017] Furthermore, the panchromatic band is injected into the multispectral band product fusion mode to construct a pair of product fusion operators P M and I M for:

[0018]

[0019] Where: μ P is the mean value of the panchromatic band image P, μ I The mean of the intermediate band I.

[0020] Furthermore, the specific method of injecting the panchromatic band into the multispectral band product fusion is:

[0021] Estimate the mean square error σ of the fused image Mf Mf for:

[0022]

[0023] Among them, σ M is the mean square error of the multispectral image M, r(M,P M ) is the multispectral image M and the product fusion operator P M The correlation coefficient, r(M,I M ) is the spectral image M and the product fusion operator I M The correlation coefficient, r(P M ,I M ) is the product fusion operator P M Sum-Product Fusion Operator I M The correlation coefficient of

[0024] Thus, the product fusion mode of the panchromatic band injected into the multispectral band is as follows (converted to linear fusion form):

[0025]

[0026] In the formula, μ M is the mean value of the multispectral image M, and They are respectively the product fusion operator P M ,I M The mean and mean square error of .

[0027] Furthermore, a multi-spectral band injection panchromatic band product fusion mode is adopted to construct a pair of product fusion operators M P and I P for:

[0028]

[0029] In the formula, μ M is the mean of the multispectral image M, μ IThe mean of the intermediate band I.

[0030] Furthermore, the specific method of multi-spectral band injection into panchromatic band product fusion is:

[0031] Estimate the mean square error σ of the fused image Mf Mf for:

[0032]

[0033] Among them, σ P is the mean square error of the panchromatic band image P, r(P,M P ) is the panchromatic band image P and the product fusion operator M P The correlation coefficient, r(P,I P ) is the panchromatic band image P and the product fusion operator I P The correlation coefficient, r(M P ,I P ) is the product fusion operator M P Sum-Product Fusion Operator I P The correlation coefficient of

[0034] Therefore, the product fusion mode of injecting multispectral images into panchromatic band images is as follows (converted to linear fusion form):

[0035]

[0036] In the formula, μ P is the mean value of the panchromatic band image P, and They are respectively the product fusion operator M P ,I P The mean and mean square error of .

[0037] Furthermore, the method also includes: selecting a plurality of multispectral images of different bands to perform spatial-spectral fusion with the panchromatic band image respectively.

[0038] Furthermore, the method further comprises: synthesizing a plurality of spatial-spectral fused images into one image and storing the image.

[0039] Furthermore, the feature extraction coefficient k is designed I And feature fusion coefficient k E It is used to ensure the fused image's ability to preserve the spatial geometric texture information of the original panchromatic image and the color information of the original multispectral image. The coefficient can be taken as 1 by default.

[0040] A second aspect of the present invention provides an image product fusion system based on the scale invariance principle, comprising a memory and a processor, wherein the memory comprises an image product fusion method program based on the scale invariance principle, and when the image product fusion method program based on the scale invariance principle is executed by the processor, the steps of the image product fusion method based on the scale invariance principle are implemented.

[0041] The third aspect of the present invention provides a computer-readable storage medium, which includes an image product fusion method program based on the scale invariance principle. When the image product fusion method program based on the scale invariance principle is executed by a processor, the steps of the image product fusion method based on the scale invariance principle are implemented.

[0042] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0043] The present invention aims at the shortcomings of Brovey space-spectral fusion of traditional remote sensing images, such as large changes in color information, limited number of fused image bands, and difficult prediction of fusion results, and provides an image product fusion method based on the scale invariance principle. Based on the scale invariance assumption of remote sensing images, the present invention obtains an image product fusion formula based on mean filtering according to the spatial detail differences and spectral detail differences among the image's low spatial resolution multispectral band M, high spatial resolution panchromatic band P, low spatial resolution panchromatic band I, and high spatial resolution multispectral band Mf.

[0044] Aiming at the defect that the traditional Brovey fusion method is difficult to predict the fusion result, the present invention uses the panchromatic band P, the multispectral band M and the intermediate band I to construct a pair of product fusion operators, transforms the product fusion into a linear fusion form, realizes the predictability of the fusion result, and facilitates the selection of fusion parameters.

[0045] The method of the present invention provides two modes: the panchromatic band is injected into the multispectral band product fusion and the multispectral band is injected into the panchromatic band product fusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 The figure is a flowchart of an image product fusion method based on the scale invariance principle in an embodiment.

[0047] Figure 2 This is a full-color band map (2-meter resolution).

[0048] Figure 3 This is a multispectral NRG composite image (8-meter resolution).

[0049] Figure 4 It is a multispectral RGB composite image (8-meter resolution).

[0050] Figure 5 This is an intermediate band I image (8-meter resolution).

[0051] Figure 6 It is a standard false color image synthesized by merging the near-infrared band (Nf), red band (Rf) and green band (Gf) according to the red, green and blue channels.

[0052] Figure 7 It is a true color image synthesized by combining the fused red band (Nf), green band (Gf) and blue band (Bf) according to the red, green and blue channels.

[0053] Figure 8 It is a standard false color image synthesized by merging the near-infrared band (Nf), red band (Rf) and green band (Gf) according to the red, green and blue channels.

[0054] Fig. 9 It is a true color image synthesized by combining the fused red band (Nf), green band (Gf) and blue band (Bf) according to the red, green and blue channels. DETAILED DESCRIPTION

[0055] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0056] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0057] Example 1

[0058] like Figure 1 As shown, the principle of the image product fusion method based on the scale invariance principle of the present invention is explained as follows:

[0059] 1. Scale invariance principle of spatial-spectral image fusion

[0060] Generally speaking, satellite remote sensing images have multiple low-resolution multispectral bands M 1 、M 2 ,……,M n And a panchromatic band P. It is assumed that corresponding to the high-resolution panchromatic band P there is a low-resolution panchromatic band I (intermediate band) corresponding to the spectral band, and corresponding to the low-resolution multispectral band M there is a high-resolution multispectral band Mf corresponding to the spectral band.

[0061] Let μ M , μ P, μ I , μ Mf are the average values ​​of low-resolution multispectral band, panchromatic band, low-resolution panchromatic band, and high-resolution multispectral image, respectively. M , σ P , σ I , σ Mf are the mean square error of low-resolution multispectral band, panchromatic band, low-resolution panchromatic band, and high-resolution multispectral image, respectively. r(P,I) is the correlation coefficient between P and I, and r(Mf,M) is the correlation coefficient between Mf and M.

[0062] assumed:

[0063] r(P,I)=1

[0064] r(Mf,M)=1

[0065] (P-μ P ) = k 1 (I-μ I )

[0066] (Mf-μ Mf ) = k 2 (M-μ M )

[0067] but:

[0068] P = k 1 (I-μ I )+μ P

[0069] Mf=k 2 (M-μ M )+μ Mf

[0070] like but:

[0071]

[0072] This means that the high resolution of the same spectral band is equal to the ratio of the low resolution image multiplied by the mean of the two.

[0073] 2. Image product fusion principle

[0074] Let M be the multispectral band of the remote sensing image, P be the high-resolution panchromatic band, I be the low-resolution panchromatic band of the corresponding spectrum of the panchromatic band, and Mf be the fused high-resolution multispectral band. The formula for remote sensing image product fusion is as follows:

[0075]

[0076] 3. Image product spatial-spectral fusion method based on scale invariance principle

[0077] Solution 1: Panchromatic band injection multispectral band product fusion mode

[0078] According to the scale invariance assumption of image space-spectrum fusion, we can get:

[0079]

[0080] Right now:

[0081]

[0082]

[0083] Construct a pair of product fusion operators as:

[0084]

[0085] In the formula, and They are respectively the product fusion operator P M ,I M The mean and mean square error of .

[0086] The mean square error σ of the fused image Mf can be estimated Mf for:

[0087]

[0088] Thus, the product fusion scheme of the panchromatic band injected into the multispectral band is as follows (converted to linear fusion form):

[0089]

[0090] In the formula, k I , k E are feature extraction coefficient and feature fusion coefficient respectively.

[0091] Solution 2: Multispectral band injection panchromatic band product fusion mode

[0092] Similarly, the product fusion scheme of multi-spectral injection into panchromatic band images is as follows:

[0093] Construct a pair of product fusion operators as:

[0094]

[0095] In the formula, and They are respectively the product fusion operator M P ,I P The mean and mean square error of .

[0096] The mean square error σ of the fused image Mf can be estimated Mf for:

[0097]

[0098] Therefore, the product fusion scheme of multispectral images injected into panchromatic band images is as follows (converted to linear fusion form):

[0099]

[0100] In the formula, k I , k E are feature extraction coefficient and feature fusion coefficient respectively.

[0101] In order to achieve the purpose of IHS spatial-spectral fusion of remote sensing images based on mean filtering, the present invention mainly uses ENVI remote sensing image processing software to achieve it, and is further described by a satellite remote sensing image with a panchromatic band (P), a blue band (B), a green band (G), a red band (R), and a near infrared (N).

[0102] 1. Input remote sensing image.

[0103] Open a domestically produced GF-6 remote sensing image with panchromatic band (P), blue band (B), green band (G), red band (R), and near infrared (N). Figure 2 , Figure 3 , Figure 4 They are full-color band map (2-meter resolution), multispectral NRG synthetic image map (8-meter resolution), and multispectral RGB synthetic image map (8-meter resolution) (the effect map of 0.1% stretching according to the default setting of envi).

[0104] 2. Select the multispectral green band (G), red band (R), and near infrared (N) to construct the intermediate band I. The calculation formula is: b = uint((1.0*b1+b2+b3) / 3), b1, b2, and b3 are the G, R, and N band images respectively. Figure 5 This is an intermediate band I image (8-meter resolution).

[0105] Solution 1: Panchromatic injection multispectral band product fusion

[0106] 3. Constructing the product fusion operator

[0107] Taking the panchromatic band injection multispectral band fusion mode as an example, the product fusion operator P is constructed. M and I M Specifically, the fusion operator P M The calculation method is: long((b1-964.517466) / b2*b3); Fusion operator I MThe calculation method is: long((b2-1064.514375) / b2*b3); where b1, b2, and b3 are the panchromatic band P, intermediate band I, and multispectral band M, respectively.

[0108] 4. Use ENVI software to convert the panchromatic band P, multispectral band M, and fusion operator P M and I M The fused source images with high spatial resolution are resampled to synthesize an image file, and then the image characteristic statistical parameters such as the mean μ and standard deviation σ of each band image are calculated. The mean and mean square error statistics of each band of the image are shown in Table 1, and the correlation coefficient statistics of each band of the image are shown in Table 2.

[0109] Table 1

[0110]

[0111]

[0112]

[0113] 5. Multispectral band fusion calculation

[0114] In this case, the fusion parameter k I , k E The value is selected as 1, and the product fusion calculation of the four bands of the multispectral spectrum is performed. The specific fusion algorithm is as follows:

[0115] (1) The fusion operation expression corresponding to the blue band is uint(0.970607*((b3-754.924488)+1*(0.748475*(b1-(-27.53764))-1*0.768249*(b2-(-30.692981))))+754.924488), where b1 is the fusion factor P B , b2 is fusion factor I B , b3 is the blue band B, and the fused blue band (Bf) image is obtained by calculation.

[0116] (2) The fusion operation expression corresponding to the green band is uint(0.981288*((b3-816.209962)+1*(0.96814*(b1-(-7.36993))-1*0.996452*(b2-(-8.509028))))+816.209962), where b1 is the fusion factor P G , b2 is fusion factor I G , b3 is the green band G, and the fused green band (Gf) image is obtained by calculation.

[0117] (3) The fusion operation expression corresponding to the red band is uint(0.989545*((b3-833.355385)+1*(1.247364*(b1-(21.558338))-1*1.294594*(b2-(23.758533))))+833.355385), where b1 is the fusion factor P R , b2 is fusion factor I R , b3 is the red band R, and the fused red band (Rf) image is obtained by calculation.

[0118] (4) The fusion operation expression corresponding to the near-infrared band is uint(0.96382*((b3-1544.975555)+1*(1.047852*(b1-(-13.958039))-1*1.11556*(b2-(-14.988883))))+1544.975555), where b1 is the fusion factor P N , b2 is fusion factor I N , b3 is the near-infrared band N, and the fused near-infrared band (Nf) image is obtained by calculation.

[0119] The fused near-infrared band (Nf), red band (Rf) and green band (Gf) are synthesized into a standard false color image according to the red, green and blue channels. Figure 6 The fused red band (Nf), green band (Gf) and blue band (Bf) are synthesized into a true color image according to the red, green and blue channels. Figure 7 .

[0120] Solution 2: Multispectral injection and panchromatic band product fusion

[0121] 3. Constructing the product fusion operator

[0122] Taking the panchromatic band injection multispectral band fusion mode as an example, the product fusion operator M is constructed. P and I P Specifically, the fusion operator B P The calculation formula is: long((b3-754.924488)*(1.0*b1 / b2)); G P The calculation formula is: long((b3-816.209962)*(1.0*b1 / b2)); R P The calculation formula is: long((b3-833.355385)*(1.0*b1 / b2)); N P The calculation formula is: long((b3-1544.975555)*(1.0*b1 / b2)); Fusion operator I PThe calculation formula is: long((b2-1064.514375)*(1.0*b1 / b2)); where b1, b2, and b3 are the panchromatic band P, intermediate band I, and multispectral band M, respectively.

[0123] 4. Use ENVI software to convert the panchromatic band P, multispectral band M, and fusion operator M P and I P The fused source images with high spatial resolution are resampled to synthesize an image file, and then the image characteristic statistical parameters such as the mean μ and standard deviation σ of each band image are calculated. The mean and mean square error statistics of each band of the image are shown in Table 3, and the correlation coefficient statistics of each band of the image are shown in Table 4.

[0124] Table 3

[0125] Image bands Mean / μ Mean square error / σ Panchromatic Band P 964.517466 394.742565 Intermediate Band I 1064.514375 377.015145 Blue Band B 754.924488 228.405575 Green Band G 816.209962 333.602392 Red Band R 833.355385 454.304795 Near infrared 1544.975555 515.779906 <![CDATA[Fusion factor B P > 0.770386 213.677926 <![CDATA[Fusion factor G P > 0.874416 311.179625 <![CDATA[Fusion factor R P > 0.561689 421.579784 <![CDATA[Fusion factor N P > 0.34903 477.119761 <![CDATA[Fusion factor I P > 0.646806 351.035897

[0126]

[0127] 5. Multispectral band fusion calculation

[0128] In this case, the fusion parameter k I , k E The value is selected as 1, and the product fusion calculation of the four bands of the multispectral spectrum is performed. The specific fusion algorithm is as follows:

[0129] (1) The fusion operation expression corresponding to the blue band is uint(0.561351*((b3-964.517466)+1*(1.847372*(b1-(0.770386))-1*1.124508*(b2-(0.646806))))+754.924488), where b1 is the fusion factor B P , b2 is fusion factor I P , b3 is the panchromatic band P, and the fused blue band (Bf) image is obtained by calculation.

[0130] (2) The fusion operation expression corresponding to the green band is uint(0.827078*((b3-964.517466)+1*(1.268536*(b1-(0.874416))-1*1.124508*(b2-(0.646806))))+816.209962), where b1 is the fusion factor G P , b2 is fusion factor I P , b3 is the panchromatic band P, and the fused green band (Gf) image is obtained by calculation.

[0131] (3) The fusion operation expression corresponding to the red band is uint(1.130501*((b3-964.517466)+1*(0.936341*(b1-(0.561689))-1*1.124508*(b2-(0.646806))))+833.355385), where b1 is the fusion factor R P , b2 is fusion factor I P , b3 is the panchromatic band P, and the fused red band (Rf) image is obtained by calculation.

[0132] (4) The fusion operation expression corresponding to the near-infrared band is uint(1.30411*((b3-964.517466)+1*(0.827345*(b1-(0.34903))-1*1.124508*(b2-(0.646806))))+1544.975555), where b1 is the fusion factor N P , b2 is fusion factor I P , b3 is the panchromatic band P, and the fused near-infrared band (Nf) image is obtained by calculation.

[0133] The fused near-infrared band (Nf), red band (Rf) and green band (Gf) are synthesized into a standard false color image according to the red, green and blue channels. Figure 8 The fused red band (Nf), green band (Gf) and blue band (Bf) are synthesized into a true color image according to the red, green and blue channels. Fig. 9 .

[0134] For remote sensing image fusion, the basic requirement is to ensure the richness and clarity of geometric spatial information such as texture, layer, and details of the fused multispectral image, and to ensure the stability of the fine spectral characteristics and color information of the remote sensing image before and after fusion. In general, the quality of remote sensing images before and after fusion can be quantitatively evaluated from the following aspects:

[0135] First, the richness and brightness of the image colors before and after fusion can be measured by the statistical characteristics of the bands - maximum value, minimum value, mean, standard deviation and the correlation indicators between the bands - correlation coefficient, covariance and other indicators; second, the richness of the overall information of the image before and after fusion can be measured by indicators such as the information entropy of the image band; third, the level (edge), details (texture) and clarity of the image before and after fusion can be measured by indicators such as gradient and average gradient; fourth, the consistency and inheritance of the image information before and after fusion, that is, the degree of correlation between the fused multispectral image and the panchromatic band and the original multispectral band information, can reflect the degree of inheritance of the fused image to the original image's spatial and spectral information. If the correlation between the fused image and the panchromatic band image is high, it means that the spatial geometric detail information of the panchromatic band image is well injected; if the correlation between the fused image and the original multispectral band is high, it means that the fine spectral information of the multispectral image is well preserved, so that the quality of image fusion can be evaluated.

[0136] By comparing the differences in the indicators of the images before and after fusion, we can analyze the changing directions of the image's spectral (grayscale, hue) information, edge (level, difference) information, and texture (detail) information, thereby judging the quality of the fused image.

[0137] There are two basic indicators for spatial-spectral fusion that reflect geometric clarity and spectral fidelity: the correlation coefficient between the fusion result and the original multispectral data and panchromatic data. It is generally believed that the higher the correlation coefficient with the multispectral data, the better the fidelity of the spectral characteristics and the greater the information entropy; the higher the correlation coefficient between the fusion result and the panchromatic band, the greater the average gradient ratio, the higher the geometric clarity, and the better the image details are preserved.

[0138] In this case, four remote sensing images, namely, the original multispectral image of GF-6, the panchromatic injection multispectral fusion image of the method of the present invention, the multispectral injection panchromatic fusion image of the method of the present invention, and the fusion image of the Gram-Schmidit method, were subjected to band data statistical analysis. The comparison of the statistical characteristic parameters of the image bands is shown in Table 5. As can be seen from the data in the table, compared with the original multispectral image, the spatial resolution of the image after fusion by the method of the present invention is increased from the original 8 meters to 2 meters, and the image spatial accuracy is greatly improved. The fused image has both the original multispectral band and the panchromatic band. Compared with the GS fusion image, the panchromatic injection multispectral fusion image (Scheme 1) of this method has smaller correlation between bands and better data structure; the information entropy of the multispectral band is slightly reduced, but the gradient information of the band is enhanced, and the spatial information of the ground objects in the fused multispectral image is richer. Compared with GS fusion image, the information entropy of multispectral bands of multispectral injected into panchromatic fusion image (Scheme 2) of this method is slightly reduced, but the gradient information of the band is greatly enhanced. The spatial information of ground objects in the fused multispectral image is richer, which is conducive to the development of various image applications such as ground object identification, interpretation, and analysis. It can be seen from the data in the table that compared with the multispectral image fused by the Gram-Schmidt method, the gradient index data of the standard false color combination image of the multispectral image fused by the method of the present invention is better, and the hierarchical expression of the ground objects is better. The two image fusion methods have their own characteristics, and the overall performance is comparable.

[0139] In view of the shortcomings of traditional Brovey spatial-spectral fusion, such as large changes in color information, limited number of fused image bands, and unpredictable fusion results, this embodiment provides an image product fusion method based on scale invariance.

[0140] The image product fusion method, system and readable storage medium based on scale invariance described in the present invention are mainly aimed at remote sensing images with panchromatic bands and multi-spectral bands such as near-infrared, red, green and blue bands. First, based on the assumption of scale invariance of images, the method uses the spatial detail differences and spectral detail differences of images of different spatial scales to obtain a product fusion formula based on mean filtering; secondly, in view of the defect that the traditional product fusion method is difficult to predict the fusion result, a pair of product fusion operators are constructed to transform the product fusion into a linear fusion form, so that the fusion result can be predicted, which is conducive to the selection of fusion parameters; thirdly, two product fusion modes of panchromatic injection into multispectral and multispectral injection into panchromatic are provided; finally, by adjusting the feature extraction coefficient k I and feature fusion coefficient k E The value can be customized to realize the image fusion result's ability to preserve the original image's spectral and spatial information.

[0141] This method improves the spectral preservation ability of product fusion while maintaining the spatial details of the panchromatic image. The fused image has bright colors, rich information, stable spectral information, and is easy to visually and automatically classify. Especially in the context of the rapid development of high-resolution satellite remote sensing, it has a huge role in promoting the promotion and application of domestic high-resolution images in various industries at home and abroad.

[0142]

[0143] Example 2

[0144] The present embodiment discloses an image product fusion system based on the scale invariance principle, including a memory and a processor. The memory includes an image product fusion method program based on the scale invariance principle. When the image product fusion method program based on the scale invariance principle is executed by the processor, the steps of the image product fusion method based on the scale invariance principle described in Example 1 are implemented.

[0145] Example 3

[0146] This embodiment discloses a computer-readable storage medium, which includes an image product fusion method program based on the scale invariance principle. When the image product fusion method program based on the scale invariance principle is executed by a processor, the steps of the image product fusion method based on the scale invariance principle described in Embodiment 1 are implemented.

[0147] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0148] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0149] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0150] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, and other media that can store program codes.

[0151] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

Claims

1. An image product fusion method based on the scale invariance principle, characterized in that: The method comprises: Acquire satellite remote sensing images, which include high spatial resolution panchromatic band images P and low spatial resolution multispectral images, which include images of n bands, denoted as M1, M2, ..., M n ; All images are spatially registered to make the geometrical spatial positions of the same ground object in the panchromatic band image P and the multispectral image consistent, and the multispectral image to be fused is resampled as a high spatial resolution image, and the multispectral image to be fused is recorded as M; Count the mean, mean square error and correlation coefficient matrix of all multispectral images and panchromatic band images P; Construct intermediate band I using low-resolution multispectral imagery: Wherein, l is the preset number of multispectral bands involved in constructing the intermediate band I, l≤n; Select the panchromatic band injected into the multispectral band product fusion mode or the multispectral band injected into the panchromatic band product fusion mode, and construct the corresponding product fusion operator; The space-spectrum fusion calculation is performed according to the product fusion operator, fusion mode and fusion parameters, and the fusion parameters include k I , k E , k I is the feature extraction coefficient, k E is the feature fusion coefficient; If the panchromatic band is injected into the multispectral band product fusion mode, a pair of product fusion operators P is constructed M and I M for: Where: μ P is the mean value of the panchromatic band image P, μ I The mean of the intermediate band I; The specific method of injecting the panchromatic band into the multispectral band product fusion is: Estimate the mean square error σ of the fused image Mf Mf for: Among them, σ M is the mean square error of the multispectral image M, r(M,P M ) is the multispectral image M and the product fusion operator P M The correlation coefficient, r(M,I M ) is the multispectral image M and the product fusion operator I M The correlation coefficient, r(P M ,I M ) is the product fusion operator P M Sum-Product Fusion Operator I M The correlation coefficient of Thus, the product fusion mode of the panchromatic band injected into the multispectral band is as follows: In the formula, μ M is the mean value of the multispectral image M, and They are respectively the product fusion operator P M ,I M The mean and mean square error of If the multi-spectral band injection panchromatic band product fusion mode is adopted, a pair of product fusion operators M is constructed P and I P for: In the formula, μ M is the mean of the multispectral image M, μ I The mean of the intermediate band I; The specific method of multi-spectral band injection panchromatic band product fusion is: Estimate the mean square error σ of the fused image Mf Mf for: Among them, σ P is the mean square error of the panchromatic band image P, r(P,M P ) is the panchromatic band image P and the product fusion operator M P The correlation coefficient, r(P,I P ) is the panchromatic band image P and the product fusion operator I P The correlation coefficient, r(M P ,I P ) is the product fusion operator M P Sum-Product Fusion Operator I P The correlation coefficient of Therefore, the product fusion mode of injecting multispectral images into panchromatic band images is as follows: In the formula, μ P is the mean value of the panchromatic band image P, and They are respectively the product fusion operator M P ,I P The mean and mean square error of .

2. The image product fusion method based on the scale invariance principle according to claim 1 is characterized in that: The method further comprises: selecting a plurality of multispectral images of different wavebands to perform spatial-spectral fusion with the panchromatic waveband image respectively.

3. The image product fusion method based on the scale invariance principle according to claim 2 is characterized in that: The method further comprises: synthesizing a plurality of spatial-spectral fused images into one image and storing the image.

4. The image product fusion method based on the scale invariance principle according to claim 1 is characterized in that: Extraction coefficient k I And feature fusion coefficient k E It is used to ensure the fused image's ability to preserve the spatial geometric texture information of the original panchromatic image and the color information of the original multispectral image. The default value of the coefficient is 1.

5. An image product fusion system based on the scale invariance principle, characterized in that: It comprises a memory and a processor, wherein the memory comprises an image product fusion method program based on the scale invariance principle, and when the image product fusion method program based on the scale invariance principle is executed by the processor, the steps of the image product fusion method based on the scale invariance principle as described in any one of claims 1 to 4 are implemented.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes an image product fusion method program based on the scale invariance principle. When the image product fusion method program based on the scale invariance principle is executed by a processor, the steps of the image product fusion method based on the scale invariance principle as described in any one of claims 1 to 4 are implemented.

Images