Remote sensing image product fusion method, system, device and storage medium

Abstract

The application discloses a remote sensing image product fusion method, system, device and storage medium, and the method comprises the steps of acquiring a remote sensing image panchromatic band and a remote sensing image multispectral band; constructing an intermediate band by using the multispectral band; synthesizing the intermediate band, the panchromatic band and the multispectral band into one image; performing image feature matching on each band in the multispectral band and the panchromatic band respectively with the intermediate band to obtain matched multispectral bands and a panchromatic band; taking the ratio of the matched panchromatic band and the intermediate band as an image space spectrum fusion product factor and multiplying the product factor with each band in the matched multispectral bands respectively to obtain fused multispectral bands; and merging the fused multispectral bands and storing the merged image. The application breaks through the band limitation of traditional fusion, and significantly improves the high-fidelity capability of multispectral image spectral information and the fine spatial geometric information of high-resolution images of the fusion result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing image fusion enhancement, and particularly relates to a remote sensing image product fusion method and system, a terminal device and a computer readable storage medium. BACKGROUND

[0002] With the development of multi-platform, multi-sensor, multi-weather, multi-temporal, multi-resolution remote sensing technology, the types of remote sensing images available for various industries are increasingly rich. These colorful remote sensing images not only provide flexible object selection space for remote sensing image application research, but also pose challenges to remote sensing image selection, integration, correction, enhancement and other image application pre-processing research.

[0003] Different remote sensing image data have different basic image characteristics such as spatial resolution, temporal resolution and spectral resolution, and have different application performances and application potentials in different application fields. Traditional remote sensing image processing focuses on the enhancement of general image features such as color, texture and level of a single image; remote sensing image fusion processing focuses on integrating the basic features of different images on a new remote sensing image map, fully tapping the integrated application potential and improving the application performance. In the past two decades, as a new direction of remote sensing image processing, remote sensing image fusion technology has made great progress and achieved a series of new results.

[0004] From the fusion principle, the remote sensing image fusion method can be divided into three categories: color space theory-based fusion method, information analysis principle-based fusion method and mathematical operation-based fusion method. The color space theory-based fusion method applies the main color models such as RGB, CMYK, Lab, IHS and HSV to image fusion, among which the IHS mode is the classic image fusion method (Te-ming TU et al., A new look at IHS-look image fusion methods, Information Fusion 2, 2001, 177-18). The color model components of this method are relatively independent and can be controlled separately, which can accurately describe the color characteristics (Li et al., Comparison of ZY-3 satellite panchromatic and multispectral image fusion methods, Transactions of the Chinese Society of Agricultural Engineering, 30(16), 2014, 157-165). The advantages of this method are that it can quickly and simply fuse images, and the disadvantages are that although the fusion results obtained by simple component replacement can well maintain the details of the panchromatic image, the spectral characteristics of the satellite remote sensing image before and after fusion will usually change. The information analysis principle-based fusion method is to apply spatial filtering, principal component analysis (PCA), Fourier transform (FFT), wavelet (Wavelet) transform, Gram-Schimdt transform, curvelet transform (Curvelet), contourlet transform (Contourlet), ridgelet transform (Ridgelet), bandelet transform (Bandelet), wedgelet transform (Wedgelet), beamlet transform (Beamlet) and other information analysis techniques to image fusion. This method injects the detail information of the panchromatic band into the multispectral image (Xiao et al., Progress and challenges of multi-source optical and spectral remote sensing image fusion, Journal of Image and Graphics, 25(05), 2020), and the obtained fusion image has good spectral continuity, but the calculation is complex and part of the image high-frequency information will be lost. The mathematical operation-based fusion method mainly includes ratio method, difference method, weighted superposition, multiple amplification and four arithmetic mixing methods. Among them, the classic methods are Brovey fusion method and CN fusion method.

[0005] Brovey fusion is a fusion method based on product transformation proposed by American scholar Brovey, also known as color normalization transformation fusion method (Vrabel J. Multispectral Imagery Band Sharpening Study. Photogrammetric Engineering & Remote Sensing, 1996, 62(9): 1075-1083), which performs product operation on three multispectral bands and high spatial resolution panchromatic image, simplifies the image fusion operation coefficient, and greatly retains the spectral information of the source image, but can only perform image fusion on three bands. CN fusion (color normalized transform) is also known as energy subdivision transform, which uses high spatial resolution panchromatic band to enhance the low spatial resolution multispectral band of the input image. This method only fuses the corresponding input band within the spectral range of the fused image band, and other input bands are directly output without fusion processing. Brovey fusion and CN fusion require the spectral response range of the panchromatic band to be consistent or similar to that of the multispectral image, otherwise it will cause significant deviation of local brightness in the fused image, change the spectral characteristics of the multispectral image before and after fusion, and cause spectral distortion. Therefore, scholars have made various optimizations and improvements to Brovey, CN and other fusion methods. Some scholars have introduced adaptive optimization methods to adjust the weight parameters of each band of the multispectral image and the panchromatic band, and have improved the Brovey fusion algorithm through weighted averaging, which not only retains the spectral physical characteristics of the multispectral source image, but also improves the spatial resolution characteristics of the image (Lin Zilei, et al. ALI Image Fusion Algorithm Based on Improved Brovey Transform. Remote Sensing Technology and Application, 2020, 35(4): 893-900). Some scholars use the nonlinear spectral data mining characteristics of the kernel principal component transform (KPCA) algorithm to extract the three principal components KPC1, KPC2 and KPC3 with the largest information in the multispectral image; then use the Brovey algorithm to normalize and fuse the three principal components and the panchromatic band, so that the spatial information and spectral information in the fusion result are more abundant (Ke Hongxia, et al. A Remote Sensing Image Fusion Method Based on KPCA and Brovey Transform. Journal of Chongqing University of Technology (Natural Science Edition), 2019, 38(2)). Some scholars introduce wavelet analysis to improve the Brovey transform, which is affected by noise and has too many scattered details in high-resolution images (Chen Sijin, et al. Improved Brovey Remote Sensing Image Fusion Method Using Wavelet Analysis. Journal of the School of Geodesy and Photogrammetry, 2004, 21(2)). Since the Brovey, CN and other fusion methods were introduced, their improvement has not stopped and has no end. SUMMARY

[0006] In view of the defects of traditional remote sensing image fusion, such as large color information change and limited number of fused image bands, the application provides a remote sensing image product fusion method, system, terminal device and computer readable storage medium, constructs an intermediate band by using a multi-spectral band, performs image feature matching on the full-color band, multi-spectral band and intermediate band, performs spatial-spectral fusion on the original multi-spectral band and feature matching result by using product algorithm principle, and obtains a multi-spectral band image with fine spatial geometric information and original spectral information.

[0007] The first object of the application is to provide a remote sensing image product fusion method.

[0008] The second object of the application is to provide a remote sensing image product fusion system.

[0009] The third object of the application is to provide a terminal device.

[0010] The fourth object of the application is to provide a computer readable storage medium.

[0011] The first object of the application can be achieved by adopting the following technical scheme:

[0012] A remote sensing image product fusion method, the method comprising:

[0013] acquiring a remote sensing image full-color band and a remote sensing image multi-spectral band; the number of bands in the remote sensing image multi-spectral band is greater than or equal to 1;

[0014] constructing an intermediate band by using a multi-spectral band, and synthesizing the intermediate band, full-color band and multi-spectral band into an image;

[0015] performing image feature matching on each band in the multi-spectral band and the full-color band respectively with the intermediate band, to obtain matched multi-spectral bands and full-color bands;

[0016] taking the ratio of the matched full-color band and intermediate band as an image spatial-spectral fusion product factor; multiplying the image spatial-spectral fusion product factor with each band in the matched multi-spectral band respectively, to obtain fused multi-spectral bands;

[0017] merging the fused multi-spectral bands and storing the merged image.

[0018] Further, the expression of the fused multi-spectral band is:

[0019]

[0020] wherein, is the i-th band in the fused multi-spectral band; hmPbyI is the matched panchromatic band, I is the intermediate band; hmB i byI is the i-th band in the matched multi-spectral bands; i is greater than or equal to 1 and less than or equal to the number of bands in the multi-spectral bands.

[0021] Further, the expression of the matched panchromatic band is:

[0022]

[0023] wherein, hmPbyI is the matched panchromatic band of the remote sensing image; μ P and σ P are the mean and standard deviation of the panchromatic band P in the synthetic image respectively; μ I and σ I are the mean and standard deviation of the intermediate band I in the synthetic image respectively;

[0024] The expression of the matched multi-spectral band is:

[0025]

[0026] wherein, hmB i byI is the i-th band in the matched multi-spectral bands; and are the mean and standard deviation of the i-th band B i in the multi-spectral bands in the synthetic image respectively; i is greater than or equal to 1 and less than or equal to the number of bands in the multi-spectral bands.

[0027] Further, the constructed intermediate band is:

[0028]

[0029] wherein, I is the intermediate band; B i is the i-th band in the multi-spectral bands; n is the number of bands in the multi-spectral bands; 1≥ω i ≥0,

[0030] Further, ω i =1 / n.

[0031] Further, the panchromatic band of the remote sensing image is replaced by any band image in the multi-spectral bands of another remote sensing image.

[0032] Further, before constructing the intermediate band, the panchromatic band of the remote sensing image and the multi-spectral band of the remote sensing image are spatially registered, so that the geometric spatial positions of the same ground object in the two images are consistent;

[0033] Resample the intermediate waveband and the multi-spectral waveband of the remote sensing image according to the high spatial resolution image, and synthesize the image file together with the panchromatic waveband of the remote sensing image.

[0034] The second object of the present application can be achieved by adopting the following technical solution:

[0035] A remote sensing image product fusion system, the system comprising:

[0036] An acquisition module for acquiring a panchromatic waveband of a remote sensing image and multi-spectral wavebands of the remote sensing image, wherein the number of wavebands in the multi-spectral wavebands is greater than or equal to 1;

[0037] A construction module for constructing an intermediate waveband using the multi-spectral wavebands, and synthesizing the intermediate waveband, the panchromatic waveband and the multi-spectral wavebands into an image;

[0038] A feature matching module for performing image feature matching between each of the multi-spectral wavebands and the panchromatic waveband with the intermediate waveband, respectively, to obtain matched multi-spectral wavebands and the panchromatic waveband;

[0039] A reconstruction module for taking the ratio of the matched panchromatic waveband and the intermediate waveband as an image spectral fusion product factor, and multiplying the image spectral fusion product factor with each of the matched multi-spectral wavebands to obtain fused multi-spectral wavebands;

[0040] A merging and storage module for merging the fused multi-spectral wavebands and storing the merged image.

[0041] The third object of the present application can be achieved by adopting the following technical solution:

[0042] A terminal device comprising a processor and a memory for storing a program executable by the processor, wherein the processor, when executing the program stored in the memory, implements the remote sensing image product fusion method described above.

[0043] The fourth object of the present application can be achieved by adopting the following technical solution:

[0044] A computer readable storage medium storing a program, wherein the program, when executed by a processor, implements the remote sensing image product fusion method described above.

[0045] The present application has the following beneficial effects compared with the prior art:

[0046] (1) The present application has clear physical meaning of each parameter, clear structure and fast operation.

[0047] The present application constructs image fusion product factor by using the product algorithm idea, constructs intermediate wave band I by using linear combination of multi-spectral wave band, then carries out histogram feature matching between full color wave band P and multi-spectral image and intermediate wave band I, then constructs image space spectrum fusion product factor of each wave band of multi-spectral image, then multiplies image space spectrum fusion product factor with multi-spectral image to reconstruct fusion image result, so that the spatial information of multi-spectral image before and after fusion is enhanced and the spectral information is kept true. Compared with original image, the geometric details, texture, edge, level and other spatial information of ground objects in the fused image are greatly enriched, and compared with original multi-spectral image, the spectral characteristics and color display of water body, vegetation, bare ground and building in the fused multi-spectral image are kept stable. The fusion expression is simple and clear, and the calculation is fast and efficient.

[0048] (2) The present application only depends on the inherent characteristics of image data, and has universality.

[0049] The present application is applicable to fusion of all multi-spectral satellite remote sensing images with full color wave band and near infrared, red, green and blue wave bands to improve the spatial accuracy of multi-spectral wave band, and is also applicable to fusion of a certain multi-spectral wave band (such as near infrared wave band) into other multi-spectral wave bands to enhance the spectral information of the latter, and is also applicable to fusion of different types of images such as optical images and microwave images, only for specific application purposes.

[0050] (3) The present application has no limitation on the number of wave bands of the fusion target image, and has openness.

[0051] The present application can fuse a multi-spectral image of one wave band, or can fuse a fusion target image of 2, 3, 4 or more wave bands, and the number of wave bands of the fused image is corresponding to the number of wave bands of the fusion target image. The present application has no limitation on the number of wave bands of the fusion target image, and has openness. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor based on the drawings shown.

[0053] Figure 1 The simple flow chart of the remote sensing image product fusion method of embodiment 1 of the present application;

[0054] Figure 2A detailed flow chart of the remote sensing image product fusion method of the embodiment 1 of the present application;

[0055] Figure 3 A panchromatic band (P) image (2m resolution) of the embodiment 1 of the present application;

[0056] Figure 4 A multispectral NRG standard false color image (8m resolution) of the embodiment 1 of the present application;

[0057] Figure 5 A multispectral RGB true color composite image (8m resolution) of the embodiment 1 of the present application;

[0058] Figure 6 A multi-intermediate band I image of the embodiment 1 of the present application;

[0059] Figure 7 A fused multispectral R'G'B' band true color composite image (2m resolution) of the embodiment 1 of the present application;

[0060] Figure 8 A fused multispectral N'R'G' band standard false color composite image (2m resolution) of the embodiment 1 of the present application;

[0061] Figure 9 A structure block diagram of the remote sensing image product fusion system of the embodiment 2 of the present application;

[0062] Figure 10 A structure block diagram of the terminal device of the embodiment 3 of the present application. DETAILED DESCRIPTION

[0063] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in connection with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without any creative work fall within the protection scope of the present application. It should be understood that the described specific embodiments are only used to explain the present application, and are not used to limit the present application.

[0064] It should be noted that for remote sensing image fusion, the basic requirement is to ensure the richness and clarity of the geometric spatial information such as texture, level and detail of the multispectral image after fusion, and to ensure the stability of the fine spectral characteristics and color information of the remote sensing image before and after fusion. Overall, the quality of the remote sensing image before and after fusion can be quantitatively evaluated from the following aspects: first, the richness and brightness of the color of the image before and after fusion can be measured by the band statistical characteristics, such as maximum value, minimum value, mean value, standard deviation, and the correlation between bands, such as correlation coefficient and covariance; second, the richness of the overall information of the image before and after fusion can be measured by the information entropy of the image band; third, the level (edge), detail (texture) and clarity of the image before and after fusion can be measured by the gradient and average gradient; fourth, the consistency and inheritance of the information of the image before and after fusion, that is, the correlation between the multispectral image after fusion and the panchromatic band and the original multispectral band, can reflect the inheritance degree of the spatial information and spectral information of the original image by the fused image. If the correlation between the fused image and the panchromatic image is high, it means that the spatial geometric detail information of the panchromatic image is well injected, and if the correlation between the fused image and the original multispectral band is high, it means that the fine spectral information of the multispectral image is well preserved, so as to evaluate the quality of image fusion.

[0065] By comparing the differences of the respective indexes of the images before and after fusion, the change direction of the spectral (gray level, hue) information, edge (level, difference) information and texture (detail) information of the image can be analyzed, so as to judge the pros and cons of the quality of the fused image.

[0066] For spectral fusion, there are two basic indexes reflecting geometric clarity and spectral fidelity, that is, the correlation coefficient between the fusion result and the original multispectral data. Generally, the larger the correlation coefficient between the fusion result and the multispectral data, the better the spectral characteristic fidelity, and the larger the information entropy; the larger the correlation coefficient between the fusion result and the panchromatic band, the larger the average gradient ratio, the higher the geometric clarity, and the better the image detail preservation.

[0067] Embodiment 1:

[0068] As shown in Figure 1 , 2 , the embodiment provides a remote sensing image product fusion method, comprising the following steps:

[0069] S101, acquiring a remote sensing image panchromatic band and a remote sensing image multispectral band.

[0070] The embodiment does not limit the number of bands in the multispectral remote sensing image, and the number of bands can be greater than 1.

[0071] The panchromatic band and the multispectral band are spatially registered to make the geometric spatial positions of the same ground object in the two images consistent.

[0072] S102, constructing an intermediate band by using multi-spectral bands of remote sensing images.

[0073] The intermediate band I is:

[0074]

[0075] wherein, B i is the i-th band in the multi-spectral bands; n is the number of multi-spectral bands participating in the fusion, n≥1; 1≥ω i ≥0,

[0076] Generally, ω i =1 / n.

[0077] S103, synthesizing an image file by the intermediate band, the panchromatic band and the multi-spectral bands.

[0078] The intermediate band and the multi-spectral bands are resampled according to high spatial resolution images, and are synthesized with the panchromatic band to form an image file, and the mean value and the standard deviation of each band in the synthesized image are calculated.

[0079] The mean value i and the standard deviation of any band image B

[0080] B i =B i (l,m)

[0081]

[0082] wherein, l and m are the horizontal and vertical coordinates of the pixels in the band B i .

[0083] The mean value μ I and the standard deviation σ I of the intermediate band I are calculated.

[0084]

[0085] S104, performing image feature matching between each band in the multi-spectral bands and the panchromatic band and the intermediate band respectively, to obtain the matched multi-spectral bands and the panchromatic band.

[0086] The specific expression is as follows:

[0087]

[0088] wherein, μ P and σ P are the mean value and the standard deviation of the panchromatic band P; hmBi byI is a band B in a multi-spectral band i hmPbyI is an image feature matching result of the panchromatic band P and the intermediate band I.

[0089] S105, multiplying the matched ratio of the panchromatic band and the intermediate band by each band of the matched multi-spectral band to obtain a fused multi-spectral band.

[0090] The image space-spectrum information fusion is performed by using the product algorithm principle, and the calculation result of the fused image is:

[0091]

[0092] wherein, is an image space-spectrum fusion product factor, is a reconstructed band (fused band image) of the intermediate band I of the original multi-spectral band B. i

[0093] S106, merging the fused multi-spectral band and storing the merged image.

[0094] The fused band image of each band of the multi-spectral band is calculated by using the fusion formula of step S105; the fused band image is merged to obtain a multi-spectral fusion image result and stored.

[0095] It can be understood that, Figure 2 B i is any band image of the multi-spectral band.

[0096] The method provided by the embodiment is based on the following principles:

[0097] (1) When the high spatial resolution panchromatic band of the remote sensing image is used to perform image space-spectrum product fusion enhancement on the low spatial resolution multi-spectral band, the image features such as the mean value and histogram of each band of the multi-spectral image before and after the fusion should be kept stable, so as to achieve the purpose of enhancing the spatial resolution and geometric detail information of the multi-spectral image, and highly maintaining the rich spectral features and color information of the multi-spectral image.

[0098] (2) The panchromatic band and the multi-spectral band are respectively matched with the intermediate band in terms of image features, and on this basis, the ratio of the matched panchromatic band and the intermediate band is taken as a product factor to multiply the matched multi-spectral band, so as to obtain a fused multi-spectral image with high spatial resolution, rich geometric detail information and high-fidelity spectral information.

[0099] ​The method provided by the embodiment is also applicable to replacing the panchromatic band with any one of the multispectral bands of another remote sensing image, and the fusion process is the same as the above method, which will not be described herein.

[0100] As a specific embodiment, for the purpose of realizing the fusion of a panchromatic image and a multispectral image, the embodiment mainly uses ENVI remote sensing image processing software to realize the purpose, and a satellite remote sensing image with a panchromatic band (P), a blue band (B), a green band (G), a red band (R) and a near-infrared band (N) is further described.

[0101] (1) Input the remote sensing image.

[0102] Open a domestic GF6 remote sensing image (the case image is taken from GF6_PMS_E112_8_N23_2_20210118_L1A1120072629) with a panchromatic band (P), a blue band (B), a green band (G), a red band (R) and a near-infrared band (N). Figures 3-5 The panchromatic band image (2-meter resolution), the multispectral NRG standard false color band image (8-meter resolution) and the multispectral RGB true color image (8-meter resolution) are respectively shown in FIGS. 2, 3 and 4 (the effect images are obtained according to the default setting of 1% stretching of ENVI).

[0103] (2) In ENVI, use the four multispectral band images to construct an intermediate band I.

[0104] The band operation expression is: I = (1.0*b1+b2+b3+b4) / 4

[0105] Wherein, b1, b2, b3 and b4 are respectively the blue band (B), the green band (G), the red band (R) and the near-infrared band (N).

[0106] The calculation result of the intermediate band I of the fusion target image is shown in FIG. 5. Figure 6 .

[0107] (3) Use ENVI software to resample the intermediate band I and the multispectral bands B, G, R and N according to the high spatial resolution fusion source image, and synthesize a image file with the panchromatic band P, and then calculate the mean value μ and the standard deviation σ of each band image. The basic characteristic statistical parameters of each band image are shown in Table 1.

[0108] Table 1 Basic characteristic statistical parameters of each band image

[0109] Image band Mean value / μ Standard deviation / σ Panchromatic band P 820.062867 352.608833 Intermediate band I 895.473494 348.621005 Blue band B 758.198332 195.501797 Green band G 826.7832 297.752981 Red band R 789.444254 415.289068 Near infrared N 1205.967693 617.606258

[0110] (4) Reconstruct the fused multispectral image.

[0111] ① Reconstruct the fused blue band (B') image, the operation expression is uint((0.988691*(b1-820.062867)+895.473494)*(1.783211*(b2-758.198332)+895.473494) / b3+0.5), wherein b1 is the panchromatic band P, b2 is the blue band (B), and b3 is the intermediate band I, to obtain the fused blue band (B') image (2m resolution).

[0112] ② Reconstruct the fused green band (G') image, the operation expression is uint((0.988691*(b1-820.062867)+895.473494)*(1.170840*(b2-826.7832)+895.473494) / b3+0.5), wherein b1 is the panchromatic band P, b2 is the green band (G), and b3 is the intermediate band I, to obtain the fused green band (G') image (2m resolution).

[0113] ③ Reconstruct the fused red band (R') image, the operation expression is uint((0.988691*(b1-820.062867)+895.473494)*(0.839466*(b2-789.444254)+895.473494) / b3+0.5), wherein b1 is the panchromatic band P, b2 is the red band (R), and b3 is the intermediate band I, to obtain the fused red band (R') image (2m resolution).

[0114] ④ Reconstruct the fused near-infrared (N') image, the operation expression is uint((0.988691*(b1-820.062867)+895.473494)*(0.564471*(b2-1205.967693)+895.473494) / b3+0.5), wherein b1 is the panchromatic band P, b2 is the near-infrared red band (N), and b3 is the intermediate band I, to obtain the fused near-infrared (N') image (2m resolution).

[0115] Synthesize the fused red band (R'), green band (G') and blue band (B') into a true color image according to the red, green and blue channels, as shown in Figure 7 ; synthesize the fused near-infrared band (N'), red band (R') and green band (G') into a standard false color image according to the red, green and blue channels, as shown in Figure 8 .

[0116] The embodiment carries out band data statistical analysis on three kinds of remote sensing images, i.e., the GF6 original multispectral image, the multispectral image fused by the method provided in the embodiment, and the multispectral image fused by the Gram-Schmidt method. The image band statistical characteristic parameter comparison is shown in Table 2. As can be seen from the data in the table, compared with the original multispectral image, the spatial resolution of the multispectral image fused by the method provided in the embodiment is increased from 8 meters to 2 meters, and the image spatial accuracy is greatly improved. The average of the correlation coefficients of each band is reduced from 0.816 to 0.815, the data structure of each band is further optimized, the total amount of image spectral information is increased, the fused image has the image information of the original multispectral band and the panchromatic band at the same time, the information consistency index is increased from 0.934 to 0.938, the information entropy of the multispectral band is slightly reduced, but the gradient information of each multispectral band is greatly enhanced, the fused multispectral image has more spatial information of ground objects, which is conducive to the development of various image applications such as ground object identification, interpretation and analysis. As can be seen from the data in the table, compared with the multispectral image fused by the Gram-Schmidt method, the correlation coefficient between the multispectral image fused by the method provided in the embodiment is lower, the information redundancy is smaller, the gradient index data of the true color composite image is better, the expression of the details and texture characteristics of the ground objects is better, and the information entropy index data is slightly lower. The two kinds of image fusion methods have their own characteristics, and the overall performance is not much different.

[0117] Table 2 Comparison of band statistical characteristic parameters of original multispectral image, fused image by the method, and GS fused image

[0118]

[0119]

[0120] By using the method provided in the embodiment, the spatial resolution of the fused multi-spectral image is greatly improved, the spatial information is greatly enriched, the geometric texture, spatial details, edge definition and hierarchy of the features on the image are comprehensively improved, meanwhile, the spectral characteristics and color display stability of the original multi-spectral image can be maintained, and the information richness and overall quality of the remote sensing image are greatly improved. The spectral information of the multi-spectral remote sensing image is rich, and the color information of the features such as water, vegetation, buildings, bare rocks and bare soil is very rich, and the spectral information and color display difference of different features are obvious, but the spatial resolution of the multi-spectral image is low, and the feature texture and detail information are lacking, which leads to inaccurate identification of feature types and positions in remote sensing analysis, and affects the application effect of the multi-spectral image. The method provided in the embodiment multiplies the panchromatic band with the multi-spectral band to enhance the pixel information point by point, so that the spatial geometric texture, edge, hierarchy and other information of the features in the image are comprehensively improved, the visual resolution and computer analysis ability of the multi-spectral image are effectively improved, and the application effect and ability of the remote sensing image are improved; the multi-spectral image is reconstructed by product fusion, the image space-spectral fusion product factor is constructed based on the feature matching principle, the spatial information amount, definition and hierarchy of the features in the multi-spectral image can be enhanced, the rich spectral characteristics and color display of the original multi-spectral image can be highly retained, the image spectral characteristics and display effect of typical features such as water, vegetation, soil and buildings are not changed, the overall quality of the multi-spectral image is effectively improved, and the potential of remote sensing image mapping application and the ability of classification application are improved.

[0121] Those skilled in the art can understand that all or part of the steps in the method of the above embodiment can be instructed by a program to relevant hardware, and the corresponding program can be stored in a computer readable storage medium.

[0122] It should be noted that although the method operations of the above embodiments are described in a specific order in the accompanying drawings, this does not require or imply that the operations must be performed in this specific order, or that all of the shown operations must be performed to achieve the desired result. On the contrary, the depicted steps can change the order of execution. Additionally or alternatively, some steps can be omitted, a plurality of steps can be combined into one step, and / or one step can be divided into a plurality of steps.

[0123] Embodiment 2:

[0124] As shown in Figure 9 The embodiment provides a remote sensing image product fusion system, which comprises an acquisition module 901, a construction module 902, a feature matching module 903, a reconstruction module 904 and a merging and storage module 905, and specifically comprises:

[0125] The acquisition module 901 is used to acquire the panchromatic band and multispectral band of the remote sensing image; the number of bands in the multispectral band of the remote sensing image is greater than or equal to 1.

[0126] Module 902 is used to construct intermediate bands using multispectral bands and combine intermediate bands, panchromatic bands and multispectral bands into one image;

[0127] The feature matching module 903 is used to perform image feature matching between each band and the panchromatic band in the multispectral band and the intermediate band, respectively, to obtain the matched multispectral band and panchromatic band;

[0128] The reconstruction module 904 is used to take the ratio of the matched panchromatic band to the intermediate band as the image spatial-spectral fusion product factor; and multiply the image spatial-spectral fusion product factor by each band in the matched multispectral band to obtain the fused multispectral band.

[0129] The merging and storage module 905 is used to merge the fused multispectral bands and store the merged image.

[0130] The specific implementation of each module in this embodiment can be found in Embodiment 1 above, and will not be repeated here. It should be noted that the system provided in this embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure can be divided into different functional modules to complete all or part of the functions described above.

[0131] Example 3:

[0132] This embodiment provides a terminal device, which can be a computer, such as... Figure 10 As shown, the system is connected via a system bus 1001 to a processor 1002, a memory, an input device 1003, a display 1004, and a network interface 1005. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium 1006 and internal memory 1007. The non-volatile storage medium 1006 stores an operating system, computer programs, and a database. The internal memory 1007 provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. When the processor 1002 executes the computer programs stored in the memory, it implements the remote sensing image product fusion method of Embodiment 1 described above, as follows:

[0133] Obtain the panchromatic band and multispectral band of the remote sensing image; the number of bands in the multispectral band of the remote sensing image is greater than or equal to 1;

[0134] An intermediate band is constructed using multispectral bands, and the intermediate band, panchromatic band, and multispectral band are combined into a single image;

[0135] image feature matching is performed on each of the multispectral bands and the panchromatic band with the intermediate band to obtain matched multispectral bands and the panchromatic band;

[0136] a ratio of the matched panchromatic band and the intermediate band is taken as an image space spectrum fusion multiplication factor; and the image space spectrum fusion multiplication factor is multiplied with each of the matched multispectral bands to obtain fused multispectral bands;

[0137] the fused multispectral bands are merged and the merged image is stored.

[0138] Embodiment 4:

[0139] The embodiment provides a computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the remote sensing image product fusion method in the above embodiment 1, as follows:

[0140] obtaining a remote sensing image panchromatic band and a remote sensing image multispectral band; the number of bands in the remote sensing image multispectral band is greater than or equal to 1;

[0141] constructing an intermediate band by using the multispectral bands, synthesizing the intermediate band, the panchromatic band and the multispectral bands into one image;

[0142] image feature matching is performed on each of the multispectral bands and the panchromatic band with the intermediate band to obtain matched multispectral bands and the panchromatic band;

[0143] a ratio of the matched panchromatic band and the intermediate band is taken as an image space spectrum fusion multiplication factor; and the image space spectrum fusion multiplication factor is multiplied with each of the matched multispectral bands to obtain fused multispectral bands;

[0144] the fused multispectral bands are merged and the merged image is stored.

[0145] It should be noted that the computer readable storage medium of the embodiment can be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection with one or more conductive wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0146] The application provides a remote sensing image product fusion method, system, device and storage medium, and is mainly used for remote sensing images with full color bands and near-infrared, red, green and blue bands and the like. First, an intermediate band I is constructed by using the to-be-fused multi-spectral image. Then, based on the image feature matching principle, the intermediate band I is used for feature matching of the full color band P and the to-be-fused multi-spectral band. Finally, the ratio of the full color band after feature matching and the intermediate band is used to construct an image fusion product operator of each multi-spectral band, and the product fusion is performed with the multi-spectral band after feature matching, so that the fused multi-spectral remote sensing image is obtained. The method is suitable for fusing the full color band and the multi-spectral band, so as to enhance the spatial geometry, texture, edge, level and the like of the ground object in the multi-spectral image, improve the image definition and spatial resolution, and meanwhile, the fused image can highly maintain the spectral characteristics and color display stability of various ground objects in the original multi-spectral image. The method of the application is also suitable for injecting a certain multi-spectral band (such as a near-infrared band) into other multi-spectral bands (such as red, green and blue bands) for fusion, so as to enhance the information amount of the image and maintain the spectral characteristics. The technical method has a solid theoretical basis, clear physical meaning, wide application object, simple operation and high operation efficiency. The fused image is bright in color, rich in information, stable in spectral information, easy to visually and automatically classify, and especially under the background of the current rapid development of high-resolution satellite remote sensing, has a great promoting effect on the promotion and application of domestic high-resolution images in various industries at home and abroad.

[0147] The above merely describes preferred embodiments of the application, but the protection scope of the application is not limited thereto, and any person skilled in the art can make equivalent replacements or changes to the technical scheme and inventive concept of the application within the scope disclosed by the application, and the replacements or changes all belong to the protection scope of the application.

Claims

1. A remote sensing image product fusion method, characterized in that, The method includes: Obtain the panchromatic band and multispectral band of the remote sensing image; the number of bands in the multispectral band of the remote sensing image is greater than or equal to 1; An intermediate band is constructed using multispectral bands, and the intermediate band, panchromatic band, and multispectral band are combined into a single image; Image feature matching is performed between each band in the multispectral band and the panchromatic band and the intermediate band to obtain the matched multispectral band and panchromatic band; The ratio of the matched panchromatic band to the intermediate band is used as the image spatial-spectral fusion product factor; the image spatial-spectral fusion product factor is multiplied by each band in the matched multispectral band to obtain the fused multispectral band. The fused multispectral bands are merged and the merged image is stored.

2. The remote sensing image multiplication and fusion method according to claim 1, characterized in that, The expression for the fused multispectral bands is: in, It represents the i-th band in the fused multispectral bands; hmPbyI represents the image spatial-spectral fusion product factor, I represents the matched panchromatic band, and hmB represents the intermediate band. i byI represents the i-th band in the matched multispectral bands; i is the number of bands in the multispectral bands that is greater than or equal to 1 and less than or equal to 1.

3. The remote sensing image multiplication and fusion method according to any one of claims 1 and 2, characterized in that, The expression for the matched panchromatic band is: Where hmPbyI is the matched panchromatic band; μ P and σ P , respectively, represent the mean and standard deviation of the panchromatic band P in the composite image; μ I and σ I These are the mean and standard deviation of the intermediate band I in the composite image, respectively; The expression for the matched multispectral bands is: Among them, hmB i byI represents the i-th band in the matched multispectral bands; and The i-th band B in the multispectral bands of the composite image is respectively i The mean and standard deviation of ; i is the number of bands in the multispectral bands that is greater than or equal to 1 and less than or equal to 1.

4. The remote sensing image multiplication and fusion method according to any one of claims 1 and 2, characterized in that, The constructed intermediate band is: Where I represents the intermediate band; B i Let be the i-th band in the multispectral bands; n is the number of bands in the multispectral bands; 1 ≥ ω i ≥0, 5. The remote sensing image product fusion method according to claim 4, characterized in that, Take ω i = 1 / n.

6. The remote sensing image multiplication and fusion method according to any one of claims 1 and 2, characterized in that, Replace the panchromatic band of a remote sensing image with any band from the multispectral bands of another remote sensing image.

7. The remote sensing image product fusion method according to any one of claims 1 and 2, characterized in that, Before constructing the intermediate band, the panchromatic band and multispectral band of the remote sensing image are spatially registered to ensure that the geometric spatial position of the same ground feature is consistent in the two images. The intermediate band and the multispectral band of the remote sensing image are resampled according to the high spatial resolution image and combined with the panchromatic band of the remote sensing image into a single image file.

8. A remote sensing image product fusion system, characterized in that, The system includes: The acquisition module is used to acquire the panchromatic band and multispectral band of the remote sensing image; the number of bands in the multispectral band of the remote sensing image is greater than or equal to 1. The building module is used to construct intermediate bands using multispectral bands and combine intermediate bands, panchromatic bands and multispectral bands into a single image; The feature matching module is used to perform image feature matching between each band in the multispectral band and the panchromatic band and the intermediate band, respectively, to obtain the matched multispectral band and panchromatic band; The reconstruction module is used to take the ratio of the matched panchromatic band to the intermediate band as the image spatial-spectral fusion product factor; the image spatial-spectral fusion product factor is multiplied by each band in the matched multispectral band to obtain the fused multispectral band. The merging and storage module is used to merge the fused multispectral bands and store the merged image.

9. A terminal device, comprising a processor and a memory for storing a processor-executable program, characterized in that, When the processor executes the program stored in the memory, it implements the remote sensing image product fusion method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the remote sensing image product fusion method according to any one of claims 1 to 7.

Images