A remote sensing image fusion method based on HIS, NSCT and PCNN

Abstract

A remote sensing image fusion method based on HIS, NSCT and PCNN, firstly, performs geographic registration on the remote sensing images to be fused, so that the geographic coordinate systems of the images to be fused are consistent; then, the registered remote sensing images A and B to be fused are respectively decomposed in different scales and directions by NSCT, and the coefficients of the NSCT decompositions of the remote sensing images to be fused at different scales and in different directions are obtained, and the NSCT decomposition coefficients and are obtained; then, the NSCT decomposition coefficients and of the fused remote sensing images are fused by using high-frequency coefficient and low-frequency fusion rules, and the fused coefficients are obtained; then, the fused coefficients are inversely transformed by NSCT to obtain the fused intensity component I'; finally, the obtained intensity component I' is inversely transformed by IHS with the H and S components of the multispectral image to obtain the final fused image. The invention not only improves the spatial resolution of the image, but also can better maintain the spectral information of the original multispectral band.

Description

Technical Field

[0001] The invention relates to a shaking image fusion method, in particular to a remote sensing image fusion method based on HIS, NSCT and PCNN. Background Art

[0002] The core idea of ​​non-subsampled Contourlet transform (NSCT) is to use non-subsampled pyramid transform (NSPFB) and non-subsampled directional filter bank (NSDFB) to decompose the image in multiple scales and multiple directions. NSPFB decomposes the image in multiple scales to ensure that the image transformation has multi-resolution characteristics; NSDFB decomposes the sub-band images of each scale in a directional manner to ensure that the image transformation has multi-directional characteristics, thereby obtaining images of sub-bands of different scales and directions. After K layers of decomposition, a low-frequency sub-band image and high frequency sub-band images, where l k is the directional decomposition level of the kth layer. NSCT not only has multi-scale, good local characteristics in the spatial and frequency domains, and multi-directional characteristics, but also has the characteristics of translation invariance and the same size between sub-band images, which is conducive to better maintaining the edge contour information and texture detail information of the original image, and is suitable for expressing multi-sensor images with rich details and directional information.

[0003] Pulse coupled neural network (PCNN) was proposed by Eckhorn to explain the process of visual signal processing by visual cortical cells in the brains of small mammals such as cats. It has been widely used in image processing, such as image segmentation, edge extraction, target recognition, image fusion and other fields. The PCNN neuron model is a feedback network composed of several branch PCNN neurons interconnected, where each neuron consists of three functional units: a receptive field, a modulation part and a pulse generator. Since the numerous parameters of the traditional PCNN model limit the depth and breadth of its application, the present invention adopts a commonly used improved PCNN model, and its mathematical expression is as follows:

[0004] F ij (n) = S ij

[0005]

[0006] U ij (n) = F ij (n)×[1+βL ij (n)]

[0007] θ ij (n) = exp(-α θ )θ ij (n-1)+V θ Yij (n)

[0008]

[0009] Where (i, j) is the coordinate of the neuron or pixel; n is the number of iterations; S ij Represents the external input stimulus signal, usually the grayscale value of the (i, j)th pixel in the image; W ijkl Represents the connection weight coefficient matrix between neurons; F ij , L ij ,θ ij and U ij Respectively represent the feedback input, link input, dynamic threshold and internal activity item of the neuron; V L 、V θ are the amplitude coefficients of the link input and variable threshold function respectively; α L , α θ are the time constants of the link input and variable threshold function respectively; β represents the link strength; Y ij Represents the output of the neuron; when Y ij When the output of (n) is 1, it means that the pixel (i, j) is ignited once. When PCNN is used to process images, it is a single-layer connected two-dimensional network. The number of neurons in the network is consistent with the number of pixels, and each neuron corresponds to each pixel one by one. Summary of the invention

[0010] In summary, the present invention aims at the multispectral and panchromatic band characteristics of Landsat remote sensing data, combines the advantages of IHS color transform, non-subsampled Contourlet transform and pulse coupled neural network in image fusion, and proposes a new remote sensing image fusion method based on IHS, NSCT and PCNN theory.

[0011] The technical solution adopted by the present invention is:

[0012] A remote sensing image fusion method based on HIS, NSCT and PCNN includes the following steps:

[0013] Step 1: geo-register the remote sensing images to be fused so that the geographic coordinate systems of the images to be fused are consistent;

[0014] Step 2: Decompose the registered remote sensing images A and B to be fused at different scales and directions by NSCT respectively, and obtain the coefficients of NSCT decomposition at different scales and directions for the remote sensing images to be fused, and obtain the NSCT decomposition coefficients and

[0015] In the formula, In the formula, it represents the NSCT high-frequency sub-generation coefficient of the k-th direction of the remote sensing image A to be fused at the j-th scale decomposition; In the figure, it represents the low-frequency sub-generation coefficient of the k-th direction of the remote sensing image B to be fused at the j-th scale decomposition; D A represents the low-frequency coefficient of the image A to be fused under NSCT decomposition, D A represents the high-frequency coefficients of the image B to be fused under NSCT decomposition, j represents the decomposition scale of the non-subsampled shearlet transform, j = 1, 2, 3 ... j, and j is the maximum decomposition scale, K represents the number of decomposition directions under the j decomposition scale, satisfying k = 2 j ;

[0016] Step 3: Use high-frequency coefficients and low-frequency fusion rules to decompose the NSCT coefficients of the fused remote sensing image. and Perform fusion and obtain the fused coefficients;

[0017] Step 4: Perform NSCT inverse transformation on the fused coefficients to obtain the fused intensity component I′;

[0018] Step 5: Perform IHS inverse transformation on the obtained intensity component I′ and the H and S components of the multispectral image to obtain the final fused image.

[0019] Furthermore, in step 3, the low-frequency coefficient fusion uses the Gaussian membership function as the adaptive weighting function. The calculation is centered on the (i, j) pixel point, and the neighborhood box takes a 3×3 window area for calculation. Its mathematical expression is:

[0020]

[0021] η 1 (i,j)=1-η 0 (i,j)

[0022] Where: L(i,j) is the low-frequency subband coefficient; u and σ are the mean and variance of the low-frequency subband image of a source image respectively; k is the Gaussian function adjustment parameter, which is the extreme value obtained by the control variable method as an empirical value, and the value is k = 0.8;

[0023] Therefore, the fusion rule of low-frequency coefficients is

[0024] L F (i,j)=η 0 (i,j)L A (i,j)+η 1 (i,j)L B (i,j)

[0025] Where: L A(i, j) is the low-frequency subband coefficient of image A; L B (i, j) is the low-frequency subband coefficient of image B; L F (i, j) represents the fused low-frequency subband coefficients;

[0026] The high frequency coefficient fusion adopts the improved local Laplace energy SML and is used as the input of the PCNN model. The SML is defined as:

[0027]

[0028] Where (2M+1,2N+1) is the window size, ML j,l (x,y) represents the discrete form of the Laplacian operator, defined as:

[0029] ML j,l (x,y)=|2H j,l (x,y)-H j,l (xs,y)-H j,l (x+s,y)|+|2H j,l (x,y)-H j,l (x,ys)-H j,l (x,y+s)|

[0030] Where s is the coefficient or variable distance between pixels;

[0031] For an M×N image block, the improved spatial frequency MSF is defined as follows:

[0032]

[0033]

[0034]

[0035]

[0036] The high-frequency coefficients of the full-color image and the I component after NSCT decomposition are calculated in the whole image respectively in the 3×3 domain of the Laplace energy and the improved spatial frequency, which are used as the input excitation signal I of the PCNN respectively. ij (n) and link strength β, the fused high-frequency coefficients are determined by comparing the firing outputs of PCNN, and the rules are as follows:

[0037]

[0038] Where: D A , D B and D F are the high-frequency subband coefficients of the full-color image, I-component image, and fused image respectively; YA , Y B They are the ignition outputs of the two images respectively.

[0039] The beneficial effect of the present invention is that the present invention not only improves the spatial resolution of the image, but also can better maintain the spectral information of the original multi-spectral bands. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is the flowchart of remote sensing image fusion based on NSCT and PCNN. DETAILED DESCRIPTION

[0041] The present invention will be further described below in conjunction with the accompanying drawings.

[0042] like Figure 1 As shown, the present invention is a remote sensing image fusion method based on HIS, NSCT and PCNN, comprising the following steps:

[0043] Step 1: Geo-reference the remote sensing images to be fused so that the geographic coordinate systems of the images to be fused are consistent.

[0044] Step 2: Decompose the registered remote sensing images A and B to be fused at different scales and directions by NSCT respectively, and obtain the coefficients of NSCT decomposition at different scales and directions for the remote sensing images to be fused, and obtain the NSCT decomposition coefficients and

[0045] In the formula, In the formula, it represents the NSCT high-frequency sub-generation coefficient of the k-th direction of the remote sensing image A to be fused at the j-th scale decomposition; In the figure, it represents the low-frequency sub-generation coefficient of the k-th direction of the remote sensing image B to be fused at the j-th scale decomposition; D A represents the low-frequency coefficient of the image A to be fused under NSCT decomposition, D A represents the high-frequency coefficients of the image B to be fused under NSCT decomposition, j represents the decomposition scale of the non-subsampled shearlet transform, j = 1, 2, 3 ... j, and j is the maximum decomposition scale, K represents the number of decomposition directions under the j decomposition scale, satisfying k = 2 j .

[0046] Step 3: Use high-frequency coefficients and low-frequency fusion rules to decompose the NSCT coefficients of the fused remote sensing image. and The fusion is performed to obtain the fused coefficients.

[0047] The low-frequency coefficient fusion uses the Gaussian membership function as the adaptive weighting function. The calculation is centered on the (i, j) pixel point, and the neighborhood box uses a 3×3 window area for calculation. Its mathematical expression is:

[0048]

[0049] η 1 (i,j)=1-η 0 (i,j)

[0050] Where: L(i,j) is the low-frequency subband coefficient; u and σ are the mean and variance of the low-frequency subband image of a source image respectively; k is the Gaussian function adjustment parameter, which is the extreme value obtained by the control variable method as an empirical value, and the value is k = 0.8;

[0051] Therefore, the fusion rule of low-frequency coefficients is

[0052] L F (i,j)=η 0 (i,j)L A (i,j)+η 1 (i,j)L B (i,j)

[0053] Where: L A (i, j) is the low-frequency subband coefficient of image A; L B (i, j) is the low-frequency subband coefficient of image B; L F (i, j) represents the fused low-frequency subband coefficients.

[0054] The high frequency coefficient fusion adopts the improved local Laplace energy SML and is used as the input of the PCNN model. The SML is defined as:

[0055]

[0056] Where (2M+1,2N+1) is the window size, ML j,l (x,y) represents the discrete form of the Laplacian operator, defined as:

[0057] ML j,l (x,y)=|2H j,l (x,y)-H j,l (xs,y)-H j,l (x+s,y)|+|2H j,l (x,y)-H j,l (x,ys)-H j,l (x,y+s)|

[0058] Here, s is the coefficient or the variable distance between pixels.

[0059] For the high-frequency coefficient fusion rule, the high-frequency subband of the image reflects the detail information of the image. The traditional fusion rule only considers the characteristics of a single or regional pixel, and uses features such as gradient, variance, absolute maximum and regional energy sum, which often loses detail information. In order to better fuse the feature information of the source image, high-frequency coefficient fusion is performed based on the PCNN model. The traditional fusion rule based on the PCNN model usually takes the grayscale value of a single pixel as feedback input, does not consider the correlation between adjacent pixels of the image, and easily introduces noise into the fused image. The local Laplace energy sum (SML) characterizes the degree of change between adjacent pixels of the image and can be used to analyze the edge information of the image. This paper uses the improved local Laplace energy sum of high-frequency coefficients as the input of the PCNN model. SML is defined as:

[0060]

[0061] Where (2M+1,2N+1) is the window size, ML j,l (x,y) represents the discrete form of the Laplacian operator, defined as

[0062] ML j,l (x,y)=|2H j,l (x,y)-H j,l (xs,y)-H j,l (x+s,y)|+|2H j,l (x,y)-H j,l (x,ys)-H j,l (x,y+s)|

[0063] Here, s is the coefficient or the variable distance between pixels.

[0064] The link strength β reflects the change between coefficients and can adjust the weight of the link channel in the internal activity item and the firing period of the central neuron. It usually takes values ​​between [0 and 1] and is crucial in the image fusion process. In the traditional PCNN model, it is determined through experience and experiments that the same β value is used for each neuron. For the human visual system, it is unreasonable to set the connection coefficients of all neurons to the same value. The connection coefficient of an actual neuron should be related to the environment in which the neuron is located, and the β value should vary with different input neurons.

[0065] Generally, the clarity of pixels is selected to adaptively adjust the connection strength of PCNN neurons, such as spatial frequency, directional information, standard deviation, etc. According to the characteristics of the human visual system, the spatial frequency (SF) reflects the information of the characteristics and details of the local area of ​​the image, but the original SF lacks the directional information present in the image, which will lead to the loss of important details of the image. Based on the original spatial frequency SF, the improved spatial frequency (MSF) adds the gradient energy in the two diagonal (DF) directions in addition to calculating the gradient energy of the image in the horizontal (RF) and vertical (CF) directions. It can better extract image information and serve as a measure of image clarity or activity energy level. This paper uses the improved spatial frequency as the connection strength of PCNN. For an M×N image block, the MSF is defined as follows:

[0066]

[0067]

[0068]

[0069]

[0070] The high-frequency coefficients of the full-color image and the I component after NSCT decomposition are calculated in the whole image respectively in the 3×3 domain of the Laplace energy and the improved spatial frequency, which are used as the input excitation signal I of the PCNN respectively. ij (n) and link strength β, the fused high-frequency coefficients are determined by comparing the firing outputs of PCNN, and the rules are as follows:

[0071]

[0072] D A , D B and D F are the high-frequency subband coefficients of the full-color image, I-component image, and fused image respectively; Y A , Y B They are the ignition outputs of the two images respectively.

[0073] Step 4: Perform NSCT inverse transformation on the fused coefficients to obtain the fused intensity component I′;

[0074] Step 5: Perform IHS inverse transformation on the obtained intensity component I′ and the H and S components of the multispectral image to obtain the final fused image.

Claims

1. A remote sensing image fusion method based on HIS, NSCT and PCNN, characterized in that: The following steps are involved: Step 1: geo-register the remote sensing images to be fused so that the geographic coordinate systems of the images to be fused are consistent; Step 2: Decompose the registered remote sensing images A and B to be fused at different scales and directions by NSCT respectively, and obtain the coefficients of NSCT decomposition at different scales and directions for the remote sensing images to be fused, and obtain the NSCT decomposition coefficients and ; In the formula, In the formula, it represents the NSCT high-frequency sub-generation coefficient of the k-th direction of the remote sensing image A to be fused at the j-th scale decomposition; In the figure, it represents the low-frequency sub-generation coefficient of the k-th direction of the remote sensing image B to be fused at the j-th scale decomposition; Represents the low-frequency coefficients of the image A to be fused under NSCT decomposition, represents the high-frequency coefficients of the image B to be fused under NSCT decomposition, j represents the decomposition scale of the non-subsampled shearlet transform, and k represents the number of decomposition directions under the j decomposition scale, satisfying k=2 j ; Step 3: Use high-frequency coefficients and low-frequency fusion rules to decompose the NSCT coefficients of the fused remote sensing image. and Perform fusion and obtain the fused coefficients; Among them, the low-frequency coefficient fusion uses the Gaussian membership function as the adaptive weighting function to calculate The pixel point is the center, and the neighborhood box is The window area is calculated, and its mathematical expression is: ； ； Where: is the low frequency subband coefficient; and are the mean and variance of the low-frequency subband image of a source image respectively; The parameters for adjusting the Gaussian function are the extreme values ​​obtained by the control variable method as empirical values, and the values ​​are ; Therefore, the fusion rule of low-frequency coefficients is ； Where: is the low-frequency subband coefficient of image A; is the low-frequency subband coefficient of image B; Represents the fused low-frequency subband coefficients; The high frequency coefficient fusion adopts the improved local Laplace energy SML and is used as the input of the PCNN model. The SML is defined as: ； In the formula, is the window size, represents the Laplace operator in discrete form, defined as: ； in, is the coefficient or variable distance between pixels; for The improved spatial frequency MSF of the image block is defined as follows: ； ； ； ； The high-frequency coefficients of the full-color image and the I component after NSCT decomposition are calculated on the entire image. The Laplace energy and improved spatial frequency of the neighborhood are used as the input excitation signal and link strength of PCNN respectively. The fused high-frequency coefficients are determined by comparing the ignition output of PCNN. The rules are as follows: ； Where: , and They are the high frequency subband coefficients of the full color image, I component image and fused image respectively; , They are the ignition outputs of 2 images respectively; Step 4: Perform NSCT inverse transformation on the fused coefficients to obtain the fused intensity components ; Step 5: Get the intensity component Perform IHS inverse transformation with the H and S components of the multispectral image to obtain the final fused image.

Images