Hyperspectral image sub-pixel location method combined with edge preservation

Abstract

The present application relates to a hyperspectral image sub-pixel positioning method with edge preservation, comprising: S1, using a Gaussian blur kernel and an original hyperspectral image to perform non-blind deconvolution to reduce the point spread function effect in the original hyperspectral image; S2, using a domain transform recursive filter to filter the result of step S1 to play a role in edge preservation and noise reduction; S3, obtaining an abundance image through a linear spectral unmixing technique; S4, using an interpolation method to up-sample the abundance image to obtain soft class values of each type of sub-pixel; S5, using a class assignment strategy to assign a class label to each sub-pixel to obtain the final sub-pixel positioning result. The present application can effectively process the point spread function effect and the noise such as texture details in the image, thereby improving the sub-pixel positioning accuracy.

Description

Technical Field

[0001] This invention relates to a method for sub-pixel localization of hyperspectral images that combines edge preservation, and belongs to the field of hyperspectral image sub-pixel localization technology. Background Technology

[0002] Due to the limitations of the instantaneous field of view of hyperspectral sensors, hyperspectral remote sensing images have low spatial resolution. Furthermore, the complexity of the actual ground conditions leads to a large number of mixed pixels. Traditional classification methods assign category labels to pixels individually, which often fails to reflect the true ground conditions. Spectral unmixing techniques are used to obtain the endmember types and abundances within mixed pixels, but they cannot clearly define the spatial distribution of various land features. Subpixel mapping (SPM) aims to determine the spatial location of different land features within each pixel using specific algorithms or models. Traditional two-step SPM directly processes the original hyperspectral image. First, it performs spectral unmixing to obtain the abundance matrices for each category. Then, it upsamples each abundance matrix at a certain scale to obtain soft class values ​​for each category at the subpixel scale. Finally, it uses a category assignment strategy to obtain the final subpixel mapping result. However, the two-step SPM does not take into account the blurring of the original hyperspectral image caused by the point spread function effect of the hyperspectral remote sensing imaging system, as well as the inevitable impact of irrelevant texture details caused by clouds and fog on the sub-pixel localization accuracy during long-distance measurements. To solve this problem, this paper proposes a hyperspectral sub-pixel localization method that combines edge-preserving (EP). Summary of the Invention

[0003] To overcome the shortcomings of existing research, this invention provides a hyperspectral image sub-pixel localization method that combines edge preservation to alleviate the phenomenon that the original hyperspectral image is susceptible to the point spread function effect from the imaging system, while also reducing irrelevant texture details on the original hyperspectral image.

[0004] The specific steps of a hyperspectral image sub-pixel localization method that combines edge preservation are as follows:

[0005] S1: Use a Gaussian blur kernel and the original hyperspectral image to perform non-blind deconvolution to reduce the point spread function effect in the original hyperspectral image;

[0006] S2: Use domain transform recursive filter filtering to preserve edges and reduce noise;

[0007] S3: Abundance images are obtained through spectral unmixing techniques;

[0008] S4: Upsample the abundance image to obtain soft class values ​​for each sub-pixel;

[0009] S5: Use a category assignment strategy to assign labels to each sub-pixel to obtain the sub-pixel localization results.

[0010] As a preferred embodiment, in step S1, the hyperspectral imaging model affected by the point spread function effect is expressed as follows:

[0011]

[0012] in, The blurred image is obtained by a hyperspectral remote sensing sensor. The fuzzy kernel corresponding to the point spread function. For the desired clear image, Indicates additive noise. Represents the convolution operation, where Indicates spatial resolution. Indicates the number of bands. This represents the order of the convolution kernel corresponding to the point spread function. The process of mitigating the effects of the point spread function on hyperspectral images involves using a blurred image and a blur kernel to obtain a sharp image.

[0013] set up , , , This indicates transpose. Assume... If the noise distribution satisfies Poisson, then:

[0014]

[0015] This formula represents the convolution of the grayscale images of each band of the hyperspectral image acquired by the imaging system and the image to be sharpened using different kernels. The result of the convolution follows a Poisson distribution. Therefore, the image to be sharpened can be obtained. The likelihood probability function is expressed as:

[0016]

[0017] in, To represent factorial operation, The representation is as follows:

[0018]

[0019] in, This represents the standard deviation of the Gaussian function. Indicated by The spatial extent of the central local window, for Taking the logarithm of the likelihood function yields its energy function, as shown below;

[0020]

[0021] That is, to find its maximum likelihood solution. for:

[0022]

[0023] Let the point spread function correspond to the normalized fuzzy kernel satisfy Taking the derivative of the above equation, we obtain the iterative expression:

[0024]

[0025] in, for The adjoint matrix, The number of iterations is generally chosen. Using this as an initial condition, iterative processes are performed, and by setting an appropriate number of iterations, an approximate value of the desired image can be obtained. .

[0026] As a preferred option, in step S2, the clear image obtained by non-blind deconvolution... Filtering is performed using a domain transform recursive filter, specifically implemented as follows:

[0027] The clear hyperspectral image obtained in step S1 is unfolded according to spatial dimensions, i.e. , , will any band Viewing it as a two-dimensional grayscale image, we convert it into multiple sets of one-dimensional signals along the horizontal and vertical directions of the image. For any given one-dimensional signal... Its domain transformation can be defined as:

[0028]

[0029] in, This represents the value of a one-dimensional signal at the initial moment. and These are two constants that control the smoothness of the filter. and Indicates the first Time and the The value of a one-dimensional signal at time t, Represents absolute value. Let the signal strength be the value after the transform domain transformation. Then, the recursive edge-preserving filter can be defined in the transform domain as:

[0030]

[0031] in, For feedback coefficients, This represents the distance between two adjacent signals in the transform domain. Indicates the first The intensity value of a one-dimensional signal at any given time. and They represent the first and the The value of the one-dimensional signal under domain transformation recursive filtering at any given time is determined by: unfolding the two-dimensional image into a one-dimensional signal by rows, performing domain transformation recursive filtering, and then recombining it into a two-dimensional signal. This process is repeated three times to obtain the filtered result.

[0032] As a preferred embodiment, in step S3, the spectral unmixing technique specifically involves: processing the image filtered by the domain transform recursive filter using a spectral unmixing method based on a linear unmixing model to obtain abundance images for each category. The linear unmixing model is defined as follows: a linear unmixing model refers to a linear relationship between the endmembers representing land cover categories within a mixed pixel and their proportion. Therefore, let... For the hyperspectral data to be unmixed, For the number of bands, For the number of pixels, For endmember matrices, The number of endmembers, This is the abundance coefficient matrix. Let be the noise matrix, then we have,

[0033]

[0034] in, Indicates 1 row A vector of all 1s in a column. Indicates 1 row The column is a vector of all 1s, and the two constraints represent the non-negativity constraint of the abundance coefficient matrix and the constraint that the sum of the abundance coefficients of each cell is 1.

[0035] As a preferred option, in step S4, the radial basis function interpolation used in the upsampling method processes the abundance images of various land cover categories obtained in step S3 through radial basis function interpolation to obtain the soft class values ​​of each category for each sub-pixel.

[0036] As a preferred option, in step S5, the category allocation strategy selected is a class-based category allocation strategy (UOC), specifically: assuming the scene contains... Land-like features, Indicates the first One pixel, Indicates the first The first of the pixels The abundance coefficient of land-like features is then:

[0037]

[0038] Here, the formula indicates that the sum of the abundance coefficients of each pixel obtained in step S3 is one, and the constraint condition indicates that the abundance coefficients of each land cover in that pixel are non-negative. Then, the pixel passes the scale factor... Upsampling is performed to divide the pixel into The sub-pixel, then the first sub-pixel within that pixel Number of sub-pixels occupied by land features It can be obtained through the following formula:

[0039]

[0040] in, This indicates rounding. In other words, during category assignment, each sub-pixel must be assigned to only one type of land cover, and the number of sub-pixels corresponding to each type of land cover within each pixel must be proportional to their abundance coefficient.

[0041] Finally, using the soft class values ​​of each sub-pixel obtained in step S4, and based on the global Moran index, which measures spatial correlation, the order of land cover category allocation is determined from high to low. Then, combining the number of sub-pixels of each class within each pixel, each sub-pixel is assigned a category according to the category allocation order and the soft class value from large to small. After the allocation is completed, the final sub-pixel positioning result can be obtained.

[0042] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0043] Traditional two-step sub-pixel localization methods do not consider the limitations of the point spread function (PSF) effect on sub-pixel localization accuracy, directly using the unprocessed raw hyperspectral image for sub-pixel localization. This invention, however, uses non-blind deconvolution to process the raw hyperspectral image, effectively mitigating the impact of the PSF effect on sub-pixel localization accuracy. For irrelevant texture details (such as artifacts) in the hyperspectral image, this invention uses a domain transform recursive filter for spatial dimension filtering to reduce irrelevant texture details and preserve edges, thereby improving sub-pixel localization accuracy. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a flowchart of a hyperspectral image sub-pixel localization method that combines edge preservation, according to an embodiment of the present invention.

[0046] Figure 2 The images are high-resolution grayscale images and low-resolution grayscale images obtained by downsampling from the experimental dataset of this invention, (a) high-resolution grayscale image, (b) low-resolution grayscale image.

[0047] Figure 3 The images show the experimental part of the present invention, including a reference classification image and a sub-pixel localization result image obtained using the method of the present invention. (a) Reference classification image, (b) Localization result image. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] Example:

[0050] The present invention proposes a method for sub-pixel localization of hyperspectral images that combines edge preservation, and the flowchart of its implementation is shown below. Figure 1 As shown. For the original low-resolution hyperspectral image , Represents the real number field. Indicates spatial resolution. Let the number of bands be denoted by and the scale factor be . The spatial resolution of the final sub-pixel localization result image is then... .

[0051] The specific steps of the method in this embodiment are as follows:

[0052] S1: Use a Gaussian blur kernel and the original hyperspectral image to perform non-blind deconvolution to weaken the point spread function effect in the original hyperspectral image;

[0053] S2: Use domain transform recursive filter filtering to preserve edges and reduce noise;

[0054] S3: Abundance images are obtained through spectral unmixing techniques;

[0055] S4: Upsample the abundance image to obtain soft class values ​​for each sub-pixel;

[0056] S5: Use a category assignment strategy to assign labels to each sub-pixel to obtain the sub-pixel localization results.

[0057] A more detailed description of each step is as follows:

[0058] In step S1, the hyperspectral imaging model affected by the point spread function effect is expressed as:

[0059]

[0060] in, The blurred image is obtained by a hyperspectral remote sensing sensor. The fuzzy kernel corresponding to the point spread function. For the desired clear image, Indicates additive noise. Represents the convolution operation, where Indicates spatial resolution. Indicates the number of bands. This represents the order of the convolution kernel corresponding to the point spread function. The process of mitigating the effects of the point spread function on hyperspectral images involves using a blurred image and a blur kernel to obtain a sharp image.

[0061] set up , , , This indicates transpose. Assume... If the noise distribution satisfies Poisson, then:

[0062]

[0063] This formula represents the convolution of the grayscale images of each band of the hyperspectral image acquired by the imaging system and the image to be sharpened using different kernels. The result of the convolution follows a Poisson distribution. Therefore, the image to be sharpened can be obtained. The likelihood probability function is expressed as:

[0064]

[0065] in, To represent factorial operation, The representation is as follows:

[0066]

[0067] in, This represents the standard deviation of the Gaussian function. Indicated by The spatial extent of the central local window, for Taking the logarithm of the likelihood function yields its energy function, as shown below;

[0068]

[0069] That is, to find its maximum likelihood solution. for:

[0070]

[0071] in, This indicates that when the objective function reaches its minimum value... The value of is determined by the point spread function corresponding to the normalized fuzzy kernel. satisfy Taking the derivative of the above equation, we obtain the iterative expression:

[0072]

[0073] in, for The adjoint matrix, The number of iterations is generally chosen. Using this as an initial condition, iterative processes are performed, and by setting an appropriate number of iterations, an approximate value of the desired image can be obtained. Because the ringing effect becomes more severe with increasing iterations, this method uses only one iteration to obtain an approximate value of the desired image. .

[0074] In step S2, the clear image obtained by non-blind deconvolution Filtering is performed using a domain transform recursive filter, specifically implemented as follows:

[0075] The clear hyperspectral image obtained in S1 is unfolded according to spatial dimensions, i.e. , , will any band Viewing it as a two-dimensional grayscale image, we convert it into multiple sets of one-dimensional signals along the horizontal and vertical directions of the image. For any given one-dimensional signal... Its domain transformation can be defined as:

[0076]

[0077] in, This represents the value of a one-dimensional signal at the initial moment. and These are two constants that control the smoothness of the filter. and Indicates the first Time and the The value of a one-dimensional signal at time t, Represents absolute value. Let the signal strength be the value after the transform domain transformation. Then, the recursive edge-preserving filter can be defined in the transform domain as:

[0078]

[0079] in, For feedback coefficients, This represents the distance between two adjacent signals in the transform domain. Indicates the first The intensity value of a one-dimensional signal at any given time. and They represent the first and the The value of the one-dimensional signal under domain transformation recursive filtering at any given time is determined by: unfolding the two-dimensional image into a one-dimensional signal by rows, performing domain transformation recursive filtering, and then recombining it into a two-dimensional signal. This process is repeated three times to obtain the filtered result.

[0080] In step S3, the spectral unmixing technique specifically involves processing the image filtered by the domain transform recursive filter using a spectral unmixing method based on a linear unmixing model to obtain abundance images for each category. The linear unmixing model is defined as follows: a linear unmixing model refers to a linear relationship between the endmembers representing land cover categories within a mixed pixel and their proportion. Let the result after step S2 be... , For the number of bands, For the number of pixels, For endmember matrices, The number of endmembers, This is the abundance coefficient matrix. Let be the noise matrix, then we have,

[0081]

[0082] in, Indicates 1 row A vector of all 1s in a column. Indicates 1 row The column is a vector of all 1s, and the two constraints represent the non-negativity constraint of the abundance coefficient matrix and the constraint that the sum of the abundance coefficients of each cell is 1.

[0083] In step S4, the radial basis function interpolation used in the upsampling method is used to process the abundance images of various land cover categories obtained in step S3 to obtain the soft class values ​​of each sub-pixel for each category.

[0084] In step S5, the category assignment strategy selected is a class-based category assignment strategy (UOC), specifically: assuming the scene contains... Land-like features, Indicates the first One pixel, Indicates the first The first of the pixels The abundance coefficient of land-like features is then:

[0085]

[0086] Here, the formula indicates that the sum of the abundance coefficients of each pixel obtained by S3 is one, and the constraint condition indicates that the abundance coefficients of each land cover in that pixel are non-negative. Then, the pixel passes the scale factor... Upsampling is performed to divide the pixel into The sub-pixel, then the first sub-pixel within that pixel Number of sub-pixels occupied by land features It can be obtained through the following formula:

[0087]

[0088] in, This indicates rounding. In other words, during category assignment, each sub-pixel must be assigned to only one type of land cover, and the number of sub-pixels corresponding to each type of land cover within each pixel must be proportional to their abundance coefficient.

[0089] Finally, using the soft class values ​​of each sub-pixel obtained in step S4, and based on the global Moran index, which measures spatial correlation, the order of land cover category allocation is determined from high to low. Then, combining the number of sub-pixels of each class within each pixel, each sub-pixel is assigned a category according to the category allocation order and the soft class value from large to small. After the allocation is completed, the final sub-pixel positioning result can be obtained.

[0090] The advantages of the method of the present invention will be explained below through specific experimental data:

[0091] The Jasper Ridge hyperspectral dataset was chosen as the validation dataset for experiments. The Jasper Ridge hyperspectral dataset has 512×614 pixels, containing 224 bands, with a spectral resolution of 9.46 nm. After removing noise bands, 198 bands remain. Due to the complexity of its actual ground conditions, 100×100 pixels were cropped for scale factors of 2 and 4, and 99×99 pixels were cropped for scale factor 3. First, the dataset was downsampled using a downsampling matrix to obtain low-resolution images, which were then used as the original low-resolution hyperspectral images for sub-pixel localization. With a scale factor of 2, the low-resolution image was 50×50; with a scale factor of 3, it was 33×33; and with a scale factor of 4, it was 25×25. For example, with a scale factor of 2, the high-resolution image is shown below. Figure 2 (b), its corresponding low-resolution image is as follows Figure 2 As shown in (b); a reference classification map is obtained by applying a support vector machine classification algorithm to the high-resolution image, as follows: Figure 3 As shown in (a), the categories are 4, including trees, water, soil, and roads; the sub-pixel localization result obtained by the method proposed in this embodiment is shown in the figure below. Figure 3As shown in (b). To reflect the final positioning effect, this invention uses four indicators for evaluation: producer accuracy (PCC), overall accuracy (OA), average accuracy (AA), and Kappa coefficient (KC). Table 1 compares the producer accuracy, overall accuracy, average accuracy, and Kappa coefficient for scale factors of 2, 3, and 4 in this embodiment. The better performers in the table are indicated in bold. RBF-EP corresponds to the method proposed in this embodiment.

[0092] Table 1. Performance comparison of the embodiments at various scales

[0093]

[0094] As can be seen from the table, the method proposed in this embodiment outperforms the edge-preserving sub-pixel localization method without the proposed method in OA, AA, and KC. Furthermore, in terms of PCC, it also improves in the other three categories except for soil, which proves the effectiveness of the method proposed in this embodiment.

[0095] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and these variations still fall within the protection scope of the present invention.

Claims

1. A method for sub-pixel localization of hyperspectral images that combines edge preservation, characterized in that: Includes the following steps: S1: Use a Gaussian blur kernel and the original hyperspectral image to perform non-blind deconvolution to weaken the point spread function effect in the original hyperspectral image; In step S1, the hyperspectral imaging model affected by the point spread function effect is expressed as: ； in, The blurred image is obtained by a hyperspectral remote sensing sensor. The fuzzy kernel corresponding to the point spread function. For the desired clear image, Indicates additive noise. Represents the convolution operation, where Indicates spatial resolution. Indicates the number of bands. This represents the convolution kernel order corresponding to the point spread function; set up , , Assuming To achieve a clear image that satisfies the Poisson noise distribution. The likelihood probability function is expressed as: ； in, To represent factorial operation, Using a Gaussian blur kernel, the maximum likelihood solution occurs when the partial derivative of the likelihood function is zero, thus yielding the iterative formula: ； in, for The conjugate transpose of the matrix. For the number of iterations, choose Using this as an initial condition, iterative processes are performed, and by setting an appropriate number of iterations, an approximate value of the desired image can be obtained. ; S2: Use domain transform recursive filter filtering to preserve edges and reduce noise; S3: Abundance images are obtained through spectral unmixing techniques; S4: Upsample the abundance image to obtain soft class values ​​for each sub-pixel; S5: Use a category assignment strategy to assign labels to each sub-pixel to obtain the sub-pixel localization results.

2. The hyperspectral image sub-pixel localization method combined with edge preservation according to claim 1, characterized in that: In step S2, the clear image obtained by non-blind deconvolution Filtering is performed using a domain transform recursive filter, specifically implemented as follows: Let , , will any band Viewing it as a two-dimensional grayscale image, we convert it into multiple sets of one-dimensional signals along the horizontal and vertical directions of the image, respectively. For any given one-dimensional signal... Its domain transformation can be defined as: ； in, This represents the value of a one-dimensional signal at the initial moment. and These are two constants that control the smoothness of the filter. and Indicates the first Time and the The value of a one-dimensional signal at time t, Represents absolute value. The recursive edge-preserving filter can be defined in the transform domain as follows: (This represents the signal strength after the transform domain transformation.) ； in, For feedback coefficients, This represents the distance between two adjacent signals in the transform domain. Indicates the first The intensity value of a one-dimensional signal at any given time. and They represent the first and the The value of the one-dimensional signal domain transformation recursive filter at any given time.

3. The hyperspectral image sub-pixel localization method combining edge preservation according to claim 1, characterized in that: In step S3, the spectral unmixing technique specifically involves processing the image filtered by the domain transform recursive filter using a spectral unmixing method based on a linear unmixing model to obtain abundance images for each category. The linear unmixing model is defined as follows: a linear unmixing model refers to a linear relationship between the endmembers representing land cover categories within a mixed pixel and their proportion. Therefore, let... The hyperspectral data to be unmixed For endmember matrices, For abundance matrix, If the noise matrix is ​​, then we have ； Among them, the abundance matrix It satisfies the non-negativity constraint and the sum of 1 constraint.

4. The hyperspectral image sub-pixel localization method combining edge preservation according to claim 1, characterized in that: In step S4, the abundance images of various land cover categories obtained in step S3 are processed by an upsampling method to obtain the soft class value of each category for each sub-pixel.

5. The hyperspectral image sub-pixel localization method combining edge preservation according to claim 1, characterized in that: In step S5, the category allocation strategy selected is a category allocation strategy based on classes as units, specifically: Assuming the scenario contains Land-like features, Indicates the first One pixel, Indicates the first The first of the pixels The abundance coefficient of land features indicates that the pixel is affected by the scale factor. Upsampling is performed to divide the pixel into The sub-pixel, then the first sub-pixel within that pixel Number of sub-pixels occupied by land features It can be obtained through the following formula: ； in, This indicates rounding operations.

Images