Hyperspectral image band selection method based on deep multi-level representation learning

Abstract

The present invention discloses a hyperspectral image band selection method based on deep multi-level representation learning. The method embeds a self-expression model into different layers of a deep convolutional autoencoder, considers the low-level and high-level information of the hyperspectral image by introducing multi-level representation learning, and learns a subspace clustering representation with richer information. A new auxiliary self-supervision task is constructed for the multi-level representation learning to further improve the representation ability of the encoder. A fusion module is then designed to fuse the multi-scale information extracted from different layers of the multi-level representation learning to learn a more discriminative self-expression coefficient matrix, and maximum entropy regularization is added to the self-expression coefficient matrix to strengthen the connectivity within each subspace so that the elements of the same subspace are evenly and densely distributed. This method can be used as an effective means of hyperspectral image band selection based on self-supervised deep multi-level representation learning fused with maximum entropy subspace clustering, and has important application value in hyperspectral image classification, target detection, and other aspects.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral image band selection, and in particular to a hyperspectral image band selection method based on self-supervised deep multi-level representation learning fused with maximum entropy subspace clustering. Background Art

[0002] Hyperspectral images typically contain hundreds of extremely narrow, continuous bands with fine resolution. Due to the availability of rich spatial and spectral information, techniques for hyperspectral image processing have become one of the most promising technologies in numerous fields, including precision agriculture, food safety, environmental monitoring, and target detection. However, the strong band correlation and high dimensionality of hyperspectral images lead to extensive redundancy, heavy computational and storage burdens, and the curse of dimensionality. Therefore, dimensionality reduction is of great significance for the processing and analysis of hyperspectral images. Band selection is an effective dimensionality reduction technique for hyperspectral images.

[0003] In recent years, self-representation-based subspace clustering models have achieved remarkable results in unsupervised band selection. This is because, within this self-representation-based framework, they accurately learn the clustering structure of bands in a low-dimensional subspace, making them robust to noise and outliers and effectively clustering high-dimensional data. Such methods typically cluster the raw hyperspectral image in a linear subspace, failing to capture the nonlinear characteristics of hyperspectral images. Furthermore, they do not consider spatial information, which prevents them from achieving excellent clustering performance. Furthermore, deep learning-based subspace clustering methods have been introduced for band selection, enabling them to learn the nonlinear characteristics of hyperspectral images in an end-to-end trainable manner. Although these methods consider the inherent spatial information of the band images, they also have certain limitations. First, because the representation learning in these methods is embedded in a deep convolutional autoencoder, the model's representational power lacks effective supervision. Second, these methods fail to consider the subspace clustering representation of both low-level and high-level information in the input band image, neglecting the important multi-scale information embedded in different layers of the deep convolutional autoencoder and wasting a large amount of deep convolutional features that could be beneficial for clustering. In addition, such methods ignore the connectivity within the subspace, which hinders the further improvement of clustering performance. Summary of the Invention

[0004] In view of the problems existing in the prior art, the present invention discloses a hyperspectral image band selection method based on deep multi-level representation learning, which specifically includes the following steps:

[0005] A deep stacked convolutional autoencoder is used as the backbone network to learn the nonlinear features of hyperspectral images and extract the spatial information of hyperspectral images;

[0006] The self-expression model is embedded in different layers of the deep stacked convolutional autoencoder, multi-level representation learning is performed on the backbone network, and low-level and high-level information of the hyperspectral image are combined to generate multiple sets of discrimination matrices and consistency matrices.

[0007] A multi-level representation learning method is used to construct a self-supervised auxiliary task, and self-supervise the feature representation of the multi-level representation learning;

[0008] The multiple sets of discriminative matrices and consistent matrices obtained through multi-level representation learning are superimposed along the channel dimension to obtain a stacked matrix. The stacked matrix is subjected to channel fusion learning using convolution kernels to obtain a more discriminative self-expression coefficient matrix.

[0009] Adding the self-expression coefficient matrix to the maximum entropy regularization constraint;

[0010] The hyperspectral band image and the selected number of bands are input into the backbone network, the total loss function of the backbone network is calculated, the backbone network is trained by the gradient method, and the backbone network parameters are updated using the standard back-propagation technique until the backbone network stops training, thereby obtaining the self-expression coefficient matrix of the backbone network;

[0011] An affinity matrix is constructed based on the self-expression coefficient matrix, and the clustering results are obtained by spectral clustering. The average band in each category of the hyperspectral image is obtained based on the clustering results.

[0012] The average band is used as the cluster center, and the distance from each band to the cluster center is calculated, wherein the band closest to the cluster center is the selected band subset.

[0013] Furthermore, The original 3D hyperspectral cube is W×H pixels and B band. The goal of band selection is to select a subset of bands.

[0014]

[0015] The encoder is defined as z = E(x; θ e ), where z represents the latent representation or bottleneck, x and θ e are the parameters of the data points and encoder respectively, and the decoder is defined as in, and θ d represent the reconstruction of the input data and the parameters of the decoder respectively. The loss function of the deep stacked convolutional autoencoder is:

[0016]

[0017] Furthermore, suppose that the hyperspectral B band images are located in the union of n subspaces In the And satisfy d1+d2+…+d n =B;

[0018] When embedding self-expression models into different layers of deep stacked convolutional autoencoders and learning multi-level representations for the backbone network: Definition As the consistency matrix and As a discriminative matrix, the loss function of multi-level representation learning is:

[0019]

[0020] in, represents the output or latent representation matrix of the encoder, and m is the depth feature dimension.

[0021] Furthermore, when constructing self-supervised auxiliary tasks: given an input for multi-level representation learning and the corresponding set of outputs for multi-level representation learning Construct positive sample pairs and negative sample pairs, and Considered as a pair, and is considered a match, and and As a negative pair, the self-supervised loss function is:

[0022]

[0023] Where σ is the temperature parameter that controls the distribution concentration level, and They are and Standardization, and This is achieved through the L2 normalization layer.

[0024] Furthermore, when the stacked matrix is subjected to channel fusion learning using convolution kernels to obtain a more distinctive self-expression coefficient matrix: C C and Superimpose along the channel dimension to obtain the stacking matrix C S , use the convolution kernel k to integrate the stacked matrix C S The channel, Among them C F represents the self-expression coefficient matrix, Represents a convolution operation.

[0025] Furthermore, the loss function of adding the maximum entropy regularization constraint to the self-expression coefficient matrix is:

[0026]

[0027] Among them, In this case, the corresponding sum Equal to 0.

[0028] The affinity matrix is constructed using the self-expression coefficient matrix:

[0029]

[0030] By adopting the above-mentioned technical solution, the present invention provides a hyperspectral image band selection method based on self-supervised deep multi-level representation learning fused with maximum entropy subspace clustering. This method embeds the self-expression model into different layers of a deep convolutional autoencoder. By introducing multi-level representation learning, it considers both low-level and high-level information of the hyperspectral image to learn a more information-rich subspace clustering representation. A new auxiliary self-supervision task is then constructed for the multi-level representation learning to further improve the encoder's representation capability. A fusion module is then designed to fuse the multi-scale information extracted from different layers of the multi-level representation learning to learn a more discriminative self-expression coefficient matrix. Maximum entropy regularization is then added to the self-expression coefficient matrix to strengthen the connectivity within each subspace, ensuring that elements in the same subspace are evenly and densely distributed. Experimental results on three real hyperspectral datasets demonstrate that this method achieves excellent classification performance. Furthermore, this method can be used as an effective means of hyperspectral image band selection based on self-supervised deep multi-level representation learning fused with maximum entropy subspace clustering, with important application value in hyperspectral image classification and target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0032] Figure 1 It is an overall flow chart of the band selection method of the present invention;

[0033] Figure 2 The present invention provides a network structure model for a hyperspectral image band selection method based on self-supervised deep multi-level representation learning fused with maximum entropy subspace clustering;

[0034] Figure 3a-3c This is the Indian Pines dataset and classification result diagram in an embodiment of the present invention;

[0035] Figure 4a-4cThis is a diagram of the Pavia University dataset and classification results in an embodiment of the present invention;

[0036] Figure 5a-5c 2 is a diagram of the Salinas dataset and classification results in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] To make the technical solutions and advantages of the present invention more clear, the technical solutions in the embodiments of the present invention are clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention:

[0038] A hyperspectral image band selection method based on deep multi-level representation learning, including:

[0039] S1: A deep stacked convolutional autoencoder is used as the backbone network to learn the nonlinear features of hyperspectral images and extract the spatial information of hyperspectral images;

[0040] S2: Embed the self-expression model into different layers of a deep stacked convolutional autoencoder, perform multi-level representation learning on the backbone network, and combine the low-level and high-level information of the hyperspectral image to generate multiple sets of discriminability matrices and consistency matrices;

[0041] S3: Constructing a self-supervised auxiliary task based on the multi-level representation learning, and performing self-supervised learning on the feature representation of the multi-level representation learning;

[0042] S4: stacking the multiple sets of discriminative matrices and consistency matrices obtained by the multi-level representation learning along the channel dimension to obtain a stacked matrix, and performing channel fusion learning on the stacked matrix using a convolution kernel to obtain a more discriminative self-expression coefficient matrix;

[0043] S5: adding the self-expression coefficient matrix to the maximum entropy regularization constraint;

[0044] S6: Input the hyperspectral band image and the selected number of band subsets into the network, calculate the network loss function, train the network through the gradient method, and use the standard back propagation technique to update the parameters until the network stops training, thereby obtaining the self-expression coefficient matrix;

[0045] S7: construct an affinity matrix based on the self-expression coefficient matrix, obtain clustering results using spectral clustering, and obtain the average band in each category of the hyperspectral image based on the clustering results;

[0046] S8: Taking the average band as the cluster center, calculating the distance between each band and the cluster center, wherein the band closest to the cluster center is the selected band subset.

[0047] Furthermore, the loss function of the deep stacked convolutional autoencoder is:

[0048] S1-1: Set The original 3D hyperspectral cube is W×H pixels and B band. The goal of band selection is to select a subset of bands.

[0049]

[0050] S1-2: The encoder is defined as z = E(x; θ e ), where z represents the latent representation or bottleneck, x and θ e are the parameters of the data points and encoder respectively. Similarly, the decoder can be defined as in, and θ d represent the reconstruction of the input data and the parameters of the decoder respectively. The loss function of the deep stacked convolutional autoencoder is:

[0051]

[0052] Furthermore, the above process of embedding the self-expression model into different layers of the deep stacked convolutional autoencoder to achieve multi-level representation learning is as follows:

[0053] S2-1: The self-expression model is based on the following assumptions: Assume that the hyperspectral B-band image is located in the union of n subspaces In the And satisfy d1+d2+…+d n =B;

[0054] S2-2: Embedding self-expression models into different layers of deep stacked convolutional autoencoders to achieve multi-level representation learning, defining As the consistency matrix and As a discriminative matrix, the loss function of multi-level representation learning is:

[0055]

[0056] in, represents the output or latent representation matrix of the encoder, and m is the depth feature dimension.

[0057] Furthermore, the self-supervised auxiliary tasks constructed by the multi-level representation learning are introduced as follows:

[0058] S3-1: Given an input Z for multi-level representation learning i l and the corresponding set of outputs for multi-level representation learning Construct positive sample pairs and negative sample pairs, and put Z i l and G l j Considered as a pair, and is considered a match, and and As a negative pair, the self-supervised loss function is:

[0059]

[0060] Where σ is the temperature parameter that controls the distribution concentration level, and They are and Standardization, and This is achieved through the L2 normalization layer.

[0061] Furthermore, the above process of using convolution kernels to perform channel fusion learning on the stacked matrix to obtain a more distinctive self-expression coefficient matrix is as follows:

[0062] S4-1: C C and Superimpose along the channel dimension to obtain the stacking matrix C S , use convolution kernel k to integrate C S The channel, in, Means convolution operation;

[0063] Furthermore: The loss function of the above self-expression coefficient matrix with the maximum entropy regularization constraint is:

[0064] S5-1: Applying Maximum Entropy Regularization to the Self-Expression Coefficient Matrix

[0065]

[0066] Among them, In this case, the corresponding sum Equal to 0.

[0067] Furthermore, the affinity matrix constructed from the above expression coefficient matrix is:

[0068] S6-1: Constructing an affinity matrix with a symmetric form

[0069]

[0070] The following uses three sets of real hyperspectral data to analyze and evaluate the practical application effect of a hyperspectral image band selection method based on self-supervised deep multi-level representation learning fused with maximum entropy subspace clustering provided by the present invention.

[0071] A. Data source: It comes from three sets of public hyperspectral datasets: Indian Pines data, Pavia University data, and Salinas data.

[0072] (1) Indian Pines Dataset

[0073] The Indian Pines dataset was collected by the Airborne Visible Infrared Imaging Spectrometer (AVIRIS) sensor on June 12, 1992, in the Indian Pines region of northwestern Indiana, USA. The images contain 224 spectral bands, with a spatial dimension of 145×145 pixels, a wavelength range of 400 to 2500 nm, and a spatial resolution of 20 meters. The pseudo-color images and real ground feature information images of this dataset are shown in the following table. Figure 3a 、 Figure 3b As shown in the figure, it contains 16 real ground object categories. Before band selection, 24 spectral bands covering the water absorption area are removed, namely 104-108, 150-163 and 220, and a total of 200 bands are used.

[0074] (2) Pavia University Dataset

[0075] The Pavia University dataset was captured by the Reflecting Optical System Imaging Spectrometer System (ROSIS) at the University of Pavia in northern Italy in 2002. The images contain 115 spectral bands, with a spatial dimension of 610×340 pixels, a wavelength range of 430-860nm, and a spatial resolution of 1.3m. The pseudo-color images and real ground feature information images of this dataset are shown in Figure 2. Figure 4a 、 Figure 4b As shown in the figure, 9 urban land cover categories are included, 12 noise bands are removed, and finally 103 bands are retained.

[0076] (3) Salinas dataset

[0077] The Salinas dataset was collected by the AVIRIS sensor in California, USA in 1998. The image contains 224 spectral bands, with a spatial dimension of 512×217 pixels, a band range of 360-2500nm, and a spatial resolution of 3.7m. The pseudo-color image and real ground feature information image of this dataset are as follows: Figure 5a 、 Figure 5b As shown in the figure, 16 real ground object categories are included. Before band selection, 20 spectral bands of water absorption are removed, namely 108–112, 154–167 and 224, and a total of 204 bands are used.

[0078] B. The deep stacked convolutional autoencoder consists of three stacked convolutional encoder layers and corresponding decoder layers, with 10, 20, and 30 filters of sizes 5×5, 3×3, and 3×3, respectively. The convolution kernel k is set to 3×3, and the network parameters and network training times are set.

[0079] C. To quantitatively evaluate the classification performance of the selected band subset, a support vector machine (SVM) with a radial basis function kernel was used to classify hyperspectral images. 10% of the samples from each class were randomly selected as the training set, and the remaining samples were used as the test set. To ensure fairness in random sampling, all experiments were run 10 times, and the final experimental results were obtained by averaging the test accuracy.

[0080] D. Use quantitative evaluation indicators of the classifier to measure the classification performance: Overall Accuracy (OA), Average Accuracy (AA) and Kappa coefficient.

[0081] E. Analysis and evaluation of classification results

[0082] The classification results of a hyperspectral band selection method based on self-supervised deep multi-level representation learning fused with maximum entropy subspace clustering (hereinafter referred to as "self-supervised deep multi-level representation learning fused with maximum entropy subspace clustering") provided by the present invention on three sets of public hyperspectral data sets are shown in Tables 2-4, and the corresponding classification results are shown in the attached figure. Figure 4c 、 5c shown.

[0083] Table 2 Classification results of Indian Pines dataset

[0084]

[0085] Table 3 Classification results of Pavia University dataset

[0086]

[0087]

[0088] Table 4 Classification results of the Salinas dataset

[0089]

[0090] To address the problems of hyperspectral imagery, such as extensive redundancy, heavy computational and storage burdens, and the curse of dimensionality, this paper proposes a hyperspectral image band selection method based on self-supervised deep multi-level representation learning and maximum entropy subspace clustering. This method embeds a self-expression model into multiple fully connected near layers between an encoder and its corresponding decoder to achieve multi-level representation learning. To learn a more discriminative self-expression coefficient matrix, a fusion module is introduced to fuse the multi-scale information extracted from different layers by multi-level representation learning. New auxiliary tasks are constructed based on multi-layer representation learning to provide self-supervisory information to further enhance the model's representational capabilities. To strengthen the connectivity within each subspace, maximum entropy regularization is introduced to ensure that elements in the same subspace are evenly and densely distributed, which helps improve the performance of subspace clustering and ensures the accurate selection of more discriminative band subsets. Classification results on three publicly available hyperspectral datasets demonstrate the effectiveness of the hyperspectral image band selection method based on self-supervised deep multi-level representation learning and maximum entropy subspace clustering.

[0091] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A hyperspectral image band selection method based on deep multi-level representation learning, characterized by include: A deep stacked convolutional autoencoder is used as the backbone network to learn the nonlinear features of hyperspectral images and extract the spatial information of hyperspectral images; The self-expression model is embedded in different layers of the deep stacked convolutional autoencoder, multi-level representation learning is performed on the backbone network, and low-level and high-level information of the hyperspectral image are combined to generate multiple sets of discrimination matrices and consistency matrices. A multi-level representation learning method is used to construct a self-supervised auxiliary task, and self-supervise the feature representation of the multi-level representation learning; The multiple sets of discriminative matrices and consistent matrices obtained through multi-level representation learning are superimposed along the channel dimension to obtain a stacked matrix. The stacked matrix is subjected to channel fusion learning using convolution kernels to obtain a more discriminative self-expression coefficient matrix. Adding the self-expression coefficient matrix to the maximum entropy regularization constraint; The hyperspectral band image and the selected number of bands are input into the backbone network, the total loss function of the backbone network is calculated, the backbone network is trained by the gradient method, and the backbone network parameters are updated using the standard back-propagation technique until the backbone network stops training, thereby obtaining the self-expression coefficient matrix of the backbone network; An affinity matrix is constructed based on the self-expression coefficient matrix, and the clustering results are obtained by spectral clustering. The average band in each category of the hyperspectral image is obtained based on the clustering results. The average band is used as the cluster center, and the distance from each band to the cluster center is calculated, wherein the band closest to the cluster center is the selected band subset.

2. The method according to claim 1, wherein: The loss function of the backbone network is: set up The original 3D hyperspectral cube is W×H pixels and B band. The goal of band selection is to select a subset of bands. The encoder is defined as z = E(x; θ e ), where z represents the latent representation or bottleneck, x and θ e are the parameters of the data points and encoder respectively, and the decoder is defined as in, and θ d represent the reconstruction of the input data and the parameters of the decoder respectively. The loss function of the deep stacked convolutional autoencoder is:

3. The method according to claim 1, wherein: Assume that the hyperspectral B band images are located in the union of n subspaces In the And satisfy d1+d2+…+d n =B; When embedding self-expression models into different layers of deep stacked convolutional autoencoders and learning multi-level representations for the backbone network: Definition As the consistency matrix and As a discriminative matrix, the loss function of multi-level representation learning is: in, represents the output or latent representation matrix of the encoder, and m is the depth feature dimension.

4. The method according to claim 1, wherein: When constructing self-supervised auxiliary tasks: Given an input for multi-level representation learning and the corresponding set of outputs for multi-level representation learning Construct positive sample pairs and negative sample pairs, and Considered as a pair, and is considered a match, and and As a negative pair, the self-supervised loss function is: Where σ is the temperature parameter that controls the distribution concentration level, and They are and Standardization, and This is achieved through the L2 normalization layer.

5. The method according to claim 3, wherein: When the stacked matrix is subjected to channel fusion learning using convolution kernels to obtain a more discriminative self-expression coefficient matrix: C C and Superimpose along the channel dimension to obtain the stacking matrix C S , use the convolution kernel k to integrate the stacked matrix C S The channel, Among them C F represents the self-expression coefficient matrix, Represents a convolution operation.

6. The method according to claim 5, characterized in that: The loss function of adding the maximum entropy regularization constraint to the expression coefficient matrix is: Among them, In this case, the corresponding sum Equal to 0.

7. The method according to claim 5, characterized in that: The affinity matrix is constructed using the self-expression coefficient matrix:

Images