A hyperspectral image unmixing method based on hierarchical denoising autoencoder

Abstract

The present invention relates to a hyperspectral image unmixing method based on a hierarchical denoising autoencoder, and belongs to the field of hyperspectral unmixing technology. The method designs a multi-layer autoencoder, and each layer of autoencoder is specifically used to separate a specific abundance component from a mixed spectrum. After the first layer of autoencoder completes the abundance separation, the spectral signal corresponding to the abundance is subtracted from the original data to generate a new data set, which is then used as the input of the next layer of autoencoder to continue extracting the next abundance until the abundance ratios of all end members are extracted. Finally, the remaining data is regarded as noise. By using a multi-level structure to decompose the hyperspectral image layer by layer, the decoupling of each layer is used to fully explore the potential information of the image, and the step-by-step separation of noise and pure signals is achieved.

Description

Technical Field

[0001] The present invention relates to a hyperspectral image unmixing method based on a hierarchical denoising autoencoder, and belongs to the technical field of hyperspectral unmixing. Background Art

[0002] Hyperspectral remote sensing has been widely used in fields such as mineral exploration, military research, and precision agriculture. Due to the low resolution of hyperspectral remote sensing images, a single pixel contains multiple ground components, resulting in mixed pixels. Hyperspectral unmixing is an effective analytical technique for processing mixed pixels. The goal of hyperspectral unmixing is to separate pure endmembers from mixed pixels and obtain their corresponding abundance ratios. Abundance represents the proportion or contribution of each endmember in a mixed pixel. Spectral mixture models are the basis for unmixing and are generally categorized as linear mixture models (LMMs) and nonlinear mixture models (NLMMs). LMMs assume that a mixed pixel is equal to a linearly weighted sum of pure endmembers. However, LMMs are based on the ideal condition that photons reaching the sensor interact only with a single species at the macroscopic scale. Because LMMs are simplistic assumptions, while real-world environments are complex and dynamic, LMMs often perform poorly in real-world applications. NLMMs encompass a variety of models, such as tight mixture models and multi-level mixture models. These models take into account the interaction of solar radiation when scattered by different materials, and their working mechanisms are more in line with the real world, so NLMM usually has higher unmixing accuracy.

[0003] In recent years, deep learning has achieved breakthroughs in many fields, driving the development of related disciplines. Currently, deep learning-based unmixing methods have gradually become the mainstream technology for processing mixed pixels in hyperspectral remote sensing. As a deep learning model, the autoencoder is the most commonly used architecture for designing unmixing methods. An autoencoder consists of an encoder and a decoder. The encoder abstracts the input data into a low-dimensional map, while the decoder learns the features of the low-dimensional data and attempts to restore it to the original input data, making it particularly suitable for hyperspectral unmixing. Furthermore, convolutional neural networks (CNNs) have been introduced into the traditional autoencoder unmixing framework. They incorporate spatial information to further enhance unmixing performance. Palsson et al. designed a deep neural network autoencoder (DAEU) for blind hyperspectral unmixing and experimented with the effects of different activation functions on unmixing performance, resulting in improved abundance estimation performance. However, DAEU does not consider the spatial information of hyperspectral images. To address the autoencoder's inability to capture spectral spatial information, Rasti et al. added a deep convolutional neural network (UnDIP) to the autoencoder to address this shortcoming. Zhu et al. proposed a hyperspectral image multimodal decomposition network (MUNet) based on an autoencoder. MUNet assists the network in focusing on more discriminative and meaningful scene spatial information by fusing the attention map generated by the height difference of laser radar (LiDAR) data.

[0004] Not only the above-mentioned unmixing method based on autoencoder, almost all existing autoencoder networks for hyperspectral unmixing adopt a single-stage framework design. However, since hyperspectral images are high-dimensional data and contain a lot of noise, the single-stage autoencoder unmixing network shows poor stability to noise and outliers. Moreover, most autoencoders only use a simple fully connected structure, which is not ideal when processing hyperspectral image data that is high-dimensional, nonlinear, has strong inter-band correlation and contains a lot of noise. Therefore, optimizing the network structure and designing noise robustness based on the characteristics of hyperspectral data are key directions to improve the performance of the unmixing method. In summary, the present invention is based on the autoencoder, improves it, and proposes an efficient and stable hyperspectral image unmixing method. Summary of the Invention

[0005] The purpose of the present invention is to provide a hyperspectral image unmixing method based on a hierarchical denoising autoencoder, aiming to solve the technical problem that ordinary autoencoder unmixing methods show poor stability to noise and outliers.

[0006] To achieve the above objectives, the technical solution of the present invention is to propose a hyperspectral image unmixing method based on a hierarchical denoising autoencoder. The method includes multiple layers of autoencoders, where the number of layers of the autoencoder is equal to the number of endmembers in the hyperspectral image. Each layer of the autoencoder is used to extract an endmember and its abundance ratio. The method is implemented by the following steps:

[0007] Step 1: Use the first layer of autoencoder to extract the abundance ratio of the first endmember, then use the original mixed spectrum to subtract the spectral features corresponding to the first endmember to obtain an updated dataset, and use the updated dataset as the input of the next layer of autoencoder;

[0008] Step 2: Based on the dataset updated by the previous autoencoder, use the current autoencoder to extract the abundance ratio of the current endmember, subtract the spectral features of the current endmember from the dataset updated by the previous layer, and re-update the dataset as the input of the next autoencoder.

[0009] Step 3: Repeat Step 2 until the abundance ratios of all end members are extracted, and the final abundance matrix and noise matrix are obtained;

[0010] Step 4: Combine the customized loss function to train the autoencoder so that the decoder's weight matrix fits the endmember feature distribution and obtains the final endmember matrix.

[0011] The Step 1 is specifically as follows:

[0012] Step 1.1: The autoencoder includes an encoder and a decoder. The first layer of CNN encoder is used to extract the spatial information and features of the hyperspectral image. The CNN encoder has two layers of two-dimensional convolutional layers. The number of input and output channels is the number of spectral bands of the hyperspectral image. The ReLU activation function is used on the output of each convolution layer to ensure the non-negativity of the output matrix.

[0013] Input hyperspectral image ,in, is the spatial dimension, is the number of spectral bands, entering the first layer of convolutional network, the network learns the weights through training and bias , using convolution operation and nonlinear activation function ReLU to input hyperspectral image Perform feature extraction to extract the spectral features of the first endmember from the hyperspectral image. The specific encoding process is expressed as:

[0014]

[0015] in, is the spectral feature after the first endmember encoding, represents the nonlinear mapping function of the first layer encoder, Represents the convolution operation;

[0016] Step 1.2: Obtain the encoded features The abundance ratio of the first end member is obtained by inputting the decoder, which includes a convolution layer and uses a Sigmoid activation function on the output of the convolution layer; the decoder maps the encoded features into an abundance map of the first end member through convolution operation and Sigmoid activation function :

[0017]

[0018] in, is the Sigmoid function, which makes the abundance value in the range of [0,1]. is the weight of the first layer encoder, is the bias term of the convolutional layer in the first decoder layer;

[0019] Step 1.3: From the original hyperspectral image Subtract the spectral feature corresponding to the first end member from , and get a new residual data set:

[0020]

[0021] in, is the residual data set.

[0022] The Step 2 is specifically as follows:

[0023] Step 2.1: Use the residual dataset As the input of the second convolutional network, the second convolutional network extracts the second end member. The encoding and decoding process of the second layer is consistent with the first layer:

[0024]

[0025] in, is the spectral feature after encoding the second end member, represents the nonlinear mapping function of the second layer encoder, is the abundance map of the second end member, is the weight of the second layer encoder, is the bias term of the convolutional layer in the second decoder layer;

[0026] Step 2.2: After extracting the abundance of the second end member, Remove the spectral feature corresponding to the second end member , and get the updated residual dataset:

[0027]

[0028] in, is the updated residual dataset.

[0029] The Step 3 is specifically as follows:

[0030] After multi-layer decomposition, the end-member components in the mixed spectrum are gradually stripped away, and the remaining is the residual data. , where m is the number of end members, and the residual data For noisy data :

[0031]

[0032] in, For the Spectral characteristics of each end member;

[0033] For an m-layer autoencoder structure, spectral unmixing is achieved through iterative optimization:

[0034]

[0035] in, is the abundance constraint, To balance the parameters, is the Frobenius norm, is the set of parameters to be optimized, For the Abundance map of end members; spectral characteristics of each layer The convolutional encoder captures the spatial-spectral joint features of the endmembers, and the decoder implements the physical meaning constraint of abundance through the Sigmoid function: ; Through the residual-driven cascade structure, the network can gradually approach the intrinsic representation of the mixed signal.

[0036] The Step 4 is specifically as follows:

[0037] During the training process, the decoder weight matrix is ​​gradually adjusted to minimize the reconstruction error. In order to eliminate the influence of spectral variability and retain the spectral features, the spectral angular distance SAD is introduced as the first term of the target loss function:

[0038]

[0039] in, is the spectral angular distance loss term, is the inverse cosine function, is the input abundance matrix, is the reconstructed abundance matrix, is the Euclidean norm;

[0040] Although SAD loss is beneficial for endmember extraction, it only considers the scale-invariant similarity of the extracted endmembers, which will lead to greater deviation in abundance estimation. Therefore, the mean square error (MSE) is introduced as the second term of the objective loss function to minimize the variance of the abundance estimation:

[0041]

[0042] in, is the mean square error loss term;

[0043] Using L 1 / 2 Regularization methods are used as the third term in the objective loss function to improve the performance of abundance estimation:

[0044]

[0045] In the formula, the matrix With P rows and Q columns, is the regularization loss term, is the reconstructed abundance matrix The abundance of elements in column b and row c of ;

[0046] The custom loss function It is the weighted sum of the above three losses:

[0047]

[0048] in, and is a hyperparameter that balances the above three losses. As training progresses, the decoder weight matrix will gradually approach the true endmember spectrum.

[0049] The beneficial effects of the present invention are as follows: compared with the existing unmixing methods, the hyperspectral image unmixing method based on hierarchical denoising autoencoder proposed in the present invention first decomposes the hyperspectral image layer by layer through a multi-level convolutional autoencoder structure. This level-by-level decomposition architecture can more accurately process complex hyperspectral data, especially when there is a large amount of noise in the hyperspectral image and nonlinear mixing between end members; secondly, the idea of ​​"subtraction" is adopted to achieve efficient noise removal and provide cleaner and more accurate input data for the subsequent unmixing process; finally, based on the residual optimization target of the spectral-spatial joint constraint, by introducing non-local similarity prior and adaptive regularization technology, the threshold and iteration step of noise separation are dynamically adjusted, thereby achieving a significant improvement in the unmixing accuracy while ensuring spectral fidelity. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a flow chart of the hyperspectral image unmixing method based on hierarchical denoising autoencoder of the present invention;

[0051] Figure 2 1 is an architectural diagram of a hyperspectral image unmixing method based on a hierarchical denoising autoencoder according to the present invention;

[0052] Figure 3 This is a graph showing the abundance inversion experimental results of the hyperspectral image unmixing method based on the hierarchical denoising autoencoder of the present invention and the comparative method;

[0053] Figure 4 This is a graph showing the experimental results of endmember extraction of the hyperspectral image unmixing method based on the hierarchical denoising autoencoder of the present invention and the comparative method. DETAILED DESCRIPTION

[0054] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments. This embodiment is only used to illustrate the technical solution of the present invention and does not constitute any limitation to its scope.

[0055] Example 1: Figure 1 and Figure 2 As shown in the figure, this embodiment proposes a hyperspectral image unmixing method based on a hierarchical denoising autoencoder. This example uses the Samson dataset: The Samson dataset is acquired using the SAMSON sensor, has a size of 95×95 pixels, contains 156 spectral bands, and has a spectral range from 0.401µm to 0.889µm. In this scene, there are three main ground features: soil, trees, and water. In addition, the Samson dataset provides ground reference data, including endmember spectra and abundance information, which can be used to quantitatively evaluate the accuracy of the unmixing results. The method specifically includes the following steps:

[0056] Step 1: Use the first layer of autoencoder to extract the abundance ratio of the first end member, then use the original mixed spectrum to subtract the spectral features corresponding to the first end member to obtain an updated dataset, and use the updated dataset as the input of the next layer of autoencoder.

[0057] Specifically, the input hyperspectral image Enter the first convolutional network. Use the first CNN encoder to extract the spatial information and features of the hyperspectral image; the CNN encoder has two layers of two-dimensional convolutional layers, the number of input and output channels is the number of spectral bands of the hyperspectral image, and the size of the convolution kernel is , the convolution kernel slides on the input feature map with a step size of 1 each time, and fills 1 layer of pixels on each side of the input feature map. The ReLU activation function is used for the output of each convolution layer to ensure the non-negativity of the output matrix. In this layer, the network learns the weights through training. and bias , using convolution operation and nonlinear activation function ReLU to input hyperspectral image Perform feature extraction to extract the spectral features of the first endmember from the hyperspectral image. Specifically, the encoding process can be expressed as:

[0058]

[0059] in, is the spectral feature after the first endmember encoding, represents the nonlinear mapping function of the first layer encoder, Represents the convolution operation;

[0060] Will Input into the decoder; the decoder contains a convolution layer, and the Sigmoid activation function is used on the output of the convolution layer; the decoder maps the encoded features to the abundance map of the first end member through convolution operation and Sigmoid activation function :

[0061]

[0062] in, is the Sigmoid function, is the weight of the first layer encoder, making the abundance value in the range of [0,1], is the bias term of the convolutional layer in the first decoder layer.

[0063] After the extraction is completed, the original hyperspectral image Subtract the spectral feature corresponding to the first end member from , and get a new residual data set:

[0064]

[0065] in, is the residual data set.

[0066] Step 2: Based on the dataset updated by the previous autoencoder, use the current autoencoder to extract the abundance ratio of the current endmember, subtract the spectral features of the current endmember from the dataset updated by the previous layer, and re-update the dataset as the input of the next autoencoder.

[0067] Specifically, using the residual dataset As the input of the second convolutional network, the second convolutional network extracts the second end member. The encoding and decoding process of the second layer is consistent with the first layer:

[0068]

[0069]

[0070] in, is the spectral feature after encoding the second end member, represents the nonlinear mapping function of the second layer encoder, is the abundance map of the second end member, is the weight of the second layer encoder, is the bias term of the convolutional layer in the second decoder layer.

[0071] After extracting the second end member, from the current residual data set Remove the spectral feature corresponding to the second end member , and get the updated residual dataset:

[0072]

[0073] in, is the updated residual dataset.

[0074] Step 3: Repeat Step 2 until the abundance ratios of all end members are extracted to obtain the final abundance matrix and noise matrix.

[0075] Specifically, the present invention uses the third layer of automatic encoder to extract the abundance ratio and noise matrix of the third end member; similarly, the third layer of automatic encoder is used to As input, extract the spectral features of the third end member and abundance diagram :

[0076]

[0077]

[0078] in, is the spectral feature after encoding the third end member, represents the nonlinear mapping function of the third layer encoder, is the abundance map of the third end member, is the weight of the third layer encoder, is the bias term of the convolutional layer in the third decoder layer.

[0079] And generate the final residual data set, that is, the noise matrix:

[0080]

[0081] in, is the final residual data set.

[0082] Step 4: Combine the customized loss function to train the autoencoder so that the decoder weight matrix fits the endmember feature distribution and obtains the final endmember matrix.

[0083] Specifically, the decoder uses Adam as an optimizer to implement gradient descent to update the endmembers. The decoder uses the low-dimensional feature representation generated by the encoder to reconstruct the input data. In this process, the decoder weight matrix is ​​regarded as the endmember spectrum. The update rule of Adam is as follows:

[0084]

[0085]

[0086]

[0087]

[0088] in, is the first-order moment estimate of the gradient at the current iteration number, is the first-order moment estimate of the gradient of the previous iteration, is the second-order moment estimate of the gradient of the current iteration number, is the second-order moment estimate of the gradient of the previous iteration, is the bias-corrected first-order moment estimate, is the bias-corrected second-order moment estimate, is the decoder weight matrix (i.e., endmember spectrum) of the t-th iteration, It is The model parameters after iterations are is the gradient of the loss function with respect to the parameters, is the learning rate, , is the momentum decay rate, is a numerical stability constant. By this rule, the endmember spectral matrix is ​​dynamically adjusted to minimize the reconstruction error.

[0089] Furthermore, during the training process, the decoder weight matrix is ​​gradually adjusted to minimize the reconstruction error, and the spectral angular distance SAD is introduced as the first term of the target loss function:

[0090]

[0091] in, is the spectral angular distance loss term, is the inverse cosine function, is the input abundance matrix, is the reconstructed abundance matrix, is the Euclidean norm;

[0092] At the same time, the mean square error MSE is introduced as the second term of the target loss function:

[0093]

[0094] in, is the mean square error loss term;

[0095] Using L 1 / 2 Regularization method as the third term of the target loss function:

[0096]

[0097] In the formula, the matrix With P rows and Q columns, is the regularization loss term, is the reconstructed abundance matrix The abundance of elements in column b and row c of ;

[0098] The custom loss function It is the weighted sum of the above three losses:

[0099]

[0100] in, and It is a hyperparameter that balances the above three losses. =0.1, = 0.01. As training progresses, the decoder weight matrix will gradually approach the true endmember spectrum.

[0101] Step 5: Use the spectral angle distance SAD and root mean square error RMSE to measure the performance of the algorithm unmixing. Excellent models have low SAD and RMSE values. SAD calculates the end member estimate and the true value of the end member The angular distance between :

[0102]

[0103] RMSE measures the true value of abundance and abundance estimates The difference between :

[0104]

[0105] in, for The total number of elements in .

[0106] In this paper, the model learning rate was set to 0.001 and trained for 300 epochs. The hyperspectral image unmixing method based on a hierarchical denoising autoencoder was experimentally compared with seven classic and advanced unmixing methods. The experimental results are shown in Table 1, with the best unmixing performance highlighted in bold. SAD was used to evaluate the performance of endmember extraction, and RMSE was used to evaluate the performance of abundance inversion.

[0107] Table 1 Unmixing results of Samson dataset

[0108]

[0109] Table 1 shows that the proposed method achieves optimal unmixing performance in both endmember extraction and abundance inversion, significantly improving upon previous methods. Specifically, the proposed method achieves a 51.89% improvement in mean squared error (SAD) compared to the best-performing method (MUNet) among seven compared methods, and a 45.71% improvement in mean root mean square error (RMSE) compared to the best-performing method (A2SAN) among the previous methods. These quantitative results demonstrate the effectiveness of the proposed strategy both theoretically and practically.

[0110] Figure 3 is the abundance map of each method, Figure 4 Comparison diagram of the endmembers extracted by each method and the true endmembers, where the dotted line represents the true endmember curve and the solid line represents the endmember curve extracted by the unmixing method. Figure 3 and Figure 4 The unmixing performance of each method is intuitively demonstrated. It can be seen that the abundance map of the present invention is closer to the real abundance map, and the end member curve extracted by the present invention has the highest overlap with the real end member curve.

[0111] The experimental data above fully demonstrates that the hyperspectral image unmixing method based on the hierarchical denoising autoencoder uses a multi-layered structure to decompose the hyperspectral image layer by layer. The decoupling of each layer fully exploits the potential information of the image, achieving a step-by-step separation of noise and pure signal. By adding a convolutional network to extract spectral spatial information, the unmixing accuracy of the hyperspectral image is significantly improved. The unmixing method proposed in this paper is noise-robust and has excellent anti-interference capabilities, and can maintain stable unmixing performance in complex noisy environments.

[0112] The above describes the specific embodiments of the present invention in detail with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in this field without departing from the scope of the present invention.

Claims

1. A hyperspectral image unmixing method based on a hierarchical denoising autoencoder, characterized in that: The specific steps include: Step 1: Use the first layer of autoencoder to extract the abundance ratio of the first endmember, then use the original mixed spectrum to subtract the spectral features corresponding to the first endmember to obtain an updated dataset, and use the updated dataset as the input of the next layer of autoencoder; Step 2: Based on the dataset updated by the previous autoencoder, use the current autoencoder to extract the abundance ratio of the current endmember, subtract the spectral features of the current endmember from the dataset updated by the previous layer, and re-update the dataset as the input of the next autoencoder. Step 3: Repeat Step 2 until the abundance ratios of all end members are extracted, and the final abundance matrix and noise matrix are obtained; Step 4: Combine the customized loss function to train the autoencoder so that the decoder weight matrix fits the endmember feature distribution and obtains the final endmember matrix. The Step 2 is specifically as follows: Step 2.1: Use the residual dataset As the input of the second convolutional network, the second convolutional network extracts the second end member. The encoding and decoding process of the second layer is consistent with the first layer: ； in, is the spectral feature after encoding the second end member, represents the nonlinear mapping function of the second layer encoder, is the abundance map of the second end member, is the weight of the second layer encoder, is the bias term of the convolutional layer in the second decoder layer; Step 2.2: After extracting the abundance of the second end member, Remove the spectral feature corresponding to the second end member , and get the updated residual dataset: ； in, is the updated residual data set; The Step 3 is specifically as follows: After multi-layer decomposition, the end-member components in the mixed spectrum are gradually stripped away, and the remaining is the residual data. , where m is the number of end members, and the residual data For noisy data : ； in, For the Spectral characteristics of each end member; For an m-layer autoencoder structure, spectral unmixing is achieved through iterative optimization: ； in, is the abundance constraint, To balance the parameters, is the Frobenius norm, is the set of parameters to be optimized, For the Abundance map of end members; spectral characteristics of each layer The convolutional encoder captures the spatial-spectral joint features of the endmembers, and the decoder implements the physical meaning constraint of abundance through the Sigmoid function: ; The Step 4 is specifically as follows: During the training process, the weight matrix of the decoder is gradually adjusted to minimize the reconstruction error, and the spectral angular distance SAD is introduced as the first term of the target loss function: ； in, is the spectral angular distance loss term, is the inverse cosine function, is the input abundance matrix, is the reconstructed abundance matrix, is the Euclidean norm; At the same time, the mean square error MSE is introduced as the second term of the target loss function: ； in, is the mean square error loss term; Using L 1 / 2 Regularization method as the third term of the target loss function: ； In the formula, the matrix With P rows and Q columns, is the regularization loss term, is the reconstructed abundance matrix The abundance of elements in column b and row c of ; The custom loss function It is the weighted sum of the above three losses: ； in, and is a hyperparameter that balances the above three losses.

2. The hyperspectral image unmixing method based on hierarchical denoising autoencoder according to claim 1 is characterized in that: The Step 1 is specifically as follows: Step 1.1: The autoencoder includes an encoder and a decoder. The first layer of CNN encoder is used to extract the spatial information and features of the hyperspectral image. The CNN encoder has two layers of two-dimensional convolutional layers. The number of input and output channels is the number of spectral bands of the hyperspectral image. The ReLU activation function is used for the output of each convolution layer. Input hyperspectral image ,in, is the spatial dimension, is the number of spectral bands, entering the first layer of convolutional network, the network learns the weights through training and bias , using convolution operation and nonlinear activation function ReLU to input hyperspectral image Perform feature extraction to extract the spectral features of the first endmember from the hyperspectral image. The specific encoding process is expressed as: ； in, is the spectral feature after the first endmember encoding, represents the nonlinear mapping function of the first layer encoder, Represents the convolution operation; Step 1.2: Obtain the encoded features The abundance ratio of the first end member is obtained by inputting the decoder, which includes a convolution layer and uses a Sigmoid activation function on the output of the convolution layer; the decoder maps the encoded features into an abundance map of the first end member through convolution operation and Sigmoid activation function : ； in, is the Sigmoid function, which makes the abundance value in the range of [0,1]. is the weight of the first layer encoder, is the bias term of the convolutional layer in the first decoder layer; Step 1.3: From the original hyperspectral image Subtract the spectral feature corresponding to the first end member from , and get a new residual data set: ； in, is the residual data set.

Images