A hyperspectral image classification method based on the Retinex model

Abstract

This invention discloses a hyperspectral image classification method based on the Retinex model, belonging to the field of image processing technology. The method includes the following steps: performing Retinex decomposition on the hyperspectral image to obtain intrinsic property components and illumination components; extracting features from the intrinsic property components and illumination components separately using a dual-branch network, wherein the intrinsic property component feature extractor is used to extract material features and texture information of the hyperspectral image, and the illumination component feature extractor is used to extract illumination features of the hyperspectral image; fusing the extracted intrinsic property features and illumination features using a feature fusion module based on an interactive attention mechanism to generate a fused feature representation; and classifying the fused features to generate a classification result for the hyperspectral image. This invention can effectively extract spatial spectral features from hyperspectral images, significantly improving classification accuracy, and has broad application prospects, suitable for hyperspectral image classification tasks under complex illumination conditions.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically a hyperspectral image classification method based on the Retinex model. Background Technology

[0002] Hyperspectral image classification is an important research direction in remote sensing science. It achieves accurate identification and classification of ground objects by capturing the continuous reflectance characteristics of surface objects across the entire electromagnetic spectrum. Hyperspectral images are characterized by a large number of bands and high information redundancy, providing extremely detailed spectral information and unprecedented insights for scientific research and practical applications.

[0003] However, hyperspectral image classification faces many challenges, especially under varying lighting conditions and limited labeled data. The classification performance of existing technologies is often limited, mainly as follows:

[0004] The impact of illumination variations: The classification performance of hyperspectral images is highly sensitive to illumination conditions. Changes in illumination significantly alter the spectral features of an image, thereby affecting the accuracy of the classification model. Existing deep learning methods perform poorly when dealing with illumination variations, especially under uneven or complex lighting conditions, where classification accuracy drops significantly.

[0005] Overfitting with limited labeled data: Acquiring labeled data for hyperspectral images is costly and difficult, resulting in limited training data. Existing deep learning methods are prone to overfitting with limited labeled data, leading to poor generalization ability and unsatisfactory classification results on unseen data.

[0006] Limitations of Single Feature Extraction: Existing hyperspectral image classification methods typically focus on only a single feature (such as spectral or spatial features), neglecting the complementary relationship between illumination features and intrinsic attribute features. This single feature extraction approach limits the classification performance of the model, especially in complex scenes, where classification accuracy is difficult to improve further.

[0007] High computational complexity: Existing deep learning methods typically require a large amount of computational resources, especially when dealing with high-dimensional hyperspectral data, where the computational complexity increases significantly, limiting their widespread application in practice. Summary of the Invention

[0008] To address the above problems, this invention provides a hyperspectral image classification method based on the Retinex model. This invention can effectively extract spatial spectral features from hyperspectral images, significantly improve classification accuracy, has broad application prospects, and is suitable for hyperspectral image classification tasks under complex lighting conditions.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0010] A hyperspectral image classification method based on the Retinex model includes the following steps:

[0011] Retinex decomposition was performed on the hyperspectral image to obtain intrinsic property components and illumination components;

[0012] The intrinsic property component and the illumination component are extracted separately by a dual-branch network. The intrinsic property component feature extractor is used to extract the material features and texture information of the hyperspectral image, and the illumination component feature extractor is used to extract the illumination features of the hyperspectral image.

[0013] The extracted intrinsic attribute features and illumination features are fused using a feature fusion module based on an interactive attention mechanism to generate a fused feature representation.

[0014] The fused features are classified to generate the classification results of the hyperspectral image.

[0015] This invention presents a hyperspectral image classification method based on the Retinex model. It separates hyperspectral images into intrinsic attribute components and illumination components through Retinex decomposition, extracts features using a dual-branch network for each component, and achieves feature fusion and classification through an interactive attention mechanism. This method effectively separates illumination variations and intrinsic attributes in hyperspectral images, improving classification accuracy and robustness, and addressing the impact of illumination variations on classification performance.

[0016] As a further improvement to the above scheme, the Retinex decomposition step includes:

[0017] Perform a logarithmic transformation on each band of the hyperspectral image;

[0018] The illumination component is obtained by filtering with Gaussian kernels of different scales, where the scale parameter σi of the Gaussian kernel is dynamically adjusted according to the local and global information of the hyperspectral image.

[0019] The reflectance image is obtained by subtracting the illumination component from the logarithmically transformed image. The reflectance image represents the inherent property components of the hyperspectral image.

[0020] Weighted averages are applied to reflectance images at different scales to obtain the final intrinsic property components;

[0021] The intrinsic property components and illumination components are subjected to an exponential transformation to restore the linear space.

[0022] The technical effects of the above improvements are as follows:

[0023] Retinex decomposition involves performing a logarithmic transformation on each band of a hyperspectral image, using multi-scale Gaussian kernel filtering to obtain the illumination component, and then subtracting the illumination component from the logarithmically transformed image to obtain the reflectance image (intrinsic property component). Finally, a weighted average is performed on the reflectance images at different scales to recover the linear space. Through multi-scale Gaussian kernel filtering and weighted averaging, both local and global information of the hyperspectral image are fully considered, effectively separating the illumination component and the intrinsic property component, thus enhancing the expressive power of image features.

[0024] As a further improvement to the above scheme, the intrinsic attribute component feature extractor includes:

[0025] The input's inherent attribute components are reduced in dimensionality using convolutional layers to generate an initial feature map;

[0026] Local features are extracted step by step through multiple convolutional layers, which include convolutional kernels of different sizes;

[0027] Global pooling operations are used to capture global and local salient information, generating global average pooling vectors and global max pooling vectors.

[0028] The global average pooling vector and the global max pooling vector are concatenated, and then channel attention weights are generated through a fully connected layer and an activation function.

[0029] The channel attention weights are multiplied element-wise with the original feature map to generate the enhanced feature map.

[0030] The improved technology achieves the following results: by using convolutional neural networks and channel attention mechanisms, material features and texture information of hyperspectral images are extracted, enhancing feature representation capabilities and improving classification accuracy.

[0031] As a further improvement to the above scheme, the illumination component feature extractor uses the same deep learning architecture as the intrinsic attribute component feature extractor for feature extraction and is trained using the cross-entropy loss function.

[0032] The improved technology achieves the following results: by using the same deep learning architecture, it extracts illumination features from hyperspectral images, ensuring the accuracy and consistency of illumination feature extraction and improving classification performance.

[0033] As a further improvement to the above scheme, the feature fusion module based on the interactive attention mechanism includes:

[0034] The inherent attribute features and illumination features are respectively converted into query vector Query, key vector Key and value vector Value through convolutional layers, where the convolutional kernel size is 3×3;

[0035] The energy matrix is ​​generated by calculating the similarity between the query vector and the key vector using the dot product;

[0036] The energy matrix is ​​normalized using the Softmax function to generate the attention weight matrix;

[0037] Multiply the attention weight matrix by the value vector to generate the fused feature map;

[0038] The fused feature map is added to the original feature map through residual connections, where the weight γ of the residual connections is a learnable parameter;

[0039] The residual concatenated feature maps are then weighted and fused using two parallel multilayer perceptrons (MLPs) to generate the final classification features.

[0040] The improved technical effects are as follows: The feature fusion module based on the interactive attention mechanism converts inherent attribute features and illumination features into query vectors, key vectors, and value vectors, respectively. It then generates an energy matrix by calculating similarity through dot product, and generates an attention weight matrix using the Softmax function. Finally, it multiplies the attention weight matrix with the value vectors to generate the fused feature map, and generates the final classification features through residual connections and weighted fusion. By dynamically adjusting the weights of inherent attribute features and illumination features through the interactive attention mechanism, information complementarity is achieved, improving the feature fusion effect and thus increasing classification accuracy.

[0041] As a further improvement to the above scheme, the generation process of the query vector Query, key vector Key, and value vector Value in the feature fusion module of the interactive attention mechanism includes:

[0042] Use a 3×3 convolution kernel to perform convolution operations on the intrinsic attribute features and the illumination features respectively to generate query vector, key vector and value vector;

[0043] The energy matrix is ​​generated by calculating the similarity between the query vector and the key vector using the dot product;

[0044] The energy matrix is ​​normalized using the Softmax function to generate the attention weight matrix;

[0045] Multiply the attention weight matrix by the value vector to generate the fused feature map;

[0046] The fused feature map is added to the original feature map through residual connections, where the weight γ of the residual connections is a learnable parameter with an initial value of 1.0;

[0047] The residual concatenated feature maps are then weighted and fused using two parallel multilayer perceptrons (MLPs) to generate the final classification features.

[0048] The improved technology achieves the following results: By using the feature fusion module with interactive attention mechanism, the weights of inherent attribute features and illumination features are dynamically adjusted to achieve information complementarity, significantly improving the accuracy and robustness of hyperspectral image classification.

[0049] As a further improvement to the above scheme, the Retinex decomposition module uses a multi-scale Gaussian kernel for filtering. The scale parameter σi of the Gaussian kernel is dynamically adjusted according to the local and global information of the hyperspectral image to fully consider the local and global information of the hyperspectral image.

[0050] The improved techniques achieve the following results: By using multi-scale Gaussian kernel filtering, the local and global information of the hyperspectral image is fully considered, enhancing the robustness of Retinex decomposition and improving classification performance.

[0051] As a further improvement to the above scheme, both the intrinsic attribute component feature extractor and the illumination component feature extractor use convolutional neural networks for feature extraction, and accelerate the training process through batch normalization and ReLU activation function.

[0052] The improved techniques described above achieve the following effects: by using batch normalization and the ReLU activation function, the training process is accelerated, the convergence speed and stability of the model are improved, and the feature extraction effect is enhanced.

[0053] As a further improvement to the above scheme, the feature fusion module based on the interactive attention mechanism dynamically adjusts the weights of inherent attribute features and illumination features to achieve information complementarity, thereby improving classification accuracy.

[0054] The technical effects of the above improvements are as follows: By dynamically adjusting the weights, the inherent attribute features and illumination features are effectively integrated, which improves classification accuracy and enhances the model's generalization ability.

[0055] As a further improvement to the above scheme, the method generates the final classification map through a visualization network, making it easier for users to intuitively understand the classification results.

[0056] The improved technical effects are as follows: by visualizing the classification diagram, the interpretability and practicality of the method are enhanced, making it easier for users to intuitively understand the classification results.

[0057] Compared with the prior art, the overall technical effect of the present invention is as follows:

[0058] 1. Robustness to Illumination Variation: This invention separates the illumination component and intrinsic attribute component of hyperspectral images through Retinex decomposition, effectively reducing the impact of illumination variations on classification performance. By extracting illumination features and intrinsic attribute features separately through a dual-branch network, the model can maintain high classification accuracy under different illumination conditions.

[0059] 2. Mitigation of Overfitting: This invention enhances the model's generalization ability by introducing an interactive attention mechanism, which fully utilizes information from limited labeled data. By dynamically adjusting the weights of intrinsic attribute features and illumination features, the model can better capture the inherent relationships in the data, reducing the risk of overfitting.

[0060] 3. Advantages of Multi-Feature Fusion: This invention achieves complementary fusion of illumination features and intrinsic attribute features through a dual-branch feature extractor and interactive attention mechanism. This multi-feature fusion method significantly improves the model's classification performance, especially in complex scenes, where classification accuracy is significantly enhanced.

[0061] 4. Improved computational efficiency: This invention simplifies the processing of hyperspectral data and reduces computational complexity through Retinex decomposition and multi-scale filtering. Furthermore, optimized design of convolutional neural networks and attention mechanisms further enhances computational efficiency, making the method more feasible in practical applications. Attached Figure Description

[0062] Figure 1 This is a diagram illustrating the overall framework of the hyperspectral image classification method proposed in this invention.

[0063] Figure 2 This is a pseudo-color image of PU.

[0064] Figure 3 This is a diagram illustrating the classification of PU.

[0065] Figure 4 This is a real label image of PU flooring.

[0066] Figure 5 This is a pseudo-color image of the IP address.

[0067] Figure 6 This is a screenshot showing the effect of IP classification.

[0068] Figure 7 This is a real IP ground truth label map. Detailed Implementation

[0069] To enable those skilled in the art to better understand the technical solution, the present invention will be described in detail below with reference to embodiments. The description in this part is only exemplary and explanatory, and should not be used to limit the scope of protection of the present invention in any way.

[0070] This invention is the first to apply the Retinex method to hyperspectral image classification. In traditional Retinex methods, various information in the input RGB image is filtered to obtain components reflecting illumination changes. Then, the illumination component is subtracted from the original image in the logarithmic domain to obtain components representing the object's reflectance properties. Hyperspectral images are characterized by a large number of bands and significant information redundancy; one hyperspectral image is equivalent to hundreds of grayscale images. Therefore, the Retinex method is used for hyperspectral image processing. The weighted linear Retinex decomposition results at different scales are combined to fully consider the local and global information of important bands in the hyperspectral image. Then, an intrinsic attribute feature extractor and an illumination feature extractor are designed. Through deep feature embedding, a convolutional neural network is used to focus on the physical properties of the image and the illumination changes in the environment. To further ensure the separation of intrinsic attribute components and illumination components, a feature fusion module based on an intrinsic attribute and illumination interaction attention mechanism is introduced. The intrinsic attribute and illumination interaction attention mechanism can complement the information of the two components, allowing each component to dynamically acquire useful information from the other. An attention matrix is ​​generated by calculating the similarity between two component features, and the final feature fusion is adjusted accordingly to achieve feature complementarity and enhancement.

[0071] 1. Overall Process:

[0072] This invention provides a hyperspectral image classification method based on the Retinex model, the overall process of which includes the following steps:

[0073] 1) Retinex decomposition: Perform Retinex decomposition on hyperspectral images to separate intrinsic property components and illumination components.

[0074] 2) Dual-branch feature extraction: Features of intrinsic attribute components and illumination components are extracted separately through a dual-branch network.

[0075] 3) Feature fusion: The feature fusion module based on the interactive attention mechanism is used to fuse inherent attribute features and illumination features.

[0076] 4) Classification: Classify the fused features to generate classification results for the hyperspectral image.

[0077] 2. Specific implementation of Retinex decomposition:

[0078] The Retinex decomposition step is implemented as follows:

[0079] 1) Logarithmic Transformation: Performing a logarithmic transformation on each band of the hyperspectral image converts the image from linear space to logarithmic space. The formula for the logarithmic transformation is:

[0080]

[0081] in, : Input hyperspectral image, representing the location First Pixel values ​​for each band, It is a reflectance image (intrinsic property component). It is the light component. The natural logarithmic transformation converts an image from a linear space to a logarithmic space.

[0082] 2) Multi-scale Gaussian kernel filtering: The logarithmically transformed image is filtered using Gaussian kernels of different scales (the scale parameter σi is dynamically adjusted based on local and global information from the hyperspectral image) to obtain the illumination component. The formula for the Gaussian kernel is:

[0083]

[0084] Scale parameter is A Gaussian kernel is used to filter images.

[0085] The scale parameter of the Gaussian kernel represents the size of the filter, which is dynamically adjusted based on local and global information of the image.

[0086] : Convolution operation, which means convolving a Gaussian kernel with the result of the logarithmic transformation of the image.

[0087] : at scale The light component obtained below.

[0088] 3) Reflectance Image Calculation: The reflectance image (intrinsic property component) is obtained by subtracting the illumination component from the logarithmically transformed image. The formula is:

[0089]

[0090] : at scale The reflectance image obtained below (intrinsic property components).

[0091] : The result of the logarithmic transformation of the input image.

[0092] : at scale The light component obtained below.

[0093] 4) Weighted average: A weighted average is applied to reflectance images at different scales to obtain the final intrinsic attribute components. The formula is:

[0094]

[0095] Reflectance image after weighted average.

[0096] Number of scales used.

[0097] : at scale The reflectance image obtained below.

[0098] 5) Exponential Transform: An exponential transformation is applied to the intrinsic attribute components and illumination components to restore them to linear space. The formula is:

[0099]

[0100]

[0101] It is the scale parameter of the multi-scale Gaussian kernel, used to capture local and global information of the image at different scales.

[0102] reflectance image and light component At different scales The result was obtained through calculation.

[0103] : Reflectance image after restoring linear space (intrinsic property components).

[0104] : Illumination components after restoring linear space.

[0105] Exponential transformation restores the result in the logarithmic space to the linear space.

[0106] Implementation of the intrinsic attribute component feature extractor:

[0107] The input's inherent attribute components are reduced in dimensionality using convolutional layers to generate an initial feature map;

[0108] Local features are extracted step by step through multiple convolutional layers, which include convolutional kernels of different sizes;

[0109] Global pooling operations are used to capture global and local salient information, generating global average pooling vectors and global max pooling vectors.

[0110] The global average pooling vector and the global max pooling vector are concatenated, and then channel attention weights are generated through a fully connected layer and an activation function.

[0111] The channel attention weights are multiplied element-wise with the original feature map to generate the enhanced feature map.

[0112] Specifically:

[0113] The specific implementation of the intrinsic attribute component feature extractor in the dual-branch network is as follows:

[0114] 1) 1×1×1 convolutional layer: Reduces the number of channels in the input band to 30, generating the initial feature map:

[0115]

[0116] 2) Multi-layer convolution: Local features are extracted stepwise using 7×3×3, 5×3×3, and 3×3×3 convolution kernels to obtain the feature map after local feature extraction.

[0117]

[0118] 3) Remodeling operation: The multi-channel feature map is reorganized into a two-dimensional feature map in order to perform the next convolution operation.

[0119] 4) 3×3×3 convolutional layer: Reduce the number of channels from 576 to 64 to generate feature maps:

[0120]

[0121] 5) Batch normalization and ReLU activation function: Apply batch normalization and ReLU activation function after each convolutional layer to accelerate the training process.

[0122] 6) Dropout layer: Adding a Dropout layer reduces the model's dependence on training data, improves the model's generalization ability, and yields the final feature map.

[0123]

[0124] 7) Global pooling operation:

[0125] Global average pooling:

[0126]

[0127] : Feature vector after global average pooling.

[0128] : Height of the feature map.

[0129] : Width of the feature map.

[0130] : Channel index of the feature map.

[0131] Global max pooling:

[0132]

[0133] : Feature vector after global max pooling.

[0134] In spatial dimension Take the maximum value.

[0135] 8) Channel attention mechanism:

[0136] Concatenate the global average pooling and global max pooling vectors:

[0137]

[0138] Intermediate features are generated using fully connected layers and the ReLU activation function:

[0139]

[0140] Channel attention weights are generated using the Sigmoid activation function:

[0141]

[0142] Channel attention weights.

[0143] The sigmoid activation function restricts the output to the range [0,1].

[0144] : Weight parameters of the fully connected layer.

[0145] : Bias parameters of the fully connected layer.

[0146] : Intermediate feature vector.

[0147] 9) Feature Enhancement: Multiply the channel attention weights element-wise with the original feature map to generate an enhanced feature map.

[0148]

[0149] 10) Cross-entropy loss function: During training, the cross-entropy loss function is used to train the feature extractor.

[0150]

[0151] 4. Implementation of the illumination component feature extractor:

[0152] The illumination component feature extractor uses the same deep learning architecture as the intrinsic attribute component feature extractor, and its implementation is as follows:

[0153] 1) Same architecture: It adopts the same convolutional neural network architecture as the intrinsic attribute component feature extractor.

[0154] 2) Cross-entropy loss function: Training is performed using the cross-entropy loss function to ensure the accuracy of illumination feature extraction. The formula is:

[0155]

[0156] 5. Implementation of the feature fusion module based on the interactive attention mechanism:

[0157] In the feature fusion step, the specific implementation of the feature fusion module based on the interactive attention mechanism is as follows:

[0158] 1) Feature Transformation: Intrinsic attribute features and illumination features are transformed into query vectors, key vectors, and value vectors respectively through a 3×3 convolutional layer. The formula is:

[0159]

[0160]

[0161]

[0162] : Query vector.

[0163] : Key vector.

[0164] Value vector.

[0165] : The weight parameters of the convolution kernel.

[0166] 2) Energy Matrix Generation: The energy matrix is ​​generated by calculating the similarity between the query vector and the key vector using the dot product. The formula is:

[0167]

[0168] Energy matrix: representing the similarity between query vector and key vector.

[0169] 3) Attention Weight Matrix Generation: The energy matrix is ​​normalized using the Softmax function to generate the attention weight matrix. The formula is:

[0170]

[0171] Attention weight matrix, representing the first... The features of the i-th position are related to the i-th Attention weights for features at each location.

[0172] 4) Feature Fusion: Multiply the attention weight matrix by the value vector to generate the fused feature map. The formula is:

[0173]

[0174] : The fused feature map.

[0175] : Transpose of the attention weight matrix.

[0176] 5) Residual Connections: The fused feature map is added to the original feature map through residual connections, where the weights γ of the residual connections are learnable parameters. The formula is:

[0177]

[0178] 6) Weighted Fusion: The feature maps after residual concatenation are weighted and fused using two parallel multilayer perceptrons (MLPs) to generate the final classification features. The formula is:

[0179]

[0180] 6. Generation process of the feature fusion module in the interactive attention mechanism:

[0181] In the feature fusion module of the interactive attention mechanism, the generation process of the query vector, key vector, and value vector is as follows:

[0182] 1) Convolution operation: Use a 3×3 convolution kernel to perform convolution operations on the intrinsic attribute features and illumination features respectively to generate query vector, key vector and value vector.

[0183] 2) Energy Matrix Generation: The similarity between the query vector and the key vector is calculated by dot product to generate the energy matrix.

[0184] 3) Attention weight matrix generation: The energy matrix is ​​normalized using the Softmax function to generate the attention weight matrix.

[0185] 4) Feature fusion: Multiply the attention weight matrix with the value vector to generate the fused feature map.

[0186] 5) Residual connection: The fused feature map is added to the original feature map through residual connection, where the weight γ of the residual connection is a learnable parameter with an initial value of 1.0.

[0187] 6) Weighted fusion: The feature maps after residual connection are weighted and fused through two parallel multilayer perceptrons (MLPs) to generate the final classification features.

[0188] 7) Multi-scale Gaussian kernel filtering of the Retinex decomposition module:

[0189] The specific implementation of multi-scale Gaussian kernel filtering in the Retinex decomposition module is as follows:

[0190] 1) Multi-scale Gaussian kernel: Filtering is performed using Gaussian kernels of different scales (the scale parameter σi is dynamically adjusted according to the local and global information of the hyperspectral image).

[0191] 2) Local and global information: By using multi-scale Gaussian kernel filtering, the local and global information of the hyperspectral image is fully considered, thereby enhancing the robustness of Retinex decomposition.

[0192] 8. Convolutional Neural Network Architecture for Feature Extractors:

[0193] Both the intrinsic attribute component feature extractor and the illumination component feature extractor use convolutional neural networks for feature extraction, as implemented below:

[0194] 1) Convolutional Neural Networks: Extract local features through multiple convolutional layers.

[0195] 2) Batch normalization and ReLU activation function: Batch normalization and ReLU activation function accelerate the training process and improve the convergence speed and stability of the model.

[0196] 9. Dynamic weight adjustment of the feature fusion module in the interactive attention mechanism:

[0197] In the feature fusion module based on the interactive attention mechanism, the specific implementation of dynamic weight adjustment is as follows:

[0198] 1) Dynamic weight adjustment: The weights of inherent attribute features and lighting features are dynamically adjusted through an interactive attention mechanism.

[0199] 2) Information complementarity: Achieve information complementarity between inherent attribute features and illumination features to improve classification accuracy.

[0200] 10. Visualize the network to generate classification diagrams:

[0201] After the classification step is completed, the final classification map is generated through a visualization network, as follows:

[0202] 1) Visualization Network: Generate a classification map from the classification results using a visualization network.

[0203] 2) Intuitive understanding: Facilitates users' intuitive understanding of the classification results, enhancing the interpretability and practicality of the method. Specific implementation examples:

[0205] The overall framework of the hyperspectral image classification method proposed in this invention is as follows: Figure 1 As shown.

[0206] First, Retinex decomposition is performed on the hyperspectral image;

[0207] Secondly, the intrinsic attribute components and illumination components obtained from the decomposition are used to extract features separately through a dual-branch network;

[0208] Then, feature fusion and classification are performed using an interactive attention-based feature fusion module;

[0209] Finally, the final classification map is generated by visualizing the network.

[0210] I. Hyperspectral Retinex Decomposition Module

[0211] Retinex theory aims to explain how the human visual system perceives color and brightness, maintaining relative constancy even under significantly varying lighting conditions. The theory posits that the human visual system, in perceiving color and brightness, relies not only on the intensity of local pixels but also on surrounding contextual information. Mathematically, the Retinex model can be represented as:

[0212]

[0213] In the formula These are the pixel values ​​of the input image. It is a reflectance image, which represents the inherent properties of an object's surface. This is a lighting image, representing changes in ambient lighting. The goal is to analyze the input image... Estimate reflectance image This is because reflectance images better reflect the inherent characteristics of an object, while illumination images include the influence of ambient light.

[0214] In hyperspectral image processing, Retinex theory can also effectively enhance image contrast and detail while reducing the impact of illumination variations, but it requires consideration of information from multiple spectral bands. Assuming the hyperspectral image is... Where B represents the spectral band. The pixels of a hyperspectral image can be decomposed into reflectance. illumination The product of:

[0215]

[0216] We use Gaussian kernels of different scales. Filter the image after logarithmic transformation for each band:

[0217]

[0218] By subtracting the logarithmically transformed image from the Gaussian filtered results at each scale, reflectance images (intrinsic attribute components) at different scales are obtained:

[0219]

[0220] The final reflectance image is obtained by weighted averaging of reflectance images at different scales.

[0221]

[0222] Finally, an exponential transform is performed on the final reflectance image and the previously filtered image of illumination changes to restore it to linear space:

[0223]

[0224]

[0225] Through the above methods, this invention successfully applies Retinex theory to the field of hyperspectral imaging, achieving the separation of inherent properties and illumination variations in hyperspectral images.

[0226] II. Two-branch feature extractor

[0227] 1) Inherent Attribute Component Feature Extractor

[0228] The intrinsic attribute component feature extractor effectively captures and represents features related to the intrinsic attributes of hyperspectral images, enabling accurate identification and differentiation of different land cover types in subsequent classification tasks. By deeply mining key features such as material characteristics and texture information from the intrinsic attribute components, it improves the model's representational power and classification accuracy.

[0229] First, the input layer receives the raw, inherent attribute components. Here, H and W represent the height and width of the image, respectively, and B represents the number of bands. To reduce the number of channels, simplify subsequent calculations, and retain important feature information, a 1×1×1 convolutional layer is used to reduce the number of input bands to 30 channels, generating an initial feature map:

[0230]

[0231] Next, more complex local features are extracted step by step through a series of multi-layer convolutional layers. Convolutional kernels of sizes 7×3×3, 5×3×3, and 3×3×3 are used to obtain feature maps after local feature extraction:

[0232]

[0233] Then, the feature maps of the multi-channel channels are reshaped. Reorganized into two-dimensional feature maps This allows for the next convolution operation. Finally, a 3×3×3 convolution kernel is used to reduce the number of channels from 576 to 64, generating feature maps.

[0234]

[0235] To stabilize the input distribution of each layer, accelerate the training process, and reduce internal covariate bias, a batch normalization layer and a ReLU activation function are applied after each convolutional layer. To reduce the model's dependence on training data and improve its generalization ability, a Dropout layer is added, resulting in the final feature map.

[0236]

[0237] In order to capture feature maps The global information is obtained by performing global average pooling on each channel of the feature map Fr in the spatial dimension, resulting in... Specifically, for each channel c, calculate its average value across all spatial locations:

[0238]

[0239] To capture the local saliency of the feature map Fr, a global max pooling operation is used to transform the feature map... Each channel is max-pooled in the spatial dimension to obtain Specifically, for each channel c, calculate its maximum value at all spatial locations:

[0240]

[0241] The two vectors obtained from global average pooling and global max pooling , They are concatenated together to form a new feature vector. :

[0242]

[0243] By using a fully connected layer (FC) and the ReLU activation function, the feature vector is... Mapping to intermediate features The purpose of this layer is to compress the feature vectors into a lower-dimensional space, while introducing non-linearity to better capture the relationships between features.

[0244]

[0245] Then, another fully connected layer and a sigmoid activation function are used to generate channel attention weights ωr. The sigmoid activation function restricts the output to the range [0,1], ensuring that the weights of each channel are within a reasonable range.

[0246]

[0247] Finally, channel attention weights Compared with the original feature map Element-wise multiplication yields the enhanced feature map. This amplifies important channels and suppresses less important channels, thereby enhancing the expressive power of the feature map.

[0248]

[0249] During training, the feature extractor is trained using the cross-entropy loss function to minimize the difference between the predicted values ​​and the true labels. The cross-entropy loss function is defined as follows:

[0250]

[0251] In the formula, the predicted value is p, the true label is y, and C represents the number of categories.

[0252] 2) Illumination component feature extractor

[0253] The role of the illumination component feature extractor is to effectively capture and represent illumination-related features in hyperspectral images, enabling accurate identification and differentiation of different land cover types in subsequent classification tasks. By focusing on key features such as environmental changes, it improves the model's representational power and classification accuracy. When processing hyperspectral images, it concentrates on extracting and enhancing features related to illumination conditions. These features are particularly crucial for accurately identifying and differentiating land cover types when illumination conditions vary significantly. It employs the same deep learning architecture for feature extraction as the intrinsic attribute component feature extractor and also uses cross-entropy loss for training.

[0254] III. Feature Fusion Module Based on Interactive Attention Mechanism

[0255] The extracted reflectance and illumination features each contain useful information, and they are complementary. To fully utilize this complementary information, we introduce an interactive attention mechanism for feature fusion. Through this mechanism, the model can dynamically adjust the weights between these two features, thereby better capturing their complementary information, extracting richer feature representations, and improving the overall classification performance of the model.

[0256] First, the input feature maps Fr1 and Fr2 are converted into query vectors, key vectors, and value vectors, respectively:

[0257]

[0258]

[0259]

[0260] In the formula It queries the convolution kernel weights of the projection. These are the convolution kernel weights of the key projection. These are the kernel weights of the value projection.

[0261] The energy matrix represents the similarity between features at each position in Fr1 and features at each position in Fr2. Similarity is calculated using the dot product, quantifying the correlation between different positions. The energy matrix E is obtained by performing a dot product operation on the query vector Q and the key vector K.

[0262]

[0263] By using softmax, we ensure that the sum of the attention weights at each position is 1, which gives the attention weights a probability distribution property, allowing for more efficient weight allocation.

[0264]

[0265] In the formula This represents the attention weight of the feature at position i to the feature at position j.

[0266] By applying the attention weight matrix A to the value vector V, a new feature map O can be obtained, which incorporates complementary information between X1 and X2.

[0267]

[0268] To maintain feature stability and prevent gradient vanishing, we introduce a learnable scaling parameter γ and perform a residual connection between the new feature map O1 and the original feature map X1:

[0269]

[0270] Finally, the feature maps obtained through the interactive attention mechanism are passed through two parallel MLPs, then weighted, fused, and classified.

[0271] Classification effect:

[0272] I. Introduction to the Dataset

[0273] 1) Pavia University: This dataset was collected in 2001 at the University of Pavia, Italy, using the Reflection Optical Systems Imaging Spectrometer (ROSIS). The dataset consists of 103 effective bands and has a size of 610×340. The spatial resolution is 1.3 m / pixel, and the wavelength range is 0.43–0.86 μm. The dataset contains a total of 42,776 labeled pixels.

[0274] 2) IndianPines: This dataset consists of images of the Indian Pine region in Indiana, USA, taken in 1992 by the Airborne Visible-Infrared Imaging Spectroradiometer (AVIRIS). The dataset comprises 200 valid bands, is 145×145 pixels in size, includes 16 categories, and ranges in wavelength from 0.4 to 2.5 μm. The dataset contains a total of 10,249 labeled pixels.

[0275] II. Classification Results

[0276] 1) Hyperspectral image dataset Pavia University:

[0277] Figure 2 This is a pseudo-color image of PU.

[0278] Figure 3 This is a diagram illustrating the classification of PU.

[0279] Figure 4 This is a real label image of PU flooring.

[0280] 2) Hyperspectral image dataset Indian_Pines(IP):

[0281] Figure 5 This is a pseudo-color image of the IP address.

[0282] Figure 6 This is a screenshot showing the effect of IP classification.

[0283] Figure 7This is a real IP ground truth label map.

[0284] The present invention was tested on two widely used hyperspectral image datasets, with the training set and test set split in a ratio of 0.1:99.9. The classification accuracy reached 95.2% and 92.8% respectively, which is significantly better than existing methods, demonstrating the effectiveness and superiority of the proposed method.

[0285] It should be noted that, in this document, the terms "comprising," "including," and any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Specific examples have been used in this document to illustrate the principles and implementation methods of the present invention. These examples are merely for the purpose of helping to understand the method and core ideas of the present invention. The above descriptions are only preferred embodiments of the present invention. It should be pointed out that, due to the limitations of written expression and the objective existence of infinite specific structures, those skilled in the art can make several improvements, modifications, or variations without departing from the principles of the present invention, and can also combine the above technical features in an appropriate manner. These improvements, modifications, variations, or combinations, or the direct application of the concept and technical solution of the present invention to other situations without modification, should all be considered within the scope of protection of the present invention.

Claims

1. A hyperspectral image classification method based on the Retinex model, characterized in that, Includes the following steps: Retinex decomposition was performed on the hyperspectral image to obtain intrinsic property components and illumination components; The intrinsic property component and the illumination component are extracted separately by a dual-branch network. The intrinsic property component feature extractor is used to extract the material features and texture information of the hyperspectral image, and the illumination component feature extractor is used to extract the illumination features of the hyperspectral image. The extracted intrinsic attribute features and illumination features are fused using a feature fusion module based on an interactive attention mechanism to generate a fused feature representation. The fused features are classified to generate the classification results of the hyperspectral image; The feature fusion module based on the interactive attention mechanism includes: The inherent attribute features and illumination features are respectively converted into query vector Query, key vector Key and value vector Value through convolutional layers, where the convolutional kernel size is 3×3; The energy matrix is ​​generated by calculating the similarity between the query vector and the key vector using the dot product; The energy matrix is ​​normalized using the Softmax function to generate the attention weight matrix; Multiply the attention weight matrix by the value vector to generate the fused feature map; The fused feature map is added to the original feature map through residual connections, where the weight γ of the residual connections is a learnable parameter; The residual concatenated feature maps are weighted and fused using two parallel multilayer perceptrons (MLPs) to generate the final classification features. In the feature fusion module of the interactive attention mechanism, the generation process of the query vector Query, key vector Key, and value vector Value includes: Use a 3×3 convolution kernel to perform convolution operations on the intrinsic attribute features and the illumination features respectively to generate query vector, key vector and value vector; The energy matrix is ​​generated by calculating the similarity between the query vector and the key vector using the dot product; The energy matrix is ​​normalized using the Softmax function to generate the attention weight matrix; Multiply the attention weight matrix by the value vector to generate the fused feature map; The fused feature map is added to the original feature map through residual connections, where the weight γ of the residual connections is a learnable parameter with an initial value of 1.0; The residual concatenated feature maps are then weighted and fused using two parallel multilayer perceptrons (MLPs) to generate the final classification features.

2. The hyperspectral image classification method based on the Retinex model according to claim 1, characterized in that, The Retinex decomposition step includes: Perform a logarithmic transformation on each band of the hyperspectral image; The illumination component is obtained by filtering with Gaussian kernels of different scales, where the scale parameter σ of the Gaussian kernel is... i Dynamically adjust based on local and global information from the hyperspectral image; The reflectance image is obtained by subtracting the illumination component from the logarithmically transformed image. The reflectance image represents the inherent property components of the hyperspectral image. Weighted averages are applied to reflectance images at different scales to obtain the final intrinsic property components; The intrinsic property components and illumination components are subjected to an exponential transformation to restore the linear space.

3. The hyperspectral image classification method based on the Retinex model according to claim 1, characterized in that, The intrinsic attribute component feature extractor includes: The input's inherent attribute components are reduced in dimensionality using convolutional layers to generate an initial feature map; Local features are extracted step by step through multiple convolutional layers, which include convolutional kernels of different sizes; Global pooling operations are used to capture global and local salient information, generating global average pooling vectors and global max pooling vectors. The global average pooling vector and the global max pooling vector are concatenated and then channel attention weights are generated through a fully connected layer and an activation function. The channel attention weights are multiplied element-wise with the original feature map to generate the enhanced feature map.

4. The hyperspectral image classification method based on the Retinex model according to claim 1, characterized in that, The illumination component feature extractor uses the same deep learning architecture as the intrinsic attribute component feature extractor for feature extraction and is trained using the cross-entropy loss function.

5. The hyperspectral image classification method based on the Retinex model according to claim 1, characterized in that, The Retinex decomposition module uses a multi-scale Gaussian kernel for filtering, with the scale parameter σ of the Gaussian kernel being... i The system is dynamically adjusted based on the local and global information of the hyperspectral image to fully consider both local and global information.

6. The hyperspectral image classification method based on the Retinex model according to claim 1, characterized in that, Both the intrinsic attribute component feature extractor and the illumination component feature extractor use convolutional neural networks for feature extraction, and the training process is accelerated by batch normalization and ReLU activation function.

7. The hyperspectral image classification method based on the Retinex model according to claim 1, characterized in that, The feature fusion module based on the interactive attention mechanism dynamically adjusts the weights of inherent attribute features and illumination features to achieve information complementarity, thereby improving classification accuracy.

8. The hyperspectral image classification method based on the Retinex model according to claim 1, characterized in that, The method generates the final classification map through a visualization network, making it easier for users to intuitively understand the classification results.

Images