Hyperspectral image cross-scene classification method based on dynamic mark enhanced mamba

Abstract

The invention discloses a hyperspectral image cross-scene classification method based on dynamic mark enhanced mamba, comprising the following steps: inputting an obtained target domain image into a trained classification model, and outputting a prediction category of the target domain image; wherein the training of the classification model comprises the following steps of: 1, expanding source domain image samples into block sequence data in a spatial dimension in sequence, and adding a position code and a spectral dimension projection through a block embedding layer; 2, performing dynamic mark enhancement on the features of the data obtained after projection; step 3, extracting the features after dynamic mark enhancement; 4, repeating the step 2 and the step 3 for N times; 5, classifying the finally extracted features through a classifier head, obtaining the prediction probability of a sample corresponding to each feature, calculating classification compensation loss, and adjusting classifier parameters by using the classification compensation loss as a feedback signal; the method has the advantages of being good in generalization ability, high in classification precision and high in efficiency.

Description

Technical Field

[0001] The invention belongs to the technical field of remote sensing image processing, and in particular relates to a hyperspectral image cross-scene classification method based on dynamic labeling enhanced mamba. Background Art

[0002] Hyperspectral images (HSI) contain detailed spectral information of hundreds of narrow bands [1], [2]. Since materials have unique spectral reflectance characteristics, using HSI to identify material categories has significant advantages [3]. Classification techniques using HSI have been widely used in agriculture, environmental monitoring, mining and other fields [4]–[7]. However, changes in environmental conditions, sensor characteristics and material properties can cause domain shifts between data from different scenes [8]. This results in the inability of models trained on data from one scene (source domain) to maintain excellent classification performance in new scenes (target domain) [9]. In order to reduce the impact of domain shift on model performance, cross-scene classification techniques based on human-computer interaction technology have been proposed. Currently, domain adaptation (DA) and domain generalization (DG) are the two main research ideas for this technology.

[0003] The goal of domain adaptation is to train the model using labeled source domain image samples and a minimum number of unlabeled target domain image samples, thereby maintaining a high classification accuracy in the unseen target domain

[10] . According to the way of reducing the difference in data distribution, DA methods can be divided into three categories: difference-based methods, adversarial methods, and reconstruction-based methods

[11] . Difference-based data analysis methods use different indicators to reduce the feature differences between different domains. Long et al.

[12] minimized the domain shift by reducing the feature distance in the kernel Hilbert space. The core idea of ​​the adversarial DA method is to constrain the similarity of feature representations of different domains by conducting adversarial training at the deep network layer. Yu et al.

[13] used adversarial learning to confuse the features learned by the model to ensure that the feature distributions on different domains are similar. The reconstruction-based DA method uses an autoencoder to minimize the cross-domain reconstruction error. Xavier et al.

[14] reconstructed datasets of different domains to force the model to learn the common features of the source and target domain data. However, whenever new TD data is obtained, the DA method must go back and retrain using the new TD data. This greatly limits the practical application scenarios of the model.

[0004] In contrast, the goal of DG is to develop models that can generalize in unseen domains without acquiring any TD data during training

[15] –

[17] . DG techniques focus on learning domain-independent features, i.e., domain-invariant features. These features are robust to changes in different scenarios, allowing the model to perform well in new scenarios without the need for domain-specific adaptation. Based on the number of source domains available during model training, DG can be divided into single-source domain generalization (SDG) and multi-source domain generalization (MDG) research directions. MDG can capture domain-invariant features from multi-source domain image samples, while SDG is limited to learning from a single source domain. Therefore, SDG is more difficult to learn domain-invariant features than MDG. In order to obtain more domain-invariant features from limited single-source domain image samples, many SDG methods have adopted data augmentation and adversarial learning methods. Zheng et al.

[18] augmented SD data with semantic transformation and used the augmented data to learn a robust model. Qiao et al.

[19] used adversarial training combined with meta-learning to generate fictitious and efficient new samples. Cugu et al.

[20] proposed a method called visually disruptive attention consistency, which transforms SD samples by visually disrupting them to generate samples with richer features.

[0005] The existing SDG technology for cross-scene classification of hyperspectral images is mainly based on convolutional neural networks (CNN). However, the convolution kernel operation causes the model to emphasize local object features, resulting in overfitting of the source domain and limiting its generalization ability. With the continuous improvement of the requirements for classification technology, how to improve the classification accuracy is a problem that needs to be solved.

[0006] To solve this problem, this paper proposes a cross-scene classification method for hyperspectral images based on dynamic labeling enhanced mamba.

[0007] This article involves the following references

[0008] [1]Hong Li, Guangrun Xiao, Tian Xia, YY Tang, Luoqing Li, "Hyperspectral Image Classification Using Functional Data Analysis," IEEETransactions on Cybernetics, vol. 44, no. 9, pp. 1544–1555, Sep. 2014.

[0009] [2]Bing Lu, Phuong D. Dao, Jiangui Liu, Yuhong He, Jiali Shang,“Recent advances of hyperspectral imaging technology and applications inagriculture,” Remote Sensing, vol. 12, no. 16, p. 2659, 2020.

[0010] [3]Shutao Li, Weiwei Song, Leyuan Fang, Yushi Chen, Pedram Ghamisi,Jon Atli Benediktsson, “Deep learning for hyperspectral image classification:An overview,” IEEE Transactions on Geoscience and Remote Sensing, vol. 57,no. 9, pp. 6690–6709, 2019.

[0011] [4]Marie Weiss, Frédéric Jacob, Grgory Duveiller, “Remote sensing foragricultural applications: A meta-review,” Remote sensing of environment,vol. 236, p. 111402, 2020.

[0012] [5]Mary B. Stuart, Andrew JS McGonigle, Jon R. Willmott,“Hyperspectral imaging in environmental monitoring: A review of recentdevelopments and technological advances in compact field deployable systems,”Sensors, vol. 19, no. 14, p. 3071, 2019.

[0013] [6]Diana Krupnik, Shuhab Khan, “Close-range, ground-basedhyperspectral imaging for mining applications at various scales: Review andcase studies,” Earth-science reviews, vol. 198, p. 102952, 2019.

[0014] [7]Lei Ma, Yu Liu, Xueliang Zhang, Yuanxin Ye, Gaofei Yin, Brian AlanJohnson, “Deep learning in remote sensing applications: A meta-analysis andreview,” ISPRS journal of photogrammetry and remote sensing, vol. 152, pp.166–177, 2019.

[0015] [8]Wojciech Masarczyk, Przemys\law G\lomb, Bartosz Grabowski, MateuszOstaszewski, “Effective training of deep convolutional neural networks forhyperspectral image classification through artificial labeling,” RemoteSensing, vol. 12, no. 16, p. 2653, 2020.

[0016] [9]Debaleena Datta, Pradeep Kumar Mallick, Akash Kumar Bhoi, MuhammadFazal Ijaz, Jana Shafi, Jaeyoung Choi, “Hyperspectral Image Classification:Potentials, Challenges, and Future Directions,” Computational Intelligenceand Neuroscience, vol. 2022, pp. 1–36, Apr. 2022.

[0017]

[10] Fuzhen Zhuang, Zhiyuan Qi, Keyu Duan, Dongbo Xi, Yongchun Zhu,Hengshu Zhu, Hui Xiong, Qing He, “A Comprehensive Survey on TransferLearning,” Proceedings of the IEEE, vol. 109, no. 1, pp. 43–76, Jan. 2021.

[0018]

[11] Mei Wang, Weihong Deng, “Deep visual domain adaptation: Asurvey,” Neurocomputing, vol. 312, pp. 135–153, 2018.

[0019]

[12] Mingsheng Long, Yue Cao, Jianmin Wang, Michael Jordan, “Learningtransferable features with deep adaptation networks,” in Internationalconference on machine learning, PMLR, 2015, pp. 97–105.

[0020]

[13] Yaroslav Ganin, Victor Lempitsky, “Unsupervised Domain Adaptationby Backpropagation”.

[0021]

[14] Xavier Glorot, Antoine Bordes, Yoshua Bengio, “Domain adaptationfor large-scale sentiment classification: A deep learning approach,” inProceedings of the 28th international conference on machine learning (ICML-11), 2011, pp. 513–520.

[0022]

[15] Behnam Gholami, Mostafa El-Khamy, Kee-Bong Song, “Latent FeatureDisentanglement for Visual Domain Generalization,” IEEE Transactions on ImageProcessing, 2023.

[0023]

[16] Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, Chen Change Loy,“Domain Generalization: A Survey,” IEEE Transactions on Pattern Analysis andMachine Intelligence, vol. 45, no. 4, pp. 4396–4415, Apr. 2023.

[0024]

[17] Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, Tao Qin,Wang Lu, Yiqiang Chen, Wenjun Zeng, Philip Yu, “Generalizing to UnseenDomains: A Survey on Domain Generalization,” IEEE Transactions on Knowledgeand Data Engineering, pp. 1–1, 2022.

[0025]

[18] Guangtao Zheng, Mengdi Huai, Aidong Zhang, "AdvST: RevisitingData Augmentations for Single Domain Generalization," Proceedings of the AAAIConference on Artificial Intelligence, vol. 38, no. 19, Art. no. 19, Mar.2024.

[0026]

[19] Fengchun Qiao, Long Zhao, Xi Peng, “Learning to learn singledomain generalization,” in Proceedings of the IEEE / CVF Conference on ComputerVision and Pattern Recognition, 2020, pp. 12556–12565.

[0027]

[20] Ilke Cugu, Massimiliano Mancini, Yanbei Chen, Zeynep Akata, "Attention consistency on visual corruptions for single-source domaingeneralization," in Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 4165–4174. Summary of the invention

[0028] The purpose of the present invention is to provide a hyperspectral image cross-scene classification method based on dynamic labeling enhanced mamba in view of the problems existing in the prior art. The scheme has the advantages of good generalization ability, high classification accuracy and high efficiency.

[0029] To achieve the above object, the technical solution adopted by the present invention is: a hyperspectral image cross-scene classification method based on dynamic labeling enhanced mamba, comprising the following steps:

[0030] Obtain target domain image;

[0031] The acquired target domain image is input into the trained classification model, and the predicted category of the target domain image is output; wherein the training of the classification model includes:

[0032] Step 1: Get source domain image samples, expand them sequentially in the spatial dimension into block sequence data, and add position encoding and spectral dimension projection through the block embedding layer;

[0033] Step 2: Dynamically mark and enhance the features of the data obtained after position encoding and spectral dimension projection;

[0034] Step 3: Extract the features enhanced by dynamic marking by constructing hidden states through the mamba layer;

[0035] Step 4: Repeat steps 2 and 3 N times to obtain the final extracted features;

[0036] Step 5: Classify the final extracted features through the classifier head, obtain the prediction probability of the samples corresponding to each feature, and calculate the classification compensation loss. Use the classification compensation loss as a feedback signal to adjust the classifier parameters and output the trained classification model.

[0037] In the above scheme, during testing, the target domain image is input into the model trained by steps 1-5, and the predicted category of the target domain image is output by the model. Steps 1-5 are the training steps of the model. Step 1 first expands the source domain image sample into block sequence data in the spatial dimension, and encodes the position of the block embedding layer and projects the spectral dimension to facilitate subsequent recognition calculations; in step 2, the dynamic label enhancement module enhances the features of the input block sequence data to facilitate recognition. Step 3, the hidden state is constructed by the mamba layer to extract the features enhanced by the dynamic label; Step 4 repeats steps 2 and 3 to process the input data; Step 5, after multiple feature dynamic label enhancement and extraction, the classifier head is used to classify the final extracted features to obtain the predicted probability of the sample corresponding to each feature. Even if the original features are enhanced, after a large amount of training, the feature range corresponding to the corresponding category learned by the model will be reduced to a small extent, which may cause the model to be biased towards a specific field and unable to generalize well to the target field. Therefore, the classification compensation loss is used to adjust the model parameters to suppress the overfitting of the model to a specific field.

[0038] Furthermore, the features of the data obtained after position encoding and spectral dimension projection are dynamically labeled and enhanced, including object label selection and background label interference. The object label selection is used to retain labels with original sample information in the features; the background label interference is used to use the features of random data to create contextual labels with more complex information, and the output of the object label selection and the output of the background label interference are combined to obtain enhanced features.

[0039] Object tag selection is used to retain tags with original sample information in the features, which is convenient for later identification of the classification probability of samples corresponding to the features. Background tag interference makes context tags for the features of random data, and combines the output of object tag selection and the output of background tag interference to obtain enhanced features. Sample features are transformed by perturbing context information while retaining object information tags.

[0040] Furthermore, the background mark interference also includes: shuffling the input feature batch sequence, generating random features, and processing and reorganizing based on the generated random features.

[0041] By shuffling the feature batch sequence, a random feature sequence is generated and recombined with the original feature sequence, so that the features are enhanced.

[0042] Furthermore, the object tag selection also includes: using a self-attention mechanism, a multi-layer perception mechanism and a memory attention map to search for spatial relationships between tags, for a given data feature obtained after position encoding and spectral dimension projection The label space relationship diagram of It is expressed as:

[0043] (1)

[0044] Through the self-attention mechanism , Multilayer Perceptron and memory attention map To facilitate the determination of features Marking spatial relations. In the object tag selection submodule, the self-attention mechanism and multi-layer perceptron are used to search for spatial relations between tags, and a memory attention map is set to alleviate the violent fluctuations in the spatial relationship search.

[0045] Furthermore, the object tag selection also includes:

[0046] Calculate the activation value of each marker in the marker space relationship graph;

[0047] According to the calculated activation values ​​of all the marks, a spatial relationship activation value map is obtained;

[0048] Perform thresholding operation based on the spatial relationship activation value map to obtain the object information activation mask;

[0049] Object information activation masks are used to filter and extract object landmarks from raw features.

[0050] Through activation value and threshold processing, a binary object information activation mask is obtained to filter and extract object tags.

[0051] Furthermore, the background mark interference also includes:

[0052] pass The function breaks up the spatial relationship between feature markers;

[0053] Adaptive instance normalization for the scattered features The spectral information is disturbed to obtain the feature , using the original features Activation mask of right Perform additional activation to obtain new background markers ;

[0054] Finally, reorganize the object tags extracted by filtering and new background markers Get enhanced features.

[0055] pass The function of breaking up the spatial relationship between feature markers is performed in the background marker interference submodule.

[0056] Furthermore, step 3 includes the following steps:

[0057] First, the enhanced features are normalized and projected into two features , ;

[0058] feature After replication and inversion, two directional features are formed: forward feature and backward features , and At the same time, it is one-dimensionally convolved And the activation function deal with;

[0059] The processed features are input into the mamba model and combined with the activated features , calculate the forward output and backward output;

[0060] Finally, the extracted features are calculated based on the forward output, backward output and enhanced features.

[0061] By normalizing and projecting the enhanced features into features , , and then combine convolution, activation and mamba models to accurately calculate the extracted features.

[0062] Further, classification compensation loss The calculation formula is as follows:

[0063]

[0064] in is the marker selection ratio in dynamic marker enhancement, is the cross entropy loss calculated using the labels and predicted probabilities of the original samples. It is the cross entropy loss calculated using the labels and predicted probabilities corresponding to random samples.

[0065] Compensate loss by calculating classification To adjust the classifier parameters.

[0066] The cross entropy loss is calculated by using the labels of the original samples and the predicted probabilities , using the labels and predicted probabilities corresponding to random samples to calculate the cross entropy loss , through the cross entropy loss , Calculate the classification compensation loss , using classification compensation loss It is used as a feedback signal to adjust the classifier parameters to improve the classifier's prediction performance for training samples.

[0067] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer executes the above method.

[0068] Cross-scene classification of hyperspectral images (HSI) based on single source domain generalization (SDG) focuses on developing a model that can effectively classify images from unseen target domains using only source domain images without retraining. Most existing SDG methods for cross-scene classification rely on convolutional neural networks (CNNs). However, the convolution kernel operation causes the model to emphasize local object features, resulting in overfitting of the source domain and limiting its generalization ability. State-space model-based methods excel in image classification by capturing global features of different image patches. Based on this concept, we propose a new method named dynamic tag augmented mamba (DTAM) to explore the potential of state-space models in cross-scene classification of human-computer interaction images. The method gradually focuses on the global features of the image by constructing hidden states for hyperspectral images unfolded into long sequences. To further enhance the global features of hyperspectral images, we design a dynamic tag augmentation module to transform sample features by perturbing contextual information while preserving object information labels. In addition, we introduce a classification compensation loss function to suppress the range of features learned by the model using the class labels of random samples.

[0069] Compared with the prior art, the present invention has the following beneficial effects:

[0070] 1. This scheme specially designs a sequence-level dynamic tag enhancement (DTA) module at the front end of the mamba layer. The dynamic tag enhancement module is mainly composed of the object tag selection (OTS) submodule and the context tag perturbation (CTP) submodule. OTS realizes the retention of tags with object information in the original features, and CTP uses random sample features to make context tags with more complex information. By reorganizing the outputs of these two submodules, the original features are implicitly enhanced, further enhancing the global features of the hyperspectral image, improving the recognition accuracy of the features, and improving the classification accuracy;

[0071] 2. This solution uses the classification compensation loss function to suppress the range of features learned by the model using the class labels of random samples, improve the stability of the features learned by the model, avoid overfitting of the source domain, and have good generalization ability;

[0072] 3. This scheme focuses on developing a model for cross-scene classification of hyperspectral images (HSI) based on single source domain generalization (SDG), which can effectively classify images from unseen target domains using only source domain images without retraining and has high classification efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 The present invention is a flowchart of a method for cross-scene classification of hyperspectral images based on dynamic labeling enhanced mamba according to an embodiment of the present invention. DETAILED DESCRIPTION

[0074] The technical solution of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. In the description of the present invention, it should be noted that the terms front, back, left, right, etc. indicate directions or positional relationships based on the directions or positional relationships shown in the drawings, or the directions or positional relationships in which the product is usually placed when in use. They are only for the convenience of describing the present invention or simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed or operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0075] Example

[0076] like Figure 1 As shown, a hyperspectral image cross-scene classification method based on dynamic labeling enhanced mamba includes the following steps:

[0077] Step 1: Train the model and input the source domain image samples into the model. The model will sequentially expand the source domain image samples into block sequence data in the spatial dimension, and add position encoding and spectral dimension projection through the block embedding layer;

[0078] Step 2: Dynamically mark and enhance the features of the data obtained after position encoding and spectral dimension projection;

[0079] Step 3: Extract the features enhanced by dynamic marking by constructing hidden states through the mamba layer;

[0080] Step 4: Repeat steps 2 and 3 N times to extract features;

[0081] Step 5: Classify the extracted features through the classifier head, obtain the prediction probability of the sample corresponding to each feature, and calculate the classification compensation loss, and use the classification compensation loss as a feedback signal to adjust the classifier parameters;

[0082] Step 6: In the test phase, the dynamic labeling enhancement module in the silent model inputs the target domain image sample into the model trained in steps 1-5 to obtain the predicted category of the sample.

[0083] In the above scheme, steps 1 to 5 are the training phase. Step 1 first expands the source domain image sample into block sequence data in the spatial dimension, and encodes the position of the block embedding layer and projects the spectral dimension to facilitate subsequent recognition calculations. In step 2, the dynamic label enhancement module enhances the features of the input block sequence data. In step 3, the features after dynamic label enhancement are extracted through the mamba layer to construct a hidden state. Step 4 repeats steps 2 and 3 to process the input data. Step 5, after multiple feature dynamic label enhancement and extraction, the extracted features are classified using the classifier head to obtain the prediction probability of the sample corresponding to each feature. Even if the original features are enhanced, after a lot of training, the range of features corresponding to the corresponding category learned by the model will be reduced to a small extent, which may cause the model to be biased towards a specific field and unable to generalize well to the target field. Therefore, the classification compensation loss is used to suppress the overfitting of the model to a specific field. In the test phase of step 6, the dynamic label enhancement module remains silent, and the target domain image sample is input into the model trained by steps 1-5 to obtain the predicted category of the sample. Step 4 repeats steps 2 and 3 to repeatedly enhance the features extracted in step 3.

[0084] Furthermore, the features of the data obtained after position encoding and spectral dimension projection are dynamically labeled and enhanced, including object label selection and background label interference. The object label selection is used to retain labels with original sample information in the features; the background label interference is used to use the features of random data to create contextual labels with more complex information, and the output of the object label selection and the output of the background label interference are combined to obtain enhanced features.

[0085] Object tag selection is used to retain tags with original sample information in the features, which is convenient for later identification of the classification probability of samples corresponding to the features. Background tag interference makes context tags for the features of random data, and combines the output of object tag selection and the output of background tag interference to obtain enhanced features. Sample features are transformed by perturbing context information while retaining object information tags.

[0086] In step 1, the input data is represented as , where X is a 3D data block, where , and Represent the spectral dimension of the sample, the spatial dimension of the sample, and the number of samples in the batch sequence respectively. The characteristics of the input data are obtained after position encoding and spectral dimension projection. ,here equal The square of Represents the dimension size after the spectrum dimension is projected. The input data is represented as It is convenient to perform calculations through To represent the data features after adding position encoding and spectral dimension projection, the size of the dimension is adjusted through spectral dimension projection.

[0087] For the cross-scene classification task of HSI, the class label corresponding to X is expressed as The input data in step 1 is sequentially expanded into a two-dimensional patch in the spatial dimension, and the patch embedding layer adds position encoding and spectral dimension projection.

[0088] Furthermore, in step 2, the data features are dynamically marked and enhanced by a dynamic marking enhancement module, which includes an object marking selection (OTS) submodule and a background marking interference (CTP) submodule. In the test phase of step 6, the dynamic marking enhancement module is used in the silent model.

[0089] During testing, the dynamic labeling enhancement module is silenced to improve data classification efficiency.

[0090] Furthermore, the background mark interference also includes: shuffling the input feature batch sequence, generating random features, and processing and reorganizing based on the generated random features.

[0091] By shuffling the feature batch sequence, a random feature sequence is generated and recombined with the original feature sequence, so that the features are enhanced.

[0092] Step 2 includes the following steps. Given the characteristics of the input data , shuffle the feature batch sequence to generate random features ,in Represents the number of new batch sequence samples, and the original features of the input data Input object tag selection (OTS) submodule, random features The input is processed and reorganized into the background tag interference (CTP) submodule to obtain the enhanced features.

[0093] By adding the feature Shuffle to generate random features in new sequences ,The enhanced features are obtained after the features are processed and reorganized by the object marker selection (OTS) submodule and the background marker perturbation (CTP) submodule.

[0094] In the subsequent labeling enhancement, batch sequences of random features and The sequence remains consistent.

[0095] Furthermore, the object tag selection also includes: using a self-attention mechanism, a multi-layer perception mechanism and a memory attention map to search for spatial relationships between tags, for a given data feature obtained after position encoding and spectral dimension projection The label space relationship diagram of It is expressed as:

[0096] (1)

[0097] Through the self-attention mechanism , Multilayer Perceptron and memory attention map To facilitate the determination of features Marking spatial relations. In the object tag selection submodule, the self-attention mechanism and multi-layer perceptron are used to search for spatial relations between tags, and a memory attention map is set to alleviate the violent fluctuations in the spatial relationship search.

[0098] Furthermore, the object tag selection also includes:

[0099] Calculate the activation value of each marker in the marker space relationship graph;

[0100] According to the calculated activation values ​​of all the marks, a spatial relationship activation value map is obtained;

[0101] Perform thresholding operation based on the spatial relationship activation value map to obtain the object information activation mask;

[0102] Object information activation masks are used to filter and extract object landmarks from raw features.

[0103] Through activation value and threshold processing, a binary object information activation mask is obtained to filter and extract object tags.

[0104] The following steps are included to calculate the activation value in the label space relationship graph, and the calculation formula is as follows:

[0105] (2)

[0106] in Represents the activation value of the nth marker in the spatial relationship graph. By calculating all markers, the spatial relationship activation value map is obtained. , perform thresholding operation to obtain the object information activation mask , finally, it is possible to use a binary mask From the original features Filter and extract object tags , the formula is as follows:

[0107] (4)

[0108] Activate the value map by setting spatial relationships To perform the activation value of each tag, the object information activation mask is obtained after the threshold operation, and finally the object information activation mask is used to obtain the original feature Filter and extract object tags .

[0109] The larger the activation value, the more object information the tag contains.

[0110] Furthermore, the thresholding operation will calculate The percentile is used as the threshold, denoted as ,if If the activation value in is greater than the threshold, the element is set to 1, otherwise it is set to 0. The formula is as follows:

[0111] (3)

[0112] in, is an adjustable marker selection ratio.

[0113] By setting a threshold to compare with the activation value in the spatial relationship activation value map, a binary mask value is obtained.

[0114] Furthermore, the background mark interference also includes:

[0115] pass The function breaks up the spatial relationship between feature markers;

[0116] Adaptive instance normalization for the scattered features The spectral information is disturbed to obtain the feature , using the original features Activation mask of right Perform additional activation to obtain new background markers ;

[0117] Finally, reorganize the object tags extracted by filtering and new background markers Get enhanced features.

[0118] pass The function of breaking up the spatial relationship between feature markers is performed in the background marker interference submodule.

[0119] New features are obtained by breaking up and disturbing the spatial relationship between feature markers. , and then use the activation mask Features Activate to get new background tag , and object tags Recombination enhanced features.

[0120] In the background marker interference submodule, first use The function breaks up the spatial relationship between feature markers at the marker level:

[0121] (5)

[0122] On this basis, the token spectrum stylized components The spectral information is perturbed and the adaptive instance normalization AdaIN is used to achieve this process, which is defined as follows:

[0123] (6)

[0124] The channel mean and standard deviation yes The original style representation of and Is the new style representation, The calculation formula of the original style is as follows:

[0125] (7)

[0126] (8)

[0127] in express The number of markers in The calculation formula for the new style representation of is as follows:

[0128] (9)

[0129] (10)

[0130] in, is the adaptive learning parameter, and then, using the original features Activation mask of right Perform additional activation. Activation mask of The calculation formula is as follows:

[0131] (11)

[0132] Then by using a binary mask Get new background markers , the formula is:

[0133] (12)

[0134] The object label obtained by the final reorganization and new background markers The enhanced features are obtained, and the formula is as follows:

[0135] (13)

[0136] Furthermore, step 3 includes the following steps:

[0137] First, the enhanced features are normalized and projected into two features , ;

[0138] feature After replication and inversion, two directional features are formed: forward feature and backward features , and At the same time, it is one-dimensionally convolved And the activation function deal with;

[0139] The processed features are input into the mamba model and combined with the activated features , calculate the forward output and backward output;

[0140] Finally, the extracted features are calculated based on the forward output, backward output and enhanced features.

[0141] By normalizing and projecting the enhanced features into features , , and then combine convolution, activation and mamba models to accurately calculate the extracted features.

[0142] Given the enhanced features , first normalize it and then project it into the following two features:

[0143] (14)

[0144] Next, is copied and inverted, forming two directional features and ,in represents the forward feature, represents the backward feature; then, and At the same time, it is one-dimensionally convolved And the activation function deal with:

[0145] (15)

[0146] (16)

[0147] The processed features are then input into the mamba model:

[0148] (17)

[0149] (18)

[0150] On this basis, the activated features Used for feature selection on the output of the mamba model:

[0151] (19)

[0152] (20)

[0153] in, Represents Hadamard multiplication, and finally fuses the forward output and the backward output, and adds the jump connection of the initial feature. The specific technology of extracting features is as follows:

[0154] .(twenty one)

[0155] Furthermore, step 5 includes the following steps:

[0156] After multiple feature dynamic labeling enhancement and extraction, the classification head is used Get a sample The predicted probability , using classification compensation loss To suppress overfitting of the model to a specific domain:

[0157] (twenty two)

[0158] in is the marker selection ratio in the dynamic marker enhancement module, is the cross entropy loss calculated using the labels and predicted probabilities of the original samples. It is the cross entropy loss calculated using the label and predicted probability corresponding to the random sample. The above two loss calculation formulas are as follows:

[0159] (twenty three)

[0160] (twenty four)

[0161] in, Represents the original sample The number of batch sequence samples, Represents a random sample The number of batch sequence samples, is the one-hot encoding of the original sample label, is the one-hot encoding of the label corresponding to the random sample, is the cross entropy loss.

[0162] The cross entropy loss is calculated by using the labels of the original samples and the predicted probabilities , using the labels and predicted probabilities corresponding to random samples to calculate the cross entropy loss , through the cross entropy loss , Calculate the classification compensation loss , using classification compensation loss It is used as a feedback signal to adjust the classifier parameters to improve the classifier's prediction performance for training samples.

[0163] Cross Entropy Loss The calculation formula is as follows:

[0164] (25)

[0165] in, Indicates the index of the category.

[0166] The beneficial effects of the present invention are verified by comparative experiments as follows.

[0167] In this experiment, four hyperspectral datasets, Houston 2013, Houston 2018, University of Pavia (UP) and Pavia Center (PC), were selected as datasets to verify the effectiveness of the model. Two sets of cross-scene classification tasks were set to verify the accuracy and reliability of the present invention. The wavelength range of the Houston 2013 dataset is 380-1050 nanometers, containing 144 spectral channels, with a spatial size of 349×1905 pixels and a spatial resolution of 2.5 meters. The wavelength range of the Houston 2018 dataset is the same as that of the Houston 2013 dataset, but there are only 48 spectral channels and a spatial resolution of 1 meter. To meet the experimental requirements, 48 ​​spectral channels were extracted from the Houston 2013 dataset, corresponding to the Houston 2018 dataset, with a wavelength range of 380-1050nm, and an overlapping spatial area of ​​209×955 was retained. The number and categories of samples of the Houston 2013 and Houston 2018 datasets are listed in Table 1. UP has 103 spectral bands with a wavelength range of 430-860 nanometers, and the spatial size of the image is 610×340 pixels. PC has 102 spectral bands with a wavelength range of 430-860 nanometers, and the pixels are 1096×715. Both have the same seven categories. The 102 spectral bands shared by both are retained. The number of samples and categories of the UP and PC datasets are listed in Table 2. In order to comprehensively evaluate the performance of the method of the present invention, a single-source domain expansion network (method 1), a generalized network in the field of language perception (method 2), a local linear unbiased random network (method 3), a spatial spectral enhancement and causal constraint network (method 4) are used here for comparison with the method of the present invention.

[0168] Table 1 Number and categories of samples in Houston 2013 and Houston 2018 datasets

[0169]

[0170] Table 2 Sample numbers and categories of UP and PC datasets

[0171]

[0172] Experimental settings: Two hyperspectral image cross-scene classification tasks were set up in the whole experiment, namely the Houston task and the Pavia task. In the Houston task, the Houston 2013 dataset is used as the SD and the Houston 2018 dataset is used as the TD. In the Pavia task, UP is used as the SD and PC is used as the TD. In all tasks, only the labeled SD is selected for training samples (80% for training and 20% for validation). The method is optimized by adaptive moment estimation (Adam) and the momentum is set to 0.9. All methods are implemented on the Pytorch platform. The test results are expressed as mean ± standard deviation. The experiment is repeated five times and different random seeds are selected.

[0173] Evaluation indicators: Overall Accuracy (OA) and Kappa coefficient are commonly used quantitative evaluation indicators in image classification and can be calculated through confusion matrix.

[0174] Table 3 Confusion matrix

[0175]

[0176] In the table, TP represents the number of pixels that have changed in the true label and are correctly predicted to have changed. FP represents the number of pixels that have not changed in the true label and are incorrectly predicted to have changed. FN represents the number of pixels that have changed in the true label and are incorrectly predicted to have not changed. TN represents the number of pixels that have not changed in the true label and are correctly predicted to have not changed.

[0177] The calculation formulas for these five evaluation indicators are as follows:

[0178]

[0179]

[0180]

[0181] The comparative experimental results and analysis are as follows:

[0182] Table 4 Comparative experimental results

[0183]

[0184] It can be clearly observed from Table 4 that the method of the present invention shows excellent performance in all tasks compared with other methods. The overall accuracy in the Houston task is 2.65% higher than the second place, and the overall accuracy in the Pavia task is 1.06% higher than the second place. In terms of overall accuracy and Kappa coefficient, the present invention has achieved higher values. Kappa coefficient is a commonly used indicator to measure classification accuracy. Its improvement means that the present invention has higher accuracy and stability in classification. In the two tasks, the overall accuracy of the method of the present invention is 1% to 3% higher than that of method six.

[0185] The embodiments of the present application also provide a computer-readable storage medium. The methods described in the above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. Computer-readable media may include computer storage media and communication media, and may also include any medium that can transfer a computer program from one place to another. The storage medium may be any target medium that can be accessed by a computer.

[0186] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A hyperspectral image cross-scene classification method based on dynamic labeling enhanced mamba, characterized in that: The steps include: Obtain target domain image; The acquired target domain image is input into the trained classification model, and the predicted category of the target domain image is output; wherein the training of the classification model includes: Step 1: Get source domain image samples, expand them sequentially in the spatial dimension into block sequence data, and add position encoding and spectral dimension projection through the block embedding layer; Step 2: Dynamically mark and enhance the features of the data obtained after position encoding and spectral dimension projection; Step 3: Extract the features enhanced by dynamic marking by constructing hidden states through the mamba layer; Step 4: Repeat steps 2 and 3 N times to obtain the final extracted features; Step 5: Classify the final extracted features through the classifier head, obtain the prediction probability of the samples corresponding to each feature, and calculate the classification compensation loss. Use the classification compensation loss as a feedback signal to adjust the classifier parameters and output the trained classification model.

2. The hyperspectral image cross-scene classification method based on dynamic labeling enhanced mamba according to claim 1 is characterized in that: Dynamically marking and enhancing the features of the data obtained after position encoding and spectral dimension projection, including object marking selection and background marking interference, wherein the object marking selection is used to retain the mark with the original sample information in the feature; The background label interference is used to use the features of random data to produce context labels with more complex information, and the output of object label selection and the output of background label interference are combined to obtain enhanced features.

3. The hyperspectral image cross-scene classification method based on dynamic labeling enhanced mamba according to claim 2 is characterized in that: The background mark interference also includes: shuffling the input feature batch sequence, generating random features, and processing and reorganizing based on the generated random features.

4. The hyperspectral image cross-scene classification method based on dynamic labeling enhanced mamba according to claim 3 is characterized in that: The object tag selection further includes: using a self-attention mechanism, a multi-layer perception mechanism, and a memory attention map to search for spatial relationships between tags, and for a given data feature obtained after position encoding and spectral dimension projection The label space relationship diagram of It is expressed as: 。 5. The hyperspectral image cross-scene classification method based on dynamic labeling enhanced mamba according to claim 4 is characterized in that: The object tag selection also includes: Calculate the activation value of each marker in the marker space relationship graph; According to the calculated activation values ​​of all the marks, a spatial relationship activation value map is obtained; Perform thresholding operation based on the spatial relationship activation value map to obtain the object information activation mask; Object information activation masks are used to filter and extract object landmarks from raw features.

6. The hyperspectral image cross-scene classification method based on dynamic labeling enhanced mamba according to claim 5 is characterized in that: The background mark interference also includes: pass The function breaks up the spatial relationship between feature markers; Adaptive instance normalization for the scattered features The spectral information is disturbed to obtain the feature , using the original features Activation mask of right Perform additional activation to obtain new background markers ; Finally, reorganize the object tags extracted by filtering and new background markers Get enhanced features.

7. The hyperspectral image cross-scene classification method based on dynamic labeling enhanced mamba according to claim 6 is characterized in that: Step 3 includes the following steps: First, the enhanced features are normalized and projected into two features , ; feature After replication and inversion, two directional features are formed: forward feature and backward features , and At the same time, it is one-dimensionally convolved And the activation function deal with; The processed features are input into the mamba model and combined with the activated features , calculate the forward output and backward output; Finally, the extracted features are calculated based on the forward output, backward output and enhanced features.

8. The hyperspectral image cross-scene classification method based on dynamic labeling enhanced mamba according to claim 1 is characterized in that: Classification of compensation losses The calculation formula is as follows: in is the marker selection ratio in dynamic marker enhancement, is the cross entropy loss calculated using the labels and predicted probabilities of the original samples. It is the cross entropy loss calculated using the labels and predicted probabilities corresponding to random samples.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the computer is caused to perform the method according to any one of claims 1 to 8.

Images