Hyperspectral image classification method and system under cross-scene condition, electronic equipment and medium

Abstract

The invention discloses a hyperspectral image classification method, system and device under a cross-scene condition and a medium, and mainly solves the problems that the existing cross-domain feature space is not rich and the classification boundary is not robust, and the scheme comprises the following steps: obtaining source domain data, a source domain label and target domain data; a classification model composed of a Mama flow network and a ViT flow network in parallel is constructed, and the Mama flow network comprises a Mama block, a spectrum SKR block and two spectrum-oriented classifiers and is used for optimizing spectrum classification boundaries; the ViT flow network comprises a self-adaptive dynamic mask, a ViT block, a space SKR block and two space guiding classifiers, and is used for enriching a feature space and optimizing a space classification boundary; inputting the source domain data, the source domain data label and the target domain data into a classification model, and performing iterative training on the classification model; and inputting the target domain data into the trained classification model and outputting a classification result. The method improves the classification capability of the cross-scene hyperspectral image, and can be used for environmental monitoring and landform change prediction.

Description

Technical Field

[0001] The present invention belongs to the technical field of cross-scene hyperspectral image processing, and in particular relates to a hyperspectral image classification method, system, electronic equipment and medium, which can be used for environmental monitoring and landform change prediction. Background Art

[0002] Cross-scene hyperspectral image classification aims to classify unlabeled target domain data by learning from labeled source domain data. Traditional cross-scene hyperspectral image classification methods can only perform shallow domain alignment representations and have limited performance. Currently, mainstream algorithms have made significant progress by relying on complex deep network architectures and are mainly divided into feature-based and classifier-based methods. Feature-based methods transfer invariant features from the source domain to the target domain, enabling the model to obtain more discriminative and efficient representations at the feature level. Classifier-based methods achieve effective classification of target domain data by gradually generalizing the learned source domain classification boundaries to the target domain.

[0003] In 2024, Cai et al. published an article titled "Mind the Gap: Multi-Level Unsupervised Domain Adaptation for Cross-scene Hyperspectral Image Classification" in the IEEE Transactions on Geoscience and Remote Sensing. They proposed a cross-scene hyperspectral image classification method called MLUDA, which aligns domain distributions at multiple levels, including images and features, to effectively classify the target domain. However, this method lacks a dynamic mask rate for self-supervised learning, resulting in a limited feature space for the model.

[0004] In 2022, Fang et al. published the article "Confident learning-based domain adaptation for hyperspectral image classification" in the IEEE Transactions on Geoscience and Remote Sensing. They proposed the cross-scenario hyperspectral image classification algorithm CLDA, which selects target samples with high pseudo-label confidence as training data to improve the network's discriminative ability. This method focuses on optimizing the model's decision-making level for target domain data categories, achieving excellent classification results. However, the use of a single model for class decision optimization limits the accuracy of classification boundaries, and the introduction of pseudo-labels also introduces the risk of misclassification.

[0005] Patent document CN202311239731.7 discloses an attention-based masked self-supervised domain-adaptive hyperspectral image classification. It restores the entire image by encoding and decoding the masked image, and constructs a dual classifier to classify the entire image. At the feature level, this method improves the model's ability to analyze hyperspectral data through masked self-supervised learning, but because it does not consider the dynamic optimization of the mask, it will lead to limited diversity in the feature space. At the classification level, this method uses dual classifiers to increase the distance between different categories, but due to the lack of consideration of the collaborative optimization of the classification decisions of multiple models, the robustness of the classification boundary will be limited. Summary of the Invention

[0006] The purpose of the present invention is to address the deficiencies of the above-mentioned existing technologies and propose a cross-scene hyperspectral image classification, system, electronic device and medium based on feature optimization and decision collaboration to generate a richer feature space, optimize the classification boundaries containing multi-model decision information, avoid the risk of erroneous pseudo-labeling, and improve the classification accuracy of target domain data.

[0007] The technical ideas for realizing the destination of the present invention are: automatic adjustment based on adaptive feature discovery of dynamic masks to generate a richer feature space, and obtain cross-domain features with high discriminability through dual-stream convolution cross-domain feature extraction of Mamba network and ViT network; category decision based on collaborative optimization of dual-stream classifiers to obtain more accurate classification boundaries while avoiding the risk of erroneous pseudo-labels.

[0008] According to the above ideas, the technical solution of the present invention includes:

[0009] 1. A cross-scene hyperspectral image classification method based on feature optimization and decision collaboration, characterized by comprising:

[0010] (1) Obtain source domain data x s , source domain data label y s , target domain data x t ;

[0011] (2) Construct a classification model consisting of the Mamba flow network and the ViT flow network in parallel:

[0012] The Mamba flow network includes a Mamba block, a spectral SKR block and two spectral-guided classifiers, which are used to optimize the classification boundary of the spectral dimension.

[0013] The ViT flow network includes an adaptive dynamic mask, a ViT block, a spatial SKR block and two spatially guided classifiers, which are used to enrich the feature space and optimize the classification boundary of the spatial dimension.

[0014] (3) The source domain data x s , source domain data label ys , target domain data x t As a training set, input it into the classification model and iteratively train it;

[0015] (4) The target domain data x t It is used as a test set to input the trained classification model and output the classification results.

[0016] Furthermore, the structure and functions of each part of the Mamba flow network are as follows:

[0017] The Mamba block is composed of multiple serially connected Mamba structures, which are used to extract global spectral relationships and cancel the pooling operation between Mamba structures to avoid information loss. Each Mamba structure includes two branches, the first branch is used to process the global relationship between adjacent spectral channels; the second branch is used to process the global relationship between spectral channels that are farther apart.

[0018] The spectral SKR block, which is composed of residual connection Mamba blocks and selective kernel networks SKNets, is used to extract spectral local relationships, help the network extract spectral local patterns between pixels, and refine the expression of local spectral features;

[0019] The two spectrally guided classifiers have the same structure, each consisting of two fully connected layers. The first fully connected layer is used to map spectral features to a low-dimensional space, and the second fully connected layer is used to optimize the clarity of spectral classification boundaries and classify the mapped low-dimensional spectral features.

[0020] Furthermore, the structure and functions of each part of the ViT flow network are as follows:

[0021] The adaptive dynamic mask is composed of an adaptively changing three-dimensional dynamic mask and is used to expand the feature representation space;

[0022] The ViT block, which consists of a self-attention mechanism and a feedforward network, is used to extract global spatial features;

[0023] The spatial SKR block, which is composed of residual connections ViT blocks and selective kernel networks SKNets, is used to extract spatial local relations;

[0024] The two space-oriented classifiers have the same structure, and each classifier consists of two fully connected layers, which are used to map spatial features to a low-dimensional space and optimize the clarity of spatial classification boundaries, respectively, and classify the mapped low-dimensional spatial features.

[0025] 2. A cross-scene hyperspectral image classification system based on feature optimization and decision collaboration, characterized by comprising:

[0026] Data acquisition module, used to obtain source domain data and target domain data;

[0027] The classification model module is used to sequentially connect the Mamba flow network module and the ViT flow network module to achieve collaborative modeling of spectral features and spatial features;

[0028] The training module is used to train the source domain data and the target domain data so that the classification model can obtain the classification ability of the source domain data and the target domain data;

[0029] The testing module is used to test the target domain data and complete cross-scene hyperspectral image classification.

[0030] Furthermore, the classification model module includes:

[0031] Mamba submodule, used to extract spectral global relationships,

[0032] Spectral SKR submodule, used to extract spectral local relations,

[0033] The spectral-guided classification submodule is used to classify spectral dimension features.

[0034] Adaptive dynamic mask submodule for expanding the feature space;

[0035] ViT submodule, used to extract spatial global structure;

[0036] The spatial SKR submodule is used to extract spatial local structures;

[0037] The spatially oriented classification submodule is used to classify spatial dimension features;

[0038] 3. An electronic device, comprising:

[0039] processor and memory;

[0040] The memory stores all program instructions;

[0041] The processor is configured to run the program instructions to perform the image classification method according to any one of claims 1 to 5.

[0042] 4. A computer-readable storage medium, characterized in that the storage medium stores program instructions, and the program instructions are executed to perform the image classification method according to any one of claims 1 to 5.

[0043] Compared with the prior art, the present invention has the following advantages:

[0044] First, the present invention introduces adaptive dynamic masks to apply masking operations to both source and target domain data simultaneously, thereby achieving adaptive multi-view feature acquisition. Therefore, it can expand the diversity of features and construct a richer feature space without adding additional annotation information.

[0045] Secondly, the present invention fully combines the advantages of the Mamba network in modeling spectral sequence information and the ViT network in modeling spatial attention information, and models hyperspectral image data from the spectral dimension and spatial dimension respectively, so as to obtain rich cross-domain features with discriminativeness.

[0046] Third, the present invention designs complementary spectral-guided classifiers and spatial-guided classifiers to achieve complementary enhancement while maintaining their respective discrimination capabilities, which can fully express the spectral boundary and spatial boundary information, thereby improving the overall classification accuracy and generalization performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flowchart of the cross-scene hyperspectral image classification method based on feature optimization and decision collaboration of the present invention;

[0048] Figure 2 It is a schematic diagram of the classification module structure in the method of the present invention;

[0049] Figure 3 This is a block diagram of the cross-scene hyperspectral image classification system based on feature optimization and decision collaboration of the present invention;

[0050] Figure 4 It is a schematic diagram of the structure of the electronic device provided by the present invention;

[0051] Figure 5 The following are simulation results of hyperspectral image classification using the present invention and the existing MLUDA and CLDA methods on the Houston dataset;

[0052] Figure 6 These are simulation results of classifying hyperspectral images using the present invention and existing MLUDA and CLDA methods on the Hyrank dataset. DETAILED DESCRIPTION

[0053] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention are clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the examples described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0054] It should be noted that the step numbers in the specification and claims of the present invention are only for the purpose of clearly describing the embodiments of the present invention and facilitating understanding, and the order of the step numbers is not limited.

[0055] Example 1: Cross-scene hyperspectral image classification method based on feature optimization and decision collaboration:

[0056] Reference Figure 1 The implementation steps of this example include the following:

[0057] Step 1: collect data.

[0058] Obtain the Houston dataset and Hyrank dataset from the public website and convert the source domain data x s , source domain label y s and target domain data x t As a training set, it is used for the iterative training of subsequent classification models. The target domain data does not participate in label supervision during the training phase. It aims to simulate the common "supervised training, unsupervised testing" transfer learning scenario in practical applications, thereby more realistically reflecting the adaptability and robustness of the model under heterogeneous data conditions.

[0059] The target domain data x of the unlabeled Houston dataset and Hyrank dataset t As a test set for subsequent classification model testing.

[0060] Step 2: Construct a classification model.

[0061] Reference Figure 2 The specific implementation of this step includes the following:

[0062] 2.1) Build the Mamba streaming network:

[0063] 2.1.1) A Mamba block consisting of multiple serially connected Mamba structures is established to extract global spectral relationships. Pooling operations between Mamba structures are eliminated to avoid information loss. Each Mamba structure consists of two branches: the first branch processes global relationships between adjacent spectral channels; the second branch processes global relationships between spectral channels that are farther apart.

[0064] 2.1.2) Establish a spectral SKR block composed of residual-connected Mamba blocks and selective kernel networks SKNets to extract spectral local relationships, help the network extract spectral local patterns between pixels, and refine the expression of local spectral features;

[0065] 2.1.3) Build two spectrally-guided classifiers with identical structures. Each classifier consists of two fully-connected layers. The first fully-connected layer maps spectral features to a low-dimensional space, while the second fully-connected layer optimizes the clarity of spectral classification boundaries and classifies the mapped low-dimensional spectral features.

[0066] 2.1.4) Connect the Mamba block, spectral SKR block and two spectral-guided classifiers in sequence to form a Mamba flow network;

[0067] 2.2) Constructing the ViT flow network:

[0068] 2.2.1) Establish an adaptive dynamic mask composed of three-dimensional dynamic masks to expand the feature representation space. This is done by multiplying the hyperspectral data and dynamically masking the hyperspectral data to increase the diversity of spectral and spatial features.

[0069] 2.2.2) Establish a ViT block consisting of a self-attention mechanism and a feedforward network to extract global spatial features. This means that a single pixel is first trained to pay attention to the global spatial information of surrounding pixels. The resulting global spatial features are then further optimized and mined.

[0070] 2.2.3) Establish a spatial SKR block consisting of a residual connection ViT block and a selective kernel network SKNets to extract spatial local relationships, helping the network to extract spatial local patterns between pixels and refine the expression of local spatial features;

[0071] 2.2.4) Build two spatially guided classifiers with the same structure. Each classifier consists of two fully connected layers, one for mapping spatial features to a low-dimensional space and the other for optimizing the clarity of spatial classification boundaries. The resulting low-dimensional spatial features are then classified.

[0072] 2.2.5) Connect the adaptive dynamic mask, ViT block, spatial SKR block, and two spatially guided classifier modules in sequence to form the ViT flow network;

[0073] 2.3) The Mamba flow network constructed in 2.1) and the ViT flow network constructed in 2.2) are connected in parallel to form a complete classification model.

[0074] Step 3: Define the classification model loss function La.

[0075] 3.1) According to the source domain data x s and the source domain label y s , get the first classifier C of the Mamba flow network m1 The source domain cross entropy loss L m1 and the second classifier C m2 The source domain cross entropy loss Lm2 :

[0076] ,

[0077] ,

[0078] Among them, G m Indicates Mamba block; L m1 and L m2 Taking source domain data and its labels as input, the Mamba stream network is guided to effectively learn the spectral representation in the source domain, improve its discrimination ability of the source domain spectral representation, and thus enhance its classification performance of the source domain spectral representation;

[0079] 3.2) Using adaptive dynamic mask Y mb Expand the feature diversity of the original hyperspectral data X and obtain hyperspectral data X containing rich spectral spatial features m :

[0080] ,

[0081] in, Represents element-by-element multiplication operations; the rich spectral spatial feature information provides sufficient support for the model's learning under cross-scene conditions. It helps the model capture subtle differences between materials and enhances the model's understanding of boundary details and texture structures;

[0082] 3.3) According to the adaptive dynamic mask source domain data x m s , get the first classifier C of ViT flow network v1 The source domain cross entropy loss L v1 and the second classifier C v2 The source domain cross entropy loss L v2 :

[0083] ,

[0084] ,

[0085] Among them, G v Represents ViT block; L v1 and L v2 By utilizing the label information of the source domain, the ViT flow network is prompted to learn the spatial features and local contextual relationships in the source domain, thereby enhancing its classification performance of the source domain spatial representation;

[0086] 3.4) According to the results of 3.1) and 3.3), the overall loss L of the classification model for the source domain data is obtained sup :

[0087] ,

[0088] According to the overall loss L of the source domain data sup , can optimize the Mamba stream and ViT stream at the same time, effectively improving the classification model's ability to understand and discriminate source domain data;

[0089] 3.5) According to the target domain data x t , we get the loss L of the Mamba flow network on the target domain data m :

[0090] ,

[0091] Among them, y t Represents the label of the target domain data; through L m Guiding the Mamba stream network to learn the representation of the target domain spectral features can adapt to the characteristics of the target domain spectral features and enhance its classification ability for the target domain spectral features. This process is used to improve the spectral feature modeling effect of the Mamba stream in the target domain and further enhance the robustness of the classification model to cross-domain spectral changes.

[0092] 3.6) According to the target domain data x after adaptive dynamic mask m t , we get the loss L of the ViT flow network on the target domain data v :

[0093] ,

[0094] By L v Guiding the ViT flow network to learn the representation of the target domain spatial features can adapt to the characteristics of the target domain spatial features and enhance its classification ability of the target domain spatial features. This process can improve the spatial feature modeling effect of the ViT flow in the target domain and further enhance the robustness of the classification model to cross-domain spatial changes;

[0095] 3.7) Based on the results of 3.4) to 3.6), the overall loss L of the classification model is obtained a :

[0096] .

[0097] Among them, L m is the loss of target domain data by Mamba flow, L v is the loss of target domain data by ViT flow, L sup The loss of source domain data by Mamba stream and ViT stream, is the coefficient for training the target domain, is the coefficient for training the source domain; the classification model acquires the ability to classify the target domain data by learning the joint representation of the spectral space of the training set through the Mamba flow network and the ViT network.

[0098] Step 4: Train the classification model.

[0099] 3.1) Assume that the number of training times is 100, the learning rate is 0.01, and the initial mask rate is 0.1;

[0100] 3.2) Perform backpropagation on the loss function La to calculate the gradient of the adaptive mask rate;

[0101] 3.3) Based on the above gradient, the adaptive mask rate is updated using the stochastic gradient descent method to obtain the current rich light

[0102] Spectral space feature representation; 3.4) Repeat the steps from 3.2) to 3.3), and increase the number of training times by one after each execution;

[0103] 3.5) Determine whether the number of training times reaches 100:

[0104] If so, the training is ended and the trained classification model is obtained;

[0105] Otherwise, return to 3.2) to continue training.

[0106] Step 5: Get the test set classification map.

[0107] The test set, namely the target domain data of the Houston dataset and the target domain data of the Hyrank dataset, is input into the trained classification model to obtain their respective classification maps.

[0108] The more consistent the obtained classification map is with the true value label map, the more accurate the classification model is in classifying the target domain data.

[0109] Example 2: Cross-scene hyperspectral image classification system based on feature optimization and decision collaboration

[0110] Reference Figure 3 This example includes: a data acquisition module 1, a classification model module 2, a training module 3, and a testing module 4. The classification model module 2 includes a Mamba submodule 21, a spectral SKR submodule 22, a spectrally guided classification submodule 23, an adaptive dynamic mask submodule 24, a ViT submodule 25, a spatial SKR submodule 26, and a spatially guided classification submodule 27. The working principle of the entire system is as follows:

[0111] The data acquisition module 1 is used to obtain source domain data and target domain data, and divide them into training sets and test sets;

[0112] The classification model module 2 is used to realize the collaborative modeling of spectral features and spatial features, wherein the Mamba submodule 21 is used to extract spectral global relationships and transmit them to the spectral SKR submodule 22; the spectral SKR submodule 22 is used to further mine spectral detail features based on the spectral global relationships and transmit them to the spectral guided classification submodule 23; the spectral guided classification submodule 23 is used to classify the spectral detail features; the adaptive dynamic mask submodule 24 is used to expand the feature representation of the training set and transmit it to the ViT submodule 25; the ViT submodule 25 is used to extract the spatial global structure of the rich feature representation and transmit it to the spatial SKR submodule 26; the spatial SKR submodule 26 is used to further extract spatial detail features from the spatial global structure and transmit them to the spatial guided classification submodule 27; the spatial guided classification submodule 27 is used to classify the spatial detail features; these submodules constitute a classification model;

[0113] The training module 3 is used to train the classification model using the training set divided by the data acquisition module 1, so that the classification model can obtain better classification capabilities for source domain data and target domain data;

[0114] The testing module 4 is used to input the test set divided by the data acquisition module 1 into the trained classification model and output the cross-scene hyperspectral image classification result.

[0115] It should be noted that the above-mentioned functional modules can be implemented in whole or in part through software, hardware, firmware, or any combination thereof. When implemented using software, they can be implemented in whole or in part in the form of a program instruction product. The program instruction product includes one or a group of program instructions. When the program instructions are loaded and executed on a computer, the process or function described is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The program instructions can be stored in a computer-readable and writable storage medium, or transferred from a computer-readable and writable storage medium to another computer-readable and writable storage medium.

[0116] The direct coupling or communication connection between the modules shown or discussed in this embodiment can be achieved through indirect coupling or communication connection of some interfaces, devices or modules. The various functional modules and submodules in this embodiment can be dynamically located in a processing component, or each module can exist physically separately, or two or more modules can be dynamically located in a processing component. When the above-mentioned dynamic components are implemented in the form of software functional modules and sold or used as independent products, they can also be stored in a computer-readable and writable storage medium. The storage medium can be a memory, a magnetic disk, an optical disk, etc.

[0117] Embodiment 3: An electronic device,

[0118] Reference Figure 4 , this example includes: a processor, a memory, an input / output interface, a communication interface and a bus, which are used to realize communication connections between each other within the device.

[0119] The processor may be implemented as a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0120] The memory can be implemented in the form of a read-only memory (ROM), a random access memory (RAM), a static storage device, a dynamic storage device, etc. The memory can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory and called and executed by the processor.

[0121] The input / output interface is used to connect to an input / output module to enable information input and output. The input / output module can be configured as a component within the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc. Output devices may include a display, speaker, vibrator, indicator light, etc.

[0122] The communication interface is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via wired methods such as USB, network cables, etc., or wireless methods such as mobile networks, WiFi, Bluetooth, etc.

[0123] The bus is used to transmit information between various components of the device, such as the processor, memory, input / output interface, and communication interface.

[0124] It should be noted that although the above device only shows a processor, memory, input / output interface, communication interface, and bus, in a specific implementation, the device may also include other components necessary for normal operation. In addition, those skilled in the art will understand that the above device may only include the components necessary to implement the embodiments of this specification, and does not necessarily include all the components shown in the figures.

[0125] An embodiment of the present invention provides a computer-readable storage medium storing a plurality of instructions, which can be loaded by a processor to execute the steps of any cross-scene hyperspectral image classification method provided by the embodiment of the present invention.

[0126] The computer-readable media of this embodiment include permanent and non-permanent, removable and non-removable media, which can implement information storage by any method or technology. The information can be a computer-readable instruction, a data structure, a module of a program, or other data. The storage media of the computer include but are not limited to phase change memory PRAM, static random access memory SRAMD, dynamic random access memory DRAM, other types of random access memory RAM, read-only memory ROM, electrically erasable programmable read-only memory EEPROM, flash memory or other memory technology, read-only compact disc read-only memory CD-ROM, digital versatile disc DVD or other optical storage, magnetic cassette, magnetic tape magnetic disk storage or other magnetic storage device or any other non-transmission medium, which can be used to store information that can be accessed by a computing device.

[0127] It should be understood by those skilled in the art that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Within the scope of the present invention, the above embodiments or technical features in different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the present invention as described above, which are not provided in detail for the sake of simplicity.

[0128] In addition, to simplify the description and discussion, and so as not to obscure the present invention, well-known power / ground connections to the integrated circuit IC chip and other components may or may not be shown in the provided figures. Furthermore, devices may be shown in block diagram form in order to avoid obscuring the present invention, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the present invention will be implemented, that is, these details should be well within the purview of those skilled in the art.

[0129] The effect of the present invention can be further illustrated by the following simulation experiments:

[0130] 1. Simulation Experiment Conditions

[0131] The present invention is carried out in a Pytorch environment using 2080tiGPU hardware.

[0132] The datasets used in the simulation are the Houston dataset and the Hyrank dataset.

[0133] 2. Simulation experiment content:

[0134] Simulation 1: The target domain data of the Houston dataset are classified using the two cross-scene hyperspectral image classification methods of the present invention and the existing MLUDA and CLDA, respectively. The results are as follows: Figure 5 As shown, where:

[0135] Figure 5 (a) is the real label map of the Houston dataset;

[0136] Figure 5 (b) is the classification diagram of the Houston dataset by the existing CLDA method;

[0137] Figure 5 (c) is the classification diagram of the Houston dataset by the existing MLUDA method;

[0138] Figure 5 (d) is the classification diagram of the Houston data set by the method of the present invention;

[0139] from Figure 5 As can be seen, for the Houston dataset, the classification map of the proposed method has the highest similarity to the ground truth map. In particular, in the complex classification area where the white box contains multiple categories, the proposed method has a higher classification accuracy on the Houston cross-scene dataset.

[0140] Simulation 2: The target domain data of the Hyrank dataset are classified using the two cross-scene hyperspectral image classification methods of the present invention and the existing MLUDA and CLDA. The results are as follows: Figure 6 As shown, where:

[0141] Figure 6 (a) is the real label map of the Hyrank dataset;

[0142] Figure 6 (b) is the classification diagram of the Hyrank dataset using the existing CLDA method;

[0143] Figure 6 (c) is the classification diagram of the existing MLUDA method for the Hyrank dataset;

[0144] Figure 6 (d) is the classification diagram of the Hyrank data set by the method of the present invention;

[0145] from Figure 6 It can be seen that for the Hyrank dataset, the classification map of the method of the present invention has the highest similarity with the ground truth map, especially in the complex classification area where the white box contains multiple categories. The classification accuracy of the method of the present invention on the Hyrank cross-scene dataset is higher.

[0146] The above simulation results show that the method of the present invention has a superior ability to complete cross-scene hyperspectral image classification, that is, a more accurate classification effect.

Claims

1. A cross-scene hyperspectral image classification method based on feature optimization and decision collaboration, characterized in that: include: (1) Obtain source domain data x s , source domain data label y s , target domain data x t ; (2) Construct a classification model consisting of the Mamba flow network and the ViT flow network in parallel: The Mamba flow network includes a Mamba block, a spectral SKR block, and two spectrally guided classifiers for optimizing the classification boundaries in the spectral dimension; The ViT flow network includes an adaptive dynamic mask, a ViT block, a spatial SKR block, and two spatially guided classifiers for enriching the feature space and optimizing the classification boundaries in the spatial dimension; (3) The source domain data x s , source domain data label y s , target domain data x t As a training set, input it into the classification model and iteratively train it; (4) The target domain data x t It is used as a test set to input the trained classification model and output the classification results.

2. The method according to claim 1, characterized in that The structure and functions of each part of the Mamba flow network in (2) are as follows: The Mamba block is composed of multiple serially connected Mamba structures, which are used to extract global spectral relationships and cancel the pooling operation between Mamba structures to avoid information loss. Each Mamba structure includes two branches, the first branch is used to process the global relationship between adjacent spectral channels; the second branch is used to process the global relationship between spectral channels that are farther apart. The spectral SKR block, which is composed of residual connection Mamba blocks and selective kernel networks SKNets, is used to extract spectral local relationships, help the network extract spectral local patterns between pixels, and refine the expression of local spectral features; The two spectral-guided classifiers have the same structure. Each classifier consists of two fully connected layers, wherein the first fully connected layer is used to map spectral features to a low-dimensional space, and the second fully connected layer is used to optimize the clarity of spectral classification boundaries and classify the mapped low-dimensional spectral features. The above-mentioned Mamba block, spectral SKR block and two spectrally guided classifiers are connected in series to form the Mamba flow network.

3. The method according to claim 1, characterized in that The structure and function of each part of the ViT flow network in (2) are as follows: The adaptive dynamic mask is composed of an adaptively changing three-dimensional dynamic mask, which is used to expand the feature representation space, that is, by multiplying with the hyperspectral data, the hyperspectral data is dynamically masked to enhance the diversity of spectral and spatial features; The ViT block, which consists of a self-attention mechanism and a forward propagation network, is used to extract global spatial features. That is, a single pixel is first enabled to pay attention to the global spatial information of other surrounding pixels during training; and then the obtained global spatial features are further optimized and mined. The spatial SKR block, which is composed of residual connections between ViT blocks and selective kernel networks SKNets, is used to extract spatial local relationships, help the network extract spatial local patterns between pixels, and refine the expression of local spatial features; The two space-oriented classifiers have the same structure. Each classifier consists of two fully connected layers, wherein the first fully connected layer is used to map spatial features to a low-dimensional space, and the second fully connected layer is used to optimize the clarity of spatial classification boundaries and classify the mapped low-dimensional spatial features. The above adaptive dynamic mask, ViT block, spatial SKR block and two spatially guided classifiers are connected in sequence to form the ViT flow network.

4. The method according to claim 1, wherein The (3) iterative training of the classification model is implemented as follows: 3a) Set the number of training times to 100, the learning rate to 0.01, and the initial mask rate to 0.1; 3b) Define the loss function L a : ； Among them, L m is the loss of target domain data by Mamba flow, L v is the loss of target domain data by ViT flow, L sup The loss of source domain data by Mamba stream and ViT stream, is the coefficient for training the target domain, is the coefficient for training the source domain; 3c) Perform backpropagation on the loss function to calculate the gradient of the adaptive mask rate; 3d) Based on the above gradient, update the adaptive mask rate using stochastic gradient descent method; 3e) Determine whether the number of training times has reached 100: If so, the training is ended and the trained classification model is obtained; Otherwise, return to 3(c) to continue training.

5. The method according to claim 4, characterized in that The loss function is defined in 3b), and its implementation includes the following: 3b1) According to the source domain data x s and the source domain label y s , get the first classifier C of the Mamba flow network m1 The source domain cross entropy loss L m1 and the second classifier C m2 The source domain cross entropy loss L m2 : , , Among them, G m Represents the Mamba block; 3b2) Using adaptive dynamic mask Y mb Expand the feature diversity of the original hyperspectral data X and obtain hyperspectral data X containing rich spectral spatial features m : , in, Represents element-wise multiplication operation; 3b3) According to the adaptive dynamic mask source domain data x m s , get the first classifier C of ViT flow network v1 The source domain cross entropy loss L v1 and the second classifier C v2 The source domain cross entropy loss L v2 : , , Among them, G v Represents a ViT block; 3b4) Based on the results of 3b1) and 3b3), the overall loss L of the classification model for the source domain data is obtained. sup : , 3b5) According to the target domain data x t , we get the loss L of the Mamba flow network on the target domain data m : , Among them, y t The label representing the target domain data; 3b6) According to the target domain data x after adaptive dynamic mask m t , we get the loss L of the ViT flow network on the target domain data v : , 3b7) Based on the results of 3b4), 3b5) and 3b6), the overall loss L of the classification model is obtained a : 。 6. A cross-scene hyperspectral image classification system based on feature optimization and decision collaboration, characterized by: include: Data acquisition module, used to obtain source domain data and target domain data; Classification model module, used to achieve collaborative modeling of spectral features and spatial features; The training module is used to train the source domain data and the target domain data so that the classification model can obtain the classification ability of the source domain data and the target domain data; The testing module is used to test the target domain data and complete cross-scene hyperspectral image classification.

7. The system according to claim 6, characterized in that The classification model module includes: Mamba submodule, used to extract spectral global relations; Spectral SKR submodule, used to extract spectral local relations; Spectral-guided classification submodule, used to classify spectral dimension features; Adaptive dynamic mask submodule for expanding the feature space; ViT submodule, used to extract spatial global structure; The spatial SKR submodule is used to extract spatial local structures; The spatially oriented classification submodule is used to classify spatial dimension features.

8. An electronic device, characterized in that: include: processor and memory; The memory stores program instructions; The processor is configured to run the program instructions to perform the image classification method according to any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that The storage medium stores program instructions, and the program instructions are executed to perform the image classification method according to any one of claims 1 to 5.

Images