A multi-network fusion hyperspectral image classification method with noise resistance

Abstract

The present invention is a multi-network fusion hyperspectral image classification method with noise resistance. By introducing the CapsuleGAN network and performing comparative learning between bands, the data set size is expanded, helping to generate more realistic sample data, and improving the feature expression ability and the generalization ability and robustness of the model. A convolutional neural network is used to extract local features, and hierarchical sampling and importance sampling are performed on the extracted local features. Graph structures are generated separately and feature fusion is performed to generate a fused graph structure. This is used as the input of the GAT and trained with the entire fusion model. This can significantly improve the performance of hyperspectral image classification. The spatial and spectral features extracted by the convolutional neural network ensure the full utilization of local feature information, and enable a more comprehensive expression of data features. It pays more attention to the use of multi-dimensional features and makes the model more stable in the face of interference factors such as atmosphere and light, thereby improving classification accuracy and robustness.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral image classification, and in particular to a multi-network fusion hyperspectral image classification method with noise resistance. Background Art

[0002] Hyperspectral image classification is a technology that uses hyperspectral imaging technology to analyze and classify images. The hyperspectral imaging system can obtain the reflectance spectral data of objects in multiple continuous bands, usually covering multiple bands from visible light to near-infrared. This technology is widely used in many fields such as environmental monitoring, agriculture, mining, urban planning, military reconnaissance, etc.

[0003] Current deep learning network training typically requires a large number of labeled samples. However, in the field of remote sensing, high-quality labeled data is often difficult and expensive to obtain. Furthermore, hyperspectral images have numerous bands and contain a vast amount of information, leading to issues such as "same object, different spectrum," "different objects, same spectrum," and information redundancy. In particular, in actual use, external influences such as noise and illumination variations render traditionally enhanced sample data less effective. Furthermore, existing simple convolutional models have poor resistance to illumination and noise, while complex models have poor processing time efficiency. They typically use all pixels as nodes in the graph attention network, which invisibly increases the computational burden. Furthermore, existing complex models are less resistant to interference such as atmospheric and illumination interference. Therefore, it is necessary to propose a multi-network fusion hyperspectral image classification method with excellent noise resistance. Summary of the Invention

[0004] The purpose of this invention is to provide a multi-network fusion hyperspectral image classification method with noise-resistant performance. This classification method uses a CapsuleGAN network to generate new sample data, introduces inter-band contrast learning, and uses the band characteristics of hyperspectral images for self-supervised pre-training to enhance feature representation. After extracting local features using a convolutional neural network with spatial group attention and spectral group attention, these extracted local features are processed using two paths: hierarchical sampling and importance sampling. Finally, a graph attention network is used to output the prediction results. This method significantly improves its adaptability to interference such as atmospheric, lighting, and noise, and has excellent noise resistance.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] In a first aspect, the present invention provides a multi-network fusion hyperspectral image classification method with noise resistance, the classification method comprising the following steps:

[0007] S1. Obtain the original hyperspectral image under the influence of different interference factors, use the CapsuleGAN network to generate virtual hyperspectral pixels for the original hyperspectral image; merge the pixels in the original hyperspectral image with the virtual hyperspectral pixels, and each pixel has its own band information and category label;

[0008] A pixel point, corresponding category label and band information constitute a sample; all samples constitute a hyperspectral image dataset;

[0009] S2. Construct a band perception module, which is used to extract band features and remove insensitive bands in the sample to obtain a hyperspectral sensitive data set;

[0010] S3. Build fusion model:

[0011] The fusion model includes a convolutional neural network with spatial group attention and spectral group attention, a graph attention network GAT, a hierarchical sampling operation, and an importance sampling operation;

[0012] The convolutional neural network containing spatial group attention and spectral group attention is used to extract local features of the input to obtain local features;

[0013] The local features are processed by a hierarchical sampling operation and an importance sampling operation respectively to obtain a hierarchical graph structure and an importance graph structure, and then the hierarchical graph structure and the importance graph structure are subjected to feature fusion to obtain a fused graph structure;

[0014] The fused graph structure is then processed by the graph attention network GAT to obtain the output of the fusion model;

[0015] S4. The fusion model is trained using a hyperspectral sensitive dataset, and the trained fusion model is used to perform classification prediction on the hyperspectral image to be classified.

[0016] Furthermore, the construction process of the band perception module is:

[0017] In the hyperspectral image dataset, 15% of the samples are selected for each category, and the band information of each band of the selected samples is taken as a separate band sample to form a band dataset, in which the band sample of a certain pixel point currently required for comparison in a certain band is defined as the anchor sample, the band sample that belongs to the same category but not in the same band as the anchor sample is the positive sample, and the band sample that does not belong to the same category but in the same band as the anchor sample is the negative sample;

[0018] Train the CNN network with the band data set to obtain the trained CNN network, and set the inter-band contrast loss function during training. for:

[0019]

[0020] Among them, f i Band features of anchor samples extracted by CNN network; represents the band features of the positive sample extracted by the CNN network; f k is the band feature extracted by the CNN network for the kth band sample in a band sample set containing one positive sample and N negative samples; N is the total number of negative samples; τ is the temperature parameter;

[0021] Sensitivity calculation unit: The band information of each band of all samples in the hyperspectral image dataset is taken as input, and the trained CNN network is used to extract all band features. The variance of the band features of each band is calculated respectively. The variance of the band features of all bands is sorted from large to small, and the variance of the band features is used as the sensitivity index. The band information of the top 70% of the sensitivity in the hyperspectral image dataset is retained, and the band information of the bottom 30% is eliminated.

[0022] The trained CNN network and the sensitivity calculation unit are combined into a band perception module.

[0023] Furthermore, the convolutional neural network containing spatial group attention and spectral group attention includes multiple convolution blocks consisting of multiple convolution layers, nonlinear activation functions, pooling layers and fully connected layers. After each convolution block, parallel spatial group attention and spectral group attention are connected. The outputs of the spatial group attention and spectral group attention are spliced ​​and connected to the input of the next convolution block. The spliced ​​output result of the spatial group attention and spectral group attention connected by the last convolution block is used as the local feature.

[0024] The cross-group fusion process of spectral group attention and spatial group attention is as follows:

[0025]

[0026] Among them, Attention spectral Attention is the spectral group attention, which is used to calculate the relationship between bands; Spatial is the spatial group attention, used to calculate the local area relationship; H fused is the result of cross-group attention fusion between the two; W O is the output projection matrix.

[0027] Furthermore, the process of feature fusion of the hierarchical graph structure and the importance graph structure is:

[0028] For each node in the hierarchical graph structure and the importance graph structure, calculate the attention weight between the nodes in the two graph structures :

[0029]

[0030] in, Indicates that the nodes in the hierarchical graph structure Features Nodes in the importance graph Features Perform feature splicing, l represents the level of the node in the current hierarchical graph structure, i represents the feature category of the node in the current importance graph structure, j and k are used to represent the current node index of the hierarchical graph structure and the importance graph structure respectively, N j Represents the total number of nodes in the current level l of the hierarchical graph structure; N k Represents the total number of nodes in the importance graph structure within the range of the current feature category i; MLP is a multi-layer perceptron function with three hidden layers and softmax as the activation function; is the feature of the j'th node at level l in the hierarchical graph structure, is the feature of the k'th node of the i-th feature category in the importance graph structure;

[0031] Features of the graph structure after fusion The calculation formula is:

[0032]

[0033] in, represents the weighted sum of node features in the importance graph structure according to the attention weight, and β is a hyperparameter.

[0034] Furthermore, the CapsuleGAN network includes a generative adversarial network and a capsule network, and uses the dynamic routing mechanism of capsules to capture the hierarchical structure of objects in hyperspectral images.

[0035] In a second aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which can implement the steps of the classification method when executed by a processor.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] The method of the present invention introduces a CapsuleGAN network and a band perception module when constructing a hyperspectral sensitive dataset to expand and filter the dataset. This not only improves the model's ability to recognize rare data types, but also improves the model's generalization and robustness. Simultaneously, a multi-network fusion model is constructed using convolutional neural networks containing spatial and spectral group attention, hierarchical sampling operations, importance sampling operations, and a graph attention network (GAT). Compared to the approach of using all pixels as nodes in the graph attention network, this reduces the computational burden, improves training efficiency, makes more effective use of local and global feature information, and improves the ability to resist interference factors such as atmosphere and illumination, ensuring effective classification under the influence of interference factors.

[0038] This invention introduces inter-band contrastive learning, which can explore differences and associations between bands, further identify and eliminate redundant bands, retain key band information, and achieve dimensionality reduction while preserving the physical characteristics of the image. For example, when processing hyperspectral images with hundreds of continuous spectral bands, the most representative bands can be screened, reducing storage and computational pressure. This approach can also enhance the model's generalization capabilities. For example, when distinguishing different land feature categories, contrastive learning allows the model to focus on the band feature combinations that play a key role in classification, helping subsequent models improve classification accuracy and reducing the number of parameters required during model training.

[0039] In the present invention, local features are subjected to hierarchical sampling and importance sampling, and a graph structure is generated. The two are fused to generate a fused graph structure. Hierarchical sampling can sample hyperspectral image features at different scales, and can capture multi-scale structural information of the image from macro to micro. Importance sampling focuses on the feature parts with key information in the hyperspectral image. The graph structure generated by fusing the two can integrate multi-scale structural information with key feature information, allowing the model to understand the image more comprehensively. For example, when analyzing land cover types, hierarchical sampling can obtain features in different regional ranges, and importance sampling highlights key land features such as vegetation and water bodies. After fusion, the classification accuracy is improved. In addition, the graph structures generated by different sampling methods can reflect different aspects of hyperspectral image features. After fusion, the model can be more stable when facing interference such as noise and illumination changes.

[0040] This paper employs a convolutional neural network that incorporates both spatial and spectral attention groups. The spectral attention group focuses on the spectral dimension of the data, capturing the importance of features across different wavelengths; the spatial attention group focuses on the spatial structure of the data, such as its position and shape. Fusion of these two approaches simultaneously considers both spectral and spatial dimensions, resulting in a more comprehensive representation of data features. Cross-group attention fusion allows for the extraction of more effective spatial-spectral features, leveraging the complementary strengths of information from different dimensions to enhance the overall effectiveness of feature extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a structural diagram of the multi-network fusion hyperspectral image classification method with noise resistance of the present invention. DETAILED DESCRIPTION

[0042] The following will clearly and completely describe the technical solutions of the present invention in conjunction with the embodiments and drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0043] The present invention has a multi-network fusion hyperspectral image classification method with noise resistance (see Figure 1 ), including the following steps:

[0044] S1. Obtain the original hyperspectral image under the influence of different interference factors (such as atmospheric and light interference), use the CapsuleGAN network to generate virtual hyperspectral pixels for the original hyperspectral image; merge the pixels in the original hyperspectral image with the virtual hyperspectral pixels, and each pixel has its own band information and category label;

[0045] A pixel point and the corresponding category and band information constitute a sample; all samples constitute a hyperspectral image dataset;

[0046] S2. Construct a band perception module, which is used to extract band features and eliminate insensitive bands in the sample to obtain a hyperspectral sensitive data set;

[0047] S3. Build fusion model:

[0048] The fusion model includes a convolutional neural network with spatial group attention and spectral group attention, a graph attention network GAT, a hierarchical sampling operation and an importance sampling operation;

[0049] The convolutional neural network containing spatial group attention and spectral group attention is used to extract local features of the input to obtain local features;

[0050] The local features are processed by a hierarchical sampling operation and an importance sampling operation respectively to obtain a hierarchical graph structure and an importance graph structure, and then the hierarchical graph structure and the importance graph structure are subjected to feature fusion to obtain a fused graph structure;

[0051] The fused graph structure is then processed by the graph attention network GAT to obtain the output of the fusion model;

[0052] S4. The fusion model is trained using a hyperspectral sensitive dataset, and the trained fusion model is used to perform classification prediction on the hyperspectral image to be classified.

[0053] After the hyperspectral sensitive dataset is constructed, in the training phase, the input of the fusion model is the samples in the hyperspectral sensitive dataset. After the training is completed, the input of the fusion model is the hyperspectral image to be classified, and no sensitivity analysis is required for the hyperspectral image to be classified.

[0054] Specifically, the CapsuleGAN network described in step S1 combines a generative adversarial network (GAN) and a capsule network. It includes a generator, a discriminator, and a capsule layer. It leverages the dynamic routing mechanism of capsules to capture the hierarchical structure of objects (such as shape and texture) in hyperspectral images. Capsule networks represent features using vector neurons (rather than scalars), preserving feature pose information (such as orientation and scale), thereby generating more realistic hyperspectral samples.

[0055] Its dynamic routing mechanism is shown in the following formula:

[0056]

[0057]

[0058]

[0059] where c mn is the routing weight from the lower-level capsule m to the higher-level capsule n. The larger the weight, the stronger the correlation between the two. is the prediction vector of the lower-level capsule m for the higher-level capsule n. q represents the index of all higher-level capsules that the lower-level capsule m can connect to. is the original coupling coefficient between the lower-level capsule m and the higher-level capsule n. is the weighted input vector of the high-level capsule n, which is obtained by weighted summation of the prediction vectors of all low-level capsules. is the output vector of the high-level capsule n, and its modulus is represents the capsule activation probability.

[0060] The construction process of the band perception module is:

[0061] Band Perception Module: Through band-perception comparative learning, the similarity between positive sample pairs is shortened and the similarity between negative sample pairs is extended, enabling the model to learn band-invariant features and improving the robustness of the model classification. In this embodiment, the band perception module uses a CNN network to extract band features;

[0062] In the hyperspectral image dataset, 15% of the samples are selected for each category, and the band information of each band of the selected samples is taken as a separate band sample to form a band dataset, in which the band sample of a certain pixel point currently required for comparison in a certain band is defined as the anchor sample, the band sample that belongs to the same category but not in the same band as the anchor sample is the positive sample, and the band sample that does not belong to the same category but in the same band as the anchor sample is the negative sample;

[0063] Train the CNN network with the band data set to obtain the trained CNN network, and set the inter-band contrast loss function during training. for:

[0064]

[0065] Among them, f i Band features of anchor samples extracted by CNN network; represents the band features of the positive sample extracted by the CNN network; f k is the band feature extracted by the CNN network for the kth band sample in a band sample set containing one positive sample and N negative samples; N is the total number of negative samples relative to the current anchor sample; τ is the temperature parameter;

[0066] Sensitivity calculation unit: The band information of each band of all samples in the hyperspectral image dataset is used as input, and the trained CNN network is used to extract all band features. The variance of the band features of each band is calculated respectively. The larger the variance, the more sensitive the band feature change is. The variance of the band features of all bands is sorted from large to small, and the variance of the band features is used as the sensitivity index. The band information of the top 70% of the sensitivity in the hyperspectral image dataset is retained, and the band information of the bottom 30% is eliminated.

[0067] The trained CNN network and the sensitivity calculation unit are combined into a band perception module. In actual use, there is no need to retrain the CNN network. The band perception module can be directly used to process the hyperspectral image dataset to obtain a hyperspectral sensitivity dataset.

[0068] The present invention sets anchor samples, positive samples and negative samples, and the band samples in the band data set used in CNN network training are used as anchor samples in turn.

[0069] In the hyperspectral image of the present invention, each pixel corresponds to many bands. A hyperspectral image has multiple detection objects, and each detection object is composed of multiple pixels. Inter-band contrast learning is introduced, and the band characteristics of the hyperspectral image are used for self-supervised pre-training to enhance feature representation.

[0070] The process of building the fusion model is:

[0071] Step S3.1 The convolutional neural network containing spatial group attention and spectral group attention can effectively capture the important information of each pixel and its neighboring areas through a series of convolutional layers and pooling layers, so that the model pays more attention to the image areas that are most important for classification decisions. The attention is divided into spectral group and spatial group, and the two are fused across groups, thereby retaining the local features of rich spatial and spectral features.

[0072] The convolutional neural network containing spatial group attention and spectral group attention includes multiple convolution blocks composed of multiple convolution layers, nonlinear activation functions, pooling layers and fully connected layers. After each convolution block, the parallel spatial group attention and spectral group attention are connected. The outputs of the spatial group attention and spectral group attention are spliced ​​and connected to the input of the next convolution block. The spliced ​​output result of the spatial group attention and spectral group attention connected by the last convolution block is used as the local feature;

[0073] The convolutional neural network with spatial group attention and spectral group attention can be pre-trained using a publicly known dataset. During pre-training, the cross-entropy loss function is used to set the learning rate, batch size, and network parameters, and local features are extracted using the pre-trained convolutional neural network with spatial group attention and spectral group attention.

[0074] The process of cross-group fusion of spectral group attention and spatial group attention is shown in the following formula:

[0075]

[0076]

[0077] Among them, Attention spectral Attention is the spectral group attention, which is used to calculate the relationship between bands. Spatial is the spatial group attention, used to calculate the local area relationship, H fused is the result of cross-group attention fusion between the two. O is the output projection matrix, d k is the attention dimension. Q is the query vector, i.e., the feature vector used to guide the attention calculation. spec is the spectral query vector, which is used to find the relevant spectral features in the attention calculation at the spectral level. spat is the spatial query vector, which is used for attention calculation at the spatial level to find the relevant spatial features. K is the key vector, which is the vector matched with Q. By calculating the similarity between Q and K, we can determine which information in V is more important to Q. spec is the spectral key vector, corresponding to the spectral dimension, used in the spectral group attention calculation with Qspec Calculate the similarity. K spat is the spatial key vector, which is used together with Q in the spatial group attention calculation spat Calculate similarity. V is a value vector containing the information actually used to calculate the output. spec is the spectral value vector. In the spectral group attention calculation, it is the spectral feature vector that actually participates in the output calculation. Its information weight is represented by Q spec and K spec The similarity is determined. spat is a spatial value vector. In the spatial group attention calculation, it is the spatial feature vector that actually participates in the output calculation. Its information weight is determined by Q spat and K spat The similarity is determined.

[0078] Step S3.2 performs importance sampling and hierarchical sampling on local features respectively. The graph structures generated by the two sampling strategies (hierarchical graph structure and importance graph structure) are used as heterogeneous inputs and fused through the shared attention mechanism to generate new nodes and construct the fused graph structure.

[0079] The process of fusion of shared attention mechanism is: for each node of the hierarchical graph structure and the importance graph structure, the attention weight between the nodes in the two graph structures is calculated. , use the multi-layer perceptron MLP to calculate the attention score:

[0080]

[0081] in, Indicates that the nodes in the hierarchical graph structure Features Nodes in the importance graph Features Perform feature splicing, l represents the level of the node in the current hierarchical graph structure, i represents the feature category of the node in the current importance graph structure, j and k are used to represent the current node index of the hierarchical graph structure and the importance graph structure respectively, N j Represents the total number of nodes in the current level l of the hierarchical graph structure; N k Represents the total number of nodes in the importance graph structure within the range of the current feature category i; MLP is a multi-layer perceptron function with three hidden layers and softmax as the activation function; is the feature of the j'th node at level l in the hierarchical graph structure, is the feature of the k'th node of the i-th feature category in the importance graph structure; is the attention weight between nodes;

[0082] Features of the graph structure after fusion The calculation formula is:

[0083]

[0084] in, Represents the node features in the importance graph structure according to the attention weight β is a hyperparameter used to balance the contribution of the hierarchical graph structure’s own features.

[0085] Step S3.3 uses the graph attention network GAT to dynamically adjust the attention weight of each node in the fused graph structure to its neighboring nodes, enhancing the model's ability to capture important node information; through the graph convolution operation GCN, the information of adjacent nodes is aggregated to the current node, enabling the model to utilize global context information and capture long-distance pixel dependencies, making up for the shortcomings of traditional CNN in processing global image information.

[0086] Input the fused graph structure into the graph convolution operation GCN, determine the node feature matrix and edge connection information, prepare the initial feature vector for each node, and form the feature matrix H;

[0087]

[0088] Where N is the number of nodes and F is the feature dimension.

[0089] Set each pair of nodes as i and its neighbor node j, and use linear transformation and activation function LeakyReLU to calculate the similarity of features, which is the attention score e ij , expressed as follows:

[0090]

[0091] Among them, W is the learnable weight matrix, || represents vector concatenation, and h i and h j is the feature vector of node i and node j, and LeakyReLU is the activation function.

[0092] Introducing the Multi-Head Attention mechanism, the weight of the adjacent node is α ij The specific formula is as follows:

[0093]

[0094] Where N(i) is the neighbor set of node i.

[0095] For each node, the updated feature representation is obtained by the weighted features of its neighbors:

[0096]

[0097] Where h′ i is the updated feature representation of node i, σ is the activation function, and W is the learnable weight matrix.

[0098] S4. The fusion model is trained using the hyperspectral sensitive data set, and the trained fusion model is used to perform classification prediction on the hyperspectral image to be classified. Classification prediction is performed on the entire image data and represented in the form of a hyperspectral image.

[0099] In summary, the present invention expands the dataset size and solves the problem of insufficient samples by introducing the CapsuleGAN network and performing inter-band comparative learning. The capsule network can retain the posture information of the retained features, help generate more realistic sample data, and improve the feature expression ability, generalization ability and robustness of the model. The convolutional neural network is used to extract local features, and the extracted local features are hierarchically sampled and importance sampled. The graph structure is generated and feature fusion is performed to generate a fused graph structure. As the input of the GAT, the entire fusion model is trained, which can significantly improve the performance of hyperspectral image classification. The spatial and spectral features extracted by the convolutional neural network ensure the full utilization of local feature information and enable a more comprehensive expression of data features, focusing more on the utilization of multi-dimensional features. The hierarchical sampling of local feature data and importance sampling, as well as the fusion of graph structure features, enable the model to have a more comprehensive understanding of the image, integrate multi-scale structural information (hierarchical sampling) with key features (importance sampling), and make the model more stable in the face of interference factors such as atmosphere and lighting, thereby improving classification accuracy and robustness.

[0100] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

[0101] Any matters not described in the present invention are applicable to the prior art.

Claims

1. A multi-network fusion hyperspectral image classification method with noise resistance, characterized in that: The classification method comprises the following steps: S1. Obtain the original hyperspectral image under the influence of different interference factors, use the CapsuleGAN network to generate virtual hyperspectral pixels for the original hyperspectral image; merge the pixels in the original hyperspectral image with the virtual hyperspectral pixels, each pixel having its own band information and category label; the different interference factors include atmosphere, light, and noise. A pixel point, corresponding category label and band information constitute a sample; all samples constitute a hyperspectral image dataset; S2. Construct a band perception module, which is used to extract band features and remove insensitive bands in the sample to obtain a hyperspectral sensitive data set; The construction process of the band perception module is: In the hyperspectral image dataset, 15% of the samples are selected for each category, and the band information of each band of the selected samples is taken as a separate band sample to form a band dataset, in which the band sample of a certain pixel point currently required for comparison in a certain band is defined as the anchor sample, the band sample that belongs to the same category but not in the same band as the anchor sample is the positive sample, and the band sample that does not belong to the same category but in the same band as the anchor sample is the negative sample; Train the CNN network with the band data set to obtain the trained CNN network. During training, set the inter-band contrast loss function L BandCLR for: Among them, f i Band features of anchor samples extracted by CNN network; represents the band features of the positive sample extracted by the CNN network; f k is the band feature extracted by the CNN network for the kth band sample in a band sample set containing one positive sample and N negative samples; N is the total number of negative samples; τ is the temperature parameter; Sensitivity calculation unit: takes the band information of each band of all samples in the hyperspectral image dataset as input, uses the trained CNN network to extract all band features, and calculates the variance of the band features of each band respectively; sorts the variances of the band features of all bands from large to small, uses the variance of the band features as the sensitivity index, retains the band information of the top 70% in the hyperspectral image dataset in terms of sensitivity, and eliminates the band information of the bottom 30%; The trained CNN network and the sensitivity calculation unit are combined into a band perception module; S3. Build fusion model: The fusion model includes a convolutional neural network with spatial group attention and spectral group attention, a graph attention network GAT, a hierarchical sampling operation, and an importance sampling operation; The convolutional neural network containing spatial group attention and spectral group attention is used to extract local features of the input to obtain local features; The convolutional neural network containing spatial group attention and spectral group attention includes multiple convolution blocks composed of multiple convolution layers, nonlinear activation functions, pooling layers and fully connected layers. After each convolution block, the parallel spatial group attention and spectral group attention are connected. The outputs of the spatial group attention and spectral group attention are spliced ​​and connected to the input of the next convolution block. The spliced ​​output result of the spatial group attention and spectral group attention connected by the last convolution block is used as the local feature; The cross-group fusion process of spectral group attention and spatial group attention is as follows: H fused =Concat(Attention spectral ,Attention spatial )W O Among them, Attention spectral Attention is the spectral group attention, which is used to calculate the relationship between bands; Spatial is the spatial group attention, used to calculate the local area relationship; H fused is the result of cross-group attention fusion between the two; W O is the output projection matrix; The local features are processed by a hierarchical sampling operation and an importance sampling operation respectively to obtain a hierarchical graph structure and an importance graph structure, and then the hierarchical graph structure and the importance graph structure are subjected to feature fusion to obtain a fused graph structure; The process of feature fusion of the hierarchical graph structure and the importance graph structure is: For each node in the hierarchical graph structure and the importance graph structure, calculate the attention weight between the nodes in the two graph structures in, Indicates that the nodes in the hierarchical graph structure Features Nodes in the importance graph Features Perform feature splicing, l represents the level of the node in the current hierarchical graph structure, i represents the feature category of the node in the current importance graph structure, j and k are used to represent the current node index of the hierarchical graph structure and the importance graph structure respectively, N j Represents the total number of nodes in the current level l of the hierarchical graph structure; N k Represents the total number of nodes in the importance graph structure within the range of the current feature category i; MLP is a multi-layer perceptron function with three hidden layers and softmax as the activation function; is the feature of the j'th node at level l in the hierarchical graph structure, is the feature of the k'th node of the i-th feature category in the importance graph structure; Features of the graph structure after fusion The calculation formula is: in, represents the weighted summation of node features in the importance graph structure according to the attention weight, and β is a hyperparameter; The fused graph structure is then processed by the graph attention network GAT to obtain the output of the fusion model; S4. The fusion model is trained using a hyperspectral sensitive dataset, and the trained fusion model is used to perform classification prediction on the hyperspectral image to be classified.

2. The multi-network fusion hyperspectral image classification method with noise resistance according to claim 1 is characterized in that: The CapsuleGAN network includes a generative adversarial network and a capsule network.

3. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the steps of the classification method according to claim 1 or 2.

Images