Multi-network fusion hyperspectral image classification method with anti-noise performance

Abstract

According to the multi-network fusion hyperspectral image classification method with the anti-noise performance, by introducing the Capsule GAN and conducting comparative learning between wavebands, the scale of a data set is expanded, generation of more vivid sample data is assisted, and the feature expression ability and the generalization ability and robustness of a model are improved. Local features are extracted by using a convolutional neural network, hierarchical sampling and importance sampling are performed on the extracted local features, graph structures are generated respectively, feature fusion is performed to generate fused graph structures, the fused graph structures are used as input of GAT, and training is performed by using a whole fusion model, so that the performance of hyperspectral image classification can be remarkably improved. Spatial and spectral features extracted by the convolutional neural network ensure full utilization of local feature information, data features are expressed more comprehensively, more attention is paid to utilization of multi-dimensional features, and the model is more stable when facing interference factors such as atmosphere and illumination, so that classification precision and robustness are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral image classification, and in particular to a multi-network fusion hyperspectral image classification method with noise resistance. Background Art

[0002] Hyperspectral image classification is a technology that uses hyperspectral imaging technology to analyze and classify images. The hyperspectral imaging system can obtain the reflectance spectral data of objects in multiple continuous bands, usually covering multiple bands from visible light to near-infrared. This technology is widely used in environmental monitoring, agriculture, mining, urban planning, military reconnaissance and other fields.

[0003] The current training of deep learning networks usually requires a large number of labeled samples, but in the field of remote sensing, there is a problem that high-quality labeled data is often difficult to obtain and costly. In addition, hyperspectral images have many bands and a huge amount of information. There are problems such as "same object, different spectrum", "different objects, same spectrum", and information redundancy. Especially in actual use, there are external influences such as noise and illumination changes, which make the sample data enhanced in the traditional way less effective. In addition, the existing simple convolutional model has poor noise resistance to illumination and noise, and the processing time efficiency of complex models is poor. Generally, all pixels are used as nodes of the graph attention network, which invisibly increases the computational burden. In addition, the existing complex models have weak ability to resist interference such as atmosphere and illumination. Therefore, it is necessary to propose a multi-network fusion hyperspectral image classification method with excellent noise resistance. Summary of the invention

[0004] The purpose of the present invention is to provide a multi-network fusion hyperspectral image classification method with anti-noise performance. The classification method uses the CapsuleGAN network to generate new sample data, introduces contrast learning between bands, uses the band characteristics of hyperspectral images for self-supervised pre-training, enhances feature representation, extracts local features using a convolutional neural network containing spatial group attention and spectral group attention, and then processes the extracted local features using two paths: hierarchical sampling and importance sampling. Finally, the graph attention network is used to output the prediction results, which significantly improves its adaptability to interference such as atmosphere, light, and noise, and has excellent anti-noise performance.

[0005] To achieve the above object, the present invention provides the following technical solutions: In a first aspect, the present invention provides a multi-network fusion hyperspectral image classification method with noise resistance, the classification method comprising the following steps: S1. Obtain the original hyperspectral image under the influence of different interference factors, and use the CapsuleGAN network to generate virtual hyperspectral pixels for the original hyperspectral image; merge the pixels in the original hyperspectral image with the virtual hyperspectral pixels, and each pixel has its own band information and category label; A pixel point and the corresponding category label and band information constitute a sample; all samples constitute a hyperspectral image dataset; S2. Construct a band perception module, which is used to extract band features and remove insensitive bands in samples to obtain a hyperspectral sensitive data set; S3. Build fusion model: The fusion model includes a convolutional neural network with spatial group attention and spectral group attention, a graph attention network GAT, a hierarchical sampling operation, and an importance sampling operation; The convolutional neural network containing the spatial group attention and the spectral group attention is used to extract local features of the input to obtain local features; The local features are processed by a hierarchical sampling operation and an importance sampling operation respectively to obtain a hierarchical graph structure and an importance graph structure, and then feature fusion is performed on the hierarchical graph structure and the importance graph structure to obtain a fused graph structure; The fused graph structure is then processed by the graph attention network GAT to obtain the output of the fusion model; S4. The fusion model is trained using a hyperspectral sensitive data set, and the trained fusion model is used to perform classification prediction on the hyperspectral image to be classified.

[0006] Furthermore, the construction process of the band perception module is: In the hyperspectral image dataset, 15% of the samples are selected for each category, and the band information of each band of the selected samples is taken as a separate band sample to form a band dataset, in which the band sample of a certain pixel point currently required for comparison in a certain band is defined as the anchor sample, the band sample that belongs to the same category as the anchor sample but is not in the same band is the positive sample, and the band sample that does not belong to the same category as the anchor sample but is in the same band is the negative sample; Train the CNN network with the band data set to obtain the trained CNN network, and set the inter-band contrast loss function during training. for:

[0007] Among them, f i Band features of anchor samples extracted by CNN network; represents the band features of the positive sample extracted by the CNN network; f kis the band feature extracted by the CNN network from the kth band sample in a band sample set containing one positive sample and N negative samples; N is the total number of negative samples; τ is the temperature parameter; Sensitivity calculation unit: take the band information of each band of all samples in the hyperspectral image dataset as input, use the trained CNN network to extract all band features, and calculate the variance of the band features of each band respectively; sort the variances of the band features of all bands from large to small, use the variance of the band features as the sensitivity index, retain the top 70% of the band information in the hyperspectral image dataset in terms of sensitivity, and remove the bottom 30% of the band information; The trained CNN network and the sensitivity calculation unit are combined into a band perception module.

[0008] Furthermore, the convolutional neural network containing the spatial group attention and the spectral group attention comprises a plurality of convolutional blocks consisting of a plurality of convolutional layers, a nonlinear activation function, a pooling layer and a fully connected layer, and the parallel spatial group attention and the spectral group attention are connected after each convolutional block, and the outputs of the spatial group attention and the spectral group attention are spliced ​​and connected to the input of the next convolutional block, and the spliced ​​output result of the spatial group attention and the spectral group attention connected to the last convolutional block is used as the local feature; The cross-group fusion process of spectral group attention and spatial group attention is as follows:

[0009] Among them, Attention spectral Attention is the spectral group attention, which is used to calculate the relationship between bands; Spatial is the spatial group attention, which is used to calculate the local area relationship; H fused is the result of cross-group attention fusion between the two; W O is the output projection matrix.

[0010] Furthermore, the process of fusing the features of the hierarchical graph structure and the importance graph structure is: For each node in the hierarchical graph and the importance graph, calculate the attention weight between the nodes in the two graphs :

[0011] in, Indicates that the nodes in the hierarchical graph structure Features Nodes in the importance graph Features Perform feature concatenation, l represents the level of the node in the current hierarchical graph structure, i represents the feature category of the node in the current importance graph structure, j and k are used to represent the current node index of the hierarchical graph structure and the importance graph structure respectively, and N j Represents the total number of nodes in the current level l of the hierarchical graph structure; N k Represents the total number of nodes in the importance graph structure within the range of the current feature category i; MLP is a multi-layer perceptron function with three hidden layers and softmax as the activation function; is the feature of the j'th node at level l in the hierarchical graph structure, is the feature of the k'th node of the i-th feature category in the importance graph structure; Characteristics of the graph structure after fusion The calculation formula is:

[0012] in, represents the weighted sum of node features in the importance graph structure according to the attention weight, and β is a hyperparameter.

[0013] Furthermore, the CapsuleGAN network includes a generative adversarial network and a capsule network, and uses the dynamic routing mechanism of capsules to capture the hierarchical structure of objects in hyperspectral images.

[0014] In a second aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which can implement the steps of the classification method when executed by a processor.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The method of the present invention introduces the CapsuleGAN network and the band perception module when constructing a hyperspectral sensitive data set to expand the screening data set, which can not only improve the model's recognition ability for rare data types, but also improve the model's generalization ability and robustness. At the same time, a convolutional neural network containing spatial group attention and spectral group attention, hierarchical sampling operations, importance sampling operations, and a graph attention network GAT are used to fuse multiple networks to form a fusion model. Compared with the method of using all pixels as graph attention network nodes, the computational burden is reduced, the training efficiency is improved, the local features and global feature information are more fully utilized, and the ability to resist the influence of interference factors such as atmosphere and light is improved, which can ensure effective classification under the influence of interference factors.

[0016] The present invention introduces contrastive learning between bands, which can mine the differences and associations between bands, further identify and eliminate redundant bands, retain key band information, and achieve dimensionality reduction while retaining the physical characteristics of the image. For example, when processing hyperspectral images with hundreds of continuous spectral bands, the most representative bands can be screened out to reduce storage and computing pressure. This approach can also enhance the generalization ability of the model. For example, when distinguishing different types of land objects, contrastive learning allows the model to focus on the band feature combination that plays a key role in classification, which helps the subsequent model improve the classification accuracy and reduce the number of parameters during model training.

[0017] In the present invention, local features are subjected to hierarchical sampling and importance sampling, and a graph structure is generated. The two are fused to generate a fused graph structure. Hierarchical sampling can sample hyperspectral image features at different scales, and can capture multi-scale structural information of the image from macro to micro. Importance sampling focuses on the characteristic parts of the hyperspectral image with key information. The graph structure generated by fusing the two can integrate multi-scale structural information with key feature information, allowing the model to understand the image more comprehensively. For example, when analyzing land cover types, hierarchical sampling can obtain features of different regional ranges, and importance sampling highlights key land features such as vegetation and water bodies, and the classification accuracy is improved after fusion. In addition, the graph structures generated by different sampling methods can reflect different aspects of hyperspectral image features. After fusion, the model can be more stable when facing interference such as noise and light changes.

[0018] The present invention adopts a convolutional neural network containing spatial group attention and spectral group attention. The spectral group attention focuses on the spectral dimension information of the data and can capture the importance of features in different bands; the spatial group attention focuses on the spatial structure information of the data, such as spatial position, shape, etc. After the two are fused, the information of spectral and spatial dimensions can be considered at the same time, so as to express the data features more comprehensively. Cross-group attention fusion can extract more effective spatial-spectral features, realize the complementary advantages of information of different dimensions, and improve the overall effect of feature extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a structural schematic diagram of the multi-network fusion hyperspectral image classification method with noise resistance of the present invention. DETAILED DESCRIPTION

[0020] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments and drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] The multi-network fusion hyperspectral image classification method with noise resistance of the present invention (see Figure 1 ), including the following steps: S1. Obtain the original hyperspectral image under the influence of different interference factors (such as atmosphere, light, etc.), use the CapsuleGAN network to generate virtual hyperspectral pixels for the original hyperspectral image; merge the pixels in the original hyperspectral image with the virtual hyperspectral pixels, and each pixel has its own band information and category label; A pixel point and the corresponding category and band information constitute a sample; all samples constitute a hyperspectral image dataset; S2. Construct a band perception module, wherein the band perception module is used to extract band features and remove insensitive bands in the sample to obtain a hyperspectral sensitive data set; S3. Build fusion model: The fusion model includes a convolutional neural network with spatial group attention and spectral group attention, a graph attention network GAT, a hierarchical sampling operation and an importance sampling operation; The convolutional neural network containing the spatial group attention and the spectral group attention is used to extract local features of the input to obtain local features; The local features are processed by a hierarchical sampling operation and an importance sampling operation respectively to obtain a hierarchical graph structure and an importance graph structure, and then feature fusion is performed on the hierarchical graph structure and the importance graph structure to obtain a fused graph structure; The fused graph structure is then processed by the graph attention network GAT to obtain the output of the fusion model; S4. The fusion model is trained using a hyperspectral sensitive data set, and the trained fusion model is used to perform classification prediction on the hyperspectral image to be classified.

[0022] After the hyperspectral sensitive data set is constructed, in the training phase, the input of the fusion model is the samples in the hyperspectral sensitive data set. After the training is completed, the input of the fusion model is the hyperspectral image to be classified, and no sensitivity analysis is required for the hyperspectral image to be classified.

[0023] Specifically, the CapsuleGAN network described in step S1 combines the Generative Adversarial Network (GAN) and the Capsule Networks, including a generator, a discriminator, and a capsule layer, and uses the dynamic routing mechanism of capsules to capture the hierarchical structure of objects in hyperspectral images (such as the shape and texture of objects). The Capsule Network represents features through vector neurons (rather than scalars), which can retain the posture information of the features (such as direction and scale), thereby generating more realistic hyperspectral samples.

[0024] Its dynamic routing mechanism is shown in the following formula:

[0025]

[0026]

[0027] where c mn is the routing weight from the lower-level capsule m to the higher-level capsule n. The larger the weight, the stronger the correlation between the two. is the prediction vector of the lower-level capsule m for the higher-level capsule n. q represents the index of all higher-level capsules that the lower-level capsule m can be connected to. is the original coupling coefficient between the lower-level capsule m and the higher-level capsule n. is the weighted input vector of the high-level capsule n, which is obtained by weighted summation of the prediction vectors of all low-level capsules. is the output vector of the high-level capsule n, whose modulus length represents the capsule activation probability.

[0028] The construction process of the band perception module is: Band perception module: Through band perception comparative learning, the similarity of positive sample pairs is shortened and the similarity of negative sample pairs is extended, so that the model can learn the band invariance characteristics and improve the robustness of the model classification. In this embodiment, the band perception module uses a CNN network to extract band features; In the hyperspectral image dataset, 15% of the samples are selected for each category, and the band information of each band of the selected samples is taken as a separate band sample to form a band dataset, in which the band sample of a certain pixel point currently required for comparison in a certain band is defined as the anchor sample, the band sample that belongs to the same category as the anchor sample but is not in the same band is the positive sample, and the band sample that does not belong to the same category as the anchor sample but is in the same band is the negative sample; Train the CNN network with the band data set to obtain the trained CNN network, and set the inter-band contrast loss function during training. for:

[0029] Among them, f i Band features of anchor samples extracted by CNN network; represents the band features of the positive sample extracted by the CNN network; f k is the band feature extracted by the CNN network from the kth band sample in a band sample set containing one positive sample and N negative samples; N is the total number of negative samples relative to the current anchor sample; τ is the temperature parameter; Sensitivity calculation unit: take the band information of each band of all samples in the hyperspectral image dataset as input, use the trained CNN network to extract all band features, and calculate the variance of the band features of each band respectively. The larger the variance, the more sensitive the band feature change is; sort the variances of the band features of all bands from large to small, use the variance of the band features as the sensitivity index, retain the top 70% of the band information in the hyperspectral image dataset in terms of sensitivity, and remove the bottom 30% of the band information; The trained CNN network and the sensitivity calculation unit are combined into a band perception module. In actual use, there is no need to train the CNN network again. The band perception module is directly used to process the hyperspectral image dataset to obtain a hyperspectral sensitive dataset.

[0030] The present invention sets anchor samples, positive samples and negative samples, and the band samples in the band data set used in CNN network training are used as anchor samples in turn.

[0031] In the hyperspectral image of the present invention, each pixel corresponds to many bands. A hyperspectral image has multiple detection objects, and each detection object is composed of multiple pixels. Inter-band contrast learning is introduced, and the band characteristics of the hyperspectral image are used for self-supervised pre-training to enhance feature representation.

[0032] The construction process of the fusion model is: Step S3.1 The convolutional neural network with spatial group attention and spectral group attention can effectively capture the important information of each pixel and its neighboring areas through a series of convolutional layers and pooling layers, so that the model pays more attention to the image areas that are most important for classification decisions. The attention is divided into spectral group and spatial group, and the two are fused across groups, thereby retaining the local features of rich spatial and spectral features.

[0033] The convolutional neural network containing the spatial group attention and the spectral group attention comprises a plurality of convolutional blocks consisting of a plurality of convolutional layers, a nonlinear activation function, a pooling layer and a fully connected layer, wherein the parallel spatial group attention and the spectral group attention are connected after each convolutional block, and the outputs of the spatial group attention and the spectral group attention are spliced ​​and connected to the input of the next convolutional block, and the spliced ​​output result of the spatial group attention and the spectral group attention connected to the last convolutional block is used as the local feature; The convolutional neural network with spatial group attention and spectral group attention can be pre-trained using a known data set. During pre-training, a cross entropy loss function is used to set the learning rate, batch size, and network parameters, and local features are extracted using the pre-trained convolutional neural network with spatial group attention and spectral group attention.

[0034] The process of cross-group fusion of spectral group attention and spatial group attention is shown in the following formula:

[0035]

[0036] Among them, Attention spectral Attention is the spectral group attention, which is used to calculate the relationship between bands. Spatial is the spatial group attention, which is used to calculate the local area relationship, H fused is the result of cross-group attention fusion between the two. O is the output projection matrix, d k is the attention dimension. Q is the query vector, i.e., the feature vector used to guide the attention calculation. spec is the spectral query vector, which is used to find the relevant spectral features in the attention calculation at the spectral level. spat is the spatial query vector, which is used for attention calculation at the spatial level to find the spatial features related to it. K is the key vector, which is the vector matched with Q. By calculating the similarity between Q and K, we can determine which information in V is more important to Q. spec is the spectral key vector, corresponding to the spectral dimension, used with Q in the spectral group attention calculation spec Calculate the similarity. K spat is the spatial key vector, which is used together with Q in the spatial group attention calculation spat Calculate the similarity. V is a value vector containing the information actually used to calculate the output. spec is the spectral value vector. In the spectral group attention calculation, it is the spectral feature vector that actually participates in the output calculation. Its information weight is determined by Q spec and K spec The similarity of V spat is a spatial value vector. In the spatial group attention calculation, it is the spatial feature vector that actually participates in the output calculation. Its information weight is determined by Q spat and K spat The similarity is determined.

[0037] Step S3.2 performs importance sampling and hierarchical sampling on local features respectively, and uses the graph structures (hierarchical graph structure and importance graph structure) generated by the two sampling strategies as heterogeneous inputs, which are fused through a shared attention mechanism to generate new nodes and construct a fused graph structure; The process of fusion by shared attention mechanism is: for each node of the hierarchical graph structure and the importance graph structure, the attention weight between the nodes in the two graph structures is calculated , use a multi-layer perceptron MLP to calculate the attention score:

[0038] in, Indicates that the nodes in the hierarchical graph structure Features Nodes in the importance graph Features Perform feature concatenation, l represents the level of the node in the current hierarchical graph structure, i represents the feature category of the node in the current importance graph structure, j and k are used to represent the current node index of the hierarchical graph structure and the importance graph structure respectively, and N j Represents the total number of nodes in the current level l of the hierarchical graph structure; N k Represents the total number of nodes in the importance graph structure within the range of the current feature category i; MLP is a multi-layer perceptron function with three hidden layers and softmax as the activation function; is the feature of the j'th node at level l in the hierarchical graph structure, is the feature of the k'th node of the i-th feature category in the importance graph structure; is the attention weight between nodes; Characteristics of the graph structure after fusion The calculation formula is:

[0039] in, Represents the node features in the importance graph structure according to the attention weight ; β is a hyperparameter used to balance the contribution of the features of the hierarchical graph structure itself.

[0040] Step S3.3 uses the graph attention network GAT to dynamically adjust the attention weight of each node in the fused graph structure to its neighboring nodes, enhancing the model's ability to capture important node information; through the graph convolution operation GCN, the information of adjacent nodes is aggregated to the current node, allowing the model to utilize global context information and capture long-distance pixel dependencies, making up for the shortcomings of traditional CNN in processing global image information.

[0041] Input the fused graph structure into the graph convolution operation GCN, determine the node feature matrix and edge connection information, prepare the initial feature vector for each node, and form the feature matrix H;

[0042] Where N is the number of nodes and F is the feature dimension.

[0043] Set each pair of nodes to be i and its neighbor node j, and use linear transformation and activation function LeakyReLU to calculate the similarity of features, which is the attention score e ij , expressed as follows:

[0044] Among them, W is the learnable weight matrix, || represents vector concatenation, and h i and h j is the feature vector of node i and node j, and LeakyReLU is the activation function.

[0045] Introduce the multi-head attention mechanism (Multi-Head Attention), the weight of the adjacent node is α ij The specific formula is as follows:

[0046] Among them, N(i) is the neighbor set of node i.

[0047] For each node, the updated feature representation is obtained by the weighted features of its neighbors:

[0048] Among them, h′ i is the updated feature representation of node i, σ is the activation function, and W is the learnable weight matrix.

[0049] S4. The fusion model is trained using a hyperspectral sensitive data set, and the trained fusion model is used to perform classification prediction on the hyperspectral image to be classified, thereby achieving classification prediction on the entire image data and representing it in the form of a hyperspectral image.

[0050] In summary, the present invention expands the scale of the data set and solves the problem of insufficient samples by introducing the CapsuleGAN network and performing comparative learning between bands. The capsule network can retain the posture information of the retained features, help generate more realistic sample data, and improve the feature expression ability and the generalization ability and robustness of the model. The convolutional neural network is used to extract local features, and the extracted local features are hierarchically sampled and importance sampled. The graph structure is generated and the feature fusion is performed to generate the fused graph structure. As the input of GAT, the entire fusion model is trained, which can significantly improve the performance of hyperspectral image classification. The spatial and spectral features extracted by the convolutional neural network ensure the full utilization of local feature information, and make the data features more comprehensively expressed, and pay more attention to the utilization of multi-dimensional features. The hierarchical sampling of local feature data and the fusion of importance sampling and graph structure features enable the model to understand the image more comprehensively, integrate multi-scale structural information (hierarchical sampling) and key features (importance sampling), and make the model more stable when facing interference factors such as atmosphere and illumination, thereby improving the classification accuracy and robustness.

[0051] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0052] Any matters not described in the present invention are applicable to the prior art.

Claims

1. A multi-network fusion hyperspectral image classification method with noise resistance, characterized in that: The classification method comprises the following steps: S1. Obtain the original hyperspectral image under the influence of different interference factors, and use the CapsuleGAN network to generate virtual hyperspectral pixels for the original hyperspectral image; merge the pixels in the original hyperspectral image with the virtual hyperspectral pixels, and each pixel has its own band information and category label; A pixel point and the corresponding category label and band information constitute a sample; all samples constitute a hyperspectral image dataset; S2. Construct a band perception module, which is used to extract band features and remove insensitive bands in samples to obtain a hyperspectral sensitive data set; S3. Build fusion model: The fusion model includes a convolutional neural network with spatial group attention and spectral group attention, a graph attention network GAT, a hierarchical sampling operation, and an importance sampling operation; The convolutional neural network containing the spatial group attention and the spectral group attention is used to extract local features of the input to obtain local features; The local features are processed by a hierarchical sampling operation and an importance sampling operation respectively to obtain a hierarchical graph structure and an importance graph structure, and then feature fusion is performed on the hierarchical graph structure and the importance graph structure to obtain a fused graph structure; The fused graph structure is then processed by the graph attention network GAT to obtain the output of the fusion model; S4. The fusion model is trained using a hyperspectral sensitive data set, and the trained fusion model is used to perform classification prediction on the hyperspectral image to be classified.

2. The multi-network fusion hyperspectral image classification method with noise resistance according to claim 1 is characterized in that: The construction process of the band perception module is: In the hyperspectral image dataset, 15% of the samples are selected for each category, and the band information of each band of the selected samples is taken as a separate band sample to form a band dataset, in which the band sample of a certain pixel point currently required for comparison in a certain band is defined as the anchor sample, the band sample that belongs to the same category as the anchor sample but is not in the same band is the positive sample, and the band sample that does not belong to the same category as the anchor sample but is in the same band is the negative sample; Train the CNN network with the band data set to obtain the trained CNN network, and set the inter-band contrast loss function during training. for: ， Among them, f i Band features of anchor samples extracted by CNN network; represents the band features of the positive sample extracted by the CNN network; f k is the band feature extracted by the CNN network from the kth band sample in a band sample set containing one positive sample and N negative samples; N is the total number of negative samples; τ is the temperature parameter; Sensitivity calculation unit: take the band information of each band of all samples in the hyperspectral image dataset as input, use the trained CNN network to extract all band features, and calculate the variance of the band features of each band respectively; sort the variances of the band features of all bands from large to small, use the variance of the band features as the sensitivity index, retain the top 70% of the band information in the hyperspectral image dataset in terms of sensitivity, and remove the bottom 30% of the band information; The trained CNN network and the sensitivity calculation unit are combined into a band perception module.

3. The multi-network fusion hyperspectral image classification method with noise resistance according to claim 1 is characterized in that: The convolutional neural network containing the spatial group attention and the spectral group attention comprises a plurality of convolutional blocks consisting of a plurality of convolutional layers, a nonlinear activation function, a pooling layer and a fully connected layer, wherein the parallel spatial group attention and the spectral group attention are connected after each convolutional block, and the outputs of the spatial group attention and the spectral group attention are spliced ​​and connected to the input of the next convolutional block, and the spliced ​​output result of the spatial group attention and the spectral group attention connected to the last convolutional block is used as the local feature; The cross-group fusion process of spectral group attention and spatial group attention is as follows: ， Among them, Attention spectral Attention is the spectral group attention, which is used to calculate the relationship between bands; Spatial is the spatial group attention, which is used to calculate the local area relationship; H fused is the result of cross-group attention fusion between the two; W O is the output projection matrix.

4. The multi-network fusion hyperspectral image classification method with noise resistance according to claim 1 is characterized in that: The process of feature fusion of the hierarchical graph structure and the importance graph structure is: For each node in the hierarchical graph and the importance graph, calculate the attention weight between the nodes in the two graphs : ， in, Indicates that the nodes in the hierarchical graph structure Features Nodes in the importance graph Features Perform feature concatenation, l represents the level of the node in the current hierarchical graph structure, i represents the feature category of the node in the current importance graph structure, j and k are used to represent the current node index of the hierarchical graph structure and the importance graph structure respectively, and N j Represents the total number of nodes in the current level l of the hierarchical graph structure; N k Represents the total number of nodes in the importance graph structure within the range of the current feature category i; MLP is a multi-layer perceptron function with three hidden layers and softmax as the activation function; is the feature of the j'th node at level l in the hierarchical graph structure, is the feature of the k'th node of the i-th feature category in the importance graph structure; Characteristics of the graph structure after fusion The calculation formula is: ， in, represents the weighted sum of node features in the importance graph structure according to the attention weight, and β is a hyperparameter.

5. The multi-network fusion hyperspectral image classification method with noise resistance according to claim 1 is characterized in that: The CapsuleGAN network includes a generative adversarial network and a capsule network.

6. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the steps of the classification method described in any one of claims 1 to 5.

Images