Hyperspectral image classification feature extraction method based on self-attention network

By using a self-attention network for spatial-spectral feature extraction, the problem of spatial and spectral attention connections was solved, the classification accuracy of hyperspectral images was improved, and local spectral difference features were extracted.

CN118968174APending Publication Date: 2024-11-15HUNAN UNIV
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411088848.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

Existing technologies struggle to effectively connect spatial attention and spectral attention, and it is difficult to obtain the importance attention weight matrix for spectral or spatial location, which affects the accuracy of hyperspectral image classification.

Method used

A hyperspectral image classification feature extraction method based on self-attention network is adopted. Through multi-granularity label generation, transformer encoder module and classifier, combined with spatial self-attention-spectral center self-attention network, spatial spectral features are extracted and local information representation is enhanced. The final classification result is obtained by training with feedforward neural network.

Benefits of technology

It improves the classification accuracy of hyperspectral images, avoids the destruction of spectral data and the influence of irrelevant spectra, focuses on the region of interest in the detection task, and extracts local spectral difference features of the image.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118968174A_ABST
    Figure CN118968174A_ABST
Patent Text Reader

Abstract

The invention discloses a hyperspectral image classification feature extraction method based on a self-attention network, provides a feature extraction network based on a spatial spectrum self-attention mechanism, directly utilizes initial input to carry out spectrum attention operation and spatial attention operation, and associates the results. A spectral attention mechanism and a space attention mechanism are introduced into a feature extraction network, so that the network pays more attention to a region in which a detection task is interested, and the problems that spectral data are damaged and spectral importance cannot be correctly extracted are avoided; a feature extraction network based on a spectrum center self-attention network is provided, the similarity between a spectrum with the maximum contribution rate and other spectrums is calculated, the influence of irrelevance between different spectrums is avoided, a neural network more pays attention to spectrum bands with large contribution, local difference features of the image spectrums are extracted, and the accuracy of the image spectrum local difference is improved. And the hyperspectral image classification precision is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a hyperspectral image classification feature extraction method based on a self-attention network. Background Art

[0002] Hyperspectral images (HSI) are obtained by imaging spectrometers, which provide tens to hundreds of fine-tuned spectral details for each pixel, forming a seamless and uninterrupted spectral curve. Hyperspectral data reveal the presence of materials that cannot be detected in traditional broadband remote sensing. Hyperspectral images can be conceptualized as three-dimensional representations, and compared with common RGB images, HSI contains richer spectral information. Since different targets have different refractive indices in different spectral bands, the rich spectral information contained in hyperspectral images plays a vital role in hyperspectral classification. HSI has been widely used in the field of image processing. Hyperspectral image classification is challenging due to the influence of factors such as space, time, environmental changes, and ground object types on the lighting conditions of high-light fall images.

[0003] In recent years, attention-based HSI classification has received increasing attention. It uses both spatial attention and spectral attention to capture the desired spatial-spectral features, thereby achieving better classification accuracy. However, it is challenging to connect these two attention mechanisms without affecting the spatial-spectral features. In addition, how to obtain the attention weight matrix that determines the importance of each spectral or spatial position is also an important issue in the use of attention mechanisms.

[0004] Based on this, a hyperspectral image classification feature extraction method based on self-attention network is proposed in the present invention. Summary of the invention

[0005] In view of the above technical problems, the present invention provides a hyperspectral image classification feature extraction method based on self-attention network.

[0006] The technical solution adopted by the present invention to solve the technical problem is:

[0007] A hyperspectral image classification feature extraction method based on a self-attention network, the method comprising the following steps:

[0008] S100: Acquire a hyperspectral image, form a data set from all pixels of the hyperspectral image, and preprocess the data set to obtain a preprocessed data set;

[0009] S200: Build a classification feature extraction model, including a multi-granularity label generator, a transformer encoder module, and a classifier;

[0010] S300: inputting the hyperspectral image in the preprocessed data set into a multi-granularity label generator to obtain a multi-granularity label;

[0011] S400: Input the multi-granularity labels into the transformer encoder module to extract features, use the spatial-spectral feature acquisition module based on the spatial self-attention-spectral center self-attention network to extract the spatial-spectral features to obtain their long-range dependencies, and use the feedforward neural network to enhance the representation of local information to obtain enhanced spatial-spectral features;

[0012] S500: Input the enhanced spatial spectrum features into the classifier to obtain the classification result, and perform training in combination with the preset loss function. When the preset training end condition is reached, a trained model is obtained, and the classification prediction of the real-time hyperspectral image is completed according to the trained model.

[0013] Preferably, S100 specifically includes:

[0014] Get a hyperspectral image, form a dataset with all pixels of the hyperspectral image, and define the entire hyperspectral image as X∈R H×W×C , where H, W, and C represent the length, width, and number of bands of the hyperspectral image, respectively, and divide X into sub-blocks x of size s×s.

[0015] Preferably, S300 includes:

[0016] The hyperspectral image in the preprocessed dataset is input, and its convolution features are continuously extracted using multiple convolutions. Each convolution is followed by batch normalization and ReLU function to extract features from the input data. Finally, the extracted features are input into deep convolution and converted into semantic tags of different granularities using different convolution kernel sizes. They are then serialized to obtain multi-granularity label sequences. Where i represents the number of different particle sizes.

[0017] Preferably, the transformer encoder module in S400 includes a spatial-spectral feature acquisition module and a feedforward neural network module, and the spatial-spectral feature acquisition module includes a spatial attention weight matrix extractor, a spectral feature extractor and a final spatial-spectral feature extractor; S400 includes:

[0018] S410: Input the granularity label sequence into the spatial attention weight matrix extractor to obtain the spatial attention weight matrix;

[0019] S420: Inputting the granularity tag sequence into a spectral feature extractor to obtain a spectral feature graph;

[0020] S430: sending the spatial attention weight matrix and the spectral feature map to the final spatial-spectral feature extractor to obtain a feature map containing spatial-spectral features and obtain its long-range dependency;

[0021] S440: Use a feedforward neural network to enhance the representation of local information of the obtained spatial spectrum feature map.

[0022] Preferably, S410 includes:

[0023] Select a branch from the multi-granularity label sequence, and the input granularity label sequence is T∈R n×d , the granularity label sequence T∈R n×d The corresponding intermediate feature map Q is obtained through the linear layer spa ∈R n×d , K spa ∈R n×d , the dimensions of the two intermediate feature maps are consistent with the granular label sequence dimensions; where d represents the number of tokens and n represents the number of channels;

[0024] Multiply the two intermediate feature maps to obtain the preliminary spatial attention weight matrix, fill it, and set the filling value to 1;

[0025] Then use 2D convolution with a convolution kernel size of 3×3 to perform feature extraction and obtain the final spatial attention weight matrix W spa ∈R n×n :

[0026] W spa =conv2D(Q spa K spa T )

[0027] Here, conv2D(·) represents a 2D convolution with a kernel of 3×3 and padding set to 1.

[0028] Preferably, S420 includes:

[0029] The granularity label sequence T∈R n×d Perform a transposition operation to obtain the input T of the spectral feature attention mechanism T ∈R d×n ;

[0030] Transpose T of the granular label sequence T ∈R d×n Input the linear layer to obtain two corresponding intermediate feature maps Q spe ∈R d ×n , V spe ∈R d×n ,The dimensions of the two intermediate feature maps are consistent with the granular label sequence dimensions;

[0031] The PCA principal component analysis algorithm was used to extract Q spe ∈Rd×n The spectrum with the largest contribution rate is taken as the central spectrum vector, and Z0∈R 1×n express;

[0032] The intermediate feature map Q spe Multiply it with the central spectral vector Z0 to obtain the preliminary spectral center attention weight matrix, fill it with a value of 1, and then use a 2D convolution with a convolution kernel size of 3×3 to perform feature extraction;

[0033] The obtained spectral center attention matrix is ​​repeated along the spatial dimension to obtain the final spectral center attention weight matrix W spe ∈R d×n , therefore, the spectral center attention weight matrix is ​​calculated as follows:

[0034]

[0035] where conv2D(·) represents a 2D convolution with a kernel of 3×3 and padding set to 1. Indicates Q spe The characteristic dimension of Z0 T represents the transpose of Z0;

[0036] Finally, the spectral center attention weight matrix W is used spe and the intermediate feature map V spe Perform point-by-point multiplication to obtain a preliminary spectral feature map, perform an average pooling operation on it, and obtain the final spectral feature map Out spe ∈R d×n ; Out of spectral characteristic graph spe The acquisition is expressed by the following formula:

[0037] Out spe =avgpool(W spe ×V spe )

[0038] Where × represents a point-wise multiplication operation, and avgpool(·) means using a 3×3 kernel with a stride of 1 and applying an average pooling operation with padding of 1 in order not to change the size of the feature map.

[0039] Preferably, S430 includes:

[0040] The spatial attention weight matrix W spa And spectral characteristic diagram Out spe Multiply them to get the final empty spectrum feature map Out spa ∈R d×n , Out of the empty spectrum feature map spa The acquisition is expressed by the following formula:

[0041]

[0042] Among them, softmax(·) represents the softmax operation, W spa The characteristic dimension of .

[0043] Preferably, S500 includes:

[0044] The enhanced spatial spectrum feature map is input into the classifier composed of fully connected layers to calculate the probability of each pixel belonging to each category, and multiple classification results are obtained at the same time. Finally, the results of different granularities are fused using the adaptive weighted fusion method to obtain the probability of each pixel belonging to each category.

[0045] The loss function obtained from the training set is continuously iterated until the loss is less than the preset threshold or the predetermined number of iterations is completed, and the final model weight is recorded; then the model weight is loaded and tested on the test set, and the predicted classification results are compared with the actual labels to obtain the final classification accuracy;

[0046] The entire hyperspectral image is sent to the final trained model for classification prediction, all pixel prediction results are spliced ​​according to position, and finally the final visualization image is obtained according to the classification prediction results.

[0047] The above-mentioned hyperspectral image classification feature extraction method based on self-attention network proposes a feature extraction network based on spatial-spectral self-attention mechanism, directly uses the initial input to perform spectral attention operation and spatial attention operation, links their results, and introduces spectral attention mechanism and spatial attention mechanism into the feature extraction network, so that the network pays more attention to the area of ​​interest of the detection task, avoiding the problem of spectral data being destroyed and spectral importance not being correctly extracted; and proposes a feature extraction network based on spectral center self-attention network, calculates the similarity between the spectrum with the largest contribution rate and other spectra, avoids the influence of irrelevant spectra between different spectra, makes the neural network pay more attention to the spectral bands with large contributions, extracts local difference features of image spectra, and improves the classification accuracy of hyperspectral images. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a flow chart of a method for extracting hyperspectral image classification features based on a self-attention network in one embodiment of the present invention;

[0049] Figure 2 Schematic diagram of a feature extraction network based on a spatial spectral self-attention mechanism in one embodiment of the present invention;

[0050] Figure 3Schematic diagram of a spectral centric self-attention network in one embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings.

[0052] In one embodiment, Figure 1 As shown, a hyperspectral image classification feature extraction method based on a self-attention network comprises the following steps:

[0053] S100: Acquire a hyperspectral image, form a data set from all pixels of the hyperspectral image, and preprocess the data set to obtain a preprocessed data set;

[0054] S200: Build a classification feature extraction model, including a multi-granularity label generator, a transformer encoder module, and a classifier;

[0055] S300: inputting the hyperspectral image in the preprocessed data set into a multi-granularity label generator to obtain a multi-granularity label;

[0056] S400: Input the multi-granularity labels into the transformer encoder module to extract features, use the spatial-spectral feature acquisition module based on the spatial self-attention-spectral center self-attention network to extract the spatial-spectral features to obtain their long-range dependencies, and use the feedforward neural network to enhance the representation of local information to obtain enhanced spatial-spectral features;

[0057] S500: Input the enhanced spatial spectrum features into the classifier to obtain the classification result, and perform training in combination with the preset loss function. When the preset training end condition is reached, a trained model is obtained, and the classification prediction of the real-time hyperspectral image is completed according to the trained model.

[0058] The above-mentioned hyperspectral image classification feature extraction method based on self-attention network proposes a feature extraction network based on spatial-spectral self-attention mechanism, directly uses the initial input to perform spectral attention operation and spatial attention operation, links their results, and introduces spectral attention mechanism and spatial attention mechanism into the feature extraction network, so that the network pays more attention to the area of ​​interest of the detection task, avoiding the problem of spectral data being destroyed and spectral importance not being correctly extracted; and proposes a feature extraction network based on spectral center self-attention network, calculates the similarity between the spectrum with the largest contribution rate and other spectra, avoids the influence of irrelevant spectra between different spectra, makes the neural network pay more attention to the spectral bands with large contributions, extracts local difference features of image spectra, and improves the classification accuracy of hyperspectral images.

[0059] In one embodiment, S100 specifically includes:

[0060] Get a hyperspectral image, form a dataset with all pixels of the hyperspectral image, and define the entire hyperspectral image as X∈R H×W×C , where H, W, and C represent the length, width, and number of bands of the hyperspectral image, respectively, and divide X into sub-blocks x of size s×s.

[0061] Furthermore, all pixels of the hyperspectral image constitute the data set D, and 1% of the labeled pixels are randomly selected as training samples as training set D train , the remaining pixels are used as the test set D test Among them, the number of pixels in the training set is N×1%, and the number of pixels in the test set is N×99%.

[0062] In one embodiment, S300 includes:

[0063] The hyperspectral image in the preprocessed dataset is input, and its convolution features are continuously extracted using multiple convolutions. Each convolution is followed by batch normalization and ReLU function to extract features from the input data. Finally, the extracted features are input into deep convolution and converted into semantic tags of different granularities using different convolution kernel sizes. They are then serialized to obtain multi-granularity label sequences. Where i represents the number of different particle sizes.

[0064] In one embodiment, Figure 2 As shown, the transformer encoder module in S400 includes a spatial-spectral feature acquisition module and a feedforward neural network module, and the spatial-spectral feature acquisition module includes a spatial attention weight matrix extractor, a spectral feature extractor, and a final spatial-spectral feature extractor; S400 includes:

[0065] S410: Input the granularity label sequence into the spatial attention weight matrix extractor to obtain the spatial attention weight matrix;

[0066] S420: Inputting the granularity tag sequence into a spectral feature extractor to obtain a spectral feature graph;

[0067] S430: sending the spatial attention weight matrix and the spectral feature map to the final spatial-spectral feature extractor to obtain a feature map containing spatial-spectral features and obtain its long-range dependency;

[0068] S440: Use a feedforward neural network to enhance the representation of local information of the obtained spatial spectrum feature map.

[0069] In one embodiment, S410 includes:

[0070] Select a branch from the multi-granularity label sequence, and the input granularity label sequence is T∈R n×d, the granularity label sequence T∈R n×d The corresponding intermediate feature map Q is obtained through the linear layer spa ∈R n×d , K spa ∈R n×d , the dimensions of the two intermediate feature maps are consistent with the granular label sequence dimensions; where d represents the number of tokens and n represents the number of channels;

[0071] Multiply the two intermediate feature maps to obtain the preliminary spatial attention weight matrix, fill it, and set the filling value to 1;

[0072] Then use 2D convolution with a convolution kernel size of 3×3 to perform feature extraction and obtain the final spatial attention weight matrix W spa ∈R n×n :

[0073] W spa =conv2D(Q spa K spa T )

[0074] Here, conv2D(·) represents a 2D convolution with a kernel of 3×3 and padding set to 1.

[0075] Specifically, this operation is used to merge relevant information between adjacent pixels.

[0076] The schematic diagram of the spectral center self-attention network in the spectral feature extractor is as follows Figure 3 As shown, in one embodiment, S420 includes:

[0077] The granularity label sequence T∈R n×d Perform a transposition operation to obtain the input T of the spectral feature attention mechanism T ∈R d×n ;

[0078] Transpose T of the granular label sequence T ∈R d×n Input the linear layer to obtain two corresponding intermediate feature maps Q spe ∈R d ×n , V spe ∈R d×n ,The dimensions of the two intermediate feature maps are consistent with the granular label sequence dimensions;

[0079] The PCA principal component analysis algorithm was used to extract Q spe ∈R d×n The spectrum with the largest contribution rate is taken as the central spectrum vector, and Z0∈R 1×n express;

[0080] The intermediate feature map Q spe Multiply it with the central spectral vector Z0 to obtain the preliminary spectral center attention weight matrix, fill it with a value of 1, and then use a 2D convolution with a convolution kernel size of 3×3 to perform feature extraction;

[0081] The obtained spectral center attention matrix is ​​repeated along the spatial dimension to obtain the final spectral center attention weight matrix W spe ∈R d×n , therefore, the spectral center attention weight matrix is ​​calculated as follows:

[0082]

[0083] where conv2D(·) represents a 2D convolution with a kernel of 3×3 and padding set to 1. Indicates Q spe The characteristic dimension of Z0 T represents the transpose of Z0;

[0084] Finally, the spectral center attention weight matrix W is used spe and the intermediate feature map V spe Perform point-by-point multiplication to obtain a preliminary spectral feature map, perform an average pooling operation on it, and obtain the final spectral feature map Out spe ∈R d×n ; Out of spectral characteristic graph spe The acquisition is expressed by the following formula:

[0085] Out spe =avgpool(W spe ×V spe )

[0086] Where × represents a point-wise multiplication operation, and avgpool(·) means using a 3×3 kernel with a stride of 1 and applying an average pooling operation with padding of 1 in order not to change the size of the feature map.

[0087] Specifically, since the spectral center self-attention module needs to avoid misclassification caused by irrelevant spectra, the present invention only calculates the similarity between the spectrum with the largest contribution rate and other spectra, and thus uses the PCA principal component analysis algorithm to extract Q spe ∈R d×n The spectrum with the largest contribution rate is taken as the central spectrum vector.

[0088] In one embodiment, S430 includes:

[0089] The spatial attention weight matrix W spa And spectral characteristic diagram Out speMultiply them to get the final empty spectrum feature map Out spa ∈R d×n , Out of the empty spectrum feature map spa The acquisition is expressed by the following formula:

[0090]

[0091] Among them, softmax(·) represents the softmax operation, W spa The characteristic dimension of .

[0092] Specifically, after completing the above operations, a feature map containing spatial spectral features is obtained, and its long-distance dependency is obtained. Finally, a feedforward neural network is used to enhance the representation of local information on the obtained spatial spectral feature map, which helps to further improve the classification performance of the model.

[0093] In one embodiment, S500 includes:

[0094] The enhanced spatial spectrum feature map is input into the classifier composed of fully connected layers to calculate the probability of each pixel belonging to each category, and multiple classification results are obtained at the same time. Finally, the results of different granularities are fused using the adaptive weighted fusion method to obtain the probability of each pixel belonging to each category.

[0095] The loss function obtained from the training set is continuously iterated until the loss is less than the preset threshold or the predetermined number of iterations is completed, and the final model weight is recorded; then the model weight is loaded and tested on the test set, and the predicted classification results are compared with the actual labels to obtain the final classification accuracy;

[0096] The entire hyperspectral image is sent to the final trained model for classification prediction, all pixel prediction results are spliced ​​according to position, and finally the final visualization image is obtained according to the classification prediction results.

[0097] The above-mentioned hyperspectral image classification feature extraction method based on self-attention network solves the connection problem between spatial attention and spectral attention, and can effectively obtain the attention weight matrix of spectral or spatial position importance, so that the neural network pays more attention to the spectral bands with large contributions, extracts the local difference characteristics of the image spectrum, and improves the classification accuracy of hyperspectral images.

[0098] The above is a detailed introduction to a hyperspectral image classification feature extraction method based on a self-attention network provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the core idea of ​​the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A hyperspectral image classification feature extraction method based on self-attention network, characterized in that: The method comprises the following steps: S100: Acquire a hyperspectral image, form a data set from all pixels of the hyperspectral image, and preprocess the data set to obtain a preprocessed data set; S200: Build a classification feature extraction model, including a multi-granularity label generator, a transformer encoder module, and a classifier; S300: inputting the hyperspectral image in the preprocessed data set into a multi-granularity label generator to obtain a multi-granularity label; S400: Input the multi-granularity labels into the transformer encoder module to extract features, use the spatial-spectral feature acquisition module based on the spatial self-attention-spectral center self-attention network to extract the spatial-spectral features to obtain their long-range dependencies, and use the feedforward neural network to enhance the representation of local information to obtain enhanced spatial-spectral features; S500: Input the enhanced spatial spectrum features into the classifier to obtain the classification result, and perform training in combination with the preset loss function. When the preset training end condition is reached, a trained model is obtained, and the classification prediction of the real-time hyperspectral image is completed according to the trained model.

2. The method according to claim 1, characterized in that S100 is specifically: Get a hyperspectral image, form a dataset with all pixels of the hyperspectral image, and define the entire hyperspectral image as X∈R H×W×C , where H, W, and C represent the length, width, and number of bands of the hyperspectral image, respectively, and divide X into sub-blocks x of size s×s.

3. The method according to claim 2, characterized in that S300 includes: The hyperspectral image in the preprocessed dataset is input, and its convolution features are continuously extracted using multiple convolutions. Each convolution is followed by batch normalization and ReLU function to extract features from the input data. Finally, the extracted features are input into deep convolution and converted into semantic tags of different granularities using different convolution kernel sizes. They are then serialized to obtain multi-granularity label sequences. Where i represents the number of different particle sizes.

4. The method according to claim 2, characterized in that: The transformer encoder module in S400 includes a spatial-spectral feature acquisition module and a feed-forward neural network module, and the spatial-spectral feature acquisition module includes a spatial attention weight matrix extractor, a spectral feature extractor, and a final spatial-spectral feature extractor; S400 includes: S410: Input the granularity label sequence into the spatial attention weight matrix extractor to obtain the spatial attention weight matrix; S420: Inputting the granularity tag sequence into a spectral feature extractor to obtain a spectral feature graph; S430: sending the spatial attention weight matrix and the spectral feature map to the final spatial-spectral feature extractor to obtain a feature map containing spatial-spectral features and obtain its long-range dependency; S440: Use a feedforward neural network to enhance the representation of local information of the obtained spatial spectrum feature map.

5. The method according to claim 4, characterized in that S410 includes: Select a branch from the multi-granularity label sequence, and the input granularity label sequence is T∈R n×d , the granularity label sequence T∈R n×d The corresponding intermediate feature map Q is obtained through the linear layer spa ∈R n×d ,K spa ∈R n×d , the dimensions of the two intermediate feature maps are consistent with the granular label sequence dimensions; where d represents the number of tokens and n represents the number of channels; Multiply the two intermediate feature maps to obtain the preliminary spatial attention weight matrix, fill it, and set the filling value to 1; Then use 2D convolution with a convolution kernel size of 3×3 to perform feature extraction and obtain the final spatial attention weight matrix W spa ∈R n×n : W spa =conv2D(Q spa K spa T ) Here, conv2D(·) represents a 2D convolution with a kernel of 3×3 and padding set to 1.

6. The method according to claim 5, characterized in that S420 includes: The granular label sequence T∈R n×d Perform a transposition operation to obtain the input T of the spectral feature attention mechanism T ∈R d×n ; Transpose T of the granular label sequence T ∈R d×n Input the linear layer to obtain two corresponding intermediate feature maps Q spe ∈R d×n ,V spe ∈R d×n ,The dimensions of the two intermediate feature maps are consistent with the granular label sequence dimensions; The PCA principal component analysis algorithm was used to extract Q spe ∈R d×n The spectrum with the largest contribution rate is taken as the central spectrum vector, and Z0∈R 1×n express; The intermediate feature map Q spe Multiply it with the central spectral vector Z0 to obtain the preliminary spectral center attention weight matrix, fill it with a value of 1, and then use a 2D convolution with a convolution kernel size of 3×3 to perform feature extraction; The obtained spectral center attention matrix is ​​repeated along the spatial dimension to obtain the final spectral center attention weight matrix W spe ∈R d×n , therefore, the spectral center attention weight matrix is ​​calculated as follows: where conv2D(·) represents a 2D convolution with a kernel of 3×3 and padding set to 1. Indicates Q spe The characteristic dimension of Z0 T represents the transpose of Z0; Finally, the spectral center attention weight matrix W is used spe and the intermediate feature map V spe Perform point-by-point multiplication to obtain a preliminary spectral feature map, perform an average pooling operation on it, and obtain the final spectral feature map Out spe ∈R d×n ; Out of spectral characteristic graph spe The acquisition is expressed by the following formula: Out spe =avgpool(W spe ×V spe ) where × represents a point-wise multiplication operation and avgpool(·) means using a 3×3 kernel with a stride of 1 and applying an average pooling operation with padding of 1.

7. The method according to claim 6, characterized in that S430 includes: The spatial attention weight matrix W spa And spectral characteristic diagram Out spe Multiply them to get the final empty spectrum feature map Out spa ∈R d×n , Out of the empty spectrum feature map spa The acquisition is expressed by the following formula: Among them, softmax(·) represents the softmax operation, W spa The characteristic dimension of .

8. The method according to claim 7, characterized in that S500 includes: The enhanced spatial spectrum feature map is input into the classifier composed of fully connected layers to calculate the probability of each pixel belonging to each category, and multiple classification results are obtained at the same time. Finally, the results of different granularities are fused using the adaptive weighted fusion method to obtain the probability of each pixel belonging to each category. The loss function obtained from the training set is continuously iterated until the loss is less than the preset threshold or the predetermined number of iterations is completed, and the final model weight is recorded; then the model weight is loaded and tested on the test set, and the predicted classification results are compared with the actual labels to obtain the final classification accuracy; The entire hyperspectral image is sent to the final trained model for classification prediction, all pixel prediction results are spliced ​​according to position, and finally the final visualization image is obtained according to the classification prediction results.

Citation Information

Cited By

  • Aluminum alloy mark identification method based on lightweight spectrum attention network

    CN119513576A

  • Hyperspectral image classification method based on channel graph and multistage convolution attention network

    CN119723176A