A hyperspectral image classification method

Abstract

The present invention discloses a hyperspectral image classification method in the field of hyperspectral image processing, aiming to solve the technical problem of low recognition accuracy caused by the difficulty in utilizing complex information in hyperspectral images. The method includes: firstly, removing useless information through preliminary feature extraction processing; utilizing the mutual learning activation characteristics between elements during KAN convolution operation processing to capture spatial and spectral features, and utilizing an adaptive attention mechanism to amplify spatial and spectral features, thereby reducing the occupation of computing resources by useless information; the adaptive attention mechanism uses the similarity between the query and the key based on the Hamming distance calculation, which can reduce the adverse effects of useless information; utilizing the refocusing convolution to adaptively adjust the response intensity of each local area in the feature map, controlling the feature expression through the response intensity, realizing the mining of image depth detail information, making full use of the complex spatial relationship and texture information in the hyperspectral image, and improving the classification efficiency and accuracy.

Description

Technical Field

[0001] The invention relates to a hyperspectral image classification method and belongs to the field of hyperspectral image processing. Background Art

[0002] With the rapid development of remote sensing technology, hyperspectral images have been widely used in environmental monitoring, resource management and other fields. Hyperspectral images have rich spectral information and can provide more detailed material composition information than traditional images. However, due to the high dimensionality and large amount of information in hyperspectral data, traditional image processing technology faces huge challenges in accurate classification and real-time processing.

[0003] Deep learning, especially convolutional neural networks (CNNs), has become an effective means of processing hyperspectral images. These networks can automatically learn the hierarchical features of data and improve classification accuracy. However, hyperspectral images usually contain hundreds of bands and have very high dimensions, which poses great challenges to feature extraction and classification. Limited training samples: In practical applications, it is often costly to obtain high-quality labeled training samples. Algorithms need to be able to learn effectively even with limited samples. Complex spatial information: There are complex spatial relationships and texture information in hyperspectral images, and conventional classification algorithms find it difficult to fully utilize this information. Summary of the invention

[0004] The purpose of this application is to overcome the deficiencies in the prior art and to provide a hyperspectral image classification method with strong feature extraction capability and full use of sample information.

[0005] In order to achieve the above objectives, this application is implemented by adopting the following technical solutions:

[0006] In a first aspect, the present application provides a hyperspectral image classification method, comprising:

[0007] Acquire a hyperspectral image, and extract a first feature map from the hyperspectral image;

[0008] Performing a KAN convolution operation on the first feature map to capture spatial and spectral features;

[0009] updating the first feature map using an adaptive attention mechanism to amplify spatial and spectral features in the first feature map to obtain a second feature map, wherein the adaptive attention mechanism calculates the similarity between the query and the key based on the Hamming distance;

[0010] Performing a refocus convolution on the second feature map, adjusting the response strength of a local area in the second feature map, and amplifying the feature expression in the local area according to the response strength of each local area to obtain a third feature map;

[0011] The features in the third feature map are linearly combined, and the features after the linear combination are identified to obtain a classification result.

[0012] In some embodiments of the first aspect, extracting the first feature map from the hyperspectral image includes: using a filter to scan each local area in the hyperspectral image to extract key feature positions; performing a KAN convolution operation on the hyperspectral image for the key feature positions to obtain the first feature map; and binarizing the first feature map using a pulse activation function;

[0013] The method of updating the first feature map using an adaptive attention mechanism to amplify the spatial and spectral features in the first feature map to obtain a second feature map also includes: binarizing the first feature map after the adaptive attention mechanism using a pulse activation function; performing a KAN convolution operation on the binarized first feature map to obtain a first feature map that highlights the spatial and spectral features; performing a second binarization operation on the first feature map that highlights the spatial and spectral features using a pulse activation function to suppress noise in the first feature map; and reducing the dimension of the first feature map after the second binarization operation by a multi-scale pooling operation; wherein the first feature map after dimensionality reduction is obtained by calculating the following formula:

[0014] ,

[0015] In the formula, is the first feature map after dimensionality reduction, and are the coordinates, is the pooling window size, is the counting number of the pooled traversal elements, is the first feature map after the second binarization process;

[0016] The second feature map is obtained according to the first feature map after dimensionality reduction.

[0017] In some embodiments of the first aspect, performing refocus convolution on the second feature map, adjusting the response strength of the local area in the second feature map, and amplifying the feature expression in the local area according to the response strength of each local area to obtain the third feature map includes:

[0018] Performing multiple refocusing convolutions on the second feature map, and after each refocusing convolution, using an impulse activation function to perform binarization processing on the second feature map to amplify the feature expression in the local area;

[0019] The process of each refocusing convolution is as follows:

[0020] The eigenvalue after refocusing convolution is calculated by the following formula:

[0021] ,

[0022] In the formula, is the eigenvalue after refocusing convolution, is the weight matrix of the refocusing convolution kernel, is the second feature map of the input, and are the coordinates, is a bias term used to adjust the output offset in the refocusing convolution operation;

[0023] The learnable parameters are introduced to dynamically adjust the eigenvalues ​​after refocusing convolution. The dynamically adjusted eigenvalues ​​are calculated by the following formula:

[0024] ,

[0025] In the formula, For the The eigenvalues ​​of the channels are dynamically adjusted. represents the channel index, For the The learnable parameters of the channel are used to control the The response strength of the channel, is the first The characteristic value of the channel;

[0026] The eigenvalues ​​of each channel after dynamic adjustment are reorganized to obtain a second feature map that completes a single refocusing convolution.

[0027] In some embodiments of the first aspect,

[0028] The second feature graph after feature expression amplification is further updated using an adaptive attention mechanism, wherein the adaptive attention mechanism calculates the similarity between the query and the key based on the Hamming distance, and optimizes the key features in the second feature graph according to the similarity between the query and the key;

[0029] The second feature map updated using the adaptive attention mechanism is then updated with a refocused convolution;

[0030] The dimension of the second feature map after refocusing convolution is reduced by multi-scale pooling operation, and the second feature map after dimensionality reduction is obtained by the following calculation:

[0031] ,

[0032] In the formula, is the second feature map after dimensionality reduction, is the pooling window size, is the counting number of the pooled traversal elements, is the second feature map after refocusing convolution update;

[0033] The third feature map is obtained according to the second feature map after dimensionality reduction.

[0034] In some embodiments of the first aspect, linearly combining the features in the third feature map and identifying the linearly combined features to obtain a classification result includes:

[0035] The linearly combined features are obtained according to the following formula:

[0036] ,

[0037] In the formula, is the weight matrix of the fully connected layer in the adaptive attention mechanism, is the third characteristic graph, is the bias term, is the feature after linear combination.

[0038] In some embodiments of the first aspect, the KAN convolution operation is expressed as follows:

[0039] ,

[0040] In the formula, and are the coordinates, is the spectral dimension, is the first feature map after the KAN convolution operation, is the weight matrix of the KAN convolution kernel, and are the coordinates of the convolution kernel, represents the sliding operation of the KAN convolution kernel, is the spectral dimension of the convolution kernel, are coordinates and the spectral dimensions are The pixel value at ;

[0041] The first convolution kernel of the KAN convolution operation adopts a Gaussian radial basis function convolution kernel.

[0042] In some embodiments of the first aspect, the processing formula for binarizing the first feature map using the impulse activation function is as follows:

[0043] ,

[0044] In the formula, and are the coordinates, is the spectral dimension, is the first feature map after binarization, express The following inequality is the execution condition. is the activation threshold, is the first feature map.

[0045] In some embodiments of the first aspect, updating the first feature map using an adaptive attention mechanism includes:

[0046] Performing linear transformation on the first feature graph to obtain a query vector, a key vector, and a value vector;

[0047] The similarity between the query vector and the key vector is calculated by the Hamming distance, and the similarity is obtained by the following calculation:

[0048] ,

[0049] In the formula, is the calculation operation of the Hamming distance, is the query vector, is the key vector, is the count number of the query vector, is the counting number of the key vector, Represents the query vector With key vector The similarities between For the The query vector elements, For the The first key vector elements, Represents the XOR operation of a binary vector. When the calculation result is 1, it means that the two elements are different. When the calculation result is 0, it means that the two elements are the same. is the element length of the vector, is the counting sequence number of the element;

[0050] Based on the similarity between the query vector and the key vector, an attention weight is generated, and the attention weight is obtained by the following calculation:

[0051] ,

[0052] In the formula, is the counting number of similarity, For the query vector and the The attention weights of the key vectors;

[0053] A new feature vector is generated according to the attention weight and the value vector, and an updated first feature map is obtained according to the new feature vector. The feature vector is obtained by the following calculation:

[0054] ,

[0055] In the formula, represents the updated feature vector, represents the count sequence of the value vector, is the key vector The corresponding value vector.

[0056] In a second aspect, the present application further provides a computer device, comprising a processor and a memory connected to the processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the hyperspectral image classification method as described in any embodiment of the first aspect are performed.

[0057] In a third aspect, the present application further provides a computer program product, comprising a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the steps of the hyperspectral image classification method described in any embodiment of the first aspect are implemented.

[0058] Compared with the prior art, the present invention has the following beneficial effects:

[0059] The hyperspectral image classification method provided by the present application first performs preliminary feature extraction processing on the hyperspectral image, preliminarily eliminates useless information, and obtains a feature map as the basis for subsequent processing; utilizes the mutual learning and activation characteristics between elements when processing the feature map by KAN convolution operation, efficiently captures spatial and spectral features from the preliminarily obtained feature map, and uses an adaptive attention mechanism to update the feature map to amplify the spatial and spectral features in the feature map, so as to concentrate computing resources on the spatial and spectral features, reduce the occupation of computing resources by useless information, and realize the utilization rate of complex information in the hyperspectral image; the adaptive attention mechanism uses a calculation based on the Hamming distance to query the similarity between keys, which can reduce the adverse effect of useless information mixed in useful information when the feature map is refocused convolution; uses refocusing convolution to adaptively adjust the response intensity of each local area in the feature map, controls the feature expression by the response intensity, realizes the mining of image depth detail information, and makes full use of the complex spatial relationship and texture information in the hyperspectral image to improve the classification efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0061] Figure 1 is a flowchart of the steps of the hyperspectral image classification method provided in this embodiment;

[0062] Figure 2 is a simplified comparison diagram of the pulse encoding process of the pulse activation function in the hyperspectral image classification method provided in this embodiment;

[0063] Figure 3 is a simplified diagram of the process of KAN convolution operation in the hyperspectral image classification method provided in this embodiment;

[0064] Figure 4 is a simplified diagram of the process of calculating the adaptive attention mechanism based on the Hamming distance in the hyperspectral image classification method provided in this embodiment;

[0065] Figure 5 This is a real hyperspectral ground image from the Pavia University hyperspectral image dataset.

[0066] Figure 6 yes Figure 5 The corresponding pseudo-color image;

[0067] Figure 7 For SVM Figure 5 Classification result map of the objects in the middle;

[0068] Figure 8 For 3D-CNN Figure 5 Classification result map of the objects in the middle;

[0069] Fig. 9 For KSANet Figure 5 Classification result map of the objects in the middle;

[0070] Fig.10 It is a schematic block diagram of the principles of the computer device provided in this embodiment. DETAILED DESCRIPTION

[0071] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0072] The term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0073] Embodiment 1:

[0074] Figure 1 This is a flow chart of a hyperspectral image classification method in the first embodiment of the present invention. This flow chart only shows the logical sequence of the method described in this embodiment. In other possible embodiments of the present invention, different methods may be used without conflict. Figure 1 The steps shown or described are accomplished in the order shown.

[0075] The hyperspectral image classification method provided in this embodiment can be applied to a terminal and can be executed by an image classification device, which can be implemented by software and / or hardware, and can be integrated in a terminal, such as any smart phone, tablet computer or computer device with communication function. Figure 1 The method of this implementation specifically includes the following steps:

[0076] A hyperspectral image is obtained, and a first feature map is extracted from the hyperspectral image. This step is to perform preliminary processing on the hyperspectral image, and its main function is to extract useful information from different bands by combining the spatial and spectral features of the image. The useful information mainly refers to the information containing the complex spatial relationship and texture information in the hyperspectral image. This step aims to exclude useless information for the first time, and provide a better input source for subsequent KAN convolution operations and similarity calculations based on Hamming distance and queries and keys. The method of extracting the first feature map can be implemented in other forms known in the art, which will not be repeated here.

[0077] A KAN convolution operation is performed on the first feature map to capture spatial and spectral features. The KAN convolution operation is a novel convolution operation method. Instead of simply applying a dot product between the kernel and the corresponding pixels in the image, a learnable nonlinear activation function is applied to each element and then they are added together. Therefore, using the KAN convolution operation instead of the traditional convolution operation can improve the efficiency of capturing spatial and spectral features, and can reduce the complexity and computational complexity of the method execution without sacrificing model performance.

[0078] The first feature map is updated using an adaptive attention mechanism to amplify the spatial and spectral features in the feature map, and the finally updated first feature map is used as the second feature map. The adaptive attention mechanism calculates the similarity between the query and the key based on the Hamming distance; the main function of the adaptive attention mechanism is to amplify the spatial and spectral features captured by the previous KAN convolution operation, and the amplified features are easily captured and processed by the next step of refocusing convolution, so that the next step of refocusing convolution mainly focuses on the spatial and spectral features; the useless information has been initially excluded when the first feature map was first extracted in the previous text, and the effect of eliminating useless information at this time is not strong. In comparison, the adaptive attention mechanism The system performs well in processing data containing noise or incomplete matching. The measure of using the adaptive attention mechanism to update the first feature map is to reduce the interference of unimportant features in the first feature map by amplifying important features, and to discard unimportant features and useless information more deeply in the next step of refocusing convolution; in addition, the similarity between the query and the key is calculated by the Hamming distance, which replaces the traditional similarity calculation method. This modified adaptive attention mechanism has a certain robustness to noise and small changes. Even if unimportant features and useless information are hidden in important features and useful information, it is difficult to affect the second feature map output by the adaptive attention mechanism for the final update of the first feature map.

[0079] The second feature map is refocused and convolved, the response intensity of the local area in the second feature map is adjusted, and the feature expression in the local area is amplified according to the response intensity of each local area to obtain the third feature map; after the first feature map is initially extracted to exclude useless information, the first feature map is updated by an adaptive attention mechanism to amplify the spatial and spectral features, and the Hamming distance calculation is used instead of the traditional similarity calculation to reduce the impact of the hidden useless information, the second feature map obtained has less useless information and clean information, and has the basis for deep feature extraction and detail feature extraction; the second feature map is refocused and convolved, and the response to different areas is dynamically adjusted by an adaptive focusing mechanism. It is not difficult for those skilled in the art to see that there are complex spatial relationships and texture information in hyperspectral images. On the one hand, conventional classification algorithms are difficult to make full use of this information. On the other hand, it also means that the image information has a certain depth, and the inability to study the depth details may reduce the classification effect; and the previous steps have reduced the difficulty of information processing by repeatedly eliminating useless information and amplifying image features. The purpose of refocusing and convolving the second feature map is to adaptively adjust the response intensity to the local area, capture detail features, and deeply utilize image information.

[0080] The third feature map finally obtained after the refocusing convolution includes multiple deeply mined and amplified features. Even multiple objects that look similar in the original hyperspectral image will appear very different after the refocusing convolution due to differences in details. The highly recognizable object features in the third feature map are obtained, linearly combined, and the features after the linear combination are identified to obtain the classification results.

[0081] The hyperspectral image classification method provided in this embodiment first performs preliminary feature extraction processing on the hyperspectral image, preliminarily eliminates useless information, and obtains a first feature map as a basis for subsequent processing; utilizes the mutual learning activation characteristics between elements during KAN convolution operation processing to efficiently capture spatial and spectral features from the preliminarily obtained first feature map, and uses an adaptive attention mechanism to update the first feature map and amplify the spatial and spectral features in the first feature map, so as to concentrate computing resources on spatial and spectral features, reduce the occupation of computing resources by useless information, and achieve utilization of complex information in the hyperspectral image; the adaptive attention mechanism uses the similarity between the query and the key based on the Hamming distance calculation, which can reduce the adverse effect of useless information mixed in the useful information when the second feature map is refocused convolution; uses the refocusing convolution to adaptively adjust the response intensity of each local area in the second feature map, controls the feature expression by the response intensity, realizes the mining of image depth detail information, makes full use of the complex spatial relationship and texture information in the hyperspectral image, and performs classification according to the linearly combined features in the third feature map, which can improve classification efficiency and accuracy.

[0082] Embodiment 2:

[0083] This embodiment provides a hyperspectral image classification method. This embodiment is optimized on the basis of the first embodiment to improve the technical effect and refine the technical solution. For the contents not fully described in this embodiment, please refer to the first embodiment.

[0084] As a preliminary step in the first embodiment, the first feature map may be extracted from the hyperspectral image by using the KAN convolution operation mentioned above.

[0085] Specifically, a filter is used to scan each local area in the hyperspectral image to extract key feature positions; a KAN convolution operation is then performed on the hyperspectral image at the key feature positions to obtain a first feature map; however, in order to reduce the amount of calculation in subsequent steps and highlight features, this embodiment uses an impulse activation function to binarize the first feature map, thereby compressing the amount of information and enhancing attention to key features.

[0086] The KAN convolution kernel is responsible for weighted summing of pixels in a specific area and its surrounding pixels to obtain local spatial-spectral features, combining the spatial and spectral features of the image to extract useful information from different bands. Through this operation, the first feature map extracted will contain preliminary local feature information, laying the foundation for subsequent feature processing. Figure 2 Example A in the example, example A performs a pulse activation function on the first feature Figure 2 Example operation of binarization, where the blue matrix is ​​the first feature map of the input, and the white matrix after the end of the last arrow is the first feature map of the output after binarization.

[0087] As one of the embodiments, the hyperspectral image can be divided into multiple small area patches, and the patch is used as an input unit to perform preliminary extraction and output of spatial and spectral features, which can reduce the single computing resource usage.

[0088] As one of the embodiments, after the adaptive attention mechanism is used to amplify the spatial and spectral features in the first feature map, further noise reduction and feature highlighting can be performed to couple the adaptive attention mechanism to the first feature map. Specifically, the pulse activation function is used to binarize the first feature map after the adaptive attention mechanism has been processed. This is slightly different from the previously mentioned binarization process. Here, it plays a role in further suppressing noise and highlighting key features, that is, further amplifying the spatial and spectral features on the basis of the adaptive attention mechanism amplification. Figure 2 Example B in the example, example B performs a pulse activation function on the first feature Figure 2 The example operation of binarization is shown in Figure 1. However, unlike Example A, the blue matrix in Example B is the input first feature map that has been binarized once. In Example B, the first feature map is binarized again and the output is the white matrix after the end of the last arrow.

[0089] As one example, in Figure 1 and Figure 2 In the pulse activation function represented by .

[0090] The KAN convolution operation is performed on the first feature map again to recapture the features that have been amplified twice from the first feature map, thereby obtaining the first feature map that highlights the spatial and spectral features.

[0091] However, the first feature map obtained after the KAN convolution operation may have entrained noise that is synchronously amplified. Therefore, as one of the embodiments, a pulse activation function is used to perform a second binarization process on the first feature map that highlights the spatial and spectral features to suppress noise in the first feature map. The first feature map after the above correction may have a higher dimension. In order to ensure that the subsequent steps can be processed with lower computing resources, this step reduces the dimension of the first feature map through a multi-scale pooling operation to ensure that global information is retained (global information is a different concept from useful information and useless information) and reduce the computational burden; wherein, the first feature map after dimensionality reduction is obtained by the following calculation:

[0092] ,

[0093] In the formula, is the first feature map after pooling, that is, the first feature map after dimensionality reduction, and are the coordinates, is the pooling window size, is the counting number of the pooled traversal elements, is the input feature map of the pooling operation, that is, the first feature map after the second binarization process. The remaining operations related to multi-scale pooling to reduce the dimension of the feature map in this embodiment can refer to this formula for processing. Finally, the second feature map is obtained based on the first feature map after dimensionality reduction.

[0094] For hyperspectral images with different information depth and complexity, the corresponding second feature maps should be subjected to different times of refocusing convolution processing. As one embodiment, each refocusing convolution process is as follows:

[0095] The eigenvalue after refocusing convolution is calculated by the following formula:

[0096] ,

[0097] In the formula, is the eigenvalue after refocusing convolution, is the weight matrix of the refocusing convolution kernel, is the second feature map of the input, and are the coordinates, is a bias term used to adjust the output offset in the refocused convolution operation.

[0098] Refocusing convolution aims to enhance local features in the image by adaptively adjusting the response strength of the convolution kernel. The second feature map is reorganized according to the eigenvalues ​​after the refocusing convolution, and learnable parameters are introduced to dynamically adjust the eigenvalues. This embodiment uses learnable parameters to dynamically adjust the eigenvalues ​​for channel adjustment. By adjusting the channel to control the convolution kernel output of a specific area to enlarge or reduce it, the response of the specific channel is more significant, thereby controlling the details in the image of interest; the dynamically adjusted eigenvalue is obtained by the following calculation:

[0099] ,

[0100] In the formula, For the The eigenvalues ​​of the channels are dynamically adjusted. represents the channel index, For the The learnable parameters of the channel are used to control the The response strength of the channel, is the first The characteristic value of the channel;

[0101] The eigenvalues ​​of each channel after dynamic adjustment are reorganized to obtain a second feature map that completes a single refocusing convolution.

[0102] Learnable parameters are introduced to dynamically adjust feature values. This mechanism is similar to weighting local features, helping the model better identify important detail features in complex image scenes.

[0103] As one of the embodiments, after each refocusing convolution is completed, the second feature map is binarized using an impulse activation function to amplify the feature expression in the local area, ensuring that important local features are effectively amplified and expressed.

[0104] There is still room for optimization for the second feature map that has been amplified by feature expression. Advanced attention mechanisms and global feature optimization can be used to integrate and optimize multi-level features and further reduce their dimensionality. The process is mainly as follows: the second feature map that has been amplified by feature expression is updated using an adaptive attention mechanism. The adaptive attention mechanism calculates the similarity between the query and the key based on the Hamming distance, and optimizes the key features in the second feature map based on the similarity between the query and the key; the key features in the second feature map are optimized based on the similarity between the query and the key, aiming to calculate the interaction between high-level features (or key features) based on the attention mechanism of the Hamming distance, optimize the key features, and filter out useless information mixed in the high-level features.

[0105] The optimized second feature map can be used again to perform deep convolution processing on the multi-level features using refocusing convolution to integrate and optimize the deep features in the image.

[0106] After the above processing, the dimension of the second feature map is relatively high. Therefore, the dimension of the second feature map after refocusing convolution is reduced through multi-scale pooling operation. The second feature map after dimensionality reduction is obtained by the following calculation formula:

[0107] ,

[0108] In the formula, is the second feature map after pooling, that is, the second feature map after dimensionality reduction, is the pooling window size, is the counting number of the pooled traversal elements, It is the input second feature map of the pooling operation, that is, the second feature map after the refocusing convolution update.

[0109] Through the global pooling operation, the feature dimension is reduced while maintaining the global key information of the image.

[0110] As mentioned in the first embodiment, the features in the third feature map are linearly combined, and the features after the linear combination are identified to obtain the classification result. In this embodiment, the method includes:

[0111] The linearly combined features are obtained according to the following formula:

[0112] ,

[0113] In the formula, is the weight matrix of the fully connected layer in the adaptive attention mechanism, is the feature map after the multi-scale pooling operation, that is, the third feature map, is the bias term, is the feature after linear combination.

[0114] In addition, most convolution processes are unified into the same or similar KAN convolution, which improves the utilization of the same module and reduces storage consumption and software optimization difficulty.

[0115] This embodiment mentions KAN convolution operation many times. As one of the embodiments, refer to Figure 3 , the related KAN convolution operation formula in this embodiment is:

[0116] ,

[0117] In the formula, and are the coordinates, is the spectral dimension, The coordinates are and , the spectral dimension is The convolution result at, for example, the first feature map after the KAN convolution operation, is the weight matrix of the KAN convolution kernel, and are the coordinates of the convolution kernel, represents the sliding operation of the KAN convolution kernel, is the spectral dimension of the convolution kernel, are coordinates and the spectral dimensions are The pixel value at ;

[0118] The first convolution kernel of the KAN convolution operation uses the Gaussian radial basis function (GaussianRBF) convolution kernel, which measures the distance between the input data point and the center of the convolution kernel and performs a weighted sum of these data to achieve convolution. This kernel function is particularly suitable for continuous spectral data unique to hyperspectral images, and can effectively extract spectral and spatial features.

[0119] In the description of this embodiment, the KAN convolution is preferably a fast KAN convolution.

[0120] For the combination of KAN convolution and spiking neural network, fast KAN convolution is used to improve the efficiency of convolution operation, and the pulse activation function in the spiking neural network is used to further optimize the accuracy of feature extraction. KAN convolution can efficiently extract spatial and spectral information in hyperspectral images with limited computing resources, and improve the focus of this method on important features through the pulse activation function.

[0121] This implementation also uses the pulse activation function to binarize the feature map multiple times. For example, the pulse activation function is used to binarize the first feature map. As one embodiment, the binarized feature map is obtained by the following calculation:

[0122] ,

[0123] In the formula, The coordinates are and , the spectral dimension is The pulse signal at, for example, the first feature map after binarization, express The following inequality is the execution condition. is the activation threshold, which is used to decide whether to generate a pulse signal.

[0124] Traditional attention mechanisms usually rely on continuous value similarity calculations such as dot product or Euclidean distance to calculate the similarity between the query vector (Q) and the key vector (K). However, when applied in spiking neural networks, this mechanism exposes the following problems:

[0125] First, continuous value similarity is not applicable to pulse signals: the signals in pulse neural networks are binary 0 or 1, and traditional continuous value similarity calculation cannot effectively process such discrete signals, resulting in a decrease in the accuracy of similarity calculation.

[0126] Secondly, the computational complexity is high: when processing pulse signals, the traditional attention mechanism still uses complex continuous value calculations, which requires a lot of computing resources, especially in high-dimensional data, seriously affecting processing efficiency.

[0127] Finally, the feature expression ability is insufficient: due to the discrete characteristics of pulse signals, traditional mechanisms have difficulty capturing subtle similarities between features, which affects the classification performance, especially in the complex feature extraction tasks of hyperspectral images.

[0128] To overcome the above limitations, refer to Figure 4 ,This embodiment proposes a binary Hamming kernel attention mechanism, which is specially designed for processing pulse signals, and can accurately measure the similarity between the query vector and the key vector and optimize the feature representation by calculating the Hamming distance between the pulse signals; Figure 4 In , softmax is an exemplary activation function. Specifically, the Hamming kernel attention mechanism has the following advantages:

[0129] First, adapt the similarity calculation of pulse signals: Different from the traditional continuous value similarity calculation, the Hamming distance calculation is based on the XOR operation of binary signals and is more suitable for processing pulse signals. By calculating the Hamming distance between pulse signals, this mechanism can effectively measure the discrete similarity between the query vector (Q) and the key vector (K), thereby enhancing the feature representation ability of the pulse neural network.

[0130] Secondly, the computational complexity is low: The calculation of the Hamming distance is very efficient, avoiding the complex product and normalization operations in the continuous value attention mechanism, significantly reducing the computational cost of the model, especially when processing high-dimensional pulse signals.

[0131] Finally, strengthen key features: Through the similarity calculation of binary Hamming distance, the attention mechanism can more accurately identify and strengthen the key features in the image, which greatly improves the classification performance of the model under high-dimensional and complex data.

[0132] In addition, by combining the innovative design of KAN convolution and adaptive convolution, this embodiment effectively addresses the complexity of hyperspectral images and further improves the accuracy and efficiency of classification.

[0133] Therefore, this embodiment uses the adaptive attention mechanism based on the Hamming distance to calculate the similarity to update the feature map multiple times. For example, the mechanism is used to update the first feature map. As one embodiment, the general use process of the adaptive attention mechanism in this embodiment includes:

[0134] Performing linear transformation on the first feature graph to obtain a query vector, a key vector, and a value vector;

[0135] The similarity between the query vector and the key vector is calculated by the Hamming distance. The similarity is calculated by the following formula:

[0136] ,

[0137] In the formula, The Hamming distance is a calculation operation that measures the number of different bits in two binary vectors (i.e., the query vector and the key vector) (i.e., the "XOR" operation between the corresponding elements of the two vector positions). is the query vector, is the key vector, is the count number of the query vector, is the counting number of the key vector, For the The query vector elements, For the The first key vector elements, : indicates the query vector With key vector The Hamming distance between them is the similarity. Represents the XOR operation of a binary vector. When the calculation result is 1, it means that the two elements are different. When the calculation result is 0, it means that the two elements are the same. is the element length of the vector, is the counting sequence number of the element;

[0138] Based on the similarity between the query vector and the key vector, the attention weight is generated. The attention weight is calculated by the following formula:

[0139] ,

[0140] In the formula, is the counting number of similarity, For the query vector and the The attention weights of the key vectors; it is worth noting that Used to attenuate the Hamming distance. The smaller the distance, the greater the weight.

[0141] A new feature vector is generated according to the attention weight and the value vector, and the first feature map is obtained according to the new feature vector. The feature vector is obtained by the following calculation:

[0142] ,

[0143] In the formula, represents the updated feature vector, represents the count sequence of the value vector, is the key vector The corresponding value vector.

[0144] The remaining feature maps can be processed by referring to the above process.

[0145] It is worth noting that in the description of this embodiment, the spiking neural network combines multi-step pulse signals to perform layer-by-layer feature extraction and classification, and uses a pooling layer to downsample the features to improve classification accuracy and efficiency. After the spiking neural network extracts key features from the pulse signal through the feature extraction layer, it combines the results of the attention mechanism for efficient classification.

[0146] In order to optimize the feature similarity calculation in hyperspectral images, this embodiment introduces a Hamming distance attention mechanism based on binary pulse signals. Different from the traditional continuous value similarity calculation, the Hamming distance can better handle pulse signals and generate attention weights by calculating the binary similarity between the query vector and the key vector.

[0147] This embodiment uses refocusing convolution to adaptively adjust local features and dynamically adjust the response strength of the convolution kernel to different areas to ensure the effective extraction of complex detail features. Through the adaptive focusing mechanism, the model can focus on local complex areas, thereby improving the depth and accuracy of feature extraction.

[0148] This embodiment significantly improves the feature extraction capability of hyperspectral images by introducing a spiking neural network (SNN) combined with the innovative design of fast Kolmogorov-Arnold (KAN) convolution and refocusing convolution. The spiking neural network simulates the impulse response characteristics of biological neurons, can efficiently process spatiotemporal data, and dynamically adjust the model's response to complex features to ensure high classification accuracy. The fast KAN convolution is responsible for efficiently extracting spatial and spectral features, while the refocusing convolution enhances the capture and optimization of local details through an adaptive mechanism. At the same time, the binary pulse coding generates a high-fidelity pulse sequence by automatically encoding each pixel, greatly improving the efficiency and processing accuracy of information transmission, especially when training samples are scarce, and can still maintain efficient learning and accurate classification performance.

[0149] This embodiment verifies the performance superiority of the hyperspectral image classification method provided in this embodiment through experiments, and selects Houston University 2013 as the hyperspectral image dataset. Figure 5 This is a real ground object image from the hyperspectral image dataset of Houston University 2013. Figure 6 for Figure 5 The pseudo-color image of the hyperspectral image is 349×1905 pixels in size, which contains a large number of background pixels. There are only 15029 pixels containing ground objects, which contain 15 types of ground objects, including trees, asphalt roads, bricks, pastures, etc. 100 samples of each type are selected for training, and the rest are used for testing, so the training samples account for nearly 10% of the total samples.

[0150] The hyperspectral image classification method provided in the embodiment is named KSANet. For comparison, SVM, 2D-CNN, 3D-CNN, SpectralFormer and SNN in the prior art are used as similar comparison items. The comparison content is the accuracy of nine types of land object categories numbered from 1 to 15, and also includes the overall accuracy, average accuracy and KaKappa coefficient obtained according to the accuracy of the nine types of land object categories from 1 to 15, and the following "Accuracy Comparison Table of Different Image Classification Methods" is obtained.

[0151] Table 1 Comparison of the accuracy of different image classification methods

[0152]

[0153] From Table 1, it can be seen that the overall accuracy of KSANet provided in this embodiment reaches 98.96%, which is significantly higher than other methods, such as 93.19% of SVM and 97.23% of 2D-CNN. This shows that KSANet has higher accuracy when processing hyperspectral image classification tasks. In particular, compared with 3D-CNN, KSANet shows a significant improvement (91.82% vs 98.96%), which shows the advantage of KSANet in extracting spatial-spectral features. In multiple categories, KSANet achieved 100% classification accuracy, such as in Class 5 (Bitumen), Class 6 (Trees) and Class 15 (Running Track). This excellent classification performance is particularly reflected in complex and fuzzy boundary objects, showing KSANet's high sensitivity to detail features. In contrast, 3D-CNN's performance in multiple categories is significantly lower than KSANet, such as Class 2 (90.83%) and Class 8 (79.94%), showing the limitations of traditional convolutional neural networks on high-dimensional data and complex objects.

[0154] Through this experiment, it can be concluded that the classification accuracy of the hyperspectral image classification method provided by this embodiment is significantly higher than that of competing products. On the basis of improved classification accuracy, the classification efficiency is also improved. It makes good use of the complex spatial relationships and texture information in the hyperspectral image to perform efficient and accurate image recognition.

[0155] To visualize the classification results, Figure 7 and Figure 8 , respectively, are the classification results shown by SVM and 3D-CNN. Fig. 9 The classification result diagram of KSANet is shown. Figure 7 , Figure 8 and Fig. 9 The classification results shown can clearly verify the conclusions of this experiment with Table 1.

[0156] This example introduces a pulse neural network, a binary Hamming kernel attention mechanism, a refocusing convolution, and a fast KAN convolution to propose an effective hyperspectral image classification method, which can significantly improve the accuracy and efficiency of classification under the conditions of complex data and fewer samples. This innovative method not only performs well in remote sensing image analysis, but can also be applied to a variety of practical scenarios such as precision agriculture and environmental monitoring.

[0157] Embodiment three:

[0158] This embodiment provides a computer device, including a processor and a memory connected to the processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the hyperspectral image classification method provided in Embodiment 1 or 2 are performed.

[0159] The computer device may be a server or an electronic terminal. As one embodiment, refer to Fig.10, the computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data acquired and generated in the hyperspectral image classification method. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the hyperspectral image classification method provided in embodiment one or two is implemented.

[0160] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0161] The computer device provided in this embodiment has the same technical effects as those in Embodiment 1 or 2, which will not be described in detail here.

[0162] Embodiment 4:

[0163] This embodiment provides a computer program product on which a computer program is stored, and when the program is executed by a processor, the steps of the hyperspectral image classification method provided in Embodiment 1 or Embodiment 2 are implemented. The computer program product provided in this embodiment can be transmitted, distributed and downloaded in the form of a signal via the Internet.

[0164] The computer program product provided in this embodiment has the same technical effects as those of Embodiment 1 or 2, and will not be described in detail here.

[0165] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0166] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0167] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0168] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0169] In the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0170] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A hyperspectral image classification method, characterized in that: include, Acquire a hyperspectral image, and extract a first feature map from the hyperspectral image; Perform KAN convolution operation on the first feature map to capture spatial and spectral features, updating the first feature map using an adaptive attention mechanism to amplify spatial and spectral features in the first feature map to obtain a second feature map, wherein the adaptive attention mechanism calculates the similarity between the query and the key based on the Hamming distance; Performing a refocus convolution on the second feature map, adjusting the response strength of a local area in the second feature map, and amplifying the feature expression in the local area according to the response strength of each local area to obtain a third feature map; Linearly combining the features in the third feature map, identifying the linearly combined features to obtain a classification result; The method of updating the first feature map using an adaptive attention mechanism to amplify the spatial and spectral features in the first feature map to obtain a second feature map further includes obtaining the first feature map after binarization processing, and performing a KAN convolution operation on the first feature map after binarization processing to obtain a first feature map that highlights the spatial and spectral features; The first feature map highlighting the spatial and spectral features is binarized for the second time using an impulse activation function to suppress noise in the first feature map. The dimension of the first feature map after the second binarization is reduced through a multi-scale pooling operation.

2. The hyperspectral image classification method according to claim 1, characterized in that: The extracting of the first feature map from the hyperspectral image includes: using a filter to scan each local area in the hyperspectral image to extract key feature positions; performing a KAN convolution operation on the hyperspectral image for the key feature positions to obtain a first feature map; and using an impulse activation function to perform a binarization process on the first feature map; The method of updating the first feature map using the adaptive attention mechanism to amplify the spatial and spectral features in the first feature map to obtain the second feature map also includes binarizing the first feature map processed by the adaptive attention mechanism using a pulse activation function; wherein the first feature map after dimensionality reduction is obtained by the following calculation formula: ， In the formula, is the first feature map after dimensionality reduction, and are the coordinates, is the pooling window size, is the counting number of the pooled traversal elements, is the first feature map after the second binarization process; The second feature map is obtained according to the first feature map after dimensionality reduction.

3. The hyperspectral image classification method according to claim 1, characterized in that: The refocusing convolution is performed on the second feature map, the response strength of the local area in the second feature map is adjusted, and the feature expression in the local area is amplified according to the response strength of each local area to obtain a third feature map, including: Performing multiple refocusing convolutions on the second feature map, and after each refocusing convolution, using an impulse activation function to perform binarization processing on the second feature map to amplify the feature expression in the local area; The process of each refocusing convolution is as follows: The eigenvalue after refocusing convolution is calculated by the following formula: ， In the formula, is the eigenvalue after refocusing convolution, is the weight matrix of the refocusing convolution kernel, is the second feature map of the input, and are the coordinates, is a bias term used to adjust the output offset in the refocusing convolution operation; The learnable parameters are introduced to dynamically adjust the eigenvalues ​​after refocusing convolution. The dynamically adjusted eigenvalues ​​are calculated by the following formula: ， In the formula, For the The eigenvalues ​​of the channels are dynamically adjusted. represents the channel index, For the The learnable parameters of the channel are used to control the The response strength of the channel, is the first The characteristic value of the channel; The eigenvalues ​​of each channel after dynamic adjustment are reorganized to obtain a second feature map that completes a single refocusing convolution.

4. The hyperspectral image classification method according to claim 3, characterized in that: The second feature graph after feature expression amplification is further updated using an adaptive attention mechanism, wherein the adaptive attention mechanism calculates the similarity between the query and the key based on the Hamming distance, and optimizes the key features in the second feature graph according to the similarity between the query and the key; The second feature map updated using the adaptive attention mechanism is then updated with a refocused convolution; The dimension of the second feature map after refocusing convolution is reduced by multi-scale pooling operation, and the second feature map after dimensionality reduction is obtained by the following calculation: ， In the formula, is the second feature map after dimensionality reduction, is the pooling window size, is the counting number of the pooled traversal elements, is the second feature map after refocusing convolution update; The third feature map is obtained according to the second feature map after dimensionality reduction.

5. The hyperspectral image classification method according to claim 4, characterized in that: The linearly combining the features in the third feature map and identifying the linearly combined features to obtain a classification result includes: The linearly combined features are obtained according to the following formula: ， In the formula, is the weight matrix of the fully connected layer in the adaptive attention mechanism, is the third characteristic graph, is the bias term, is the feature after linear combination.

6. The hyperspectral image classification method according to claim 1, characterized in that: The KAN convolution operation formula is as follows: ， In the formula, and are the coordinates, is the spectral dimension, is the first feature map after the KAN convolution operation, is the weight matrix of the KAN convolution kernel, and are the coordinates of the convolution kernel, represents the sliding operation of the KAN convolution kernel, is the spectral dimension of the convolution kernel, are coordinates and the spectral dimensions are The pixel value at ; The first convolution kernel of the KAN convolution operation adopts a Gaussian radial basis function convolution kernel.

7. The hyperspectral image classification method according to claim 2, characterized in that: The processing formula for binarizing the first feature map using the impulse activation function is as follows: ， In the formula, and are the coordinates, is the spectral dimension, is the first feature map after binarization, express The following inequality is the execution condition. is the activation threshold, is the first feature map.

8. The hyperspectral image classification method according to claim 1, characterized in that: The updating of the first feature map using an adaptive attention mechanism includes: Performing linear transformation on the first feature graph to obtain a query vector, a key vector, and a value vector; The similarity between the query vector and the key vector is calculated by the Hamming distance, and the similarity is obtained by the following calculation: ， In the formula, is the calculation operation of the Hamming distance, is the query vector, is the key vector, is the count number of the query vector, is the counting number of the key vector, Represents the query vector With key vector The similarities between For the The query vector elements, For the The first key vector elements, Represents the XOR operation of a binary vector. When the calculation result is 1, it means that the two elements are different. When the calculation result is 0, it means that the two elements are the same. is the element length of the vector, is the counting sequence number of the element; Based on the similarity between the query vector and the key vector, an attention weight is generated, and the attention weight is obtained by the following calculation: ， In the formula, is the counting number of similarity, For the query vector and the The attention weights of the key vectors; A new feature vector is generated according to the attention weight and the value vector, and an updated first feature map is obtained according to the new feature vector. The feature vector is obtained by the following calculation: ， In the formula, represents the updated feature vector, represents the count sequence of the value vector, is the key vector The corresponding value vector.

9. A computer device, characterized in that: The method comprises a processor and a memory connected to the processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the hyperspectral image classification method according to any one of claims 1 to 8 are executed.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the hyperspectral image classification method according to any one of claims 1 to 8 are implemented.

Images