A hyperspectral image classification method and system

Abstract

The present invention discloses a hyperspectral image classification method and system. The method includes: acquiring a hyperspectral image; preprocessing the hyperspectral image to obtain a preprocessed image; extracting spectral, spatial, and frequency domain features from the preprocessed image; performing superpixel segmentation on the preprocessed image, treating each superpixel of the image as a node, constructing a graph structure, and extracting global features of the graph structure; combining the global and local features of the graph structure to obtain fusion features of the hyperspectral image; and determining a classification result of the hyperspectral image based on the fusion features of the hyperspectral image. This method improves the classification and recognition effect of hyperspectral images.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral image classification, and in particular to a hyperspectral image classification method and system. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of deep learning technology, hyperspectral image analysis has been widely used in fields such as remote sensing, agricultural monitoring, and environmental protection. Hyperspectral images contain rich spectral information, with each pixel containing spectral data from multiple continuous bands. This high spectral dimension provides strong support for accurate classification. However, due to the complexity of high-dimensional data and the scarcity of labeled samples, the classification task faces significant challenges. Especially under small sample conditions, the classification accuracy and robustness of existing methods are often insufficient. Early hyperspectral image classification methods relied on machine learning algorithms such as support vector machines (SVM), K-nearest neighbor (KNN), and random forests (RF). These methods use artificially designed features for classification. Although they perform well in certain applications, they struggle to capture the deep information in hyperspectral images and primarily focus on utilizing spectral features, ignoring the spatial correlation between pixels. This results in low classification accuracy and susceptibility to noise, especially in complex scenes.

[0004] In recent years, graph neural networks (GNNs) have been gradually introduced to hyperspectral image classification. Graph convolutional networks (GCNs) and GATs capture global features through graph structures and excel at processing the complex spatial relationships in hyperspectral images. GATs, in particular, dynamically adjust the weights between nodes through an attention mechanism, enhancing feature extraction capabilities in small-sample scenarios. However, existing GNN methods rely on a fixed adjacency matrix and are unable to dynamically adjust node relationships based on data changes. Furthermore, most methods focus solely on temporal features and fail to fully exploit frequency domain features, which are crucial for capturing periodic variations and global patterns in images. Furthermore, with the advancement of small-sample learning techniques, metric learning methods such as Siamese networks have been gradually applied to hyperspectral image classification. Siamese networks learn the similarities between pairs of samples and can achieve effective classification with limited labeled data. However, most current Siamese networks only utilize spectral and spatial features and fail to fully integrate frequency domain information for comprehensive feature extraction, limiting further improvement in their classification performance. The field of hyperspectral image classification urgently needs a solution that can fully leverage the synergistic effects of spectral, spatial, and frequency domain features to enhance classification accuracy and robustness in small-sample conditions. Summary of the Invention

[0005] In order to solve the above problems, the present invention proposes a hyperspectral image classification method and system, which improves the accuracy of hyperspectral image classification.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] First, a hyperspectral image classification method is proposed, including:

[0008] Acquire hyperspectral images;

[0009] Preprocessing the hyperspectral image to obtain a preprocessed image;

[0010] Extracting spectral spatial features and frequency domain features from the preprocessed image;

[0011] The spectral spatial features are fused with the frequency domain features extracted from the preprocessed image to obtain the local features of the hyperspectral image;

[0012] Perform superpixel segmentation on the preprocessed image, treat each superpixel of the image as a node, construct a graph structure, and extract the global features of the graph structure;

[0013] The global features of the graph structure and the local features of the hyperspectral image are combined to obtain the fusion features of the hyperspectral image;

[0014] According to the fusion characteristics of the hyperspectral image, the classification result of the hyperspectral image is determined.

[0015] Furthermore, the process of performing superpixel segmentation on the preprocessed image includes:

[0016] Obtain the spectral and spatial characteristics of each pixel in the preprocessed image;

[0017] Determine the distance between pixels in the preprocessed image based on the spectral characteristics and spatial characteristics of each pixel in the preprocessed image;

[0018] The preprocessed image is segmented into superpixels according to the distance between pixels in the preprocessed image.

[0019] Furthermore, the average value of all pixel features in the superpixel is used as the node feature of the node corresponding to the superpixel, wherein each pixel feature includes the spectral feature and spatial feature of the pixel.

[0020] Furthermore, the similarity between nodes is calculated based on the node features;

[0021] Build an edge between two nodes whose similarity is greater than a set value.

[0022] Furthermore, the graph attention network is used to extract the global features of the graph structure.

[0023] Furthermore, spectral features are extracted from the preprocessed image;

[0024] The spatial features are extracted from the spectral features to obtain the spectral spatial features.

[0025] Secondly, a hyperspectral image classification system is proposed, including:

[0026] An image acquisition module, used for acquiring hyperspectral images;

[0027] A data preprocessing module is used to preprocess the hyperspectral image to obtain a preprocessed image;

[0028] The spectral, spatial and frequency domain feature extraction module is used to extract spectral spatial features and frequency domain features from the preprocessed image; the spectral spatial features are fused with the frequency domain features extracted from the preprocessed image to obtain the local features of the hyperspectral image;

[0029] The global feature extraction module is used to perform superpixel segmentation on the preprocessed image, treating each superpixel of the image as a node, constructing a graph structure, and extracting the global features of the graph structure;

[0030] The feature fusion module is used to combine the global features of the graph structure and the local features of the hyperspectral image to obtain the fusion features of the hyperspectral image;

[0031] The classification module is used to determine the classification results of the hyperspectral image based on the fusion features of the hyperspectral image.

[0032] In a third aspect, a computer device is provided, comprising:

[0033] a processor adapted to execute a computer program;

[0034] A computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the hyperspectral image classification method proposed in the first aspect is implemented.

[0035] In a fourth aspect, a computer-readable storage medium is proposed, wherein the computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing a hyperspectral image classification method proposed in the first aspect.

[0036] In a fifth aspect, a computer program product is proposed, which includes a computer program. When the computer program is executed by a processor, it implements the hyperspectral image classification method proposed in the first aspect.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The present invention provides a hyperspectral image classification method and system. The method first preprocesses the hyperspectral image, then performs superpixel segmentation on the preprocessed image to reduce data complexity, construct a graph structure for the image, and utilize a graph attention network to extract global features of the graph structure while reducing redundant features. Furthermore, the present invention extracts spectral-spatial features and frequency-domain features from the preprocessed image, then fuses the spectral-spatial and frequency-domain features to form local features. These local features contain the image's spectral, spatial, and frequency-domain information, and can capture global patterns and periodic variations in the image. This method can effectively improve the robustness and accuracy of classification when processing complex scenes with periodic variations. Finally, the global features are fused with the local features to generate fused features, which are then used to determine the classification results of the hyperspectral image. Because the fused features contain more feature information about the image, the accuracy of hyperspectral image recognition can be improved.

[0039] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings in the specification, which constitute a part of this application, are used to provide further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute improper limitations on this application.

[0041] Figure 1 A flow chart of a hyperspectral image classification method disclosed in an embodiment;

[0042] Figure 2 A block diagram of a local feature extraction network structure in a hyperspectral image classification method disclosed in an embodiment;

[0043] Figure 3 This is the overall framework of existing hyperspectral image classification and recognition methods;

[0044] Figure 4 This is a framework diagram of a hyperspectral image classification system disclosed in an embodiment. DETAILED DESCRIPTION

[0045] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0046] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0047] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0048] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0049] Example 1

[0050] Explanation of terms involved in this embodiment:

[0051] Hyperspectral Image (HSI): A remote sensing imaging technique that captures the spectral information of an object in multiple continuous narrow bands. Each pixel contains a spectral curve representing the reflectance or radiance at different wavelengths.

[0052] Graph Attention Network (GAT): A graph neural network that performs convolution operations on graph-structured data. GAT uses an attention mechanism to dynamically adjust the weight distribution between nodes, allowing the network to process complex graph data more efficiently.

[0053] Siamese Network: A neural network architecture for metric learning that processes pairs of input samples through two weighted sub-networks and calculates their feature similarity. This network is commonly used for few-shot learning and similarity tasks.

[0054] Few-Shot Learning (FSL): A machine learning method that aims to use a very small number of labeled examples for training and show good generalization ability on new data.

[0055] Frequency-Domain Features: Converting an image from the time domain to the frequency domain through Fourier transform allows us to capture the global patterns and periodic variations of the image. This is particularly important for processing image categories with high-frequency information.

[0056] Superpixel: Adjacent pixels in a hyperspectral image are grouped using a clustering algorithm. Each superpixel represents a set of pixels in a local area, which is used to reduce computational complexity.

[0057] With the rapid development of deep learning technology, hyperspectral image analysis has been widely used in fields such as remote sensing, agricultural monitoring, and environmental protection. Hyperspectral images contain rich spectral information, with each pixel containing spectral data from multiple continuous bands. This high spectral dimension provides strong support for accurate classification. However, due to the complexity of high-dimensional data and the scarcity of labeled samples, the classification task faces significant challenges. Especially under small sample conditions, the classification accuracy and robustness of existing methods are often insufficient. Early hyperspectral image classification methods relied on machine learning algorithms such as support vector machines (SVM), K-nearest neighbor (KNN), and random forests (RF). These methods use artificially designed features for classification. Although they perform well in certain applications, they struggle to capture the deep information in hyperspectral images and primarily focus on utilizing spectral features, ignoring the spatial correlation between pixels. This results in low classification accuracy and susceptibility to noise, especially in complex scenes.

[0058] The introduction of convolutional neural networks (CNNs) has significantly improved the performance of hyperspectral image classification. By automatically extracting spectral and spatial features, CNNs reduce the reliance on handcrafted features. However, CNN applications in hyperspectral imagery still face two major challenges: first, the scarcity of annotated hyperspectral image data makes CNNs prone to overfitting when trained with small sample sizes; second, CNNs have limited processing capabilities for non-Euclidean structures (such as graph-structured data), making it difficult to fully capture the global contextual information in hyperspectral images. To address these issues, graph neural networks (GNNs) have been gradually introduced in recent years for hyperspectral image classification. Graph convolutional networks (GCNs) and GATs capture global features through graph structures and excel at processing the complex spatial relationships in hyperspectral images. In particular, GATs dynamically adjust the weights between nodes through an attention mechanism, enhancing feature extraction capabilities in small sample sizes. However, existing GNN methods rely on a fixed adjacency matrix and are unable to dynamically adjust node relationships based on data changes. Furthermore, most methods focus solely on temporal features and fail to fully utilize frequency domain features, which are crucial for capturing periodic changes and global patterns in images. Furthermore, with the development of small-sample learning techniques, metric learning methods such as twin networks are gradually being applied to hyperspectral image classification. By learning the similarities between pairs of samples, twin networks can achieve effective classification with limited labeled data. However, most current twin networks only utilize spectral and spatial features, failing to fully integrate frequency domain information for comprehensive feature extraction, which limits further improvement in their classification performance. Currently, the field of hyperspectral image classification urgently needs a solution that can fully utilize the synergistic effects of spectral, spatial, and frequency domain features under small sample conditions to enhance classification accuracy and robustness.

[0059] In the research based on hyperspectral image classification, many methods currently use a single-branch structure. These methods usually focus on either extracting global features or capturing local pixel relationships, and lack the effective combination of global and local features. This single feature extraction method limits the performance of the model when processing hyperspectral data with complex spectral-spatial information, especially when the amount of sample data is small, the classification accuracy is significantly limited. In order to solve the problem that global and local features cannot be effectively combined, a method based on a cascade structure has been proposed recently, and has achieved good results in hyperspectral image classification. This method first extracts global features through a superpixel-based graph convolutional network, and then further extracts local detail features through a pixel-based convolutional neural network, such as Figure 3 As shown. Specifically, the method designs a multi-layer structure in the GCN module and adopts DropEdge technology and residual connection to alleviate the problems of over-smoothing and overfitting, so that GCN can extract global features at the superpixel level more deeply. After completing the global feature extraction, the CNN module applies a hybrid attention mechanism to the pixel-level data, combining spatial coordinate information and channel attention to extract detailed spectral-spatial features. Through this cascade structure, the model can first capture global context information and then accurately locate the relationship between local pixels, thereby significantly improving the classification performance, especially when the sample size is small. However, although this method is better than most current classification methods in terms of overall performance, it still has some limitations. Although GCN can process graph-structured data, its ability to capture global structural information is slightly insufficient compared to the graph attention network. In addition, the CNN branch mainly focuses on spatial and spectral features and fails to make full use of frequency domain features, which may affect the classification effect when processing complex spectral-spatial data. Therefore, in response to these shortcomings, the present invention can not only combine local and global features, but also significantly improve the classification performance by introducing frequency domain information, and can effectively cope with the challenge of limited sample size.

[0060] In summary, the shortcomings of the existing technology are:

[0061] (1) Single feature extraction method: Existing methods usually focus only on spectral or spatial features and fail to effectively combine the two. This results in the inability to fully utilize the multidimensional information in hyperspectral images, thus affecting the accuracy and robustness of classification.

[0062] (2) Insufficient utilization of frequency domain information: Although hyperspectral images contain a large amount of frequency domain information, existing methods still rely mainly on time domain features and cannot effectively use frequency domain features to capture global patterns and periodic changes, resulting in poor performance of the model in complex scenarios.

[0063] (3) Poor classification performance with small samples: Due to the scarcity of labeled data for hyperspectral images, existing deep learning models are prone to overfitting under small sample conditions, and it is difficult to maintain a high level of classification performance.

[0064] This embodiment realizes the coordinated use of spectral, spatial and frequency domain information in hyperspectral images by designing a dual-branch architecture that combines a graph attention network and a twin network, so as to improve the accuracy and robustness of hyperspectral image classification. At the same time, the present invention introduces frequency domain feature extraction to capture global patterns and periodic changes, thereby reducing data redundancy and enhancing the generalization ability of the model in complex scenarios. Through the metric learning mechanism of the twin network, this embodiment improves the classification performance under small sample conditions and avoids the overfitting problem. In addition, the introduction of the graph attention mechanism allows the weights between nodes to be dynamically adjusted, thereby improving the expressive ability of the model when processing complex hyperspectral data and overcoming the limitations of traditional graph neural networks.

[0065] like Figure 1-Figure 2 As shown, the present embodiment discloses a hyperspectral image classification method, including:

[0066] S1: Acquire hyperspectral images.

[0067] The size of the acquired hyperspectral image is H×W×C, where H is the height, W is the width, and C is the number of spectral channels.

[0068] S2: Preprocess the hyperspectral image to obtain a preprocessed image.

[0069] The preprocessing of hyperspectral images in this embodiment includes noise reduction and dimensionality reduction to remove noise and redundant information in the original data, reduce the complexity of high-dimensional data, and enhance the ability to distinguish spectral features in classification tasks. Specifically: First, batch normalization (BN) is performed on the hyperspectral image to reduce the noise and redundant information of the data. This step enhances the training stability and convergence of the model by standardizing the mean and variance of the data; then, convolution calculation is performed on the batch normalized data X to reduce the data dimension and obtain the preprocessed image X. conv .

[0070] Preferably, two 2D convolution layers with 1×1 convolution kernels are used to reduce the dimensionality of the batch normalized data, and combined with the LeakyReLU activation function to obtain the preprocessed image, further reduce the noise and extract more discriminative spectral features. The formula is X conv =LeakyReLU(Conv2D(X)), the processed data is used for subsequent image superpixel segmentation and feature extraction.

[0071] This embodiment preprocesses the hyperspectral image and can compress the hyperspectral image data from hundreds of bands to a few principal components, which helps to enhance the robustness and discriminability of the spectral characteristics of the hyperspectral image, thereby reducing redundant information and improving the efficiency of subsequent processing.

[0072] S3: Extract spectral spatial features and frequency domain features from preprocessed images;

[0073] The spectral spatial features are fused with the frequency domain features extracted from the preprocessed image to obtain the local features of the hyperspectral image;

[0074] The preprocessed image is segmented into superpixels, each superpixel of the image is regarded as a node, a graph structure is constructed, and the global features of the graph structure are extracted.

[0075] The process of acquiring the spectral spatial features in this embodiment includes:

[0076] Extract spectral features from the preprocessed image;

[0077] The spatial features are extracted from the spectral features to obtain the spectral spatial features.

[0078] Preferably, a convolutional neural network is used to extract spectral features of the preprocessed image, and spatial features are extracted from the spectral features;

[0079] The preprocessed image is converted into frequency domain data using fast Fourier transform, and the frequency domain features of the preprocessed image are extracted from the frequency domain data.

[0080] This embodiment uses a local feature extraction network to extract local features of the hyperspectral image from the preprocessed image, such as Figure 2 As shown in the figure, the local feature extraction network is a twin network architecture based on convolutional neural network (CNN), which extracts spectral and spatial features Y from the preprocessed image. spatial-spectral and frequency domain features Y frequency , and the spectral spatial feature Y spatial-spectral and frequency domain features Y frequency Perform weighted fusion to obtain the local features F of the hyperspectral image local By extracting frequency domain features from preprocessed images, the model can capture high-frequency details in the image, making it particularly suitable for classifying complex landforms. Through point convolution and depthwise convolution, the model can capture local details in the image, making it particularly suitable for distinguishing between categories with similar spectra but different spatial distributions. This branch effectively extracts local features and provides fine-grained discrimination capabilities.

[0081] Specifically, the local feature extraction network adopts a dual-branch structure with shared weights, including a spectral spatial feature extraction branch and a frequency domain feature extraction branch; the spectral spatial feature extraction branch includes a first convolutional network and a second convolutional network; the first convolutional network extracts spectral features from the preprocessed image through pointwise convolution. This convolution operation processes each spectral channel independently to capture the spectral details of each band. The second convolutional network uses depthwise convolution to extract spatial features from the spectral features output by the first convolutional network to obtain the spectral spatial feature Y spatial-spectral , which can capture the local pattern of the image in the spatial dimension, thereby effectively distinguishing categories with similar spectra but different spatial distributions, and improving classification accuracy by capturing the multi-domain features of hyperspectral images. The frequency domain feature extraction branch uses fast Fourier transform (FFT) to convert the pre-processed image into frequency domain data, and extracts the amplitude spectrum features from the frequency domain data. The amplitude spectrum features are the frequency domain features of the image. The frequency domain features capture the high-frequency detail information in the image and enhance the classification ability of objects with complex textures and repetitive patterns. The data processing process of the frequency domain feature extraction branch is as follows:

[0082] Y spa =Select(F spa (X conv ))

[0083] Among them, F spa is the Fourier transform performed on the spatial dimensions, Y spa Represents the converted frequency domain data, and by randomly selecting m patterns, most of the information is retained to ensure effectiveness. spa Perform frequency modulation to obtain the adjusted frequency domain data Y spa :

[0084] Y spa =F M (Y spa )=Y spa ⊙R

[0085] Among them, R is the parameterized convolution kernel, initialized to the standard normal distribution, and frequency modulation is performed using the Hadamard product ⊙ to enhance the expressiveness of the feature. Then, the modulated frequency domain data Y spa Perform zero padding and convert it back to the time domain through inverse Fourier transform to obtain the initial frequency domain features

[0086]

[0087] This step ensures that important information is not lost when the features are converted back to the spatial domain. Subsequently, the above steps are repeated in the spectral dimension, first performing Fourier transform on the initial frequency domain features to obtain the spectral domain Y spe :

[0088]

[0089] For spectral domain Y spe Perform frequency modulation to obtain the modulated spectrum domain:

[0090] Y spe =F M (Y spe )=Y spe ⊙R′

[0091] Among them, R′ is the parameterized convolution kernel, and finally the modulated spectrum domain Y is transformed into spe Convert back to the time domain to obtain the frequency domain feature Y frequency :

[0092]

[0093] The spectral spatial feature Y output by the spectral spatial feature extraction branch spatial-spectral And the frequency domain feature Y output by the frequency domain feature extraction branch frequency Perform weighted fusion to obtain the local features F of the hyperspectral image local :

[0094] F local =CNN(X conv )=α·Y spatial-spectral +(1-α)·Y frequency ;

[0095] Here, α is an adjustable fusion weight. By effectively combining local information in the time and frequency domains, detailed features in the input block can be more fully captured. The role of the local feature extraction network is to optimize the network's ability to distinguish between similar and dissimilar samples by comparing input sample pairs, thereby improving classification accuracy. The contrastive loss in the local feature extraction network further optimizes feature discriminability by minimizing the distance between similar samples and maximizing the distance between dissimilar samples. Therefore, while capturing information in the time and frequency domains, the CNN strengthens the capture of detailed differences between samples through the local feature extraction network structure, effectively improving classification performance under small sample conditions.

[0096] This embodiment adopts the SLIC (super pixel segmentation) algorithm to perform super pixel segmentation on the pre-processed image. The process of performing super pixel segmentation on the pre-processed image includes:

[0097] Obtain the spectral and spatial characteristics of each pixel in the preprocessed image;

[0098] Determine the distance between pixels in the preprocessed image based on the spectral characteristics and spatial characteristics of each pixel in the preprocessed image;

[0099] The preprocessed image is segmented into superpixels according to the distance between pixels in the preprocessed image.

[0100] Among them, the three main components X of the preprocessed image are extracted PCA ;

[0101] The three extracted principal components are mapped to the spectral features of each pixel in the preprocessed image;

[0102] Calculate and determine the color distance between pixels in the preprocessed image based on the spectral characteristics of each pixel in the preprocessed image;

[0103] Calculate and determine the spatial distance between pixels in the preprocessed image based on the spatial characteristics of each pixel in the preprocessed image;

[0104] The distance between pixels in the preprocessed image is calculated based on the color distance and spatial distance.

[0105] Specifically, this embodiment constructs the graph structure of the pre-processed image through the global feature extraction network and extracts the global features of the graph structure. The global feature extraction network uses the PCA (principal component analysis) method to extract the global features of the pre-processed image X conv Dimensionality reduction to three main components. These three principal components retain the maximum variance in the data while significantly reducing the feature dimension, thereby reducing processing complexity. PCA prepares for superpixel segmentation by calculating the covariance matrix of the data and finding the first three principal components.

[0106] Among them, X PCA =PCA(X conv ).

[0107] This embodiment uses the color distance calculation method to calculate the distance between space and spectrum through the preprocessed image to assist in the construction of the graph. The processing method is: the three principal components of PCA dimensionality reduction are mapped to the r, g, and b values ​​of the preprocessed image respectively, which are used to calculate the color distance d between pixels in the preprocessed image. c At the same time, the spatial distance d between pixels in the preprocessed image is calculated based on the spatial characteristics of the pixels s . c and d s Fusion, to obtain the distance metric that combines spectral and spatial information, that is, the comprehensive distance D between pixels in the preprocessed image, as the basis for subsequent superpixel segmentation. cThe calculation formula is: Among them, r j 、g j 、b j is the spectral feature of pixel j in the preprocessed image, r i 、g i 、b i is the spectral feature of pixel i in the preprocessed image. Next, the spatial distance d is calculated by the coordinates x and y in the spatial feature. s , the formula is: Among them, x j 、y j is the spatial feature of pixel j in the preprocessed image, x i 、y i is the spatial feature of pixel i in the preprocessed image. Finally, the distance D between pixels in the fused preprocessed image is calculated by combining the color distance and the spatial distance. The formula is: Among them, N c and N s They are normalization parameters, which are used to adjust the influence weight of color and spatial information in distance calculation.

[0108] Next, based on the distance D between pixels in the preprocessed image, pixels with similar color and spatial position are grouped into the same cluster, ultimately obtaining superpixels of the preprocessed image. These superpixels are compact and coherent regions that visually correspond to the edges of objects in the image.

[0109] This embodiment treats each superpixel of the image as a node, and takes the average value of all pixel features in the superpixel as the node feature of the node corresponding to the superpixel, where each pixel feature includes the spectral feature (r, g, b) and spatial feature (x, y) of the pixel. Based on the node features, the similarity between nodes is calculated; an edge is constructed between two nodes whose similarity is greater than a set value; and the edge weight is calculated using the Gaussian kernel function, using the formula: Among them, A i，j represents the edge weight between nodes i and j, and σ controls the scale of similarity. Finally, the output graph structure G = (V, E), where V is the set of superpixel nodes and E is the set of edges based on feature similarity.

[0110] This embodiment uses the Graph Attention Network (GAT) to extract global features of the graph structure. Specifically, GAT is used to extract global features, and its input is the node features in the graph structure generated after superpixel segmentation. The Graph Attention Network uses a multi-head attention mechanism to dynamically adjust the weights between neighboring nodes, thereby effectively extracting global context information and ultimately generating global features of the graph structure. The formula is as follows: global =GAT(V, E), where Fglobal Represents the global features of the graph structure extracted by the global feature extraction network.

[0111] S4: Combine the global features and local features of the graph structure to obtain the fusion features of the hyperspectral image.

[0112] In this embodiment, the global feature F is extracted from the global feature extraction network. global and the local features F obtained from the local feature extraction network local are linearly combined to form a richer feature representation. This process is performed by the following formula: fused =β·F global +(1-β)·F local , where β is the fusion weight parameter, which determines the contribution ratio of global features and local features in the final representation. Fusion feature F fused The global structural information and local spectral, spatial and frequency domain details are integrated to better describe the complex spectral and spatial relationships in the hyperspectral image. fused Passed to the classification module for further processing and prediction.

[0113] S5: Determine the classification result of the hyperspectral image based on the fusion features of the hyperspectral image.

[0114] This embodiment uses the Softmax classifier to identify the fusion features of the hyperspectral image and determine the classification result p of the hyperspectral image. i .

[0115] Specifically, the fusion feature F obtained by S4 fused First, adaptive average pooling is performed to reduce the dimension of the features. Subsequently, these reduced-dimensional features are classified through the fully connected layer, and finally the category probability distribution of each pixel is generated through Softmax. The specific formula is: i =Softmax(W·F pooled,i +b), where W is the weight matrix and b is the bias term. Through this process, the model can generate a probability distribution of categories for each pixel and perform pixel-level classification. Finally, the classification result p i The Softmax function is used to output the classification results of the hyperspectral image.

[0116] This embodiment also constructs a contrast loss function and a cross entropy loss function to optimize the network model. First, the local feature extraction network uses a contrast loss function to ensure that the feature distances of samples of the same type are closer and the feature distances of samples of different types are farther by comparing the similarities between the sample pairs. contrastive The formula is as follows: Where d represents the characteristic distance of the sample pair, y represents the label of the sample pair (1 for the same class and 0 for different classes), and m is the distance threshold. By minimizing the distance of samples of the same class and maximizing the distance of samples of different classes, the network can better distinguish between classes of objects with similar spectral features but different spatial distributions. In the final classification part, the model uses the cross entropy loss function to supervise the classification output of the network. Cross entropy loss L cross-entropy By calculating the difference between the class probability distribution predicted by the model and the actual label, the classification performance is optimized. The formula is: Among them, y i is the true category label, p i is the class probability predicted by the model. By minimizing the cross-entropy loss, the model can gradually optimize its classification accuracy and generate more accurate pixel-level classification results for hyperspectral images. The synergy of these two loss functions significantly improves the model's classification performance under small sample conditions.

[0117] Before development, technicians need to configure the experimental environment, including installing VSCode and the PyTorch deep learning framework. Furthermore, because deep learning model training is time-consuming, it is best performed on a GPU. During development, attention should be paid to the code implementation of the forward propagation formula corresponding to the loss function. The most challenging aspect of the entire development process is the setting of hyperparameters, such as the thresholds for different modules, the learning rate, and the number of iterations. These settings significantly impact experimental results.

[0118] This example not only extracts spatial and spectral features of hyperspectral images through CNN, but also extracts frequency-domain features. This allows the capture of global patterns and periodic variations in hyperspectral images, significantly reduces redundant information in the spectral dimension, improves classification robustness and accuracy, and facilitates the identification of complex ground objects.

[0119] The twin network in this embodiment is optimized through a contrastive loss function, which allows it to better distinguish between classes with similar spectral features but different spatial distributions. This contrastive learning approach significantly improves the model’s generalization and classification accuracy under small sample sizes.

[0120] Finally, this example combines the global features extracted by the global feature extraction network with the local features extracted by the local feature extraction network to better capture the complex spatial and spectral relationships in hyperspectral images. This fusion strategy ensures the robustness and accuracy of the model in various complex scenarios, improves the overall classification performance of hyperspectral images, and ensures the model's strong robustness in various application scenarios.

[0121] This embodiment effectively addresses the challenges faced by existing methods in hyperspectral image classification, especially in small sample conditions, by combining twin networks, dual-branch architecture, and multi-domain feature fusion technology. Existing methods are often limited to a single spectral or spatial dimension in feature extraction and cannot fully capture the complex information in the image. This embodiment uses a graph attention network to extract global structural information and captures local spectral-spatial features through a convolutional neural network, achieving an effective combination of global and local information and greatly improving classification accuracy. Through the fusion of multi-domain features, this embodiment not only utilizes spatial and spectral features, but also fully introduces frequency domain information, thereby enhancing the model's recognition ability for hyperspectral data.

[0122] At the same time, the introduction of the twin network enables the model to show stronger generalization ability under small sample conditions. Compared with traditional classification methods, the twin network effectively distinguishes categories with similar spectra but different spaces through contrastive learning, thereby improving the robustness in scenarios with limited samples. Although the existing methods have improved the classification performance to a certain extent, they have ignored the importance of frequency domain features. The present invention overcomes the shortcomings of the existing technology by combining spectral, spatial and frequency domain information at multiple levels, especially showing higher classification accuracy and stability in complex scenarios and limited sample conditions.

[0123] Example 2

[0124] In this embodiment, a hyperspectral image classification system is disclosed, such as Figure 4 Shown, including:

[0125] An image acquisition module, used for acquiring hyperspectral images;

[0126] A data preprocessing module is used to preprocess the hyperspectral image to obtain a preprocessed image;

[0127] The spectral, spatial and frequency domain feature extraction module is used to extract spectral spatial features and frequency domain features from the preprocessed image; the spectral spatial features are fused with the frequency domain features extracted from the preprocessed image to obtain the local features of the hyperspectral image;

[0128] The global feature extraction module is used to perform superpixel segmentation on the preprocessed image, treating each superpixel of the image as a node, constructing a graph structure, and extracting the global features of the graph structure;

[0129] The feature fusion module is used to combine the global features of the graph structure and the local features of the hyperspectral image to obtain the fusion features of the hyperspectral image;

[0130] The classification module is used to determine the classification results of the hyperspectral image based on the fusion features of the hyperspectral image.

[0131] The present invention also discloses a computer device, comprising:

[0132] a processor adapted to execute a computer program;

[0133] A computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the hyperspectral image classification method disclosed in Example 1 is implemented.

[0134] The present invention also discloses a computer-readable storage medium, which stores a computer program. The computer program is suitable for being loaded by a processor and executing a hyperspectral image classification method disclosed in Example 1.

[0135] The present invention also discloses a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the hyperspectral image classification method disclosed in Example 1.

[0136] The method disclosed in Example 1 can be directly implemented as a hardware processor, or can be implemented using a combination of hardware and software modules in the processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, it will not be described in detail here.

[0137] Those skilled in the art will appreciate that the units and algorithm steps of the various examples described in conjunction with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0138] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. A hyperspectral image classification method, characterized in that: include: Acquire hyperspectral images; Preprocessing the hyperspectral image to obtain a preprocessed image; A local feature extraction network is used to extract local features of the hyperspectral image from the preprocessed image; the local feature extraction network is a twin network architecture based on a convolutional neural network, which extracts spectral spatial features and frequency domain features from the preprocessed image; the spectral spatial features and the frequency domain features extracted from the preprocessed image are weightedly fused to obtain the local features of the hyperspectral image; wherein, the local feature extraction network adopts a dual-branch structure with shared weights, including a spectral spatial feature extraction branch and a frequency domain feature extraction branch; the spectral spatial feature extraction branch includes a first convolutional network and a second convolutional network; the first convolutional network extracts spectral features from the preprocessed image through point convolution, and the convolution operation processes each spectral channel independently to capture the spectral details of each band; the second convolutional network uses deep convolution to extract spatial features from the spectral features output by the first convolutional network to obtain spectral spatial features, which can capture the local pattern of the image in the spatial dimension; the frequency domain feature extraction branch uses fast Fourier transform to convert the preprocessed image into frequency domain data, and extracts amplitude spectrum features from the frequency domain data, and the amplitude spectrum features are the frequency domain features of the image; Perform superpixel segmentation on the preprocessed image, treat each superpixel of the image as a node, construct a graph structure, and use a graph attention network to extract the global features of the graph structure. The input of the graph attention network is the node features in the graph structure generated after superpixel segmentation; The global features of the graph structure and the local features of the hyperspectral image are fused to obtain the fusion features of the hyperspectral image; According to the fusion characteristics of the hyperspectral image, the classification result of the hyperspectral image is determined.

2. A hyperspectral image classification method according to claim 1, characterized in that: The process of superpixel segmentation of the preprocessed image includes: Obtain the spectral and spatial characteristics of each pixel in the preprocessed image; Determine the distance between pixels in the preprocessed image based on the spectral characteristics and spatial characteristics of each pixel in the preprocessed image; The preprocessed image is segmented into superpixels according to the distance between pixels in the preprocessed image.

3. A hyperspectral image classification method according to claim 1, characterized in that: The average value of all pixel features in the superpixel is used as the node feature of the node corresponding to the superpixel, where each pixel feature includes the spectral feature and spatial feature of the pixel.

4. A hyperspectral image classification method according to claim 3, characterized in that: Calculate the similarity between nodes based on node features; Build an edge between two nodes whose similarity is greater than a set value.

5. A hyperspectral image classification method according to claim 1, characterized in that: Spectral features are extracted from the preprocessed image; spatial features are extracted from the spectral features to obtain spectral spatial features.

6. A hyperspectral image classification system, characterized in that: include: An image acquisition module, used for acquiring hyperspectral images; A data preprocessing module is used to preprocess the hyperspectral image to obtain a preprocessed image; The spectral, spatial and frequency domain feature extraction module is used to extract the local features of the hyperspectral image from the preprocessed image using the local feature extraction network; the local feature extraction network is a twin network architecture based on convolutional neural networks, which extracts spectral spatial features and frequency domain features from the preprocessed image; the spectral spatial features and the frequency domain features extracted from the preprocessed image are weightedly fused to obtain the local features of the hyperspectral image; wherein, the local feature extraction network adopts a dual-branch structure with shared weights, including a spectral spatial feature extraction branch and a frequency domain feature extraction branch; the spectral spatial feature extraction branch includes a first convolutional network and a second convolutional network; the first convolutional network extracts spectral features from the preprocessed image through point convolution, and the convolution operation processes each spectral channel independently to capture the spectral details of each band; the second convolutional network uses deep convolution to extract spatial features from the spectral features output by the first convolutional network to obtain spectral spatial features, which can capture the local pattern of the image in the spatial dimension; the frequency domain feature extraction branch uses fast Fourier transform to convert the preprocessed image into frequency domain data, and extracts amplitude spectrum features from the frequency domain data, and the amplitude spectrum features are the frequency domain features of the image; The global feature extraction module is used to perform superpixel segmentation on the preprocessed image. Each superpixel in the image is regarded as a node, a graph structure is constructed, and the global features of the graph structure are extracted using a graph attention network. The input of the graph attention network is the node features in the graph structure generated after superpixel segmentation. The feature fusion module is used to fuse the global features of the graph structure with the local features of the hyperspectral image to obtain the fusion features of the hyperspectral image; The classification module is used to determine the classification results of the hyperspectral image based on the fusion features of the hyperspectral image.

7. An electronic device, characterized in that: The device comprises: a processor adapted to execute a computer program; A computer-readable storage medium having a computer program stored therein, wherein when the computer program is executed by the processor, the method for classifying hyperspectral images according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the hyperspectral image classification method according to any one of claims 1 to 5.

9. A computer program product, characterized in that The computer program product includes a computer program, and when the computer program is executed by a processor, the method for classifying hyperspectral images according to any one of claims 1 to 5 is implemented.

Images