Hyperspectral image classification method and system

Abstract

The invention discloses a hyperspectral image classification method and system. The method comprises the following steps: acquiring a hyperspectral image; preprocessing the hyperspectral image to obtain a preprocessed image; extracting spectrum, space and frequency domain features from the preprocessed image; superpixel segmentation is carried out on the preprocessed image, each superpixel of the image is regarded as a node, a graph structure is constructed, and global features of the graph structure are extracted; obtaining fusion features of the hyperspectral image according to the global features and the local features of the graph structure; and determining a classification result of the hyperspectral image according to the fusion features of the hyperspectral image. And the classification identification effect of the hyperspectral image is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral image classification, and in particular to a hyperspectral image classification method and system. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of deep learning technology, hyperspectral image analysis has been widely used in remote sensing, agricultural monitoring, and environmental protection. Hyperspectral images have rich spectral information, and each pixel contains spectral data of multiple continuous bands. This hyperspectral dimension provides strong support for accurate classification. However, due to the complexity of high-dimensional data and the scarcity of labeled samples, the classification task faces great challenges, especially under small sample conditions, the classification accuracy and robustness of existing methods are often insufficient. Early hyperspectral image classification methods relied on machine learning algorithms such as support vector machines (SVM), K nearest neighbors (KNN), and random forests (RF). These methods use artificially designed features for classification. Although they perform well in some applications, they are difficult to capture the deep information in hyperspectral images, and mainly focus on the use of spectral features, ignoring the spatial correlation between pixels. This leads to low classification accuracy and susceptibility to noise, especially in complex scenes.

[0004] In recent years, graph neural networks (GNNs) have been gradually introduced into hyperspectral image classification. Graph convolutional networks (GCNs) and GATs capture global features through graph structures and perform well in processing complex spatial relationships in hyperspectral images. In particular, GAT dynamically adjusts the weights between nodes through the attention mechanism, which enhances the feature extraction capability in small sample scenarios. However, existing GNN methods rely on a fixed adjacency matrix and cannot dynamically adjust the node relationship according to the changes in data. At the same time, most methods only focus on time domain features and fail to fully utilize frequency domain features, which is crucial for capturing periodic changes and global patterns in images. In addition, with the development of small sample learning technology, metric learning methods such as twin networks have been gradually applied to hyperspectral image classification. Twin networks can achieve effective classification under limited labeled data by learning the similarities between sample pairs. However, most of the current twin networks only use spectral and spatial features, and fail to fully combine frequency domain information for comprehensive feature extraction, which limits the further improvement of their classification performance. At present, the field of hyperspectral image classification urgently needs a solution that can fully utilize the synergy of spectral, spatial and frequency domain features under small sample conditions to enhance classification accuracy and robustness. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a hyperspectral image classification method and system, which improves the accuracy of hyperspectral image classification.

[0006] To achieve the above object, the present invention adopts the following technical solution:

[0007] Firstly, a hyperspectral image classification method is proposed, comprising:

[0008] Acquire hyperspectral images;

[0009] Preprocessing the hyperspectral image to obtain a preprocessed image;

[0010] Extracting spectral spatial features and frequency domain features from the preprocessed image;

[0011] The spectral spatial features are fused with the frequency domain features extracted from the preprocessed image to obtain the local features of the hyperspectral image;

[0012] Perform superpixel segmentation on the preprocessed image, regard each superpixel of the image as a node, build a graph structure, and extract the global features of the graph structure;

[0013] The global features of the graph structure and the local features of the hyperspectral image are combined to obtain the fusion features of the hyperspectral image;

[0014] According to the fusion characteristics of the hyperspectral image, the classification result of the hyperspectral image is determined.

[0015] Furthermore, the process of performing superpixel segmentation on the preprocessed image includes:

[0016] Obtain the spectral characteristics and spatial characteristics of each pixel in the preprocessed image;

[0017] Determine the distance between pixels in the preprocessed image according to the spectral characteristics and spatial characteristics of each pixel in the preprocessed image;

[0018] The preprocessed image is segmented into superpixels according to the distance between pixels in the preprocessed image.

[0019] Furthermore, the average value of all pixel features in the superpixel is used as the node feature of the node corresponding to the superpixel, wherein each pixel feature includes the spectral feature and spatial feature of the pixel.

[0020] Furthermore, the similarity between nodes is calculated based on the node features;

[0021] Build an edge between two nodes whose similarity is greater than a set value.

[0022] Furthermore, the graph attention network is used to extract the global features of the graph structure.

[0023] Furthermore, spectral features are extracted from the preprocessed image;

[0024] The spatial features are extracted from the spectral features to obtain the spectral spatial features.

[0025] Secondly, a hyperspectral image classification system is proposed, including:

[0026] An image acquisition module, used for acquiring hyperspectral images;

[0027] A data preprocessing module is used to preprocess the hyperspectral image to obtain a preprocessed image;

[0028] The spectral, spatial and frequency domain feature extraction module is used to extract spectral spatial features and frequency domain features from the preprocessed image; the spectral spatial features are fused with the frequency domain features extracted from the preprocessed image to obtain the local features of the hyperspectral image;

[0029] The global feature extraction module is used to perform superpixel segmentation on the preprocessed image, regard each superpixel of the image as a node, construct a graph structure, and extract the global features of the graph structure;

[0030] The feature fusion module is used to combine the global features of the graph structure and the local features of the hyperspectral image to obtain the fusion features of the hyperspectral image;

[0031] The classification module is used to determine the classification result of the hyperspectral image according to the fusion characteristics of the hyperspectral image.

[0032] In a third aspect, a computer device is provided, the device comprising:

[0033] a processor adapted to execute a computer program;

[0034] A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, a hyperspectral image classification method proposed in the first aspect is implemented.

[0035] In a fourth aspect, a computer-readable storage medium is proposed, wherein the computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing a hyperspectral image classification method proposed in the first aspect.

[0036] In a fifth aspect, a computer program product is proposed. The computer program product includes a computer program. When the computer program is executed by a processor, the hyperspectral image classification method proposed in the first aspect is implemented.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The present invention provides a method and system for classifying hyperspectral images. The method first preprocesses the hyperspectral image, then performs superpixel segmentation on the preprocessed image, reduces the complexity of the data, constructs the graph structure of the image, and uses the graph attention network to extract the global features of the graph structure while reducing redundant features. In addition, the present invention also extracts spectral spatial features and frequency domain features from the preprocessed image, then fuses the spectral spatial features and frequency domain features to form local features, so that the local features contain the spectral information, spatial information and frequency domain information of the image, and can capture the global pattern and periodic changes in the image. This method can effectively improve the robustness and accuracy of classification when processing complex scenes with periodic changes; finally, the global features are fused with the local features to generate fused features; the fused features are used to determine the classification results of the hyperspectral image. Since the fused features contain more feature information of the image, the accuracy of hyperspectral image recognition can be improved.

[0039] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings in the specification, which constitute a part of the present application, are used to provide further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0041] Figure 1 A flow chart of a hyperspectral image classification method disclosed in an embodiment;

[0042] Figure 2 A block diagram of a local feature extraction network in a hyperspectral image classification method disclosed in an embodiment;

[0043] Figure 3 This is the overall framework diagram of the existing hyperspectral image classification and recognition methods;

[0044] Figure 4 A framework diagram of a hyperspectral image classification system disclosed in an embodiment. DETAILED DESCRIPTION

[0045] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0046] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0047] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0048] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0049] Example 1

[0050] Explanation of terms involved in this embodiment:

[0051] Hyperspectral Image (HSI): A remote sensing imaging technology that can capture the spectral information of an object in multiple continuous narrow bands. Each pixel contains a spectral curve representing the reflectance or radiance at different wavelengths.

[0052] Graph Attention Network (GAT): A graph neural network that performs convolution operations on graph-structured data. GAT introduces an attention mechanism to dynamically adjust the weight distribution between nodes, allowing the network to process complex graph data more efficiently.

[0053] Siamese Network: A neural network architecture for metric learning that processes input sample pairs through two sub-networks with shared weights to calculate their feature similarity. This network is often used for small sample learning and similarity tasks.

[0054] Few-Shot Learning (FSL): A machine learning method that aims to use a very small number of labeled examples for training and show good generalization ability on new data.

[0055] Frequency-Domain Features: The image is converted from the time domain to the frequency domain through Fourier transform to obtain the global pattern and periodic feature information of the image. This is especially important for processing image categories with high-frequency information.

[0056] Superpixel: Adjacent pixels in a hyperspectral image are grouped using a clustering algorithm. Each superpixel represents a set of pixels in a local area, which is used to reduce computational complexity.

[0057] With the rapid development of deep learning technology, hyperspectral image analysis has been widely used in remote sensing, agricultural monitoring, and environmental protection. Hyperspectral images have rich spectral information, and each pixel contains spectral data of multiple continuous bands. This hyperspectral dimension provides strong support for accurate classification. However, due to the complexity of high-dimensional data and the scarcity of labeled samples, the classification task faces great challenges, especially under small sample conditions, the classification accuracy and robustness of existing methods are often insufficient. Early hyperspectral image classification methods relied on machine learning algorithms such as support vector machines (SVM), K nearest neighbors (KNN), and random forests (RF). These methods use artificially designed features for classification. Although they perform well in some applications, they are difficult to capture the deep information in hyperspectral images, and mainly focus on the use of spectral features, ignoring the spatial correlation between pixels. This leads to low classification accuracy and susceptibility to noise, especially in complex scenes.

[0058] The introduction of convolutional neural networks has significantly improved the performance of hyperspectral image classification. By automatically extracting spectral and spatial features, CNN reduces the reliance on manually designed features. However, CNN still faces two major problems in the application of hyperspectral images: first, the annotated data of hyperspectral images is scarce, and CNN is prone to overfitting under small sample conditions; second, CNN has limited processing capabilities for non-Euclidean structures (such as graph structure data), making it difficult to fully capture the global context information in hyperspectral images. To address these problems, graph neural networks (GNNs) have been gradually introduced into hyperspectral image classification in recent years. Graph convolutional networks (GCNs) and GATs capture global features through graph structures and perform well in processing complex spatial relationships in hyperspectral images. In particular, GAT dynamically adjusts the weights between nodes through an attention mechanism, which enhances the feature extraction capability in small sample scenarios. However, existing GNN methods rely on a fixed adjacency matrix and cannot dynamically adjust node relationships according to data changes. At the same time, most methods only focus on time domain features and fail to fully utilize frequency domain features, while frequency domain information is crucial for capturing periodic changes and global patterns in images. In addition, with the development of small sample learning technology, metric learning methods such as twin networks are gradually being applied to hyperspectral image classification. By learning the similarities between sample pairs, twin networks can achieve effective classification with limited labeled data. However, most of the current twin networks only use spectral and spatial features, and fail to fully combine frequency domain information for comprehensive feature extraction, which limits the further improvement of their classification performance. At present, the field of hyperspectral image classification urgently needs a solution that can fully utilize the synergy of spectral, spatial and frequency domain features under small sample conditions to enhance classification accuracy and robustness.

[0059] In the research based on hyperspectral image classification, many methods currently adopt a single branch structure. These methods usually either focus on the extraction of global features or focus on capturing local pixel relationships, and lack the effective combination of global and local features. This single feature extraction method limits the performance of the model when processing hyperspectral data with complex spectral-spatial information, especially when the amount of sample data is small, the classification accuracy is obviously limited. In order to solve the problem that global and local features cannot be effectively combined, a method based on a cascade structure has been proposed recently, and has achieved good results in hyperspectral image classification. This method first extracts global features through a superpixel-based graph convolutional network, and then further extracts local detail features through a pixel-based convolutional neural network, such as Figure 3 As shown. Specifically, the method designs a multi-layer structure in the GCN module and adopts DropEdge technology and residual connection to alleviate the problems of over-smoothing and over-fitting, so that GCN can extract global features at the superpixel level more deeply. After completing the global feature extraction, the CNN module applies a hybrid attention mechanism to the pixel-level data, combines spatial coordinate information and channel attention, and extracts detailed spectral-spatial features. Through this cascade structure, the model can first capture global context information, and then accurately locate the relationship between local pixels, thereby significantly improving the classification performance, especially when the sample size is small. However, although this method is superior to most current classification methods in terms of overall performance, it still has some limitations. Although GCN can process graph structure data, its ability to capture global structural information is slightly insufficient compared to the graph attention network. In addition, the CNN branch mainly focuses on spatial and spectral features, and fails to make full use of frequency domain features, which may affect the classification effect when processing complex spectral-spatial data. Therefore, in view of these shortcomings, the present invention can not only combine local and global features, but also significantly improve the classification performance by introducing frequency domain information, and can effectively cope with the challenge of limited sample size.

[0060] In summary, the shortcomings of the prior art are:

[0061] (1) Single feature extraction method: Existing methods usually only focus on spectral or spatial features and fail to effectively combine the two. This results in the inability to fully utilize the multidimensional information in hyperspectral images, thus affecting the accuracy and robustness of classification.

[0062] (2) Insufficient utilization of frequency domain information: Although hyperspectral images contain a large amount of frequency domain information, existing methods still mainly rely on time domain features and cannot effectively utilize frequency domain features to capture global patterns and periodic changes, resulting in poor performance of the model in complex scenes.

[0063] (3) Poor classification performance with small samples: Due to the scarcity of labeled data for hyperspectral images, existing deep learning models are prone to overfitting under small sample conditions, and it is difficult to maintain a high level of classification performance.

[0064] This embodiment realizes the coordinated use of spectral, spatial and frequency domain information in hyperspectral images by designing a dual-branch architecture that combines a graph attention network and a twin network, so as to improve the accuracy and robustness of hyperspectral image classification. At the same time, the present invention introduces frequency domain feature extraction to capture global patterns and periodic changes, thereby reducing data redundancy and enhancing the generalization ability of the model in complex scenarios. Through the metric learning mechanism of the twin network, this embodiment improves the classification performance under small sample conditions and avoids the overfitting problem. In addition, the introduction of the graph attention mechanism allows the weights between nodes to be dynamically adjusted, thereby improving the expressiveness of the model when processing complex hyperspectral data and overcoming the limitations of traditional graph neural networks.

[0065] like Figure 1-Figure 2 As shown, a hyperspectral image classification method disclosed in this embodiment includes:

[0066] S1: Acquire hyperspectral images.

[0067] The size of the acquired hyperspectral image is H×W×C, where H is the height, W is the width, and C is the number of spectral channels.

[0068] S2: Preprocess the hyperspectral image to obtain a preprocessed image.

[0069] The preprocessing of the hyperspectral image in this embodiment includes noise reduction and dimensionality reduction processing to remove noise and redundant information in the original data, reduce the complexity of high-dimensional data, and enhance the ability to distinguish spectral features in classification tasks. Specifically: first, batch normalization (BN) processing is performed on the hyperspectral image to reduce the noise and redundant information of the data. This step enhances the training stability and convergence of the model by standardizing the mean and variance of the data; then, convolution calculation is performed on the batch normalized data X to reduce the data dimension and obtain the preprocessed image X conv .

[0070] Preferably, two two-dimensional convolutional layers with 1×1 convolution kernels are used to reduce the dimension of the batch normalized data, and combined with the LeakyReLU activation function to obtain the preprocessed image, further reduce the noise and extract more discriminative spectral features. The formula is X conv =LeakyReLU(Conv2D(X)), the processed data is used for subsequent image superpixel segmentation and feature extraction.

[0071] This embodiment preprocesses the hyperspectral image and can compress the hyperspectral image data from hundreds of bands to a few principal components, which helps to enhance the robustness and discrimination ability of the spectral characteristics of the hyperspectral image, thereby reducing redundant information and improving the efficiency of subsequent processing.

[0072] S3: Extract spectral spatial features and frequency domain features from preprocessed images;

[0073] The spectral spatial features are fused with the frequency domain features extracted from the preprocessed image to obtain the local features of the hyperspectral image;

[0074] The preprocessed image is segmented into superpixels, each superpixel of the image is regarded as a node, a graph structure is constructed, and the global features of the graph structure are extracted.

[0075] The process of acquiring the spectral spatial features in this embodiment includes:

[0076] Extract spectral features from the preprocessed image;

[0077] The spatial features are extracted from the spectral features to obtain the spectral spatial features.

[0078] Preferably, a convolutional neural network is used to extract spectral features of the preprocessed image, and spatial features are extracted from the spectral features;

[0079] The preprocessed image is converted into frequency domain data using fast Fourier transform, and the frequency domain features of the preprocessed image are extracted from the frequency domain data.

[0080] This embodiment uses a local feature extraction network to extract local features of the hyperspectral image from the preprocessed image, such as Figure 2 As shown in Figure 2, the local feature extraction network is a twin network architecture based on a convolutional neural network (CNN), which extracts spectral and spatial features Y from the preprocessed image. spatial-spectral and frequency domain feature Y frequency , and the spectral spatial feature Y spatial-spectral and frequency domain feature Y frequency Perform weighted fusion to obtain the local features F of the hyperspectral image local . By extracting the frequency domain features of the preprocessed image, the high-frequency detail information in the image can be captured, which is particularly suitable for the classification of complex landform types. Through point convolution and depth convolution, the model can capture local detail information in the image, which is particularly suitable for distinguishing categories with similar spectra but different spatial distributions. This branch effectively extracts local features and provides fine-grained discrimination capabilities.

[0081] Specifically, the local feature extraction network adopts a dual-branch structure with shared weights, including a spectral spatial feature extraction branch and a frequency domain feature extraction branch; the spectral spatial feature extraction branch includes a first convolutional network and a second convolutional network; the first convolutional network extracts spectral features from the preprocessed image through pointwise convolution, and the convolution operation processes each spectral channel independently to capture the spectral details of each band. The second convolutional network uses depthwise convolution to extract spatial features from the spectral features output by the first convolutional network to obtain the spectral spatial features Y spatial-spectral , which can capture the local pattern of the image in the spatial dimension, thereby effectively distinguishing categories with similar spectra but different spatial distributions, and improving classification accuracy by capturing the multi-domain features of hyperspectral images. The frequency domain feature extraction branch uses fast Fourier transform (FFT) to convert the preprocessed image into frequency domain data, and extracts the amplitude spectrum features from the frequency domain data. The amplitude spectrum features are the frequency domain features of the image. The high-frequency detail information in the image is captured through the frequency domain features, and the classification ability of objects with complex textures and repeated patterns is enhanced. The data processing process of the frequency domain feature extraction branch is as follows:

[0082] Y spa =Select(F spa (X conv ))

[0083] Among them, F spa is the Fourier transform performed on the spatial dimensions, Y spa Represents the converted frequency domain data, and most of the information is retained by randomly selecting m patterns to ensure effectiveness. spa Perform frequency modulation to obtain the adjusted frequency domain data Y spa :

[0084] Y spa =F M (Y spa )=Y spa ⊙R

[0085] Among them, R is a parameterized convolution kernel, which is initialized to a standard normal distribution and uses the Hadamard product ⊙ for frequency modulation to enhance the expressiveness of the feature. Then, the modulated frequency domain data Y spa Zero padding is performed and it is converted back to the time domain through inverse Fourier transform to obtain the initial frequency domain features

[0086]

[0087] This step ensures that no important information is lost when the features are converted back to the spatial domain. Subsequently, the above steps are repeated in the spectral dimension, firstly Fourier transforming the initial frequency domain features to obtain the spectral domain Y spe :

[0088]

[0089] For spectral domain Y spe Perform frequency modulation to obtain the modulated spectrum domain:

[0090] Y spe =F M (Y spe )=Y spe ⊙R′

[0091] Among them, R′ is the parameterized convolution kernel, and finally the modulated spectral domain Y is transformed by inverse Fourier transform. spe Convert back to the time domain and get the frequency domain feature Y frequency :

[0092]

[0093] The spectral spatial feature Y output by the spectral spatial feature extraction branch spatial-spectral And the frequency domain feature Y output by the frequency domain feature extraction branch frequency Perform weighted fusion to obtain the local features F of the hyperspectral image local :

[0094] F local =CNN(X conv )=α·Y spatial-spectral +(1-α)·Y frequency ;

[0095] Among them, α is an adjustable fusion weight. By effectively combining the local information in the time domain and frequency domain, the detailed features in the input block can be more fully captured. The role of the local feature extraction network is to optimize the network's ability to distinguish similar and dissimilar samples by comparing the input sample pairs, thereby improving the classification accuracy. The contrastive loss in the local feature extraction network further optimizes the distinguishability of features by minimizing the distance between similar samples and maximizing the distance between dissimilar samples. Therefore, while capturing the time domain and frequency domain information, CNN strengthens the capture of detailed differences between samples through the local feature extraction network structure, thereby effectively improving the classification performance under small sample conditions.

[0096] This embodiment adopts the SLIC (super pixel segmentation) algorithm to perform super pixel segmentation on the preprocessed image. The process of performing super pixel segmentation on the preprocessed image includes:

[0097] Obtain the spectral characteristics and spatial characteristics of each pixel in the preprocessed image;

[0098] Determine the distance between pixels in the preprocessed image according to the spectral characteristics and spatial characteristics of each pixel in the preprocessed image;

[0099] The preprocessed image is segmented into superpixels according to the distance between pixels in the preprocessed image.

[0100] Among them, the three main components X of the preprocessed image are extracted PCA ;

[0101] The three extracted principal components are respectively mapped to the spectral features of each pixel in the preprocessed image;

[0102] According to the spectral characteristics of each pixel in the preprocessed image, the color distance between pixels in the preprocessed image is calculated and determined;

[0103] According to the spatial characteristics of each pixel in the preprocessed image, the spatial distance between pixels in the preprocessed image is calculated and determined;

[0104] Based on the color distance and the spatial distance, the distance between pixels in the preprocessed image is calculated and determined.

[0105] Specifically, this embodiment constructs the graph structure of the preprocessed image through the global feature extraction network and extracts the global features of the graph structure. The global feature extraction network uses the PCA (principal component analysis) method to extract the global features of the graph structure. conv Dimensionality reduction to three main components. These three principal components retain the largest variance in the data while significantly reducing the feature dimension, thereby reducing processing complexity. PCA prepares for superpixel segmentation by calculating the covariance matrix of the data and obtaining the first three principal components.

[0106] Among them, X PCA = PCA(X conv ).

[0107] This embodiment uses the color distance calculation method to calculate the distance between space and spectrum through the preprocessed image to assist in the construction of the graph. The processing method is: the three principal components of PCA dimensionality reduction are mapped to the r, g, and b values ​​of the preprocessed image respectively, which are used to calculate the color distance d between pixels in the preprocessed image. c , and calculate the spatial distance d between pixels in the preprocessed image based on the spatial characteristics of the pixels s . c and d s Fusion, to obtain the distance metric that combines spectral and spatial information, that is, the comprehensive distance D between pixels in the preprocessed image, as the basis for subsequent superpixel segmentation. cThe calculation formula is: Among them, r j , g j , b j is the spectral feature of pixel j in the preprocessed image, r i , g i , b i is the spectral feature of pixel i in the preprocessed image. Next, the spatial distance d is calculated by the coordinates x and y in the spatial feature. s , the formula is: Among them, x j ,y j is the spatial feature of pixel j in the preprocessed image, x i ,y i is the spatial feature of pixel i in the preprocessed image. Finally, the distance D between pixels in the fused preprocessed image is calculated by combining the color distance and the spatial distance. The formula is: Among them, N c and N s They are normalization parameters, which are used to adjust the influence weight of color and spatial information in distance calculation.

[0108] Afterwards, pixels with similar colors and spatial positions are grouped into the same cluster according to the distance D between pixels in the preprocessed image, and finally the superpixels of the preprocessed image are obtained. These superpixels are compact and coherent areas that have a high visual correspondence with the edges of objects in the image.

[0109] In this embodiment, each superpixel of the image is regarded as a node, and the average value of all pixel features in the superpixel is used as the node feature of the node corresponding to the superpixel, wherein each pixel feature includes the spectral feature (r, g, b) and spatial feature (x, y) of the pixel. Based on the node features, the similarity between nodes is calculated; an edge is constructed between two nodes whose similarity is greater than a set value; and the edge weight is calculated using the Gaussian kernel function, and the formula is: Among them, A i，j represents the edge weight between nodes i and j, and σ controls the scale of similarity. Finally, the output graph structure G = (V, E), where V is the superpixel node set and E is the edge set based on feature similarity.

[0110] This embodiment uses a graph attention network (GAT) to extract global features of the graph structure. Specifically, GAT is used to extract global features, and its input is the node features in the graph structure generated after superpixel segmentation. The graph attention network uses a multi-head attention mechanism to dynamically adjust the weights between neighboring nodes, thereby effectively extracting global context information and ultimately generating global features of the graph structure. The formula is as follows: global =GAT(V, E), where Fglobal Represents the global features of the graph structure extracted by the global feature extraction network.

[0111] S4: Combine the global features and local features of the graph structure to obtain the fusion features of the hyperspectral image.

[0112] In this embodiment, the global feature F extracted from the global feature extraction network global and the local features F obtained from the local feature extraction network local are linearly combined to form a richer feature representation. This process is performed by the following formula: fused =β·F global +(1-β)·F local , where β is the fusion weight parameter, which determines the contribution ratio of global features and local features in the final representation. Fusion feature F fused The global structural information and local spectral, spatial and frequency domain details are integrated to better describe the complex spectral and spatial relationships in the hyperspectral image. fused It is passed to the classification module for further processing and prediction.

[0113] S5: Determine the classification result of the hyperspectral image according to the fusion characteristics of the hyperspectral image.

[0114] This embodiment uses the Softmax classifier to identify the fusion features of the hyperspectral image and determine the classification result p of the hyperspectral image. i .

[0115] Specifically, the fusion feature F obtained by S4 fused First, adaptive average pooling is performed to reduce the dimension of the features. Then, these reduced-dimensional features are classified through the fully connected layer, and finally the category probability distribution of each pixel is generated through Softmax. The specific formula is: i =Softmax(W·F pooled,i +b), where W is the weight matrix and b is the bias term. Through this process, the model can generate a probability distribution of categories for each pixel and perform pixel-level classification. Finally, the classification result p i The Softmax is used for output to generate the classification results of the hyperspectral image.

[0116] This embodiment also constructs a contrast loss function and a cross entropy loss function to optimize the network model. First, the local feature extraction network uses a contrast loss function to ensure that the feature distances of samples of the same type are closer and the feature distances of samples of different types are farther by comparing the similarities between sample pairs. contrastive The formula is as follows: Where d represents the characteristic distance of the sample pair, y represents the label of the sample pair (1 for the same class, 0 for different classes), and m is the distance threshold. By minimizing the distance of samples of the same class and maximizing the distance of samples of different classes, the network can better distinguish between classes of objects with similar spectral features but different spatial distributions. In the final classification part, the model uses the cross entropy loss function to supervise the classification output of the network. Cross entropy loss L cross-entropy The classification performance is optimized by calculating the difference between the class probability distribution predicted by the model and the true label. The formula is: Among them, y i is the true category label, p i is the class probability predicted by the model. By minimizing the cross entropy loss, the model can gradually optimize its classification accuracy and generate more accurate pixel-level classification results for hyperspectral images. The synergy of these two loss functions significantly improves the classification performance of the model under small sample conditions.

[0117] Before development, technicians need to configure the experimental environment, including installing VScode software and PyTorch deep learning framework. In addition, because the deep learning model training process takes a long time, it is best to train under GPU. During the development process, you need to pay attention to the code implementation of the forward propagation formula corresponding to the loss function. The most difficult thing to control during the entire development process is the setting of some hyperparameters, such as the thresholds of different modules, the value of the learning rate, the number of iterations, etc. The setting of these parameters has a great impact on the experimental results.

[0118] This embodiment extracts the frequency domain features of hyperspectral images based on the spatial and spectral features extracted by CNN, thereby capturing the global patterns and periodic changes in hyperspectral images, significantly reducing the redundant information in the spectral dimension, improving the robustness and accuracy of classification, and facilitating the identification of complex land object categories.

[0119] The twin network of this embodiment can better distinguish categories with similar spectral features but different spatial distributions by optimizing the contrast loss function. Under small sample conditions, this contrast learning method greatly improves the generalization ability and classification accuracy of the model.

[0120] Finally, in this embodiment, the global features extracted by the global feature extraction network and the local features extracted by the local feature extraction network are weighted and fused to better capture the complex spatial and spectral relationships in the hyperspectral image. This fusion strategy ensures the robustness and accuracy of the model in various complex scenarios, improves the overall classification performance of hyperspectral images, and ensures that the model has strong robustness in various application scenarios.

[0121] This embodiment effectively addresses the challenges faced by existing methods in hyperspectral image classification, especially the performance under small sample conditions, by combining twin networks, dual-branch architectures, and multi-domain feature fusion technology. Existing methods are often limited to a single spectral or spatial dimension in feature extraction and cannot fully capture the complex information in the image. This embodiment uses a graph attention network to extract global structural information and captures local spectral-spatial features through a convolutional neural network, achieving an effective combination of global and local information and greatly improving classification accuracy. Through the fusion of multi-domain features, this embodiment not only utilizes spatial and spectral features, but also fully introduces frequency domain information, thereby enhancing the model's ability to recognize hyperspectral data.

[0122] At the same time, the introduction of the twin network enables the model to show stronger generalization ability under small sample conditions. Compared with traditional classification methods, the twin network effectively distinguishes spectrally similar but spatially different categories through contrastive learning, which improves the robustness in scenarios with limited samples. Although the existing methods have improved the classification performance to a certain extent, they have ignored the importance of frequency domain features. The present invention overcomes the shortcomings of the existing technology by combining spectral, spatial, and frequency domain information at multiple levels, especially showing higher classification accuracy and stability in complex scenarios and limited sample conditions.

[0123] Example 2

[0124] In this embodiment, a hyperspectral image classification system is disclosed, such as Figure 4 As shown, including:

[0125] An image acquisition module, used for acquiring hyperspectral images;

[0126] A data preprocessing module is used to preprocess the hyperspectral image to obtain a preprocessed image;

[0127] The spectrum, space and frequency domain feature extraction module is used to extract the spectrum space features and frequency domain features from the preprocessed image; the spectrum space features are fused with the frequency domain features extracted from the preprocessed image to obtain the local features of the hyperspectral image;

[0128] The global feature extraction module is used to perform superpixel segmentation on the preprocessed image, regard each superpixel of the image as a node, construct a graph structure, and extract the global features of the graph structure;

[0129] The feature fusion module is used to combine the global features of the graph structure and the local features of the hyperspectral image to obtain the fusion features of the hyperspectral image;

[0130] The classification module is used to determine the classification result of the hyperspectral image according to the fusion characteristics of the hyperspectral image.

[0131] The present invention also discloses a computer device, which includes:

[0132] a processor adapted to execute a computer program;

[0133] A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, a hyperspectral image classification method disclosed in Embodiment 1 is implemented.

[0134] The present invention also discloses a computer-readable storage medium, which stores a computer program. The computer program is suitable for being loaded by a processor and executing a hyperspectral image classification method disclosed in Example 1.

[0135] The present invention also discloses a computer program product, which includes a computer program. When the computer program is executed by a processor, the hyperspectral image classification method disclosed in Example 1 is implemented.

[0136] The method disclosed in Example 1 can be directly embodied as a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0137] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0138] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A hyperspectral image classification method, characterized in that: include: Acquire hyperspectral images; Preprocessing the hyperspectral image to obtain a preprocessed image; Extracting spectral spatial features and frequency domain features from the preprocessed image; The spectral spatial features are fused with the frequency domain features extracted from the preprocessed image to obtain the local features of the hyperspectral image; Perform superpixel segmentation on the preprocessed image, regard each superpixel of the image as a node, build a graph structure, and extract the global features of the graph structure; The global features of the graph structure and the local features of the hyperspectral image are combined to obtain the fusion features of the hyperspectral image; According to the fusion characteristics of the hyperspectral image, the classification result of the hyperspectral image is determined.

2. A hyperspectral image classification method as claimed in claim 1, characterized in that: The process of superpixel segmentation of the preprocessed image includes: Obtain the spectral characteristics and spatial characteristics of each pixel in the preprocessed image; Determine the distance between pixels in the preprocessed image according to the spectral characteristics and spatial characteristics of each pixel in the preprocessed image; The preprocessed image is segmented into superpixels according to the distance between pixels in the preprocessed image.

3. A hyperspectral image classification method as claimed in claim 1, characterized in that: The average value of all pixel features in the superpixel is used as the node feature of the node corresponding to the superpixel, wherein each pixel feature includes the spectral feature and spatial feature of the pixel.

4. A hyperspectral image classification method as claimed in claim 3, characterized in that: Calculate the similarity between nodes based on node features; Build an edge between two nodes whose similarity is greater than a set value.

5. A hyperspectral image classification method as claimed in claim 1, characterized in that: Graph attention network is used to extract global features of graph structure.

6. A hyperspectral image classification method as claimed in claim 1, characterized in that: Spectral features are extracted from the preprocessed image; spatial features are extracted from the spectral features to obtain spectral spatial features.

7. A hyperspectral image classification system, characterized in that: include: An image acquisition module, used for acquiring hyperspectral images; A data preprocessing module is used to preprocess the hyperspectral image to obtain a preprocessed image; Spectral, spatial and frequency domain feature extraction module, used to extract spectral spatial features and frequency domain features from the preprocessed image; The spectral spatial features are fused with the frequency domain features extracted from the preprocessed image to obtain the local features of the hyperspectral image; The global feature extraction module is used to perform superpixel segmentation on the preprocessed image, regard each superpixel of the image as a node, construct a graph structure, and extract the global features of the graph structure; The feature fusion module is used to combine the global features of the graph structure and the local features of the hyperspectral image to obtain the fusion features of the hyperspectral image; The classification module is used to determine the classification result of the hyperspectral image according to the fusion characteristics of the hyperspectral image.

8. An electronic device, characterized in that: The device comprises: a processor adapted to execute a computer program; A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, a hyperspectral image classification method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing a hyperspectral image classification method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method for classifying hyperspectral images according to any one of claims 1 to 6 is implemented.

Images