Superpixel fusion method based on spectrum induced alignment

By employing a superpixel fusion method based on spectral-induced alignment, and utilizing deeply abstracted spectral features and a min-max fusion strategy, the problem of insufficient information utilization in existing technologies is solved, thereby improving the classification accuracy and algorithm performance of hyperspectral images.

CN119027763APending Publication Date: 2024-11-26HOHAI UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310586574.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-23
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

Existing superpixel segmentation and feature fusion methods cannot effectively utilize the rich spectral information in hyperspectral image classification, resulting in information loss, affecting classification performance, and lacking effective fusion methods.

Method used

A superpixel fusion method based on spectral-induced alignment is adopted. Superpixel segmentation is performed through DAE with deep abstraction of spectral features. The minimum fusion and maximum fusion strategies are combined to improve the accuracy of feature representation and classification.

Benefits of technology

It improves the accuracy and algorithm performance of hyperspectral image classification. By fusing deep abstract spectral features and original features, it enhances the discrimination ability of superpixels and achieves more refined classification results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027763A_ABST
    Figure CN119027763A_ABST
Patent Text Reader

Abstract

The invention discloses a superpixel fusion method based on spectrum-induced alignment, and belongs to the field of artificial intelligence. The method comprises the steps that superpixel segmentation is carried out on a hyperspectral (HS) image with original and depth abstract spectral features at the same time, a superpixel map is obtained, the depth abstract spectral features are excavated by a depth auto-encoder (DAE), and the depth abstract spectral features are extracted by a depth auto-encoder (DAE); constructing two fusion strategies, and performing minimum fusion and maximum fusion on the segmented superpixel images; and classifying the targets in the hyperspectral image according to the fused features. By adopting the method, the utilization rate of the features and the accuracy of super-pixel segmentation can be improved, so that the model performance is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a superpixel fusion method based on spectral induced alignment. Background Art

[0002] In recent years, with the rapid development of computer hardware, more and more image classification algorithms based on convolutional neural networks have been proposed. Today, image classification has been widely used in military, remote sensing, medical, social services and other fields. Hyperspectral (HS) images are usually composed of hundreds of bands and have rich spectral information and spatial information of ground objects, which makes it possible to distinguish different ground objects on the earth's surface.

[0003] HS images have a large number of pixels, and it is difficult for Graph Convolution Network (GCN) to directly process the entire image. In order to improve this problem, some researchers have used superpixel segmentation technology to cluster adjacent pixels that are likely to belong to the same land cover type into a pixel block. These pixel blocks are composed of areas with strong spectral-spatial similarity. Networks using superpixel segmentation technology include Dual Interactive Graph Convolutional Networks (DIGCN), Multiscale Graph Sample and Aggregate Network With Context-Aware Learning for Hyperspectral Image Classification (MSAGE-CAL), etc., which are mainstream applications of superpixel segmentation technology on HS images. These networks mainly act on the original HS images, by reducing the number of redundant pixels and increasing the diversity of network input information to achieve better algorithm performance. In addition, during network training, backpropagation can be used to feed back errors to the learning network, readjust the network coefficients, and reconstruct the graph for different classification targets to improve image classification accuracy.

[0004] However, mainstream networks such as DIGCN and MSAGE-CAL rely heavily on the original HS image. Once the original HS cannot be well adapted to the superpixel segmentation technology, it will have a fatal impact on the segmentation result. In addition, these mainstream networks cannot effectively utilize rich spectral information, resulting in important information loss and affecting the image classification performance. Moreover, these mainstream networks do not have an effective fusion method. Summary of the invention

[0005] Purpose of the invention: In order to overcome the shortcomings of existing superpixel segmentation and feature fusion methods, a superpixel fusion method based on spectral induced alignment is provided, which can effectively solve the shortcomings of not being able to provide sufficient discriminative feature representation and not being able to accurately obtain superpixels for constructing the initial image during image classification, and thus improve the algorithm performance.

[0006] Technical solution: To achieve the above purpose, the present invention provides a superpixel fusion method based on spectral induced alignment, comprising the following steps:

[0007] S1: Perform superpixel segmentation on the original HS image to obtain a superpixel map. Use DAE to mine deep abstract spectral features, perform superpixel segmentation on the HS image with deep abstract spectral features, and obtain a superpixel map;

[0008] S2: Construct two fusion strategies to perform minimum fusion and maximum fusion on the segmented superpixel images;

[0009] S3: Classify the objects in the HS image based on the fused features;

[0010] Furthermore, in step S1, the HS image with original and deep abstract spectral features is used to obtain a superpixel segmentation result, and the method specifically includes:

[0011] A1: For image segmentation, according to SLIC, it segments the HS image I of the original and deep abstracted spectral features into M superpixels;

[0012] A2: Deep abstract spectral features are mined by DAE, which consists of two deep convolutional neural networks, an encoder and a decoder, namely: Z = DAE Enc(X) ,

[0013] Here, XX^ represents the original features and the reconstructed features, respectively, and Z represents the extracted deep abstract spectral features.

[0014] Furthermore, in step S2, two fusion strategies are used to perform minimum fusion and maximum fusion on the segmented results. The two fusion formulas are as follows:

[0015] B1: Naturally apply SLIC on Z to obtain the superpixel segmentation result M1 based on abstract spectral features;

[0016] B2: Perform SLIC on the original HS image to obtain the superpixel result M2;

[0017] B3: The proposed superpixel fusion strategy is adopted on M1 and M2 to produce more refined superpixel results. The formulas of the two fusion strategies are:

[0018]

[0019]

[0020] Here, the minimum fusion (M Min ) result is determined by the number of pixels in a superpixel, while the maximum fusion (M Max ) is similar to finding the intersection of two superpixel results.

[0021] Furthermore, the DAE designed in step A2 for mining deep abstract spectral features consists of two deep convolutional neural networks, an encoder and a decoder. The encoder composed of the deep convolutional neural network extracts potential features through multiple 1D convolutions, and the decoder composed of the deep convolutional neural network reconstructs the input through 1D transposed convolutions. The training process of DAE is to use MSE X^ X

[0022] BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 A schematic diagram of a method flow chart provided for an example of the present invention;

[0025] Figure 2 A schematic diagram of a superpixel fusion structure based on spectral induced alignment provided by an example of the present invention;

[0026] Figure 3 Schematic diagram of two superpixel fusion strategies provided for examples of the present invention;

[0027] Figure 4 This is a hyperspectral remote sensing image data diagram of Indian Pines and Pavia University in the simulation experiment of the present invention;

[0028] Figure 5 This is a pseudo-color labeled image of the Indian Pines and Pavia University hyperspectral remote sensing image data from the simulation experiment of the present invention;

[0029] Figure 6 The classification result diagram of the Indian Pines and Pavia University hyperspectral remote sensing image data obtained by the method of the present invention and other methods in the simulation experiment of the present invention; DETAILED DESCRIPTION

[0030] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0031] like Figure 1 As shown, the present invention provides a superpixel fusion method based on spectral induced alignment, comprising the following steps:

[0032] S1: Perform superpixel segmentation on the original HS image to obtain a superpixel map. Use DAE to mine deep abstract spectral features, perform superpixel segmentation on the HS image with deep abstract spectral features, and obtain a superpixel map;

[0033] S2: Construct two fusion strategies to perform minimum fusion and maximum fusion on the segmented superpixel images;

[0034] S3: Classify the objects in the HS image based on the fused features;

[0035] In step S1 of this embodiment, the HS image with original and deep abstract spectral features is used to obtain the superpixel segmentation result. The method specifically includes:

[0036] A1: For image segmentation, according to SLIC, it segments the HS image I of the original and deep abstracted spectral features into M superpixels;

[0037] A2: Deep abstract spectral features are mined by DAE, which consists of two deep convolutional neural networks, an encoder and a decoder, namely: Z = DAE Enc(X) ,

[0038] Here, XX^ represents the original features and the reconstructed features, respectively, and Z represents the extracted deep abstract spectral features.

[0039] In this embodiment, the two fusion strategies in step S2 perform minimum fusion and maximum fusion on the segmented results. The two fusion formulas are as follows:

[0040] B1: Naturally apply SLIC on Z to obtain the superpixel segmentation result M1 based on abstract spectral features;

[0041] B2: Perform SLIC on the original HS image to obtain the superpixel result M2;

[0042] B3: The proposed superpixel fusion strategy is adopted on M1 and M2 to produce more refined superpixel results. The formulas of the two fusion strategies are:

[0043]

[0044]

[0045] Here, the minimum fusion (M Min ) result is determined by the number of pixels in a superpixel, while the maximum fusion (M Max ) is similar to finding the intersection of two superpixel results.

[0046] In step A2 of this embodiment, the DAE designed for mining deep abstract spectral features consists of two deep convolutional neural networks, an encoder and a decoder. The encoder composed of the deep convolutional neural network extracts potential features through multiple 1D convolutions, and the decoder composed of the deep convolutional neural network reconstructs the input through 1D transposed convolutions. The training process of DAE is to use MSE X^ X

[0047]

[0048] Based on the above scheme, the effect of the method of the present invention is verified by ablation comparison experiments. The method of the present invention is verified using the Indian Pines and Pavia University datasets. Among them, 30 samples of each category in the hyperspectral image data Indian Pines and Pavia University datasets are used as training sets. For categories with less than 30 samples, 5 samples of each category are used as training sets, and the remaining samples are used as test sets for simulation experiments; superpixel segmentation based on the original HS image features (ablation method 1), superpixel segmentation based on HS images with deep abstract spectral features (ablation method 2), superpixel fusion based on spectral induced alignment and using the maximum fusion method to replace the spectral spatial graph convolutional network (S2GCN) (replacement comparison method 1), superpixel fusion based on spectral induced alignment and using the maximum fusion method to replace MSAGE-CAL (replacement comparison method 2), based on the spectral spatial graph convolutional network (S 2 GCN) (comparison method 1), MSAGE-CAL (comparison method 2), superpixel fusion based on spectral induced alignment and using the minimum fusion method (method 1 of the present invention), superpixel fusion based on spectral induced alignment and using the maximum fusion method (method 2 of the present invention) are used for ablation comparison experiments to highlight the effectiveness of the method of the present invention. Among them, the replacement method is only verified using Indian Pines. The method of the present invention uses the Pytorch deep learning framework, and the experimental parameter settings are shown in Table 1:

[0049] Table 1 Experimental parameter settings

[0050] Dataset Initial number of superpixels Number of deep autoencoder layers Indian Pines 1800 3 Pavia University 8000 3

[0051] Implementation Results

[0052] In this embodiment, the overall classification accuracy (OA), average classification accuracy (AA) and Kappa coefficient are used. Table 2 and Table 3 respectively show the classification accuracy comparison between the method of the present invention and other ablation comparison methods. Figure 4-5 They are the Indian Pines and Pavia University hyperspectral remote sensing image data and the corresponding pseudo-color labeled images used in the experiment. Figure 4 Applying the above method, we get Figure 6 The classification results of Indian Pines and Pavia University hyperspectral remote sensing images are shown.

[0053] Table 2 OA, AA, Kappa indexes of the method of the present invention and the ablation method

[0054]

[0055] Table 3 OA, AA, Kappa indexes of the network replaced by the method of the present invention and the original comparison method

[0056]

[0057] According to the data in Table 2, the evaluation indicators of the method of the present invention are better than those of the ablation method, which shows that the classification effect of the method of the present invention is better, reflecting the effectiveness of the method of the present invention. According to the data in Table 3, the evaluation indicators of the network replaced by the method of the present invention are better than those of the original comparison method, which shows that the classification effect of the network replaced by the method of the present invention is better, which indirectly shows the superiority of the method of the present invention. It should be noted that both the minimum fusion method and the maximum fusion method have achieved satisfactory classification performance. In addition, Figure 6 It can be seen from the classification result graph that the classification accuracy is improved by fusing the superpixel results of the original image and the image with latent features, thereby verifying the effect of the method of the present invention. The effectiveness of the method of the present invention can be further illustrated by observing that the classification result graph obtained by replacing the comparison method with the method of the present invention is better than the classification result graph obtained by the original comparison method.

Claims

1. A superpixel fusion method based on spectral induced alignment, characterized in that: The steps include: S1: Perform superpixel segmentation on the original hyperspectral (HS) image to obtain a superpixel map. Use deep auto encoders (DAE) to mine deep abstract spectral features, perform superpixel segmentation on the HS image with deep abstract spectral features, and obtain a superpixel map; S2: Construct two fusion strategies to perform minimum fusion and maximum fusion on the segmented superpixel images; S3: Classify the objects in the HS image based on the fused features.

2. The superpixel fusion method based on spectral induced alignment according to claim 1, characterized in that: In step S1, the HS image with original and deep abstract spectral features is used to obtain a superpixel segmentation result. The method specifically includes: A1: For image segmentation, according to Simple Linear Iterative Clustering (SLIC), it divides the HS image I of the original and deep abstracted spectral features into M superpixels; A2: Deep abstract spectral features are mined by DAE, which consists of two deep convolutional neural networks, an encoder and a decoder, namely: Z = DAE Enc(X) , Here, XX` represents the original features and the reconstructed features, respectively, and Z represents the extracted deep abstract spectral features.

3. The superpixel fusion method based on spectral induced alignment according to claim 1, characterized in that: The two fusion strategies in step S2 perform minimum fusion and maximum fusion on the segmented results. The two fusion formulas are as follows: B1: Naturally apply SLIC on Z to obtain the superpixel segmentation result M1 based on abstract spectral features; B2: Perform SLIC on the original HS image to obtain the superpixel result M2; B3: The proposed superpixel fusion strategy is adopted on M1 and M2 to produce more refined superpixel results. The formulas of the two fusion strategies are: Here, the minimum fusion (M Min ) result is determined by the number of pixels in a superpixel, while the maximum fusion (M Max ) is similar to finding the intersection of two superpixel results.

4. The superpixel fusion method based on spectral induced alignment according to claim 2, characterized in that: The DAE designed in step A2 for mining deep abstract spectral features consists of two deep convolutional neural networks, an encoder and a decoder. The encoder composed of the deep convolutional neural network extracts potential features through multiple 1D convolutions, and the decoder composed of the deep convolutional neural network extracts potential features through 1D DAE Mean-Squared Loss MSE X`X