Classification learning method, system, and computer storage medium
By generating high-quality HSI images using generative adversarial networks and combining them with a discriminator network for iterative training, the problem of insufficient feature extraction in hyperspectral image classification is solved, resulting in higher classification accuracy and clearer boundaries.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHENZHEN RESEARCH INSTITUTE OF NORTHWEST A & F UNIVERSITY
- Filing Date
- 2023-06-16
- Publication Date
- 2026-05-19
AI Technical Summary
Existing hyperspectral image classification methods suffer from insufficient feature extraction, resulting in low classification accuracy. Furthermore, traditional CNN algorithms ignore the spectral and spatial structure of HSI data, failing to maximize the utilization of its information.
A generative adversarial network-based approach is adopted, which generates high-quality HSI images through a generator network, combines them with a discriminator network for iterative training, and uses a two-layer extraction and classification network to extract spectral and spatial information to achieve feature fusion and classification.
It improves the classification accuracy of hyperspectral images, significantly reduces the impact of noise, and generates more accurate classification results with clearer boundaries and higher classification accuracy.
Smart Images
Figure CN116704255B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to a classification learning method, system, and computer storage medium. Background Technology
[0002] Hyperspectral imaging (HSI) is a digital imaging technology that captures a vast amount of continuous spectral information. Traditional color images only contain information in the red, green, and blue bands, while hyperspectral images can simultaneously record spectral data from dozens to hundreds of different bands. In a hyperspectral image, each pixel corresponds to a spectral curve of spectral reflectance value. These curves provide detailed information about the surface material and chemical composition of objects. By analyzing this spectral data, we can identify and distinguish different land features, materials, and environmental characteristics. Hyperspectral imaging has wide applications in many fields, including remote sensing, earth sciences, environmental monitoring, agriculture, and medicine. In remote sensing applications, hyperspectral images can be used for land cover classification, land use and cover analysis, and environmental change detection. The main advantage of hyperspectral imaging is that it provides richer spectral information, capturing more subtle features and changes. However, due to the large volume and complexity of the data, processing and analyzing hyperspectral images also presents certain challenges, requiring appropriate algorithms and techniques to extract useful information.
[0003] Many traditional HSI classification methods, such as the k-nearest neighbor algorithm and support vector machine algorithm, have been proposed in the past, but these methods have low sensitivity to high-dimensional information and are not entirely applicable. Meanwhile, dimensionality reduction and spectral information extraction methods, including principal component analysis, independent component analysis, and linear discriminant analysis, ignore the spatial correlation between pixels in the spatial dimension, failing to fully utilize spatial characteristics.
[0004] While existing CNN (Convolutional Neural Network) classification models have achieved relatively advanced performance, they still have some limitations. These CNN-based algorithms ignore some information in the input HSI (Highly Sequentially Intersecting Indicators) data. CNN is a vector-based method that takes input as a set of pixel vectors. However, HSI data has an ordered structure in the spectral domain. Therefore, CNN algorithms cannot be perfectly applied to HSI data for classification, resulting in insufficient HSI image feature extraction and low HSI image classification accuracy. Summary of the Invention
[0005] This invention provides a classification learning method, system, and computer storage medium to address the problem of insufficient HSI image feature extraction in the prior art, which leads to low HSI image classification accuracy.
[0006] On one hand, embodiments of the present invention provide a classification learning method, including:
[0007] Randomly collect raw HSI from different scenarios;
[0008] The PAN image is obtained by averaging all spectral bands of the original HSI using the Wald protocol.
[0009] The original HSI and the PAN image are input into the generator network to obtain an HQHSI image;
[0010] The discriminator D1 is used to synchronize the spectral information of the HQHSI with the original HSI;
[0011] The discriminator D2 is used to synchronize the spatial structure of the HQHSI with the PAN image;
[0012] By repeatedly iterating the discriminator D1 and the discriminator D2, the highest quality HQHSI can be obtained;
[0013] The highest quality HQHSI is input into a two-layer extraction and classification network to obtain the classification result.
[0014] In one possible implementation, the generator network is used for feature extraction and feature fusion.
[0015] In one possible implementation, the feature extraction is to downsample the original HSI and the PAN image to obtain a feature map, and the feature fusion is to connect the feature maps and add a residual block for fusion.
[0016] In one possible implementation, the discriminator D1 has two downsampled convolutional layers with a stride of 2 and two convolutional layers with a stride of 1, and the discriminator D2 has three downsampled convolutional layers with a stride of 2 and two convolutional layers with a stride of 1.
[0017] In one possible implementation, the highest quality HQHSI is the consistency between different modal images of the same ground feature.
[0018] On the other hand, embodiments of the present invention also provide a classification learning system, including:
[0019] The acquisition module is used to randomly acquire raw HSI data from different scenarios.
[0020] The acquisition module is used to average all spectral bands of the original HSI using the Wald protocol to obtain a PAN image; input the original HSI and the PAN image into the generator network to obtain an HQHSI; apply discriminator D1 to synchronize the spectral information of the HQHSI with the original HSI; apply discriminator D2 to synchronize the spatial structure of the HQHSI with the PAN image; and repeatedly iterate the application of discriminator D1 and discriminator D2 to obtain the highest quality HQHSI.
[0021] The classification module is used to input the highest quality HQHSI into a two-layer extraction and classification network to obtain the classification result.
[0022] In one possible implementation, the acquisition module applies a generative adversarial network to acquire the highest quality HQHSI.
[0023] In one possible implementation, the generator network is the generator part of the generative adversarial network, and the application discriminator D1 and application discriminator D2 are the discriminator parts of the generative adversarial network.
[0024] In one possible implementation, the dual-layer extraction and classification network obtains channel feature maps through two convolutional layers, maps and connects the channel feature maps, and convolves them into a one-dimensional tensor. Then, the one-dimensional tensor is input into a fully connected layer for classification.
[0025] On the other hand, embodiments of the present invention also provide a computer storage medium storing a plurality of computer instructions, which are used to cause a computer to execute the above-described method.
[0026] The classification learning method, system, and computer storage medium of this invention have the following advantages:
[0027] (1) To address the problem of insufficient information in classifying single HSI images, a classification method based on generative adversarial networks is proposed, which utilizes combined spectral spatial information to improve classification accuracy.
[0028] (2) A generator based on a two-stream generative adversarial network architecture is used to generate high-quality feature maps, and two discriminators are used to learn an adaptive loss function to improve the quality of the features.
[0029] (3) By using a two-layer extraction classification network, spectral spatial information can be extracted simultaneously, thereby improving classification accuracy. Attached Figure Description
[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0031] Figure 1 A flowchart of the classification learning method provided in the embodiments of the present invention;
[0032] Figure 2A diagram of a GAN-based two-stream extraction network framework provided in an embodiment of the present invention;
[0033] Figure 3 This is a diagram of a two-stream classification network framework provided in an embodiment of the present invention.
[0034] Figure 4 Example figures showing the classification results of the classification learning method provided in this embodiment of the invention and other algorithms on the Indian Pines dataset;
[0035] Figure 5 Example figures showing the classification results of the classification learning method provided in this embodiment of the invention and other algorithms on the KSC dataset;
[0036] Figure 6 This is a schematic diagram of the classification learning system structure provided in an embodiment of the present invention. Detailed Implementation
[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0038] Figure 1 This is a flowchart illustrating the classification learning method provided in an embodiment of the present invention; the present invention provides a classification learning method, including:
[0039] Randomly collect raw HSI from different scenarios;
[0040] The PAN image is obtained by averaging all spectral bands of the original HSI using the Wald protocol.
[0041] The original HSI and the PAN image are input into the generator network to obtain HQHSI;
[0042] The discriminator D1 is used to synchronize the spectral information of the HQHSI with the original HSI;
[0043] The discriminator D2 is used to synchronize the spatial structure of the HQHSI with the PAN image;
[0044] By repeatedly iterating the discriminator D1 and the discriminator D2, the highest quality HQHSI can be obtained;
[0045] The highest quality HQHS is input into a two-layer extraction and classification network to obtain the classification result.
[0046] For example, the original HSI is the initially acquired hyperspectral image, and the PAN image is a single-channel panchromatic image obtained by averaging the spectral bands of the original HSI. After acquiring the original HSI and PAN images, they are input into the generator network of a generative adversarial network (GAN). The output of the generator network is added to the HQHSI (High-Quality Hyperspectral Image) through a high-pass domain to better preserve spatial spectral information. Then, the HQHSI is iterated through discriminators D1 and D2, which are the discriminators of the GAN, to obtain the highest quality HQHSI. Finally, the HQHSI is input into a two-layer extraction and classification network for classification.
[0047] In one possible embodiment, the generator network is used for feature extraction and feature fusion.
[0048] For example, the generator network has feature extraction and feature fusion functions.
[0049] In one possible embodiment, the feature extraction is to downsample the original HSI and the PAN image to obtain a feature map, and the feature fusion is to connect the feature maps and add a residual block for fusion.
[0050] For example, the feature extraction involves downsampling the input original HSI and PAN images. The feature extraction has two convolutional layers with a stride of 2 for downsampling. After feature extraction, a feature map is obtained. The feature maps obtained by feature extraction are concatenated and a residual block is added to achieve fusion. Then, two consecutive convolutions with a stride of 1 / 2 are applied to upsample the feature map to meet the required HQHSI size. The HQHSI is obtained by adding the high-frequency components of the HSI in the high-pass domain to the upsampled feature map.
[0051] In one possible embodiment, the discriminator D1 has two downsampling convolutional layers with a stride of 2 and two convolutional layers with a stride of 1, and the discriminator D2 has three downsampling convolutional layers with a stride of 2 and two convolutional layers with a stride of 1.
[0052] In one possible embodiment, the highest quality HQHSI is the consistency between different modal images of the same ground feature.
[0053] For example, see Figure 2 and Figure 3As shown, the generator network includes a feature extraction module, a feature fusion module, and an encoder module. The encoder module encodes the input HSI and PAN images into low-dimensional feature vectors. The feature fusion module fuses the feature vectors of both to form a feature map HQHSI. The criterion for the highest quality HQHSI is the consistency between different modal images of the same land cover. Based on this, a statistical similarity measure between the HSI and PAN images can be obtained, which is applicable to the fused features. Therefore, spatial and spectral consistency can be directly calculated from the fused HQHSI, HSI, and PAN images without requiring ground truth values.
[0054] To quantify this, a non-gt loss function is introduced on top of QNR, defined as follows:
[0055]
[0056] QNR is an abbreviation for No-Reference Quality, D λ It is the spectral distortion index, D s It is for D λ Supplemental space quality measurement.
[0057] QNR=(1-D λ (1-D) s (2)
[0058] The ideal QNR value is 1.
[0059]
[0060] Where F represents the fusion result, R represents the HSI input, and B represents the number of frequency bands. i and R i These represent their i-th bands, respectively. Q represents the image QI value, defined as follows.
[0061]
[0062] Where Q(x,y) represents the covariance of x and y. and Let x and y represent the variances, respectively. and These are the average values of x and y, respectively.
[0063]
[0064] Where P is the PAN input, It is a degraded version of HSI. This loss function allows for measuring the quality of the fused image (HQ HSI) of the input PAN image and the HSI.
[0065] Discriminator D1 is used for spectral storage, and discriminator D2 is used for spatial storage.
[0066] The generator learns to retain more spectral information to fool D1, thus distinguishing between real and fake HSI images. Furthermore, the generator learns to retain more spatial details to fool D2, which distinguishes between real PAN images and spectral degradation fusion results. The loss function of the generator is...
[0067]
[0068] Where N is the number of samples. F, R, and P are the HQHSI, HSI, and PAN images, respectively. and Let α and β represent the spatial and spectral degradation of F, respectively. α and β are hyperparameters.
[0069] To achieve stable training, the WGAN-GP framework is used, which utilizes the Wasserstein distance and incorporates a gradient penalty in the discriminator. The loss functions for D1 and D2 are denoted as follows:
[0070]
[0071]
[0072] Where GP is the gradient penalty of the discriminator.
[0073] The loss function used in the two-layer extraction network is the cross-entropy loss function. In deep learning, cross-entropy is often used as a loss function for classification problems. It measures the model's error by measuring the difference between the actual output probability and the expected output probability. The cross-entropy loss function is expressed as follows:
[0074]
[0075] N is the number of samples, and y is the input feature. For the predicted features, i = 1, 2, ..., N
[0076] Figure 1 This is a schematic diagram of the structure of a classification learning system provided in an embodiment of the present invention. The classification learning system includes:
[0077] The acquisition module is used to randomly acquire raw HSI data from different scenarios.
[0078] The acquisition module is used to average all spectral bands of the original HSI using the Wald protocol to obtain a PAN image; input the original HSI and the PAN image into the generator network to obtain an HQHSI; apply discriminator D1 to synchronize the spectral information of the HQHSI with the original HSI; apply discriminator D2 to synchronize the spatial structure of the HQHSI with the PAN image; and repeatedly iterate the application of discriminator D1 and discriminator D2 to obtain the highest quality HQ HSI.
[0079] The classification module is used to input the highest quality HQ HSI into a two-layer extraction and classification network to obtain the classification result.
[0080] In one possible embodiment, the acquisition module applies a generative adversarial network to acquire the highest quality HQHSI.
[0081] In one possible embodiment, the generator network is the generator part of the generative adversarial network, and the application discriminator D1 and application discriminator D2 are the discriminator parts of the generative adversarial network.
[0082] In one possible embodiment, the dual-layer extraction and classification network obtains channel feature maps through two convolutional layers, maps and connects the channel feature maps, and convolves them into a one-dimensional tensor. Then, the one-dimensional tensor is input into a fully connected layer for classification.
[0083] For example, the first convolutional layer of the dual-layer extraction and classification network includes a 1×1 convolutional kernel, a ReLU activation function, and a GroupNorm (group normalization) layer. The number of input channels is the same as the number of channels in the HSI, and the number of output channels is the same as the number of image classes in the HSI. The second convolutional layer of the dual-layer extraction and classification network includes a 1×1 convolutional kernel, a ReLU activation function, a regularization layer, three separable convolutional layers, and a normalization layer. During forward propagation, the input is first processed using xZero software, then a 16-channel feature map is obtained through the first convolutional layer, and a 64-channel feature map is obtained through the second convolutional layer. Finally, the two feature maps are concatenated and convolved into a one-dimensional tensor, which is then input into a fully connected layer for classification.
[0084] In one possible embodiment, such as Figure 4 and Figure 5 As shown, Figure 4 and Figure 5 The results of all methods using the same input image across different categories are shown. The classification map generated by this invention is smoother, more comprehensive, and closer to the real ground map.
[0085] In contrast, maps generated by comparison techniques exhibit significant noise. This noise affects their ability to accurately identify object types, resulting in slightly lower performance than the method of this invention. It is clear from the figures that the method of this invention produces the most accurate and well-defined boundaries, further confirming the high classification performance of my invention. These observations demonstrate that, compared to other methods, the network of this invention effectively preserves detailed information and significantly reduces noise.
[0086] The dataset selection is shown in Table 1. This invention uses two hyperspectral datasets to verify the effectiveness of the proposed classification algorithm. The evaluation index results of the classification results obtained by different methods are shown in Tables 2 and 3. This invention uses four performance indicators to evaluate the effectiveness of the method: classification accuracy (ACC), overall accuracy (OA), average accuracy (AA), and Kappa coefficient (K) for each class. The higher the ACC, OA, AA, and K, the better. As can be seen in Table 3, the method provided by this invention has achieved better image classification results compared with other methods.
[0087] Table 1 Table 2 Classification results of the Indian Pines dataset
[0088]
[0089]
[0090] Table 3 Classification results of the KSC dataset
[0091]
[0092] This invention primarily compares its performance with five widely used HSI classification methods: 2D CNN, 3D CNN, HybridSN, pResNet, and SSAN. Based on the results of the IN dataset shown in Table 2, the OA accuracy of the proposed method is 99.01%, which is 8.38% higher than the 2D CNN method, 6.42% higher than the 3D CNN method, 5.20% higher than the HybridSN method, 4.43% higher than the pResNet method, and 3.70% higher than the SSAN method. Despite a relatively small number of training samples and an imbalanced distribution of samples between classes due to random sampling, the method maintains high accuracy for each class. This demonstrates the effectiveness of the algorithm in classifying small samples. Similar results can be obtained from the KSC dataset in Table 3.
[0093] In summary, this invention is optimal compared to existing advanced methods in terms of visual evaluation and evaluation metrics, thereby improving classification accuracy.
[0094] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.
[0095] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.
Claims
1. A classification learning method, characterized in that, include: Randomly collect raw HSI from different scenarios; The PAN image is obtained by averaging all spectral bands of the original HSI using the Wald protocol. The original HSI and the PAN image are input into the generator network to obtain HQHSI; The discriminator D1 is used to synchronize the spectral information of the HQHSI with the original HSI; The discriminator D2 is used to synchronize the spatial structure of the HQHSI with the PAN image; By repeatedly iterating the discriminator D1 and the discriminator D2, the highest quality HQHSI can be obtained; The highest quality HQHSI is input into a two-layer extraction and classification network to obtain the classification result; The generator network is used for feature extraction and feature fusion; The feature extraction is to downsample the original HSI image and the PAN image to obtain feature maps, and the feature fusion is to connect the feature maps and add a residual block for fusion. The discriminator D1 has two downsampling convolutional layers with a stride of 2 and two convolutional layers with a stride of 1, and the discriminator D2 has three downsampling convolutional layers with a stride of 2 and two convolutional layers with a stride of 1. The highest quality HQHSI image is one that exhibits consistency across different modal images of the same ground feature.
2. A classification learning system, said system being used to implement the method of claim 1, characterized in that, include: The acquisition module is used to randomly acquire raw HSI images from different scenarios; The acquisition module is used to average all spectral bands of the original HSI image using the Wald protocol to obtain a PAN image; and inputs the original HSI image and the PAN image into the generator network to obtain an HQHSI image. The discriminator D1 is used to synchronize the spectral information of the HQHSI image with the original HSI; the discriminator D2 is used to synchronize the spatial structure of the HQHSI image with the PAN image; the discriminator D1 and the discriminator D2 are repeatedly applied iteratively to obtain the highest quality HQHSI image; The classification module is used to input the highest quality HQHSI image into a two-layer extraction and classification network to obtain the classification result.
3. The classification learning system according to claim 2, characterized in that, The acquisition module uses a generative adversarial network to acquire the highest quality HQHSI image.
4. The classification learning system according to claim 3, characterized in that, The generator network is the generator part of the generative adversarial network, and the discriminator D1 and discriminator D2 are the discriminator parts of the generative adversarial network.
5. A computer storage medium, characterized in that, The computer storage medium stores a plurality of computer instructions, which are used to cause the computer to execute the method of claim 1.