A hyperspectral image classification method
By using FPCA and 3D convolutional kernels to jointly reduce the dimensionality of hyperspectral image data, and combining neighborhood pixel blocks of different scales and attention mechanisms, the problem of spectral and spatial features being ignored in hyperspectral image classification is solved, achieving higher classification accuracy and application effect.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHONGQING TECH & BUSINESS UNIV
- Filing Date
- 2022-10-24
- Publication Date
- 2026-05-22
AI Technical Summary
Existing convolutional neural network models ignore the implicit spatial spectral information between spectral features and spatial features in hyperspectral image classification, leading to inaccurate classification.
We employ FPCA and 1*1*d 3D convolutional kernels to perform joint dimensionality reduction on hyperspectral image data. We combine neighborhood pixel blocks of different scales and use multiple 3DCNN layers, convolutional block attention weight modules, and fully connected layers for classification. We calculate channel attention mapping of spatial spectral features through an attention mechanism.
Effective preservation and utilization of the spatial spectral information of hyperspectral images improves classification accuracy, particularly in agricultural and military applications, enhancing productivity and target reconnaissance accuracy.
Smart Images
Figure CN115457336B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image classification, specifically relating to a hyperspectral image classification method. Background Technology
[0002] Hyperspectral remote sensing technology is a type of remote sensing technology that can continuously image ground features using an imaging spectrometer. It has wide applications in fields such as urban planning, environmental monitoring, agricultural production, and military construction. Therefore, the identification and classification of hyperspectral images is of great research significance.
[0003] In recent years, with the rapid development of deep learning technology, its application to hyperspectral image classification has become a common technique. Current methods for classifying hyperspectral images primarily employ convolutional neural networks (CNNs) to extract spectral and spatial features, thereby determining the image's category based on these features. However, most CNN models neglect the implicit spatial-spectral information between spectral and spatial features, leading to inaccurate hyperspectral image classification. Summary of the Invention
[0004] To address the problem that existing hyperspectral image classification methods neglect the implicit spatial-spectral information between spectral and spatial features, leading to inaccurate classification, this invention provides a hyperspectral image classification method, comprising the following steps:
[0005] S1: The hyperspectral image data is subjected to joint dimensionality reduction processing using FPCA and 1*1*d three-dimensional convolution kernels to obtain the final dimensionality-reduced hyperspectral image data;
[0006] S2: Take any pixel in the final dimensionality-reduced hyperspectral image data as the center point and take neighborhoods of different scales along all spectral dimensions to obtain neighborhood pixel blocks of different scales.
[0007] S3: Classify the hyperspectral image using a trained hyperspectral image classification model based on neighborhood pixel blocks of different scales; output the classification result; wherein, the hyperspectral image classification model includes: multiple 3DCNN layers of different scales, convolutional block attention weight modules, and fully connected layers;
[0008] The training process of the hyperspectral image classification model includes:
[0009] Step 1: Obtain raw hyperspectral image data with label information, wherein the label information is the category of the raw hyperspectral image data;
[0010] Step 2: Obtain original neighborhood pixel blocks at different scales based on the original hyperspectral image data;
[0011] Step 3: Select a 3DCNN layer of the corresponding scale according to the scale of the original neighboring pixel block to extract features from the original neighboring pixel block and obtain the second spatial spectral features of the original neighboring pixel block.
[0012] Step 4: Perform feature stitching along the spectral dimension on the second spatial spectral features of the original neighboring pixel blocks at different scales to obtain the first spatial spectral feature map;
[0013] Step 5: Input the first spatial spectral feature map into the attention weight module of the convolutional block and use the attention mechanism to calculate the channel attention mapping of the first spatial spectral feature map;
[0014] Step 6: Calculate the second spatial spectral feature map based on the channel attention mapping of the first spatial spectral feature map and the first spatial spectral feature map;
[0015] Step 7: Input the second spatial spectral feature map into the fully connected layer for classification; obtain the classification results of the original hyperspectral image data;
[0016] Step 8: Construct the loss function of the hyperspectral image classification model based on the classification results and label information of the original hyperspectral image data. Adjust the parameters of the hyperspectral image classification model through backpropagation to complete the training of the hyperspectral image classification model.
[0017] The present invention has at least the following beneficial effects:
[0018] This invention performs joint dimensionality reduction processing on hyperspectral image data using FPCA and 1*1*d three-dimensional convolutional kernels, which can simultaneously preserve the spectral and spatial features of the hyperspectral image data. By stitching together the second spatial spectral features of neighboring pixel blocks at different scales, the resulting spatial spectral feature map can completely preserve the spatial spectral information of the hyperspectral image at different scales. The channel attention mapping of the spatial spectral feature map is calculated through the convolutional block attention weight module, and the calculated second spatial spectral feature map can reflect the importance of the spatial spectral information of the hyperspectral image at different scales. By classifying the spatial spectral feature map with weighted information through a fully connected layer, the accuracy of spectral image data classification is improved. When applied to agriculture, it ensures agricultural production efficiency; when applied to military industry, it improves the accuracy of military target reconnaissance, and so on. Attached Figure Description
[0019] Figure 1 This is a flowchart of the method of the present invention;
[0020] Figure 2 This is a flowchart of the classification model training process of the present invention;
[0021] Figure 3 This is a flowchart of the dimensionality reduction process of the present invention;
[0022] Figure 4 This is a block diagram of the model structure of the present invention;
[0023] Figure 5 This is a schematic diagram of the structure of the convolutional block attention weight module of the present invention. Detailed Implementation
[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0025] Please see Figure 1 This embodiment provides a hyperspectral image classification method, which includes, but is not limited to, the following steps:
[0026] S1: The hyperspectral image data is jointly dimensionality-reduced using FPCA and a 1*1*d 3D convolutional kernel to obtain the final dimensionality-reduced hyperspectral image data. Specifically, Folded Principal Component Analysis (FPCA) and the 1*1*d 3D convolutional kernel are combined in parallel to reduce the dimensionality of the hyperspectral image. The dimensionality reduction strategy is as follows: Figure 3 As shown, it comprises two branches, an upper branch and an lower branch. The upper branch uses FPCA to reduce the number of spectral dimensions in the hyperspectral image. This method reassigns the bands of each pixel into a matrix form. Based on this, the covariance matrix of the hyperspectral image data can be directly determined for subsequent eigenvalue decomposition and data projection. This not only improves data storage efficiency but also reduces computational complexity, making it suitable for high-dimensional data such as hyperspectral images.
[0027] Please see Figure 3 Preferably, the step of using FPCA and a 1*1*d three-dimensional convolution kernel to perform joint dimensionality reduction processing on the hyperspectral image data to obtain the final dimensionality-reduced hyperspectral image data includes:
[0028] S11: FPCA is used to reduce the dimensionality of the hyperspectral image data to obtain the first dimensionality-reduced hyperspectral image data. Where M is the width of the hyperspectral image, N is the height of the hyperspectral image, and L1 is the number of spectral bands in the first-dimensional reduced hyperspectral image data; in the hyperspectral image data I∈R M*N*D Applying FPCA to the spectral dimension reduces the number of spectral bands from D to L1 while maintaining the same spatial dimensions (i.e., width M and height N). This reduces only the number of spectral bands while preserving spatial information.
[0029] S12: Use a 1*1*d three-dimensional convolution kernel to reduce the dimensionality of the hyperspectral image data to obtain the second-dimensional hyperspectral image data. Where M is the width of the hyperspectral image, N is the height of the hyperspectral image, and L2 is the number of spectral bands in the second-dimensional reduced hyperspectral image data; since adjacent pixels are not calculated, this operation is essentially a one-dimensional convolution of the spectral dimension. Considering that neural networks have gradient descent and backpropagation mechanisms, using one-dimensional convolution to reduce dimensionality can fully explore the relevant information between channels in the hyperspectral image.
[0030] S13: Perform feature fusion on the first and second dimension-reduced hyperspectral image data to obtain the final dimension-reduced hyperspectral image data X∈R. M*N*B Where M is the width of the hyperspectral image, N is the height of the hyperspectral image, and B = L1 + L2 is the number of spectral bands in the final dimensionality-reduced hyperspectral image data; by performing joint dimensionality reduction processing on the hyperspectral image data using FPCA and a 1*1*d three-dimensional convolution kernel, the spectral and spatial features of the hyperspectral image data can be preserved simultaneously, where d represents the number of channels.
[0031] S2: Using any pixel in the final dimensionality-reduced hyperspectral image data as the center point, take neighborhoods of different sizes along all spectral dimensions to obtain neighborhood pixel blocks of different sizes; for example, for any spatial location (a, b) in the hyperspectral image, take neighborhoods of three different sizes along all spectral dimensions with that center point to obtain neighborhood pixel blocks of different sizes corresponding to that center point. The neighborhood pixel block sizes used in this invention are 25*25*30, 27*27*30, and 29*29*30 respectively, and the edge pixels are filled by mirror copying.
[0032] Please see Figure 4 S3: Classify the hyperspectral image using a trained hyperspectral image classification model based on neighborhood pixel blocks of different scales; output the classification result; wherein, the hyperspectral image classification model includes: multiple 3DCNN layers of different scales, convolutional block attention weight module, and fully connected layer;
[0033] Please see Figure 2 Preferably, the training process of the hyperspectral image classification model includes:
[0034] Step 1: Obtain raw hyperspectral image data with label information, where the label information is the category of the raw hyperspectral image data; for example, classify the hyperspectral image according to the category of land cover in the hyperspectral image data. In this design, the raw hyperspectral image data is the IndianPines dataset, and the raw hyperspectral image data in the IndianPines dataset is divided into 16 categories.
[0035] Step 2: Obtain original neighborhood pixel blocks of different scales based on the original hyperspectral image data; this is the same as the step of obtaining neighborhood pixel blocks in step S2 above.
[0036] Step 3: Select a 3DCNN layer of the corresponding scale according to the scale of the original neighboring pixel block to extract features from the original neighboring pixel block and obtain the second spatial spectral features of the original neighboring pixel block.
[0037] Each 3DCNN layer includes a first convolutional kernel and a second convolutional kernel; the first convolutional kernel of 3DCNN layers of different scales are three-dimensional convolutional kernels of different scales; the second convolutional kernel of 3DCNN layers of different scales are three-dimensional convolutional kernels of the same scale.
[0038] The step of selecting a 3DCNN layer of the corresponding scale based on the scale of the original neighboring pixel block to extract features from the original neighboring pixel block, and obtaining the second spatial spectral features of the original neighboring pixel block, includes:
[0039] S31: Based on the scale of the original neighboring pixel block, the first spatial spectral features of the original neighboring pixel block are extracted using the first convolution kernel of the corresponding scale.
[0040] S32: Extract the second spatial spectral features of the original neighboring pixel blocks by using the second convolution kernel to extract the first spatial spectral features of the original neighboring pixel blocks.
[0041] Step 4: Concatenate the second spatial spectral features of the original neighboring pixel blocks at different scales along the spectral dimension to obtain the first spatial spectral feature map F∈R. L*L*C For example, in this invention, a 3DCNN layer with a first convolutional kernel of (1*1*1, 16) and a second convolutional kernel of (1*1*30) is used to extract features from a 25*25*30 neighborhood pixel block; a 3DCNN layer with a first convolutional kernel of (3*3*1, 32) and a second convolutional kernel of (1*1*30) is used to extract features from a 27*27*30 neighborhood pixel block; a 3DCNN layer with a first convolutional kernel of (5*5*1, 64) and a second convolutional kernel of (1*1*30) is used to extract features from a 27*27*30 neighborhood pixel block; and a 3DCNN layer with a first convolutional kernel of (5*5*1, 64) and a second convolutional kernel of (1*1*30) is used to extract features from a 25*25*30 neighborhood pixel block. A 3DCNN layer of size 1*30 extracts features from a neighborhood pixel block of size 29*29*30. In this invention, the specific parameters of the convolution kernel can be adjusted according to the scale of the neighborhood pixel block. For a neighborhood pixel block of size A*B*C, an n*n*m convolution kernel is used as the first convolution kernel; then for a neighborhood pixel block of size (A+2)*(B+2)*(C+2), a (n+2)*(n+2)*(m*2) convolution kernel is used as the first convolution kernel.
[0042] Step 5: Input the first spatial spectral feature map F into the convolutional block attention weight module and use the attention mechanism to calculate the channel attention mapping of the first spatial spectral feature map F;
[0043] Please see Figure 5 The convolutional block attention weight module includes: a global max pooling layer, a global average pooling layer, a sigmoid activation function, and a 1*1*k one-dimensional convolutional kernel;
[0044] The specific steps for calculating the channel attention mapping of the first spatial spectral feature map F include:
[0045] S51: The global max pooling feature F′ of the first spatial spectral feature map F is calculated by passing the first spatial spectral feature map F through the global max pooling layer and the global average pooling layer, respectively. max ∈R 1*1*C and global average pooling feature F′ avg ∈R 1 *1*C ;
[0046] F′ max =Maxpool(F)
[0047] F′ avg =Avgpool(F)
[0048] S52: Transpose the global max pooling feature and multiply it by the global average pooling feature to obtain the second-order statistical feature f∈R. 1*C*C By performing bilinear integration on the global average pooling features and the global max pooling features, second-order interactive information is obtained, enhancing the network's ability to represent subtle details.
[0049] f = F′ avg ×F′ max T
[0050] S53: The second-order aggregated feature F″∈R is obtained by averaging along the column dimensions of the second-order statistical feature. 1*1*C The i-th element F″ i The calculation formula is as follows:
[0051]
[0052] S54: The second-order aggregated features are convolved with a 1*1*k one-dimensional convolution kernel and then input into the Sigmoid activation function to calculate the channel attention mapping of the first spatial spectral feature map.
[0053] α=W1F″+b1
[0054] Where W1 represents the weight parameters of the one-dimensional convolution kernel, and b1 represents the bias parameters of the one-dimensional convolution kernel. The value represents the size of the one-dimensional convolution kernel, where a and b are constants, and C is the number of channels in the first spatial spectral feature map.
[0055] Step 6: Calculate the second spatial spectral feature map based on the channel attention mapping of the first spatial spectral feature map and the first spatial spectral feature map;
[0056] The channel attention mapping of the second spatial spectral feature map is added pixel by pixel to the second spatial spectral feature map.
[0057] Step 7: Input the second spatial spectral feature map into the fully connected layer for classification; obtain the classification results of the original hyperspectral image data;
[0058] The second spatial spectral feature map is input into the Flatten layer for global pooling. Then, the resulting one-dimensional vector is sequentially input into two fully connected layers with 256 and 128 nodes respectively. To prevent overfitting, Dropout with a ratio of 0.4 is used in both fully connected layers. Finally, the number of nodes in the Softmax classification layer is the same as the number of categories in the original hyperspectral image data. In this design, the original hyperspectral image data is the IndianPines dataset, so the number of nodes in the Softmax classification layer is set to 16.
[0059] Step 8: Construct the loss function of the hyperspectral image classification model based on the classification results and label information of the original hyperspectral image data. Adjust the parameters of the hyperspectral image classification model through backpropagation to complete the training of the hyperspectral image classification model.
[0060] Preferably, the loss function of the hyperspectral image classification model includes:
[0061]
[0062] Where y = [y1,..y i .,y C ] represents the label information of the original hyperspectral image data. When the sample belongs to the i-th class, y i =1, otherwise y i =0; P i represents the probability that the original hyperspectral image data belongs to class i; C represents the total number of classes in the original hyperspectral image data.
[0063] In a specific embodiment, the loss function error can be less than 0.01 to complete the training of the model. The hyperspectral image classification method of the present invention is designed as a computer program and stored in smart devices such as mobile phones, computers, tablets, and servers. Hyperspectral image data is acquired in real time through smart devices such as mobile phones, computers, tablets, and servers, and the computer program is run to realize the real-time classification or offline classification function of hyperspectral image data.
[0064] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. A hyperspectral image classification method, characterized in that, Includes the following steps: S1: Utilizing FPCA and The three-dimensional convolutional kernels are used to perform joint dimensionality reduction processing on the hyperspectral image data to obtain the final dimensionality-reduced hyperspectral image data; The use of FPCA and The three-dimensional convolution kernels used for joint dimensionality reduction of hyperspectral image data include: S11: FPCA is used to reduce the dimensionality of the hyperspectral image data to obtain the first dimensionality-reduced hyperspectral image data. Where M is the width of the hyperspectral image and N is the height of the hyperspectral image. This represents the number of spectral bands in the first-dimensional reduced hyperspectral image data; S12: Adopt The three-dimensional convolutional kernel is used to reduce the dimensionality of the hyperspectral image data, resulting in a second-dimensional reduced hyperspectral image data. ∈ Where M is the width of the hyperspectral image and N is the height of the hyperspectral image. This represents the number of spectral bands in the second-dimensional reduced hyperspectral image data; S13: Perform feature fusion on the first and second dimension-reduced hyperspectral image data to obtain the final dimension-reduced hyperspectral image data. Where M is the width of the hyperspectral image and N is the height of the hyperspectral image. This refers to the number of spectral bands in the final dimensionality-reduced hyperspectral image data. S2: Take any pixel in the final dimensionality-reduced hyperspectral image data as the center point and take neighborhoods of different scales along all spectral dimensions to obtain neighborhood pixel blocks of different scales. Step S2 includes: taking any spatial location (a, b) of the hyperspectral image as the center point and taking a neighborhood of three different scales along all spectral dimensions to obtain the neighborhood pixel blocks of different scales corresponding to the center point. For any spatial location (a, b) in the hyperspectral image, take a neighborhood of three different scales along all spectral dimensions with that center point to obtain the neighborhood pixel blocks of different scales corresponding to that center point. S3: Classify the hyperspectral image using a trained hyperspectral image classification model based on neighborhood pixel blocks of different scales; output the classification result; wherein, the hyperspectral image classification model includes: multiple 3DCNN layers of different scales, convolutional block attention weight modules, and fully connected layers; The training process of the hyperspectral image classification model includes: Step 1: Obtain raw hyperspectral image data with label information, wherein the label information is the category of the raw hyperspectral image data; Step 2: Obtain original neighborhood pixel blocks at different scales based on the original hyperspectral image data; Step 3: Select a 3DCNN layer of the corresponding scale according to the scale of the original neighboring pixel block to extract features from the original neighboring pixel block and obtain the second spatial spectral features of the original neighboring pixel block. Step 4: Perform feature stitching along the spectral dimension on the second spatial spectral features of the original neighboring pixel blocks at different scales to obtain the first spatial spectral feature map; Step 5: Input the first spatial spectral feature map into the attention weight module of the convolutional block and use the attention mechanism to calculate the channel attention mapping of the first spatial spectral feature map; Step 6: Calculate the second spatial spectral feature map based on the channel attention mapping of the first spatial spectral feature map and the first spatial spectral feature map; Step 7: Input the second spatial spectral feature map into the fully connected layer for classification; obtain the classification results of the original hyperspectral image data; Step 8: Construct the loss function of the hyperspectral image classification model based on the classification results and label information of the original hyperspectral image data. Adjust the parameters of the hyperspectral image classification model through backpropagation to complete the training of the hyperspectral image classification model.
2. The hyperspectral image classification method according to claim 1, characterized in that, Each 3DCNN layer includes a first convolutional kernel and a second convolutional kernel; the first convolutional kernel of 3DCNN layers of different scales are three-dimensional convolutional kernels of different scales; the second convolutional kernel of 3DCNN layers of different scales are three-dimensional convolutional kernels of the same scale. The step of selecting a 3DCNN layer of the corresponding scale based on the scale of the original neighboring pixel block to extract features from the original neighboring pixel block, and obtaining the second spatial spectral features of the original neighboring pixel block, includes: S31: Based on the scale of the original neighboring pixel block, the first spatial spectral features of the original neighboring pixel block are extracted using the first convolution kernel of the corresponding scale. S32: Extract the second spatial spectral features of the original neighboring pixel blocks by using the second convolution kernel to extract the first spatial spectral features of the original neighboring pixel blocks.
3. The hyperspectral image classification method according to claim 1, characterized in that, The convolutional block attention weight module includes: a global max pooling layer, a global average pooling layer, a Sigmoid activation function, and... A one-dimensional convolution kernel; The specific steps for calculating the channel attention mapping of the first spatial spectral feature map include: S51: The global maximum pooling feature and global average pooling feature of the first spatial spectral feature map are calculated by passing the global maximum pooling layer and the global average pooling layer respectively. S52: Transpose the global max pooling feature and multiply it with the global average pooling feature to obtain the second-order statistical feature; S53: The second-order aggregated feature is obtained by averaging across the column dimensions of the second-order statistical feature; S54: Transform the second-order aggregated features through... After performing convolution operations on a one-dimensional convolution kernel, the channel attention mapping of the first spatial spectral feature map is calculated by inputting it into the Sigmoid activation function.
4. The hyperspectral image classification method according to claim 1, characterized in that, The second spatial spectral feature map is calculated based on the channel attention mapping of the first spatial spectral feature map and the first spatial spectral feature map: The channel attention mapping of the second spatial spectral feature map is added pixel by pixel to the second spatial spectral feature map.
5. The hyperspectral image classification method according to claim 1, characterized in that, The step of inputting the second spatial spectral feature map into the fully connected layer for classification includes: The second spatial spectral feature map is input into the Flatten layer for global pooling. Then, the resulting one-dimensional vector is sequentially input into two fully connected layers with 256 and 128 nodes respectively. To prevent overfitting, both fully connected layers use Dropout with a ratio of 0.
4. Finally, the number of nodes in the Softmax classification layer is the same as the number of categories in the original hyperspectral image data.
6. The hyperspectral image classification method according to claim 1, characterized in that, The loss function of the hyperspectral image classification model includes: Where y=[ This represents the label information of the raw hyperspectral image data, when the sample belongs to the i-th class. 1, otherwise ; represents the probability that the original hyperspectral image data belongs to class i; C represents the total number of classes in the original hyperspectral image data.