A hyperspectral image classification method based on dense connection and self-integrated network

Abstract

The application discloses a hyperspectral image classification method based on a dense connection and a self-integrated network and belongs to the field of remote sensing images. The method comprises the following steps: acquiring a hyperspectral image, pre-processing the hyperspectral image to obtain a pre-processed image, and dividing the pre-processed image into a training sample data set and a test sample data set; constructing a DSEN network; inputting the obtained training sample data set and test sample data set into the DSEN network for training to obtain a trained DSEN network model; inputting an image to be classified into the trained DSEN network, inputting obtained features into a Softmax classifier layer, and obtaining a final prediction result; and introducing a dense connection module into the self-integrated network to make up for the problem that the prediction quality of an integrated model is difficult to guarantee to a certain extent, and further improve the classification precision and generalization ability of the model in a hyperspectral image classification task.

Description

Technical Field

[0001] This invention relates to a hyperspectral image classification method based on densely connected and self-integrated networks, belonging to the field of remote sensing image technology. Background Technology

[0002] Hyperspectral imaging covers the ultraviolet, visible, near-infrared, and mid-infrared regions of the electromagnetic spectrum. A hyperspectral image is a three-dimensional data cube obtained by simultaneously imaging ground features in three dimensions. While providing information on the spatial distribution of ground features, it also contains rich spectral information reflecting the essential characteristics of these features. Hyperspectral remote sensing technology, with its high spectral resolution, can effectively describe the spectral information reflected by ground features and improve the ability to identify and classify them, gradually becoming a cutting-edge research area in contemporary remote sensing technology. For hyperspectral remote sensing technology, it can not only obtain rich spectral information from target ground features but also acquire a large amount of spatial information reflecting the shape characteristics of ground features. The rich spectral and spatial information in hyperspectral images has led to the rapid development of hyperspectral remote sensing technology, especially in hyperspectral image classification, which has now become a hot research topic in the field of hyperspectral remote sensing.

[0003] Deep learning models play a crucial role in feature extraction from hyperspectral remote sensing images, with convolutional neural networks (CNNs) being one of the representative algorithms. A typical CNN consists of convolutional layers, pooling layers, and fully connected layers, offering two major advantages: weight sharing and local perception. It is a type of feedforward neural network architecture that incorporates convolutional computations and has a deep structure, exhibiting good feature extraction capabilities and generalization performance, and is widely used in the field of hyperspectral imaging.

[0004] However, applying deep learning to hyperspectral image classification still faces many challenges. Deep learning can extract abstract deep feature representations from hyperspectral images, but in order to significantly improve classification accuracy, it is usually necessary to use a deeper network structure with more layers. This will make the model contain a large number of network parameters that need to be learned. At the same time, increasing the number of layers in the deep network will make network training difficult, require longer training time, and also lead to problems such as gradient vanishing and classification accuracy decay. Summary of the Invention

[0005] The purpose of this invention is to provide a hyperspectral image classification method based on dense connections and self-integrating networks, specifically including the following steps:

[0006] Step 1: Acquire hyperspectral images from a hyperspectral camera.

[0007] Step 2: Use Local Linear Embedding (LLE) to perform dimensionality reduction preprocessing on the image to obtain the preprocessed image, which is divided into training sample dataset and test sample dataset.

[0008] Step3: Building a DSEN neural network model, the neural network model comprising a student network module and a teacher network module; wherein the student network module and the teacher network module are respectively composed of a spatial information extraction module, a spectral information extraction module, a feature fusion module and a classification module.

[0009] Step4: inputting the obtained training data set into the built network for training to obtain a trained network.

[0010] Step5: inputting the image to be classified into the trained network, and outputting the classification result of the image to be classified.

[0011] Preferably, the specific process of Step Step2 of the present application is as follows:

[0012] Step2.1: assuming that the spectral dimension of the hyperspectral image X is P, i.e. the number of bands, the height and width of the image are H and K respectively, each image contains H×K pixels; the hyperspectral image can be represented as a three-dimensional data cube with a size of H×K×P.

[0013] Step2.2: preprocessing the hyperspectral data H×K×P by using local linear embedding (LLE), which reduces the number of hyperspectral bands P to L, and the output after preprocessing is H×K×L; then, 3D cube extraction is performed on X LLE to obtain N 3D cubes S×S×L, S×S being the spatial size, and the specific steps of dimension reduction are as follows:

[0014] Step2.2.1: finding the nearest neighbor point: according to the Euclidean distance, find the nearest neighbor point k of each sample point x i (i=1,2,3···n) in the high-dimensional space hyperplane, k is a constant value.

[0015] Step2.2.2: constructing a weight matrix W:

[0016] For each sample point x i , LLE starts from the k nearest neighbor points, and the sample x i will be approximated by the linear combination of its nearest neighbor points, and then the weight matrix W that satisfies the constraint condition of the sample point can be calculated:

[0017]

[0018] In the formula: i=1,2,3···n, x i ,x j are sample points; w ij is a weight coefficient, and if x j is not xi If the neighbor is w ij =0; if x j It is x i Neighbors, The weight matrix W describes the sample x j For sample x i Contributions to the reconstruction.

[0019] Step 2.2.3: Calculate the low-dimensional embedding

[0020] Based on the principle that weight coefficients in high-dimensional space remain unchanged in low-dimensional space, the low-dimensional space sample Y is determined by W. After determining the weight matrix, the loss function is minimized, resulting in:

[0021]

[0022]

[0023] In the formula: y i ,y j These are low-dimensional space sample points.

[0024] Step 2.3: Divide the data obtained by dimensionality reduction using Local Linear Embedding (LLE) into training sample set and test sample set.

[0025] The partitioning rule is as follows: n% of the pixel samples for each land cover class are randomly selected from the public dataset as the training sample set, and the remaining 100% - n% of the pixel samples for each land cover class are used as the test sample set, where N... train = N1×n% + N2×n% + ... + N M ×n%,N test =N Q -N train ; where N train N represents the total number of training samples. test M represents the total number of test samples, M represents the total number of ground feature pixel categories, and N represents the total number of samples. Q This represents the total number of pixels for ground features.

[0026] Preferably, the construction of the neural network model described in Step 3 of this invention involves the following specific steps:

[0027] Step 3.1: As Figure Two As shown, the DSEN neural network model has two sub-networks, including a student network module and a teacher network module. The two networks share the same architecture, and each network contains two branches: an upper branch and a lower branch. The upper branch simply uses a fully connected layer to extract the spectral features f of the hyperspectral image. spe; in the lower branch, two SEDB modules and two average pooling layers are used to extract the spatial features f spa .

[0028] Step3.2: In order to perform spectral-spatial classification, f spe and f spa are further concatenated into a fusion layer f fusion = [f spe ; f spa ]; let f cls become the feature of the last fully connected layer; the probability that the input sample belongs to the i-th class can be obtained by the Softmax function.

[0029] Preferably, the step 4 of the present application inputs the obtained training data set into the built network for training to obtain a trained network, and the steps are as follows:

[0030] Step4.1: first, in the training stage, the labeled samples are input into the student network, and the supervised cross-entropy loss between the prediction and the ground truth label is calculated; on the other hand, the unlabeled samples are input into the student network and the teacher network; then, the unsupervised mean square error between the predictions of the two networks is calculated.

[0031] Step4.2: in each iteration, the student network is updated using the gradient of the supervised and unsupervised loss; the teacher network does not participate in the back propagation step; in this way, the student network and the teacher network can gradually learn from each other, achieving the purpose of self-integrated learning.

[0032] The present application has the following beneficial effects:

[0033] (1) The method of the present application performs local linear embedding to reduce the dimension of the hyperspectral image, and by minimizing the reconstruction error of the data points in the low-dimensional space, the local structure between the hyperspectral data is preserved, and the dimension of the data is reduced; it effectively handles the nonlinear relationship of the hyperspectral data, and reduces the redundancy of the data while maintaining the characteristics of the data itself.

[0034] (2) The present application extracts and optimizes the spectral features and spectral-spatial features through the spectral sub-network branch and the spatial sub-network branch respectively, and introduces a dense connection module into the self-integrated network to some extent to make up for the problem that the prediction quality of the integrated model is difficult to guarantee; by using the features learned by the model to guide the calculation of the supervised loss of the labeled data and the unsupervised loss of the unlabeled data, under the constraint of the two kinds of losses, the student network and the teacher network can learn from each other, which not only guarantees the classification accuracy of the model, but also speeds up the training speed of the model, thereby realizing the self-integrated mechanism and further improving the classification accuracy and generalization ability of the model in the hyperspectral image classification task. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure One is a flow chart of the present application;

[0036] Figure Two is a structural block diagram of the present application. DETAILED DESCRIPTION

[0037] The present application will be further described in conjunction with the accompanying drawings and specific embodiments, but the scope of protection of the present application is not limited to the described content.

[0038] Example 1

[0039] A hyperspectral image classification method based on dense connection and self-integrated network, as shown in Figure One , includes the following steps:

[0040] Step 1: Collect hyperspectral images from a hyperspectral camera; in this embodiment, two open source data sets in the hyperspectral field, Pavia data set and Indian Pines data set, are used.

[0041] Step 2: Use local linear embedding (LLE) to perform dimensionality reduction preprocessing on the image to obtain a preprocessed image, which is divided into a training sample data set and a test sample data set; high-dimensional data contains too much redundant information, making the calculation amount very large, therefore, local linear embedding (LLE) is used to perform spectral dimensionality reduction on the hyperspectral image, to reduce the dimension of the spectral band and retain complete spatial information, which includes the following steps:

[0042] Step 2.1: Assuming that the spectral dimension of the hyperspectral image X is P, i.e. the number of bands, the height and width of the image are H and K respectively, then each image contains H×K pixels; the hyperspectral image can be represented as a three-dimensional data cube with size H×K×P.

[0043] Step 2.2: Preprocess the hyperspectral data H×K×P using local linear embedding (LLE), which reduces the number of hyperspectral bands P to L, and the output after preprocessing is H×K×L; then, 3D cube extraction is performed on X LLE to obtain N 3D cubes S×S×L, S×S being the spatial size; the specific implementation of dimensionality reduction is as follows:

[0044] Step 2.2.1: Find the nearest neighbor

[0045] For the LLE algorithm, the selection of the neighborhood size needs to be determined first, i.e. how many neighborhood samples are needed to linearly represent a certain sample; according to the Euclidean distance, find the nearest neighbor of each sample point x i(i = 1, 2, 3 ··· n) as its neighbor points.

[0046] Step 2.2.1: Constructing weight matrix W

[0047] For each sample point x i , LLE starts from k neighbor points, sample x i will be approximated by the linear combination of its neighbor points, and then the weight matrix W that satisfies the constraint condition of the sample point can be calculated:

[0048]

[0049] In the formula: i = 1, 2, 3 ··· n, x i ,x j is a sample point; w ij is a weight coefficient, if x j is not a neighbor of x i , w ij = 0; if x j is a neighbor of x i , The weight matrix W describes the contribution of the sample x j to the reconstruction of the sample x i ;

[0050] Step 2.2.3: Calculate low-dimensional embedding

[0051] According to the principle that the weight coefficient in the high-dimensional space remains unchanged in the low-dimensional space, the low-dimensional space sample Y is determined by W. After the weight matrix is determined, the loss function is minimized, that is:

[0052]

[0053]

[0054] In the formula: y i ,y j is a low-dimensional sample point;

[0055] Step 2.3: Divide the data obtained by using local linear embedding (LLE) for dimension reduction into training sample set and test sample set.

[0056] The division rule is: randomly select n% from each class of ground object pixel sample in the public data set as the training sample set, and the remaining part 100%-n% in each class of ground object pixel sample as the test sample set, N train =N1×n%+N2×n%+···+N M ×n%, N test =N Q -Ntrain ; where N train N represents the total number of training samples. test M represents the total number of test samples, M represents the total number of ground feature pixel categories, and N represents the total number of samples. Q This represents the total number of pixels for ground features.

[0057] Step 3: As Figure Two As shown, the DSEN neural network model is constructed, which includes a student network module and a teacher network module. The student network module and the teacher network module are each composed of a spatial information extraction module, a spectral information extraction module, a feature fusion module, and a classification module.

[0058] Step 3.1: The student network and teacher network share the same architecture, consisting of two branches, one above the other. The upper branch simply uses a fully connected layer to extract the spectral features f of the hyperspectral image. spe In the next branch, we use two SEDB modules and two average pooling layers to extract the spatial features f of the hyperspectral image. spa .

[0059] Step 3.2: To perform spectral spatial classification, f is then... spe and f spa Further connected into a fusion layer f fusion =[f spe ;f spa Let f cls The features become the last fully connected layer; the probability that an input sample belongs to the i-th class can be obtained by the softmax function.

[0060] Step 4: Input the obtained training dataset into the built network for training to obtain a trained network model;

[0061] Step 4.1: First, during the training phase, labeled samples are input into the student network, and the supervised cross-entropy loss between the predictions and the ground truth labels is calculated. On the other hand, unlabeled samples are input into both the student and teacher networks. Then, the unsupervised mean squared error between the predictions of the two networks is calculated.

[0062] Step 4.2: In each iteration, the student network is updated using gradients of supervised and unsupervised losses; the teacher network does not participate in the backpropagation step; in this way, the student network and the teacher network can learn from each other step by step, achieving the goal of self-ensemble learning.

[0063] Step 5: Input the image to be classified into the trained network and output the classification result of the image to be classified.

[0064] The specific embodiments of the present application are described in detail above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the spirit of the present application.

Claims

1. A hyperspectral image classification method based on densely connected and self-integrated networks, characterized in that, Specifically, the following steps are included: Step 1: Acquire hyperspectral images from a hyperspectral camera; Step 2: Use Local Linear Embedding (LLE) to perform dimensionality reduction preprocessing on the image to obtain the preprocessed image, which is divided into training sample dataset and test sample dataset; The specific process of Step 2 is as follows: Step 2.1: Assuming the spectral dimension of the hyperspectral image X is P, i.e. the number of bands, and the height and width of the image are H and K respectively, then each image contains H×K pixels; the hyperspectral image can be represented as a cube of size H×K×P. Step 2.2: Dimensionality reduction of the hyperspectral remote sensing image data is performed using Local Linear Embedding (LLE), and the data is divided into three-dimensional data blocks. The specific steps of dimensionality reduction are as follows: Step 2.2.1: Finding Nearest Neighbors: Based on Euclidean distance, find each sample point in the high-dimensional hyperplane. ( The nearest neighbor k of ) is taken as its nearest neighbor, where k is a constant; Step 2.2.2: Construct the weight matrix : For each sample point LLE starts from the k nearest neighbors and samples It can then be approximated by a linear combination of its nearest neighbors, and thus the weight matrix that satisfies the constraints can be calculated. : ； In the formula: , For sample points; For weighting coefficients, if no Neighbors, ; if yes Neighbors, Weight matrix Describes the sample For the sample Contributions to the reconstruction; Step 2.2.3: Calculate the low-dimensional embedding Based on the principle that weight coefficients in high-dimensional space remain unchanged in low-dimensional space, through... After determining the low-dimensional sample Y and the weight matrix, minimizing the loss function yields: ； ； In the formula: These are low-dimensional space sample points; Step 2.3: Divide the data obtained by dimensionality reduction using Local Linear Embedding (LLE) into training sample set and test sample set; The partitioning rule is as follows: n% of the pixel samples of each land cover class in the public dataset are randomly selected as the training sample set, and the remaining 100%-n% of the pixel samples of each land cover class are used as the test sample set. , ;in, The total number of training samples. M represents the total number of test samples and the total number of ground feature pixel categories. This represents the total number of pixels representing ground features. Step 3: Building the DSEN neural network model, which includes a student network module and a teacher network module; wherein the student network module and the teacher network module are each composed of a spatial information extraction module, a spectral information extraction module, a feature fusion module and a classification module; The specific steps for building the neural network model described in Step 3 are as follows: Step 3.1: The DSEN neural network model has two sub-networks, including a student network module and a teacher network module. The two networks share the same architecture, and each network contains two branches. Step 3.2: The upper branch uses a fully connected layer to extract the spectral features f of the hyperspectral image. spe In the next branch, two SEDB modules and two average pooling layers are used to extract the spatial features f of the hyperspectral image. spa ; Step 3.3: To perform spectral spatial classification, f is then... spe and f spa Further connected into a fusion layer f fusion =[f spe ;f spa ]; let f cls The features become the last fully connected layer; the probability of an input sample belonging to the i-th class can be obtained by the Softmax function; Step 4: Input the obtained training dataset into the built network for training to obtain a trained network; In Step 4, the obtained training dataset is input into the constructed network for training to obtain a trained network. The steps are as follows: Step 4.1: First, during the training phase, labeled samples are input into the student network, and the supervised cross-entropy loss between the prediction and the ground truth label is calculated; on the other hand, unlabeled samples are input into both the student network and the teacher network; then, the unsupervised mean squared error between the predictions of the two networks is calculated. Step 4.2: In each iteration, the student network is updated using gradients of supervised and unsupervised losses; the teacher network does not participate in the backpropagation step; the student network and the teacher network can then learn from each other step by step, achieving the goal of self-ensemble learning. Step 5: Input the image to be classified into the trained network and output the classification result of the image to be classified.

Images