A farmland classification method for hyperspectral remote sensing images based on generative adversarial networks

Abstract

The present invention belongs to the technical field of hyperspectral image processing, and relates to a hyperspectral remote sensing image farmland classification method based on a generative adversarial network. The method constructs a generative adversarial network model to perform superpixel segmentation on the hyperspectral image. In the training process of the generator network model and the discriminator network model, the loss function of the discriminator network model is modified to learn the categories of the obtained adjacent superpixel segmentation blocks, thereby making full use of spectral information and spatial neighborhood information. In the case of a small number of training sets, the method realizes accurate classification of farmland objects in hyperspectral data and achieves good results.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral remote sensing image processing, and in particular to a hyperspectral object classification method based on a deep learning network. Background Art

[0002] Compared with traditional images, hyperspectral images have more channels, so hyperspectral images can clearly classify ground objects. However, due to the excessive number of hyperspectral channels, data redundancy is easily caused, and traditional machine learning methods cannot classify well. In addition, due to the limited number of training samples, hyperspectral ground object classification is a huge challenge for emerging deep learning methods. Due to the uniqueness of hyperspectral data, spectral and spatial features are important factors to consider in classification. Traditional machine learning methods can only consider spectral features, such as support vector machines (SVM) and one-dimensional convolutional networks (1DCNN). The most important spatial features of ground objects are the texture information of pixels and the location information of pixels. Pixels with the same texture information are very likely to belong to the same category, and pixels with the same category are likely to be concentrated in the same area.

[0003] Compared with traditional land object classification, farmland classification based on hyperspectral remote sensing data has higher spectral similarity. In the case of small samples, only using spectral information for distinction will result in low classification accuracy. However, farmland objects of the same category are mostly concentrated in a regional shape with obvious spatial characteristics. Therefore, a hyperspectral farmland classification method is needed that can combine spectral information and spatial information to improve the classification accuracy. Summary of the invention

[0004] The purpose of the present invention is to provide a hyperspectral remote sensing image farmland classification method based on a generative adversarial network, which utilizes the spatial and spectral feature information of farmland hyperspectral data samples to achieve spatial and spectral combination in the classification process and improve the classification accuracy.

[0005] To achieve the above functions, the present invention designs a hyperspectral remote sensing image farmland classification method based on a generative adversarial network. For the hyperspectral remote sensing image of farmland, the following steps are performed:

[0006] Perform super-pixel segmentation on hyperspectral satellite remote sensing images of farmland to obtain super-pixel segmentation blocks;

[0007] Perform PCA dimensionality reduction processing on each superpixel segmentation block;

[0008] Calculate the center position of each superpixel segmentation block and find the adjacent superpixel segmentation blocks of the superpixel segmentation block;

[0009] Construct a generator network model and a discriminator network model, and divide the hyperspectral satellite remote sensing images of farmland into training sets and test sets;

[0010] Use the data from the training set and the test set to train the generator network model and pre-train the discriminator network model;

[0011] Use the data and labels of the training set to train the classification performance of the discriminator network model;

[0012] The loss value of the discriminator network model is calculated based on the superpixel category information of the training set and the category information of the adjacent superpixel blocks;

[0013] Write the category information output by the discriminator network model into the superpixel segmentation block, and use the back propagation algorithm to update the parameters of the discriminator network model;

[0014] The classification performance of the discriminator network model is tested using the data and labels of the test set, and the classification results of hyperspectral satellite remote sensing images of farmland are output.

[0015] The hyperspectral image is segmented into superpixels, and each superpixel segmentation block is subjected to PCA dimensionality reduction, including: using a superpixel segmentation algorithm for the hyperspectral image, dividing the hyperspectral image into different superpixel blocks according to texture, each superpixel block contains pixels with similar textures and adjacent distances, and performing PCA dimensionality reduction on all pixels in each superpixel block.

[0016] Calculate the center position of each superpixel and its adjacent superpixel segmentation blocks, including:

[0017] According to the pixel positions contained in each superpixel segmentation block, the center coordinates of each superpixel segmentation block are calculated, and the formula is as follows:

[0018]

[0019]

[0020] in, and is the center coordinate of each superpixel segmentation block, and is the coordinate information of the pixels contained in the superpixel segmentation block, is the number of pixels contained in the superpixel segmentation block.

[0021] According to the center coordinates of the superpixel segmentation block and the coordinate information of the pixels contained in it, the radius of the circumscribed circle of each superpixel segmentation block is calculated using the following formula:

[0022]

[0023] Where n is the number of pixels contained in each superpixel segmentation block;

[0024] According to the radius of its circumcircle , calculate the distance between each superpixel segmentation block , find the neighboring blocks of each superpixel segmentation block, the formula is as follows:

[0025]

[0026] in, , , , , and They are the center coordinates of different superpixel segments and their corresponding circumscribed circle radii.

[0027] Construct the generator network model and the discriminator network model, including:

[0028] The generator network model and the discriminator network model use a three-layer one-dimensional convolutional network model to convolve the spectral bands. Regularization and ReLU activation functions need to be added after each layer of convolution.

[0029] Use the training set and test set to train the generator network model and pre-train the discriminator network model, including:

[0030] The optimized generative adversarial network loss function is used to replace the traditional loss function with the improved Wasserstein distance loss function to alleviate mode collapse, complete the training of the generator network model, and pre-train the discriminator network model. The formula is as follows:

[0031]

[0032] in, is the true distribution of the sample, The distribution of samples generated for the generator network model.

[0033] The superpixel category information of the training set and the category information of the adjacent superpixel blocks are used to calculate the loss value of the discriminator, including:

[0034] For a given training set, determine the superpixel segmentation block to which it belongs and find the adjacent superpixel segmentation blocks. The discriminator network model outputs the category of the training set. , according to the category of the adjacent superpixel segmentation block , calculate the loss value of the discriminator

[0035]

[0036] in is the number of adjacent superpixel blocks.

[0037] Calculate the loss value based on the category information of the superpixel segmentation block , and The loss values ​​are added together to form the total loss value , the formula is as follows:

[0038]

[0039] in, is the scale of neighborhood learning, which can be adjusted by The value of controls the scale of neighborhood learning.

[0040] Write the category information output by the discriminator network model into the superpixel segmentation block, including:

[0041] The discriminator network model outputs the category of the training set , write it into the superpixel segmentation block to which the current training set belongs. Each superpixel segmentation block uses a stack structure to store category information.

[0042] The beneficial effects of the present invention are as follows: by constructing a generative adversarial network model, a hyperspectral remote sensing image is segmented into superpixels, and in the training process of the generator network model and the discriminator network model, the categories of the obtained adjacent superpixel segmentation blocks are learned by modifying the loss function of the discriminator network model, thereby making full use of the spectral information and spatial neighborhood information. With a small number of training sets, accurate classification of farmland objects in hyperspectral remote sensing data is achieved, and good results are achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0044] Figure 1 A flow chart of a method for farmland classification based on a hyperspectral remote sensing image and a generative adversarial network provided by the present invention;

[0045] Figure 2 A generative adversarial network framework diagram of a hyperspectral remote sensing image farmland classification method based on a generative adversarial network provided by the present invention;

[0046] Figure 3Schematic diagram of the classification results of the classification method of the present invention and the comparative classification method on the Indian_Pines dataset; wherein (a) is the ground truth map, (b) is the classification result map of the SVM method, (c) is the classification result map of the 3DGAN method, (d) is the classification result map of the 3DCNN method, (e) is the classification result map of the SSFTT method, and (f) is the classification result map of the SNGAN method proposed in the present invention;

[0047] Figure 4 Schematic diagram of the classification results of the classification method of the present invention and the comparative classification method on the Salinas data set; wherein (a) is the ground truth map, (b) is the classification result map of the SVM method, (c) is the classification result map of the 3DGAN method, (d) is the classification result map of the 3DCNN method, (e) is the classification result map of the SSFTT method, and (f) is the classification result map of the SNGAN method proposed in the present invention. DETAILED DESCRIPTION

[0048] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0049] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0050] like Figure 1 A method for classifying farmland in hyperspectral remote sensing images based on a generative adversarial network is shown, comprising the following steps:

[0051] The superpixel segmentation algorithm is used to segment the farmland satellite hyperspectral images. The superpixel segmentation algorithm can segment adjacent pixels with the same texture information into different superpixel blocks according to the texture information of the hyperspectral images, and record the superpixel segmentation block to which each pixel belongs.

[0052] According to the superpixel segmentation blocks shown by each pixel, PCA dimensionality reduction is performed separately. Compared with the traditional PCA dimensionality reduction of the entire image, it can enhance the spatial characteristics of the classified pixels and retain the feature information to the greatest extent while reducing the data dimension.

[0053]

[0054] According to the pixel positions contained in each superpixel segmentation block, the center coordinates of each superpixel segmentation block are calculated. The calculation formula is shown in the following figure:

[0055]

[0056]

[0057] in and is the center coordinate of each superpixel segmentation block, and is the coordinate information of the pixels contained in the superpixel segmentation block, is the number of pixels contained in each superpixel segmentation block.

[0058] According to the center coordinates of the superpixel segmentation block and the coordinate information of the pixels contained in it, the circumscribed circle radius of each superpixel segmentation block is calculated, and the circumscribed circle radius is calculated according to the formula .

[0059]

[0060] According to the circumcircle radius , calculate the distance between each superpixel segmentation block , record the neighboring blocks of each superpixel segmentation block.

[0061]

[0062] in , , , , , , are the center coordinates of two different superpixel segments and their corresponding circumscribed circle radii.

[0063] Use Python language to build a generative adversarial network based on the pytorch deep learning library.

[0064] The network model is shown in the figure Figure 2As shown in Figure 2. The generative adversarial network consists of a generator network model and a discriminator network model. Both the generator network model and the discriminator network model are composed of convolutional networks. The last layer of the discriminator model network is divided into two parts, the discrimination layer and the classification layer. The discrimination layer is learned in the pre-training stage, and the classification layer is learned in the classification process of neighborhood learning.

[0065] The training set and test set are generated in a certain ratio. Since it is necessary to give full play to the best effect of neighborhood learning, the training set of small samples is set as dispersed as possible.

[0066] The generator network model and the pre-trained discriminator network model are trained using noise data and data from the training set and the test set. The generator network model and the discriminator network model compete with each other. The generator network model uses noise data to generate false data to deceive the discriminator network model, while the discriminator network model distinguishes the false data generated by the generator network model based on the real data of the training set and the test set. Finally, the generator network model can generate similar false data, and the discriminator network model can distinguish between false data and real data. The traditional generative adversarial network loss function may cause the data generated by the generator network model to be single, the mode to collapse, and the discriminator network model and the generator network model to be unable to learn. Therefore, we use Wasserstein distance as the loss function, which can not only alleviate the mode collapse, but also speed up the training speed. The formula is shown below. Train 2 epochs to complete the training of the generator network model and the pre-training of the discriminator network model.

[0067]

[0068] in, is the true distribution of the sample, The distribution of samples generated for the generator network model.

[0069] Train the discriminator network model to perform classification learning of neighborhood learning. Use the data and labels of the training set to perform classification training on the discriminator network model. Perform the following steps during the training process:

[0070] S1. For a given training set, determine the superpixel segmentation block and find the adjacent superpixel segmentation blocks.

[0071] S2. The discriminator network model gives the category of the training set , calculate the loss according to the category of the adjacent superpixel segmentation block , the calculation formula is as follows:

[0072]

[0073] in, is the number of adjacent superpixel blocks.

[0074] S3. Calculate the loss value based on the category information of the superpixel segmentation block , and The loss values ​​are added together to form the total loss value , the formula is as follows:

[0075]

[0076] in, is the scale of neighborhood learning, by adjusting The value of controls the scale of neighborhood learning.

[0077] S4. The training categories given by the discriminator network model Write the superpixel segmentation block to which the current training set belongs. Each superpixel segmentation block uses a stack to store category information.

[0078] Repeat the above training steps for 500 epochs, and the discriminator network model learns from the neighborhood to achieve the best category classification effect.

[0079] Use the data and corresponding labels in the test set to test the discriminator network and calculate the classification effect.

[0080] The hyperspectral remote sensing image datasets used in this embodiment are two public datasets, Indian_Pines and Salinas. Indian_Pines is the earliest test dataset for hyperspectral image classification. An airborne visible near-infrared imaging spectrometer imaged an Indian pine tree in Indiana, USA in 1992, and then cut out a size of 145×145 and marked it for hyperspectral image classification testing. The Indian_Pines feature category includes two-thirds of agriculture and one-third of forests or other natural perennial plants. The Salinas dataset was captured by a 224-band sensor over the Salinas Valley in California, with high spatial resolution and a coverage area of ​​512×217 samples. The Salinas feature categories include agricultural plants such as vegetables, bare soil, and vineyards.

[0081] The classification method comparison experiment used support vector machine classification method (SVM), three-dimensional generative adversarial network classification method (3DGAN), three-dimensional convolutional feature extraction network classification method (3DCNN) and spectral-spatial feature labeling Transformer classification method (SSFTT). The accuracy analysis of the experimental results used the classification accuracy, overall accuracy (OA), average accuracy (AA) and Kappa coefficient of each type of sample.

[0082] The model training settings are as follows:

[0083] In Indian_Pines, the method proposed in the present invention and the spectral-spatial feature labeling Transformer classification method (SSFTT) are both trained using a small sample training set, with a total number of 300 small samples. The remaining comparative experiments are trained using 5% of the data set.

[0084] In Salinas, the method proposed in the present invention and the spectral-spatial feature labeling Transformer classification method (SSFTT) are both trained using a small sample training set, with a total number of small samples of 400. The other comparative experiments are trained using three percent of the data set.

[0085] The learning rate is set to 0.001, the batch size is set to 64, the number of superpixel segmentations of the input is set to 300, and the parameters of the other comparison methods are configured according to the original environment.

[0086] Under this condition, 10 repeated experiments were conducted. The classification accuracy of the method of the present invention and the comparative experimental method on the Indian_Pines dataset is shown in Table 1. Figure 3 The classification accuracy of the method of the present invention and the comparative experimental method on the Salinas dataset is shown in Table 2. The classification accuracy of the method of the present invention and the comparative experimental method on the Indian_Pines dataset is shown in Table 3. Figure 4 shown.

[0087] From the classification accuracy table 1 and classification accuracy table 2, it can be seen that the classification accuracy of the method proposed in the present invention is greatly improved in the case of small samples compared with SVM, 3DGAN and 3DCNN using more training samples. At the same time, compared with the latest transformer network SSFTT, the classification accuracy is improved by about 1.5% when the number of training samples is the same.

[0088] Table 1 Classification accuracy of each method on the Indian_Pines dataset

[0089]

[0090] Table 2 Classification accuracy of each method on the Salinas dataset

[0091]

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A hyperspectral remote sensing image farmland classification method based on generative adversarial network, characterized in that: include: Perform super-pixel segmentation on hyperspectral satellite remote sensing images of farmland to obtain multiple super-pixel segmentation blocks; Perform PCA dimensionality reduction processing on each superpixel segmentation block; Calculate the center position of each superpixel segmentation block and obtain the adjacent superpixel segmentation blocks of the superpixel segmentation block; Construct a generator network model and a discriminator network model, and divide the hyperspectral satellite remote sensing images of farmland into training sets and test sets; Use the data from the training set and the test set to train the generator network model and pre-train the discriminator network model; Use the data and labels of the training set to train the classification performance of the discriminator network model; The loss value of the discriminator network model is calculated based on the superpixel category information of the training set and the category information of the adjacent superpixel blocks; Write the category information output by the discriminator network model into the superpixel segmentation block, and use the back propagation algorithm to update the parameters of the discriminator network model; Use the data and labels of the test set to test the classification performance of the discriminator network model and output the classification results of the hyperspectral satellite remote sensing images of farmland; Calculate the center position of each superpixel and its adjacent superpixel segmentation blocks, including: According to the pixel positions contained in each superpixel segmentation block, the center coordinates of each superpixel segmentation block are calculated, and the formula is as follows: Among them, C x and C y is the center coordinate of each superpixel segmentation block, x i and i is the coordinate information of the pixels contained in the superpixel segmentation block, and n is the number of pixels contained in the superpixel segmentation block; According to the center coordinates of the superpixel segmentation block and the coordinate information of the pixels contained in it, the radius r of the circumscribed circle of each superpixel segmentation block is calculated using the following formula: Where n is the number of pixels contained in each superpixel segmentation block; According to the radius r of its circumscribed circle, the distance d between each superpixel segmentation block is calculated, and the adjacent blocks of each superpixel segmentation block are found. The formula is as follows: Among them, Cm x , Cm y 、r m , Cn x , Cn y and r n are the center coordinates of different superpixel segments and their corresponding circumscribed circle radii; Construct the generator network model and the discriminator network model, including: The generator network model and the discriminator network model use a three-layer one-dimensional convolutional network model to convolve the spectral bands. Regularization and ReLU activation functions need to be added after each layer of convolution. Use the training set and test set to train the generator network model and pre-train the discriminator network model, including: The optimized generative adversarial network loss function is used, and the improved Wasserstein distance loss function is used instead of the traditional loss function to alleviate the mode collapse and complete the training of the generator network model. When pre-training the discriminator network model, the formula is as follows: Among them, P r is the true distribution of the sample, P g The distribution of samples generated for the generator network model; According to the superpixel category information of the training set and the category information of the adjacent superpixel blocks, the loss value of the discriminator network model is calculated, including: For a given training set, determine the superpixel segmentation block to which it belongs and find the adjacent superpixel segmentation blocks. The discriminator network model outputs the category y of the training set predi , according to the category y of the adjacent superpixel segmentation block i , calculate the loss value L of the discriminator network model neighb orhood Where N is the number of adjacent superpixel blocks; According to the category information of the superpixel segmentation block, calculate the loss value L discriminator , and L neighborhood The loss values ​​are added together to form the total loss value L total , the formula is as follows: L total =L discriminator +α·L neighborhood Among them, α is the scale of neighborhood learning, and the scale of neighborhood learning is controlled by adjusting the value of α.

2. According to claim 1, a method for farmland classification based on hyperspectral remote sensing images based on generative adversarial networks is characterized by: The hyperspectral image is segmented into superpixels, and each superpixel segmentation block is subjected to PCA dimensionality reduction, including: the hyperspectral image is segmented into different superpixel blocks according to texture using a superpixel segmentation algorithm, each superpixel block contains pixels with similar textures and adjacent distances, and all pixels in each superpixel block are subjected to PCA dimensionality reduction.

3. According to the method of claim 1, the method for farmland classification based on hyperspectral remote sensing images and generative adversarial networks is characterized in that: Write the category information output by the discriminator network model into the superpixel segmentation block, including: the category of the training set output by the discriminator network model Write it into the superpixel segmentation block to which the current training set belongs. Each superpixel segmentation block uses a stack structure to store category information.

Images