A small sample hyperspectral image classification method and device based on neural network

Abstract

The present application provides a method and device for classifying small sample hyperspectral images based on neural networks, wherein the method mainly includes: obtaining all hyperspectral images to be classified; extracting one-dimensional vectors according to all hyperspectral images to be classified; combining the one-dimensional vectors into a two-dimensional matrix; obtaining a two-dimensional feature map by reducing the dimension and extracting features from the two-dimensional matrix; reorganizing the two-dimensional feature map into a one-dimensional vector; and classifying according to the number of one-dimensional vectors. The method is suitable for small sample HSI classification, and can obtain better classification performance, while avoiding the problems of being easily disturbed by low efficiency, huge amount of calculation, and harsh hardware conditions in traditional methods.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and device for classifying small sample hyperspectral images based on a neural network. Background Art

[0002] With the development of remote sensing technology, hyperspectral images (HSI) have received extensive attention in the field of remote sensing. HSI data contains both spectral and spatial information and is widely used in agriculture, monitoring, geography, industry, environmental protection and other fields. Therefore, in the practical application of the above fields, remote sensing image analysis is crucial. Hyperspectral images can simultaneously describe the two-dimensional radiation information and spectral information of the spatial distribution of objects, forming a unique spectrum that integrates spatial, radiation and spectral information. Classification is an important part of hyperspectral image processing technology. Its ultimate goal is to assign a unique category identifier to each pixel in the image, which has important applications in both military and civilian fields.

[0003] In recent years, many classification methods based on spectral spatial information have emerged, achieving significant performance improvements. Spectral spatial classification methods are generally divided into two categories. The first method extracts spectral and spatial features separately, and the model classifier is responsible for feature fusion, including remote sensing image fusion and dual-channel convolutional neural network. The second method extracts spectral spatial features simultaneously and is processed by the classifier, which performs better than the first method. Traditional HSI classification methods continue to develop, and many innovative algorithms based on deep learning have been proposed in recent years, especially the application of convolutional neural network (CNN) to image classification tasks, which has achieved more outstanding results in HSI classification than other methods. The advantage of CNN is that based on big data and deep network structure, large labeled samples of HSI can significantly improve classification performance. However, for researchers, the collection of labeled data involves difficult, expensive and time-consuming ground motion. Therefore, in the literature on hyperspectral scenes, a small number of labeled samples can be used for image classification. The famous Hughes effect also shows that when the training samples are limited, the classification accuracy decreases with the increase of dimensionality. In addition, CNN based on spectral spatial features has better classification performance, which requires a more complex network and huge amount of computation to improve the effect of CNN-based HSI classification model. Summary of the invention

[0004] In order to solve the problem in the prior art that in the classification of hyperspectral images based on convolutional neural networks, large labeled samples of HSI are often used. This method requires a complex network and a huge amount of calculation, is time-consuming, has high hardware requirements and requires too high network quality. This patent proposes a segmentation convolutional neural network for the classification of small-size labeled samples HSI, which can reduce the calculation of various dimensions of feature maps and improve the classification performance of small-size labeled HSI through a deep network structure. In addition, the segmentation convolutional neural network can make full use of spectral spatial information to extract fusion features. Comparative experiments with other classification methods show that the segmentation convolutional neural network is suitable for small sample HSI classification and can obtain better classification performance.

[0005] In a first aspect of the present invention, a small sample hyperspectral image classification method based on a neural network is disclosed, comprising the following steps:

[0006] S10, acquiring all hyperspectral images to be classified;

[0007] S20, extracting a one-dimensional vector according to all the hyperspectral images to be classified;

[0008] S30, combining the one-dimensional vectors into a two-dimensional matrix;

[0009] S40, obtaining a two-dimensional feature map by performing dimensionality reduction and feature extraction on the two-dimensional matrix;

[0010] S50, reorganizing the two-dimensional feature map into a one-dimensional vector;

[0011] S60, classifying according to the number of one-dimensional vectors.

[0012] Preferably, the one-dimensional vector extracted in S20 is:

[0013] Extract a one-dimensional vector through convolution:

[0014]

[0015] g(x)=x+s x (h-1) (2)

[0016] In formula (1) and formula (2), is the convolution kernel k l,i One of the elements of , i is the index of the l-th layer kernel, Yes (l-1),i One of the elements of ; (x, y, z) and (g(x), g(y), g(z),) both indicate the position of the element in it, where It is one of the feature map elements generated by convolution in the three-dimensional process, h is the index; S x Indicates the stride of the convolution in the x direction;

[0017] output(m (l+1),i )=f(m l,i +b l,i ) (3)

[0018] Among them, l represents the layer considered, m l,i is the input, b l,i is with m l,i Corresponding to the deviation, the function f(·) represents the linear unit of correction, which is a non-saturated function, and the function m (l+1),i is the result of convolution.

[0019] Preferably, the feature extraction in S40 is:

[0020] Convolve in two dimensions:

[0021]

[0022] Among them, (α, β) represents the coordinates in the two-dimensional feature matrix.

[0023] In a second aspect of the present invention, an image defect detection device based on frequency domain filtering is disclosed, comprising:

[0024] Obtain hyperspectral image module to obtain all hyperspectral images to be classified;

[0025] One-dimensional vector module, extracts one-dimensional vectors based on all hyperspectral images to be classified;

[0026] The two-dimensional matrix module combines one-dimensional vectors into a two-dimensional matrix;

[0027] The two-dimensional feature map module obtains a two-dimensional feature map by reducing the dimension of the two-dimensional matrix and extracting features;

[0028] One-dimensional vector module, which reorganizes the two-dimensional feature map into a one-dimensional vector;

[0029] The classification module performs classification based on the number of one-dimensional vectors.

[0030] Preferably, the feature extraction in the one-dimensional vector module is:

[0031] Extract a one-dimensional vector through convolution:

[0032]

[0033] g(x)=x+s x (h-1) (2)

[0034] In formula (1) and formula (2), is the convolution kernel k l，i One of the elements of , i is the index of the l-th layer kernel, Yes (l-1),i One of the elements of ; (x, y, z) and (g(x), g(y), g(z),) both indicate the position of the element in it, where It is one of the feature map elements generated by convolution in the three-dimensional process, h is the index; S x Indicates the stride of the convolution in the x direction;

[0035] output(m (l+1),i )=f(m l,i +b l,i ) (3)

[0036] Among them, l represents the layer considered, m l,i is the input, b l,i is with m l,i Corresponding to the deviation, the function f(·) represents the linear unit of correction, which is a non-saturated function, and the function m (l+1),i is the result of convolution.

[0037] Preferably, the feature extraction in the one-dimensional vector module is:

[0038] Convolve in two dimensions:

[0039]

[0040] Among them, (α, β) represents the coordinates in the two-dimensional feature matrix.

[0041] This application proposes a neural network-based small sample hyperspectral image classification method, in which the dimension of the feature map undergoes four stages of change. In the case of small samples, this method is simpler and has better classification results than other CNN-based classification methods. By comparing the classification accuracy of this method with some state-of-the-art small-label HSI classifiers, it is proved that this method can achieve higher accuracy with a simpler CNN structure. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0043] Figure 1 A flowchart of a small sample hyperspectral image classification method based on a neural network disclosed in an embodiment of the present invention;

[0044] Figure 2It is a structural diagram of S20-S50 of a small sample hyperspectral image classification method based on a neural network disclosed in an embodiment of the present invention;

[0045] Figure 3 The classification result of the Indian pines dataset diagram disclosed in the embodiment of the present invention;

[0046] Figure 4 The classification result of the Pavia University dataset graph disclosed in the embodiment of the present invention;

[0047] Figure 5 The present invention discloses a structural block diagram of a small sample hyperspectral image classification based on a neural network. DETAILED DESCRIPTION

[0048] In order to solve the problem in the prior art that in the classification of hyperspectral images based on convolutional neural networks, large labeled samples of HSI are often used. This method requires a complex network and a huge amount of calculation, is time-consuming, has high hardware requirements and requires too high network quality. This patent proposes a segmentation convolutional neural network for the classification of small-size labeled samples HSI, which can reduce the calculation of various dimensions of feature maps and improve the classification performance of small-size labeled HSI through a deep network structure. In addition, the segmentation convolutional neural network can make full use of spectral spatial information to extract fusion features. Comparative experiments with other classification methods show that the segmentation convolutional neural network is suitable for small sample HSI classification and can obtain better classification performance.

[0049] See also Figure 1 , Figure 1 This is a flowchart of a small sample hyperspectral image classification method based on a neural network provided by the present application, which is applied to a corresponding device. Figure 1 As shown, the following steps are included:

[0050] S10, obtaining all hyperspectral images to be classified.

[0051] S20, extracting a one-dimensional vector according to all the hyperspectral images to be classified.

[0052] See also Figure 2 In this method, S20-S50 are all related to feature extraction. The main function of S20 is to extract three-dimensional features from the original HSI data. The input of the network is a three-dimensional cube of HSI, which maintains the original spectral spatial features. The input is convolved with a three-dimensional kernel function, and the spectral-spatial feature map is extracted at the same time. This step has 3 convolutional layers and 1 maximum pooling layer for feature extraction. As the number of layers increases, the number of kernels also increases. The final result is a number of one-dimensional vectors generated by the three-dimensional feature extraction.

[0053] The one-dimensional vector extracted in S20 is:

[0054] Extract a one-dimensional vector through convolution:

[0055]

[0056] g(x)=x+s x (h-1) (2)

[0057] In formula (1) and formula (2), is the convolution kernel k l,i One of the elements of , i is the index of the l-th layer kernel, Yes (l-1),i One of the elements of ; (x, y, z) and (g(x), g(y), g(z),) both indicate the position of the element in it, where It is one of the feature map elements generated by convolution in the three-dimensional process, h is the index; S x Indicates the stride of the convolution in the x direction;

[0058] output(m (l+1),i )=f(m l,i +b l,i ) (3)

[0059] Among them, l represents the layer considered, m l,i is the input, b l,i is with m l,i Corresponding to the deviation, the function f(·) represents the linear unit of correction, which is a non-saturated function and is faster than other saturated functions. The function m (l+1),i is the result of convolution.

[0060] S30, combining the one-dimensional vectors into a two-dimensional matrix.

[0061] The two-dimensional matrix contains spectral spatial features and has a lower dimension than HSI data.

[0062] S40, obtaining a two-dimensional feature map by performing dimensionality reduction and feature extraction on the two-dimensional matrix.

[0063] The feature extraction process in this step has three convolutional layers and three maximum pooling layers. The result of the third stage is a series of two-dimensional feature maps after dimensionality reduction and feature extraction. Two fully connected layers, dropout layers, and softmax layers for classification are in S50.

[0064] The feature extraction in S40 is as follows:

[0065] Extract a one-dimensional vector through convolution:

[0066]

[0067] Among them, (α, β) represents the coordinates in the two-dimensional feature matrix.

[0068] S50, reorganizing the two-dimensional feature map into a one-dimensional vector;

[0069] S60, classifying according to the number of one-dimensional vectors.

[0070] Specifically, Figure 3 This is a structural diagram of S20-S50 of a small sample hyperspectral image classification method based on a neural network in this application.

[0071] In the 3D feature extraction process of this method, there are three convolutional layers and one maximum pooling layer; in the 2D feature extraction process, there are three convolutional layers and three maximum pooling layers. The number and size of kernels in feature extraction are shown in the following table.

[0072] Indian Pines Dataset Pavia University Dataset OA (%) 87.87±0.41 98.18±0.29 AA(%) 82.88±1.44 97.20±0.31 K 0.862±0.005 0.976±0.004

[0073] In addition, the adaptive moment estimation (Adam) algorithm is applied to this method for random optimization, with a dropout rate of 0.8 and a learning rate of 0.0001. This method selects the spatial area of ​​10*10 Indian Pines HSI cubes and sets the batch size to 40. The spatial area size of Pavia University is set to 8*8 and the batch size is set to 150. When the cross entropy value is less than 5*10 -6 When , the training stops.

[0074] In addition, the method was implemented based on the TensorFlow framework and experiments were conducted on Intel Core i7-6700@3.4GHz for CPU, NVIDIA GTX 1080Ti for GPU and 16GB RAM. Each experiment was conducted 10 times to achieve the average accuracy, and several state-of-the-art classification methods were considered for comparison. In the experiment, we used overall accuracy (OA), average accuracy (AA) and kappa coefficient to evaluate the classification performance of the method. The classification results of the method on the Indian Pines dataset are shown in the following table.

[0075]

[0076] The classification results on the Pavia University dataset are shown in the following table.

[0077]

[0078] The OA of this method for Indian Pines is 87.87%, AA is 82.88%, and AA is 0.862. The OA of Pavia University is 98.18%, AA is 97.20%, and AA is 0.976.

[0079] In addition, in the experiment, some state-of-the-art methods were selected for HSI classification and compared with this method using small-sized labeled samples. The table lists the classification results of support vector machine

[11] , OMP

[12] , NRS

[13] , CRT

[14] , KCRT

[14] and DKCRT

[15] on the Indian pines and Pavia University datasets. Figure 3 and Figure 4 These methods are effective methods for small sample labeled hyperspectral classification and have been certified by researchers.

[0080] It is obvious from the data in the table that the classification performance of this method is better than other methods. In 5 out of 16 classes in Indian Pines and 8 out of 9 classes in Pavia University, this method achieved better classification accuracy than other methods. At the same time, it is obvious that for this method, the more samples in each class, the higher the classification accuracy. In addition, the classification performance of this method is closely related to the number of samples.

[0081] For further reference, Figure 5 , a small sample hyperspectral image classification device based on a neural network disclosed in an embodiment of the present invention further includes:

[0082] Obtain hyperspectral image module to obtain all hyperspectral images to be classified;

[0083] One-dimensional vector module, extracts one-dimensional vectors based on all hyperspectral images to be classified;

[0084] The two-dimensional matrix module combines one-dimensional vectors into a two-dimensional matrix;

[0085] The two-dimensional feature map module obtains a two-dimensional feature map by reducing the dimension of the two-dimensional matrix and extracting features;

[0086] One-dimensional vector module, which reorganizes the two-dimensional feature map into a one-dimensional vector;

[0087] The classification module performs classification based on the number of one-dimensional vectors.

[0088] Many specific details are described in the above description to facilitate a full understanding of the present invention. However, the above description is only a preferred embodiment of the present invention. The present invention can be implemented in many other ways different from those described herein, so the present invention is not limited to the specific implementation disclosed above. At the same time, any person familiar with the art can make many possible changes and modifications to the technical solution of the present invention using the methods and technical contents disclosed above without departing from the scope of the technical solution of the present invention, or modify it into an equivalent embodiment of equivalent changes. Any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still falls within the scope of protection of the technical solution of the present invention.

Claims

1. A small sample hyperspectral image classification method based on neural network, characterized in that: The following steps are involved: S10, acquiring all hyperspectral images to be classified; S20, extracting a one-dimensional vector according to all the hyperspectral images to be classified; The one-dimensional vector extracted in S20 is: Extract a one-dimensional vector through convolution: g(x)=x+s x (h-1) (2) In formula (1) and formula (2), is the convolution kernel k l,i One of the elements of , i is the index of the l-th layer kernel, Yes (l-1),i One of the elements of ; (x, y, z) and (g(x), g(y), g(z),) both indicate the position of the element in it, where It is one of the feature map elements generated by convolution in the three-dimensional process, h is the index; S x Indicates the stride of the convolution in the x direction; output(m (l+1),i )=f(m l,i +b l,i ) (3) Among them, l represents the layer considered, m l,i is the input, b l,i is with m l,i Corresponding to the deviation, the function f(·) represents the linear unit of correction, which is a non-saturated function, and the function m (l+1),i is the result of convolution; S30, combining the one-dimensional vectors into a two-dimensional matrix; S40, obtaining a two-dimensional feature map by performing dimensionality reduction and feature extraction on the two-dimensional matrix; The feature extraction in S40 is as follows: Convolve in two dimensions: Among them, (α, β) represents the coordinates in the two-dimensional feature matrix; S50, reorganizing the two-dimensional feature map into a one-dimensional vector; S60, classifying according to the number of one-dimensional vectors.

2. A small sample hyperspectral image classification device based on neural network, characterized in that: include: Obtain hyperspectral image module to obtain all hyperspectral images to be classified; One-dimensional vector module, extracts one-dimensional vectors based on all hyperspectral images to be classified; The feature extraction in the one-dimensional vector module is: Extract a one-dimensional vector through convolution: g(x)=x+s x (h-1) (2) In formula (1) and formula (2), is the convolution kernel k l,i One of the elements of , i is the index of the l-th layer kernel, Yes (l-1),i One of the elements of ; (x, y, z) and (g(x), g(y), g(z),) both indicate the position of the element in it, where It is one of the feature map elements generated by convolution in the three-dimensional process, h is the index; S x Indicates the stride of the convolution in the x direction; output(m (l+1),i )=f(m l,i +b l,i ) (3) Among them, l represents the layer considered, m l,i is the input, b l,i is with m l,i Corresponding to the deviation, the function f(·) represents the linear unit of correction, which is a non-saturated function, and the function m (l+1),i is the result of convolution; The two-dimensional matrix module combines one-dimensional vectors into a two-dimensional matrix; The two-dimensional feature map module obtains a two-dimensional feature map by reducing the dimension of the two-dimensional matrix and extracting features; One-dimensional vector module, which reorganizes the two-dimensional feature map into a one-dimensional vector; The feature extraction in the one-dimensional vector module is: Convolve in two dimensions: Among them, (α, β) represents the coordinates in the two-dimensional feature matrix; The classification module performs classification based on the number of one-dimensional vectors.

Images