Label noise-containing hyperspectral image crop classification based on 3DCNN combined with MLP

Abstract

The invention provides a label noise-containing hyperspectral image crop classification method based on 3DCNN combined with MLP, and belongs to the technical field of image classification. According to the method, firstly, spatial and spectral features are extracted at the same time through a 3D convolutional neural network (3D Convolutional Neural Network, 3DCNN), and three-dimensional information of a hyperspectral image is fully utilized. Then, a multi-layer perceptron (MLP) is introduced to carry out classification decision making, so that the advantages of the multi-layer perceptron (MLP) in processing a high-dimensional feature space are fully exerted; in order to further enhance the robustness of the model to the noise label, the method introduces a loss function mixed by Normalized Cross Entropy (NCE) and Reversed Cross Entropy (RCE) while training the sample, and reduces the interference of the noise sample to the model. The powerful feature extraction capability of the 3DCNN and the excellent classification performance of the MLP are comprehensively utilized, and a mixed loss function is combined to finally obtain the crop category. According to the method, feature extraction and classification decision advantages of 3DCNN and MLP in hyperspectral image classification are fully utilized, and a mixed loss function of RCE and NCE is combined, so that a classification model which has robustness to noise labels and is more accurate is constructed. According to the innovative method, the processing capacity of the noise label in the hyperspectral image is remarkably improved, and the understanding and characterization of the model on the high-dimensional feature space are enhanced. In a hyperspectral image classification task of a noise label, the method shows excellent classification performance, and has important scientific and practical values for improving the accuracy and robustness of hyperspectral image crop classification.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing image classification, and in particular to a method for classifying crops in label-noised hyperspectral images based on 3DCNN combined with MLP. Background Art

[0002] Hyperspectral images (HSI) have rich information and subtle spectral differentiation capabilities, and are widely used in mineral identification, military reconnaissance, precision agriculture and other fields. Traditional classification algorithms, such as support vector machines and decision trees, have largely met basic classification needs. However, in practical applications, hyperspectral images often reduce classification accuracy due to noise, especially in crop classification tasks, where the noise problem is more prominent due to the complexity of spectral features and interference from environmental factors. In particular, when the label data itself is noisy, the performance of the classification model will be further impaired. This noisy label originates from sensor errors, data transmission problems or human labeling errors. Therefore, how to achieve efficient and accurate hyperspectral image crop classification in the presence of noisy labels has become a crucial research direction. To address this problem, researchers have proposed a variety of methods and conducted extensive explorations, including data preprocessing (such as spectral smoothing, principal component analysis) and robust classification models based on deep learning. These methods have shown significant effects in reducing noise interference and improving classification accuracy.

[0003] Faced with this challenge, researchers began to explore more powerful models, such as Multilayer Perceptron (MLP) and 3D Convolutional Neural Network (3DCNN). Both models have shown significant advantages in classifying hyperspectral images with high noise. Multilayer Perceptron (MLP) is a classic shallow neural network. Its powerful information fusion capability, adjustable network depth and number of nodes, and optimized gradient descent algorithm all give MLP excellent noise information processing and classification performance. 3DCNN can more comprehensively retain the spatial and spectral information in hyperspectral images by performing convolution operations in both spatial and spectral domains, further enhancing the expression ability of the model and having significant advantages in processing hyperspectral images. However, 3DCNN and MLP each have their own limitations. The main problem faced by 3DCNN is the high computational complexity, while MLP has the problem of overfitting when processing large data sets.

[0004] Therefore, the present invention obtains a more robust model by combining these two models. That is, 3DCNN is used to extract spatial and spectral features simultaneously in the early stage, and MLP is used for classification decision in the later stage. The strategy of the two combined models is better than using any model alone, because the advantages of 3DCNN and MLP can complement their shortcomings, such as the powerful spatial feature extraction ability of 3DCNN can solve the data processing problem of MLP, and the excellent classification performance of MLP can make up for the computational complexity of 3DCNN. In addition, the method of combining the mixed loss function of normalized cross entropy (NCE) and reverse cross entropy (RCE) can further improve the robustness of the model, thereby improving the accuracy and stability of classification. Summary of the invention

[0005] The present invention makes full use of the feature extraction and classification decision-making advantages of 3DCNN and MLP in hyperspectral image classification, and combines the mixed loss function of RCE and NCE to construct a classification model that is robust to noisy labels and more accurate. This innovative method not only significantly improves the processing ability of noisy labels in hyperspectral images, but also enhances the model's understanding and representation of high-dimensional feature space. In the task of hyperspectral image classification with noisy labels, the present invention demonstrates excellent classification performance, which has important scientific and practical value for improving the accuracy and robustness of hyperspectral image crop classification.

[0006] The object of the present invention is achieved in that:

[0007] A method for crop classification of label-noisy hyperspectral images based on 3DCNN combined with MLP includes the following steps:

[0008] Step a: Input the HSI data to be classified, randomly select training samples, and construct training sets and test sets;

[0009] Step b: construct 3DCNN to extract the spatial and spectral features of HSI and make full use of the three-dimensional information of hyperspectral images;

[0010] Step c: Design an MLP model to make classification decisions on the extracted features, giving full play to the advantages of MLP in processing high-dimensional feature space;

[0011] Step d: In order to enhance the robustness of the model to noisy labels, a hybrid loss function of NCE and RCE is combined to reduce the interference of noisy labels on model training.

[0012] Step e: Comprehensively utilize the feature extraction capability of 3DCNN and the classification performance of MLP, and combine the combined hybrid loss function to complete HSI classification and obtain the final classification result.

[0013] The above-mentioned method for crop classification of label-noisy hyperspectral images based on 3DCNN combined with MLP is characterized in that step b specifically includes the following steps:

[0014] Step b1: Patch extraction is performed on the training and test samples of the hyperspectral data, with a window size of 7×7×nBand.

[0015] Step b2: Construct a 3DCNN model, which includes 3D convolution and ReLU activation functions. By stacking multiple layers of 3D convolution, the network can gradually extract and learn more abstract and discriminative features from shallow to deep layers. At the same time, this multi-layer structure also enables the model to effectively learn the correlation between multiple bands, thereby significantly enhancing the processing capabilities of noise suppression and complex classification tasks.

[0016] The above-mentioned method for crop classification of label-noisy hyperspectral images based on 3DCNN combined with MLP is characterized in that in the step c, an MLP model is constructed, which includes an input layer, multiple hidden layers, and an output layer. The main task of the input layer is to receive data and distribute it to the hidden layer according to different weights. Then, the hidden layer processes the input data through a nonlinear activation function, passes the processed information to the output layer, and finally gives the classification result from the output layer.

[0017] The above-mentioned method for crop classification of hyperspectral images with label noise based on 3DCNN combined with MLP is characterized in that in the step d, a hybrid loss function is constructed, and the model's processing ability for noise samples can be better improved after combining the NCE and RCE loss functions, because NCE allows the model to learn to distinguish between real samples and noise samples, thereby enhancing the robustness of the model, while RCE makes the model more robust when processing abnormal or noisy data by reversing the traditional cross entropy loss. The combination of these two loss functions helps the model to extract useful features more effectively when facing noise samples, reduce overfitting, and thus improve classification performance.

[0018] The above-mentioned method for crop classification of labeled noisy hyperspectral images based on 3DCNN combined with MLP is characterized in that, in the step e, 3DCNN and MLP are connected in a cascade manner to form an overall architecture, and the noise samples are trained with a joint mixed loss function for classification to obtain the final crop classification result.

[0019] Beneficial effects:

[0020] The present invention relates to the field of crop classification of remote sensing images based on a label noisy hyperspectral image based on 3DCNN combined with MLP. First, the present invention simultaneously extracts spatial and spectral features through 3DCNN, making full use of the three-dimensional information of the hyperspectral image. Then, MLP is introduced to make classification decisions to give full play to its advantages in processing high-dimensional feature space. In order to further enhance the robustness of the model to noisy labels, the present invention introduces a mixed loss function of NCE and RCE while training samples to reduce the interference of noise samples on the model. The powerful feature extraction capability of 3DCNN and the excellent classification performance of MLP are comprehensively utilized, combined with the mixed loss function, and finally the crop category is obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is an overall flow chart of the crop classification method for label-noised hyperspectral images based on 3DCNN combined with MLP in the method of the present invention.

[0022] Figure 2 It is a pseudo-color image of the Pavia Center dataset in the method of the present invention and its corresponding ground truth map.

[0023] Figure 3 It is a schematic diagram of the principle of 3DCNN in the method of the present invention.

[0024] Figure 4 It is a diagram of the MLP structure in the method of the present invention.

[0025] Figure 5 It is the classification result diagram of the Pavia Center data set in the method of the present invention. DETAILED DESCRIPTION

[0026] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings.

[0027] A method for crop classification of hyperspectral images with label noise based on 3DCNN combined with MLP in a specific embodiment of the present invention is shown in the flowchart as follows: Figure 1 As shown, the following steps are included:

[0028] Step a: input the hyperspectral image data to be classified, randomly select training samples, and construct training sets and test sets;

[0029] In a specific embodiment of the present invention, the Pavia Center dataset is used as experimental data. The Pavia Center dataset was captured by the ROSIS sensor in a flight mission in Pavia, northern Italy. The data pixel is 1906×715 and the spatial resolution is 1.3m. The dataset has 102 spectral bands with a wavelength range of 430nm to 860nm. The image contains 9 categories of samples, with a total of 148,152 labeled samples. Table 1 shows the category and number of each category of samples in the Pavia Center dataset. Figure 2 Pseudo-color images and their corresponding ground truth maps are shown.

[0030] Table 1. Pavia Center dataset

[0031]

[0032] Step b: The schematic diagram of the 3DCNN principle of the present invention is as follows: Figure 3 As shown in Figure 3, the three-dimensional information of hyperspectral images is fully utilized by extracting the spatial and spectral features of HSI.

[0033] The specific implementation is:

[0034] Step b1: Patch extraction is performed on the training and test samples of the hyperspectral data, with a window size of 7×7×nBand.

[0035] Step b2: Input the HSI patch data into a 3DCNN consisting of 3D convolution and RelU activation function. By stacking multiple layers of 3D convolution, the network can gradually extract and learn more abstract and discriminative features from shallow to deep layers.

[0036] The specific steps are as follows:

[0037] The three-dimensional convolution operator is used to extract features. Its convolution kernel can perform convolution in three directions at the same time, and the obtained feature map is still a third-order tensor. The output of the i-th layer and the j-th feature map of the 3D-CNN convolution layer at the position (x, y, j) is:

[0038]

[0039] Where: f(·) is the activation function. In this paper, the ReLU activation function is used. The main advantage of ReLU is that it has a fixed derivative for all inputs greater than zero, which speeds up network training; K is the number of feature maps in the previous layer; W i , H i , C i are the width, height and spectral depth of the convolution kernel respectively; is the value of the convolution kernel at the spatial position (w, h, c) connecting the j-th convolution kernel of the i-th layer and the k-th feature map of the previous layer; is the value of the kth feature map output by the (i-1)th layer at the spatial position (x+w, y+h, z+c); C ij is the bias item corresponding to the jth feature map of the i-th layer.

[0040] Step c: The schematic diagram of the MLP structure constructed by the present invention is as follows: Figure 4 As shown in the figure, by inputting the features output from step b2 into the MLP model, combining the feature extraction capability of 3DCNN with the nonlinear fitting and fault tolerance capabilities of MLP, the model can process complex hyperspectral data more comprehensively, gradually reduce the impact of noise on model accuracy, and achieve more efficient noise suppression and classification effects.

[0041] The specific implementation is:

[0042] MLP consists of an input layer, a hidden layer, and an output layer. The main task of the input layer is to receive data and distribute it to the hidden layer according to different weights. Then, the hidden layer processes the input data through the GELU activation function, the formula is: The processed information is passed to the output layer, and the classification result is finally given from the output layer. The formula is: MLP(X) = FC2(GELU(FC1(X))). In the formula: Φ(X) represents the standard Gaussian cumulative distribution function,

[0043] Step d: For the classification of hyperspectral data containing noise samples, the present invention introduces a hybrid loss function that combines NCE and RCE to train the model of this article. Compared with other loss functions, the combined use can better improve the model's ability to handle noise samples, because NCE allows the model to learn to distinguish between real samples and noise samples, thereby enhancing the robustness of the model, while RCE reverses the traditional cross entropy loss to make the model more robust when dealing with abnormal or noisy data. The combination of these two loss functions helps the model to more effectively extract useful features when facing noise samples, reduce overfitting, and thus improve classification performance.

[0044] The specific implementation is:

[0045] Assuming the sample is x, the probability output of the classifier is Correspondingly, the true distribution of sample x on different labels is RCE changes the order of the true distribution q and the predicted probability p. Compared with the cross entropy loss, it is a reverse version. Therefore, RCE also reduces the possibility of wrong labels interfering with the model. The formula is: NCE is a standardized CE that maximizes the probability of the correct label, as follows: Subsequently, RCE is combined with NCE to form a hybrid noise robust loss function, with the formula: loss(p,q)=RCE(p,q)+NCE(p,q).

[0046] Step e: Comprehensively utilize the feature extraction capability of 3DCNN and the classification performance of MLP, and combine the combined hybrid loss function to complete HSI classification and obtain the final classification result.

[0047] The specific implementation is:

[0048] 3DCNN and MLP are connected in a cascade manner to form an overall architecture, and the mixed loss function is used to train noise samples for classification to obtain the final crop classification results.

[0049] The classification accuracy of the Pavia Center dataset used in this invention is shown in Table 2. The overall accuracy, average accuracy and Kappa coefficient are used as classification evaluation indicators. The classification results of the Pavia Center dataset are shown in Table 2. Figure 5 The experimental results show that the present invention has only a small amount of misclassification, is close to the actual crop distribution, and greatly reduces the area of ​​misclassification.

[0050] The experimental environment of the present invention is Intel (R) Xeon (R) CPU E5-2620 v4 @ 2.10GHz processor, 128GB memory and NVIDIA GeForce RTX 2080Ti GPU graphics card. In addition, the deep learning framework is Pytorch, which uses Python as the programming language. The initial learning rate is set to 0.00012, and the Adam optimization algorithm is used for optimization. The number of iterations is set to 1000. Other methods follow the parameters set by the authors of the paper. At the same time, in order to reduce the randomness brought by the training samples, each test is repeated 10 times and the average value is taken as the final test result.

[0051] Table 2 Classification accuracy of Pavia Center dataset classification methods

[0052]

Claims

1. Crop classification of label-noisy hyperspectral images based on 3DCNN combined with MLP, characterized by: The following steps are involved: Step a: Input the hyperspectral image (HSI) data to be classified, randomly select training samples, and construct training sets and test sets; Step b: construct a 3D convolutional neural network (3DCNN) to extract the spatial and spectral features of HSI and make full use of the three-dimensional information of the hyperspectral image; Step c: Design a multilayer perceptron (MLP) model to make classification decisions on the extracted features, giving full play to the advantages of MLP in processing high-dimensional feature space; Step d: In order to enhance the robustness of the model to noisy labels, a hybrid loss function of normalized cross entropy (NCE) and reverse cross entropy (RCE) is combined to reduce the interference of noisy labels on model training. Step e: Comprehensively utilize the feature extraction capability of 3DCNN and the classification performance of MLP, and combine the combined hybrid loss function to complete the HSI classification training and obtain the final classification result.

2. The crop classification of label-noised hyperspectral images based on 3DCNN combined with MLP according to claim 1, characterized in that: The step b specifically comprises the following steps: Step b1: Patch extraction is performed on the training and test samples of the hyperspectral data, with a window size of 7×7×nBand. Step b2: Construct a 3DCNN model, which includes 3D convolution and ReLU activation functions. By stacking multiple layers of 3D convolution, the network can gradually extract and learn more abstract and discriminative features from shallow to deep layers. At the same time, this multi-layer structure also enables the model to effectively learn the correlation between multiple bands, thereby significantly enhancing the processing capabilities of noise suppression and complex classification tasks.

3. The crop classification of label-noised hyperspectral images based on 3DCNN combined with MLP according to claim 1, characterized in that: In step c, an MLP model is constructed, which includes an input layer, multiple hidden layers, and an output layer. The main task of the input layer is to receive data and distribute it to the hidden layer according to different weights. Then, the hidden layer processes the input data through a nonlinear activation function, passes the processed information to the output layer, and finally gives the classification result from the output layer.

4. The crop classification of label-noised hyperspectral images based on 3DCNN combined with MLP according to claim 1, characterized in that: In step d, a hybrid loss function is constructed. Combining the NCE and RCE loss functions can better improve the model's ability to handle noise samples, because NCE allows the model to learn to distinguish between real samples and noise samples, thereby enhancing the robustness of the model, while RCE reverses the traditional cross entropy loss to make the model more robust when dealing with abnormal or noisy data. The combination of these two loss functions helps the model to more effectively extract useful features when facing noise samples, reduce overfitting, and thus improve classification performance.

5. The crop classification of label-noised hyperspectral images based on 3DCNN combined with MLP according to claim 1, characterized in that: In the step e, 3DCNN and MLP are connected in a cascade manner to form an overall architecture, and the noise samples are trained and classified using a combined mixed loss function to obtain the final crop classification result.

Images