A hyperspectral image classification method based on passive migration network

Abstract

A hyperspectral image classification method based on passive migration network belongs to the technical field of remote sensing image processing and application, the present application designs the target function optimization source domain model for preventing the overfitting of neural network, reduces the weight of the class of real sample label when calculating the loss function, and plays the effect of inhibiting overfitting. The source domain model is migrated to the target data, the target function is adjusted, the target domain feature output has individual certainty and global diversity. At the same time, in order to prevent the target domain data from being matched with the wrong source domain to a certain extent, the model training is carried out by using the pseudo label technology. The present application only utilizes the source domain model, does not need to store and utilize the source domain data, migrates the classification model to the target domain data, and provides an effective hyperspectral image passive migration classification solution.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of remote sensing image processing and application, and particularly relates to a hyperspectral image classification method based on a passive migration network. BACKGROUND

[0002] Hyperspectral images have high-resolution spectral information and large-width spatial information, and can realize large-width, fine-grained and high-precision earth observation tasks. Hyperspectral remote sensing image classification is an important research content in the field of remote sensing, and through analysis of corresponding spectral-spatial information, a semantic label is assigned to each pixel of the hyperspectral image, which is a fundamental step and key problem of various remote sensing applications. High-performance hyperspectral image classification technology will greatly improve the research accuracy of ground objects, thereby improving the efficiency of earth observation.

[0003] People have made a lot of explorations on hyperspectral classification methods and models, and have achieved excellent results:

[0004] Zhang Chunmin et al. (Reference: Zhang Chunmin, Mu Tingkui, Yan Tingyu, Chen Zeyu. Development and prospect of hyperspectral remote sensing technology [J]. Spaceflight and remote sensing, 2018, 39(3): 104-114.) analyzed the current development status of Chinese hyperspectral remote sensing technology, summarized the international future development plan of hyperspectral remote sensing technology, and looked forward to the future development of hyperspectral remote sensing technology, and pointed out the development trend of hyperspectral remote sensing image, spectral, polarization multi-information integration, intelligentization, networking and small and light, which can provide reference for the further maturity and practicality of Chinese hyperspectral remote sensing technology.

[0005] Yang Jinnan et al. (Reference: Yang Jinnan, Lai Wenkui. Application status and development trend of deep learning algorithm in remote sensing image classification and recognition [J]. Surveying and Spatial Geoinformation, 2020, 43(04): 114-117.) summarized the problems and development trend of deep learning in remote sensing image classification and recognition. This paper briefly describes the application of three classical deep learning model structures in remote sensing image classification and recognition. Deep belief network algorithm is composed of multiple groups of restricted Boltzmann machines with good unsupervised learning ability; convolutional neural network can well reduce the dimension of high-dimensional image and realize its classification and recognition; self-encoder produces less reconstruction error in feature extraction and dimension reduction, but needs to be combined with other classifiers to obtain high accuracy.

[0006] Liu et al. (Reference: Liu, Y., Liu, R., Li, D. X., Yang, F. C. Hyperspectral image classification algorithm based on joint spatial-spectral collaborative representation[J]. Computer Engineering and Design, 2020, 41(03): 815-820.) proposed an improved joint spatial-spectral collaborative representation classification algorithm to address the problem of insufficient utilization of spatial and spectral information in hyperspectral image classification. The spectral information was obtained by calculating the Euclidean distance between the training pixels and the test pixels, and the spatial information was calculated between the test pixels and the training pixels. A spatial joint representation classification model was established by combining the spatial information and the spectral information, and the least residual method was used to realize the classification of hyperspectral images.

[0007] Zhao et al. (Reference: Zhao, C. J., Zhou, S. G., Liu, L. L., Ding, Q. Hyperspectral image classification based on homogeneous region and transfer component analysis[J]. Computer Engineering and Applications, 2019.) improved the traditional transfer component analysis (TCA) using image homogeneous region information, and implemented one transfer component analysis and classification using a randomly selected point to represent a target domain homogeneous region patch. The majority of the multiple classification results of the pixels in the same patch was used as the common class of the target domain homogeneous region patch.

[0008] The above methods mostly use deep networks to extract spatial-spectral features and classify hyperspectral images, and try to realize hyperspectral image transfer classification to assist the target domain data classification task using the information in the source domain data. However, most existing transfer classification methods need to access both source domain data and target domain data during model transfer. Due to storage conditions and data ownership, etc., the source domain dataset cannot be shared, and these limitations limit the use of hyperspectral transfer classification methods in practical scenarios. Therefore, in the era of big data, how to realize hyperspectral image transfer classification without touching the source domain data, but only touching the pre-trained model of the source domain, is a key problem for hyperspectral image classification to move towards practical application. SUMMARY

[0009] To solve the above problems, the present application proposes a hyperspectral image classification method based on a passive transfer network, which includes three stages. The first stage is to optimize the source domain model through a loss function that prevents overfitting. The second stage is to transfer the pre-trained source domain model to the target domain data, and adjust the feature extractor of the model through the optimization of the target function to make the target domain feature output have individual certainty and global diversity. The third stage is to train the model through pseudo-label technology to prevent the target domain data from matching with the wrong source domain, and to retrain the feature extractor of the model through the labeled target domain data.

[0010] Further, the first stage uses the source domain data to train the source domain model as follows: S1.1, read the hyperspectral images of the source domain X s and the target domain X t for preprocessing:

[0011] Linear normalization is performed on the spectral features of the source domain and the target domain, and the linear normalization process is as follows: the spectral features are linearly transformed by using the maximum value and the minimum value of the spectral features, so that the spectral features are distributed between 0 and 1, and the definition formula of linear normalization is as follows:

[0012]

[0013] Wherein: X is the input original data, X min and X max are the minimum value and the maximum value of the data respectively.

[0014] Further, S1.2, a convolution layer of a single convolution kernel is constructed to extract spatial features:

[0015] The layer is taken as the first layer of feature extraction, the value of each pixel point is obtained by weighted average of itself and other pixel values in its neighborhood according to different weights, and the classification uncertainty caused by the same object and different spectrum and the same spectrum and different object is reduced.

[0016] Further, S1.3, two fully connected layers are constructed to extract spectral features:

[0017] The first fully connected layer takes the output of the convolution layer as the input, and the second fully connected layer takes the output of the first fully connected layer as the input, and the spectral feature of a certain source domain sample is extracted. The embedding function of the feature extraction process is f s (·), the fully connected weight is W, the corresponding bias parameter is b, and the output feature vector is:

[0018]

[0019] Further, S1.4, a softmax classifier is used to classify the spectral features;

[0020] The embedding function of the classifier is g s (·), the classifier weight is W', the corresponding bias parameter is b', and the output of the classifier is:

[0021]

[0022] S1.5, the network parameters are optimized;

[0023] A regularization strategy is used to optimize the network parameters, and the weight of the class of the real sample label in the calculation of the loss function is reduced, so that the effect of inhibiting overfitting is achieved, y s is the real label of the sample, represents the vector output by the classifier The k-th element, where K is the number of classes, M is the number of samples, ε is set to 0.1, and Loss source The formula is as follows:

[0024]

[0025]

[0026] Furthermore, the second stage of source domain model migration to target data processing steps is as follows: S2.1, Optimize the objective equation;

[0027] When inputting target domain data, the loss function of the network model is adjusted as follows:

[0028]

[0029]

[0030] Where E represents the mean. Let y be the mean of the vectors processed by the softmax classifier. t The true labels for the target domain samples. The vector representing the output of the classifier The k-th element, where K is the number of categories and N is the number of samples.

[0031] Furthermore, the third stage of self-supervised pseudo-label training processing steps are as follows: S3.1, obtain the center of each class;

[0032] Suppose the number of samples assigned to a certain class is S, the centroid c of that class is obtained using a method similar to weighted K-means clustering. 0 :

[0033]

[0034] Furthermore, S3.2, assign labels to the target domain samples;

[0035] Pseudo-labels are assigned to the target samples based on the cosine distance between the target domain samples and the obtained class centers. The feature extractor of the model is then retrained using the labeled target domain data. The specific process of assigning labels is as follows:

[0036]

[0037] in Let D(a,b) represent the value of variable k that minimizes the objective function F(·), and let D(a,b) represent the cosine distance between vectors a and b, i.e., the cosine of the angle between a and b in the vector space. These are pseudo-labels assigned to samples in the target domain.

[0038] The beneficial effects of the present application are: the target function optimization source domain model designed to prevent overfitting of the neural network reduces the weight of the class of real sample labels when calculating the loss function, which plays a role in inhibiting overfitting. The source domain model is migrated to the target data, and the target function is adjusted to make the target domain feature output have individual certainty and global diversity. At the same time, in order to prevent the target domain data from being matched with the wrong source domain to a certain extent, the pseudo-label technology is used for model training. The present application only uses the source domain model and does not need to store the source domain data, and migrates the classification model to the target domain data, providing an effective hyperspectral image source-free migration classification solution. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 The schematic diagram of the present application.

[0040] Figure 2 The source domain Pavia University hyperspectral scene data;

[0041] Figure 3 The target domain Pavia Center hyperspectral scene data;

[0042] Figure 4 The Pavia Center classification result map using the method of the present application. DETAILED DESCRIPTION

[0043] The purpose of the present application is to overcome the defects of the prior art and provide a hyperspectral image classification method based on source-free migration network. The method is used to solve the problem that the existing hyperspectral image migration classification is difficult to be separated from the source domain data, and the target domain feature module is learned through the adjustment of the network model loss function and the self-supervised pseudo-label method.

[0044] The embodiment as shown in Figure 1 The technical effects of the present application are illustrated by taking Pavia University and Pavia Center hyperspectral image data sets as examples, Figure 2 The pseudo-color image of Pavia University and the ground truth map with color identification are shown in Figure 3Pseudo-color image and ground truth map with color labels of Pavia Center are shown, containing land cover type labels 1-9. Pavia University and Pavia Center hyperspectral images contain 7 same land cover types. 39332 known samples of Pavia University are used as source domain dataset, and 39355 known samples of Pavia Center are used as target dataset. The present application performs land cover target recognition on Pavia Center hyperspectral image, including the following steps:

[0045] 1. Read the hyperspectral images of the source domain and the target domain, and perform feature normalization preprocessing:

[0046] 1.1 Linearly normalize the spectral features of the hyperspectral images of the source domain and the target domain respectively, so that they are distributed between 0 and 1.

[0047] 2. Train the source domain model using 7 labeled training samples in the source domain data:

[0048] 2.1 Build a convolution layer with a single convolution kernel as the first layer of feature extraction for spatial feature extraction.

[0049] 2.2 Build 2 fully connected layers to extract spectral features, the first fully connected layer takes the output of the convolution layer as input, and the second fully connected layer takes the output of the first fully connected layer as input. For a source domain sample x s , the embedding function of the feature extraction process is f s (·), the fully connected weight is W, the corresponding bias parameter is b, and the output feature vector r s is:

[0050] r s =f s (Wx s +b)

[0051] 2.3 Classify the extracted spatial-spectral features using a softmax classifier, the embedding function of the classifier is g s (·), and the output of the classifier is z s =g s (f s (x))。

[0052] 2.4 A regularization strategy is used to optimize the network parameters, reduce the weight of the class of the true sample label in the calculation of the loss function, and suppress the effect of overfitting. The specific Loss source formula is as follows:

[0053]

[0054] Losssource = {(1-ε)*Loss1,if(y=k); ε*Loss1,if(y≠k)}

[0055] where y s is the true label of the sample, is the k-th element of the vector output by the classifier, K is the number of classes, M is the number of samples, and ε is set to 0.1.

[0056] 3. Input the unlabeled target domain data to train the feature extractor of the source domain model.

[0057] 3.1 Adjust the loss function of the network model when inputting the target domain data, so that the classification probability entropy of each sample on the target domain is as small as possible, and the average probability predicted by all samples is as uniform as possible. The specific formula is as follows:

[0058]

[0059]

[0060] where E is the mean value, is the mean value of the vector after the softmax classifier, y t is the true label of the target domain sample, is the k-th element of the vector output by the classifier, K is the number of classes, and N is the number of samples.

[0061] 4. Assign labels to the target domain data using the pseudo-labeling technique and train the feature extractor.

[0062] 4.1 Assign labels to the target domain samples using the pseudo-labeling technique, where D(·) represents the cosine distance. Assume that the number of samples assigned to a certain class is S, c 0 is the center value of a certain class, is the pseudo label assigned to the target domain sample. Train the feature extractor using the labeled target domain samples. The specific process of assigning labels is as follows:

[0063]

[0064]

[0065] 5. In the testing phase, use the classifier of image 2 to classify the images of the target domain. According to the predicted label vector and the spatial position of the test samples, output the classification result map of the target domain hyperspectral image, as shown in Figure 4 . The specific results of the target domain hyperspectral image classification are shown in Table 1.

[0066] The results show that the hyperspectral image classification model migration method based on the passive migration network can migrate the classification model to new data, and identify new categories without exposing the source domain data and without any target domain label samples.

[0067] Table 1 comparison of classification accuracy

[0068]

[0069]

[0070] The above describes only the preferred specific embodiments of the present application, but the protection scope of the present application is not limited thereto, any skilled person in the art within the technical range disclosed by the present application, according to the technical solution and concept of the present application, equivalent replacement or change, should be covered within the protection scope of the present application.

Claims

1. A hyperspectral image classification method based on passive transfer networks, characterized in that, It includes three stages: The first stage involves optimizing the source domain model using a loss function to prevent overfitting. Optimize network parameters: A regularization strategy is employed to optimize network parameters, reducing the weight of the true sample label category in the loss function calculation, thereby suppressing overfitting. For the true label of the sample, The vector representing the output of the classifier The One element, For the number of categories, For the sample size, Set to 0.1, The formula is as follows: ； The second stage involves transferring the pre-trained source domain model to the target domain data. This is achieved by optimizing the objective function and adjusting the model's feature extractor to ensure that the target domain feature output possesses both individual determinism and global diversity. The steps for this second stage of source domain model transfer to target data processing are as follows: Optimizing the objective equation: When inputting target domain data, the loss function of the network model is adjusted, as shown in the following formula: in, To find the mean, The mean of the vectors after passing through the softmax classifier. The true labels for the target domain samples. The vector representing the output of the classifier The One element, For the number of categories, The number of samples; The third stage involves training the model using pseudo-labeling techniques to prevent the target domain data from matching the incorrect source domain, and then retraining the model's feature extractor using labeled target domain data.

2. The hyperspectral image classification method based on passive transfer networks as described in claim 1, characterized in that, The steps for training the source domain model using source domain data in the first stage are as follows: S1.1, Read the source domain data. and target domain Preprocessing of hyperspectral images: The spectral features of the hyperspectral images in the source and target domains are linearly normalized. The linear normalization process involves linearly transforming the spectral features using their maximum and minimum values ​​to ensure they fall between 0 and 1. The definition formula for linear normalization is as follows: in: The original input data, and These are the minimum and maximum values ​​of the data, respectively.

3. The hyperspectral image classification method based on passive transfer networks as described in claim 2, characterized in that, S1.2 Construct a convolutional layer with a single convolutional kernel for spatial feature extraction: This layer is used as the first layer for feature extraction. The value of each pixel is obtained by weighted averaging of its own value and the values ​​of other pixels in its neighborhood according to different weights, thereby reducing the classification uncertainty caused by the phenomenon of different spectra of the same object and different objects of the same spectra.

4. The hyperspectral image classification method based on passive transfer networks as described in claim 3, characterized in that, S1.3 Construct two fully connected layers to extract spectral features: The first fully connected layer takes the output of the convolutional layer as input, and the second fully connected layer takes the output of the first fully connected layer as input, for a given source domain sample. The embedding function for the feature extraction process is: The weight of the fully connected layer is The corresponding bias parameters are The output feature vector for: 。 5. The hyperspectral image classification method based on passive transfer networks as described in claim 4, characterized in that, S1.

4. Use a softmax classifier to classify spatial spectral features; The embedding function of the classifier is The classifier weights are The corresponding bias parameters are The classifier output is: 。 6. The hyperspectral image classification method based on passive transfer networks as described in claim 1, characterized in that, The third stage of self-supervised pseudo-label training involves the following steps: S3.1, obtaining the center of each class; Suppose the number of samples assigned to a certain class is The centroids of the clusters are obtained using weighted K-means clustering. : 。 7. The hyperspectral image classification method based on passive transfer networks as described in claim 6, characterized in that, S3.2 Assign labels to the target domain samples: Pseudo-labels are assigned to the target samples based on the cosine distance between the target domain samples and the obtained class centers. The feature extractor of the model is then retrained using the labeled target domain data. The specific process of assigning labels is as follows: in This indicates that the objective function The variable that can reach the minimum value The value, Representative vector sum vector The cosine distance, i.e., in vector space and The cosine of the included angle, These are pseudo-labels assigned to samples in the target domain.

Images