A small sample high spectral classification method based on cross-domain spectral band alignment

Abstract

The present application relates to remote sensing information processing technical field, for solving the problem of low algorithm accuracy of existing hyperspectral classification method, poor auxiliary training effect of source domain data set and so on, and proposes a kind of small sample hyperspectral classification method based on cross-domain spectral band alignment, comprising: S1, reading the source domain data set D source And target domain data set D target Of hyperspectral data set, calculate the aligned spectral wavelength range and the aligned band number N align ;S2, by parabolic interpolation and resampling alignment method, the spectral wavelength range and band number N align Of two data sets are unified;S3, select training sample, collect training support set and training query set to construct small sample learning task S4, train embedding network F emb Determine network model parameters;S5, collect test support set and test query set from target domain data set, determine the predicted class label of each sample using nearest neighbor algorithm, and then complete the classification of small sample hyperspectral.

Description

Technical Field

[0001] This invention relates to the field of remote sensing information processing technology, specifically to a small-sample hyperspectral classification method based on cross-domain spectral band alignment. Background Technology

[0002] Hyperspectral remote sensing imagery boasts high spectral resolution and a large number of bands, enabling the acquisition of almost continuous spectral feature curves of ground objects. Furthermore, specific bands can be selected or extracted to highlight target features as needed. However, as the amount of training data required for target models increases, the time and manpower costs of labeling hyperspectral remote sensing data also gradually rise. This makes existing spectral training samples insufficient to support hyperspectral classification algorithms with massive datasets, and the inadequacy of existing spectral training samples has become a major bottleneck in the development of hyperspectral classification algorithms.

[0003] Currently, for pixel-level hyperspectral classification problems with only a small number of labeled samples, also known as few-shot hyperspectral classification, the prototype network algorithm is usually used to solve the few-shot hyperspectral classification problem. The prototype network is a method of few-shot learning. Its feature is that it uses an embedding network to extract the embedding features of the input data, calculates the average value of the embedding features of each class of samples as the feature prototype of that class, and uses the nearest neighbor algorithm to find the class prototype that is closest to the embedding features of the sample to be labeled, and uses this to determine the predicted label.

[0004] However, existing prototype network methods for small-sample hyperspectral classification mainly have the following problems: 1. Different datasets have different spectral ranges and band numbers, and existing algorithms do not handle them sufficiently; 2. The role of the source domain dataset is limited, resulting in low algorithm accuracy. These problems lead to poor performance of small-sample hyperspectral classification algorithms when using source domain datasets for training, or even a decrease in accuracy instead of an increase. The uneven distribution of data and other defects make it impossible to meet the classification accuracy requirements for spectral data. Summary of the Invention

[0005] The purpose of this invention is to address the problems of low algorithm accuracy, poor performance when using source domain datasets for training, and inability to meet the classification accuracy requirements of spectral data in existing small-sample hyperspectral classification methods. Therefore, this invention proposes a small-sample hyperspectral classification method based on cross-domain spectral band alignment.

[0006] The design concept of this invention is as follows: by using parabolic interpolation and resampling, the spectral response curves of hyperspectral images acquired by different sensors are unified in terms of spectral range and number of bands, which enhances the distribution consistency of cross-domain data, enabling source domain data to provide features that are more conducive to distinguishing target domain data. The alternating training strategy is used to improve the network's ability to learn cross-domain generalization knowledge, and the training strategy of lightweight embedded network and prototype network is used to reduce the overfitting phenomenon of data.

[0007] To achieve the above objectives, the technical solution proposed by this invention is as follows:

[0008] A small-sample hyperspectral classification method based on cross-domain spectral band alignment, characterized by the following steps:

[0009] S1. Read multiple hyperspectral datasets as source domain datasets D source Read the hyperspectral dataset to be classified as the target domain dataset D target Calculate the source domain dataset D source and target domain dataset D target Aligned spectral wavelength range The number of aligned bands N align ;

[0010] S2. Using parabolic interpolation and resampling alignment, align all source domain datasets D from step S1. source and target domain dataset D target Unify to the same spectral wavelength range Within, and uniformly set to the same number of bands N. align ;

[0011] S3, from source domain dataset D source Select qualified training samples from the dataset, and combine the categories of these qualified training samples together to form a hybrid source domain dataset. In mixed source domain datasets Select the number of categories and from the mixed source domain dataset and target domain dataset D target Training support set in China and training query set Constructing few-shot learning tasks

[0012] S4, via embedded network F emb Extract training support set Training support set samples Embedding features With training support set The average of the embedded features of samples from each category is used as the feature prototype f for that category. proto ; By calculating the training query set samples Embedding features With various feature prototypes f proto The Euclidean distance determines the class probability of the sample. For embedded network F emb Train the network model and determine its parameters;

[0013] S5. Based on the trained network model, from the target domain dataset D target Collection and testing support set and test query set By embedding network F emb Extract test support set samples Embedding features With test support set The average value of the features of each class of samples is used as the feature prototype f of that class. proto ; By embedding network F emb Extract test query set samples Embedding features It calculates the Euclidean distance between the sample and the feature prototype, uses the nearest neighbor algorithm to determine the predicted class label for each sample, and then completes the classification of small sample hyperspectral data with cross-domain spectral band alignment.

[0014] Furthermore, step S1 specifically includes:

[0015] Read multiple hyperspectral datasets as source domain dataset D source ={D1,D2,…,D n}, where the source domain dataset D source The number of bands in each hyperspectral dataset is N1, N2, ..., N. n The spectral wavelength ranges are respectively

[0016] Read the hyperspectral dataset to be classified as the target domain dataset D target Its number of bands is N target The spectral wavelength range is

[0017] Spectral wavelength range after alignment The following formula can be used to calculate:

[0018]

[0019] The number of bands N after alignment align The following formula can be used to calculate:

[0020] N align =min(N1,N2,…,N) n N target ).

[0021] Furthermore, step S2 specifically includes:

[0022] S2.1 Using parabolic interpolation, in the source domain dataset D... sourceFor each dataset, the coordinates (x1, y1), (x2, y2), and (x3, y3) of every three adjacent points within the original spectral wavelength range are collected and constructed into the following formula:

[0023]

[0024] By calculating the values ​​of a, b, and c, a local quadratic curve y = ax is obtained. 2 +bx+c, and then extend this process to the entire spectral wavelength range until the source domain dataset D is fitted. source All spectral response curves f1 of the corresponding dataset;

[0025] S2.2 Using parabolic interpolation, in the target domain dataset D target On each dataset within the target domain, the same fitting method as in step S2.1 is used to fit the target domain dataset D. target All spectral response curves f2;

[0026] S2.3, For the source domain dataset D source All spectral response curves f1 and target domain dataset D target All spectral response curves f2 in the aligned spectral wavelength range Equal-interval resampling is performed within the dataset to unify the number of bands in the two different datasets to the aligned number of bands N. align .

[0027] Furthermore, step S3 specifically includes:

[0028] S3.1, From the source domain dataset D source In each dataset, at least 200 training samples are selected for each category, and categories with fewer than 200 training samples are removed to maintain a balance in the number of training samples for different categories.

[0029] S3.2. Combine the categories of the training samples that meet the requirements selected in step S3.1 to form a mixed source domain dataset. This allows each time a dataset from a mixed source domain to be generated... The training samples randomly selected from D source Different datasets in China;

[0030] S3.3, Set the target domain dataset D target Each category in the dataset contains 5 labeled samples that meet the training selection criteria. The target domain dataset D is augmented using a rotation and cropping data augmentation method. target The number of labeled samples for each category has been expanded from 5 to 200;

[0031] S3.4, Set the target domain dataset Dtarget The number of land cover categories is n, making it follow the "n-way, k-shot" paradigm, for the mixed source domain dataset in step S3.2. Each time, n categories are randomly selected for the target domain dataset D. target Choose all n categories, and randomly select k samples from each category as the support set. Select m samples as the query set This together constitutes a small sample learning task.

[0032] Furthermore, step S4 specifically includes:

[0033] By embedding network F emb Using a mixed source domain dataset and target domain data D target Alternately perform iterative training; through embedding the network F emb Extract support set Training support set samples for each category Embedding features Extract the average of the embedded features of k samples from each class as the feature prototype f for that class. proto Then, through the embedded network F emb Extracting the training query set Training query set samples Embedding features And calculate the embedded features With the feature prototypes f of each category proto The Euclidean distance is calculated; finally, the Euclidean distance for each training query set sample is obtained using softmax. Predicted classification probability It satisfies the following formula:

[0034]

[0035] Where d(·) represents the calculation of Euclidean distance;

[0036] During training, the embedded network F emb loss function L fsl for:

[0037]

[0038] Among them, y i Indicates training query set samples The true category, L CE This represents the cross-entropy loss.

[0039] The beneficial effects of this invention are:

[0040] 1. This invention provides a small-sample hyperspectral classification method based on cross-domain spectral band alignment, which enhances the distribution alignment effect of cross-domain data. It uses parabolic interpolation to obtain more refined spectral data, and achieves alignment of the spectral response curves of hyperspectral images collected by different sensors in terms of spectral wavelength range and number of spectral bands. This enables deep neural networks to extract more generalized features, improves the positive effect of source domain data on the network classification effect, and effectively improves the classification accuracy and data alignment effect of the algorithm.

[0041] 2. In this invention, by adopting a multi-source domain hybrid alternating training strategy, the scope of use of source domain datasets is expanded, allowing more hyperspectral data to participate in training. Alternating training makes the network more robust to data domain switching, enhances the network's cross-domain learning ability, and improves the auxiliary training effect of source domain datasets.

[0042] 3. This invention trains the network model by embedding the network. Through lightweight design thinking, it uses one-dimensional convolution to reduce the network parameter scale, which can effectively improve the inference speed of hyperspectral datasets, reduce the overfitting phenomenon of each data domain, and also effectively improve the classification accuracy of spectral data. Attached Figure Description

[0043] Figure 1 This is a flowchart of an embodiment of a small-sample hyperspectral classification method based on cross-domain spectral band alignment according to the present invention;

[0044] Figure 2 This is a flowchart illustrating the method for aligning the spectral response curves of the source domain dataset and the target domain dataset in an embodiment of the present invention.

[0045] Figure 3 This is a schematic diagram illustrating the effect of aligning the spectral response curves of the source domain dataset and the target domain dataset before and after in an embodiment of the present invention;

[0046] Among them, (a) is a schematic diagram of the effect before the spectral response curves of the two are aligned;

[0047] (b) is a schematic diagram of the effect after aligning the spectral response curves of the two above. Detailed Implementation

[0048] like Figure 1 As shown, a small-sample hyperspectral classification method based on cross-domain spectral band alignment is implemented using the following steps:

[0049] The target domain data used in this embodiment is the University of Pavia dataset;

[0050] Step 1: Read multiple hyperspectral datasets as source domain datasets D sourceRead the hyperspectral dataset to be classified as the target domain dataset D target Calculate the source domain dataset D source and target domain dataset D target Aligned spectral wavelength range The number of aligned bands N align ;

[0051] The specific steps are as follows: First, read multiple available hyperspectral datasets as the source domain dataset D. source ={D1,D2,…,D n}, where the spectral wavelength range of the i-th dataset is The number of bands is N i ;

[0052] In this embodiment, the Chikusei, Botswana, and Kennedy SpaceCenter datasets are used as the source domain datasets, with wavelength ranges of (363nm, 1018nm), (400nm, 2500nm), and (400nm, 2500nm), respectively, and 128, 145, and 176 bands, respectively; the University of Pavia dataset to be classified is used as the target domain dataset D. target Its spectral wavelength range is (430nm, 860nm), and the number of bands is 103.

[0053] After the reading is complete, calculate the aligned spectral wavelength range and the number of spectral bands; first, determine the aligned spectral wavelength range using the following formula.

[0054]

[0055] The aligned spectral wavelength range is (430nm, 860nm);

[0056] The number of bands N after alignment for each dataset is then determined using the following formula. align :

[0057] N align =min(N1,N2,…,N) n N target )

[0058] The calculated number of aligned bands is 103.

[0059] Step Two: As Figure 2 As shown, by using parabolic interpolation and resampling alignment, all hyperspectral datasets are unified to the same spectral wavelength range. Within, and the same number of bands N align ;

[0060] In the source domain dataset D respectively source and target domain dataset D target For each dataset, parabolic interpolation is performed on its spectral response curve based on the spectral wavelength range and number of bands of each dataset. The following formula is constructed using the coordinates (x1, y1), (x2, y2), and (x3, y3) of every three adjacent points on the spectral response curve:

[0061]

[0062] By calculating the values ​​of a, b, and c, a local quadratic curve y = ax can be fitted. 2 +bx+c, then generalize to the entire spectral wavelength range, until a more accurate source domain dataset D is fitted. source The spectral response curve f1 and the target domain dataset D target The spectral response curve f2;

[0063] like Figure 3 As shown in (a) and (b), all datasets are resampled according to the aligned spectral wavelength range and the aligned number of bands, so that the two different datasets mentioned above have the same spectral range and number of bands after resampling.

[0064] Step 3: Mix multiple source domain datasets, select and divide them into training and test samples;

[0065] S3.1, From the source domain dataset D source In each dataset, at least 200 training samples are selected for each category, and categories with fewer than 200 training samples are removed to maintain a balance in the number of training samples for different categories.

[0066] S3.2. Combine the categories of the training samples that meet the requirements selected in step S3.1 to form a mixed source domain dataset. This allows each time a dataset from a mixed source domain to be generated... The training samples randomly selected from D source Different datasets in China;

[0067] S3.3, Set the target domain dataset D target Each category in the dataset contains 5 labeled samples that meet the training selection criteria. The target domain dataset D is augmented using a rotation and cropping data augmentation method. target The number of labeled samples for each category has been expanded from 5 to 200;

[0068] S3.4, Set the target domain dataset D targetThe number of land cover categories is n, making it follow the "n-way, k-shot" paradigm, for the mixed source domain dataset in step S3.2. Each time, n categories are randomly selected for the target domain dataset D. target Choose all n categories, and randomly select k samples from each category as the support set. Select m samples as the query set This together constitutes a small sample learning task.

[0069] In this embodiment, for each category in the source domain dataset with 200 or more labeled samples, 200 labeled samples from each category are randomly selected as source domain training data. In the target domain dataset, each category has 5 labeled samples that can be used for training. These samples are augmented using data augmentation methods such as rotation and cropping to obtain 200 labeled samples for each category. For the target domain training data, all labeled pixels are used as target domain test samples. In this embodiment, all training and test data samples are image patches with a size of 9×9×103.

[0070] In this embodiment, the number of classes in each training iteration is 9, following the "n-way, k-shot" paradigm. In the batches trained using the source domain data, an equal number of classes as the target domain classes are randomly selected from all eligible classes in all datasets for training. Therefore, this is considered as mixing the source domain datasets. Each time, 9 classes are randomly selected from all categories, and when training using the target domain dataset, all 9 classes are selected. One sample from each class is randomly selected as the training support set. Nineteen samples were selected as the training query set. Together they constitute a few-shot learning task

[0071] Step 4: Utilize embedded network F emb Extract training support set samples Embedding features The average value of each class sample in the training support set is used as the feature prototype f for that class. proto By embedding network F emb Extracting practice query set samples Embedding features The Euclidean distance between the embedded features and the feature prototypes of each category is calculated. Finally, the training query set samples are calculated using softmax. Predicted classification probability For embedded network F emb Train the model to determine its parameters;

[0072] In this embodiment, the input network data size is 9×9×103, where 9×9 represents the spatial size and 103 represents the number of bands. First, a one-dimensional convolutional mapping is performed on the input data to map the number of bands from 103 channels to 100 channels. That is, after passing through a two-dimensional convolutional layer with a kernel size of 1×1 and an output of 100 channels, and a batch normalization layer, the output data size is obtained as 9×9×100.

[0073] Embedded Feature Extraction Network F emb It contains four convolutional operations. Each operation specifically includes: a two-dimensional convolutional layer with a kernel size of 1×1 and an output of 200 channels → a batch normalization layer → a ReLU activation layer, repeated four times, and finally entering the mean pooling, flattening operation and linear layer, with the final output being an embedded feature vector of size 1×100.

[0074] The network is trained by alternating between mixed source domain data and target domain data; that is, the i-th iteration uses mixed source domain data, and the (i+1)-th iteration uses target domain data. In this embodiment, each training batch contains 9 categories, with one labeled data point for each category serving as the training support set for that batch. Training support set samples extracted by the network Embedding features Directly used as the feature prototype f of this class proto Based on training query set samples Embedding features and various feature prototypes f proto The Euclidean distance is calculated using softmax for each training query set sample. Predicted classification probability

[0075]

[0076] Where d(·) represents the calculation of Euclidean distance;

[0077] During training, the embedded network F emb loss function L fsl for:

[0078]

[0079] Among them, y i Indicates sample x i The true category, L CE This indicates the calculation of cross-entropy loss.

[0080] Step 5: Based on the trained network model, randomly select 5 labeled data points from each class in the augmented labeled data of the target domain dataset as the test support set, and select 100 unlabeled data points from the target domain dataset as the test query set. First, the network extracts test support set samples. Embedding features Take the average value of the embedded features of each class as the feature prototype f of that class. proto Subsequently, the network extracted test query set samples. Embedding features The Euclidean distance between the target sample and the feature prototype is calculated. Finally, the nearest neighbor algorithm is used to determine the predicted class label of the sample to be classified in the target domain dataset.

[0081] The small-sample hyperspectral classification method based on cross-domain spectral band alignment of this invention is compared with existing small-sample hyperspectral classification methods. The specific comparison method is as follows:

[0082] Two deep learning-based hyperspectral classification methods, HybridSN and SSRN, were selected, along with five small-sample hyperspectral classification methods, DFSL, DCFSL, CMFSL, Gia-CFSL, and RPCL-FSL, as comparison algorithms. The code for all comparison algorithms has been open-sourced on relevant websites.

[0083] The tests were conducted using three commonly used hyperspectral datasets: Indian Pines, Salinas, and the University of Pavia. During the experiments, for HybridSN and SSRN, five samples per class in the target domain were used for training, with the remainder used for testing. For DFSL and CMFSL, in addition to the aforementioned datasets, the University of Houston 2013 dataset was used for training, and a graph representation band selection method was used to unify the bands to 100 dimensions. For DCFSL and Gia-CFSL, only the Chikusei dataset was used as the source domain dataset, and a feature mapping layer was used to ensure dimensionality consistency between the training and testing datasets. For all small-sample hyperspectral classification methods, 200 samples per class were selected from the source domain for training, and all other experimental conditions remained the same.

[0084] OA (Overall Classification Accuracy), AA (Average Classification Accuracy), and Kappa coefficient were selected as evaluation metrics. Overall classification accuracy refers to the proportion of correctly classified samples out of all samples; average classification accuracy (AA) represents the average classification accuracy for each category; and the Kappa coefficient is used to measure the consistency between the predicted results and the true values.

[0085] The results after training and classification are listed in Tables 1 to 3.

[0086] Table 1. Classification results of the IndianPines dataset:

[0087]

[0088]

[0089] Table 2 Classification results for the Salinas dataset:

[0090] Table 3 Classification results for the University of Pavia dataset:

[0091]

[0092] Tables 1 to 3 show that, compared with the control algorithm, the classification method proposed in this application achieved better classification results on all three datasets, and the classification method has high classification accuracy.

Claims

1. A small-sample hyperspectral classification method based on cross-domain spectral band alignment, characterized in that, Includes the following steps: S1. Read multiple hyperspectral datasets as source domain datasets D source Read the hyperspectral dataset to be classified as the target domain dataset D target Calculate the source domain dataset D source and target domain dataset D target Aligned spectral wavelength range The number of aligned bands N align ; S2. Using parabolic interpolation and resampling alignment, align all source domain datasets D from step S1. source and target domain dataset D target Unify to the same spectral wavelength range Within, and uniformly set to the same number of bands N. align ; S3, from source domain dataset D source Select qualified training samples from the dataset, and combine the categories of these qualified training samples together to form a hybrid source domain dataset. In mixed source domain datasets Select the number of categories and from the mixed source domain dataset and target domain dataset D target Training support set in China and training query set Constructing few-shot learning tasks S4, via embedded network F emb Extract training support set Training support set samples Embedding features With training support set The average of the embedded features of samples from each category is used as the feature prototype f for that category. proto ; By calculating the training query set samples Embedding features With various feature prototypes f proto The Euclidean distance determines the class probability of the sample. For embedded network F emb Train the network model and determine its parameters; S5. Based on the trained network model, from the target domain dataset D target Collection and testing support set and test query set By embedding network F emb Extract test support set samples Embedding features With test support set The average value of the features of each class of samples is used as the feature prototype f of that class. proto ; By embedding network F emb Extract test query set samples Embedding features It calculates the Euclidean distance between the sample and the feature prototype, uses the nearest neighbor algorithm to determine the predicted class label for each sample, and then completes the classification of small sample hyperspectral data with cross-domain spectral band alignment.

2. The small-sample hyperspectral classification method based on cross-domain spectral band alignment according to claim 1, characterized in that, Step S1 is as follows: Read multiple hyperspectral datasets as source domain dataset D source ={D1,D2,…,D n }, where the source domain dataset D source The number of bands in each hyperspectral dataset is N1, N2, ..., N. n The spectral wavelength ranges are respectively Read the hyperspectral dataset to be classified as the target domain dataset D target Its number of bands is N target The spectral wavelength range is Spectral wavelength range after alignment The following formula can be used to calculate: The number of bands N after alignment align The following formula can be used to calculate: N align =min(N1,N2,…,N n ,N target )。 3. The small-sample hyperspectral classification method based on cross-domain spectral band alignment according to claim 2, characterized in that, Step S2 is as follows: S2.1 Using parabolic interpolation, in the source domain dataset D... source For each dataset, the coordinates (x1, y1), (x2, y2), and (x3, y3) of every three adjacent points within the original spectral wavelength range are collected and constructed into the following formula: By calculating the values ​​of a, b, and c, a local quadratic curve y = ax is obtained. 2 +bx+c, and then extend this process to the entire spectral wavelength range until the source domain dataset D is fitted. source All spectral response curves f1 of the corresponding dataset; S2.2 Using parabolic interpolation, in the target domain dataset D target On each dataset within the target domain, the same fitting method as in step S2.1 is used to fit the target domain dataset D. target All spectral response curves f2; S2.3, For the source domain dataset D source All spectral response curves f1 and target domain dataset D target All spectral response curves f2 in the aligned spectral wavelength range Equal-interval resampling is performed within the dataset to unify the number of bands in the two different datasets to the aligned number of bands N. align .

4. The small-sample hyperspectral classification method based on cross-domain spectral band alignment according to claim 3, characterized in that, Step S3 is as follows: S3.1, From the source domain dataset D source In each dataset, at least 200 training samples are selected for each category, and categories with fewer than 200 training samples are removed to maintain a balance in the number of training samples for different categories. S3.

2. Combine the categories of the training samples that meet the requirements selected in step S3.1 to form a mixed source domain dataset. This allows each time a dataset from a mixed source domain to be generated... The training samples randomly selected from D source Different datasets in China; S3.3, Set the target domain dataset D target Each category in the dataset contains 5 labeled samples that meet the training selection criteria. The target domain dataset D is augmented using a rotation and cropping data augmentation method. target The number of labeled samples for each category has been expanded from 5 to 200; S3.4, Set the target domain dataset D target The number of land cover categories is n, making it follow the "n-way, k-shot" paradigm, for the mixed source domain dataset in step S3.

2. Each time, n categories are randomly selected for the target domain dataset D. target Choose all n categories, and randomly select k samples from each category as the support set. Select m samples as the query set This together constitutes a small sample learning task.

5. The small-sample hyperspectral classification method based on cross-domain spectral band alignment according to claim 4, characterized in that, Step S4 is as follows: By embedding network F emb Using a mixed source domain dataset and target domain data D target Alternately perform iterative training; through embedding the network F emb Extract support set Training support set samples for each category Embedding features Extract the average of the embedded features of k samples from each class as the feature prototype f for that class. proto Then, through the embedded network F emb Extracting the training query set Training query set samples Embedding features And calculate the embedded features With the feature prototypes f of each category proto The Euclidean distance is calculated; finally, the Euclidean distance for each training query set sample is obtained using softmax. Predicted classification probability It satisfies the following formula: Where d(·) represents the calculation of Euclidean distance; During training, the embedded network F emb loss function L fsl for: Among them, y i Indicates training query set samples The true category, L CE This represents the cross-entropy loss.

Images