Small sample hyperspectral classification method based on cross-domain spectral band alignment
Through cross-domain spectral band alignment and alternating training strategies, the low accuracy and overfitting problems of small sample hyperspectral classification methods are solved, achieving higher classification accuracy and robustness.
Patent Information
- Application Number
- CN202411033588.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-07-30
AI Technical Summary
The existing small sample hyperspectral classification method has low algorithm accuracy and poor results when using source domain data sets to assist in training, resulting in the spectral data classification accuracy that cannot meet the requirements.
The cross-domain spectral band alignment method is used to unify the spectral response curves of hyperspectral images acquired by different sensors through parabolic interpolation and resampling, enhance the consistency of data distribution, and alleviate the overfitting phenomenon through alternating training strategies and lightweight embedded networks.
The classification accuracy and data alignment effect are improved, the network's cross-domain learning ability and robustness to data domain switching are enhanced, overfitting is reduced, and the classification accuracy of spectral data is improved.
Smart Images

Figure CN119027805A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of remote sensing information processing, and in particular to a small sample hyperspectral classification method based on cross-domain spectral band alignment. Background Art
[0002] Hyperspectral remote sensing images have high spectral resolution and many bands, and can obtain almost continuous spectral characteristic curves of ground objects, and can select or extract specific bands as needed to highlight target features. However, as the amount of training data required for the target model increases, the time and labor costs of annotating hyperspectral remote sensing data are gradually increasing, making the existing spectral training samples unable to bear the huge amount of data in the hyperspectral classification algorithm. The lack of existing spectral training samples has become the main development bottleneck of the hyperspectral classification algorithm.
[0003] At present, the problem of pixel-level hyperspectral classification with only a small number of labeled samples, also known as small-sample hyperspectral classification, is usually solved by the prototype network algorithm. The prototype network is a method of small-sample learning. Its characteristic is that it uses an embedding network to extract the embedded features of the input data, calculates the average value of the embedded features of each type of sample as the feature prototype of the class, and uses the nearest neighbor algorithm to find the category prototype closest to the embedded features of the sample to be labeled, and uses this to determine the predicted label.
[0004] However, the existing prototype network small sample hyperspectral classification methods have the following main problems: 1. Different data sets have different spectral ranges and number of bands, and the existing algorithms do not handle them adequately; 2. The role of the source domain data set is limited, and the algorithm accuracy is low; and the above problems will lead to the small sample hyperspectral classification algorithm when using the source domain data set to assist in training. The effect is not good, and the accuracy may even decrease instead of increase. The data distribution is uneven and other defects cannot meet the spectral data classification accuracy. Summary of the invention
[0005] The purpose of the present invention is to solve the problems of low algorithm accuracy of existing small sample hyperspectral classification methods, poor effect when using source domain datasets for auxiliary training, and inability to meet the classification accuracy of spectral data, and propose a small sample hyperspectral classification method based on cross-domain spectral band alignment.
[0006] The design idea of the present invention is: parabolic interpolation and resampling are used to achieve the unification of spectral response curves of hyperspectral images collected by different sensors in terms of spectral range and number of bands, thereby enhancing the distribution consistency of cross-domain data, so that the source domain data can provide features that are more conducive to distinguishing the target domain data, and the alternating training strategy is used to enhance the network's ability to learn cross-domain generalized knowledge, and the lightweight embedding network and prototype network training strategy is used to reduce the overfitting phenomenon of data.
[0007] To achieve the above objectives, the technical solution proposed by the present invention is:
[0008] A small sample hyperspectral classification method based on cross-domain spectral band alignment is special in that it includes the following steps:
[0009] S1. Read multiple hyperspectral datasets as source domain datasets D source , read the hyperspectral dataset to be classified as the target domain dataset D target , calculate the source domain dataset D source and the target domain dataset D target Spectral wavelength range after alignment and the number of bands after alignment N align ;
[0010] S2, through parabola interpolation and resampling alignment method, all source domain datasets D in step S1 are aligned source and the target domain dataset D target Unified to the same spectral wavelength range and unified to the same number of bands N align ;
[0011] S3. From the source domain dataset D source Select the training samples that meet the requirements, and combine the categories of the training samples that meet the requirements to form a mixed source domain dataset In mixed source domain datasets The number of categories is selected and the mixed source domain dataset is and the target domain dataset D target Acquisition training support set and the training query set Constructing a few-shot learning task
[0012] S4, by embedding network F emb Extracting training support set The training support set samples Embedding features Training support set The average value of the embedded features of samples of each category in is taken as the feature prototype f of this category. proto ; By calculating the training query set samples Embedding features With various feature prototypes f proto The Euclidean distance determines the class probability of the sample Embedded network F emb Conduct training and determine the network model parameters;
[0013] S5. Based on the trained network model, target Collection test support set and the test queryset By embedding the network F emb Extracting test support set samples Embedding features Test support set The average value of each class of sample features is taken as the feature prototype f of this class proto ; By embedding network F emb Extract test query set samples Embedding features The Euclidean distance between the sample and the feature prototype is calculated, and the predicted category label of each sample is determined using the nearest neighbor algorithm, thereby completing the classification of small sample hyperspectral images aligned across spectral bands.
[0014] Furthermore, step S1 is specifically as follows:
[0015] Read multiple hyperspectral datasets as source domain datasets D source ={D 1 ,D 2 ,…,D n}, where the source domain dataset D source The number of bands in each hyperspectral dataset is N 1 ,N 2 ,…,N n The spectral wavelength ranges are
[0016] Read the hyperspectral dataset to be classified as the target domain dataset D target , the number of bands is N target , the spectral wavelength range is
[0017] Spectral wavelength range after alignment Calculated by the following formula:
[0018]
[0019] The number of bands after the two are aligned is N align Calculated by the following formula:
[0020] N align =min(N 1 ,N 2 ,…,N n ,N target ).
[0021] Furthermore, step S2 is specifically as follows:
[0022] S2.1, using parabolic interpolation method, in the source domain dataset D source For each data set in the original spectral wavelength range, the coordinates (x 1 ,y 1 ),(x 2 ,y 2 ),(x 3 ,y 3 ) is constructed as follows:
[0023]
[0024] By calculating the values of a, b, and c respectively, we can then fit the local quadratic curve y=ax 2 +bx+c, and then extend it to the entire spectral wavelength range until the source domain data set D is fitted source All spectral response curves f1 of the corresponding data set;
[0025] S2.2, using parabolic interpolation method, in the target domain dataset D target The same fitting method as step S2.1 is used to fit the target domain dataset D target All spectral response curves f2;
[0026] S2.3. Source domain dataset D source All spectral response curves f1 and the target domain dataset D target All spectral response curves f2 in the spectral wavelength range after alignment The number of bands of the two different data sets is unified to the number of bands after alignment N. align .
[0027] Furthermore, step S3 is specifically as follows:
[0028] S3.1. From the source domain dataset D source In each category of each data set, no less than 200 training samples are selected, and categories with less than 200 training samples are eliminated to maintain a balance in the number of training samples in different categories;
[0029] S3.2. Combine the categories of training samples that meet the requirements selected in step S3.1 to form a mixed source domain dataset So that each time from the mixed source domain dataset The training samples randomly selected from D source Different datasets in
[0030] S3.3. Set the target domain dataset D targetThere are 5 labeled samples for each category that meet the training screening criteria. The target domain dataset D is augmented by the rotation and cropping method. target The number of labeled samples for each category was expanded from 5 to 200;
[0031] S3.4. Set the target domain dataset D target The number of object categories is n, so that it follows the “n-way, k-shot” paradigm. For the mixed source domain dataset in step S3.2 Each time, n categories are randomly selected for the target domain dataset D target Select all n categories and randomly select k samples from each category as the support set Select m samples as the query set This together constitutes a small sample learning task
[0032] Further, step S4 is specifically as follows:
[0033] By embedding the network F emb Using mixed source domain datasets and target domain data D target Alternate iterative training; by embedding the network F emb Extracting support sets The training support set samples for each category in Embedding features Extract the average of the embedded features of k samples of each category as the feature prototype f of the category proto ; Then, by embedding the network F emb Extracting the training query set The training query set samples Embedding features And calculate the embedding features With the characteristic prototype f of each category proto The Euclidean distance of each training query set is obtained by softmax calculation. The predicted classification probability It satisfies the following formula:
[0034]
[0035] Among them, d(·) represents the calculation of Euclidean distance;
[0036] During the training process, the embedding network F emb The loss function L fsl for:
[0037]
[0038] Among them, yi Represents the training query set samples The real category, L CE represents the cross entropy loss.
[0039] Beneficial effects of the present invention:
[0040] 1. The small sample hyperspectral classification method based on cross-domain spectral band alignment of the present invention enhances the distribution alignment effect of cross-domain data, uses the parabolic interpolation method to obtain more refined spectral data, and realizes the alignment of spectral response curves of hyperspectral images collected by different sensors in terms of spectral wavelength range and number of spectral bands, so that deep neural networks can extract more generalized features, improve the positive effect of source domain data on network classification effect, and effectively improve the classification accuracy of the algorithm and the data alignment effect.
[0041] 2. The present invention expands the scope of use of source domain data sets by adopting a multi-source domain mixed alternating training strategy, allowing more hyperspectral data to participate in training. Alternating training makes the network more robust to data domain switching, enhances the network's cross-domain learning capability, and improves the auxiliary training effect of source domain data sets.
[0042] 3. The present invention embeds a network training network model. It uses lightweight design thinking and one-dimensional convolution to reduce the scale of network parameters, which can effectively improve the inference speed of hyperspectral data sets, reduce the overfitting phenomenon of each data domain, and effectively improve the classification accuracy of spectral data. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 It is a flow chart of an embodiment of a small sample hyperspectral classification method based on cross-domain spectral band alignment of the present invention;
[0044] Figure 2 is a schematic flow chart of a method for aligning spectral response curves of a source domain dataset and a target domain dataset in an embodiment of the present invention;
[0045] Figure 3 is a schematic diagram of the effect before and after the spectral response curves of the source domain data set and the target domain data set are aligned in an embodiment of the present invention;
[0046] Among them, (a) is a schematic diagram of the effect before the above two spectral response curves are aligned;
[0047] (b) is a schematic diagram of the effect after the spectral response curves of the above two are aligned. DETAILED DESCRIPTION
[0048] like Figure 1 As shown in the figure, a small sample hyperspectral classification method based on cross-domain spectral band alignment is implemented as follows:
[0049] The target domain data used in this embodiment is the University of Pavia dataset;
[0050] Step 1: Read multiple hyperspectral datasets as source domain datasets D source , read the hyperspectral dataset to be classified as the target domain dataset D target , calculate the source domain dataset D source and the target domain dataset D target Spectral wavelength range after alignment and the number of bands after alignment N align ;
[0051] The specific steps are as follows: First, read multiple available hyperspectral datasets as source domain datasets D source ={D 1 ,D 2 ,…,D n}, where the spectral wavelength range of the i-th data set is The number of bands is N i ;
[0052] In this example, the Chikusei, Botswana and Kennedy Space Center datasets are used as source domain datasets, with wavelength ranges of (363nm, 1018nm), (400nm, 2500nm), (400nm, 2500nm), and the number of bands are 128, 145 and 176 respectively; the University of Pavia dataset to be classified is read as the target domain dataset D target , its spectral wavelength range is (430nm, 860nm) and the number of bands is 103.
[0053] After reading, calculate the aligned spectral wavelength range and the number of spectral bands. First, determine the aligned spectral wavelength range using the following formula:
[0054]
[0055] The wavelength range of the spectrum after alignment is (430nm, 860nm);
[0056] Then the number of bands N after alignment of each data set is determined by the following formula: align :
[0057] N align =min(N 1 ,N 2 ,…,N n ,N target )
[0058] The number of bands after alignment is calculated to be 103.
[0059] Step 2: If Figure 2 As shown in Figure 2, all hyperspectral datasets are unified to the same spectral wavelength range through parabolic interpolation and resampling alignment. and the same number of bands N align ;
[0060] In the source domain dataset D source and the target domain dataset D target For each data set in the dataset, parabolic interpolation is performed on its spectral response curve according to the spectral wavelength range and number of bands of each data set, and the coordinates of each adjacent three points of its spectral response curve (x 1 ,y 1 ),(x 2 ,y 2 ),(x 3 ,y 3 ) form the following formula:
[0061]
[0062] The values of a, b, and c are calculated, and the local quadratic curve y=ax can be fitted. 2 +bx+c, and then extended to the entire spectral wavelength range until a more accurate source domain data set D is fitted. source The spectral response curve f1 and the target domain dataset D target The spectral response curve f2;
[0063] like Figure 3 As shown in (a) and (b) in FIG. 5 , all data sets are resampled according to the aligned spectral wavelength range and the aligned number of bands, so that the resampled two different data sets have the same spectral range and number of bands.
[0064] Step 3: Mix multiple source domain datasets, select and divide training and test samples;
[0065] S3.1. From the source domain dataset D source In each category of each data set, no less than 200 training samples are selected, and categories with less than 200 training samples are eliminated to maintain a balance in the number of training samples in different categories;
[0066] S3.2. Combine the categories of training samples that meet the requirements selected in step S3.1 to form a mixed source domain dataset So that each time from the mixed source domain dataset The training samples randomly selected from Dsource Different datasets in
[0067] S3.3. Set the target domain dataset D target There are 5 labeled samples for each category that meet the training screening criteria. The target domain dataset D is augmented by the rotation and cropping method. target The number of labeled samples for each category was expanded from 5 to 200;
[0068] S3.4. Set the target domain dataset D target The number of object categories is n, so that it follows the “n-way, k-shot” paradigm. For the mixed source domain dataset in step S3.2 Each time, n categories are randomly selected for the target domain dataset D target Select all n categories and randomly select k samples from each category as the support set Select m samples as the query set This together constitutes a small sample learning task
[0069] In this embodiment, the categories with more than or equal to 200 labeled samples in the source domain data set are counted, and 200 labeled samples in each category are randomly selected as source domain training data; in the target domain data set, each category has 5 labeled samples that can be used for training, and data enhancement methods such as rotation and cropping are used to expand the samples to obtain 200 labeled samples in each category; for the target domain training data, all labeled pixels are used as target domain test samples; in this embodiment, the samples of all training and test data are image blocks of size 9×9×103.
[0070] In this embodiment, the number of categories for each training is 9, following the "n-way, k-shot" paradigm. In the batch trained with source domain data, the same number of categories as the target domain categories are randomly selected from all categories that meet the conditions in all data sets for training. Therefore, it is considered to mix the source domain data set, and 9 categories are randomly selected from all categories each time. When training with the target domain data set, all 9 categories are selected. One sample is randomly selected from each category as the training support set. Select 19 samples as the training query set Together they constitute a small sample learning task
[0071] Step 4: Using the embedded network F emb Extract training support set samples Embedding features The average value of each category sample in the training support set is used as the feature prototype f of this category. proto By embedding the network F embExtract training query set samples Embedding features And calculate the Euclidean distance between the embedded features and the feature prototypes of each category. Finally, the training query set samples are calculated through softmax The predicted classification probability Embedded network F emb Conduct training to determine model parameters;
[0072] In this embodiment, the data size of the input network is 9×9×103, where 9×9 represents the spatial size and 103 represents the number of bands. First, a one-dimensional convolution mapping is performed on the input data to map the number of bands from 103 channels to 100 channels, that is, after a two-dimensional convolution layer and a batch normalization layer with a convolution kernel size of 1×1 and an output of 100 channels, the output data size is 9×9×100.
[0073] Embedded feature extraction network F emb It contains four layers of convolution operations, each of which includes: a two-dimensional convolution layer with a convolution kernel size of 1×1 and an output of 200 channels → batch normalization layer → ReLU activation layer, repeated four times, and finally entering the mean pooling, straightening operation and linear layer, and finally outputting an embedded feature vector with a size of 1×100.
[0074] The network is trained alternately by mixing the source domain data and the target domain data, that is, the mixed source domain data is used in the i-th iteration, and the target domain data is used in the i+1-th iteration. In this embodiment, each training batch contains 9 categories, and 1 labeled data of each category is used as the training support set of the batch. Training support set samples extracted by the network Embedding features Directly as the feature prototype f of this class proto ; Based on the training query set samples Embedding features and various feature prototypes f proto The Euclidean distance is calculated by softmax to obtain the Euclidean distance of each training query set sample. The predicted classification probability
[0075]
[0076] Among them, d(·) represents the calculation of Euclidean distance;
[0077] During the training process, the embedding network F emb The loss function L fsl for:
[0078]
[0079] Among them, y i Represents sample x i The real category, L CE Represents the calculation of cross entropy loss.
[0080] Step 5: Based on the trained network model, randomly extract 5 labeled data of each category from the labeled data in the target domain dataset after data augmentation as the test support set, and take 100 unlabeled data from the target domain dataset as the test query set. First, the network extracts the test support set samples Embedding features Take the average value of the embedded features of each class of samples as the feature prototype f of that class proto ; Then, the network extracts the test query set samples Embedding features And calculate its Euclidean distance with the feature prototype; finally, use the nearest neighbor algorithm to determine the predicted category label of the sample to be classified in the target domain data set.
[0081] The small sample hyperspectral classification method based on cross-domain spectral band alignment of the present invention is compared with the existing small sample hyperspectral classification method. The specific comparison method is as follows:
[0082] Two deep learning-based hyperspectral classification methods were selected: HybridSN and SSRN, and five small sample hyperspectral classification methods: DFSL, DCFSL, CMFSL, Gia-CFSL and RPCL-FSL were selected as comparison algorithms. The codes of the comparison algorithms have been open sourced on relevant websites.
[0083] The three commonly used hyperspectral datasets, Indian Pines, Salinas, and University of Pavia, were used for testing. During the experiment, for HybridSN and SSRN, 5 data of each category in the target domain were used for training, and the rest were used for testing; for DFSL and CMFSL, in addition to the above datasets, the University of Houston 2013 dataset was used for training, and the bands were unified to 100 dimensions using the graph representation band selection method; for DCFSL and Gia-CFSL, only the Chikusei dataset was used as the source domain dataset, and the feature mapping layer was used to ensure the dimensional consistency of the training dataset and the test dataset. For all small sample hyperspectral classification methods, 200 samples of each category were selected from the source domain for training, and the other experimental conditions remained the same.
[0084] OA (overall classification accuracy), AA (average classification accuracy) and Kappa coefficient were selected as evaluation indicators. Among them, the overall classification accuracy refers to the proportion of correctly classified samples to all samples, the average classification accuracy AA refers to the average classification accuracy of each category, and the Kappa coefficient is an indicator used to measure the consistency between the prediction result and the true value.
[0085] The results after training and classification are listed in Tables 1 to 3.
[0086] Table 1 IndianPines dataset classification results:
[0087]
[0088]
[0089] Table 2 Classification results of Salinas dataset:
[0090] Table 3 Classification results of University of Pavia dataset:
[0091]
[0092] From Tables 1 to 3, it can be found that compared with the control algorithm, the classification method proposed in the present application has achieved better classification results on the three data sets, and the classification method has higher classification accuracy.
Claims
1. A small sample hyperspectral classification method based on cross-domain spectral band alignment, characterized in that: The following steps are involved: S1. Read multiple hyperspectral datasets as source domain datasets D source , read the hyperspectral dataset to be classified as the target domain dataset D target , calculate the source domain dataset D source and the target domain dataset D target Spectral wavelength range after alignment and the number of bands after alignment N align ; S2, through parabola interpolation and resampling alignment method, all source domain datasets D in step S1 are aligned source and the target domain dataset D target Unified to the same spectral wavelength range and unified to the same number of bands N align ; S3. From the source domain dataset D source Select the training samples that meet the requirements, and combine the categories of the training samples that meet the requirements to form a mixed source domain dataset In mixed source domain datasets The number of categories is selected and the mixed source domain dataset is and the target domain dataset D target The training support set and the training query set Constructing a few-shot learning task S4, by embedding network F emb Extracting training support set The training support set samples Embedding features Training support set The average value of the embedded features of samples of each category in is taken as the feature prototype f of this category. proto ; By calculating the training query set samples Embedding features With various feature prototypes f proto The Euclidean distance determines the class probability of the sample Embedded network F emb Conduct training and determine the network model parameters; S5. Based on the trained network model, target Collection test support set and the test queryset By embedding the network F emb Extracting test support set samples Embedding features Test support set The average value of each class of sample features is taken as the feature prototype f of this class proto ; By embedding network F emb Extract test query set samples Embedding features The Euclidean distance between the sample and the feature prototype is calculated, and the predicted category label of each sample is determined using the nearest neighbor algorithm, thereby completing the classification of small sample hyperspectral images aligned across spectral bands.
2. According to claim 1, a small sample hyperspectral classification method based on cross-domain spectral band alignment is characterized in that: Step S1 is specifically as follows: Read multiple hyperspectral datasets as source domain datasets D source ={D1,D2,…,D n }, where the source domain dataset D source The number of bands in each hyperspectral dataset is N1, N2, ..., N n The spectral wavelength ranges are Read the hyperspectral dataset to be classified as the target domain dataset D target , the number of bands is N target , the spectral wavelength range is Spectral wavelength range after the two are aligned Calculated by the following formula: The number of bands after the two are aligned is N align Calculated by the following formula: N align =min(N1,N2,…,N n ,N target )。 3. According to claim 2, a small sample hyperspectral classification method based on cross-domain spectral band alignment is characterized in that: Step S2 is specifically as follows: S2.1, using parabolic interpolation method, in the source domain dataset D source For each data set in the original spectral wavelength range, the coordinates of each adjacent three points (x1, y1), (x2, y2), (x3, y3) of each data set are collected to construct the following formula: By calculating the values of a, b, and c respectively, we can then fit the local quadratic curve y=ax 2 +bx+c, and then extend it to the entire spectral wavelength range until the source domain data set D is fitted source All spectral response curves f1 of the corresponding data set; S2.2, using parabolic interpolation method, in the target domain dataset D target The same fitting method as step S2.1 is used to fit the target domain dataset D target All spectral response curves f2; S2.
3. Source domain dataset D source All spectral response curves f1 and the target domain dataset D target All spectral response curves f2 in the aligned spectral wavelength range The number of bands of the two different data sets is unified to the number of bands after alignment N. align .
4. According to claim 3, a small sample hyperspectral classification method based on cross-domain spectral band alignment is characterized in that: Step S3 is specifically as follows: S3.
1. From the source domain dataset D source In each category of each data set, no less than 200 training samples are selected, and categories with less than 200 training samples are eliminated to maintain a balance in the number of training samples in different categories; S3.
2. Combine the categories of training samples that meet the requirements selected in step S3.1 to form a mixed source domain dataset So that each time from the mixed source domain dataset The training samples randomly selected from D source Different datasets in S3.
3. Set the target domain dataset D target There are 5 labeled samples for each category that meet the training screening criteria. The target domain dataset D is augmented by the rotation and cropping method. target The number of labeled samples for each category was expanded from 5 to 200; S3.
4. Set the target domain dataset D target The number of object categories is n, so that it follows the "n-way, k-shot" paradigm. For the mixed source domain dataset in step S3.2 Each time, n categories are randomly selected for the target domain dataset D target Select all n categories and randomly select k samples from each category as the support set Select m samples as the query set This together constitutes a small sample learning task 5. According to claim 4, a small sample hyperspectral classification method based on cross-domain spectral band alignment is characterized in that: Step S4 is specifically as follows: By embedding the network F emb Using mixed source domain datasets and target domain data D target Alternate iterative training; by embedding the network F emb Extracting support sets The training support set samples for each category in Embedding features Extract the average of the embedded features of k samples of each category as the feature prototype f of the category proto ; Then, by embedding the network F emb Extracting the training query set The training query set samples Embedding features And calculate the embedding features With the characteristic prototype f of each category proto The Euclidean distance of each training query set is obtained by softmax calculation. The predicted classification probability It satisfies the following formula: Where d(·) represents the calculation of Euclidean distance; During the training process, the embedding network F emb The loss function L fsl for: Among them, y i Represents the training query set samples The real category, L CE represents the cross entropy loss.
Citation Information
Patent Citations
Hyperspectral image classification method and system based on deep cross-domain few-sample learning
CN115170961A
Cross-domain video action recognition method, device and equipment and computer readable storage medium
CN115439791A
Hyperspectral image classification method based on small sample deep learning
CN116310510A
Small sample hyperspectral image classification method, system and device and storage medium
CN116977693A
Cross-domain long-tail image classification method based on self-supervised learning and self-training mechanism
CN117115547A
