Cross-domain hyperspectral image accurate classification method based on small sample learning

Abstract

The invention provides a cross-domain hyperspectral image accurate classification method based on small sample learning, and belongs to the technical field of image processing. Firstly, target domain data containing a small number of labels and source domain data containing sufficient labels are input respectively, feature information is extracted through an embedded model, a self-attention mechanism is introduced in the feature extraction stage, a clear area in a feature map is automatically concerned, an uncertain fuzzy area is ignored, the perceptual ability of different-domain structure information is improved, and the feature extraction efficiency is improved. Information dislocation caused by space structure and resolution difference is effectively reduced, and support is provided for solving spectrum offset. The method comprises the following steps of: obtaining a source domain and a target domain, realizing distribution alignment of the source domain and the target domain through a conditional adversarial domain adaptation strategy, overcoming the problem of spectrum offset, and finally performing classification through a K-nearest neighbor (KNN) algorithm to obtain a ground feature category. Experimental results show that the performance of the method on a Pavia University data set is superior to that of an existing cross-domain hyperspectral image classification method of a small sample, and a new thought is provided for research and application of cross-domain hyperspectral image classification.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of remote sensing image classification, and particularly relates to a cross-domain hyperspectral image accurate classification method based on small sample learning. BACKGROUND

[0002] Hyperspectral images (HSI) are widely used in mineral identification, military reconnaissance, precision agriculture, and other fields due to the rich information they provide in spatial and spectral dimensions. However, traditional hyperspectral image classification methods often exhibit low generalization ability and classification accuracy when faced with cross-domain scenarios and limited labeled data.

[0003] In practical applications, obtaining a large amount of labeled hyperspectral data is both expensive and time-consuming, especially in target domains, where the scarcity of labeled data makes it difficult for models to effectively learn. To this end, researchers have proposed cross-domain small sample learning methods, aiming to utilize the rich labeled data in the source domain and transfer knowledge to the target domain through meta-learning strategies, thereby improving the model's classification performance in the target domain. However, significant distribution differences may exist between the source and target domains, including spectral features, spatial resolution, and noise levels. Such inter-domain differences can lead to a decline in model performance in the target domain.

[0004] In recent years, to address this issue, domain adaptation-based methods have been widely applied in deep learning models and have achieved significant success. For example, a domain adversarial loss function effectively reduces the difference between the feature distributions of the source and target domains by introducing a discriminator, thereby improving the model's generalization ability. Specifically, the discriminator learns to distinguish between the feature representations of the source and target domains through adversarial training; while the model optimizes to make the discriminator unable to accurately distinguish between the two, thereby achieving alignment of the feature distributions. This approach ensures that the model can better migrate and adapt between different domains. At the same time, to more comprehensively preserve spatial and spectral information in hyperspectral images and further enhance the model's expression ability, three-dimensional convolutional neural networks perform convolution operations in both spatial and spectral domains, enabling more comprehensive extraction and integration of multi-dimensional features. In processing hyperspectral images, this approach has shown significant advantages, providing strong support for accurate classification and analysis.

[0005] In addition, the self-attention mechanism can enhance the model's ability to focus on important features by calculating the correlation within the features, thereby improving the model's perception of key areas. The three-dimensional convolutional neural network (3D CNN) model combined with the self-attention mechanism can not only extract multi-dimensional features in the spatial and spectral domains, but also automatically focus on clear and stable areas while ignoring uncertain and ambiguous areas during feature extraction. This synergistic mechanism significantly enhances the model's ability to perceive information from different domains, effectively reduces the information misalignment caused by spatial structure and resolution differences, and provides strong support for solving the spectral shift problem. In the cross-domain hyperspectral image classification task, this method has shown excellent performance and broad application potential.

[0006] Therefore, the present application combines the self-attention mechanism, 3D CNN, and domain-adversarial loss function to build a more robust hyperspectral image classification model. During feature extraction, the self-attention mechanism automatically focuses on key areas, enhancing the model's ability to perceive information from different domains in hyperspectral images. The 3D CNN extracts rich multi-dimensional features in both spatial and spectral domains. The domain-adversarial loss function is used to align the distributions of the source and target domains, effectively overcoming the spectral shift problem and ensuring the model's ability to transfer between different domains. This strategy can better improve the model's performance compared to using the self-attention mechanism, 3D CNN, or domain-adversarial loss function alone. The introduction of the self-attention mechanism allows the model to ignore uncertain areas and focus on important information. The multi-dimensional feature extraction advantage of the 3D CNN improves the overall classification effect, while the domain-adversarial loss function further enhances the model's robustness, reducing the differences between the source and target domains, thereby improving the accuracy and reliability of the classification. SUMMARY

[0007] The present application proposes a cross-domain hyperspectral image classification method based on small sample learning, which fully combines the feature extraction capability of the embedding model, the stability enhancement of the self-attention mechanism, and the distribution alignment advantage of the conditional adversarial domain adaptation strategy. By introducing the self-attention mechanism, the model's ability to perceive information from different domains is effectively improved, and the information misalignment caused by spatial structure and resolution differences is reduced. Meanwhile, the conditional adversarial domain adaptation strategy is used to overcome the spectral shift problem. Finally, the present application realizes accurate land cover class prediction through KNN classification, showing excellent classification performance in the cross-domain hyperspectral image classification task, providing important technical support for solving the small sample and spectral shift problems, and having significant scientific and practical value.

[0008] The purpose of the present application is achieved as follows:

[0009] The cross-domain hyperspectral image accurate classification method based on small sample learning comprises the following steps:

[0010] Step a, input the hyperspectral data sets of source domain and target domain respectively, define the source domain data set as D s , and the target domain data set as D t , D t includes a small amount of labeled data D f and a large amount of unlabeled data D u , wherein D t =D f ∪D u ;

[0011] Step b, small sample learning is performed on D s and D t respectively, C classes are randomly selected from D s , K labeled samples are selected from each class to form a support set S s , N unlabeled samples are selected from the same C classes to form a query set Q s , wherein and I s =S s ∪Q s , the same small sample learning task is performed on the target domain D t to generate I t =S t ∪Q t ;

[0012] Step c, D s and D t after performing the small sample learning task are preprocessed through mapping networks M s and M t , the feature dimension conversion is performed on D s and D t to generate I s ′ and I t ′ respectively, so as to ensure that the input feature dimensions of I S and I t are consistent, and the formula is as follows:

[0013]

[0014] wherein, c×b s and c×b t are the width and height of the source domain and the target domain respectively, and d is the target dimension;

[0015] Step d, a feature extractor f embed is constructed, a spatial-spectral feature is extracted based on the improved self-attention 3D residual convolutional network structure to generate an embedded feature F(x), wherein F(x)=f embed(M(x)). Through the introduced self-attention mechanism, f embed The clear and stable region in the automatic attention feature map is focused on, and the uncertain and fuzzy region is ignored, the perception ability of the heterogeneous structure information is improved, the information dislocation caused by the spatial structure and resolution difference is effectively reduced, and support is provided for solving the spectral shift;

[0016] Step e, aligning the distribution difference of the embedded feature f=F(x) by using a domain adversarial loss function, reducing the spectral difference between different domains, and the domain adversarial loss function is as follows:

[0017]

[0018] Wherein, D represents a discriminator, The embedded features of the source domain and the target domain samples are respectively, wherein h=(f, g) is a joint variable, g is the category information predicted by the discriminator D, and T is a multi-linear dimension transformation.

[0019] Step f, classifying the target domain samples by using a nearest neighbor classifier to obtain a final ground object classification result.

[0020] The above-mentioned cross-domain hyperspectral image precise classification method based on small sample learning, characterized in that, step a specifically comprises the following steps:

[0021] Step a1, inputting a source domain data set D s , Wherein represents the hyperspectral image data of the i-th sample, is the category label of the sample.

[0022] Step a2, inputting a target domain data set D t , Wherein is the hyperspectral data of the i-th sample, is the category label of the target domain. The target domain data set D t includes a small amount of labeled data D f and a large amount of unlabeled data D u , wherein D t =D f ∪D u and

[0023] The above-mentioned cross-domain hyperspectral image precise classification method based on small sample learning, characterized in that, step b specifically comprises the following steps:

[0024] Step b1, performing a source domain D s small sample learning task, randomly selecting C classes from the source domain data set D S , selecting K labeled samples for each class, and forming a support set SS N unlabelled samples are selected from the same C class to form a query set Q S The formula is as follows:

[0025]

[0026] Step b2, performing target domain D t Small sample learning task, target domain D t Perform the same small sample learning task to generate the support set S of the target domain t And the query set Q t The formula is as follows:

[0027]

[0028] The cross-domain hyperspectral image accurate classification method based on small sample learning has the characteristics that step c specifically includes the following steps:

[0029] Step c1, in order to ensure that the feature dimensions of the source domain and the target domain are consistent, a mapping network M s And M t Convert the hyperspectral data of the source domain and the target domain to a unified dimension. The data processed by the mapping network is represented as:

[0030] I' s =M s (I s ), I' t =M t (I t )

[0031] Where M s and M t are the mapping networks of the source domain and the target domain, I s and I t are the original data of the source domain and the target domain, I s ' and I t ' are the data processed by the mapping network.

[0032] Step c2, the dimension of the mapped data is And Where c and b are the width and height of the image respectively, and d is the feature dimension. The role of the mapping network is to align the feature dimensions of the source domain and the target domain data, reduce the feature differences caused by different band numbers, and ensure that the two can be effectively processed in the same feature space.

[0033] The cross-domain hyperspectral image accurate classification method based on small sample learning has the characteristics that step d specifically includes the following steps:

[0034] Step d1, construct the feature extractor, the network architecture includes multiple convolutional layers, each layer is a 3D convolution operation, the 3D convolution network is used to extract spatial-spectral features from the input hyperspectral image data. ReLU activation function is used between layers to enhance the non-linear characteristics. In order to alleviate the problem of gradient disappearance in deep neural network, residual connection strategy is adopted. By introducing direct jump connection between network layers, the model can more effectively learn the difference between input and output, thereby simplifying the training process. The specific process is as follows:

[0035] F s = f embed (M s (I s )), F t = f embed (M t (I t ))

[0036] Where f embed represents the feature extractor, M s and M t are the mapping networks of the source domain and the target domain, I s and I t are the data of the source domain and the target;

[0037] y = f(x) + x

[0038] Where f(x) is the feedforward operation of the network, and x is the input feature. By adding input x residual connection, the network can more easily learn complex mapping. In convolutional neural networks, residual connection helps to avoid gradient disappearance and accelerate training by directly passing the input to the subsequent layer through "jump connection".

[0039] Step d2, introduce self-attention mechanism in the feature extractor. Self-attention mechanism can make the network pay attention to specific areas in the image, thereby improving the feature expression of important areas while ignoring unimportant areas. Self-attention mechanism weights the features of different areas by calculating the relationship between each pixel point, thereby improving the spatial perception ability of the network. Assume that the input is X ∈ R n×d , where n is the number of pixel points in the hyperspectral image, and d is the dimension of each feature. Calculate Query, Key and Value:

[0040] Q = XW Q , K = XW K , V = XW V

[0041] Where W Q , W K , W V are learnable weight matrices;

[0042] The attention score is calculated by calculating the similarity between the query and the key to generate the attention score:

[0043]

[0044] Where A is the attention matrix, QK T represents the dot product between the query and the key, d is the size of the feature dimension, is used to scale the dot product to avoid gradient explosion caused by too large values;

[0045] The weighted output O is calculated, O = AV, where O is the final weighted output, and the features are generated by weighting different values V, and the self-attention mechanism adjusts the importance of each feature by weighting different features, thereby improving the aggregation ability of the target region.

[0046] The above cross-domain hyperspectral image precise classification method based on small sample learning, characterized in that, in step e, the domain adversarial loss function is used to reduce the distribution difference between the source domain and the target domain, so that the features of the source domain can be effectively transferred to the target domain, and the domain adversarial loss function is as follows:

[0047]

[0048] Where D represents the discriminator, are the embedding features of the source domain and the target domain samples respectively, where h = (f, g) is the joint variable, g is the class information predicted by the discriminator D, and p is the probability that the sample belongs to the source domain. By minimizing the distribution difference between the source domain and the target domain, the target domain can share the features of the source domain, thereby realizing the cross-domain classification task. T is a multi-linear dimension transformation.

[0049] The above cross-domain hyperspectral image precise classification method based on small sample learning, characterized in that, in step f, a nearest neighbor classifier is used to classify the samples of the target domain. The classifier selects the nearest sample for classification by calculating the distance between each target sample and the support set sample, and obtains the final land cover class.

[0050] Beneficial effects:

[0051] The application provides a small sample learning-based cross-domain hyperspectral image accurate classification method, and belongs to the technical field of image processing. Firstly, target domain data containing a small amount of labels and source domain data with sufficient labels are respectively input, feature information is extracted through an embedding model, and a self-attention mechanism is introduced in the feature extraction stage, so that clear and stable regions in a feature map are automatically focused on, uncertain and fuzzy regions are ignored, the perception ability of structural information in different domains is improved, information dislocation caused by spatial structure and resolution difference is effectively reduced, and support is provided for solving spectral shift. Subsequently, a conditional adversarial domain adaptation strategy is used to realize distribution alignment of the source domain and the target domain, overcome the spectral shift problem, and finally, KNN is used for classification to obtain accurate ground object categories. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 is a small sample learning-based cross-domain hyperspectral image accurate classification method in the method of the application.

[0053] Figure 2 is a pseudo-color image and a corresponding ground truth map of the WHU-Hi-HanChuan data set in the method of the application.

[0054] Figure 3 is a pseudo-color image and a corresponding ground truth map of the Pavia University data set in the method of the application.

[0055] Figure 4 is a principle schematic diagram of the feature extractor in the method of the application.

[0056] Figure 5 is a classification result map of the Pavia University data set in the method of the application.

[0057] Figure 6 is an attached drawing of the summary in the method of the application. DETAILED DESCRIPTION

[0058] The small sample learning-based cross-domain hyperspectral image accurate classification method in the specific embodiment of the application is described in further detail below with reference to the drawings.

[0059] The small sample learning-based cross-domain hyperspectral image accurate classification method in the specific embodiment of the application is described in further detail below with reference to the drawings. Figure 1 As shown in the flowchart, the method comprises the following steps:

[0060] Step a, input the hyperspectral data sets of the source domain and the target domain respectively, define the source domain data set as D s , the target domain data set as D t , randomly select training samples, and construct a training set and a test set:

[0061] In the detailed description of the present application, the WHU-Hi-HanChuan dataset is used as the source domain data. The WHU-Hi-HanChuan dataset was collected in Hanchuan, Hubei Province, China, from 17:57 to 18:46 on June 17, 2016, using a 17mm focal length Headwall Nano-Hyperspec imaging sensor installed on a Leica Aibot X6 UAV V1 platform. During data collection, the weather was sunny and cloudless, the temperature was about 30°C, and the relative humidity was about 70%. The study area is located at the urban-rural interface, covering buildings, water bodies, and farmland, and contains seven crops: strawberries, cowpeas, soybeans, sorghum, water spinach, watermelons, and leafy greens. The UAV flight height is 250m, the image size is 1217x303 pixels, the number of bands is 274, the wavelength coverage range is 400nm-1000nm, and the spatial resolution is about 0.109m. It is worth noting that since the dataset was collected at a low solar elevation angle in the afternoon, there are a large number of shadow areas in the image. Table 1 shows the class and number of samples of each class in the WHU-Hi-HanChuan dataset, Figure 2 The pseudo-color image and its corresponding ground truth map are shown.

[0062] Table 1. WHU-Hi-HanChuan dataset

[0063]

[0064]

[0065] The Pavia University dataset is used as the target domain dataset. The Pavia University dataset is collected by a ROSIS sensor, containing 610x340 pixels with a spatial resolution of 1.3m. The dataset has 103 spectral bands with a wavelength range of 430nm-860nm. The dataset includes 9 classes. Table 2 shows the class and number of samples of each class in the Pavia University dataset, Figure 3 The pseudo-color image and its corresponding ground truth map are shown.

[0066] Table 2. Pavia University dataset

[0067] Number Category Number of samples C1 Asphalt 6631 C2 Meadows 18649 C3 Gravel 2099 C4 Trees 3064 C5 Sheets 1345 C6 Bare soil 5029 C7 Bitumen 1330 C8 Bricks 3682 C9 Shadow 947

[0068] Step b, by performing a small sample learning task on D s and D t , enables the model to quickly learn and identify shared features between different domains from limited samples.

[0069] The specific implementation is:

[0070] Step b, D s and D t respectively, randomly select C classes from D s , and select K labeled samples from each class to form the support set S s , and select N unlabeled samples from the same C classes to form the query set Q s , where and I s = S s ∪ Q s , perform the same small sample learning task on the target domain D t to generate I t = S t ∪ Q t .

[0071] Step b1, perform source domain task and target domain task, build support set and query set, where each task (episode) contains data from source domain and target domain, and the small sample learning task of the target domain is simulated through the source domain.

[0072] Source domain task: randomly select C classes from the source domain dataset, and select K labeled samples from each class as the support set. Select N unlabeled samples from the same C classes as the query set Q s . This support set and query set constitute the source domain task, which is used to train the small sample classification model.

[0073] Target domain task: randomly select a small number of labeled samples from the target domain dataset as the support set, select K samples from each class, and select N query set samples for small sample classification. The target domain task structure is similar to the source domain task.

[0074] Step b2, train the source domain, when training the source domain, the goal is to learn a shared embedding space so that samples from different classes can have a large distance in the space, while samples of the same class are relatively close in the space. Each task is classified by calculating the distance between the query set and the support set. On the source domain, the network will be trained with the support set to make predictions on the query set and gradually optimize the model. Target domain training is similar to source domain training, and the goal of the target domain task is also to learn a shared embedding space. Through meta-learning, the model will learn the feature representation that can be transferred from the source domain to the target domain. The training of the target domain will rely more on the meta-knowledge learned by the source domain, so that the target domain can still effectively classify even with only a small amount of labeled data.

[0075] Step c, by inputting the features output by step b2 into a mapping model composed of a 2D convolution layer and a batch normalization layer, the alignment of the spectral dimensions of different domain data is completed, the influence of the dimension difference is effectively reduced, and cross-domain classification is realized.

[0076] Specifically implemented as:

[0077] The mapping network includes two main parts: a convolution layer and a batch normalization layer. The main task of the input layer is to receive input data and pass it to the subsequent convolution layer. Through the convolution layer, the feature dimension of the input data is converted, and the channel number is converted to 100. Then, the hidden layer normalizes the convolution result through the batch normalization operation, aiming to speed up the training process and improve the stability of the model. The batch normalization layer will standardize the data, thereby eliminating the inconsistency brought by different data features. After these processes, the data will be passed to the output layer, and finally the processed feature map is output.

[0078] Step d, the feature extraction network structure diagram constructed by the application is as shown in Figure 4 The data output by step c is input into the feature extractor to generate embedded features F(x), where F(x)=f embed (M(x)). At the same time, the self-attention mechanism is introduced to automatically focus on clear and stable areas in the feature map and ignore uncertain and fuzzy areas, thereby improving the perception ability of the structure information of the different domains and effectively reducing the information dislocation caused by the difference in spatial structure and resolution, thereby providing support for solving the spectral shift.

[0079] Specifically implemented as:

[0080] Assuming that the input sample is x, high-dimensional feature vectors are gradually extracted through multiple modules. First, the input data is subjected to the first residual block for preliminary feature extraction. The residual block adds the input x and the feature map f conv1 (x) obtained through convolution operation through a jump connection to obtain the output x1=x+f conv1 (x). Subsequently, the features are reduced in dimension through 3D maximum pooling operation, and the calculation formula is x2=MaxPool3D(x1). Then, the features of the key areas are extracted through the self-attention mechanism to obtain x3=SelfAttention(x2). In the second residual block, the feature map x3 is subjected to convolution operation to obtain x4=x3+f conv2 (x3), and further important features are extracted through the self-attention mechanism to obtain x5=SelfAttention(x4). Then, the features are subjected to dimension reduction through the 3D maximum pooling layer again, and the calculation formula is x6=MaxPool3D(x5). Subsequently, the features are processed through the 3D convolution layer to obtain the final output feature vector x outThe entire network effectively alleviates the gradient vanishing problem through residual connection, significantly reduces the feature dimension through max pooling, enhances the model's perception of key areas through self-attention mechanism, and fully extracts spatial and spectral features through 3D convolutional layers, providing efficient feature representation for subsequent classification tasks. The SelfAttention formula is as follows:

[0081] Assuming the input is X ∈ R n×d , where n is the number of pixels in the hyperspectral image and d is the dimension of each feature. Calculate Query, Key, and Value:

[0082] Q = XW Q , K = XW K , V = XW V

[0083] where W Q , W k , W V are learnable weight matrices;

[0084] Calculate the attention score by calculating the similarity between Query and Key:

[0085]

[0086] where A is the attention matrix, QK T represents the dot product between Query and Key, d is the size of the feature dimension, to scale the dot product to avoid gradient explosion caused by large numerical values;

[0087] Calculate the weighted output O, O = AV, where O is the final weighted output, and the feature is generated by weighting different values V. The self-attention mechanism adjusts the importance of each feature by weighting different features, thereby improving the ability to focus on target regions.

[0088] Step e, reduce the distribution difference between the source domain and the target domain by using the domain adversarial loss function to reduce the distribution difference between the source domain and the target domain, so that the features of the source domain can be effectively transferred to the target domain.

[0089] The specific implementation is as follows:

[0090] The source domain samples and target domain samples are respectively generated through the feature extractor T, to generate corresponding feature representations and The discriminator D is responsible for distinguishing whether these features belong to the source domain or the target domain, where represents the probability that the feature comes from the source domain, represents the probability that the feature comes from the target domain. By minimizing L d, the feature extractor T is optimized to generate domain-independent feature representations; at the same time, by maximizing L d , the discriminator D is optimized to enhance the discrimination ability. In this adversarial training process, the feature distributions of the source domain and the target domain are finally aligned, thereby improving the robustness of the cross-domain classification task. The adversarial domain loss function is as follows:

[0091]

[0092] Where D represents the discriminator, are the embedded features of the source domain and target domain samples respectively, where h = (f, g) joint variable, g is the category information predicted by the discriminator D, indicating the probability that the sample belongs to the source domain, by minimizing the distribution difference between the source domain and the target domain.

[0093] In step f, the data from step e, where the spectral differences between different domains have been reduced, is input into a classifier. The target domain samples are classified using a nearest neighbor classifier. This classifier calculates the distance between each target sample and the support set sample, selects the sample with the closest distance, and classifies the final feature category.

[0094] The specific implementation is:

[0095] The classification accuracy of the Pavia University dataset used in this invention is shown in Table 3. The overall accuracy OA, average accuracy AA and Kappa coefficient are used as classification evaluation indicators. The classification results of the Pavia University dataset are shown in Table 3. Figure 5 As shown in the experimental results, the present invention has only a small amount of misclassification phenomenon, is close to the actual distribution of objects, and the area of ​​misclassification is greatly reduced. In addition, Figure 6 This figure is a summary of the method of the present invention, which is intended to summarize and illustrate the key steps and processes of the present invention. The figure concisely presents the relationship between the various modules and their working principles. Every link from data input to final output is effectively organized and described.

[0096] The experimental environment of the present application is an Intel(R) Xeon(R) CPU E5-2620 v4@2.10GHz processor, 128GB memory and NVIDIA GeForce RTX 2080Ti GPU card. In addition, the deep learning framework is Pytorch, which uses Python as the programming language. The Adam optimization algorithm is used for optimization, the number of iterations is set to 10000, and other methods are set according to the parameters set by the paper authors. At the same time, in order to reduce the randomness brought by the training samples, each test is repeated 10 times to take the average value as the final test result, and the self-attention mechanism based on the present application cross-domain small sample learning (Self-Attention Deep Cross-Domain Few-Shot Learning, SADCFS L ) is compared with the extreme gradient boosting algorithm (eXtreme Gradient Boosting, XGBoost), support vector machine (Support Vector Machine, SVM) and cross-domain small sample learning method (Deep Cross-Domain Few-Shot Learning, DCFSL), and the results are shown in Table 3.

[0097] Table 3. Classification accuracy of classification methods of Pavia University dataset

[0098]

[0099]

Claims

1. A cross-domain hyperspectral image accurate classification method based on small sample learning, characterized by: The following steps are involved: Step a: Input the hyperspectral datasets of the source domain and target domain respectively, and define the source domain dataset as D s , the target domain dataset is D t , D t Including a small amount of labeled data D f and a large amount of unlabeled data D u , where D T =D f ∪D u ; Step b: D s and D t Perform small sample learning separately, randomly select s Select C categories and K labeled samples from each category to form a support set S s , select N unlabeled samples from the same C class to form a query set Q s ,in and I s =S s ∪Q s , for the target domain D t Perform the same small sample learning task and generate I t =S t ∪Q t ; Step c: After completing the small sample learning task, s and D t By mapping the network M s and M t Preprocessing, D s and D t Perform feature dimension conversion to generate I s ′ and I t ′, ensure I S and I t The input feature dimensions are consistent, and the formula is as follows: Among them, c×b s and c×b t are the width and height of the source domain and target domain respectively, and d is the target dimension; Step d: Construct feature extractor f embed , based on the improved self-attention 3D residual convolutional network structure, spatial-spectral features are extracted to generate embedded features F(x), where F(x) = f embed (M(x)); By introducing the self-attention mechanism, f embed Automatically focus on clear and stable areas in the feature map and ignore uncertain and fuzzy areas, improving the ability to perceive foreign structural information, effectively reducing information dislocation caused by differences in spatial structure and resolution, and providing support for resolving spectrum offsets; Step e: Use the domain adversarial loss function to align the distribution differences of the embedded features f = F(x) to reduce the spectral differences between different domains. The domain adversarial loss function is as follows: Where D represents the discriminator, are the embedded features of the source domain and target domain samples respectively, where h = (f, g) joint variable, g is the category information predicted by the discriminator D, and T is the multilinear dimensional transformation. Step f: Classify the target domain samples using the K-nearest neighbor (KNN) algorithm to obtain the final object classification results.

2. The cross-domain hyperspectral image accurate classification method based on small sample learning according to claim 1 is characterized in that: The step a specifically comprises the following steps: Step a1: Input source domain dataset D s , in is the hyperspectral image data of the i-th sample, is the category label of the sample. Step a2: Input target domain dataset D t , in is the hyperspectral data of the i-th sample, Is the category label of the target domain. Target domain dataset D t Including a small amount of labeled data D f and a large amount of unlabeled data D u , where D t =D f ∪D u and 3. The cross-domain hyperspectral image accurate classification method based on small sample learning according to claim 1 is characterized in that Described step b specifically comprises the following steps: Step b1: Execute source domain D s The small sample learning task is to start from the source domain dataset D S Randomly select C categories and select K labeled samples for each category to form a support set S S , select N unlabeled samples from the same C class to form a query set Q S , the formula is as follows: Step b2: Execute target domain D t Small sample learning tasks, for the target domain D T Perform the same small sample learning task to generate the support set S of the target domain T and query set Q T , the formula is as follows:

4. The cross-domain hyperspectral image accurate classification method based on small sample learning according to claim 1 is characterized in that The step c specifically comprises the following steps: Step c1: To ensure that the feature dimensions of the source domain and the target domain are consistent, a mapping network M is used. s and M t The hyperspectral data of the source and target domains are converted into a unified dimension. The data processed by the mapping network is represented as: I' s =M s (I s ),I' t =M t (I t ) Among them, M s and M t are the mapping networks of the source domain and the target domain respectively, I s and I t are the original data of the source domain and the target domain respectively, I s 'and I t ' is the data processed by the mapping network. Step c2, the data dimension after mapping is and Where c and b are the width and height of the image, respectively, and d is the feature dimension. The mapping network aligns the feature dimensions of the source and target domain data, reducing the feature differences caused by the different number of bands and ensuring that both can be effectively processed in the same feature space.

5. The cross-domain hyperspectral image accurate classification method based on small sample learning according to claim 1 is characterized in that The step d is specifically as follows: Step d1: Construct a feature extractor. The network architecture includes multiple convolutional layers, each of which is a 3D convolution operation. The 3D convolutional network is used to simultaneously extract spatial and spectral features from the input hyperspectral image data. Reluctant linear unit (ReLU) activation functions are used between layers to enhance nonlinear characteristics. To alleviate the vanishing gradient problem common in deep neural networks, a residual connection strategy is adopted. By introducing direct skip connections between network layers, the model can more effectively learn the difference between input and output, thereby simplifying the training process. The specific process is as follows: F s =f embed (M s (I s )),F t =f embed (M t (I t )) Among them, f embed represents the feature extractor, M s and M t are the mapping networks of the source domain and the target domain respectively, I s and I t It is the data of source domain and target domain. y=f(x)+x Here, f(x) represents the network's feedforward operation, and x represents the input features. Adding residual connections to the input x allows the network to more easily learn complex mappings. In convolutional neural networks, residual connections pass inputs directly to subsequent layers via "skip connections," helping to prevent vanishing gradients and accelerate training. Step d2: Introduce the self-attention mechanism into the feature extractor. The self-attention mechanism enables the network to focus on specific areas in the image, thereby improving the feature expression of important areas and ignoring unimportant areas. The self-attention mechanism calculates the relationship between each pixel to weight the features of different areas, thereby improving the network's spatial perception ability. Assume that the input is X∈R n×d , where n is the number of pixels in the hyperspectral image and d is the dimension of each feature. Calculate the query, key, and value: Q=XW Q ,K=XW K ,V=XW V Among them, W Q ,W K ,W V is a learnable weight matrix; Calculate the attention score, which is generated by calculating the similarity between the query and the key: Among them, A is the attention matrix, QK T represents the dot product between the query and the key, d is the size of the feature dimension, Used to scale dot products to avoid gradient explosion caused by excessive values; Calculate the weighted output O, O = AV, where O is the final weighted output, which is a feature generated by weighting different values ​​​​V. The self-attention mechanism adjusts the importance of each feature by weighting different features, thereby improving the ability to aggregate the target area.

6. The cross-domain hyperspectral image accurate classification method based on small sample learning according to claim 1 is characterized in that In step e, the domain adversarial loss function is used to reduce the distribution difference between the source domain and the target domain, so that the features of the source domain can be effectively transferred to the target domain. The domain adversarial loss function is as follows: Where D represents the discriminator, where h = (f, g) is the joint variable, g is the category information predicted by the discriminator D, and represents the probability that the sample belongs to the source domain. By minimizing the distribution difference between the source and target domains, the target domain can share the features of the source domain, thus achieving cross-domain classification tasks. T is a multi-linear dimensionality transformation.

7. The cross-domain hyperspectral image accurate classification method based on small sample learning according to claim 1 is characterized in that In step f, the samples in the target domain are classified using a nearest neighbor classifier, which calculates the distance between each target sample and the support set sample, selects the nearest sample for classification, and obtains the final feature category.

Images