Feature alignment based cross-domain hyperspectral image crop fine classification method

Abstract

The application provides a cross-domain hyperspectral image crop fine classification method based on feature alignment, and belongs to the technical field of image processing. First, input a small amount of labeled target domain data and sufficient labeled source domain data, and extract feature information by using an embedding model. Second, introduce an asymmetric convolution, flexibly adapt to feature extraction in different directions, accurately capture crop edges and contours through different direction convolution kernels, and ensure the retention of boundary information under different scales. Then, realize the distribution alignment of the source domain and the target domain through a conditional adversarial domain adaptation strategy, and overcome the spectral shift. In addition, a sharpness perception minimization smoothing parameter optimization is adopted, so that the model is not sensitive to the change of feature distribution, and the fluctuation caused by the spectral shift is reduced. Finally, K‑nearest neighbor (KNN) algorithm is used for classification to obtain the crop category. The experimental results show that the classification accuracy of the method on the Indian Pines data set is better than that of the existing method, and a new idea is provided for cross-domain hyperspectral image crop classification.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image classification, specifically to a method for fine classification of crops in cross-domain hyperspectral images based on feature alignment. Background Technology

[0002] Hyperspectral images (HSI) are widely used in mineral identification and precision agriculture due to the rich information they provide in both spatial and spectral dimensions. However, traditional hyperspectral image classification methods often exhibit low generalization ability and classification accuracy when faced with cross-domain scenarios and limited labeled data.

[0003] Traditional classification methods, such as Support Vector Machines (SVM) and Random Forests, have achieved some success in hyperspectral image classification. However, their reliance on shallow feature extraction limits their ability to capture complex spatial-spectral features. In recent years, deep learning methods have gradually become the mainstream approach for hyperspectral image classification. For example, Convolutional Neural Networks (CNNs), by stacking multiple convolutional layers, can effectively extract spatial and spectral features and achieve significant performance improvements. Transformer-based methods utilize their unique structure to model hyperspectral images, comprehensively capturing multidimensional relationships in the data and preserving fine-grained features well. This approach performs exceptionally well in complex scenarios, especially when in-depth mining of spatial-spectral information is required. However, despite the significant advantages of Transformers in extracting global features, the performance of these methods typically depends on a large number of labeled samples, limiting their application under data-scarce conditions.

[0004] In practical applications, acquiring labeled hyperspectral data is both expensive and time-consuming. The scarcity of labeled data in the target domain makes it difficult for models to effectively learn features. To address this, researchers have proposed domain-adaptive learning methods, aiming to leverage the abundant labeled data in the source domain to transfer knowledge to the target domain, thereby improving the model's classification performance in the target domain. However, significant distributional differences may exist between the source and target domains, including spectral features, spatial resolution, and noise levels. These inter-domain differences lead to a decline in model performance in the target domain. Transfer learning methods, by utilizing source domain data to improve target domain performance, have become an important strategy for addressing the problem of scarce labeled data. However, when faced with significant distributional differences between the source and target domains, they often fail to achieve ideal transfer effects. When spectral features or spatial resolution differences are significant, the performance of transfer learning models is greatly limited. Therefore, how to effectively mitigate inter-domain distributional differences and improve the model's generalization ability remains a significant challenge in current cross-domain hyperspectral image classification research.

[0005] To overcome the aforementioned problems, this invention proposes a cross-domain hyperspectral image-based fine classification method for crops based on feature alignment. This method first inputs a small amount of labeled target domain data and a sufficient amount of labeled source domain data, and then extracts feature information using an embedding model. Subsequently, asymmetric convolution is introduced, and by designing convolution kernels with different directions, the edges and contours of crops are flexibly captured, ensuring the integrity of boundary information at different scales. Simultaneously, a conditional adversarial domain adaptation strategy is introduced to align the feature distributions of the source and target domains, thereby overcoming the spectral shift problem and improving the robustness of cross-domain classification. Furthermore, to further enhance the model's stability and adaptability to changes in feature distribution, this method employs sharpness-aware minimization of smoothing parameters to reduce the uncertainty caused by spectral shift. In the classification stage, a KNN classifier is used to classify the embedded features, ultimately obtaining the crop category. Summary of the Invention

[0006] This invention proposes a feature-aligned method for fine classification of crops in cross-domain hyperspectral images. It combines the feature extraction capabilities of an embedding model, the edge-capturing advantages of asymmetric convolution, the distribution alignment effect of a conditional adversarial domain adaptation strategy, and the stability improvement of sharpness-aware minimization smoothing optimization. First, the method extracts feature information from both the source and target domains using an embedding model. Then, asymmetric convolution is used to flexibly capture the edges and contours of crops, ensuring complete boundary information is maintained at different scales. Next, a conditional adversarial domain adaptation strategy is combined to align the feature distributions of the source and target domains, effectively mitigating the spectral shift problem and improving the robustness of cross-domain classification. Finally, sharpness-aware minimization smoothing optimization reduces the uncertainty caused by spectral shift, improving the model's stability and adaptability to changes in feature distribution. In the classification stage, a KNN classifier is used to accurately classify the embedded features, ultimately achieving fine prediction of crop categories. This method demonstrates superior performance in cross-domain hyperspectral image classification tasks, especially under conditions of data scarcity and spectral shift, offering significant technical advantages. It provides an innovative solution to the cross-domain problem in hyperspectral image classification and has significant scientific research value and practical application prospects.

[0007] The objective of this invention is achieved as follows:

[0008] A feature-aligned method for fine classification of crops across hyperspectral images includes the following steps:

[0009] Step a: Input the target domain data and source domain data into the model respectively. The target domain data contains a small number of labeled samples, and the source domain data contains a sufficient number of labeled samples. The target domain data is used to construct... Source domain data is used to build The target domain samples include a small number of labeled samples. and a large number of unlabeled samples ,satisfy ;

[0010] Step b, to reduce and The difference in feature distribution between them is addressed using a mapping layer. and right and The data undergoes feature dimension transformation to map it to a unified feature space, generating feature vectors. and The specific formula is as follows:

[0011]

[0012] in, and The input features are from the source domain and the target domain, respectively, with dimensions of [dimensions missing]. and , and The mapping layer between the source and target domains is responsible for the dimensionality transformation of features, converting the number of bands in the source and target domains. and Mapping to target dimension ; and These are the alignment features for the source and target domains, respectively, with a unified dimension. ;

[0013] Step c: Using the embedding model Extract target domain and source domain The feature information is used to map the data into a high-dimensional space to generate embedded features. Furthermore, asymmetric convolution is introduced, and feature information is comprehensively extracted by flexibly designing convolution kernels in different directions. This method captures the edges and contours of crops to ensure the integrity and effectiveness of boundary information at different scales. The convolution kernel structure of asymmetric convolution is tailored to the characteristics of the input data to achieve accurate description of boundary details and local features, thereby improving the classification performance and generalization ability of the model.

[0014] Step d: Align embedded features using domain adversarial loss function To mitigate the distribution differences and reduce spectral differences between different domains, the domain adversarial loss function is as follows:

[0015]

[0016] Where D represents the discriminator, , These are the embedding features of samples from the source and target domains, respectively. The joint variable is g, which is the class information predicted by the discriminator D, and T is the multilinear dimensional transformation.

[0017] Step e: To further enhance feature alignment, a sharpness-aware minimization strategy is introduced. By optimizing model parameters, the loss function is made smoother, improving the model's robustness to changes in feature distribution and effectively reducing classification performance fluctuations caused by spectral shift. The model's worst-case performance is optimized by considering the most challenging perturbation direction at each parameter update. By optimizing smoothing parameters, the model maintains stable classification results even with slight changes in feature distribution. This strategy not only enhances the model's generalization ability but also reduces misclassification caused by spectral shift, thus better adapting to feature differences between the source and target domains.

[0018] Step f: Classify the target domain samples using a nearest neighbor classifier to obtain the final crop classification result.

[0019] The above-mentioned method for fine classification of crops in cross-domain hyperspectral images based on feature alignment is characterized in that step a specifically includes the following steps:

[0020] Step a1: Input source domain dataset , in For the first Hyperspectral image data of one sample This is the category label for the sample.

[0021] Step a2: Input the target domain dataset , ,in For the first Hyperspectral data of one sample, These are the category labels for the target domain. Target domain dataset. Includes a small amount of labeled data and a large amount of unlabeled data ,in .

[0022] The above-mentioned method for fine classification of crops in cross-domain hyperspectral images based on feature alignment is characterized in that step b specifically includes the following steps:

[0023] Step b1: To ensure that the feature dimensions of the source and target domains are consistent, a mapping layer is used. and The hyperspectral data from both the source and target domains are transformed into a unified dimension. The data processed by the mapping layer is represented as follows:

[0024]

[0025] in, and These are mapping layers for the source and target domains, respectively, responsible for feature dimension transformation. and The input features are the source and target domains. and This is the data after it has been processed by the mapping layer.

[0026] Step b2, the dimensions of the mapped data are and Where W and H are the width and height of the image, respectively. The feature dimension is used to align the feature dimensions of the source and target domain data, thus matching the number of bands in the source and target domains. and Mapping to target dimension This reduces the differences in characteristic distribution.

[0027] The above-mentioned method for fine classification of crops in cross-domain hyperspectral images based on feature alignment is characterized in that step c specifically includes the following steps:

[0028] Step c1: Construct a feature extractor. The network architecture consists of multiple convolutional layers, combining 3D convolution and asymmetric convolution operations to efficiently extract features from the input image. Asymmetric convolution, through flexible design of convolution kernels with different orientations and sizes, can accurately capture detailed features of the image, including edge and contour information. Combining asymmetric convolution with residual connections further enhances the model's ability to learn the differences between input and output, while ensuring the integrity of boundary information at different scales. With the support of asymmetric convolution, the embedded features of the source and target domains can fully characterize the details and boundary features of the image. These embedded features not only significantly improve the accuracy of image representation but also provide stronger support for subsequent classification tasks. The specific process is as follows:

[0029]

[0030] in, Indicates feature extractor, and These are mapping layers for the source and target domains, respectively. and It is source domain data.

[0031] y=f(x)+x

[0032] Here, f(x) represents the network's feedforward operation, and x is the input feature. Adding residual connections to the input x allows the network to learn complex mappings more easily. In convolutional neural networks, residual connections, through "skip connections," directly pass the input to subsequent layers, helping to avoid gradient vanishing and accelerate training.

[0033] Step c2: Introduce asymmetric convolution into the residual connections. Asymmetric convolution uses multiple kernels in different directions to extract features, accurately capturing detailed information about crops, including edges and contours. The formula for asymmetric convolution is:

[0034]

[0035] in, These are input features. It represents the convolution kernel, and * represents the convolution operation. These are the output features after convolution. This can be achieved by designing convolution kernels in different directions. Convolution operations are performed at different scales to obtain multiple feature maps. These feature maps are then fused to enhance the ability to capture edge information. With the help of asymmetric convolution, the embedding features of the source and target domains can effectively capture details in the image, including edge and contour information. These embedding features not only improve the accuracy of image representation but also provide stronger support for subsequent classification tasks.

[0036] The above-mentioned cross-domain hyperspectral image crop fine classification method based on feature alignment is characterized in that, in step d, the distribution difference between the source domain and the target domain is reduced by using a domain adversarial loss function, so that the features of the source domain can be effectively transferred to the target domain. The domain adversarial loss function is as follows:

[0037]

[0038] Where D represents the discriminator, , These are the embedding features of samples from the source and target domains, respectively. The joint variable, g, represents the class information predicted by the discriminator D, indicating the probability that a sample belongs to the source domain. By minimizing the distribution difference between the source and target domains, the target domain can share features of the source domain, thereby achieving cross-domain classification tasks. T represents the multilinear dimension transformation.

[0039] The above-mentioned method for fine classification of crops in cross-domain hyperspectral images based on feature alignment is characterized in that step e specifically includes the following steps:

[0040] The core idea of ​​SAM is to find the most challenging perturbation direction for model training (even the direction with the steepest loss function) at each step of gradient descent, and then further optimize the model in that direction. This is achieved by adding additional perturbation terms. To approximate the loss value in the worst-case scenario:

[0041]

[0042] in: These are model parameters; For the disturbance term, subject to the radius limit, This is the loss function.

[0043] The SAM strategy corrects the direction of model parameter updates, enabling the updated model parameters to avoid overfitting to sharp regions of local features, thus resulting in a smoother feature distribution. The specific process is as follows:

[0044] First, calculate the perturbation direction of the current gradient. Adjust parameters in the direction of disturbance Then, the loss was recalculated. Finally, the model parameters are updated to make it more robust to changes in feature distribution. Through the above optimizations, SAM can effectively reduce the model's sensitivity to changes in the target domain's spectrum, enabling it to maintain high classification accuracy even when the spectral characteristics of the source and target domains differ significantly.

[0045] The aforementioned method for fine classification of crops in cross-domain hyperspectral images based on feature alignment is characterized in that, in step f, a nearest neighbor classifier is used to classify samples in the target domain. This classifier calculates the distance between each target sample and the support set samples, selects the nearest sample for classification, and obtains the final crop category.

[0046] Beneficial effects:

[0047] This invention proposes a feature-aligned method for fine classification of crops in cross-domain hyperspectral images, belonging to the field of image processing technology. First, a target domain with a small amount of labeled data and a source domain with sufficient labeled data are input, and feature information is extracted using an embedding model. Second, asymmetric convolution is introduced to flexibly adapt to feature extraction in different directions. Convolutional kernels in different directions accurately capture crop edges and contours, ensuring the preservation of boundary information at different scales. Subsequently, a conditional adversarial domain adaptation strategy is used to align the distribution of the source and target domains, overcoming spectral shift. Furthermore, a sharpness-aware minimization smoothing parameter optimization is employed to make the model insensitive to changes in feature distribution, reducing fluctuations caused by spectral shift. Finally, a KNN classifier is used for classification to obtain crop categories. Experimental results show that the method outperforms existing methods in classification accuracy on the Indian Pines dataset, providing a new approach for crop classification in cross-domain hyperspectral images. Attached Figure Description

[0048] Figure 1 This is an overall flowchart of the cross-domain hyperspectral image crop fine classification method based on feature alignment in the present invention.

[0049] Figure 2 These are the pseudo-color images of the WHU-Hi-HanChuan dataset and their corresponding ground truth maps in the method of this invention.

[0050] Figure 3 It refers to the pseudo-color image of the Indian Pines dataset and its corresponding ground truth map in the method of this invention.

[0051] Figure 4 This is a schematic diagram of the feature extractor in the method of the present invention.

[0052] Figure 5 This is a classification result diagram of the Indian Pines dataset in the method of this invention.

[0053] Figure 6 The accompanying drawings are a summary of the method of this invention. Detailed Implementation

[0054] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0055] The flowchart of the cross-domain hyperspectral image-based fine classification method for crops based on feature alignment in a specific embodiment of the present invention is as follows: Figure 1 , Figure 6 As shown, it includes the following steps:

[0056] Step a: Input the hyperspectral datasets for the source and target domains respectively, and define the source domain dataset as... The target domain dataset is Randomly select training samples to construct training and test sets:

[0057] In this specific embodiment of the invention, the WHU-Hi-HanChuan dataset is used as the source domain data. The WHU-Hi-HanChuan dataset was collected on June 17, 2016, using a 17mm focal length Headwall Nano-Hyperspec imaging sensor mounted on a Leica Aibot X6 UAV V1 platform. The UAV flew at an altitude of 250m, the image size was 1217×303 pixels, the number of bands was 274, covering a wavelength range of 400 nm to 1000 nm, and the spatial resolution was approximately 0.109m. It is worth noting that because the dataset was collected in the afternoon when the sun's altitude angle was low, there are many shadowed areas in the images. Table 1 shows the categories and number of samples in each class of the WHU-Hi-HanChuan dataset. Figure 2 The pseudo-color image and its corresponding ground truth map are shown.

[0058] Table 1. WHU-Hi-HanChuan dataset

[0059] C1 Strawberry 44735 C9 Grass 9469 C2 Cowpea 22753 C10 Red roof 10516 C3 Soybean 10287 C11 Gray roof 16911 C4 Sorghum 5353 C12 Plastic 3679 C5 Water spinach 1200 C13 Bare soil 9116 C6 Watermelon 4533 C14 Road 18560 C7 Greens 5903 C15 Bright object 1136 C8 Trees 17978 C16 Water 75401

[0060] The Indian Pines dataset was used as the target domain dataset. Collected by AVIRIS in northwestern Indiana, USA in 1992, the Indian Pines dataset contains 200 bands with wavelengths ranging from 400 to 2500 nanometers, a spatial resolution of 20 meters per pixel, and an image size of 145 × 145 pixels. The dataset includes 16 vegetation categories. Table 2 shows the category and number of samples in each class of the Indian Pines dataset. Figure 3 The pseudo-color image and its corresponding ground truth map are shown.

[0061] Table 2. Indian Pines Dataset

[0062] C1 Alfalfa 46 C9 Oats 20 C2 Corn-notill 1428 C10 Sovbean-notill 972 C3 Corn-mintill 830 C11 Soybean-mintill 2455 C4 Corn 237 C12 Soybean-cleam 593 C5 Grass-pasture 483 C13 Wheat 205 C6 Grass-tree 730 C14 Woods 1265 C7 Grass-pasture-mowed 28 C15 Buildings-Grass-Trees-Drives 386 C8 Hay-windrowed 478 C16 Stone-Steel-Towers 93

[0063] Step b: By inputting the data output from step a into a mapping model consisting of 2D convolutional layers and batch normalization layers, the spectral dimensions of data from different domains are aligned, effectively reducing the impact of dimensional differences and thus achieving cross-domain classification.

[0064] The specific implementation is as follows:

[0065] The mapping network consists of two main parts: convolutional layers and batch normalization layers. The main task of the input layer is to receive the input data and pass it to the subsequent convolutional layers. Through the convolutional layers, the feature dimensions of the input data are transformed, and the number of channels is converted to 100. Next, the hidden layers normalize the convolutional results through batch normalization, aiming to accelerate the training process and improve the model's stability. The batch normalization layer standardizes the data, thereby eliminating inconsistencies caused by different data features. After these processing steps, the data is passed to the output layer, which finally outputs the processed feature map.

[0066] Step c: A schematic diagram of the feature extraction network structure constructed in this invention is shown below. Figure 4 As shown, by inputting the data output from step b into the feature extractor, embedded features are generated. ,in The feature extraction network combines asymmetric convolution and residual connections to effectively extract key features from source and target domain images, enhancing the model's ability to perceive edges, contours, and regional details. At the same time, it reduces information misalignment caused by differences in spatial structure and resolution, providing high-quality feature representations for subsequent classification tasks.

[0067] The specific implementation is as follows:

[0068] First, input the data. Preliminary feature extraction is performed using the first residual block. Within each residual block, the feature map after convolution is compared to the input. The elements are added together by skip connections to form the output. The specific operation is as follows:

[0069]

[0070] in, This represents a standard 3D convolution operation. These are input features. It is the output of the residual block.

[0071] Next, asymmetric convolution is used to further extract detailed features. Asymmetric convolution uses multi-directional kernels to process feature maps to capture details such as edges and contours in the image. The asymmetric convolution operation used here is as follows:

[0072]

[0073] in, An asymmetric convolution kernel is used for convolution operations in different directions. This allows for the flexible design of convolution kernels with different directions and sizes, enabling the capture of more complex features.

[0074] Feature map after asymmetric convolution Through a standard 3D convolution operation Further processing yields new feature maps. :

[0075]

[0076] Then, use residual join to... and The feature maps are fused to obtain the final output feature map. :

[0077]

[0078] This residual connection helps the model better learn complex mapping relationships while avoiding gradient vanishing. Finally, after multiple residual blocks and asymmetric convolutions, the network's final output feature vector is obtained. :

[0079]

[0080] Step d: The embedded features generated in step c By using a domain adversarial loss function, the distribution difference between the source and target domains is reduced, enabling the features of the source domain to be effectively transferred to the target domain.

[0081] The specific implementation is as follows:

[0082] Source domain samples and target domain samples are respectively processed by feature extractors Generate the corresponding feature representation and Discriminator The function responsible for distinguishing whether these features belong to the source domain or the target domain is... This represents the probability that a feature originates from the source domain. This represents the probability that a feature originates from the target domain. This is achieved by minimizing... Feature extractor It is optimized to generate domain-independent feature representations; simultaneously, by maximizing Discriminator It is optimized to enhance discriminative power. This adversarial training process ultimately aligns the feature distributions of the source and target domains, thereby improving the robustness of cross-domain classification tasks. The adversarial domain loss function is as follows:

[0083]

[0084] Where D represents the discriminator, , These are the embedding features of samples from the source and target domains, respectively. The joint variable, g, represents the class information predicted by the discriminator D, indicating the probability that a sample belongs to the source domain, and is calculated by minimizing the distribution difference between the source and target domains.

[0085] Step e: To further enhance feature alignment, a sharpness-aware minimization strategy is introduced. By optimizing the model parameters, the loss function is made smoother, improving the model's robustness to changes in feature distribution, thereby effectively reducing the fluctuations in classification performance caused by spectral shift.

[0086] The specific implementation is as follows:

[0087] The core idea of ​​SAM is to find the most challenging perturbation direction for model training (even the direction with the steepest loss function) at each step of gradient descent, and then further optimize the model in that direction. This is achieved by adding additional perturbation terms. To approximate the loss value in the worst-case scenario:

[0088]

[0089] in: These are model parameters; For the disturbance term, subject to the radius limit, This is the loss function.

[0090] The SAM strategy corrects the direction of model parameter updates, enabling the updated model parameters to avoid overfitting to sharp regions of local features, thus resulting in a smoother feature distribution. The specific process is as follows:

[0091] First, calculate the perturbation direction of the current gradient. Adjust parameters in the direction of disturbance Then, the loss was recalculated. Finally, the model parameters are updated to make it more robust to changes in feature distribution. Through the above optimizations, SAM can effectively reduce the model's sensitivity to changes in the target domain's spectrum, enabling it to maintain high classification accuracy even when the spectral characteristics of the source and target domains differ significantly.

[0092] Step f: Input the data from step e, in which the spectral differences between different domains have been reduced, into the classifier, and use a nearest neighbor classifier to classify the target domain samples. This classifier calculates the distance between each target sample and the support set samples, selects the nearest sample for classification, and obtains the final crop category.

[0093] The specific implementation is as follows:

[0094] The classification accuracy of the Indian Pines dataset used in this invention is shown in Table 3. Overall accuracy (OA), average accuracy (AA), and the Kappa coefficient were used as classification evaluation metrics. The classification results of the Indian Pines dataset are shown below. Figure 5 As shown in the figure. Experimental results show that the present invention has only a small number of misclassifications, closely reflects the actual distribution of crops, and greatly reduces the area of ​​misclassification.

[0095] The experimental environment for this invention consisted of an Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz processor, 128 GB of RAM, and an NVIDIA GeForce RTX 2080Ti GPU. Furthermore, the deep learning framework used was PyTorch, with Python as the programming language. The Adam optimization algorithm was employed, with 10,000 iterations. Other methods followed the parameters set by the authors of the paper. To reduce the randomness of the training samples, each experiment was repeated 10 times, and the average value was used as the final experimental result. To verify the effectiveness of this invention, the Feature Alignment based Cross-Domain Learning (FABCDL) was compared with eXtreme Gradient Boosting (XGBoost), Support Vector Machine (SVM), and Deep Cross-Domain Few-Shot Learning (DCFSL). The results are shown in Table 3.

[0096] Table 3. Classification accuracy of classification methods for the Indian Pines dataset

[0097] 1、 68.29±12.91 67.48±17.13 94.15±8.81 94.39±6.82 2、 24.43±18.83 28.48±8.69 40.93±6.69 46.40±8.79 3、 25.13±15.00 34.34±11.28 45.70±6.39 47.13±9.94 4、 29.60±2.87 60.20±3.51 72.16±16.83 71.77±15.01 5、 48.26±19.90 50.28±4.51 71.23±7.87 71.53±6.35 6、 63.91±6.22 77.61±2.68 83.35±7.09 82.68±9.96 7、 68.12±20.55 85.51±17.57 98.70±3.91 98.26±2.88 8、 36.36±3.96 70.54±12.76 81.16±13.84 82.56±11.53 9、 62.22±23.41 91.11±15.40 99.33±2.00 99.33±2.00 10、 29.85±15.35 45.81±3.38 56.29±10.77 55.45±11.80 11、 19.69±3.84 35.01±16.04 57.49±12.78 60.13±12.27 12、 24.94±8.33 38.72±7.23 43.84±15.17 43.67±9.85 13、 82.33±9.65 92.50±9.10 96.35±4.99 96.20±3.99 14、 57.17±6.98 63.97±14.02 86.21±6.27 88.44±5.10 15、 27.56±7.83 37.97±16.19 68.92±8.71 73.99±6.30 16、 89.39±8.83 80.30±10.92 98.64±1.42 97.95±2.43 OA 34.70±0.76 46.82±6.07 62.65±2.60 64.56±2.35 AA 47.33±1.64 59.99±3.93 74.65±1.93 75.62±1.92 KAPPA 28.47±1.26 40.99±6.07 58.08±2.72 60.07±2.51

Claims

1. A method for fine classification of crops in cross-domain hyperspectral images based on feature alignment, characterized in that, Includes the following steps: Step a: Input the target domain data and source domain data into the model respectively. The target domain data contains a small number of labeled samples, and the source domain data contains a sufficient number of labeled samples. The target domain data is used to construct... Source domain data is used to build The target domain samples include a small number of labeled samples. and a large number of unlabeled samples ,satisfy ; Step b, to reduce and The difference in feature distribution between them is addressed using a mapping layer. and right and The data undergoes feature dimension transformation to map it to a unified feature space, generating feature vectors. and The specific formula is as follows: in, and The input features are from the source domain and the target domain, respectively, with dimensions of [dimensions missing]. and , and The mapping layer between the source and target domains is responsible for the dimensionality transformation of features, converting the number of bands in the source and target domains. and Mapping to target dimension ; and These are the alignment features for the source and target domains, respectively, with a unified dimension. ; Step c: Using the embedding model Extract target domain and source domain The feature information is used to map the data into a high-dimensional space to generate embedded features. Furthermore, asymmetric convolution is introduced, and feature information is comprehensively extracted by flexibly designing convolution kernels in different directions. This method captures the edges and contours of crops, ensuring the integrity and effectiveness of boundary information at different scales. The convolution kernel structure of asymmetric convolution is tailored to the characteristics of the input data, achieving accurate description of boundary details and local features, thereby improving the classification performance and generalization ability of the model. Step C is as follows: Step c1: Construct a feature extractor. The network architecture consists of multiple convolutional layers, combining 3D convolution and asymmetric convolution operations to efficiently extract features from the input image. Asymmetric convolution, through flexible design of convolution kernels with different directions and sizes, can accurately capture detailed features of the image, including edge and contour information. Combining asymmetric convolution with residual connections further enhances the model's ability to learn the differences between input and output, while ensuring the integrity of boundary information at different scales. With the support of asymmetric convolution, the embedded features of the source and target domains can fully characterize the details and boundary features of the image. These embedded features not only significantly improve the accuracy of image representation but also provide stronger support for subsequent classification tasks. The specific process is as follows: in, Indicates feature extractor, and These are mapping layers for the source and target domains, respectively. and It is source domain data; y=f(x)+x Here, f(x) is the feedforward operation of the network, and x is the input feature. By adding residual connections to the input x, the network can learn complex mappings more easily. In convolutional neural networks, residual connections pass the input directly to subsequent layers through "skip connections," which helps to avoid gradient vanishing and accelerate training. Step c2: Introduce asymmetric convolution into the residual connections. Asymmetric convolution uses multiple convolution kernels in different directions to extract features, accurately capturing detailed information about crops, including edges and contours. The formula for asymmetric convolution is: in, These are input features. It represents the convolution kernel, and * represents the convolution operation. It refers to the output features after convolution, which are obtained by designing convolution kernels in different directions. Convolution operations are performed at different scales to obtain multiple feature maps. Then, these feature maps are fused to enhance the ability to capture edge information. With the help of asymmetric convolution, the embedding features of the source and target domains can effectively capture the details in the image, including edge and contour information. These embedding features not only improve the accuracy of image representation, but also provide stronger support for subsequent classification tasks. Step d: Align embedded features using domain adversarial loss function To mitigate the distribution differences and reduce spectral differences between different domains, the domain adversarial loss function is as follows: Where D represents the discriminator, , These are the embedding features of samples from the source and target domains, respectively. The joint variable, g, is the class information predicted by the discriminator D. By minimizing the distribution difference between the source domain and the target domain, the target domain can share the features of the source domain, thereby achieving cross-domain classification tasks. T is the multilinear dimension transformation. Step e: To further enhance feature alignment, a Sharpness Aware Minimization (SAM) strategy is introduced. By optimizing model parameters, the loss function is made smoother, improving the model's robustness to changes in feature distribution. This effectively reduces classification performance fluctuations caused by spectral shift. By considering the most challenging perturbation direction at each parameter update, the model's performance in the worst case is optimized. By optimizing the smoothing parameters, the model can maintain stable classification results even with slight changes in feature distribution. This strategy not only enhances the model's generalization ability but also reduces misclassification caused by spectral shift, thus better adapting to feature differences between the source and target domains. The core idea of ​​SAM is to find the most challenging perturbation direction for model training at each gradient descent step, and then further optimize the model in that direction by adding additional perturbation terms. To approximate the loss value in the worst-case scenario: in: These are model parameters; For the disturbance term, subject to the radius limit, The loss function; The SAM strategy corrects the direction of model parameter updates, enabling the updated model parameters to avoid overfitting to sharp regions of local features, thus resulting in a smoother feature distribution. The specific process is as follows: First, calculate the perturbation direction of the current gradient. Adjust parameters in the direction of disturbance Then, the loss was recalculated. Finally, the model parameters are updated to make it more robust to changes in feature distribution. Through the above optimization, SAM can effectively reduce the model's sensitivity to changes in the target domain spectrum, so that it can still maintain a high classification accuracy when the spectral characteristics of the source domain and the target domain are very different. Step f: Classify the target domain samples using the K-nearest neighbor (KNN) algorithm to obtain the final crop classification result.

2. The method for fine classification of crops in cross-domain hyperspectral images based on feature alignment according to claim 1, characterized in that, Step a specifically includes the following steps: Step a1: Input source domain dataset , in For the first Hyperspectral image data of one sample This is the category label for the sample; Step a2: Input the target domain dataset , ,in For the first Hyperspectral data of one sample, These are the category labels for the target domain; the target domain dataset. Includes a small amount of labeled data and a large amount of unlabeled data ,in .

3. The method for fine classification of crops in cross-domain hyperspectral images based on feature alignment according to claim 1, characterized in that, Step b specifically includes the following steps: Step b1: To ensure that the feature dimensions of the source and target domains are consistent, a mapping layer is used. and The hyperspectral data from the source and target domains are transformed into a unified dimension, and the data after processing by the mapping layer is represented as follows: in, and These are mapping layers for the source and target domains, respectively, responsible for feature dimension transformation. and The input features are the source and target domains. and This is the data after it has been processed by the mapping layer; Step b2, the dimensions of the mapped data are and Where W and H are the width and height of the image, respectively. As a feature dimension, the mapping layer's role is to align the feature dimensions of the source and target domain data, thereby increasing the number of bands in the source and target domains. and Mapping to target dimension This reduces the differences in characteristic distribution.

Images