Cross-domain hyperspectral image crop fine classification based on feature alignment

Abstract

The invention provides a cross-domain hyperspectral image crop fine classification method based on feature alignment, and belongs to the technical field of image processing. Firstly, target domain data with a small number of labels and source domain data with sufficient labels are input, and feature information is extracted through an embedded model. Secondly, asymmetric convolution is introduced to flexibly adapt to feature extraction in different directions, and crop edges and contours are accurately captured through convolution kernels in different directions to ensure boundary information retention under different scales; and then, distribution alignment of the source domain and the target domain is realized through a conditional adversarial domain adaptation strategy, and spectrum offset is overcome. In addition, sharpness perception minimization smoothing parameter optimization is adopted, so that the model is insensitive to feature distribution change, and fluctuation caused by spectrum offset is reduced. And finally, carrying out classification by utilizing a K-nearest neighbor (KNN) algorithm, and obtaining crop categories. Experimental results show that the classification precision of the method on an Indian Pines data set is superior to that of an existing method, and a new thought is provided for agricultural classification of cross-domain hyperspectral images.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing image classification, and in particular to a cross-domain hyperspectral image crop fine classification method based on feature alignment. Background Art

[0002] Hyperspectral images (HSI) are widely used in fields such as mineral identification, military reconnaissance, and precision agriculture because of the rich information they provide in spatial and spectral dimensions. However, traditional hyperspectral image classification methods often show low generalization ability and classification accuracy when faced with cross-domain scenarios and limited labeled data.

[0003] Traditional classification methods, such as support vector machines (SVM) and random forests, have achieved certain results in hyperspectral image classification, but they are limited in capturing complex spatial-spectral features because they mainly rely on shallow feature extraction. In recent years, deep learning methods have gradually become the mainstream means of hyperspectral image classification. For example, convolutional neural networks (CNNs) can effectively extract spatial and spectral features by stacking multiple convolutional layers, and achieve significant performance improvements. Transformer-based methods use their unique structure to model hyperspectral images, which can fully capture the multidimensional relationships in the data and better preserve fine-grained features. This method performs well in complex scenes, especially when it is necessary to deeply mine spatial spectral information. However, despite the obvious advantages of Transformer in extracting global features, the performance of these methods usually depends on a large number of labeled samples, which limits their application under data-scarce conditions.

[0004] In practical applications, the acquisition of hyperspectral data labels is expensive and time-consuming, especially in the target domain, where the scarcity of labeled data makes it difficult for the model to effectively learn features. To this end, researchers have proposed domain adaptive learning methods, which aim to use the rich labeled data in the source domain to transfer knowledge to the target domain, thereby improving the classification performance of the model in the target domain. However, there may be significant distribution differences between the source and target domains, including spectral features, spatial resolution, and noise levels. This inter-domain difference will lead to a decrease in the performance of the model in the target domain. Transfer learning methods have become an important strategy to solve the problem of scarce labeled data by using source domain data to improve the performance of the target domain. However, when there is a significant distribution difference between the source and target domains, it is often difficult to achieve an ideal transfer effect, especially when the spectral features or spatial resolution are significantly different, the performance of the transfer learning model will be greatly limited. Therefore, how to effectively alleviate the distribution difference between domains and improve the generalization ability of the model is still an important challenge in the current cross-domain hyperspectral image classification research.

[0005] In order to overcome the above problems, the present invention proposes a cross-domain hyperspectral image crop fine classification method based on feature alignment. The method first inputs a small amount of labeled target domain data and sufficient labeled source domain data, and uses the embedded model to extract feature information. Subsequently, asymmetric convolution is introduced to flexibly capture the edges and contours of crops by designing convolution kernels in different directions to ensure the integrity of boundary information at different scales. At the same time, a conditional adversarial domain adaptation strategy is introduced to achieve the alignment of the feature distribution of the source domain and the target domain, thereby overcoming the problem of spectral offset and improving the robustness of cross-domain classification. In addition, in order to further improve the stability of the model and its adaptability to changes in feature distribution, this method uses sharpness-aware minimization to optimize the smoothing parameters to reduce the uncertainty caused by spectral offset. In the classification stage, the embedded features are classified using the KNN classifier to finally obtain the crop category. Summary of the invention

[0006] This paper proposes a cross-domain hyperspectral image crop fine classification method based on feature alignment, which combines the feature extraction ability of the embedded model, the edge capture advantage of asymmetric convolution, the distribution alignment effect of the conditional adversarial domain adaptation strategy, and the stability improvement of sharpness-aware minimization and smooth optimization. The method first extracts the feature information of the target domain and source domain data through the embedded model, and uses asymmetric convolution to flexibly capture the edges and contours of crops to ensure that the complete boundary information is maintained at different scales. Then, combined with the conditional adversarial domain adaptation strategy, the source domain and target domain feature distributions are aligned, thereby effectively alleviating the spectrum offset problem and improving the robustness of cross-domain classification. Finally, through sharpness-aware minimization and smooth optimization, the uncertainty caused by spectrum offset is reduced, and the stability of the model and its adaptability to feature distribution changes are improved. In the classification stage, the KNN classifier is used to accurately classify the embedded features, and finally the fine prediction of crop categories is achieved. This method shows excellent performance in the cross-domain hyperspectral image classification task, especially under the conditions of data scarcity and spectrum offset. It has important technical advantages, provides an innovative solution to solve the cross-domain problem in hyperspectral image classification, and has significant scientific research value and practical application prospects.

[0007] The object of the present invention is achieved in that:

[0008] The cross-domain hyperspectral image crop fine classification method based on feature alignment includes the following steps:

[0009] Step a: Input the target domain data and source domain data into the model respectively, where the target domain data contains a small number of labeled samples and the source domain data contains sufficient labeled samples. The target domain data is used to construct D t , the source domain data is used to construct D s The target domain samples include a small number of labeled samples and a large number of unlabeled samples satisfy

[0010] Step b: To reduce D s and D t The feature distribution difference between them is obtained by using the mapping layer M s and M t Right D s and D t The data are transformed into feature dimensions respectively, and the data are mapped to a unified feature space to generate a feature vector X′ s and X′ t The specific formula is as follows:

[0011] X′ s =M s (X s ),X′ t =M t (X t )

[0012] Among them, X s and X t are the input features of the source domain and the target domain, with dimensions B s ×H×W and B t ×H×W,M s and M t The mapping layer of the source domain and the target domain is responsible for the dimension conversion of the features, and the number of bands B in the source domain and the target domain is s and B t Mapped to the target dimension d; X′ s and X′ t They are the alignment features of the source domain and the target domain, and their dimensions are unified as d×H×W;

[0013] Step c: Use the embedding model f embed Extract the target domain D t and source domain D s The feature information of the data is mapped to a high-dimensional space to generate an embedded feature F(x) = f embed (M(x)), further introduces asymmetric convolution, and comprehensively extracts feature information by flexibly designing convolution kernels in different directions; this method ensures the integrity and effectiveness of boundary information at different scales by capturing the edges and contours of crops; the convolution kernel structure of asymmetric convolution is tailored according to the characteristics of the input data to achieve accurate description of boundary details and local features, thereby improving the classification performance and generalization ability of the model;

[0014] Step d: Use the domain adversarial loss function to align the distribution differences of the embedded features f = F(x) to reduce the spectral differences between different domains. The domain adversarial loss function is as follows:

[0015]

[0016] Where D represents the discriminator, are the embedded features of samples in the source domain and the target domain respectively, where h = (f, g) joint variable, g is the category information predicted by the discriminator D, and T is the multilinear dimensional transformation.

[0017] Step e: In order to further increase feature alignment, a sharpness-aware minimization strategy is introduced. By optimizing the model parameters, the loss function is made smoother and the model's robustness to changes in feature distribution is improved, thereby effectively reducing the classification performance fluctuations caused by spectral shift. By considering the most challenging perturbation direction at each parameter update, the model's performance in the worst case is optimized. By optimizing the smoothing parameters, the model can maintain stable classification results when the feature distribution changes slightly. This strategy not only enhances the generalization ability of the model, but also reduces the misclassification caused by spectral shift, thereby better adapting to the feature differences between the source and target domains.

[0018] Step f: classify the target domain samples through the nearest neighbor classifier to obtain the final crop classification result.

[0019] The above-mentioned cross-domain hyperspectral image crop fine classification method based on feature alignment is characterized in that step a specifically includes the following steps:

[0020] Step a1: Input source domain dataset D S , in is the hyperspectral image data of the i-th sample, is the category label of the sample.

[0021] Step a2: Input the target domain dataset D t , in is the hyperspectral data of the i-th sample, is the category label of the target domain. Target domain dataset D t Including a small amount of labeled data and a large amount of unlabeled data in

[0022] The above-mentioned cross-domain hyperspectral image crop fine classification method based on feature alignment is characterized in that step b specifically includes the following steps:

[0023] Step b1: To ensure that the feature dimensions of the source domain and the target domain are consistent, a mapping layer M is used. s and M tThe hyperspectral data of the source and target domains are converted into a unified dimension. The data processed by the mapping layer is represented as:

[0024] X′ s =M s (X s ),X′ t =M t (X t )

[0025] Among them, M s and M t They are the mapping layers for the source domain and the target domain, responsible for the dimension conversion of features. and are the input features of the source domain and the target domain, X′ s and X′ t It is the data processed by the mapping layer.

[0026] Step b2: The dimension of the mapped data is X′ s ∈R H×W×d and X′ t ∈R H×W×d , where W and H are the width and height of the image, respectively, and d is the feature dimension. The mapping layer is used to align the feature dimensions of the source domain and the target domain data, and to convert the number of bands in the source domain and the target domain into s and B t Mapped to the target dimension d, reducing the difference in feature distribution.

[0027] The above-mentioned cross-domain hyperspectral image crop fine classification method based on feature alignment is characterized in that step c specifically includes the following steps:

[0028] Step c1, build a feature extractor. The network architecture consists of multiple convolutional layers, combining 3D convolution and asymmetric convolution operations to efficiently extract features from the input image. Asymmetric convolution can accurately capture the detailed features of the image, especially the edge and contour information, by flexibly designing convolution kernels of different directions and sizes. Combining asymmetric convolution with residual connection further enhances the model's ability to learn the difference between input and output, while ensuring the integrity of boundary information at different scales. With the support of asymmetric convolution, the embedded features of the source domain and the target domain can fully characterize the details and boundary characteristics of the image. These embedded features not only significantly improve the accuracy of image representation, but also provide stronger support for subsequent classification tasks. The specific process is as follows:

[0029] F s =f embed (M s (X s )),F t =f embed (Mt (X t ))

[0030] Among them, f embed represents the feature extractor, M s and M t are the mapping layers of the source domain and the target domain respectively, X s and X t It is the source domain data.

[0031] y=f(x)+x

[0032] Where f(x) is the feedforward operation of the network and x is the input feature. Adding residual connections to the input x makes it easier for the network to learn complex mappings. In convolutional neural networks, residual connections pass inputs directly to subsequent layers through "skip connections" to help avoid gradient vanishing and accelerate training.

[0033] Step c2: Introduce asymmetric convolution in residual connection. Asymmetric convolution uses multiple convolution kernels in different directions to extract features, which can accurately capture the details of crops, especially edges and contours. The asymmetric convolution formula is:

[0034]

[0035] Among them, x is the input feature, w is the convolution kernel, * represents the convolution operation, and y is the output feature after convolution. By designing convolution kernels w1, w2, ..., w n , perform convolution operations on different scales to obtain multiple feature maps y1,y2,...,y n ,Then these feature maps are fused to enhance the ability to capture edge information. With the help of asymmetric convolution, the embedded features of the source and target domains can effectively capture the details in the image, especially the edge and contour information. These embedded features not only improve the accuracy of image representation, but also provide stronger support for subsequent classification tasks.

[0036] The above-mentioned cross-domain hyperspectral image crop fine classification method based on feature alignment is characterized in that in the step d, the distribution difference between the source domain and the target domain is reduced by using the domain adversarial loss function, so that the features of the source domain can be effectively migrated to the target domain. The domain adversarial loss function is as follows:

[0037]

[0038] Where D represents the discriminator, are the embedded features of samples in the source domain and the target domain, respectively, where h = (f, g) joint variable, g is the category information predicted by the discriminator D, indicating the probability that the sample belongs to the source domain. By minimizing the distribution difference between the source domain and the target domain, the target domain can share the features of the source domain, thereby achieving cross-domain classification tasks. T is a multi-linear dimensional transformation.

[0039] The above-mentioned cross-domain hyperspectral image crop fine classification method based on feature alignment is characterized in that step e specifically includes the following steps:

[0040] The core idea of ​​SAM is to find the perturbation direction that is most challenging for model training (i.e. the direction in which the loss function is steepest) in each step of gradient descent, and then further optimize the model in this direction. The worst-case loss value is approached by adding an additional perturbation term ∈:

[0041]

[0042] Where: θ is the model parameter; ∈ is the disturbance term, which is limited by the radius ρ, and L is the loss function.

[0043] The SAM strategy corrects the parameter update direction of the model so that the updated model parameters can avoid overfitting to the sharp areas of local features, thereby obtaining a smoother feature distribution. The specific process is as follows:

[0044] First calculate the perturbation direction of the current gradient After adjusting the parameter θ in the perturbation direction, the loss L(θ+∈) is recalculated, and the model parameters are finally updated to make it more robust to changes in feature distribution. Through the above optimization, SAM can effectively reduce the model's sensitivity to changes in the target domain spectrum, so that it can still maintain a high classification accuracy when the spectral characteristics of the source domain and the target domain are greatly different.

[0045] The above-mentioned cross-domain hyperspectral image crop fine classification method based on feature alignment is characterized in that, in the step f, a nearest neighbor classifier is used to classify the samples in the target domain. The classifier calculates the distance between each target sample and the support set sample, selects the nearest sample for classification, and obtains the final crop category.

[0046] Beneficial effects:

[0047] The present invention proposes a method for fine classification of crops in cross-domain hyperspectral images based on feature alignment, which belongs to the field of image processing technology. First, target domain data with a small amount of labels and source domain data with sufficient labels are input, and feature information is extracted using an embedding model. Secondly, asymmetric convolution is introduced to flexibly adapt to feature extraction in different directions, and the edges and contours of crops are accurately captured through convolution kernels in different directions to ensure that boundary information at different scales is retained. Subsequently, the distribution alignment of the source domain and the target domain is achieved through the conditional adversarial domain adaptation strategy to overcome the spectrum offset. In addition, the sharpness-aware minimization smoothing parameter optimization is adopted to make the model insensitive to changes in feature distribution and reduce the fluctuations caused by spectrum offset. Finally, the KNN classifier is used for classification to obtain the crop category. Experimental results show that the classification accuracy of the method on the Indian Pines dataset is better than that of the existing methods, providing a new idea for cross-domain hyperspectral image crop classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is the overall flow chart of the cross-domain hyperspectral image crop fine classification method based on feature alignment in the method of the present invention.

[0049] Figure 2 It is the pseudo color image of the WHU-Hi-HanChuan dataset in the method of the present invention and its corresponding ground truth map.

[0050] Figure 3 It is a pseudo-color image of the Indian Pines dataset and its corresponding ground truth map in the method of the present invention.

[0051] Figure 4 It is a schematic diagram of the principle of the feature extractor in the method of the present invention.

[0052] Figure 5 This is a diagram of the classification results of the Indian Pines dataset in the method of the present invention.

[0053] Figure 6 It is an accompanying drawing of the abstract of the method of the present invention. DETAILED DESCRIPTION

[0054] The specific implementation modes of the present invention are further described in detail below with reference to the accompanying drawings.

[0055] The cross-domain hyperspectral image crop fine classification method based on feature alignment according to the specific implementation mode of the present invention is as follows: Figure 1 As shown, the following steps are included:

[0056] Step a: Input the hyperspectral datasets of the source domain and the target domain respectively, and define the source domain dataset as D s , the target domain dataset is D t, randomly select training samples and construct training sets and test sets:

[0057] In a specific embodiment of the present invention, the WHU-Hi-HanChuan dataset is used as the source domain data. The WHU-Hi-HanChuan dataset was collected in Hanchuan City, Hubei Province, China on June 17, 2016, using a 17mm focal length Headwall Nano-Hyperspec imaging sensor installed on the Leica AibotX6 UAV V1 platform. The study area is located at the junction of urban and rural areas, covering buildings, water bodies and cultivated land, and contains seven crops: strawberry, cowpea, soybean, sorghum, water spinach, watermelon and green leafy vegetables. The flight altitude of the drone is 250m, the image size is 1217×303 pixels, the number of bands is 274, the wavelength range is 400nm~1000nm, and the spatial resolution is about 0.109m. It is worth noting that since the dataset was collected in the afternoon when the solar altitude angle is low, there are a lot of shadow areas in the image. Table 1 shows the categories and number of samples of each category in the WHU-Hi-HanChuan dataset. Figure 2 Pseudo-color images and their corresponding ground truth maps are shown.

[0058] Table 1. WHU-Hi-HanChuan dataset

[0059] serial number category Number of samples serial number category Number of samples C1 Strawberry 44735 C9 Grass 9469 C2 Cowpea 22753 C10 Red roof 10516 C3 Soybean 10287 C11 Gray roof 16911 C4 Sorghum 5353 C12 Plastic 3679 C5 Water spinach 1200 C13 Bare soil 9116 C6 Watermelon 4533 C14 Road 18560 C7 Greens 5903 C15 Bright object 1136 C8 Trees 17978 C16 Water 75401

[0060] The Indian Pines dataset is used as the target domain dataset. The Indian Pines dataset was collected by AVIRIS in northwest Indiana, USA in 1992. The dataset contains 200 bands with a wavelength range of 400 to 2500 nanometers, a spatial resolution of 20 meters per pixel, and an image size of 145×145 pixels. The dataset includes 16 vegetation categories. Table 2 shows the category and number of samples in each category of the Indian Pines dataset. Figure 3 Pseudo-color images and their corresponding ground truth maps are shown.

[0061] Table 2. Indian Pines dataset

[0062] serial number category Number of samples serial number category Number of samples C1 Alfalfa 46 C9 Oats 20 C2 Corn-notill 1428 C10 Sovbean-notill 972 C3 Corn-mintill 830 C11 Soybean-mintill 2455 C4 Corn 237 C12 Soybean-cleam 593 C5 Grass-pasture 483 C13 Wheat 205 C6 Grass-tree 730 C14 Woods 1265 C7 Grass-pasture-mowed 28 C15 Buildings-Grass-Trees-Drives 386 C8 Hay-windrowed 478 C16 Stone-Steel-Towers 93

[0063] Step b: By inputting the data output from step a into a mapping model composed of a 2D convolutional layer and a batch normalization layer, the spectral dimensions of data in different domains are aligned, effectively reducing the impact of dimensional differences, thereby achieving cross-domain classification.

[0064] The specific implementation is:

[0065] The mapping network consists of two main parts: convolutional layer and batch normalization layer. The main task of the input layer is to receive input data and pass it to the subsequent convolutional layer. Through the convolutional layer, the feature dimension of the input data is converted and the number of channels is converted to 100. Then, the hidden layer normalizes the convolution result through the batch normalization operation, which aims to speed up the training process and improve the stability of the model. The batch normalization layer standardizes the data to eliminate the inconsistency caused by different data features. After these processes, the data will be passed to the output layer, and finally the processed feature map will be output.

[0066] Step c: The present invention constructs a feature extraction network structure diagram as shown in Figure 4 As shown, by inputting the data output in step b into the feature extractor, representative feature information is extracted from the data input b through a multi-level processing structure to generate an embedded feature F(x), where F(x) = f embed (M(x)). The feature extraction network combines asymmetric convolution and residual connection to effectively extract key features in the source and target domain images, enhance the model's perception of edges, contours, and regional details, and reduce information dislocation caused by spatial structure and resolution differences, providing high-quality feature representation for subsequent classification tasks.

[0067] The specific implementation is:

[0068] First, the input data x is passed through the first residual block for preliminary feature extraction. In each residual block, the feature map after the convolution operation is added to the input x through a jump connection to form the output. The specific operation is as follows:

[0069] y1 = ReLU (Conv3 × 3 × 3 (x) + x)

[0070] Among them, Conv3x3x3(x) represents a standard 3D convolution operation, x is the input feature, and y1 is the output of the residual block.

[0071] Next, the detail features are further extracted through asymmetric convolution. Asymmetric convolution uses multi-directional convolution kernels to process feature maps to capture detail information such as edges and contours in the image. The asymmetric convolution operation used here is:

[0072] y2=ReLU(AsymmetricConv3×3×3(y1))

[0073] Among them, AsymmetricConv3×3×3 is an asymmetric convolution kernel that performs convolution operations in different directions. In this way, convolution kernels of different directions and sizes can be flexibly designed to further capture more complex features.

[0074] The asymmetric convolution feature map y2 is further processed by a standard 3D convolution operation Conv3x3x3 to obtain a new feature map y3:

[0075] y3=Conv3×3×3(y2)

[0076] Then, use the residual connection to fuse y1 and y3 to get the final output feature map y4:

[0077] y4=ReLU(y1+y3)

[0078] This residual connection helps the model learn complex mapping relationships better while avoiding gradient vanishing. Finally, after multiple residual blocks and asymmetric convolution processing, the final output feature vector x of the network is obtained. out :

[0079] x out =Flatten(y4)

[0080] Step d: The embedded feature f = F(x) generated in step c is used to reduce the distribution difference between the source domain and the target domain through the domain adversarial loss function, so that the features of the source domain can be effectively migrated to the target domain.

[0081] The specific implementation is:

[0082] The source domain samples and target domain samples are respectively passed through the feature extractor T to generate corresponding feature representations and The discriminator D is responsible for distinguishing whether these features belong to the source domain or the target domain, where represents the probability that the feature comes from the source domain, represents the probability that the feature comes from the target domain. By minimizing L d , the feature extractor T is optimized to generate domain-independent feature representations; at the same time, by maximizing L d , the discriminator D is optimized to enhance the ability to distinguish. In this adversarial training process, the feature distributions of the source domain and the target domain are finally aligned, thereby improving the robustness of the cross-domain classification task. The adversarial domain loss function is as follows:

[0083]

[0084] Where D represents the discriminator, are the embedded features of samples in the source domain and the target domain, respectively, where h = (f, g) joint variable, g is the category information predicted by the discriminator D, indicating the probability that the sample belongs to the source domain, by minimizing the distribution difference between the source domain and the target domain.

[0085] In step e, feature alignment is further added and a sharpness-aware minimization strategy is introduced. By optimizing the model parameters, the loss function is made smoother and the model’s robustness to changes in feature distribution is improved, thereby effectively reducing the classification performance fluctuations caused by spectral shift.

[0086] The specific implementation is:

[0087] The core idea of ​​SAM is to find the perturbation direction that is most challenging for model training (i.e. the direction in which the loss function is steepest) in each step of gradient descent, and then further optimize the model in this direction. The worst-case loss value is approached by adding an additional perturbation term ∈:

[0088]

[0089] Where: θ is the model parameter; ∈ is the disturbance term, which is limited by the radius ρ, and L is the loss function.

[0090] The SAM strategy corrects the parameter update direction of the model so that the updated model parameters can avoid overfitting to the sharp areas of local features, thereby obtaining a smoother feature distribution. The specific process is as follows:

[0091] First calculate the perturbation direction of the current gradient After adjusting the parameter θ in the perturbation direction, the loss L(θ+∈) is recalculated, and the model parameters are finally updated to make it more robust to changes in feature distribution. Through the above optimization, SAM can effectively reduce the model's sensitivity to changes in the target domain spectrum, so that it can still maintain a high classification accuracy when the spectral characteristics of the source domain and the target domain are greatly different.

[0092] Step f: Input the data with reduced spectrum differences between different domains in step e into the classifier, and classify the target domain samples using the nearest neighbor classifier. The classifier calculates the distance between each target sample and the support set sample, selects the sample with the closest distance for classification, and obtains the final crop category.

[0093] The specific implementation is:

[0094] The classification accuracy of the Indian Pines dataset used in the present invention is shown in Table 3. The overall accuracy OA, average accuracy AA and Kappa coefficient are used as classification evaluation indicators. The classification results of the Indian Pines dataset are shown in Table 3. Figure 5 The experimental results show that the invention has only a small amount of misclassification, is close to the actual crop distribution, and greatly reduces the area of ​​misclassification. In addition, Figure 6This is an abstract of the method of the present invention, which is intended to summarize and demonstrate the key steps and processes of the present invention. The figure concisely presents the relationship between the various modules and their working principles, and each link from data input to final output is effectively organized and described.

[0095] The experimental environment of the present invention is Intel (R) Xeon (R) CPU E5-2620 v4 @ 2.10GHz processor, 128GB memory and NVIDIA GeForce RTX 2080Ti GPU graphics card. In addition, the deep learning framework is Pytorch, which uses Python as the programming language. The Adam optimization algorithm is used for optimization, and the number of iterations is set to 10000. Other methods are based on the parameters set by the authors of the paper. At the same time, in order to reduce the randomness brought by the training samples, each test is repeated 10 times and the average value is taken as the final test result. In order to verify the effectiveness of the present invention, the feature alignment based cross-domain learning (FeatureAlignment based Cross-Domain Learning, FABCDL) of the present invention is compared with the extreme gradient boosting algorithm (eXtremeGradient Boosting, XGBoost), support vector machine (Support Vector Machine, SVM), and cross-domain small sample learning (Deep Cross-Domain Few-Shot Learning, DCFSL), and the results are shown in Table 3.

[0096] Table 3. Classification accuracy of classification methods on the Indian Pines dataset

[0097]

[0098]

Claims

1. A cross-domain hyperspectral image crop fine classification method based on feature alignment, characterized by: The following steps are involved: Step a: Input the target domain data and source domain data into the model respectively, where the target domain data contains a small number of labeled samples and the source domain data contains sufficient labeled samples. The target domain data is used to construct D t , the source domain data is used to construct D s The target domain samples include a small number of labeled samples and a large number of unlabeled samples satisfy Step b: To reduce D s and D t The feature distribution difference between them is obtained by using the mapping layer M s and M t Right D s and D t The data are transformed into feature dimensions respectively, and the data are mapped to a unified feature space to generate a feature vector X′ s and X′ t The specific formula is as follows: X′ s =M s (X s ),X′ t =M t (X t ) Among them, X s and X t are the input features of the source domain and the target domain, with dimensions B s ×H×W and B t ×H×W,M s and M t The mapping layer of the source domain and the target domain is responsible for the dimension conversion of the features, and the number of bands B in the source domain and the target domain is s and B t Mapped to the target dimension d; X′ s and X′ t They are the alignment features of the source domain and the target domain, and their dimensions are unified as d×H×W; Step c: Use the embedding model f embed Extract the target domain D t and source domain D s The feature information of the data is mapped to a high-dimensional space to generate an embedded feature F(x) = f embed (M(x)), further introduces asymmetric convolution, and comprehensively extracts feature information by flexibly designing convolution kernels in different directions; this method ensures the integrity and effectiveness of boundary information at different scales by capturing the edges and contours of crops; the convolution kernel structure of asymmetric convolution is tailored according to the characteristics of the input data to achieve accurate description of boundary details and local features, thereby improving the classification performance and generalization ability of the model; Step d: Use the domain adversarial loss function to align the distribution differences of the embedded features f = F(x) to reduce the spectral differences between different domains. The domain adversarial loss function is as follows: Where D represents the discriminator, are the embedded features of samples in the source domain and the target domain respectively, where h = (f, g) joint variable, g is the category information predicted by the discriminator D, and T is the multilinear dimensional transformation. Step e: In order to further increase feature alignment, a sharpness-aware minimization strategy is introduced. By optimizing the model parameters, the loss function is made smoother and the model's robustness to changes in feature distribution is improved, thereby effectively reducing the classification performance fluctuations caused by spectral shift. By considering the most challenging perturbation direction at each parameter update, the model's performance in the worst case is optimized. By optimizing the smoothing parameters, the model can maintain stable classification results when the feature distribution changes slightly. This strategy not only enhances the generalization ability of the model, but also reduces the misclassification caused by spectral shift, thereby better adapting to the feature differences between the source and target domains. Step f: classify the target domain samples using the K-nearest neighbor (KNN) algorithm to obtain the final crop classification results.

2. The cross-domain hyperspectral image crop fine classification method based on feature alignment according to claim 1 is characterized in that: The step a specifically comprises the following steps: Step a1: Input source domain dataset D S , in is the hyperspectral image data of the i-th sample, is the category label of the sample. Step a2: Input the target domain dataset D t , in is the hyperspectral data of the i-th sample, is the category label of the target domain. Target domain dataset D t Including a small amount of labeled data and a large amount of unlabeled data in 3. The cross-domain hyperspectral image crop fine classification method based on feature alignment according to claim 1 is characterized in that: The step b specifically comprises the following steps: Step b1: To ensure that the feature dimensions of the source domain and the target domain are consistent, a mapping layer M is used. s and M t The hyperspectral data of the source and target domains are converted into a unified dimension. The data processed by the mapping layer is represented as: X′ s =M s (X s ),X′ t =M t (X t ) Among them, M s and M t They are the mapping layers for the source domain and the target domain, responsible for the dimension conversion of features. and are the input features of the source domain and the target domain, X′ s and X′ t It is the data processed by the mapping layer. Step b2: The dimension of the mapped data is X′ s ∈R H×W×d and X′ t ∈R H×W×d , where W and H are the width and height of the image, respectively, and d is the feature dimension. The mapping layer is used to align the feature dimensions of the source domain and the target domain data, and to convert the number of bands in the source domain and the target domain into s and B t Mapped to the target dimension d, reducing the difference in feature distribution.

4. The method for fine crop classification based on cross-domain hyperspectral images based on feature alignment according to claim 1 is characterized in that: The step c is specifically as follows: Step c1, build a feature extractor. The network architecture consists of multiple convolutional layers, combining 3D convolution and asymmetric convolution operations to efficiently extract features from the input image. Asymmetric convolution can accurately capture the detailed features of the image, especially the edge and contour information, by flexibly designing convolution kernels of different directions and sizes. Combining asymmetric convolution with residual connection further enhances the model's ability to learn the difference between input and output, while ensuring the integrity of boundary information at different scales. With the support of asymmetric convolution, the embedded features of the source domain and the target domain can fully characterize the details and boundary characteristics of the image. These embedded features not only significantly improve the accuracy of image representation, but also provide stronger support for subsequent classification tasks. The specific process is as follows: F s =f embed (M s (X s )),F t =f embed (M t (X t )) Among them, f embed represents the feature extractor, M s and M t are the mapping layers of the source domain and the target domain respectively, X s and X t It is the source domain data. y=f(x)+x Where f(x) is the feedforward operation of the network and x is the input feature. Adding residual connections to the input x makes it easier for the network to learn complex mappings. In convolutional neural networks, residual connections pass inputs directly to subsequent layers through "skip connections" to help avoid gradient vanishing and accelerate training. Step c2: Introduce asymmetric convolution in residual connection. Asymmetric convolution uses multiple convolution kernels in different directions to extract features, which can accurately capture the details of crops, especially edges and contours. The asymmetric convolution formula is: Among them, x is the input feature, w i is the convolution kernel, * represents the convolution operation, and y is the output feature after convolution. By designing convolution kernels w1, w2, …, w n , perform convolution operations on different scales to obtain multiple feature maps y1,y2,…,y n ,Then these feature maps are fused to enhance the ability to capture edge information. With the help of asymmetric convolution, the embedded features of the source and target domains can effectively capture the details in the image, especially the edge and contour information. These embedded features not only improve the accuracy of image representation, but also provide stronger support for subsequent classification tasks.

5. The method for fine crop classification based on cross-domain hyperspectral images based on feature alignment according to claim 1 is characterized in that: In step d, the distribution difference between the source domain and the target domain is reduced by using the domain adversarial loss function, so that the features of the source domain can be effectively migrated to the target domain. The domain adversarial loss function is as follows: Where D represents the discriminator, are the embedded features of samples in the source domain and the target domain, respectively, where h = (f, g) joint variable, g is the category information predicted by the discriminator D, and by minimizing the distribution difference between the source domain and the target domain, the target domain can share the features of the source domain, thereby achieving cross-domain classification tasks. T is the dimension transformation.

6. The method for fine crop classification based on cross-domain hyperspectral images based on feature alignment according to claim 1 is characterized in that: In step e, in the cross-domain hyperspectral image crop classification, due to the spectrum offset problem between the source domain and the target domain, the feature distribution may change significantly, resulting in unstable model classification performance. To solve this problem, the present invention introduces a sharpness aware minimization (SAM) strategy, which improves the robustness of the model to feature distribution changes by optimizing smoothing parameters, thereby reducing the performance fluctuation caused by spectrum offset. Step e specifically includes the following steps: The core idea of ​​SAM is to find the perturbation direction that is most challenging for model training in each step of gradient descent, and then further optimize the model in this direction. The worst-case loss value is approximated by adding additional perturbation terms ∈: Where: θ is the model parameter; ∈ is the disturbance term, which is limited by the radius ρ, and L is the loss function. The SAM strategy corrects the parameter update direction of the model so that the updated model parameters can avoid overfitting to the sharp areas of local features, thereby obtaining a smoother feature distribution. The specific process is as follows: First calculate the perturbation direction of the current gradient After adjusting the parameter θ in the perturbation direction, the loss L(θ+∈) is recalculated, and the model parameters are finally updated to make it more robust to changes in feature distribution. Through the above optimization, SAM can effectively reduce the model's sensitivity to changes in the target domain spectrum, so that it can still maintain a high classification accuracy when the spectral characteristics of the source domain and the target domain are greatly different.

7. The method for fine crop classification based on cross-domain hyperspectral images based on feature alignment according to claim 1 is characterized in that: In step f, the samples in the target domain are classified using a nearest neighbor classifier, which calculates the distance between each target sample and the support set sample, selects the nearest sample for classification, and obtains the final crop category.

Images