Remote sensing image crop classification method based on space-spectrum joint enhancement contrast learning

Abstract

The invention discloses a remote sensing image crop classification method based on space-spectrum joint enhancement contrast learning. According to the method, a three-dimensional space-time spectrum feature encoder is designed to perform preliminary extraction of crop features. Secondly, extracting spatial and spectral features of the remote sensing image in a multi-view joint contrast learning mode based on the features of the remote sensing image, and introducing an optimal index method into a spectrum enhancement module to perform feature optimization from the multi-spectral image to construct a spectrum enhancement view so as to improve the utilization efficiency of spectral information; the spatial enhancement module utilizes the spatial distribution relationship of crops and combines spectral similarity modeling to construct a spatial enhancement view for enhancing feature association between adjacent farmland samples. Compared with an existing most advanced self-supervised comparative learning method, the method has the advantages that effective spatial-spectral joint semantic features can be mined from unlabeled remote sensing data, the performance of the model in crop classification tasks is improved, and the method has good application value.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image classification, and in particular to a remote sensing image crop classification method based on spatial-spectral joint enhanced contrast learning. Background Art

[0002] In recent years, with the continuous advancement of satellite remote sensing technology, high-resolution remote sensing images have provided a rich data foundation for accurately extracting crop information. By using remote sensing image data to monitor and accurately classify different crops in farmland, it is helpful to gain a deeper understanding of agricultural production structure, crop distribution and planting conditions, which is of great significance for realizing precision agriculture and promoting agricultural modernization. Deep learning models have been widely used in the field of remote sensing intelligent interpretation due to their advantages in automatically extracting deep features of images. However, due to the difficulties in remote sensing data labeling, complex crop growth and planting structures, and large differences in crop characteristics in different regions in the field of crop classification, existing deep learning methods are difficult to adapt to the needs of crop classification in different regions and spatiotemporal conditions. Therefore, how to realize automatic classification of crops without relying on label data based on the spatiotemporal spectral characteristics of remote sensing data and the growth and planting laws of crops has become a research hotspot.

[0003] So far, researchers at home and abroad have conducted extensive research on crop classification based on high-resolution remote sensing images. The initial algorithms are usually based on the temporal and spectral characteristics of remote sensing images, using vegetation indices or spectral reflectance to identify and classify crops. For example, the literature: Kang Y, Hu X, Meng Q, et al. Land cover and crop classification based on red edge indices features of GF-6WFV time series data [J]. Remote Sensing, 2021, 13 (22): 4522. proves the above point. However, this type of method relies on the researchers' manual design and manipulation of spectral features. It is limited by human subjective experience and domain knowledge, and it is difficult to fully explore the potential information of large-scale, high-dimensional remote sensing data, thus affecting the classification accuracy. For example, the literature: Luo C, Meng S, Hu X, et al. Cropnet: Deep Spatial-Temporal-Spectral Feature Learning Network for Crop Classification from Time-Series Multi-Spectral Images[C]. IEEE International Geoscience and Remote Sensing Symposium(IGARSS). Waikoloa, HI, USA: IEEE, 2020: 4187~4190. proves this statement.In recent years, intelligent computing technology has developed rapidly, and deep learning technology represented by convolutional neural network (CNN) has attracted much attention. It can adaptively learn high-level semantic features from data, and has made great progress in the field of remote sensing intelligent interpretation with its powerful recognition and feature extraction capabilities. The literature: Khan AH, Fraz MM, Shahzad M. Deep Learning Based LandCover and Crop Type Classification: A Comparative Study [C]. 2021 International Conference on Digital Futures and Transformative Technologies (ICoDT2). Islamabad, Pakistan: IEEE, 2021: 1~6. and the literature: Wang L, Bai Y, Wang J, et al. Histogram matching-based semantic segmentation model for crop classification with Sentinel-2 satellite imagery [J]. GIScience & Remote Sensing, 2023, 60: 1. proves the above statement. However, most deep learning models use supervised training to automatically learn features from large-scale remote sensing data. However, most remote sensing image data cannot be automatically labeled accurately, and the cost of manual labeling is high, which greatly limits the application of deep learning methods in real crop classification scenarios. Recent studies have attempted to use self-supervised contrastive learning to automatically mine spectral and spatial discriminative features from massive unlabeled remote sensing data.

[0004] In the spectral dimension, the bands of remote sensing images are usually randomly grouped to construct two spectral views as positive sample pairs for contrastive learning. The above view is proved by the literature: Xue Z, Liu B, Yu A, et al. Self-Supervised Feature Representation and Few-Shot Land Cover Classification of Multimodal Remote Sensing Images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1-18. and the literature: Liu B, Yu A, Yu X, et al. Deep multiview learning for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 59(9): 7758-7772. However, the growth of crops is accompanied by complex spectral feature changes. Different crop types have different sensitivities to different bands. The existing spectral contrast learning method ignores the changes in crop response to different bands and cannot make full use of the spectral information of crops. The detailed and structural information brought by the gradual improvement of spatial resolution has not been fully considered in practical applications, making it difficult to achieve accurate land use classification.

[0005] In the spatial dimension, contrastive learning methods are usually designed based on the geographic spatial relationship of remote sensing images. By defining the spatial neighborhood as the anchor sample, adjacent samples and distant samples are selected to mine the consistency information of adjacent samples in spatial distribution. References: Jean N, Wang S, Samar A, et al. Tile2Vec: Unsupervised representation learning for spatially distributed data [J]. arXiv, 2018. and Kang J, Fernandez-Beltran R, Duan P, et al. Deep Unsupervised Embedding for Remotely Sensed Images Based on Spatially Augmented Momentum Contrast [J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59 (3): 2598-2610. prove the above point. However, relying solely on the correlation between adjacent samples to constrain the geographic distance, it is assumed that the characteristics of samples with close distances must be similar, which may mistakenly bring the characteristics of different crop types closer and reduce the robustness of the model.

[0006] In view of the above problems, the present invention proposes a remote sensing image crop classification method based on spatial-spectral joint enhanced contrast learning to solve the above technical problems. Summary of the invention

[0007] The purpose of the present invention is to provide a remote sensing image crop classification method based on spatial-spectral joint enhanced contrast learning to solve the problem that the existing contrast learning method cannot fully explore the spatial and spectral features of crops. First, a three-dimensional spatial-temporal-spectral feature extraction module (3D Spatial-Spectral-Temporal Feature Extraction Module, 3D SSTFEM) is designed. Secondly, a spectral augmentation module based on optimal band feature selection (Spectral Augment Module based on Optimal Band Feature Selection, SAMOBS) and a spatial augmentation module based on spectral similarity modeling (Spatial Augment Module based on Spectral Similarity Modelling, SAMSSM) are designed to respectively enhance the model's ability to extract spectral detail features and enhance the similarity between adjacent samples in geographic space. Finally, a spatial spectral feature fusion module (Spatial Spectral Feature Fusion Module, SSFFM) is designed to fuse the features extracted by different branch modules and flexibly learn the complementary relationship between spectral detail features and geographic spatial features. Compared with other latest crop classification methods, the remote sensing image crop classification method based on spatial-spectral joint enhanced contrast learning proposed in the present invention has improved accuracy, time complexity and generalization ability.

[0008] The object of the present invention is achieved in that: A remote sensing image crop classification method based on space-spectrum joint enhanced contrast learning, characterized in that: it includes a contrast learning model SSACL based on space-spectrum joint enhancement, the contrast learning model SSACL based on space-spectrum joint enhancement includes a three-dimensional time-space-spectrum feature extraction module 3D SSTFEM, a spectral enhancement module SAMOBS based on optimal band feature selection, a spatial enhancement module SAMSSM based on spectral similarity modeling and a space-spectrum feature fusion module SSFFM, the space-spectrum feature fusion module SSFFM performs weighted fusion of features of the spectral enhancement module SAMOBS and the spatial enhancement module SAMSSM through contrast loss; The remote sensing image crop classification method based on spatial-spectral joint enhanced contrast learning comprises the following steps: Step S1, designing a three-dimensional time-space-spectrum feature extraction module 3D SSTFEM: The three-dimensional space-time-spectral feature extraction module 3D SSTFEM is based on a three-dimensional convolutional neural network, which utilizes the spatial distribution and spectral characteristics of crops and captures the growth dynamic information of crops from the time dimension at the same time; the three-dimensional space-time-spectral feature extraction module 3D SSTFEM includes four blocks, each of which includes a convolution layer, a normalization layer and an activation function. A batch normalization BN layer is added after the convolution layer to speed up the model fitting speed and avoid overfitting. The ReLu activation function is used to effectively avoid gradient explosion or disappearance, making the network training faster; the space-time spectral features of remote sensing images are extracted simultaneously through three-dimensional convolution, which can retain the time change information in the growth process of crops while mining spatial and spectral features, and fully model the complex space-time spectral change characteristics in the growth process of crops; Step S2: Design a spectrum enhancement module SAMOBS based on optimal band feature selection: The spectrum enhancement module SAMOBS based on optimal band feature selection designs a feature band selection method based on the OIF optimal index factor; the optimal band is selected by calculating the correlation and information content of each band in the multispectral image, and the feature map composed of the optimal bands is used as the spectral enhancement view of the original image to enhance the model's ability to extract spectral detail features; the bands of remote sensing data have high correlation. By reasonably selecting and combining the bands, extracting bands with high discrimination and sensitivity helps to highlight the characteristics of different objects and improve the classification performance of the model; Step S3: Design a spatial enhancement module SAMSSM based on spectral similarity modeling: The spatial enhancement module SAMSSM based on spectral similarity modeling uses spatial relationships to mine the consistent features between crops with similar geographical distribution. Based on multiple adjacent image samples, the spectral similarity matrix is ​​used to model the semantic relationship between adjacent samples. By dynamically adjusting the correlation between adjacent samples, the similarity between adjacent samples in the geographic space is enhanced while reducing the noise impact caused by the incorrect selection of the nearest neighbor samples. While making full use of the correlation between adjacent crop samples, the features of different crop types are avoided due to the incorrect aggregation, thereby improving the classification accuracy and robustness of crops with similar spatial distribution. Step S4, designing the spatial spectrum feature fusion module SSFFM: The spatial-spectral feature fusion module SSFFM designs a spatial-spectral joint contrast loss to fuse the features extracted by different branch modules in step S2 and step S3, and learn the complementary relationship between spectral detail features and geographic spatial features, which brings better model performance and higher accuracy for crop classification tasks; by dynamically learning the complementary relationship between spatial dimension and spectral dimension, it effectively promotes the fusion of spatial-spectral self-supervised features; Step S5: Method for verifying and comparing models using data sets: A comparative verification of the contrastive learning model SSACL based on spatial-spectral joint enhancement is performed to verify the outstanding performance of the proposed method in complex crop classification scenarios.

[0009] The specific steps of step S2 are: A spectral enhancement module SAMOBS based on optimal band feature selection is proposed to select and combine bands, extract bands with discrimination and sensitivity to highlight key crop features. For each sampled image, the standard deviation of each band and the correlation coefficient between bands are calculated: In the formula, S i represents the standard deviation of the ith band, R i，j is the correlation coefficient of the two bands i and j, w represents the width and height of the anchor tile, is the reflectivity of the ith band, is the mean reflectivity of the i-th band; The larger the standard deviation of the image data, the greater the amount of information it contains, and the smaller the correlation coefficient between bands, indicating that the image independence between bands is higher and the information redundancy is smaller; then, the OIF index of different band combinations is calculated based on the standard deviation and correlation coefficient: Where p is the number of band combinations; OIF can balance the information content and correlation of each spectral band, extract bands with large information content and low inter-band information redundancy, and thus determine the optimal characteristic band combination, which is used as the spectral enhancement view X in this paper. spe , and then use contrastive learning to bring the spectrally enhanced view group {x q , x spe} consistency characteristics.

[0010] The specific steps of step S3 are: The spatial enhancement module SAMSSM based on spectral similarity modeling is based on multiple adjacent image samples and uses the spectral similarity matrix to model the semantic relationship between adjacent samples. By dynamically adjusting the correlation between adjacent samples, the similarity between adjacent samples in the geographic space is enhanced while reducing the noise impact caused by the wrong selection of the nearest neighbor samples. First, a spatial neighborhood radius R is given, with the anchor image x q As the center, a set of n neighbor images are cropped out within its neighborhood The cosine similarity is used to evaluate the spectral similarity between the neighbor image and the anchor image in the neighborhood and establish the spectral similarity matrix: S irepresents the cosine similarity between the i-th neighbor image and the anchor image in the matrix, and selects a neighbor image with the maximum cosine similarity, i.e., the highest spatial semantic similarity, to the anchor image as the spatial enhancement view x spa , and then use contrastive learning to close the spatial enhancement view group {x q , x spa} consistency characteristics.

[0011] The specific steps of step S4 are: After the spatial-spectral enhancement transformation, the multi-view set {X q , X spe , X spa} is encoded by the feature extraction network f(·) into a representation {h q ,h spe ,h spa}; the nonlinear projection network g(·) projects each intermediate representation into two normalized embedding subspaces {Z1, Z2}, each of which is specialized for a single enhancement; the embedding subspace Z1 is used to mine spectral information, The opposite of Negative pairs are the embeddings of other samples in the embedding space and The embedding subspace Z2 is used to mine geospatial information. The opposite of Negative pairs are the embeddings of other samples in the embedding space and In order to make the gap between positive sample features smaller and the gap between negative sample features larger in each embedding subspace to achieve the purpose of contrastive learning, the contrastive loss functions L1 and L2 are constructed: Where L1 is the spectral contrast loss, L2 is the spatial contrast loss, N is the number of negative samples, represents the negative sample queue, is the temperature hyperparameter; In order to fuse the features of the two embedded subspaces for joint training, a contrast loss function L for spatial-spectral joint enhancement is proposed. total : L total =αL1+βL2 Where α and β are weight hyperparameters, which are used to balance the complementary relationship between spectral detail features and geographic spatial features.

[0012] The beneficial effects of the present invention are as follows: the present invention constructs a three-dimensional time-space-spectral feature extraction network, which aims to comprehensively mine the spatial and spectral features of remote sensing data while integrating temporal features. Then, a multi-view joint representation contrast learning method is constructed for the spectral and spatial domains of remote sensing data. In the spectral domain, an optimal feature selection strategy integrating the OIF index is designed, and the spectral bands are optimized according to the amount of information and correlation to enhance the model's ability to express key spectral features of crops; in the spatial domain, the spatial correlation of remote sensing data is utilized and combined with spectral similarity modeling to enhance the semantic relationship between adjacent samples under spatial constraints. Finally, in the spatial-spectral feature fusion module, the spatial-spectral joint contrast loss function is used to enable the model to flexibly learn the complementary relationship between spectral detail features and spatial semantic features. The proposed contrast learning method is applied to crop classification, and based on the experimental dataset CalCR O P21, Germany, and compared with the latest and most widely used contrastive learning method, the proposed method has the best overall performance in the task of high-resolution remote sensing image crop classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a structural diagram of the contrastive learning model SSACL based on spatial-spectral joint enhancement of the present invention; Figure 2 The CalCR of the present invention O Sample pixel statistics of the P21 dataset and the Germany dataset; Figure 3 and Figure 4 The present invention and other methods are respectively O P21, Comparison of visualization results of crop classification in Germany dataset; Figure 5 Result diagram of the influence of hyperparameters of the present invention on the performance of SSACL model, (a) the result diagram of the influence of loss weights α and β on the performance of the model; (b) the result diagram of the influence of neighborhood radius R on the performance of the model. DETAILED DESCRIPTION

[0014] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0015] like Figure 1As shown, a remote sensing image crop classification method based on spatial-spectral joint enhanced contrastive learning includes a contrastive learning model SSACL based on spatial-spectral joint enhancement. The contrastive learning model SSACL based on spatial-spectral joint enhancement includes a three-dimensional spatiotemporal-spectral feature extraction module 3D SSTFEM, a spectral enhancement module SAMOBS based on optimal band feature selection, a spatial enhancement module SAMSSM based on spectral similarity modeling, and a spatial-spectral feature fusion module SSFFM. The spatial-spectral feature fusion module SSFFM weightedly fuses the features of the spectral enhancement module SAMOBS and the spatial enhancement module SAMSSM through contrast loss.

[0016] The method for crop classification based on remote sensing images based on spatial-spectral joint enhanced contrast learning comprises the following steps: Step S1, designing a three-dimensional time-space-spectrum feature extraction module 3D SSTFEM: The three-dimensional space-time-spectral feature extraction module 3D SSTFEM is based on a three-dimensional convolutional neural network, which can utilize the spatial distribution and spectral characteristics of crops and capture the growth dynamic information of crops from the time dimension. The three-dimensional space-time-spectral feature extraction module 3D SSTFEM includes four blocks, each of which contains a convolution layer, a normalization layer and an activation function. A batch normalization BN layer is added after the convolution layer to speed up the model fitting speed and avoid overfitting. The ReLu activation function is used to effectively avoid gradient explosion or disappearance, making the network training faster; Step S2: Design a spectrum enhancement module SAMOBS based on optimal band feature selection: The spectral enhancement module SAMOBS based on optimal band feature selection designs a feature band selection method based on the OIF optimal index factor. The optimal band is selected by calculating the correlation and information content of each band in the multispectral image, and the feature map composed of the optimal bands is used as the spectral enhancement view of the original image, which enhances the model's ability to extract spectral detail features.

[0017] The specific steps of step S2 are: A spectral enhancement module SAMOBS based on optimal band feature selection is proposed to reasonably select and combine bands, extract bands with high discrimination and sensitivity to highlight key crop features. For each sampled image, the standard deviation of each band and the correlation coefficient between bands are calculated: In the formula, S i represents the standard deviation of the ith band, R i，j is the correlation coefficient of the two bands i and j, w represents the width and height of the anchor tile, is the reflectivity of the ith band, is the mean reflectivity of the i-th band; The larger the standard deviation of the image data, the greater the amount of information it contains, and the smaller the correlation coefficient between bands, indicating that the image independence between bands is higher and the information redundancy is smaller; then, the OIF index of different band combinations is calculated based on the standard deviation and correlation coefficient: Where p is the number of band combinations. OIF can balance the information content and correlation of each spectral band, extract bands with large information content and low inter-band information redundancy, and thus determine the optimal characteristic band combination, which is used as the spectral enhancement view x in this paper. spe , and then use contrastive learning to bring the spectrally enhanced view group {x q , x spe} consistency characteristics.

[0018] Step S3: Design a spatial enhancement module SAMSSM based on spectral similarity modeling: The spectral enhancement module SAMSSM based on optimal band feature selection uses spatial relationships to mine the consistent features between crops with similar geographical distribution. Based on multiple adjacent image samples, the semantic relationship between adjacent samples is modeled using the spectral similarity matrix. By dynamically adjusting the correlation between adjacent samples, the similarity between adjacent samples in the geographic space is enhanced while reducing the noise impact caused by the incorrect selection of the nearest neighbor samples.

[0019] The specific steps of step S3 are: In order to better utilize spatial relationships to mine the consistent features between crops with similar geographical distribution, a spatial enhancement module SAMSSM based on spectral similarity modeling is proposed. This module is based on multiple adjacent image samples and uses the spectral similarity matrix to model the semantic relationship between adjacent samples. By dynamically adjusting the correlation between adjacent samples, the similarity between adjacent samples in the geographic space is enhanced while reducing the noise impact caused by the wrong selection of the nearest neighbor samples. First, a spatial neighborhood radius R is given, and the anchor image x is used as the q As the center, a set of n neighbor images are cropped out within its neighborhood The cosine similarity is used to evaluate the spectral similarity between the neighbor image and the anchor image in the neighborhood and establish the spectral similarity matrix: S i represents the cosine similarity between the i-th neighbor image and the anchor image in the matrix, and selects a neighbor image with the maximum cosine similarity, i.e., the highest spatial semantic similarity, to the anchor image as the spatial enhancement view x spa , and then use contrastive learning to close the spatial enhancement view group {x q ,xspa} consistency characteristics.

[0020] Step S4, designing the spatial spectrum feature fusion module SSFFM: The spatial-spectral feature fusion module SSFFM designs a spatial-spectral joint contrast loss, aiming to fuse the features extracted by different branch modules in steps S2 and S3, and flexibly learn the complementary relationship between spectral detail features and geographic spatial features, bringing better model performance and higher accuracy for crop classification tasks.

[0021] The specific steps of step S4 are: After the spatial-spectral enhancement transformation, the multi-view set {X q ,X spe ,X spa} is encoded by the feature extraction network f(·) into a representation {h q ,h spe ,h spa}. The nonlinear projection network g(·) projects each intermediate representation into two normalized embedding subspaces {Z1, Z2}, each of which is specialized for a single enhancement. Embedding subspace Z1 is used to mine spectral information, The opposite of Negative pairs are the embeddings of other samples in the embedding space and The embedding subspace Z2 is used to mine geospatial information. The opposite of Negative pairs are the embeddings of other samples in the embedding space and In order to make the gap between positive sample features smaller and the gap between negative sample features larger in each embedding subspace to achieve the purpose of contrastive learning, the contrastive loss functions L1 and L2 are constructed: Where L1 is the spectral contrast loss, L2 is the spatial contrast loss, N is the number of negative samples, represents the negative sample queue, is the temperature hyperparameter.

[0022] In order to fuse the features of the two embedded subspaces for joint training, a contrast loss function L for spatial-spectral joint enhancement is proposed. total : L total =αL1+βL2 Where α and β are weight hyperparameters, which are used to balance the complementary relationship between spectral detail features and geographic spatial features.

[0023] Step S5: Method for verifying and comparing models using data sets: The contrastive learning model SSACL based on spatial-spectral joint enhancement proposed in the above steps is compared and verified to verify the outstanding performance of the proposed method in complex crop classification scenarios.

[0024] like Figure 2 The CalCROP21 dataset covers the entire California Central Valley crop belt, using multispectral images observed by the Sentinel-2 satellite in 2018. The label data comes from the CDL data layer provided by the United States Department of Agriculture (USDA), covering 10 crop types such as corn, wheat, grapes, and almonds. The distribution of crop sample points in the dataset is shown in Figure 1. Figure 2 As shown in (a), crops with a sample ratio greater than 10% are selected for classification to reduce the inter-class classification differences caused by the imbalanced distribution of crop samples.

[0025] The Germany dataset defines a large study area north of Munich, Germany, using Sentinel-2 satellite images from 2017. The Bavarian Ministry of Food, Agriculture and Forestry (StMELF) of Germany provides ground truth information in the form of semantic labels, including 15 crop types such as corn, wheat, beans, oats, and sugar beets. The distribution of crop sample points in this dataset is shown in the figure. Figure 2 As shown in (b), crop types with sample proportions greater than 5% are selected to construct the final crop classification dataset.

[0026] like Figure 3 , 4 As shown in the figure, the most advanced contrastive learning model and crop classification model are selected to compare and verify the SSACL classification method based on spatial-spectral joint enhancement of the contrastive learning model proposed in this invention, proving the effectiveness and outstanding performance of the proposed method in high-resolution remote sensing image crop classification. The models selected for comparison are Tile2Vec, SimCLR, MoCo v2, BYOL, and 3D UNet.

[0027] Figure 3This is a visualization of the classification results of the contrastive learning model SSACL based on spatial-spectral joint enhancement proposed in the present invention and different models on the CalCROP21 dataset. SSACL has achieved the highest classification accuracy in most crop categories, especially for corn and cotton, with F1 values ​​as high as 91.6% and 92.2%, respectively. This shows that the contrastive learning model SSACL based on spatial-spectral joint enhancement can learn the intrinsic structure and representation of data by comparing the similarities or differences between different samples when the sample size is insufficient, thereby effectively distinguishing different categories of crops. The 3D UNet supervised learning model extracts smoother crop plot boundaries, but in terms of crop segmentation, 3DUNet and SimCLR, MoCo v2, and BYOL methods are all affected by noise to varying degrees, resulting in more crop misclassifications. In contrast, the contrastive learning model SSACL based on spatial-spectral joint enhancement proposed in the present invention achieves a more accurate crop classification effect.

[0028] Figure 4 The visualization effect of the classification results of the contrastive learning model SSACL based on spatial-spectral joint enhancement proposed in the present invention and different models on the Germany dataset. The contrastive learning model SSACL based on spatial-spectral joint enhancement has achieved the highest classification accuracy on winter wheat, winter barley, rapeseed and potato, among which the most significant effect is potato crop, which has increased the F1 value by 10.9% and the IoU by 7.6% compared with the supervised learning model. It shows that the SSACL model is less restricted by the number of samples, and improves the classification effect of crops by deeply mining the spatial-spectral features of crops under the condition of limited number of samples. In addition, compared with Tile2Vec, SimCLR, BYOL and MoCo v2 contrastive learning methods, the contrastive learning model SSACL based on spatial-spectral joint enhancement proposed in the present invention has achieved the highest classification accuracy on most crops and extracted the most complete crop plot boundaries, which has a good crop recognition effect on fragmented plots and effectively reduces the misclassification of crops.

[0029] like Figure 5 As shown, the present invention conducts a hyperparameter analysis of the SSACL model based on the CalCROP21 dataset to explore the influence of loss weights α, β and spatial neighborhood radius distance R on the model performance.

[0030] α, β are parameters used to control the relative importance of spectral and spatial features in the loss function. Figure 5As shown in (a), when α=0.4, β=0.6, the model focuses more on learning spatial features, and the ability to capture spectral information is insufficient, resulting in poor model classification performance. As higher weights are assigned to the spectral enhancement module, the model performance gradually improves. When α=0.6, β=0.4, SSACL achieves the best performance, indicating that full utilization of spectral information is crucial to improving model performance. However, when the spectral enhancement loss weight is too large, the model may focus too much on spectral detail features and ignore geographic spatial context features, resulting in a decrease in model classification performance.

[0031] R is the neighborhood radius for sampling neighbor samples centered on the anchor image sample. Figure 5 (b) shows the experimental results of the contrastive learning model SSACL based on spatial-spectral joint enhancement under different neighborhood radii. It can be seen that with the increase of the sampling range and the neighborhood radius R, the performance of the contrastive learning model SSACL based on spatial-spectral joint enhancement for crop classification shows a trend of first increasing and then decreasing. When R is too small, the model can only capture the local feature information in the remote sensing image, and cannot fully capture the spatial context semantic features, resulting in a decrease in the model classification performance. If R is too large, the spatial samples will be too dispersed, and it will be difficult for the model to learn the spatial distribution law of crops. When R = 100 pixels, the contrastive learning model SSACL based on spatial-spectral joint enhancement achieved the best classification performance.

[0032] Aiming at the problem that the large-scale label data is lacking in the field of remote sensing crop classification and the complex changes of spatiotemporal spectral features during crop growth affect the performance of existing self-supervised learning representation, the present invention proposes a remote sensing image crop classification method based on spatial-spectral joint enhanced contrast learning. First, the three-dimensional spatiotemporal-spectral feature extraction module 3D SSTFEM is used to simultaneously extract the spatiotemporal spectral features of crops. Then, the spectral enhancement module SAMOBS based on optimal band feature selection improves the utilization efficiency of spectral information, and the spatial enhancement module SAMSSM based on spectral similarity modeling enhances the spatial correlation between adjacent plots. Finally, the spatial-spectral feature fusion module SSFFM is designed and the spatial-spectral joint contrast loss is used to perform joint training to learn the complementary relationship between the spectral domain and the spatial domain. Experiments on crop classification datasets in different regions show that the proposed method can effectively capture the spatial structure and spectral features of crops and achieve more accurate crop type recognition. In addition, when there are fewer labeled samples, the contrast learning model SSACL based on spatial-spectral joint enhancement exceeds supervised learning in classification performance and model robustness, significantly reducing the dependence of deep learning model training on large-scale labeled data, and can be effectively applied to crop classification tasks.

Claims

1. A remote sensing image crop classification method based on spatial-spectral joint enhanced contrast learning, characterized by: It includes a contrastive learning model SSACL based on space-spectrum joint enhancement, wherein the contrastive learning model SSACL based on space-spectrum joint enhancement includes a three-dimensional time-space-spectrum feature extraction module 3D SSTFEM, a spectral enhancement module SAMOBS based on optimal band feature selection, a spatial enhancement module SAMSSM based on spectral similarity modeling, and a space-spectrum feature fusion module SSFFM, wherein the space-spectrum feature fusion module SSFFM weightedly fuses the features of the spectral enhancement module SAMOBS and the spatial enhancement module SAMSSM through contrast loss; The remote sensing image crop classification method based on spatial-spectral joint enhanced contrast learning comprises the following steps: Step S1, designing a three-dimensional time-space-spectrum feature extraction module 3D SSTFEM: The three-dimensional space-time-spectral feature extraction module 3D SSTFEM is based on a three-dimensional convolutional neural network, which utilizes the spatial distribution and spectral characteristics of crops and captures the growth dynamic information of crops from the time dimension at the same time; the three-dimensional space-time-spectral feature extraction module 3D SSTFEM includes four blocks, each of which includes a convolution layer, a normalization layer and an activation function. A batch normalization BN layer is added after the convolution layer to speed up the model fitting speed and avoid overfitting. The ReLu activation function is used to effectively avoid gradient explosion or disappearance, making the network training faster; the space-time spectral features of remote sensing images are extracted simultaneously through three-dimensional convolution, which can retain the time change information in the growth process of crops while mining spatial and spectral features, and fully model the complex space-time spectral change characteristics in the growth process of crops; Step S2: Design a spectrum enhancement module SAMOBS based on optimal band feature selection: The spectrum enhancement module SAMOBS based on optimal band feature selection designs a feature band selection method based on the OIF optimal index factor; the optimal band is selected by calculating the correlation and information content of each band in the multispectral image, and the feature map composed of the optimal bands is used as the spectral enhancement view of the original image to enhance the model's ability to extract spectral detail features; the bands of remote sensing data have high correlation. By reasonably selecting and combining the bands, extracting bands with high discrimination and sensitivity helps to highlight the characteristics of different objects and improve the classification performance of the model; Step S3: Design a spatial enhancement module SAMSSM based on spectral similarity modeling: The spatial enhancement module SAMSSM based on spectral similarity modeling uses spatial relationships to mine the consistent features between crops with similar geographical distribution. Based on multiple adjacent image samples, the spectral similarity matrix is ​​used to model the semantic relationship between adjacent samples. By dynamically adjusting the correlation between adjacent samples, the similarity between adjacent samples in the geographic space is enhanced while reducing the noise impact caused by the incorrect selection of the nearest neighbor samples. While making full use of the correlation between adjacent crop samples, the features of different crop types are avoided due to the incorrect aggregation, thereby improving the classification accuracy and robustness of crops with similar spatial distribution. Step S4, designing the spatial spectrum feature fusion module SSFFM: The spatial-spectral feature fusion module SSFFM designs a spatial-spectral joint contrast loss to fuse the features extracted by different branch modules in step S2 and step S3, and learn the complementary relationship between spectral detail features and geographic spatial features, which brings better model performance and higher accuracy for crop classification tasks; by dynamically learning the complementary relationship between spatial dimension and spectral dimension, it effectively promotes the fusion of spatial-spectral self-supervised features; Step S5: Method for verifying and comparing models using data sets: A comparative verification of the contrastive learning model SSACL based on spatial-spectral joint enhancement is performed to verify the outstanding performance of the proposed method in complex crop classification scenarios.

2. The method for crop classification based on remote sensing imagery based on spatial-spectral joint enhanced contrast learning according to claim 1, characterized in that: The specific steps of step S2 are: A spectral enhancement module SAMOBS based on optimal band feature selection is proposed to select and combine bands, extract bands with discrimination and sensitivity to highlight key crop features. For each sampled image, the standard deviation of each band and the correlation coefficient between bands are calculated: In the formula, S i represents the standard deviation of the ith band, R i，j is the correlation coefficient of the two bands i and j, w represents the width and height of the anchor tile, is the reflectivity of the ith band, is the mean reflectivity of the i-th band; The larger the standard deviation of the image data, the greater the amount of information it contains, and the smaller the correlation coefficient between bands, indicating that the image independence between bands is higher and the information redundancy is smaller; then, the OIF index of different band combinations is calculated based on the standard deviation and correlation coefficient: Where p is the number of band combinations; OIF can balance the information content and correlation of each spectral band, extract bands with large information content and low inter-band information redundancy, and thus determine the optimal characteristic band combination, which is used as the spectral enhancement view x in this paper. spe , and then use contrastive learning to bring the spectrally enhanced view group {x q , x spe } consistency characteristics.

3. The method for crop classification based on remote sensing imagery based on spatial-spectral joint enhanced contrast learning according to claim 1, characterized in that: The specific steps of step S3 are: The spatial enhancement module SAMSSM based on spectral similarity modeling is based on multiple adjacent image samples and uses the spectral similarity matrix to model the semantic relationship between adjacent samples. By dynamically adjusting the correlation between adjacent samples, the similarity between adjacent samples in the geographic space is enhanced while reducing the noise impact caused by the wrong selection of the nearest neighbor samples. First, a spatial neighborhood radius R is given, with the anchor image x q As the center, a set of n neighbor images are cropped out within its neighborhood The cosine similarity is used to evaluate the spectral similarity between the neighbor image and the anchor image in the neighborhood and establish the spectral similarity matrix: s i represents the cosine similarity between the i-th neighbor image and the anchor image in the matrix, and selects a neighbor image with the maximum cosine similarity, i.e., the highest spatial semantic similarity, to the anchor image as the spatial enhancement view x spa , and then use contrastive learning to close the spatial enhancement view group {x q , x spa } consistency characteristics.

4. The method for crop classification based on remote sensing imagery based on spatial-spectral joint enhanced contrast learning according to claim 1, characterized in that: The specific steps of step S4 are: After the spatial-spectral enhancement transformation, the multi-view set {X q , X spe , X spa } is encoded by the feature extraction network f(·) into a representation {h q ,h spe ,h spa }; The nonlinear projection network g(·) projects each intermediate representation into two normalized embedding subspaces {Z1, Z2}, each of which is dedicated to a single enhancement; The embedded subspace Z1 is used to mine spectral information. The opposite of Negative pairs are the embeddings of other samples in the embedding space and The embedding subspace Z2 is used to mine geospatial information. The opposite of Negative pairs are the embeddings of other samples in the embedding space and In order to make the gap between positive sample features smaller and the gap between negative sample features larger in each embedding subspace to achieve the purpose of contrastive learning, the contrastive loss functions L1 and L2 are constructed: Where L1 is the spectral contrast loss, L2 is the spatial contrast loss, N is the number of negative samples, represents the negative sample queue, is the temperature hyperparameter; In order to fuse the features of the two embedded subspaces for joint training, a contrast loss function L for spatial-spectral joint enhancement is proposed. total : L total =αL1+βL2 Where α and β are weight hyperparameters, which are used to balance the complementary relationship between spectral detail features and geographic spatial features.

Images