A fine classification method for crops based on multi-dimensional spectral features

Abstract

The present application belongs to the technical field of image processing, and particularly relates to a crop fine classification method based on multi-dimensional spectral features. Single-phase high-resolution remote sensing images are acquired first, and multiple spectral indexes are calculated to generate image grids. Based on the cultivated land block extraction result, the remote sensing image is superimposed to select the classification label sample. Then, the remote sensing image and the spectral index grid are cut respectively and synthesized in multiple channels. After that, the data set is divided, and the deep learning model is constructed. After training and verification, the global inference output result is obtained. The method in the present application improves the accuracy through multi-dimensional spectral indexes, and is suitable for fine classification of different crops. Based on the cultivated land map spot classification, the influence of noise and broken map spots on crop recognition can be reduced, the recognition result is more suitable for planting conditions, and only single-phase high-resolution remote sensing images are required, which is not affected by the missing of time series data, has high adaptability to data sources, the number of model input channels is flexible, and is suitable for crop recognition and classification tasks in complex geographical environments.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically a method for fine classification of crops based on multidimensional spectral features. Background Technology

[0002] Refined crop classification has always been a crucial technical challenge in remote sensing applications. Early traditional crop classification primarily relied on low- to medium-resolution open-source remote sensing imagery (such as Landsat and MODIS), utilizing single- or multi-temporal data to construct supervised classification training datasets, and then employing machine learning methods such as Random Forest (RF) and Support Vector Machine (SVM) for crop classification. With the continuous development of computer information technology, big data, and remote sensing technology, using satellite remote sensing imagery to identify crop planting has become one of the most important technical means.

[0003] However, current technical solutions are mostly pixel-oriented and object-oriented classification methods based on machine learning models such as random forests and support vector machines, or crop classification schemes based on time-series phenological features using deep learning models. While the former can achieve good classification results at the microscale, pixel-based classification results in a large number of scattered pixels, exhibiting obvious "salt and pepper" noise, poor readability, and difficulty in applying to actual production. Object-based classification, due to its global segmentation and lack of focus on specific farmland plots, results in fragmented classifications that do not reflect actual crop planting conditions. The latter approach extracts crop growth cycle features from time-series data; however, in actual production, the lack of continuous time-series data due to cloud and fog often hinders crop identification. Detailed introductions to these two approaches are as follows:

[0004] The paper "Hyperspectral Remote Sensing Crop Classification Based on Random Forest Algorithm" proposes a crop classification model based on hyperspectral imagery, constructing a random forest crop classification model using decision trees. This model analyzes the hyperspectral characteristics of different crops and uses a voting decision method to determine the crop planting category. However, this method primarily classifies based on pixels. When crops are planted in a heterogeneous distribution, the classification results contain many fragmented pixels, affecting the readability of the classification results. Furthermore, this method relies on hyperspectral remote sensing imagery, which is often difficult to obtain in actual production operations, limiting its universality in crop classification. The paper "Crop Classification Extraction Based on Multi-Source Remote Sensing Imagery" proposes an object-oriented classification method based on synthetic aperture radar and multispectral data, extracting spectral and texture features through multi-scale segmentation, achieving good classification results. However, this object-oriented segmentation strategy tends to produce fragmented patches in heterogeneously planted crop types, resulting in less than ideal crop classification results.

[0005] The patent CN113159154, titled "A Time-Series Feature Reconstruction and Dynamic Recognition Method for Crop Classification," proposes using a Long Short-Term Memory (LSTM) network to predict and fill in missing data regions, and then using dynamically acquired data for crop classification. This method overcomes the impact of cloud and fog on the lack of continuous time-series data. However, because it relies on optical time-series remote sensing imagery, data gaps exist in multi-temporal images during continuous cloudy and foggy weather, thus affecting crop identification. Furthermore, it cannot solve the problem of fine-grained crop classification when only single-temporal images are available.

[0006] In the context of precision agriculture and the business needs of agricultural insurance, there is an urgent need for a refined crop classification solution that can be applied to actual business production, has high identification accuracy, and provides easily readable classification results. Summary of the Invention

[0007] The purpose of this invention is to provide a fine classification method for crops based on multidimensional spectral features, so as to solve the problems of fragmented classification results, reliance on continuous time-series data, and insufficient accuracy in existing crop classification methods proposed in the background art.

[0008] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0009] A method for fine classification of crops based on multidimensional spectral features includes the following steps:

[0010] Step S1: Acquire high-resolution remote sensing images; select single-temporal high-resolution remote sensing images of crop growth stages, calculate various spectral indices using high-resolution remote sensing images, and generate corresponding image raster images.

[0011] Step S2: Select classification and labeling samples; Select classification and labeling samples based on the farmland plot extraction results. The samples are farmland plot labeling samples corresponding to the crop category.

[0012] Step S3, image segmentation and multi-channel synthesis: Based on the samples marked by cultivated land patches, high-resolution multispectral images and corresponding spectral index images are segmented according to the unique patch codes, and the segmented images are resampled to a fixed size; the RGB channels and multiple spectral indices of the segmented images are synthesized in multiple channels to obtain multi-channel image data;

[0013] Step S4: Divide the multi-channel image data obtained in step S3 into training set, validation set and test set according to the proportion;

[0014] Step S5: Construct a deep learning crop classification model, which includes a backbone network, a neck network, and a classification head.

[0015] Step S6: Train the deep learning crop classification model using the training set, evaluate the training performance using the validation set, and obtain the trained crop classification model.

[0016] Step S7: Apply the trained crop classification model to the multi-channel image data corresponding to the cultivated land patches across the entire region, and output the refined crop classification results across the entire region.

[0017] According to the above technical solution, in step S1, the spectral index includes at least three of the following: Normalized Difference Vegetation Index (NDVI), Dual-Band Enhanced Vegetation Index (EVI2), Normalized Water Index (NDWI), Modified Soil Adjusted Vegetation Index (MSAVI), Ratio Vegetation Index (RVI), and Vegetation Near-Infrared Reflectance Index (NIRv).

[0018] According to the above technical solution, the formula for calculating the Normalized Difference Vegetation Index (NDVI) is as follows:

[0019] NDVI = (NIR - RED) / (NIR + RED)

[0020] Where NIR stands for near-infrared band and RED stands for red band;

[0021] The formula for calculating the dual-band enhanced vegetation index EVI2 is as follows:

[0022] EVI2=G*(NIR-RED) / (NIR+2.4*RED+L)

[0023] Where G is the gain coefficient, G=2.5 for MODIS sensors, and L is the soil adjustment factor, used to adjust for soil background effects, L=1 for MODIS sensors.

[0024] According to the above technical solution, in step S3, the high-resolution image of the whole area and the corresponding spectral index image are cut, resampled and multi-channel synthesized according to the order of the cultivated land patch number. The synthesized image is numbered according to the cultivated land patch number and used for crop classification inference.

[0025] According to the above technical solution, in step S3, the size of the image data after resampling and synthesis is H*W*C, where H is the image height, W is the image width, and C is the number of channels, which is the sum of the number of RGB channels and the number of spectral index types.

[0026] According to the above technical solution, in step S4, the ratio of training set, validation set and test set is 8:1:1, that is, the training set accounts for 80%, the validation set accounts for 10%, and the test set accounts for 10%.

[0027] According to the above technical solution, in step S5, the deep learning crop classification model can freely specify the number of channels of the input image to match the number of channels of the multi-channel synthesized image data.

[0028] According to the above technical solution, in step S6, the training process uses the SGD optimizer, the learning rate uses the multi-step learning rate (MultiStepLR), and the loss function uses the cross-entropy loss (Cross Entropy Loss).

[0029] According to the above technical solution, in step S6, the generalization ability of the model is verified using a test set, and Top-1 and Top-3 accuracy are used as evaluation indicators.

[0030] According to the above technical solution, in step S7, the crop classification results are saved in ESRI Shapefile format, and the crop category is labeled by cultivated land plot.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] This invention effectively improves the accuracy of crop classification and identification by utilizing multiple spectral indices, allowing for free selection of multi-channel synthesis schemes. By combining different spectral indices with remote sensing imagery, fine classification of different crop types at different times can be achieved. Based on cultivated land plots rather than pixels or global objects, the classification results avoid salt-and-pepper noise and fragmented patches, resulting in highly readable results that accurately reflect actual crop planting conditions.

[0033] Relying solely on single-phase imagery, without requiring continuous time-series data, and unaffected by data loss due to cloud or fog, combined with multidimensional spectral indices (such as NDVI, EVI2, NDWI, etc.), it effectively distinguishes between crops and non-crops, different crop types, and soil / water body interference, thereby improving classification accuracy.

[0034] The technical solution in this invention also supports multi-source satellite data supplementation of the near-infrared band, and has high adaptability to image data sources; the model can flexibly specify the number of input channels, and is suitable for different spectral index combination scenarios. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the selection of labeled samples in this invention, wherein Figure (a) is a cultivated land patch and Figure (b) is a selected labeled sample;

[0036] Figure 2 This is a schematic diagram of multi-channel image synthesis according to the present invention; (the left side shows a single-phase RGB image and various spectral indices, and the right side shows the synthesized multi-channel image)

[0037] Figure 3This is a schematic diagram of the fine classification results of crops in this invention, where Figure (a) is a high-resolution remote sensing image and Figure (b) is the classification result;

[0038] Figure 4 This is a flowchart illustrating the fine classification process for crops according to the present invention. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] Example 1

[0041] like Figure 4 As shown, a method for fine classification of crops based on multidimensional spectral features includes the following steps:

[0042] Step S1: Acquire high-resolution remote sensing images; select single-temporal high-resolution remote sensing images of crop growth stages, calculate various spectral indices using high-resolution remote sensing images, and generate corresponding image raster images.

[0043] Step S2: Select classification and labeling samples; select classification and labeling samples based on the farmland plot extraction results. The samples are farmland plots corresponding to the crop category.

[0044] Specifically, staff will overlay farmland plots and images on software or platforms, and manually select the corresponding crop category based on the image features selected from the plots.

[0045] Step S3, Image Segmentation and Multi-channel Synthesis: Based on the farmland patch annotation samples, high-resolution multispectral images and corresponding spectral index images are segmented, and the segmented images are resampled to a fixed size; the RGB channels and multiple spectral indices of the segmented images are synthesized in multiple channels to obtain multi-channel image data;

[0046] Specifically, the segmentation method is as follows: segmentation by mask, that is, extracting only the remote sensing image of the area covered by cultivated land patches, and then saving it as a separate image. The purpose is to extract images of each cultivated land patch for model training and inference.

[0047] Step S4: Divide the multi-channel image data obtained in step S3 into training set, validation set and test set according to the proportion;

[0048] Step S5: Construct a deep learning crop classification model, which includes a backbone network, a neck network, and a classification head.

[0049] Step S6: Train the deep learning crop classification model using the training set, evaluate the training performance using the validation set, and obtain the trained model.

[0050] Step S7: Apply the trained model to the multi-channel image data corresponding to the cultivated land patches across the entire area and output the crop classification results.

[0051] This invention effectively improves the accuracy of crop classification and identification by utilizing multiple spectral indices, allowing for free selection of multi-channel synthesis schemes. By combining different spectral indices with remote sensing imagery, fine classification of different crop types at different times can be achieved. Based on cultivated land plots rather than pixels or global objects, the classification results avoid salt-and-pepper noise and fragmented patches, resulting in highly readable results that accurately reflect actual planting conditions.

[0052] Relying solely on single-phase imagery, without requiring continuous time-series data, and unaffected by data loss due to cloud or fog, combined with multidimensional spectral indices (such as NDVI, EVI2, NDWI, etc.), it effectively distinguishes between crops and non-crops, different crop types, and soil / water body interference, thereby improving classification accuracy.

[0053] The technical solution in this invention also supports multi-source satellite data supplementation of the near-infrared band, and has high adaptability to image data sources; the model can flexibly specify the number of input channels, and is suitable for different spectral index combination scenarios.

[0054] Example 2

[0055] This embodiment is a further refinement of Embodiment 1.

[0056] This invention aims to enable the precise classification of crops by utilizing single-temporal high-resolution remote sensing images and cultivated land plot data.

[0057] Specifically, the first step is to acquire high-resolution remote sensing imagery. Select single-temporal high-resolution remote sensing images of crops during their growth stages (image resolution better than 0.75m, free from cloud and fog effects, and preferably capable of distinguishing crop types). Utilize the red, green, blue, and near-infrared (NIR) bands of the high-resolution remote sensing images to calculate various spectral indices and generate corresponding raster images. If the high-resolution remote sensing imagery lacks a near-infrared band, the technical solution of this invention can also use other satellite sensors from the same period (such as Sentinel-2 / SPOT) to calculate the corresponding spectral indices as a substitute. Commonly used optional spectral index calculation methods are shown in Table 1:

[0058] Table 1. Calculation methods and characteristics of spectral indices

[0059]

[0060] In Table 1, NIR represents the near-infrared band, RED represents the red band, and GREEN represents the green band. G is the gain coefficient, which is 2.5 for the MODIS sensor, and L is the soil adjustment factor, which is 1 for the MODIS sensor.

[0061] Specifically, the roles of each index in crop classification are as follows: Normalized Difference Vegetation Index (NDVI), Dual-Band Enhanced Vegetation Index (EVI2), and Ratio Vegetation Index (RVI) can distinguish the image features of crop-grown and non-crop-grown plots; Vegetation Near-Infrared Reflectance Index (NIRv) effectively reduces noise interference from the soil background and is sensitive to vegetation phenology, effectively extracting crop type features at different growth stages; Normalized Water Index (NDWI) enhances the difference features between water and non-water surface types, used to distinguish farmland and dryland plots, effectively improving the accuracy of crop classification results; Soil Modified Vegetation Index (MSAVI) is adjusted for soil effects, is more sensitive to early crops in the field, compensates for the sensitivity of NDVI in low vegetation cover conditions, and improves the accuracy of crop classification.

[0062] The second step is to select labeled samples. Based on the extracted farmland plots, corresponding classification and labeling samples are selected, as illustrated in the diagram below. Figure 1 As shown in (a). Since the farmland plot data has already been divided into image segments, manual delineation is not required when selecting labeled samples. Only samples corresponding to the crop category need to be selected, such as... Figure 1 As shown in (b).

[0063] The third step is image segmentation and multi-channel synthesis. Based on the selected farmland patch annotation samples, high-resolution multispectral images and corresponding spectral indices are segmented according to the fixed numbering order of the patches. The segmented images are then resampled to a fixed image size (determined based on the field size, generally set to 128) using the LANCZOS sampling method to match the input of the deep learning network. The segmented image patches are numbered according to the annotation sample numbers. Subsequently, the RGB channels of the segmented high-resolution images and the corresponding multiple spectral indices are synthesized in multiple channels. The size of the synthesized image data after resampling is H*W*C, where H is the image height, W is the image width, and C is the number of channels. Similarly, based on the entire farmland patch, the entire high-resolution image and the corresponding spectral index image are segmented, resampled, and synthesized in multiple channels according to the patch numbering order. The synthesized image is numbered according to the farmland patch number. The dataset generated by the entire field segmentation will be used for crop classification inference. A schematic diagram of image segmentation and multi-channel synthesis is shown below. Figure 2 As shown.

[0064] The fourth step is to divide the dataset. After the image samples have been segmented, resampled, and synthesized into multiple channels, they are divided into training, validation, and test sets in an 8:1:1 ratio.

[0065] The fifth step is to construct a deep learning crop classification model. Based on the ResNet-18 backbone network, this invention constructs a classification model suitable for fine-grained crop classification using multi-channel images. The model allows for flexible specification of the number of channels in the input image. The specific components of the model include:

[0066] The backbone network uses ResNet-18 and consists of one initial convolutional layer, eight residual block convolutional layers (eight in total), and one fully connected layer, for a total of nine convolutional layers. The network incorporates residual connections to reduce network degradation. The convolutional layers effectively extract spatial features from the images, while the residual structure ensures the stability and accuracy of training deep networks.

[0067] Classification Head. The classification head consists of a fully connected layer with an input feature dimension of 512 and an output of the classification probabilities for n classes. The loss function used is cross-entropy loss.

[0068] Neck network. A global average pooling layer is used before the classification head to further compress the spatial dimension of the feature maps to reduce the number of parameters.

[0069] Step 5, Model Training. Train the model using the training dataset. The training parameters are set using the SGD optimizer, with a maximum of 300 training epochs, a learning rate (LR) of MultiStepLR, and a batch size of 64.

[0070] Step 6: Model Evaluation and Accuracy Assessment. The scheme used in this invention uses a validation dataset to evaluate the model's performance during training, a test dataset to verify the model's generalization ability, and Top-1 and Top-3 accuracy metrics for evaluation.

[0071] Specifically, Top-3 accuracy is calculated as follows: From the categories in the test set, select the top 3 categories with the highest probabilities. If the true category is one of these 3, the prediction is correct; otherwise, it is incorrect. The formula is:

[0072] Top-3 accuracy = Number of samples whose true class ranks in the top 3 of the predicted results / Total number of samples in the test set.

[0073] Top-1 accuracy: The proportion of correctly predicted data in the test set out of the total number of samples in the test set. The formula is: Top-1 accuracy = Number of correctly predicted data / Total number of test set samples.

[0074] Step 7, Model Application. The trained crop classification model can classify crops from remote sensing images taken during the same period. The classification results are output and saved in ESRI Shapefile format. The classification results are shown below. Figure 3 As shown, where, Figure 3 (a) is a high-resolution remote sensing image. Figure 3 (b) shows the classification results.

[0075] The key points and intended protection points of this invention lie in using multidimensional spectral indices and high-resolution remote sensing imagery for multi-channel synthesis based on cultivated land plots to achieve fine classification of crops and improve the classification accuracy of the crop classification model. Furthermore, the classification method of this invention solves the problem of fragmented and scattered crop classification in traditional machine learning, improving the readability of the crop classification results.

[0076] Existing technologies generally rely on multi-temporal high-resolution remote sensing imagery, which still cannot solve the problem of fine crop classification under widespread cloudy and foggy weather. The technical solution of this invention relies only on single-temporal high-resolution remote sensing imagery, and does not limit the source of satellite remote sensing data sensors or the spatial resolution of the imagery. The remote sensing imagery only requires four channels: red, green, blue, and near-infrared (NIR), which is sufficient to achieve the crop classification of this invention.

[0077] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0078] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for fine classification of crops based on multi-dimensional spectral features, characterized in that, The method comprises the following steps: Step S1, obtaining high-resolution remote sensing images; selecting single-phase high-resolution remote sensing images of crop growth stages, calculating various spectral indices by using the high-resolution remote sensing images, and generating corresponding image raster images; Step S2, selecting classification annotation samples; based on the extraction results of the cultivated land plots, selecting the classification annotation samples, and the samples are the cultivated land plot annotation samples corresponding to the crop categories; Step S3, image cutting and multi-channel synthesis; based on the cultivated land plot annotation samples, cutting the high-resolution multi-spectral images and the corresponding spectral index images according to the unique plot codes respectively, and resampling the cut images to a fixed size; performing multi-channel synthesis on the RGB channels and the various spectral indices of the cut images to obtain multi-channel image data; Step S4, dividing the multi-channel image data obtained in step S3 into a training set, a verification set and a test set according to a proportion; Step S5, constructing a deep learning crop classification model, the model comprising a backbone network and a classification head, and a global average pooling layer being arranged between the backbone network and the classification head; Step S6, training the deep learning crop classification model by using the training set, evaluating the training performance by using the verification set, and obtaining the trained crop classification model; Step S7, applying the trained crop classification model to the multi-channel image data corresponding to the global cultivated land plots, and outputting a global refined crop classification result.

2. The method according to claim 1, wherein the method is characterized by: In step S1, the spectral indices include at least three of a normalized difference vegetation index NDVI, a dual-band enhanced vegetation index EVI2, a normalized water index NDWI, a modified soil-adjusted vegetation index MSAVI, a ratio vegetation index RVI and a vegetation near-infrared reflectance index NIRv.

3. The method for fine classification of crops based on multidimensional spectral features according to claim 2, characterized in that: The calculation formula of the normalized difference vegetation index NDVI is: NDVI=(NIR-RED) / (NIR+RED) wherein NIR is a near-infrared band, and RED is a red band; The calculation formula of the dual-band enhanced vegetation index EVI2 is: EVI2=G*(NIR-RED) / (NIR+2.4*RED+L) wherein G is a gain coefficient, G=2.5 for a MODIS sensor, and L is a soil adjustment factor for adjusting the influence of soil background, L=1 for the MODIS sensor.

4. The method according to claim 1, wherein the method is characterized by: In step S3, the global high-resolution images and the corresponding spectral index images are cut, resampled and multi-channel synthesized according to the cultivated land plot numbering order, the synthesized images are numbered according to the cultivated land plot numbering, and are used for inference of crop classification.

5. The method according to claim 4, wherein: In step S3, the size of the image data synthesized after resampling is H*W*C, wherein H is the image height, W is the image width, and C is the number of channels, which is the sum of the number of RGB channels and the number of spectral index types.

6. The method of claim 1, wherein the method comprises: In step S4, the proportions of the training set, the verification set and the test set are 8:1:1, that is, the training set accounts for 80%, the verification set accounts for 10%, and the test set accounts for 10%.

7. The method according to claim 1, wherein the method is characterized by: In step S5, the deep learning crop classification model can specify the number of input image channels to match the number of channels of the multi-channel synthesized image data.

8. The method according to claim 1, wherein the method is characterized by: In step S6, the training process uses the SGD optimizer, the learning rate uses the multi-step learning rate MultiStepLR, and the loss function uses the cross-entropy loss Cross Entropy Loss.

9. The method according to claim 1, wherein the method is characterized by: In step S6, the generalization ability of the model is verified using the test set, and the Top-1 and Top-3 accuracy are used as evaluation indicators.

10. The method of claim 1, wherein the method comprises: In step S7, the crop classification results are saved in the ESRI Shapefile format, and the classification results are labeled with crop categories in the farmland plot unit.

Images