A method for monitoring corn harvest progress with high temporal and spatial resolution

Abstract

A method for monitoring corn harvesting progress with high spatiotemporal resolution, the present invention relates to a method for monitoring corn harvesting progress with high spatiotemporal resolution. The present invention comprises the following steps: 1. obtaining Sentinel‑1 and Sentinel‑2 images; 2. preprocessing data; 3. extracting sample point data; 4. analyzing polarization characteristics of Sentinel‑1 images and spectral reflectance curves of Sentinel‑2; 5. constructing a preliminary feature set; 6. forming a feature set; 7. classifying data and comparing accuracy; 8. selecting a classification method; 9. performing general classification post-processing; 10. optimizing classification accuracy with a special classification post-processing method; 11. drawing a spatiotemporal dynamic distribution diagram of corn harvesting progress, and statistically calculating the cumulative corn harvesting area and harvesting ratio. The present invention combines Sentinel‑1 radar images and Sentinel‑2 optical images to monitor corn harvesting progress.

Description

Technical Field

[0001] The invention relates to a corn high temporal and spatial resolution monitoring method. Background Art

[0002] Northeast China is the main corn producing area in my country, and its corn output ranks first among major crops. Corn is one of my country's main food crops and is of great significance to my country's food security. Timely monitoring of corn harvesting progress is conducive to timely adjustment of agricultural management decisions and ensuring corn quality and yield.

[0003] However, the main method for crop monitoring is still manual sampling survey. This method requires a lot of time, manpower and material resources. It is not only unable to achieve large-scale monitoring, but also has a long monitoring cycle and cannot meet the timeliness requirements of data. Remote sensing technology, as a "remote observation" technology, can obtain spatiotemporal dynamic change data of large-area objects in a relatively short period of time. Therefore, remote sensing technology can achieve large-area dynamic mapping and is a better method to monitor the progress of corn harvesting.

[0004] At present, remote sensing technology is used to monitor the progress of crop harvesting, mainly using optical images to monitor the progress of winter wheat harvesting. The main methods are divided into two categories, one is to classify single-phase data based on NDVI threshold segmentation, and the other is to classify multi-phase data. NDVI time series curves are constructed based on multi-period data, and thresholds are found based on the fluctuations of the NDVI time series curves before and after winter wheat for classification. However, NDVI is easily affected by vegetation moisture content, and its changes after corn harvest are unstable, so its monitoring accuracy needs to be improved. Moreover, optical images also have certain limitations in monitoring corn harvesting progress.

[0005] Optical data is greatly affected by weather conditions such as clouds, fog, and snow, which can easily cause image loss, thus easily resulting in a longer monitoring cycle for corn harvest progress. Radar data, as another type of image in remote sensing technology, is not easily affected by weather conditions and can achieve all-day and all-weather monitoring of corn harvest progress. However, radar images do not have rich spectral information, and a single radar data has a long revisit cycle, which cannot meet the monitoring cycle of corn harvest progress. Summary of the invention

[0006] The present invention provides a method for monitoring corn harvesting progress with high temporal and spatial resolution, which is a method for monitoring corn harvesting progress with high temporal and spatial resolution that cooperates with Sentinel-1 and Sentinel-2 data and is based on a time series misclassification correction algorithm.

[0007] The method for monitoring corn harvesting progress with high spatiotemporal resolution of the present invention is carried out according to the following steps:

[0008] 1. Determine the corn harvest time in the area to be monitored and obtain Sentinel-1 and Sentinel-2 images before and after the harvest;

[0009] 2. Preprocess the Sentinel-1 and Sentinel-2 data;

[0010] 3. Visually interpret the changes in the images before and after corn harvest and extract the sample point data;

[0011] 4. Extract and analyze the polarization characteristics of Sentinel-1 images and the spectral reflectance curves of Sentinel-2 based on the sample point data;

[0012] 5. Construct a preliminary feature set for Sentinel-1 and Sentinel-2 images;

[0013] 6. Select the feature sets of Sentinel-1 images and Sentinel-2 images to form a feature set;

[0014] 7. Use the threshold segmentation method and random forest method to classify Sentinel-1 and Sentinel-2 data based on the feature set and compare the accuracy;

[0015] 8. Select the classification method for Sentinel-1 and Sentinel-2 images;

[0016] 9. Perform general post-classification processing on the classified images;

[0017] 10. Special post-classification processing methods for time series misclassification correction to optimize classification accuracy;

[0018] 11. Draw a spatiotemporal dynamic distribution map of corn harvesting progress, and count the cumulative corn harvesting area and harvesting ratio; that is, high spatiotemporal resolution monitoring of corn harvesting progress is achieved.

[0019] The present invention combines Sentinel-1 radar images and Sentinel-2 optical images to monitor the progress of corn harvesting. While shortening the monitoring period, it combines backscatter coefficient information and rich spectral information to greatly improve the accuracy of corn harvesting progress monitoring. Taking the Friendship Farm in Heilongjiang Province as an example, the present invention dynamically monitors the progress of corn harvesting in the Friendship Farm. The Sentinel-1 radar image selects the threshold segmentation method, and the Sentinel-2 optical image selects the random forest method. The overall accuracy (Kappa) can reach 0.97 (0.94) and 0.99 (0.99) respectively.

[0020] The invention plays an extremely important role in agricultural production decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 The time series misclassification correction method in Example 1;

[0022] Figure 2 This is the cumulative harvested area and harvest ratio of corn in Example 1. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0025] Specific implementation method 1: The high spatiotemporal resolution monitoring method for corn harvesting progress in this implementation method is performed according to the following steps:

[0026] 1. Determine the corn harvest time in the area to be monitored and obtain Sentinel-1 and Sentinel-2 images before and after the harvest;

[0027] 2. Preprocess the Sentinel-1 and Sentinel-2 data;

[0028] 3. Visually interpret the changes in the images before and after corn harvest and extract the sample point data;

[0029] 4. Extract and analyze the polarization characteristics of Sentinel-1 images and the spectral reflectance curves of Sentinel-2 based on the sample point data;

[0030] 5. Construct a preliminary feature set for Sentinel-1 and Sentinel-2 images;

[0031] 6. Select the feature sets of Sentinel-1 images and Sentinel-2 images to form a feature set;

[0032] 7. Use the threshold segmentation method and random forest method to classify Sentinel-1 and Sentinel-2 data based on the feature set and compare the accuracy;

[0033] 8. Select the classification method for Sentinel-1 and Sentinel-2 images;

[0034] 9. Perform general post-classification processing on the classified images;

[0035] 10. Special post-classification processing methods for time series misclassification correction to optimize classification accuracy;

[0036] 11. Draw a spatiotemporal dynamic distribution map of corn harvesting progress, and count the cumulative corn harvesting area and harvesting ratio; that is, high spatiotemporal resolution monitoring of corn harvesting progress is achieved.

[0037] Specific implementation method 2: This implementation method is different from specific implementation method 1 in that: the pre-processing method in step 2: the Sentinel-1 data is subjected to Lee filtering with a window size of 7*7 to suppress speckle noise; the Sentinel-2 data is subjected to cloud removal processing; and then the Sentinel-1 radar data and the Sentinel-2 optical data are respectively clipped according to the crop distribution data of the monitored area. The other steps and parameters are the same as those in specific implementation method 1.

[0038] Specific implementation method 3: This implementation method is different from specific implementation method 1 in that: in step 4, the polarization characteristics of the Sentinel-1 image are extracted and analyzed: VV polarization mode, VH polarization mode, VH / VV backscattering coefficient and backscattering coefficient ratio. Other steps and parameters are the same as those in specific implementation method 1.

[0039] Specific embodiment 4: This embodiment differs from specific embodiment 1 in that: the Sentinel-2 spectral reflectance curve in step 4: the spectral reflectance curves of B2 (Blue), B3 (Green), B4 (Red), B8 (NIR), B11 (SWIR1) and B12 (SWIR2) of the Sentinel-2 image. The other steps and parameters are the same as those in specific embodiment 1.

[0040] Specific implementation method 5: This implementation method is different from specific implementation method 1 in that: in step 5, a preliminary feature set is constructed: on the Sentinel-1 optical image, the spectral reflectance of B2 (Blue), B3 (Green), B4 (Red), B8 (NIR), B11 (SWIR1) and B12 (SWIR2) and the NDVI, NDTI, STI, NDI7, VTI, SVTI indexes are selected as the preliminary feature set; on the Sentinel-2 radar image, VV, VH and RI (VH / VV) are selected as the preliminary feature set. The other steps and parameters are the same as those in specific implementation method 1.

[0041] Specific implementation method 6: This implementation method is different from specific implementation method 1 in that: the selection method for forming a feature set: by drawing a histogram of the preliminary feature set and sorting the feature importance, a feature set that can distinguish corn before and after harvest is selected, which are the spectral reflectance of B2 (Blue), B3 (Green), B4 (Red), B8 (NIR), B11 (SWIR1) and B12 (SWIR2) of Sentinel-2 optical images and the NDVI, NDTI, STI, NDI7, VTI, SVTI index and the backscatter coefficient of the VH polarization mode of Sentinel-1 radar images. The other steps and parameters are the same as those in specific implementation method 1.

[0042] In this embodiment, the feature set is formed by drawing a histogram and ranking the feature importance, and a feature set that is better for distinguishing corn before harvest and corn after harvest is selected.

[0043] Specific implementation method 7: This implementation method is different from specific implementation method 1 in that: in step 7, the Sentinel-1 radar image and the Sentinel-2 optical image are classified by threshold segmentation method and random forest classification respectively, and the applicability of the two classification methods for radar images and optical images is compared. The other steps and parameters are the same as those in specific implementation method 1.

[0044] Specific implementation eight: This implementation differs from specific implementation one in that: in step eight, a classification method is selected: Sentinel-1 radar images select threshold segmentation method for corn harvesting progress monitoring, and Sentinel-2 optical images select random forest classification method for corn harvesting progress monitoring. Other steps and parameters are the same as those in specific implementation one.

[0045] Specific implementation method 9: This implementation method is different from specific implementation method 1 in that the general post-classification processing in step 9 is post-classification processing for removing small spots (Majority and Clump). Other steps and parameters are the same as those in specific implementation method 1.

[0046] Example 1 In 2021, the method of the present invention was used to monitor the corn harvesting progress at Youyi Farm in Heilongjiang Province with high temporal and spatial resolution. The specific steps are as follows:

[0047] 1. Determine the corn harvest time in the area to be monitored and obtain Sentinel-1 and Sentinel-2 images;

[0048] 2. Preprocess the Sentinel-1 and Sentinel-2 data;

[0049] 3. Visually interpret the changes in the images before and after corn harvest and extract the sample point data;

[0050] 4. Extract and analyze the polarization characteristics of Sentinel-1 images and the spectral reflectance curves of Sentinel-2 based on the sample point data;

[0051] 5. Construct a preliminary feature set such as backscatter coefficient and index for Sentinel-1 and Sentinel-2 images;

[0052] 6. Select the feature sets of Sentinel-1 and Sentinel-2 images to form the optimal feature set;

[0053] 7. Use the threshold segmentation method and random forest method to classify Sentinel-1 and Sentinel-2 data based on the feature set and compare the accuracy;

[0054] 8. Select the classification method;

[0055] 9. Carry out general post-classification processing;

[0056] 10. Special post-classification processing methods for time series misclassification correction to optimize classification accuracy;

[0057] 11. Draw a spatiotemporal dynamic distribution map of corn harvesting progress, and count the cumulative corn harvesting area and harvesting ratio; that is, high spatiotemporal resolution monitoring of corn harvesting progress is achieved.

[0058] Among them, in step 1, the harvest time of corn in Youyi Farm in Heilongjiang Province is concentrated from late September to early November. The Sentinel-1 data and Sentinel-2 data of Youyi Farm from late September to early November 2021 were obtained from the Google Earth Engine platform;

[0059] In step 2, Lee filtering is performed on the Sentinel-1 data with a window size of 7*7 to suppress speckle noise; the Sentinel-2 data is de-clouded; and then the Sentinel-1 radar data and Sentinel-2 optical data are cropped according to the crop distribution data of Friendship Farm.

[0060] In step 3, after corn is harvested, the mature corn is turned into crushed stalks, and then the soil is turned over. In the true color composite image of optical remote sensing, it will first become brighter and then darker; after corn is harvested, the brightness will become darker in radar images; based on these characteristics, visual interpretation is performed to select sample point data before and after corn harvest;

[0061] In step 4, based on the sample point data, the VV polarization mode, VH polarization mode, VH / VV backscattering coefficient and backscattering coefficient ratio of the Sentinel-1 image are extracted and analyzed, and the spectral reflectance curves of B2-Blue, B3-Green, B4-Red, B8-NIR, B11-SWIR1 and B12-SWIR2 of the Sentinel-2 image are extracted and analyzed;

[0062] In step five, through analysis, on the optical image, the spectral reflectance and NDVI, NDTI, STI, NDI7, VTI, SVTI indexes of B2-Blue, B3-Green, B4-Red, B8-NIR, B11-SWIR1 and B12-SWIR2 are selected as the primary features; on the radar image, VV, VH and RI-VH / VV are selected as the primary features; among them, the introduction of each band of Sentinel-2 is shown in Table 1, and the introduction of the spectral index formula is shown in Table 2, among which NIR is the reflectance of the near-infrared band, SWIR1 is the reflectance of the short-wave infrared 1 band, and SWIR2 is the reflectance of the short-wave infrared 2 band.

[0063] In step six, the optimal feature set for distinguishing corn before and after harvesting was selected by drawing histograms and ranking the feature importance, and finally the spectral reflectance of B2-Blue), B3-Green-, B4-Red, B8-NIR, B11-SWIR1 and B12-SWIR2 and NDVI, NDTI, STI, NDI7, VTI, SVTI index and VH were selected as the feature set;

[0064] In step seven, the threshold segmentation method and random forest classification are used to classify Sentinel-1 radar images and Sentinel-2 optical images, respectively, and the applicability of the two classification methods for radar images and optical images is compared;

[0065] In step eight, after experiments, the threshold segmentation method was selected for radar images to monitor the progress of corn harvesting, and the random forest classification method was selected for optical images to monitor the progress of corn harvesting;

[0066] In step nine, the classified image is post-processed to remove small spots (Majority and Clump);

[0067] In step 10, because the spectral reflectance is easily affected by external conditions such as the position of the sun, the position of the sensor, the geographical location, the terrain, the season, the climate change, the change of the ground humidity, the variation of the ground object itself, and the atmospheric conditions, the addition of the spectral reflectance as a feature requires further elimination of the uncertainty of its monitoring results; in order to improve the accuracy of further monitoring of the corn harvesting progress, a time series misclassification correction method is constructed to correct it; for the classification results, it is assumed that the pixel value of the harvest is 0 and the value of the pixel that is not harvested is 255; based on the classification results, 100 sample pixels are extracted and analyzed; four scenarios are obtained: all harvested, all unharvested, unharvested-harvested and harvested-unharvested, with multiple fluctuations; the correction method focuses on the fourth category; to make it one of the first three categories; because the possibility of three consecutive 0s or three consecutive 255s is small, three consecutive cycles are used as thresholds, and the following algorithm can be executed to modify the pixel value:

[0068] 1. If 0 or 255 appears three or more times in a row; when 0 appears three or more times in a row, all values ​​after 0 are assigned to 0; when 255 appears three or more times in a row, all values ​​before 255 are assigned to 255; then, continue to search for three consecutive 0s or three consecutive 255s;

[0069] 2. Based on the result of the first step, search for two consecutive 0s or two consecutive 255s again; when 0 appears twice in a row, all values ​​after 0 are assigned to 0; when 255 appears twice in a row, all values ​​before 255 are assigned to 255; then, continue to search for two consecutive 0s or two consecutive 255s;

[0070] 3. According to the result of the second step, we need to find the position where 0 appears for the first time and assign all values ​​after 0 to 0. The time series misclassification correction method is as follows: Figure 1 shown.

[0071] In step 11, ArcGIS software is used to draw a spatiotemporal dynamic distribution map of corn harvest progress, and the cumulative corn harvest area and harvest ratio are counted, such as Figure 2 shown.

[0072] Table 1 Introduction to each band of Sentinel-2

[0073]

[0074] Table 2 Introduction of spectral index

[0075]

Claims

1. A method for monitoring corn harvest progress with high temporal and spatial resolution, characterized in that The high spatiotemporal resolution monitoring method for corn harvest progress is carried out in the following steps:

1. Determine the corn harvest time in the area to be monitored and obtain Sentinel-1 and Sentinel-2 images before and after the harvest; 2. Preprocess the Sentinel-1 and Sentinel-2 data; 3. Visually interpret the changes in the images before and after corn harvest and extract the sample point data; 4. Extract and analyze the polarization characteristics of Sentinel-1 images and the spectral reflectance curves of Sentinel-2 based on the sample point data; 5. Construct a preliminary feature set for Sentinel-1 and Sentinel-2 images; 6. Select the feature sets of Sentinel-1 and Sentinel-2 images to form the optimal feature set; the selection method for forming the optimal feature set is as follows: by drawing a histogram of the preliminary feature set and sorting the feature importance, a feature set that can distinguish corn before and after harvest is selected, which are the spectral reflectance of B2-Blue, B3-Green, B4-Red, B8-NIR, B11-SWIR1 and B12-SWIR2 of Sentinel-2 optical images and NDVI, NDTI, STI, NDI7, VTI, SVTI indexes and the backscatter coefficient of the VH polarization mode of Sentinel-1 radar images, wherein the calculation method of NDVI, NDTI, STI, NDI7, VTI, SVTI is as follows: Normalized Difference Vegetation Index NDVI = (NIR - RED) / (NIR + RED); Normalized difference index of cultivated land: NDTI = (SWIR1-SWIR2) / (SWIR1+SWIR2); Simple tillage index STI = SWIR1 / SWIR2; Normalized difference index NDI7=(NIR-SWIR2) / (NIR+SWIR2); Vegetation tillage index VTI = (NDVI-NDTI) / (NDVI+NDTI); Simple vegetation tillage index SVTI = (NDVI-STI) / (NDVI+STI); 7. Use the threshold segmentation method and random forest method to classify Sentinel-1 and Sentinel-2 data based on the feature set and compare the accuracy; 8. Select the classification method for Sentinel-1 and Sentinel-2 images. For Sentinel-1 radar images, select the threshold segmentation method for corn harvesting progress monitoring. For Sentinel-2 optical images, select the random forest classification method for corn harvesting progress monitoring.

9. Post-classification processing of the classified images to remove small spots Majority and Clump; 10. Perform special post-classification processing for time series misclassification correction to optimize classification accuracy. That is, for the classification results, assume that the value of the harvested pixel is 0 and the value of the unharvested pixel is 255. Perform the following steps to modify the pixel value:

1. If 0 or 255 appears three times in a row; when 0 appears three times in a row, all values ​​after 0 are assigned to 0; when 255 appears three times in a row, all values ​​before 255 are assigned to 255; then, continue to search for three consecutive 0s or three consecutive 255s; 2. Based on the result of step 1, search for two consecutive 0s or two consecutive 255s again; when 0 appears twice in a row, all values ​​after 0 are assigned to 0; when 255 appears twice in a row, all values ​​before 255 are assigned to 255; then, continue searching for two consecutive 0s or two consecutive 255s; 3. According to the result of step 2, you need to find the first occurrence of 0 and assign all values ​​after 0 to 0; 11. Draw a spatiotemporal dynamic distribution map of corn harvesting progress, and count the cumulative corn harvesting area and harvesting ratio; that is, high spatiotemporal resolution monitoring of corn harvesting progress is achieved.

2. A method for monitoring corn harvest progress with high temporal and spatial resolution according to claim 1, characterized in that The preprocessing method in step 2 is as follows: Lee filtering is performed on the Sentinel-1 data with a window size of 7*7 to suppress speckle noise; the Sentinel-2 data is declouded; and then the Sentinel-1 radar data and Sentinel-2 optical data are cropped according to the crop distribution data of the area to be detected.

3. A method for monitoring corn harvest progress with high temporal and spatial resolution according to claim 1, characterized in that In step 4, the polarization characteristics of the Sentinel-1 image are extracted and analyzed, including the backscattering coefficient of the VV polarization mode, the backscattering coefficient of the VH polarization mode, and the ratio of the backscattering coefficients VH / VV.

4. A method for monitoring corn harvest progress with high spatiotemporal resolution according to claim 1, characterized in that In step 5, a preliminary feature set is constructed: on Sentinel-2 optical images, the spectral reflectances of B2-Blue, B3-Green, B4-Red, B8-NIR, B11-SWIR1 and B12-SWIR2 and the NDVI, NDTI, STI, NDI7, VTI and SVTI indices are selected as the preliminary feature set; on Sentinel-1 radar images, VV, VH and VH / VV are selected as the preliminary feature set.

Images