A real-time classification method for wheat phenological period based on UAV RGB images

The RGB images are obtained through the drone and the feature selection algorithm of the compaction-separation principle and mRVM classifier are used to realize real-time monitoring of the phenological period of wheat, solving the problem of monitoring lag in the existing technology, and providing efficient and accurate Field management guidance.

CN114782840BActive Publication Date: 2024-11-26NANJING AGRICULTURAL UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210414686.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-20
Publication Date
2024-11-26
Estimated Expiration
2042-04-20

AI Technical Summary

Technical Problem

The existing technology cannot monitor the key breeding periods of wheat in real time, such as jointing, flowering and grouting periods, resulting in lagging field management and timely guidance.

Method used

The drone is used to obtain RGB images, and the feature selection algorithm based on the principles of preprocessing, feature extraction and compaction-separation, combined with the mRVM classifier, real-time classification and recognition of wheat phenological period is achieved.

Benefits of technology

Real-time monitoring of wheat phenological period is achieved, cost reduction, real-time and accuracy of monitoring is improved, and timely field management guidance is provided.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114782840B_ABST
    Figure CN114782840B_ABST
Patent Text Reader

Abstract

The present invention proposes a real-time classification method for wheat phenological period based on unmanned aerial vehicle RGB images, and the steps are as follows: (1) according to the actual growth of wheat fields treated with different sowing periods, a time series high spatial resolution RGB image is obtained and preprocessed to obtain unmanned aerial vehicle images of the same area in different years; (2) the spectral information and texture information of the time series unmanned aerial vehicle images are extracted, and all the spectral features and texture features derived are used as the full set of features; (3) the feature selection algorithm based on the compact separation principle is used to sort the importance of all features, and the optimal features and the number of features are determined; (4) the mRVM classifier is applied to automatically classify and identify the features of different phenological stages, and the overall classification accuracy and the classification accuracy of each period are obtained. The classification method constructed by the present invention is simple and efficient, and can obtain timely crop phenological information, providing a basis for effectively guiding agricultural management decisions, such as irrigation, fertilization and pesticide management activities at a specific stage.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of precision agriculture, and mainly relates to a wheat phenological period monitoring method, in particular to a wheat phenological period classification method based on RGB images and machine learning. Background Art

[0002] Accurate information related to farmland management, including crop water consumption, phenology, and yield data, is essential for managing crop growth under sustainable and precise farming methods. One aspect of farmland management involves information on crop phenology, which is one of the most important applications in agronomy. Phenology refers to seasonal biological life stages driven by environmental factors and is considered a sensitive and accurate indicator of climate change. In China, where climate warming has accelerated since the 1980s, affecting crop development and productivity, crop phenology monitoring can be used to measure climate change. Second, changes in phenology have the potential to widely affect terrestrial ecosystems and human society, for example, by changing global carbon, water, and nitrogen cycles, crop yields (risk of frost damage), duration of the pollination season, and diseases. Therefore, phenology research has also become an important focus of recent ecological research. Wheat is an important food crop in the world, with the largest planting area among all crops and the third largest food crop in my country. Under the new situation, food security has become the core of the national food policy, and the status of wheat in food security will become increasingly prominent. In view of this, evaluating wheat phenology is of great significance for steadily improving my country's wheat production capacity and ensuring food security.

[0003] At present, the threshold method, maximum slope method and curve fitting method based on optical remote sensing have been used to effectively detect crop phenology. However, these methods can only detect the beginning of growth, the peak of greening and the end of the season based on the inflection point, so they cannot detect other key stages closely related to agricultural practice (such as jointing period, flowering period and filling period, etc.). Secondly, the empirical model and curve fitting method based on the time series vegetation index curve are sensitive to environmental noise. The above methods still have the most unavoidable problem, that is, the phenological period can only be monitored from historical data at the end of the entire growth period, which has a lag and cannot provide timely guidance for field management. Therefore, in order to overcome this bottleneck of the phenological period, real-time monitoring is particularly important.

[0004] In recent years, the rapid development of UAV technology has provided favorable conditions for obtaining real-time high-precision remote sensing image data. Using digital cameras to shoot in multiple phases to observe plant phenology not only saves human resources, but also provides high-frequency and high-spatial-resolution phenology data. It is also suitable for complex farmland environments and has high technical efficiency and low cost in the rapid analysis of field crop phenotypic information. In this case, UAVs equipped with digital cameras are an effective tool for crop phenology monitoring. Summary of the invention

[0005] The purpose of the present invention is to provide a real-time classification method for wheat phenological period based on UAV RGB images, which realizes real-time discrimination of phenological period by using the spectral features after high-resolution image screening and advanced classification algorithms. This method proves that UAVs can effectively and timely monitor the dynamic response of wheat to phenology and identify phenological stages, which has important guiding significance for the implementation of field water and fertilizer management and crop management.

[0006] To achieve the above purpose, the technical solution adopted by the present invention is: a real-time classification method of wheat phenological period based on drone RGB images, which splices the original RGB images obtained by the drone, and extracts the spectrum and texture features of different growth stages after preprocessing such as radiation correction and geometric correction, and then uses a feature selection algorithm based on the compactness-separation principle to measure the importance of each feature, and then determine the optimal feature and the number of features. The mRVM classifier is used to realize the classification of eight important growth stages of wheat (seedling stage, tillering stage, jointing stage, booting stage, heading stage, flowering stage, filling stage and maturity stage); the specific steps are as follows:

[0007] Step 1: According to the actual growth conditions of wheat fields treated at different sowing periods, obtain time-series high-spatial-resolution RGB images and preprocess them to obtain drone images of the same area from different years;

[0008] Step 2: Extract the spectral information and texture information of the time-series UAV image, and derive all spectral features and texture features as the full feature set;

[0009] Step 3: The feature selection algorithm based on the compactness-separation principle ranks the importance of all features and determines the optimal features and number of features;

[0010] Step 4: Apply the mRVM classifier to automatically classify and identify the characteristics of different phenological stages to obtain the overall classification accuracy and classification accuracy for each period.

[0011] Furthermore, in step 1, the specific steps of pre-processing include:

[0012] Step 1-1: stitch the single high spatial resolution images of the same period to obtain the orthophoto of the complete test area;

[0013] Step 1-2: Georeference and radiometrically calibrate the orthophotos of each period to obtain the drone spectral reflectance data.

[0014] Furthermore, the ground resolution of the acquired image data is 2 to 3 cm. After the images are stitched, the first image is selected as the reference image, and all the remaining images are sequentially registered with the reference image to reduce the coordinate error caused by environmental conditions in the actual shooting.

[0015] Furthermore, in step 2, the specific steps of generating the full set of features include:

[0016] Step 2-1: Calculate the color spectral index based on the reflectance of each band after preprocessing, including R cc , G cc , GRVI, VARI and ExG, as well as the original band reflectance data, as the full set of spectral features;

[0017] Step 2-2: Calculate the gray level co-occurrence matrix of the R channel, G channel and B channel of the corrected image respectively;

[0018] Step 2-3: Calculate the texture features under different gray-level co-occurrence matrices with a window of 3*3 and a motion direction of D1 (0°), including mean (MEA), variance (VAR), homogeneity (HOM), contrast (CON), angular second moment (SEC), correlation (COR), heterogeneity (DIS) and entropy (ENT), and then calculate the normalized texture index between each pair of texture features. All the above features are taken as the complete set of texture features.

[0019] Furthermore, in step 3, the specific steps of determining the optimal features and the number of features include:

[0020] Step 3-1: Taking spectral features and texture features as input data;

[0021] Step 3-2: Apply feature selection algorithm, calculate the compactness-separation coefficient for each feature, obtain the importance score of each feature and sort it;

[0022] Step 3-3: Determine the optimal number of features with the ultimate goal of obtaining the highest classification accuracy.

[0023] Furthermore, in step 3-2, the calculation formula of the compaction-separation coefficient is:

[0024]

[0025]

[0026] ρ cs (k) = αρ s (k)-(1-α)ρ c (k)

[0027] Among them, the compactness coefficient ρ cand the separation coefficient ρ s They are used to measure the compactness between each stage and the separation between different stages. Given a set of training samples, x i ∈R N , n is the number of training samples, N is the number of features; phenological stage category labeling Y = {y 1 ,y 2 ,…,y M}, M is the number of phenological stages, y i is the training sample set belonging to the i-th stage; the number of training samples in each stage is {β i},i=1,2,…,M;ρ cs is the compactness-separation coefficient, α is a factor from 0 to 1, which can adjust ρ c and ρ s The relationship between the characteristics of cs The higher the value, the more discriminative the feature is for phenological stages.

[0028] Furthermore, in step 4, the specific steps of obtaining the classification result include:

[0029] Step 4-1: Take the optimal feature subset and growth period labels of all sample points as input data;

[0030] Step 4-2: Use the mRVM classifier to construct a wheat phenological period classification model, and use the Gaussian kernel function as the kernel; the Gaussian kernel function parameter of the mRVM model is set to 1 / D, where D is the number of features; the verification method uses a ten-fold cross-validation method, and finally the optimal overall classification accuracy and classification accuracy for each period are obtained and saved.

[0031] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0032] (1) The real-time classification and identification method of wheat phenological period based on UAV RGB images of the present invention is low-cost, simple to operate, efficient, and can be automated and easily promoted.

[0033] (2) The real-time classification and identification method of wheat phenological period based on UAV RGB images of the present invention overcomes the lag of previous growth period monitoring and realizes the real-time discrimination of phenological stage;

[0034] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1Schematic diagram of the experimental design of the present invention; a two-year wheat experiment was conducted in the Grain Industry Park of Xinghua City, Taizhou City, Jiangsu Province in 2017-2018 and 2018-2019. The experiment involved different sowing dates (2017-2018: sowing date one: November 6, sowing date two: November 19, sowing date three: December 3; 2018-2019: sowing date one: November 2, sowing date two: November 20, sowing date three: November 30), nitrogen levels (N1: 150 kg / hectare, N2: 300 kg / hectare) and varieties (C1: Yangmai 23, C2: Ningmai 13). A total of 12 plots, each with an area of ​​2700m in the first year. 2 (90m×30m), 1350m in the second year 2 (45m×30m).

[0036] Figure 2 It is a schematic diagram of the real-time classification method of wheat phenological period based on unmanned aerial vehicle RGB images of the present invention.

[0037] Figure 3 It is the spatial distribution map of the original RGB image of the UAV and the changes of spectral and texture characteristics with the phenological period; taking the Yangmai 23 variety field under the treatment of high nitrogen level in the sowing period of 2017-2018 as an example; a1-a8 are the original RGB images, b1-b8 are the spectral characteristics (GRVI), c1-c8 are the texture characteristics (NDTI (CON G ,SEC G )).

[0038] Figure 4 The response law of the spectral and texture characteristics derived from the UAV to the phenological period; (a) is the spectral characteristic (GRVI), (b) is the texture characteristic (NDTI (CON G ,SEC G )).

[0039] Figure 5 is the score of all features generated by the feature selection algorithm based on the compactness-separation principle of the present invention; the black part is the spectral feature, the gray part is the texture feature, a total of 308. The score is located at the dotted line (ρ cs =0.5560) or above are the optimal features.

[0040] Figure 6 This is a confusion matrix result diagram of the real-time classification of wheat phenological period based on UAV RGB images of the present invention.

[0041] Figure 7 It is a comparison of the phenological classification results based on different feature inputs of the present invention.

[0042] Figure 8In the figure: (a) is a graph of the overall classification accuracy with the number of feature inputs under different parameter values ​​in the feature selection algorithm of the present invention, which is used to determine the optimal number of features; the top dot represents the fastest optimal estimated accuracy value reached based on different values ​​of parameter α among the first 60 features. (b) is a graph of the overall classification accuracy with the number of feature inputs obtained by other feature selection methods (mRMR, ReliefF, RFE, RF_GINI) and the feature selection method of the present invention. DETAILED DESCRIPTION

[0043] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0044] The present invention takes wheat planting area as an example, and the research area is as follows Figure 1 As shown in the figure, the drone data used is the drone image of the study area obtained by the RGB camera (FC300X, Shenzhen, China) carried by DJI Phantom 3 (Shenzhen, China). The drone automatically sets the flight route, with a field of view of 94°, a flight altitude of 60 to 70 meters, a flight time of 5 to 20 minutes, and fixed-point hovering shooting. The lateral repetition rate and heading repetition rate are 82% and 75% respectively.

[0045] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0046] Figure 2 The present invention is a flow chart of a method for real-time monitoring of wheat phenological period based on UAV RGB images. The method may include the following steps:

[0047] Step 1: Acquire high-spatial-resolution RGB images of the time series during the entire growth period of wheat and perform preprocessing.

[0048] Step 1-1: stitch the single high spatial resolution images of the same period to obtain the orthophoto of the complete test area;

[0049] Step 1-2: Georeference the orthophotos of each period to obtain the drone spectral reflectance data;

[0050] It should be added that the ground resolution of the acquired image data is 2 to 3 cm. After the images are stitched, the first image is selected as the reference image, and all the remaining images are aligned with the reference image in turn to reduce the coordinate error caused by environmental conditions in the actual shooting.

[0051] Step 2: Extract the spectral information and texture information of the time-series UAV image, and derive all spectral features and texture features as the full feature set. The specific steps include:

[0052] Step 2-1: Calculate the color spectral index based on the corrected reflectance of each band, including R cc , G cc , GRVI, VARI and ExG, as well as the original band reflectance data, as the full set of spectral features;

[0053] Step 2-2: Calculate the gray level co-occurrence matrix of the R channel, G channel and B channel of the corrected image respectively;

[0054] Step 2-3: Calculate the texture features under different gray-level co-occurrence matrices with a window size of 3*3 and a moving direction of D1 (0°), including mean (MEA), variance (VAR), homogeneity (HOM), contrast (CON), angular second-order moment (SEC), correlation (COR), heterogeneity (DIS) and entropy (ENT), and then calculate the normalized texture index (NDTI) between each pair of texture features. All the above features are taken as the complete set of texture features;

[0055] It should be added that ENVI / IDL software is used to extract the original DN value of each region of interest (ROI) in the digital image and to calculate the spectral features. The calculation method of the spectral features is shown in Table 1.

[0056] Table 1 Spectral characteristics obtained based on UAV RGB images

[0057]

[0058] The gray-level co-occurrence matrix involves three important parameters, namely, the moving step size, the moving direction, and the window size. The window size is related to the image resolution. A smaller window represents more subtle texture features, and a larger window can represent coarse texture features. The moving step size depends on the texture coarseness of the image. There are four moving directions, namely 0°, 45°, 90°, and 135°. The texture features extracted perpendicular to the planting row direction have been proven to be effective in monitoring the growth parameters of row-sown crops. In the present invention, the experimental fields are planted in vertical rows in the north-south direction, so only the texture features in the D1 direction are extracted, and the calculation method of the texture features is shown in Table 2. In addition, the normalized texture index (NDTI) can maximize the sensitivity to phenology, and the present invention obtains a total of 276 NDTIs. A total of 300 texture features.

[0059] Table 2 Texture features obtained based on UAV RGB images

[0060]

[0061] Step 3: The feature selection algorithm based on the compactness-separation principle ranks the importance of all features and determines the optimal features and number of features. The specific steps include:

[0062] Step 3-1: Take the spectral features and texture features as input data and normalize them;

[0063] Step 3-2: Apply feature selection algorithm, calculate the compactness-separation coefficient for each feature, obtain the importance score of each feature and sort it;

[0064] It should be added that the specific calculation process of the compaction-separation coefficient is:

[0065]

[0066]

[0067] ρ cs (k) = αρ s (k)-(1-α)ρ c (k)

[0068] Among them, the compactness coefficient ρ c and the separation coefficient ρ s They are used to measure the compactness between each stage and the separation between different stages. Given a set of training samples, x i ∈R N (where n is the number of training samples and N is the number of features); phenological stage category labeling Y = {y 1 ,y 2 ,…,y M ](where M is the number of phenological stages, y i is the training sample set belonging to the i-th stage); the number of training samples in each stage is {β i},i=1,2,…,M;ρ cs is the compactness-separation coefficient, α is a factor from 0 to 1, which can adjust ρ c and ρ s The relationship between α and α in the present invention is 0.6. cs The higher the value, the more discriminative the feature is for phenological stages.

[0069] Step 3-3: Determine the optimal number of features with the ultimate goal of obtaining the highest classification accuracy.

[0070] Step 4: Apply the mRVM classifier to automatically classify and identify the characteristics of different phenological stages to obtain the overall classification accuracy and classification accuracy for each period. The specific steps include:

[0071] Step 4-1: Take the optimal feature subset and growth period labels of all sample points as input data;

[0072] Step 4-2: Use the mRVM classifier to train and validate the model, and use the Gaussian kernel function as the kernel. The Gaussian kernel function parameter of the mRVM model is set to 1 / D, where D is the number of features. The validation method uses a ten-fold cross-validation method, and finally obtains the optimal overall classification accuracy and classification accuracy for each period and saves them.

[0073] The specific feature screening results of this embodiment are as follows Figure 5 As shown in Table 3:

[0074] Table 3 Optimal feature selection results

[0075]

[0076] Both texture features and spectral features can play a role in the classification of wheat phenological stages. Finally, the first 13 features were selected as the optimal features. The following is a comparison of the performance of the feature selection method FS-CS proposed in the present invention with the mRMR, ReliefF, RFE, and RF_GINI feature selection methods in screening the optimal features sensitive to phenological periods. Figure 8 (b) The above feature selection methods are summarized to evaluate the wheat phenological stage. When the feature input is the least, the feature selection method of the present invention can obtain the highest classification accuracy.

[0077] The wheat phenological stage classification results of this example are as follows Figure 6 As shown, it can be seen from the confusion matrix that mRVM can well classify the seedling stage, tillering stage, filling stage and maturity stage. The following compares the performance of the classification algorithm mRVM proposed in the present invention with the BN, KNN and SVM algorithms in the classification of wheat phenological stages based on UAV RGB images. The comparison of the classification accuracy and overall accuracy of the eight important phenological stages obtained by the prior art and the method of the present invention is shown in Table 4. It can be seen from Table 4 that the overall classification accuracy obtained by the method of the present invention is the highest. The method of the present invention can improve the accuracy of wheat phenological monitoring and provide a basis for management decisions for large-scale fields in a timely manner.

[0078] Table 4 Comparison of classification results of different classifiers

[0079]

[0080] The above descriptions are only some embodiments of the present invention. It should be pointed out that a person skilled in the art can make several improvements without departing from the principles of the present invention. These improvements should be regarded as within the protection scope of the present invention.

[0081] The parts not involved in the present invention are the same as the prior art or can be implemented by using the prior art.

Claims

1. A real-time classification method for wheat phenological period based on drone RGB images, characterized in that: The following steps are involved: Step 1: According to the actual growth conditions of wheat fields treated at different sowing dates, high-spatial-resolution RGB images were acquired and preprocessed to obtain UAV images of the same area from different years; Step 2: Extract the spectral information and texture information of the time-series UAV image, and derive all spectral features and texture features as the full feature set; The specific steps to generate the full set of features include: Step 2-1: Calculate the color spectral index based on the reflectance of each band after preprocessing, including R cc , G cc , GRVI, VARI and ExG, as well as the original band reflectance data, as the full set of spectral features, where R cc is the relative redness index, G cc is the relative greenness index, GRVI is the red-green vegetation index, VARI is the visible light atmospheric impedance index, and ExG is the absolute greenness index; Step 2-2: Calculate the gray level co-occurrence matrix of the R channel, G channel and B channel of the corrected image respectively; Step 2-3: Calculate the texture features of different gray-level co-occurrence matrices under the window of 3*3 and the motion direction of D1 at 0°, including mean (MEA), variance (VAR), homogeneity (HOM), contrast (CON), angular second moment (SEC), correlation (COR), heterogeneity (DIS) and entropy (ENT), and then calculate the normalized texture index between the two texture features. All the above texture features are taken as the complete set of texture features; Step 3: The feature selection algorithm based on the compactness-separation principle ranks the importance of all features and determines the optimal features and number of features; the specific steps include: Step 3-1: Taking spectral features and texture features as input data; Step 3-2: Apply the feature selection algorithm to calculate the compactness-separation coefficient feature by feature, obtain the importance score of each feature and sort it. The calculation formula of the compactness-separation coefficient is: r cs (k)=number s (k)-(1-a)p c (k) Among them, the compactness coefficient ρ c and the separation coefficient ρ s They are used to measure the compactness between each stage and the separation between different stages. Given a set of training samples, n is the number of training samples, N is the number of features; phenological stage category labeling Y = {y 1 ,y 2 ,…,y M }, M is the number of phenological stages, y i is the training sample set belonging to the i-th stage; the number of training samples in each stage is {β i },i=1,2,…,M;ρ cs is the compactness-separation coefficient, α is a factor from 0 to 1, which can adjust ρ c and ρ s The relationship between the characteristics of cs The higher the value, the greater the discrimination of the trait for phenological stages; Step 3-3: Determine the optimal number of features with the ultimate goal of obtaining the highest classification accuracy; Step 4: Apply the mRVM classifier to automatically classify and identify the characteristics of different phenological stages to obtain the overall classification accuracy and classification accuracy for each period; the specific steps include: Step 4-1: Take the optimal feature subset and growth period labels of all sample points as input data; Step 4-2: Use the mRVM classifier to construct a wheat phenological period classification model, and use the Gaussian kernel function as the kernel; the Gaussian kernel function parameter of the mRVM model is set to 1 / D, where D is the number of features; the verification method uses a ten-fold cross-validation method, and finally the optimal overall classification accuracy and classification accuracy for each period are obtained and saved.

2. The real-time classification method for wheat phenological period based on drone RGB images according to claim 1 is characterized in that: In step 1, the specific steps of pretreatment include: Step 1-1: stitch the single high spatial resolution RGB images of the same period to obtain the orthophoto of the complete test area; Step 1-2: Georeference and radiometrically calibrate the orthophotos of each period to obtain the drone spectral reflectance data.

3. The real-time classification method of wheat phenological period based on drone RGB images according to claim 2 is characterized in that: The ground resolution of the acquired high spatial resolution RGB images is 2 to 3 cm. After the images are stitched, the first image is selected as the reference image, and all the remaining images are aligned with the reference image in turn to reduce the coordinate error caused by environmental conditions in actual shooting.