Wheat breeding material growth period intra-season classification and estimation method in combination with time sequence RGB image and machine learning

Abstract

The invention provides a wheat breeding material growth period intra-season classification and estimation method in combination with a time sequence RGB image and machine learning, which comprises the following steps: (1) according to wheat breeding material plot tests in different growth seasons and different ecological points, obtaining a time sequence high spatial resolution RGB image and preprocessing the image, constructing a sample library of a large number of germplasm resources in different growth periods; (2) carrying out growth period stage classification by fusing plant height information and image information to realize high-precision classification of growth stages; (3) constructing an estimation model of the starting date of the key growth period based on the time sequence prediction probability obtained after classification; and (4) determining an optimal data acquisition interval for monitoring the growth period of the breeding material. The method is efficient, timely crop phenological information can be obtained, planting management can be guided, variety selection can be optimized, the yield and quality can be improved, and scientific basis and technical support are provided for wheat production.

Description

Technical Field

[0001] The present invention belongs to the technical field of precision agriculture, and mainly relates to a growth period monitoring method for wheat breeding materials, and specifically to an intra-season classification and estimation method for the growth period of wheat breeding materials combining time series RGB images and machine learning. Background Art

[0002] The growth period refers to the time period from sowing to maturity of wheat, which is of great significance for wheat cultivation and breeding. Accurate monitoring of the growth period of wheat is conducive to efficient planting management, and scientific planting management measures can be formulated to ensure that the needs of wheat at various growth stages are met and increase yield and quality. Secondly, by studying the length and adaptability of the growth period of different varieties, the impact of adversity can be avoided. Reasonable arrangement of the length of the growth period can reduce the impact of wheat on pests and diseases, drought, high temperature, etc. Finally, the monitoring and research of the wheat growth period can also optimize the variety mating and breeding strategies, select excellent varieties with shorter growth periods, high yields and high quality, and provide reference and basis for accelerating the wheat breeding process. In short, the growth period is of great significance to wheat breeding. It can guide planting management, optimize variety selection, and improve yield and quality, providing scientific basis and technical support for wheat production.

[0003] Currently, most of them are developed based on satellite platforms, and usually focus on the overall growth process of crops in the region, ignoring the local characteristics and details of small-field breeding plots at different growth stages. Low-altitude remote sensing or close-range sensing of unmanned aerial vehicles (UAVs) provides new opportunities for the growth period monitoring of small-field breeding plots. In recent years, drone-based growth period monitoring has also made significant progress, but most of them still use regression models based on empirical methods. The more mature empirical methods are mainly aimed at modern single or multiple cultivars, whose growth period processes are relatively consistent, and crop growth laws are easier to explore. However, for some breeding materials that have not experienced the green revolution, empirical regression methods are difficult to apply. With the deepening of research, researchers continue to explore the deep features of high-resolution images to reduce the impact of canopy heterogeneity on image analysis. More accurate extraction of detailed features that reflect the differences between different growth periods is very important for the construction of a robust model of the growth period of breeding materials. The emergence of deep learning technology provides a powerful tool for near-real-time monitoring of the growth period of breeding materials and even in the field of crop phenotyping. Compared with the results of growth period discrimination using spectral information and texture features, the application of deep learning has greatly improved the monitoring accuracy. However, research based on image depth features has always focused on the two-dimensional information of the crop canopy, while the differences in the growth period are not only reflected in the canopy differences, but also in the three-dimensional height features. After the crop enters the jointing stage, the internodes of the stem begin to elongate rapidly upward, so one of the important characteristics of the jointing stage is the increase in plant height. When entering the heading stage, the plant height tends to stabilize. Plant height information can also be used as another feature for classification of the pre-growth period before the flowering period. However, there is currently no research on the classification of the pre-growth period based on three-dimensional information, and there is also a lack of research on growth period monitoring that integrates three-dimensional height features and two-dimensional image features. The monitoring effect of the growth period after integration is also worth exploring.

[0004] At the same time, the ultimate goal of monitoring the growth period in breeding work is to obtain the starting point of each growth period. Real-time classification of the growth period based on deep learning alone cannot achieve quantitative monitoring of the starting date, and few studies have applied deep learning classification algorithms to achieve quantitative monitoring. In this context, high-precision identification of the growth period of wheat breeding materials and monitoring of their starting dates based on drone RGB images are of great significance to breeders. Summary of the invention

[0005] The purpose of the present invention is to provide a high-throughput, high-precision monitoring method for the growing period of wheat breeding materials within the season, to construct an integrated algorithm that fuses height information and image information, which can quickly identify the main growing stages of wheat breeding plots, and to establish a quantitative monitoring model for the starting date of the key growing period, thereby providing technical support for variety selection and smart breeding.

[0006] To achieve the above object, the technical solution adopted by the present invention is: a method for intra-season classification and estimation of the growth period of wheat breeding materials combining time series RGB images and machine learning, which comprises the following steps:

[0007] Step 1: Based on the wheat breeding material plot tests in different growing seasons and ecological points, obtain time-series high-spatial-resolution RGB images and perform preprocessing to build a sample library of a large number of germplasm resources at different growth stages;

[0008] Step 2: Classification of growth stages by integrating plant height information and image information to achieve high-precision classification of growth stages;

[0009] Step 3: Based on the time series prediction probability obtained after classification, an estimation model for the start date of the key growth period is constructed;

[0010] Step 4: Determine the optimal data collection interval for monitoring the growth period of breeding materials.

[0011] Furthermore, in step 1, the specific steps of pre-processing include:

[0012] Step 1-1, image stitching and registration: Agisoft PhotoScan 1.2.6 software (Agisoft LLC, ST, Petersburg, Russia) was used to stitch the RGB images. During the stitching process, the longitude and latitude information of GCPs was imported for image geo-registration to reduce the coordinate error caused by environmental conditions during the actual shooting process. After the stitching project was completed, the orthophoto map (DOM) and digital surface model (DSM) were generated.

[0013] Step 1-2, plot boundary extraction: The plot boundary is plotted using the GRID python package. First, the orthophotos of different ecological points at the jointing period are used for image binarization to distinguish the plots and soil after the row closure. Then, the center point of the field is determined based on a mathematical algorithm, and the appropriate boundary length and width are input to generate a plot boundary shapefile with coordinate information.

[0014] Steps 1-3, batch masking of plots: Use the plot boundary shapefile generated by GRID for masking, build a sample library grouped by growth period, and mark the current phenological period information and plot number information of each field when the image acquisition time is the same.

[0015] Furthermore, in step 1, drone image acquisition is performed after wheat seedlings emerge, drone image acquisition is performed every 1-3 days during the transition period of the growth period, and once every 3-6 days in the rest of the period. Try to select the time period of 10:00 and 12:00 local time on a sunny day to obtain, the flight altitude of all flight missions is 30 meters, the flight time is about 20 minutes, the speed is 2.5m / s, the lateral and forward overlaps are 80% and 78% respectively, and the ground sampling resolution of the RGB image is 1.37 cm per pixel. Ground control points (GCPs) are evenly distributed in the field for geometric correction.

[0016] Furthermore, the specific steps of step 2 include: growth stage classification based on three-dimensional height features and KNN, growth stage classification based on the minimalist neural network VanillaNet-13, and growth stage classification using classifier integration.

[0017] Furthermore, the growth stage classification based on three-dimensional height features and KNN: Growth stage classification based on three-dimensional height features and KNN. Based on the high-overlap RGB images of drones, the running recovery structure algorithm is used to obtain the dense point cloud data of wheat, thereby generating the wheat crop surface model DSM; a drone operation is carried out after wheat sowing at each ecological point to obtain the digital terrain model (Digital terrain model, DTM) of the study area; the crop surface model is subtracted from the digital surface model obtained after sowing to obtain the crop canopy height model (Crop height model, CHM); the plant height of each stage is divided by the maximum plant height to obtain the relative plant height; the KNN (k nearest neighbor) clustering algorithm is used to classify the relative plant height into three growth stages: tillering stage (TS), jointing and booting stage (JBS), and heading to maturity stage (HMS). The KNN algorithm determines the category to which the sample data to be classified belongs, and outputs the predicted probability of each sample in the three categories, recorded as KNN TS、 KNN JBS and KNN HMS .

[0018] Furthermore, the classification of growth stages based on the minimalist neural network VanillaNet-13: The classification model of growth stages is constructed using the minimalist neural network VanillaNet-13, which is divided into tillering stage (TS), jointing and booting stage (JBS), heading stage (HS), anthesis and filling stage (AFS) and maturity stage (MS), and the predicted category and predicted probability of each sample are generated, and the predicted probabilities are recorded as CNN TS ,CNN JBS ,CNN HS ,CNN AFS and CNN MS; In order to evaluate the performance of the classification model, the true positive (TP), true negative (TN), false positive (FP) and false negative (FN) rates are calculated uniformly, and then the accuracy (Accuracy), precision (Precision), recall (Recall) and F1 score (F1-score) of the test data set are obtained according to the following formulas. In this classification, in terms of image preprocessing and enhancement, random 10-degree rotation, Gaussian blur, changes in color saturation and brightness, Mixup enhancement methods are added, and resizing and normalization are performed; the calculation formulas for accuracy, precision, recall and F1 score are:

[0019]

[0020] Furthermore, the classification of the growth stage of the classifier ensemble: the two classification results are integrated to obtain higher classification accuracy. The prediction probabilities of the two classifiers in the same stage TS and JBS are integrated, and the prediction probability of the samples classified as HMS by KNN is determined by CNN. The integration method is to use the normalized values ​​of the F1 scores of the two classifiers in the categories TS and JBS as weighted weights to combine the prediction probabilities, and obtain the prediction probability after the integration of the categories TS and JBS. TS and Ensemble JBS , Ensemble HS 、Ensemble AFS and Ensemble MS It is still based on the classification result of CNN. Based on the prediction probabilities of the five categories after the final integration, the final prediction category of the sample is determined as the category with the maximum prediction probability, and finally the accuracy, precision, recall and F1 score of the test data set are obtained. The calculation formula of the integration method is:

[0021]

[0022] Ensemble_Pj=CNN_Pj (6)

[0023] i is the first two growth stages TS and JBS, j is the last three growth stages HS, AFS and MS, are the weight coefficients of KNN and CNN in stage i respectively.

[0024] Furthermore, the step 3 specifically includes:

[0025] Step 3-1: Take the time series prediction probability and start date of each wheat breeding material dynamics as input data;

[0026] Step 3-2: Apply random forest to estimate the start date of the reproductive period and predict the probability of time series Ensemble JBS 、Ensemble HS 、Ensemble AFS and Ensemble MS They were used as feature inputs for the estimation models of jointing, heading, flowering and maturity to obtain the estimation results of each start date;

[0027] Step 3-3: Calculate R for the observed and estimated values ​​for each starting date 2 , RMSE and relative RMSE are used to evaluate the performance of model estimation.

[0028] Furthermore, the step 4 specifically includes:

[0029] Step 4-1: Divide the total number of days in the wheat season by the number of drones flown to obtain the average time window;

[0030] Step 4-2: Reduce the number of flights in sequence, gradually extend the time window, and estimate the monitoring accuracy of different growth stages;

[0031] Step 4-2: By estimating the monitoring accuracy of the growth period under different time windows, determine the maximum time window while keeping the monitoring accuracy of the growth period as unchanged as possible, that is, the optimal time window.

[0032] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0033] (1) The high-precision monitoring method for the growing period of wheat breeding materials based on drone RGB images of the present invention is low-cost, high-precision, simple and efficient to operate, and can be automated and easily promoted.

[0034] (2) The high-precision intra-season monitoring method of the growth period of wheat breeding materials based on drone RGB images of the present invention overcomes the lag of previous growth period monitoring and realizes real-time identification of the growth stage;

[0035] (3) The high-precision intra-season monitoring method of the growth period of wheat breeding materials based on drone RGB images of the present invention realizes quantitative monitoring of the start date of the key growth period, which makes up for the shortcoming that the start date of the growth period cannot be quantified based on the deep learning classification algorithm;

[0036] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1: It is a schematic diagram of the experimental design of the present invention; Experiment 1 was conducted at the Quzhou Experimental Station of China Agricultural University in Handan City, Hebei Province, China (36°52′8″N, 115°1′23″E) from 2021 to 2022, and was recorded as HB. The total number of breeding plots was 628, and no repetition was set. The length and width of the plot where each material was located were 2.0m and 1.8m respectively, with 5 rows planted, the row spacing was 25cm, and 180 seeds were manually sown at equal intervals in each row. The interval between plots was 30cm, covering a total area of ​​about 5 mu. Sowing was carried out on November 4, 2021, and harvesting began in batches on July 20, 2022. The application of nitrogen, phosphorus and potassium fertilizers was carried out according to local conventional fertilizer treatment. Experiment 2 was conducted at the Experimental Comprehensive Base of the Chinese Academy of Agricultural Sciences in Shenzhen, Guangdong Province, China (22°35′50″N, 114°30′6″E) from 2021 to 2022, and was recorded as GD. The wheat population, plot size and sowing amount were consistent with Experiment 1, and there was no repeated treatment. Sowing was carried out on November 17, 2021, and harvesting began in batches on May 20, 2022. The application of nitrogen, phosphorus and potassium fertilizers followed the local conventional fertilizer treatment. Experiment 3 was conducted in 2022-2023 at the National Information Agriculture Engineering Technology Center, Rugao City, Nantong City, Jiangsu Province, China (32°15′51″N, 120°46′1″E), denoted as JS. There was no repeated treatment for the Watkins material, and there were repeated treatments for the modern species. A total of 234 plots were planted with modern species. The plot size was consistent with the previous two experiments, with a reduced sowing density, 50 seeds per row, and a plant spacing of 3 cm, covering an area of ​​about 6 mu. Sowing was carried out on November 3 and 4, 2022, and harvesting began in batches on July 20, 2023. Conventional fertilizer treatments of 220 kg / ha of nitrogen fertilizer, 60 kg / ha of phosphorus fertilizer, and 90 kg / ha of potassium fertilizer were applied before planting.

[0038] Figure 2 It is a schematic diagram of the intra-season classification method of the growth period of wheat breeding materials based on unmanned aerial vehicle RGB images of the present invention.

[0039] Figure 3 It is a framework diagram of the classification algorithm VanillaNet-13 based on two-dimensional deep features of images of the present invention.

[0040] Figure 4 This is a schematic diagram of the present invention for constructing a key growth period start date estimation model based on time series prediction probability. J ,ID H ,ID A , and ID M They represent the starting dates of the jointing stage, heading stage, flowering stage and maturity stage respectively.

[0041] Figure 5 (a) is the determination of the parameter K value in the KNN algorithm of the present invention, and (b) is the classification confusion matrix based on KNN and plant height information.

[0042] Figure 6 It is the classification confusion matrix based on Vanilinet-13 and image deep features of the present invention.

[0043] Figure 7 It is the classification confusion matrix based on classifier integration of the present invention.

[0044] Figure 8 It is the RF model estimation result of the start date of the critical growth period of the present invention.

[0045] Fig. 9 It is a spatial distribution diagram of the start date of the key growth period of the present invention.

[0046] Fig.10 It is a comparison of the error days of estimating the start date of the growth period under different time windows of the present invention. DETAILED DESCRIPTION

[0047] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0048] The present invention takes the wheat breeding material planting area as an example, and the research area is as follows Figure 1 As shown in the figure, the drone data used is the drone image of the study area obtained by the Zenmuse P1 camera (Zenmuse P1, Shenzhen, China) equipped with DJI Matrice 300RTK (Shenzhen, China). The drone image collection was carried out after the wheat seedlings emerged, every 1-3 days during the transition period of the growth period, and every 3-6 days in the rest of the period. The acquisition was tried to choose the time period of 10:00 and 12:00 local time on sunny days. The flight altitude of all flight missions was 30 meters, the flight time was about 20 minutes, the speed was 2.5m / s, the lateral and forward overlaps were 80% and 78% respectively, and the ground sampling resolution of the RGB image was 1.37 cm per pixel. The ground control points (GCPs) were evenly distributed in the field for geometric correction.

[0049] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0050] Figure 2 The figure is a schematic diagram of a method for intra-season classification of wheat breeding materials during the growth period based on drone RGB images of the present invention. The method may include the following steps:

[0051] Step 1: Based on the wheat breeding material plot tests in different growing seasons and ecological points, obtain time-series high-spatial-resolution RGB images and perform preprocessing to build a sample library of a large number of germplasm resources at different growth stages;

[0052] Step 1-1: Image stitching and registration: Agisoft PhotoScan 1.2.6 software (Agisoft LLC, ST, Petersburg, Russia) was used to stitch the RGB images. During the stitching process, the latitude and longitude information of GCPs was imported for image geo-registration to reduce the coordinate error caused by environmental conditions during the actual shooting process. After the stitching project was completed, the orthophoto map (DOM) and digital surface model (DSM) were generated.

[0053] Step 1-2: Extraction of plot boundaries: The plot boundaries were plotted using the GRID python package. First, orthophotos of different ecological points at the jointing period were used for image binarization to distinguish the plots and soil after row closure. The center point of the field was determined based on a mathematical algorithm, and the appropriate boundary length and width were input to generate a plot boundary shapefile with coordinate information.

[0054] Step 1-3: Batch masking of plots: Use the plot boundary shapefile generated by GRID to mask, build a sample library grouped by growth period, and mark the current phenological period information and plot number information of each field when the image acquisition time is the same.

[0055] Step 2: Classification of growth stages by integrating plant height information and image information to achieve high-precision classification of growth stages;

[0056] Step 2-1: Growth stage classification based on three-dimensional height features and KNN. Based on the high-overlap RGB images of drones, the running recovery structure algorithm is used to obtain the dense point cloud data of wheat, thereby generating the wheat crop surface model DSM. A drone operation is carried out after wheat sowing at each ecological point to obtain the digital terrain model (DTM) of the study area. The crop surface model is subtracted from the digital surface model obtained after sowing to obtain the crop canopy height model (CHM). The plant height of each stage is divided by the maximum plant height to obtain the relative plant height. The KNN (knearest neighbor) clustering algorithm is used to classify the relative plant height into three growth stages: tillering stage (TS), jointing and booting stage (JBS), and heading to maturity stage (HMS). The KNN algorithm determines the category to which the sample data to be classified belongs, and outputs the predicted probability of each sample in the three categories, recorded as KNN TS、 KNN JBS and KNN HMS ;

[0057] Step 2-2: Growth stage classification based on the minimalist neural network VanillaNet-13. Use the minimalist neural network VanillaNet-13 to build a classification model for the growth stage, which is divided into tillering stage (TS), jointing and booting stage (JBS), heading stage (HS), anthesis and filling stage (AFS) and maturity stage (MS), and generate the predicted category and predicted probability of each sample. The predicted probabilities are recorded as CNN TS ,CNN JBS ,CNN HS ,CNN AFS and CNN MS In order to evaluate the performance of the classification model, the true positive (TP), true negative (TN), false positive (FP) and false negative (FN) rates are calculated uniformly, and then the accuracy (Accuracy), precision (Precision), recall (Recall) and F1 score (F1-score) of the test data set are obtained;

[0058] Step 2-3: Classification of reproductive stages by classifier integration. The two classification results are integrated to obtain higher classification accuracy. The prediction probabilities of the two classifiers at the same stage TS and JBS are integrated, and the prediction probability of the samples classified as HMS by KNN is determined by CNN. Based on the final integrated 5-category prediction probability, the final prediction category of the sample is determined as the category with the maximum prediction probability. Finally, the accuracy (Accuracy), precision (Precision), recall (Recall) and F1 score (F1-score) of the test data set are obtained;

[0059] It should be added that in the classification of growth stages based on three-dimensional height features and KNN, the final plant height varies significantly due to different varieties. To solve this problem, the plant height of each stage is normalized by dividing it by the maximum plant height to obtain a relative plant height value ranging from 0 to 1. The relative plant height is used as a one-dimensional feature input for the KNN algorithm. KNN is an instance-based classification method that predicts the category of a sample by calculating the distance and using a voting mechanism. Using the majority voting method, the K nearest neighbors of the sample are selected, and the most frequently occurring phenological stage is determined as the predicted class. The choice of K is crucial: a smaller K value may lead to overfitting of the training data, while a larger K value may lead to underfitting. Therefore, the cross-validation method is used to determine the optimal K value. In addition, considering the trend of wheat plant height to stabilize after the heading stage, the category 3 is determined according to the distribution range of the relative plant height.

[0060] It should be added that in the classification of growth stages based on the minimalist neural network VanillaNet-13, in terms of image preprocessing and enhancement, some commonly used enhancement methods such as random 10-degree rotation, Gaussian blur, changes in color saturation and brightness, Mixup, etc. were added, and resizing and normalization were performed. Figure 3 This is the network framework diagram of the minimalist neural network VanillaNet-13. The calculation formulas for accuracy, precision, recall and F1-score are:

[0061]

[0062] It should be added that in the classification of growth stages of the classifier ensemble, the calculation formula of the ensemble method is:

[0063]

[0064] Ensemble_Pj=CNN_Pj (6)

[0065] The values ​​of i are the first two growth stages, TS and JBS, and the values ​​of j are the last three growth stages, HS, AFS and MS. are the weight coefficients of KNN and CNN in stage i respectively.

[0066] Step 3: Based on the time series prediction probability obtained after classification, the specific steps of constructing an estimation model for the start date of the key growth period include:

[0067] Step 3-1: Take the time series prediction probability and start date of each wheat breeding material dynamics as input data;

[0068] Step 3-2: Apply random forest to estimate the start date of the reproductive period and predict the probability of time series Ensemble JBS 、Ensemble HS 、Ensemble AFS and Ensemble MS They were used as feature inputs for the estimation models of jointing, heading, flowering and maturity to obtain the estimation results of each start date;

[0069] Step 3-3: Calculate R for the observed and estimated values ​​for each starting date 2 , RMSE and relative RMSE (rRMSE) were used to evaluate the performance of model estimation.

[0070] Step 4: The specific steps to determine the optimal data collection interval for monitoring the growth period of breeding materials include:

[0071] Step 4-1: Divide the total number of days in the wheat season by the number of drones flown to obtain the average time window;

[0072] Step 4-2: Reduce the number of flights in sequence, gradually extend the time window, and estimate the monitoring accuracy of different growth stages;

[0073] Step 4-2: By estimating the monitoring accuracy of the growth period under different time windows, determine the maximum time window while keeping the monitoring accuracy of the growth period as unchanged as possible, that is, the optimal time window.

[0074] The data statistics of different growth periods of the present embodiment at three test points are shown in Table 1. There are differences in the evolution of growth periods (jointing period, heading period, flowering period and maturity period) at different ecological points, especially GD and the other two tests are significantly different, about 40 to 60 days earlier than HB and JS on average. The variability of wheat materials in GD is the largest in the same growth period, indicating that the growth process differences between germplasm resources are huge. The standard deviation of spring wheat in the heading period is up to 20.59, and the data dispersion is relatively high. On the contrary, the maturity data of winter wheat is more concentrated, and it enters the maturity period at an average of 179 days, with a standard deviation of 8.14. The growth process law of wheat materials at HB and JS ecological points is relatively consistent, and the total growth period length is about 220 days after sowing. The growth period difference between wheat germplasm resources at each ecological point is also relatively small. Whether it is spring wheat or winter wheat, the standard deviation of each growth period is within 8 days.

[0075] Table 1 Statistics of different growth stages of the three test sites GD, HB and JS (s and w represent spring wheat and winter wheat, respectively)

[0076]

[0077] The K value in the KNN algorithm for three-dimensional height feature classification in this embodiment is determined as follows: Figure 5 As shown in (a), the optimal value of K is determined to be 26, at which the minimum error is 0.157. The classification results of growth stages based on three-dimensional height features and KNN are shown in Figure 5 As shown in (b), the three growth stages were classified based on plant height information, with an overall classification accuracy of 0.93, which was a good classification effect. The plant height of wheat after heading was most different from that of the first two stages, so the classification accuracy of HMS was the highest (0.97), and only 0.03 samples were incorrectly classified as JBS. The accuracy of TS and JBS were 0.91 and 0.90, respectively.

[0078] The classification results of the growth stages based on the minimalist neural network VanillaNet-13 in this embodiment are as follows: Figure 6 As shown in the confusion matrix, it can be seen that the accuracy of Vanilnet13 for the five growth stages is 87%, 80%, 66%, 87% and 96% respectively. The accuracy of using RGB image depth information alone in the mid-growth stage is relatively poor, especially in the JBS and HS stages. The performance comparison results of the VanillaNet-13 model and other classic image classification models are shown in Table 2. FLOPs refers to the number of floating-point operations, which is used to measure the complexity of the algorithm / model and is often used as an indirect measure of the speed of the neural network model. FPS refers to the number of frames per second, that is, the number of pictures that can be processed per second or the time required to process a picture to evaluate the detection speed. The shorter the time, the faster the rate. These results highlight the potential of VanillaNet for real-time processing in existing deep networks.

[0079] Table 2 Comparison of VanillaNet-13 and classic image classification models

[0080]

[0081] The weight values ​​of the integrated classification results of KNN and VanillaNet-13 in this embodiment are shown in Table 3. The F1 scores of KNN in these two stages are higher than those of VanillaNet-13, so a greater weight is given to the KNN result.

[0082] Table 3 Weight values ​​of KNN and VanillaNet-13 classifier integration

[0083]

[0084] The classification results of the growth stage of the classifier integration in this embodiment are as follows: Figure 7As shown in Table 4, the classification results of different classifiers are compared and analyzed. It can be seen that the classification accuracy of the stage with significant differences in plant height in the early growth period is improved most significantly after integration. The F1 scores of the first three stages are increased by 7.41%, 5.81% and 13.16% respectively.

[0085] Table 4 Comparative analysis of classification results of different classifiers

[0086]

[0087]

[0088] The RF model estimation results and spatial distribution diagrams of the start date of the key growth period in this embodiment are as follows: Figure 8 and Fig. 9 As shown in the figure, the estimation accuracy of the jointing stage is relatively poor, the data is relatively concentrated, and the DAS is concentrated between 130 and 140 days. The range of the flowering period is basically the same as the heading period, with rRMSE of 0.02 and an error of about 4 days. The estimation accuracy of the maturity period is the highest, with DAS values ​​distributed after 200 days and an error of less than 2 days. Fig. 9 The spatial distribution map can more intuitively show the significant differences in the growth process of 1,000+ breeding plots, making it easier for breeders to quickly screen early-maturing and late-maturing varieties and varieties with other special growth processes.

[0089] The determination of the optimal data acquisition time interval for the growth period monitoring research of breeding materials in this embodiment is as follows: Fig.10 As shown. The present invention flies 47 times in a wheat growth period, and the total time span from the first flight to the last flight is 131 days. The average time interval is defined as the total time span divided by the number of flights, and the average time interval is 2.8 days. In the stage with higher time frequency, the number of flights is reduced successively until it is reduced to 1 time. By gradually increasing the time interval, the monitoring accuracy of the growth period under different time intervals is estimated, so as to determine the maximum time interval under the condition that the monitoring accuracy of the growth period remains unchanged as much as possible. The results show that as the average time interval gradually increases, the RMSE of the monitoring error during the growth period also gradually increases. The RMSE of the maturity period and the flowering period increases slightly after more than 14 days, indicating that the optimal time interval is within 14 days to ensure high-precision monitoring of the maturity period and the flowering period. When the time interval is less than 5 days, the monitoring accuracy of each growth period can be guaranteed to the maximum extent. Therefore, the optimal time interval is within 5 to 13 days for the optimal data acquisition frequency for monitoring the growth period. This will provide guidance for drone flight missions that save the utilization of agricultural resources and the consumption of manpower and material resources.

[0090] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the protection scope of the present invention in any form, and all technical solutions obtained by equivalent replacement and the like fall within the protection scope of the present invention. The parts not involved in the present invention are the same as the prior art or can be implemented by the prior art.

Claims

1. A method for intra-seasonal classification and estimation of the growth period of wheat breeding materials combining time series RGB images and machine learning, characterized in that: The following steps are involved: Step 1: Based on the wheat breeding material plot tests in different growing seasons and ecological points, obtain time-series high-spatial-resolution RGB images and perform preprocessing to build a sample library of a large number of germplasm resources at different growth stages; Step 2: Classification of growth stages by integrating plant height information and image information to achieve high-precision classification of growth stages; Step 3: Based on the time series prediction probability obtained after classification, an estimation model for the start date of the key growth period is constructed; Step 4: Determine the optimal data collection interval for monitoring the growth period of breeding materials.

2. The intra-season classification and estimation method of the growth period of wheat breeding materials combining time series RGB images and machine learning according to claim 1 is characterized in that: In step 1, the specific steps of pretreatment include: Step 1-1, image stitching and registration: stitch the RGB images, and import the longitude and latitude information of GCPs for image geo-registration during the stitching process. After the stitching project is completed, generate orthophotos and digital surface models; Step 1-2, plot boundary extraction: The plot boundary is plotted using the GRID python package. First, the orthophotos of different ecological points at the jointing period are used for image binarization to distinguish the plots and soil after the row closure. Then, the center point of the field is determined based on a mathematical algorithm, and the appropriate boundary length and width are input to generate a plot boundary shapefile with coordinate information. Steps 1-3, batch masking of plots: Use the plot boundary shapefile generated by GRID for masking, build a sample library grouped by growth period, and mark the current phenological period information and plot number information of each field when the image acquisition time is the same.

3. The intra-season classification and estimation method of the growth period of wheat breeding materials combining time series RGB images and machine learning according to claim 1 is characterized in that: In step 1, drone image collection is performed after wheat seedlings emerge, drone image collection is performed every 1-3 days during the transition period of the growth period, and once every 3-6 days during the rest of the period, and the images are acquired during the time period of 10:00-12:00 local time on sunny days.

4. The intra-season classification and estimation method of the growth period of wheat breeding materials combining time series RGB images and machine learning according to claim 1 is characterized in that: The specific steps of step 2 include: growth stage classification based on three-dimensional height features and KNN, growth stage classification based on the minimalist neural network VanillaNet-13, and growth stage classification using classifier integration.

5. The intra-season classification and estimation method of the growth period of wheat breeding materials combining time series RGB images and machine learning according to claim 4 is characterized in that: The growth stage classification based on three-dimensional height features and KNN is as follows: based on the UAV RGB high-overlapping images, the running recovery structure algorithm is used to obtain the dense point cloud data of wheat, so as to generate the wheat crop surface model; after wheat is sown at each ecological point, a UAV operation is carried out to obtain the digital terrain model of the study area, and the crop surface model is subtracted from the digital surface model obtained after sowing to obtain the crop canopy height model, and the plant height of each stage is divided by the maximum plant height to obtain the relative plant height, and the KNN clustering algorithm is applied to classify the relative plant height into three growth stages: tillering stage TS, jointing and booting stage JBS, and heading to maturity stage HES. The KNN algorithm determines the category to which the sample data to be classified belongs, and outputs the predicted probability of each sample in the three categories, which is recorded as KNN TS 、KNN JBS and KNN HMS .

6. The intra-season classification and estimation method of the growth period of wheat breeding materials combining time series RGB images and machine learning according to claim 4, characterized in that: The classification of growth stages based on the minimalist neural network VanillaNet-13: The classification model of growth stages is constructed using the minimalist neural network VanillaNet-13, which is divided into tillering stage TS, jointing and booting stage JBS, heading stage HS, flowering and filling stage AFS and maturity stage MS, and the predicted category and predicted probability of each sample are generated. The predicted probabilities are recorded as CNN TS ,CNN JBS ,CNN HS ,CNN AFS and CNN MS ; To evaluate the performance of the classification model, the true positive TP, true negative TN, false positive FP and false negative FN rates are calculated uniformly, and then the accuracy, precision, recall and F1-score of the test dataset are obtained.

7. The intra-season classification and estimation method of the growth period of wheat breeding materials combining time series RGB images and machine learning according to claim 6, characterized in that: In terms of image preprocessing and enhancement, random 10-degree rotation, Gaussian blur, changes in color saturation and brightness, Mixup enhancement methods are added, and resizing and normalization are performed; the calculation formulas for accuracy, precision, recall and F1 score are:

8. The intra-season classification and estimation method of the growth period of wheat breeding materials combining time series RGB images and machine learning according to claim 4, characterized in that: The classification of the growth stage of the classifier integration: by integrating the two classification results, a higher classification accuracy is obtained; specifically, the prediction probabilities of the two classifiers at the same stage TS and JBS are integrated, and the prediction probability of the sample classified as HMS by KNN is determined by CNN; based on the 5 categories of prediction probabilities after the final integration, the final prediction category of the sample is determined as the category where the maximum prediction probability is located, and finally the accuracy, precision, recall and F1 score of the test data set are obtained; wherein the calculation formula of the integration method is: Ensemble_Pj=CNN_Pj (6) i is the first two growth stages TS and JBS, j is the last three growth stages HS, AFS and MS, are the weight coefficients of KNN and CNN in stage i respectively.

9. The intra-season classification and estimation method of the growth period of wheat breeding materials combining time series RGB images and machine learning according to claim 1, characterized in that: The step 3 specifically includes: Step 3-1: Take the time series prediction probability and start date of each wheat breeding material dynamics as input data; Step 3-2: Apply random forest to estimate the start date of the reproductive period and predict the probability of time series Ensemble JBS 、Ensemble HS 、Ensemble AFS and Ensemble MS They were used as feature inputs for the estimation models of jointing, heading, flowering and maturity to obtain the estimation results of each start date; Step 3-3: Calculate R for the observed and estimated values ​​for each starting date 2 , RMSE and relative RMSE are used to evaluate the performance of model estimation.

10. The intra-season classification and estimation method of the growth period of wheat breeding materials combining time series RGB images and machine learning according to claim 1, characterized in that: The step 4 specifically includes: Step 4-1: Divide the total number of days in the wheat season by the number of drones flown to obtain the average time window; Step 4-2: Reduce the number of flights in sequence, gradually extend the time window, and estimate the monitoring accuracy of different growth stages; Step 4-2: By estimating the monitoring accuracy of the growth period under different time windows, determine the maximum time window while keeping the monitoring accuracy of the growth period as unchanged as possible, that is, the optimal time window.

Images