Method for predicting crop maturity

Abstract

Disclosed is a computer-implemented method for predicting crop maturity, comprising: providing aerial image data based on one or more aerial images of a crop plot of a field; determining an analytic function representative of a vegetation index of the crop plot as a function of time, the vegetation index derived from the aerial image data; applying a machine learning model, the machine learning model trained to use coefficients of the analytic function as input for predicting crop maturity, to predict a time of crop maturity per crop plot.

Description

[0001] Method for predicting crop maturity

[0002] TECHNICAL FIELD

[0003] Disclosed are a method for predicting crop maturity, a training method for a machine learning method, use of the method, a trained machine learning model, a use of the machine learning model, a system, a computer program product, and a computer-readable medium.

[0004] TECHNICAL BACKGROUND

[0005] Predicting crop maturity is useful in different scenarios. Such scenarios include controlling harvesting operations and breeding scenarios, e.g. for trait development in crops. Crop maturity may be used to improve efficiency and accuracy of seed selection decisions and / or to group varieties of similar maturity in the field, e.g., to improve efficiency of running yield trials. Known methods for predicting crop maturity of agricultural fields are rather subjective. This can make them error-prone, difficult to reproduce, and potentially imprecise. Accordingly, there is need for improvement of known methods.

[0006] In the Proceedings of Spie, Narayanan et al., “Improving Soybean breeding using UAS measurements of physiological maturity” (14 May 2019), some concepts and challenges with present methods are described. Methods presented in the publication include modeling greenness / NGA decay over time. Here, greenness time series during canopy senescence phase are described by simple linear or piece-wise linear regression. Maturity is directly estimated from linear regression.

[0007] It is therefore an object of the present disclosure to provide an improved method for determining crop maturity.

[0008] SUMMARY OF THE INVENTION

[0009] The present disclosure provides a computer-implemented method for predicting crop maturity, comprising: providing aerial image data based on one or more aerial images of a crop plot of a field; determining an analytic function representative of a vegetation index of the crop plot as a function of time, the vegetation index derived from the aerial image data; applying a machine learning model, the model trained to use coefficients of the analytic function as input for predicting crop maturity, to predict a time of crop maturity per crop plot. In particular, the aerial image data may be based on two or more, in particular three or more, in particular four or more aerial images of the crop plot.

[0010] In other words, the present disclosure provides a computer-implemented method for predicting crop maturity, comprising: determining an analytic function representative of a vegetation index of a crop plot of a field as a function of time, the vegetation index derived from aerial image data that is based on one or more aerial images of the crop plot; applying a machine learning model, the model trained to use coefficients of the analytic function as input for predicting crop maturity, to predict a time of crop maturity per crop plot.

[0011] The method of the present disclosure provides a machine learning approach for predicting maturity based on a model that takes inputs from coefficients of an analytic function, for example regression coefficients of a logistic function.

[0012] The present disclosure, provides a method that is particularly tolerant to sources of environmental variability, e.g. affecting measurements of canopy. Accordingly, a robust prediction of senescence, and / or, a robust prediction of crop maturity can be obtained. This is for example the case if the analytic function is a logistic function, e.g. where regression coefficients of the logistic function are used as input for the machine learning.

[0013] It will be understood that, rather than using time series, e.g. of vegetation and / or meteorological variables and / or vegetation indices, as input for machine learning, according to the present disclosure time series of vegetation indices may be used to obtain / build an analytic function and the coefficients (terms) of those analytic functions are used as input for machine learning. The coefficients, thus, can be seen as input variables (i.e., predictor variables) for the machine learning model. Accordingly, initial variables (vegetation indices) are transformed into new predictor variables used as model inputs. Thus, it will be understood that according to the present disclosure, function fit (e.g. logistic function) may be used to determine coefficients of this function and use these coefficients as inputs for a prediction model. The model may be trained on image-derived vegetation indices and ground truth data (i.e., manual estimation of crop maturity). Among others, this allows for a more robust and generalized description of canopy senescence process for each field plot and potentially more accurate replication of manual estimation approach.

[0014] While logistic functions of time series have been employed for entirely different purposes, such as for identifying a time frame, e.g. season, for example over which to aggregate input data, logistic functions are not used to describe a senescence process or to use their coefficients as inputs for a prediction model.

[0015] Known approaches often predict determining seasons (start, peak, or end of season) and do not predict crop maturity or senescence. For the purpose of breeding support, predicting a season is not particularly useful, as end of season and crop maturity will usually not correspond.

[0016] Known approaches often evaluate land cover classification, such as by applying classification ML models. Such approaches are not particularly useful for the purpose of breeding support, in particular generally irrelevant to crop maturity estimation.

[0017] In yet other words, the present disclosure provides a computer-implemented method for predicting crop maturity, comprising: providing aerial image data based on one or more aerial images of a crop plot of a field; determining an analytic function representative of a time series of a vegetation index of the crop plot, the vegetation index derived from the aerial image data; applying a machine learning model, the model trained to use coefficients of the analytic function as input for predicting crop maturity, to predict a time of crop maturity per crop plot.

[0018] In yet other words, the present disclosure provides a computer-implemented method for predicting crop maturity, comprising: determining an analytic function representative of a time series of a vegetation index of a crop plot of a field, the vegetation index derived from aerial image data that is based on one or more aerial images of the crop plot; applying a machine learning model, the model trained to use coefficients of the analytic function as input for predicting crop maturity, to predict a time of crop maturity per crop plot.

[0019] The present disclosure further provides use of the method of the present disclosure for at least one of: determining when to harvest the crop on a field, particularly the field and / or one or more other fields having the same type of crop and / or having crop being grown concurrently with the crop on the field, determining a harvesting schedule for a field, particularly the field and / or one or more other fields having the same type of crop and / or having crop being grown concurrently with the crop on the field, outputting, to a harvester and / or control device for controlling a harvester and / or to a user an indication when to harvest the crop on a field, particularly the field and / or one or more other fields having the same type of crop and / or having crop being grown concurrently with the crop on the field, outputting selection support data for making a selection decision in breeding and / or trait development, outputting grouping support data for grouping crop varieties in the field in accordance with crop maturity.

[0020] In other words, the predicted crop maturity may be used in different contexts and scenarios.

[0021] For example, the predicted crop maturity may be used in breeding scenarios, such as in soybean breeding. The predicted crop maturity can be used is for making (seed) selection decisions, e.g. to select for certain maturity characteristics for breeding. Alternatively or in addition, the predicted crop maturity can be used is grouping varieties of similar maturity in the field to improve efficiency of running yield trials.

[0022] The predicted crop maturity may alternatively or in addition be used in harvesting scenarios. A first harvesting scenario in which crop maturity can be used to provide data to a harvester, e.g. an autonomous / self-driving harvester, the data indicating when to harvest a crop in a field, and / or providing scheduling data to the harvester. A second harvesting scenario in which crop maturity, potentially together with additional data, such as phenotyping data, is data indicating to the harvester, e.g. a harvester having automatic sample bagging capability, to save harvested samples, e.g. putting it in a sample bag, or discard the harvested samples.

[0023] The present disclosure further provides a system comprising a computing system configured to carry out the method of the present disclosure.

[0024] In other words, the present disclosure provides a system comprising a computer system for predicting crop maturity, the computer system configured to provide aerial image data based on one or more aerial images of a crop plot of a field; determine an analytic function representative of a vegetation index of the crop plot as a function of time, the vegetation index derived from the aerial image data; applying a machine learning model, the model trained to use coefficients of the analytic function as input for predicting crop maturity, to predict a time of crop maturity per crop plot.

[0025] According to the present disclosure, the computing system may be configured to predict crop maturity by executing a first module, a second module, a third module, and optionally an optional fourth module, wherein the first module is configured to provide aerial image data based on one or more aerial images of a crop plot of a field, wherein the second module is configured to determine an analytic function representative of a vegetation index of the crop plot as a function of time, the vegetation index derived from the aerial image data, wherein the third module is configured to apply a machine learning model, the model trained to use coefficients of the analytic function as input for predicting crop maturity, to predict a time of crop maturity per crop plot, and wherein the optional fourth module is configured to train the machine learning model to predict crop maturity, particularly to predict a time of crop maturity per crop plot.

[0026] The computer system may be configured as a distributed computer system. In particular, the method may be executed by different computing devices of the distributed computer system. In particular, the distributed computer system may comprise one or more servers and one or more client devices and the method may be executed in part by at least one of the servers and in part by at least one of the client devices.

[0027] The system may comprise one or more devices other than the computer system. The system may comprise one or more imaging devices for acquiring the aerial images and / or one or more harvester control devices for controlling operation of one or more harvesters, particularly on the basis of the predicted crop maturity and / or data derived therefrom, such as a harvesting schedule, and / or one or more user interfaces for outputting predicted crop maturity and / or data derived therefrom, such as a harvesting schedule, to a user.

[0028] The present disclosure also provides a training method comprising training of a machine learning model, wherein the training of the machine learning model comprises training the machine learning model to predict a crop maturity on the basis of an input comprising coefficients of an analytic function representative of a vegetation index of a crop plot of a field as a function of time, the vegetation index derived from training aerial image data, in particular aerial image data derived from acquired images of one or more fields and / or the aerial image data derived from artificially generated aerial images and / or the aerial image data being artificially generated, particularly without previous generation of artificially generated aerial images.

[0029] The present disclosure also provides a use of the trained machine learning model, trained using a training method of the present disclosure, for at least one of: determining when to harvest the crop on a field, particularly the field and / or one or more other fields having the same type of crop and / or having crop being grown concurrently with the crop on the field; outputting, to a harvester and / or control device for controlling a harvester and / or to a user an indication when to harvest the crop on a field, particularly the field and / or one or more other fields having the same type of crop and / or having crop being grown concurrently with the crop on the field; determining a harvesting schedule for a field, particularly the field and / or one or more other fields having the same type of crop and / or having crop being grown concurrently with the crop on the field, outputting selection support data for making a selection decision in breeding and / or trait development, outputting grouping support data for grouping crop varieties in the field in accordance with crop maturity. Reference is made to the above description of using the method. Features described therein also apply for use of the trained machine learning model.

[0030] The present disclosure further provides a computer program product comprising instructions which, when carried out by a computer, cause the computer to carry out and / or control the method of the present disclosure.

[0031] The present disclosure further provides a computer-readable medium having stored thereon instructions which, when carried out by a computer, cause the computer to carry out and / or control the method of the present disclosure.

[0032] Any features described in the context of one of the categories, such as system or method, equally apply to the other categories.

[0033] As outlined above, the present disclosure provides a computer-implemented method for predicting crop maturity, comprising: providing aerial image data based on one or more aerial images of a crop plot of a field; determining an analytic function representative of a vegetation index of the crop plot as a function of time, the vegetation index derived from the aerial image data; applying a machine learning model, the model trained to use coefficients of the analytic function as input for predicting crop maturity, in particular to predict a time of crop maturity per crop plot. In particular, the aerial image data may be based on two or more, in particular three or more, in particular four or more aerial images of the crop plot. According to the present disclosure, predicting crop maturity is to be understood as comprising predicting a respective time of crop maturity for one or more crop plots, particularly all crop plots of a field. The time of crop maturity of a crop plot may be seen as the time it takes for the crop of the crop plot to become mature. For example, when the degree of greenness of a crop plot has fallen below a predetermined level, this may indicate that the crop plot has become mature. This will be described in detail below.

[0034] The analytic function according to the present disclosure may be a function that analytically describes a canopy senescence process of a crop plot.

[0035] The analytic function according to the present disclosure may comprise any function, such as logistic function, polynomial, or the like. A logistic function may, in particular, be employed. It was found that compared to other functions, such as Gaussian, this allows for a more accurate analytical description of the canopy senescence process. As mentioned above, the present disclosure provides a method that is tolerant to sources of environmental variability, e.g. affecting measurements of canopy. Accordingly, a robust prediction of senescence can be obtained. This is for example the case if logistic regression is used, which improves robustness significantly.

[0036] The coefficients of the analytic function may comprise coefficients of the logistic function. Particularly, the analytic function may be a four-parameter logistic function.

[0037] In particular, all coefficients of the analytic function may be used. E.g., all coefficients may be provided to the machine learning model as input. The set of all coefficients uniquely identifies the function. This allows for a more simplified approach for selection of model inputs.

[0038] The function being representative of the vegetation index may also be referred to as the function describing the vegetation index. In particular, the function may be or be derived from the values of the vegetation index for different times.

[0039] The machine learning model may be trained, in particular, to use the coefficients of the analytic function, particularly a logistic function, as input for predicting crop maturity to predict a time of crop maturity per crop plot.

[0040] A vegetation index, according to the present disclosure, may be understood broadly to entail any parameter whose values represent characteristics of a vegetation, such as a crop. For example, the vegetation index may be representative of greenness, e.g. canopy greenness. This will be explained in more detail below. The color will change over time due to maturing, for example greenness might be reduced and other colors like yellow or brown may increase.

[0041] The vegetation index being derived from aerial image data may entail, as an example, a calculation based on colors of pixels of the aerial images. This will be explained in detail below.

[0042] The machine learning model may be any suitable machine learning model known in the art. In particular, regression models (models that solve regression tasks) may be used. This allows for predicting a continuous variable.

[0043] As described below, an SVM (Support Vector Machine) or PLS (Partial Least Squares) regression or MLR may be used.

[0044] In the present disclosure, the coefficient “model” is used in short for “machine learning model”.

[0045] A trained machine learning model is a model that has undergone training. A machine learning model trained to use coefficients of the analytic function as input for predicting crop maturity, particularly to predict a time of crop maturity, may have been trained using, as an input, training aerial image data derived from acquired images of one or more fields and / or the aerial image data derived from artificially generated aerial images and / or aerial image data being artificially generated, particularly without previous generation of artificially generated aerial images. Training methods according to the present disclosure may be used to obtain the trained model.

[0046] Predicting a time of crop maturity per crop plot may comprise, per crop plot, predicting a time when the respective crop plot has reached maturity. The time may be provided in any suitable format, such as a date, a week, a time point, a time interval, or the like.

[0047] Providing aerial image data may comprise making aerial image data available for use in determining the vegetation index. Aerial image data to be provided may be aerial image data retrieved from a data storage. Aerial image data may be received directly from one or more imaging devices acquiring aerial images and / or may be received from one or more other devices.

[0048] The method may optionally comprise the step of acquiring aerial images prior to providing the aerial image data, e.g. by means of an imaging device. The imaging device may, for example, be mounted to an airborne vehicle like a drone.

[0049] Aerial image data being based on aerial images is to be understood broadly and may entail that the aerial image data comprise the aerial images or a subset of the aerial images and / or that the aerial image data comprises data derived from the aerial images. For example, aerial images may be processed to obtain the aerial image data, such as converted, compressed, filtered, enhanced, or the like.

[0050] Aerial images of a crop plot are to be understood as images that depict at least part of, particularly the entire crop plot. In other words, the images may be acquired with a field of view of the imaging device comprising the at least part of the crop plot, particularly the entire crop plot. The images, in particular, may depict the at least part of the crop plot in a top view. An aerial image may depict multiple crop plots or parts of multiple crop plots.

[0051] In the following, the coefficients crop plot and crop will be explained in detail.

[0052] The field, according to the present disclosure may be an agricultural field.

[0053] The field, particularly the agricultural field, may comprise a piece of land that may be planted with a crop. A crop may comprise one or more crop plants. The field may be a trial field, also referred to as test field or experimental field. A trial field may be a field for which at least some growth parameters are controlled and / or monitored and / or where the crop plots may be subject to examination. In particular, a trial field may be a field used for breeding trials.

[0054] The crop may, for example comprise, for example annual grain crops for which senescence leads to physiological maturity. The crop may be one of soy, corn, canola, wheat, dry peas, for example. Such crops may mature based on growing degree days or photoperiodism. For example, the method may be employed for soy. In the example of soy, a plant may be defined as fully mature when 95 percent of the pods haver reached their mature pod color.

[0055] The field, particularly the agricultural field, may be divided into plots of the agricultural field, wherein plots cover the entire agricultural field. These plots, herein, are referred to as crop plots so as to indicate that the plots are planted with a crop. The crop plots may all have the same shape and size. Alternatively, the crop plots may comprise portions of different shapes and / or of different sizes.

[0056] The method of the present disclosure may be carried out for the entire field or only for part of the field, for example, for one or more, particularly all, of the crop plots.

[0057] Thus, to summarize, the present disclosure allows for predicting crop maturity. This can be done in an objective way by analyzing aerial image data to quantitatively derive crop maturity. The proposed method is less error prone, more reproducible, and, accordingly, more reliable than presently known methods. It has a higher throughput and faster turnaround times. Therefore, it can also be used as an objective basis for scheduling harvesting or breeding, for example.

[0058] According to the present disclosure, the aerial images may be aerial images having been acquired at multiple time points over a senescence period of the field, in particular, the aerial images having been acquired / collected for one or more crop plots of the field, in particular all crop plots of the field. If the method is carried out for the entire field or a majority of the crop plots of the field, this may provide insights concerning how different crop plots mature relative to each other and / or how the overall field matures.

[0059] A senescence period of the field, such as a trial field, according to the present disclosure, may be the period between the earliest onset of senescence in the field and all crop plots having reached maturity, as will be explained in more detail below.

[0060] Senescence is a reliable indicator of maturing for various crops, such as annual grain crops for which senescence leads to physiological maturity, e.g., soy, corn, canola, wheat, dry peas, etc.

[0061] Accordingly, this feature allows for reliably deriving crop maturity.

[0062] According to the present disclosure, the analytic function may be a logistic function, particularly having at least 3 coefficients, particularly at least 4 coefficients, particularly exactly 4 coefficients. The number of coefficients may comprise less than 6, in particular less than 5 coefficients. In other words, the machine-learning model may be trained based on at least 3, particularly at least 4, particularly exactly 4 coefficients, and / or less than 6, in particular less than 5 coefficients.

[0063] While other numbers of coefficients yield suitable results, the number of coefficients presented above provide higher accuracy than a lower number of coefficients, and the use of higher than the specified number of coefficients unnecessarily complicates the method.

[0064] It is noted that using coefficients of the function, rather than directly inputting the time-series of vegetation indices or the aerial image data, as such, leads to improved results.

[0065] According to the present disclosure, the method may utilize, for determining the analytic function, maturity data comprising, for each of multiple time points, which time points may be selected in the same manner as described above, a value representative of a greenness for the respective crop plot, in particular a value of the vegetation index.

[0066] The analytic function, thus, may be seen as describing a change, particularly drop, in greenness over time, e.g. a senescence process. It may be fitted to a time series of the vegetation index.

[0067] Maturity data is data representative of characteristics that are related to crop maturity. Greenness is a good predictor for crop maturity, particularly for the crops described above. Accordingly, using a value representative of greenness is suitable for predicting crop maturity.

[0068] Maturity data may be image-derived data, e.g. a plot’s mean greenness value calculated from an image of the plot at multiple time points.

[0069] The vegetation index represented by the analytic function may be a vegetation index representative of greenness. Vegetation indices will be discussed in more detail below. As an example, a VARI index or MCARI2 index may be represented by the analytic function, but other indices are also possible, as outlined further below. In particular, VARI based on RGB images may be used, particularly images collected by drones. Multispectral and / or satellite image data are not required. Thus, while the present method may be carried out using, for example, VARI, MCARI2, and / or NDVI for obtaining the coefficients (MCARI2 and NDVI, for example, derived from multispectral images), VARI derived from RGB images may allow for easy and cheap data collection while providing accurate results in the context of the method of the present disclosure.

[0070] According to the present disclosure training data used for training the machine learning model may comprise visually assessed maturity data.

[0071] Model training may be based on single time point maturity date estimates, using the analytic function coefficients as input, for example. Training data is representative of maturity of the plants in the plot as assessed by the breeder. It may not necessarily (directly) representative of greenness of the crop plot as such. For example, for soybean maturity can be defined as estimating the time / time interval when 95% of pods reach mature (pod) color.

[0072] Particularly, the visually assessed data may be used in the training of the machine learning model to confirm accuracy of the prediction made by the machine learning model, thereby allowing the machine learning model to be trained to make higher accuracy predictions.

[0073] Thus, the trained machine learning model can make high-quality predictions.

[0074] In an embodiment, the visual assessment may be carried out by a controlled setup and / or controlled person or set of people, particularly a trained person or set of people, to improve quality of the visually assessed data.

[0075] According to the present disclosure, the machine learning model may be trained using ground truth data, in particular target data, derived from the visually assessed maturity data. The coefficient ground truth may refer to the reality to be modelled with a supervised machine learning algorithm. Ground truth may also be referred to as the target for training or validating the model with a labeled dataset.

[0076] This allows for particularly high-quality predictions.

[0077] According to the present disclosure, the senescence period of the field may start with the onset of senescence and end with all crop plots of the field having reached maturity.

[0078] The onset of senescence may be the time when the first crop plot of the field starts showing senescence, also referred to as the earliest onset of senescence in the field. Physiological maturity may be reflected by plant colour. For example, full physiological maturity of a soybean plant is reached when 95% of all pods have their mature pod color, such as brown, tan, or tawny. The change in color is indicative of senescence and, accordingly, maturing of the crop.

[0079] Thus, the relevant period is covered by the data and the results have high quality.

[0080] According to the present disclosure, the vegetation index may comprise a greenness index, particularly a canopy greenness index. The vegetation index may be Visible Atmospherically Resistant Index, VARI, and / or Modified Chlorophyll Absorption Ratio Index Improved, MCARI2, and / or normalized difference red edge index, NDRE, and / or normalized difference vegetation index, NDVI. Particularly, the vegetation index may be VARI and / or MCARI2.

[0081] Canopy greenness, as outlined above, is a good indicator for maturity. The indices mentioned above are particularly suitable for quantifying greenness with good accuracy.

[0082] MCARI2 is considered a predictor of green leaf area. It indicates relative abundance of chlorophyll. VARI allows for emphasizing vegetation in the visible portion of the spectrum. It is advantageous for RGB or color images and utilizes all three color bands. It allows for mitigating illumination differences and atmospheric effects. NDRE is a vegetation index that can be used to analyze multi-spectral image data to determine whether vegetation therein is healthy. NDRE is based on a ratio of spectral reflectance at near-infrared and red edge wavelengths derived from multispectral images. NDVI is an index that quantifies vegetation greenness. For example, it may be useful in understanding vegetation density and plant health. NDVI is based on a ratio of spectral reflectance of red and nearinfrared derived from multispectral images.

[0083] According to the present disclosure, the vegetation index, in particular a / the (canopy) greenness index, may be determined for each crop plot of the field individually.

[0084] This allows for better precision of the results. Particularly, it allows for precisely reflecting differences between different parts of the field when it comes to maturity.

[0085] According to the present disclosure, the aerial images may be images having been acquired at predetermined variable intervals within a range of allowed intervals.

[0086] For example, according to the present disclosure, the aerial images may be images having been acquired every 1 to 20 days, in particular every 3 to 14, in particular 3 to 7 days, in particular every 4 to 6 days, in particular every 4 to 5 days. Alternatively, they may be images having been acquired at fixed intervals. For example, the aerial images may be images having been acquired every X days, X being an integer between 1 and 20, in particular 3 and 14, in particular 3 and 7, in particular 4 days or 5 days.

[0087] Making the interval fixed may make it easier to process data. However, using variable intervals allows for more flexibility, such as taking into account weather conditions for image acquisition. The method used herein is particularly advantageous as it does not require that the intervals be fixed due to using the coefficients of the analytic function, rather than individual values at certain points in time.

[0088] According to the present disclosure, the aerial images may be images having been acquired at variable intervals at time points that are selected taking into account environmental conditions and / or taking into account a predetermined target interval.

[0089] The environmental conditions may, for example, comprise weather conditions, such as rain, wind, visibility or the like. The predetermined target interval may be used to limit the variability, e.g., ensuring that the time between image acquisitions does not become too long. Thus, the method may allow for flexibility and the quality can be kept high.

[0090] According to the present disclosure, the analytic function may be determined based on data comprising, for each of the multiple time points, a corresponding value of the vegetation index. This may be done making use of a fitting method. In other words, the analytic function may be based on, particularly fitted to, a time series of the vegetation index.

[0091] The method may comprise determining when to harvest the crop on a field, particularly the field and / or one or more other fields having the same type of crop and / or having crop being grown concurrently with the crop on the field.

[0092] For example, a time for harvesting the crop may be determined. In particular, the time may be selected in accordance with maturity, e.g., the time may be selected as the time when the entire field or a predetermined portion thereof is expected to have reached maturity.

[0093] The method may comprise creating a harvesting schedule for a field, particularly the field and / or one or more other fields having the same type of crop and / or having crop being grown concurrently with the crop on the field. For example, a schedule for harvesting the crop may be determined in accordance with maturity, i.e., when the field or a portion thereof has reached maturity. The method may comprise outputting, to a harvester and / or control device for controlling a harvester and / or to a user an indication when to harvest the crop on a field, particularly the field and / or one or more other fields having the same type of crop and / or having crop being grown concurrently with the crop on the field. In particular, the schedule and / or time described above may be output to the harvester and / or control device for controlling a harvester and / or user.

[0094] The above allows for using the maturity information for improved harvesting operations.

[0095] As explained above, the predicted crop maturity may alternatively or in addition be used in breeding scenarios, such as in soybean breeding. The predicted crop maturity can be used is for making (seed) selection decisions, e.g. to select for certain maturity characteristics for breeding.

[0096] Alternatively or in addition, the predicted crop maturity can be used is grouping varieties of similar maturity in the field to improve efficiency of running yield trials.

[0097] As outlined above, the method of the present disclosure may also comprise training the machine learning model using, as input for the training, the coefficients of an analytic function representative of a vegetation index as a function of time, the vegetation index derived from training aerial image data, in particular aerial image data derived from acquired images of one or more fields and / or the aerial image data derived from artificially generated aerial images and / or the aerial image data being artificially generated, particularly without previous generation of artificially generated aerial images. The machine learning model may be trained at least for the vegetation index to be used for predicting crop maturity with the trained model. The machine learning model may be trained at least for the type of crop for which the trained model is used for predicting crop maturity.

[0098] In other words, the model may be trained on actually acquired data, which may be a good indicator for regular use cases, and / or artificially generated training data. Artificially generated training data may allow for supplementing and / or replacing actually acquired data, for example to cover cases that were not observed in the actually acquired data and / or for correcting for errors in actually acquired data.

[0099] A model trained accordingly is particularly suitable for the above-described methods for predicting crop maturity.

[0100] As explained above, the present disclosure also provides a training method comprising training of a machine learning model, wherein the training of the machine learning model comprises training the machine learning model to predict a crop maturity, particularly time of crop maturity per crop-plot, on the basis of an input comprising coefficients of an analytic function representative of a vegetation index of a crop plot of a field as a function of time, the vegetation index derived from training aerial image data, in particular aerial image data derived from acquired images of one or more fields and / or the aerial image data derived from artificially generated aerial images and / or the aerial image data being artificially generated, particularly without previous generation of artificially generated aerial images.

[0101] In other words, the model may be trained on actually acquired data, which may be a good indicator for regular use cases. Artificially generated training data may allow for supplementing and / or replacing actually acquired data, for example to cover cases that were not observed in the actually acquired data and / or for correcting for errors in actually acquired data.

[0102] While many known types of machine learning are suitable, an SVM or PLS regression or MLR may be used in the present disclosure. They work particularly well with the data associated with the present invention.

[0103] A model trained accordingly is particularly suitable for the above-described methods for predicting crop maturity. BRIEF DESCRIPTION OF THE DRAWINGS

[0104] In the following, the present disclosure is further described with reference to the enclosed figures:

[0105] Fig. 1 illustrates an exemplary system in which the method of the present disclosure may be carried out.

[0106] Fig. 2 illustrates an exemplary method for predicting a crop maturity according to the present disclosure.

[0107] Fig. 3 illustrates an exemplary method for training a machine learning model according to the present disclosure.

[0108] Figs. 4a and 4b illustrate exemplary RGB images at multiple time points, per-plot greenness data and a per-plot logistic function as a schematic drawing and depicted in a photograph.

[0109] Fig. 5 illustrates an example of performance of prediction models.

[0110] DETAILED DESCRIPTION OF EMBODIMENTS

[0111] The following embodiments are mere examples and shall not be considered limiting.

[0112] Figure 1 shows a system 1 according to the present disclosure. The system comprises a computing system 2 configured to carry out a method for predicting crop maturity according to the present disclosure, such as shown in and described in the context of Figure 2, for example comprising: providing aerial image data based on one or more aerial images of a crop plot of a field; determining an analytic function representative of a vegetation index of the crop plot as a function of time, the vegetation index derived from the aerial image data; applying a machine learning model, the model trained to use coefficients of the analytic function as input for predicting crop maturity, to predict a time of crop maturity per crop plot. The system may also be configured to carry out a training method for training the machine learning model according to the present disclosure, such as described below making reference to Figure 3.

[0113] The computing system may be a centralized or a distributed computing system.

[0114] The system optionally comprises one or more imaging devices 3 for capturing / acquiring the aerial images. The one or more imaging devices are shown in Figure 1 as mounted on a drone 4 and a satellite 5 as non-limiting examples. However, the imaging devices may be mounted, for example, to any kind of airborne vehicle. The imaging devices may, for example, be configured to provide multispectral images of the crop plot. As an example, the imaging devices may comprise a camera that may, for example, be mounted to a drone.

[0115] Figure 1 , for the sake of illustration, also shows a data storage device 6 that is external to the imaging devices and the computing system. The data storage device may be configured to exchange data with the imaging devices and the imaging systems, for example by wired or wireless data connections 7. The system may be configured such that aerial image data is provided by the imaging devices to the storage device and provided by the storage device to the computing system. Alternatively or in addition, the system may be configured such that the imaging devices provide the aerial image data directly to the computing system via wired or wireless data connections 8.

[0116] For the sake of illustration, Figure 1 also shows an agricultural field 10, divided into crop plots 11.

[0117] Optionally, the system, particularly the computing system 2, may comprise a unit 2a configured to output the crop maturity and / or data derived therefrom, such as scheduling data, to a user via a user interface and / or to one or more control devices 12a for controlling one or more harvesters 12.

[0118] Optionally, at least some of the one more control devices 12a may be part of the computing system 2 and / or part of the harvester 12 and / or external to the computing system 2 and the harvester 12.

[0119] The system of the present disclosure may comprise a training computing system 13 configured to train the machine learning model, for example external to or as part of the computing system 2.

[0120] Figure 2 illustrates an exemplary method for predicting crop maturity according to the present disclosure.

[0121] The method of the present disclosure comprises, in step S12, providing aerial image data based on one or more aerial images of a crop plot of a field. As an example, the crop may be soybeans, but other crops are also conceivable, as outlined above. The field may be a trial field associated with breeding trials or it may be any other type of agricultural field. The data may obtained, for example, by retrieval from data storage and / or by receiving it from an external device, such as an imaging device. In particular, the aerial image data may be based on two or more, in particular three or more, in particular four or more aerial images of the crop plot.

[0122] The method of the present disclosure comprises, in step S13, determining an analytic function representative of a vegetation index of the crop plot as a function of time, the vegetation index derived from the aerial image data.

[0123] For example, the analytic function may be a logistic function with at least three, in particular at least four, in particular exactly four, coefficients. However, other analytic functions may be used.

[0124] The vegetation index may comprise a vegetation index representative of (canopy) greenness (canopy greenness index). The canopy greenness index may be VARI or MCARI2, but other indices are also conceivable, as outlined above. The vegetation index is determined individually for each crop plot according to the present example. In other words, the vegetation index value is plotspecific.

[0125] The analytic function may be determined based on data comprising, for each of multiple time points, a corresponding value of the vegetation index. Any known suitable fitting methods may be used for determining the function, for example.

[0126] In particular, the above may be carried out for multiple, in particular all crop plots of a field.

[0127] The method of the present disclosure comprises, in step S14, applying a machine learning model, the model trained to use coefficients of the analytic function as input for predicting crop maturity, to predict a time of crop maturity per crop plot. An exemplary training method according to the present disclosure for obtaining such a trained model is described in the context of Figure 3.

[0128] In optional step S15, it is determined when to harvest the crop on a field based on the determined crop maturity.

[0129] In optional step S16, a harvesting schedule for the field and optionally for one or more other fields having the same crop is determined based on the determined crop maturity. In optional step S17, the crop maturity or data derived therefrom, such as a time to harvest a crop or scheduling data, like a harvesting schedule, is output to a user via a user interface and / or is output to a harvester control device for controlling a harvester.

[0130] In optional step S18 the crop maturity or data derived therefrom, such as scheduling data or time of harvesting, is used to trigger a harvesting operation, particularly trigger automatic steps thereof.

[0131] In optional step S11 , which precedes claim S12, aerial images of the crop plot are acquired by an imaging device. Acquiring aerial images may be carried out at multiple time points over a senescence period of the field. For example, the aerial images may be acquired at predetermined variable intervals within the senescence period, the intervals within a range of allowed intervals or being fixed intervals, in particular, every 1 to 20 days, in particular every 3 to 14, in particular every 3 to 7 days, in particular every 4 to 5 days, or every 4 days or every 5 days. As an example, at variable intervals at time points that are selected taking into account environmental conditions and / or taking into account a predetermined target interval.

[0132] In optional step S10, which precedes claim S12, the machine learning model is trained to use coefficients of the analytic function as input for predicting crop maturity, particularly to predict a time of crop maturity per crop plot. For the training, ground truth data may, for example, comprise data derived from visually assessed maturity data.

[0133] Figure 3 illustrate an exemplary method for training a machine learning model according to the present disclosure.

[0134] The method comprises training a model, in step S22, to predict a crop maturity on the basis of an input comprising coefficients of an analytic function representative of a vegetation index of a crop plot of a field as a function of time, the vegetation index derived from training aerial image data.

[0135] In optional step S20a, aerial image data are acquired for one or more fields and in step S21 a, at least part of the training aerial image data are derived from the acquired image data.

[0136] In optional step S20b, artificial aerial images are generated and in step S21 b, at least part of the training aerial image data are derived from the artificial aerial images.

[0137] In optional step S21c, at least part of the training aerial image data are derived from artificially created aerial image data, particularly without previous generation of artificially generated aerial images.

[0138] In step S23, the machine learning model is trained using the coefficients of the analytic function and ground truth data.

[0139] The present disclosure also provides a machine learning model trained to predict a crop maturity on the basis of an input comprising coefficients of an analytic function representative of a vegetation index of a crop plot of a field as a function of time, for example using the method described in the context of Figure 3.

[0140] Moreover, the present disclosure provides a use of the trained machine learning model for the method of the present disclosure, for example as described in the context of Figure 1 .

[0141] The present disclosure provides a use of the method of the present disclosure for determining a harvesting schedule and / or a time to harvest a crop, and optionally outputting the crop maturity and or data derived therefrom, such as the harvesting schedule and / or time to harvest the crop, to a user via a user interface and / or to a harvester control device, e.g. for controlling a harvester based on the harvesting schedule. Furthermore, the method may be used for triggering, e.g. automatically, harvesting, wherein the crop maturity or data derived therefrom, such as scheduling data or time of harvesting, is used to trigger harvesting.

[0142] The method may also be used for breeding purposes, such as to estimate maturity time for new varieties being developed.

[0143] The predicted crop maturity can be used is for making (seed) selection decisions, e.g. to select for certain maturity characteristics for breeding. Alternatively or in addition, the predicted crop maturity can be used is grouping varieties of similar maturity in the field to improve efficiency of running yield trials.

[0144] In the following further advantages and features of the method according to the present disclosure will be provided.

[0145] Maturity can be interesting for harvesting operations. Moreover, breeders may want to estimate maturity of new varieties that are being developed. The crop maturity is interesting for making (seed) selection decisions, e.g. to select for certain maturity characteristics for breeding. Alternatively or in addition, the crop maturity is interesting for grouping varieties of similar maturity in the field to improve efficiency of running yield trials.

[0146] Currently, maturity is estimated subjectively, by visual assessment, which is time-consuming and subjective. The present disclosure provides a high-throughput objective method. It is time saving and yields higher-quality reproducible data.

[0147] Among others, the present disclosure makes the subjective manual (visual) assessment of maturity redundant, i.e. , it can be replaced by high-throughput digital (objective) assessment. Turnaround times for receiving information on the crop maturity are short.

[0148] Drone-based techniques may be used, which allows for collecting data often and for many plots at little effort.

[0149] In general, the present disclosure is applicable for different types of crops, but hereinbelow soybeans are often discussed as an example. Maturity of a soybean variety is the length of time from planting to physiological maturity (just prior to harvest).

[0150] A general example of a prediction method (for an unknown field / plot) is described below. The trained machine learning model may be the one obtained by the described training example.

[0151] Aerial images of one or more unknown plots may be acquired and the canopy greenness values may be analytically described as a logistic function (or, in the alternative, a polynomial function). The coefficients of the logistic function may be input into the trained machine learning model and the machine learning model may output the predicted maturity for the unknown plot.

[0152] It has been found that the method takes significantly (in some experiments about 3-5 times) less time compared to manual (visual) assessment. It allows for covering more / all plots and results in higher-quality (objective) data.

[0153] The data may, for example, be used as assistance to breeders. Usually, assignment to maturity groups is done in early breeding stages. When using the present method, for example, in early breeding stages (progeny rows), this allows placement of genotypes in correct maturity groups which allows to avoid spending resources on “wrong” material later.

[0154] An example for a method for prediction and an example for a training method are provided below. In this example, aerial RGB images of fields may be collected during senescence process, for example, every 3 - 7 days, although other intervals may be used. Time series of canopy greenness values are extracted from field images for every field plot. For example, per-plot greenness values may comprise values of the VARI vegetation index.

[0155] Each time series is analytically described as a logistic function, although alternatively a polynomial function or other functions may be used.

[0156] For example, for every plot, the VARI time curve may be described as a 4-term logistic function: y - Vmin + (Vdelta / (1 + exp((TLS - x) / b))), where y is VARI and x is days after a start date.

[0157] Figs. 4a and 4b illustrate exemplary RGB images at multiple time points, per-plot greenness data and a per-plot logistic function obtained in accordance with the above, as a schematic drawing and depicted in a photograph, respectively.

[0158] A trained machine learning model is used to predict a maturity time. To that end, regression coefficients of the logistic functions are used as inputs for the machine learning model, based on which the per plot maturity time is predicted. For example, the model may use Vmin, Vdelta, TLS, and b as inputs (features)

[0159] As can be understood from the above, the trained model is used to predict maturity of an unknown plot from its senescence curve described by logistic function.

[0160] For providing such a trained machine learning model, as an example, the following steps may be performed.

[0161] First, the above steps (prior to prediction) may be carried out for training image data. For example, training aerial RGB images of fields may be collected during senescence process, for example, every 3 - 7 days, although other intervals may be used. Time series of canopy greenness values are extracted from field images for every field plot. For example, per-plot greenness values may comprise values of the VARI vegetation index. Each time series is analytically described as a logistic function, although alternatively a polynomial function or other functions may be used.

[0162] Again, for example, for every plot, the VARI time curve may be described as the 4-term logistic function: y = Vmin + (Vdelta / (1 + exp((TLS - x) / b))), where y is VARI and x is days after a start date.

[0163] Regression coefficients of the logistic functions (as independent variables) and visual maturity assessments (as dependent target variables) are used to train the machine learning model, also referred to prediction model. For example, the model may be based with Vmin, Vdelta, TLS, and b as inputs (features) and visual maturity as target. The visually assessed maturity data may be, thus, considered ground truth or target data.

[0164] Two modelling approaches will be outlined below that are suitable for predicting crop maturity on the basis of vegetation indices, specifically MCARI and VARI in these examples.

[0165] Modelling approach 1 :

[0166] In this approach, as in the first approach, features describing a canopy greenness time curve are used as X and visual maturity ground truth data as y. For example: VARI as greenness, PLS regression; VARI as greenness, SVM regression; MCARI2 as greenness, PLS regression; MCARI2 as greenness, SVM regression. However, in this approach regression coefficients of the logistic regression are used (instead of 2ndorder polynomial regression).

[0167] The features (greenness time curve features) may be extracted as follows: For every plot, calculate 4-term logistic regression equation y = Vmin + (Vdelta / (1 + exp((TLS - x) / b))), where y is VARI or MCARI2 and x is days after a start date.

[0168] Use regression coefficients Vmin, Vdelta, TLS, and b as model inputs.

[0169] For training, the data may be split into training and validation sets (such as 75 % training, 25 % validation). The regression models may be trained using PLS and SVM or MLR algorithms. The models may then be validated by testing performance using independent validation set.

[0170] Modelling approach 2:

[0171] In this approach, features describing a canopy greenness time curve, in this case regression coefficients of the 2ndorder polynomial regression, are used as X and visual maturity ground truth data as y. For example: VARI as greenness, PLS regression; VARI as greenness, SVM regression; MCARI2 as greenness, PLS regression; MCARI2 as greenness, SVM regression.

[0172] The features (greenness time curve features) may be extracted as follows:

[0173] For every plot, calculate 2nd order polynomial regression equation days after start date.

[0174] The regression coefficients bO, b1, and b2 are used as model inputs.

[0175] For training, the data may be split into training and validation sets (such as 75 % training, 25 % validation). The regression models may be trained using PLS and SVM or MLR algorithms. The models may then be validated by testing performance using independent validation set.

[0176] The plots of Fig. 5 demonstrate the predictive ability of test models, e.g. from Modelling approach 1 , as tested using one of the independent data sets.

[0177] Specifically, Fig. 5 illustrates an example of performance of prediction models, that is PLS VARI, SVM VARI, PLS MCARI2, and SVM MACARI2, respectively. R2 is the determination coefficient and SEP is the standard error of prediction. This is an example of performance of prediction models as tested using one of the independent test sets. It is to be understood that usually performance varies depending on a test set and version of a prediction model.

[0178] The present disclosure has been described in conjunction with preferred embodiments and examples as well. However, other variations can be understood and effected by those persons skilled in the art and practicing the claimed invention, from the studies of the drawings, this disclosure, and the claims. Notably, unless otherwise specified, the any steps presented can be performed in any order, i.e., the present invention is not limited to a specific order of these steps. Moreover, unless specified otherwise, it is also not required that the different steps are performed at a certain place or at one computing device of a distributed system, i.e., each of the steps may be performed at different computing devices.

Claims

CLAIMS1 . A computer-implemented method for predicting crop maturity, comprising: providing (S12) aerial image data based on one or more aerial images of a crop plot of a field; determining (S13) an analytic function representative of a vegetation index of the crop plot as a function of time, the vegetation index derived from the aerial image data; applying (S14) a machine learning model, the machine learning model trained to use coefficients of the analytic function as input for predicting crop maturity, to predict a time of crop maturity per crop plot.

2. The computer-implemented method according to claim 1 , the aerial images having been acquired at multiple time points over a senescence period of the field, in particular, the aerial images having been acquired for one or more crop plots of the field.

3. The computer-implemented method according to claim 1 or 2, wherein the analytic function is a logistic function, particularly having at least 3 coefficients, particularly at least 4 coefficients, particularly exactly 4 coefficients.

4. The computer-implemented method according to claim 2 or 3, the method utilizing, for determining the analytic function, maturity data, the maturity data comprising, for each of the multiple time points, a value representative of a greenness for the respective crop plot, in particular a value of the vegetation index.

5. The computer-implemented method according to any of the preceding claims, wherein training data used for training the machine learning model may comprise visually assessed maturity data.

6. The computer-implemented method according to claim 5, the machine learning model having been trained using ground truth data, in particular target data, derived from the visually assessed maturity data.

7. The computer-implemented method according to any of claims 2 to 6, wherein the senescence period of the field starts with the onset of senescence and ends with all crop plots of the field having reached maturity.

8. The computer-implemented method according to any of the preceding claims, wherein the vegetation index may comprise a greenness index, particularly a canopy greenness index, such as Visible Atmospherically Resistant Index, VARI and / or Modified Chlorophyll Absorption Ratio Index Improved, and / or MCARI2, and / or normalized difference red edge index, NDRE, and / or normalized difference vegetation index, NDVI.

9. The computer-implemented method according to any of the preceding claims, wherein the vegetation index, in particular a / the greenness index, is determined for each crop plot of the field individually.

10. The computer-implemented method according to any of claims 2 to 9, the aerial images having been acquired at predetermined variable intervals within a range of allowed intervals or at fixed intervals,for example, at least every 20 days, in particular, at least every 14 days, in particular at least every 7 days, in particular at least every three days, in particular at least every five days, in particular at least every other day, in particular every day, or for example, every 1 to 20 days, in particular every 3 to 14 days, in particular every 3 to 7 days, in particular every 4 to 6 days, in particular every 4 to 5 days, or for example, every X days, X being an integer between 1 and 20, in particular s and 14, in particular 3 and 7, in particular every 3 days or every 5 days.11 . The computer-implemented method according to any of claims 2 to 10, the aerial images having been acquired at variable intervals at time points that are selected taking into account environmental conditions and / or taking into account a predetermined target interval.

12. The computer-implemented method according to any of claims 2 to 11 , wherein the analytic function is determined based on data comprising, for each of the multiple time points, a corresponding value of the vegetation index.

13. The method of any of the preceding claims, comprising at least one of: determining, based on the determined crop maturity or data derived therefrom, when to harvest the crop on a field, determining, based on the determined crop maturity or data derived therefrom, a harvesting schedule for one or more fields, outputting, to a harvester and / or control device for controlling a harvester and / or to a user, an indication when to harvest the crop on a field, the indication based on the determined crop maturity or data derived therefrom, in particular outputting the maturity time and / or outputting the scheduling data, outputting selection support data for making a selection decision in breeding and / or trait development, outputting grouping support data for grouping crop varieties in the field in accordance with crop maturity.

14. The method according to any of the preceding claims, comprising training the machine learning model using, as input for the training, the coefficients of an analytic function representative of a vegetation index as a function of time, the vegetation index derived from training aerial image data, in particular aerial image data derived from acquired images of one or more fields and / or the aerial image data derived from artificially generated aerial images and / or the aerial image data being artificially generated.

15. Use of the method of any for the preceding claims for at least one of: determining when to harvest the crop on a field, determining a harvesting schedule for one or more fields, outputting, to a harvester and / or control device for controlling a harvester and / or to a user an indication when to harvest the crop on a field and / or the harvesting schedule, outputting selection support data for making a selection decision in breeding and / or trait development,outputting grouping support data for grouping crop varieties in the field in accordance with crop maturity.

16. A training method comprising training of a machine learning model, wherein the training of the machine learning model comprises training the machine learning model to predict a crop maturity on the basis of an input comprising coefficients of an analytic function representative of a vegetation index of a crop plot of a field as a function of time, the vegetation index derived from training aerial image data, in particular aerial image data derived from acquired images of one or more fields and / or the aerial image data derived from artificially generated aerial images and / or the aerial image data being artificially generated.

17. A trained machine learning model configured to predict a crop maturity on the basis of an input comprising coefficients of an analytic function representative of a vegetation index of a crop plot of a field as a function of time.

18. The trained machine learning model of claim 17, trained using the training method of claim 16.

19. Use of the trained machine learning model of claim 17 or 18 for the method of claims 1 to 14.

20. A system comprising a computing system configured to carry out the method of any of the preceding claims.21 . A computer program product comprising instructions which, when the program is executed by a computing system, cause the computing system to carry out the method of claims 1 to 14.

22. A computer-readable medium having stored thereon instructions which, when the program is executed by a computing system, cause the computing system to carry out the method of claims 1 to 14.

Images