Digital twin framework for identifying best agricultural fields for conducting experiments across large geographical regions, depending on research objectives
By using clustering algorithms to identify fields closest to their centroids within a multi-dimensional parameter space, the method enhances the representation and diversity of agronomic field trials, addressing the limitations of current trial designs and improving data quality.
Patent Information
- Application Number
- PCT/IL2024/050463
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-22
- Filing Date
- 2024-05-13
- Publication Date
- 2024-11-28
AI Technical Summary
Current agronomic field trials lack a validated mechanism to ensure that selected fields adequately cover the necessary parameters affecting experimental subjects, leading to limited utility of gathered data due to inadequate representation of effecting factors.
The method employs clustering techniques to determine optimal locations for agronomic experiments by specifying relevant parameters, mapping them into a multi-dimensional space, and applying algorithms to maximize diversity, representation, or balance both, thereby identifying fields closest to their respective centroids for targeted experimentation.
This approach ensures better analysis and elucidation of product responses by selecting fields that collectively cover the parameter space, significantly improving the data quality and utility compared to random selection methods.
Smart Images

Figure IL2024050463_28112024_PF_FP_ABST
Abstract
Description
Digital Twin Framework for Identifying Best Agricultural Fields For Conducting Experiments Across Large Geographical Regions, Depending on ResearchObjectivesField of the Invention
[0001] The present invention relates to the field of agronomic trials, and more specifically, but not exclusively, to systems and methods for identifying optimal locations of agricultural fields for conducting agronomic experiments over a large geographic area, depending on research objectives.Background of the Invention
[0002] Agronomic field trials are used to evaluate the impact of soil and crop treatments, such as fertilizer or pesticides, on multiple aspects of crop production. Agronomic field trials are critical for testing hypotheses, validating the success of a new crop variety, generating new management practices, and establishing the efficacy and safety of new commercial products.
[0003] When designing an agronomic field trial, the parameters effecting the main subject of the experiment must be considered. These parameters include, for example, rainfall amount and intensity, temperature along the growing season, multiple soil characteristics such as organic content, texture, porosity or acidity, different field management procedures such as tillage, planting of cover crops, and many more. For a given geographic region, these parameters will each have a range of values, which needed to be covered in the field experiments to allow understanding of how they interact and effect the subject of the experiment.
[0004] Typically, companies seeking to implement new agricultural trials lack any validated mechanism for ensuring that the trials implemented cover the main factors effecting the experimental subject. Instead, the companies typically find fields through Contract Research Organizations (CRO), which find field owners willing to participate in the experiments, and are located in a geographical area where product validation is needed. Correspondingly, the fields chosen for the experiments do not necessarily cover adequately the parameter space of effecting factors, which is needed in order to elucidate the product behavior and response. As such, the data that is gathered from these trials is of more limited utility.Summary of the Invention
[0005] The present disclosure provides systems and methods for determining optimal agricultural fields for conducting agronomic field trials over a large geographic area. The disclosure uses different clustering techniques from the field of statistics, applied to any population of fields in the relevant geographic area, to determine both an ideal number of fields to test and ideal locations for such fields. The parameters used for the selection may be specifically chosen and tailored to meet the needs of the agricultural trial that is being conducted. In addition, the number of fields may be tailored according to the needs and resources of the parties conducting the trials. As a result, the agricultural field trials are conducted on fields that collectively, enable better analysis and elucidation of tested product responses to multiple effecting parameters.
[0006] According to a first aspect, a method for determining optimal locations of agricultural fields for conducting agronomic experiments across largegeographical regions is disclosed. The method includes: specifying a plurality of effecting parameters relevant to an agronomic experiment; joining values of the plurality of effecting parameters, for each field in a geographical region, into layers; mapping values of the layers for each field to an N-dimension parameter space, where N is the number of layers; selecting a clustering algorithm for application onto the parameters based on desired goals for coverage of the parameter space, from among a first clustering algorithm configured to maximize diversity of the selected regions, a second clustering algorithm configured to maximize representation of the parameter space, and a third clustering algorithm configured to balance diversity and representation of the parameter space; performing the selected clustering algorithm to thereby divide the parameter space into clusters and assign each field to a specific cluster; determining a centroid of each of the clusters; calculating a Euclidean distance between coordinates of each field in the parameter space and the coordinates of each centroid; ranking the fields in accordance with the Euclidean distances, and identifying the field that is closest to its centroid; and suggesting the closest field to each centroid as a potential location for conducting the agronomic experiments.
[0007] In another implementation according to the first aspect, the first clustering algorithm comprises dividing the parameter space into equal-sized regions.
[0008] In another implementation according to the first aspect, the second clustering algorithm comprises dividing the parameter space into regions sized according to a predefined tolerance, such that, for each region, the centroid of theregion is representative of the entire region when considering the predefined tolerance. Optionally, the predefined tolerance is + / - 5% for each parameter in the parameter space.
[0009] In another implementation according to the first aspect, the third clustering algorithm comprises determining an ideal number of clusters according to the distribution of the fields in the parameter space. Optionally, the plurality of effecting parameters comprises a single parameter, and the clustering algorithm is a Jenks Natural Break Algorithm. Optionally, the step of analyzing outputs of the clustering algorithm to determine the ideal number of clusters comprises evaluating the Goodness of Variance Fit (GVF) factor. Further optionally, the method further includes setting the ideal number of clusters based on a predetermined threshold for improvement in GVF factor resulting from increasing the number of clusters.
[0010] Optionally, the plurality of effecting parameters comprises two or more parameters, and the clustering algorithm is a K-means clustering algorithm. Further optionally, the step of analyzing outputs of the clustering algorithm to determine the ideal number of clusters comprises applying an ensemble approach based on different validation algorithms for determining the ideal number of clusters. Optionally, the validation algorithms include Silhouette, VRC, and GAP.
[0011] In another implementation according to the first aspect, the method further includes, in response to user feedback specifying a target number of fields to sample, adjusting the number of clusters.
[0012] In another implementation according to the first aspect, the step of specifying a plurality of parameters comprises specifying a range of acceptable values for each parameter.
[0013] In another implementation according to the first aspect, the method further includes identifying the locations of the suggested fields according to latitude and longitude. Optionally, the method further includes identifying the locations of the fields at a resolution of 100 meters x 100 meters. Optionally, the method further includes displaying the locations of the fields on a pictorial representation of the geographical region.
[0014] According to a second aspect, a system for determining optimal locations of agricultural fields for conducting agronomic experiments across large geographical regions is disclosed. The system includes a non-transitory computer- readable medium storing instructions, that, when implemented by a processor, causes the performance of the following steps: specifying a plurality of effecting parameters relevant to an agronomic experiment; joining values of the plurality of effecting parameters, for each field in a geographical region, into layers; mapping values of the layers for each field to an N-dimension parameter space, where N is the number of layers; selecting a clustering algorithm for application onto the parameters based on desired goals for coverage of the parameter space, from among a first clustering algorithm configured to maximize diversity of the selected regions, a second clustering algorithm configured to maximize representation of the parameter space, and a third clustering algorithm configured to balance diversity and representation of the parameter space; performing the selected clusteringalgorithm to thereby divide the parameter space into clusters and assign each field to a specific cluster; determining a centroid of each of the clusters; calculating a Euclidean distance between coordinates of each field in the parameter space and the coordinates of each centroid; ranking the fields in accordance with the Euclidean distances, and identifying the field that is closest to its centroid; and suggesting the closest field to each centroid as a potential location for conducting the agronomic experiments.Brief Description of the Drawings
[0015] FIGS. 1A-1D illustrates steps in a method of determining optimal locations for performing agronomic field trials, according to embodiments of the present disclosure;
[0016] FIG. 2A illustrates a map of agricultural fields within a large geographic area, according to embodiments of the present disclosure;
[0017] FIG. 2B illustrates a random number of fields selected from the map of FIG. 2A, according to embodiments of the present disclosure;
[0018] FIGS. 3A-3B illustrate application of a first clustering algorithm for purposes of selecting fields with a high diversification, according to embodiments of the present disclosure;
[0019] FIGS. 4A-4B illustrate application of a second clustering algorithm for purposes of selecting fields with a high representation of the parameter space, according to embodiments of the present disclosure;
[0020] FIG. 5 illustrates application of a third clustering algorithm for purposes of selecting fields with a combination of high diversification and highrepresentation of the parameter space, for a single parameter, according to embodiments of the present disclosure;
[0021] FIGS. 6A-6C illustrate comparison of selection of the fields according to the third clustering algorithm with random selection of the fields;
[0022] FIGS. 7A-7C illustrate locations of the fields selected according to each of the clustering algorithms, according to embodiments of the present disclosure;
[0023] FIG. 8: Potato fields in the state of Idaho, USA.
[0024] FIG. 9 shows a 2D parameter space delineated into 9 regions;
[0025] FIG. 10 shows the region (indicated by numeral 8) where the pareto optimization is demonstrated;
[0026] FIG. 11 shows a 2D projection of the data in the region 8 of FIG. 10, presenting for each field its cost and its distance from the region centroid. The fields indicated by the strong dots are unique fields dominating all other fields in the region;
[0027] FIG. 12 shows 41 fields identified as optimal across all nine regions of FIG. 9;
[0028] FIG. 13 shows 430K different combinations of 9 locations - one in each region - mapped into a 2D space of cost and distance from centroid;
[0029] FIGS. 14A-14C are the final 9 fields selected according to different optimization - experimental cost, distance from centroid, or both, respectively; and
[0030] FIGS. 15A-15C show 9 selected fields in 3 different optimizations on a map.Detailed Description of the Invention
[0031] The present invention relates to the field of agronomic trials, and more specifically, but not exclusively, to systems and methods for identifying the optimal locations of agricultural fields for conducting agronomic experiments over a large geographic area, according to the research objectives.
[0032] Before explaining at least one embodiment of the invention in detail, it is to be understood that the invention is not necessarily limited in its application to the details of construction and the arrangement of the components and / or methods set forth in the following description and / or illustrated in the drawings and / or the Examples. The invention is capable of other embodiments or of being practiced or carried out in various ways.
[0033] The systems and methods described herein are implemented on a computer. The computer may include a memory, which is non-transitory computer- readable medium having instructions stored thereon. The computer may further include a processor configured to execute the instructions to carry out the instructions stored on the memory. The computer may be a local computer, a virtualized computer, and / or a cloud-based computer. The computer may include a plurality of memories and processors configured to operate together in order to perform the calculations described herein. The system may further include a plurality of databases for storing agronomic data, and at least one database management system for accessing the data.
[0034] In particular, the system utilizes a database in which agronomic data is organized in a knowledge graph. As used in the present disclosure, a "knowledgegraph" is a knowledge base that uses a graph-structured data model or topology to organize data. Knowledge graphs are used to store interlinked descriptions of entities while also encoding the semantics underlying the used terminology. As relevant to the present disclosure, the knowledge graph may be understood as an extensive spatial database of different layers - representing climate and soil parameters - and the interrelationships between these parameters and crop and soil outcomes. The knowledge graph includes the interrelationships between various environmental parameters, agronomic treatments, crop diseases, and crop growth. The environmental parameters may include, for example, rainfall, sunlight, temperature, acidity of soil, and organic content in soil. The agronomic treatments may include certain fertilizers or pesticides. The crop diseases may include various types of bacteria or fungi that affect crops. Crop growth refers to the rate of growth of particular crops, such as wheat, corn, or potatoes. Thus, for example, the knowledge graph may include the effect that an acidic soil may have on growth of wheat, or on the ability of a particular fungus to proliferate on corn.
[0035] The information that forms the basis of the knowledge graph may come from various sources. For example, much agronomic information is collected by public agencies, such as the US Department of Agriculture, and made publicly available. Additional information is published in peer-reviewed journals, and still other sources of information may be privately-held data that is stored in proprietary databases.
[0036] FIGS. 1A-1D illustrate steps in a method 100 for selecting agricultural fields for conducting agronomic experiments across large geographical regions. FIG.1A represents steps that are common to each of the selection methods, while FIGS. 1B-1D represent steps that differ based on the particular goals of the experiment.
[0037] At step 101, using the knowledge graph, a user specifies a plurality of effecting parameters for a specific experiment. The user identifies a specific hypothesis to test regarding how certain conditions will affect certain results. Based on the knowledge graph, the user selects a parameter that is expected to affect the hypothesis. In addition, the computer system may automatically suggest the parameter based on the hypothesis. Typically, the hypotheses relate to growth of particular crops, or the proliferation of a certain crop disease, and the parameters which affect those results are environmental or soil conditions. The knowledge graph enables identification of all the relevant affecting parameters.Example 1
[0038] The following example illustrates application of the method. The chosen region is the U.S. state of Idaho. The chosen crop is potato (solamum tuberosum). The chosen crop disease is Powdery scab. Powdery scab is caused by the potato mop top virus (PMTV) and vectored by a cercozoan called Spongospora subterranea. The parameters that affect the growth of powdery scab (apart from an agricultural treatment that is being considered, which may be the purpose of conducting the test) are available root zone water storage, and percent of days within the growing season between 12 and 20 °C.
[0039] At step 102, the system joins values for the effecting parameters as layers, for all cultivated fields in a region of interest. As used herein, a "layer" refers to a set of information upon which the statistical analysis is to be applied. Therelevant information includes both the location of the fields (longitude and latitude) and values for each of the affecting parameters.
[0040] FIG. 2A illustrates a map 200 showing all of the approximately 200 thousand cultivated fields in the state of Idaho. Although the map is at relatively low resolution, the locations of each of the fields may be identified at a much higher resolution. For example, the fields may be identified at a resolution of 100 meters by 100 meters.
[0041] FIG. 2B illustrates a random subset 210 of one thousand fields in the state of Idaho. The one thousand fields are randomly selected out of the entire set of fields, for computational efficiency. In order to ensure that the sampling is representative, a subset of 1000 random fields was iteratively generated out of the dataset of 200,000 Idaho fields. This subset was then used to compare the different sampling approaches. For robustness, the results are reported from 100 iterations of the simulation. The calculations described in the remainder of this disclosure were performed on this subset of one thousand fields.
[0042] The set of mapped fields, with the relevant parameters of each field stored as layer for mathematical analysis, functions as a digital twin for simulating the entire geographical area. A "digital twin" is a digital representation of a physical object, process, service or environment that behaves and looks like its counterpart in the real-world. Here, the mathematical model of the fields represents the expected behavior of the actual fields.
[0043] At step 103, all the fields and their respective values of relevant layers are mapped to a N-dimension parameter space. The parameter space is also referredto herein as a feature space. N is the number of layers (i.e. effecting parameters). For purposes of the present illustration, the number of effecting parameters is two, thus, the parameter space is two-dimensional. Specifically, the two effecting parameters are available root zone water storage and percent of days within the growing season between 12 and 20 °C. Each of the fields is treated as a single data point with respect to both parameters. In addition, as will be described further herein, each of the two effecting parameters may be considered in isolation as an independent effecting parameter. Each data point representing a field also has associated therewith the latitude and longitude of the field; however, this location data is not considered for purposes of the analysis described herein.
[0044] At step 104, the user selects the desired clustering algorithm based on the goals of the experiment. In particular, while the purpose of an agricultural trial is to determine the efficacy of a product, each specific trial may have different objectives. For purposes of product development, it is desirable to determine whether the product is effective under diverse conditions. To that end, it is most important that the experiments cover a diverse range of the parameter space. In other circumstances, the product may be known to be generally effective, but there is an interest in marketing the product. In such circumstances, it may be desired to gather data on the highest possible coverage of data points in the parameter space. A third approach is to perform the experiments in a manner that maximizes, to the extent possible, both the diversity of the parameter space and the percent coverage of the parameter space.
[0045] Each of these approaches is now explicated in detail. For purposes of each approach, it is assumed that there is an interest in testing of nine fields, for two effecting parameters.
[0046] Referring now to FIGS. IB and FIGS. 3A-3B, the first approach, shown in FIG. IB as method 110, is to optimize the distribution of fields for product development. As seen in FIG. 3A, the effecting parameters are distributed on an X-Y scatter graph, with one axis representing the values of the first parameter, and the second axis representing the values of the second parameter. Assuming a request to identify nine fields, at step 111 of FIG. IB, the parameter space is then divided into regions, each representing a portion of the range of the parameter space. Here, because there is an interest in finding nine data points for two parameters, the parameter space is divided into a grid of nine two-dimensional regions having equal distribution along the X and Y axes. Obviously, the parameter space may be of any level of dimensions (e.g., 1, 2, 3, 4, etc.) depending on the number of effecting parameters. Similarly, the parameter space may be divided into any number of equal-sized regions, depending on the desired number of fields in which experiments are to be conducted.
[0047] At step 112, for each of the distributions of dots within a given region, the centroid of the region is calculated. At step 113, the field nearest to each centroid is identified. In FIG. 3A, these fields are indicated with the larger, more heavily shaded dots.
[0048] As discussed above, a diverse distribution of fields within the parameter space leads to effective diversity of the tested parameters. This diversityof coverage is especially desirable when testing for purposes of product development. The coverage of the parameter space may be quantified by calculating the total area of a polygon circumscribed by the fields that were selected in FIG. 3A. This polygon is illustrated in FIG. 3B, and, as can be seen, covers nearly the entire parameter space. In general, one advantage of this method is that, on the one hand, the chosen locations are spread around the "edges" of the parameter space, thus maximizing the diversity of the range that is selected, while, on the other hand, selection of the centroids of each region ensures that the values that are selected are not extreme outliers.
[0049] Finally, at step 114, the program maps the closest data points back to Cartesian space (e.g., latitude and longitude). The program suggests those fields to be potential locations for experiments. The suggested points are overlaid over a map of the state of Idaho in FIG. 7A.
[0050] FIG. 1C and FIGS. 4A-4B illustrate a second approach 120 for determining the ideal testing locations. In this method, the experiments are designed in order to prove that the product was tested under conditions that are applicable to the highest percentage of fields in the desired geographical region. The theory underpinning this model is that minor variations of characteristics of the fields, within certain tolerances, may still be considered to be reliable for purposes of marketing. For purposes of the present example, it is assumed that a selected point on the parameter space may be representative of up to 5% of its upper and lower neighboring values, for a total representation range of 10%. For example, a point with the parameter space coordinates of (3, 50) may be assumed to represent theparameter space range (2.85-3.15) on the x-axis, and (47.5-52.5) on the Y-axis. FIG. 4A illustrates nine fields selected from the parameter space, represented as points, with boxes around the points illustrating the boundaries of 10% ranges around the fields.
[0051] Referring now to FIG. 1C, at step 121, a user may select a tolerance for representation of values relative to a parameter space. For example, the range may be 10%, or stricter, or less strict. At step 122, based on the tolerance selected by the user, the program divides the parameter space into regions, with each region containing, at each boundary or edge, one range of the selected tolerance. For example, as shown in FIG. 4B, when the user selects a tolerance of 10%, the parameter space is divided into 100 blocks, with each block having 10% of the X-axis and 10% of the Y-axis. If the tolerance were 8%, then there would be 144 blocks (12x12), with the number of blocks being rounded to the closest integer value. If the tolerance were 12%, then there would be 64 blocks (8x8). The user may also select the tolerances for each of the respective parameters to be different from each other (for example, 10% for the first parameter and 8% for the second parameter), in which case there would be different divisions of regions on the X and Y axes. Regardless of the numbers that are used, the blocks are shaped so that, when the tolerance is considered, the center of each block effectively covers the entire parameter space delineated by that block.
[0052] At step 123, following the division into regions, a centroid is calculated for each region. The centroids of each of these 100 regions may be assumed to represent all of the fields contained in that region. Accordingly, in step124, these fields are accordingly translated into Cartesian space and recommended as locations for the testing. If there were a budget to test 100 fields, then those fields would be effectively representative of the entire parameter space, considering the 10% tolerance as discussed above. When it is possible to test only a smaller number of fields, these fields are selected, from among the 100 centroids, to maximize the coverage of the parameter space. Finally, at step 124, the program maps the centroids back to Cartesian space and suggests those locations as potential locations for experiments. The outcome of this mapping is illustrated in FIG. 7B.
[0053] Referring now to FIG. ID and FIGS. 5A-5D, a third approach is to use a clustering technique to identify the regions of interest in the parameter space. The clustering approach balances the approaches described in the first two methods. This is done using the following steps of method 130.
[0054] At step 131, the system identifies the ideal number of clusters and assigns each field to a specific cluster, using a clustering algorithm. A clustering algorithm is a statistical technique that involves the grouping of data points into clusters. Clustering algorithms interpret the input data and find natural groups or clusters of that data in the feature space. A cluster is identified as an area of density in the feature space where examples from the feature domain are closer to the centroid of that cluster compared to the centroids of other clusters.
[0055] Many clustering algorithms are presently in use, and, in principle, may be used for clustering of the agronomic data. These include: library installation, clustering dataset, affinity propagation, agglomerative clustering, BIRCH, DBSCAN, K-Means, Mini-Batch K-Means, K-Medians, Mean Shift, OPTICS, Spectral Clustering, and Gaussian Mixture Model.
[0056] Without limitation, for parameter spaces having a single effecting parameter, the present disclosure uses the Jenks Natural Break algorithm, which is illustrated in FIG. 5. Determining the optimal number of clusters in the natural break algorithm is decided using the Goodness of Variance Fit (GVF) factor. The GVF factor is described in Coulson, M. R., In The Matter of Class Intervals for Choropleth Maps: With Particular Reference to the Work of George F Jenks, Cartographica: The International Journal for Geographic Information and Geovisualization, 24(2), 16-39 (1987). The GVF is found by taking the difference between the Squared Deviations from the Array Mean (SDAM) and the Squared Deviations from the Class Means (SDCM), and dividing by the SDAM. The GVF factor calculation thus "rewards" clusters in which the values are close to the centroid of that cluster, relative to the other data values, and "penalizes" clusters for which the individual data points are not particularly close, relatively speaking, to the centroids of those clusters. The closer the GVF approaches 1.0, the better the optimization. A GVF of 1.0 would only be attained in the unlikely case that there was zero within-class variation. The value of the GVF factor increases as more clusters are added, because each data point is necessarily closer to the centroid of its cluster. This increase is illustrated in FIG. 5A, in which the clusters are labeled as "Jenks bins."
[0057] In the case here of optimizing the number of trials needed for an experiment, the benefit of adding a cluster has implication for experimental cost and logistical burden. It accordingly makes sense to find the number of clusters, wherebyadding an additional cluster does not lead to a "meaningful" increase in GVF value. Interpretation of what considers a meaningful increase in GVF needs to be considered in the context of the experimental topic. To consider this more fully, three thresholds of GVF score improvement - 1%, 3% and 5% - were considered. These approaches were considered using potato production in Idaho USA. Two hundred thousand cultivated fields in Idaho were mapped, and, for each field, the Available Water Storage (AWS, in mm) was retrieved, for the range of depths of 0-30 cm. For computational consideration, 1000 fields were randomly selected from the dataset of 200,000 fields, as discussed above in connection with FIGS. 2A and 2B. The Available Water Storage values of the selected subset of fields were subsequently subjected to a Jenks natural break classification, and the number of clusters was alternatively determined using the three thresholds (1%, 3% and 5%). The results for the root zone water storage layer are presented in Table 1.Table 1 -The Number of Clusters Based on 1%, 3% and 5% Thresholds for Increase in the Goodness of Variance Fit (GVF) FactorAs can be seen, the number of clusters is inversely related to the threshold value.
[0058] Without limitation, the present disclosure uses K-means clustering as a preferred clustering algorithm for parameter spaces having two or more effecting parameters. The algorithm of K-means clustering is well-understood on a basic level and need not be explicated here in detail. In brief, the algorithm begins with assignment of a certain number ("k") of centroids ("means") for the data points. For each data point, the Euclidean distance is calculated between that data point and each centroid, and the data point is assigned to the centroid with the closest Euclidean distance to it. Following initial classification, the centroids are recalculated for the data points assigned to each cluster. The process repeats for multiple iterations, until the locations of the centroids no longer change.
[0059] The k-means algorithm may further be optimized in order to determine the best number of clusters. Various algorithms may also be used in order to evaluate whether the selected clusters have values that are relatively close to each other, and whether a different number of clusters may result in cluster centers that more closely represent the different values in the feature space. These include: Silhouette, described in Rousseeuw, P. J., Silhouettes: A Graphical Aid to the Interpretation and Validation of Cluster Analysis, J. Comput. Appl. Math. 20 (1987), 53-65); Variance Ratio Criterion (VRC), as described in Calinski, T., and Harabasz, J., A Dendrite Method for Cluster Analysis, Communications in Statistics 3, 1 (1974), 1-27; and the "Gap Statistic," as described in Tibshirani, R., Walther, G., and Hastie, T., Estimating the Number of Clusters in a Data Set Via the Gap Statistic, Journal of the Royal Statistical Society: Series B (Statistical Methodology) 63, 2 (2001), 411-423.Other computational methods may also be used in order to verify the ideal number of clusters.
[0060] Preferably, an ensemble method is employed with respect to both the clustering and with respect to determining the ideal number of clusters. An ensemble method involves using multiple learning algorithms to obtain better predictive performance than could be obtained from any of the constituent learning algorithms alone. Thus, for example, each of the three algorithms described above, and / or any other suitable algorithms, may be executed. The results may be considered in the aggregate in order to obtain the most optimal value for the number of clusters.
[0061] To demonstrate the usefulness of such an ensemble approach, as previously, the subset of 1000 fields from the Idaho field dataset was considered. Two effecting variables were considered for the clustering - Available Water Storage (AWS) and the percent of days with mean daily temperature of 12-20 degrees Celsius. Using these two covariates, the optimal number of clusters was calculated according to three different methods. This procedure of random sampling was repeated 10 times to corroborate the results. The number of clusters calculated using each approach, as well of the mean value representing the ensemble approach, are summarized in Table 2. It is evident that in most cases, the three methods give different results. Therefore, an ensemble approach is applied, averaging the results of all three methods.Table 2. The Number of Clusters Calculated for Ten Randomly-Generated Subsets of the Data (subset size = 1000 fields)
[0062] The clustering framework is adaptive to the number of field locations the user wishes to identify. If the amount of locations is lower than the optimal K classes value identified by the algorithm for a specific experiment, the clustering is re-run using a value equal to user prescription. If the amount of locations is higher than the optimal K classes value identified by the algorithm for a specific experiment, the user is notified and decides whether to reduce the number of fields, or to add more clusters.
[0063] Referring again to FIG. ID, following the setting of the desired number of clusters, at step 132, a centroid is determined for each cluster. At step 133, a Euclidean distance is calculated between each agricultural field in a given cluster and the centroid of that cluster. The fields are ranked in accordance with the Euclidean distances. Although the calculation of Euclidean distance may be part of the k-means clustering algorithm, as discussed, here it is specifically performed as part of the process for determining an optimal field for the agronomic experiment. The datapoint that is closest to the centroid of each cluster is identified. This is the data point that is expected to represent the cluster centroid.
[0064] At step 134, the data points are mapped back to Cartesian space, and are suggested as potential locations for experiments. This process is illustrated in FIG. 7C, in which the field locations are superimposed over a map of the state of Idaho.
[0065] Notably, and referring in aggregate to FIGS. 7A, 7B, and 7C, the fields are not distributed evenly geographically. Many of the agricultural fields that are selected are relatively close to each other, and certain of the fields are comparatively far from each other. It is unlikely that a researcher selecting locations ad-hoc would pick locations with this exact geographic distribution. This distribution is made possible only through use of the clustering algorithm or other algorithms described herein.
[0066] The process described above outputs the ideal number of experiments and ideal locations for those experiments. In practice, however, it may not be possible for a researcher to conduct that number of experiments. For example, although the ideal number of clusters may be nine, the researcher may have a budget for performing experiments in only six fields. If the amount of available locations is lower than the optimal K classes value (or value of Jenks bins) identified by the algorithm for a specific experiment, the clustering is re-run using the k classes value equal to user prescription. Conversely, the user may have a budget for even more experiments than nine, and may wish to utilize the entire budget. If the amount of locations is higher than the optimal K classes value (or valueof Jenks bins) identified by the algorithm for a specific experiment, each cluster is split and new centroids are identified and used for the calculation. Thus, the framework is adaptive to the amount of field locations the user wishes to identify. Of course, it is possible to set the number of classes initially, without separately calculating the ideal number of classes, if desired.
[0067] The parameters that define the feature space may likewise be adjusted at will. For example, suppose that the product that is being tested has an ideal temperature range of 14-18 °C, even though the disease that it operates on has an ideal growth temperature range of 12 -20 °C. The user is able to identify the region of the parameter space on which the experiments will be performed, and to tailor the clustering to the desired ranges of the parameter space. The modification of the boundaries of the parameter space may be imagined as a "sliding scale," with the user having the ability to adjust the boundaries with a slider, re-determine the distribution of the data within the feature space, and correspondingly re-determine the ideal clustering and locations for the testing.
[0068] In the following examples, each of the three approaches described above is compared to random selection of fields for sampling. Data is provided below, which demonstrates that each of the approaches described above provided better coverage of the field parameters compared to random selection of fields.Example 2
[0069] First considering the first approach, for maximizing diversity of the parameter space, testing was performed with a simulation. For 100 iterations, 1000 fields from the Idaho dataset were sampled, and 9 fields were sampled out of thissubset either randomly or using the regions approach described in relation to FIG. IB and 3A-3B. The total area of the parameter space (as illustrated in FIG. 3B) was calculated using i) all the 1000 fields in the subset; ii) the 9 randomly chosen fields; and iii) the approach of FIGS. IB and 3A-3B. Once the parameter space area was quantified for all three cases, the percentage of the total parameter space covered by either the random or the disclosed approach was calculated. The value reported is the difference between the approach of the present disclosure and the random one.
[0070] Table 3 presents a summary statistic (minimal value, 25thpercentile, median, mean, 75thpercentile, and maximal value) of the difference based on the 100 simulations. On average, the approach of FIG. IB covers 74% of the total possible parameter space, compared to 43% for the random approach. In terms of difference, the disclosed approach leads to an increase of 86% percent in the coverage of the parameter space compared to the case of random sampling. This difference was found statistically significant (a=0.05, p<0.05) and represents a substantial increase in parameter space coverage.Table 3. Statistical Summary for the Difference in Parameter Space Coverage Between the First Disclosed Approach and Random Sampling.Example 3
[0071] Turning now to the second approach disclosed in FIG. 1C and in FIGS.4A-4B, in this case, as well, a simulation was employed. For 100 iterations, 1000fields from the Idaho dataset were sampled, and 9 fields were sampled out of this subset either randomly or using the regions approach described in connection with FIG. 1C and FIGS. 4A-4B. Then, for each case, the total number of adjacent fields enclosed by the representative area (defined as plus and minus 5% in the x and y axes) was summed, and its relative fraction from the total 1000 fields was calculated. The values presented in the analysis are the increase in percentage in the number of fields covered by the chosen experiment locations.
[0072] Table 4 presents a summary statistic (minimal value, 25thpercentile, median, mean, 75thpercentile, and maximal value) of the difference based on the 100 simulations. On average, the 9 experiment locations chosen by the approach of FIG. 1C represents 49% of all fields in Idaho, while random sampling represents only 26%. In terms of difference, for a required 9 experimental locations, the selected approach leads to an increase of 86% percent in the representation of Idaho fields. This difference was found statistically significant (a=0.05, p<0.05).Table 4: Statistical Summary for the Difference in Idaho Fields RepresentationBetween the Second Disclosed Approach and Random SamplingExample 4
[0073] Turning now to the quantitative analysis of the third approach, to quantitatively assess the advantage of this approach over a random choice of field, a simulation approach was employed. For 100 iterations, 1000 fields from the Idaho dataset were sampled. In each iteration, several numbers of requested fields were tested, ranging from 5 to 20. For the random approach, the random fields wereselected to match the required number of fields. For the disclosed approach, we the K means algorithm was used to generate the specific number of clusters (according to the requested number of fields). For each cluster centroid, in addition, the number of fields the centroid represents were calculated (as before, assuming plus or minus 5% in both the Y and the X axes).
[0074] The results of this comparison are presented in Table 5 ((%disciosed - %random) / %random) X 100). There are some sporadic cases where the random sampling approach performed better than the disclosed approach, as indicated in the minimal reported values. But for the majority of the cases, the third disclosed approach increased substantially the number of fields covered, and on average, the third disclosed approach increased the number of represented fields by 31% compare to a random sampling approach.
[0075] In addition, the difference in the number of fields covered by the two approaches - the third disclosed approach and the random one - was tested for statistical significance, and was found significant for all the range of requested number of fields (p<0.001, a= 0.05). Results of this data are shown in Table 6.Table 5. The Difference in Percent of Fields Covered by Either the Disclosed or Random Sampling Approach, Calculated by ((%disciosed - %random) / %random) X 100Table 6. Percentage of Fields Covered by Either the Third Disclosed or Random Sampling Approach.
[0076] To further demonstrate the usefulness of the disclosed approaches, FIGS. 6A-6C depict the fields chosen either by the disclosed approaches or the random one, for one subset of 1000 fields. The presented centroids were calculated based on a 2D K means clustering approach (FIG. 6A), or using Jenks natural classification when based solely on the root zone storage layer (FIG. 6B) or the percent of days between 12-20 degrees (FIG. 6C). In FIG. 6A, the centroids mapped by the third disclosed approach are presented in circles 301-309, while circles 311- 319 represent 9 fields selected randomly. In FIG. 6B, circles 321-329 are calculated based on the third approach, while circles 331-338 are random. In FIG. 6C, circles 341-349 are calculated based on the third approach, and circles 351-359 are random. In all 3 cases, the clustering approach identifies much better the distribution of the data. There are cases where a random approach completely misses out on regions of the parameter space - i.e. right side of FIG. 6A (circle 306) or of FIG. 6B (circle 326).
[0077] FIGS. 7A-7C illustrate conversion of the selected fields from the parameter space to geographical locations, for each of the three sampling approaches. These Figures illustrate that the different sampling approaches lead todifferent real-life fields. FIGS. 7A-7C each show nine fields selected from one of the 1000 subsets, for either the first approach for product development (FIG. 7A), the second approach for marketing (FIG. 7B), or the third, balanced approach (FIG. 7C). It is interesting to notice that fields from different geographical regions of Idaho were selected. While this might be the pure science-based solution, pragmatically users might not want to use fields so far apart. Therefore, the framework is built in a way that the use can specify a region (i.e. one or several counties) and the framework will identify the best fields in that specific region.
[0078] Following the determining of the best locations for the agronomic field trials, the field trials themselves may be conducted at the designated fields.
[0079] In a "real world" scenarios, additional considerations must be taken into account when operating according to the invention, such as experiment cost or travel distance. The following example will illustrate a method to incorporate these considerations in the digital twin.Example 5
[0080] As before, the starting point is a requirement to find the optimal locations to perform an experiment. The research is for product development, and the budget allows 9 locations. For demonstration, a population of 10k fields used to grow potatoes in Idaho, USA is used (FIG. 8). The digital twin is tasked with the optimization of two affecting factors: i) Available water storage (mm), and ii) % of days within the growing season between 12-20 degrees Celsius. Since in this example the experiments are done for product development, the digital twin delineates this parameter space into 9 regions (FIG. 9).
[0081] The digital twin is now requested to accommodate an operational constraint: optimize also for the cost of the experiment. To create variability in experimental cost, the 10k fields were randomly coupled to a value between $2600- $4000, representing realistic experimental cost (payments to the land owner).
[0082] The digital twin now has to choose 9 locations which optimize 3 parameters: temperature, available water storage, and cost. It is known that each location needs to be in a different region (FIG. 9). Accounting for the all combinations of one field per region is not feasible, as there are 4xlOE23 combinations in this example. To reduce the number of combinations evaluated, the Pareto concept is used to identify the fields which are both closest to the region centroid, and have the lowest cost.
[0083] For example, region 8 in FIG. 10 is taken. Within this region, each field can be mapped and projected into a 2D space of its cost and its distance to region centroid. On this 2D space, a Pareto optimization is conducted to find the minimal number of fields which dominate all other fields. For each one of the marked fields, no other field is cheaper or closer to the centroid. For the fields in this region, 8 fields dominate all others. FIG. 10 shows the region (indicated by numeral 8) where the Pareto optimization is demonstrated. FIG. 11 shows a 2D projection of the data in the region 8 of FIG. 10, presenting for each field its cost and its distance from the region centroid. The fields indicated by the strong dots are unique fields dominating all other fields in the region.
[0084] The process of Pareto optimization was repeated for each one of the 9 regions, resulting in 41 locations (FIG. 12). These locations are the best locationswithin each region. To identify which group of locations - one for each region - is the best in terms of budget and distance from each region centroid, all possible combinations were explored using the 41 locations, with the condition that a single location is chosen within each region (total of 9 locations). This results in 430K combinations. Plotting these combinations on a 2D space of cost and distance from centroid (FIG. 13) allows finding the group of 9 locations which has either the lowest cost, or closest to the centroid, or any other weighted target prescribed by the user.
[0085] FIG. 14 presents possible outcomes of field selections by the digital twin. The fields can be optimized for cost, distance from centroid, or both. When mapped into a geographical projection - different locations in the parameter space lead to different locations (FIG. 15).
[0086] In a similar manner, other operational constraint can be accommodated by the twin. For example - the user can restrict the initial field population to be within a certain travel distance from a point of interest.
Claims
What is claimed is:
1. A method for determining optimal locations of agricultural fields for conducting agronomic experiments across large geographical regions, comprising: specifying a plurality of effecting parameters relevant to an agronomic experiment; joining values of the plurality of effecting parameters, for each field in a geographical region, into layers; mapping values of the layers for each field to an N-dimension parameter space, where N is the number of layers; selecting a clustering algorithm for application onto the parameters based on desired goals for coverage of the parameter space, from among a first clustering algorithm configured to maximize diversity of the selected regions, a second clustering algorithm configured to maximize representation of the parameter space, and a third clustering algorithm configured to balance diversity and representation of the parameter space; performing the selected clustering algorithm to thereby divide the parameter space into clusters and assign each field to a specific cluster; determining a centroid of each of the clusters; calculating a Euclidean distance between coordinates of each field in the parameter space and the coordinates of each centroid; ranking the fields in accordance with the Euclidean distances, and identifying the field that is closest to its centroid; andsuggesting the closest field to each centroid as a potential location for conducting the agronomic experiments.
2. The method of claim 1, wherein the first clustering algorithm comprises dividing the parameter space into equal-sized regions.
3. The method of claim 1, wherein the second clustering algorithm comprises dividing the parameter space into regions sized according to a predefined tolerance, such that, for each region, the centroid of the region is representative of the entire region when considering the predefined tolerance.
4. The method of claim 3, wherein the predefined tolerance is + / - 5% for each parameter in the parameter space.
5. The method of claim 1, wherein the third clustering algorithm comprises determining an ideal number of clusters according to the distribution of the fields in the parameter space.
6. The method of claim 5, wherein the plurality of effecting parameters comprises a single parameter, and the clustering algorithm is a Jenks Natural Break Algorithm.
7. The method of claim 6, wherein the step of analyzing outputs of the clustering algorithm to determine the ideal number of clusters comprises evaluating the Goodness of Variance Fit (GVF) factor.
8. The method of claim 7, further comprising setting the ideal number of clusters based on a predetermined threshold for improvement in GVF factor resulting from increasing the number of clusters.
9. The method of claim 5, wherein the plurality of effecting parameters comprises two or more parameters, and the clustering algorithm is a K-means clustering algorithm.
10. The method of claim 9, wherein the step of analyzing outputs of the clustering algorithm to determine the ideal number of clusters comprises applying an ensemble approach based on different validation algorithms for determining the ideal number of clusters.
11. The method of claim 10, wherein the validation algorithms include Silhouette, VRC, and GAP.
12. The method of claim 1, further comprising, in response to user feedback specifying a target number of fields to sample, adjusting the number of clusters.
13. The method of claim 1, wherein the step of specifying a plurality of parameters comprises specifying a range of acceptable values for each parameter.
14. The method of claim 1, further comprising identifying the locations of the suggested fields according to latitude and longitude.
15. The method of claim 14, further comprising identifying the locations of the fields at a resolution of 100 meters x 100 meters.
16. The method of claim 14, further comprising displaying the locations of the fields on a pictorial representation of the geographical region.
17. A system for determining optimal locations of agricultural fields for conducting agronomic experiments across large geographical regions, comprising a non-transitory computer-readable medium storing instructions, that, when implemented by a processor, causes the performance of the following steps:specifying a plurality of effecting parameters relevant to an agronomic experiment; joining values of the plurality of effecting parameters, for each field in a geographical region, into layers; mapping values of the layers for each field to an N-dimension parameter space, where N is the number of layers; selecting a clustering algorithm for application onto the parameters based on desired goals for coverage of the parameter space, from among a first clustering algorithm configured to maximize diversity of the selected regions, a second clustering algorithm configured to maximize representation of the parameter space, and a third clustering algorithm configured to balance diversity and representation of the parameter space; performing the selected clustering algorithm to thereby divide the parameter space into clusters and assign each field to a specific cluster; determining a centroid of each of the clusters; calculating a Euclidean distance between coordinates of each field in the parameter space and the coordinates of each centroid; ranking the fields in accordance with the Euclidean distances, and identifying the field that is closest to its centroid; and suggesting the closest field to each centroid as a potential location for conducting the agronomic experiments.
Citation Information
Patent Citations
Identifying management zones in agricultural fields and generating planting plans for the zones
US20190014727A1
Clustering for k-anonymity in location trajectory data
US20200019815A1
Use of soil and other environmental data to recommend customized agronomic programs
US20210010993A1
