Digital twins for precise agricultural management

Abstract

Methods of creating and using digital twins of vegetation / crops in relation to their growing environments. Some embodiments include creating a digital twin of the growing environment using sensor data from a subsurface sensor suite, creating a digital twin of the vegetation crop using data from an overhead sensor suite, and using at least one mathematical model to relate the digital twin of the crop to the digital twin of the growing environment. In some embodiments, the mathematical model(s) implement a root cause analysis that uses a limiting factor approach. In some embodiments, the subsurface sensor suite is on a multi-sensor soil probe. In some embodiments, the multi-sensor soil probe comprises a video microscope, a tip stress sensor, a sleeve stress sensor, and a volumetric- water-content sensor. Other related systems, devices, methods, and software are also disclosed.

Description

DIGITAL TWINS FOR PRECISE AGRICULTURAL MANAGEMENTCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of pnority of U.S. Provisional Patent Application Serial No. 63 / 542,757, filed October 5, 2023, and titled '‘Digital Twin for Precise Agricultural Management”, U.S. Provisional Patent Application Serial No. 63 / 542,937, filed October 6, 2023, and titled “Digital Twin for Precise Agricultural Management”, and U.S. Provisional Patent Application Serial No. 63 / 615.832, filed December 29, 2023, and titled “Multi-Sensor Probes and Methods of Making and Using the Same”, each of which is incorporated herein by reference in its entirety for all purposes.

[0002] U.S. Patent No. 11,800.246 titled “Systems and Methods for Multispectral Landscape Mapping”; U.S. Patent Application Serial No. 17 / 404,470, filed February 24, 2023, and titled “Site Characterization for Agriculture”; U.S. Patent No. 12,092,625 titled “Precision Site Characterization Using Digital Twin”; U.S. Patent No. 11,686,676 titled “In-situ Near Infrared Sensor Unit and Method of Making the Same”; International Application No.PCT / US2022 / 018197, filed February 28, 2022, entitled "In-situ Near Infrared Sensor Unit and Method of Making the Same”; U.S. Patent Application Serial No. 17 / 411,476, filed August 25, 2021, and titled “Soil Penetrating Probe and System for Measuring Electrical Properties to Determine Soil Water Content”; and U.S. Patent Application No. 18 / 086,219, filed December 21, 2022. entitled “Fiber-Optic Bundle for In-Situ Spectral Sensor, Spectral Sensor and Method of Making and Using the Same” are also incorporated herein by reference in their entireties for all of their teachings relevant to the present disclosure.FIELD OF THE INVENTION

[0003] The present disclosure generally relates to the field of agricultural management. In particular, the present disclosure is directed to digital twins for precise agricultural management.BACKGROUND

[0004] Creating an effective digital twin capable of resolving individual plant performance requires accurate 3D soil mapping and individual plant models which necessitates accurately measuring and representing the vertical structure of the soil and the (horizontal) spatial variability of the vertical structure, which is to say, how the soil profile varies with location and thus by plant. The vertical soil profile captures crucial information such as moisture gradients, nutrient distribution, and root-zone conditions at different depths, which directly affect plant grow th and performance. The most effective known means to create accurate digital twins of soilin profile is through the application of soil cores, soil pits, or proximal soil sensing using a soil probe that penetrates the soil in vertical profile, also known as a penetrometer probe. However, soil pits and soil samples have inherent drawbacks to producing accurate 3D digital twins of soil. These drawbacks include dependence on independent analysis of multiple soil parameters ex situ in a laboratory that therefore decouples the soil parameters from natural in situ relationships, low vertical spatial resolution due to the volumes of soil that most laboratory analysis methods require, and low accuracy and repeatability of laboratory methods due to various factors including human subjectivity such as where to partition the length of a core sample into individual samples that will then be homogenized, variations among laboratories, and limited precision of the laboratory methods themselves. Most laboratory methods require some sort of calibration, too.

[0005] Even with theoretically perfect laboratory processes and calibrations, samples retrieved and sent to the lab will always present an inaccurate representation of the in situ plant production conditions in the field due to retrieval-related disturbance, compositing, homogenization, and uncertain depth registration. Some amount of structural detail is always lost through sampling. Moreover, a requirement of 500 grams to 1,000 grams per sample is commonly needed to support a comprehensive suite of physical, chemical, and biological parameters, which corresponds to a whopping 20 centimeters to 40 centimeters of core assuming a midrange bulk density. It is therefore often impossible to obtain a full suite of soil parameters from any layer that is thinner than around 6 inches.SUMMARY OF THE DISCLOSURE

[0006] In one implementation, the present disclosure is directed to a multi-sensor soil probe, which includes a plurality of sensors, wherein the plurality of sensors comprises: a diffuse reflectance spectroscopy sensor comprising a spectrometer that is in optical communication with soil by way of a fiber optic bundle that conveys reflected light to the spectrometer; and a sensor that collects data that is indicative of particle size distribution of the soil.

[0007] In another implementation, the present disclosure is directed to a method of creating a digital twin of a crop in relation to a grow ing environment of the crop. The method includes obtaining sensor data from a subsurface sensor suite pertaining to the growing environment: obtaining sensor data from an overhead sensor suite pertaining to the crop; using the data from the subsurface sensor suite to create a digital tw in of the growing environment; using the data from the overhead sensor suite to create a digital twin of thecrop; and using at least one mathematical model to relate the digital twin of the crop to the digital twin of the growing environment.

[0008] In yet another implementation, the present disclosure is directed to a digital twin for agriculture, which includes a plurality of sensor profiles collected from a plurality of modalities of sensors, wherein the plurality of sensor profiles comprises variation in soil and variation in a crop located in the soil; wherein variation in the soil comprises spatial variation in the soil and a plurality of temporal variations in a plurality of conditions of the soil; wherein variation in the crop comprises spatial variation in the crop and a plurality of temporal variations in a plurality of management practices for the crop; and wherein variation in the crop is correlated to variation in the soil.

[0009] In still another implementation, the present disclosure is directed to a multi-sensor soil probe, which includes a first sapphire window; a conical tip formed by a pair of electrodes separated by an insulating element; an outer cylindrical surface; a plurality of sensors, wherein the plurality' of sensors comprises: a video microscope that collects imagery' through the first sapphire window; a tip stress sensor that measures force on the conical tip; and a sleeve stress sensor that measures friction exerted by soil surrounding the outer cylindrical sleeve surface; herein the conical tip is configured to measure volumetric water content.

[0010] In another implementation, the present disclosure is directed to an agricultural management system, which includes a multi-sensor soil probe configured to obtain data pertaining to a growing environment; an aerial crop sensor configured to obtain data pertaining to a crop in the growing environment; and a machine learning model configured to relate the crop to the growing environment; wherein the agricultural management system is configured to quantify at least one of particle size distribution in situ and soil organic carbon in situ.

[0011] In yet another implementation, the present disclosure is directed to a method of valuing agricultural land. The method includes creating a digital twin of a crop in relation to a growing environment of the crop, comprising: obtaining sensor data from a subsurface sensor suite pertaining to the growing environment; obtaining sensor data from an overhead sensor suite pertaining to the crop; using the data from the subsurface sensor suite to create a digital twin of the growing environment; using the data from the overhead sensor suite to create a digital twin of the crop; and using at least one mathematical model to relate thedigital twin of the crop to the digital twin of the growing environment; and valuing a parcel of agricultural land that comprises the crop based on the digital twin of the crop in relation to the growing environment of the crop.

[0012] In another implementation, the present disclosure is directed to a method of prospecting land for the expansion of a crop into a new' region. The method includes creating a digital twin of the crop in relation to a growing environment of the crop, comprising: obtaining sensor data from a subsurface sensor suite pertaining to the growing environment; obtaining sensor data from an overhead sensor suite pertaining to the crop; using the data from the subsurface sensor suite to create a digital twin of the growing environment; using the data from the overhead sensor suite to create a digital twin of the crop; and using at least one mathematical model to relate the digital twin of the crop to the digital twin of the growing environment; and prospecting land for the expansion of the crop into a new region based on the digital twin of the crop in relation to the growing environment of the crop.

[0013] In another implementation, the present disclosure is directed to a method of determining what to plant on a management unit. The method includes creating a digital twin of a crop in relation to a growing environment of the crop, comprising: obtaining sensor data from a subsurface sensor suite pertaining to the growing environment; obtaining sensor data from an overhead sensor suite pertaining to the crop; using the data from the subsurface sensor suite to create a digital twin of the growing environment; using the data from the overhead sensor suite to create a digital twin of the crop; and using at least one mathematical model to relate the digital twin of the crop to the digital twin of the growing environment; and determining what to plant on a management unit based on the digital twin of the crop in relation to the growing environment of the crop, wherein the management unit comprises the growing environment of the crop.

[0014] In another implementation, the present disclosure is directed to a soil layer detection and soil classification system, which includes an acoustic sensing module disposed in a soil probe; a data communication module; and a logic device in communication with the acoustic sensing module and the communication module, wherein the logic device is configured to: capture audio recordings of the soil probe advancing through soil using the acoustic sensing module; process the audio recordings; and detect transitions between soil layers through which the soil probe advances based on the processed audio recordings.BREIF DESCRIPTION OF THE DRAWINGS

[0015] For the purpose of illustration, the accompanying drawings show aspects of one or more embodiments of the disclosure. However, it should be understood that the scope of this disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, wherein:

[0016] FIG. 1 A is an elevational / partial cutaway perspective view of a multi-sensor soil probe according to an embodiment of the present disclosure;

[0017] FIG. IB is an elevational perspective view showing the tip stress sensor of the multisensor soil probe of FIG. 1A;

[0018] FIG. 1C is an elevational perspective view showing the sleeve stress sensor of the multi-sensor soil probe of FIG. 1A;

[0019] FIG. ID is an elevational perspective view showing the electrical-properties-sensing device of the multi-sensor soil probe of FIG. 1A;

[0020] FIG. IE is an elevational perspective view showing the acoustic sensor of the multisensor soil probe of FIG. 1A;

[0021] FIG. IF is an elevational perspective view showing the video microscope of the multisensor soil probe of FIG. 1A;

[0022] FIG. 1G is an elevational perspective view showing the visible and near-infrared (VNIR) diffuse reflectance spectroscopy (DRS) sensor of the multi-sensor soil probe of FIG. 1 A;

[0023] FIG. 2 shows eight example microscope images acquired in situ by the video microscope of FIG. I F;

[0024] FIG. 3 shows two example multi-target random forest plots according to embodiments of the present disclosure;

[0025] FIG. 4 is a diagram illustrating an example of a decision tree according to an embodiment of the present disclosure;

[0026] FIG. 5 shows six example histograms before and after dithering raw measured data according to embodiments of the present disclosure;

[0027] FIG. 6A is a flow diagram for an example method for simulating plant performance for a digital twin of a growing environment according to embodiments of the present disclosure;

[0028] FIG. 6B is a flow diagram for another example method for simulating plant performance for a digital twin of a growing environment according to embodiments of the present disclosure;

[0029] FIG. 6C is a graph of predicted initial bud count versus measured initial bud count illustrating an example simulation of plant performance based on the depth-dependent raw sensor data according to an embodiment of the present disclosure;

[0030] FIG. 7 is a flow diagram of an example method of assessing and predicting plant health and performance from sensor data, used to create a digital twin of a crop according to an embodiment of the present disclosure;

[0031] FIG. 8 is a flow diagram of an example application of a Root Cause Analytics™ model according to an embodiment of the present disclosure.

[0032] FIG. 9 is a scatterplot showing the relationship between productivity and a plurality of genetic, environmental, management, and / or socioeconomic factors according to an embodiment of the present disclosure;

[0033] FIG. 10 is a scatterplot showing Management Limited Productivity vs. actual current productivity' according to an embodiment of the present disclosure; and

[0034] FIG. 11 is a stacked graph showing productivity and opportunity estimates in productivity per unit area that add up to a Site Limited Productivity according to an embodiment of the present disclosure.DETAILED DESCRIPTION

[0035] The entire contents of the appended claims are incorporated into this Detailed Description section as if originally presented herein.

[0036] As used herein, the singular forms “a,” “an,” and “the” include the plural referents unless the context clearly indicates otherwise. The terms “include,” “such as,” and the like are intended to convey inclusion without limitation, unless otherwise specifically indicated.

[0037] As used herein, the term “comprising” also specifically includes embodiments “consisting of’ and “consisting essentially of' the recited elements, unless specifically indicated otherwise.

[0038] As used herein, the term “about” when used with a corresponding numeric value refers to ±20% of the numeric value, typically ±10% of the numeric value, often ±5% of the numeric value, and most often ±2% of the numeric value. In some embodiments, the term “about” canmean the numeric value itself. In certain embodiments, where applicable, the term '‘about’7indicates the designated value(s) ± one standard deviation of that / those value(s).

[0039] As used herein, the term “management unit” includes the smallest possible physical area that a human or machine management method or system can act on. Management units can be defined according to one or more characteristics for the portion of the region represented by the management unit. For example, management units can be characterized by biological, chemical, geological, topographical, weather and climate, socio-economic, and other scientific, technical, business, and financial characteristics. The unit can be as small as a single plant and scale in size based on biological, chemical, geological, topographical, weather and climate, socio-economic, and other scientific, technical, business, and financial characteristics to includes fields and blocks, farms, ranches, landscapes, geographic regions, states, countries, and even continents.

[0040] Management units can be temporally dynamic. In other words, the portion of the region represented by the management unit can vary according to temporal variation in the characteristics defining the unit. The productivity or other performance measures of physical sites can be optimized over time through the dynamic implementation of management methods and systems on one or more management units representing a region.

[0041] For example, if a management unit represents a fixed economic value, then depending on economic conditions, the size of the region represented by the management unit changes as a function of the amount of region matching that fixed value. One thousand dollars in lemons may correspond to a management unit of 10 lemon trees during the peak season of lemon production. Later, the same management unit can represent 20 lemon trees, for example during the offseason as the value of lemons decreases. Other examples can include a wide range of financial metrics.

[0042] As another example, management units representing a labor cost to maintain the region represented by the units can also result in the units changing over time, for example, because labor costs can vary over time with socio-economic conditions. As another example, improvements in technology can affect management units defined by the yield of the region represented by the units. As technology for planting, cultivation, and harvesting improve, the portion of the region represented by a unit can increase because the same amount of labor and other resources can be used to greater effect.

[0043] Previously, defining management units within a physical site was imprecise for site characterization or subsequent agronomic recommendations for managing the site, because sensor data collected and analyzed for characterizing a site was collected at a level not granular enough to guide informed decisions about how best to manage a management unit at the level of, for example, a row of grapes within a vineyard or an individual grapevine. The incorporation of market conditions or other socio-economic factors, conditions and data, for example, pricing, labor availability, or export demand, has also had limited utility in defining management units due to the lack of methods and systems to accurately and dynamically combine key characteristics in a timely manner to inform or make management decisions. By techniques described in this disclosure, sensor data can be collected and analyzed to provide inferred, derived, or predicted characteristics for individual plants or other management units.

[0044] As used herein, the term “precision agriculture”, also known as precision farming, refers to an agricultural management approach that uses spatially aware technologies to observe, measure, analyze, simulate, predict, and respond to variability’ both within and across management units. Techniques and tools of precision agriculture such as GPS, sensors, drones, manned aircraft, satellite imagery', and data analytics, and the use of digital twins are used to optimize within and across management units using management practices based on genetics, environment, management, and socio-economic factors. Key components of precision agriculture include data collection and monitoring using subsurface, at surface, and overhead sensors, climate and weather information, machinery' equipped with GPS or other map and location-based technologies that can perform tasks with high spatial precision, geographic information systems that integrate horizontal and vertical location spatial data to analyze and determine yvith a high-degree of accuracy within and betw een management unit variability that can be used to create maps for varying management yvithin and across management units, such as variable rate soil amendments, variable rate irrigation, variable rate seeding, variable depth tillage, variable pruning, and variable harvest, in addition to other types of management practices that can be varied based on conditions of the management unit, utilizing agronomic, machine learning, or Al models or other mathematical algorithms to simulate and predict plant groyvth and performance, and support or automate decisions that optimize socio-economic outcomes including: yield, quality, production costs, environmental impacts, compliance with regulations and policies, and simplification of voluntary market-based assessment, verification, and certification programs. By using yield, quality, and other socio-economic outcomes, along with mapping and monitoring of environmental and management factors, while accounting for effects of genetics, precision agriculture can be used to identify, classify, and compare within andbetween management units performance for improved outcomes, reduced inputs, site selection, crop selection, land valuation, and other management decisions. Precision agriculture can also be used for forensic investigation of differences in plant performance and behavior within and across management units, and identification, diagnosis and treatment of pests and diseases.

[0045] As used herein, the acronym "GEMS" stands for Genetics x Environment x Management x Socio-economic factors (GEMS). GEMS refers to an integrated and holistic framework that can be used to analyze, simulate, predict, and optimize how genetic factors (G), environmental conditions (E), management practices (M). and socio-economic factors (S) interact to influence agricultural outcomes. Genetics (G): refers to the genetic makeup of crops or livestock, including specific genes and traits that determine characteristics like yield potential, disease resistance, drought tolerance, and nutritional content. Environment (E): encompasses all external abiotic and biotic factors affecting agricultural production, such as climate, weather patterns, soil types, water availability, and other abiotic and biotic factors such as pests and diseases. Management (M): involves all the management planning, decision making, and implementation including techniques and practices employed by farmers, managers, consultants, and others. Example techniques and practices include financial planning, decision making, and implementation using financial metrics, complying with regulations, engaging in voluntary assessments, certification programs, and ecosystem services markets, and other socio-economic factors, and farm management practices including planting methods, irrigation, fertilization, pest control, harvesting, resource allocation and all other management techniques and practices utilized in agriculture production. Socio-economic Factors (S): include economic, social, cultural, and environmental impact factors that influence or result from agricultural management activities, such as market access, pricing, policies, labor, cultural practices, risk management, health and safety, regulatory7context and compliance, pollution and contamination, financial resources and deployment of capital, consumer preferences, and other supply chain factors and conditions. A GEMS framework provides a comprehensive view of agricultural systems by considering biological, physical, and socio-economic dimensions used to characterize, analyze, simulate, predict, and optimize agricultural production.

[0046] The significance of GEMS includes gaining a holistic understanding of agricultural systems by considering biological, physical, and socio-economic dimensions, enhanced decisionmaking to support all participants in the agriculture supply chain including input providers, farmers, farm managers, researchers, policymakers, and consumers in making informed decisions that account for the complexity of agricultural systems, improved crop and livestock performance to facilitate the development of varieties and breeds that are better adapted tospecific environments and management regimes, considering socio-economic realities, improving Organic, Sustainable, Regenerative and other categories of voluntary assessment, certification, and markets focused on environmentally sustainable, economically viable, and socially acceptable management practices and outcomes, and speeding adaptation and resilience in the face of climate impacts, resource limitations, market fluctuations, and other factors that can create uncertainty and risks.

[0047] As a component of socio-economic factors and conditions, “financial metrics” are more specifically defined as metrics that assess how management processes and practices impact profitability and efficiency of investments or business operations and serve various purposes, such as investment appraisal, performance evaluation, risk assessment, and strategic planning. Examples include Return on Investment (ROI) that measures the amount of return on an investment relative to its cost, Internal Rate of Return (IRR) that measures the discount rate at which the net present value (NPV) of all cash flows (both positive and negative) from an investment equals zero. Net Present Value (NPV) that measures the difference between the present value of cash inflows and outflows over a period of time, Return on Equity (ROE) that measures the profitability relative to shareholders' equity; calculated as Net Income divided by Shareholder's Equity, Return on Assets (ROA) evaluates the management efficiency in using assets to generate earnings. Return on Capital Employed (ROCE), measures profitability and efficiency in capital utilization; calculated as Earnings Before Interest and Tax (EBIT) divided by Capital Employed, Total Return provides a comprehensive measure of investment performance, Profit Margin measures the percentage of revenue that turns into profit; calculated as Net Income divided by Revenue and includes Gross Profit Margin. Operating Profit Margin, and Net Profit Margin, Cash Flow Return on Investment (CFROI) assesses the cash-generating ability of a company's assets, Payback Period measures the time it takes for an investment to generate an amount of cash equal to the initial investment cost, Profitability Index (PI) measures the ratio of the present value of future cash flows to the initial investment, Compound Annual Growth Rate (CAGR) measures the mean annual growth rate of an investment over a specified time period longer than one year, and Economic Value Added (EVA) measures a company’s financial performance based on residual wealth and includes Net Operating Profit After Taxes (NOPAT) minus the capital charge (cost of capital times the economic capital employed).

[0048] As a component of socio-economic factors and conditions, regulatory policies and compliance programs are more specifically defined as described as areas in food safety, protecting the environment, meeting government subsidy and compliance programs, and other areas. Examples of environmental regulations include restrictions on the types, applicationmethods, and quantities of pesticides used, regulations on water quality and quantity that promote water conservation through efficient irrigation technologies, protection of water sources from surface and groundwater contamination and onsite water storage, regulations to conserve soil resource including preventing soil erosion, degradation, and contamination, nutrient management requirements to manage manure, compost, and fertilizer application to prevent nutrient loss through water runoff, leaching, and off gassing, air quality regulations that limit emissions from agricultural activities including dust, greenhouse gases, and other air pollutants, meeting eligibility7criteria for government subsidies, grants, and support programs, agricultural finance and credit regulations governing agricultural loans and credit availability, programs promoting the use of solar, wind, and bioenergy in agricultural operations, labeling and marketing standards that regulate nutritional information and origin of production, greenhouse gas regulations for tracking and reporting greenhouse gas emissions from agricultural activities. Specific regulations vary depending on the country7, region, and local governance structures.

[0049] Ssocio-economic factors and conditions include voluntary market-based assessment, verification, and certification programs in the areas of Organic, sustainable agriculture, regenerative agriculture, ecosystem services markets, climate-smart agriculture, and related programs such as the USDA National Organic Program and other accredited organic certification programs. Regenerative Organic Certification, Regenified, and other regenerative agriculture programs, Non-GMO Project, Bee Friendly Farming, Demeter Biody namic Certification, GlobalG.A.P, Rainforest Alliance Certification, Salmon-Safe, Sustainably Grown. Sustainability in Practice (SIP) Certified, LodiRules, California Certified Sustainable Winegrowing, Fish Friendly Farming, Roundtable on Sustainable Palm Oil (RSPO) Certification. Bonsucro Certification, Better Cotton Initiative (BCI), Sustainable Agriculture Network (SAN) Farm Sustainability Assessment, Carbon Trust Standard, Verified Carbon Standard (VCS), Climate Action Reserve (CAR), American Carbon Registry7(ACR), Cool Farm Tool and Cool Farm Alliance, USDA Climate Smart Agriculture and Forestry (CSAF) Initiative, Global Carbon Council (GCC), Savory Institute's Land to Market Program, Ecosystem Services Market Consortium (ESMC), and others.

[0050] This disclosure relates generally to methods, systems, and machine-executable instructions for characterizing, analyzing, simulating, predicting, and, optionally, managing physical agricultural, forest, and similar sites. More particularly, this disclosure relates to methods, systems, and computer-program products for creating and iterating a digital twin of crops and other plants, a digital twin of the grow ing environment, and combining crop, plant, and growing environmental digital twins to simulate and predict the outcome of management, andrecommend specific management practices to meet crop production objectives included in the GEMS framework such as increasing yield, improving quality, achieving regulatory compliance, increasing resource sustainability, appraising and increasing land values, and other objectives in agriculture and / or forestry.

[0051] Digital twins represent a paradigm shift in various industries, including agriculture. A digital twin is essentially a digital representation of a physical object or system that can be used for various purposes like simulation, monitoring, predicting, and optimization. Digital twins can range in sophistication from merely modeling structures, such as a 3D computer model of human structural anatomy created using magnetic resonance imaging, to twins that embody fully functional simulation of the physical system, such as aerospace digital twins that represent and simulate the dynamical behavior of a jet engine under different conditions, including the current operational state as a function of the engine's entire history from manufacture to past flights, maintenance actions, and more. A sophisticated digital twin can serve as a bridge between the physical and digital worlds by integrating real-world data, usually from sensors, with a virtual model.

[0052] Digital twins have gained significant attention and adoption in industries such as manufacturing, healthcare, transportation, and urban planning, but are a relatively new concept in agriculture, where digital twins have not yet been sophisticated enough to enable, for example, predictive analytics with respect to the outcomes of different potential management practices on every plant in a field.

[0053] A digital twin in agriculture is a virtual representation of a physical agricultural system that can be updated continuously or intermittently with data from sensors and other sources, including soil sensors and remote sensors, such as multispectral and thermal aerial and drone imagery and light detection and ranging (LiDAR) or photogrammetric measurements of the crop geometry above ground, such as canopy height, volume, spectral characteristics, and other attributes which allow for precise monitoring and simulations, enabling optimized decisions on irrigation, fertilization, and all other categories of crop management by predicting outcomes based on collected data and system dynamics, effectively integrating environmental, biological, agronomic, and social-economic factors. In other words, in the context of agriculture, digital twins according to the present disclosure can model farm operations, crop growth, water management systems, yield, quality, price categories, regulator ' compliance, selection of crops to plant, and more, to provide growers with a virtual platform to monitor, simulate, predict, and optimize their agricultural management practices.

[0054] Digital twins can play a crucial role in precision agriculture, enabling growers to manage fields at more granular spatial and temporal scales. By linking each plant’s behavior to the specific characteristics of the soil in which it is growing — both horizontally and vertically — the digital twin enables precise modeling of how variations in soil properties (e g., compaction, water retention, nutrient stratification) influence plant performance. This detailed mapping allows for fine-tuned management decisions that reflect the dynamic interactions between plants and the spatial heterogeneity of the soil they rely on, optimizing inputs like water through irrigation, fertilizers, and other management practices to meet management objectives including targeted yield, quality, crop safety, regulatory requirements, product pricing categories, and more. With detailed models of fields, crops, climate, and weather, farmers can simulate various scenarios, such as different irrigation regimes or pest outbreaks, and make more informed and spatially and temporally granular decisions.

[0055] In contrast to the drawbacks of soil sampling and sensing outlined above, soil probes of the present disclosure can take in situ measurements of soil properties and parameters at fine vertical resolution, such as centimeter scale. In some embodiments, this is achievable by having multiple direct-contact and proximal sensors that are individually sensitive in different ways to different soil properties. These sensors are capable of resolving the 3D distribution of soil properties and parameters with sufficient resolution and specificity to enable creation of a digital twin that accurately resolves individual plant performance in the context of each plant’s growing environment, thereby solving a problem in the prior art of not having enough resolution or specificity to enable creation of a digital twin. Limitations of soil probes in the prior art prevent the creation of useful digital twins because they do not generate data with enough resolution or specificity.

[0056] Inherent in a digital twin is one or more computational models or systems of models that relate a plant’s behavior to its root environment (e.g.. soil, canopy environment, atmosphere) and management practices (such as, but not limited to, irrigation, fertilization, and other forms of amendment, tillage, pruning practices, planting of cover crops, application of crop protection products, other management practices, and environmental control). Various types of computational models may undergird the functioning of a digital twin, and these models can be of different types and combinations of types. Such models can include empirical models that statistically relate plant performance characteristics to soil and management practices, processbased models such as biodynamic models that express such relationships mechanistically, and machine learning models that are capable of relating plant performance metrics to multiplemodel inputs and observations that interact in various linear, non-linear, continuous, discrete, limiting, and enabling ways and combinations thereof.

[0057] One particular model disclosed herein for relating plant performance, including socioeconomic outcomes, to genetic, environmental, and management factors is a Root Cause Analytics™ model (RCA). In that particular RCA method, a limiting factor approach is applied to the GEMS analysis, wherein a minimization function is used as the form of the function f in Plant Performance and Behavior (PPB) = f(G,E,M,S), thereby overcoming limitations in current practices of GEMS analysis. The RCA is conducted by first summarizing the soil data from the higher resolution profiles to potential rooting zones of varying depths. Then, the digital productivity target is related to each environmental and management factor for the same genetics, forming a multidimensional matrix. For each factor, the maximum, or statistical threshold of productivity is calculated for the range of each factor. In two dimensions, the relationship between each factor and productivity can be shown in scatter plots, and the productivity potential for each factor at each step in its range can be show n as sets of points. A bootstrapped confidence interval is calculated at each step in the range of each factor. For instance, the 95thpercentile of productivity can be calculated at a 95% confidence interval. The confidence interval can also be displayed on the same plot as upper and lower bounds around the statistical threshold. Each step in each factor is then categorized as sufficient, deficient, or excess, based on the difference betw een low er bounds of the confidence interval of the step with the maximum productivity potential and the upper bounds of the confidence interval for that step. If the difference is negative, then a value for that factor in that range is considered sufficient. If the difference is positive, then the value for that factor in that range is considered either deficient or excess, depending on whether that step is less than or greater than the step with the maximum productivity potential for that factor. For instance, RCA may relate plant performance, such as measured yield or yield expected based on bud count as the dependent variable, against a soil property, such as potassium concentration over a given soil layer or vertical interval of soil as the independent variable, with each point defined by an individual plant and one condition of the soil in which it is growing. Yield and potassium concentration are non-limiting examples of dependent and independent variables to which the RCA can be applied, as many other embodiments are possible. Other plant performance characteristics can include, for example, a multispectral vegetation vigor index, a multispectral pigmentation index, a bud count, a canopy height, a canopy volume, and any other metric of plant performance or socioeconomic outcomes, as w ill be readily understood by those of ordinary skill in the art. Other soil parameters can include, for example, soil water holding capacity, soil moisture content, soiltexture, soil compaction state, soil carbon content, soil nitrate concentration, soil cation exchange capacity, or any other vertically averaged or depth-dependent soil parameter or property which may affect plant performance, as will be readily understood by those of ordinary skill in the art. Once the relationship between each factor and productivity is established, the productivity potential across all factors is calculated using a limiting factor approach. In the particular example of RCA, productivity potential (Management Potential Productivity) for each plant is calculated by finding the minimum productivity potential in the matrix of productivity potentials per factor for that plant.

[0058] Another model of the present disclosure that can undergird the functionality of a digital twin in agriculture is one that uses machine learning to directly link the high-density multisensor soil probe data directly to crop performance and behavior metrics, bypassing slow, costly, coarsely resolved laboratory soil analysis. Partial Least Squares Regression with leave-one-out cross-validation is a nonlimiting example of a machine-learning technique that is effective at implementing such a model for a digital twin in agriculture.

[0059] With digital twins of the present disclosure for irrigation management, growers can model their irrigation system and simulate and predict water flow, infiltration, root water uptake, and deep drainage. Of these components of the mass flow of water, the latter three require knowledge of soil water characteristics or soil hydraulic behavior, which has historically been limited in availability because it is hard to measure and characterize in situ or to replicate in a laboratory. By closely mirroring the actual water needs of crops and the water transmitting and holding behavior of soils, growers can ensure efficient water use and mitigate wastage.

[0060] In some embodiments, digital twins of the present disclosure are used by growers in pest and disease management. By simulating crop growth and monitoring actual field conditions using sensors, farmers can forecast potential pest or disease outbreaks, thus enabling timely interventions, reducing reliance on pesticides, and ensuring healthier crops.

[0061] In some embodiments, digital twins of the present disclosure model underground conditions in the soil, including soil health and soil function. Informed by spatial and spatiotemporal data from sensors that are sensitive to various soil characteristics and properties such as texture, compaction state, moisture conditions, nutrient levels, microbiology and / or others, digital twins of soil structure and function can help growers understand soil health better and recommend changes to irrigation, nutrient management, and even structural interventions such as soil ripping.

[0062] By coupling digital twins of the present disclosure to climate and weather prediction models, growers can prepare for changing climate and weather patterns, adjusting their practices in response to forecasts for rainfall, temperature, or other critical factors.

[0063] In some embodiments, digital twins of the present disclosure can also help farmers simulate, understand, and predict the environmental impact of their practices. This could range from water use efficiency to carbon sequestration, among many others, helping farmers adopt more sustainable and regenerative practices.

[0064] Digital twins of plants can be constructed from sensor data that elucidates aspects of the structure and function of the plant. For example, Bayer-pattem or red, green, blue (RGB) imaging sensors deployed from an aerial vehicle can acquire overlapping images that can be used to photogrammetrically construct 3D models of plant canopy, multi-spectral (MS) cameras can acquire spectral reflectance data that can be used to build spectral indices of vanous aspects of plant health and performance, such as normalized difference vegetation index (NDVI) or normalized red edge index (NDRE), whereas thermal infrared (TIR) imaging sensors can be used to map variations in canopy temperature that are indicative of the comparative water use among plants.

[0065] Likewise, a multi-sensor penetrometer probe can be inserted in the soil to measure mechanical, electrical, and other properties of soil that are indicative of soil texture, compaction state, moisture content, and other soil characteristics and parameters, as well as to elucidate soil layer structure by interpreting variations in vertical profiles of sensor data with depth below ground. Soil sensors can thereby be used to construct digital twins of soils that express both their structures and their functions.

[0066] The marriage of sensor technology with computational modeling as disclosed herein enables the construction of digital twins in agriculture that are both structurally and functionally representative of real conditions. This level of heretofore unattained detailed digital replication can provide growers with powerful tools to understand, predict, and manage their agricultural systems with a marked increase of statistical confidence.

[0067] However, the full utility and potential of digital twins in agriculture using sensor data has not been realized due to several limitations on sensor capability, sensor data, and the combination and management of data from both plant sensors and soil sensors into a common analytical framework that can then incorporate socio-economic factors and conditions, as well as limitations on the analytical and modeling approaches and techniques used to relate both sensordata to physical properties and characteristics of plants and soils, and to relate indicators of plant performance above ground to the belowground environment in which the plant is rooted.

[0068] In some embodiments, the methods, systems, and machine-executable instructions of the present disclosure include obtaining sensor data and predicting characteristics of physical sites, and implementing management methods and systems using one or more deterministic models, probabilistic models, and machine-learning models alone or in combination: to relate plant performance and / or behavior to sensor data pertaining to the crop: to relate plant performance and / or behavior to sensor data pertaining to the soil; to relate sensor data pertaining to the crop to sensor data pertaining to the soil; to relate sensor data pertaining to the crop and / or sensor data pertaining to the soil to data pertaining to the crop collected by means other than the sensors included in this disclosure; and to relate sensor data pertaining to the crop and / or sensor data pertaining to the soil to data pertaining to the soil collected by means other than the sensors included in this disclosure, for example, soil maps, soil map databases, soil pits, soil samples, soil cores, soil laboratory analyses.

[0069] In some embodiments, methods of creating a digital twin of a crop in relation to its growing environment include collecting sensor data. In some embodiments, the sensor data collected comprises two suites of sensor data: a subsurface suite pertaining to soil, also called a Digital Soil Core (DSC), and an airborne or “overhead” suite pertaining to crop spectral, thermal, and structural, for example, geometric, characteristics, also called a Digital Vegetation Signature (DVS). The subsurface suite is focused on soil conditions, while the overhead suite is focused on crop conditions. However, since the crop characteristics are at least in part determined by the condition of the soil in which each plant is growing, both suites of data may provide information about either or both the soil and / or the crop. For instance, spectral information from a dronebased multi-spectral imager can provide information about bare soil as well as about crop canopy. And in fact, bare soil information is used to calibrate and interpret sensor response to canopy in some embodiments. Similarly, subsurface video microscope imagery primarily reveals color and textural information about the soil, but can also produce information about crop root structure, moisture, and redox conditions. Furthermore, spatial and / or temporal variations in vegetation vigor or other vegetation attributes that correlate to contrast in spectral reflectance or the aboveground geometric structure of the plant frequently correlate spatially to subsurface spatial distributions of soil properties, ground surface topography, and / or soil structure.

[0070] Specific limitations of past digital twins in agriculture that the current disclosure addresses include using time series of individual plant spectral characteristics to differentiateamong plants in a field on the basis of their temporal response to variations in growing conditions, including spatial and temporal variation in soils, management practices, disease and pest exposures, and other sources of variability among what plants experience, as does one nonlimiting aspect of the present disclosure.

[0071] Another specific limitation of digital twins in agriculture that the present disclosure addresses is the unnecessary additional uncertainty introduced by relating soil sensor responses first to traditional sample-based measures of soil properties and conditions as an intermediate step to relating plant health and performance to these same traditional measures of soil properties and conditions, rather than relating variations in plant health and performance under uniform management practices to variations in the raw soil sensor response to soil properties and conditions as does one nonlimiting aspect of the present disclosure.

[0072] An additional specific limitation of digital twins in agriculture that the present disclosure addresses is the unnecessary additional uncertainty introduced by relating crop imaging sensor responses and derived sensor data products first to traditional measures of plant health and performance as an intermediate step to relating plant health and performance to soil properties, rather than relating variations in sensor response to plant health and performance under uniform management practices to variations in soil properties and conditions as does one nonlimiting aspect of the present disclosure.

[0073] An additional limitation of digital twins in agriculture that the present disclosure addresses is the unnecessary additional uncertainty introduced by relating soil sensor responses first to traditional sample-based measures of soil properties and conditions as an intermediate step to relating plant health and performance to these same traditional measures of soil properties and conditions, rather than directly relating sensor output and sensor data products that are sensitive to variations in plant health and performance under uniform management practices to variations in raw soil sensor response that are sensitive to variations in soil properties and conditions as does one aspect of the present disclosure.

[0074] In addition, the utility of digital twins in agriculture has been limited by: lack of spatial, spectral, and temporal resolution in the collection of aboveground sensor data that relates to the health and performance of the crop; and lack of spatial fidelity and measurement precision and repeatability with which a sufficient diversity of in situ soil properties and conditions can be ascertained to structurally and functionally characterize the growing environment. The present disclosure overcomes these previous limitations.

[0075] In addition, the utility of GEMS analyses in agriculture have been limited in at least one of spatial, temporal, or depth resolution of GEMS factors, including but not limited to the ability to address management units dow n to the size of individual plants and address soil factors to multiple depths. The lack of resolution of the management units, for instance, has resulted in limited statistical and machine learning capabilities in previous analyses. Furthermore, the lack of resolution of the management units has not supported variable rate recommendations. The present disclosure overcomes these previous limitations.

[0076] Another limitation of GEMS analyses in agriculture is that the forms of the analysis that would allow for the analysis of a combination of a plurality of GEMS factors is poorly understood. While GEMS is a framework for analysis, it does not specify the form or the function. One aspect of the present disclosure overcomes this previous limitation.

[0077] In addition, a limitation of variable rate nutnent applications in agriculture is that the calculation of rates typically requires not only knowledge of the current soil conditions, but of target soil conditions. These target soil conditions, the points at which adding additional inputs are unlikely to support a further change in outcomes, are typically sourced from university extensions, crop consultants, or other published sources and may not be applicable to conditions on a specific management unit. One aspect of the present disclosure overcomes this previous limitation.

[0078] Sub-Surface Suite

[0079] When obtaining soil information via sampling and laboratory analysis, there is always a trade-off between multiple factors w hose interplay conspires to limit the combination of fidelity, diversify, precision, and accuracy with which soil information can be obtained. When analyzing retrieved soil samples from, for example, soil cores, these factors include: the number of samples that are practical to obtain, the volume of each sample needed to support the diversify of laboratory analyses desired, and thus the number of parameters that can be ascertained from laboratory' testing of the volume of soil supplied; the minimum vertical spatial resolution a soil sample can represent given the volume needed for the desired analyses; the representativeness of the actual in situ condition of the soil once the soil sample reaches the laboratory: and the repeatability of the sampling process that affect the aforementioned representativeness.

[0080] The trade-offs involved in soil sampling and analysis that limit the soil information available from which to construct a digital twin have limited the statistical confidence ofpredictive analytical power and thus the operational usefulness of past soil digital twins for providing management decision support through, for example, predictive analytics.

[0081] When obtaining soil information via proximal sensing, a limitation on the combination of fidelity’, diversity, precision, and accuracy with which soil information can be obtained in situ has been the number of sensors, diversity of sensor modalities, and selectivity' of each sensor modality for a specific soil parameter that can be incorporated into a given soil probe. Available computational methods have been a factor in this limitation, as there is a tradeoff between the amount of information that can be obtained by using multiple sensors employing modalities that may be sensitive to more than one soil parameter and the ability to computationally deconvolve the contributions of individual soil properties to specific sensor responses. For example, a tip stress sensor and a sleeve friction sensor in a soil penetrometer will both be sensitive to soil texture, compaction state, and present moisture condition, but each in a different way. An audio sensor that records the sound of the penetration process will be sensitive to texture, compaction state, and present moisture condition, but in a different way than will the tip or sleeve sensors. A dielectric sensor will be sensitive to both soil moisture content and pore water salinity', whereas a direct current (DC) electrical resistance sensor will be sensitive only to soil electrical conductivity.

[0082] The effect of electrical conductivity' as measured by the DC resistivity' sensor can be deconvolved from the response of the dielectric sensor to yield moisture content, which can then be deconvolved from the combination of tip, sleeve, and audio response to yield information about texture and compaction state. But there have previously been no empirical or closed form analytical solutions to this challenge of deconvolving the partial sensitivities of multiple sensor modalities to the partial influences of multiple soil parameters.

[0083] When information dense sensor modalities such as diffuse reflectance spectroscopy (DRS) and video microscopy (VM) are added to the suite of sensors in a soil probe, the degrees of freedom available in the proximal sensing data set as well as the number of soil parameters to which the sensor suite is responsive multiply substantially. DRS in the visible and near infrared (NIR) ranges of the spectrum exhibits absorption features that are associated with particle size effects as well as moisture, in addition to the absorption overtones of organic and inorganic molecules comprising the soil chemistry7and biochemistry which have fundamental absorption bands in the mid-infrared (MIR) spectral range. VM can yield numerous image features that are qualitative and quantitative indicators of soil properties and conditions, including but not limited to color characteristics, fractal metrics such as succolarity and lacunarity, and frequency domainfeatures such as the magnitude spectrum and the phase spectrum resulting from Fourier transform of the image. Combined with high spatial fidelity sensor sampling rates, sensors such as DRS and VM increase sensor data availability to a level that is sufficient to support the training of data hungry machine learning algorithms. Some embodiments of the suite of sensors in a soil probe can include DRS in the MIR as well as in the visible and NIR or other spectral ranges.

[0084] The spatial resolution and coverage with which sensor data pertaining to the crop and / or sensor data pertaining to the soils helps differentiate actual or expected patterns of spatial variation in crop and, by inference, actual or expected patterns of spatial variation in soil assist the grower in determining the appropriate size of management units based on utility for optimizing some objective function, which can be one or more of: maximizing yield, minimizing inputs, maximizing return on investment (ROI) and other appropriate financial metrics, meeting regulatory compliance objectives, and achieving voluntary market-based assessment, verification, and certification programs, or any combination of these objectives or any other management objectives.

[0085] The subsurface suite of soil sensor measurements is obtained using a thin, elongate vertical profiling penetrometer probe, also known as a digital soil core (DSC) probe. In one embodiment, the DSC probe includes sensors for: soil mechanical behavior (i.e., tip bearing stress and sleeve friction) during penetrometer advancement, soil imaging via a charge-coupled device (CCD) video camera and microscope optical components, soil moisture content via the measurement of dielectric permittivity obtained by making the soil a capacitive element in a modified Clapp oscillator circuit, an electrical conductivity (EC) obtained via two independent means: one including making the soil a resistive element comprising a discrete series resistance in a voltage divider, and another making the soil a resistive element in a circuit that measures the time for a capacitor to charge to a specific value of voltage, as well as audio data obtained from a micro electro-mechanical system (MEMS) microphone embedded in the DSC probe that records the sound of the DSC probe penetrating the soils wherein the sound is recognized to change in frequency content and amplitude in response to variation in soil texture, bulk density, in situ moisture condition, the inclusion of stones larger than 2 mm, the presence of chemical precipitates such as fertilizer and other salts, or other soil-related factors.

[0086] Probes of the present disclosure may also include additional optical sensors in an integrated subsurface sensor suite, such as, but not limited to, fiber-based visible and nearinfrared optical absorption and / or diffuse reflection spectroscopy, Raman spectroscopy, etc.Probes of the present disclosure and the various sensor systems therein and coupled thereto may also include any one or more features of the above-referenced documents that are incorporated herein by reference (see para.

[0002] ).

[0087] For example, in some embodiments the DSC probe comprises a Diffuse Reflectance Spectroscopy (DRS) sensor comprising spectrometers located above the ground surface on a push rig that are in optical communication with the soil by way of a fiber optic bundle extending between the spectrometers and an aboveground light source, and optical components located in the probe that direct light between the fibers and the soil. In some embodiments, the fiber optic bundle uses low hydroxyl silicon fiber to transmit light in the visible and near-infrared (NIR) spectral range. In other embodiments, the fiber optic bundle is made of other fiber materials to enable effective use in the mid-infrared (MIR) spectral range or other spectral ranges. In some non-limiting embodiments, the fiber optic bundle includes one or more fibers for conducting light to the soil and one or more fibers for collecting light from the soil.

[0088] The use of the fiber optic bundle and optical components in the DSC probe need not be limited to conducting reflectance spectroscopy. Additional optical methods of evaluating the soil can be performed through the fiber optic bundle and optical components in the probe. Additional optical methods include Raman Spectroscopy, Fourier Transform Infrared Spectroscopy (FTIR), Fluorescence Spectroscopy, Circular Dichroism (CD) Spectroscopy, laser-induced breakdown spectroscopy (LIBS). Colorimetry and / or other methods. Chemiluminescence and Bioluminescence can be measured through the fiber optic bundle by including in the DSC probe by a means of injecting one or more reagents into the soil and examining luminescent, fluorescent, or colorimetric response.

[0089] Tip and sleeve stress measurements are indicators of soil strength, which is neither spatially uniform nor temporally static. Soil strength, comprising both bearing strength (which is closely related to tip stress) and sheer strength (which is closely related to sleeve friction), is controlled by soil texture (which summarizes grain size distribution), compaction state (related to bulk density), and moisture condition (which is both a function of intrinsic properties of the soil such as the soil water retention curve also known as the soil moisture characteristic curve and a function of dynamic processes such as precipitation, irrigation, and evapotranspiration). In addition, compaction state can vary both spatially and over time, as does moisture content.

[0090] A critical physical characteristic of soils that is relied upon heavily in agricultural research and production is the soil textural classification. Textural classification is based on aparameterization of soil grain size, specifically the percentage by volume of the soil matrix whose grains fall within each of three grain-size ranges designated as sand, silt, and clay. Grain size distribution — or its simplification as a textural classification — does not, however, provide all the information about the grain structure of a soil. Due to the compressibility of soils and the ability of their grains to assume different packing configurations, bulk density’ is also an important parameter in further describing the in situ structural state of the soil matrix. In other words, texture and bulk density together provide more useful information about the soil than does each without the other.

[0091] Changes in strength, color, and / or moisture with depth can elucidate changes in texture and hydraulic parameters (e.g., soil behavior with respect to wetting, drying, transmission of water, and w ater holding capacity) with depth.

[0092] Established methods of interpreting profiles of tips stress, sleeve stress, and moisture content correlate these measurements to soil texture. Additional known methods correlate soil textural information to hydraulic properties. For instance, the US Department of Agriculture developed the Rosetta software that estimates Van Genuchten parameters from textural classification and / or bulk density by using an artificial neural network.

[0093] Aspects of the present disclosure include application of machine learning (ML) using convolutional neural networks (CNNs) to ascertain depth-dependent soil textural classification and / or bulk density from sensor data such as any combination of tip stress, sleeve friction, video, audio, moisture, and / or EC profiles alone or in various combinations. Other machine learning models and architectures can be used. For example, machine learning algorithms based on linear regression, gradient boosting, and physics-informed CNNs can also be applied to predict physical and mechanical properties such as hydraulic properties of porous media from images of the media.

[0094] In some embodiments, systems with machine learning models are programmed with instructions and configured to determine vertical profiles of texture, moisture content, and / or hydraulic behavior types directly from profiles of soil probe video and / or audio data.

[0095] In addition to vertical profiles of soil properties and conditions, in another example horizontal spatial variability and its effects on the movement and retention of water and nutrients, as well as on the growth of roots is determined. Established geostatistical estimation techniques attempt to use observed anisotropy and observed spatial scales of variability’ and correlation to predict subsurface soil properties and their structure everywhere in a plot ofinterest that has not been sampled, based on what has been sampled. The present disclosure includes examination of joint and conditional probabilities involving multiple sensed quantities in a spatial context.

[0096] The present disclosure includes sensor systems, data acquisition and data processing to quantify soil organic carbon (SOC) and other important soil health or soil nutritional parameters in situ, in vertically resolved spatial profiles up to 1 meter or more in depth, using data obtained from a suite of proximal soil sensors integrated into a soil penetrometer, thus improving the speed, cost, and accuracy of voluntary market-based assessment, verification, and certification programs targeting soil carbon stock, soil health, and other information useful to crop management such as nutrient and salt concentrations, root distributions, and soil biological activity7without the need to retrieve, process, and analyze physical soil samples.

[0097] SOC has emerged as a critical component in mitigating climate change and ensuring agricultural sustainability'. Conversion of natural ecosystems to agricultural lands, and conventional management of agricultural lands has led to loss of soil organic matter (SOM) and a deficiency in SOC. Changes in management methods have been identified that can increase SOC in affected areas, presenting the opportunity to use these lands as a sink for atmospheric carbon dioxide. Practices that increase SOC are also associated with increased soil health and functionality, leading to increased productivity', sustainable agricultural intensification, and food security . Efforts to estimate the potential global carbon sink through increased global SOC put the ceiling between 8 Pg and 28 Pg. From another perspective, a global increase of 0.4% in SOC annually would offset annual global fossil fuel emissions. These global scale calculations, though eye-catching, belie the complex dynamics of SOC, and the size of this potential carbon sink is uncertain due to the highly localized nature of the determining factors.

[0098] Nonetheless, the depletion of SOC in agricultural soils has attracted the attention of farmers wishing to improve the productivity of their land, as well as emerging carbon markets seeking to sell carbon offset credits. The trustworthiness and success of these SOC-based carbon credits is dependent on measurement of SOC stocks. Representative measurement of SOC stocks is no small task, since SOC is both highly spatially variable, and slow to change over time.

[0099] Systems of the present disclosure are configured to quantify SOC and other soil parameters in-situ, removing costs and barriers to collecting numerous, accurate measurements. The present disclosure includes advanced ML-based processing of data from a suite of sensors in a single device that provides accurate determination of in situ soil water content and particle sizedistribution (PSD) independent of concurrently acquired diffuse reflectance spectra or other spectral optical measurements such as Raman spectroscopy. Optical measurements can be in the ultraviolet, visible, NIR and / or MIR. A method of the present disclosure utilizes the independent determination of the PSD and volumetric water content (VWC) to condition the interpretation of the reflectance spectra. The foregoing may be utilized to determine the actual carbon present in agricultural soils via VNIR spectrometry because the variation in reflectance spectra that is due to differences in chemistry can be isolated from the variation due to particle size and moisture effects by independently quantifying the latter two influences.

[0100] As set forth above, the present disclosure includes a vertical profiling soil penetrometer probe, also known as the DSC probe, that includes multiple sensors. In some embodiments, the DSC probe incorporates independent sensors for VWC and several indicators of soil texture and structure (e g., particle size distribution) concurrent with in situ VNIR reflectance spectra. Through effective exploitation of sensor data from an integrated video microscope, tip stress and friction sensors, and audio emissions, an independent determination of soil roughness and wetness concurrent with DRS can be performed, and to separate their effects from the chemical signatures also contained in the reflectance spectra, without the need to retrieve, process, and analyze physical soil samples.

[0101] Several sensors of the DSC probe of the present disclosure may be sensitive to attributes of soil texture, structure, and compaction state. These may include a tip stress sensor and sleeve friction sensor, to indicate textural classification in geotechnical applications. As the DSC probe penetrates the soil, an internal microphone records the audio emissions produced. The sound is affected by soil texture and structure, compaction state, and w ater content. The conical tip of the DSC may comprise tw o concentrically arranged metallic electrodes used to make rapidly multiplexed measurements of DC electrical conductivity and real dielectric permittivity at a frequency above about 50MHz from which w ater content is inferred. The DSC probe may in some embodiments further include a digital RGB video microscope that performs direct soil imaging through a sapphire w indow . In an example, the digital RGB video microscope collects 1920 x 1080 video at 30 frames per second (fps) with a resolution of about 1.24-pm per pixel. In an example, subsequent images overlap by about 50%, providing more than 1900 images over the 1.3-m depth of each vertical soil profile.

[0102] In some embodiments, the DSC measures soil moisture ahead of its tip to quantify in situ VWC in a manner least distorted by the penetration process, thus acquiring with high qualitythe information needed to accurately determine soil carbon present via VNIR absorbance spectra using machine learning models and other data interpretation methods.

[0103] A method of the present disclosure includes determination of particle size information from tip stress, sleeve stress, audio emissions, and summary visual metrics derived from video microscope imagery acquired in situ using the DSC probe. A multi-task approach, both linear and ANN may be utilized to simultaneously predict multiple soil properties including nutrient content and sand / clay fractions. Leveraging interrelationships between soil properties, including the soil water content, as a prediction task alongside the soil properties, imposes an inferential bias that forces the model’s internal feature representations to consider the soil water content from the spectra, thereby providing more internal context to the inference of soil properties.

[0104] Highly resolved particle size distribution may be determined from this same combination of non-DRS sensor measurements, using full-frame image data collected by the video microscope (rather than just summary image metrics) with sophisticated ML methods, such as fully convolutional networks (FCNs) and transformer networks. VNIR DRS may also be collected from dry samples to consider relationships between spectra and soil properties considering PSD or not.

[0105] In an example, machine learning and analytical architecture integrates the information available from all sensors in the DSC probe (including, for example, VNIR DRS), as well as PSD to perform accurate quantitation of soil carbon and / or other soil parameters important to climate-smart farming.

[0106] DSC Probe Overview

[0107] An example implementation of a DSC probe 100 is depicted in FIG. 1A. The DSC probe 100 includes a plurality of sensors. In this example, the plurality of sensors includes a VNIR DRS sensor 104, a video microscope 108, a sleeve stress sensor 112, an acoustic sensor 116, a tip stress sensor 120, and an electrical-properties-sensing device 124, which senses both volumetric water content (VWC) and DC electrical conductivity (EC). In other embodiments of the DSC probe 100, the plurality of sensors comprises other combinations of sensors — it need not contain each of these seven sensor types and could also comprise other sensors. This is just one non-limiting example.

[0108] FIG. IB shows the tip stress sensor 120. The tip stress sensor 120 measures force on a conical tip 128 of the DSC probe 100. In some embodiments, the conical tip 128 is 60°. Force on the conical tip 128 is an indicator of soil bearing strength, including soil densify and compaction.In contrast to the American Society of Agricultural Engineering (ASAE) standard penetrometer, force is only registered at the conical tip 128, so it is local and independent of any friction developed along the trailing portions of the DSC probe 100. Unlike applied load measured above ground, stress at the conical tip 128 is isolated from side friction.

[0109] FIG. 1C shows the sleeve stress sensor 112, which is just outside of the tip stress sensor 120 and above the conical tip 128. The sleeve stress sensor 112 measures friction exerted by soil surrounding a cylindrical sleeve surface 132 of the DSC probe 100. Measurements from the tip stress sensor 120 and the sleeve stress sensor 112 are both reliable indicators of “soil behavior type”, which combines information about grain size and compaction.

[0110] FIG. ID shows the electrical-properties-sensing device 124. In this example, the electrical-properties-sensing device 124 measures VWC and DC EC. Bearing strength and cohesion are both modified by moisture, so understanding water content is important to textural classification. EC is an indicator of salinity and clay content. VMC and EC are related, so measuring them simultaneously is important to holistic interpretation of data from the DSC probe 100.[OHl] Forming the conical tip 128 are a pair of steel electrodes 136 separated by an insulating element 140. The conical tip 128 is used to measure VWC and EC in rapid alternation. The electrical-properties-sensing device 124 uses the surrounding soil as a capacitive element in a modified Clapp oscillator — a type of analog oscillating circuit. The dielectric permittivity of the soil controls the oscillatory frequency of the modified Clapp oscillator, which is measured using a digital microcontroller. The dielectric permittivity of the soil is recovered via calibration to known standards, and converted to VWC using well-established relationships that are understood by persons of ordinary skill in the art. EC may be measured by a capacitive circuit on the DSC probe 100 and / or by a resistance circuit on the DSC probe.

[0112] By sensing VWC at a very7leading surface of the DSC probe 100, distortion of the actual value by soil compaction due to the penetration process is minimized. In other words, sensing happens ahead of the conical tip 128, where the soil is least disturbed. Sensing through the conical tip 128 also enables more accurate fusion with other sensor data.

[0113] FIG. IE shows the acoustic sensor 116. which is embedded within the sleeve stress sensor 112 and the tip stress sensor 120. Different textures and coarse fragments emit different sounds when penetrated. In this example, the acoustic sensor 116 includes a MEMS digital microphone 144 that records acoustic emissions resulting from re-arrangement particles in the'llsoil and along the DSC probe 100 as a result of displacement due to penetration. Advanced digital signal processing is used to extract meaningful features. For example, the recorded audio provides information that relates to PSD, compaction, and VWC.

[0114] FIG. IF shows the video microscope 108, which is located behind a first sapphire window 148. Video microspore imagery is collected through the first sapphire window 148 under uniform, consistent illumination synchronized to the frame rate. In this example, the video microscope 108 produces Bayer pattern color imagery with approximately I -urn pixel resolution and a spatial density of about 15 images per cm. which reveals color, texture, structure, porosity, permeability, and other important features (see FIG. 2). The video microscope 108 provides continuous video coverage over the depth of a soil profile.

[0115] FIG. 1G shows the VNIR DRS sensor 104. which is located behind a second sapphire window 152. In this example, VNIR DRS data is conducted through the second sapphire window 152 at a rate of four scans per second. The VNIR DRS sensor 104 has a downhole optical design that makes it optimized for maximum signal-to-noise ratio (SNR) in collected spectra. Rapid scanning by high quantum efficiency, linear detector array spectrometers of the VNIR DRS sensor 104 provide fine spatial resolution with multi-scan averaging to further increase SNR. The VNIR DRS sensor 104 produces analytic-grade reflectance spectra for estimating SOC, nutrients, and / or other parameters.

[0116] With its seven different types of sensors each employing largely orthogonal modalities, the DSC probe 100 provides over about 1,200 independent pieces of information for every centimeter of soil that it encounters. Each of the sensing modalities produces independent pieces of information, and there also may be independent pieces of information within each modality. For example, the diffuse reflectance measured at each band of wavelength in the visible and NIR spectra of the data collected by the VNIR DRS sensor 104 is independent of the diffuse reflectance measured at every other band. As another example, the hue, value, intensity, succolarity, lacunarity. and other characteristics of microscope imagery' collected by the video microscope 108 are each independent pieces of information.

[0117] FIG. 2 shows example microscope images 200, 204, 208, 212, 216, 220, 224, 228 acquired in-situ by the video microscope 108 of the DSC probe 100. The example microscope images 200, 204, 208, 212. 216, 220, 224, 228 show a variety of textures, colors, structures, and macropores, including penetration-related strain features.

[0118] Aspects of the present disclosure include: methods of estimating particle size distribution based on video microscope images and other sensors; a process-based model that correlates the physical relationship of particle size distribution to reflectance spectra; a system configured with a correlation between laboratory SOC measurements and VNIR DRS spectra with and without PSD information; a system configured to use independent measures of particle size distribution and water content to estimate soil properties including SOC using VNIR reflectance spectra; a system configured to determine practical maximum limit of water content for predicting PSD. SOC, and other soil properties; methods of characterizing error and uncertainty in estimates of SOC pools; methods of coordinating with standards bodies to ensure uti 1 ity for offset verification protocols; and methods for developing strategies for incorporating the information processing into the DSC software, and presenting data in useful formats and units.

[0119] Soil core sample data, soil samples, and DSC probe data, including imagery, are collected. Laboratory analysis of the soil cores may include physical properties (sand / silt / clay (SSC) fractions, bulk density) and chemical composition (soluble salts, B, Ca, Cu, Fe, K, Mg, Na, NO3-N, Total P, HCO3-P, S, Organic Matter, cation exchange capacity).

[0120] Image Preprocessing and Feature Extraction

[0121] To align continuously collected images from the video microscope 108 with soil core strata, images are subsampled to have a plurality, for example, 14 images representing each cm of depth. The images may be calibrated for both color value and color consistency. In embodiments where more than one DSC probe is used, each probe’s imagery may be color calibrated by applying a probe-specific color shading table in firmware. This color shading table may be computed by imaging and analyzing several standard color chips. Color calibration includes chief-ray-angle compensation, color correction, and vignette-removal. Each color- calibrated frame of the input video may be processed by a variety of algorithms for generating representative features.

[0122] Image features may be extracted that describe an image and its content, through both its color or value and its structural level of detail. In addition to these explicit features, a set of ambiguous features may be extracted using a pre-trained image classifying CNN.

[0123] HSV color values may be determined for both the mean image color, as well as the color values of three K-means regions. Approximate porosity may be determined by measuringnumber of pixels with color below a specific threshold. Several metrics associated with the grain structure of the images may be computed such as those described below.

[0124] Succolarity is a metric intended to measure the ability of a hypothetical fluid to flow through the structure of a medium, and has been shown to be an effective way to approximate porosity in other materials. The further the fluid can travel as determined by the contrast structure of the image, the larger the succolarity value. Succolarity curve values were based on building the succolarity function around 1.000 times on a given frame, increasing the thresholding value. Succolarity curve metrics parametrically describe the shape of the resulting sigmoid curve. Succolarity curve metrics include the start of the curve rise, the end of the curve rise, the midpoint, the difference of the start and end, and the ratio between the start and mid. The ratio measures the skew weighting of the curve, representing how even the two ends of the curve are.

[0125] Entropy is a metric to describe perceivable detail in an image, or its digital compressibility, and is calculated as the complexity in a given neighborhood. An image entropy function such as the one in the Python scikit-image package may be used.

[0126] Fractal dimension is a real-number representation of the relationship between an object’s volume and its total surface area. In image processing, a fractal dimension is computed as the fractality or fractionation between pixels within an image based on the amount of fine structure in the image. In an example, a Python-based implementation of the algorithm utilizing the hierarchical multi-scale box-sliding method to determine the fractal dimension may be used.

[0127] Image lacunarity is a secondary fractal metric that is intended to represent the extent of gaps in an image, where larger gaps between structure may result in a higher lacunarity. In some embodiments, image lacunarity methods use the hierarchical multi-scale box-sliding method described above for image entropy.

[0128] Gabor filters are commonly used for image texture analysis and have been successfully used to extract features related to soil texture. Python’s scikit-image package may be used to generate and apply Gabor filters to images.

[0129] Processing Multi-Sensor Probe Data

[0130] The plurality of sensors of a multi-sensor probe of the present disclosure, such as the seven sensors of the DSC probe 100, each employing largely orthogonal modalities are sensitive in different ways to numerous mechanical, structural, and chemical soil parameters both directly,and indirectly by way of spatial correlation between the distributions of some soil parameters and others due to the morphological and solute transport processes including root action that determine the spatial distribution of, for instance, chemical species in the soil. However, sorting out the causal and correlative relationships of these multiple sensor outputs with the multiple soil parameters, which may be multivariate and have non-linear relationships, with the aim to develop predictive models of soil parameters as outputs based on multiple sensor responses as inputs is a task that is well-suited for various supervised machine learning techniques including but not limited to partial least squares regression (PLSR), random forests (RF), various types of artificial neural networks (ANN), etc.

[0131] Sensor Data Acquisition

[0132] Training a ML model to estimate soil parameters from sensor outputs requires training and testing datasets comprising a very large number of training examples that relate sensor output to independently measured soil properties in the same material. To collect such data, the DSC probe 100 is advanced into the soil to measure a vertical profile of sensor response. In some embodiments, the DSC probe 100 is digitized and electronically records data several times per second from each of the plurality of sensors, with the probe moving at a controlled rate of about two centimeters per second.

[0133] For sensors that produce a single data value per sample in time, in some embodiments the sensor readings are recorded in a tabular data file with each row corresponding to a time of acquisition that is also recorded, but other data structures and formats can also be used. Probe depth relative to a datum such as the soil surface is also recorded as a function of time from a depth sensor such as a string potentiometer or an ultrasonic distance sensor. In some embodiments, time indexed sensor readings are recorded several times per second.

[0134] Readings from sensors that produce multi-dimensional data are recorded using other formats. For example, data acquired by the video microscope 108 represents RGB pixel values in a two-dimensional array representing an image of soil encountered at discrete intervals of time corresponding to the video frame rate, and is stored in video format files bearing a file creation time stamp corresponding to the start of the push. Knowing the frame rate and the video start time, the exact time of acquisition of any video frame can be calculated and aligned with a depth by correlation to the time indexed depth sensor data. Conversely, the video image frame corresponding most directly to a specific depth in the soil can be recovered from the video file by locating the frame whose time of acquisition corresponds most closely to the time that specific depth was encountered by the video microscope 108. Resolution is based on video frame rate. Insome embodiments, the video frame rate is set to 30 fps, but can be another value in other embodiments, such as, but not limited to, at least 15 fps. In the example of setting the video frame rate to 30 fps, the DSC probe 100 is able to resolve an image to within l / 30th of a second, which corresponds to within 1 / 15th of a centimeter (or less than 1 mm) of depth when the push rate of the DSC probe is 2 cm per second. Other frame rates and push rates are also possible.

[0135] Audio data from the MEMS digital microphone 144 is similarly recorded in linear audio format files each labeled with an acquisition start time. In an embodiment, audio is sampled at about 48 kHz, enabling the start and end of audio associated with a specific centimeter of soil depth to be located and extracted with about 0.02-millisecond precision. In other embodiments, audio is sampled at other frequencies.

[0136] VNIR reflectance spectra are acquired from the VNIR DRS sensor 104 in subsequent sub-second scan intervals. In some embodiments, the sub-second scan intervals are 52 milliseconds each. Four such scan intervals are acquired each second and the resulting reflectance spectra from each second of acquisition are then averaged to represent two centimeters of soil depth.

[0137] ML Target (Soil Parameter) Data Acquisition

[0138] ML target data are acquired by collection and analysis of soil samples taken in vertical profile via soil coring in close proximity to the vertical profiles of sensor data acquired using the DSC probe 100.

[0139] Sensor Data Pre-Processing

[0140] Conversion of Sensor Data to Engineering Units

[0141] The raw data from the plurality of sensors represent electrical transducer output as voltages, resistances, analog-to-digital converter (ADC) integer output, or frequency. These have a direct correspondence to engineering units such as, in the case of tip stress and sleeve stress, force per unit area, which is obtained from the output of strain gauges attached to metal parts that exhibit strain in linear relation to applied force. In some embodiments, calibrations that relate strain gauge output in millivolts per volt of excitation are conducted prior to in situ data acquisition in soils. These calibrations involve placing the DSC probe 100 in a calibration fixture that uses user-controlled compression of fluid in a hydraulic cylinder to apply a controlled load through a third-party load cell alternately to the conical tip 128 or cylindrical sleeve surface 132 and record the output of the strain gauges.

[0142] In some embodiments, a least-squares fit of a first order polynomial to a series of calibration data is used to relate ADC values that directly correspond to mV of strain gauge output to the applied load normalized to the projected area of the load sensing element such as the conical tip 128 or cylindrical sleeve surface 132. In some embodiments, the first order polynomial applied to the raw sensor output data produces measurements of stress in pounds per square inch. In similar fashion, raw resistivity sensor values are transformed to soil resistivity and the frequency of the modified Clapp oscillator is converted to dielectric permittivity of the surrounding media via a three-point calibration curve through which a linearly transformed halfsine curve is fitted. VWC may be then quantified by applying a well-known equation relating soil dielectric permittivity to VWC, such as the Topp equation, the Ledieu equation, or the Ferre equation. Conversion to engineering units can be performed during data acquisition or after data acquisition.

[0143] Depth Registration and Co-Regi strati on of Sensor Data

[0144] Pre-processing of the sensor data includes averaging the readings from each sensor over fixed increments of vertical travel, which involves first converting time indices to depth indices using the time-registered depth sensor data.

[0145] Data is not always acquired in uniformly spaced increments of time. Similarly, the speed of penetration of the DSC probe 100 into the soil can vary slightly in time. Each reading from each of the plurality of sensors is therefore not likely to align with respect to depth precisely with readings from other sensors.

[0146] Since each of the plurality of sensors are each incorporated at different positions along the axial dimension of the DSC probe 100, to correlate sensor readings from multiple sensors with the same soil depth interval, a sensor position can be accounted for by subtracting a sensor position offset measured from the conical tip 128 of the DSC probe from the recorded depth measurement associated with the time of acquisition of each sensor reading. Sensor data processing also generally includes re-sampling each sensor’s output on evenly spaced discrete depth intervals. In some embodiments, multiple readings from each sensor are averaged over or re-sampled for each centimeter of distance traveled and recorded in a tabular data file with each row corresponding to a depth interval, but other data structures and formats can also be used. Averaged and re-sampled readings from multiple sensors are collated at each depth increment and may be recorded in different columns of the same data file.

[0147] Converting Sensor Data to Conventional Soil Parameters

[0148] A benefit of the multi-sensor probes of the present disclosure, such as the DSC probe 100, is to eliminate the need for soil samples to be collected and sent to a laboratory for analysis in order to understand the distribution of soil properties such as SOC, nutrient concentrations, color, redox state, pH, salinity, texture, structure, etc. and the sequencing and thicknesses of layers of soil wherein each layer is defined by substantial differences in average layer properties relative to properties of other. To achieve this benefit, multivariate calibration models relating multiple sensor outputs to each property of interest may be developed. In an embodiment, these calibration models comprise supervised machine learning (ML) models such as partial least squares regression (PLSR), random forests (RF), artificial neural networks (ANN) or other supervised machine learning models that use sensor output as input and produce soil properties as output. Such models can be configured and trained to estimate a single soil property based on multiple sensor inputs or to simultaneously estimate multiple soil properties based on multiple sensor inputs.

[0149] In order to train the supervised ML models, a training dataset is assembled, which collates sensor readings to soil properties as determined by independent means, such as through sampling and laboratory analysis. In one example, training datasets are assembled by acquiring pairs of soil profile data in which one member of the pair is the depth-referenced soil sensor outputs from the multi-sensor probe and the other member of the pair is the results of independent analysis of depth-indexed samples of soil obtained in close proximity, for example, 0 cm to 100 cm, to the location of the vertical sensor profile. Soil samples for analysis by independent means can be obtained via soil coring, excavation, for example, soil pits, or any other methods of retrieving a soil sample, or may be analyzed in situ, such as in the wall of a soil pit using a field analytical instrument. Samples from multiple depths are analyzed using conventional laboratory' methods or field analytical methods involving laboratory instrumentation and equipment or field instrumentation and equipment. Pairs of a vector of sensor values and a vector of soil properties acquired from the same depth or the same soil horizon, i.e., layer, comprise training examples for training the ML model.

[0150] Training of a ML model providing calibration of one or more soil properties to multiple sensor readings proceeds as would training of any supervised ML model according to methods that are well known in the art, with some key differences in the training data sets. For instance, standard practice in submitting a soil sample isolated to represent a given layer or horizon from a continuous core for laboratory analysis is to gather and send the minimum quantity of soil required by all the laboratory methods in sum that may be applied to obtain all the soil propertiesof interest. This quantity is typically 500 g to 2000 g of soil, which can typically account for all the soil within a 5 cm to 10 cm depth interval of a continuous core. The laboratory may return a single result for each soil parameter that is analyzed representing that discrete interval of soil. In contrast, a set of readings from the collection of sensors comprising the multi-sensor probes may represent a single cm of soil. Therefore, combinations of sensor readings representing different values of the combination for each centimeter of sensor values may be paired with a single depth interval of soil sample representing multiple centimeters of depth. In other words, a diversity of sensor readings may be associated with each value of a soil property obtained from laboratory7or field analysis of a core sample.

[0151] To improve the performance of the ML model, in some embodiments the method amplifies the training data set by adding variability to the values obtained from laboratory7analysis such that, for example, each value of a specific soil property obtained from laboratory analysis of a 10-cm segment of core is used to produce 10 values each representing one centimeter of the depth profile collated to each 1 cm of depth for which a combination of sensor readings has been obtained. The transformation of one value of soil property7to ten values of soil properties is conducted within constraints by randomly selecting a value within the uniform or gaussian distribution of deviations from the laboratory value as determined and limited by the statistical accuracy or repeatability of the laboratory method by which the soil property value w as obtained. For example, if sieve analysis for texture produces a percentage of sand, silt, and clay w hose published laboratory repeatability is + / - 8% of the value reported, then a specimen representing 10 cm of soil core that was reported by the laboratory to have 45% sand, 30% silt and 25% clay can be used to generate dithered values of texture for each of the 10 cm represented in the specimen, having for instance, sand-silt-clay percentage ratios of 40-30-30, 50-30-20, 45-34-21, 45-26-29, 50-30-20, 49-32-19, 44-31-25, 46-31-23, 46-29-25, 41-29-30 with each combination of sand-silt-clay percentages then paired in the training data set with a combination of sensor values obtained from a specific centimeter of depth within the depth interval from which the soil sample was obtained. In this manner, noise is added to the training data and the number of unique training examples of paired sensor and soil parameter data is increased to provide more robust ML model development.

[0152] Other aspects of how data can be used and paired, and introduced as inputs and outputs to train supervised ML models to estimate soil properties are described herein.

[0153] Convolutional neural networks (CNNs), including fully convolutional networks(FCNs), transformer networks, and other forms of convolutional netw orks, are effective imageclassifiers. Several networks pre-trained on a massive labeled image database are freely available, such as several versions of ResNet and Inception. These trained networks are routinely used as is for extracting image features to be used in subsequent processing for a variety of technical applications. In an example of transfer learning, a pretrained network may be applied to soil images to extract a feature set that is fed to an additional stage of processing for classification or regression that can be trained using a more parsimonious set of labeled soil images. Similar processes which leverage pre-training a network on a large corpus of more general imagery and performing soil-specific classification or regression using a smaller corpus of labeled soil images may also be referred to as finetuning a pre-trained network. Similarly, methods that apply self-learning with application-specific finetuning may also be applied effectively.

[0154] Align Soil Core Data and Image Features

[0155] Continuously collected data and images from multi-sensor probes of the present disclosure may be aligned with soil core and PARIO data. Each vertical cm of soil may be used as a sampling increment. Features from a plurality, for example, 15 images per cm, may be averaged. The CNN extracted features may not be able to be averaged the same way, and one of the 10 images may be selected at random. In some embodiments, supervised machine learning methods are applied to PSD estimation.

[0156] A multi -target random forest (RF) may be utilized, such as shown in FIG. 3. An RF is an algorithm composed of a multitude of decision trees, which are predictive models in the structure of an upside-down tree. FIG. 3 shows two example RF plots 300 and 304. RF plot 300 shows sand fraction predictions versus actual measured fractions. RF plot 304 shows clay fraction predictions versus actual measured fractions.

[0157] FIG. 4 shows an example of a decision tree 400. The decision tree 400 contains a root node 404, decision nodes 408 (only two labelled to avoid clutter), and leaf nodes 412. Decision nodes 408 are nodes with two or more branches, and leaf nodes 412 represent a classification or regression decision. Branches 416 (only two labelled to avoid clutter) connect the various nodes. To create each decision, the algorithm splits samples on available variables and then selects the split that results in the most homogenous sub-nodes.

[0158] During training, each decision tree within a random forest is constructed from a random subset of features sampled independently from the same set for all trees in the forest. This creates an inherent variation among trees and corrects for the tendency of a sole decision tree to overfitto the training data, where “overfit’ ’ describes a model with an inability to generalize from the training data to unseen test data.

[0159] Preparing Data for ML

[0160] In some embodiments, data may be preprocessed and missing values may be imputed using appropriate methods, such as the K-nearest neighbor model. These methods work by finding the samples in the training set that are closest to the sample with the missing value and averaging the nearby points to impute it. The soil core fractions of SSC as reported by the laboratory are given within a plus / minus percentage threshold of accuracy. In order to more closely represent reality, this percentage threshold may be used to randomly jitter these textural fractions according to a random uniform distribution of the magnitude of the measurement uncertainty7, while still ensuring that the SSC fractions sum to one. An example of the transformation before and after the application of this jittering process is shown in the frequency histograms in FIG. 5. The histograms of FIG. 5 are organized in a top row 500 and a bottom row 504. The top row 500 includes raw measured data for sand 508, raw measured data for silt 512, and raw measured data for clay 516. In the bottom row 504, a realistic magnitude of random noise has been added to the raw measured data. In other words, the raw measured data is randomly jittered or dithered. The bottom row 504 therefore includes dithered data for sand 520, dithered data for silt 524, and dithered data for clay 528.

[0161] Optimize Tree Features

[0162] After this data cleaning and preprocessing, manual and automated hyperparameter tuning may be conducted to choose a set of optimal hyperparameters for the RF to achieve a high model performance. To do this, a randomized search may be run with cross-validation using Scikit-Leam’s RandomizedSearchCV and / or a grid search with cross-validation using Scikit- Leam’s GridSearchCV. Random search algorithms try7a random combination of a range of values for a defined number of iterations, and are generally chosen for their ability to reach a good combination computationally quickly and allow a narrowed-down range to be established for each hyperparameter to concentrate the search. Grid search may then allow explicitly specified combinations of hyperparameters to be evaluated instead of sampling randomly from a distribution.

[0163] Various hyperparameters of the RF may be optimized, such as max features (the number of input features to consider at every7split within a decision tree), max_depth (the maximum number of levels in a tree), min_samples_split (the minimum number of samples required to split a node), and criterion (the function used to measure the quality of a split) byrunning the randomized cross-validation search for ten iterations. Controlling the upper bounds of the search space for hyperparameters such as max depth and min_samples_split, in particular, are important in preventing overfitting. Once established, these hyperparameters may be used to run the RF to train and test the resulting data.

[0164] Feature Importance and Target Selection

[0165] Feature selection helps to improve accuracy, and including too many variables reduces computational speed and can result in overfitting. In some embodiments, to determine the relative contribution of different input variables to make predictions, the Boruta method is used to assess feature importance. This method includes randomized replicates of the input variables to a RF in order to determine which input could be removed with the least loss of predictive power. A Python implementation of the Boruta method may be used.

[0166] Evaluate PSD Estimates

[0167] To quantify the performance of methods disclosed herein, a comparison of image- derived PSD with depth fractions from physical soil cores may be performed. PSDs may be consolidated into sand / silt / clay fractions.

[0168] Established metrics may be used for evaluating performance of the methods. The accuracy of the predictions could be determined using metrics such as the root mean square difference (RMSD) or R2 between measurements and precisions. The ratio of spread over error (RSE), the ratio of the standard deviation of a set of averaged samples to the RMSD of replicate samples provides a basis for quantify ing a degree of measurement precision. Model validity may be quantified with the ratio of performance to interquartile distance (RPIQ), computed from prediction errors and the variation of observed values.

[0169] Collect VNIR Reflectance Spectra and Video Imagery

[0170] Multi-sensor probes of the present disclosure may be used on dried, unground samples in a lab to collect reflectance spectra and video imagery. Reflectance spectra may be collected with a reflectance spectrometer. In an example, the system may comprise two high-sensitivity spectrometers, a high-powered light source, and customized optics. A linear diode array spectrometer, Ocean Insights QEPro (operating between 350 and 900 nm) and an Ocean Insights NIRQuest+ (operating between 900 and 2500 nm) may be used. The reflectance light source may be a fiber optic coupled, 20 W tungsten-halide lamp, also from Ocean Insights (HL- 2000HP). The illumination radiation and reflectance signal may be carried in a customized bundle of 600 micron low-hydroxyl optical fibers separated from the soil sample by a sapphirewindow. Instrument and data collection parameters may be optimized by Insight - QEPro for maximum signal to noise ratios, and all subsequent data processing may be based on raw (digitized) spectrometer output. Reflectivity may be normalized between 0-1 and checked with the calibration process.

[0171] Audio-Vision Soil Penetrometer

[0172] In some embodiments, multi-sensor probes of the present disclosure comprise an audiovision soil penetrometer. Techniques are disclosed for systems and methods using acoustic sensing modules to detect layer transitions between soil materials from audio recordings of audio-vision soil penetrometers advancing into soil, and to classify soil material from the audio recordings. A soil layer detection and soil classification system may include one or more acoustic sensing modules, a logic device, and a communication module. The logic device may be adapted to process recorded audio to detect transitions between soil materials and to classify soil material encountered by the audio-vision soil penetrometer as it advances into the soil. The logic device may also be adapted to use the communication module to report detected soil layer transitions and soil classifications to an indicator, a display, a user interface, and / or a penetrometer data acquisition and control system. The system may further comprise a window disposed in the outer surface of the audio-vision soil penetrometer and a camera module disposed within the audio-vision soil penetrometer in such a manner that the field of view (FOV) of the camera module includes soil outside the audio-vision soil penetrometer in the scene imaged by the camera module, from which images of the soil are captured and processed by the logic device in conjunction with the logic device’s processing of audio recordings to detect layer transitions between soil materials and / or to classify soil material. In some embodiments, additional sensors are configured to acquire information that is useful for classifying soil material and / or determining in situ parameters of soil state, such as in situ bulk densify and moisture content.

[0173] The logic device may be a microcontroller. In some embodiments, the logic device classifies soil material into texture-based categories, grain-sized-based categories, and / or colorbased categories. In some embodiments, the logic device classifies soil material into the categories of the United States Department of Agriculture Natural Resources Conservation Service (NRCS) soil texture triangle, the categories of the Official Soil Series Descriptions (OSD) United States Department of Agriculture Natural Resources Conservation Service (NRCS), categories of the Soil Series Classification Database (SC) of the United States Department of Agriculture Natural Resources Conservation Service (NRCS), categories of the Universal Soil Classification System of the International Union of Soil Sciences (IUSS), and / orcategories of the World Soil Classification schema of the Food and Agriculture Organization of the United Nations (FAO). Tn some embodiments, sensor data is snapped to a grid to fit into any of the official soil classification schemes of specific standards bodies or governmental organizations, by a nearest neighbor or vector distance calculation through supervised machine learning techniques.

[0174] In some embodiments, the logic device is further adapted to: determine the in situ bulk density of soil through which the penetrometer is advanced; determine the in situ water content of soil through which the penetrometer is advanced; and / or use the communication module to report detected soil layer transitions and soil classifications.

[0175] In some embodiments, the acoustic sensing module is sensitive to frequencies of acoustic signal within the range of human hearing. In other embodiments, the acoustic sensing module is sensitive to frequencies of acoustic signal beyond the range of human hearing.

[0176] In some embodiments, the processing of audio recordings performed by logic device further includes conversion of time domain representation of the audio recordings to frequency domain representations of the audio recordings.

[0177] The camera module may be sensitive to visible light, infrared light, and / or ultraviolet light. In some embodiments, the camera module is a multispectral imaging module sensitive to light in any discrete waveband in a range including ultraviolet to mid infrared.

[0178] In some embodiments, detection of soil layer transitions and / or classification is performed by means of an ANN. The ANN may be a CNN, a deep learning network (DLN), a recurrent neural network (RNN). a long short term memory (LSTM) network, or other network. In some embodiments, the activation function(s) of the ANN comprise one or more of an ReLU function, a sigmoidal function, or similar class of functions.

[0179] Strobing Video Cone Penetrometer

[0180] In some embodiments, instead of using a multi -sensor probe, a strobing video cone penetrometer is used to collect data for the subsurface suite. In particular, the strobing video cone penetrometer may be used for acquiring a series of images of soils in situ with images arranged in a sequence corresponding to a vertical profile through the soil.

[0181] The strobing video cone penetrometer comprises a camera, and illuminates an image using one or more strobed light sources synchronized with the exposure time interval of the camera. Activating the scene illumination source via strobing that is synchronized with theexposure time interval of the camera, also known as the shutter interval, enables the exposure to be limited to an illumination interval that is of a shorter duration than the minimum shutter speed of the camera that, for a camera pose that is moving relative to the scene, enables limiting the relative motion of the camera relative to the scene during image capture, thereby reducing the amount of motion blur that occurs while the shutter is open. But that is not the only benefit of illumination by shutter synchronized strobes.

[0182] Alternating Between Multiple Limited Ranges of Illumination Angles

[0183] In some embodiments, the strobing cone penetrometer further comprises a sapphire window against which soil particles are pressed by ambient forces. Since the image is of the interface of an inherently 3-dimensional medium with the 2-dimensional face of the sapphire window, a textured surface of soil is exposed to the camera. Illuminating this textured surface or “scene” from a relatively shallow angle relative to the plane of the sapphire window during image capture produces more structure in the patterns of light and shadow in the acquired image that are correlated to the size distribution and arrangement of soil grains in the scene than does illumination from an angle closer to that of the viewing angle.

[0184] The information content of a digital image relates directly to the length of description required to enable its exact replication. An image with relatively more structure to the light distribution requires a longer description than one with relatively less structure to the light. Therefore, an image produced by illuminating from a shallow angle relative to the face of the sapphire window carries more information content than an image illuminated more directly from an angle closer to the camera viewing angle. Consequently, when an image illuminated from a limited range of angles is used as input to a machine learning algorithm, such as a convolutional neural network designed to learn about soil grain size distribution from images of the soil, the performance of the machine learning algorithm is improved by using images of soil in which the face of the soil is illuminated from a shallow angle.

[0185] Further, the data set for training and application of the machine learning model can be amplified by acquiring images of the same face of the same soil illuminated from multiple independent directions. In other words, in some embodiments images are acquired by alternating between multiple limited ranges of illumination angles.

[0186] Soil Probe Sensor Suite for Soil Classification

[0187] Soil Classification Background

[0188] In conventional geotechnical technology, soil layer structure is classified into textures of ‘types' by examining a combination of mechanical strength parameters measured by a penetrometer, sometimes including excess pore water pressure, for example, that pressure in excess of a hydrostatic distribution that occurs due to the penetration process. Several researchers have developed soil classification charts based on cone resistance and sleeve friction. Others accounted for pore pressure measurements for soil classification, and also provided correlations between cone measurements from cone penetration tests (CPT) and the soil liquidity index.

[0189] Thus, using three parameters, a soil texture generally regarded as comprising three mass fractions (i.e., sand, silt, and clay) is inferred. However, the strength parameters measured by the cone are dependent on more than textural composition. They are also affected by compaction state, which manifests as in situ bulk density, and by moisture condition generally expressed as volumetric water content that also happens to be variable over time. Five parameters of state (i.e., three mass fractions of particle size, in situ bulk density, and moisture) cannot be fully characterized by measuring three proxies (tip stress, sleeve stress, and excess pore water pressure — the last of which only exists under saturated conditions that are of nearly no practical interest in agriculture). In an example, systems, devices, and methods of the present disclosure may be utilized to solve the five parameters of state utilizing video and acoustic sensing, water content sensing, and near infrared spectroscopy. Any combination of the foregoing may be utilized in a penetrometer probe with a plurality of sensors.

[0190] Probes of the present disclosure may include a video camera that introduces not just one more variable, but four variables (e.g., red, green, and blue color intensities and overall luminance), each at an individual pixel level, as well as information expressed in the spatial patterns present of the visual scene structure (e.g., resolution of soil particles, fractures, water, etc.).

[0191] Probes of the present disclosure may include a microphone to capture audio data generated by the steady-rate penetration process, the overall energy and spectral content of which varies as a function of the soil being penetrated. In an example, the combination of acoustic and visual information may obviate the dependency on tip and sleeve stress.

[0192] Probes of the present disclosure may include a water content sensor that uses dielectric permittivity as a proxy for water, since there is strong contrast between the dielectric permittivity of water and the permittivity of dry soil particles and air. Probes of the present disclosure may include an electrical resistivity measurement. A method of making electrical contact with the soil to obtain the dielectric measurement may be beneficial because it co-locates the electrical measurements with the tip stress measurement at the conical face of the probe where it encounters the soil before it is significantly disturbed and rearranged by the passage of the probe. A coaxial arrangement of electrical contacts also enables a coaxial cable to run from the contact with the soil all the way back to the measurement electronics located some distance away, which facilitates isolating the radio frequency signal applied to the soil to sense dielectric from noise and interference from the ambient environment and other probe electronics.

[0193] Digital Twin of Growing Environment

[0194] In some embodiments, methods of creating a digital twin of a crop in relation to its growing environment include creating a digital twin of the crop, creating a digital twin of its growing environment, and using at least one mathematical model to relate crop behavior and performance of the digital twin of the crop to characteristics of soil in the digital twin of the growing environment. The digital twin of the growing environment is constructed with data collected from a soil probe, such as a multi-sensor probe like the DSC probe 100.

[0195] FIG. 6A and FIG. 6B show two example methods for simulating plant performance for the digital twin of the growing environment. In FIG. 6A, a first method 600 relies on two sequential models and intermediate prediction of traditional soil characteristics that can be unreliable due to deficiencies in how “ground truth” for developing the model is traditionally obtained. In FIG. 6B, a second method 604 employs the mathematical principle of composability to reduce the two sequential models into a single model, eliminating the intermediate steps.

[0196] In the first method 600 shown in FIG. 6A, at step 608, a soil probe 610 is deployed. In some embodiments, the soil probe 610 comprises the DSC probe 100. The soil probe 610 is deployed at one or more locations selected on the basis of spatial variation in topography, plant behavior or performance, surface geophysical indicators, or other indicators of spatial variability in soils. At step 612, depth-dependent raw sensor data 614 is acquired at each of the one or more locations as outputs from the soil probe 610. In some embodiments, step 612 is also accompanied by the collection of soil samples to be submitted to conventional laboratory analysis of soil characteristics. At step 616, a first model 618 is developed to predict traditional soil characteristics based on raw sensor values. In other words, the raw sensor data is the inputand traditional soil characteristics are the output — the first model 618 relates traditional sensor characteristics to sensor data. The first model 618 can be a statistical model, a machine learning model, multivariate, linear, non-linear, or any other type of computational model that predicts one or more soil characteristics from the output of one or more sensors. Once the first model 618 is developed based on a subset of locations with which the soil probe 610 has been deployed, it can be applied in step 620. At step 620, the raw sensor data is converted to depth-dependent traditional soil characteristics 622. The first model 618 converts the raw sensor data obtained from the one or more other locations to which the soil probe 610 has been deployed, to predict one or more depth dependent soil characteristics 622 at additional locations where the soil probe 610 has been deployed but corresponding soil samples are not obtained. The first model 618 can also be applied to refine the vertical spatial fidelity with which the distributions of soil properties are represented.

[0197] In other words, the first model 618 relates depth-dependent sensor values, a nonlimiting subset of which are shown as depth-dependent raw sensor data 614, from one or more in situ proximal soil sensors in soil probe 610 to depth-dependent conventional soil properties and parameters, collectively called soil characteristics, a non-limiting subset of which are shown as depth-dependent traditional soil characteristics 622, as would be determined by analysis of soil samples, obtained by extracting a soil core or digging a soil pit, using soil analysis methods commonly applied in a fixed laboratory setting. This method requires the development of a first one or more mathematical models 618 that relate the depth-dependent raw sensor data 614 to soil characteristics 622 determined by analyzing soil samples obtained from a location proximal to the location at which the sensor profiles are obtained, within 1 meter for example. Alternatively, the first one or more models 618 can be developed by using soil characteristics known from a prior analysis of soil samples obtained proximal to the sensor location or from a database of known soil characteristics or descriptions previously associated with the location from which the soil sensor data is obtained, such as the SSURGO database, not shown. The first model 614 can also be developed from site-specific or region-specific soils data or it can be developed from a global database of soil characteristics and sensor responses such as Kellogg Soil Surv ey Laboratory (KSSL) spectral library', which is a database of soil samples that contains analytical data for over 20,000 pedons of U.S. soils and about 1,100 pedons from other countries.

[0198] At step 624, a second model 626 is developed to predict aspects of plant behavior and / or performance based on variations in depth-dependent traditional soil characteristics 622 within a subset of a given field. The second model 626 and the first model 618 are typically developed independently from one another, as the nature and form of the first model 618 do notdepend on the nature and form of the second model 626 and vice versa. The second model 626 predicts the plant behavior or performance based on soil characteristics 622 determined using the first model 618. In other embodiments, the second model 626 predicts plant behavior or performance based on soil characteristics determined in a more traditional manner, such as by analysis of soil samples in a fixed laboratory or by retrieval of characteristics from a database such as SSURGO based on the NRCS soil map unit associated with the location of the plant. At step 628, the second model 626 is applied to a remainder of the field to simulate plant performance based on the depth-dependent traditional soil characteristics. In this nonlimiting example, the second model 626 predicts the initial bud count on each tree in an almond crop from characteristics of the soil in which each tree is growing, as shown in graph 630.

[0199] In the manner of predicting plant behavior or performance from soil sensor data shown in FIG. 6A. each type of model involved, by nature of being an incomplete approximation of a complex reality, produces uncertainties and inaccuracies because the process involves multiple models applied in succession. In particular, the intermediate prediction of traditional soil characteristics can be unreliable due to deficiencies in how “ground truth” for developing the model is traditionally obtained. Sample sizes required to support conventional laboratory analysis often substantially exceed the spatial scale within which soil properties vary. In contrast, proximal sensors are in close proximity to soil in situ with the potential to continuously characterize the variability of soils along minimally disturbed in situ soil profiles, while conventional extracted soil samples used for laboratory-based soil analytics provide much more limited vertical resolution. A typical laboratory requires 500 grams of soil sample which equates to about a 30 cm section of a standard core to run a full laboratory testing panel.

[0200] FIG. 6B shows the second method 604 of predicting plant behavior and performance from soil sensor data. While the second method 604 in FIG. 6B includes many of the same steps as those of the first method 600 in FIG. 6A, the second method 604 has the advantage that it employs the mathematical principle of composability to reduce the tw o independent and sequentially applied models 618, 626 of the first method 600 into a single model, thereby reducing the compounding effect of uncertainty in each of the model types. The second method 604 bypasses the intermediate prediction of traditional soil characteristics as the basis for predicting aspects of plant behavior and performance, thereby forgoing the uncertainty and inaccuracy introduced into the process due to the limitations on “ground truth” discussed with respect to the first model 618 in the first manner of predicting. This manner of predicting plant behavior or performance from soil sensor data, when embodied as a model that undergirds the functionality of a digital twin in agriculture, uses machine learning to directly link the soil sensordata to crop performance and behavior metrics, bypassing slow, costly, coarsely resolved laboratory soil analysis. Partial Least Squares Regression with leave-one-out cross-validation is one example of a machine learning technique that is effective at implementing such a model as one basis for a digital twin in agriculture.

[0201] In the second method 604, at step 632, a soil probe 634 is deployed. In some embodiments, the soil probe 634 comprises DSC probe 100. The soil probe 634 is deployed at one or more locations selected on the basis of spatial variation in topography, plant behavior or performance, surface geophysical indicators, or other indicators of spatial variability in soils. At step 636, depth-dependent raw sensor data 638 is acquired at each of the one or more locations as outputs from the soil probe 634. At step 640, a model 642 is developed to predict plant performance based on the depth-dependent raw sensor data. At step 644, plant performance is simulated based on the depth-dependent raw sensor data. In this nonlimiting example, the model 642 predicts the initial bud count on each tree in an almond crop from depth-dependent raw sensor data from the soil in which each tree is growing, as shown in graph 646.

[0202] FIG. 6C shows an example simulation 648 of plant performance based on the depthdependent raw sensor data. The simulation 648 shows the results of using a PLSR model to predict almond bud count based on depth-dependent raw sensor data obtained from a visible-NIR reflectance spectroscopy sensor in a soil probe. Measured initial bud count is on the x-axis and predicted initial bud count is on the y-axis. In this example, bias w as equal to 0. 14, RMSE was equal to 16, and R2was equal to 0.6.

[0203] In some embodiments, a digital twin of the soil can be related to a digital tw in of a crop in the soil using at least one mathematical model, as set forth in more detail below7. In some embodiments, while data from the sub-surface suite is used in creating the digital tw in of the soil, the overhead suit is used in creating the digital twin of the corresponding crop.

[0204] Overhead Suite

[0205] Beyond simply defining vertical profiles of soil properties and conditions, it is equally important to understand horizontal spatial variability and its effects on the movement and retention of water and nutrients, as well as on the growth of roots. Established geostatistical estimation techniques attempt to use observed anisotropy and observed spatial scales of variability and correlation to predict subsurface soil properties and their structure everywhere in a plot of interest that has not been sampled, based on what has been sampled. Often thesetechniques do not examine joint and conditional probabilities involving multiple sensed quantities in a spatial context.

[0206] In one embodiment, the overhead suite of sensors is used to construct the digital twin of the crop. In some embodiments, the digital twin of the crop includes digital twins of individual plants within the field, which comprise images obtained from imaging spectrometers (e.g, IR, multi-spectral, and / or hyperspectral), and visible light (i.e., electro-optic) sensors (e.g., cameras), and LiDAR for high-resolution surface topography.

[0207] Overhead sensor data produces high-accuracy, high-resolution maps of crop health over time that enable the greater framework of how to identify and quantify crop vigor, growth patterns, and stresses as well as other aspects of plant performance and / or behavior that are not always discernible to direct human observation or human interpretation of imagery. The combination of the hardware employed to acquire overhead sensor data (including camera, lensing, triggering systems, drone, etc.), the acquisition methodology (including flight patterns, times of flight, weather conditions, etc.), and the data processing (including rolling shutter correction, a Bayer pattern decomposition method, spatial registration and co-registration techniques, radiometric calibration and correction techniques, mosaicking and 3D model building techniques, choice and use of spectral bands, and the like) are referred to as the Digital Vegetation Signature (DVS). (US Patent 11,800,246 discloses methods, devices, and systems comprising aspects of the DVS).

[0208] The DVS produces image maps that are high in qualify in three critical respects: spectral resolution and accuracy, spatial resolution and accuracy, and temporal resolution and accuracy. The disclosed method, system, and instructions precisely, repeatably co-register images that have accurate, precise, spatially and temporally repeatable spectral information, at a fine spatial resolution, enabling discovery, monitoring, diagnostics, and quantitative economically linkable decision support.

[0209] The panchromatic or RGB sensor data is sufficient in lateral and longitudinal overlap to enable the construction of a high-fidelity 3D photogrammetric model of the aboveground structure of the crop. Every pixel in the RGB imagery as well as the multispectral (MS) imagery, derived MS image products, and TIR imagery can then be associated with an elevation from the 3D model due to the co-registration of all image products. Using computational filters that operate on the spatial location of pixels, for example, height above ground, the spectral content of the pixels, for example, whether they contain greenery, soil, fruit, or other factors, as well asthe brightness of individual pixels, for example, to determine whether or not they are directly sunlit and thus in aggregate for each plant can support the derivation of repeatable statistical measures of plant performance and / or behavior from time to time, can be used to isolate “information rich” pixels that are representative of various aspects of the crop’s health and performance, and moreover can each be associated with a specific individual plant thereby enabling the computation of statistical measures of each plant’s health and performance from consideration of multiple information rich pixels associated with the each plant.

[0210] From the multispectral data collected by the crop imaging sensors, various spectral indices can be computed. Spectral indices are mathematical combinations of different bands from multispectral or hyperspectral imagery. Numerous spectral indices are known in the art to correlate to specific aspects of plant health and performance and can help in assessing plant health, estimating biomass, detecting diseases, monitoring water stress, and more. These indices can be computed and used to characterize each plant by statistical summary of the index values from aggregates of pixels that represent each plant.

[0211] Some commonly known spectral indices that can be computed for each sufficiently lit multispectral pixel (also known as information rich pixels) on each plant include normalized difference vegetation index (NDVI), normalized red edge index (NDRE), enhanced vegetation index (EVI), soil adjusted vegetation index (SAVI), transformed SAVI (TSAVI), green NDVI (GNDVI), normalized difference water index (NDWI), triangular vegetation index (TVI), leaf chlorophyll content index (LCCI), canopy chlorophyll content index (CCCI), photochemical reflectance index (PRI), and normalized difference infrared index (NDII). Other indices can also be computed, including proprietary' indices. Some less common indices include vegetation vigor index (VVI), pigmentation index (PGI), and leaf pigmentation index (LPI). These indices are designed to enhance specific spectral signatures and to quantify various surface properties. Other embodiments incorporating other spectral indices are also possible.

[0212] Each spectral index is designed to elucidate a specific aspect of plant health and performance. NDVI is commonly used for vegetation monitoring. NDVI is intended to indicate the presence and condition of green vegetation. EVI is designed to improve sensitivity in high biomass regions and reduces atmospheric influences. EVI uses a blue spectral band in addition to red and NIR bands. SAVI is designed to minimize soil brightness influences by using a soilbrightness correction factor. GNDVI focuses on the green band and is useful for estimating Leaf Area Index (LAI) and chlorophyll content in crops. CCCI and LCCI are useful for assessing the chlorophyll content of the canopy and leaves, which can provide information on the nutritionalstatus of the plant. NDWI is designed to monitor water content in vegetation. Additional spectral indices can also be devised that are indicative of other aspects of the plant health or performance.

[0213] Additionally, the 3D model can be used to determine the vertically projected area of each plant’s canopy and the height distribution of the crown of each individual plant, thereby enabling computation of foliar area, canopy volume, and similar metrics of plant performance and / or behavior such as leaf area index (LAI). The expected yields of nut trees and other agricultural crops has been correlated to the amount of sunlight the plant’s canopy can intercept during the fruit production season. Similarly, yield potential in almond trees has been highly correlated to the bloom count that first appears on each tree early in the growing season. Bloom count is a metric that can be determined from RGB and / or multispectral imagery.

[0214] FIG. 7 shows an example method 700 of assessing and predicting plant health and performance from sensor data used to create a digital twin of vegetation. Vegetation may comprise a crop or individual plants of a crop. At step 704, in a preferred embodiment, remotesensing (RS) crop sensors are deployed. In one nonlimiting example, the RS crop sensors are mounted on a mobile aerial platform, such as a small unmanned aerial vehicle, also known as a drone. Deployed RS crop sensors can include visible panchromatic or (RGB) cameras, multispectral (MS) cameras, thermal infrared (TIR) cameras, LiDAR sensors, radar sensors, and / or other imaging, time of flight, and shape sensors in any combination.

[0215] At step 708. the RS crop sensors are flown in a flight pattern that enables the acquisition of images showing every plant in a crop. An example of a crop is an almond orchard. Another example of a crop is a wine grape vineyard. In embodiments where LiDAR is deployed, it will result in the direct generation of a 3D model of the plants comprising a crop. In other embodiments, other means of constructing a 3D model of a crop is through the application of photogrammetry to overlapping images of the plants in the crop. The overlapping images can be RGB images, panchromatic images, multi-spectral images, or other 2D image representations. If the 3D model of the crop will be built photogrammetrically, the flight pattern is laid out in such a manner as to enable the acquisition of overlapping images of each plant that can be used to photogrammetrically construct the 3D models of the plant canopy and the spaces between each plant, which may comprise bare earth or cover crop. MS cameras can acquire spectral reflectance data that can be used to build spectral indices of various aspects of plant health and performance, such as normalized difference vegetation index (NDVI) or normalized difference red edge index (NDRE), and TIR imaging sensors can be used to map variations in canopy temperature that are indicative of the comparative water use among plants. In some embodiments, RS images fromthe RS crop sensors are acquired and used to construct a 3D photogrammetric model and MS mosaic.

[0216] At step 712, the acquired LiDAR or overlapping image data is used to construct a 3D model of the crop and the intervening bare soil or cover crop, co-registered geospatially with MS imagery, TIR imagery, or other acquired imagery as described elsewhere in the present disclosure. In some embodiments, data is processed into a 3D model and pixel maps of RS spectral index products.

[0217] At step 716, each plant is identified and RS indicators of health and performance are summarized. The 3D model or co-registered imagery is used to delineate the geospatial bounds of each plant in the crop. One example method of using the 3D model is to fit a 2D surface to the ground elevation using digital filtering or heuristic rules that identify elevation minima and consider any part of the 3D model that exceeds some threshold elevation about the minima to be a plant. Other means of identifying and fitting a surface model to elevation minima may also be applied. Another example of identifying and delineating individual plants is on the basis of spectral or thermal information. Generally, bare soil or a cover crop will exhibit different spectral or temperature than that of plant canopy. 3D model information, spectral information, or thermal information can be used in any combination to identify and delineate individual plants in the crop. Also in step 716, the spectral reflectance or TIR characteristics of each plant can be summarized statistically. For instance, the mean, medium, minimum, maximum, standard deviation, or any other statistical measure of the NDRE of all pixels within the spatial bounds of an individually identified plant can be computed. In some embodiments, statistical descriptions of the pixels comprising each and every plant in the crop are computed. These spectral and thermal indicators are measures of different aspects of the health of each plant.

[0218] At optional step 720, one or more models are developed to simulate plant performance based on RS indicators. In some embodiments, the one or more models are statistical or machine learning models of plant behavior or performance based on the data summarized for each plant in the crop. For example, by correlating a measure of each plant's behavior or performance determined by remote sensing (RS) or non-remote sensing (non-RS) means as output of a machine learning (ML) model, with spectral, thermal, or 3D structural information in any combination about the plant or the soil as input to the ML model, a ML model can be trained to predict an aspect of plant behavior or performance from the RS data acquired and processed according to the steps described herein. More conventional statistical correlation models and other types of models can also be developed to predict other metrics of plant behavior orperformance from RS-derived information and soil information acquired or developed using the devices, systems, and methods described in the present disclosure.

[0219] Some nonlimiting examples of non-RS means of assessing a metric of plant behavior or performance that can be predicted by the ML or statistical model include bloom coverage and overlap, shoot or bud counts, and fruit or nut counts per plant, such as the total number of almonds, grapes (clusters), lemons, or avocados per plant. Other non-limiting examples include weight of harvest per plant, fruit or nut size distribution within each plant, and kernel or flesh recovery, such as almond kernels or avocado flesh weight as a percentage of total fruit weight. Other examples may include the sugar content of wine grapes or the phenolics content present in the skins of wine grapes. These measures allow for assessing yield potential, quality, and overall productivity7of each plant, and informing targeted management practices.

[0220] Non-limiting examples of RS means of assessing plant behavior or health can include, for example, canopy height, area, and volume as determined from 3D model data, vegetation vigor or chlorophyll content as determined from multi-spectral data, or water use efficiency as determined from differential TIR data.

[0221] At step 724, the one or more models created in step 720, which are based on a smaller subset of plants, are then used to predict the plant performance and behavior indicators which were not directly measured by remote sensing, on a larger number of plants across at least one management unit. This new data then becomes part of the digital twin for agriculture, informing further analysis and changes to management practices as described elsewhere in the present disclosure. In some embodiments, crop health and / or performance is simulated. The simulation could, for example, estimate expected yield and later compare it to realized actual yield via a scatterplot, as shown in FIG. 7.

[0222] The functional aspects of the digital twin of the crop or of individual plants is developed by correlating spatial and temporal variations in spectral indices and canopy geometric properties to spatial and temporal variations in soil conditions and in management practices such as applications of water, fertilizer, and other nutrients, pollination, pest and disease control actions, harvesting, pruning, trellising and training (in the case of wine grapes, for instance), and other mechanical operations. By establishing covariances between variations in growing environment and management practice and variations in plant performance and / or behavior indicators, and in more sophisticated embodiments, first principles models of causality or machine learning models of correlation, a digital twin has the ability to support simulation and predictive analytics.

[0223] The combination of fine spatial and spectral resolution and accuracy enables georegistration accuracy and repeatability that is essential to quantitatively tracking the spectral trajectory' through time of sub-crow n-sized spectral anomalies, which can be indicators of pest infestation or disease. Sub-crown-sized spectral anomalies are variations in DVS spectral indices or 3D structural characteristics that are smaller in scale than an individual plant, and thus can be detected and characterized within the spatial bounds of an individual plant’s canopy or attributed to an individual plant. This capability enables characterizing and tracking the performance and behavior of each plant individually through time using the DVS.

[0224] Analogous to the well-established tradeoff between sensitivity and selectivity in laboratory -grade analytical chemistry, the disclosed provision of high-resolution overhead sensor data spatially, spectrally, and temporally expands the trade space of sensitivity versus selectivity7, enabling increases in both sensitivity and selectivity of what can be detected and quantified (instantaneously and spatiotemporally) in plant performance and / or behavior and in the relation of each to the grow ing environment.

[0225] Correlating spatially structured, persistent, or temporally progressive spectral variation in a canopy over time with human observations of known plant stresses (e.g.. pests, diseases, nutrient and water deficiencies), and adjusting the sensitivity of the algorithm that delineates and bounds the identified anomalous spatial structures, provides useful quantitation of the riskreward function that informs decision making.

[0226] Similar to adjusting the sensitivity along a receiver operating characteristic (ROC) curve for each pixel of overhead imagery, expansion or contraction of the spatial area of concern as a function of confidence in the classification can be effected. This can also be interpreted as producing contours of likely severity. Plant material at the spatial hearts, for example, close to the centroid, of these spatially structured, for example, contoured, anomalies can be inspected or sampled and analyzed to find corroborating evidence and identification of a given cause of the spectral anomaly, such as a pest, disease, soil-related toxicity7, or other cause. Thus, specific spectral signatures can be correlated to specific distresses. Specific spectral signatures may change with the severity of the given distress, so the temporal evolution of a spectrally evident distress can also be studied and known for future use in crop diagnostic monitoring using the present disclosure.

[0227] In other words, the grower can obtain a contoured high-spatial-resolution probability map of the areas in which a specific distress is indicated by the multi-spectral imagery. With thiscapability to image a management unit with temporally consistent spectral accuracy and pixellevel spatial co-regi strati on over time, the temporal derivative of the probability of a given distress can also be mapped to show how fast the distress is progressing in local severity and / or spreading spatially.

[0228] In another aspect of the disclosure, the aboveground and belowground sensor data is used in combination and analyzed in real-time or near-real-time for the purposes of guiding the placement of additional observations. For example, the above-ground imagery is used in combination in the context of an artificial intelligence and machine-learning context for the purposes of determining where subsequent underground observations should be made. Similarly, sensor data obtained from below the surface can be used to guide ‘forensic’ observations via acquisition of overhead imagery.

[0229] An aerial survey consists of a satellite or aircraft, which can be manned or un-manned, and a sensor such as a camera, a spectrometer, or a LiDAR system. The aerial survey may be performed one or more times, having different sensors, or the same sensors from the same or different aircraft or satellite. To make use of an aerial survey, it must be adequately georeferenced, and sufficient distortion must be removed from the resulting data so that it can be accurately aligned with other data sets. Furthermore, to make use of multiple aerial surveys that measure the same characteristic of soil or vegetation, the sensor data must be calibrated spectrally.

[0230] In one embodiment, an aerial survey is conducted using an unmanned aerial vehicle (UAV) provisioned with a positioning system (e.g., a GPS and IMU), and one or more cameras are mounted to the UAV using a gimble. The UAV may also be referred to as an aerial drone or simply a drone. The cameras are configured on the gimble to point downward and take pictures once per second. The gimble is configured to record the position of the camera in three axes with respect to the drone, and the IMU and GPS is configured to record the position of the drone in three axes with respect to the ground at each imaging event. The system is configured to store these geometries for later use in processing the imagery.

[0231] In some embodiments, the camera imaging sensor is a 3-band Complementary Metal- Oxide Semiconductor (CMOS) array that is sensitive to visible and near infrared light. For example, in an embodiment, the camera imaging sensor is an RGB camera having one or more lenses with one or more filters. The ground is illuminated by the sun and reflects light that is observed by the camera. Light reflected from the ground passes through the lenses and filters andilluminates the sensor array based on the geometry of the lenses and filters. It is common for cameras to have three sensor elements sensitive to particular ranges of spectra and one lens in most embodiments, resulting in three image data sets wi th each image data set corresponding to the sensitivity of each sensor. However, it is also possible to use more than one lens and filter combination configured to project an image of the same ground area over discrete areas of the imaging sensor, resulting in a number of datasets equal to the number of lenses times the number of sensor elements, configured in that manner. (See U.S. Patent 11,800,246 “Systems and Methods for Multispectral Landscape Mapping’" issued October 24, 2023, to Paris et al). It should be noted that the present invention is applicable to at least both of these embodiments. The configuration could also include a second camera that is sensitive to thermal radiation.

[0232] The data sets from a single camera sensor array are generally co-registered to each other. However, sometimes they are not. In addition, if multiple cameras are mounted on the same aerial platform they may record on separate sensors with separate lens geometries. If these geometries are known, then the images may be aligned according to their physical geometries to co-register the images. In the case where the same ground area is imaged multiple times on different parts to the same sensor, additional work is needed to align the data sets, which may include the alignment according to optical geometries, but may also require compensation for the effects of “rolling shutter”. It is common for CMOS sensors to use a rolling shutter where each sensor element is read sequentially starting at the top of the sensor array and ending at the bottom of the sensor array at a later time. While the amount of time required to read the entire sensor array is a fraction of a second, it can be long enough to cause distortion in the image as the sensor is moved with respect to the ground during image acquisition. This distortion must be removed before attempting to perform an orthorectification process, which corrects only for certain other types of distortions due to the camera altitude position and orientation. One method of correcting for rolling shutter is to calculate the displacement of pixels representing the same place on the ground by calculating the correlation between two data arrays. The displacement may be constant or it may vary across the image (e.g., in the direction of travel). The calculated displacement is used to correct the imagery.

[0233] It is appreciated in the art that the Earth’s surface is curved and that the surface varies in relative elevation from a datum that describes that curved surface. The reference system of a map is a flat two-dimensional grid, which can be used to visually or mathematically compare two or more data sets representing positions on the Earth’s surface. Maps of the Earth’s surface are a projection of a three-dimensional surface onto a two dimensional plane. In order to use aerial survey data, the sensor data must be corrected for distortions of the three-dimensional surfaceimaged as well as the camera position, orientation, altitude, and the camera's optical properties (e.g., the pixel size of the imaging sensor, commonly called the pitch, and the focal length of the sensor or sensor focusing apparatus). This process is commonly called orthorectification.

[0234] Orthorectification removes the effects of the terrain and projects the sensor data onto a two-dimensional plane, removing also the distortion of the lens and orientation, and correcting for the differing in altitude between multiple image frames, resulting in images that are correctly aligned to the other data in a map reference system and therefore usable for comparison with other data. Orthorectification is typically performed by collecting multiple overlapping images, finding the same ground location in each of the overlapping images, and then using a 3D correlation between the overlapping points and sensor metadata to calculate an aerial triangulation. The process can be repeated to optimize the solution across multiple images, until the residual errors of the solution are acceptable. In some cases, the area imaged is small enough to fit inside a single image, in which case full orthorectification is often forgone in favor of simply projecting the single image onto a flat plane aligned with the map reference system. While more computationally efficient, forgoing orthorectification does not correct for distortions in the sensor data caused by changes in surface elevation.

[0235] The condition of agricultural crops changes through time. Some crops are annual, and some are perennial. Annual crops start out as a seed at the start of the growing season and grow through multiple phenological stages throughout the growing season, usually increasing in vegetative maturity, until they are harvested at the end of the season. Perennial crops continue to grow in place between seasons. Some perennial crops are dormant between seasons, and some continue to produce year-round. Crops may be harvested for any number of their physiological components (e.g., vegetation, fruit, seeds, tubers, roots). These components may vary' in size and color throughout the growing cycle as the plant partitions its growth between its various components. The plant can variously partition its growth based on environmental conditions and management practices. Tn addition, the plant can exhibit changes in color and size or shape, or presence or absence of various physiological components in response to environmental conditions and management practices.

[0236] It is common in the current art to collect monthly, weekly, or more frequent images over the same area of an agricultural field. However, to facilitate the use of machine learning to use that imagery' for change detection and to model the growth of individual plants, the imagery' must be adequately georeferenced and spectrally calibrated. Shifts in georeferencing of two different grids could result in apparent change, where there is none, or could obfuscate or cancelthe appearance of change where it exists. In theory, the georeferencing error should be less than the error in the ground spacing distance of the analyzed imagery. Similarly, when the imagery is repeated, differences in illumination, instrument orientation, altitude, direction of travel, and especially variations between sensors can interfere with the abi 1 i ty to analyze change quantitatively.

[0237] The present disclosure enables the appropriate spatial and spectral calibration of the sensor data. In some embodiments, the methods include masking of information-rich pixels, using information-rich pixels to define spectral control points defined by end-point conditions observed on the ground (e.g., bright vs. dark soil, soil vs. vegetation, green vegetation vs. yellow vegetation), the transformation of the multi-dimensional data from one feature space into a new multi-dimensional feature space, in which at least tw o dimensions are orthogonal and then scaling the orthogonal dimensions between spectral control points, resulting in calibrated vegetation indices that are less noisy than other known methods. Additional crop-specific parameters may also be used to adjust the scaling to particular crops. These indices may also be combined with thermal data in further statistical analysis to identify potential causes of plant stress and yield development. The thermal data may need to be scaled by the size of the canopy. The canopy size can be estimated from vegetative indices and the resulting surface model from the orthorectification process.One product of an aerial survey is a ground elevation model, and another product is a surface model. The surface model represents the ground in some places, low vegetation in others, and trees and building in yet other locations. The surface model can be used in creating the information rich pixel mask, and the information rich pixel mask can be used as above to obtain a vegetation index. The land surface can be segmented in the ground, vegetation, trees, and buildings through a statistical algorithm that incorporates vegetation indices and surface elevations. The algorithm may also take into account textural and shape features derived from the data. The land surface segmentation may be classified by the same and the land classification may be used for further masks and calibrations. The resulting data can be used to develop management units or to identify individual plants and to associate attributes of individual plant performance and behavior to the plant location in a plant database.

[0238] An additional aspect of this disclosure involves machine learning (ML) and artificial intelligence (Al) to exploit spatial pattern recognition in the joint spatial and multi-modal correlation of all sensor-derived measurements to improve spatial interpolation and extrapolation of measured and inferred parameters. For instance, small scale topographic features inconjunction with (or independent of) spectral reflectance characteristics of surface soils may be strong indicators of the distribution of water holding capacity or nutrients within a field.

[0239] Additionally, the temporal component of variability from time-to-time, for example, day-to-day. week-to-week. and / or month-to-month, in crop canopy structure, spectral reflectance, and / or thermal response to weather and irrigation conditions or various management actions are also aspects of plant performance and / or behavior that may be spatially differentiated among plants and in relation to spatial patterns of variation in soil and topographic conditions.

[0240] Conventional statistical methods as well as machine learning can be used to correlate sensor data as inputs for the purpose of characterizing attributes of the soil and plant. For example, the ability of the soil to hold water in an area is a function of the thickness of each layer of soil combined with the contents of each layer. One sensor may estimate density, another friction, another color, another moisture. While any one of these measurements would not reliably indicate how much water the soil can hold, in combination they can have useful predictive value. These data are one component of training data sets that also include the results of conventional laboratory and field analyses of soil samples as machine learning prediction or estimation targets obtained alongside DSC sensor profiles and to build accuracy and support continued machine learning over time.

[0241] This disclosure describes technologies, methods, and systems for obtaining sensor data for a physical site and inferring, deriving, or predicting characteristics for the physical site, including characteristics of present and future plants grown at the site. These technologies generally involve obtaining, at different physical locations in a region, sensor data that may comprise data from proximal soil sensing devices and data from overhead imaging devices.

[0242] The sensor data includes raw signals and measurements of characteristics of the physical locations using contact and / or non-contact sensors, including surface-level characteristics, sub-surface level characteristics, remote data from drones, planes or satellites and / or other characteristics ubiquitous to the region, such as weather conditions. The combination of spatiotemporal data can be aggregated into “data cores” incorporating imagery collected from satellites, planes, and / or drones, with subsurface sensor data including imagery through the root zone. In some embodiments, data cores include measured, inferred, derived, and / or predicted characteristics in addition to imagery. The sensor data is provided to one or more machine learning models that are configured to receive the sensor data and to predict characteristics of the region at the physical locations, including characteristics that are notrepresented in the sensor data measured at the physical site, and characteristics that are measured for at some, but not all, locations at the physical site.

[0243] Inferred, derived, or predicted characteristics can include characteristics that were measured by different sensors, for example, sensors for classifying soil texture, but "sharpened" to provide more accurate measurements by combining the separate measurements. The characteristics can also include latent characteristics that are not directly measured by the different sensors, but are inferred, derived, or predicted by the machine learning models based on learned correlations between other, directly measured, characteristics, for example, a maximum water holding capacity for a growing medium predicted given measured characteristics for moisture content and the quantification of different layers in the growing medium. The growing medium may be soil, or it may be another ty pe of material. The described techniques for site characterization can result in identifying numerous and complex interactions, relationships, or correlations between directly measured characteristics of a physical site, which can further result in new or refined, inferred, derived, or predicted characteristics.

[0244] The inferred, derived, or predicted characteristics and additional external constraints, such as socio-economic factors, conditions, and data, or a defined objective for a physical site, such as an expected plant yield at the end of a season, can be used to generate recommendations for "best practices” and dynamic management methods and systems in maintaining and managing a physical site, so as to satisfy the external constraints or achieve the defined objective.

[0245] Recommendations can be automatically translated into a set of instructions for managing and controlling agronomic or forestry management equipment, such as irrigation systems, fertilization systems including but not limited to variable rate application machinery’, and pest control systems, to result in changes of a physical site’s management to a level of precision previously considered intractable, or can be recommendations related to manually managed activity' such as when to harvest. Rather than manage a physical site at a block level, i.e., by sweeping management decisions that generalize an inherently heterogeneous block, the provided site characterization and predicted characteristics can offer a granularity for a recommendation system to generate decisions or recommendations for managing a smaller portion of a region, such as row of trees in an orchard, grouping of vines in a vineyard, clusters of annual plants in a field, or individual trees, vines, plants, or animals in a physical site, or even to a plant organ level such as leaves, shoots, roots, or fruit.

[0246] Additionally, the sensor data collected can be used for more accurate predictions about the current and future state of a physical site. The predictions can include latent characteristics of the physical site that are not directly measured, but are instead inferred, derived, or predicted based on found interactions, relationships, or correlations of measured characteristics for the physical site at the management unit, plant, animal, and even individual fruit level.

[0247] The predictions can be leveraged for more accurate operation of automated machinery' for the planting, maintaining, and harvesting of plants or animals in a given region. For example, the predicted characteristics of growing medium at the physical site, for example, a predicted water-holding capacity for the growing medium, can be used as input for an irrigation system that modifies the rate of irrigation of the site in response to the predicted characteristics. Further, the predicted characteristics can correspond to management units smaller than a typical block or field in which physical sites have been typically organized. As a result, operation of automated machinery, for example, an irrigation system in the previous example, can vary dramatically across the physical site, which can result in improved crop yield relative to operation across the entire site. The predictions can also drive improvements to a site characterization system, by identifying additional locations for obtaining sensor data that can be used to train models that can further improve the characterization and prediction of site characteristics.

[0248] A digital soil core, also referred to as a DSC in the present disclosure, can be generated through sensor data collected using probes that, as described above in this disclosure, are configured with a plurality of different sensor units that take a sequence of measurements starting from the surface level of a growing medium, up to a terminal depth level. Digital soil core sensor data can include the condition, behavior, interactions, and emergent properties of: light and other forms of electromagnetic radiation; molecular elements; molecules and assemblages of molecules; organismal components including cells and assemblages of cells into microbial, plant, or animal organs; individual organisms; population of organisms; species communities and ecosystems; biogeochemical cycles such as the water, nitrogen, carbon, phosphorus, energy , and other cycles; weather and climate; and physical and mechanical conditions including soil structure and site topography . Different measurements can be taken by different sensor units.

[0249] A sensor profile is a heterogeneous or homogeneous collection of data acquired using one or more sensors that comprise a plurality of measurement values along one or more axes of differentiation. Examples of axes of differentiation include sensor modalities, ranges of sensitivity including frequency and wavelength ranges, and spatial or temporal variability. Forexample, the measurement values returned by a collection of soil sensors that includes tip stress, sleeve friction, electrical conductivity, and moisture content from a first depth level at a coordinate may constitute a sensor profile, as may also the values measured by a single sensor from a plurality of depth levels at a coordinate. A still more complex example of a sensor profile is the values measured by a collection of sensors from a plurality of depths at a coordinate, which may also comprise dimensionality along one or more spectral axes.

[0250] Likewise, the wavelength-dependent reflectance spectrum measured from a portion of a plant is another example of a sensor profile. The reflectance spectrum may represent the sensor data from one pixel that together with the other pixels in a multi-spectral image comprise a two- dimensional sensor profile of spatially differentiated spectra in a scene. As a further non-limiting example, a time series of multispectral images acquired from and geo-referenced to coordinates in the same management unit is a sensor profile of the temporal trajectory of spatial variation in the spectral reflectance of plants in the management unit.

[0251] The resulting “sensor profile” not only represents a rich vector of features for measured characteristics to be processed by a machine learning model, but the machine learning model can be trained to produce more accurate predictions by using the temporal relationship between each feature. The temporal relationship refers to the fact that the measurements by the different sensor units are taken in close temporal proximity to one another at the measured location, which can yield more accurate predictions by a model over a processed profile that includes different measurements taken minutes, hours, or days apart from each other.

[0252] The latter case often arises when sampling is performed ex situ such as when growing medium, plants, plant parts, and / or animals are removed from a site and analyzed at a different location versus in situ. In the example of removing growing medium as part of measurement, the removal can degrade the accuracy of the measurements taken or make some measurements outright impossible, and by consequence, the accuracy of any predictions taken from measurements obtained in that matter. By techniques described in this disclosure, not only are sensor profiles of measured locations within a physical site enriched by quick succession of in situ measurements, but measurements taken in this manner can reduce or eliminate measurement inaccuracy by minimally disturbing the measured characteristics of the physical location.

[0253] Root Cause Analysis Model

[0254] In some embodiments, a Root Cause Analysis model (RCA) relates plant performance to genetic, environmental, management, and / or socio-economic factors. The RCA is appliedacross at least one management unit of the same crop type. FIG. 8 shows an example flow diagram of a method 800 of applying the RCA using the limiting factor analysis described below, where the RCA is used for decision support with digital twins of vegetation and the soil the vegetation is in. At step 802, data from geophysical sensors on penetrometers or near surface devices are collected across a set of soil profiles.

[0255] At step 804, soil property data is obtained from another reference source, such as a soil laboratory' analysis of physical soil samples or another calibrated sensor or field measurement (e.g., infiltrometer).

[0256] At step 806, a model is created between the soil sensor data collected at box 802 and soil properties collected at step 804.

[0257] At step 810, a satellite, or manned or unmanned arial vehicle with passive or active optical sensors sensitive to various ranges of the electromagnetic spectrum collects sensor readings on the canopy, fruit, and / or structure of a set of plants across a set of agricultural or forestry management units.

[0258] At step 812, a set of reference observations (e.g., bloom count, bud count, spur count) related to productivity or another agricultural outcome, are made at one or more predetermined growth stages. In some embodiments, the reference observations are used to estimate the yield potential at the end of the growing seasons. In other embodiments, the set of reference observations are used to estimate potential outcomes such as fruit size or quality.

[0259] At step 814, a model is created between the sensor data collected at step 810 and the reference observations collected at step 812.

[0260] At step 860, soil sensor data similar to the data collected at step 802 are collected across one or more agricultural or forestry management units.

[0261] At step 808, the model created at step 806 is used with the sensor data collected at step 860 to create soil property predictions for profiles and a 3D model of soil properties across the management units at step 860. In some non-limiting embodiments, the 3D model may be summarized to horizons of varying thicknesses and depths. In other non-limiting embodiments, the 3D model may be summarized to root zones of varying thicknesses.

[0262] At step 862, vegetation sensor data similar to what is collected at step 810 is collected across the same agricultural or forestry management units as at step 860.

[0263] At step 816. the model created at step 812 is used with the sensor data collected at step 862 to create estimates of vegetative outcomes, including but not limited to yield and yield quality, for all of the plants or sets of plants across the management units at step 860. It should be considered that steps 862 and 816 are independent from steps 860 and steps 808. However, there could be variations where results of step 816 inform the collection at step 860 or vice versa.

[0264] At step 818, the limiting factor analysis of the RCA then relates the digital productivity target and to other GEMS factors, such as to each environmental and management factor for the same genetics, forming a multidimensional matrix. For each factor, the maximum, or statistical threshold of productivity is calculated for the range of each factor. In two dimensions, the relationship between each factor and productivity can be shown in scatter plots, and the productivity potential for each factor at each step in its range can be shown as sets of points. In FIG. 9, an example scatter plot 900 is shown with sets of points 902 (only one labeled to avoid clutter). A bootstrapped confidence interval is calculated at each step in the range of each of a plurality of factors 904 (only one labelled to avoid clutter). For instance, the 95th percentile of productivity can be calculated at a 95% confidence interval. The confidence interval can also be displayed on the same plot as upper and lower bounds around the statistical threshold.

[0265] Each step in each factor is then categorized as sufficient, deficient, or excess, based on the difference between lower bounds of the confidence interval of the step with the maximum productivity potential and the upper bounds of the confidence interval for that step. If the difference is negative, then a value for that factor in that range is considered sufficient. If the difference is positive, then the value for that factor in that range is considered either deficient or excess, depending on whether that step is less than or greater than the step with the maximum productivity potential for that factor. For consistency and expediency, if a step is between the first step considered sufficient and the last step considered sufficient for that factor, that step is also considered sufficient for that factor. For excess and deficient steps, the positive difference calculated from the confidence interval bounds above is stored as the management opportunity for each factor. The management opportunity for sufficient steps for each factor is set to zero. Deficient and excess critical levels are defined as the lower and upper ends of the sufficient range for each factor. The critical levels can be represented as vertical lines on a scatter plot. In the example scatter plot 900 shown in FIG. 9, the scatter plot has a deficient critical level 906 and an excess critical level 908.

[0266] Turning back to FIG. 8. at step 834 and step 836. the deficient critical level 906 fromFIG. 9 can be used as the target soil test level in variable rate prescription calculations for soilamendments. Likewise, at step 834 and step 836, the excess critical level 908 can be used in determining leaching recommendations. In another example of step 834 and step 836, both critical levels may be used in adjusting management practices such as irrigation applications.

[0267] Other non-limiting variations of RCA step 818 exist where different calculations are used to determine the envelope or frontier of productivity potential given the multidimensional matrix of productivity and factors, including, by way of example, using a kernel density estimator on the matrix for each factor and contouring at a density threshold, then either finding the maximum value of the contour in each step for each factor, or estimating a function that describes a continuous production frontier and then finding the corresponding value for each factor value of each plant. Other non-limiting examples of frontier finding algorithms could include Variable Return to Scale Data Envelope Analysis, Stochastic Frontier Analysis, or Stochastic Nonparametric Envelope of Data (StoNED). In other embodiments, at step 818 any other type of envelope and / or frontier analyses may be applied. After defining the production frontier, the rest of the analysis would proceed in a similar manner.

[0268] At this point it is instructive to again consider the plot of productivity versus each factor in FIG. 9. The plants that plot within the range of the confidence interval for each step are considered to be on a frontier 910. The plants that are on the frontier are considered to be 100% efficient for that factor and are limited by that factor. Plants that are below the confidence interval for their step for that factor are not limited by that factor and are instead limited by another factor, such as the plant represented at point 912.

[0269] Once the relationship between each factor and productivity- is established, the productivity- potential across all factors is calculated using a limiting factor approach, also referred to as a limiting factor analysis. In the particular example of the RCA, productivity potential (Management Potential Productivity) for each plant is calculated by finding the minimum productivity potential in the matrix of productivity potentials per factor for that plant. This enables the calculation of various ty pes of productivity limits and gaps (called opportunities) through the grouping of factors and determining the limits in a similar fashion to how the Management Potential Productivity is calculated for example above. In particular, a Genetic Potential Productivity can be calculated as the maximum production of a cultivar and rootstock regardless of other environmental or management-related conditions. Ranch and Field Potential Productivities can then be calculated by Genetic groups as above without regard to the other factors, but limited to their geographic or management groups. Then. Site Limited Productivity can be calculated by finding the minimum productivity- potential for each plant for agroup of factors that are considered to be intrinsic to the plant's location and cannot be changed through management. Then, a Management Limited Productivity can be calculated by finding the minimum productivity potential for each plant for groups of factors that are considered to be manageable. The Management Limited Productivity can be divided into different groups by further subdividing the manageable factors into smaller groups such as, for non-limiting examples: Nutrient Management, Water Management, Tillage Management, Pest Management, or Soil Health Management. For each group, a management limited opportunity can be calculated related to that group.

[0270] Once the limits to production based on various factors have been established, various productivity gaps or opportunities and efficiencies can be calculated for each plant. By way of non-limiting examples, a Management Opportunity can be calculated by subtracting the Management Limited Productivity from the Site Limited Productivity. Management Opportunity components can be calculated in a similar manner based on the groups of manageable factors. Another example of productivity gap can exist between the Management Limited Productivity and actual current productivity of each plant. This gap represents the effects of factors that were not included in the model and is called the Scouting Opportunity because it can be used to optimize a farm manager's scouting operations or suggest the need for additional measurements and analysis.

[0271] It is again instructive to now consider a two-dimensional plot of Management Limited Productivity vs. actual cunent productivity. In FIG. 10, an example plot 1000 of Management Limited Productivity vs. actual current productivity is shown. If all of the factors that limit production were included in the analysis, then all of the points in the plot would fall along a line that approximates the 1 : 1 line 1002 and every7plant would have fallen on at least one frontier in the plots for each factor described above. However, in practice, any number of plants will fall below the line 1002 in the above plot, such as the plant represented at point 1004, and not fall on any of the frontiers defined for individual factors. Various efficiencies can be calculated as the ratio between the terms described above. For instance, by way of non-limiting examples, Genetic efficiency is Current Actual Productivity divided by Genetic Potential Outcome, Field Efficiency is Current Actual Productivity divided by Field Potential Outcome, and Site Efficiency is Current Actual Productivity divided by Site Potential Outcome. Management Efficiency is the Management Limited Productivity divided by the Site Limited Productivity. The Analysis Efficiency is the Current Actual Productivity divided by the Management Limited Productivity. These productivity levels and opportunity gaps can be summed and divided by area (such as in hectares or acres) to provide productivity and opportunity estimates inproductivity per unit area. These data can then be summarized and shown in a stacked graph 1 100 that shows the components adding up the Site Limited Productivity. Collectively, this information is used to summarize field performance and allow for comparison between fields and ranches. For example, FIG. 11 shows an example stacked graph 1100 that shows a plurality of components adding up a Site Limited Productivity.

[0272] Referring back to FIG. 8, other variations of this limiting factor analysis at step 818 exist where other outcomes are optimized on values other than productivity. In one non-limiting instance, the outcome value could be a soil health index. In another non-limiting instance, yield pricing is incorporated in the model and the outcome value is total economic income, the value of which can be composed by adding multiple sources of income such as productivity and quality, or productivity and sustainability incentives. In another non-limiting variation on the previous instance, the outcome variable is a relative variable with no direct correlation to productivity or monetary value, but is constructed as the weighted output of a combination of other factors. In a further non-limiting instance, the outcome value can be an undesirable outcome such as the prevalence of a disease such as hull rot in an almond crop. In the case of an undesirable outcome that should be minimized instead of maximized, the lower frontier in the plot would be found instead of the upper frontier, and the analysis would proceed to find the maximum of all of the minima of the factors. In fact, there could be many variations on the analysis that use the same key concepts described herein.

[0273] Another non-limiting application of the RCA step 818 is to use its outputs to summarize the Management Opportunity for each factor across a ranch, field, zone, or other management unit and then rank the factors based on their total Management Opportunity (or Management Opportunity per unit area).

[0274] A further non-limiting application of the outputs of the RCA step 818 is to summarize the adjustment of each factor required to go from the current state to the optimal state. By examining the distribution of the adjustments required, acute (strong opportunity' across a small portion of a management unit) and chronic (varying opportunity across a large portion of a management unit) problems can be identified.

[0275] Still another non-limiting use of the outputs of the RCA step 818 is to analyze the relationship between soil factors at different proposed rooting depths and plant performance to determine the most likely effective rootzone depth for at least one management unit.

[0276] Still another non-limiting use of the outputs of the RCA step 818 described above is to find the optimal soil test level for variable rate soil amendments. Although it is common for applicators to use a removal rate multiplied by expected production or a removal rate multiplied by expected production plus the difference between the recommended soil test levels and the cunent soil test levels, the recommended soil test levels typically are derived from an outside source such as a university extension report, a soil laboratory, or a consulting agronomist, rather than from a digital twin, and may not be applicable to site specific conditions. The RCA provides a means of determining the recommended soil test levels from the data from the ranch and fields to which it is being applied and unlike many other sources can provide targets for all factors included in the analysis along with details of the analysis to allow a digital agronomist (e.g., one who works with the digital twin to make recommendations) to make informed decisions. Some other advantages of the RCA over current practices include, but are not limited to, the ability to evaluate factors of production on each plant or plant group basis, applications on each plant or plant group basis, analysis of soil factor by depth, and the ability to find targets for all of the factors from the digital twin.

[0277] Soil nutrient removal can be calculated in step 822 by multiplying soil removal rates from step 824 by productivity estimation from step 818. Similarly, water removal rates (e.g., evapotranspiration and drainage) can be determined from weather and climate data from step 842 and genetic rate parameters (e.g., crop coefficients) and soil properties (e.g., soil water holding capacity' and hydraulic conductivity ) from step 808.

[0278] Then, in step 826, as a non-limiting example of a nutrient model, a nutrient mass balance model can be run that incorporates the output of step 820 and step 822 with, optionally, as-applied data from step 830. The as-applied data may comprise as-applied nutrient additions from soil amendments, as-applied water additions from irrigation, or any other type of as-applied data.

[0279] Similarly, in step 828, as a non-limiting example of a water model, a water balance model can be run that incorporates the outputs of step 820 and step 822 along with, optionally, as-applied water additions from irrigation collected at step 830 and rainfall collected from step 832.

[0280] In other variations of method 800, steps 826 and 828 can be run prior to step 818 and results can be included as factors in step 818.

[0281] One particular example of how an application rate is calculated using the RCA in step 834 is to equate Application Rate to the Removal rate of step 822 plus the difference between the Deficient Critical Level determined by RCA in step 818 and the DSC determined value of the relevant factor in step 808. Application rates include, but are not limited to, irrigation rates and soil amendment rates. Most soil amendments are additive in nature and the Deficient Critical level is targeted because it is the level at which adding additional inputs is unlikely to affect production. The Excess Critical level would be used in leaching calculations for irrigation in step 836 or calculations for amendments in step 834 that encourage conversion of one excessive soil property into another. One such non-limiting example of that process is the conversion of excess lime into gypsum using an acidifying sulfur-based fertilizer.

[0282] Another variation of how the RCA can be run to determine application rates in steps 834 and 836 is to create a total available nutrient factor equal to the DSC Soil Test for that nutrient plus the amount of that nutrient applied during the growing season minus the removal for that nutrient for the growing season. These additive factors are then evaluated in the RCA using the procedure outlined above. In this variation, the application rate is then determined by taking the difference between the Optimal Nutrient Value and adding the expected Removal. This allows for a more closed-loop approach to using the maximum amount of information specific to the management unit and also facilitates the use of the RCA across multiple management units in the ranch that may be subjected to different management practices.

[0283] Yet another variation on the method in the previous paragraph is to include application timing in the RCA. This is achieved by modeling the removal of managed nutrients and the associated fertilizer applications through time. In some variations, the nutrient use and application can be modeled by week or by month. In other variations, this is done by phenological stage. In one particular example, the nitrogen removal is calculated per growth stage in steps 826 and 828 and the nitrogen applications applied in the previous growing season in step 830 are added per grow th stage. Total nitrogen values for each growth stage are included in step 818 as described above and a critical level for total nitrogen is determined per growth stage in step 820. Then, an application rate per growth stage can be calculated as described in the previous paragraph in step 834.

[0284] In fact, there could be many variations on how factors are combined in the analysis at steps 834 and 836 and used to produce variable rate or optimized single rate prescriptions.

[0285] In addition to running a single pass optimization of application rates using a single primary nutrient for each scheduled nutrient application as one way to apply the RCA, other variations of applying the RCA to the calculation of nutrient and water prescriptions can include, but are not limited to, single and multiple pass optimizations using techniques such as linear programming, non-linear programing, or other generalized algebraic models used in the art for single or multi-target optimizations.

[0286] It should be appreciated that the backward-looking models of steps 826 and 828 can be run in any order that the outputs can be used in one or the other to enhance the model for the purposes of proposing nutrient or water prescriptions in steps 834 and 836. By way of nonlimiting examples, more calcium-bearing materials may need to be applied based on sodium concentrations in the irrigation water, or more lime may need to be applied when the fertilizerirrigation solution is acidic, or more irrigation water may need to be applied when the soluble salts in the soil are elevated and the hydraulic conductivity of the soil is sufficient to support leaching.

[0287] The application rates of steps 834 and 836 can be fed into forw ard looking nutrient and water models in steps 838 and 840 to estimate effects of the applications, including but not limited to the calculation of over application and under application of nutrients and water, estimation of greenhouse gas emissions, and expected productivity limits given changes to limiting factors under management per the analysis in step 818. These estimated impacts and outcomes can be used to simulate, predict, and optimize compliance with regulatory and other compliance requirements, meeting the standards, requirements, and / or information needs for voluntary market-based assessment, verification, and certification programs, modeling financial metrics associated with the deployment of capital, and other socio-economic factors and conditions.

[0288] The application rates of steps 834 and 836 and outcomes of steps 838 and 840 can be grouped in any number of ways. By way of non-limiting examples, they may be grouped into rates of fixed increments at a number of steps from the minimum to maximum rate, or grouped by management units, or grouped by scenario. It should be appreciated that if a number of rates are summarized into a group rate, including but not limited to the scenario where there is only one group, that summarized rate for the group could be calculated from various statistics, including but not limited to the mean, median, or percentile of any rank for each group.

[0289] At step 846. for example, the return on investment (ROI) is calculated using the grouped application results at step 844, yield prices at step 848, and regulatory / voluntary marketbased program / ecosystem services value at step 850. As a general concept, ROI is equal to the sum of all of the valuable effects, including but not limited to yield minus the sum of all of the costs, including but not limited to the costs of the proposed prescriptions. In other embodiment, financial metrics in addition to or as alternatives to ROI are used. Other non-limiting variations of step 848 exist where the quality or size of the individual items of the crop are factored into the value. In other non-limiting variations, aspects of production other than yield can be valued and incorporated in the ROI and / or other-financial-metrics calculation. The process at step 846 may be iterated through pathway 852 until an acceptable or optimal ROI and / or other financial metric targets are achieved.

[0290] At step 854, the grouped set of applications can be exported to a computer readable file or digital data structure for transmission and storage in another format, such as. but not limited to, a tabular file format (e g., delimited text file, or spreadsheet, or database table) or a spatial representation linked to a tabular data format (e.g. a Shapefile, geojson, or kml). At minimum, the file would contain a spatial representation and an application rate per unit area. Other variations may include identifiers for each area, and identifiers of the product to be applied, area covered by each spatial representation, or total applications. Some variations of the file may be loaded into a machine controller and applied to an agricultural field via pathway 856.

[0291] After application, the effects on the soil and vegetation can be measured again in steps 860 and 862 after a period of time, which in some non-limiting variations, may occur approximately one year after the initial collection of soil sensor data. Other variations could include monitoring at some or all grow th stages in subsequent iterations. It should be appreciated that it may not be necessary to collect new training data in subsequent years and that soil data collection in step 860 may be done at fewer locations that in the first year. It should also be appreciated that soil data collection in the subsequent years may be targeted by the results of the analysis, including but not limited to the exported or saved water and nutrient prescriptions of step 854.

[0292] A unique temporal component of the RCA is that previous simulations, predictions, and optimization recommendations can be compared to actual measured outcomes once one or more of the management recommendations have been implemented in the management unit and subsequent DVS and DSC information is acquired and run through the RCA. The statistical degree of accuracy and variation can be calculated for measured, derived, and / or inferred factorsbefore and after management practices have been implemented. This process establishes the basis to generate a statistically derived guarantee of accuracy within a stated confidence interval for the RCA-based future management recommendations. Across management units in space and time, the RCA can then determine the degree of improvements in management recommendation accuracy over time and over spatial geography. The accumulated statistical degree of accuracy by management practice recommendations and actual implementation can be determined and used for business decisions, including rating and ranking management units by “manageability”, where the variability due to genetics, environment, and socio-economic factors and conditions can be computationally deconvolved to determine the respective contributions to variability from the genetics x environment x management x socio-economic factors, conditions, and data.

[0293] The ability to determine the sources and degree of statistically variability7over time and space de-risks decisions for what crops to plant, what genetics to use, and what management practices to deploy given environmental factors including climate, weather, and soils based on socio-economic factors such as yield, quality, potential environmental impacts, the ability to achieve regulatory and compliance requirements, and the ability to meet the criteria for voluntary market-based assessment, verification, and certification programs.

[0294] In addition, the ability’ to computationally deconvolve the respective GEMS factors’ contributions to statistical variability and accuracy, can be used to appraise and value land, and prospect in new geographic regions to determine similar environmental factors that are likely to result in the production of desired crops based on known genetics, known management, and known socio-economic objectives such as yield and quality.

[0295] As the digital twins created and iterated using the DVS, the DSC, and the RCA evolve and adapt over time and space, the ability to predict variability within a defined statistical range of accuracy enables dramatically improved and de-risked decision making around crop insurance, land appraisals, and a wide range of financial metrics.

[0296] In some embodiments, methods for valuing agricultural land using digital twin technologies that represent one or more agricultural management units within a land parcel involve creating a virtual representation that integrates one or more genetic, environmental, management, or socio-economic factors, conditions, or data for one or more management units. In some embodiments, the methods for valuing agricultural land include running simulations and predictions to model various agricultural scenarios and assessing factors that influence land value. By combining one or more simulations, predictions, recommendations and actual plantperformance outcomes with economic and financial metrics models, the methods for valuing agricultural land calculate an accurate valuation of the land. The digital twin provides an interactive platform for users to explore the property, customize scenarios, and receive comprehensive valuation reports. The methods for valuing agricultural land enhance decisionmaking for stakeholders by offering dynamic, data-driven insights into the property's value and potential. In some embodiments, methods for valuing agricultural land further comprise rating, ranking, and comparing lands by digital twin determined valuation.

[0297] The one or more digital twins present a systematic approach to valuing agricultural land through the use of digital twin technologies that generates a virtual replica of the property that mirrors its physical and functional characteristics. In some embodiments, the methods for valuing agricultural land encompass the collection and integration of diverse datasets, simulation of agricultural scenarios, economic valuation modeling, and the generation of detailed reports accessible via interactive platforms. By leveraging advanced technologies including the DVS, the DSC, and the RCA, the methods for valuing agricultural land provide a comprehensive and accurate assessment of land value. They account for various factors, including soil health, climate conditions, crop yield potentials, market dynamics, sustainability metrics, and risk factors.

[0298] The described methods for valuing agricultural land offer an innovative approach to valuing agricultural land by harnessing the power of digital twin technology. It provides a sophisticated tool that captures the complexity of agricultural systems and translates it into actionable economic insights.

[0299] In some embodiments, the digital twin technology described herein may also be used for methods of prospecting land for the expansion of a crop into anew region, expansion within a reason, and / or determining what to plant / replace on an existing management unit.

[0300] User Interface

[0301] In general, the above-described systems can be interacted with on a user interface, for example, displayed on a computing device. The user interface can be displayed as part of a user application installed on the computing device that can be configured to send to and receive data from a user of the system 100. For example, the user interface can display measured and predicted characteristics from the system, as well as generated recommendations. In addition, the user can input requests to receive and filter specific data of interest to the user. In addition, the user interface is configured to receive additional data, for example, economic data such asmarket information, for use in generating recommendations, as described above. The user interface can receive the additional data directly from the user, or the client application implementing the user interface can be configured to pull information from online databases, for example, government data tracking agricultural characteristics for a site of interest.

[0302] Embodiments of the subject matter and the actions and operations described in this specification can be implemented in digital electronic circuitry, in tangibly-embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, for example, one or more modules of computer program instructions, encoded on a computer program carrier, for execution by, or to control the operation of, data processing apparatus. The carrier may be a tangible non- transitory computer storage medium. Alternatively, or in addition, the carrier may be an artificially -generated propagated signal, for example, a machine-generated electrical, optical, or electromagnetic signal that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus. The computer storage medium can be or be part of a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them. A computer storage medium is not a propagated signal.

[0303] The term “data processing apparatus” encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. Data processing apparatus can include specialpurpose logic circuitry, for example, an FPGA (field programmable gate array), an ASIC (application-specific integrated circuit), or a GPU (graphics processing unit). The apparatus can also include, in addition to hardware, code that creates an execution environment for computer programs, for example, code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0304] A computer program can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages; and it can be deployed in any form, including as a stand-alone program, for example, as an app, or as a module, component, engine, subroutine, or other unit suitable for executing in a computing environment, which environment may include one or more computers interconnected by a data communication network in one or more locations.

[0305] A computer program may, but need not, correspond to a file in a file system. A computer program can be stored in a portion of a file that holds other programs or data, for example, one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, for example, files that store one or more modules, sub-programs, or portions of code.

[0306] The processes and logic flows described in this specification can be performed by one or more computers executing one or more computer programs to perform operations by operating on input data and generating output. The processes and logic flows can also be performed by special-purpose logic circuitry, for example, an FPGA, an ASIC, or a GPU, or by a combination of special-purpose logic circuitry and one or more programmed computers.

[0307] Computers suitable for the execution of a computer program can be based on general or special-purpose microprocessors, quantum computers, or any combination, or any other kind of central processing unit. Generally, a central processing unit will receive instructions and data from a read-only memory or a random-access memoty or both. The essential elements of a computer are a central processing unit for executing instructions and one or more memory devices for storing instructions and data. The central processing unit and the memory can be supplemented by, or incorporated in, special-purpose logic circuitry .

[0308] Generally, a computer will also include, or be operatively coupled to, one or more mass storage devices, and be configured to receive data from or transfer data to the mass storage devices. The mass storage devices can be, for example, magnetic, magneto-optical, or optical disks, or solid-state drives. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, for example, a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, for example, a universal serial bus (USB) flash drive, to name just a few.

[0309] To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on one or more computers having, or configured to communicate with, a display device, for example, a LCD (liquid crystal display) or organic lightemitting diode (OLED) monitor, a virtual-reality' (VR) or augmented-reality (AR) display, for displaying information to the user, and an input device by which the user can provide input to the computer, for example, a keyboard and a pointing device, for example, a mouse, a trackball or touchpad. Other kinds of devices can be used to provide for interaction with a user as well; forexample, feedback and responses provided to the user can be any form of sensory feedback, for example, visual, auditory, speech or tactile; and input from the user can be received in any form, including acoustic, speech, or tactile input, including touch motion or gestures, or kinetic motion or gestures or orientation motion or gestures. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user’s device in response to requests received from the web browser, or by interacting with an app running on a user device, for example, a smartphone or electronic tablet. Also, a computer can interact with a user by sending text messages or other forms of message to a personal device, for example, a smartphone that is running a messaging application, and receiving responsive messages from the user in return.

[0310] This specification uses the term “configured to” in connection with systems, apparatus, and computer program components. That a system of one or more computers is configured to perform particular operations or actions means that the system has installed on it software, firmware, hardware, or a combination of them that in operation cause the system to perform the operations or actions. That one or more computer programs is configured to perform particular operations or actions means that the one or more programs include instructions that, when executed by data processing apparatus, cause the apparatus to perform the operations or actions. That special-purpose logic circuitry is configured to perform particular operations or actions means that the circuitry has electronic logic that performs the operations or actions.

[0311] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back-end component, for example, as a data server, or that includes a middleware component, for example, an application server, or that includes a frontend component, for example, a client computer having a graphical user interface, a web browser, or an app through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, for example, a communication network. Examples of communication networks include a local area network (LAN) and a wide area network (WAN), for example, the Internet.

[0312] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some embodiments, a servertransmits data, for example, an HTML page, to a user device, for example, for purposes of displaying data to and receiving user input from a user interacting with the device, which acts as a client. Data generated at the user device, for example, a result of the user interaction, can be received at the server from the device.

[0313] Various modifications and additions can be made without departing from the spirit and scope of this disclosure. Features of each of the various embodiments described above may be combined with features of other described embodiments as appropriate in order to provide a multiplicity of feature combinations in associated new embodiments. Furthermore, while the foregoing describes a number of separate embodiments, what has been described herein is merely illustrative of the application of the principles of the present disclosure. Additionally, although particular methods herein may be illustrated and / or described as being performed in a specific order, the ordering is highly variable within ordinary skill to achieve aspects of the present disclosure. Accordingly, this description is meant to be taken only by way of example, and not to otherwise limit the scope of this disclosure. The appended claims as originally filed with this application shall be considered to form part of this Written Description section as if contained herein upon the filing of this application.

[0314] Exemplary embodiments have been disclosed above and illustrated in the accompanying drawings. It will be understood by those of ordinary skill in the art that various changes, omissions, and additions may be made to that which is specifically disclosed herein without departing from the spirit and scope of the present disclosure.

Claims

What is claimed is:

1. A multi-sensor soil probe, comprising: a plurality of sensors, wherein the plurality of sensors comprises: a diffuse reflectance spectroscopy sensor comprising a spectrometer that is in optical communication with soil by way of a fiber optic bundle that conveys reflected light to the spectrometer; and a sensor that collects data that is indicative of particle size distribution of the soil.

2. The multi-sensor soil probe of claim 1, wherein the diffuse reflectance spectroscopy sensor is in a mid-infrared spectral range.

3. The multi-sensor soil probe of claim 1. wherein the diffuse reflectance spectroscopy sensor is in a near-infrared range.

4. The multi-sensor soil probe of claim 1. further comprising a first sapphire window, a second sapphire window, and a video microscope; wherein the video microscope collects imagery through the first sapphire window and the diffuse reflectance spectroscopy sensor collects data through the second sapphire window.

5. The multi-sensor soil probe of claim 4. wherein the video microscope has a frame rate of at least fifteen frames per second and a resolution that is less than one millimeter of soil.

6. The multi-sensor soil probe of claim 4. wherein the video microscope comprises: a camera; and a light source that, during operation, provides strobed lighting; wherein the strobed lighting illuminates the soil at an oblique angle relative to a camera viewing angle.

7. The multi-sensor soil probe of claim 4, further comprising a conical tip and a tip stress sensor, wherein the tip stress sensor measures force on the conical tip.

8. The multi-sensor soil probe of claim 7, further comprising an outer cylindrical sleeve surface and a sleeve stress sensor, wherein the sleeve stress sensor measures friction exerted by soil surrounding the outer cylindrical sleeve surface.

9. The multi-sensor soil probe of claim 8. wherein the conical tip is formed by a pair of electrodes separated by an insulating element, and the conical tip is configured to measure volumetric water content and electrical conductivity in alternation.

10. The multi-sensor soil probe of claim 9. further comprising an acoustic sensor that comprises a micro electro-mechanical system digital microphone.

11. The multi-sensor soil probe of claim 10, wherein the multi-sensor soil probe is configured to provide at least 1,200 independent pieces of information for every centimeter of soil that it encounters.

12. A method of creating a digital twin of a crop in relation to a growing environment of the crop, the method comprising: obtaining sensor data from a subsurface sensor suite pertaining to the growing environment; obtaining sensor data from an overhead sensor suite pertaining to the crop; using the data from the subsurface sensor suite to create a digital twin of the growing environment; using the data from the overhead sensor suite to create a digital twin of the crop; and using at least one mathematical model to relate the digital twin of the crop to the digital twin of the growing environment.

13. The method of claim 12, wherein: the digital twin of the growing environment comprises a plurality of soil characteristics; the digital twin of the crop comprises a performance of the crop; the performance of the crop comprises at least one of a group consisting of crop health, crop productivity and a socioeconomic outcome; and using the at least one mathematical model to relate the digital twin of the crop to the digital twin of the growing environment comprises relating the performance of the crop to the plurality of soil characteristics.

14. The method of claim 12, wherein the digital twin of the crop comprises a first plurality of digital twins corresponding to a plurality of plants of the crop.

15. The method of claim 14. wherein the digital twin of the growing environment comprises a second plurality of digital tw ins corresponding to a plurality7of areas of soil in which each of the plurality of plants is rooted.

16. The method of claim 12, wherein obtaining sensor data from a subsurface sensor suite pertaining to the growing environment comprises using the multi -sensor probe of any one of claims 1-11.

17. The method of claim 12, wherein using the data from the subsurface sensor suite to create the digital twin of the growing environment comprises: deploying a soil sensor; acquiring depth-dependent raw sensor data from the soil sensor; developing a model to predict crop performance based on the depth-dependent raw sensor data; and simulating crop performance based on the depth-dependent raw sensor data; wherein the crop performance comprises at least one of a group consisting of crop health, crop productivity and a socioeconomic outcome.

18. The method of claim 17, wherein using the data from the overhead sensor suite to create a digital twin of the crop comprises: deploying a remote-sensing crop sensor; acquiring a plurality of images from the remote-sensing crop sensor; using the plurality of images to construct a 3D photogrammetric model of the crop; creating co-registered imagery based on the 3D photogrammetric model and a plurality of other acquired imagery, wherein the co-registered imagery shows a plurality of remotesensor indicators of plant performance; identifying each one of a plurality of plants in the co-registered imagery and statistically summarizing the plurality of remote-sensor indicators of health and performance for at least one of the plurality7of plants; developing a model to simulate performance of at least one of the plurality of plants based on the plurality of remote-sensor indicators of plant performance; and simulating health and performance of the at least one of the plurality of plants.

19. The method of claim 18, further comprising collecting non-remote sensing data, wherein the model is further based on the non-remote sensing data.

20. The method of claim 15, wherein using at least one mathematical model to relate the digital twin of the crop to the digital twin of the growing environment comprises apply ing a root cause analysis model to relate the performance of the plurality of plants to a plurality of factors.

21. The method of claim 18, wherein the plurality of factors comprises at least one of a group consisting of a genetic factor, an environmental factor, and a management factor, and each of the plurality of factors comprises a corresponding range including a plurality of steps; wherein applying the root cause analysis model comprises: calculating a statistical threshold of performance for the range of each of the plurality of factors; calculating a bootstrapped confidence interval at each of the plurality of steps in the ranges for each of the plurality of factors; and categorizing each of the plurality of steps as either sufficient, deficient, or excess, based on a difference between a lower bound of the bootstrapped confidence interval at each of the plurality of steps and the statistical threshold of performance for the range of each of the plurality of factors.

22. The method of claim 21, wherein applying the root cause analysis model further comprises calculating a plurality of opportunities to increase the performance of the plurality of plants related to a plurality of deficient and excess steps for each of the plurality of factors.

23. The method of claim 21, wherein applying the root cause analysis model further comprises using a plurality of critical levels to separate a plurality of sufficient steps from a plurality of deficient steps and from a plurality of excess steps, wherein the plurality' of critical levels are used as target levels for the plurality of factors that may be changed through a management practice.

24. The method of claim 23, wherein applying the root cause analysis model further comprises calculating a recommended adjustment for the plurality of plants for the plurality of factors.

25. The method of claim 23, wherein the management practice comprises a plurality of application rates.

26. The method of claim 21, wherein at least one of the plurality of factors is calculated using a backwards-looking model.

27. The method of claim 26, wherein each of the plurality of factors is used in the backwards-looking model to calculate a new plurality of factors that represent a plurality of steps in time for each of the new plurality of factors, wherein the plurality of steps in time comprise growth stages of the crop.

28. The method of claim 25, wherein applying the root cause analysis model further comprises calculating the plurality of application rates for a fixed plurality of materials.

29. The method of claim 28, wherein the plurality of application rates are determined in a single pass.

30. The method of claim 28, wherein the plurality of application rates are optimized in a plurality of passes.

31. The method of claim 25, wherein the plurality of application rates are calculated for an optimized set of materials chosen from a larger set of available materials in one or more passes comprising optimization techniques.

32. The method of claim 25, wherein applying the root cause analysis model further comprises feeding the plurality of application rates into a forward-looking model to predict a plurality of outcomes.

33. The method of claim 25 or 32, wherein the plurality of application rates and plurality' of outcomes are grouped into a plurality’ of grouped results.

34. The method of claim 33, wherein applying the root cause analysis model further comprises calculating a return on investment (ROI) for each of the plurality of grouped results.

35. The method of claim 34, wherein applying the root cause analysis model further comprises comparing the ROI of at least two of the plurality of grouped results against each other.

36. The method of claim 34, wherein applying the root cause analysis model further comprises reviewing the ROT for at least one of the plurality of groups to determine whether an iteration is needed.

37. The method of claim 33, wherein at least one of the plurality of grouped results is exported in a machine readable format.

38. The method of claim 37, wherein at least one of the plurality of grouped results is transferred to a machine controller and used to guide a material application rate.

39. The method of claim 38, wherein the machine controller reports at least one material application rate that is associated with a management unit.

40. The method of claim 39, wherein at least one reported material application rate is exported in a machine readable format.

41. The method of claim 40, wherein at least one material application rate is used as an input in the forward-looking model in a subsequent iteration.

42. The method of claim 22, wherein the plurality of opportunities are grouped into a plurality of grouped results.

43. The method of claim 20, wherein applying the root cause analysis model comprises determining an effective root zone based on a relationship between soil factors of varying depths and productivity.

44. The method of claim 20, wherein applying the root cause analysis model comprises targeting a plurality of scouting operations based on a difference between actual and expected plant performance.

45. The method of claim 44, wherein the plurality of scouting operations result in a plurality7of new measurements that are then included in subsequent iterations of applying the root cause analysis model.

46. The method of claim 22 or 42, wherein applying the root cause analysis model comprises categorizing the plurality of opportunities into a plurality of acute opportunities and a plurality of chronic opportunities.

47. The method of claim 20 or 42. wherein applying the root cause analysis model comprises calculating an efficiency for each of the plurality of plants.

48. A digital twin for agriculture, comprising: a plurality’ of sensor profiles collected from a plurality of modalities of sensors, wherein the plurality of sensor profiles comprises variation in soil and variation in a crop located in the soil; wherein variation in the soil comprises spatial variation in the soil and a plurality’ of temporal variations in a plurality of conditions of the soil; wherein variation in the crop comprises spatial variation in the crop and a plurality of temporal variations in a plurality of management practices for the crop; and wherein variation in the crop is correlated to variation in the soil.

49. The digital twin for agriculture of claim 48, wherein variation in the soil further comprises a plurality of spectral characteristics of the soil; and wherein variation in the crop further comprises: a plurality of spectral indices of the crop; and a plurality of temporal variations in the plurality of spectral indices of the crop.

50. A multi-sensor soil probe, comprising: a first sapphire window'; a conical tip formed by a pair of electrodes separated by an insulating element; an outer cylindrical surface; a plurality7of sensors, wherein the plurality of sensors comprises: a video microscope that collects imagery’ through the first sapphire window7; a tip stress sensor that measures force on the conical tip; anda sleeve stress sensor that measures friction exerted by soil surrounding the outer cylindrical sleeve surface; wherein the conical tip is configured to measure volumetric water content.

51. The multi-sensor soil probe of claim 50, wherein the video microscope has a frame rate of at least fifteen frames per second and a resolution that is less than one millimeter of soil.

52. The multi-sensor soil probe of claim 50, wherein the video microscope comprises: a camera; and a light source that, during operation, provides strobed lighting; wherein the strobed lighting illuminates the soil at an oblique angle relative to a camera viewing angle.

53. The multi-sensor soil probe of claim 50, wherein at least one of the plurality of sensors is a sensor that collects data that is indicative of particle size distribution of the soil.

54. The multi-sensor soil probe of claim 50, wherein the conical tip is configured to measure volumetric water content and electrical conductivity in alternation.

55. The multi-sensor soil probe of claim 54, further comprising: a second sapphire window; and a diffuse reflectance spectroscopy sensor comprising a spectrometer that is in optical communication with soil by way of a fiber optic bundle that conveys reflected light to the spectrometer; wherein the diffuse reflectance spectroscopy sensor collects data through the second sapphire window.

56. The multi-sensor soil probe of claim 55, wherein the diffuse reflectance spectroscopy sensor is in a mid-infrared spectral range.

57. The multi-sensor soil probe of claim 55, further comprising an acoustic sensor that comprises a micro electro-mechanical system digital microphone.

58. The multi-sensor soil probe of claim 57, wherein the multi-sensor soil probe is configured to provide at least 1,200 independent pieces of information for every’ centimeter of soil that it encounters.

59. The multi-sensor soil probe of claim 55, wherein the diffuse reflectance spectroscopy sensor is in a near-infrared spectral range.

60. The method of claim 12, wherein obtaining sensor data from a subsurface sensor suite pertaining to the growing environment comprises using the multi -sensor probe of any one of claims 50-59.

61. An agricultural management system, comprising: a multi-sensor soil probe configured to obtain data pertaining to a growing environment; an aerial crop sensor configured to obtain data pertaining to a crop in the growing environment; and a machine learning model configured to relate the crop to the growing environment; wherein the agricultural management system is configured to quantify at least one of particle size distribution in situ and soil organic carbon in situ.

62. A method of valuing agricultural land, comprising: creating a digital twin of a crop in relation to a growing environment of the crop, comprising: obtaining sensor data from a subsurface sensor suite pertaining to the grow ing environment; obtaining sensor data from an overhead sensor suite pertaining to the crop; using the data from the subsurface sensor suite to create a digital twin of the growing environment; using the data from the overhead sensor suite to create a digital twin of the crop; and using at least one mathematical model to relate the digital twin of the crop to the digital tw in of the growing environment; and valuing a parcel of agricultural land that comprises the crop based on the digital twin of the crop in relation to the growing environment of the crop.

63. The method of valuing agricultural land of claim 62, wherein creating the digital twin of the crop in relation to the growing environment of the crop further comprises the method of any one of claims 13-47.

64. A method of prospecting land for the expansion of a crop into anew region, comprising: creating a digital twin of the crop in relation to a growing environment of the crop, comprising: obtaining sensor data from a subsurface sensor suite pertaining to the growing environment;obtaining sensor data from an overhead sensor suite pertaining to the crop; using the data from the subsurface sensor suite to create a digital twin of the growing environment; using the data from the overhead sensor suite to create a digital twin of the crop; and using at least one mathematical model to relate the digital twin of the crop to the digital twin of the growing environment; and prospecting land for the expansion of the crop into a new region based on the digital twin of the crop in relation to the growing environment of the crop.

65. The method of prospecting land for the expansion of a crop into a new region of claim 64, wherein creating the digital twin of the crop in relation to the growing environment of the crop further comprises the method of any one of claims 13-47.

66. A method of determining what to plant on a management unit, comprising: creating a digital twin of a crop in relation to a growing environment of the crop, comprising: obtaining sensor data from a subsurface sensor suite pertaining to the growing environment; obtaining sensor data from an overhead sensor suite pertaining to the crop; using the data from the subsurface sensor suite to create a digital twin of the growing environment; using the data from the overhead sensor suite to create a digital twin of the crop; and using at least one mathematical model to relate the digital tw in of the crop to the digital twin of the growing environment; and determining what to plant on a management unit based on the digital twin of the crop in relation to the growing environment of the crop, w herein the management unit comprises the growing environment of the crop.

67. The method of determining what to plant on a management unit of claim 66, wherein creating the digital twin of the crop in relation to the growing environment of the crop further comprises the method of any one of claims 13-47.

68. A soil layer detection and soil classification system, comprising: an acoustic sensing module disposed in a soil probe;a data communication module; and a logic device in communication with the acoustic sensing module and the communication module, wherein the logic device is configured to: capture audio recordings of the soil probe advancing through soil using the acoustic sensing module; process the audio recordings; and detect transitions between soil layers through which the soil probe advances based on the processed audio recordings.

Images