System for monitoring, modeling, and managing agricultural plants and related methods and devices

Abstract

A agricultural vision system including an aerial and a land vehicle equipped with imaging and geolocation units to generate lateral and overhead maps of crops. These maps fused into a multidimensional model for object tracking, yield estimation, and growth analysis. AI-driven algorithms provide actionable insights and enable automation, including pruning, harvesting, and augmented reality visualization for precision agriculture.

Description

SYSTEM FOR MONITORING, MODELING, AND MANAGING AGRICULTURAL PLANTS AND RELATED METHODS AND DEVICESCROSS-REFERENCES & RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Application No. 63 / 571,857 filed March29, 2024, and entitled “Technologies for Agriculture,” which is hereby incorporated by reference in its entirety under 35 U.S.C. §119(e).TECHNICAL FIELD

[0002] The disclosure relates to agricultural technologies.BACKGROUND

[0003] There are various agricultural vehicles capable of driving within fields that employ computer vision techniques to identify ripe fruit (e.g., apples, tomatoes, strawberries) and then pick that fruit accordingly via end effectors (e.g., claws). Additionally, there are various known agricultural monitoring stations in vertical agriculture that employ computer vision to monitor fruit and then report on such monitoring.

[0004] These technologies suffer from various shortcomings, including manual, laborious, and time-consuming monitoring of plants I trees I bushes to determine whether (i) the vehicles I robots should be activated, (ii) the plants I trees / bushes are diseased (e.g., blight diseases), (iii) crop / buds yield I size meets expectations, (iv) the trees / bushes are growing as expected, (v) the blooming stages are timely and appropriate, and (vi) branches should be pruned.BRIEF SUMMARY

[0005] The present disclosure relates to an agricultural vision system that utilizes coordinated aerial and land-based vehicles to capture, process, and analyze visual and geolocation data for crop monitoring and management. In certain implementations, the system uses land vehicles and aerial vehicles equipped with capture units (e.g., cameras, radar) and geolocation units, which feed data to either a centralized or distributed computing instance. The collected data is used to generate lateral and overhead maps of crop rows, which may be fused into a multidimensional map. This multidimensional map enables advanced analytics, including object identification, growth tracking, defect detection, yield estimation, and decision-making for agricultural tasks such as pruning, harvesting, and chemical thinning. The system in various implementations may also support augmented reality interfaces and LLM (Large Language Model) or RAG (Retrieval-Augmented Generation) based tools I chatbots to assist users with real-time insights and interaction. The architecture I protocols / algorithms driving the system utilize multiple data models, many of which employ artificial intelligence, to optimize crop yield and quality on a per-plant basis.

[0006] According to certain Examples, a system of one or more computers can be configured to perform particular operations or actions by virtue of having software, firmware, hardware, or a combination of them installed on the system that in operation causes or cause the system to perform the actions. One or more computer programs can be configured to perform particular operations or actions by virtue of including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions.

[0007] Other implementations of these Examples include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

[0008] Implementations of the described Examples and associated techniques may include hardware, a method or process, or computer software on a computer-accessible medium.

[0009] Various terminology used herein can imply direct or indirect, full or partial, temporary or permanent, action or inaction. For example, when an element is referred to as being "on," "connected" or "coupled" to another element, then the element can be directly on, connected or coupled to the other element or intervening elements can be present, including indirect or direct variants. In contrast, when an element is referred to as being "directly connected" or "directly coupled" to another element, there are no intervening elements present.

[0010] As used herein, the term "or" is intended to mean an inclusive "or" rather than an exclusive"or." That is, unless specified otherwise, or clear from context, "X employs A or B" is intended to mean any of the natural inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, then "X employs A or B" is satisfied under any of the foregoing instances. For example, X includes A or B can mean X can include A, X can include B, and X can include A and B, unless specified otherwise or clear from context.

[0011] As used herein, each of singular terms "a," "an," and "the" is intended to include a plural form (e.g., two, three, four, five, six, seven, eight, nine, ten, tens, hundreds, thousands, millions) as well, including intermediate whole or decimal forms (e.g., 0.0, 0.00, 0.000), unless context clearly indicates otherwise. Likewise, each of singular terms "a," "an," and "the" shall mean "one or more," even though a phrase "one or more" may also be used herein.

[0012] As used herein, each of terms "comprises," "includes," or "comprising," "including" specify a presence of stated features, integers, steps, operations, elements, or components, but do not preclude a presence or addition of one or more other features, integers, steps, operations, elements, components, or groups thereof.

[0013] As used herein, when this disclosure states herein that something is "based on" something else, then such statement refers to a basis which may be based on one or more other things as well. In other words, unless expressly indicated otherwise, as used herein "based on" inclusively means "based at least in part on" or "based at least partially on."

[0014] As used herein, terms, such as "then," "next," or other similar forms are not intended to limit an order of steps. Rather, these terms are simply used to guide a reader through this disclosure. Although process flow diagrams may describe some operations as a sequential process, many of those operations can be performed in parallel or concurrently. In addition, the order of operations may be rearranged.

[0015] As used herein, the terms “response” or “responsive” are intended to include a machine- sourced action or inaction, such as an input (e.g., local, remote), or a user- sourced action or inaction, such as an input (e.g., via user input device).

[0016] As used herein, the terms "about" or "substantially" refer to a + / -10% variation from a nominal value / term.

[0017] Although various terms, such as first, second, third, and so forth can be used herein to describe various elements, components, regions, layers, or sections, note that these elements, components, regions, layers, or sections should not necessarily be limited by such terms. Rather, these terms are used to distinguish one element, component, region, layer, or section from another element, component, region, layer, or section. As such, a first element, component, region, layer, or section discussed below could be termed a second element, component, region, layer, or section, without departing from this disclosure.

[0018] Unless otherwise defined, all terms (including technical and scientific terms) used herein have a same meaning as commonly understood by skilled artisans to which this disclosure belongs. These terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in context of relevant art and should not be interpreted in an idealized or overly formal sense, unless expressly defined as such herein.

[0019] Features or functionality described with respect to certain implementations may be combined and sub-combined in or with various other implementations. Also, different aspects, components, or elements of implementations, as disclosed herein, may be combined and sub-combined in a similar manner as well. Further, some implementations, whether individually or collectively, may be components of a larger system, wherein other procedures may take precedence over or otherwise modify their application. Additionally, a number of steps may be required before, after, or concurrently with implementations, as disclosed herein. Note that any or all methods or processes, as disclosed herein, can be at least partially performed via at least one entity or actor in any manner.

[0020] Example 1 relates to an agricultural vision system comprising a land vehicle configured to take lateral view images of crops to produce a first set of capture data; an aerial vehicle configured to take overhead view images of crops to produce a second set of capture data; and at least one computing instance in electronic communication with the land vehicle and aerial vehicle, the at least one computing instance configured to generate a multidimensional map of the crops.

[0021] Example 2 relates to Examples 1 and 3-8, wherein the land vehicle is configured to record its location to produce a first set of geolocation data and the aerial vehicle is configured to record its location to produce a second set of geolocation data.

[0022] Example 3 relates to Examples 1-2 and 4-8, wherein the first set of capture data and first set of geolocation data are combined into a lateral map and the second set of capture data and second set of geolocation data are combined into an overhead map

[0023] Example 4 relates to Examples 1-3 and 5-8, wherein the lateral map and overhead map are combined to make the multidimensional map.

[0024] Example 5 relates to Examples 1-4 and 6-8, wherein the at least one computing instance processes the multidimensional map with one or more data models to produce one or more outputs.

[0025] Example 6 relates to Examples 1-5 and 7-8, wherein the outputs are interpreted and given to a user through a language learning model.

[0026] Example 7 relates to Examples 1-6 and 8, wherein the outputs are commands to one or more robots.

[0027] Example 8 relates to Examples 1-7, wherein the commands to the one or more robots can be to perform one or more tasks selected from the list consisting of picking fruit, pruning branches, and applying chemical thinning.

[0028] Example 9 relates to an agricultural vision system comprising one or more capture units configured to collect capture data from a lateral view and collect capture data from an overhead view; one or more geolocation units configured to collect geolocation data corresponding to locations of the one or more capture units during capturing of the capture data from a lateral view and capture data from an overhead view; and a computing instance in electronic communications with the one or more capture units and one or more geolocation units, the computing instance configured to analyze the capture data from a lateral view, capture data from an overhead view, and geolocation data to produce one or more outputs.

[0029] Example 10 relates to Examples 9 and 11-16, wherein the computing instance is hosted in the cloud.

[0030] Example 11 relates to Examples 9-10 and 12-16, wherein the computing instance uses one or more data models in analyzing the capture data from a lateral view, capture data from an overhead view, and geolocation data to produce one or more outputs.

[0031] Example 12 relates to Examples 9-11 and 13-16, wherein the one or more data models are chosen from the list consisting of a dormant buds model, an optimal buds model, a growth stage model, a flowers per tree model, a fruit after thinning model, a fruit size model, a yield prediction model, and a chemical thinning model.

[0032] Example 13 relates to Examples 9-12 and 14-16, wherein the analysis of the capture data from a lateral view, capture data from an overhead view, and geolocation data is done by data fusion to produce a multidimensional map, and the multidimensional map is analyzed by the one or more data models.

[0033] Example 14 relates to Examples 9-13 and 15-16, wherein the computing instance analyzes the capture data from a lateral view to identify identified objects and mark them with one or more bounding boxes.

[0034] Example 15 relates to Examples 9-14 and 16, wherein the bounding boxes are integrated into the multidimensional map.

[0035] Example 16 relates to Examples 9-15, wherein the identified objects are chosen from the list consisting of trees, bushes, branches, buds, and fruits.

[0036] Example 17 relates to a method of analyzing an agricultural field, comprising gathering data with internet of things sensors; fusing data into a multidimensional map; processing the multidimensional map with data models to produce outputs; and outputting the outputs.

[0037] Example 18 relates to Examples 17 and 19-20, wherein the internet of things sensors are one or more capture units and one or more geolocation units.

[0038] Example 19 relates to Examples 17-18 and 20, wherein the one or more capture units and one or more geolocation units are disposed on both a land vehicle and an aerial vehicle.

[0039] Example 20 relates to Examples 17-19, wherein fusing data into a multidimensional map and processing the multidimensional map with data models to produce outputs are done by a computing instance located in the cloud.

[0040] While multiple implementations are disclosed, still other implementations of the disclosure will become apparent to those skilled in the art from the following detailed description, which shows and describes illustrative implementations of the invention. As will be realized, the disclosure is capable of modifications in various obvious aspects, all without departing from the spirit and scope of the disclosure. Accordingly, the drawings and detailed description are to be regarded as illustrative in nature and not restrictive.BRIEF DESCRIPTION OF DRAWINGS

[0041] FIG. 1 is a diagram of the system in use on a field, according to one implementation.

[0042] FIG. 2 is a diagram of the components of the system in communication with one another, according to one implementation.

[0043] FIG. 3 is a diagram of the components of the system in communication with one another, according to one implementation.

[0044] FIG. 4 is a diagram showing data transfer of the components of a land vehicle to the computing instance, according to one implementation.

[0045] FIG. 5 is a diagram showing data transfer of the components of an aerial vehicle to the computing instance, according to one implementation.

[0046] FIG. 6 shows the data fusion in the computing instance, according to one implementation.

[0047] FIG. 7A shows a lateral image as would be collected by a land vehicle with bounding boxes overlayed, according to one implementation.

[0048] FIG. 7B shows a lateral image as would be collected by a land vehicle with ID tags overlayed, according to one implementation.

[0049] FIG. 8A shows an overhead image as would be collected by an aerial vehicle with bounding boxes overlayed, according to one implementation.

[0050] FIG. 8B shows an overhead image as would be collected by an aerial vehicle showing vegetation density, according to one implementation

[0051] FIG. 9 shows a screen of the Ul terminal, according to one implementation.

[0052] FIG. 10 shows a screen of the Ul terminal, according to one implementation.

[0053] FIG. 11 shows a screen of the Ul terminal, according to one implementation.

[0054] FIG. 12 shows a screen of the Ul terminal, according to one implementation.

[0055] FIG. 13 shows a screen of the Ul terminal, according to one implementation.

[0056] FIG. 14 shows a diagram of the data processing done through data models to yield outputs with the multidimensional map data used as inputs, according to one implementation.

[0057] FIG. 15 shows a diagram of the data processing done through data models to yield outputs with the capture data and geolocation data used as inputs, according to one implementation.

[0058] FIG. 16 shows a flowchart of a method that may be employed by the system, with the multidimensional map data used as inputs, according to one implementation.

[0059] FIG. 17 shows a flowchart of a method that may be employed by the system, with the capture data and geolocation data used as inputs, according to one implementationDETAILED DESCRIPTION

[0060] This disclosure is now described more fully with reference to all attached figures, in which some implementations of this disclosure are shown. This disclosure may, however, be embodied in many different forms and should not be construed as necessarily being limited to various implementations disclosed herein. Rather, these implementations are provided so that this disclosure is thorough and complete and fully conveys various concepts of this disclosure to skilled artisans. Note that like numbers or similar numbering schemes can refer to like or similar elements throughout.

[0061] According to certain implementations, such as in FIG. 1 , the disclosed agricultural vision system 10 comprises one or more vehicles 12, including one or more land vehicles 14 and, optionally, one or more aerial vehicles 16, a computing system or computing instance 18, and all associated hardware, software, and / or firmware components required to perform the various functions described herein.

[0062] The land vehicle 14 may be any vehicle that is configured to move on land, such as but not limited to an unmanned vehicle, a tractor, an all-terrain vehicle, a utility terrain vehicle, or the like.

[0063] The aerial vehicle 16 may be any vehicle capable of flight. In certain implementations, the aerial vehicle 16 may be capable of stationary flight, known as hovering. The aerial vehicle 16 may be, but is not limited to being, a drone, a helicopter, a quadcopter, or similar vehicle.

[0064] In various implementations, the at least one land vehicle 14 is an agricultural land vehicle14 hosting a capture unit 20 and a geolocation unit 22, such as shown in FIGS. 2 and 3. In some implementations, the system 10 may have one or more aerial vehicles 16 hosting a capture unit 24 and a geolocation unit 26.

[0065] The capture units 20, 24 may be devices configured to capture and process visual information. In various implementations, the capture units 20, 24 may be cameras of various types.

[0066] The geolocation units 22, 26 may be devices configured to track the location of the geolocation units 22, 26. In certain implementations, the geolocation units 22, 26 may use satellite navigation, such as but not limited to global navigation satellite systems (GNSS) like a global positioning system (GPS), global navigation satellite system (GLONASS), the BeiDou navigation satellite system (BDS), the Galileo satellite navigation system, or the like as would be understood by those of skill in the art.

[0067] In various implementations, the system 10 may have a computing instance 28 that may give commands and collect data from the various components of the land vehicle 14, such as the capture unit 20 and geolocation unit 22.

[0068] In implementations like those of FIG. 2, the computing instance 28 may be remote from the land vehicle 14 but still in electronic communication with the land vehicle 14 and various components of the land vehicle 14, such as but not limited to the capture unit 20 and geolocation unit 22.

[0069] Similarly, a computing instance 32 may be remote from the aerial vehicle 16 but still in electronic communication with the aerial vehicle 16 and various components of the aerial vehicle 16, such as but not limited to the capture unit 24 and geolocation unit 26.

[0070] In implementations like those of FIG. 3, the computing instance 28 may be integrated into the land vehicle 14 and in electronic communication with the land vehicle 14 and various components of the land vehicle 14, such as but not limited to the capture unit 20 and geolocation unit 22.

[0071] Similarly, the computing instance 32 may be integrated into the aerial vehicle 16 and still in electronic communication with the aerial vehicle 16 and various components of the aerial vehicle 16, such as but not limited to the capture unit 24 and geolocation unit 26.

[0072] In various implementations, the computing instances 28, 32 may be subcomponents of a broader computing instance 18, such as but not limited to being separate programs or subroutines within a computing instance 18. In certain implementations, a first computing instance 28 and a second computing instance 32 may be hosted on the computing instance 18, which is a cloud server.

[0073] In alternative implementations, the computing instances 28, 32 may be separate components, such as individual cloud servers, virtual cloud servers, or computing devices. In such implementations, the computing instances 28, 32 may be in electronic communication through various devices and systems known to those in the art, such as but not limited to internet communications, wireless communication, wired communication, Bluetooth, Wi-Fi, cellular networks, and the like as would be understood by those of skill in the art.

[0074] In various implementations, the system 10 may have a Ul terminal 33 in electronic communication with the computing instances 18, 28, 32 which a user of the system 10 may use to interface with the system 10. The Ul terminal 33 may be used to examine data and issue commands to the system 10. The Ul terminal may be a computing device, such as but not limited to a laptop computer, desktop computer, tablet, smartphone, headset device (such as a Google Glass, Apple Vision Pro, or the like), or similar device.

[0075] Turning now to FIG. 4, in various implementations, capture unit 20 and geolocation unit 22 may be in communication with the computing instance 28 programmed to receive a set of capture data 34 from the capture unit 20 and a set of geolocation data 36 from the geolocation unit 22 based on the land vehicle 14. The land vehicle optionally moving on a path 30 lateral to a row of trees, crops, or other plants as the set of capture data 34 describes at least some of the trees, crops, or other plants or parts thereof in the row from a lateral view. The set of geolocation data 36 geolocates at least some of the tree, crops, or other plants or parts thereof in the row. In various implementations, the set of capture data 36 may include visual image data of relevant plants, crops, trees, and the like.

[0076] Shown in FIG. 5, in various implementations, the capture unit 24 and geolocation unit 26 may be in communication with a computing instance 32 programmed to receive a set of capture data 38 from the capture unit 24 and a set of geolocation data 40 from the geolocation unit 26 on the aerial vehicle 16. The set of capture data 38 may include visual image data taken from an overhead view of relevant plants, crops, trees, and the like. The set of capture data 38 may also contain vegetation density data.

[0077] As shown in FIG. 6, the system 10 may associate the set of capture data 34 and the set of geolocation data 36 to form a lateral map 42 for the row of trees, crops, or other plants or parts thereof. As would be understood, the lateral map 42 shows a two-dimensional representation of the field from a particular observation point on or near the ground, with the observation field of view roughly parallel to the plane of the ground.

[0078] FIG. 7A shows an exemplary lateral map 42 of a row of trees, where various trees and flowers have been digitally identified by the system 10. Optionally, the system 10 may place a bounding box 44 around each identified object 45. In various implementations, an identified object 45 may be any relevant physical object of interest in the system 10, such as but not limited to trees, bushes, branches, buds, or fruits. As would be understood, the bounding box 44 may be a digital device to catalog the location of a physical object in a digital framework. In implementations like those shown in FIG. 7B, the system 10 may use the set of capture data 34 and the set of geolocation data 36 to give each bounding box 44 an ID Tag 46, which may be a unique identifier that allows each object to be tracked individually.

[0079] In some implementations, the bounding box 44 and ID Tag 46 may be placed on a reference object, which may be a tree, plant, bush, or the like that is selected to serve as the frame of reference for later bounding boxes 44, optionally at the beginning of the path 30. Individual trees, plants, bushes, or the like and branches or similar subcomponents may then be identified through a segmentation process in the algorithm 100 of the computing instance 18. As the system 10 progresses across each tree,the imagery enables identification of each branch on each tree, bush, plant, or part thereof. Optionally, the system 10 may execute tree, plant, bush, or the like segmentation as part of the identification process. The system 10 executing an algorithm 100 optionally identifies individual trees, plants, bushes, or the like and then further segments each tree, plant, bush, or the like into various parts such as branches, fruits, and the like.

[0080] Likewise shown in FIG. 6, the system 10 may associate the set of capture data 38 and the set of geolocation data 40 to form an overhead map 48 for the field. The overhead map 48 optionally including each tree, plant, bush or the like in the rows that has been described from the overhead view. As would be understood, the overhead map 48 shows a two-dimensional representation of the field from a particular observation substantially above the ground, with the observation field of view roughly perpendicular to the plane of the ground. The overhead map 48 may also contain and display vegetation density data.

[0081] FIG. 8A shows an example of an overhead map 48 with bounding boxes 44 overlaid to track various trees or bushes. FIG. 8B shows an example of an overhead map 48 with vegetation density displayed.

[0082] Returning to FIG. 6, in various implementations, the system 10 may fuse the lateral map42 and the overhead map 48 such that a multidimensional map 50 including both the lateral view and the overhead view is formed. The system 10 may then take actions based on the multidimensional map, as is discussed in more detail below.

[0083] Returning to FIG. 1 , the land vehicle 14 may travel on a path 30 within a field, where the field has an array of trees, bushes, plants, or the like, each having or capable of having fruit. In various implementations, the path 30 may extend within the array of trees, bushes, plants, and the like (e.g., between rows) in a serpentine manner. While traveling along the path 30, the land vehicle 14 may use capture units 20 to capture images of the trees, bushes, plant, or parts thereof from the lateral side, while the geolocation unit 22 tracks the land vehicle 14. In some implementations, the geolocation unit 22 may be integrated into the capture unit 20, which may increase data accuracy.

[0084] As will be discussed in more detail below, an algorithm 100 in the computing instance 18,28 may process the video data and the location data of the land vehicle 14, identify individual trees, bushes, branches, buds, fruits, and the like, and assign coordinates to such objects based on the location of the land vehicle 14 and data from the geolocation unit 22. In various implementations, the algorithm 100 may be configured to determine the number of one or more objects of interest (e.g., branches, buds, flowers, fruits). These algorithms 100 may also determine the geolocation of each tree, bush, plant, or the like. Improved accuracy when placing the geolocation unit 22 on the capture unit 20 of the land vehicle 14 may occur as the system does not need to compensate for the distance between the geolocation unit 22 and the capture unit 20 when correlating image and location data.

[0085] The aerial vehicle 16 may fly over the field and may use the capture units 24 to capture images of the field and objects within the field from the top side while the geolocation unit 26 tracks theaerial vehicle 16. In some implementations, the geolocation unit 26 may be integrated into the capture unit 24, which may increase data accuracy.

[0086] In various implementations, an algorithm 100 in the computing instance 18, may process the capture data 38 and the location data 40 of the aerial vehicle 16, identify individual trees, bushes, branches, buds, fruits, or the like, and assign coordinates to objects of interest based on the location of the aerial vehicle 16. Improved accuracy when placing the geolocation unit 22 on the capture unit 20 of the aerial vehicle 16 may occur as the system does not need to compensate for the distance between the geolocation unit 22 and the capture unit 20 when correlating image and location data.

[0087] In certain implementations, the sets of capture data 34, 38 may be sent to the computing instance 18, which may be hosted remotely, i.e. in the cloud, to form a lateral map 42 and an overhead map 48 to be fused by the computing instance 18. Once fused, the computing instance 18 forms a multidimensional map 50 of the field, the path 30, and identified objects 45. In combining the two- dimensional lateral map 42 and two-dimensional overhead map 48, taken from different perspectives, the multidimensional map 50 may be three-dimensional, as the relative perspectives may allow for the missing dimension of each two-dimensional map to be filled by the other. Geolocation data in the lateral map 42 and overhead map 48 may be used to align and combine the maps creating a more precise fused map.

[0088] In various implementations, the various computations and steps used to combine the lateral map 42 and overhead map 48 into the multidimensional map 50 may be referred to as data fusion 80 (discussed more below).

[0089] The multidimensional map 50 attempts to identify each identified object 45 from the imagery and associate each identified object 45 with a specific geolocation. At that time, various computer vision algorithms are employed on the imagery, based on the multidimensional map 50, on a per identified object 45 basis based on the respective geolocation of the object, which then enable various automated determinations. These automated determinations may include (i) if activation of a robots or other deployment to pick the fruits from the identified object 45 is needed, (ii) if the identified object 45 (such as trees, bushes, buds, or the fruits) are diseased or blighted, (iii) if the identified objects 45 meet anticipated yield expectations, (iv) if the identified objects 45 are growing as expected, (v) if the blooming stages of the identified objects 45 are timely and appropriate, and / or (vi) if any branches that should be pruned.

[0090] In various implementations one or more of the capture units 20, 24 may use visible spectrum cameras. In further implementations one or more the capture unit 20, 24 may also include a radar to scan its field of view in order to detect objects of interest that may be obscured / occluded from being imaged by a line of slight camera. The radar may be but is not limited to a radio frequency (RF) or digital radar. The radar may be implemented to scan the identified objects 45 for the computing instance 18 to supplement the imagery or to fuse the imagery with the radar data to build a model of a respective identified object 45. The data from the radar may also be geotagged as would be appreciated in light of this disclosure.

[0091] Turning to FIGS. 9-13, the computing instance 18 may interface with the user of the system10 on the Ul terminal 33. In some implementations where the Ul terminal 33 is portable, such as a GoogleGlass, Apple Vision Pro, cellphone, tablet, or the like, the Ul terminal 33 may be employed in the field. In such implementations, the user of the system 10 may inspect the field and the identified object 45 with augmented reality to readily correlate the gathered data with the physical identified objects 45.

[0092] The system 10 may enable an augmented reality superimposed over the tree / bush / branch / bud / flower / fruit when gazed or oriented thereto, which when the user looks at a tree using the Ul terminal 33, the system 10 enables each respective tree / bush / branch / flower / fruit / bud to be seen with augmented reality content superimposed. This may allow the user to be able to see relevant information for each respective tree / bush / branch / flower / fruit / bud on a given tree that the user is looking at. The system 10 may use color coding in the superimposed content to convey the information to the user. The user using the Ul terminal 33 may be able to see a drawn bounding box 44 (or another suitable indicator) for each tree / bush / branch / flower / fruit / bud. Thus, the augmented reality feature may provide an immersive experience, and the user can walk row by row for a real time and interactive experience.

[0093] FIGS. 9-13 show various presentations and interface options that may be presented to the user of the system 10 by the Ul terminal 33.

[0094] As discussed briefly above, in various implementations, the computing instance 18 may work based on cloud technology and may be web-based. The computing instance 18 may use an Application Programming Interface (or API) that transfers data from the field into the computing instance 18, which in turn contains end point database and algorithms. The API and computing instance 18 may communicate back and forth between each other or between the field and the database and vice versa. The algorithms 100 in the computing instance 18 may contain a number of data models 102, optionally the computing instance 18 includes eight data model 102 although more or less are possible. In certain implementations, the data models 102 may use artificial intelligence (Al) for some or all of the computations.

[0095] In some implementations, the goal of the data models 102 and the algorithms 100 in the computing instance 18 is to have each individual tree, bush, plant, or the like in orchards or fields produce an optimal yield and the highest quality produce as possible. Alternative desired outcomes / goals are possible and would be understood. The data models 102 may be modified or tailored to the desired outcome / goal to be achieved. In one specific example, the goal is to obtain the optimal number of 100-120 apples and the fanciest apples possible, the latter of which are apples of the best quality and that can be sold for a higher price by a farmer or grower, explained further below.

[0096] As would be appreciated, the dormant buds model 110, optimal buds model 112, growth stage model 114, flowers per tree model 116, and fruit after thinning model 118 may be applied during a fruitless stage of crop development. The fruit size model 120, yield prediction model 122, and chemical thinning model 124 may be applied during a fruit stage of crop development. As would likewise be appreciated, the fruitless stage of crop development occurs when there is no noticeable fruit growth and the fruit stage of crop development occurs when there is noticeable fruit growth.

[0097] In various implementations the data models may also incorporate additional data sources with the multidimensional map 50, such as but not limited to weather data, diseases databases, crop varieties and other public database to enhance predictive accuracy.

[0098] FIG. 14 shows a diagram of the relationship between the algorithm 100, data models 102, and the individual models 110, 112, 114, 116, 118, 120, 122, 124. The system 10 may take the multidimensional map 50 and the data contained therein and use it as an input to an algorithm 100, which may contain the data models 102. After data processing from the algorithm 100 and data models 102, the system 10 may create one or more outputs 52, which will be discussed in detail below.

[0099] In various implementations, the algorithm 100 and data model 102 (including the individual models 110, 112, 114, 116, 118, 120, 122, 124) may be constructed using artificial intelligence techniques, such as machine learning and neural network programming techniques. In other implementations, the algorithm 100 and data model 102 (including the individual models 110, 112, 114, 116, 118, 120, 122, 124) may be programmed with traditional programming methods. In still further implementations, the algorithm 100 and data model 102 (including the individual models 110, 112, 114, 116, 118, 120, 122, 124) may be constructed with a combination of traditional programming and artificial intelligence techniques.

[0100] It would be appreciated that while these data models 102 are discussed herein in relation to trees and apples for the sake of clarity and brevity. The data models 102 may also apply to various other crop-producing plants and other fruits / crops as would be understood and appreciated.

[0101] The dormant buds model 110 estimates the number of dormant buds by tree. The dormant buds model 110 accounts for and summarizes how many buds are on each tree and on each branch of each individual tree.

[0102] The optimal buds model 112 estimates the number of optimal buds per tree. This optimal buds model 112 may estimate the optimal number of buds for each tree and on each branch of each tree for ideal growth of the apples. In certain implementations, the ideal growth of apples is about 100-120 apples per tree for high density apple trees (although this estimate can vary depending on tree, growing conditions, and other parameters that would be understood by those of skill in the art). The optimal buds model 112 may be used to ensure that each tree and branch yields the optimal amount of fruit.

[0103] The growth stage model 114 determines thegrowth stage of an object of interest. In various implementations the growth stage model 114 tracks the changes in the flower growth stages of the flowers on each tree and the flowers on each branch of each tree (as conventionally the buds become swollen before they start to flower).

[0104] The flowers per tree model 116 determines the number of flowers on a tree. The flowers per tree model 116 counts the number of flowers per tree at bloom. The flowers per tree model 116 may be used to determine when to start or apply a pollen-to-growth model, and when to start pollination of the flowers. As would be understood, the pollen-to-growth model helps farmers or growers know when to start pollination of the flowers on the trees by the use of bees, as well as when to remove the bees after pollination and apply chemicals to the flowers which helps the flowers grow into better quality apples, known aschemical thinning. The application of chemical thinning informs when and how to thin the flowers with chemicals to yield larger and healthier apples. Through the chemical thinning process, also referred to as the carbohydrate model and fruit growth model, is when carbohydrate solutions or compounds are applied to the flowers which increases or decreases the formation of carbohydrate in the flowers, which then impacts or helps their growth potential. Chemical thinning may be done after pollination so the bees that pollinate the flowers are not exposed to chemicals.

[0105] The fruit after thinning model 118 estimates the count of fruit after blossom thinning or spraying. Before the fruit begins to grow and when the chemical thinning is applied, the fruit after thinning model 118 may be used as a fruit-length-after-blossom cleaning model, in which the desired length of the blossoming fruit is between 5-10 mm (although this may vary). The purpose of monitoring the fruit count and growth after blossoming may be to guide whether further chemical thinning is needed and when it should be done, as chemical thinning can be done more than once and at this stage as well.

[0106] The fruit size model 120 is a fruit size determination model. After the fruitless stage, this fruit size model 120 is used to look at the size increase of the fruit after each chemical thinning process or spraying. The fruit size 120 model operates by taking digital measurements of the fruit size and tracking the increase in fruit size to determine the effectiveness of the chemical thinning applied. Between the fruitless stage and fruit stage, apple farmers or growers may apply chemical thinning to the apples, and what informs their decision of when to harvest particular apples the size or growth rate of this model. One purpose of this model is to determine the fruit size and when to pick the apples as they reach the desired size.

[0107] The yield prediction model 122 determines the count of fruits and measurement of fruit size. It may track how much each branch, tree, and field or orchard as a whole has yielded, including the number of fancy apples that were yielded. As would be understood, fancy apples are certain apples that do not have defects. Apples without defects may be, but are not limited to, those that are free from decay, internal browning, internal breakdown, and scabs. Fancy apples are the type of apples that will be sold as whole apples on grocery store shelves or produce sections, whereas non-fancy apples or apples that have defects are sold for use for different purposes (for example, going into making apple juice or other products). The fancy apple vs. non-fancy apple distinction determines the use of each apple. This model of the algorithm 100 helps with the classification of apples, which in turn helps farmers or growers with the sorting of apples. This yield prediction model 122 also determines when apples are ready for harvest.

[0108] The chemical thinning model 124 may be a chemical thinning or chemical precision thinning model. This chemical thinning model 124 applies at each part of the process where chemical thinning may be required, and purpose of the model is to determine the amount of the carbohydrate solution to apply to the flowers or blossoms on each tree and each branch of each tree.

[0109] The system 10 may be used as a whole for tree fruits, and all eight models of the algorithm100 above apply to tree fruits, in that all the fruit will need to go through all of the model changes. The system 10 could be used for other ground crops. In implementations where the system 10 is applied toother ground crops, different combinations of the data models 102 may be used, where appropriate. In implementations where leaves are analyzed rather than flowers, it may be that only the flowers per tree model 116, fruit after thinning model 118, fruit size model 120, and yield prediction model 122 would be used for leaf count (rather than fruit) estimation and that the application of thinning chemicals could be omitted.

[0110] Each respective model of the algorithm 100 may show actual photo or image frames of each tree and each branch, converted from captured video and geolocation data, which then enables the algorithm to identify which actions are needed to be taken on each individual tree and branch.

[0111] In some implementations, the capture units 20, 24 capture the changes in color, size, and measurements of the buds, flowers, blossoms, or fruits, and the algorithm 100 may compare the current and past status of the same. This may enable the algorithm 100 to track growth and quality changes. Thus, color and shape matter because the changes in color and shape enable the computing instance 18 and the algorithm 100 used therein to determine the existence of defects and in turn to determine fancy apples vs. non-fancy apples, as well as the size and measurements of apples. The capture units 20, 24 may capture video data and both the land vehicle 14 and aerial vehicle 16 enable such capture.

[0112] The system 10 may present information such as outputs 52 (discussed more below) of the data models 102 to the user through an augmented reality interface (as discussed above) and I or with the assistance of a chatbot, such as a large language model (LLM) chatbot or a retrieval-augmented generation (RAG) chatbot. LLM chatbots may be a machine learning program that specializes in mimicking human speech and conveying information known to a system to users in a natural manner. RAG chatbots may be a machine learning program that combines information retrieval with generative artificial intelligence, allowing it to fetch relevant external data and generate more context-aware responses. For instance, an LLM or RAG chatbot within the system may be configured to highlight areas of concern or interest in the data and present it to the user in a conversational and understandable fashion. In alternative implementations, the system 10 information may be presented to users through user interfaces, such as are shown in FIGS. 9-13.

[0113] The determinations by the system 10 are enabled via the data models 102 referenced above to provide data for users of the system 10 to enable various determinations or decision-making. For example, in addition to the automated determinations discussed below, there are may also be automated determinations regarding when chemical thinning should be done, where and when pollination should be done and where, and other determination that would be appreciated by those of skill in the art. Specifically, the bloom stages can help determine when the bees are ready to pollinate the flowers, and subsequently when chemical thinning is applied. Similarly, the application of pesticides or fungicides during certain stages can be performed by the cloud computing instance as well.

[0114] The system 10 may output one or more outputs 52 that enable robot-pickers to be activated to pick fruit, prune branches, chemically thin, or perform various other tasks that may be needed. For example, when the optimal bud count number is met as determined by the optimal buds model 112, therobots may then be able to remove excess buds above that optimal number of buds from the trees. For each decision point in the various models of the algorithm 100, the system 10 may feed data to the robots based on optimal count models or growth models, and once a certain threshold is hit under those models, the system 10 may tell the robots or activates the robots to, for example, prune, chemically thin, and so forth. This may be the case for each decision point, including the fruit size model 120 that determines size and yield. In the case of the yield prediction model 122, the system 10 may tell the robots when to harvest. In some situations, the user may activate robots based on the system 10 notifying the user via the Ul terminal 33 to activate the robots for a specific task.

[0115] Alternatively, the output 52 may be a display of information to a user, and the user may perform these tasks manually without robots.

[0116] If the tasks are done by the robots, the robots may also travel path 30 the field in a serpentine manner. The robots may see the video data that the land vehicle 14 and aerial vehicle 16 captured when they had their last travelling or gathering of data around the field. The user may follow along with the robots and, when prompted by the robots or the system 10, allow them to carry out the action. The user can do this either by following along and monitoring robot activity at each tree or by interacting with a Ul terminal 33 and deciding whether the activity will be carried out. In certain implementations, the user may prompt the robots or the system 10 to carry out various actions as they are conducted, i.e. in real time. In other implementations, the user may prompt the robots or system 10 to carry out various actions independently of the actions being conducted, such as in advance of the conduct, also known as not real time.

[0117] The system 10 may enable determination as to whether the trees / bushes / branches / buds / fruit are diseased (e.g., blight diseases). Whether the tree / bush or the branches or the flowers or fruit on the tree is diseased may be based on color. For example, there can be scabs (which are small dots) on flowers or blight (brown or black marks). The dormant buds model 110, optimal buds model 112, growth stage model 114, flowers per tree model 116, and fruit after thinning model 118, as referenced above, are operative when disease can be determined, because this occurs mainly at the fruitless stage, since the flowers or leaves also show the scabs or defects. Despite this, it may be determined at any stage of the algorithm 100, although the algorithm 100 is structured so that it determines it as early as possible, which would hopefully be at the fruitless stage.

[0118] The system 10 may enable determination as to whether crop yield or bud size meet expectations. This may be determined using the dormant buds model 110, optimal buds model 112, growth stage model 114, flowers per tree model 116, fruit after thinning model 118, fruit size model 120, and yield prediction model 122, as referenced above. The size expectation may be based on different varieties of the apple. The algorithm 100 may be fed average sizing measurements in diameter for each type and subtype of tree fruit or crop that it deals with in a certain field.

[0119] The system 10 may enable determination as to whether the trees / bushes / branches / buds / fruits are growing as expected. This may be determined with fruit after thinningmodel 118 and fruit size model 120, as referenced above hosted on the cloud computing instance, where it is presenting the data at three levels, by row, by tree, and by acre. The algorithm 100 may monitor all growing estimates by tree / bush, row, and acre, and compare estimates to later captured data. The purpose of this is to determine the production output by each of the three levels, which in turn provides information that certain factors may be affecting individual trees or parts of orchard, which may need to be remedied.

[0120] The system 10 may enable determination as to whether the blooming stages are timely and appropriate. This may be determined by the growth stage model 114. The purpose of ensuring the blooming stages are timely and appropriate is so that the pollination and chemical thinning models are applied at the appropriate time to ensure optimal growth, size, quality and yield. This may determine when to apply further models under the algorithm 100.

[0121] The system 10 may enable determination as to whether branches should be pruned, either mechanically or chemically, as described above. There may be use cases when the camera technology and visual data are used to give the farmer a count, and then the farmer does his own count, and radar technology may be used when there is a discrepancy between the visual data count and the farmer’s count. Additional or alternative to aerial vehicle 16, a satellite may be used, in which case satellite imagery would be used.

[0122] In certain implementations, such as shown in FIG. 15, the system 10 may directly use the capture data 34, 38 and geolocation data 36, 40 may be used as the inputs to the algorithm 100. In such implementations, the data model 102, containing the individual models 110, 112, 114, 116, 118, 120, 122, 124, may use the capture data 34, 38 and geolocation data 36, 40 to construct the various outputs 52 that may be appropriate.

[0123] FIG. 16 shows an overview of a mapped method 200 that the system 10 may follow, which was described in various aspects above. The specific steps and order of steps described in this particular mapped method 200 is only exemplary of the methods for which the system 10 may be used. For this reason, those skilled in the art would understand that each step may be omitted, and other steps may be added. Additionally, the presented order of the steps may be rearranged as may be required in different implementations. That is the various steps may be performed in any order or not at all, some steps may be preformed sequentially, iteratively, and / or concurrently.

[0124] The mapped method 200 may begin with gathering data using the sensors, which may beInternet of Things (loT) sensors, that are deployed on the land vehicles 14 and aerial vehicles 16 (box 202). Various examples of the loT sensors include capture units 20, 24 and the geolocation units 22, 26, as well as others that may be understood or appreciated in light of this disclosure. Examples of the data include but are not limited to the set of capture data 34, 38 and geolocation data 36, 40.

[0125] The data may be sent to the computing instance 18, 28 32 for the data fusion steps to be performed to produce the multidimensional map 50 and related data, as described above (box 204).

[0126] The multidimensional map 50 data may then be processed with the data models 102 as described above to yield outputs 52 that may be displayed to a user, output through an LLM or RAG chatbot, or used to generate an augmented reality display (box 208).

[0127] The system 10 may then take action from the results, such as ordering robots to take action like harvesting, trimming, pruning, and / or chemically thinning, as well as other various actions. These actions may be automatic or may require input and / or approval from the user (box 210). That is, in some implementations, the system 10 may generate robotic scripts as outputs allow for integration with various known robotic systems for use in agriculture.

[0128] FIG. 17 shows an overview of a direct method 300 that the system 10 may follow. The specific steps and order of steps described in this particular mapped method 200 is only exemplary of the methods for which the system 10 may be used. For this reason, those skilled in the art would understand that each step may be omitted, and other steps may be added. Additionally, the presented order of the steps may be rearranged as may be required in different implementations. That is the various steps may be performed in any order or not at all, some steps may be preformed sequentially, iteratively, and I or concurrently.

[0129] The direct method 300 may begin with gathering data using the sensors, which may be loT sensors, that are deployed on the land vehicles 14 and aerial vehicles 16 (box 302). Various examples of the loT sensors may include capture units 20, 24 and geolocation units 22, 26. Examples of the data include but are not limited to the set of capture data 34, 38 and geolocation data 36, 40.

[0130] The data, which may include some or all of the set of capture data 34, 38 and geolocation data 36, 40., may be sent to the computing instance 18, 28, 32, where it may be used as inputs for the algorithm 100 and data models 102 to produce outputs 52 (box 304).

[0131] The outputs 52 may then be displayed to a user, output through an LLM and RAG chatbot, or used to generate an augmented reality display (box 306).

[0132] The system 10 may then take action from the results, such as ordering robots to take action like harvesting, trimming, pruning, and / or chemically thinning, as well as other various actions. These actions may be automatic or may require input and / or approval from the user (box 308).

[0133] Various implementations of the present disclosure may be implemented in a data processing system suitable for storing and / or executing program code that includes at least one processor coupled directly or indirectly to memory elements through a system bus. The memory elements include, for instance, local memory employed during actual execution of the program code, bulk storage, and cache memory which provide temporary storage of at least some program code in order to reduce the number of times code must be retrieved from bulk storage during execution.

[0134] I / O devices (including, but not limited to, keyboards, displays, pointing devices, DASD, tape, CDs, DVDs, thumb drives and other memory media, etc.) can be coupled to the system either directly or through intervening I / O controllers. Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storagedevices through intervening private or public networks. Modems, cable modems, and Ethernet cards are just a few of the available types of network adapters.

[0135] This disclosure may be embodied in a system, a method, and / or a computer program product. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present disclosure. The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punchcards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0136] Computer readable program instructions for carrying out operations of the present disclosure may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. A code segment or machine-executable instructions may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, among others. The computer readable program instructions may execute entirely on the user's computer, partly onthe user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In various implementations, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0137] Aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to implementations of the disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions. The various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the implementations disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

[0138] The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various implementations of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustration, and combinations of blocks in the block diagrams and / or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.

[0139] Words such as “then,” “next,” etc. are not intended to limit the order of the steps; these words are simply used to guide the reader through the description of the methods. Although process flow diagrams may describe the operations as a sequential process, many of the operations can be performedin parallel or concurrently. In addition, the order of the operations may be re-arranged. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to a return of the function to the calling function or the main function.

[0140] Although the disclosure has been described with references to various implementations, persons skilled in the art will recognize that changes may be made in form and detail without departing from the spirit and scope of this disclosure.

Claims

CLAIMSWhat is claimed is:1 . An agricultural vision system comprising: a) a land vehicle configured to take lateral view images of crops to produce a first set of capture data; b) an aerial vehicle configured to take overhead view images of crops to produce a second set of capture data; and c) at least one computing instance in electronic communication with the land vehicle and aerial vehicle, the at least one computing instance configured to generate a multidimensional map of the crops.

2. The system of claim 1 , wherein the land vehicle is configured to record its location to produce a first set of geolocation data and the aerial vehicle is configured to record its location to produce a second set of geolocation data.

3. The system of claim 2, wherein the first set of capture data and first set of geolocation data are combined into a lateral map and the second set of capture data and second set of geolocation data are combined into an overhead map.

4. The system of claim 3, wherein the lateral map and overhead map are combined to make the multidimensional map.

5. The system of claim 1 , wherein the at least one computing instance processes the multidimensional map with one or more data models to produce one or more outputs.

6. The system of claim 5, wherein the outputs are interpreted and given to a user through a chatbot.

7. The system of claim 5, wherein the outputs are commands to one or more robots.

8. The system of claim 7, wherein the commands to the one or more robots can be to perform one or more tasks including one or more of picking fruit, pruning branches, and applying chemical thinning.

9. An agricultural vision system comprising: a) one or more capture units configured to collect capture data from a lateral view and collect capture data from an overhead view;b) one or more geolocation units configured to collect geolocation data corresponding to locations of the one or more capture units during capturing of the capture data from a lateral view and capture data from an overhead view; and c) a computing instance in electronic communications with the one or more capture units and one or more geolocation units, the computing instance configured to analyze the capture data from a lateral view, capture data from an overhead view, and geolocation data to produce one or more outputs.

10. The system of claim 9, wherein the computing instance is hosted in the cloud.

11. The system of claim 9, wherein the computing instance uses one or more data models in analyzing the capture data from a lateral view, capture data from an overhead view, and geolocation data to produce one or more outputs.

12. The system of claim 11, wherein the one or more data models are chosen from the list consisting of a dormant buds model, an optimal buds model, a growth stage model, a flowers per tree model, a fruit after thinning model, a fruit size model, a yield prediction model, and a chemical thinning model.

13. The system of claim 11 , wherein the analysis of the capture data from a lateral view, capture data from an overhead view, and geolocation data is done by data fusion to produce a multidimensional map, and the multidimensional map is analyzed by the one or more data models.

14. The system of claim 9, wherein the computing instance analyzes the capture data from a lateral view to identify identified objects and mark them with one or more bounding boxes.

15. The system of claim 14, wherein the bounding boxes are integrated into the multidimensional map.

16. The system of claim 14, wherein the identified objects are chosen from the list consisting of trees, bushes, branches, buds, and fruits.

17. A method of analyzing an agricultural field, comprising: gathering data with internet of things sensors; fusing data into a multidimensional map; processing the multidimensional map with data models to produce outputs; and outputting the outputs.

18. The method of claim 17, wherein the internet of things sensors are one or more capture units and one or more geolocation units.

19. The method of claim 18, wherein the one or more capture units and one or more geolocation units are disposed on both a land vehicle and an aerial vehicle.

20. The method of claim 17, wherein fusing data into a multidimensional map and processing the multidimensional map with data models to produce outputs are done by a computing instance located in the cloud.

Images