System and method for distributed processing in crop assessment

Abstract

According to an aspect, there is provided systems and methods for assessing crops. The crop assessment includes, for each given plant of a plurality of plants, capturing an image of the given plant, processing the captured image to generate at least one output for the given plant using a crop assessment device, transmitting the at least one output for the given plant to a user device, and processing the at least one output for the given plant and additional information to generate at least one crop metric for the given plant using the user device.

Description

SYSTEM AND METHOD FOR DISTRIBUTED PROCESSING IN CROP ASSESSMENTCROSS-REFERENCE

[0001] This application claims priority from US Provisional Patent Application No. 63 / 730756, entitled “SYSTEM AND METHOD FOR DISTRIBUTED PROCESSING IN CROP ASSESSMENT”, filed on December 11 , 2024, the entire contents of which are incorporated herein by reference.FIELD

[0002] Embodiments of the present disclosure generally relate to the field of crop assessment and / or management, and more specifically, embodiments relate to devices, systems, and methods for distributed processing in crop assessment.BACKGROUND

[0003] Agricultural production, such as fruit production, is harder than ever. Aggressive weather events can hurt or outright destroy crop yields. Pests and disease can spread through crops and severely reduce crop yields. All of these may impact the volume of crops produced (e.g., the total number of suitable crop units produced and / or a reduction in the size of suitable crop units).

[0004] Growers collect data manually to determine farm strategies. In this regime, the plants are inspected by agents in the field making data collection a manually intensive process. Only a small percentage of plants can be analysed. This may impact the accuracy and precision of crop estimates based on the few plants actually analyzed.

[0005] Post-harvest costs and sales are based on these estimates which may lead to misalignment of supply and demand. This may further misallocate food resources which may lead to wasted food in some places and too little food provided elsewhere.

[0006] Improvement in the area of crop assessment is desirable.SUMMARY

[0007] According to an aspect, there is provided a computer-implemented system for assessing crops. The system includes a crop assessment device and a user device. The crop assessment device includes at least one imaging sensor and a processing subsystem that includes one or more processors and one or more memories coupled with the one or more processors. The processing subsystem is configured to cause the system to, for each given plant of a plurality of plants, capture an image of the given plant using the at least one imaging sensor, process the captured image to generate at least one output for the given plant, and transmit the at least one output for the given plant to a user device. The user device includes one or more processors and one or more memories coupled with the one or more processors. The user device has access to additional information. The user device is configured to cause the system to, for each at least one output of the given plant of the plurality of plants, process the at least one output for the given plant and the additional information to generate at least one crop metric for the given plant.

[0008] In some embodiments, the additional information includes additional information associated with the given plant.

[0009] In some embodiments, the crop assessment device further includes a geo-positioning subsystem. The processing subsystem is further configured to estimate a location of the given plant based on a signal from the geo-positioning subsystem.

[0010] In some embodiments, the additional information is based in part on the location of the given plant.

[0011] In some embodiments, the crop assessment device is provided on a vehicle.

[0012] In some embodiments, the additional information includes at least one of historic data and plant metadata.

[0013] In some embodiments, the historic data includes historic predictions on a per-plant basis and the plant metadata includes at least one of local plant spacing, plant age or year planted, root stalk, trellis type, plant variety, plant type, weather data, or soil data.

[0014] In some embodiments, the system further controls an action of the crop assessment device or another system component based on the at least one output or the at least one crop metric.

[0015] In some embodiments, the at least one crop metric measures at least one of a crop unit number, a crop unit volume, a crop load, crop surface area, crop diameter, or crop density.

[0016] In some embodiments, the at least one crop metric comprises at least one of a disease metric, a pest metric, a blemish metric, a canopy density / volume metric, a nutrient deficiency metric, a colour grading metric, a quality grading, a ripeness grading, a trunk or branch cross- sectional area metric, a glucose (brix) metric, a dry matter metric, or a water content metric.

[0017] In some embodiments, the at least one output includes at least one preliminary crop metric.

[0018] In some embodiments, the crop assessment device is a first crop assessment device and the system further includes a second crop assessment device with an available connection to the user device. The first crop assessment device is configured to transmit the at least one output for the given plant to the user device via the second crop assessment device.

[0019] In some embodiments, the crop assessment device is a first crop assessment device and the system further includes a second crop assessment device with available memory. The first crop assessment device is configured to transmit data to the second crop assessment device to be stored in the free memory of the second crop assessment device.

[0020] In some embodiments, the crop assessment device is a first crop assessment device and the system further includes a second crop assessment device with an available connection to the user device or a server. The first crop assessment device is configured to transmit data to the second crop assessment device to transmit the data to the user device or the server through the available connection.

[0021] According to an aspect, there is provided a method of assessing crops. The method includes, for each given plant of a plurality of plants, capturing an image of the given plant, processing the captured image to generate at least one output for the given plant using a crop assessment device, transmitting the at least one output for the given plant to a user device, and processing the at least one output for the given plant and additional information to generate at least one crop metric for the given plant using the user device.

[0022] In some embodiments, the additional information includes additional information associated with the given plant.

[0023] In some embodiments, the method further includes estimating a location of the given plant based on a signal from a geo-positioning subsystem.

[0024] In some embodiments, the additional information is based in part on the location of the given plant.

[0025] In some embodiments, the crop assessment device is provided on a vehicle.

[0026] In some embodiments, the additional information includes at least one of historic data and plant metadata.

[0027] In some embodiments, the historic data includes historic predictions on a per plant basis and the plant metadata includes at least one of local plant spacing, plant age or year planted, root stalk, trellis type, plant variety, plant type, weather data, or soil data.

[0028] In some embodiments, the method further includes controlling an action of the crop assessment device or another system component based on the at least one output or the at least one crop metric.

[0029] In some embodiments, the at least one crop metric measures at least one of a crop unit number, a crop unit volume, a crop load, crop surface area, crop diameter, and crop density.

[0030] In some embodiments, the at least one crop metric includes at least one of a disease metric, a pest metric, a blemish metric, a canopy density / volume metric, a nutrient deficiency metric, a colour grading metric, a quality grading, a ripeness grading, a trunk or branch cross- sectional area metric, a glucose (brix) metric, a dry matter metric, a water content metric.

[0031] In some embodiments, the at least one output includes at least one preliminary crop metric.

[0032] In some embodiments, transmitting the at least one output for the given plant to the user device includes transmitting the at least one output for the given plant from a first crop assessment device through a second crop assessment device to the user device.

[0033] Many further features and combinations thereof concerning embodiments described herein will appear to those skilled in the art following a reading of the instant disclosure.DESCRIPTION OF THE FIGURES

[0034] In the figures, embodiments are illustrated by way of example. It is to be expressly understood that the description and figures are only for the purpose of illustration and as an aid to understanding.

[0035] Embodiments will now be described, by way of example only, with reference to the attached figures, wherein in the figures:

[0036] FIG. 1A is a block schematic diagram of a distributed crop assessment system, according to some embodiments.

[0037] FIG. 1B is a block schematic diagram of a distributed crop assessment system with a plurality of crop assessment devices, according to some embodiments.

[0038] FIG. 2A is a perspective view of a crop assessment device, according to some embodiments.

[0039] FIG. 2B is a front view of the crop assessment device of FIG. 2A, according to some embodiments.

[0040] FIG. 2C is a side view of the crop assessment device of FIG. 2A, according to some embodiments.

[0041] FIG. 2D is a rear view of the crop assessment device of FIG. 2A, according to some embodiments.

[0042] FIG. 3 illustrates the crop assessment device of FIG. 2A mounted on a vehicle, according to some embodiments.

[0043] FIG. 4 is a block schematic diagram of the system of the crop assessment device of FIG. 2A, according to some embodiments.

[0044] FIG. 5 is a block schematic diagram of the interfaces of the crop assessment device of FIG. 2A, according to some embodiments.

[0045] FIG. 6 illustrates different information outputs provided by the crop assessment system, according to some embodiments.

[0046] FIG. 7 illustrates a dashboard output provided by the crop assessment system, according to some embodiments.

[0047] FIG. 8 is a process diagram of a method of assessing crops, according to some embodiments.

[0048] FIG. 9 is a schematic diagram of computing device, according to some embodiments.DETAILED DESCRIPTION

[0049] Provided herein are systems and methods to aid in crop assessment. In particular, provided herein is a system that is capable of scanning through a crop (e.g., an orchard) and assessing the crop (e.g., size of crop unit (e.g., fruit), number of crop units per plant, etc.) in a distributed fashion. The system is configured to travel through a crop (e.g., mounted on a vehicle) and scan the crops for information. Such a system may use machine learning methods to identify individual plants and the crop units associated with each of the plants. The system may further be configured to obtain the location data of the plants and provide geographic representations (such as a 2D map) of the crops with information related to each of the plants provided thereon.

[0050] In some embodiments, such systems may be advantageous because the information from the crops can be obtained semi-automatically (without manual inspection by agents) and can be carried out on most or all of the plants in the crop as opposed to only a few. This may help in developing more effective farming strategies than could be developed if only a few plants were assessed. This may enhance the accuracy and precision of crop estimates. Post-harvest costs and sales that are based on these enhanced estimates may lead to better alignment of supply and demand. This may further properly allocate food resources and reduce food waste.

[0051] Further, in some embodiments, it may be beneficial to distribute the processing of the data between different components (e.g., a suitable combination of crop assessment devices and user devices or servers). For example, a crop assessment device may carry out a first part of an analysis of the crops or an initial analysis of the crops on a per plant basis and a user device may carry out a second part of the analysis or a more refined analysis with the benefit of additional information about the crops or external factors. Crop assessment devices may not be equipped to store or receive information on a per-plant basis or relevant context information (e.g., local climate or other environmental data) while in the field (e.g., too little memory to store all the information and / or no reliable internet connection while in the field). Accordingly, configuring thecrop assessment device to carry out an assessment based on the information retrieved from, for example, a captured image of the plant without additional context may enable the crop assessment device to provide a preliminary assessment (that may benefit a user in the field) while still providing a more refined assessment using additional information as that data is available. Furthermore, some metrics may be suitable to derive from captured images alone (e.g., from a preliminary assessment) while others may benefit from added context (e.g., so processing occurs in a secondary assessment).

[0052] Further, in some embodiments, the crop assessment devices can be configured to share information and computing resources between themselves. For example, multiple crop assessment devices may be deployed in the field to image the crops. This imaging process can be memory intensive (e.g., recording video). The crop assessment device can be configured to monitor its data stores (e.g., its memory) and transmit data between itself and other crop assessment devices (or other user devices) to optimize the amount of storage available. This may be useful particularly if the devices are configured to take additional or more detailed images of certain crops under certain trigger conditions (e.g., if it detects the presence of pests on a plant, the device may be configured to stop at that plant to investigate further or may take images with higher fidelity).

[0053] As a further example, multiple crops assessment devices may be configured to share CPU or GPU resources or other processing resources for analyzing data. For example, when undertaking CPU intensive processes (e.g., the crop assessment device is carrying out a realtime action necessitating a high amount of processing power such as approaching and positioning a camera to take specific photos), a crop assessment device may use the computing power of a nearby crop assessment device to carry out the any preliminary crop assessment. Such sharing may be extended to GPU resources as well (e.g., to make use of the parallel structure of the GPU). Such resource sharing may enable load balancing between crop assessment devices in the field and / or parallel processing.

[0054] Furthermore, structuring the system as described herein can help protect the privacy of clients. The crop assessment device may be provided as a bespoke device that may be shared between different crops (potentially owned by competitors). Structuring the data analysis in this way ensures that some confidential or proprietary crop information (such as, for example, historic crop predictions, pre-plant basis metadata, etc.) need not be processed by the crop assessmentdevice which further means that it need not be provided to the crop assessment device and hence it need not be wiped from the crop assessment device prior to providing to a different client.

[0055] FIG. 1A is a block schematic diagram of a distributed crop assessment system 10, according to some embodiments.

[0056] The crop assessment system 10 may include a crop assessment device 100 and a user device 300 that may be in communication through a network 330. The crop assessment system 10 may collectively be configured to image (or otherwise survey) crops and provide output on the crops on a per-plant basis. The crop assessment device 100 may be configured to provide a preliminary assessment on a per-plant basis based on the captured images of the crops and the user device 300 may be configured to provide a secondary assessment on a per-plant basis taking into account additional information about the crop and / or the plants.

[0057] The crop assessment device 100 may be configured with imaging sensors 104 (e.g., cameras) and a processor 117. The crop assessment device 100 may also optionally have memory and a network interface. The crop assessment device 100 may be a device configured to traverse a crop. For example, the crop assessment device 100 may be configured to be carried through a crop by a worker or on a vehicle. In some embodiments, the crop assessment device 100 may be configured within a vehicle. In some embodiments, the crop assessment device 100 may part of an autonomous vehicle.

[0058] The imaging sensors 104 can be configured to capture images of the crops. The imaging sensors 104 may be configured to capture images in the, for example, visible light spectrum or infrared light spectrum. The imaging sensors 104 may be configured to capture images at specific rates. The rate of image capture may be based in part on the speed with which the crop assessment device 100 is travelling through the crops. In some embodiments, the rate of image capture may be configured to prevent image capture artefacts such as motion blur. In some embodiments, multiple image frames taken from different angles (e.g., structure from motion) can be combined to estimate the position of the plant in combination with the geo-positioning subsystem.

[0059] The processor 117 may be equipped with a preliminary plant assessor 150. The processor 117 may be, for example, any type of general-purpose microprocessor or microcontroller, a digital signal processing (DSP) processor, an integrated circuit, a fieldprogrammable gate array (FPGA), a reconfigurable processor, a programmable read-only memory (PROM), or any combination thereof.

[0060] The preliminary plant assessor 150 may be configured to process the captured images of the plants and assess the plants. In some embodiments, the preliminary plant assessor 150 may use the captured images directly. In some embodiments, the preliminary plant assessor 150 may use outputs from a model trained to extract features from the captured images (e.g., a preliminary computer vision model used to detect plants in the captured images). In some embodiments, the preliminary plant assessor 150 can generate an output. In some embodiments, the preliminary plant assessor 150 can output features or a feature vector (which may not itself represent a known crop metric). A feature vector may be output when an intermediate hidden state / feature of a neural network model is passed to the user device 300 and further processed there (by, for example, secondary plant assessor 308) to generate the crop metric. In some embodiments, the output may be features extracted by the preliminary plant assessor 150 (which may or may not be indicative of crop yield on their own). In some embodiments, the output can be a preliminary crop metric. In some embodiments, the preliminary crop metric may be indicative of crop yield (e.g., they can be used as a preliminary assessment of the performance of the crop on a plant-by-plant basis). In some embodiments, the output may include both crop metrics (which may not require further processing) and outputs to be processed by a secondary crop assessor. In some embodiments, preliminary plant assessor 150 may be a model trained using machine learning methods. In some embodiments, the preliminary plant assessor 150 may be algorithmic or rules based. In some embodiments, the preliminary plant assessor 150 may be trained in tandem with any preliminary image processing models and any subsequent plant assessor models. In some embodiments, the preliminary plant assessor 150 may be trained separate from other models used in the system.

[0061] In some embodiments, these metrics (or raw data) may be associated with the position data of the plant. In such embodiments, the positional data may also be transmitted to the user device 300.

[0062] In some embodiments, the crop assessment device 100 may send the raw data to the user device 300 (or elsewhere for processing) rather than undertaking preliminary plant assessment with the preliminary plant assessor 150.

[0063] In some embodiments, the crop assessment device 100 may use the one or more outputs to control the operation of the crop assessment device 100 (or other external devices). For example, the crop assessment device 100 may determine that more images are required to assess the plant or images at a different angle. As another example, the crop assessment device 100 may implement active lighting based on the quality of the captured images. As another example, the crop assessment device 100 may activate an onboard sprayer or spreader based on the outputs (e.g., if they are a preliminary crop metric) to apply water, fertilizer, nutrients, or other chemical additives (e.g., pesticides, herbicides, insecticides, fungicides, growth moderators, etc.). Such a configuration may be useful to enable the crop assessment device to dynamically react to real-time situations without requiring further refined processing on the user device 300.

[0064] The memory may store relevant models (e.g., any computer vision models and the preliminary plant assessment model for the preliminary plant assessor 150). The memory may include a suitable combination of any type of computer memory that is located either internally or externally such as, for example, random-access memory (RAM), read-only memory (ROM), compact disc read-only memory (CDROM), electro-optical memory, magneto-optical memory, erasable programmable read-only memory (EPROM), and electrically-erasable programmable read-only memory (EEPROM), Ferroelectric RAM (FRAM) or the like.

[0065] Any network interface can enable the crop assessment device 100 to communicate with other components, to exchange data with other components, to access and connect to network resources, to serve applications, and perform other computing applications by connecting to a network 330 (or multiple networks 330) capable of carrying data including the Internet, Ethernet, plain old telephone service (POTS) line, public switched telephone network (PSTN), integrated services digital network (ISDN), digital subscriber line (DSL), coaxial cable, fiber optics, satellite, mobile, wireless (e.g. Wi-Fi, WiMAX, Bluetooth), SS7 signaling network, fixed line, local area network, wide area network, and others, including any combination of these.

[0066] The user device 300 may be a personal computer, a mobile device, or other computing device, including a portion of another device or apparatus such as, e.g., processing module or a control module that is part of another piece of equipment. In some embodiments, the user device 300 may be any computing device that includes additional data that supplements data available at the crop assessment device 100. Such additional data may include, for example, data specific to a particular user, a particular client, or a particular field. In some embodiments, the user device 300 may be a computing device that the user uses in the field. In some embodiments, the userdevice 300 may be a device configured within a vehicle that the user is operating. The user device 300 may include a processor 302, a memory 304, and an I / O interface 306. The user device 300 may also optionally have a network interface.

[0067] The processor 302 may be equipped with a secondary plant assessor 308 and a mobile client / app 402. The processor 302 may be, for example, any type of general-purpose microprocessor or microcontroller, a digital signal processing (DSP) processor, an integrated circuit, a field programmable gate array (FPGA), a reconfigurable processor, a programmable read-only memory (PROM), or any combination thereof.

[0068] The secondary plant assessor 308 may be configured to process the output (or parts thereof) of the preliminary plant assessor 150 (e.g., extracted features, a preliminary crop metric, etc.). In some embodiments, the user device 300 may be configured to receive raw data from the crop assessment device 100 and may process that data in conjunction with the additional data to provide a crop metric which may be indicative of crop yield. In some embodiments, the secondary plant assessor 308 may process the output along with additional information to provide a better output indicative of crop yield. For example, the additional information can be information associated with the given plant being inspected or the location being inspected. In some embodiments, the additional information may include historic information (e.g., historic predictions on a per-plant basis, etc.) or plant metadata (e.g., local plant spacing, plant age or year planted, root stalk, trellis type, plant variety, plant type, weather data, soil data, etc.). Such additional information may be confidential or proprietary in nature (e.g., historic performance of the crops or specific plant type (e.g., specific cultivars)). In some embodiments, the additional information can be entered by the user into the user device 300. In some embodiments, the additional information can be retrieved from the memory 304. In some embodiments, the additional information can be retrieved from an external source such as a server 350. In some embodiments, the additional information may be retrieved from the server 350 while the connection is good (e.g., ahead of assessment in the field) and saved to the memory 304 for retrieval during crop assessment (e.g., when there may not be a reliable connection with the server 350). In some embodiments, the user device 300 may retrieve additional information in real-time.

[0069] In some embodiments, two or more crop assessment devices 100 may have images of the same plant (e.g., two devices 100 in adjacent rows each imaging one side of the plant). In such embodiments, the user device 300 may be configured to compile the output from the preliminary plant assessors 150 (and any raw data and / or additional information) in each of thedevices 300 to generate a singular crop metric for the given plant. In such embodiments, the user device 300 may provide a destination for all outputs from the preliminary plant assessors 150 configured to compile the data in a coherent and unified manner (e.g., unifying outputs related to the same plant). In some embodiments, the location of the plant may be used to associate and compile different outputs from the same plant. In some embodiments, timestamps (e.g., from imaging data) associated with the outputs can be used to associate and compile different outputs from the same plant.

[0070] In some embodiments, the crop metric may be indicative of crop yield. In some embodiments, the crop metric may measure a crop unit number (e.g., number of fruits), a crop unit volume (e.g., size of fruits), a crop load (e.g., number of fruits per tree), crop surface area (e.g., canopy leaf area), crop diameter (e.g. fruit diameter), crop density (e.g., blossom cluster density), and / or any diseases or other abnormalities on the crop units.. In some embodiments, the crop metric may include a disease metric, a pest metric, a blemish metric, a canopy density / volume metric, a nutrient deficiency metric, a colour grading metric, a quality grading, a ripeness grading, a trunk or branch cross-sectional area metric, a glucose (brix) metric, a dry matter metric, a water content metric. In some embodiments, the crop metric is estimated for each plant based on, for example, its overall appearance in the image and various features extracted from the image.

[0071] In some embodiments, secondary plant assessor 308 may be a model trained using machine learning methods. In some embodiments, the secondary plant assessor 308 may be algorithmic or rules based. In some embodiments, the secondary plant assessor 308 may be trained in tandem with any preliminary models (e.g., machine vision and / or the preliminary plant assessor 150). In some embodiments, the secondary plant assessor 308 may be trained separately from other models used in the system. In some embodiments, the preliminary plant assessor 150 and the secondary plant assessor 308 may share the same model, but more information is delivered to the secondary plant assessor 308 enabling it to produce a more precise and / or accurate prediction.

[0072] The mobile client / app 402 may configure the user device 300 to provide outputs to the user and to enable the user device 300 to communicate with the crop assessment device 100 and other components of the system described herein. The mobile client / app 402 may also moderate the upload and download of information from the user device 300 (e.g., upload ofoutputs and / or crop metrics and other information such as additional information and the download of models for the secondary plant assessor 308).

[0073] The memory 304 may store relevant models (e.g., the secondary plant assessment model for the secondary plant assessor 308) or any additional information. The memory 304 may include a suitable combination of any type of computer memory that is located either internally or externally such as, for example, random-access memory (RAM), read-only memory (ROM), compact disc read-only memory (CDROM), electro-optical memory, magneto-optical memory, erasable programmable read-only memory (EPROM), and electrically-erasable programmable read-only memory (EEPROM), Ferroelectric RAM (FRAM) or the like.

[0074] Each I / O interface 306 enables the user device 300 to interconnect with one or more input devices, such as a keyboard, mouse, camera, touch screen and a microphone, or with one or more output devices such as a display screen and a speaker. The user may use such devices to input, for example, additional information into the user device 300 or may receive updates by way of such devices.

[0075] Any network interface can enable the user device 300 to communicate with other components, to exchange data with other components, to access and connect to network resources, to serve applications, and perform other computing applications by connecting to a network 330 (or multiple networks 330) capable of carrying data including the Internet, Ethernet, plain old telephone service (POTS) line, public switched telephone network (PSTN), integrated services digital network (ISDN), digital subscriber line (DSL), coaxial cable, fiber optics, satellite, mobile, wireless (e.g. Wi-Fi, WiMAX, Bluetooth), SS7 signaling network, fixed line, local area network, wide area network, and others, including any combination of these.

[0076] In some embodiments, multiple crop assessment devices 100 are transmitting data to one user device 300. This may be used, for example, in a situation where multiple autonomous crop assessment devices 100 are canvassing the crops while one user is in the field. In some embodiments, multiple crop assessment device 100 may image the same plant (e.g., from different sides) and this data is aggregated in a user device 300 (or server 350) to provide a holistic metric for the plant. In some embodiments, the crop assessment devices 100 may transmit their information directly to a server 350 for secondary crop assessment. Such a configuration may still protect confidential and proprietary information because the crop assessment devices 100 may not have all the additional information about the crops.

[0077] The network 330 may include a network (or multiple networks) capable of carrying data including the Internet, Ethernet, plain old telephone service (POTS) line, public switched telephone network (PSTN), integrated services digital network (ISDN), digital subscriber line (DSL), coaxial cable, fiber optics, satellite, mobile, wireless (e.g., Wi-Fi, WiMAX, Bluetooth), SS7 signaling network, fixed line, local area network, wide area network, and others, including any combination of these. Furthermore, components of the network may communicate through different modalities. For example, the crop assessment device 100 and the user device 300 may communicate with each other through a wired or short-range network 330 while the crop assessment device 100 and the user device 300 may communicate with any servers 350 (either directly or indirectly) through a long-range network 330 (e.g., over an Internet connection).

[0078] In some embodiments, the system 10 may further include a server 350. The server 350 may be, for example, a server storing confidential or proprietary information related to the crops. The server 350 may optionally be any computing device that stores such information (e.g., a computing device within the farm property, a centralized storage hub storing information across multiple industrial-scale farms, etc.). The server 350 may include a memory 352. The server 350 may optionally include a processor, an I / O interface, and a network interface.

[0079] The memory 352 may store the additional information related to the crops. The memory 352 may optionally store models associated with the operation of the system 10. The memory 352 may include, for example, historic data 354 and plant metadata 356. The memory 352 may include a suitable combination of any type of computer memory that is located either internally or externally such as, for example, random-access memory (RAM), read-only memory (ROM), compact disc read-only memory (CDROM), electro-optical memory, magneto-optical memory, erasable programmable read-only memory (EPROM), and electrically-erasable programmable read-only memory (EEPROM), Ferroelectric RAM (FRAM) or the like.

[0080] The information stored on the memory 352 may be confidential or proprietary in nature. The information may include, for example historic data 354 and plant metadata 356. The historic data 354 may include historic predictions of plant performance on a plant-by-plant basis, historic weather results, historic crop yields, or other information. The plant metadata 356 may include local plant spacing, plant age or year planted, root stalk, trellis type, plant variety, plant type, weather data, soil data, or other information generally relating to the present crop and its configuration. Such information may be retrieved by the user device 300 to be processed with the features (or raw data) to generate the crop metric which may be indicative of crop yield. Dividingthe processing in this manner ensures that the confidential or proprietary additional information need not be stored on a crop assessment device 100 at any time ensuring that this data is not inadvertently leaked to competitors or third parties if the crop assessment device 100 is shared. Using this additional information can help generate predictions on a per-plant basis which are more accurate and / or precise.

[0081] The processor may be, for example, any type of general-purpose microprocessor or microcontroller, a digital signal processing (DSP) processor, an integrated circuit, a field programmable gate array (FPGA), a reconfigurable processor, a programmable read-only memory (PROM), or any combination thereof. In some embodiments, the crop assessment device 100 may send the outputs directly to a server 350 (for example, after assessing the crops) and the server 350 may be configured with a secondary plant assessor 308 to generate the crop metric as described above. This may obviate the need for the user device while still keeping the confidential and proprietary information off of the crop assessment device 100.

[0082] Each I / O interface enables the server 350 to interconnect with one or more input devices, such as a keyboard, mouse, camera, touch screen and a microphone, or with one or more output devices such as a display screen and a speaker.

[0083] Any network interface can enable the server 350 to communicate with other components, to exchange data with other components, to access and connect to network resources, to serve applications, and perform other computing applications by connecting to a network 330 (or multiple networks 330) capable of carrying data including the Internet, Ethernet, plain old telephone service (POTS) line, public switched telephone network (PSTN), integrated services digital network (ISDN), digital subscriber line (DSL), coaxial cable, fiber optics, satellite, mobile, wireless (e.g. Wi-Fi, WiMAX, Bluetooth), SS7 signaling network, fixed line, local area network, wide area network, and others, including any combination of these.

[0084] In operation, some embodiments of the crop assessment device 100 may travel through a crop. The imaging sensors 104 may capture one or more images of the crops. The imaging sensors 104 may be configured to generate a stream of images (e.g., live feed). The imaging sensors 104 may make use of infrared image capture to obviate the hindrance caused by occlusions to the crop units. The images may be processed by the processor 117 on the crop assessment device 100 to generate an output. The output may be transmitted to a user device 300 that processes the output along with any additional information to generate the crop metricwhich may be indicative of crop yield. The images may be analyzed using models trained with machine learning to identify individual plants (e.g., based on the trunks of trees) and to identify the crop units thereon. The models may be trained to assess the number of crop units, the volume of crop units, the crop load (e.g., the number of crop units per plant), the crop surface area (e.g., canopy leaf area), the crop diameter (e.g. fruit diameter), and the crop density (e.g., blossom cluster density). The models may also be configured to detect other features such as canopy density, the presence of pests and / or disease, nutrient deficiencies, blemishes (and percentage of crop units with blemishes), and colour grading. In some embodiments, the system 10 may further be configured to assess the full crop for further insights (e.g., detection of pests via regions of underperforming crops). The system 10 may take the information pulled from the images and associate it with the locations of the plant (e.g., as determined by the positioning system). This may all be combined to produce a 2D or 3D map of the crop with the crop metrics super imposed thereon.

[0085] In some embodiments, the system 10 may be configured to generate a crop metric indicative of crop yield. The crop assessment device 100 may further be configured to use the positioning system antenna and receiver 108 to determine the position of the plants that the crop assessment device 100 is currently investigating and associate the outputs (e.g., preliminary crop metrics or other metrics) with the location of the plant. Advantages of this approach can provide localized information about the crops. Such information might be useful to identify regions of the crop which are more productive or less productive than expected (e.g., due to soil conditions, disease, etc.). In some embodiments, the positional information of the plants may be associated with the metrics provided by the system 10. In such embodiments, the metrics may be superimposed or otherwise mapped onto an output to provide the user with an overview of the whole crop’s performance to date. In some embodiments, the system 10 may further be configured to provide overview metrics for the whole crop based on the metrics collected for each of the plants (e.g., average number of crop units per plant, average size of crop units, diseased or abnormal regions, etc.).

[0086] The output may be used for farming strategies such as dormant pruning, pest / disease detection and thinning. The output may also be used for yield prediction and post-harvest cost optimization. It may further be used for program planning.

[0087] The outputs may give a prediction for the yield of the crop as it presently exists. The output may further be configured to forecast the yield of the crop when it is harvested.

[0088] In some embodiments, the system 10 may include and / or control the operation of additional devices. For example, the system 10 may include or control agricultural equipment (e.g., variable-rate spraying, robotic pruning, picking, etc.), industrial equipment (e.g., factory automation, mine monitoring, etc.), etc.

[0089] The system 10 may also train models for the crop assessment devices 100 and user devices 300 is a distributed fashion. For example, the system 10 may collect all of the crop assessment devices’ 100 outputs and check the preliminary crop metric against the final crop metric to further refine the models generating the preliminary crop metric to generate more accurate preliminary results. The data may be collected from the system 10 to further train models against future results (e.g., future detected performance of the crops). Other model training paradigms are envisioned. This may be useful to implement refined models among a fleet of crop assessment devices 100 and / or among many different client user devices 300.

[0090] Model optimization can be carried out using, for example, federated learning. For example, model optimization may be carried out in a distributed fashion, where models (or parts thereof) are optimized locally on different crop assessment devices 100, based on the (e.g., local) data available to each of the devices 100. Refined models or model updates could be communicated to other devices 100 or to a (e.g., central) user device 300 or server 350, be combined into a new, joint model, and transferred back to all or some devices 100.

[0091] In some embodiments, the model used in the preliminary plant assessor 150 may generate an output. The output may include metrics indicative of crop yield or other metrics and metrics which may then be input into the secondary plant assessor 308 for further modelling. The metrics indicative of crop yield or other metrics predicted by the preliminary plant assessor 150 may be sufficiently refined without additional processing while the metrics which may then be input into the secondary plant assessor 308 may benefit from additional information to generate refined results. The secondary plant assessor 308 may process the additional outputs with the additional information (e.g., historic data, plant metadata, etc.) to provide more refined predictions about certain aspects of the crops. In some embodiments, the system 10 may output a bundle of metrics and some may be suitable to generate with the crop assessment device 100 and some may be suitable to generate on the user device 300.

[0092] In some embodiments, the models used in the preliminary plant assessor 150 and the secondary plant assessor 308 may use different model paradigms. In some embodiments, onemay be rules-based while another uses machine learning models. In some embodiments, the preliminary plant assessor 150 may be a machine learning model trained to provide generic outputs without taking additional information (e.g., historic data, plant metadata, etc.) into its predictions. In some embodiments, it may provide an output. The output may include one or more of metrics indicative of crop yield and metrics to be further processed. In some embodiments, the secondary plant assessor 308 may be a machine learning model trained to provide bespoke outputs taking into account additional information (e.g., historic data, plant metadata, etc.) into its predictions.

[0093] In some embodiments, the crop assessment device 100 may be configured to transmit data such as the imaging data to an external system (such as the user device 300 or the server 350) over the network 330, for example, to free up space within the memory of the crop assessment device 100, to make the data available for further processing, to enable processing load balancing between crop assessment devices 100, and / or to enable parallel processing. This may be useful if the crop assessment device 100 has stopped to take further imaging data of a particular plant or has taken higher resolution images of a plant (i.e., higher memory load) because the preliminary crop metric triggered such a protocol. Rebalancing the memory in this way may be useful because it can ensure that the crop assessment device 100 can complete its survey without running out of memory.

[0094] The crop assessment devices 100 may be configured to transmit data (e.g., raw data, imaging data, etc.) to another crop assessment device 100 with space left in its memory so that it need not return to a docking station to upload data before continuing its assessment. T ransmission of raw data between crop assessment devices 100 may occur automatically to send all information to the crop assessment device 100 with the most storage space left (i.e., data is tunneled to the crop assessment device 100 with the most memory regardless of how much memory is available in the originating crop assessment device 100) or transmission of raw data may only occur when the originating crop assessment device 100 is at or nearing capacity (e.g., data is stored on the originating crop assessment device 100 until it runs out of memory or is close to running out of memory, and is then transmitted to another crop assessment device 100 with storage space). Configuring the system 100 in this way can balance the storage utilization across different crop assessment devices 100 and can achieve longer operation overall (making sure no single device 100 runs out of space before the others).

[0095] The redistribution of data within the system 10 in this way may be beneficial if there would otherwise be a bottleneck within the system 10. For example, crop assessment devices 100 may be configured to transmit raw data to a server 350 however, the maximum upload speed may be slower than the generation of new raw data or the connection between the crop assessment device 100 and the server 350 may be interrupted. Accordingly, the crop assessment device 100 can transmit some data to other crop assessment devices 100 (which may themselves upload the data) leaving space so that it can continue its survey.

[0096] In some embodiments, the crop assessment devices 100 may be configured to transmit raw data through the user device 300. In such embodiments, upload of data may be limited by the connection strength that the user device 300 has with the server 350. Accordingly, the crop assessment devices 100 may be configured to redistribute data between the crop assessment devices 100 so that they do not need to halt their survey and wait for the user device 300 to transmit the data.

[0097] In some embodiments, the transmission of the output from the preliminary plant assessor 150 to the user device 300 may not be sufficiently fast to keep memory usage low (e.g., if there is an interruption in the connection between the crop assessment device 100 and the user device 300). In such embodiments, the crop assessment device 100 may send the output from its preliminary plant assessor 150 to another crop assessment device 100 until the user device 300 is ready to receive the output.

[0098] In some embodiments, the transmission of data between the crop assessment devices 100 may be beneficial to enable load balancing between the crop assessment devices 100 and to enable parallel processing. For example, a crop assessment device 100 may transmit raw imaging data to a neighbouring crop assessment device 100 so the crop assessment device 100 can carry out a computationally taxing maneuver (e.g., positioning a sprayer in a location to spray a pest and not spray anything else while trying to account for environmental variables like pest movement and movement of the plant from the wind) and the neighbouring crop assessment device 100 can process using that preliminary plant assessor 150. Such transmission may balance processing taking place on the CPU, GPU, or other processing resources (e.g., dedicated processors for embedded applications with, for example, efficient implementations of neural network operations or other matrix arithmetic, etc.).

[0099] FIG. 1 B is a block schematic diagram of a distributed crop assessment system 10 with a plurality of crop assessment devices 100a and 100b, according to some embodiments.

[0100] As described above, in some embodiments there may be a plurality of crop assessment devices 100 (illustrated in the figure as crop assessment device 100a and crop assessment device 100b, each of which being an example of a crop assessment device 100) configured to survey the crops. In such embodiments, the crop assessment devices 100 may be configured to transmit data as between the crop assessment devices 100 as is determined optimum for the system 10.

[0101] In the figure, the crop assessment devices 100a and 100b may be the same as one another. In some embodiments, the crop assessment devices 100a and 100b may be different. For example, a fleet of crop assessment devices 100 may be contain crop assessment devices 100 of different generations or makes that may nonetheless still be compatible with the crop assessment system 10. In such embodiments, the outputs from the crop assessment devices 100 may be configured to be processed by the user device 300 in a manner to render one complete set of analyses (a coherent and unitary crop metric). Such crop assessment devices 100 may be advantageous as the fleet can be expanded or upgraded without being hindered by a variety of disparate crop assessment device 100 models.

[0102] The signal connection within the crops may not always be strong. Accordingly, a first crop assessment device 100a may fall out of a long-distance network 330b with the user device 300 (or the server 350) but may still be within the short-range network 330a distance of a second crop assessment device 100b. In such embodiments, the first crop assessment device 100a may be configured to transmit the output from preliminary crop assessor 150 to the user device 300 through the second crop assessment device 100b. In such configurations, the first crop assessment device 100a may append the output (and any other data such as the raw data, e.g., raw imaging data) with a device identifier to keep the source of the data clear. The first crop assessment device 100a may transmit the output to the second crop assessment device 100b through the short-range network 330a. The second crop assessment device 100b may then transmit this output to the user device 300 (or a server 350) using the second network 330b. The second network 330b may also be a short-range network (e.g., if the second crop assessment device 100b is closer to the user device 300 than the first crop assessment device 100a is to the user device 300) or the second network 330b may be a long-distance network (e.g., if the secondcrop assessment device 100b is receiving a better wireless network connection from the longdistance network).

[0103] In some embodiments, the user device 300 may periodically transmit a signal through the network 330 to the crop assessment devices 100. Each of the crop assessment devices 100 may then transmit a signal to each nearby crop assessment device 100. Based on the origin of the signal, each crop assessment device 100 may identify the shortest route to the user device 300 through the nearby crop assessment devices 100 (provided there is a possible connection).

[0104] For example, each crop assessment device 100 may track whether it is in direct communication with the user device 300 and, if not, track the nearest crop assessment device 100. The system 10 may be configured to transmit outputs of the preliminary crop assessor 150 through the crop assessment devices 100 (and not back through any source of the output) until the preliminary crop metric reaches the user device 300.

[0105] As another example, crop assessment devices 100 may include a user device distance variable along with a device identifier in their respective memory. All crop assessment devices 100 in direct communication with the user device 300 may write a 1 into the user device distance variable and the device identifier for the user device 300 into their respective memory. These crop assessment devices 100 may then transmit an incremented user device distance variable (e.g., 2) with the device identifier of itself to all nearby crop assessment devices 100. Any crop assessment devices 100 that are not in direct communication with the user device 300 may write this 2 as the user device distance variable, and transmit that incremented variable (e.g., 3) along with its own device identifier to nearby crop assessment devices 100. Any crop assessment device 100 with no user device distance variable or with a user device distance variable that is greater than the one received from a nearby crop assessment device 100 can overwrite the user device distance variable with the incoming one and the incoming device identifier. In following the protocol, the crop assessment devices 100 will generate the shortest pathway through the crop assessment devices 100 to the user device 300. In some embodiments, this protocol may take signal strength between the crop devices 100 into account (to provide a short and stable path).

[0106] In some embodiments, the crop assessment devices 100 may also be configured to transmit raw data (e.g., imaging data) to the user device 300 or the server 350 through such a network hopping configuration.

[0107] According to an aspect, there is provided a computer-implemented system 10 for assessing crops. The system 10 includes a crop assessment device 100 and a user device 300. The crop assessment device 100 includes at least one imaging sensor 104 and a processing subsystem that includes one or more processors 117 and one or more memories coupled with the one or more processors 117. The processing subsystem is configured to cause the system 10 to, for each given plant of a plurality of plants, capture an image of the given plant using the at least one imaging sensor 104, process the captured image to generate at least one output for the given plant, and transmit the at least one output for the given plant to a user device 300. The user device 300 includes one or more processors 302 and one or more memories 304 coupled with the one or more processors 302. The user device 300 has access to additional information. The user device 300 is configured to cause the system 10 to, for each at least one output of the given plant of the plurality of plants, process the at least one output for the given plant and the additional information to generate at least one crop metric for the given plant.

[0108] In some embodiments, the additional information includes additional information associated with the given plant.

[0109] In some embodiments, the crop assessment device 100 further includes a geopositioning subsystem. The processing subsystem is further configured to estimate a location of the given plant based on a signal from the geo-positioning subsystem.

[0110] In some embodiments, the additional information is based in part on the location of the given plant.

[0111] In some embodiments, the crop assessment device 100 is provided on a vehicle 200.

[0112] In some embodiments, the additional information includes at least one of historic data and plant metadata.

[0113] In some embodiments, the historic data includes historic predictions on a per-plant basis and the plant metadata includes at least one of local plant spacing, plant age or year planted, root stalk, trellis type, plant variety, plant type, weather data, or soil data.

[0114] In some embodiments, the system 10 further controls an action of the crop assessment device 100 or another system component based on the at least one output or the at least one crop metric.

[0115] In some embodiments, the at least one crop metric measures at least one of a crop unit number, a crop unit volume, a crop load, crop surface area, crop diameter, or crop density.

[0116] In some embodiments, the at least one crop metric comprises at least one of a disease metric, a pest metric, a blemish metric, a canopy density / volume metric, a nutrient deficiency metric, a colour grading metric, a quality grading, a ripeness grading, a trunk or branch cross- sectional area metric, a glucose (brix) metric, a dry matter metric, or a water content metric.

[0117] In some embodiments, the at least one output includes at least one preliminary crop metric.

[0118] In some embodiments, the crop assessment device 100 is a first crop assessment device 100a and the system 10 further includes a second crop assessment device 100b with an available connection to the user device 300. The first crop assessment device 100a is configured to transmit the at least one output for the given plant to the user device 300 via the second crop assessment device 100b.

[0119] In some embodiments, the crop assessment device 100 is a first crop assessment device 100a and the system 10 further includes a second crop assessment device 100b with available memory. The first crop assessment device 100a is configured to transmit data to the second crop assessment device 100b to be stored in the free memory of the second crop assessment device 100b.

[0120] In some embodiments, the crop assessment device 100 is a first crop assessment device 100a and the system 10 further includes a second crop assessment device 100b with an available connection to the user device 300 or a server 350. The first crop assessment device 100a is configured to transmit data to the second crop assessment device 100b to transmit the data to the user device 300 or the server 350 through the available connection.

[0121] In some embodiments, transmitting data (e.g., the at least on output for the given plant) via the second crop assessment device 100b can include transmitting the data from the first crop assessment device 100a to the second crop assessment device 100b and the second crop assessment device 100b transmitting the data to its destination (e.g., the user device 300 or server 350). In some embodiments, the second crop assessment device 100 b may transmit the data to the destination indirectly (e.g., it may transmit it to a subsequent one or more second crop assessment devices 100b that may transmit the data to the destination). In some embodiments,transmitting data via the second crop assessment device 100b can include relaying the data through a series of one or more second crop assessment devices 100b.

[0122] FIG. 2A is a perspective view of a crop assessment device 100, according to some embodiments. FIG. 2B is a front view of the crop assessment device 100 of FIG. 2A, according to some embodiments. FIG. 2C is a side view of the crop assessment device 100 of FIG. 2A, according to some embodiments. FIG. 2D is a rear view of the crop assessment device 100 of FIG. 2A, according to some embodiments.

[0123] FIG. 2A-2D provide an exemplary device 100 that can be used to image the crops and assess them. These figures also provide the optional utility of providing active lighting for the crops. While the active lighting is described, assessment devices 100 that lack the active lighting functionality are included in the scope of the description herein.

[0124] The crop assessment device 100 can include, for example, light emitters 102, imaging sensors 104, a protective glass 106, a positioning system antenna and receiver 108, visual indicators and antenna 110, heatsink 112, an enclosure 114, a display 118a, a button 118b (the display 118a and the display 118b together forming part of the buttons, LED, display 118), an ethernet connector 125, and a power connector 121. The crop assessment device 100 can be configured to survey and assess crops. For example, the crop assessment device 100 can use active lighting to image the crops and process the image data to generate one or more outputs. Such outputs may be indicative of crop yield. This information may be combined with information from the positioning system (e.g., a geo-positioning subsystem) to associate plants with their locations and generate outputs for further processing.

[0125] The optional light emitters 102 may be configured to emit active lighting. The light emitted by the light emitters 102 may be visible light. The light emitted by the light emitters 102 may be infrared light. The light emitted by the light emitters 102 may be selected to transmit through leaves or other occlusions to the crop units (e.g., fruits). In some embodiments, the light may be selected because it specifically reflects or is absorbed and reemitted by the crops in question. IR (or near-IR) light absorption and emission of plant tissue can be linked to its photosynthetic activity, and can be used to assess plant / tissue health and detect various issues. For example, the commonly used normalized difference vegetation index (NDVI) compares the relative amounts of red and near-IR light to assess photosynthetic activity or plant health. Tissue damage can be detected in near-IR, as it can lead to reduced near-IR emission / reflection. Inindoor settings, near-IR / IR light might be desirable, as it doesn’t interfere with other (visible) light sources, so the system can acquire consistent images regardless of other light sources present in the room.

[0126] The crop assessment device 100 may have a plurality of light emitters 102. Each of the plurality of light emitters 102 may emit the same range of wavelengths of light or each light emitters 102 may emit a different range of wavelengths of light. Different light emitters 102 may be activated based on the type of crop being assessed (e.g., where different ranges may be optimal for different types of crops). While FIG. 2A shows four configurations of light emitters 102 and imaging sensors 104, other configurations and numbers of light emitters 102 and / or imaging sensors 104 can be used. For example, the front panel can be modified to accept more or less light emitters 102. As another example, there may be no light emitters 102.

[0127] In some embodiments, the light emitters 102 that emit near-IR can be positioned in the central two positions and an imaging sensor 104 capable of receiving near-IR in the right-most position. Not aligning the light source with the imaging sensor 104 can mean better contrast in certain settings (if the light comes from an angle, it can be easier to see surface structure and distinguish between objects due to the slight shadow).

[0128] In some embodiments, the light emitters 102 may activate based on energy or imaging needs. For example, the crop assessment device 100 may be configured to determine whether the images received from the imagining sensors 104 meet a quality threshold and if not the crop assessment device 100 may be configured to adjust the lighting provided by the light emitters 102 until the images meet the quality threshold (or some other criteria is satisfied). In some embodiments, the crop assessment device 100 can control the direction of the light emitters 102.

[0129] In some embodiments, the light emitters 102 can be turned on and off with precise timing (e.g., for 1 ms when capturing a frame to overlap with the camera exposure interval). Pulsed operation can allow the system to put all the energy into the duration of the frame exposure and can achieve much higher “effective” output power, similar to a conventional camera flash.

[0130] In some embodiments, the light emitters 102 may include, for example, three 850 nm light emitters. Using 850 nm light may only affect the near-IR images and leave the RGB images unaffected. This can enable applications where the reflected / transmitted radiation of different light sources to be measured simultaneously, but independently (e.g., using natural sunlight and artificial 850 nm light).

[0131] The imaging sensors 104 can be configured to capture images of the crops. The imaging sensors 104 may be configured to capture images in the, for example, visible light spectrum or infrared light spectrum. The imaging sensors 104 may be configured to capture images at specific rates. The rate of image capture may be based in part on the speed with which the crop assessment device 100 is travelling through the crops. In some embodiments, the rate of image capture may be configured to prevent image capture artefacts such as motion blur.

[0132] In some embodiments, there may be a plurality of imaging sensors 104. In some embodiments, each of the plurality of imaging sensors 104 may generally be oriented to capture images in the same direction. In some embodiments, each of the plurality of imaging sensors 104 may be configured to capture images at different orientations. In some embodiments, each of the plurality of imaging sensors 104 may be sensitive to different spectral bands. For example, each of the imaging sensors 104 may be sensitive to the spectral band provided by one of the light emitters 102 or sensitive to a spectral band reflected or reemitted by fruit from one of the light emitters 102.

[0133] In some embodiments, the at least one imaging sensor 104 is configured to detect multispectral light. Such embodiments may include a standard sensor without any filter (it can detect any light compatible with its spectral response characteristics, i.e. , any visible and near-IR light for a conventional complementary metal-oxide-semiconductor (CMOS) sensor).

[0134] In some embodiments, the crop assessment device 100 may include three different imaging units comprised of imaging sensors 104 and optics including a wide-angle, colour (RGB Bayer filter array), a telephoto, colour (RGB Bayer filter array), and a wide-angle, monochrome (near-IR bandpass filter). The framerates of the imaging sensors 104 may be about 1 to about 160 fps.

[0135] The light emitters 102 and the imaging sensors 104 may be encased in the crop assessment device 100 behind protective glass 106. The protective glass 106 may be fabricated from any material (e.g., plastic, plexiglass, glass, etc.) suitable for field use. The protective glass 106 may generally be transmissible for the spectral bands for use with the crop assessment device 100 (e.g., intended to be emitted by the light emitters 102 and / or received by the imaging sensors 104).

[0136] The crop assessment device 100 may also include a positioning system antenna and receiver 108. The positioning system antenna and receiver 108 may provide, for example,position, velocity, and time for the crop assessment device 100. For example, the positioning system antenna and receiver 108 may provide the position of the crop assessment device 100 in an absolute context (e.g., as a global positioning system) or in a relative context (e.g., it may be configured to determine the location of the crop assessment device 100 within the crop itself). The positioning system antenna and receiver 108 may, for example, be part of a global navigation satellite system (GNSS) such as a global positioning system (GPS). This may include a multiband (and / or multi-constellation) GNSS antenna and receiver. It may also include RTK technology for cm-level positioning.

[0137] In some embodiments, the crop assessment device 100 may be configured to generate one or more outputs. The crop assessment device 100 may further be configured to use the positioning system antenna and receiver 108 to determine the position of the plants that the crop assessment device 100 is currently investigating and associate the outputs (or raw data or other metric) with the location of the plant. Advantages of this approach can provide localized information about the crops. Such information might be useful to identify regions of the crop which are more or less productive than expected (e.g., due to soil conditions, disease, etc.).

[0138] The visual indicators and antenna 110 can provide information about the status of the crop assessment device 100 and can communicate with external devices. The visual indicator and antenna 110 may include, for example a WiFi and / or Bluetooth antenna to allow for radio communication. The visual indicators and antenna 110 may include LEDs to serve as indicators that can signal the system status, such as “normal operation" or “recording”.

[0139] The heatsink 112 can optionally provide cooling and heat dissipation to the device 100. In some embodiments, the components of the crop assessment device 100 may produce heat and for proper and continued operation of the crop assessment device 100, it may be beneficial to remove the heat generated by these components. Further, the crop assessment device 100 may be configured to operate under a variety of different conditions which may include in hot weather and the heatsink 112 may be helpful to dissipate heat in such situations.

[0140] The enclosure 114 may encase some of the components of the crop assessment device 100. For example, it may encase the light emitters 102 along with the imaging sensors 104. It may do so with the protective glass 106. The enclosure 114 may be waterproof and ruggedized for field use.

[0141] The display 118a may provide output information or status information to the user. For example, it may provide the user with error messages to help the user troubleshoot an issue with the crop assessment device 100.

[0142] The button 118b may provide the user with a means of inputting commands to the device 100.

[0143] The ethernet connector 125 may enable an ethernet connection for uploading and / or downloading information to and from the crop assessment device 100 (e.g., to the user device 300 and / or the server 350). This can help download information from the device 100 once it has passed through the crop. Other data transfer methods are also possible.

[0144] The power connector 121 can enable power to connect to the crop assessment device 100 to power the system.

[0145] The crop assessment device 100 may be modular. For example, the crop assessment device 100 illustrated has four separate imaging sensor 104 subsystems. Each of these subsystems can be individually modified, for example, with the same or different light emitters 102 and / or imaging sensors 104. Several different light emitters 102 can be placed inside the front panel and aligned with the imaging sensors for optimal illumination of the scene. In some embodiments, the light emitters 102, the imaging sensors 104, and other sensors can be reconfigured and swapped easily. For example, two identical near-IR imaging sensors 104 could be used in two slots for standard stereo vision applications. As a further example, arbitrary bandpass / lowpass / highpass filters with multiple monochrome sensors could be used to capture some arbitrary bands simultaneously. In some embodiments, different light emitters 102 of different wavelengths may simultaneously be controlled. In some embodiments, the imaging sensors 104 can, but do not need to be synchronized with each other and / or the light emitters 102.

[0146] In some embodiments, the crop assessment device 100 may have more or less input modalities than described above. For example, in some embodiments, the crop assessment device 100 can include an inertial measurement unit (IMU) to measure acceleration and / or rotation of the crop assessment device 100, a temperature sensor to measure the temperature, a radar to measure, for example, depth, light detections and ranging (LiDAR) sensor to measure, for example, depth, an ambient light sensor to measure ambient light, a humidity sensor to measure humidity. Crop assessment devices 100 with one or more differing input modalities maybe capable of using the information from these modalities to further control the crop assessment devices 100 or to provide additional information when generating one or more outputs or sending to user device 300. For example, the ambient light sensor may be used to determine the number of light emitters 102 to activate and at what intensity.

[0147] The crop assessment device 100 can provide a specific temporal resolution for the control signal (e.g., imaging sensor 104 control, light emitter 102 control, etc.). The resolution may be optimized based on the expected field use of the crop assessment device 100 (e.g., expected speed of movement through the crops). In some embodiments, the crop assessment device 100 can provide 83 ns resolution for the control signal.

[0148] In some embodiments, the crop assessment device 100 can quickly adjust the light source to capture certain properties of an imaged object (e.g. defects of a certain kind), based on the scene / object imaged. In some embodiments, the crop assessment device 100 may use the outputs to determine imaging or light adjustments. In some embodiments, the crop assessment device 100 may adjust imaging and lighting parameters based on the condition of the plant imaged, its surroundings, or location to capture specific properties or to improve the image quality. In some embodiments, the crop assessment system 10 can be used as a real-time control unit for agricultural equipment, including light-based equipment, such as laser-based weeding systems, UV-based pest control, etc. In some embodiments, the crop assessment device 100 may be mounted to a picking robot and can be used to dynamically illuminate the scene if an object of interest is present (turning off the light otherwise to minimize interference with other robots). In some embodiments, the crop assessment device 100 can be used for high-resolution spectral imaging of fast-moving objects with a single or small number of imaging sensors 104. For example, depending on the object, capturing an image using either spectral band A or band B could be desirable. In such cases, to maximize image quality, a monochrome sensor could be used with light sources of different wavelengths and depending on the shape (or another feature) of the object inferred in a first frame (1), light source A or B can be conditionally switched on for the next frame (2). This may be similar to a fast-moving camera with static scenes.

[0149] FIG. 3 illustrates the crop assessment device 100 of FIG. 2A mounted on a vehicle 200, according to some embodiments.

[0150] In some embodiments, the crop assessment device 100 may be configured to be mounted on a vehicle 200. In such embodiments, the crop assessment device 100 may beinteroperable with a mount 202 which can hold the crop assessment device 100 in an orientation to carry out crop assessment as the vehicle 200 travels through the crops. In some embodiments, the crop assessment device 100 may be built directly into a vehicle 200. In some embodiments, the crop assessment device 100 may be built into an autonomous vehicle. In such embodiments, the crop assessment device 100 may control or provide input into the control unit of the autonomous vehicle.

[0151] FIG. 4 is a block schematic diagram of the system of the crop assessment device 100 of FIG. 2A, according to some embodiments.

[0152] In some embodiments, the crop assessment device 100 may include a real-time controller 116, main processor 117, buttons, LEDs, display 118, LISB-C port 120, GB Ethernet 122, non-volatile memory express storage 124, WiFi / BT 126 and antenna 128, LTE modem 130 and antenna 132, IMU 138, and GNSS receiver 134 and active antenna 136. These internal components can be used by the crop assessment device 100 to control its operation and to output information to external devices / signal statuses to a user.

[0153] The real-time controller 116 may be configured to control the operation of the light emitters 102 and the imaging sensors 104. For example, the real-time controller 116 may activate or deactivate components based on present system needs. The real-time controller 116 may further be configured to modify the intensity and / or timing of the light emitters 102 based on current lighting or other considerations.

[0154] The main processor 117 may be configured to interact with and / or control other components of the crop assessment device 100. For example, the data from one or multiple imaging sensors 104 can be transferred to an image processor within the main processor 117, which, using neural networks or other computer vision techniques can extract features from the images. Based on inferred properties, the processor 117 can send control signals to a real-time control unit 116, which can precisely control the timing of the imaging sensors 104, light emitters 102, and other periphery such as external agricultural equipment, robotic systems, processing lines etc. Signals can be locked to time signals received from positioning systems, and additional sensor data might be used as inputs (e.g. accelerometer or position data).

[0155] The main processor 117 may also include the preliminary plant assessor 150. The preliminary plant assessor 150 may make use of the features extracted from the captured images (or the captured images themselves) to assess the crops and provide one or more outputs. Asdescribed in greater detail above, the outputs may be preliminary crop metrics indicative of crop yield or it may be an output to be fed into a secondary plant assessor 308 to which may then provide a crop metric indicative of yield. The outputs may be used to determine adjustments made by the crop assessment device 100. The preliminary plant assessor 150 may make use of machine learning models or algorithmic models depending on the configuration of the crop assessment system 10.

[0156] In some embodiments, the real-time controller 116 and the main processor 117 can be implemented in the same component rather than being separated as illustrated in the Figures.

[0157] The buttons, LEDs, and display 118 (which may include, but may not be limited to, display 118a and button 118b) may form a part of the visual indicators and antenna 110. For example, the buttons, LEDs, and display 118 may serve as indicators that can signal the system status, such as “normal operation" or “recording” or enable the user to input things directly into the crop assessment device 100.

[0158] The USB-C port 120 may provide a means to power the crop assessment device 100. The USB-C port 120 may provide a means to upload / download data onto / off of the crop assessment device 100. The USB-C port 120 may provide a port to interoperate with further devices (e.g., the user device 300). The USB-C port 120 may be another type of port which may provide one or more of the foregoing functions.

[0159] The GB Ethernet 122 may provide the main processor 117 with the ability to transmit Ethernet frames, for example, at a rate of a gigabit per second. Other possible data link layer protocol data units are also usable.

[0160] The non-volatile memory express storage 124 can store instructions for carrying out one of more operations of the crop assessment device 100 (e.g., the preliminary plant assessor 150). The non-volatile memory express storage 124 may also be used to store information generated during operation of the device 100, for example, prior to download onto an external device. The non-volatile memory express storage 124 may alternatively be a different type of storage device.

[0161] The WiFi / BT 126 and antenna 128 may be configured to enable the crop assessment device 100 to use WiFi and / or Bluetooth. The WiFi / BT 126 and antenna 128 may be configured to transmit information to or from an external device (e.g., the user device 300). For example, the WiFi / BT 126 and antenna 128 may be configured to receive control instructions from, for example,the user device 300. As another example, the WiFi / BT 126 and antenna 128 may be configured to transmit data to an external computing device to offer the user a real-time view of the data generated thus far during an assessment. The crop assessment device 100 may use other shortdistance wireless communication modalities. Such short-range communication may be beneficial to provide updates on the user device 300 and to transmit data to the user device 300 for further processing by the user device 300.

[0162] The LTE modem 130 and antenna 132 can provide the crop assessment device 100 with long-term evolution wireless broadband communication. This may enable the crop assessment device 100 to transmit data over longer distances (e.g., to an external device in a remote location). The crop assessment device 100 may use other long-distance wireless communication modalities.

[0163] The I MU 138 may provide the crop assessment device 100 with an inertial measurement unit. The IMU 138 may provide the crop assessment device 100 with information regarding its acceleration and orientation. Such information may be helpful to orient the images captured by the imaging sensor 104. The information may further assist the crop assessment device 100 to map outputs onto the appropriate plants or in the appropriate region in the crop on a readout.

[0164] The GNSS receiver 134 and active antenna 136 may be used by the positioning system to determine the location of the crop assessment device 100. This can be used by the crop assessment device 100 to correspond metrics indicative of crop yield to specific regions in the crop or to particular plants in the crop. This may be useful to provide particularized information about the crop. Other positioning system modalities are also possible.

[0165] In operation, the real-time controller 116 may control the light emitters 102 to emit active lighting (e.g., IR lighting) to illuminate the crops (if such functionality is present). The real-time controller 116 may image the crops using the imaging sensors 104. These images may be passed to the main processor 117 for analysis by, for example, preliminary plant assessor 150. The crop assessment device 100 may also track its location using the GNSS receiver 134 and active antenna 136 and may track the orientation of the imaging sensors 104 using the IMU 138. The images may be processed by the main processor 117 to identify plants in the image (for example, the device 100 may make use of machine learning models to detect the trunks of trees to identify plants) and use the positional data from the GNSS receiver 134 and active antenna 136 and / or IMU 138 to associate the plant with a position. The main processor 117 may further process theimages (for example using a model trained using machine learning) to determine one or more outputs using the preliminary plant assessor 150. By optionally using active lighting in the IR ranges, the crop assessment device 100 may be better able to determine crop yields as the IR may not be occluded by leaves or other plant debris that is not the crop unit (e.g., the fruit). Further, as described above, IR (or near-IR) light absorption and emission of plant tissue can be linked to its photosynthetic activity and may be helpful to determine plant health-related criteria (e.g., tissue damage, photosynthetic activity, pest, disease, nutrient deficiency) and may help distinguish between objects that are hard to distinguish using just the visible range.

[0166] Once the metrics are computed, then the crop assessment device 100 may be configured to transmit the metrics to the user device 300. The crop assessment device 100 may also transmit the results to a user via the buttons, LEDs, and display 118. Alternatively, the crop assessment device 100 may store the information on the non-volatile memory express storage 124 for later download using the LISB-C 120. Alternately, the crop assessment device 100 may transmit the information using the WiFi / BT 126 and antenna 128, the LTE modem 130 and antenna 132, and / or with GB Ethernet 122.

[0167] FIG. 5 is a block schematic diagram of the interfaces of the crop assessment device 100 of FIG. 2A, according to some embodiments.

[0168] The data coming out of the crop assessment device 100 can be routed through the mobile client / app 402 and transferred to cloud infrastructure for further processing, visualization, aggregation, etc. This mobile client / app 402 may be configured using an app backend 404. In some embodiments, the data may also be directly uploaded to a cloud infrastructure as well. The crop assessment device 100 may further be updated by this infrastructure as well.

[0169] In some embodiments, the crop assessment device 100 may be configured to generate one or more outputs and transmit those to the user device 300 (e.g., to the mobile client / app 402). The user device 300 may further process these outputs to generate a crop metric. The crop metric may be indicative of crop yield. The crop assessment device 100 may also transmit other data to the user device 300 (e.g., location data). In some embodiments, the crop assessment device 100 may be configured to upload some or all data (e.g., collected raw data, the outputs, location data, etc.) to a cloud infrastructure.

[0170] For data, the information may be downloaded from the crop assessment device 100 via the data upload 406. This may be via any of the data transfer methods described above of viaother data transfer methods. The data upload 406 may transmit the data to a data warehouse 416. The mobile client / app 402 may also provide data to and receive data from the data warehouse 416 via the app backend 404. Such configurations may enable data warehouse 416 to receive the crop metric from the user device 300.

[0171] The data warehouse 416 may transmit the data to an analytics backend 414 to conduct analytics on the data. Analysis of the data may be carried out, for example, in the data warehouse 416 and / or the analytics backend 414. Analysis of the data could include monitoring of trends (e.g., incorporating historic data), growth curves / rates, yield estimation, data aggregation across multiple locations, conversion of the data into action plans (e.g. for farm staff) or input prescriptions (e.g., for sprayer control). The analytics backend 414 may push the data (and its analysis) to an analytics dashboard 412 for review by a user.

[0172] In some embodiments, the data in the data warehouse 416 may be fed into machine learning models 418. These machine learning models 418 may use the obtained data to further refine the models. In particular, if the data include error reports by users, these errors may be used for further training of any machine learning models used to identify the plants and / or the crop units (e.g., identify the trunks of trees and fruit growing thereon). These refined models may be transferred into the model management 420. The model management 420 may push new models to the crop assessment device 100 and / or to the mobile client 402 via model deployment 410. Model development may be particularly useful when an initial model was trained using limited training data or the model is being adapted to a new crop or spectral band used in active lighting. Furthermore, this management may be useful when implementing fleets of crop assessment devices 100.

[0173] FIG. 6 illustrates different information outputs provided by the crop assessment system 10, according to some embodiments.

[0174] As described above, the crop assessment system 10 may be configured to generate metrics indicative crop yields associated with specific plants and / or locations in the crops. As such, the crop assessment device 100 or the user device 300 may be configured to produce, for example, 2D maps of the crops with the metrics placed thereon. In some embodiments, the metrics may be illustrated by means of colour, gradient (e.g., greyscale), or other visually identifiable feature (see, e.g., image 602). For example, the crop unit count may be represented on a scale between two colours wherein purely the first colour means no or few crop units whilepurely the second colour means a maximum or threshold crop unit count. In such configurations, the colour of the location of the plant may indicate the count for that plant thereby providing an intuitively digestible output for a user (e.g., regions of poor or good production will all illuminate with the same colour; see for example images 602 and 604). In some embodiments, the metrics may be illustrated by means of size of a shape such as a circle (see, e.g., image 604). In some embodiments, the division may be associated with a unit of ground rather than individual plants (to control for densely packed plants).

[0175] FIG. 7 illustrates a dashboard output provided by the crop assessment system 10, according to some embodiments.

[0176] The dashboard output 700 may include a control panel 701 , global statistics 702, a crop map 704, plant distribution of crop unit count 706, and a crop unit volume distribution 708. The dashboard 700 may further provide more information based on the information pulled using the crop assessment device 100 and / or the user device 300. For example, the dashboard output 700 may compare information pulled from different crops to provide comparative insights.

[0177] The control panel 701 may enable the user to select which crop they want to view and at what time. For example, the user may be able to select a particular crop (e.g., fruit) season, month, scan date, farm name, farm block, section name, variety, stage name, and specific parameters (e.g., size units).

[0178] The global statistics 702 may provide statistics by type of crop unit (e.g., type of fruit or subtype of fruit (e.g., variety of apple)). The statistics provided may include the average crop unit per plant, the rows scanned, the plants scanned, the crop units counted, and the average crop unit size (e.g., volume or diameter).

[0179] The crop map 704 may be a 2D geographic output which maps the metrics predictive of crop yield onto a map of the crop to give geographic insights. The crop map 704 may further color code (or otherwise visually indicate) different types of crops. Such displays are described above with reference to FIG. 6.

[0180] The plant distribution of crop unit count 706 may provide information relating to the distribution of crop unit counts over plants. This may be helpful to establish a visual means of comparing different crops to each other (e.g., using the shape of the graph to ascertain further insights about different crops).

[0181] The crop unit volume distribution 708 may provide information relating to the distribution of crop unit volumes or sizes (or any spatial measurement of the crop unit). This may be helpful to establish a visual means of comparing different crops to each other (e.g., using the shape of the graph to ascertain further insights about different crops).

[0182] Other possible statistics which the crop assessment device 100 or the user device 300 may be configured to provide include one or more of canopy volume / density, trunk or branch cross-sectional area, (fruit) color, quality grading, ripeness, glucose content (brix), dry matter, water content, disease statistics, pest statistics, nutrient deficiencies.

[0183] In some embodiments, it may further be possible to select subset regions of the crop map and obtain the above statistical analyses for the subregion. Such implementations may be helpful when comparing different subregions of the crops to one another (e.g., to determine whether a region may be diseased, nutrient deficient, etc.).

[0184] FIG. 8 is a process diagram of a method of assessing crops 800, according to some embodiments.

[0185] According to an aspect, there is provided a method of assessing crops 800. The method 800 includes, for each given plant of a plurality of plants, capturing an image of the given plant (block 802), processing the captured image to generate at least one output for the given plant using a crop assessment device (block 804), transmitting the at least one output for the given plant to a user device (block 806), and processing the at least one output for the given plant and additional information to generate at least one crop metric for the given plant using the user device (block 808).

[0186] In some embodiments, the method 800 may generate outputs or crop metrics (blocks 804 and 808) as each plant is measured. In some embodiments, the method 800 may wait until all (or some subset of) plants are analyzed before generating the outputs or crop metrics (blocks 804 and 808).

[0187] In some embodiments, the additional information includes additional information associated with the given plant.

[0188] In some embodiments, the method 800 further includes estimating a location of the given plant based on a signal from a geo-positioning subsystem.

[0189] In some embodiments, the additional information is based in part on the location of the given plant.

[0190] In some embodiments, the crop assessment device is provided on a vehicle.

[0191] In some embodiments, the additional information includes at least one of historic data and plant metadata.

[0192] In some embodiments, the historic data includes historic predictions on a per plant basis and the plant metadata includes at least one of local plant spacing, plant age or year planted, root stalk, trellis type, plant variety, plant type, weather data, or soil data.

[0193] In some embodiments, the method 800 further includes controlling an action of the crop assessment device or another system component based on the at least one output or the at least one crop metric.

[0194] In some embodiments, the at least one crop metric measures at least one of a crop unit number, a crop unit volume, a crop load, crop surface area, crop diameter, and crop density.

[0195] In some embodiments, the at least one crop metric includes at least one of a disease metric, a pest metric, a blemish metric, a canopy density / volume metric, a nutrient deficiency metric, a colour grading metric, a quality grading, a ripeness grading, a trunk or branch cross- sectional area metric, a glucose (brix) metric, a dry matter metric, or a water content metric.

[0196] In some embodiments, the at least one output includes at least one preliminary crop metric.

[0197] In some embodiments, transmitting the at least one output for the given plant to the user device includes transmitting the at least one output for the given plant from a first crop assessment device through a second crop assessment device to the user device.

[0198] FIG. 9 is a schematic diagram of computing device 900, according to some embodiments.

[0199] The technical solution of embodiments may be in the form of a software product. The software product may be stored in a non-volatile or non-transitory storage medium, which can be a compact disk read-only memory (CD-ROM), a USB flash disk, or a removable hard disk. Thesoftware product includes a number of instructions that enable a computer device (personal computer, server, or network device) to execute the methods provided by the embodiments.

[0200] The embodiments described herein are implemented by physical computer hardware, including computing devices, servers, receivers, transmitters, processors, memory, displays, and networks. The embodiments described herein provide useful physical machines and particularly configured computer hardware arrangements. The embodiments described herein are directed to electronic machines and methods implemented by electronic machines adapted for processing and transforming electromagnetic signals which represent various types of information. The embodiments described herein pervasively and integrally relate to machines, and their uses; and the embodiments described herein have no meaning or practical applicability outside their use with computer hardware, machines, and various hardware components. Substituting the physical hardware particularly configured to implement various acts for non-physical hardware, using mental steps for example, may substantially affect the way the embodiments work. Such computer hardware limitations are clearly essential elements of the embodiments described herein, and they cannot be omitted or substituted for mental means without having a material effect on the operation and structure of the embodiments described herein. The computer hardware is essential to implement the various embodiments described herein and is not merely used to perform steps expeditiously and in an efficient manner.

[0201] For simplicity only one computing device 900 is shown but the crop assessment device 100, the user device 300, the server 350 (or greater system 10) may include more computing devices 900 operable by users to access remote network resources and exchange data. The computing devices 900 may be the same or different types of devices. The computing device 900 at least one processor, a data storage device (including volatile memory or non-volatile memory or other data storage elements or a combination thereof), and at least one communication interface. The computing device components may be connected in various ways including directly coupled, indirectly coupled via a network, and distributed over a wide geographic area and connected via a network (which may be referred to as “cloud computing”).

[0202] For example, and without limitation, the computing device may be a server, network appliance, set-top box, embedded device, computer expansion module, personal computer, laptop, personal data assistant, cellular telephone, smartphone device, LIMPC tablets, video display terminal, gaming console, electronic reading device, and wireless hypermedia device orany other computing device capable of being configured to carry out the methods described herein.

[0203] As depicted, computing device 900 includes at least one processor 902, memory 904, at least one I / O interface 906, and at least one network interface 908.

[0204] Each processor 902 may be, for example, any type of general-purpose microprocessor or microcontroller, a digital signal processing (DSP) processor, an integrated circuit, a field programmable gate array (FPGA), a reconfigurable processor, a programmable read-only memory (PROM), or any combination thereof.

[0205] Memory 904 may include a suitable combination of any type of computer memory that is located either internally or externally such as, for example, random-access memory (RAM), read-only memory (ROM), compact disc read-only memory (CDROM), electro-optical memory, magneto-optical memory, erasable programmable read-only memory (EPROM), and electrically- erasable programmable read-only memory (EEPROM), Ferroelectric RAM (FRAM) or the like.

[0206] Each I / O interface 906 enables computing device 900 to interconnect with one or more input devices, such as a keyboard, mouse, camera, touch screen and a microphone, or with one or more output devices such as a display screen and a speaker.

[0207] Each network interface 908 enables computing device 900 to communicate with other components, to exchange data with other components, to access and connect to network resources, to serve applications, and perform other computing applications by connecting to a network (or multiple networks) capable of carrying data including the Internet, Ethernet, plain old telephone service (POTS) line, public switched telephone network (PSTN), integrated services digital network (ISDN), digital subscriber line (DSL), coaxial cable, fiber optics, satellite, mobile, wireless (e.g. Wi-Fi, WiMAX, Bluetooth), SS7 signaling network, fixed line, local area network, wide area network, and others, including any combination of these.

[0208] Computing device 900 is operable to register and authenticate users (using a login, unique identifier, and password for example) prior to providing access to applications, a local network, network resources, other networks and network security devices. Computing devices 900 may serve one user or multiple users.

[0209] Applicant notes that the described embodiments and examples are illustrative and nonlimiting. Practical implementation of the features may incorporate a combination of some or all ofthe aspects, and features described herein should not be taken as indications of future or existing product plans. Applicant partakes in both foundational and applied research, and in some cases, the features described are developed on an exploratory basis.

[0210] In the foregoing, numerous references were made to fruit and the trees that bare them. The systems and methods described herein could be used on other crops. For example, the systems described herein may make use of light when can reflect off of tubers underground and assess them using similar strategies.

[0211] In some embodiments, the systems and methods described herein can be used to analyze produce post-harvest, for example, in a warehouse, on a truck, in a processing plant, etc. In some embodiments, the systems and methods described herein can be used to analyze produce crop-growth, for example, as seeds or pre-seedings.

[0212] Such systems may be configured to survey crops for example to assess seeds, preseedlings, or seedlings. Such systems may be configured to be mounted onto vehicles that survey warehouses and / or processing plants and may provide, for example, assessment of a harvest once in the warehouse or productivity of a processing plant. In some embodiments, such systems may be stationary and assess crops as they pass by (e.g., counting crops as they are loaded onto a vehicle).

[0213] The term “connected” or "coupled to" may include both direct coupling (in which two elements that are coupled to each other contact each other) and indirect coupling (in which at least one additional element is located between the two elements).

[0214] Although the embodiments have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the scope. Moreover, the scope of the present application is not intended to be limited to the particular embodiments of the process, machine, manufacture, composition of matter, means, methods and steps described in the specification.

[0215] Moreover, the scope of the present application is not intended to be limited to the particular embodiments of the process, machine, manufacture, composition of matter, means, methods and steps described in the specification. As one of ordinary skill in the art will readily appreciate from the disclosure, processes, machines, manufacture, compositions of matter, means, methods, or steps, presently existing or later to be developed, that perform substantiallythe same function or achieve substantially the same result as the corresponding embodiments described herein may be utilized. Accordingly, the appended claims are intended to include within their scope such processes, machines, manufacture, compositions of matter, means, methods, or steps.

[0216] As can be understood, the examples described above and illustrated are intended to be exemplary only.

Claims

WHAT IS CLAIMED IS:

1. A computer-implemented system for assessing crops, the system comprising: a crop assessment device and a user device; wherein the crop assessment device comprises: at least one imaging sensor; and a processing subsystem that includes one or more processors and one or more memories coupled with the one or more processors, the processing subsystem configured to cause the system to: for each given plant of a plurality of plants: capture an image of the given plant using the at least one imaging sensor; process the captured image to generate at least one output for the given plant; and transmit the at least one output for the given plant to a user device; and wherein the user device includes one or more processors and one or more memories coupled with the one or more processors, the user device having access to additional information, the user device configured to cause the system to: for each at least one output of the given plant of the plurality of plants, process the at least one output for the given plant and the additional information to generate at least one crop metric for the given plant.

2. The computer-implemented system of claim 1 , wherein the additional information comprises additional information associated with the given plant.

3. The computer-implemented system of claim 1 or 2, wherein the crop assessment device further comprises a geo-positioning subsystem, and wherein the processing subsystem is furtherconfigured to estimate a location of the given plant based on a signal from the geo-positioning subsystem.

4. The computer-implemented system of claim 3, wherein the additional information is based in part on the location of the given plant.

5. The computer-implemented system of any one of claims 1 to 4, wherein the crop assessment device is provided on a vehicle.

6. The computer-implemented system of any one of claims 1 to 5, wherein the additional information comprises at least one of historic data and plant metadata.

7. The computer-implemented system of claim 6, wherein: the historic data comprises historic predictions on a per-plant basis; and the plant metadata comprises at least one of local plant spacing, plant age or year planted, root stalk, trellis type, plant variety, plant type, weather data, or soil data.

8. The computer-implemented system of any one of claims 1 to 7, wherein the system further controls an action of the crop assessment device or another system component based on the at least one output or the at least one crop metric.

9. The computer-implemented system of any one of claims 1 to 8, wherein the at least one crop metric measures at least one of a crop unit number, a crop unit volume, a crop load, crop surface area, crop diameter, or crop density.

10. The computer-implemented system of any one of claims 1 to 9, wherein the at least one crop metric comprises at least one of a disease metric, a pest metric, a blemish metric, a canopy density / volume metric, a nutrient deficiency metric, a colour grading metric, a quality grading, a ripeness grading, a trunk or branch cross-sectional area metric, a glucose (brix) metric, a dry matter metric, or a water content metric.

11. The computer-implemented system of any one of claims 1 to 10, wherein the at least one output comprises at least one preliminary crop metric.

12. The computer-implemented system of any one of claims 1 to 11 , wherein the crop assessment device is a first crop assessment device and the system further comprises a second cropassessment device with an available connection to the user device and wherein the first crop assessment device is configured to transmit the at least one output for the given plant to the user device via the second crop assessment device.

13. The computer-implemented system of any one of claims 1 to 11 , wherein the crop assessment device is a first crop assessment device and the system further comprises a second crop assessment device with available memory and the first crop assessment device is configured to transmit data to the second crop assessment device to be stored in the free memory of the second crop assessment device.

14. The computer-implemented system of any one of claims 1 to 11 , wherein the crop assessment device is a first crop assessment device and the system further comprises a second crop assessment device with an available connection to the user device or a server and the first crop assessment device is configured to transmit data to the second crop assessment device to transmit the data to the user device or the server through the available connection.

15. A method of assessing crops, the method comprising: for each given plant of a plurality of plants: capturing an image of the given plant; processing the captured image to generate at least one output for the given plant using a crop assessment device; transmitting the at least one output for the given plant to a user device; and processing the at least one output for the given plant and additional information to generate at least one crop metric for the given plant using the user device.

16. The method of claim 15, wherein the additional information comprises additional information associated with the given plant.

17. The method of claim 15 or 16, further comprising estimating a location of the given plant based on a signal from a geo-positioning subsystem.

18. The method of claim 17, wherein the additional information is based in part on the location of the given plant.

19. The method of any one of claims 15 to 18, wherein the crop assessment device is provided on a vehicle.

20. The method of any one of claims 15 to 19, wherein the additional information comprises at least one of historic data and plant metadata.21 . The method of claim 20, wherein: the historic data comprises historic predictions on a per plant basis; and the plant metadata comprises at least one of local plant spacing, plant age or year planted, root stalk, trellis type, plant variety, plant type, weather data, or soil data.

22. The method of any one of claims 15 to 21 , further comprising controlling an action of the crop assessment device or another system component based on the at least one output or the at least one crop metric.

23. The method of any one of claims 15 to 22, wherein the at least one crop metric measures at least one of a crop unit number, a crop unit volume, a crop load, crop surface area, crop diameter, and crop density.

24. The method of any one of claims 15 to 23, wherein the at least one crop metric comprises at least one of a disease metric, a pest metric, a blemish metric, a canopy density / volume metric, a nutrient deficiency metric, a colour grading metric, a quality grading, a ripeness grading, a trunk or branch cross-sectional area metric, a glucose (brix) metric, a dry matter metric, a water content metric.

25. The method of any one of claims 15 to 24, wherein the at least one output comprises at least one preliminary crop metric.

26. The method of any one of claims 15 to 25, wherein transmitting the at least one output for the given plant to the user device comprises transmitting the at least one output for the given plant from a first crop assessment device through a second crop assessment device to the user device.

Images