A produce monitoring system and method

A portable computing device with switchable imaging modes and tailored machine learning models addresses the inefficiencies of manual produce inspection, enhancing accuracy and flexibility in monitoring agricultural produce parameters.

GB2641202APending Publication Date: 2025-11-26HARVESTEYE LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
GB2024001649
Authority / Receiving Office
GB · GB
Patent Type
Applications
Current Assignee / Owner
Filing Date
2024-02-07
Publication Date
2025-11-26

AI Technical Summary

Technical Problem

Traditional manual inspection of agricultural produce is labor-intensive, time-consuming, and prone to human error, making it inefficient for grading and categorization.

Method used

A portable computing device mounted to a bracket on a harvester captures images using a mounted camera, determines parameters with a machine learning model, and switches to a handheld mode for additional imaging and parameter determination, utilizing different models for each mode to enhance versatility and accuracy.

Benefits of technology

The system provides efficient, accurate, and versatile monitoring of agricultural produce parameters, reducing human error and increasing operational flexibility by combining harvester-mounted and handheld imaging modes with tailored machine learning models.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

A method of monitoring produce comprises mounting a portable computing device to a mounting bracket and initiating a mounted operation mode. In the mounted operation mode, a first image of first produ
Need to check novelty before this filing date? Find Prior Art

Description

FIELD The invention relates to systems and methods for monitoring produce, which may be agricultural produce such as potatoes, for example. Aspects of the invention also relate to a case or protective housing. BACKGROUND Produce, such as agricultural produce, can be monitored to determine one or more parameters associated with the produce. These parameters could include size, weight, colour, quality, or health, for example. The determined parameters can then be used to grade or categorise the produce. For example, potatoes can be graded or categorised by size. Traditionally, a producer, such as a farmer, would determine any parameters associated with their produce manually. This could be by a manual or physical inspection of the produce, carried out by the producer, for example. This process could involve manually sizing, weighing, or inspecting the colour, quality, or health of the produce, for example. These traditional processes are labour-intensive, time-consuming, and prone to human error. The present invention seeks to alleviate problems associated with the prior art. BRIEF DESCRIPTION OF THE INVENTION Disclosed is a method of monitoring produce, comprising: mounting a portable computing device to a mounting bracket and initiating a mounted operation mode; receiving, in the mounted operation mode, a first image, the first image being of first produce, at the portable computing device; determining, with the portable computing device, at least one parameter associated with the first produce based on the first image; demounting the portable computing device from the mounting bracket and initiating a second operation mode; capturing, after demounting the portable computing device, a second image, the second image being of second produce; receiving, in the second operation mode, the second image at the portable computing device; and determining, with the portable computing device, at least one parameter associated with the second produce based on the second image. The portable computing device may determine the at least one parameter using a machine learning model. The portable computing device may be communicatively connected to a mounted camera in the mounted operation mode. The mounted camera may be mounted to a part of a harvester. The portable computing device may be configured to control the mounted camera, and the method may include the portable computing device capturing images automatically in the mounted operation mode. The portable computing device may be communicatively connected to a second camera in the second operation mode. The portable computing device may be configured to control the second camera, and the method may include capturing an image in response to a user input in the second operation mode. The mounting bracket may be installed on a harvester. The portable computing device may be communicatively connected to a depth sensor and the method may include receiving depth data associated with the first and / or second image at the portable computing device. The method may further include transmitting the first and / or second image from the portable computing device to a remote device, and determining the at least one parameter for the first and / or second produce using the remote device. The remote device may be configured to determine the at least one parameter using a machine learning model. The machine learning model used by the remote device may be different to the machine learning model used by the portable computing device. The portable computing device may be configured to use a first machine learning model to determine the at least one parameter in the mounted operation mode and to use a second machine learning model to determine the at least one parameter in the second operation mode. The second machine learning model may be trained to identify features that the first machine learning model is not trained to identify. The features may include one or more parts of a user’s body. The second camera may be inbuilt into the portable computing device, and the mounted camera may be external to the portable computing device. The mounted camera may be demountable such that it can be used in the second operation mode as the second camera. Also disclosed is a system for monitoring produce, comprising: a portable computing device switchable between a mounted operation mode and a second operation mode, wherein the portable computing device is configured to be mounted to a mounting bracket in use in the mounted operation mode, and to be dismounted from the mounting bracket in use in the second operation mode, wherein the portable computing device is configured to receive an image of produce and to determine at least one parameter associated with the produce based on the received image. The portable computing device may determine the at least one parameter using a machine learning model. The system may further include a mounted camera communicatively connected to the portable computing device. The mounted camera may be mounted to a part of a harvester. The portable computing device may be configured to control the mounted camera to capture images automatically in the mounted mode. The system may further include a second camera, the portable computing device may be configured to control the second camera to capture an image in response to a user input in the second operation mode. The mounting bracket may be installed on a harvester. The portable computing device may be communicatively connected to a depth sensor and the portable computing device may be further configured to receive depth data associated with the image. The system may further include a remote device, the portable computing device may be configured to transmit the image from the portable computing device to the remote device, and the remote device may be configured to determine the at least one parameter. The remote device may be configured to determine the at least one parameter using a machine learning model. The machine learning model used by the remote device may be different to the machine learning model used by the portable computing device. The portable computing device may be configured to use a first machine learning model to determine the at least one parameter in the mounted operation mode and to use a second machine learning model to determine the at least one parameter in the second operation mode. The second machine learning model may be trained to identify features that the first machine learning model is not trained to identify. The features may include one or more parts of a user’s body. The second camera may be inbuilt into the portable computing device, and the mounted camera may be external to the portable computing device. The mounted camera may be demountable such that it can be used in the handheld operation mode as the second camera. The method may further include mounting the second camera such that the second camera has at least part of a separate transporter within its field of view, such that produce transported by the separate transporter passes through the field of view of the second camera. The second camera may be an inbuilt camera of the portable computing device. The second camera may be external to the portable computing device, and may be connected to the portable computing device by a wired or wireless communications link. The second operation mode may be a handheld operation mode in which the portable computing device is not mounted to a mounting bracket. Also disclosed is a method of monitoring produce, comprising: mounting a portable computing device to a first mounting bracket and initiating a mounted operation mode; receiving, in the mounted operation mode, a first image, the first image being of first produce, at the portable computing device; determining, with the portable computing device, at least one parameter associated with the first produce based on the first image; demounting the portable computing device from the first mounting bracket and initiating a second operation mode; mounting the portable computing device to a second mounting bracket; capturing, after mounting the portable computing device to the second mounting bracket, a second image, the second image being of second produce; receiving, in the second operation mode, the second image at the portable computing device; and determining, with the portable computing device, at least one parameter associated with the second produce based on the second image. Also disclosed is a method of monitoring produce, comprising: mounting a portable computing device to a mounting bracket and initiating a mounted operation mode; receiving, in the mounted operation mode, a first image, the first image being of first produce, at the portable computing device; determining, with the portable computing device, at least one parameter associated with the first produce based on the first image; demounting the portable computing device from the mounting bracket and initiating a second operation mode; capturing, using a handheld camera after demounting the portable computing device, a second image, the second image being of second produce; receiving, in the second operation mode, the second image at the portable computing device; and determining, with the portable computing device, at least one parameter associated with the second produce based on the second image. Also disclosed is a system for monitoring produce, comprising: a portable computing device configured to be mounted to a mounting bracket of a harvester and for communicating with a first camera mounted to the harvester; and a second camera communicatively coupled to or part of the portable computing device and configured for use when the portable computing device is unmounted from the mounting bracket, wherein the first and second cameras are configured to capture respective images of produce for use in determining respective parameters associated with the produce. The portable computing device may be configured to determine the respective parameters associated with the produce. The second camera may be attachable to the portable computing device when the portable computing device is unmounted from the mounting bracket. The system may further include the mounting bracket. The system may further include the harvester. The portable computing device may be a tablet computer. The system may further include a transporter for produce separate from the harvester, and the second camera may be configured to capture an image of produce on the separate transporter. The second camera or portable computing device may include a depth sensor configured to generate depth data associated with the image captured by the second camera. The portable computing device may be configured to be in a first mode of operation when mounted to the mounting bracket, and in a second mode of operation when unmounted from the mounting bracket, the first and second modes of operation being different modes of operation. The portable computing device may be configured to determine the respective parameters and the different modes of operation determine the method used to determine the respective parameters. Also disclosed is a case or protective housing for a portable computing device, the case or protective housing being configured to receive the portable computing device and including a first configuration for engagement by a mounting bracket to secure the case or protective housing to a harvester, and a second configuration for engagement with a camera to secure the camera to the case or protective housing. The first and second configurations may share one or more components. Use of the first configuration to secure the case or protective housing to the harvester may obstruct use of the second configuration to secure the camera to the case or protective housing. The case or protective housing may be combined with one or more of the mounting bracket, the harvester, and the camera. The second camera may be mounted such that the second camera has at least part of a separate transporter within its field of view, such that produce transported by the separate transporter passes through the field of view of the second camera. The second camera may be an inbuilt camera of the portable computing device. The second camera may be external to the portable computing device, and may be connected to the portable computing device by a wired or wireless communications link. The second operation mode may be a handheld operation mode in which the portable computing device is not mounted to a mounting bracket. Also disclosed is a system for monitoring produce, comprising: a first mounting bracket for mounting a portable computing device; a second mounting bracket for mounting the portable computing device; and the portable computing device, wherein the portable computing device is configured to operate in a mounted operation mode when mounted to the first mounting bracket and to receive, in the mounted operation mode, a first image, the first image being of first produce, wherein the portable computing device is configured to determine at least one parameter associated with the first produce based on the first image, wherein the portable computing device is configured to operate in a second operation mode when mounted to the second mounting bracket, wherein the portable computing device is configured in use to capture, when mounted to the second mounting bracket, a second image, the second image being of second produce; wherein the portable computing device is configured to receive the second image in the second operation mode; and wherein the portable computing device is configured to determine at least one parameter associated with the second produce based on the second image. Also disclosed is a system for monitoring produce, comprising: a mounting bracket for mounting a portable computing device; a handheld camera; and the portable computing device, wherein the portable computing device is configured to operate in a mounted operation mode when mounted to the mounting bracket and to receive, in the mounted operation mode, a first image, the first image being of first produce, wherein the portable computing device is configured to determine at least one parameter associated with the first produce based on the first image, wherein the portable computing device is configured to operate in a second operation mode when demounted from the mounting bracket, wherein the portable computing device is configured in use to capture, using the handheld camera, a second image, the second image being of second produce; wherein the portable computing device is configured to receive the second image in the second operation mode; and wherein the portable computing device is configured to determine at least one parameter associated with the second produce based on the second image. BRIEF DESCRIPTION OF THE FIGURES In order that the present disclosure may be more readily understood, preferable embodiments thereof will now be described, by way of example only, with reference to the accompanying drawings, in which: Fig. 1 is a schematic illustration of produce; Fig. 2 is a schematic illustration of the produce of Fig. 1 after harvesting; Fig. 3 is a schematic illustration of produce; Fig. 4 is a schematic illustration of a harvester harvesting the produce of Fig. 3; Fig. 5 is a schematic illustration of a part of a produce monitoring system in a mounted operation mode; Fig. 6 is a schematic illustration of a part of a produce monitoring system in a handheld operation mode; Fig. 7 is a flowchart outlining a method of produce monitoring; Fig. 8 is a schematic illustration of a part of a produce monitoring system; 5 Fig. 9 is a schematic illustration of a part of a produce monitoring system; Fig. 10 is a schematic illustration of a part of a produce monitoring system; and Fig. 11 is a schematic illustration of a part of a produce monitoring system in a handheld operation mode. 10 DETAILED DESCRIPTION OF THE DISCLOSURE The disclosed technology includes a monitoring system 1 and / or a monitoring method. The monitoring system 1 and / or method may be for monitoring produce 2, such as agricultural produce 2, and may be for determining one or more parameters associated with the produce 2. The one or more parameters may, therefore, be determined by the monitoring system 1 and / or method. The produce 2 may include one or more produce items 21. The produce 2 may be or include agricultural produce 2, such as foodstuffs, for example fruits, vegetables, meat, or eggs. The produce items 21 may, therefore, include foodstuff items, such as individual fruits, vegetables, eggs, or cuts of meat. Some examples of produce items 21 are potatoes and apples. Each individual potato or apple may be a produce item 21, and a group or crop of individual produce items 21 may be referred to as the produce 2. The one or more parameters may be determined for the produce 2 as a whole and / or separately for each produce item 21. For example, the one or more parameters may include an average value for a given group of produce items 21, which may be referred to as a crop, for example a crop of potatoes. The average value could be a mean, median, or mode average. The one or more parameters may include individual values associated with each produce item 21. For example, an average size for the produce 2 may be determined, and / or the size of each produce item 21 may be determined individually. Fig. 1 provides a schematic illustration of produce items 21 in the form of apples 21 on an apple tree. Fig. 2 provides a schematic illustration of produce 2 in the form of produce items 21, in this example apples, in a container 3 following harvesting. Fig. 3 provides a schematic illustration of produce 2 in the form of produce items 21, in this example potatoes, before harvesting. Fig. 4 provides a schematic illustration of produce 2, and produce items 21, being harvested by a harvester 13. The system 1 may include the harvester 13. The harvester 13 may include a cab 134 for a user. The cab 134 may include operating means for the user to operate the harvester 13. The harvester 13 may include a transporter 132 for transporting the produce 2, and this may be in the form of a conveyor, such as a conveyor belt, for example. The transporter 132 (e.g. conveyor) may include slats or other spaced members to allow debris to be separated from the produce 2. The system 1 may include a trailer 131. The produce 2 may be loaded into the trailer 131 after harvesting. The system 1 may include a portable computing device 11 (see e.g. Fig. 8). The portable computing device 11 may include one or more of a processor 111, a memory 112, an imaging unit 113, such as a camera 113, a depth sensor 114, a power source 115, a communications module 116, and / or a display 117. The portable computing device 11 may have a mounted operation mode and a handheld operation mode. The portable computing device 11 may be switchable between the mounted operation mode and the handheld operation mode, for example by interaction with a user interface of the portable computing device 11 (and / or the modes of operation may be switched automatically). In other words, the portable computing device 11 may be selectively operable in the mounted operation mode and the handheld operation mode. Automatic switching between modes may be achieved, for example, through the sensing by the portable computing device 11 of a connection to another system 1 component which is unique to the mounted operation mode or the handheld operation mode (such as a particular camera 14m / h, see below) and / or through the use of a motion sensor (such as an inertial measurement unit of the portable computing device 11) to determine a motion characteristic of the portable computing device 11 which is indicative of being mounted or handheld. The user interface may be displayed on the display 117. The display 117 may be a touch-sensitive display, and / or the portable computing device 11 may include input means such as a keyboard and / or mouse. The user may, therefore, interact with the portable computing device 11 using the user interface, and may control the operation of the portable computing device 11 using the user interface. The system 1 may include a mounting bracket 12 (see e.g. Fig. 10). The mounting bracket 12 may be configured to releasably mount the portable computing device 11. The mounting bracket 12 may be installed on the harvester 13. For example, the mounting bracket 12 may be installed in the cab 134. The mounting bracket 12 may, therefore, mount the portable computing device 11 to a part of the harvester 13. The system 1 may include a mounted camera 14m, which may be external to the portable computing device 11. The mounted camera 14m may be a mounted external camera 14m, which may be mounted to a part of the harvester 13. The mounted camera 14m may be mounted to the harvester using a camera mount 133 (see e.g. Fig. 5). The camera mount 133 may be located such that, when mounted to the camera mount 133, the mounted camera 14m has a field of view that includes the transporter 132. The mounted camera 14m may, therefore, be configured to produce images of the produce 2 in use, and those images may be of the produce 2 in transit on the transporter 132. The mounted camera 14m may be connected to the portable computing device 11, for example by a wired or wireless communications link. There may, therefore, be a wiring harness to connect the mounted camera 14m to the portable computing device 11 and connection to that wiring harness may be detectable by the portable computing device 11 for automatically determining the mode of operation, for example. Similarly, the identity of the camera 14m to which the portable computing device 11 is connected may be determined by the portable computing device 11, and the identity of the camera 14m may be used for automatically determining the mode of operation. The identity of the camera 14m to which the portable computing device 11 is connected may be determined for either a wired or wireless connection, for example. The identity of the camera 14m may be determined through the use of an identification code for that camera 14m (which may be a unique identifier or an identifier common to a group of such cameras 14m). Images captured by the mounted camera 14m may, therefore, be sent to the portable computing device 11. The portable computing device 11 may be configured to receive images from the mounted camera 14m. The portable computing device 11 may be connected to the mounted camera 14m in the mounted operation mode. The system 1 may include a handheld camera 14h, which may be a handheld external camera 14h. In some versions the handheld camera 14h may be provided by an inbuilt camera 113 of the portable computing device 11. The handheld camera 14h may be releasably attached to the portable computing device 11, for example by a camera mount attached to the portable computing device 11. The handheld camera 14h may be connected to the portable computing device 11, for example by a wired or wireless communications link. In the case of a wired connection, this may be via a wiring harness which may be different to a wiring harness used to connect the portable computing device 11 to the mounted camera 14m (such that connection to either wiring harness may determine the mode of operation, for example). Similarly, as described above, the identity of the camera 14m, 14h, or 113 to which the portable computing device 11 is connected may be determined by the portable computing device 11, and the identity of the camera 14m may be used for automatically determining the mode of operation. The identity of the camera 14m, 14h, or 113 to which the portable computing device 11 is connected may be determined for either a wired or wireless connection, for example. The identity of the camera 14m, 14h, or 113 may be determined through the use of an identification code for that camera 14m, 14h, or 113 (which may be a unique identifier or an identifier common to a group of such cameras 14m, 14h, or 113). Images captured by the handheld camera 14h may, therefore, be sent to the portable computing device 11. The portable computing device 11 may be configured to receive images from the handheld camera 14h. The handheld camera 14h may be considered to be a handheld camera 14h if the camera is sized such that it can be carried by a user (e.g. in their hand or hands). The handheld camera 14h is an example of a portable camera and other portable cameras might be used, in some versions, in place of the handheld camera 14h. In some versions, the mounted camera 14m is a first camera and the handheld camera 14h is a second camera. In some versions, the portable computing device 11 may be in the form of a tablet computer. In some versions, the portable computing device 11 may include a case or other protective housing. In some versions, the case or other protective housing may include mounting locations configured to be engaged by (or to engage) the camera mount to attach the handheld camera 14h to the portable computing device 11. In some versions, the camera mount includes one or more c- or L-shaped members configured to be rotated with respect to a part of the handheld camera 14h and into engagement with a part of the portable computing device 11 (e.g. a part of the case or other protective housing) to attach the handheld camera 14h to the portable computing device 11. With the portable computing device 11 and handheld camera 14h attached to each other, in some versions, a user may manipulate the portable computing device 11 (e.g. moving that device 11) to move the handheld camera 14h. In some versions, display 117 of the portable computing device 11 may be configured to face a user during operation with the handheld camera 14h attached thereto such that a field of view of the handheld camera 14h faces away from the user (e.g. in a direction opposite to the direction in which the display 117 is facing). The portable computing device 11 may be connected to the mounted camera 14m in the mounted operation mode. The portable computing device 11 may be connected to the handheld camera 14h in the handheld operation mode. The mounted camera 14m may be demountable (e.g. from the camera mount 133) such that it is usable as the handheld camera 14h in the handheld mode. In some versions, separate cameras may be provided for the mounted and handheld modes. The portable computing device 11 may, therefore, switch between connections to the mounted camera 14m and the handheld camera 14h when the operation mode is switched between the mounted and handheld operation modes. In the handheld mode, the portable computing device 11 may be handheld by a user 4 (see e.g. Fig. 6 and 11). The handheld camera 14h,113 may also be handheld in use in the handheld mode. This may allow the user 4 freedom to remove the portable computing device 11 from its mounting bracket 12, for example, and to capture images by directing the handheld camera 14h at any produce 2 to be imaged. This may allow greater freedom to image produce 2 in different locations, for example to capture images of samples of produce 2 as shown in Fig. 6 or to capture images of produce 2 that has not yet been harvested as shown in Fig. 11. In some versions, handheld use does not preclude the use of one or more straps to aid the user in carrying the portable computing device 11 and / or the handheld camera 14,113. In some versions, the mounting bracket 12 may be configured to receive the portable computing device 11 including the case or other protective housing - such that this case or other protective housing does not need to be removed from the rest of the portable computing device 11 for the device 11 to be mounted to the mounting bracket 12. In some versions, the case or other protective housing may include a battery or other portable power supply to assist in powering the portable computing device 11. In some versions, the portable computing device 11 may receive electrical power from the harvester 13 when mounted in the mounting bracket 12. In some versions, the electrical power for the operation of the portable computing device 11 may be provided at least in part by the battery or other portable power supply when the portable computing device 11 is not mounted to the mounting bracket 12. The portable computing device 11 may include one or more further batteries or other portable power supplies (in addition to any provided as part of the case or other protective housing). In some versions, the case or protective housing includes a first configuration for use in securing the case or protective housing to the mounting bracket 12 and a second configuration for use in securing the handheld camera 14h to the case or protective housing. The first and second configurations may include one or more respective arms, clamps, protrusions, recesses, or other members used in the securing of the case or protective housing to the mounting bracket 12 or handheld camera 14h. In some versions, the securing of the case or protective housing to the mounting bracket 12 may cover or otherwise obstruct access to at least part of the second configuration, such that - for example - the handheld camera 14h cannot be secured to the case or protective housing when the case or protective housing is secured to the mounting bracket 12. In some cases, the first and second configurations are the same configurations. In summary, therefore, the portable computing device 11 may be switchable between the mounted operation mode, in which it is mounted to the mounting bracket 12, and for example mounted to a part of a harvester 13, and the handheld operation mode, in which the portable computing device 11 is released (and removed) from the mounting bracket 12 such that it can be carried by the user in a handheld manner. In some versions the mounted camera 14m may be the inbuilt camera 113 of the portable computing device 11. In such versions, therefore, the mounting bracket 12 and the camera mount 133 may be the same entity. For example, the entire portable computing device 11 may be mounted to the camera mount 133, which may also function as the mounting bracket 12, such that the inbuilt camera 113 of the portable computing device 11 is configured to capture images of the produce 2 from its mounted position. As described, this may involve the transporter 132 being visible within a field of view of the camera 113. In some versions, a mounting bracket (which may be similar to or identical to the mounting bracket 12 or 133) may be provided which is positioned relative to a separate transporter for transporting the produce 2 (e.g. a conveyor, such as a conveyor belt) which may be a standalone transporter or part of another system such as a processing system for the produce 2 (e.g. to clean and / or prepare the produce). The positioning may be such that the handheld camera 14h,113 has at least part of the separate transporter within its field of view, such that produce transported by the separate transporter passes through the field of view. In some versions, the handheld camera 14h is separate from the portable computing device 11 but communicatively connected thereto through a wired or wireless connection, with the handheld camera 14h mounted to a further camera mount positioned with the separate transporter within its field of view. Thus, the portable computing device 11 may have a further mode of operation in which the portable computing device 11 has been demounted from the harvester 13 but is, instead, mounted to other equipment (such as the separate transporter) or otherwise used in relation to that other equipment (e.g. with the handheld camera 14h mounted relative to the separate transporter). This further mode of operation may be a variation of the handheld mode of operation or may be a separate mode of operation (e.g. an other equipment mode of operation). In such a mode of operation, the portable computing device 11 may be handheld by the user 4, but the handheld camera 14h may be mounted relative to the separate transporter such that produce 2 transported by the separate transporter passes through its field of view. In versions where the handheld camera 14h is the inbuilt camera 113, the portable computing device 11, including the inbuilt camera 113, may be mounted relative to the separate transporter such that produce 2 transported by the separate transporter passes through its field of view. The mounting bracket 12 may include one or more arms or other members configured to clamp or lock the portable computing device 11 in place - the one or more arms or other members being releasable to allow removal of the portable computing device 11. A method of monitoring produce may, therefore, include mounting the portable computing device 11 to the mounting bracket 12 and initiating the mounted operation mode. The mounted operation mode may be activated by the user 4 interacting with the user interface, which may be displayed on the display 117 (or may be automatically activated). In the mounted operation mode, the portable computing device 11 may be configured to receive images, which may be images of produce 2, from the mounted camera 14m (which images may have been captured by the mounted camera 14m). Produce items 21 may, therefore, be visible in the images. The transporter 132 may also be visible in such images. The portable computing device 11 may be configured to control the mounted camera 14m in the mounted operation mode. The portable computing device 11 may be configured to control the mounted camera 14m to capture images automatically in the mounted mode. The automatic image capture may be triggered by variations in light levels, for example, or activated by motion (e.g. detected by use of the mounted camera 14m). The portable computing device 11 may control the mounted camera 14m to continue capturing images at a predetermined frequency until a deactivation criterion is met, such as when no motion is detected, for example, or when a light level criterion is met. The portable computing device 11 may be configured to determine at least one parameter associated with the produce 2. The at least one parameter may be determined based on the image(s) of the produce 2. The image(s) may, for example, be received using the communications module 116, and may be stored (temporarily or permanently) in the memory 112, which may be a non-transitory computer-readable memory. The at least one parameter may include one or more of size, weight, colour, quality, or health, for example. The parameter may be associated with the produce 2 as a whole, for example as an average value, or may be determined individually for individual produce items 21. The determination of the at least one parameter may be performed using the processor 111. The parameter may be determined using a machine learning model. The machine learning model may be trained, in a training phase, using training data. The training data may be or include ground truth data, which may be annotated with labels or values provided by humans, for example. For example, the ground truth data may include images of produce 2 with associated parameters identified for each image. The associated parameters may form metadata, for example. The ground truth data may, therefore, include a collection of images of produce 2 for which the parameter to be determined has been established manually. The machine learning model may then learn from the training data so that it can determine the one or more parameters for produce 2 in an operative phase. In the operative phase, the machine learning model may determine at least one parameter associated with the produce 2 based on the image or images received at the portable computing device 11, which may be images from the inbuilt camera 113, the mounted camera 14m, or the handheld camera 14h. The parameter determined by the portable computing device 11 may be displayed on the display 117 or transmitted to a remote device using the communications module 116. The portable computing device 11 may be demounted from the mounting bracket 12. The portable computing device 11 may be switched from the mounted operation mode to the handheld operation mode. In the handheld operation mode (which may include the other equipment mode), the portable computing device 11 may be configured to control the handheld camera 14h, and optionally to capture an image in response to a user input. The user input may be via the user interface, for example, such as by interacting with a “capture” button. The image capture in the handheld mode may, therefore, be triggered manually, whereas the image capture in the mounted mode may be automatic. In some versions, the image capture may be automatic in the handheld mode, as described for the mounted mode. The user 4 may, therefore, use the portable computing device 11 and handheld camera 14h to capture an image of produce 2 in the handheld mode, such as is illustrated in Figs. 6 and 11. The produce 2 may be harvested produce, such as in Fig. 6, or may be unharvested produce, such as in Fig. 11. The produce 2 may be a sample of a larger crop, for example. The parameters determined by the portable computing device 11 for the sample can then be used to estimate corresponding parameters for the larger crop. For example, a sample of a crop taken from one part of a field could be used to estimate corresponding parameters for the crop for the whole field. The image captured by the handheld camera 14h may be sent to the portable computing device 11. Accordingly, the image may be received by the portable computing device 11. The portable computing device 11 may then determine the at least one parameter associated with the produce 2 based on the captured image. As described, this may involve the use of a machine learning model. However, in some versions, the machine learning model used in the mounted operation mode may differ from the machine learning model used in the handheld operation mode. For example, the handheld mode machine learning model may be trained to identify features that the mounted mode machine learning model is not trained to identify. Those features may include a part of a user’s body, such as a user’s feet (which may include identification of shoes, for example). The features that the handheld mode machine learning model is additionally trained to identify may be features that are not expected to be visible in the mounted mode (such as a user’s feet or thumb / finger). These features may be excluded from the determining of at least one parameter, as they are not produce items 21. In some versions, when in the handheld operation mode a part of the captured image(s) may be excluded from analysis to identify produce items 21, this part of the image(s) representing the region closest to the user for example, to reduce the risk of parts of the user (e.g. their feet) from being incorrectly identified as produce items 21. The portable computing device 11 may, therefore, be used to determine at least one parameter for first produce 2 in the mounted operation mode, and to determine at least one parameter for second produce 2 in the handheld operation mode. The first produce 2 may include one or more produce items 21 visible in an image captured by the mounted camera 14m, for example, and the second produce 2 may include one or more produce items 21 visible in an image captured by the handheld camera 14h (which may be different produce items 21 to those visible in the image captured by the mounted camera 14m). In this manner, the portable computing device 11 may be used to determine the at least one parameter for produce 2 both during harvesting, for example when mounted to the mounting bracket 12, and before harvesting, for example using the handheld mode. The produce monitoring system 1 is, therefore, more versatile than previous systems, which can be confined to operation only when mounted to a harvester 13, and cannot be demounted for operation in a handheld mode. The mounted camera 14m and / or handheld camera 14h may include a depth sensor, which may produce depth data associated with the captured images. The depth data may be used by the portable computing device 11 to determine the parameter associated with the produce 2, in addition to the received image or images. In some versions the depth sensor may be a depth sensor 114 that is inbuilt into the portable computing device 11. Each image received by the portable computing device 11 may, therefore, be received in conjunction with corresponding depth data, which may be used by the portable computing device 11 (e.g. by the machine learning model) to determine the at least one parameter. The machine learning model may, therefore, be trained using ground truth depth data associated with training images. The depth sensor may include, for example, a sensor such as a LiDAR sensor. In some versions, the depth sensor is part of the mounted camera 14m and / or handheld camera 14h which may each include a respective pair of spaced apart camera devices to provide parallax information from which depth data can be determined. In some versions, the mounted camera 14m and / or handheld camera 14h may include a projector which is configured to project a speckle pattern of infrared light onto the produce 2, from which the camera(s) 14m,h may determine the depth data. The mounted camera 14m and the handheld camera 14h may use the same technology as each other or may use different technologies. The mounted camera 14m and the handheld camera 14h may be the same type of camera, for example. In some instances, the handheld camera 14h may have different imaging characteristics (e.g. resolution) compared to the mounted camera 14m. The system 1 may include a remote device 5 (see Fig. 9), which may be cloud-based. The portable computing device 11 may be configured to send data to the remote device 5. The portable computing device 11 may, therefore, be configured to send data to the cloud. The portable computing device 11 may be configured to send images received from the cameras 14m,14h,113 and / or associated depth data to the remote device 5. The portable computing device 11 may be configured to send the determined parameters to the remote device 5. The remote device 5 may be configured to determine one or more parameters associated with the produce 2 based on the data received from the portable computing device 11, such as the image or images and / or associated depth data. The parameter or parameters determined by the remote device 5 may be the same as those determined by the portable computing device 11, or may be different. The remote device 5 may be configured to determine the same parameters as the portable computing device 11, and to determine additional parameters as well. The parameters determined by the remote device 5 for the produce 2 may be sent back to the portable computing device 11. The parameter or parameters determined by the remote device 5 may be displayed on the display 117. The remote device 5 may be configured to determine the one or more parameters using a machine learning model, which may be trained as described previously. In some versions, the machine learning model used by the remote device 5 to determine the one or more parameters may be different to the machine learning model used by the portable computing device 11. For example, the machine learning model used by the remote device 5 may require more computing power, and may be more sophisticated than that used by the portable computing device 11. The portable computing device 11 may, therefore, use a relatively simple machine learning model to determine the one or more parameters, whereas the remote device 5 may use a relatively complex machine learning model to determine the one or more parameters. The machine learning model used by the remote device 5 may be trained using a larger dataset than the model used by the portable computing device 11. The machine learning model used by the remote device 5 may be updated more frequently than the model used by the portable computing device 11. In some versions, the machine learning model used by the portable computing device 11 may not be updated, and the remote device 5 may therefore use a newer machine learning model to determine the one or more parameters, which may be more effective than the model used by the portable computing device 11. The portable computing device 11 may, therefore, be used to determine an initial estimate of the one or more parameters associated with the produce 2, and the remote device 5 may verify and / or improve upon the estimate provided by the portable computing device 11. As will be appreciated, whilst the term “handheld” has been used herein, other terms may be used in relation to some versions in which the feature or mode is not necessarily “handheld” by a user. In some versions, the handheld features or modes may be referred to as second features or second modes, for example. Likewise, where “handheld” has been used, “mobile” may be used instead. When used in this specification and claims, the terms "comprises" and "comprising" and variations thereof mean that the specified features, steps or integers are included. The terms are not to be interpreted to exclude the presence of other features, steps or components. The invention may also broadly consist in the parts, elements, steps, examples and / or features referred to or indicated in the specification individually or collectively in any and all combinations of two or more said parts, elements, steps, examples and / or features. In particular, one or more features in any of the embodiments described herein may be combined with one or more features from any other embodiment(s) described herein. Protection may be sought for any features disclosed in any one or more published documents referenced herein in combination with the present disclosure. Although certain example embodiments of the invention have been described, the scope of the appended claims is not intended to be limited solely to these embodiments. The claims are to be construed literally, purposively, and / or to encompass equivalents.

Claims

1. A method of monitoring produce, comprising:mounting a portable computing device to a mounting bracket and initiating a mounted operation mode;receiving, in the mounted operation mode, a first image, the first image being of first produce, at the portable computing device;determining, with the portable computing device, at least one parameter associated with the first produce based on the first image;demounting the portable computing device from the mounting bracket and initiating a second operation mode;capturing, after demounting the portable computing device, a second image, the second image being of second produce;receiving, in the second operation mode, the second image at the portable computing device; anddetermining, with the portable computing device, at least one parameter associated with the second produce based on the second image.

2. A method according to claim 1, wherein the portable computing device determines the at least one parameter using a machine learning model.

3. A method according to any preceding claim, wherein the portable computing device is communicatively connected to a mounted camera in the mounted operation mode.

4. A method according to claim 3, wherein the mounted camera is mounted to a part of a harvester.

5. A method according to claim 3 or 4, wherein the portable computing device is configured to control the mounted camera, and wherein the method includes the portable computing device capturing images automatically in the mounted operation mode.

6. A method according to any preceding claim, wherein the portable computing device is communicatively connected to a second camera in the second operation mode.

7. A method according to claim 6, wherein the portable computing device is configured to control the second camera, and wherein the method includes capturing an image in response to a user input in the second operation mode.

8. A method according to any preceding claim, wherein the mounting bracket is installed on a harvester.

9. A method according to any preceding claim, wherein the portable computing device is communicatively connected to a depth sensor and the method includes receiving depth data associated with the first and / or second image at the portable computing device.

10. A method according to any preceding claim, further including transmitting the first and / or second image from the portable computing device to a remote device, and determining the at least one parameter for the first and / or second produce using the remote device.

11. A method according to claim 10, wherein the remote device is configured to determine the at least one parameter using a machine learning model.

12. A method according to claim 11 when dependent on claim 2, wherein the machine learning model used by the remote device is different to the machine learning model used by the portable computing device.

13. A method according to any preceding claim when dependent on claim 2, wherein the portable computing device is configured to use a first machine learning model to determine the at least one parameter in the mounted operation mode and to use a second machine learning model to determine the at least one parameter in the second operation mode.

14. A method according to claim 13, wherein the second machine learning model is trained to identify features that the first machine learning model is not trained to identify.

15. A method according to claim 14, wherein the features include one or more parts of a user’s body.

16. A method according to claim 3 or any of claims 4-15 when dependent on both claims 3 and 6, wherein the second camera is inbuilt into the portable computing device, and the mounted camera is external to the portable computing device.

17. A method according to claim 3 or any of claims 4-15 when dependent on both claims 3 and 6, wherein the mounted camera is demountable such that it can be used in the second operation mode as the second camera.

18. A system for monitoring produce, comprising:a portable computing device switchable between a mounted operation mode and a second operation mode,wherein the portable computing device is configured to be mounted to a mounting bracket in use in the mounted operation mode, and to be dismounted from the mounting bracket in use in the second operation mode,wherein the portable computing device is configured to receive an image of produce and to determine at least one parameter associated with the produce based on the received image.

19. A system according to claim 18, wherein the portable computing device determines the at least one parameter using a machine learning model.

20. A system according to claim 18 or 19, further including a mounted camera communicatively connected to the portable computing device.

21. A system according to claim 20, wherein the mounted camera is mounted to a part of a harvester.

22. A system according to claim 20 or 21, wherein the portable computing device is configured to control the mounted camera to capture images automatically in the mounted mode.

23. A system according to any of claims 18-22, further including a second camera, wherein the portable computing device is configured to control the second camera to capture an image in response to a user input in the second operation mode.

24. A system according to any of claims 18-23, wherein the mounting bracket is installed on a harvester.

25. A system according to any of claims 18-24, wherein the portable computing device is communicatively connected to a depth sensor and the portable computing device is further configured to receive depth data associated with the image.

26. A system according to any of claims 18-25, further including a remote device, wherein the portable computing device is configured to transmit the image from the portable computing device to the remote device, and the remote device is configured to determine the at least one parameter.

27. A system according to claim 26, wherein the remote device is configured to determine the at least one parameter using a machine learning model.

28. A system according to claim 27 when dependent on claim 19, wherein the machine learning model used by the remote device is different to the machine learning model used by the portable computing device.

29. A system according to any of claims 18-28 when dependent on claim 19, wherein the portable computing device is configured to use a first machine learning model to determine the at least one parameter in the mounted operation mode and to use a second machine learning model to determine the at least one parameter in the second operation mode.

30. A system according to claim 29, wherein the second machine learning model is trained to identify features that the first machine learning model is not trained to identify.

31. A system according to claim 30, wherein the features include one or more parts of a user’s body.

32. A system according to claim 23 when dependent on claim 20, wherein the second camera is inbuilt into the portable computing device, and the mounted camera is external to the portable computing device.

33. A system according to claim 23 when dependent on claim 20, wherein the mounted camera is demountable such that it can be used in the handheld operation mode as the second camera.

34. A method according to claim 6 or any of claims 7-17 when dependent on claim 6, further including mounting the second camera such that the second camera has at least part of a separate transporter within its field of view, such that produce transported by the separate transporter passes through the field of view of the second camera.

35. A method according to claim 34, wherein the second camera is an inbuilt camera of the portable computing device.

36. A method according to claim 34, wherein the second camera is external to the portable computing device, and is connected to the portable computing device by a wired or wireless communications link.

37. A method according to any of claims 1-17, wherein the second operation mode is a handheld operation mode in which the portable computing device is not mounted to a mounting bracket.

38. A method of monitoring produce, comprising:mounting a portable computing device to a first mounting bracket and initiating a mounted operation mode;receiving, in the mounted operation mode, a first image, the first image being of first produce, at the portable computing device;determining, with the portable computing device, at least one parameter associated with the first produce based on the first image;demounting the portable computing device from the first mounting bracket and initiating a second operation mode;mounting the portable computing device to a second mounting bracket;capturing, after mounting the portable computing device to the second mounting bracket, a second image, the second image being of second produce;receiving, in the second operation mode, the second image at the portable computing device; anddetermining, with the portable computing device, at least one parameter associated with the second produce based on the second image.

39. A method of monitoring produce, comprising:mounting a portable computing device to a mounting bracket and initiating a mounted operation mode;receiving, in the mounted operation mode, a first image, the first image being of first produce, at the portable computing device;determining, with the portable computing device, at least one parameter associated with the first produce based on the first image;demounting the portable computing device from the mounting bracket and initiating a second operation mode;capturing, using a handheld camera after demounting the portable computing device, a second image, the second image being of second produce;receiving, in the second operation mode, the second image at the portable computing device; anddetermining, with the portable computing device, at least one parameter associated with the second produce based on the second image.

40. A system for monitoring produce, comprising:a portable computing device configured to be mounted to a mounting bracket of a harvester and for communicating with a first camera mounted to the harvester; anda second camera communicatively coupled to or part of the portable computing device and configured for use when the portable computing device is unmounted from the mounting bracket, wherein the first and second cameras are configured to capture respective images of produce for use in determining respective parameters associated with the produce.

41. A system according to claim 40, wherein the portable computing device is configured to determine the respective parameters associated with the produce.

42. A system according to claim 40 or 41, wherein the second camera is attachable to the portable computing device when the portable computing device is unmounted from the mounting bracket.

43. A system according to any of claims 40-42, further including the mounting bracket.

44. A system according to claim 43, further including the harvester.

45. A system according to any of claims 40-44, wherein the portable computing device is a tablet computer.

46. A system according to any of claims 40-45, further including a transporter for produce separate from the harvester, wherein the second camera is configured to capture an image of produce on the separate transporter.

47. A system according to any of claims 40-46, wherein the second camera or portable computing device includes a depth sensor configured to generate depth data associated with the image captured by the second camera.

48. A system according to an of claims 40-47, wherein the portable computing device is configured to be in a first mode of operation when mounted to the mounting bracket, and in a second mode of operation when unmounted from the mounting bracket, the first and second modes of operation being different modes of operation.

49. A system according to claim 48, wherein the portable computing device is configured to determine the respective parameters and the different modes of operation determine the method used to determine the respective parameters.

50. A case or protective housing for a portable computing device, the case or protective housing being configured to receive the portable computing device and including a first configuration for engagement by a mounting bracket to secure the case or protective housing to a harvester, and a second configuration for engagement with a camera to secure the camera to the case or protective housing.

51. The case or protective housing according to claim 50, wherein the first and second configurations share one or more components.

52. The case or protective housing according to any of claims 50-51, wherein use of the first configuration to secure the case or protective housing to the harvester obstructs use of the second configuration to secure the camera to the case or protective housing.

53. The case or protective housing according to any of claims 50-52 in combination with one or more of the mounting bracket, the harvester, and the camera.

54. A system according to claim 23 or any of claims 24-33 when dependent on claim 23, wherein the second camera is mounted such that the second camera has at least part of a separate transporter within its field of view, such that produce transported by the separate transporter passes through the field of view of the second camera.

55. A system according to claim 54, wherein the second camera is an inbuilt camera of the portable computing device.

56. A system according to claim 54, wherein the second camera is external to the portable computing device, and is connected to the portable computing device by a wired or wireless communications link.

57. A system according to any of claims 18-33 or 54-56, wherein the second operation mode is a handheld operation mode in which the portable computing device is not mounted to a mounting bracket.

58. A system for monitoring produce, comprising:a first mounting bracket for mounting a portable computing device;a second mounting bracket for mounting the portable computing device; andthe portable computing device,wherein the portable computing device is configured to operate in a mounted operation mode when mounted to the first mounting bracket and to receive, in the mounted operation mode, a first image, the first image being of first produce,wherein the portable computing device is configured to determine at least one parameter associated with the first produce based on the first image,wherein the portable computing device is configured to operate in a second operation mode when mounted to the second mounting bracket,wherein the portable computing device is configured in use to capture, when mounted to the second mounting bracket, a second image, the second image being of second produce;wherein the portable computing device is configured to receive the second image in the second operation mode; andwherein the portable computing device is configured to determine at least one parameter associated with the second produce based on the second image.

59. A system for monitoring produce, comprising:a mounting bracket for mounting a portable computing device;a handheld camera; andthe portable computing device,wherein the portable computing device is configured to operate in a mounted operation mode5 when mounted to the mounting bracket and to receive, in the mounted operation mode, a first image, the first image being of first produce,wherein the portable computing device is configured to determine at least one parameter associated with the first produce based on the first image,wherein the portable computing device is configured to operate in a second operation mode10 when demounted from the mounting bracket,wherein the portable computing device is configured in use to capture, using the handheld camera, a second image, the second image being of second produce;wherein the portable computing device is configured to receive the second image in the second operation mode; and15 wherein the portable computing device is configured to determine at least one parameterassociated with the second produce based on the second image.

Citation Information

Patent Citations

  • Grain quality control system and method

    US11197417B2

  • Systems, devices, and methods for monitoring and assessing characteristics of harvested specialty crops

    US20180042176A1

  • Crop grading via deep learning

    US20200273172A1

  • drone

    US20200364456A1

  • Crop monitoring system and method

    US20220394922A1