Apparatus and method of recording storage location by size for harvested tubers

Abstract

To provide an apparatus and method of determining a storage location by size for harvested tubers deposited by a bin piler.SOLUTION: The method may comprise: capturing one or more images of the harvested tubers; segmenting individual tuber depictions visible in the images; determining one or more shape characteristics for a plurality of the individual tuber depictions; identifying one or more unoccluded tuber depictions by determining that at least one of the individual tuber depictions is an unoccluded tuber depiction based on the shape characteristics of the individual tuber depiction; measuring a size of tubers corresponding to at least one of the unoccluded tuber depictions; attributing an average size to at least tubers corresponding to the occluded tuber depictions based on the measured size; determining the storage location of the harvested tubers based in part on bin piler location data; and recording in memory the storage location and average size.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] This application relates to an apparatus and method for recording the storage location of size-specific harvested tubers deposited by a bin piler. [Background technology]

[0002]

[0002] A tuber is a type of enlarged structure used as a storage organ for nutrients in some plants. Tuberous crops are some of the most important crops around the world and form a large part of the world's food supply. Tubers include tuberous roots (e.g., potato, yam) and tubers (e.g., sweet potato, cassava).

[0003]

[0003] In agricultural applications, tubers may often be graded based on physical attributes such as size. The intended application and pricing of the tubers may be based on the grading results. For example, large, elongated potatoes that can be used to make French fries and chips may be priced higher. Smaller, irregularly shaped potatoes that can be used for other processed products such as mashed potatoes or potato flakes may be priced lower.

[0004]

[0004] Timely grading of tubers after harvesting allows for proper pricing and logistics operations (storage, processing and distribution), which improves operational efficiency and reduces post-harvest waste. Summary of the Invention

[0005]

[0005] The following summary is provided to introduce the reader to the more detailed discussion that follows. The summary is not intended to limit or define any claims or as yet unclaimed inventions. One or more inventions may reside in any combination or subcombination of the elements or process steps disclosed in any part of this application, including the claims and drawings.

[0006]

[0006] In accordance with some aspects, a method is provided for recording storage locations by size of harvested tubers deposited by a bin piler. The method may include using a camera to capture one or more images of harvested tubers moving on a conveyor towards a bin piler; segmenting, by a processor, individual tuber representations visible in the one or more images; determining, by a processor, one or more shape characteristics for a plurality of individual tuber representations; identifying, by a processor, one or more non-obstructed tuber representations by determining, by a processor, that at least one of the individual tuber representations is an non-obstructed tuber representation based on the one or more shape characteristics of the individual tuber representations, where the tuber representations include an obstructed tuber representation; measuring, by a processor, a size of a tuber corresponding to at least one of the non-obstructed tuber representations; attributing, by a processor, an average size to tubers corresponding to at least the obstructed tuber representations based on the size of the tuber corresponding to at least one of the non-obstructed tuber representations; determining, by a processor, a storage location of the harvested tubers based at least in part on the bin piler location data; and recording in memory the storage location and average size of the harvested tubers.

[0007]

[0007] In accordance with some aspects, an apparatus is provided for recording storage locations of harvested tubers by size. The apparatus may include a bin piler, a conveyor, a camera, a location system, and a control system. The bin piler may be configured to deposit the harvested tubers at the storage location. The conveyor may be configured to move the harvested tubers towards the bin piler. The camera may be configured to capture one or more images of the harvested tubers moving on the conveyor. The location system is configured to generate bin piler location data indicative of a location of the bin piler. The control system may have at least one processor and a memory. The at least one processor may be collectively configured to perform the following: receive one or more images from the camera; segment individual tuber representations visible in the one or more images; determine one or more shape characteristics for a plurality of individual tuber representations; identify one or more non-obstructed tuber representations by determining that at least one of the individual tuber representations is a non-obstructed tuber representation based on the one or more shape characteristics of the individual tuber representations, where the tuber representations include an obstructed tuber representation; measure a size of a tuber corresponding to at least one of the non-obstructed tuber representations; attribute an average size to tubers corresponding to at least the obstructed tuber representations based on the size of the tuber corresponding to at least one of the non-obstructed tuber representations; determine a storage location of the harvested tubers based at least in part on the bin piler location data; and record the storage location and average size of the harvested tubers in memory.

[0008]

[0008] In accordance with some aspects, a non-transitory computer readable medium is provided comprising instructions executable by a processor, which when executed may configure a processor to: receive from a camera one or more images of harvested tubers moving on a conveyor towards a bin piler, segment individual tuber representations visible in the one or more images, determine one or more shape characteristics for a plurality of individual tuber representations, identify one or more non-obstructed tuber representations by determining that at least one of the individual tuber representations is an non-obstructed tuber representation based on the one or more shape characteristics of the individual tuber representations, where the tuber representations include an obstructed tuber representation, measure a size of a tuber corresponding to at least one of the non-obstructed tuber representations, attribute an average size to tubers corresponding to at least the obstructed tuber representation based on the size of the tuber corresponding to at least one of the non-obstructed tuber representations, determine storage locations of the harvested tubers based at least in part on bin piler location data, and record the storage locations and average size of the harvested tubers in memory.

[0009]

[0009] The drawings included in this specification are intended to illustrate various examples of the articles, methods, and apparatus of the specification and are not intended to limit the scope of the teachings in any way. [Brief description of the drawings]

[0010] [Figure 1]

[0010] FIG. 1 is a schematic diagram showing an apparatus for recording storage locations of harvested tubers by size, according to one embodiment. [Diagram 2]

[0011] FIG. 2 is a block diagram showing a control system of the device of FIG. 1. [Diagram 3]

[0012] 1 is a flow chart illustrating an example method for recording storage locations by size of harvested tubers deposited by a bottle piler. [Figure 4A]

[0013] 1 is an example captured image of a number of potatoes moving on a conveyor according to one embodiment. [Figure 4B]

[0014] 4B is a segmented image generated for the example captured image of FIG. 4A. [Figure 4C]

[0015] FIG. 4C shows a binary mask for the five non-occluded potato depictions identified among the individual potato depictions shown in FIG. 4B. [Figure 5A]

[0016] FIG. 13 illustrates a bounding box that fits around an example binary mask of an individual tuber depiction, according to one embodiment. [Figure 5B]

[0017] FIG. 13 illustrates an angle-adjusted bounding box fit around an example binary mask of an individual tuber depiction, according to one embodiment. [Figure 5C]

[0018] FIG. 5C illustrates an ellipse that fits the outer contour of the example binary mask of FIG. 5B. [Figure 5D]

[0019] FIG. 13 illustrates the deviation of an example binary mask of an individual tuber depiction from its convex hull, according to one embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011]

[0020] Numerous embodiments are described herein and are presented for illustrative purposes only. The described embodiments are not intended to be limiting in any way. The present invention is broadly applicable to numerous embodiments, as is readily apparent from the disclosure herein. Those skilled in the art will appreciate that the present invention can be practiced with modifications and alterations without departing from the teachings disclosed herein. Although certain features of the present invention may be described with reference to one or more specific embodiments or drawings, it should be understood that such features are not limited to the use in the one or more specific embodiments or drawings described with reference thereto.

[0012]

[0021] "An embodiment," "embodiment," "embodiments," "the embodiment," "the outline," "one or more embodiments," "some embodiments," and "one embodiment" mean "one or more (but not all) embodiments of the invention," unless otherwise specified.

[0013]

[0022] The terms "including," "comprising," and variations thereof mean "including but not limited to" unless otherwise specified. A list of items does not imply that any or all of the items are mutually exclusive unless otherwise specified. The terms "a," "an," and "the" mean "one or more" unless otherwise specified.

[0014]

[0023] As used herein and in the claims, two or more parts are considered to be "coupled," "connected," "attached," "joined," "attached," or "fastened" when the parts are joined or operate together directly or indirectly (i.e., through one or more intermediate parts) so long as a link occurs. As used herein and in the claims, two or more parts are considered to be "directly coupled," "directly connected," "directly attached," "directly joined," "directly attached," or "directly fastened" when the parts are connected in physical contact with one another. As used herein, two or more parts are considered to be "firmly coupled," "firmly connected," "firmly attached," "firmly joined," "firmly attached," or "firmly fastened" when the parts are connected such that they move as one while maintaining a constant orientation with respect to one another. None of the terms "coupled," "connected," "attached," "joined," "attached," and "fastened" distinguish the manner in which two or more parts are joined together.

[0015]

[0024] Additionally, although method steps may be described (in this disclosure and / or claims) sequentially, such methods may be configured to operate in an alternating order. In other words, any order or sequence of steps that may be described does not necessarily indicate a requirement that the steps be performed in that order. Steps of methods described herein may be performed in any order that is practical. Additionally, some steps may be performed simultaneously.

[0016]

[0025] As used in this specification and claims, a group of elements is considered to "collectively" perform an action if the action is performed by any one of the elements in the group or cooperatively by two or more (or all) elements in the group.

[0017]

[0026] As used in this specification and claims, unless otherwise specified, a first element will be considered to be "received within" a second element if at least a portion of the first element is received within the second element.

[0018]

[0027] Some elements in this specification are designated by a base number followed by an alphabetic or subscripted numeric suffix (e.g., 112a or 112 1 Elements herein may be identified by part numbers that share a base number in common and differ by their subscripts (e.g., 112 1 , 112 2 , and 112 3 ), and may be identified by a part number. All elements with a common base number may be referred to collectively or generally using the base number without a subscript (e.g., 112).

[0019]

[0028] As used in this specification and claims, terms such as "top," "bottom," "upper," "lower," "upward," "vertical," "elevation," "above," "lower," and similar terms generally relate to an orientation that is aligned (e.g., parallel) with gravity. However, none of the terms referred to in this paragraph imply any particular alignment between elements. For example, if a first element is positioned higher than a second element, and regardless of whether the first element is vertically aligned with the second element, the first element is considered to be "vertically above" the second element.

[0020]

[0029] After harvesting, grading the tubers in a timely manner can require significant labor resources. Harvested tubers can be transported and deposited in a storage facility using high throughput conveyors. For example, the conveyor throughput can be greater than 200 kg per minute. Manually measuring the size of every harvested tuber, or even a small portion of every harvested tuber, can be costly, time consuming, and labor intensive, requiring significant labor and time investment. The disclosed apparatus and method can reduce (or nearly eliminate) cost, time, and labor requirements by automating the size measurement and location tracking of harvested tubers.

[0021]

[0030] Timely and accurate grading of harvested tubers can be difficult. Conveyors used to transport harvested tubers into storage can be narrow, causing clustering and blockage of tubers. For example, when a pile of tubers forms on a conveyor, the top tubers overlap the lower tubers, thereby blocking (i.e., obscuring) at least a portion of the lower tubers. Blocked portions can prevent accurate size assessment when using images to aid this process. Other problems can include mechanical vibrations, environmental light, and foreign objects (e.g., mud), which can vary according to the section of the field where the tubers are harvested.

[0022]

[0031] The disclosed apparatus and method can improve the accuracy of size measurement by distinguishing fully visible (i.e., non-occluded) tubers in the image from occluded tubers. A sampling of non-occluded tubers can be selected for size measurement, and an average size can be attributed to adjacent harvested tubers (including occluded tubers) based on size measurements of the sampled non-occluded tubers. This avoids inaccuracies associated with assessing the size of occluded tubers.

[0023]

[0032] 1, there is shown a schematic diagram of an apparatus 100 for recording the size-by-size storage locations of harvested tubers 10. In the illustrated example, the apparatus 100 includes a bin piler 104, a conveyor 108, a camera 112, a control system 116, and a location system 120.

[0024]

[0033] The harvested tubers 10a may be deposited at a first end 128 of a conveyor 108. For example, the harvested tubers may be transported from a field and deposited at the first end 128 by a truck 14. A second end 132 of the conveyor 108 may be connectable to the bin piler 104. The conveyor 108 may have any suitable design for moving the harvested tubers 10 from the first end 128 towards the bin piler 104 at the second end 132. For example, the conveyor 108 may be a belt conveyor or a motorized roller conveyor.

[0025]

[0034] The conveyor 108 may include one or more conveyor segments. For example, the conveyor 108 may include a single straight conveyor segment extending from the first end 128 to the second end 132. In the illustrated embodiment, the conveyor 108 is shown to include multiple conveyor segments (e.g., conveyor segments 110a-110d extending from the grading station 22 to the storage facility 18). Each conveyor segment 110 may extend along the same or a different direction than adjacent segments and / or may be inclined at the same or a different angle than adjacent segments. In some examples, each conveyor segment may move at the same or a different speed than adjacent segments.

[0026]

[0035] Optionally, the apparatus 100 may include one or more sensors 148 configured to measure the speed of the conveyor segments 110. The apparatus 100 may include any suitable number of sensors 148 (e.g., 1 to 100 sensors). For example, the apparatus 100 may include one speed sensor 148 for each conveyor segment 110 that is movable at a different speed than adjacent segments 110. In the illustrated embodiment, the sensors 148a-148d are configured to measure the speed of each of the conveyor segments 110a-110d.

[0027]

[0036] The sensor 148 may have any design suitable for measuring the speed of the conveyor segment 110. For example, the sensor 148 may be a Reed 59060 magnetic shaft sensor capable of providing a non-contact measurement of conveyor speed. The Reed 59060 magnetic shaft sensor is capable of measuring the number of revolutions per second of any shaft around which the conveyor belt rotates. The speed of the conveyor segment may be determined based on the measured number of revolutions per second and the radius of the shaft according to Equation 1 below. Conveyor speed (m / s) = measured rotation speed (1 / s) x 2π x shaft radius (m) (Equation 1)

[0028]

[0037] In other examples, different sensors may be used. The sensor 148 may provide measured conveyor speed data to the control system 116 by wired or wireless (e.g., LoRa, WiFi, or Bluetooth). Measuring the speed of the conveyor segment 110 allows the control system 116 to more accurately predict when a tuber passing the grading station 22 will be discharged from the bin piler 104 into a tuber pile 10c in the storage facility 18. This allows the storage location 30 of that tuber in the storage facility 18 to be more accurately determined and recorded. For example, the speed of the conveyor segment 110 may change over time depending on factors such as the density of the tubers carried by the conveyor segment 110. Thus, the transit time between the grading station 22 and the discharge end of the bin piler 104 may also change over time. Such changes can be reflected by continuously measuring the speed of the conveyor segment 10.

[0029]

[0038] In other embodiments, the speed of the conveyor segment 110 is constant or the transit time from the grading station 22 to the discharge end of the bin piler 104 is constant, making it possible to accurately predict the time when the tubers will be discharged from the bin piler 104 after passing through the grading station without measuring the speed of the conveyor segment 110 (i.e., without using sensor 148).

[0030]

[0039] A grading station 22 may be installed along the conveyor 108 to inspect and grade the harvested tubers. A camera 112 may be positioned at the grading station 22. The camera 112 may be any suitable camera for capturing images of the harvested tubers 10b moving along the conveyor 108. In some embodiments, the camera 112 may be an RGB camera that captures images of the harvested tubers in the visible region of the electromagnetic spectrum. In other embodiments, the camera 112 may be any other camera capable of capturing images in the visible and infrared (IR) regions of the electromagnetic spectrum, such as a hyperspectral camera. A hyperspectral camera may provide additional information compared to an RGB camera, such as information regarding internal defects of the imaged tubers. An RGB camera may allow for lower cost and system complexity compared to a hyperspectral camera.

[0031]

[0040] The camera 112 may provide captured images (e.g., individually captured photographs or selected video frames) to the control system 116 directly (e.g., wired or wirelessly) or indirectly via the network 26. The network 26 may include a communications network, such as the Internet, a wide area network (WAN), a local area network (LAN), or other types of networks. The camera 112 may communicate with the network 26 using a communications module 144. In some examples, the communications module 144 may be incorporated into the camera 112. In other examples, the communications module 144 may be a separate module that is communicatively coupled to the camera 112 (e.g., a Wi-Fi or network dongle).

[0032]

[0041] In some embodiments, the control system 116 may control the image capture rate of the camera 112 and the measured speed of the conveyor 108 based on the positioning and image capture parameters of the camera 112. For example, the control system 116 may control the image capture rate of the camera 112 based on conveyor speed data received from a sensor 148 associated with a conveyor segment 110a within the field of view of the camera 112. The image capture rate of the camera 112 may be controlled to avoid capturing multiple images containing the same tuber.

[0033]

[0042] For example, the field of view of the camera 112 may include a 1.3 m length of the conveyor 108 based on the camera 112 positioning and image capture parameters. This length of the conveyor 108 may therefore be included in each image captured by the camera 112. For a conveyor speed of 1 m / s, the control system may set the image capture rate of the camera 112 to 0.5 images per second (i.e., one image captured every two seconds) to avoid capturing the same tuber in multiple images. For a conveyor speed of 0.5 m / s, the control system may set the image capture rate of the camera 112 to 0.25 images per second (i.e., one image captured every four seconds) to avoid capturing the same tuber in multiple images. Equation 2 below provides the relationship between the image capture rate in images per second (f) and the conveyor speed in meters per second (v) for the exemplary case where a conveyor length of 1.3 m is captured in each image. f=0.5v (Equation 2)

[0034]

[0043] The bin piler 104 may have any design suitable for depositing harvested tubers into the storage facility 18. For example, the bin piler 104 may include an elevator 136 and a boom 140. The elevator 136 may control the vertical altitude at which the harvested tubers are discharged from the bin piler 104. The boom 140 may control the lateral (e.g., horizontal XY) location of the discharge of the harvested tubers from the bin piler 104. The combination of the elevator 136 and the boom 140 may control the storage location 30 of the harvested tubers 10c. Other bin piler designs may have alternative and / or additional moveable elements for controlling the three dimensional location at which the harvested tubers are discharged into the storage facility 18.

[0035]

[0044] Location system 120 may use any suitable technique to track the movements of binpilers 104 and generate location data for the binpilers. For example, location system 120 may track the movements of binpilers 104 using dead reckoning or distance measurements to nearby anchor nodes (i.e., nodes with known fixed locations, such as WiFi access points, Bluetooth beacons, or ultra-wideband beacons).

[0036]

[0045] In the illustrated schematic, the location system 120 includes a tag 120a positioned at the emission end of the bin piler 104 and multiple anchor tags 120b (three anchor tags in FIG. 1, but four or more tags are preferred to derive 3D coordinates) positioned at known locations within the storage facility 18. Each tag 120 may include an Impulse Radio Ultra Wideband (IR-UWB) transceiver (e.g., an ESP32 Ultra Wideband board with a Qorvo™ DW1000 UWB module). The location of the emission end of the bin piler 104 is trackable in real time based on two-way time-of-flight (TOF) measurements between the tag 120a and the multiple anchor tags 120b. The high bandwidth provided by the IR-UWB transceivers may enable tracking of the bin piler's movements with high resolution in three dimensions.

[0037]

[0046] The location system 120 may provide the generated binpiler location data to the control system 116 using any suitable communication technology. For example, the location system 120 (e.g., tag 120a) may transmit the binpiler location data to the communication module 144 using LoRa to enable low power operation of the location system 120. Alternatively or additionally, the communication module 144 may provide the received binpiler location data to the control system 116 via the network 26. In other examples, the location system 120 (e.g., tag 120a) may provide the generated binpiler location data directly to the control system 116. Alternatively or additionally to using LoRa, the location system 120 may transmit the binpiler location data using other technologies, such as WiFi or via a wired connection.

[0038]

[0047] Alternatively or additionally, the location system 120 may include sensors that detect the inclination of the elevator 136 and / or the extension of the boom 140 and / or the extension of one or more of the conveyor segments 110. For example, one or more of the conveyor segments 110 and / or the bin piler boom 140 may be extendable in length to accommodate the pile of tubers at a targeted storage location. Such variability in length may affect the accuracy of the predicted transit time from the grading station 22 to the discharge end of the bin piler 104, where the transit time is predicted based on the measured speed of the conveyor segment 110. In some embodiments, there may be sensors associated with each of one or more (or all) of the extendable conveyor segments 110 and / or the bin piler boom 140 to determine the length of these elements, whereby the predicted transit time (measured as the length divided by the speed to travel over each element) can be determined based on the accurate length.

[0039]

[0048] Any suitable sensor may be used to determine the current length of the extendable conveyor segment 110 and / or the bin piler boom 140. The illustrated example shows several conveyor segments 110, including sensors 121, which may be hardware similar to sensors 120a or 120b described above. For example, data from the sensors 121 may be transmitted to the control system 116, which may determine the 2D or 3D location of each sensor 121 and derive the length of the associated segment 110 therefrom. Similarly, the length of the bin piler boom 140 may be derived based on location data from sensor 120a and / or additional sensors.

[0040]

[0049] 1 and 2, FIG. 2 is a block diagram of an example control system 116. In some embodiments, the control system 116 may be implemented as a server device (e.g., a remote server). In other embodiments, the control system 116 may be implemented using any other suitable hardware components. For example, the control system 116 may be implemented as a device that is physically integrated with the grading station 22.

[0041]

[0050] 2, control system 116 includes memory 152, applications 156, output devices 160, display devices 164, secondary storage devices 168, processors 172, input devices 176, and communications devices 180. One or more (or all) of memory 152, applications 156, output devices 160, display devices 164, secondary storage devices 168, processors 172, input devices 176, and communications devices 180 may be communicatively coupled via wires and / or wirelessly.

[0042]

[0051] In some embodiments, the control system 116 includes any one or more of the following: memory 152, applications 156, output devices 160, display devices 164, secondary storage devices 168, processors 172, input devices 176, and communications devices 180. In some embodiments, the control system 116 does not include any one or more of the applications 156, secondary storage devices 168, network connections, input devices 176, output devices 160, display devices 164, and communications devices 180.

[0043]

[0052] The memory 152 may include one or more of a random access memory (RAM) and a read only memory (ROM). In some embodiments, the memory 152 stores one or more applications 156 for execution by the processor(s) 172. The applications 156 correspond to software modules that include computer-executable instructions that, when executed by one or more processors, collectively cause the one or more processors to perform operations related to the functions and methods described herein.

[0044]

[0053] The memory 152 may store one or more machine learning models. The machine learning models may be trained to segment individual tubers visible in the captured image and / or to identify non-occluded tubers among the segmented individual tubers. The machine learning models may be generated and trained by the control system 116. In some embodiments, the control system 116 may receive a generated machine learning model from an external device (e.g., via the network 26) and train the received machine learning model. The control system 116 may perform supervised training of the received model using labeled training data. In some embodiments, the control system 116 may receive a trained machine learning model from an external device (e.g., via the network 26) and store the trained machine learning model in the memory 152.

[0045]

[0054] In some embodiments, the memory 152 may also store the determined storage location and average size of the harvested tubers. The stored data may be provided to a user, for example, via the network 26.

[0046]

[0055] Secondary storage device 168 may include any suitable non-transitory computer-readable medium containing instructions executable by a processor (e.g., processor 172). For example, secondary storage device 168 may include a hard drive, a floppy drive, a CD drive, a DVD drive, a Blu-ray drive, a solid-state drive, flash memory, or other type of non-volatile data storage. Processor 172 may collectively execute instructions contained on secondary storage device 168 to perform operations for the functions and methods described herein.

[0047]

[0056] In some embodiments, control system 116 stores information in a remote storage device, such as cloud storage, accessible across a network, such as network 26 or another network. In some embodiments, control system 116 stores information distributed across multiple storage devices, such as memory 152 and secondary storage device 168 (i.e., each of the multiple storage devices stores a portion of the information, and collectively the multiple storage devices store all of the information). Thus, storing data on a storage device as used herein and in the claims can mean storing the data in a local storage device, storing the data in a remote storage device, or storing the data in a distributed manner across multiple storage devices, each of which can be local or remote.

[0048]

[0057] The input devices 176 may include any device for inputting information into the control system 116. For example, the input devices 176 may be a keyboard, a keypad, a cursor device, a touch screen, a camera, or a microphone. The input devices 176 may also include input ports and wireless communications (e.g., Bluetooth or 802.11x) for wired and wireless connections to external devices.

[0049]

[0058] Display device 164 may include any type of device for presenting visual information. For example, display device 164 may be a computer monitor, a flat screen display, a projector, or a display panel.

[0050]

[0059] Output device 160 may include any type of device for presenting a hard copy of information, such as, for example, a printer. Output device 160 may also include other types of output devices, such as, for example, speakers. In at least one embodiment, output device 160 includes one or more of an output port and wireless communication (e.g., Bluetooth, or 802.11x) for wired and wireless connection to external devices.

[0051]

[0060] Communications device 180 may have any design suitable for receiving analog and / or digital inputs and for providing analog and / or digital outputs. In some embodiments, communications device 180 may include separate modules for analog and digital signals.

[0052]

[0061] Processor 172 may be any device capable of collectively executing applications, computer readable instructions, or programs. The applications, computer readable instructions, or programs may be stored in memory 152 or in secondary storage device 168, or may be received from remote storage accessible, for example, via network 26. Processor 172 may be a high performance general purpose processor, a standard processor (e.g., an Intel® or AMD® processor), specialized hardware (e.g., a GPU), or multiple processing devices that collectively perform the functions provided by processor 172.

[0053]

[0062] 2 illustrates an example of a control system 116. In alternative embodiments, the control system 116 includes fewer, additional, or different components. Additionally, while aspects of the implementation of the control system 116 are described as being stored in memory, those skilled in the art will appreciate that these aspects may also be stored on or read from other types of computer program products or computer readable media, such as secondary storage devices, including hard disks, floppy disks, CDs, or DVDs; carrier waves from the Internet or other networks; or other forms of secondary storage devices, including RAM or ROM. For example, the control system 116 may include a non-transitory computer readable medium that stores computer readable instructions that, when executed by the processor 172, collectively configure the processor 172 to perform the methods described herein.

[0054]

[0063] 3, a flow chart illustrating an example method 200 of recording size-specific storage locations for harvested tubers deposited by a bin piler is shown. The harvested tubers may be, for example, all tubers harvested from a field. In some embodiments, the harvested tubers may be a portion of the tubers harvested from a field. In some examples, the portion may be defined by duration (e.g., harvested tubers moving on a conveyor for a defined duration (e.g., 1 minute or 1 hour) or by number (e.g., a truckload of tubers)).

[0055]

[0064] Method 200 may be performed, for example, using apparatus 100 (FIG. 1), also referenced below in FIGs. 1 and 2. Method 200 may be performed automatically in response to detecting movement of conveyor 108. In some embodiments, method 200 may be performed in response to user input or based on a stored time schedule.

[0056]

[0065] In some cases, the harvested tubers may be potatoes. In other cases, the harvested tubers may be any other tuber (e.g., sweet potato, yam, cassava, etc.).

[0057]

[0066] At 204, one or more images may be captured of the harvested tubers 10 moving on the conveyor towards the bin piler. For example, the images may be captured by the camera 112. The camera 112 may provide the captured images to a control system (e.g., control system 116).

[0058]

[0067] In some embodiments, a portion of the harvested tubers may be imaged at 204. Imaging a portion of the harvested tubers may provide a reduction in computational resource requirements. In other embodiments, all of the harvested tubers may be imaged at 204. This may provide a large amount of data for determining the size of the harvested tubers, which may allow for more accurate size determination.

[0059]

[0068] At 208, individual tuber representations visible in the captured images may be segmented. For example, the processor 172 of the control system 116 may segment individual tuber representations visible in the captured images. The individual tuber representations may be segmented by generating a binary mask indicating the detected area of ​​each individual tuber representation and generating a bounding box that encompasses the detected area of ​​that individual tuber representation.

[0060]

[0069] In some embodiments, the individual tuber representations may be segmented using a suitable machine learning model. For example, the processor 172 may segment the individual tuber representations using a suitable machine learning model stored in the memory 152.

[0061]

[0070] A suitable machine learning model may be selected based on factors including accuracy, speed, and computational resource requirements. In some embodiments, a Mask Region-based Convolutional Neural Network (R-CNN) model may be used to segment individual tuber depictions visible in the captured image. In other embodiments, a different model (e.g., U-net) may be used to segment individual tuber depictions visible in the captured image.

[0062]

[0071] Reference is now made to Figures 4A and 4B. Figure 4A illustrates an example captured image 304 of multiple potatoes moving on a conveyor. The moving potatoes are clustered, with some potatoes overlapping on top of others. As visible in the captured image 304, the clustering may cause occlusion of some of the imaged potatoes. Figure 4B illustrates an example segmented image 308 generated for the example captured image 304 using a Mask R-CNN model. The trained Mask R-CNN model may receive the captured image 304 as an input and provide a segmented image 308 as an output. The segmented image 308 includes individual potato depictions segmented from one another (instance segmentation) and from the conveyor (semantic segmentation).

[0063]

[0072] The Mask R-CNN model architecture may include a backbone stage and a head stage. In some embodiments, the backbone stage may be constructed using a Feature Pyramid Network (FPN), a Residual Network (ResNet), a Regional Proposal Network (RPN), and a Region of Interest (ROI) alignment layer. The backbone stage may generate predictions of regions within a captured image that may include tuber delineations. The head stage may include fully connected layers that may generate classification, bounding box, and mask predictions using the region predictions from the backbone stage.

[0064]

[0073] Any suitable platform may be used to implement the Mask R-CNN model architecture. In some embodiments, the detectron2 library may be used. In other embodiments, a different platform (e.g., MMDetection) may be used.

[0065]

[0074] 1-3, at 212 of the method 200, one or more shape characteristics may be determined for each individual tuber representation. For example, the processor 172 may determine one or more shape characteristics for each individual tuber representation segmented at 208.

[0066]

[0075] The shape properties may include, for example, one or more color-based feature parameters (S / N_1 and S / N_2), edge-based feature parameters (avg_val_bbox, ellipticity, ellipse ratio, circularity, convexity failure), and / or seven image invariant moments (H1-H7) obtained from the HuMoments method of the OpenCV Python library. More information regarding these shape properties is provided below. In some embodiments, step 212 is omitted because one or more shape properties are pre-stored (e.g., in memory 152 or storage device 168) and accessible to processor 172 (see FIG. 2).

[0067]

[0076] Reference is now made to Figures 5A-5D. Figure 5A shows a bounding box 320 that is fitted around an example binary mask of an individual tuber representation. Figure 5B shows an angle-adjusted bounding box 332 that is fitted around an example binary mask of an individual tuber representation. Figure 5C shows an ellipse 336 that is fitted to the outer contour of the example binary mask of an individual tuber representation. Figure 5D shows the deviation 340 of the example binary mask of an individual tuber representation from its convex hull.

[0068]

[0077] The avg_val_bbox edge-based feature parameter may be determined based on the average density of tuber mask pixels along the edge of the bounding box. For example, the processor 172 may calculate the total number of pixels along the edge of the bounding box 320 shown in FIG. 5A. The processor 172 may also calculate the number of tuber mask pixels along the edge of the bounding box 320 (corresponding to the edge portions 324 and 328). The processor 172 may determine the avg_val_bbox edge-based feature parameter using a ratio of the tuber mask pixels along the edge of the bounding box 320 to the total number of pixels along the edge of the bounding box 320. A higher value for this ratio (e.g., higher than a threshold) may indicate that the tuber represented by the binary mask is substantially cut off (e.g., by an occluded tuber or by an edge of the captured image) and should be treated as an occluded tuber.

[0069]

[0078] The S / N_1 color-based feature parameter may be determined based on the ratio of the number of tuber mask pixels to the total number of pixels enclosed by the bounding box. For the example bounding box 320 shown in FIG. 5A, the processor 172 may calculate the number of tuber mask pixels (white pixels 352) to the total number of pixels (black pixels 356 and white pixels 352) enclosed by the bounding box 320.

[0070]

[0079] The S / N_2 color-based feature parameter may be determined based on the ratio of the number of tuber mask pixels to the total number of pixels enclosed by the angle-adjusted bounding box. The angle-adjusted bounding box may be defined to capture the angular orientation of the tuber mask. For the example angle-adjusted bounding box 332 shown in FIG. 5B, the processor 172 may calculate the number of tuber mask pixels (white pixels 360) to the total number of pixels (gray pixels 364 and white pixels 360) enclosed by the angle-adjusted bounding box 332. The angle-adjusted bounding box may be smaller than the corresponding unadjusted bounding box, resulting in a higher S / N_2 parameter value than the corresponding S / N_1 parameter value.

[0071]

[0080] The elliptical edge based feature parameter may be determined based on the ratio of the area of ​​an ellipse (that fits the outer contour of the tuber mask pixels) to the number of tuber mask pixels. For example, the processor 172 may calculate the area of ​​an ellipse 336 that fits the outer contour of the tuber mask pixels shown in FIG. 5C. The processor 172 may calculate the area of ​​the ellipse 336 using Equation 3 below: A=πab / 4 (Equation 3) where a is the length of the major axis 368 of the ellipse 336, b is the length of the minor axis 372 of the ellipse 336, and A is the computer area of ​​the ellipse 336. The processor 172 may determine the ratio of the area A to the number of tuber mask pixels to determine ellipticity.

[0072]

[0081] The ellipse ratio edge-based feature parameter may be determined based on the ratio of the number of tuber mask pixels enclosed by the contour of the fitted ellipse to the total number of tuber mask pixels. For example, the processor 172 may calculate the number of tuber mask pixels enclosed by the contour of the ellipse 336 shown in FIG. 5C (the number of white pixels enclosed by the ellipse 336) to the total number of tuber mask pixels (the total number of white pixels).

[0073]

[0082] The circularity edge-based feature parameter may be determined based on how closely the shape of the tuber mask resembles a circle. For example, the processor 172 may determine the circularity parameter value using Equation 4 below. Circularity = 4π × Area / Perimeter 2 (Formula 4) In the above formula, Area is the area of ​​the tuber mask (the number of tuber mask pixels (e.g., white pixels in FIG. 5C)) and Perimeter is the perimeter of the tuber mask (e.g., the number of edge pixels that form the perimeter of the tuber mask).

[0074]

[0083] The convexity defect edge based feature parameter may be determined based on the deviation of the tuber mask from its convex hull. For the example shown in FIG. 5D, the processor 172 may determine the convexity defect parameter value based on the deviation 340 of the tuber mask from its convex hull 344.

[0075]

[0084] 1-3, at 216 of the method 200, one or more non-occluded tuber representations may be identified by determining that at least one of the individual tuber representations is a non-occluded tuber representation. As shown in FIG. 4A, the captured image 304 may depict a plurality of occluded and non-occluded tubers. The segmented individual tuber representations may be identified as non-occluded tuber representations based on the shape characteristics determined for the individual tuber representations at 212. For example, the processor 172 of the control system 116 may identify at 216 the one or more non-occluded tuber representations.

[0076]

[0085] Non-occluded tuber representations may be identified based on a binary classification of the segmented individual tuber representations as fully visible and partially visible tubers. In some embodiments, a threshold-based sampling approach may be used to identify non-occluded tuber representations. In other embodiments, a machine learning based sampling approach may be used to identify non-occluded tuber representations. The machine learning based sampling approach may provide a higher classification accuracy compared to the threshold-based sampling approach.

[0077]

[0086] Any suitable number of non-occluded tuber representations may be sampled within each image. In some embodiments, up to 1 to 5 non-occluded tuber representations may be sampled within each image. A smaller sample size may reduce computing resource requirements. In other embodiments, a greater number (e.g., up to 5 to 10) non-occluded tuber representations may be sampled within each image. If fewer than a defined maximum number of non-occluded tuber representations are present in an image, fewer than a defined maximum number of non-occluded tuber representations may be sampled. Overall, this may provide greater accuracy in determining the average size attributed to harvested tubers, but may require a greater amount of computing resources.

[0078]

[0087] In the threshold-based sampling approach, one or more of the shape characteristics may be used as threshold parameters. The threshold parameters may be selected based on a predetermined correlation coefficient between the parameter value of the shape characteristic and the response variable (binary classification as occluded / non-occluded). For example, the threshold parameters may include S / N_1, S / N_2, ellipticity, circularity, convexity failure, and two Hu image moments (H3 and H4). In other examples, different threshold parameters may be used.

[0079]

[0088] The processor 172 may identify occluded tuber depictions by comparing the determined value for the threshold parameter with a predefined threshold value, which in some embodiments may be based on the mean, area, 25th, 50th, and 75th percentile values ​​for both fully visible and partially visible tuber classes.

[0080]

[0089] In the machine learning based sampling approach, any suitable machine learning model may be used. In some embodiments, a trained random forest model may be used. In other embodiments, other trained models may be used, such as a support vector machine (SVM) or a logistic regression model. The random forest model may provide a higher classification accuracy compared to other models for identifying non-obstructed tuber delineation.

[0081]

[0090] In some embodiments, processor 172 may provide a subset of one or more shape features as input to a selected machine learning model (e.g., a random forest model). The subset of shape features may be predetermined based on sequential forward feature selection to identify features with the highest predictive power. For example, processor 172 may provide parameter values ​​corresponding to convexity defect, ellipse ratio, ellipticity, circularity, avg_val_bbox, and S / N_1 as input to the machine learning model.

[0082]

[0091] 4C, there is shown an image 312 providing a graphical representation of the output generated by the machine learning model. Image 312 shows five non-occluded potato depictions identified among the segmented individual tuber depictions shown in FIG. 4B.

[0083]

[0092] 1-3, at 220 of the method 200, a size of the tuber corresponding to at least one of the non-occluded tuber depictions (identified at 216) may be measured. For an example embodiment in which five non-occluded tuber depictions are identified in each image, the processor 172 may measure a size of the tuber corresponding to each of the five non-occluded tuber depictions.

[0084]

[0093] The measured size may include at least one of one dimensional length, projected area, and volume. For the example image shown in FIG. 4C, processor 172 may measure the major axis of an ellipse that fits each non-obstructed tuber representation. In some embodiments, processor 172 may measure the minor axis of the ellipse, or may measure both the major and minor axes.

[0085]

[0094] In some embodiments, the processor 172 may measure the projected area of ​​the tubers corresponding to the unoccluded tuber representations. For example, the processor 172 may measure the two-dimensional area corresponding to the tuber mask for each unoccluded potato representation shown in FIG.

[0086]

[0095] In some embodiments, the processor 172 may measure a volume of the tuber corresponding to the unobstructed tuber representation. For example, the processor 172 may measure the volume based on a volume of rotation of the projected area of ​​the unobstructed tuber around the major axis of a fitted ellipse.

[0087]

[0096] At 224, an average size may be attributed to at least the tubers that correspond to the blocked tuber depiction based on the size measured at 220. For example, the processor 172 may attribute an average size to at least the tubers that correspond to the blocked tuber depiction based on the size measured at 220.

[0088]

[0097] The average size may be attributed based on the measured sizes of a plurality of unobstructed tuber depictions. For example, a plurality of unobstructed tuber depictions may be measured in each captured image, and / or a plurality of images of harvested tubers may be captured with each image including one or more measured unobstructed tuber depictions. The average size may be attributed to tubers corresponding to the obstructed tuber depictions in the captured images. The average size may also be attributed to tubers corresponding to the unobstructed tuber depictions in the captured images, and / or any tubers that were not imaged. For example, the average size may be attributed to a particular number or quantity of tubers, or tubers that passed within a certain time interval before, during, and / or after the images were captured.

[0089]

[0098] At 228, a storage location for the harvested tubers may be determined based at least in part on the bin piler location data. For example, the processor 172 may determine a storage location for the harvested tubers based in part on the bin piler location data received from the location system 120.

[0090]

[0099] The storage location of the harvested tubers may include any suitable data indicative of the physical arrangement of the harvested tubers. For example, a bin piler may deposit the harvested tubers into bins located within a storage facility, and the determined storage location may include a bin identifier (e.g., an identification number). As another example, the storage location may include a position coordinate (e.g., a two-dimensional or three-dimensional coordinate) within the storage facility, or a location identifier (e.g., a quadrant or zone number) associated with a physical location within the storage facility.

[0091]

[0100] The storage location may include data with any suitable level of granularity. For example, the storage location may include x,y coordinate data indicating a floor location within a storage facility. In some examples, the storage location may also include z coordinate data indicating a height within a pile of harvested tubers.

[0092]

[0101] In some embodiments, a storage location for the harvested tubers may be determined based on the bin piler location data and one or more movement characteristics of the conveyor. For example, the processor 172 may determine the storage location based on the bin piler location data received from the location system 120, the conveyor speed data received from the sensor 148, and the extendable conveyor segment length data and boom length data received from the sensor 121 and the location system 120. The processor 172 may use the conveyor speed and length data to track the movement of the harvested tubers from the grading station 22 to the bin piler 104. The movement tracking may be based on real-time speed and length data received from the sensor 148 or a predefined transit time associated with the movement of the tubers from the grading station 22 to the bin piler 104. The processor 172 may further track the movement of the harvested tubers from the bin piler 104 to the storage location based on the bin piler location data received from the location system 120. The processor 172 may determine, based on the motion characteristics of the conveyor, which average size is attributable to the tubers currently being deposited by the bin piler 104. The transit time between the grading station 22 and the bin piler 104 may be such that multiple groups of tubers, each one associated with an average size, may pass through the grading station 22 before the first of those tuber groups reaches the bin piler 104. In this example, the transit time between the grading station 22 and the bin piler 104 may be used by the processor 172 to determine which set of determined average sizes is attributable to the tubers currently being deposited by the bin piler 104 (e.g., the average sizes determined for the groups that passed through the grading station 22 one transit time ago).

[0093]

[0102] In some embodiments, the height (i.e., z coordinate, altitude, or gravity orientation) location of the tubers deposited by the bin piler 104 may be determined to be a prescribed or measured offset from the z coordinate location of the discharge end of the bin piler 104. For example, the bin piler 104 may maintain the discharge end of the boom 140 at a programmed height (e.g., automatically using a sensor such as an ultrasonic distance sensor or manually by an operator) above the tubers or the floor below, which height is intended to mitigate damage from falling tubers (e.g., 15 cm or less). In other examples, the bin piler 104 may include a sensor 122 (e.g., an ultrasonic distance sensor) for sensing a height offset between the discharge end of the bin piler 104 and the tubers or the floor below. This may allow for precise determination of the elevation (e.g., z coordinate location) at which the tubers are being deposited within the storage facility 18.

[0094]

[0103] At 232, the storage location and average size of the harvested tubers may be recorded in memory. For example, the processor 172 may record in memory 152 (e.g., as a database entry) the storage location of the harvested tubers associated with the average size attributed to the harvested tubers. The stored information may be provided to one or more users of the control system 116, which may enable tubers of a desired grade (e.g., size) to be selectively located and removed according to the intended product to be made from the tubers (e.g., French fries or potato flakes).

[0095]

[0104] Although the above description provides examples of embodiments, it will be apparent that some features and / or functions of the described embodiments are susceptible to modification without departing from the spirit and principles of operation of the described embodiments. Therefore, the above description is intended to be illustrative and not limiting of the present invention, and those skilled in the art will appreciate that other variations and modifications may be made without departing from the scope of the present invention as defined in the appended claims. The claims are not limited by the preferred embodiments and examples, but are to be given the broadest interpretation consistent with the description as a whole.

[0096] item

[0105] Item 1: A method for recording storage locations by size of harvested tubers deposited by a bin piler, the method comprising: using a camera to capture one or more images of harvested tubers moving on a conveyor towards a bin piler; segmenting, by a processor, individual tuber representations visible in the one or more images; determining, by a processor, one or more shape characteristics for a plurality of individual tuber representations; identifying, by a processor, one or more non-obstructed tuber representations by determining, by a processor, that at least one of the individual tuber representations is an non-obstructed tuber representation based on the one or more shape characteristics of the individual tuber representations, wherein the tuber representations include an obstructed tuber representation; measuring, by a processor, a size of a tuber corresponding to at least one of the non-obstructed tuber representations; and attributing, by a processor, an average size to tubers corresponding to at least the obstructed tuber representation based on the size of the tuber corresponding to at least one of the non-obstructed tuber representations; determining, by a processor, storage locations of the harvested tubers based at least in part on bin piler location data; and recording in memory the storage locations and average size of the harvested tubers.

[0106] Item 2: The method of any preceding item, wherein the segmenting includes deriving a binary mask for each individual tuber representation using a Mask Region-based Convolutional Neural Network (R-CNN) model.

[0107] Item 3: A method as described in any preceding item, wherein determining one or more shape characteristics includes at least one of determining color-based feature parameters and edge-based feature parameters for a binary mask.

[0108] Item 4: The method of any preceding item, wherein identifying one or more non-obstructed tuber depictions comprises a binary classification of fully and partially visible tuber depictions using a random forest model.

[0109] Item 5: The method according to any of the preceding items, wherein the size includes at least one of a volume, a projected area, and a one-dimensional length.

[0110] Item 6: The method of any preceding item, wherein measuring includes fitting at least one external contour of the unobstructed tuber representation with an ellipse and determining major and minor axes of the ellipse.

[0111] Item 7: The method of any preceding item, wherein determining the storage location of the harvested tubers is further based on one or more motion characteristics of the conveyor.

[0112] Item 8: The method of any preceding item, wherein the bin piler location data is received from one or more sensors positioned on the bin piler or proximate to the storage location.

[0113] Item 9: The method of any preceding item, wherein the harvested tubers comprise potatoes.

[0114] Item 10: An apparatus for recording storage locations by size of harvested tubers, comprising: a bin piler configured to deposit harvested tubers at a storage location; a conveyor configured to move the harvested tubers towards the bin piler; a camera configured to capture one or more images of the harvested tubers moving on the conveyor; a location system configured to generate bin piler location data indicative of a location of the bin piler; and a control system having at least one processor and memory, wherein the at least one processor is collectively configured to: receive one or more images from the camera; segment individual tuber representations visible in the one or more images; determine one or more shape characteristics for a plurality of individual tuber representations; identifying one or more non-obstructed tuber representations by determining that at least one of the individual tuber representations is a non-obstructed tuber representation based on one or more shape characteristics of the individual tuber representations, where the tuber representations include an obstructed tuber representation; measuring a size of a tuber corresponding to at least one of the non-obstructed tuber representations; attributing an average size to tubers corresponding to at least the obstructed tuber representations based on the size of the tubers corresponding to at least one of the non-obstructed tuber representations; determining a storage location of harvested tubers based at least in part on bin piler location data; and recording the storage location and average size of the harvested tubers in memory.

[0115] Item 11: The apparatus of any preceding item, wherein the segmenting includes deriving a binary mask for each individual tuber representation using a Mask Region-based Convolutional Neural Network (R-CNN) model.

[0116] Item 12: An apparatus described in any preceding item, wherein determining one or more shape characteristics includes determining at least one of color-based feature parameters and edge-based feature parameters for a binary mask.

[0117] Item 13: The apparatus of any preceding item, wherein identifying the one or more unobstructed tuber depictions comprises binary classification of fully and partially visible tuber depictions using a random forest model.

[0118] Item 14: An apparatus according to any preceding item, wherein the size includes at least one of a volume, a projected area, and a one-dimensional length.

[0119] Item 15: The apparatus of any preceding item, wherein measuring includes fitting at least one external contour of the unobstructed tuber representation with an ellipse and determining major and minor axes of the ellipse.

[0120] Item 16: The apparatus of any preceding item, wherein determining a storage location for the harvested tubers is further based on one or more movement characteristics of the conveyor.

[0121] Item 17: An apparatus described in any preceding item, wherein the location system includes one or more sensors positioned on the bin piler or proximate to the storage location, the one or more sensors configured to generate location data for the bin piler.

[0122] Item 18: The apparatus of any preceding item, wherein the harvested tubers include potatoes.

[0123] Item 19: A non-transitory computer-readable medium comprising instructions executable by a processor, the instructions, when executed, configuring the processor to: receive from a camera one or more images of harvested tubers moving on a conveyor toward a bin piler; segment individual tuber representations visible in the one or more images; determine one or more shape characteristics for a plurality of individual tuber representations; identify one or more non-obstructed tuber representations by determining that at least one of the individual tuber representations is an non-obstructed tuber representation based on the one or more shape characteristics of the individual tuber representations, wherein the tuber representations include an obstructed tuber representation; measure a size of a tuber corresponding to at least one of the non-obstructed tuber representations; attribute an average size to tubers corresponding to at least the obstructed tuber representations based on the size of the tuber corresponding to at least one of the non-obstructed tuber representations; determine a storage location of the harvested tubers based at least in part on the bin piler location data; and record the storage location and average size of the harvested tubers in memory.

[0124] Item 20: The non-transitory computer-readable medium of any preceding item, wherein the segmenting includes deriving a binary mask for each individual tuber representation using a Mask Region-based Convolutional Neural Network (R-CNN) model.

[0125] Item 21: A non-transitory computer-readable medium described in any preceding item, wherein determining one or more shape characteristics includes determining at least one of color-based feature parameters and edge-based feature parameters for a binary mask.

[0126] Item 22: The non-transitory computer-readable medium of any preceding item, wherein identifying one or more unobstructed tuber depictions includes binary classification of fully and partially visible tuber depictions using a random forest model.

[0127] Item 23: A non-transitory computer-readable medium according to any preceding item, wherein the size includes at least one of a volume, a projected area, and a one-dimensional length.

[0128] Item 24: The non-transitory computer-readable medium of any preceding item, wherein measuring includes fitting at least one outer contour of the unobstructed tuber representation with an ellipse and determining major and minor axes of the ellipse.

[0129] Item 25: The non-transitory computer-readable medium of any preceding item, wherein determining a storage location for the harvested tubers is further based on one or more motion characteristics of the conveyor.

[0130] Item 26: The non-transitory computer-readable medium of any preceding item, wherein the bin piler location data is received from one or more sensors positioned on the bin piler or proximate the storage location.

[0131] Item 27: The non-transitory computer-readable medium of any preceding item, wherein the harvested tubers include potatoes.

Claims

1. 1. A method for recording storage locations of harvested tubers by size deposited by a bottle piler, comprising: using a camera to capture one or more images of the harvested tubers moving on a conveyor towards the bin piler; segmenting, by a processor, individual tuber depictions visible in said one or more images; determining, by the processor, one or more shape characteristics for a plurality of the individual tuber representations; identifying, by the processor, one or more non-obstructed tuber representations by determining that at least one of the individual tuber representations is a non-obstructed tuber representation based on the one or more shape characteristics of the individual tuber representations, wherein the tuber representations include an obstructed tuber representation; measuring, by the processor, a size of a tuber corresponding to at least one of the non-obstructed tuber representations; attributing, by the processor, an average size to tubers corresponding to at least the obstructed tuber representation based on a size of the tubers corresponding to the at least one of the non-obstructed tuber representations; determining, by the processor, the storage location of the harvested tubers based at least in part on bin piler location data; and recording in a memory the storage location and the average size of the harvested tubers.

2. 2. The method of claim 1, wherein the segmenting comprises deriving a binary mask for each individual tuber representation using a Mask Region-based Convolutional Neural Network (R-CNN) model.

3. The method of claim 2 , wherein determining the one or more shape characteristics comprises at least one of determining color-based feature parameters and edge-based feature parameters for the binary mask.

4. The method of claim 1 , wherein identifying the one or more non-obstructed tuber depictions comprises a binary classification of fully and partially visible tuber depictions using a random forest model.

5. The method of claim 1 , wherein the size includes at least one of a volume, a projected area, and a length in one dimension.

6. The method of claim 1 , wherein said measuring comprises fitting the at least one outer contour of the unobstructed tuber representation with an ellipse and determining major and minor axes of the ellipse.

7. The method of claim 1 , wherein determining the storage location of the harvested tubers is further based on one or more motion characteristics of the conveyor.

8. The method of claim 1 , wherein the bin piler location data is received from one or more sensors positioned on the bin piler or proximate to the storage location.

9. The method of claim 1 , wherein the harvested tubers comprise potatoes.

10. 1. An apparatus for recording storage locations of harvested tubers by size, comprising: a bin piler configured to deposit the harvested tubers at the storage location; a conveyor configured to move the harvested tubers towards the bottle piler; a camera configured to capture one or more images of the harvested tubers moving on the conveyor; a location system configured to generate binpiler location data indicative of a location of the binpiler; a control system having at least one processor and a memory; Equipped with The at least one processor collectively: receiving the one or more images from the camera; segmenting individual tuber depictions visible in said one or more images; determining one or more shape characteristics for a plurality of said individual tuber representations; identifying one or more non-occluded tuber representations by determining that at least one of the individual tuber representations is a non-occluded tuber representation based on the one or more shape characteristics of the individual tuber representations, wherein the tuber representations include an obstructed tuber representation; measuring a size of a tuber corresponding to at least one of said non-obstructed tuber representations; attributing an average size to tubers corresponding to at least one of the obstructed tuber depictions based on the size of the tubers corresponding to the at least one of the non-obstructed tuber depictions; determining the storage location of the harvested tubers based at least in part on the bin piler location data; and recording in said memory the storage location and the average size of the harvested tubers; configured to execute Device.

11. 11. The apparatus of claim 10, wherein the segmenting comprises deriving a binary mask for each individual tuber representation using a Mask Region-based Convolutional Neural Network (R-CNN) model.

12. The apparatus of claim 11 , wherein determining the one or more shape characteristics comprises determining at least one of a color-based feature parameter and an edge-based feature parameter for the binary mask.

13. The apparatus of claim 10 , wherein identifying the one or more non-obstructed tuber depictions comprises a binary classification of fully and partially visible tuber depictions using a random forest model.

14. The apparatus of claim 10 , wherein the size includes at least one of a volume, a projected area, and a length in one dimension.

15. The apparatus of claim 10 , wherein said measuring comprises fitting the at least one outer contour of the unobstructed tuber representation with an ellipse and determining major and minor axes of the ellipse.

16. The apparatus of claim 10 , wherein determining the storage location of the harvested tubers is further based on one or more motion characteristics of the conveyor.

17. 11. The apparatus of claim 10, wherein the location system comprises one or more sensors positioned on the bin piler or proximate to the storage location, the one or more sensors configured to generate location data for the bin piler.

18. The apparatus of claim 10 , wherein the harvested tubers include potatoes.

19. A non-transitory computer-readable medium comprising instructions executable by a processor, the instructions, when executed, receiving one or more images from the camera of the harvested tubers moving on the conveyor towards the bin piler; segmenting individual tuber depictions visible in said one or more images; determining one or more shape characteristics for a plurality of said individual tuber representations; identifying one or more non-occluded tuber representations by determining that at least one of the individual tuber representations is a non-occluded tuber representation based on the one or more shape characteristics of the individual tuber representations, wherein the tuber representations include an obstructed tuber representation; measuring a size of a tuber corresponding to at least one of said non-obstructed tuber representations; attributing an average size to tubers corresponding to at least one of the obstructed tuber depictions based on the size of the tubers corresponding to the at least one of the non-obstructed tuber depictions; determining a storage location for the harvested tubers based at least in part on the bottle piler location data; and recording in memory the storage location and the average size of the harvested tubers; configuring the processor to execute Non-transitory computer-readable medium.

20. 20. The non-transitory computer-readable medium of claim 19, wherein the segmenting comprises deriving a binary mask for each individual tuber representation using a Mask Region-based Convolutional Neural Network (R-CNN) model.

21. 21. The non-transitory computer-readable medium of claim 20, wherein determining the one or more shape characteristics comprises determining at least one of a color-based feature parameter and an edge-based feature parameter for the binary mask.

22. 20. The non-transitory computer-readable medium of claim 19, wherein identifying the one or more non-obstructed tuber depictions comprises a binary classification of fully and partially visible tuber depictions using a random forest model.

23. 20. The non-transitory computer-readable medium of claim 19, wherein the size comprises at least one of a volume, a projected area, and a length in one dimension.

24. 20. The non-transitory computer readable medium of claim 19, wherein the measuring comprises fitting the at least one outer contour of the non-obstructed tuber representation with an ellipse and determining major and minor axes of the ellipse.

25. 20. The non-transitory computer readable medium of claim 19, wherein determining the storage location of the harvested tubers is further based on one or more motion characteristics of the conveyor.

26. 20. The non-transitory computer-readable medium of claim 19, wherein the instructions, when executed, further configure the processor to receive location data of the bin piler from one or more sensors positioned on the bin piler or proximate to the storage location.

27. 20. The non-transitory computer readable medium of claim 19, wherein the harvested tubers comprise potatoes.

Images