Apparatus and method for mechanized vegetable harvesting

Abstract

Methods, devices, and systems, that relate to mechanization of agricultural tasks, including a robotic tool carrier configured to be mounted on a tractor or other type of vehicle and employing imaging and artificial intelligence to perform agricultural tasks, such as harvesting vegetables such as celery plants.

Description

APPARATUS AND METHOD FOR MECHANIZED VEGETABLE HARVESTINGCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority from U.S. Provisional Application No. 63 / 523,886, filed on June 28, 2023, the disclosures of which are incorporated herein by reference in their entirety.FIELD

[0002] This disclosure relates to mechanization of agricultural tasks. More specifically, the disclosure relates to a tool carrier configured to be mounted on a vehicle such as a tractor and employing imaging and artificial intelligence to perform agricultural tasks, such as harvesting vegetables.BACKGROUND

[0003] The labor force for farm laborers has been steadily decreasing since 2000. Specialty crops, including fruits, vegetables, tree nuts, and nursery crops, are some of the most labor-intensive crops to farm with labor costs being a large percentage of the overall expenses for those crops. Farmers want to mechanize tasks traditional performed by laborers, but current mechanized farm equipment does not have automatic adjustment features which leads to poor quality and crop damage.

[0004] One example where agricultural mechanization is desired is with vegetable harvesting.The height of a vegetable plant varies and it has to be cut at the correct height. An example of harvesting vegetables is the harvesting of celery.

[0005] Most of the celery plants grown in the USA have been harvested manually for years.While there are some approaches to automate the celery harvesting process that use industry robots, the cost and throughput of such systems cannot match the current cost and throughput requirements. And these automated approaches are nowhere close to the cost and throughput human harvesting crews can achieve. However, with increasing labor costs and diminishing labor supply it is difficult to continuewith manual harvesting operations. The need for automation is strong, but a viable solution must provide a good combination of throughput, cost, and ease of use.

[0006] Currently in harvesting celery a harvesting crew makes three cuts to produce the finished harvested celery product. The first cut is to remove celery from the root structure. In the first step of the harvesting process a crew member usually holds the celery top with one hand to gain access to the root of the plant, as shown in FIG. 1A.

[0007] The second step is to cut the celery plant at the root. In this second step the crew member makes a pushing cut to remove the celery from the ground, as shown in FIG. IB. During this cut the crew member stands and bends his or her back. Although they have limited vision of the soil and celery joint, an experienced worker can cut roughly at the joint, and bring the celery up and hold it with one hand. Because a human crew cannot adjust the cut height accurately, they usually cut the plant so it is longer than it should be.

[0008] The third step is to chop off the extra part of the plant that has soil attached to it as shown in FIG. 1C.

[0009] The fourth step is to chop off the top of the celery to make it the right length, as shown in FIG. ID. In this step, the crew holds it with one hand so he can make a chop cut.

[0010] The fifth step in the process is to throw the celery back to the packer. The packer does some clean-up and peeling before packing it into the packing box or plastic bag.

[0011] If the celery plant is cut too high the celery stalks can fall apart into pieces. If it is cut too low, too much soil comes with the cut vegetable and can be included with the vegetable in the vegetable packaging. Farmers would like a mechanized solution that automatically adjusts the cutting height for the vegetable despite variations in the appropriate cutting height for each vegetable.

[0012] What is needed is a mechanized solution that automatically determines the optimal locations where a vegetable, such as celery, is to be cut to harvest it.SUMMARY

[0013] Some embodiments of the present disclosure solve the previously mentioned problems and other problems of the background art. However, not all embodiments of the present disclosure are required to solve those problems to practice the inventive techniques of the present application.

[0014] Some embodiments of the present disclosure enable a tool carrier apparatus, that includes a tool for working on a plant planted in the ground, wherein the tool includes a first cutting mechanism configured to cut a portion of the plant; an adjustable carrier configured to hold the tool and move the tool in a horizontal direction and a vertical direction with respect to the ground and configured to mount to a vehicle; a camera configured to capture an image of a plant, the plant having an optimal cutting location; a memory having a program stored therein; a processor that when executing the program; a control algorithm outputting a control command based on the output from the artificial intelligence engine; implements: an artificial intelligence engine trained to identify the optimal cutting location of the plant, receive the captured image of the plant, and output an indication of the optimal cutting location of the plant; and a robotic controller configured to control the adjustable carrier based on the control command to position the tool to cut the plant at the optimal cutting location.

[0015] In some embodiments the camera is mounted on the adjustable carrier.

[0016] In some embodiments the plant is a celery plant.

[0017] In some embodiments the adjustable carrier includes an adjustable horizontal arm moveable in the horizontal direction, an adjustable vertical arm moveable in the vertical direction with respect to the ground, and an end effector attached to one of the adjustable horizontal arm and the adjustable vertical arm and configured to hold the tool.

[0018] In some embodiments the camera is attached to the end effector by a rigid support and in close proximity to the tool.

[0019] In some embodiments the plant is a vegetable and the artificial intelligence engine is trained to identify the protected portion of the plant so that the robotic controller causes the position of the cutting tool to correspond to a predicted portion of the vegetable between a lower point of the vegetable and an upper point of the vegetable.

[0020] In some embodiments the lower point of the vegetable corresponds to a point where soil is not taken when the vegetable is cut and the upper point of the vegetable corresponds to a point where the cut vegetable is not likely to divide into separate pieces.

[0021] In some embodiments the plant is a crop planted in one of a plurality of rows of the crop, and the tool is configured to extract a weed disposed between the rows of crops while avoiding damaging the plant.

[0022] In some embodiments the vehicle is a tractor.

[0023] In some embodiments the tool includes a grabbing apparatus.

[0024] In some embodiments the grabbing apparatus is configured to grab the plant prior to performing work on the plant, and adjust a position of the plant to provide a clear path between the camera and the optimal cutting location.

[0025] In some embodiments the tool further includes a secondary cutting mechanism configured to operate a second cutting operation subsequent to operation of the first cutting mechanism.

[0026] In some embodiments the robotic controller is further configured to control the adjustable carrier to, subsequent to the second cutting operation, place the cut plant in a transport device for transport of the cut plant to a storage location.

[0027] In some embodiments the transport device is a conveyor.

[0028] Some embodiments of the present disclosure relate to a tool positioning method, including, capturing an image of a plant, the plant having an optimal cutting location; receiving, at an artificial intelligence engine trained to identify the optimal cutting location of the plant, a capturedimage of the plant, and output an indication of the optimal cutting location of the plant; outputting a control command based on the output from the artificial intelligence engine; and controlling an adjustable carrier based on the control command to position a tool to cut the plant at the optimal cutting location.

[0029] Some embodiments of the present disclosure relate to a tool positioning non-transitory computer-readable medium, including: at least one non-transitory computer-readable medium containing program code, the program code configured to, when executed by at least one processor, cause at least one of the least one processors to: capture an image of a plant, the plant having an optimal cutting location; receive, at an artificial intelligence engine trained to identify the optimal cutting location of the plant, a captured image of the plant, and output an indication of the optimal cutting location of the plant; output a control command based on the output from the artificial intelligence engine; and control an adjustable carrier based on the control command to position a tool to cut the plant at the optimal cutting location.BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Further features, the nature, and various advantages of the disclosed subject matter will be more apparent from the following detailed description with reference to the accompanying drawings in which:

[0031] FIGS. 1A, IB, 1C, and ID illustrate steps of manually harvesting celery.

[0032] FIG. 2 illustrates the location of a part of a plant, in this case a the celery / soil joint of a celery plant, predicted by an Al model, from images captured by an imaging system.

[0033] FIG. 3 is a diagram illustrating a robotic tool carrier system for harvesting celery.

[0034] FIG. 4 illustrates a location on a celery plant to make a cut to harvest the plant, the location being generated by an Al powered tool carrier system.

[0035] FIG. 5 illustrates an example embodiment of a grabber of tool carrier system.

[0036] FIGS. 6A and 6B illustrate operation of a grabber according to an example embodiment of the adjustable tool carrier.

[0037] FIG. 7 illustrates a conveyor system used with various embodiments of the tool carrier system.

[0038] FIG. 8 is a diagram illustrating a hardware configuration of an information processing system that can be used to implement various devices of at least some embodiments of the disclosure.

[0039] FIG. 9a illustrates another embodiment of a grabber, or gripping device.

[0040] FIG. 9b illustrates another view of the gripping device of FIG. 9a.

[0041] FIG. 10a illustrates a top-down view of the gripping device of FIG. 9a in an opened position.

[0042] FIG. 10b illustrates a top-down view of the gripping device of FIG. 9a in a closed position.

[0043] FIG. 11 illustrates a side profile view of the gripping device of FIG. 9a .

[0044] FIG. 12a illustrates a top-down view of a top cutting device in an opened position of at least some embodiments of the disclosure.

[0045] FIG. 12b illustrates a top-down view of the top cutting device in a closed position.

[0046] FIG. 13 illustrates a side profile view of the top cutting device.

[0047] FIG. 14 is a flow diagram illustrating an embodiment of interactions between various elements that operate on images input from a camera and that output control signals to control the adjustable tool carrier and tool.

[0048] FIG. 1 is a flow diagram illustrating an example process of performing image analysis.

[0049] FIG. 16 is a flow diagram illustrating an example process of adjusting a position of a camera and / or working toolDETAILED DESCRIPTION

[0050] Illustrative embodiments of the disclosure will now be described in detail with reference to the attached drawings in which like reference numerals refer to like elements.Robotic Tool Carrier System

[0051] In certain embodiments of a mechanized vegetable harvester a robotic tool carrier system, which controls a tool for harvesting vegetables, mounts to a farm vehicle, such as a tractor, that provides for mobile operation of the tool. The tool carrier system includes an adjustable tool carrier with the tool attached to the carrier. The tool carrier system includes an imaging system coupled to a computer with an artificial intelligence (Al) engine implementing a deep learning prediction model that is trained for harvesting a particular vegetable. The imaging system and Al engine determine where to position the tool to optimally harvest the vegetable.

[0052] In some embodiments, the critical portion of the plant is a body or fruit of the plant, and the system is configured to avoid cutting the critical of the plant while cutting or harvesting the plant. In some embodiments, the critical portion of the plant is the optimal cutting location of the plant. A celery plant includes a root and a heart. An optimal cutting location of the celery plant separates the body of the plant from the root and dirt covered portions of the plant, while maintaining connection of the body of the plant to at least a portion of the heart of the plant.

[0053] In the case of harvesting celery, the imaging system and Al engine can detect the celery plant and the celery / soil joint in real-time. FIG. 2 is an image captured by the imaging system with boxes superimposed on the image showing where the Al engine has detected the celery / soil joint. With the use of the imaging system and Al engine the computer can understand the celery structure even in difficult lighting conditions.

[0054] A two-axis robotic arm embodiment of a robotic tool carrier system 300 for harvesting celery is illustrated in FIG. 3. The robotic tool carrier system 300 includes a tool carrier apparatus 302that is mounted on one end to a farm vehicle, such as a tractor 304, and has an end effector 306 attached at an end opposite to the tractor. The end effector 306 includes a grabber device 308, configured to grasp a celery plant 310, and is attached to an end of a tool carrier apparatus. The tool carrier apparatus includes two hydraulic cylinders: a horizontal cylinder 312, and a vertical cylinder 314. The horizontal hydraulic cylinder 312 is arranged to extend or retract in a direction substantially parallel to the ground. The horizontal cylinder 312 is attached to the vertical hydraulic cylinder 314 which is arranged to extend or retract in a direction substantially orthogonal to the ground. A cutting tool 316 also is attached to the end effector 306 and is positioned below the grabber device 308 to cut a celery plant 310 near the soil. This tool carrier apparatus provides a robotic arm controllable along two axes. Once cut, the grabber 308 places the cut celery plant onto a conveyer belt system to move the cut celery plant to a location suitable for further processing, such as packing the cut celery plants. In some embodiments, the horizontal cylinder is driven pneumatically by an air compressor, while the vertical cylinder is driven by an electrical motor.

[0055] A camera 318 is mounted on the end effector near the grabber 308 and cutting tool 316 to capture images of plants being operated on by the grabber and cutting tool. The camera 318 is positioned close to the grabber and cutting tool.

[0056] A controller 320 includes information processing equipment, such as a computer, the executes computer software that implements the Al engine as well as implements controller functions to control operations of the tool carrier system 300.

[0057] The tool carrier system 300 is configured to be mounted on a tractor 304 that moves along a row of celery plants. While the controller 320is shown mounted on the tool carrier system 300, the controller also may be located within the tractor 304.Mechanized Celery Harvester Operation

[0058] 1. Two-axis robotic arm. The tool carrier system 300 is mounted onto a vehicle such as a tractor 304 to move the end effector 306 along two axes, as shown in Fig. 3. Movement of the end effector along one axis provides forward and backward motion for the cutting. Movement along the vertical axis will provide height adjustment to cut the celery at the right location. The grabber and a conveyor belt system collect the celery as the tractor moves forward.

[0059] A camera 318, such as a stereo camera, is attached to the end effector 306 close to the cutter 316, around ground level to provide accurate vision observation of the celery root location, as shown in Fig. F. The camera is coupled to an artificial intelligence (Al) engine implementing a deep learning prediction model as described in International Application No. PCT / US2022 / 054273, filed December 29, 2022, and claiming priority to U.S. Provisional Application No. 63 / 294,627 filed December 29, 2021, each of which is incorporated by reference in its entirety. The Al engine in the present application has been trained with images of celery plants.

[0060] 2. Accurate control software adjusts the cut height. The height at which celery plants are cut can be adjusted to an accurate height to avoid bringing up soil with the celery. This avoids the human’s cutting in step 3 shown in FIG. 1C. The camera 318, mounted on the end effector and near the cutter, captures images of a celery plant in a row along which the tractor moves. Images captured from the camera are input to the Al engine. The Al engine predicts a location where the celery plant and ground meet to determine a cut baseline. Based on a heart size adjustment determines a cut location relative to the cut baseline. FIG. 4 shows an example of a baseline for performing the cut in step 2. The baseline is determined based on the Al engine prediction of the celery / soil joint. An adjustment to the predicted cut location, relative to the baseline, as shown in FIG. 4, can be determined based on the size of the celery heart. A final cut location is generated according to the cut baseline and adjusted by the heart size adjustment. A control algorithm generates a control signal to control the hydraulic cylinders to position the cutter at the determined final cut location.

[0061] 3. Compressed air driven grabber holds the celery during cutting. In one embodiment the grabber 308 is configured as a hand with rubber fingers and a palm to protect the celery plant while in the grasp of the grabber, as shown in FIG. 5. This rubber hand is controlled to open before cutting (FIG. 6A) and to close while cutting (FIG. 6B) to hold the celery in position while the cut is made at the determined final cut location. In one embodiment the rubber hand is normally held open, such as by spring mechanisms within the rubber hand. The fingers of the rubber hand are caused to close and grip the celery by use of compressed air to hold the celery while it is being cut and to continue to hold the celery after it has been cut.

[0062] 4. Cut celery is moved to conveyor. Once cut, the tool carrier system moves the grabber, which grasps the cut celery plant, up from the ground level to a conveyor system that conveys the cut celery to a packing station where the celery is packed for shipment. The rubber hand holding the celery and the robotic arm bring the cut celery up from the ground level to the conveyor system and the rubber hand rotates to drop the celery on rollers or a belt of the conveyor system. The conveyor system brings the cut celery back to a packing station where a crew hand packs the celery into wrapping plastic. See FIG. 7.Imaging System and Al Engine

[0063] The camera 318, which can be a stereo camera, is mounted to the adjustable tool carrier in close proximity to the grabber 308 and cutter 316 to capture real-time images of the crop to be operated on by the tool. In certain embodiments the camera is attached to a support that is rigidly attached to the tool so that the camera moves with the tool. Images from the camera are input into a computing device, which includes, in addition to one or more processors and memories, an Al engine with a deep learning prediction model. The model uses the images to predict a characteristic about the crop. The Al engine and model are described in International Application No. PCT / US2022 / 054273, incorporated by reference herein.

[0064] In the case of a celery harvesting application, the camera captures in real-time images of the celery plant and the model, having been trained to recognize the critical portion of a celery plant, predicts the location of the critical portion from the captured images. The captured images can be from a video stream output from the first camera. This prediction is used as input to a robotic controller that controls the horizontal and vertical positioning arms, or the articulable robotic arm, to adjust the position of the cutting tool 316 disposed on the end effector 405, to cut the celery plant at an optimal cutting location at the celery / soil joint. In some embodiments, the horizontal positioning arm is moved by an pneumatic driver and / or air compressor, while the vertical positioning arm is moved by an electrical motor.

[0065] In some embodiments, the optimal cutting location is difficult to see in the image due to leaves, braches, soil, or other plants obscuring the optimal cutting location. In order to more accurately analyze blurry or otherwise obscured images, the Al model may be trained using training data including images of plants, such as a celery plant, where the plants are heavily obscured by blur or obstacles. The Al model may produce an Al mask showing the predicted optimal cutting location. The robotic controller controls the position of the robotic arm and tool based on the Al mask output of the optimal cutting location to make a cut at the appropriate location on the plant while avoiding the body or other portion of the plant such as a protected portion of the plant. Based on the predicted optimal cutting location, the cutting tool can be positioned at a location that follows the optimal cutting location. In some embodiments the location of the cutting tool is set to be placed a predetermined distance above or below the root of the plant, the heart of the plant, or the body of the plant.

[0066] In some embodiments, based on the images captured by the first camera, the Al predication model outputs an Al mask indicating the optimal cutting location. This output may be provided to a control algorithm which also receives inputs from sensors, which can be sensors on the robotic tool carrier system, sensors on the vehicle, or other external sensors. The control algorithm maycompare the predicted location of the optimal cutting location from the output Al mask with the information input from the sensors to determine the difference between the sensed location of the robotic arm and the predicted optimal cutting location.Hardware / Software Environment

[0067] A hardware configuration of an information processing system 800 according to one exemplary embodiment is shown in FIG. 8. This embodiment can be used to implement, for example, the controller 320, and other computer implemented structures disclosed herein. While the information processing system 800 shown in FIG. 8 illustrates various components, not all components are necessary to use in various embodiments of the computing structures described herein.

[0068] Fig. 8 is a block diagram illustrating a hardware configuration of an information processing system 800 according to an example embodiment. The Al powered robotic tool carrier system 300 can be structured, in certain embodiments, with one or more of the components of the information processing system 800 shown in FIG. 8. For example, the Al engine implementing a deep learning prediction model can be structured, in certain embodiments, with one or more of the components of the information processing system 800. Similarly, a control algorithm and a robotic controller can be implemented with one or more of the components of the information processing system 800. Further, the information processing system 800 has a function of a computer. For example, the information processing system 800 may be configured integrally within an embedded controller, and in other embodiments it may be configured with a general purpose computer such as a personal computer (PC), a laptop PC, a tablet PC, a smartphone, or the like.

[0069] The information processing system 800 has a processor 802, a random access memory (RAM) 806, a read only memory (ROM) 808, and a possibly a mass storage device (MSD) 810 such as a hard disk drive (HDD), an optical disk drive, an electrically erasable ROM (EEROM) or other semiconductor memory, or another known device for persistently storing large quantities of data in orderto perform storage and retrieval of electronic data. Further, the information processing system 800 can include a serial input / output (VO) interface (VF) 812 for connection to a serial bus. In certain embodiments the information processing system 800 can include communication interfaces 814 for communications protocols other than serial data communication. In certain embodiments the information processing system 800 can include a display device 816, an input device 818, and other output devices 820. The processor 802, the RAM 806, the ROM 808, the MSD 810, the serial VO communication VF 814, the other communication interfaces 814, the display device 816, the input device 818, and the other output devices 820 are connected to each other via a bus 804. According to an example embodiment, the display device 816, the input device 818, the other output devices 820 may be connected to the bus 804 via a drive device (not illustrated) used for driving these devices. According to an example embodiment, the processor 802 may be a central processing unit (CPU), a microcontroller, other types of controllers, or the like. Moreover, in some embodiments the processor 802 may be comprised of one or more processors, such as a plurality of CPUs or microcontrollers. According to another example embodiment, the processor 802 may be a hardware processor. According to another example embodiment, the processor 802 may be implemented by a combination of hardware, software, and / or firmware components. According to another example embodiment, the processor 802 may be implemented by a configuration of electronic components including one or more circuitry components.

[0070] While respective components forming the information processing system 800 are illustrated in Fig. 8 as an integrated device, some of the components and / or some of the functions performed by the components thereof may be performed by an externally attached device. For example, the display device 816, the input device 818, and the other output devices 820 may be externally attached devices that are separate from apart from the components performing the functions of a computer including the processor 802 or the like.

[0071] The processor 802 has a function of performing an operation in accordance with a program stored in the ROM 808, the MSD 810, or the like, and controlling each component of the information processing system 800. According to an example embodiment, the processor 802 may obtain one or more instructions stored in the ROM 808, the MSD 810, or the like and execute the one or more instructions to perform one or more operations. The one or more operations may include controlling one or more components of the information processing system 800 to perform one or more operations. The RAM 806 is formed of a volatile storage medium and provides a temporary memory field used in the operation of the processor 802. The ROM 808 is formed of a nonvolatile storage medium and stores information such as a program used in the operation of the information processing system 800. The MSD 810 is a storage device that is formed of a nonvolatile storage medium and stores electronic data, such as message captured by a message collection device, or the like.

[0072] The other communication I / F 814 may be a communication interface based on a specification such as an 802.11 wireless communication standard, a 3GPP standard for cellular communication, or the like, which is a module for communicating with other devices. The display device 816 may be a liquid crystal display, an organic light emitting diode (OLED) display, or any other computer controlled device capable of displaying a moving image, a static image, a text, or the like. Examples of the input device 818 are a button, a touchscreen, a keyboard, a pointing device, or the like and capable of use by a user to operate the information processing system 800. The display device 816 and the input device 818 may be integrally formed such as in a touchscreen.

[0073] According to an example embodiment, the hardware configuration illustrated in Fig. 8 is an example embodiment of a processing system, and components or devices, other than those illustrated in FIG. 8, may be added, or some of the components or devices shown may not be provided in certain embodiments. Further, some of the components or devices may be replaced with another component or device having a similar function. Furthermore, some of the functions may be provided by anothercomponent or device via a network, or the functions forming the example embodiment may be implemented by being distributed in a plurality of components or devices. For example, the MSD 810 may be replaced with cloud storage.Grabbing Mechanisms

[0074] Additional embodiments of the grabbing mechanism shown in FIG. 5 are illustrated in FIGS. 9A, 9B, 10A, 10B, and 11.

[0075] In certain embodiments, as illustrated in Fig. 9a and Fig. 9b, one or more grabbing devices 901 may be attached to the robotic tool carrier system.

[0076] In some embodiments, the grabbing devices each have two arms 905 each arm including one or more bars 902. In some embodiments, the bars may be curved. In certain embodiments, the curvature of the bars ranges from 30 degrees to 60 degrees, inclusive. Such a range facilitates efficient grabbing of the crop without need for excessive force.

[0077] In some embodiments, the bars may be constructed out of and / or coated with foam. Use of foam and / or a foam coating offers enhanced harvesting efficiency by reducing bruising and / or breaking of crops during the harvesting process.

[0078] In some embodiments, the grabbing mechanism is configured to close the two arms around a plant in order to apply a certain amount of force to the plant. In some embodiments, the amount of force is determined by the Al 302 and / or the control algorithm 305. In some embodiments, the amount of force is adjustable using a current provided to an electrical motor of the grabber.

[0079] In some embodiments, the grabbing mechanism includes a plurality of sections 903 and 904, each of which may have a different curvature, bar material, bar coating, and / or bar spacing from any of the other sections of the grabbing mechanism. In some embodiments, one or more sections 903 nearest or nearer to the robotic tool carrier system has a greater density of bars and / or greater size of bar than a section further from the robotic tool carrier. Increasing coverage of the grabbing mechanismnearest the robotic tool carrier facilitates reliable grabbing of the plant, allowing for a reduction in force needed to reliably grab the plant, thus reducing the risk of damaging the grabbed plant. In some embodiments, one or more sections 904 distant or most distant from the robotic tool carrier system may have reducing density and / or bar spacing. In some embodiments, the one or more distant or most distant sections may be configured such that the sections of the first arm may be placed in an alternating fashion with the section of the second arm, enabling the alternating sections of the arms to interlock. In some embodiments, all sections of both arms may be placed in alternating fashion, enabling the entirely of the arms to interlock. In some embodiments the grabbing mechanism may be configured to close only a portion of the plurality of sections.Cutting Mechanism

[0080] In certain embodiments, as illustrated in Figs. 12a, 12b, and 13 one or more top cutting devices 1000 may be attached to the robotic tool carrier system.

[0081] In certain embodiments, the one or more top cutting devices may include scissor blades 1201. The scissor blades 1201 are disposed at the top end of grabber and configured to cut the top end of a celery plant when the celery plant is grasped by the grabber and separated from the soil. In certain embodiments, one or more of the scissor blades may be driven by a piston mechanism 1204 into a open position as in Fig. 12a or into a closed position as in Fig. 12b. In some embodiments, one or more of the scissor blades may be attached or connected to a trough 1203. In some embodiments, one or more prongs 1202 may be attached to one or more of the scissor blades. In some embodiments, the one or more prongs may be sized in order to fit in the trough. In some embodiments, the trough, scissor blades, prongs, and piston mechanism are placed and / or sized such that driving the piston in a horizontal direction causes two or more scissor blades to close in a pincer cutting motion. In some embodiments, the piston may be pneumatically driven. In some embodiments, the piston may be electrically driven. Insome embodiments, the piston may be hydraulically driven. In some embodiments, when the piston is pneumatically driven, an air compressor drives the pneumatic action of the piston.

[0082] As will be appreciated by one of ordinary skill in the art, a pneumatically driven piston offers enhanced cleanliness when compared to a hydraulic system by eliminating the possibility of hydraulic fluid leakage from the system. Thus, a pneumatically driven system increases the safety of the system by allowing for cleaner harvested plants.

[0083] The cutting mechanism also includes a bottom cutting device operating at the bottom end of the grabber. In one embodiment the bottom cutting device includes a blade that extends from the tool carrier apparatus in the direction of the movement of the tractor towards the celery plant. Once the tool carrier apparatus adjusts the position of the bottom cutting device to the final cut location determined using the imaging and Al systems, when the celery plant is in range for cutting the bottom cutting device is activated and extends towards the celery plant and cuts the celery plant at the final cut location. In this way the celery plant is separated from the soil and freed to be harvested. The bottom cutting device may be driven by a piston mechanism supplying enough force to consistently drive the bottom cutting device through a root of the plant. In some embodiments, the piston may be pneumatically driven. In some embodiments, the piston may be electrically driven. In some embodiments, the piston may be hydraulically driven.Mechanized Vegetable Harvesting Process

[0084] Fig. 14 illustrates an example mechanized vegetable harvesting process according to some embodiments and some configurations of the invention. As will be appreciated by one of ordinary skill in the art, although the operations are listed sequentially the operations may be performed in alternative ordering, simultaneously, or while omitting or repeating certain operations.

[0085] At operation 1702, the camera images the target crop.

[0086] At operation 1703, the robotic tool carrier system analyzes the images received from the camera.

[0087] At operation 1704, the robotic tool carrier system adjusts a location of the first camera and / or working tool based on results of the analysis.

[0088] At operation 1705, the robotic tool carrier system determines whether the working tool is located at an optimal location based on results of the analysis.

[0089] At operation 1706, the robotic tool carrier system returns to operation 1701 based on determining that the working tool is not located at an optimal location.

[0090] At operation 1707, the robotic tool carrier system performs a grabbing operation on the target plant using the grabbing apparatus.

[0091] At operation 1708, the robotic tool carrier system performs a work operation on the target plant using the working tool.

[0092] At operation 1709, the robotic tool carrier system performs a cutting operation on the target plant using the secondary cutting device.

[0093] At operation 1710, the robotic tool carrier system raises the grabbed portion of the plant.

[0094] At operation 1711, the robotic tool carrier system moves the grabbed portion of the plant to a storage area, and releases the plant.

[0095] Fig. 15 illustrates an example process of the image analyzing performed during operation 1703. At operation 1501, an image is received from the camera. At operation 1502, the image is input to the Al Engine. At operation 1503, the Al engine generates mask information based on the input image, and outputs the mask image to the robotic controller.

[0096] Fig. 16 illustrates an example process of the position adjustment performed during operation 1704. At operation 1601, the robotic controller receives the mask information from the AlEngine. At operation 1602, the controller adjusts the position of the end effector, grabbing mechanism, and / or cutting tool based on the received mask information.

[0097] While the subject matter of the present application has been particularly shown and described with reference to illustrative embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. The illustrative embodiments should be considered in a descriptive sense only and not for purposes of limitation.

[0098] While the various embodiments described herein may contain different components and features, upon reading the specification, one skilled in the art readily will realize that such components and features in one embodiment may be incorporated into or combined with components and features of another embodiment. Also, the description of various embodiments is provided to enable a person skilled in the art to make and use the present disclosure. Moreover, various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles and specific examples defined herein may be applied to other embodiments without the use of inventive faculty.Therefore, the present disclosure is not intended to be limited to the embodiments described herein but is to be accorded the widest scope as defined by the limitations of the claims and equivalents thereof

[0099] The whole or part of the example embodiments disclosed above can be described as, but not limited to, the following supplementary notes.Supplemental Notes[SUPPLEMENTAL NOTE 1]A tool carrier apparatus, including: a tool for working on a plant planted in the ground, wherein the tool includes a first cutting mechanism configured to cut a portion of the plant;an adjustable carrier configured to hold the tool and move the tool in a horizontal direction and a vertical direction with respect to the ground and configured to mount to a vehicle; a camera configured to capture an image of a plant, the plant having an optimal cutting location; a memory having a program stored therein; a processor that when executing the program a control algorithm outputting a control command based on the output from the artificial intelligence engine; implements: an artificial intelligence engine trained to identify the optimal cutting location of the plant, receive the captured image of the plant, and output an indication of the optimal cutting location of the plant; and a robotic controller configured to control the adjustable carrier based on the control command to position the tool to cut the plant at the optimal cutting location.[SUPPLEMENTAL NOTE 2]The tool carrier apparatus according to supplemental note 1, wherein the camera is mounted on the adjustable carrier.[SUPPLEMENTAL NOTE 3]The tool carrier apparatus according to supplemental note 2, wherein the plant is a celery plant. [SUPPLEMENTAL NOTE 4]The tool carrier apparatus according to supplemental note 2, wherein the adjustable carrier comprises an adjustable horizontal arm moveable in the horizontal direction, an adjustable vertical arm moveable in the vertical direction with respect to the ground, and an end effector attached to one of the adjustable horizontal arm and the adjustable vertical arm and configured to hold the tool. [SUPPLEMENTAL NOTE 5]The tool carrier apparatus according to supplemental note 4, wherein the camera is attached to the end effector by a rigid support and in close proximity to the tool.[SUPPLEMENTAL NOTE 6]The tool carrier apparatus according to supplemental note 2, wherein the plant is a vegetable and the artificial intelligence engine is trained to identify the protected portion of the plant so that the robotic controller causes the position of the cutting tool to correspond to a predicted portion of the vegetable between a lower point of the vegetable and an upper point of the vegetable.[SUPPLEMENTAL NOTE 7]The tool carrier apparatus according to supplemental note 6, wherein the lower point of the vegetable corresponds to a point where soil is not taken when the vegetable is cut and the upper point of the vegetable corresponds to a point where the cut vegetable is not likely to divide into separate pieces. [SUPPLEMENTAL NOTE 8]The tool carrier apparatus according to supplemental note 2, wherein the plant is a crop planted in one of a plurality of rows of the crop, and the tool is configured to extract a weed disposed between the rows of crops while avoiding damaging the plant.[SUPPLEMENTAL NOTE 9]The tool carrier apparatus according to supplemental note 2, wherein the vehicle is a tractor. [SUPPLEMENTAL NOTE 10]The tool carrier apparatus according to supplemental note 1, wherein the tool further comprises a grabbing apparatus.[SUPPLEMENTAL NOTE 11]The tool carrier apparatus according to supplemental note 10, wherein the grabbing apparatus is configured to grab the plant prior to performing work on the plant, and adjust a position of the plant to provide a clear path between the camera and the optimal cutting location.[SUPPLEMENTAL NOTE 12]The tool carrier apparatus according to supplemental note 1, wherein the tool further comprises a secondary cutting mechanism configured to operate a second cutting operation subsequent to operation of the first cutting mechanism.[SUPPLEMENTAL NOTE 13]The tool carrier apparatus according to supplemental note 12, wherein the robotic controller is further configured to control the adjustable carrier to, subsequent to the second cutting operation, place the cut plant in a transport device for transport of the cut plant to a storage location. [SUPPLEMENTAL NOTE 14]The tool carrier apparatus according to supplemental note 13, wherein the transport device is a conveyor.[SUPPLEMENTAL NOTE 15]A tool positioning method, including: capturing an image of a plant, the plant having an optimal cutting location; receiving, at an artificial intelligence engine trained to identify the optimal cutting location of the plant, a captured image of the plant, and output an indication of the optimal cutting location of the plant; outputting a control command based on the output from the artificial intelligence engine; and controlling an adjustable carrier based on the control command to position a tool to cut the plant at the optimal cutting location.[SUPPLEMENTAL NOTE 16]The tool positioning method according to supplemental note 15, wherein the plant is a celery plant.[SUPPLEMENTAL NOTE 17]A tool positioning non-transitory computer-readable medium, including: at least one non-transitory computer-readable medium containing program code, the program code configured to, when executed by at least one processor, cause at least one of the least one processors to: capture an image of a plant, the plant having an optimal cutting location; receive, at an artificial intelligence engine trained to identify the optimal cutting location of the plant, a captured image of the plant, and output an indication of the optimal cutting location of the plant; output a control command based on the output from the artificial intelligence engine; and control an adjustable carrier based on the control command to position a tool to cut the plant at the optimal cutting location.[SUPPLEMENTAL NOTE 18]The tool positioning non-transitory computer-readable medium according to supplemental note 17, wherein the plant is a celery plant.

[0100] The foregoing disclosure provides illustration and description, but is not intended to be exhaustive or to limit the implementations to the forms explicitly described. Modifications and variations are possible in light of the above disclosure or may be acquired from practice of the implementations.

[0101] It will be apparent that systems and / or methods, described herein, may be implemented in different forms of hardware, firmware, or a combination of hardware and software. The actual specialized control hardware or software code used to implement these systems and / or methods is not limiting of embodiments of the present disclosure.

[0102] Even though combinations of features are recited in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of possible implementations.Many of the described features may be combined in ways not explicitly recited in the claims and / or explicitly described in the above disclosure. Although each dependent claim listed below may directly depend on only one claim, the disclosure of possible implementations includes each dependent claim in combination with every other claim in the claim set.

[0103] No element, act, or instruction used herein should be construed as critical or essential unless explicitly described as such. Also, as used herein, the articles “a” and “an” are intended to include one or more items, and may be used interchangeably with “one or more.” Also, as used herein, the terms “has,” “have,” “having,” “including” or the like are intended to be open-ended terms. Further, the phrase “based on” is intended to mean “based, at least in part, on” unless explicitly stated otherwise.The term “or” as used herein is an inclusive “or”, and has a meaning equivalent to “and / or.”

Claims

WHAT IS CLAIMED IS:

1. A tool carrier apparatus, comprising: a tool for working on a plant planted in the ground, wherein the tool includes a first cutting mechanism configured to cut a portion of the plant; an adjustable carrier configured to hold the tool and move the tool in a horizontal direction and a vertical direction with respect to the ground and configured to mount to a vehicle; a camera configured to capture an image of a plant, the plant having an optimal cutting location; a memory having a program stored therein; a processor that when executing the program a control algorithm outputting a control command based on the output from the artificial intelligence engine; implements: an artificial intelligence engine trained to identify the optimal cutting location of the plant, receive the captured image of the plant, and output an indication of the optimal cutting location of the plant; and a robotic controller configured to control the adjustable carrier based on the control command to position the tool to cut the plant at the optimal cutting location.

2. The tool carrier apparatus according to claim 1, wherein the camera is mounted on the adjustable carrier.

3. The tool carrier apparatus according to claim 2, wherein the plant is a celery plant.4 The tool carrier apparatus according to claim 2, wherein the adjustable carrier comprises an adjustable horizontal arm moveable in the horizontal direction, an adjustable vertical arm moveablein the vertical direction with respect to the ground, and an end effector attached to one of the adjustable horizontal arm and the adjustable vertical arm and configured to hold the tool.

5. The tool carrier apparatus according to claim 4, wherein the camera is attached to the end effector by a rigid support and in close proximity to the tool.

6. The tool carrier apparatus according to claim 2, wherein the plant is a vegetable and the artificial intelligence engine is trained to identify the protected portion of the plant so that the robotic controller causes the position of the cutting tool to correspond to a predicted portion of the vegetable etween a lower point of the vegetable and an upper point of the vegetable.

7. The tool carrier apparatus according to claim 6, wherein the lower point of the vegetable corresponds to a point where soil is not taken when the vegetable is cut and the upper point of the egetable corresponds to a point where the cut vegetable is not likely to divide into separate pieces.

8. The tool carrier apparatus according to claim 2, wherein the plant is a crop planted in one of a plurality of rows of the crop, and the tool is configured to extract a weed disposed between the rows of crops while avoiding damaging the plant.

9. The tool carrier apparatus according to claim 2, wherein the vehicle is a tractor.

10. The tool carrier apparatus according to claim 1, wherein the tool further comprises a grabbing apparatus.

11. The tool carrier apparatus according to claim 10, wherein the grabbing apparatus is configured to grab the plant prior to performing work on the plant, and adjust a position of the plant to provide a clear path between the camera and the optimal cutting location.

12. The tool carrier apparatus according to claim 1, wherein the tool further comprises a secondary cutting mechanism configured to operate a second cutting operation subsequent to operation of the first cutting mechanism.

13. The tool carrier apparatus according to claim 12, wherein the robotic controller is further configured to control the adjustable carrier to, subsequent to the second cutting operation, place the cut plant in a transport device for transport of the cut plant to a storage location.

14. The tool carrier apparatus according to claim 13, wherein the transport device is a conveyor.

15. A tool positioning method, comprising: capturing an image of a plant, the plant having an optimal cutting location; receiving, at an artificial intelligence engine trained to identify the optimal cutting location of the plant, a captured image of the plant, and output an indication of the optimal cutting location of the plant; outputting a control command based on the output from the artificial intelligence engine; and controlling an adjustable carrier based on the control command to position a tool to cut the plant at the optimal cutting location.

16. The tool positioning method according to claim 15, wherein the plant is a celery plant.

17. A tool positioning non-transitory computer-readable medium, comprising: at least one non-transitory computer-readable medium containing program code, the program code configured to, when executed by at least one processor, cause at least one of the least one processors to: capture an image of a plant, the plant having an optimal cutting location; receive, at an artificial intelligence engine trained to identify the optimal cutting location of the plant, a captured image of the plant, and output an indication of the optimal cutting location of the plant; output a control command based on the output from the artificial intelligence engine; and control an adjustable carrier based on the control command to position a tool to cut the plant at the optimal cutting location.

18. The tool positioning non-transitory computer-readable medium according to claim 17, herein the plant is a celery plant.

Images