Robotic Harvesting System with Gantry System

Abstract

The robotic harvesting system includes a base, a linear translation section, a robotic arm, and an end effector. The base is configured to move in a direction of travel. The linear translation section is attached to the base. The linear translation section is configured to move along the base in a direction substantially the same as or opposite to the direction of travel. The robotic arm is attached to the linear translation section. The robotic arm has a proximal end and a distal end. The distal end of the robotic arm is configured to rotate from a first joint toward and away from the base. The end effector is attached to the distal end of the robotic arm.

Description

[Background technology]

[0001] The process of harvesting objects (e.g., fruits, flowers, vegetables, etc.) from multiple plants is a time-consuming process. Often, humans are deployed to harvest the objects, but harvesting the objects can become tiring. As a result, the efficiency with which humans can harvest the objects decreases over time. It would be beneficial to deploy a robotic system to harvest the objects. [Brief explanation of the drawings]

[0002] Various embodiments of the present invention are disclosed in the following detailed description and the accompanying drawings.

[0003] [Figure 1] FIG. 1 illustrates an embodiment of a robotic harvesting system, according to some embodiments.

[0004] [Figure 2] 1 illustrates a gantry system, according to some embodiments.

[0005] [Figure 3] 1 illustrates an arm module, according to some embodiments.

[0006] [Figure 4] 10A-10C illustrate multiple approach angles according to some embodiments.

[0007] [Figure 5] FIG. 1 illustrates a growing environment, according to some embodiments.

[0008] [Figure 6] 1 is a flowchart illustrating a process for harvesting an object according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0009] The present invention may be embodied in various forms, including as a process, an apparatus, a system, a composition of matter, a computer program product embodied on a computer-readable storage medium, and / or a processor configured to execute instructions stored in and / or provided by a memory coupled to the processor. These embodiments, or any other form the present invention may take, may be referred to herein as technology. In general, the order of steps in a disclosed process may be varied within the scope of the present invention. Unless otherwise noted, components, such as a processor or memory, described as configured to perform a task may be implemented as general components temporarily configured to perform the task at a given time, or as specific components manufactured to perform the task. As used herein, the term “processor” refers to one or more devices, circuits, and / or processing cores configured to process data, such as computer program instructions.

[0010] The following is a detailed description of one or more embodiments of the present invention with reference to figures that illustrate the principles of the invention. While the present invention has been described in connection with such embodiments, it is not limited to any particular embodiment. The scope of the present invention is limited only by the claims, and the present invention includes many alternatives, modifications, and equivalents. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. These details are for the purpose of example, and the present invention may be practiced according to the claims without some or all of these specific details. For simplicity, technical matters that are well known in the art related to the present invention have not been described in detail so as not to unnecessarily obscure the present invention.

[0011] Disclosed herein is a robotic harvesting system. The robotic harvesting system includes at least a base, one or more linear motion units, one or more robotic arms, and one or more end effectors. The base includes a plurality of wheels or tracks that enable the robotic harvesting system to move (autonomously or by remote control) through a growing environment. The robotic harvesting system may be positioned at a first location to harvest a first set of objects (e.g., fruits, flowers, etc.). The robotic harvesting system includes one or more cameras and / or other sensors to determine corresponding positions of the plurality of objects included in the first set at the first location.

[0012] A linear moving unit is attached to the base and configured to move along the base in substantially the same direction as or opposite to the direction of travel of the robotic harvesting system. The linear moving unit is coupled to the elongated member. A robotic arm is coupled to the elongated member via a first joint. The elongated member is configured to move up and down along the vertical direction of the robotic harvesting system. The robotic arm is configured to move up and down along the vertical direction of the elongated member. The robotic harvesting system includes a control system. The control system is coupled to the linear moving unit, the elongated member, and the robotic arm. The control system can send one or more commands to move the linear moving unit along the base, one or more commands to move the elongated member up and down along the vertical direction of the robotic harvesting system, and / or one or more commands to move the robotic arm up and down along the vertical direction of the elongated member. As a result, the robotic arm can be moved to harvest objects within its reach at a particular location.

[0013] The robotic arm is composed of at least a first segment and a second segment. The first segment of the robotic arm is coupled to the elongated member via a first joint. The first segment of the robotic arm can be rotated clockwise or counterclockwise about the first joint. The second segment of the robotic arm is coupled to the first segment of the robotic arm via a second joint. The second segment of the robotic arm can be rotated clockwise or counterclockwise about the second joint. As a result, the robotic arm can approach the harvesting object from any angle (e.g., 360°) or nearly any angle (e.g., 356°) depending on the cultivation environment in which the harvesting object exists (e.g., whether or not there are any obstacles).

[0014] Other robotic harvesting systems may have robotic arms that are limited by kinematic or mechanical design to approach harvest targets from a single angle (e.g., a 90° angle). One or more obstacles may be present in the harvesting path for harvesting the target. The obstacle(s) may include vines, branches, mechanical structures supporting the plant, other harvest targets, etc. These other systems may not be able to manipulate the robotic arm to prevent contact with the obstacle(s). Contact may cause damage to one or more other harvest targets. Contact may also move the obstacle (e.g., a vine), which may make harvesting the target in a particular location difficult because the moving obstacle may prevent the harvesting of one or more other targets. In contrast, the robotic arms disclosed herein provide minimal degrees of freedom to approach harvest targets from any angle (e.g., 360°) or nearly any angle (e.g., 356°), depending on the growing environment in which the harvest targets reside. This allows the robotic harvesting system to select an obstacle-free harvesting path for harvesting objects that prevents the robotic arm from contacting one or more obstacles.

[0015] An end effector is coupled to the distal end of the robotic arm. A second segment of the robotic arm may include one or more cameras and / or one or more other sensors (e.g., a depth sensor, SONAR, RADAR, LIDAR, IMU, GNSS, etc.) pointed toward the end effector. A processing system of the robotic arm may receive one or more images from the one or more cameras and / or one or more other sensors. The processing system may identify one or more objects from the one or more images and assign an associated cost to the one or more identified objects. The processing system may select one of the one or more identified objects for harvesting based on the associated cost and determine a relative distance between the current position of the end effector and the current position of the object. The processing system may send a command to a control system of the robotic harvesting system to move the robotic arm the relative distance. In response to receiving the command, the control system may move the linear translation section, the elongated member, and / or the robotic arm to reposition the robotic arm the determined relative distance.

[0016] The end effector may be comprised of a cutting mechanism and a gripping mechanism, each with a corresponding set of jaws. The end effector may be configured to simultaneously open and close the cutting mechanism and the gripping mechanism. When the cutting mechanism and the gripping mechanism are open, a plant appendage connected to the object may be positioned within an opening associated with the corresponding set of jaws. An actuator associated with the end effector may apply a force that causes the cutting mechanism and the gripping mechanism to close. The applied force causes the cutting mechanism to perform a cut (e.g., a bypass cut) that separates the plant appendage into a first portion and a second portion, while the applied force causes the gripping mechanism to grip the second portion of the plant appendage connected to the harvest object. The robotic harvesting system may move the end effector to a storage location and open the end effector, thereby depositing the harvested object in the storage location.

[0017] After harvesting a first set of objects at a first location in the growing environment, the robotic harvesting system may move to a second location in the growing environment to harvest a second set of objects. The robotic harvesting system may be configured to move a predetermined distance (e.g., 1 m) after the first set of objects is harvested. In some embodiments, the predetermined distance is the length of the robotic harvesting system. In some embodiments, the predetermined distance is based on the density of the harvested objects. In some embodiments, the predetermined distance is a fraction of the length of the base (e.g., 8 / 10) to create some overlap in the reachable areas at each location.

[0018] Each time the robotic harvesting system is moved to a new location, one or more cameras and / or one or more other sensors of the robotic harvesting system may acquire additional images to enable the processing system to determine the corresponding positions of one or more harvest objects at the new location. This introduces additional delays and processing costs into the overall time it takes to harvest the objects. The configuration of the robotic harvesting system disclosed herein is optimized to harvest as many objects as possible at a particular location without having to move the robotic harvesting system. In some embodiments, images of objects to be harvested at the new location are acquired at the current location of the robotic harvesting system. Images acquired from a depth camera may be used to determine the corresponding positions of the objects to be harvested at the new location. This allows the robotic harvesting system to harvest objects at the new location without having to acquire additional images when the robotic harvesting system is at the new location.

[0019] The robotic harvesting system may repeat the process of harvesting multiple objects and moving on to the next location until the storage capacity for the harvested objects is reached and the robotic harvesting system is unable to harvest additional objects. The robotic harvesting system is configured to wait until an operator or another robotic system removes the harvested objects from the robotic harvesting system. The robotic harvesting system may then resume the harvesting process. When the robotic harvesting system reaches the end of a row in the growing environment, it moves to the next row in the growing environment and resumes the harvesting process. When the robotic harvesting system reaches the end of the last row, it completes that round of object harvesting and awaits further instructions to begin a new round of object harvesting.

[0020] 1 is a diagram illustrating one embodiment of a robotic harvesting system, according to some embodiments. In the illustrated example, robotic harvesting system 100 includes base 102, gantry system 104, robotic arms 122, 124, and end effectors 132, 134.

[0021] The components of the robotic harvesting system may be manufactured using low-cost manufacturing techniques, such as those used to manufacture sheet metal. When assembled, these components have a total weight of approximately 4-5 kg. This allows the robotic harvesting system to be quickly moved to various positions, allowing the robotic arm to be quickly moved to various positions at a specific location. In contrast, other systems may utilize pre-fabricated components, including complex machined castings. These components may be significantly heavier. For example, the total weight of the components of other robotic harvesting systems may be approximately 40 kg. As a result, the speed at which these other robotic harvesting systems can harvest objects may be slower than robotic harvesting system 100.

[0022] A growing area may be comprised of multiple rows. The width of the base 102 may be designed to accommodate multiple row widths. For example, the row width may be 1 meter. The row width may vary based on location (e.g., different countries, different farms) or the type of object being grown. For example, the row width in Europe (e.g., the UK) may be different from the row width in the United States. In some embodiments, the width of the base 102 may be between 0.5 meters and 1.5 meters. This may allow the robotic harvesting machine 100 to move between rows without contacting the rows on either side of the harvesting machine 100. The length of the base 102 may be selected so that the robotic harvesting system 100 harvests a specific amount of object (e.g., 10 to 15 fruits) at a specific location when parked. For example, the length of the base 102 may be between 1 meter and 3 meters. The length of the base 102 may be less than 1 meter, but this may require the robotic harvesting system 100 to be moved more frequently to harvest the same amount of objects compared to a harvesting system with a base between 1 meter and 3 meters. This may also increase the time required to harvest the same amount of objects compared to a harvesting system with a base between 1 meter and 3 meters, because the robotic harvesting system 100 may need to reorient itself relative to the growing system to determine where objects can be harvested each time the harvesting system is moved and stopped. This additional calculation time increases the time required to harvest the objects. The length of the base 102 may be greater than 3 meters, but such a length may make maneuvering the robotic harvesting system 100 too cumbersome and / or difficult.

[0023] Some growing areas (such as greenhouses) may have existing infrastructure between rows. For example, greenhouses may have rails between rows. The shape of base 102 may be designed to allow robotic harvesting system 100 to move between rows without contacting the existing infrastructure. For example, as seen in FIG. 1 , the shape of base 102 may include a U-shaped channel portion. This may allow robotic harvesting system 100 to be used in different types of growing environments without having to manufacture the harvesting system for the specific dimensions of the growing environment.

[0024] Base 102 is coupled to a number of rolling components 106. The rolling components 106 may be wheels, rail wheels, treads, rollers, tracks, or any other rolling components that enable robotic harvesting system 100 to move in various directions (e.g., forward, backward, left, right, and / or any combination thereof).

[0025] Base 102 may include some or all of control system 108. Control system 108 may be comprised of one or more processors. In some embodiments, the one or more processors are graphics processing units. The one or more processors may send one or more commands to move the plurality of rotating components 106. For example, the one or more commands may move robotic harvesting system 100 forward, backward, left, right, and / or any combination thereof. In some embodiments, the one or more commands move robotic harvesting system 100 a predetermined distance (e.g., 1 meter).

[0026] Control system 108 is connected to a power source (e.g., one or more batteries). The power source is configured to provide power to various components of robotic harvesting system 100. Robotic harvesting system 100 may include a charging port that allows the power source to be recharged. In some embodiments, the charging port allows the power source to be recharged via a power cord plugged into an outlet, a charging station, or a mobile charging device. In some embodiments, the charging port allows for wireless charging of the power source.

[0027] The control system 108 may include one or more circuit boards. In some embodiments, the control system 108 includes a main processing circuit board and one or more sub-processing circuit boards. The main processing circuit board with one or more processors may be located on the base 102, and the one or more sub-processing circuit boards with one or more processors may be located on corresponding portions of the robotic arms 122, 124 and / or gantry system 104. The main processing circuit board may be configured to perform more complex calculations, such as computer vision, while the one or more sub-processing circuit boards may be configured to perform less complex calculations, such as motion control. In some embodiments, the main processing circuit board and the one or more sub-processing circuit boards are integrated onto a single circuit board.

[0028] The control system 108 is connected to the drive module 110. The drive module 110 may include a self-navigation system that utilizes output from a sensor system 114. The sensor system 114 may be comprised of one or more cameras and / or one or more other sensors and utilizes computer vision algorithms for maneuvering and obstacle avoidance. The one or more cameras may be RGB cameras, RGBD cameras, RGB / IR "time-of-flight" cameras, high-end high-resolution cameras using multispectral imaging, or a combination thereof. In some embodiments, the sensor system 114 is configured to capture images at regular intervals (e.g., every second, every minute, every hour, etc.). The sensor system 114 may obtain information such as structural, color, and / or depth information using infrared data from structured infrared, time-of-flight, stereo, LIDAR integration, motion, or other means. The sensor system 114 may include one or more other sensors such as radar, ultrasonic, LIDAR (Light Detection and Ranging), IMU (Inertial Measurement Unit), GNSS (Global Navigation Satellite System), etc.

[0029] Robotic harvesting system 100 may include headlights 112 that allow robotic harvesting system 100 to be used in low light conditions. Robotic harvesting system 100 may include status lights 116 that indicate the current state of robotic harvesting system 100. For example, status lights 116 may indicate whether robotic harvesting system 100 is parked, moving, and / or harvesting an object. The color of status lights 116 may change depending on the current state of robotic harvesting system 100.

[0030] The robotic harvesting system 100 may include a gantry system 104. The gantry system 104 may be comprised of two vertical support beams, a cross beam, and a base frame. In some embodiments, the two vertical support beams and cross beam of the gantry system 104 are narrower than the width of the base 102. This allows the robotic harvesting system 100 to move next to the rows without accidentally contacting the harvesting objects when the harvesting objects overhang the rows. The gantry system 104 may include a movement assembly (not shown) comprised of a motor and belt system, which allows the base frame of the gantry system 104 to move up and down along the vertical axis of the robotic harvesting system 100. Objects harvested by the robotic harvesting system 100 may be located at different heights. For example, grapes hanging from a vine may be located at a first height, and cherries hanging from a branch may be located at a second height. The movement assembly allows the robotic harvesting system 100 to adjust the reach of the robotic arms 122, 124. For example, the robotic harvesting system 100 may be capable of harvesting objects from a range of heights (e.g., from 1 foot (304.8 mm) above ground to 10 feet (3048 mm) above ground). In some embodiments, the gantry system 104 includes multiple in-line central shafts with single or multiple fixed cross beams.

[0031] The robotic harvesting system 100 includes a linear translation unit 118. In some embodiments, the cross beam of the gantry system 104 includes a linear rail to which the linear translation unit 118 is coupled. The linear rail may be coupled to a second translation assembly (not shown) comprised of a motor and belt system. The second translation assembly may be capable of moving the linear translation unit 118 in substantially the same direction as or opposite to the direction of travel of the base 102. The robotic harvesting system 100 may include a second linear translation unit 120. The second linear translation unit 120 may function similarly to the linear translation unit 118.

[0032] The linear traveler 118 may comprise an elongated member. In some embodiments, the elongated member is a rod. In some embodiments, the linear traveler 118 comprises a translation assembly that moves the elongated member up and down in the vertical direction of the robotic harvesting system 100. The translation assembly may comprise a lead screw, a belt, and a servo motor. The robotic arm 122 may be attached to the elongated member. The robotic arm 122 may be moved up or down depending on the height of the harvesting object and / or the angle at which the object is harvested.

[0033] The robotic arm 122 comprises a distal end and a proximal end. The robotic arm 122 may comprise one or more segments. In the illustrated example, the robotic arm 122 comprises two segments. A first segment of the robotic arm 122 may be coupled to the elongate member via a first joint. A second segment of the robotic arm 122 may be coupled to the elongate member via a second joint. ThroughThe robotic arm 122 may be coupled to a first segment of the robotic arm 122 via a rotational axis. In some embodiments, the first segment of the robotic arm 122 moves up and down the vertical direction of the elongate member. The proximal end of the robotic arm 122 may be rotated clockwise or counterclockwise, causing the first segment to rotate toward and away from the base 102. The distal end of the robotic arm 122 may be rotated clockwise or counterclockwise, causing the second segment to rotate toward and away from the base 102. The configuration of the robotic arm 122 enables the robotic harvesting system 100 to approach and pick targets from any angle or nearly any angle (e.g., 356°), depending on the growing environment in which the targets are present (e.g., whether or not there are any obstacles). The control system 108 may determine the approach angle for a particular target and provide one or more commands to the control board that cause the robotic arm 122 to approach and pick the target from the determined approach angle. The determined approach angle may provide an obstacle-free path for the robotic arm to approach and harvest the harvesting target. Approaching and harvesting the target at the determined approach angle may prevent the robotic arm 122 from accidentally contacting an obstacle (e.g., a vine, branch, or other harvesting target) when attempting to harvest the target. Such contact may cause the obstacle to move, which may make harvesting the target more difficult because the obstacle may block (partially, completely, temporarily, or permanently) the obstacle-free path to the harvesting target. Approaching and harvesting the target at the determined approach angle may also prevent the robotic arm 122 from potentially damaging other harvesting targets when attempting to harvest the target.

[0034] Robotic harvesting system 100 may include a second robotic arm 124. Second robotic arm 124 may function similarly to robotic arm 122. In some embodiments, robotic arm 122 and robotic arm 124 harvest objects on opposite sides of base 102. In some embodiments, robotic arm 122 or robotic arm 124 is rotated so that robotic arms 122, 124 harvest objects on the same side of base 102.

[0035] The end effector 132 is attached to the distal end of the robotic arm 122. The end effector 132 may comprise a cutting mechanism and a gripping mechanism. The cutting mechanism and the gripping mechanism include corresponding sets of jaws. The end effector 132 is configured to simultaneously open and close the cutting mechanism and the gripping mechanism. The plant may include appendages (e.g., stems, vines, branches, stalks, etc.) and harvest objects (e.g., flowers, fruits, vegetables, etc.). When the cutting mechanism and the gripping mechanism are open, the plant appendage may be positioned within an opening associated with the corresponding set of jaws. An actuator associated with the end effector 132 may apply a force that closes the cutting mechanism and the gripping mechanism. The applied force causes the cutting mechanism to perform a cut that separates the plant appendage into a first portion and a second portion, while the applied force causes the gripping mechanism to grip the second portion of the plant appendage attached to the harvest object. As a result, the end effector 132 can remove the harvesting objects from the plant without damaging the harvesting objects. The harvested objects may be placed in a container (such as container 142).

[0036] Robotic harvesting system 100 may include a second end effector 134. Second end effector 134 may function similarly to end effector 132.

[0037] 2 is a diagram illustrating a gantry system, according to some embodiments. In the illustrated example, gantry system 200 includes a base frame 202. The top of base frame 202 includes a storage section 204. Storage section 204 may be capable of accommodating one or more trays supporting multiple containers (e.g., boxes, baskets, etc.). Harvested objects may be placed into one of the containers by a robotic arm. In some embodiments, a tray can support eight or ten containers. In some embodiments, storage section 204 can accommodate two trays.

[0038] The gantry system 200 may include a communication device 206. The communication device 206 may enable the harvesting system to communicate wirelessly over a local area network, a wide area network, an intranet, the Internet, and / or combinations thereof. The communication device 206 may enable a control system (such as control system 108) to receive software downloads and / or updates. In some embodiments, the communication device 206 transmits harvest information to a remote server (e.g., a cloud-based database). The harvest information may include recorded information about the harvest event, such as a timestamp, success / failure status, location, fruit size, quality, estimated weight, unique identifier, approach angle, and / or other information related to the harvest event. The remote server may store current and past plant structure and morphology information (e.g., leaf count, plant structure).

[0039] The remote server may include a database that may include a graphical user interface that can display queried information or sets of information regarding past performance, yield, inventory, projected inventory, plant and fruit condition, and / or plant and fruit health. The database may include a front-end API that can feed structured information back to a customer system or receive and incorporate information from an external customer system.

[0040] The remote server may store and run AI algorithms that identify patterns and correlations between growing conditions, yield, plant health, diseases, pests, and other factors. The remote server may predict berry growth based on descriptive historical growth data collected by the harvesting system, temperature, sunlight, and / or other environmental and operational inputs from the customer or from the sensing system. The remote server may publish detection software that makes predictions and provides key growth information to the user.

[0041] The gantry system 200 may include a sensing device 208. The sensing device 208 may consist of one or more sensors. The one or more sensors may acquire environmental growth information (such as sunlight, air temperature, relative humidity, etc.). The one or more sensors may include a quantum meter that measures sunlight. The sensing device 208 may include a processing board that utilizes algorithms to calculate heating and cooling degree days in real time, connect to a remote system via a control system, and store the calculated information in the remote system (such as a cloud-based data architecture).

[0042] The gantry system 200 may include a linear rail 210, on which the linear transport 212 can move in the same direction as or opposite to the direction of travel. The linear rail 210 may include a movement assembly made up of a carriage, a belt, and a motor. Other movement assemblies, such as a rack and pinion, a screw, etc., may also be used. The gantry system 200 may include a second linear rail, on which the linear transport 222 can move in the same direction as or opposite to the direction of travel. The second linear rail may include a movement assembly made up of a carriage, a belt, and a motor.

[0043] In some embodiments, the upper portion of the base frame 202 includes a first linear rail to which the linear translation unit 212 is coupled and a second linear rail to which the linear translation unit 222 is coupled. The linear rails are located on either side of the upper portion of the base frame 202 (e.g., on either side of the storage portion 204).

[0044] Gantry system 200 may include a first elongated member 214 and a second elongated member 216. In some embodiments, first and second elongated members 214, 216 are rods. Linear translation units 212, 222 include corresponding translation assemblies that move elongated members 214, 216 up and down the vertical axis of robotic harvesting system 100. The translation assemblies may include lead screws, belts, and servo motors.

[0045] The gantry system 200 may include a UVC germicidal system consisting of an array of ultraviolet C (UVC) lamps that may be used to periodically irradiate fruit, plants, growing systems, and plant environments with UVC radiation to mitigate mildew, fungi, pests, and other plant problems.

[0046] FIG. 3 illustrates an arm module, according to some embodiments. The arm module 300 may include a linear translation section, an elongated member, a robotic arm, and an end effector. A harvesting system (such as the robotic harvesting system 100) may include one or more arm modules. The harvesting system's control system may execute software that enables the harvesting system to operate multiple arm modules simultaneously. The software may include instructions that prevent collisions between the arm modules. The software may include instructions that enable coordination between the arm modules (e.g., a first arm module harvests a first portion of the growing environment while a second arm module harvests a second portion of the growing environment without accidental overlap).

[0047] The arm module 300 is configured to be attached to and detached from the harvesting system. The arm module may be designed to harvest a particular type of object. For example, the arm module may be designed to harvest tomatoes, grapes, cucumbers, chili peppers, etc.

[0048] The arm module 300 may be constructed from sheet metal. The arm module 300 is produced using low-cost manufacturing techniques. The arm module 300 may be constructed, machined, and printed from sheet metal to achieve the best combination of performance and low cost for the application and current manufacturing scale. Other systems may utilize off-the-shelf components, including complex castings that are then machined to form the arm module. However, such off-the-shelf components are more expensive than sheet metal. As a result, the total cost of a harvesting system including one or more arm modules constructed from sheet metal is lower than the total cost of a harvesting system including one or more arm modules constructed from off-the-shelf components.

[0049] In the illustrated example, the arm module 300 includes a linear travel section 302. The linear travel section 302 is capable of moving along a linear rail (e.g., linear rail 210). The linear travel section 302 is coupled to an elongated member 304. The elongated member 304 is coupled to a first segment 306 of the robot arm via a first joint.

[0050] The first segment 306 may include a rotation assembly that allows the first segment 306 to rotate in a clockwise or counterclockwise direction about the elongate member 304 via a first joint. The rotation assembly may include a belt and a servo motor.

[0051] The first robotic arm segment 306 may be coupled to a second robotic arm segment 308 via a second joint. The second robotic arm segment 308 may include an electronic / mechanical housing 310. The electronic / mechanical housing 310 may include a rotating assembly consisting of a two-stage belt transmission. The rotating assembly allows the second robotic arm segment 308 to rotate in a clockwise or counterclockwise direction around the first robotic arm segment 308 via a second joint.

[0052] The electronic / mechanical component housing 310 may include one or more cameras and / or one or more other sensors. The one or more cameras and / or one or more other sensors may be pointed toward the end effector 314. As a result, the one or more cameras and / or one or more other sensors may capture images of the growing environment, plants, fruit, or harvesting system to diagnose issues and obtain information. The one or more cameras and / or one or more other sensors may include an RGB fisheye camera with a wide-angle lens, an RGBD depth camera, a high-end high-resolution camera utilizing multispectral imaging, and / or a combination thereof.

[0053] Output from the one or more cameras and / or one or more other sensors may be provided to a sub-processing circuit board in the electronic / mechanical component housing 310. The sub-processing circuit board may include one or more processors employing one or more computer vision algorithms. The one or more processors may employ one or more computer vision algorithms to identify fruit. The one or more processors may employ one or more computer vision algorithms to uniquely mark each fruit or each group of fruits in an identifier based on physical characteristics of the fruit itself, surrounding fruit, and / or leaves. The computer vision algorithms may be configured to assess size, weight, precise location, ripeness, and / or any developmental issues associated with the fruit. Ripeness and commercial grade associated with the fruit may be determined at a given location using output from the one or more cameras and / or one or more other sensors as input. Values ​​associated with ripeness and commercial grade may be based on one or more factors, such as color, shape, texture, uniformity, pattern, etc. In some embodiments, sugar content associated with fruit may be determined based on IR light refraction.

[0054] The one or more processors may utilize one or more computer vision algorithms to identify plant structures, barriers, obstacles, and / or obstructions. The one or more processors may use this information to determine the cost associated with navigating through these identified items to harvest the object. The one or more processors may utilize one or more computer vision algorithms to identify specific plant health issues (such as disease or pest symptoms), abiotic stress symptoms (such as drought, temperature, or humidity issues). The one or more processors may utilize one or more computer vision algorithms to identify and distinguish between equipment and plants. The one or more processors may utilize one or more computer vision algorithms to identify overall environmental structures and paths.

[0055] The one or more processors may utilize one or more control algorithms to perform collision avoidance, path planning, inverse kinematics, and high-level planning customized for the particular module hardware implementation. In some embodiments, the one or more control algorithms comprise a fully customized stack, including proprietary messaging, IK, path planning, low-level drivers, and high-level planning.

[0056] In some embodiments, the one or more processors use one or more control algorithms to direct the robot arm to coordinates adjacent to each stalk so that as the robot arm moves forward, it moves toward the stalk avoiding collisions between the gripper or the front and sides of the robot arm and obstructions.

[0057] In some embodiments, the one or more processors use one or more control algorithms to calculate the cost associated with colliding with a particular type of plant to determine whether the robotic arm and end effector can navigate through the material to reach the location of the harvesting target.

[0058] In some embodiments, one or more processors use one or more control algorithms to plan the sequence associated with harvesting the multiple objects. In some embodiments, the plan is determined before harvesting begins. In some embodiments, the plan associated with the next object may be determined while the current object is being harvested. This may reduce the overall time to harvest the multiple objects because the growing environment in which the multiple objects are being harvested or assumptions associated with the multiple objects may change while the object is being harvested. For example, the cost associated with harvesting a second object may change while the robotic arm harvests the first object and additional images of the second object are acquired.

[0059] The electronic / mechanical component housing 310 may be coupled to an end effector 314. The end effector 314 may be comprised of a cutting mechanism and a gripping mechanism, which may include corresponding sets of jaws. The end effector 314 may be configured to simultaneously open and close the cutting mechanism and the gripping mechanism. When the cutting mechanism and the gripping mechanism are open, a plant appendage attached to an object may be positioned within an opening associated with the corresponding set of jaws.

[0060] One or more processors in the electronic / mechanical component housing 310 may receive one or more images from one or more cameras and / or one or more other sensors in the electronic / mechanical component housing 310. The one or more processors may identify one or more objects from the one or more images and assign an associated cost to the one or more identified objects. The one or more processors may select one of the one or more identified objects for harvesting based on the associated cost and determine a relative distance between the current position of the end effector and the current position of the object. The one or more processors may send a command to a control system (such as control system 108) of the robotic harvesting system to move the robot arm the relative distance. In response to receiving the command, the control system may move the linear translation section, the elongated member, and / or the robot arm to reposition the robot arm the determined relative distance. An actuator associated with the end effector 314 may then apply a force to close the cutting and gripping mechanisms. The applied force causes the cutting mechanism to perform a cut that separates the plant appendage into a first portion and a second portion, while the applied force causes the gripping mechanism to grip the second portion of the plant appendage attached to the harvest object. The robotic harvesting system may move the end effector 314 to a storage location (such as the container 142) and open the end effector 314, thereby depositing the harvested object in the storage location.

[0061] 4 is a diagram illustrating multiple approach angles, according to some embodiments. In the illustrated example, a growing area at a particular location includes object 402 and harvested object 404. One or more processors associated with the robotic harvesting system may determine corresponding costs associated with harvesting object 402 and harvested object 404. In the illustrated example, the cost associated with harvesting object 404 is lower than the corresponding cost associated with object 402. Object 402 may be harvested after object 404 is harvested. The corresponding cost associated with object 402 may change after object 404 is harvested.

[0062] The robotic arm of the robotic harvesting system may approach the object 404 from multiple angles 406. The configuration of the robotic arm allows the robotic harvesting system to approach and harvest the object 406 from any angle (e.g., 360°) or near any angle (e.g., 356°), depending on the growing environment in which the harvest object resides. The control system may determine the approach angle for a particular object based on the cost associated with each approach angle and provide one or more commands to the robotic arm to approach and harvest the object from the determined approach angle. The one or more commands may move a linear translation section along a linear rail 408, cause a linear screw to rotate the robotic arm, cause a drive assembly to rotate a first segment of the robotic arm, cause a drive assembly to rotate a second segment of the robotic arm, and / or cause an end effector to open or close relative to the stem of the object 406.

[0063] FIG. 5 is a diagram illustrating a growing environment, according to some embodiments. In the illustrated example, growing environment 500 includes multiple rows 502a, 502b, 502c, 502d, 502e, and 502f. The multiple rows may have a specific width. For example, the specific width may be 1 meter. The base of harvesting system 501 may be designed to fit between the specific widths. The specific width may vary based on the country in which the growing environment is located. The specific width may vary from farm to farm. The base of harvesting system 501 may be designed to fit between the rows, regardless of the country in which the growing environment is located.

[0064] 6 is a flow chart illustrating a process for harvesting an object, according to some embodiments. In the illustrated example, process 600 may be performed by a harvesting machine (such as robotic harvesting machine 100).

[0065] At step 602, a robotic harvesting machine is positioned at a particular location. The particular location may include a first set of harvest objects.

[0066] In step 604, one or more objects available at the location are determined. The robotic harvesting machine includes one or more cameras and / or one or more other sensors. The one or more cameras and / or one or more other sensors may be located on a robotic arm. The robotic arm may be coupled to an end effector. The one or more cameras and / or one or more other sensors may be pointed toward the end effector. The one or more cameras and / or one or more other sensors may capture one or more images, which may include image data indicating that one or more objects are available for harvest at a particular location.

[0067] In step 606, a cost associated with each of the one or more objects is determined. One or more processors of the robotic harvesting machine may utilize one or more computer vision algorithms to identify one or more objects included in the image data. The one or more processors may assign a cost to each of the one or more identified objects. The assigned cost may indicate the likelihood of harvesting the object. The assigned cost may be based on one or more factors, such as the location of the object relative to other harvest objects, the approach angle required to harvest the object, the presence of any obstacles, the ripeness of the object, etc.

[0068] In step 608, the objects are harvested based on the determined cost. An object having the lowest cost among the plurality of objects may be selected. A relative distance between the current position of the end effector and the current position of the selected object may be determined based on the image data. One or more processors associated with the robotic arm may send a command to a control system of the robotic harvesting system to move the robotic arm the relative distance. In response to receiving the command, the control system of the robotic harvesting system may move the linear translation member, the elongated member, and / or the robotic arm to reposition the robot arm the determined relative distance.

[0069] An actuator associated with the end effector may then apply a force that causes the cutting mechanism and the gripping mechanism of the end effector to close. The applied force causes the cutting mechanism to perform a cut that separates the plant appendage associated with the harvesting object into a first portion and a second portion, while the applied force causes the gripping mechanism to grip the second portion of the plant appendage attached to the harvesting object. The robotic harvesting system may move the end effector to a storage location in the robotic harvesting system and open the end effector, thereby depositing the harvested object in the storage location.

[0070] After the object is harvested, the costs associated with the remaining harvested objects may be recalculated and the process may be repeated to harvest the next object.

[0071] At step 610, it is determined whether the storage bin of the robotic harvesting system is full of harvested objects. If it is determined that the storage bin of the robotic harvesting system is full of harvested objects, process 600 proceeds to step 612. If it is determined that the storage bin of the robotic harvesting system is not full of harvested objects, process 600 proceeds to step 614.

[0072] At step 612, the robotic harvesting system waits for the harvested objects to be unloaded. In some embodiments, an operator associated with the robotic harvesting system removes one or more trays containing the harvested objects from the robotic harvesting system and replaces the removed tray or trays with one or more new trays. In some embodiments, the robotic harvesting system stores one or more trays beneath the one or more trays containing the harvested objects, and an operator associated with the robotic harvesting system removes the one or more trays containing the harvested objects from the robotic harvesting system.

[0073] In some embodiments, the robotic tray system removes one or more trays containing the harvested objects from the robotic harvesting system and replaces the removed tray or trays with one or more new trays. In some embodiments, the robotic harvesting system stores one or more trays below the one or more trays containing the harvested objects, and the robotic tray system removes the one or more trays containing the harvested objects from the robotic harvesting system.

[0074] At step 614, it is determined whether the harvester is at the end of the row. If the harvester is not at the end of the row, process 600 proceeds to step 616 where the robotic harvester is moved a predetermined distance.

[0075] If the harvesting machine is at the end of the row, process 600 proceeds to step 618, where it is determined whether there are more rows in the growing environment. If there are more rows in the growing environment, process 600 returns to step 602. If there are no more rows in the growing environment, process 600 proceeds to step 620, where the robotic harvesting system waits for a command to begin the next round of object harvesting. For example, the robotic harvesting system may enter a sleep mode. In response to receiving a command to begin the next round of object harvesting, process 600 may be repeated.

[0076] Although the above embodiments have been described in some detail for ease of understanding, the invention is not limited to the details provided. There are many alternative ways of implementing the invention. The disclosed embodiments are illustrative and are not intended to be limiting. [Application Example 1] A system, a base configured to move in a direction of travel; a linear translation unit attached to the base, the linear translation unit configured to move along the base in a direction substantially the same as or opposite to the direction of travel; a robot arm attached to the linear motion section, the robot arm having a proximal end and a distal end, the distal end of the robot arm configured to rotate from a first joint toward and away from the base; an end effector attached to the distal end of the robotic arm. [Application Example 2] A system according to Application Example 1, wherein the robot arm comprises a first segment coupled to the first joint and the second joint, and the first segment is configured to rotate toward and away from the base by the first joint. [Application Example 3] A system according to Application Example 2, wherein the robot arm comprises a second segment connected to the first segment via the first joint, and the second segment is configured to rotate toward and away from the base by the second joint. [Application Example 4] The system according to Application Example 3, wherein the end effector is coupled to the second segment. [Application Example 5] The system described in Application Example 3, further comprising a control system, wherein the control system is configured to control a corresponding rotation angle associated with the first segment and a corresponding rotation angle associated with the second segment to approach an object from a desired angle. [Application Example 6] The system described in Application Example 5, further comprising one or more cameras and / or one or more sensors, wherein the one or more cameras and / or the one or more sensors are coupled to the distal end of the second segment. [Application Example 7] A system according to Application Example 6, wherein the control system is configured to move the robot arm a relative distance from the current position of the robot arm to the current position of the object. [Application Example 8] A system according to Application Example 7, wherein the relative distance is determined based on image data obtained from the one or more cameras and / or the one or more sensors. [Application Example 9] The system according to Application Example 1, wherein the linear movement unit comprises a gantry system. [Application Example 10] A system according to Application Example 9, wherein the gantry system is configured to move up and down in a vertical direction relative to the base. [Application Example 11] A system according to Application Example 1, wherein the first joint comprises a rod, and the first segment of the robot arm is configured to move up and down in a vertical direction relative to the rod. [Application Example 12] The system according to Application Example 1, further comprising a second robot arm attached to the linear moving part. [Application Example 13] The system according to Application Example 12, wherein the robot arm and the second robot arm are positioned on opposite sides of the base. [Application Example 14] The system according to Application Example 12, wherein the robot arm and the second robot arm are arranged on the same side of the base. [Application Example 15] A system according to Application Example 12, wherein the robot arm and the second robot arm are configured to operate simultaneously. [Application Example 16] The system according to Application Example 1, wherein the base has a width of 0.5 m to 1.5 m. [Application Example 17] The system according to Application Example 1, wherein the base has a length of 1 m to 3 m. [Application Example 18] A system according to Application Example 1, wherein the base has a U-shaped groove portion. [Application Example 19] A system according to Application Example 1, wherein, in response to a command, the base is configured to move a specific distance along the direction of travel. [Application Example 20] A system according to Application Example 19, wherein the specific distance is the length of the base or the length of a portion of the base.

Claims

1. 1. A system comprising: a base configured to move in a direction of travel; a linear movement unit attached to the base via a linear rail extending parallel to the traveling direction and disposed on the base, the linear movement unit being configured to move along the linear rail in a direction substantially the same as or opposite to the traveling direction; a rod coupled to the linear translation section, the rod extending downward from the linear rail; a robotic arm rotatably attached to the rod at a first joint, the robotic arm having a proximal end and a distal end, the distal end of the robotic arm configured to rotate from the first joint toward and away from the base in a horizontal plane; an end effector attached to the distal end of the robotic arm.

2. 2. The system of claim 1, wherein the robotic arm comprises a first segment coupled to the first joint and a second joint, the first segment configured to rotate toward and away from the base by the first joint.

3. 3. The system of claim 2, wherein the robot arm comprises a second segment coupled to the first segment via the second joint, the second segment configured to rotate toward and away from the base by the second joint.

4. The system of claim 3 , wherein the end effector is coupled to the second segment.

5. 4. The system of claim 3, further comprising a control system configured to control a corresponding rotation angle associated with the first segment and a corresponding rotation angle associated with the second segment to approach an object from a desired angle.

6. 6. The system of claim 5, further comprising one or more cameras and / or one or more sensors, the one or more cameras and / or the one or more sensors coupled to a distal end of the second segment.

7. 7. The system of claim 6, wherein the control system is configured to move the robot arm a relative distance from a current position of the robot arm to a current position of the object.

8. 8. The system of claim 7, wherein the relative distance is determined based on image data obtained from the one or more cameras and / or the one or more sensors.

9. The system of claim 1 , wherein the linear motion section comprises a gantry system.

10. 10. The system of claim 9, wherein the gantry system is configured to move up and down vertically relative to the base.

11. 10. The system of claim 1, wherein a first segment of the robotic arm is configured to move up and down in a vertical direction relative to the rod.

12. The system of claim 1 , further comprising a second robotic arm attached to the linear motion section.

13. 13. The system of claim 12, wherein the robotic arm and the second robotic arm are located on different sides of the base.

14. 13. The system of claim 12, wherein the robotic arm and the second robotic arm are located on the same side of the base.

15. 13. The system of claim 12, wherein the robotic arm and the second robotic arm are configured to operate simultaneously.

16. 2. The system of claim 1, wherein the base has a width of 0.5 m to 1.5 m in a direction perpendicular to the direction of travel.

17. 2. The system of claim 1, wherein the base has a length of 1 m to 3 m in the direction of travel.

18. The system of claim 1 , wherein the base comprises a U-channel portion.

19. 10. The system of claim 1, wherein in response to a command, the base is configured to move a specified distance along the direction of travel.

20. 20. The system of claim 19, wherein the particular distance is the length of the base or a portion of the base in the direction of travel.

Images