Leaf collection device and method for controlling the leaf collection device

Abstract

To improve efficiency of work.SOLUTION: A leaf collection device according to one aspect of the present disclosure comprises an end effector for collecting a leaf from a crop, and a robot for changing the relative position of the end effector with respect to the crop. The end effector comprises: a case member provided with an opening in a tip part, and forming a space for taking in the leaf of the crop; a suction part for generating a suction force so as to suck and take in the leaf of the crop into the space from the opening; and a cutting member capable of cutting the leaf of the crop taken in into the space.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present disclosure relates to a leaf collection device, an end effector for leaf collection, and a control method for the leaf collection device.

Background Art

[0002] Patent Document 1 discloses a device for collecting leaves from crops. This device includes a camera for observing the crop, a central processing unit to which the camera is connected, and a movable trolley to which a peeling means controllable by the central processing unit is attached.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The present disclosure provides a leaf collection device, an end effector for leaf collection, and a control method for the leaf collection device that are useful for improving work efficiency.

Means for Solving the Problems

[0005] The leaf collection device according to one aspect of the present disclosure includes an end effector for collecting leaves from a crop, and a robot for changing the relative position of the end effector with respect to the crop. The end effector has a case member provided with an opening at the tip portion, forming a space for taking in the leaves of the crop, a suction portion for generating a suction force to suck the leaves of the crop from the opening into the space, and a cutting member capable of cutting the leaves of the crop taken into the space.

[0006] An end effector for leaf collection according to one aspect of this disclosure is formed to be attachable to a robot whose position of the object being operated on can be changed, and is an end effector for collecting leaves from crops. This end effector has a case member with an opening at its tip that forms a space for taking in crop leaves, a suction unit that generates a suction force to suck the crop leaves into the space through the opening, and a cutting unit that can cut the crop leaves taken into the space.

[0007] A control method for a leaf harvesting device relating to one aspect of the present disclosure is a control method for a leaf harvesting device comprising an end effector for harvesting leaves from a crop, and a robot for changing the relative position of the end effector with respect to the crop. The end effector has a case member with an opening at its tip that forms a space for taking in crop leaves, a suction unit that generates a suction force to suck and take in crop leaves from the opening into the space, and a cutting member capable of cutting the crop leaves taken into the space. The control method includes a first control in which the robot moves the end effector to a work start position set according to a predetermined setting method, a second control in which the robot moves the end effector from the work start position toward the crop leaves, and a third control in which, after the execution of the second control, controls either or both the end effector and the robot so that the leaves are cut from the crop by the cutting member. [Effects of the Invention]

[0008] This disclosure provides a leaf collection device, a leaf collection end effector, and a method for controlling the leaf collection device that are useful for improving work efficiency. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a schematic diagram showing an example of a leaf collection device. [Figure 2] Figure 2 is a photograph showing an example of a leaf that was collected. [Figure 3] Figure 3 is a schematic plan view illustrating an example of the operation using the sampling device. [Figure 4]Figure 4 is a schematic perspective view showing an example of an end effector. [Figure 5] Figure 5 is a schematic side view showing an example of the inside of an end effector. [Figure 6] Figure 6 is a schematic side view showing an example of the front view of an end effector. [Figure 7] Figure 7 is a block diagram showing an example of the controller's functional configuration. [Figure 8] Figures 8(a), 8(b), and 8(c) are schematic diagrams illustrating an example of the leaf collection process. [Figure 9] Figure 9 is a photograph illustrating an example of a method for detecting leaves from captured images. [Figure 10] Figure 10 is a schematic diagram illustrating an example of the direction in which one approaches a leaf. [Figure 11] Figure 11 is a schematic diagram illustrating one example of a method for determining the order in which leaves should be collected. [Figure 12] Figures 12(a) and 12(b) are schematic diagrams illustrating an example of a method for determining the order in which leaves should be collected. [Figure 13] Figure 13 is a graph showing an example of the time variation of rotational speed and torque command value in the suction section. [Figure 14] Figure 14 is a flowchart showing an example of a control method for the sampling device. [Figure 15] Figure 15 is a block diagram showing an example of the controller's hardware configuration. [Figure 16] Figure 16(a) is a schematic diagram using a photograph to explain the approach direction. Figure 16(b) is a schematic diagram using a photograph to explain the approach angle. [Modes for carrying out the invention]

[0010] An embodiment will be described below with reference to the drawings. In this description, the same elements or elements having the same function will be denoted by the same reference numeral, and redundant descriptions will be omitted.

[0011] [Leaf collection device] Referring to FIGS. 1 to 3, the outline of the leaf collection device according to an embodiment and the crop as the work target will be first described. The leaf collection device 1 shown in FIG. 1 is a device (system) that automatically executes at least a part of the work for collecting one or more leaves from the crop 200. By the leaf collection device 1 executing the work for collecting leaves (hereinafter referred to as "collection work"), one or more leaves are removed from the crop 200. The crop 200 that is the object (work) of the collection work is, for example, a fruit vegetable such as a cucumber, or a fruiting vegetable such as a strawberry. For industrial products with a constant work state, when the work is the crop 200 (natural product, agricultural crop), various states are assumed due to growth conditions such as the environment and individual differences, and it is necessary to execute the collection work according to these various states. In the present disclosure, the leaf collection device 1 will be described by taking the case where the crop 200 is a cucumber as an example.

[0012] The leaf 210 (crop leaf) collected by the collection work is directly or indirectly connected to the main stem 202. As shown in FIG. 2, the leaf 210 includes a leaf blade 212 and a leaf stalk 214. In the collection work on the leaf 210 by the leaf collection device 1, the leaf blade 212 and at least a part of the leaf stalk 214 are collected from the crop 200. As shown in FIG. 3, the crop 200 may be planted so as to line up along a horizontal one direction (predetermined direction). The crop 200 is, for example, planted on a shelf 290 extending along a horizontal one direction in a greenhouse. The crop 200 includes a plurality of plants 201 (seedlings) lined up along a horizontal one direction on the shelf 290. The plurality of plants 201 included in the crop 200 may be planted at a substantially constant interval. One plant 201 in the crop 200 has one or more leaves 210. In FIG. 3, the direction in which the plurality of plants 201 included in the crop 200 line up is represented by the "X-axis", and a horizontal one direction perpendicular to the X-axis direction is represented by the "Y-axis". The direction perpendicular to the plane (X-Y plane) constituted by the X-axis and the Y-axis corresponds to the vertical direction.

[0013] Returning to FIG. 1, the leaf collection device 1 includes, for example, a robot 2, an end effector 50, a collection unit 4, a traveling carriage 6, a first measurement unit 8, a second measurement unit 9, and a controller 100. Hereinafter, an example of each of the plurality of units (devices) constituting the leaf collection device 1 will be described.

[0014] (Robot) The robot 2 is a device that changes the relative position of the end effector 50 with respect to the crop 200. The robot 2 may change the relative position and orientation of the end effector 50 with respect to the crop 200. The robot 2 is, for example, a six-axis vertical articulated robot and has a base 11, a swivel unit 12, a first arm 13, a second arm 14, a third arm 17, a tip 18, and actuators 41, 42, 43, 44, 45, 46. The base 11 is installed on the traveling carriage 6. Thereby, as the traveling carriage 6 moves, the entire robot 2 moves.

[0015] The swivel unit 12 is provided on the base 11 so as to rotate around a vertical axis 21. The first arm 13 is connected to the swivel unit 12 so as to rotate around an axis 22 that intersects (for example, is orthogonal to) the axis 21, and extends in a direction away from the axis 22. In the present disclosure, the intersection includes a case where there is a twisted relationship such as a so-called three-dimensional intersection.

[0016] The second arm 14 is connected to the tip of the first arm 13 so as to rotate around an axis 23 that is substantially parallel to the axis 22, and extends in a direction away from the axis 23. The second arm 14 includes an arm base 15 and an arm end 16. The arm base 15 is connected to the tip of the first arm 13. The arm end 16 is connected to the tip of the arm base 15 so as to rotate around an axis 24 that intersects (for example, is orthogonal to) the axis 23, and extends in a direction away from the arm base 15 along the axis 24.

[0017] The third arm 17 is connected to the tip of the arm end 16 so as to rotate around an axis 25 that intersects (for example, is perpendicular to) the axis 24. The tip 18 is connected to the tip of the third arm 17 so as to rotate around an axis 26 that intersects (for example, is perpendicular to) the axis 25. An end effector 50 (a leaf-collecting end effector) is attached to the tip 18 as the target of the robot 2's operation.

[0018] As described above, the robot 2 has a joint 31 connecting the base 11 and the swivel section 12, a joint 32 connecting the swivel section 12 and the first arm 13, a joint 33 connecting the first arm 13 and the second arm 14, a joint 34 in the second arm 14 connecting the arm base 15 and the arm end 16, a joint 35 connecting the arm end 16 and the third arm 17, and a joint 36 connecting the third arm 17 and the tip 18.

[0019] Actuators 41, 42, 43, 44, 45, and 46 include, for example, electric motors and reduction gears, and drive joints 31, 32, 33, 34, 35, and 36, respectively. For example, actuator 41 rotates the swivel section 12 around axis 21, actuator 42 rotates the first arm 13 around axis 22, and actuator 43 rotates the second arm 14 around axis 23. Also, actuator 44 rotates the arm end 16 around axis 24, actuator 45 rotates the third arm 17 around axis 25, and actuator 46 rotates the tip 18 around axis 26.

[0020] The specific configuration of robot 2 can be changed as appropriate. For example, robot 2 may be a 7-axis redundant robot, which is a 6-axis vertical articulated robot with an additional joint added, or it may be a so-called scalar-type articulated robot.

[0021] (End effector) The end effector 50 is a device (tool) for harvesting leaves 210 from crops 200. The end effector 50 is formed to be attachable to the robot 2. The end effector 50 is configured to harvest leaves 210 from crops 200 by sucking them into an internal space it forms (hereinafter referred to as "internal space S") through an opening. The end effector 50 may also have a function to cut the leaves 210 that have been sucked into the internal space S. The end effector 50 may also have a function to shred (crush) the leaves 210 that have been sucked into the internal space S. An example of the end effector 50 is schematically shown in Figures 4 to 6. As shown in Figures 4 and 5, the end effector 50 includes, for example, a case member 52, a suction part 60, a cutting member 80, and a protective member 88.

[0022] The case member 52 is a member that forms an internal space S for enclosing the leaves 210. The internal space S is formed to the extent that it can accommodate a portion of the petiole 214 and the leaf blade 212 contained in one leaf 210. The case member 52 is formed in a cylindrical shape as a whole. The case member 52 may also be formed in a cylindrical shape. The case member 52 is formed to extend in one direction, and the internal space S is open at one end (one end and its vicinity) in the direction of extension. One end of the case member 52 in the direction of extension will be referred to as "end 54" and the other end as "end 56" (see Figure 5).

[0023] An opening 54a is provided at the end portion 54 to open up the internal space S. The internal space S and the space outside the case member 52 are connected through the opening 54a. It is possible to take the leaves 210 into the internal space S through the opening 54a. The end portion 54 constitutes the tip of the end effector 50 (case member 52). That is, an opening 54a is provided at the tip of the end effector 50. The end portion 56 may be attached to the tip portion 18 of the robot 2. Note that in Figures 4 to 6, the end effector 50 is depicted in a simplified form, and the diameter of the case member 52 (inner and outer diameters in a cross section perpendicular to the extending direction) may differ depending on the position of the case member 52 in the extending direction. In one example, the end portion 54 may be formed such that its diameter increases from the inside to the outside of the internal space S.

[0024] The suction unit 60 is a part (unit) that generates a suction force to draw the leaves 210 of the crop 200 into the internal space S through the opening 54a. When the suction unit 60 is generating a suction force, the opening 54a at the tip of the end effector 50 approaches the leaf 210, causing the leaf 210 to be drawn into the internal space S. The suction unit 60 includes, for example, an impeller 62 and a drive unit 70. The drive unit 70 rotates the impeller 62, generating a suction force in the internal space S.

[0025] The impeller 62 is, for example, located in the internal space S and is rotatably mounted around an axis (hereinafter referred to as "axis Ax1") that intersects the opening 54a. Axis Ax1 may be perpendicular to the opening 54a. Axis Ax1 may be set to pass through approximately the center of the opening 54a. The impeller 62 may be located between the end 54 and the end 56, closer to the end 56. The impeller 62 may include a shaft member 64 that is rotatably mounted to the case member 52 and extends along axis Ax1, and a rotating body 68 composed of a plurality of blades 66. As the rotating body 68 (impeller 62) rotates around axis Ax1, a gas flow is generated that is directed radially outward around the circumference centered on axis Ax1, resulting in an attractive force being generated in the internal space S. A discharge section 58 is connected to the peripheral wall of the case member 52 at the position where the rotating body 68 is located (see also Figure 4). The discharge section 58 forms an outward-facing gas flow path formed by the impeller 62.

[0026] The impeller 62 may be formed to shred the leaves 210 taken into the internal space S. In this case, the impeller 62 has both the function of generating suction force and the function of shredding the leaves 210 in the internal space S. The impeller 62 may be formed of a material harder than the leaves 210 (for example, metal or resin). When the leaves 210 taken into the internal space S come into contact with the rotating body 68, the leaf blade 212 and petiole 214 of the leaves 210 are shredded (crushed) and discharged outside the internal space S via the discharge section 58.

[0027] The drive unit 70 drives the impeller 62 to generate an attractive force in the internal space S. The drive unit 70 includes, for example, a drive motor 72, a fixed member 74, and a transmission member 76. The drive motor 72 is the drive source and generates a driving force to drive the impeller 62. The fixed member 74 is a member that fixes the drive motor 72 to the case member 52. One end of the fixed member 74 is connected, for example, to the outer circumferential surface of the end 56 of the case member 52. The other end of the fixed member 74 is connected to the drive motor 72, and a space may be formed between the drive motor 72 and the case member 52. The transmission member 76 is a member that transmits the driving force from the drive motor 72 to the impeller 62 (the shaft member 64 of the impeller 62). The transmission member 76 includes, for example, a belt, a pulley, and a reduction gear.

[0028] The cutting member 80 is a member capable of cutting the leaves 210 that have been taken into the internal space S. The cutting member 80 may be provided on the peripheral wall 53 surrounding the axis Ax2 of the case member 52 so as to be rotatable around the axis intersecting the opening 54a (hereinafter referred to as "axis Ax2"). Axis Ax2 (second axis) may substantially coincide with axis Ax1. In the example shown in Figures 4 and 5, the peripheral wall 53 is a cylindrical portion of the case member 52 that extends in the above-mentioned extending direction, and includes an inner surface (inner wall surface) along the circumference around axis Ax2. One open end of the peripheral wall 53 corresponds to the end 54. At least a part of the cutting member 80 is located on the peripheral edge of the opening 54a when viewed from the direction in which axis Ax2 extends, and is movable (rotatable) along the inner surface of the peripheral wall 53. The cutting member 80 is positioned between the impeller 62 and the opening 54a in the direction in which the axis Ax2 extends. The cutting member 80 may be positioned closer to the opening 54a between the impeller 62 and the opening 54a, or it may be provided at the end 54 (one end of the peripheral wall 53), as shown in Figure 5. The cutting member 80 includes, for example, a plurality of blades 84, each extending along a direction intersecting the axis Ax2, and a support member 82 that supports the plurality of blades 84 and is rotatably mounted on the case member 52. The plurality of blades 84 are positioned at different locations on the circumference around the axis Ax2, as shown in Figure 6. Each of the plurality of blades 84 is formed to extend along the circumference (inner surface of the peripheral wall 53) around the axis Ax2. Each of the plurality of blades 84 is attached to the support member 82 such that at least a portion of the cutting edge faces inward (in the direction from the inner surface of the peripheral wall 53 toward the axis Ax2).

[0029] The drive unit 70 of the suction unit 60 may further drive the cutting member 80 so that it rotates around the axis Ax2. The transmission member 76 of the drive unit 70 may transmit the driving force from the drive motor 72 to both the impeller 62 and the cutting member 80. The drive unit 70 may rotate the impeller 62 and the cutting member 80 in opposite directions. In Figure 6, the direction in which the impeller 62 rotates is indicated by "R1", and the direction in which the cutting member 80 rotates is indicated by "R2". When the end effector 50 is viewed from the front (when the opening 54a is viewed from outside the internal space S), if direction R1 is clockwise, direction R2 may be counterclockwise. Unlike the example shown in Figure 6, direction R1 may be counterclockwise and direction R2 may be clockwise.

[0030] The protective member 88 is a member that prevents damage to parts of the crop 200 other than the leaves 210 to be harvested from contact with the multiple blades 84. The protective member 88 is provided on the peripheral wall 53 (for example, the end 54) so ​​as to cover the cutting member 80 when the opening 54a is viewed from outside the internal space S. In this case, the protective member 88, the cutting member 80 (multiple blades 84), and the impeller 62 are arranged in this order in the direction in which the axis Ax2 extends. The protective member 88 is formed in an annular shape so as to extend along the circumference around the axis Ax2. When the opening 54a is viewed from a direction perpendicular to the opening 54a (for example, the direction in which the axis Ax2 extends), the protective member 88 may cover the entirety of each of the multiple blades 84. In this case, the shortest distance between the axis Ax2 and the blade 84 is greater than the shortest distance between the axis Ax2 and the protective member 88.

[0031] The protective member 88 is formed to be expandable and contractible in a direction intersecting the axis Ax2. When no external force is applied to the protective member 88, it covers the entire blade 84. The protective member 88 is formed to contract in the radial direction when a force is applied outward in the radial direction of the circumference centered on the axis Ax2. The protective member 88 is made of a material softer than the material forming the blade 84 (for example, sponge or rubber). When the cutting member 80 cuts the leaf 210, for example, after the petiole 214 of the leaf 210 is taken into the internal space S, the end effector 50 moves along a direction intersecting the axis Ax2 while the cutting member 80 is rotating. As the end effector 50 moves along a direction intersecting the axis Ax2, the petiole 214 comes into contact with the protective member 88, applying an external force to the protective member 88 so that it contracts along the radial direction. As a result, the cutting edges of the multiple blades 84 are exposed beyond the protective member 88, and the petiole 214 comes into contact with the rotating multiple blades 84 and is cut.

[0032] (Collection Department) Returning to Figure 1, the recovery unit 4 is a unit that recovers the leaves 210 collected by the end effector 50. The recovery unit 4 recovers, for example, the leaves 210 (fine leaf powder) after they have been cut and shredded by the end effector 50. The recovery unit 4 has, for example, a bag for containing the shredded leaves 210 and a pipe (hose) connecting the bag to the discharge port of the discharge section 58 of the end effector 50. The shredded leaves 210 recovered in the recovery unit 4 are transported together with the robot 2 by the traveling carriage 6.

[0033] (Traction cart) The mobile carriage 6 is a mobile carriage capable of moving the robot 2 to which the end effector 50 is attached. The mobile carriage 6 can travel along the direction in which the multiple plants 201 contained in the leaf 210 are lined up (the X-axis direction in Figure 3), as shown in Figure 3. The mobile carriage 6 has a motor, which is the drive source for travel, and may travel on a rail 7 that extends in the direction in which the multiple plants 201 contained in the leaf 210 are lined up. As the mobile carriage 6 moves, the relative position of the base 11 of the robot 2 with respect to the crop 200 changes.

[0034] (First Measurement Unit) The first measurement unit 8 (imaging unit) is a unit that measures data for detecting the leaves to be harvested from the crop 200. The first measurement unit 8 has the function of imaging a field of view that includes the leaves 210 of the crop 200. The field of view that the first measurement unit 8 can image is referred to as the "imaging range PR1". The imaging range PR1 is set to include multiple leaves 210 of the crop 200. The imaging range PR1 may also be set to include multiple plants 201 of the crop 200. The first measurement unit 8 further has the function of measuring the distance (depth) in the imaging range PR1. As shown in Figure 1, the first measurement unit 8 is fixed to the mobile carriage 6 via a fixing member 8a. As the mobile carriage 6 moves, the position of the imaging range PR1 moves. On the other hand, even if the mobile carriage 6 moves, the relative position between the base 11 of the robot 2 and the first measurement unit 8 does not change.

[0035] The first measurement unit 8 includes, for example, an image sensor and a distance sensor. The image sensor of the first measurement unit 8 is a sensor (camera) capable of acquiring image data for visualizing the imaging range RP1. The image sensor of the first measurement unit 8 images the imaging range RP1 using, for example, visible light or light other than visible light (for example, infrared light). The distance sensor of the first measurement unit 8 is a sensor capable of acquiring distance data (depth map) in the imaging range PR1. The distance sensor of the first measurement unit 8 acquires data indicating the distance (distance from the sensor) to multiple locations on an object present in the imaging range PR1. The distance sensor of the first measurement unit 8 is, for example, a TOF (Time of Flight) sensor.

[0036] In the example shown in Figure 3, the various sensors of the first measurement unit 8 may be oriented in the Y-axis direction, or they may be oriented in a direction inclined with respect to the Y-axis direction. The first measurement unit 8 may have multiple sets of image sensors and distance sensors oriented at different angles to each other with respect to the Y-axis direction. The various sensors of the first measurement unit 8 may be installed such that at least a portion of the imaging range PR overlaps with at least a portion of the traveling carriage 6 in the X-axis direction (the direction in which the traveling carriage 6 travels). The first measurement unit 8 outputs the acquired image data and distance data to the controller 100.

[0037] (Second Measurement Unit) The second measurement unit 9 (second imaging unit) is a unit that detects the state of the portion of the crop 200 located in front of the traveling trolley 6. The second measurement unit 9 has the function of imaging a field of view that includes the portion of the crop 200 located in front of the traveling trolley 6. The field of view that the second measurement unit 9 can image is referred to as the "imaging range PR2". The area in front of the traveling trolley 6 is the direction in which the traveling trolley 6 will move (in the X-axis direction). In the X-axis direction, the imaging range PR2 may be set so as not to overlap with the traveling trolley 6. As shown in Figure 1, the second measurement unit 9 is fixed to the traveling trolley 6 via a fixing member 9a. As the traveling trolley 6 moves, the position of the imaging range PR2 moves. The second measurement unit 9 has, for example, an image sensor (camera) capable of acquiring image data for visualizing the imaging range PR2. The image sensor of the second measurement unit 9 images the imaging range PR2 using, for example, visible light or light other than visible light (for example, infrared light).

[0038] (controller) Returning to Figure 1, the controller 100 is one or more computer devices configured to control the robot 2, the end effector 50, the traveling carriage 6, the first measurement unit 8, and the second measurement unit 9, respectively. The controller 100 may consist of multiple computer devices that individually control the robot 2, the end effector 50, the traveling carriage 6, the first measurement unit 8, and the second measurement unit 9, and the multiple computer devices may be connected to each other in a manner that allows them to communicate with one another.

[0039] As shown in Figure 7, the controller 100 has a functional configuration (hereinafter referred to as "functional modules") including, for example, a drive control unit 102, a robot control unit 104, a first measurement control unit 106, a state estimation unit 108, a model holding unit 112, a model construction unit 114, a work procedure determination unit 110, a state monitoring unit 122, a second measurement control unit 126, and a travel control unit 124. The processes performed by these functional modules correspond to the processes performed by the controller 100.

[0040] The drive control unit 102 controls the drive motor 72 of the drive unit 70 so that the impeller 62 rotates around axis Ax1 and the cutting member 80 rotates around axis Ax2. The drive control unit 102 controls the drive motor 72 so that, for example, the rotational speed of the drive motor 72 follows a predetermined target value. The drive control unit 102 may obtain a detected value indicating the current rotational speed of the drive motor 72 from a sensor provided in the drive unit 70. The drive control unit 102 may calculate a torque command value (or current command value) to the drive motor 72 in order to reduce the deviation between the current rotational speed of the drive motor 72 and the target value. In one example, the drive control unit 102 starts or stops the drive of the impeller 62 and the cutting member 80 by the drive motor 72 in response to a command from the robot control unit 104 so as to be linked to the operation of the end effector 50 by the robot 2.

[0041] The robot control unit 104 controls the robot 2 so that the end effector 50 performs the leaf collection operation 210. Figures 8(a), 8(b), and 8(c) schematically show the collection operation for one leaf 210, and the robot 2 is omitted. The robot control unit 104 starts the collection operation for one leaf 210, for example, when the end effector 50 is positioned in a standby position. The standby position may be set so that at least a part of the end effector 50 is positioned vertically above the traveling carriage 6.

[0042] During the harvesting operation, as shown in Figure 8(a), the robot control unit 104 performs a first control to move the end effector 50 to a position where the opening 54a faces the surface 212a of the leaf blade 212 contained in the leaf 210 (hereinafter referred to as the "work start position"). The surface 212a is the upper side of the leaf blade 212, and is the side facing away from the main stem 202 (see Figure 2). When the end effector 50 is positioned at the work start position, the opening 54a overlaps with at least a portion of the surface 212a of the leaf blade 212 when viewed from a direction perpendicular to the opening 54a (the direction in which the axis Ax1 extends). Here, the part where the leaf blade 212 and the petiole 214 are connected is defined as the "connection part CP". When the robot control unit 104 executes the first control, with the end effector 50 positioned at the work start position, it controls the robot 2 so that the connection portion CP overlaps with the opening 54a when viewed from a direction perpendicular to the opening 54a.

[0043] As shown in Figure 2, the portion located at the tip of the leaf blade 212 is defined as the "tip portion TP". The tip portion TP is the portion located at the tip of the leaf vein 218 (main vein) that extends approximately parallel to the end portion (end and its vicinity) of the petiole 214, including the connection portion CP. The inclination of the leaf 210 (leaf blade 212) in the vertical direction is defined by the inclination of the imaginary line segment between the connection portion CP and the tip portion TP. When the crop 200 is a cucumber, due to the weight of the leaf blade 212, the imaginary line segment from the connection portion CP to the tip portion TP often extends vertically downward, starting from the connection portion CP.

[0044] The robot control unit 104 may adjust the posture of the end effector 50 by the robot 2 so that, at the work start position, the opening 54a is aligned in the vertical direction, or the angle of the opening 54a is tilted by ± a few degrees with respect to the vertical direction. In the vertical direction, at least a portion of the opening 54a of the end effector 50 positioned at the work start position and at least a portion of the leaf 210 (leaf blade 212) may be at the same height. Before, during, or after the execution of the first control, the drive control unit 102 may control the drive motor 72 of the drive unit 70 to start driving the impeller 62 and the cutting member 80.

[0045] After the execution of the first control, as shown in Figure 8(b), the robot control unit 104 executes a second control to move the end effector 50 from the work start position toward the leaf 210. The robot control unit 104 may execute the first and second controls consecutively (i.e., it may transition from the first control to the second control without stopping the end effector 50 at the work start position). For example, when executing the second control, the robot control unit 104 controls the robot 2 so that the end effector 50 moves along a virtual line connecting the center of the opening 54a at the work start position to any point on the leaf 210. When executing the second control, the robot control unit 104 may control the robot 2 so that the end effector 50 moves from the work start position toward the connection point CP between the leaf blade 212 and the petiole 214. In one example, the robot control unit 104 controls the robot 2 so that the end effector 50 moves along a virtual line connecting the center of the opening 54a at the work start position to the connection point CP.

[0046] As the end effector 50 approaches the leaf 210, the entire leaf blade 212 and a portion of the petiole 214 are drawn into the internal space S within the end effector 50. The portion of the leaf 210 taken into the internal space S is then shredded by the rotating impeller 62. The controller 100, which has a robot control unit 104, performs first and second controls based on the captured image obtained by the first measurement unit 8. Specific examples of the first and second controls performed based on the captured image will be described later.

[0047] The robot control unit 104 terminates the second control when a predetermined stop condition is met. The robot control unit 104 determines that the stop condition is met, for example, when the end effector 50 has moved by a predetermined set amount. After the execution of the second control, the robot control unit 104 performs a third control, as shown in Figure 8(c), to move the end effector 50 along a direction intersecting a virtual line perpendicular to the opening 54a. The virtual line perpendicular to the opening 54a may coincide with axis Ax1 or axis Ax2. As a result of the operation accompanying this third control, multiple blades 84 of the rotationally driven cutting member 80 that extend along the direction intersecting the virtual line cut the petiole 214 of the leaf 210.

[0048] The first measurement control unit 106 acquires data from the first measurement unit 8 to detect the leaves 210 to be harvested from the crop 200. The first measurement control unit 106 causes the image sensor of the first measurement unit 8 to perform imaging and the distance sensor of the first measurement unit 8 to perform distance measurement, for example, at measurement timings corresponding to the movement of the trolley 6 or at predetermined measurement timings. As a result, as shown in Figure 9, an image D1 (imported image) obtained by imaging a part of the crop 200 is obtained from the image sensor. In addition, data representing the distance of each pixel in the region corresponding to image D1 (depth map) is obtained from the distance sensor.

[0049] The state estimation unit 108 detects one or more leaves 210 that are candidates for collection from image D1. Figure 9 shows an example in which multiple candidate leaves 210 are detected as detection object id1, detection object id2, detection object id3, and detection object id4 from image D1. The controller 100 may also detect a rectangular bounding box surrounding the leaf 210 from image D1 for each of detection object id1, detection object id2, detection object id3, and detection object id4. In addition to the candidate leaf 210 itself, the state estimation unit 108 may also detect the connection portion CP (the portion connecting the leaf blade 212 and the petiole 214) on the leaf 210 from image D1.

[0050] In the example shown in Figure 9, the state estimation unit 108 detects the approach portion ap1 of the rectangular frame surrounding the connection portion CP of the detected object id1. The size of the rectangular frame showing the approach portion ap1 is smaller than the bounding box surrounding the leaf 210 which is a candidate for sampling. Similarly, the state estimation unit 108 detects the approach portions ap2, ap3, and ap4 surrounding the connection portion CP of the corresponding detected objects id2, id3, and id4, respectively. The approach portions ap1, etc., become the target positions when the end effector 50 is brought closer to the leaf 210 in the second control when the corresponding detected object is determined to be a sampling target.

[0051] The state estimation unit 108 may estimate the orientation of the leaf blade 212 included in the candidate leaf 210 to be collected, in addition to the leaf 210 itself and the approach portion, based on the image D1. In this disclosure, the orientation of the leaf blade 212 represents the inclination (orientation) of the imaginary line segment connecting the connecting portion CP and the tip portion TP in the horizontal plane. Figure 10 illustrates five leaves 210 with different orientations of leaf blade 212. To individually identify the five leaves 210, they are represented in Figure 10 as "210a", "210b", "210c", "210d", and "210e".

[0052] The orientation of the leaf blade 212 can be represented by classifying it into multiple categories, for example. In one example, the orientation of the leaf blade 212 can be represented by one of the following classifications (orientations): central, diagonally left, left, diagonally right, and right. In this disclosure, the terms "left" and "right" are used with reference to the view of the crop 200 (shelf 290) from the traveling trolley 6 in the Y-axis direction. In the example shown in Figure 10, the orientation of the leaf blade of leaf 210a is central, the orientation of the leaf blade of leaf 210b is diagonally left, and the orientation of the leaf blade of leaf 210c is left. Also, the orientation of the leaf blade of leaf 210d is diagonally right, and the orientation of the leaf blade of leaf 210e is right. The central orientation of the leaf blade includes not only the case where the imaginary line segment connecting the connecting portion CP and the tip portion TP is parallel to the Y-axis direction, but also the case where it is inclined with respect to the Y-axis direction. The orientation of the leaf blade to the right and to the left includes not only the case where the imaginary line segment connecting the connecting portion CP and the apical portion TP is parallel to the X-axis, but also the case where it is inclined with respect to the X-axis.

[0053] The state estimation unit 108 may estimate the orientation of the leaf blades of one or more leaves 210 detected in image D1 based on an estimation model pre-built by machine learning and image D1 (image data representing image D1) obtained by the first measurement unit 8. The model holding unit 112 holds the estimation model. The estimation model held by the model holding unit 112 is pre-built by machine learning to detect leaves 210 contained in an image in response to an input image obtained by imaging, and to output information indicating the orientation of the leaf blades contained in the detected leaves 210. The estimation model held by the model holding unit 112 may also be built to further detect the connection portion CP of the leaves 210 contained in the input image. The bounding boxes for each of the detected objects id1 to id4 and the approach portion frames surrounding the connection portion CP exemplified in Figure 9 represent an example of the results output by the estimation model.

[0054] The model building unit 114 constructs the estimation model by performing machine learning based on multiple training images obtained by imaging the crop 200 and ground truth data indicating the orientation of the leaf blades contained in each of the multiple training images. The crop 200 to be imaged in order to obtain multiple training images may be of the same type as the crop 200 that is harvested by the leaf harvesting device 1, and may be the same individual as the crop 200 that is harvested, or it may be a different individual. There may be one or more leaves 210 in each training image. The ground truth data indicating the orientation of the leaf blades of each of the one or more leaves 210 present in each training image is classified by an operator such as a worker and then input into the controller 100. The operator may classify the orientation of the leaf blades (assign ground truth labels) by looking at the leaves 210 visualized in the images obtained by imaging, without measuring the angle in the horizontal direction of the leaf blades.

[0055] Machine learning is a method in which a machine (computer) autonomously discovers laws or rules by iteratively learning based on given information. The estimation model described above can be constructed using algorithms and data structures. For example, the estimation model can be realized using a neural network, which is an information processing model that mimics the structure of the human brain. The specific machine learning algorithms used when constructing the estimation model are not particularly limited, but examples of algorithms for detecting objects include R-CNN (Regional CNN), YOLO (You Only Look Once), and SSD (Single Shot Multibox Detector).

[0056] The estimation model is constructed by performing machine learning based on training data for building the model. For example, by performing machine learning using data given as input to machine learning (the multiple training images mentioned above) and the ground truth data of the machine learning output (detection positions of leaves 210 and connecting portion CPs, and the orientation of the leaf blade), an estimation model that detects leaves 210, identifies their position on the image, and estimates the orientation of the leaf blade may be autonomously constructed. The input to machine learning is various image data containing one or more leaves 210. The output of machine learning may be the detection positions of leaves 210 and connecting portion CPs, and the orientation of the leaf blade. The detection position on the image may be the position (vertical and horizontal coordinates) of the frame surrounding the leaf 210 or connecting portion CP on the image.

[0057] The stage in which the estimation model is autonomously manufactured corresponds to the learning phase. In the evaluation phase (the phase in which the actual collection work is performed), the state estimation unit 108 uses the estimation model held in the model holding unit 112 to detect the leaf 210 and the connecting portion CP in the input image, from an input image where the position of the leaf 210, the position of the connecting portion CP, and the orientation of the leaf blade are unknown, and estimates the orientation of the leaf blade of the detected leaf 210. The estimation model, being a trained model, is portable between computers. Therefore, the controller 100 may, instead of itself, acquire an estimation model constructed on another computer and store that estimation model in the model holding unit 112.

[0058] The work procedure determination unit 110 determines the procedure for having the robot 2 and the end effector 50 perform the harvesting operation. For example, the work procedure determination unit 110 determines which leaves 210 to actually harvest from among a plurality of leaves 210 detected in image D1. For example, the work procedure determination unit 110 determines the target leaves 210 to harvest a predetermined number of leaves 210 (for example, 2 leaves) per plant. When determining the target leaves 210, the work procedure determination unit 110 may calculate the position of the connection portion CP for each of the plurality of candidate leaves 210 to be harvested, based on distance data (depth map) obtained from the distance sensor of the first measurement unit 8. For example, the work procedure determination unit 110 calculates the three-dimensional position of the connection portion CP (for example, the position relative to the reference position of the robot 2) using the measured distance at the coordinate corresponding to the coordinate of the connection portion CP on image D1 from the distance data (depth map).

[0059] When determining the target leaf 210, the work procedure determination unit 110 may calculate the work start position and the direction of approach from the work start position to the leaf 210 (hereinafter referred to as the "approach direction") for each of the multiple candidate leaves 210 that are the target of harvesting, based on the estimated orientation of the leaf blade. An example of the relationship between the orientation of the leaf blade and the work start position and approach direction will be explained using Figure 10. The work procedure determination unit 110 calculates (determines) the work start position and approach direction according to the orientation of the leaf blade. In one example, for a leaf 210a that is estimated to be facing the center, the work procedure determination unit 110 calculates the work start position so that it is a position that approximately coincides with the position of the connection part CP in the X-axis direction. The work procedure determination unit 110 also calculates the orientation of the end effector 50 (opening 54a) and the approach direction at the work start position so that the movement trajectory of the end effector 50 when approaching the leaf 210a from the work start position follows the Y-axis direction.

[0060] The work procedure determination unit 110 calculates the work start position for leaf 210b, which is estimated to be facing diagonally to the left, so that it is located to the left of leaf 210b in the X-axis direction. The work procedure determination unit 110 also calculates the orientation and approach direction of the end effector 50 (opening 54a) at the work start position so that the movement trajectory of the end effector 50 when approaching leaf 210b from the work start position is inclined with respect to both the X-axis and Y-axis directions. The work procedure determination unit 110 calculates the work start position for leaf 210c, which is estimated to be facing left, so that it is located to the left of leaf 210c in the X-axis direction and approximately coincides with leaf 210c in the Y-axis direction. The work procedure determination unit 110 also calculates the orientation and approach direction of the end effector 50 (opening 54a) at the work start position so that the movement trajectory of the end effector 50 when approaching leaf 210c from the work start position is along the X-axis direction.

[0061] In a plan view, the direction toward the main stem 202 of the crop 200 is defined as "inward," and the direction away from the main stem 202 is defined as "outward." For each of the multiple leaves 210 that are candidates for harvesting, the work procedure determination unit 110 calculates the orientation of the end effector 50 (opening 54a) at the work start position and the approach direction so that the end effector 50 moves from the outside to the inside when approaching the leaf 210 from the work start position.

[0062] Here, using Figures 11 and 12, we will explain the method by which the work procedure determination unit 110 determines the work procedure, and an example of harvesting work based on the determined work procedure. The controller 100 performs the following first, second, and third processes while the traveling carriage 6 continues to move. In the first process, the first measurement control unit 106 of the controller 100 causes the first measurement unit 8 to perform imaging of the imaging range PR1. In the second process, the work procedure determination unit 110 of the controller 100 determines the execution order of harvesting operations from the crop 200 for multiple leaves 210 included in the image D1 obtained by imaging by the first measurement unit 8 in the first process. In the third process, the drive control unit 102 and robot control unit 104 of the controller 100 perform leaf harvesting operations on the crop 200 by operating the end effector 50 and robot 2 according to the execution order determined in the second process.

[0063] The controller 100 repeats a series of processes, including the first process, the second process, and the third process, in accordance with the movement of the trolley 6. As the above series of processes is repeated, the portion of the crop 200 included in the imaging range PR1 is updated, and the leaves 210 to be collected and the work order (order of leaf collection) are updated. For example, the controller 100 repeats the series of processes, including the first process, the second process, and the third process, each time the trolley 6 travels a predetermined distance. Alternatively, the controller 100 may repeat the series of processes, including the first process, the second process, and the third process, each time a collection operation is performed on a predetermined number of leaves 210.

[0064] Figure 11 shows the determined execution order of operations for each of the multiple leaves 210 at a certain time t. In Figure 11, the range AR in which the end effector 50 can move due to the operation of the robot 2 is shown. The imaging range PR1, which is the field of view of the image sensor of the first measurement unit 8, is set to correspond to the range AR that indicates the movable range of the end effector 50. In the second process, the work procedure determination unit 110 extracts a predetermined number of leaves 210 (for example, 2 leaves) from among the multiple leaves 210 included in image D1, for each plant included in image D1, according to predetermined conditions. The work procedure determination unit 110 may extract 2 leaves 210 for each plant based on at least one of the position in the X-axis direction and the Y-axis direction of the connection portion CP of each of the multiple leaves 210 included in image D1.

[0065] In the X-axis direction, the direction in front of the moving trolley 6 is defined as "positive," and the direction opposite to the direction of travel of the moving trolley 6, towards the rear, is defined as "negative." The position in the X-axis direction increases as the direction of travel of the moving trolley 6 progresses. In the Y-axis direction, the direction from the moving trolley 6 toward the crop 200 (shelf 290) is defined as "positive," and the direction from the crop 200 (shelf 290) toward the moving trolley 6 is defined as "negative." The position in the Y-axis direction increases as the distance from the moving trolley 6 toward the crop 200 progresses. In one example, if the work procedure determination unit 110 contains three or more leaves 210 on a single plant in image D1, it extracts two leaves 210 with small positions in the X-axis direction of the connection portion CP. The work procedure determination unit 110 may also extract two leaves 210 with small positions in the Y-axis direction of the connection portion CP, or two leaves 210 with small sums of positions in the X-axis and Y-axis directions.

[0066] In the example shown in Figure 11, four leaves 210 are extracted from image D1 as leaves to be collected. The work procedure determination unit 110 determines the order in which the work is performed on the four extracted leaves 210. For example, in the second process, the work procedure determination unit 110 determines which of the four leaves 210 will be collected first, based on the position of the leaf 210 in the X-axis direction and the position of the leaf 210 in the Y-axis direction which is perpendicular to the X-axis direction. In one example, the work procedure determination unit 110 determines that the leaf 210 with the smallest position of the connecting portion CP in the X-axis direction is the leaf 210 to be collected first. There may be other leaves 210 whose positions in the X-axis direction overlap with the leaf 210 with the smallest position of the connecting portion CP in the X-axis direction, or whose difference in position in the X-axis direction is smaller than a predetermined amount. In this case, the work procedure determination unit 110 may determine the leaf 210 with the smaller position in the Y-axis direction of the connection portion CP as the leaf 210 to be harvested first.

[0067] The work procedure determination unit 110 may determine the order of the second and subsequent leaves in ascending order of the position of the connection portion CP in the X-axis direction. Furthermore, if there are two or more leaves 210 whose positions in the X-axis direction of the connection portion CP overlap with each other, or whose difference in position in the X-axis direction is less than a predetermined amount, the work procedure determination unit 110 may determine the leaf 210 with the smallest position of the connection portion CP in the Y-axis direction to be harvested first. In Figure 11, the leaf determined to be harvested first is shown as "210A", and the leaves determined to be harvested second, third, and fourth are shown as "210B", "210C", and "210D", respectively.

[0068] The drive control unit 102 and the robot control unit 104 control the end effector 50 and the robot 2 to perform the operation of harvesting leaf 210A (the first leaf) (first operation) after the order of leaf harvesting has been determined in the second process described above. After the operation of harvesting leaf 210A has been performed, the drive control unit 102 and the robot control unit 104 control the end effector 50 and the robot 2 to perform the operation of harvesting leaf 210B (the second leaf) (second operation). In the example shown in Figure 11, the harvesting operations for leaves 210A and 210B have been performed, and the next measurement timing, time (t+1), has been reached before the harvesting operation for leaf 210C is performed. At time (t+1), the controller 100 performs the series of operations, including the first process, the second process, and the third process, again. At time t1, the two leaves 210 that were determined to be harvested third and fourth are determined to be harvested first and second at time (t+1). When the next measurement timing, time (t+2), occurs, the controller 100 repeats the above series of processes. The controller 100 repeats the above series of processes until the trolley 6 reaches a predetermined end position.

[0069] The work procedure determination unit 110 may, in the second processing, calculate the approach direction for bringing the end effector 50 closer to each of the multiple leaves 210 contained in the image D1 (for example, the four leaves 210 extracted as described above). Based on the calculation result of the approach direction, the work procedure determination unit 110 may determine which of the four leaves 210 contained in the image D1 will be the second (or subsequent) leaf to be collected. In Figures 12(a) and 12(b), the collection order of each of the four leaves 210 is shown using "210A" to "210D", similar to Figure 11. The calculation results of the approach direction for each of the leaves 210A to 210D are shown by white arrows.

[0070] In the example shown in Figure 12(a), the order of the second and subsequent leaves is determined based on the position of the connection part CP in the X-axis direction and the position in the Y-axis direction, without considering the approach direction. In the example shown in Figure 12(b), the order of the second and subsequent leaves is determined based on the approach direction. In one example, the work procedure determination unit 110 may determine the order of the second and subsequent leaves by first taking leaves 210 that have a distance in the X-axis direction from the previous leaf 210 that is smaller than a predetermined value, and that have the same approach direction as the previous leaf 210, instead of sorting by the position in the X-axis direction. In Figure 12(a), the two leaves 210 (leaves 210A and 210C) located closer to the trolley 6 are determined to be taken first and third. In contrast, in Figure 12(b), the two leaves 210 (leaves 210A and 210B) located closer to the trolley 6 are determined to be taken first and second.

[0071] Returning to Figure 7, the state monitoring unit 122 monitors fluctuations in a state value representing the operating state of the drive motor 72 of the drive unit 70. Based on the fluctuations in the state value, the state monitoring unit 122 detects the state of work on the leaf 210 in the end effector 50. The state value representing the operating state of the drive motor 72 may be a detected value from a sensor capable of detecting the state of the drive motor 72, or it may be a command value representing an operation command from the controller 100 to the drive motor 72. The state value may be, for example, a detected value of the current flowing through the drive motor 72, or a torque command value or current command value for the drive motor 72. When the rotational speed of the drive motor 72 is controlled to follow a target rotational speed, the torque command value (current command value) fluctuates when a load is applied to the impeller 62, so the torque command value (current command value) represents the operating state of the drive motor 72.

[0072] The state monitoring unit 122 detects at least one of the start and completion of shredding by the impeller 62 on the leaves 210 taken into the internal space S, based on the fluctuation of the state value of the drive motor 72. Figure 13 shows the time change of the rotation speed feedback value (value detected by the sensor) and the torque command value when the rotation speed of the drive motor 72 is controlled to follow a target value. In the graph shown in Figure 13, the rotation speed on the vertical axis increases as you move downward from the position indicated by "0", and the torque command value on the vertical axis increases as you move downward from the position indicated by "0". During the time period indicated by "Accelerating", the torque command value also increases as the rotation speed of the drive motor 72 rises to the target rotation speed. During the time period indicated by "No Load", no leaves 210 are taken into the internal space S, and there is no load on the impeller 62. Therefore, the range of fluctuation of the torque command value is smaller compared to other time periods.

[0073] During the period indicated as "leaf blade shredding," the leaf 210 is taken into the internal space S, and shredding of the leaf blade 212 is performed. Immediately after the start of shredding of the leaf blade 212, the torque command value fluctuates significantly compared to the no-load period. After the shredding of the leaf blade 212 is completed, the fluctuation of the torque command value decreases to the same level as during the no-load period. Then, during the period indicated as "petiole shredding" after the no-load state has continued, the petiole 214 is cut, and shredding of the petiole 214 is performed. Immediately after the start of shredding of the petiole 214, the torque command value fluctuates significantly compared to the no-load period. After the shredding of the petiole 214 is completed, the fluctuation of the torque command value decreases to the same level as during the no-load period.

[0074] The status monitoring unit 122 may use the fluctuation in the torque command value, as shown in Figure 13, to detect the start of shredding of the leaf blade 212, or to detect the completion of shredding of the leaf blade 212. The status monitoring unit 122 may also detect the start of shredding of the petiole 214, or to detect the completion of shredding of the petiole 214. Note that detecting at least one of the start and completion of shredding includes detecting either the start or completion of shredding, and detecting both the start and completion of shredding.

[0075] The controller 100 controls the operation of either or both the robot 2 and the end effector 50 based on the detection result of monitoring and completion of shredding by the impeller 62 on the leaf 210 taken into the internal space S. For example, if the controller 100 detects the completion of shredding by the impeller 62 on the leaf 210A (the petiole 214 of the leaf 210A) based on the change in the above state value obtained during the execution of the operation to collect leaf 210A, it terminates the operation to collect leaf 210A. The controller 100 then controls the robot 2 to terminate the operation to collect leaf 210A and move on to the operation to collect the next target, leaf 210B. An example of an operation to move on to the operation to collect leaf 210B is to move the end effector 50 to a standby position that overlaps with the traveling carriage 6.

[0076] In one example, the drive control unit 102 of the controller 100 controls the drive motor 72 to reduce the rotational speed of the impeller 62 when it detects the completion of shredding of the leaf 210A (the petiole 214 of the leaf 210A) based on the change in the above state value during the operation of collecting the leaf 210A. The drive control unit 102 may also control the drive motor 72 so that the rotational speed of the impeller 62 becomes a value smaller than the target rotational speed during the execution of shredding of the leaf 210 when it detects the completion of shredding. In one example, the drive control unit 102 controls the drive motor 72 so that the rotation of the impeller 62 stops when it detects the completion of shredding. As described above, reducing the rotational speed of the impeller 62 (decelerating the rotation of the impeller 62) also includes stopping the rotation of the impeller 62 and setting the rotational speed to zero.

[0077] Returning to Figure 7, the second measurement control unit 126 obtains an image (second image) obtained by imaging the imaging range PR2 set in front of the trolley 6 by having the second measurement unit 9 perform imaging of the imaging range PR2. The second measurement control unit 126 may repeatedly perform imaging by the second measurement unit 9 in accordance with the movement of the trolley 6. For example, the second measurement control unit 126 performs imaging by the second measurement unit 9 each time the trolley 6 moves by a predetermined set amount. The second measurement control unit 126 may calculate the density of leaves 210 contained in the image obtained by imaging by the second measurement unit 9. In one example, the second measurement control unit 126 detects one or more leaves 210 contained in the image and then counts the number of detected leaves to calculate the density. The second measurement control unit 126 may calculate the density of leaves 210 in the image each time imaging of the imaging range PR2 is performed.

[0078] The travel control unit 124 controls the travel cart 6 to travel at a preset speed. The reference value for the travel cart 6's movement speed is preset. The travel control unit 124 may also control the travel cart 6 to continue traveling without stopping until all harvesting work on the crops 200 planted on one trellis 290 is completed. The robot control unit 104 takes into account the movement of the robot 2 accompanying the travel cart 6's movement and executes the movement operation of the end effector 50 by the robot 2 (control of actuators 41-46). The travel control unit 124 may also adjust the travel cart 6's movement speed based on the calculated result of the density of leaves 210 within the imaging range PR2. In one example, the travel control unit 124 maintains the travel cart 6's movement speed at the reference value when the calculated value of the density of leaves 210 is greater than a predetermined threshold. The travel control unit 124 changes the travel cart 6's movement speed to a value greater than the reference value when the calculated value of the density of leaves 210 is less than or equal to the predetermined threshold. The second measurement control unit 126 may adjust the movement speed of the traveling trolley 6 each time imaging of the imaging range PR2 and calculation of the density of the leaves 210 are performed.

[0079] [Control method for leaf harvesting device] Figure 14 is a flowchart illustrating an example of a control method performed in the leaf collection device 1. The controller 100 executes step S11, for example, when the travel carriage 6 has started moving. In step S11, for example, the first measurement control unit 106 causes the image sensor of the first measurement unit 8 to perform imaging and acquires an image D1 obtained by imaging the imaging range PR1. The first measurement control unit 106 also causes the distance sensor of the first measurement unit 8 to perform distance measurement and acquires a depth map in the region corresponding to the imaging range PR1.

[0080] Next, the controller 100 executes step S12. In step S12, for example, the state estimation unit 108 detects one or more leaves 210 that are candidates for collection included in image D1, and the connection portion CP in each of the one or more leaves 210, based on the image D1 acquired in step S11. In step S12, for example, the state estimation unit 108 estimates the horizontal orientation of the leaf blade 212 for each of the one or more leaves 210 detected in image D1. The state estimation unit 108 detects one or more leaves 210 and the connection portion CP from the output result obtained by inputting the image D1 acquired in step S11 into an estimation model constructed by machine learning, and then estimates the orientation of the leaves 210. Furthermore, in step S12, for example, the state estimation unit 108 calculates the position of the connection portion CP from the depth map obtained in step S11 for each of the one or more leaves 210 detected in image D1.

[0081] Next, the controller 100 executes step S13. In step S13, for example, the work procedure determination unit 110 executes the second process described above. In step S13, the work procedure determination unit 110 determines the execution order of the harvesting operations from the crop 200 for the multiple leaves 210 contained in the image D1 obtained in step S11. In one example, after extracting the multiple leaves 210 to be harvested, the work procedure determination unit 110 determines which leaf 210 to harvest first from the extracted multiple leaves 210 based on the position of the leaf 210 in the X-axis direction and the position of the leaf 210 in the Y-axis direction. Then, for each of the extracted multiple leaves 210, the work procedure determination unit 110 calculates the approach direction for bringing the end effector 50 closer based on the estimation result of the orientation of the leaf blade 212. After that, the work procedure determination unit 110 determines which leaves 210 to harvest second and subsequent from the extracted multiple leaves 210 based on the calculation result of the approach direction.

[0082] Next, the controller 100 executes step S14. In step S14, for example, the robot control unit 104 executes control to move the end effector 50 to the work start position for the leaf 210A that was determined to be the first target for harvesting in step S13. In the control of step S14 (first control), the robot control unit 104 moves the end effector 50 to the work start position set according to a predetermined setting method. In step S14, the robot control unit 104 controls the robot 2 to move the end effector 50 to the work start position where the opening 54a faces the surface 212a of the leaf 210A.

[0083] Next, the controller 100 performs step S15. In step S15, for example, the drive control unit 102 controls the drive motor 72 to start rotating the impeller 62 and the cutting member 80. This puts the end effector 50 in a state where it can attract leaves 210 into the internal space S through the opening 54a. After the execution of step S15, the drive control unit 102 controls the drive motor 72 so that the rotational speed of the impeller 62 follows the target rotational speed. In conjunction with the execution of step S15, the state monitoring unit 122 may start monitoring a state value (for example, a torque command value) that represents the operating state of the drive motor 72.

[0084] Next, the controller 100 executes step S16. In step S16, for example, the robot control unit 104 controls the robot 2 to move the end effector 50 toward the leaf 210A from the starting position. As the end effector 50 approaches the leaf 210A, the leaf 210A is attracted (taken into) the internal space S. Then, as the leaf blade 212 of the leaf 210A begins to come into contact with the impeller 62, the shredding of the leaf blade 212 by the impeller 62 begins.

[0085] Next, the controller 100 executes step S17. In step S17, for example, the robot control unit 104 moves the robot 2 to move the end effector 50 so that the leaves 210A are cut from the crop 200 by the cutting member 80. In one example, the robot control unit 104 controls the robot 2 to move the end effector 50 along a direction intersecting axis Ax1 or axis Ax2. During the execution of step S17, the drive control unit 102 controls the drive motor 72 to continue the rotation of the cutting member 80 around axis Ax2.

[0086] Next, the controller 100 performs step S18. In step S18, for example, the state monitoring unit 122 waits, based on the fluctuation of the state value of the drive motor 72, until it detects that the shredding by the impeller 62 of the portion of the leaf 210A that is in the internal space S after cutting is complete. The state monitoring unit 122 may also detect the start of shredding when the fluctuation range of the torque command value to the drive motor 72 becomes larger than a predetermined size, or, after detecting the start of shredding, it may detect the completion of shredding when the fluctuation range of the torque command value becomes smaller than a predetermined size.

[0087] Next, the controller 100 executes step S19. In step S19, for example, the controller 100 determines that the harvesting operation on the leaf 210A is complete. Then, in step S19, for example, the drive control unit 102 controls the drive motor 72 to stop the rotation of the impeller 62 and the cutting member 80, and the robot control unit 104 controls the robot 2 to move the end effector 50 to a standby position that overlaps with at least a part of the traveling carriage 6 in a plan view.

[0088] Next, the controller 100 executes step S20. In step S20, for example, the controller 100 determines whether or not it is a measurement timing. In one example, the controller 100 determines that it is a measurement timing when the distance traveled by the trolley 6 from the start of the execution of step S11 has reached a predetermined amount. The controller 100 determines that it is not a measurement timing when the distance traveled by the trolley 6 from the start of the execution of step S11 has not reached a predetermined amount.

[0089] If it is determined in step S20 that it is not time to measure (step S20: NO), the controller 100 returns to step S14. The controller 100 then performs the series of processes from steps S14 to S19 for the next leaf 210 to be collected. The controller 100 continues to perform the series of processes from steps S14 to S19, changing the leaf 210 to be collected, until it is determined in step S20 that it is time to measure.

[0090] If it is determined in step S20 that it is time for measurement (step S20: YES), the controller 100 returns to step S11. The controller 100 then performs a series of steps S11 to S13 to update the leaves 210 to be collected and the collection procedure. Subsequently, the controller 100 repeats a series of steps S14 to S19 according to the updated collection procedure until the next time for measurement. Thereafter, the controller 100 repeatedly updates the target and collection procedure and performs the collection work according to the updated collection procedure until the trolley 6 travels to a predetermined end position.

[0091] While the flow shown in Figure 14 is being executed, the travel control unit 124 may control the travel carriage 6 to continue traveling without stopping it. At a timing (period) different from the measurement timing of step S20, the second measurement control unit 126 may cause the second measurement unit 9 to perform imaging of the imaging range PR2 set in front of the travel carriage 6. The travel control unit 124 may calculate the density of leaves 210 from the image obtained by imaging the imaging range PR2, and if the calculated value of the density of leaves 210 is smaller than a predetermined threshold, it may control the travel carriage 6 to increase its moving speed above a reference value.

[0092] The controller 100 includes a circuit 150, as shown in Figure 15. The circuit 150 includes one or more processors 152, a memory 154, a storage 156, one or more drivers 158, and input / output ports 162. The storage 156 is a computer-readable non-volatile storage medium (e.g., flash memory). The storage 156 stores programs and data that cause the controller 100 to control the robot 2, the end effector 50, the traveling carriage 6, the first measurement unit 8, and the second measurement unit 9. The memory 154 temporarily stores programs loaded from the storage 156 and calculation results from the processor 152.

[0093] The processor 152, in cooperation with the memory 154, executes the above-mentioned program to configure each of the function modules described above. One or more drivers 158 output drive power to the drive motor 72 of the end effector 50 and the actuators 41-46 of the robot 2, respectively, in response to commands from the processor 152. One or more drivers 158 may be provided separately from the controller 100 (outside the controller 100). A so-called power conversion device such as a servo amplifier or inverter may be used as the driver 158. The input / output port 162 inputs and outputs electrical signals to and from the suction unit 60 of the end effector 50, the robot 2, the trolley 6, the first measurement unit 8, and the second measurement unit 9, etc., in response to commands from the processor 152. Note that the controller 100 is not necessarily limited to configuring each function by program. The controller 100 may, for example, configure at least some of its functions by a dedicated logic circuit or an ASIC (Application Specific Integrated Circuit) that integrates such a circuit. If the controller 100 is composed of multiple computer devices corresponding to each of the multiple devices included in the leaf collection device 1, such as the robot 2 and the end effector 50, a driver 158 may be provided corresponding to each of the multiple devices.

[0094] [Rating 1] Next, we will explain the results of evaluating the success rate of suction when the direction in which the end effector 50 is brought close to the leaf 210 (the approach direction described above) is changed relative to the orientation of the leaf 210. In evaluating the success rate, we evaluated the case where the target position for bringing the end effector 50 close was set to the connection part CP (petiole base) and the case where the target position was set to the center of the leaf blade 212 (hereinafter referred to as "leaf blade center MI"; see Figure 16(a)). The target position for bringing the end effector 50 close is the position where the axis Ax1 (corresponding to the central axis of the end effector 50) and the leaf 210 intersect when the opening 54a of the end effector 50 is brought close to the leaf 210. The leaf blade center MI was set to the most protruding point on the main vein of the leaf blade 212. Regarding the vertical angle (elevation angle) of the direction in which the end effector 50 is brought close to the leaf 210, the end effector 50 was moved along the direction normal to the leaf 210 at the target position to bring it close to the leaf 210.

[0095] Regarding the approach direction, as shown in Figure 16(a), the direction corresponding to the orientation of the leaf 210 was set to 0°, and the directions were varied to 0°, 45°, 90°, and 135°. Three leaves were evaluated per condition, and success was evaluated when a portion of the leaf blade 212 and petiole 214 of the leaf 210 were attracted and taken into the internal space S of the end effector 50.

[0096] Table 1 shows the evaluation results of the success rate (number of successes / number of evaluated leaves) when the approach direction was changed. From the results shown in Table 1, it can be seen that the success rate is higher when the end effector 50 is brought closer so that the approach direction is closer to the orientation of leaf 210. Furthermore, it can be seen that the success rate is further increased by making the approach direction approximately the same as the orientation of leaf 210 (by setting the approach direction to 0°). [Table 1]

[0097] [Rating 2] The success rate of suction was evaluated when the angle in the vertical direction of approaching the end effector 50 (hereinafter referred to as the "approach angle") was changed relative to the normal direction to the leaf 210 at the target position. As shown in Figure 16(b), the normal direction to the leaf 210 at the target position was set to 0°, and the approach angle was changed to -30°, 0°, 30°, 45°, and 60°. The approach direction was also set to 0°. The target position was set at both the connection point CP and the leaf blade center MI. When the target position was the connection point CP, the number of evaluations per condition was set to 5, and when the target position was the leaf blade center MI, the number of evaluations per condition was set to 3. The method for determining success or failure was the same as in Evaluation 1 described above.

[0098] Table 2 shows the evaluation results of the success rate (number of successes / number of evaluated leaves) when the approach angle is changed. From the evaluation results shown in Table 2, it can be seen that suction may be successful when the end effector 50 is brought close from a position where the opening 54a faces the surface 212a of the leaf 210 (opposite the surface 212a). It can also be seen that the success rate tends to increase when the approach angle is brought closer to the normal of a predetermined point on the leaf 210 (bringing the approach angle closer to 0°). [Table 2]

[0099] [Differentiation] The series of processes shown in Figure 14 is an example and can be modified as appropriate. In the above series of processes, the controller 100 may execute one step and the next step in parallel, or execute each step in a different order than the example described above. For example, the controller 100 may execute step S15 (start driving the end effector) and then step S14 (move to the work start position). The controller 100 may omit any of the steps, or may execute a different process in any of the steps than the example described above.

[0100] The suction section 60 of the end effector 50 may be configured in any way as long as it is possible to generate suction force in the internal space S formed in the end effector 50. The suction section 60 may, for example, have an impeller (fan) arranged in a space separated from the internal space S.

[0101] The end effector 50 may have a drive motor for rotating the cutting member 80 independently of the drive of the impeller 62. The direction of rotation of the impeller 62 around its axis Ax1 may be the same as the direction of rotation of the cutting member 80 around its axis Ax2. The cutting member 80 may be fixed to the case member 52 without rotating. The end effector 50 does not have a protective member 88 that covers the cutting member 80 from the front.

[0102] The drive control unit 102 of the controller 100 may control the drive motor 72 to continue rotating the impeller 62 without reducing its rotational speed between the completion of the operation to collect one leaf 210 and the start of the operation to collect the next leaf 210. The controller 100 may determine one leaf 210 to be collected each time imaging of the imaging range PR1 is performed by the first measurement unit 8, without determining the order of collection for multiple leaves 210. If the leaf collection device 1 does not have a second measurement unit 9, the controller 100 may control the traveling cart 6 to travel at a constant target speed regardless of the state of the crop 200. The controller 100 may stop the traveling cart 6 while the robot 2 and end effector 50 perform the collection work. If the leaf collection device 1 does not have a traveling cart 6, the base 11 of the robot 2 may be installed at a location where its position does not change relative to the crop 200.

[0103] [Effects of the Embodiment] The leaf harvesting device 1 described above comprises an end effector 50 for harvesting leaves 210 from crops 200, and a robot 2 for changing the relative position of the end effector 50 with respect to the crops 200. The end effector 50 has a case member 52 with an opening 54a at its tip that forms an internal space S for taking in leaves 210, a suction unit 60 that generates a suction force to suck the leaves 210 into the internal space S through the opening 54a, and a cutting member 80 capable of cutting the leaves 210 taken into the internal space S.

[0104] In this leaf harvesting device 1, the robot 2 can automatically bring the end effector 50 close to the leaf 210. With the end effector 50 close to the leaf 210, the leaf 210 can be sucked up and then cut. In this case, compared to not using suction, a larger portion of the leaf 210 can be taken into the end effector 50 with fewer movements or in a shorter time, and the leaf 210 can be cut. Therefore, the leaf harvesting device 1 is useful for improving work efficiency.

[0105] The leaf harvesting device 1 described above may further include a trolley 6 to move the robot 2 to which the end effector 50 is attached. In this case, as the trolley 6 moves, the range in which the robot 2 can move the end effector 50 also moves. This allows harvesting work to be performed on crops 200 planted over a wide area with a single device. Therefore, this is useful for simplifying the leaf harvesting device 1.

[0106] In the leaf collection device 1 described above, the suction unit 60 may include an impeller 62 located in the internal space S and rotatable around an axis Ax1 intersecting the opening 54a, and a drive unit 70 that drives the impeller 62 to generate the suction force. In this case, the rotational movement of the impeller 62 located in the internal space S allows predetermined work to be performed on the leaves 210 taken into the internal space S. Therefore, the leaves 210 can be cut while being sucked up and predetermined work is performed on them. Thus, this is even more useful for improving work efficiency.

[0107] In the leaf collection device 1 described above, the impeller 62 may be formed to be able to shred the leaves 210 taken into the internal space S. In this case, the impeller 62 has the function of generating suction force and the function of shredding the sucked leaves 210. Therefore, it is useful for simplifying the device when shredding leaves 210 by suction.

[0108] In the leaf harvesting device 1 described above, the cutting member 80 may be provided on the peripheral wall 53 surrounding the axis Ax2 of the case member 52 so as to be rotatable around the axis Ax2 intersecting the opening 54a. The drive unit 70 may further drive the cutting member 80 to rotate around the axis Ax2. The drive unit 70 may include a drive motor 72 that generates a driving force and a transmission member 76 that transmits the driving force from the drive motor 72 to both the impeller 62 and the cutting member 80. In this case, one motor is shared to drive two rotating members. Therefore, this is useful for simplifying the device.

[0109] In the leaf harvesting device 1 described above, the cutting member 80 may be provided on the peripheral wall 53 surrounding the axis Ax2 of the case member 52 so as to be rotatable around the axis Ax2 intersecting the opening 54a. The drive unit 70 may further drive the cutting member 80 to rotate around the axis Ax2. The drive unit 70 may rotate the impeller 62 and the cutting member 80 in opposite directions. If the impeller 62 and the cutting member 80 are rotated in the same direction, the leaves 210 taken into the internal space S may rotate together with the impeller 62 and the cutting member 80 in the same direction as their rotation. In contrast, in the above configuration, by rotating in opposite directions, a force opposite to the rotation direction of the cutting member 80 is also applied to the leaves 210 taken into the internal space S. Therefore, the operation of the impeller 62 on the leaves 210 after they have been cut by the cutting member 80 is performed more reliably. Thus, it is even more useful for improving work efficiency.

[0110] In the leaf harvesting device 1 described above, the cutting member 80 may be provided on the peripheral wall 53 surrounding the axis Ax2 of the case member 52 so as to be rotatable around the axis Ax2 intersecting the opening 54a. In this case, by providing the rotating cutting member 80 on the peripheral wall of the case member 52 of the end effector 50, it is easy to position the rotating shaft or drive unit for rotating the cutting member 80 while avoiding the internal space S. For example, if the impeller 62 is located in the internal space S, it is easy to position the cutting member 80 while avoiding the space between the opening 54a and the impeller 62. This reduces the possibility that leaves 210 cannot be smoothly collected into the internal space S and the impeller 62. Therefore, it is even more useful for improving work efficiency.

[0111] In the leaf harvesting device 1 described above, the end effector 50 may further have a protective member 88 provided on the peripheral wall 53 so as to cover the cutting member 80 when the opening 54a is viewed from outside the internal space S. The protective member 88 may be formed to be expandable and contractible in a direction intersecting the axis Ax2. In this case, it is suppressed that parts of the crop 200 other than the target of harvesting become entangled with the cutting member 80 while the end effector 50 is moving. This reduces the possibility of a decrease in work efficiency due to parts of the crop 200 becoming entangled with the cutting member 80. Therefore, it is even more useful for improving work efficiency.

[0112] The leaf collection device 1 described above may further include a controller 100 that controls at least the end effector 50. The suction unit 60 may include a rotatable impeller 62 and a drive motor 72 that drives the impeller 62 to generate the suction force. The controller 100 may monitor fluctuations in a state value representing the operating state of the drive motor 72 and detect the state of work on the leaves 210 at the end effector 50 based on the fluctuations in the state value. In this case, the status of work on the leaves 210 can be grasped without adding sensors or the like to detect the state of work. Therefore, it is useful for both simplifying the device and improving work efficiency.

[0113] In the leaf collection device 1 described above, the impeller 62 may be arranged in the internal space S so as to rotate around an axis Ax1 that intersects the opening 54a. The impeller 62 may be formed to shred the leaves 210 taken into the internal space S. The operating state of the drive motor 72 changes depending on the state of the shredding operation on the leaves 210 by the impeller 62. In the above configuration, the status of the shredding operation on the leaves 210 can be grasped by monitoring the state value of the drive motor 72 without providing sensors or the like for monitoring.

[0114] In the leaf collection device 1 described above, the controller 100 may detect, based on the change in the above state value, the start and completion of shredding of the leaves 210 taken into the internal space S by the impeller 62. In this case, the controller 100 can perform control in accordance with the actual working conditions of shredding the leaves 210. Therefore, it is even more useful for improving the efficiency of the work.

[0115] In the leaf collection device 1 described above, the controller 100 may control the operation of either or both of the robot 2 and the end effector 50 based on the detection result of at least one of the start and completion of shredding by the impeller 62 on the leaves 210 taken into the internal space S. In this case, at least one of the robot 2 and the end effector 50 is controlled in accordance with the actual working conditions of shredding the leaves 210, so that at least one of the robot 2 and the end effector 50 can be operated efficiently. Therefore, this is even more useful for improving work efficiency.

[0116] In the leaf harvesting device 1 described above, the controller 100 may control the end effector 50 and the robot 2 to perform a first operation of harvesting leaves 210A (first leaves) from the crop 200, and a second operation of harvesting leaves 210B (second leaves) from the crop 200 after the first operation has been performed. Alternatively, based on the changes in the state value obtained during the execution of the first operation, the controller 100 may control the robot 2 to terminate the first operation and proceed to the second operation when it detects that the shredding of leaves 210A by the impeller 62 is complete. If the completion of shredding is not detected, for example, the first operation must be continued for a set time with a certain margin before being terminated. In contrast, the above configuration can reduce the waiting time after the actual completion of the first operation, and the first operation can be shortened. Therefore, it is even more useful for improving work efficiency.

[0117] In the leaf harvesting device 1 described above, the controller 100 may control the drive motor 72 to reduce the rotational speed of the impeller 62 when it detects the completion of shredding of the leaf 210A in the first operation based on the change in the above state value. In this case, when transitioning from the first operation to the second operation, the possibility that the end effector 50 may suck up parts of the crop 200 other than the target of harvesting (for example, another leaf 210 surrounding leaf 210B, or another leaf 210 that the operating end effector 50 approaches and comes into contact with) can be reduced. Therefore, this is useful for protecting the crop 200 while the harvesting operation by the leaf harvesting device 1 is being performed.

[0118] The leaf collection device 1 described above may further include a controller 100 that controls the end effector 50 and the robot 2. The controller 100 may perform a first control that moves the end effector 50 to a work start position where the opening 54a faces the surface 212a of the leaf blade 212 contained in the leaf 210, and a second control that moves the end effector 50 toward the leaf 210 from the work start position. It has been found that the success rate of collecting the leaf 210 is higher when the end effector 50 approaches the leaf 210 from a position facing the surface 212a rather than approaching the leaf 210 from a direction along the surface 212a of the leaf blade 212. In the above configuration, the end effector 50 is controlled to approach the leaf 210 from a position where the opening 54a faces the surface 212a. Therefore, it is even more useful for improving the efficiency of the work.

[0119] In the leaf collection device 1 described above, the controller 100 may control the robot 2 to move the end effector 50 from the work start position toward the connection point CP between the leaf blade 212 and the petiole 214 on the leaf 210 when executing the second control. It has been found that the success rate of collecting the leaf 210 is higher when the end effector 50 is brought closer toward the connection point CP and the leaf 210 is collected, compared to when the end effector 50 is brought closer toward other points. In the above configuration, since the end effector 50 moves toward the connection point CP in the second control, the possibility of problems occurring where the leaf 210 is not collected is reduced. Therefore, it is even more useful for improving work efficiency.

[0120] In the leaf collection device 1 described above, when the controller 100 performs the first control, with the end effector 50 positioned at the start of the work, the robot 2 may be controlled so that, when viewed from a direction perpendicular to the opening 54a, the connection portion CP between the leaf blade 212 and the petiole 214 of the leaf 210 overlaps with the opening 54a. It has been found that approaching the end effector 50 from a position where the connection portion CP overlaps with the opening 54a and then performing suction results in a higher success rate of leaf 210 suction compared to when the connection portion CP does not overlap with the opening 54a. Therefore, by performing the second control as described above, the possibility of problems occurring where the leaf 210 is not suctioned is reduced. Consequently, this is even more useful for improving work efficiency.

[0121] In the leaf harvesting device 1 described above, the cutting member 80 may include a blade 84 extending along a direction intersecting a virtual line perpendicular to the opening 54a. After performing the second control, the controller 100 may further perform a third control to move the end effector 50 along the direction intersecting the virtual line. In this case, the cutting member 80 can more reliably cut the leaves 210. Therefore, the possibility of problems occurring where the leaves 210 cannot be cut is reduced. Thus, it is even more useful for improving work efficiency.

[0122] In the leaf harvesting device 1 described above, the robot 2 may be an articulated robot capable of changing the position and orientation of the end effector 50 relative to the crop 200. In this case, the end effector 50 can be brought close to multiple leaves 210, each with a leaf blade surface 212a facing in various directions, from a direction suitable for suction and cutting. This is therefore even more useful for improving work efficiency.

[0123] [Note] Furthermore, this disclosure includes the configurations described in the following appendices 1 to 6. <Note 1> An end effector that sucks the leaves of a crop into the internal space through an opening and collects the leaves from the crop, A robot that changes the relative position of the end effector with respect to the crop, The system comprises the end effector and a controller for controlling the robot, The end effector comprises a rotatable impeller and a drive motor that drives the impeller to generate a suction force for sucking up the leaves of the crop. The aforementioned controller, The change in the state value representing the operating state of the drive motor is monitored, A leaf harvesting device that detects the state of work being performed on the leaves of the crop at the end effector based on the fluctuation of the aforementioned state value. <Note 2> The impeller is positioned within the space so as to rotate around a predetermined axis that intersects the opening. The harvesting device according to Appendix 1, wherein the impeller is formed to be capable of shredding the leaves of the crop taken into the space. <Note 3> The sampling device according to Appendix 1 or 2, wherein the controller detects, based on the change in the state value, the start and completion of shredding of the leaves of the crop taken into the space by the impeller. <Note 4> The harvesting device according to Appendix 3, wherein the controller controls the operation of either or both of the robot and the end effector based on the detection result of at least one of the start and completion of shredding by the impeller on the leaves of the crop taken into the space. <Note 5> The aforementioned controller, The end effector and the robot are controlled to perform a first operation of harvesting a first leaf from the crop, and a second operation of harvesting a second leaf from the crop after the first operation has been performed. The harvesting device according to Appendix 4, which controls the robot to terminate the first operation and proceed to the second operation when it detects the completion of shredding of the first leaf by the impeller based on the change in the state value obtained during the execution of the first operation. <Note 6> The sampling device according to Appendix 5, wherein the controller controls the drive motor to reduce the rotational speed of the impeller when it detects the completion of shredding of the first leaf in the first operation based on the change in the state value. [Explanation of symbols]

[0124] 1...Leaf harvesting device, 2...Robot, 6...Traveling cart, 8...First measuring unit, 9...Second measuring unit, 50...End effector, 52...Case member, 53...Peripheral wall, S...Internal space, 60...Suction unit, 62...Impeller, Ax1...Axis, 70...Drive unit, 72...Drive motor, 76...Transmission member, 80...Cutting member, Ax2...Axis, 84...Blade, 88...Protective member, 100...Controller, 200...Crop, 210...Leaf, 212...Leaf blade, 212a...Surface, 214...Petiole, CP...Connection part.

Claims

1. An end effector for harvesting leaves from crops, A robot that changes the relative position of the end effector with respect to the crop, The system comprises at least a controller for controlling the end effector, The aforementioned end effector is, A case member having an opening at its tip to form a space for taking in the leaves of the crop, A suction unit comprising an impeller arranged in the space and rotatable about a predetermined axis intersecting the opening, and a drive motor that drives the impeller to generate a suction force, the suction unit that generates the suction force to draw in the leaves of the crop from the opening into the space, It has a cutting member capable of cutting the leaves of the crop that have been taken into the space, The impeller is formed to be able to shred the leaves of the crop that have been taken into the space, The cutting member is provided between the opening and the impeller in the direction in which the axis extends. The aforementioned controller, The change in the state value representing the operating state of the drive motor is monitored, A leaf harvesting device that detects the state of work being performed on the leaves of the crop at the end effector based on the fluctuation of the aforementioned state value.

2. The leaf harvesting apparatus according to claim 1, further comprising a trolley capable of moving the robot to which the end effector is attached.

3. The suction unit has a drive unit that includes the drive motor and a transmission member that transmits the driving force from the drive motor to both the impeller and the cutting member, The cutting member is provided on the peripheral wall surrounding the second axis of the case member so as to be rotatable about a predetermined second axis that intersects the opening, The leaf harvesting device according to claim 1, wherein the drive unit drives the cutting member to rotate around the second axis.

4. The suction unit has a drive unit including the drive motor, The cutting member is provided on the peripheral wall surrounding the second axis of the case member so as to be rotatable about a predetermined second axis that intersects the opening, The aforementioned drive unit is The cutting member is driven to rotate around the second axis, The leaf harvesting device according to claim 1, wherein the impeller and the cutting member are rotated in opposite directions to each other.

5. The leaf harvesting device according to any one of claims 1 to 4, wherein the cutting member is provided on the peripheral wall surrounding the second axis of the case member so as to be rotatable about a predetermined second axis that intersects the opening.

6. The end effector further includes a protective member provided on the peripheral wall so as to cover the cutting member when the opening is viewed from outside the space, The leaf harvesting device according to claim 5, wherein the protective member is formed to be expandable and contractible in a direction intersecting the second axis.

7. The leaf harvesting device according to any one of claims 1 to 4, wherein the controller detects, based on the change in the state value, the start and completion of shredding of the crop leaves taken into the space by the impeller.

8. The leaf harvesting apparatus according to claim 7, wherein the controller controls the operation of either or both of the robot and the end effector based on the detection result of at least one of the start and completion of shredding of the crop leaves taken into the space by the impeller.

9. The aforementioned controller, The end effector and the robot are controlled to perform a first operation of harvesting a first leaf from the crop, and a second operation of harvesting a second leaf from the crop after the first operation has been performed. The leaf harvesting device according to claim 8, wherein the robot is controlled to terminate the first operation and proceed to the second operation when it detects the completion of shredding of the first leaf by the impeller based on the change in the state value obtained during the execution of the first operation.

10. The leaf harvesting apparatus according to claim 9, wherein the controller controls the drive motor to reduce the rotational speed of the impeller when it detects the completion of shredding of the first leaf in the first operation based on the change in the state value.

11. The controller further controls the robot, The aforementioned controller, A first control moves the end effector to a starting position where the opening faces the surface of the leaf blade contained in the leaf of the crop, A leaf harvesting device according to any one of claims 1 to 4, comprising: performing a second control to move the end effector from the work start position toward the leaves of the crop.

12. The leaf harvesting apparatus according to claim 11, wherein the controller controls the robot to move the end effector from the work start position toward the connection portion between the leaf blade and the petiole on the leaf of the crop when performing the second control.

13. The leaf harvesting apparatus according to claim 11, wherein the controller controls the robot so that, when the end effector is positioned at the work start position, the connection between the leaf blade and petiole of the crop leaf overlaps with the opening when viewed from a direction perpendicular to the opening.

14. The cutting member includes a blade extending along a direction intersecting a virtual line perpendicular to the opening, The leaf harvesting apparatus according to claim 11, wherein the controller further performs a third control, after performing the second control, to move the end effector along a direction intersecting the virtual line.

15. The leaf harvesting device according to claim 11, wherein the robot is a multi-joint robot capable of changing the position and orientation of the end effector relative to the crop.

16. The imaging unit further comprises an imaging unit configured to capture a field of view including the leaves of the crop, The leaf harvesting apparatus according to claim 11, wherein the controller performs the first control and the second control based on the captured image obtained by the imaging unit.

17. An end effector for harvesting leaves from a crop, A robot that changes the relative position of the end effector with respect to the crop, The system comprises at least a controller for controlling the end effector, The aforementioned end effector is, A case member having an opening at its tip to form a space for taking in the leaves of the crop, A suction unit that generates suction force to draw the leaves of the crop into the space through the opening, It has a cutting member capable of cutting the leaves of the crop that have been taken into the space, The suction unit includes a rotatable impeller and a drive motor that drives the impeller to generate the suction force. The aforementioned controller, The change in the state value representing the operating state of the drive motor is monitored, A leaf harvesting device that detects the state of work being performed on the leaves of the crop at the end effector based on the fluctuation of the aforementioned state value.

18. A method for controlling a leaf harvesting device comprising an end effector for harvesting leaves from a crop, and a robot for changing the relative position of the end effector with respect to the crop, The aforementioned end effector is, A case member having an opening at its tip to form a space for taking in the leaves of the crop, A suction unit comprising: an impeller disposed within the space and rotatable about a predetermined axis intersecting the opening, the impeller being formed to shred the leaves of the crop taken into the space; and a drive motor that drives the impeller to generate a suction force, the suction unit that generates the suction force to draw the leaves of the crop into the space from the opening; The space includes a cutting member capable of cutting the leaves of the crop taken into the space, and provided between the opening and the impeller in the direction in which the axis extends. A first control that moves the end effector to a work start position set according to a predetermined setting method using the robot, A second control that moves the end effector by the robot from the aforementioned work start position toward the leaves of the crop, After the execution of the second control, a third control is performed to control either or both of the end effector and the robot so that the leaves are cut from the crop by the cutting member. The change in the state value representing the operating state of the drive motor is monitored, A method for controlling a leaf harvesting device, comprising detecting the state of work being performed on the leaves of the crop at the end effector based on the fluctuation of the state value.

19. A method for controlling a leaf harvesting device comprising an end effector for harvesting leaves from a crop, and a robot for changing the relative position of the end effector with respect to the crop, The aforementioned end effector is, A case member having an opening at its tip to form a space for taking in the leaves of the crop, A suction unit that generates suction force to draw the leaves of the crop into the space through the opening, It has a cutting member capable of cutting the leaves of the crop that have been taken into the space, The suction unit includes a rotatable impeller and a drive motor that drives the impeller to generate the suction force. A first control that moves the end effector to a work start position set according to a predetermined setting method using the robot, A second control that moves the end effector by the robot from the aforementioned work start position toward the leaves of the crop, After the execution of the second control, a third control is performed to control either or both of the end effector and the robot so that the leaves are cut from the crop by the cutting member. The change in the state value representing the operating state of the drive motor is monitored, A method for controlling a leaf harvesting device, comprising detecting the state of work being performed on the leaves of the crop at the end effector based on the fluctuation of the state value.

Images