Harvesting device, harvesting method, and program

Abstract

To provide a harvesting device and a harvesting method, enabling harvesting objects to be efficiently and stably harvested, and a program.SOLUTION: A harvesting device includes: a harvesting unit harvesting a harvesting object; a harvesting unit moving unit moving the harvesting unit to a position appropriate to harvest the harvesting object; a main body in which the harvesting unit and the harvesting unit moving unit are mounted; a main body moving unit moving a main body; and a control unit. The control unit performs: a first step of determining whether or not interference between the harvesting unit and an obstacle occurs at the appropriate position; a second step of determining whether or not interference between the harvesting unit moving unit and the obstacle occurs in a route for moving the harvesting unit to the appropriate position when determining that the interference does not occur in the first step; and a third step of controlling the harvesting unit so that the harvesting unit performs harvesting when determining that interference does not occur in the second step.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a harvesting device, a harvesting method, and a program for harvesting harvest objects such as fruits. [Background technology]

[0002] There is a demand for automation of agricultural harvesting work.

[0003] One known example of a device that automatically harvests harvest targets is the harvesting device described in Patent Document 1. The harvesting device disclosed in Patent Document 1 includes a vacuum pad that uses a vacuum suction device to suck up fruit as the harvest target, and a motor that rotates and vibrates the vacuum pad. The harvesting device disclosed in Patent Document 1 harvests fruit growing on branches by rotating and vibrating the vacuum pad while the fruit is sucked up by the vacuum pad.

[0004] In the harvesting device disclosed in Patent Document 1, a portion of the surface of the fruit is vacuum-sucked, which can leave suction marks on the fruit and damage the fruit.

[0005] An example of a harvesting device that reduces damage to harvested crops is the harvesting device disclosed in Patent Document 2. Patent Document 3 also discloses a harvesting method that uses a bendable harvesting arm to avoid obstacles such as main stems and bunches. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 63-141517 [Patent Document 2] Japanese Patent Application Laid-Open No. 2017-51103 [Patent Document 3] Japanese Patent Application Publication No. 8-238014 Summary of the Invention [Problem to be solved by the invention]

[0007] There is a demand for a harvesting method that can more efficiently and stably harvest harvest objects while avoiding obstacles.

[0008] An object of the present disclosure is to provide a harvesting device, a harvesting method, and a program that can efficiently and stably harvest harvest objects while avoiding obstacles. [Means for solving the problem]

[0009] A harvesting device according to one aspect of the present invention includes a harvesting unit that harvests harvest objects, a harvesting unit moving unit that moves the harvesting unit to an appropriate position for harvesting the harvest objects, a main body in which the harvesting unit and the harvesting unit moving unit are provided, a main body moving unit that moves the main body, and a control unit, wherein the control unit: The harvesting unit to the appropriate position On the route when moving The method includes a first step of determining whether interference will occur between the harvesting unit and an obstacle, a second step of determining whether interference will occur between the harvesting unit moving unit and the obstacle on the path when moving the harvesting unit to the appropriate position if it is determined in the first step that no interference will occur, and a third step of controlling the harvesting unit to perform harvesting if it is determined in the second step that no interference will occur.

[0010] A harvesting method according to one aspect of the present disclosure is a harvesting method executed by a computer that controls a harvesting device, the method including: a harvesting unit that harvests an object to be harvested; of , in a proper position for harvesting the harvest object. Move Time On the route The method includes a first step of determining whether interference will occur between the harvesting unit and an obstacle; a second step of determining, if it is determined in the first step that no interference will occur, whether interference will occur between the obstacle and a harvesting unit moving unit that moves the harvesting unit to the appropriate position on a route when moving the harvesting unit to the appropriate position; and a third step of controlling the harvesting unit to perform harvesting, if it is determined in the second step that no interference will occur.

[0011] A program according to one aspect of the present disclosure is a program executed by a computer that controls a harvesting device, the program including a harvesting unit that harvests harvest objects. of , in a proper position for harvesting the harvest object. Move Time On the route The computer is caused to execute a first step of determining whether interference will occur between the harvesting unit and an obstacle; a second step of determining, if it is determined in the first step that interference will not occur, whether interference will occur between the obstacle and a harvesting unit moving unit that moves the harvesting unit to the appropriate position on a route when moving the harvesting unit to the appropriate position; and a third step of controlling the harvesting unit to perform harvesting, if it is determined in the second step that interference will not occur. [Effects of the Invention]

[0012] According to the present invention, harvest objects can be harvested efficiently and stably while avoiding obstacles. [Brief explanation of the drawings]

[0013] [Figure 1] A diagram showing the structure of a bunch of cherry tomatoes as an example of a harvest target. [Figure 2] FIG. 1 is a diagram for explaining the configuration of a harvesting device according to an embodiment of the present disclosure. [Figure 3] Schematic top view of the working arm and end effector [Figure 4] Flowchart for explaining an example of the operation of the harvesting device [Figure 5A] A diagram explaining how to create an obstacle map [Figure 5B] A diagram explaining how to create an obstacle map [Figure 6] A diagram to explain the concept of how to determine the harvest direction [Figure 7A] A diagram to explain the concept of how to determine the harvest direction [Figure 7B] A diagram to explain the concept of how to determine the harvest direction [Figure 8A] FIG. 1 illustrates an end effector map including end effector regions showing the locations of end effectors. [Figure 8B] FIG. 10 shows a first interference map obtained by overlaying an obstacle map and an end effector map. [Figure 9] Diagram to explain the harvesting posture map [Figure 10] FIG. 1 is a diagram illustrating an interference map. [Figure 11A] FIG. 1 is a diagram for explaining a first region and a second region of a working arm; [Figure 11B] FIG. 10 is a diagram showing a first work arm region corresponding to a first region of the work arm and a second work arm region corresponding to a second region in a harvesting posture map. [Figure 12] A diagram showing the harvesting posture map and the second interference map superimposed on each other. [Figure 13A] FIG. 10 is a diagram for explaining a method for determining the movement amount of the harvesting device. [Figure 13B] FIG. 10 is a diagram for explaining a method for determining the movement amount of the harvesting device. [Figure 14] A diagram to explain how to determine the new harvesting direction after moving the main body. [Figure 15] Perspective view of the end effector [Figure 16A] Diagram to explain the operation of the end effector during harvesting [Figure 16B] Diagram to explain the operation of the end effector during harvesting [Figure 16C] Diagram to explain the operation of the end effector during harvesting DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, a harvesting device according to an embodiment of the present disclosure will be described in detail with reference to the drawings.

[0015] <Explanation about the harvest target> First, the harvest object of the harvesting device will be described. In the following description, cherry tomatoes, which grow in clusters in a dense state, will be used as an example of the harvest object.

[0016] Figure 1 shows the structure of a cherry tomato cluster. As shown in Figure 1, one cluster 406 is composed of a stem called a peduncle 405. Pedicels 401 and calyx 403 grow from this peduncle 405, and fruit 404 grows at the end of them.

[0017] In the middle of the pedicel 401, there is a portion called an abscission layer 402. When force is applied to the pedicel 401, the pedicel 401 is divided at the abscission layer 402.

[0018] The harvesting device of the present disclosure targets fruits with such an abscission layer. Therefore, the harvesting device of the present disclosure can target not only cherry tomatoes but also other fruits with an abscission layer. Furthermore, the harvesting device of the present disclosure can target not only fruits that grow in clusters but also fruits that grow individually.

[0019] <Configuration of harvesting device 1> Fig. 2 is a diagram for explaining the configuration of a harvesting device 1 according to an embodiment of the present disclosure. The harvesting device 1 includes a main body moving unit 10, a work arm 20, an end effector 200, a main body 30, an imaging device 40, and a control unit 60. Note that Fig. 2 is a side view of the harvesting device 1, and the up-down direction in the following description corresponds to the up-down direction of the harvesting device 1 shown in Fig. 2.

[0020] The main body moving unit 10 is configured to move the entire main body 30 of the harvesting device 1, for example, by wheels or caterpillars. The main body 30 is provided with the other components of the harvesting device 1, namely, the work arm 20, the end effector 200, the imaging device 40, and the control unit 60, so that movement of the main body 30 is equivalent to movement of the entire harvesting device 1.

[0021] In this embodiment, the main body moving unit 10 is described as being capable of moving the main body 30 only in one predetermined direction and its opposite direction, such as moving the main body 30 on rails laid between ridges. However, the present disclosure is not limited to this, and the main body moving unit 10 may be capable of moving the main body 30 in any direction. In the following description, the direction in which the main body moving unit 10 moves the main body 30 is referred to as the "moving direction." The moving direction is used when creating the planar map 500, which will be described later. Therefore, if the main body 30 can move in any direction, the moving direction of the main body moving unit 10 when creating the planar map 500 can be defined by, for example, referring to the direction in which the harvest target objects are located.

[0022] In this embodiment, the main body moving unit 10 can move the main body 30 in the front-to-rear direction. The front-to-rear direction of the harvesting device 1 is the direction in which the harvesting device 1 moves when the harvesting target is on the left or right side as viewed from the harvesting device 1. The right side of the paper in FIG. 2 corresponds to the front direction of the harvesting device 1, and the left side of the paper in FIG. 2 corresponds to the rear direction of the harvesting device 1.

[0023] The end effector 200 is a part that harvests the object to be harvested without damaging it. The end effector 200 is an example of a harvesting unit of the present disclosure. The detailed structure and operation of the end effector 200 will be described later.

[0024] The working arm 20 is a part for moving the end effector 200 to a position relative to the harvesting target that makes it easier to harvest the target. The working arm 20 is an example of a harvesting unit movement unit of the present disclosure. One end of the working arm 20 is attached to the main body 30, and the other end is attached to the end effector 200. The working arm 20 moves the end effector 200 to an appropriate position for harvesting the target, which is located away from the harvesting device 1, thereby allowing the end effector 200 to harvest the target.

[0025] Figure 3 is a schematic top view of the working arm 20 and the end effector 200. Figure 3 shows the working arm 20, which has a three-axis horizontal articulated arm, as an example of a working arm. The working arm 20 has a first link 21 and a second link 22. The end effector 200 is mounted on the tip of the second link 22. The end effector 200 is rotatable around the tip of the second link 22 (point Q in Figure 3) as the axis of rotation.

[0026] 3, the working arm 20 and the end effector 200 are shown extending to the left of the harvesting device 1. In the following description of this embodiment, the working arm 20 and the end effector 200 are also assumed to extend to the left of the harvesting device 1. However, the present disclosure is not limited to this, and the working arm 20 and the end effector 200 may extend in any direction from the main body 30.

[0027] The main body 30 is the main body of the harvesting device 1, and is equipped with the main body moving unit 10, the working arm 20, the imaging device 40, and the control unit 60.

[0028] The imaging device 40 is a camera device that generates an image by capturing an image of at least one of a harvesting target, an obstacle, the end effector 200, and the work arm 20. In particular, the imaging device 40 is a camera that can acquire RGB (Red, Green, Blue) color images, IR (infrared) images, and distance data.

[0029] The control unit 60 is a computer that controls each component of the harvesting device 1. The control unit 60 is a processor configured, for example, by a CPU (Central Processing Unit), ROM (Read Only Memory), and RAM (Random Access Memory). The control unit 60 reads programs stored in the ROM, expands them in the RAM, and controls each component of the harvesting device 1 according to the expanded programs. The RAM forms a work area that temporarily stores various programs executed by the CPU and data related to the programs. The ROM is configured by non-volatile memory or the like, and stores various programs and data used during control. The computer that controls the harvesting device 1 may be located outside the harvesting device 1, and the harvesting device 1 may be remotely operated via a communication network or the like.

[0030] More specifically, the control unit 60 analyzes the captured images generated by the imaging device 40, and controls the operations of the main body movement unit 10, the working arm 20, and the end effector 200 based on the analysis results.

[0031] <Example of harvesting device 1 operation> Next, an example of the operation of the harvesting device 1 when harvesting harvest targets will be described with reference to Fig. 4. Fig. 4 is a flowchart for explaining an example of the operation of the harvesting device 1. Note that the example of the operation of the harvesting device 1 shown in Fig. 4 will be described assuming that one of a plurality of fruits is determined as the harvest target, and the harvesting device 1 is positioned in a position substantially directly opposite the harvest target.

[0032] First, in step S1, the control unit 60 uses the imaging device 40 to capture images of the target fruit, the bunch containing the fruit, the stem, the entire plant, etc., and obtains the images, color information, distance information, etc. of each subject.

[0033] Next, in step S2, the control unit 60 obtains harvest target object position information indicating the position of the harvest target object and obstacle position information indicating the position of any obstacles, based on the image and distance information generated by the imaging device 40. An obstacle is an object that could become an obstacle when harvesting the harvest target object, and in this embodiment, is a main stem or a leaf.

[0034] Known methods can be used to recognize harvest objects or obstacles contained in an image. For example, the technology described in JP 2018-206015 A can be used as a method for recognizing fruit as a harvest object based on an image. Furthermore, for example, the technology described in JP 2020-000170 A can be used as a method for recognizing a main stem as an obstacle based on an image.

[0035] Next, in step S3, the control unit 60 creates an obstacle map based on the harvest object position information and the obstacle position information. A method for creating the obstacle map will be described in detail below.

[0036] [How to create an obstacle map] 5A and 5B are diagrams for explaining a method for creating an obstacle map. Note that the following describes a method for creating a two-dimensional obstacle map.

[0037] First, the control unit 60 creates a grid-like planar map 500 as shown in Fig. 5A based on the harvest target position information. In the example shown in Fig. 5A, the planar map 500 is a map consisting of two axes (first axis 501, second axis 502) along each of two directions (first direction D1, second direction D2). The planar map 500 is created to include the harvest target position information.

[0038] The first direction D1 corresponds to the direction in which the harvesting device 1 can move (forward and backward), and the first axis 501 is an axis along the first direction D1. The second direction D2 corresponds to the direction of the harvesting object as seen from the harvesting device 1, and the second axis 502 is an axis perpendicular to the first direction D1. That is, FIG. 5A shows the planar map 500 as viewed obliquely from above. The distance between axes in the same direction, i.e., the grid size, can be set to an appropriate size, such as 1 mm.

[0039] Next, the control unit 60 assigns a value of "1" to each grid of the planar map 500 where an obstacle exists and a value of "0" to each grid where no obstacle exists, based on the obstacle position information. As a result, an obstacle map 504 is created, as shown in FIG. 5B. Similarly to FIG. 5A, FIG. 5B also shows the planar map 500 as viewed from diagonally above. In FIG. 5B, grids where obstacles exist, i.e., grids with a value of "1," are indicated by diagonal lines. Since obstacles generally have a certain size, grids where obstacles exist are adjacent to each other. The example shown in FIG. 5B shows an obstacle map including an obstacle region 503 as a group of multiple grids indicating the locations of obstacles.

[0040] The above explanation describes how to create a two-dimensional obstacle map, but if a three-dimensional obstacle map is required, the two-dimensional obstacle maps created using the above method can be stacked along a third axis parallel to the height direction to create a three-dimensional obstacle map.

[0041] Next, in steps S4 to S6 and steps S16 to S18, the control unit 60 determines a harvesting direction, which is the direction in which the end effector 200 will be moved when harvesting the harvest object. A method for determining the harvesting direction will be described with reference to Figures 6, 7A, and 7B.

[0042] [How to determine harvest direction] 6, 7A, and 7B are diagrams for explaining the concept of the method for determining the harvesting direction. Figures 6, 7A, and 7B show a schematic diagram of the positional relationship between the fruit 404 as the harvest target and the end effector 200 when viewed from above.

[0043] When harvesting fruit 404 using end effector 200, it is necessary to move end effector 200 to an appropriate position for harvesting. The appropriate position for harvesting is a position where fruit 404 is placed inside a ring-shaped member (harvesting ring 202 in FIG. 15, which will be described later) of end effector 200, as shown in FIG. 6. By moving end effector 200 to such an appropriate position, end effector 200 can easily pluck fruit 404 at abscission layer 402 (see FIG. 1).

[0044] To minimize damage to the harvested object when moving the end effector 200 to the appropriate position shown in Fig. 6, it is ideal to move the end effector 200 in a direction from the fruit 404 toward the stalk 405 along a straight line L1 that passes through the stalk 405 and the center of the fruit 404, as shown in Fig. 7A. In the following description, the movement direction of the end effector 200 shown in Fig. 7A is referred to as the ideal harvesting direction Di. The ideal harvesting direction Di is an example of the third direction of the present disclosure.

[0045] However, in actual harvesting work, if there are obstacles such as stems or leaves near the fruit 404, it is not always possible to move the end effector 200 along the ideal harvesting direction Di shown in FIG. 7A.

[0046] Therefore, in step S4, the control unit 60 assumes that the harvesting direction of the end effector 200 is the ideal harvesting direction. Then, in step S5, the control unit 60 simulates whether or not the end effector 200 will interfere with an obstacle in the assumed harvesting direction and in a position (appropriate position) where the end effector 200 can harvest the harvest target. Specifically, whether or not the end effector 200 will interfere with an obstacle in the appropriate position is determined by simulating whether or not an obstacle exists within the end effector area indicated by the dashed line in FIG. 6. The end effector area is an area of ​​a predetermined size centered on the harvest target when the end effector 200 is in the appropriate position, and the size and center position of the area can be set as appropriate.

[0047] Then, when the control unit 60 determines from the simulation result that the end effector 200 will not interfere with the obstacle (step S5: NO), in step S6, it determines the assumed harvesting direction as the actual harvesting direction.

[0048] On the other hand, when the control unit 60 determines that the end effector 200 will interfere with an obstacle (step S5: YES), it re-assumes a harvesting direction by slightly shifting the harvesting direction from the ideal harvesting direction in steps S16 and S17. Specifically, in step S16, the control unit 60 determines whether the magnitude of the deviation between the currently assumed harvesting direction and the ideal harvesting direction is equal to or less than the threshold angle φ, and if it determines that the magnitude of the deviation is equal to or less than the threshold angle φ (step S16: YES), it proceeds to step S17. In step S17, the control unit 60 assumes a new harvesting direction that is slightly shifted from the currently assumed harvesting direction, and returns the process to step S5. Then, in step S5, the control unit 60 again simulates whether or not the end effector 200 will interfere with an obstacle if the end effector 200 is moved in the shifted harvesting direction. By repeating the processes of steps S4 and S5 in this manner, the control unit 60 can determine the harvesting direction of the end effector 200 that does not interfere with obstacles.

[0049] Note that the greater the deviation of the harvesting direction from the ideal harvesting direction, the greater the likelihood that harvesting by the end effector 200 will fail. For this reason, in this embodiment, the threshold angle φ is set so that the deviation from the ideal harvesting direction is equal to or less than a predetermined value. Specifically, as shown in FIG. 7B , the harvesting direction is determined so that the angle with the ideal harvesting direction (the direction along the line L1 passing through the stalk 405 and the center of the fruit 404) is within the threshold angle φ. The threshold angle φ is set to, for example, 30°. Furthermore, when the harvesting direction is shifted in step S17, it is set to be shifted by, for example, 10° each time. The threshold angle φ and the magnitude of the angle shift per shift can be set as appropriate.

[0050] In this way, if moving the end effector 200 in the ideal harvesting direction would result in interference between the end effector 200 and an obstacle, the movement direction of the end effector 200, i.e., the harvesting direction, is set to a direction deviated from the ideal harvesting direction within the threshold angle, as shown in FIG. 7B. This allows the end effector 200 to be moved to an appropriate position while avoiding the obstacle. This enables stable harvesting. The harvesting direction after the shift, which does not cause interference with the obstacle, is an example of the fourth direction of the present disclosure.

[0051] If a harvesting direction that does not cause interference between the end effector 200 and an obstacle cannot be found within the threshold angle φ (step S16: NO), the control unit 60 stops the harvesting operation for the current harvest target in step S18. In this case, for example, the control unit 60 may set a new harvest target, move the harvesting device 1 to a position directly opposite the new harvest target, and restart a new harvesting operation for the new harvest target from step S1.

[0052] In step S5, the control unit 60 simulates whether the end effector 200 will interfere with an obstacle in the assumed harvesting direction and appropriate position by comparing the obstacle map created in step S3 with a newly created end effector map indicating the position of the end effector 200. The end effector map is a map created by simulating the position information of the end effector 200 while moving the end effector 200 from its current position to the appropriate position in the assumed harvesting direction, and is a map indicating the end effector area shown in FIG. 6. The control unit 60 simulates the position information of the end effector 200 when the end effector 200 is in the appropriate position based on the harvesting object position information included in the planar map 500, and simulates the position of the end effector 200 during movement based on the harvesting direction.

[0053] The control unit 60 creates a first interference map by overlaying the obstacle map and the end effector map. The first interference map is a map that indicates whether an obstacle will interfere with the end effector 200 in the assumed harvesting direction and appropriate position. If the first interference map contains a grid where a grid indicating the position of an obstacle overlaps with a grid where a grid indicating the position of the end effector (end effector region), the control unit 60 determines that the obstacle and the end effector 200 will interfere with each other when the end effector 200 is moved in the assumed harvesting direction. On the other hand, if the first interference map contains no grid where a grid indicating the position of an obstacle overlaps with a grid where a grid indicating the position of the end effector (end effector region), the control unit 60 determines that the obstacle and the end effector 200 will not interfere with each other in the assumed harvesting direction and appropriate position.

[0054] The control unit 60 may determine that the obstacle and the end effector 200 will not interfere with each other if the number of overlapping grids in the first interference map between grids indicating the position of the obstacle and grids indicating the position of the end effector 200 is equal to or less than a predetermined threshold. This prevents a situation in which an incorrect recognition that may occur when recognizing an obstacle or the end effector 200 included in an image results in an incorrect determination that interference will occur when in fact there is no interference. This also allows for problem-free harvesting when the degree of interference is small and the likelihood of the interference adversely affecting the harvest is low. The threshold can be set to any value, such as 3, or a wider range of values, such as 1 to 10.

[0055] Fig. 8A is a diagram showing an end effector map 506 including an end effector region 505 indicating the position of the end effector 200. In Fig. 8A, the end effector region 505 corresponding to the position of the end effector 200 is shown shaded.

[0056] 8B is a diagram showing a first interference map 507 obtained by overlaying the obstacle map 504 and the end effector map 506. In the first interference map 507 shown in FIG. 8B, an obstacle region 503 indicating the position of an obstacle and an end effector region 505 indicating the position of the end effector 200 overlap in region R. In FIG. 8B, the overlapping region is indicated by black dots. In this case, the control unit 60 determines that the end effector 200 will interfere with the obstacle if the end effector 200 is moved to the appropriate position in the assumed harvesting direction.

[0057] Returning to the description of Figure 4, next, in step S7, the control unit 60 calculates the harvesting posture of the working arm 20 for moving the end effector 200 along the harvesting direction determined in step S6.

[0058] [Method for calculating the harvesting posture of the working arm 20] The following describes a method for calculating the harvesting posture of the working arm 20. As shown in FIG.

[0059] The first link 21 and the second link 22 are rod-shaped members. One end of the first link 21 is connected to the main body 30 at point O, and the other end is connected to the second link 22 at point P. One end of the second link 22 is connected to the first link 21 at point P, and the other end is connected to the end effector 200 at point Q.

[0060] Point O is the rotation axis of first link 21, located near one end of first link 21 and at the center in the short direction of first link 21. Point P is located near the other end of first link 21 and at the center in the short direction of first link 21. At the same time, point P is the rotation axis of second link 22, located near one end of second link 22 and at the center in the short direction of second link 22. Point Q is located near the other end of second link 22 and at the center in the short direction of second link 22.

[0061] The first link 21 is rotatable relative to the main body 30, with point O as the fulcrum. The second link 22 is rotatable relative to the first link 21, with point P as the fulcrum. The end effector 200 is rotatable relative to the second link 22, with point Q as the fulcrum. With this configuration, the working arm 20 can take various postures, and the position of the end effector 200 can be moved to various positions relative to the main body 30.

[0062] In this embodiment, the harvesting posture of the working arm 20 refers to the respective positions of the first link 21 and the second link 22 when the end effector 200 is moved to an appropriate position for harvesting. Specific examples of the harvesting posture of the working arm 20 include the respective positions of points O, P, and Q when the end effector 200 is in the appropriate position.

[0063] The control unit 60 determines the harvesting posture of the work arm 20 based on the appropriate position of the end effector 200 as seen from the harvesting device 1, the harvesting direction (movement direction of the end effector 200) determined in step S6, and the link lengths of the first link 21 and the second link 22. The control unit 60 calculates position information of the end effector 200 when it is in the appropriate position based on the harvest object position information held by the planar map 500. The control unit 60 also uses inverse kinematics to calculate the positions of points O, P, and Q when the end effector 200 is in the appropriate position, and sets the calculated positions or angles as the harvesting posture of the work arm 20. Note that instead of the positions of points O, P, and Q, the angle between the line connecting points O and P and the line connecting points P and Q may also be set as the harvesting posture of the work arm 20.

[0064] Returning to the explanation of Figure 4, next, in step S8, the control unit 60 creates a harvesting posture map corresponding to the harvesting posture of the working arm 20 calculated in step S7.

[0065] [How to create a harvesting posture map] The harvesting posture map can be created using a method similar to that used for the obstacle map described above. That is, for the planar map 500 shown in Fig. 5A, a value of "1" is assigned to a grid in which the work arm 20 is located, and a value of "0" is assigned to a grid in which the work arm 20 is not located, based on the harvesting posture of the work arm 20 calculated in step S7 (i.e., the positions of points O, P, and Q of the work arm 20 when the end effector 200 is in the appropriate position). Fig. 9 is a diagram for explaining the harvesting posture map 509. As a result, a harvesting posture map 509 is created that includes a work arm region 508 made up of multiple grids in which the work arm 20 is located (a group of grids having a value of "1"), as shown in Fig. 9.

[0066] Returning to the explanation of Figure 4, next, in step S9, the control unit 60 creates a second interference map for simulating whether or not the work arm 20 will interfere with an obstacle when the end effector 200 is moved to an appropriate position.

[0067] The second interference map is created by overlaying the obstacle map created in step S3 with the harvesting posture map created in step S8, and extracting a grid where an obstacle region 503 indicating the position of an obstacle and a work arm region 508 indicating the position of the work arm 20 overlap. In other words, the second interference map is a map that indicates whether an obstacle will interfere with the work arm 20 when the end effector 200 is moved to the appropriate position. FIG. 10 is a diagram for explaining the second interference map 511. In FIG. 10, a grid (interference region 510) where the grid indicating the position of the obstacle (obstacle region 503) and the grid indicating the position of the work arm 20 (work arm region 508) overlap is shown in black. The presence of the interference region 510 in the created second interference map 511 means that the work arm 20 will interfere with the obstacle when the end effector 200 is moved to the appropriate position.

[0068] The control unit 60 may determine that the obstacle and the work arm 20 will not interfere with each other if the number of overlapping grids in the second interference map between the grid indicating the position of the obstacle and the grid indicating the position of the work arm 20 is equal to or less than a predetermined threshold. This prevents a situation in which an incorrect recognition that may occur when recognizing an obstacle or the work arm 20 included in the image results in an incorrect determination that interference will occur when in fact there is no interference. This also allows for problem-free harvesting when the degree of interference is small and the likelihood of the interference adversely affecting the harvest is low. The threshold can be set to any value, such as 3, or a wider range of values, such as 1 to 10.

[0069] Returning to the explanation of Figure 4, next, in step S10, the control unit 60 determines, based on the second interference map created in step S9, whether or not the work arm 20 will interfere with an obstacle when moving the end effector 200 to an appropriate position. The determination in step S10 is made by detecting the presence or absence of an interference region 510 in the second interference map 511, as described above.

[0070] If it is determined that interference will occur (step S10: YES), the control unit 60 proceeds to step S11. On the other hand, if it is determined that interference will not occur (step S10: NO), the control unit 60 proceeds to step S14.

[0071] If it is determined that interference will occur, in step S11, the control unit 60 determines the direction and amount of movement of the harvesting device 1 to move the entire harvesting device 1 so that the determination result that interference will occur between the work arm 20 and the obstacle does not occur.

[0072] [Method for determining the movement direction of the harvesting device 1] The control unit 60 determines the movement direction of the harvesting device 1 based on the harvesting posture map created in step S8 and the second interference map created in step S9. To determine the movement direction of the harvesting device 1, the control unit 60 first divides the work arm 20 into two regions (a first region and a second region) in the harvesting posture map that indicates the position of the work arm 20.

[0073] 11A is a diagram for explaining the first and second regions of the working arm 20. When the first link 21 constituting the working arm 20 is divided along a line (hereinafter referred to as the first centre line 21C) passing through points O and P, one region is the first region 21R1 and the other region is the second region 21R2. Similarly, when the second link 22 constituting the working arm 20 is divided along a line (hereinafter referred to as the second centre line 22C) passing through points P and Q, one region is the first region 22R1 and the other region is the second region 22R2. In the following description, the first region 21R1 of the first link 21 and the first region 22R1 of the second link 22 may be collectively referred to as the first region 20R1 of the working arm 20, and the second region 21R2 of the first link 21 and the second region 22R2 of the second link 22 may be collectively referred to as the second region 20R2 of the working arm 20.

[0074] FIG. 11B is a diagram showing a work arm first region 508R1 corresponding to the first region 20R1 of the work arm 20, and a work arm second region 508R2 corresponding to the second region 20R2 in the harvesting posture map 509.

[0075] Next, the control unit 60 overlays the harvesting posture map and the second interference map, and determines whether the interference area is contained more in the first work arm area or the second work arm area. Figure 12 is a diagram showing the harvesting posture map 509 overlaid on the second interference map 511. In Figure 12, the first work arm area 508R1 and the second work arm area 508R2 on the harvesting posture map are represented by white borders, and the interference area 510 is shown in black.

[0076] 12, the interference region 510 is included more in the work arm first region 508R1 than in the work arm second region 508R2. In this case, the degree of interference between the work arm 20 and the obstacle is smaller in the work arm second region than in the work arm first region. Therefore, in order to further reduce interference between the work arm 20 and the obstacle, it can be determined that it is best to move the harvesting device 1 in the direction from the work arm first region toward the work arm second region.

[0077] 12, moving the harvesting device 1 in the direction from the first working arm region toward the second working arm region corresponds to moving the harvesting device 1 forward. This allows the control unit 60 to determine the moving direction of the harvesting device 1 to be the forward direction.

[0078] As described above, each map in the present disclosure is created using the first axis corresponding to the direction in which the harvesting device 1 can move (first direction D1), so it is easy to determine which side of the map the area with a low degree of interference is located in the first direction relative to the area with a high degree of interference. Therefore, based on which side of the map the area with a low degree of interference is located in the first direction relative to the area with a high degree of interference, it is easy to determine the direction in which the harvesting device 1 should be moved to reduce interference.

[0079] [Method for determining the movement amount of the harvesting device 1] Next, we will explain a method for determining the amount of movement of the harvesting device 1. Figures 13A and 13B are diagrams for explaining a method for determining the amount of movement of the harvesting device 1.

[0080] First, the control unit 60 measures the angle between a first direction D1, which corresponds to the direction in which the harvesting device 1 can move, and the center line of the link of the work arm 20 in which the interference area exists. FIG. 13A is a diagram showing how the angle θ between the first direction D1 and the second center line 22C of the second link 22 is measured. Similar to FIG. 12, FIG. 13A also shows a map in which the harvesting posture map 509 and the second interference map 511 are superimposed. In the example shown in FIG. 13A, the interference area 510 is included in the area corresponding to the second link 22, so the angle θ between the first direction D1 and the second center line 22C is calculated. However, if the interference area is included in the area corresponding to the first link 21, the angle θ between the first direction D1 and the first center line 21C (see FIG. 11A) can be calculated.

[0081] If the interference area spans both the area corresponding to the first link 21 and the area corresponding to the second link 22, the angle θ can be calculated as the angle between the center line of the link that contains the greater proportion of the interference area and the first direction D1.

[0082] 13B, the control unit 60 forms on the map a circumscribing rectangle 512 that has sides parallel to the second center line 22C and circumscribes the interference region 510. Then, of the sides of the circumscribing rectangle 512, the length t of the side perpendicular to the second center line 22C is measured.

[0083] Then, the control unit 60 calculates the movement amount x of the harvesting device 1 using the following formula (1). x=t / sinθ (1) In this way, the movement amount x is calculated based on the size of the circumscribing rectangle 512.

[0084] 13B in the movement direction determined in FIG. 12, the entire harvesting device 1 moves the length of the circumscribing rectangle 512 along the first direction. Because the circumscribing rectangle 512 is a rectangle that circumscribing the interference region 510, by setting the movement amount of the harvesting device 1 to the movement amount x, the harvesting device 1 moves to a position where the interference region 510 does not occur. In other words, by moving the harvesting device 1 by more than the movement amount x, it is possible to avoid interference that may occur at the position before the movement.

[0085] After calculating the movement amount x of the harvesting device 1, it may be determined whether the movement amount x is equal to or less than a predetermined threshold value. Here, the threshold value is preferably a value smaller than the total link length of the first link 21 and the second link 22. This makes it possible to make a decision to, for example, stop harvesting at that position when the movement amount of the harvesting device 1 becomes large and the end effector 200 cannot be moved to an appropriate position even if the working arm 20 is extended.

[0086] Returning to the description of Fig. 4, in step S12, the control unit 60 controls the main body moving unit 10 to move the entire harvesting device 1 in the movement direction determined in step S11 by the movement amount determined in step S11.

[0087] Next, in step S13, the control unit 60 determines a new harvesting posture after the main body 30 has moved.

[0088] In step S13, the control unit 60 determines a new harvesting posture in the same manner as in step S7. That is, the control unit 60 determines a new harvesting posture of the work arm 20 based on the appropriate position of the end effector 200 as seen from the harvesting device 1 after movement, and the link lengths of the first link 21 and the second link 22. Figure 14 is a diagram for explaining the new harvesting posture. In Figure 14, the work arm 20 before movement is shown by a dotted line, and the work arm 20 after movement is shown by a solid line.

[0089] As shown in Figure 14, by moving point O connected to main body 30 by a distance x, the posture of the entire working arm 20 changes, and the end effector 200 can be moved to the appropriate position without interfering with obstacle OB.

[0090] The end effector 200 may adopt the harvesting direction determined in step S6 as is, since it has been confirmed in step S5 that the end effector 200 will not interfere with any obstacles.

[0091] However, before step S13, the same processes as steps S4 to S6 may be newly performed. By determining a new harvesting direction after moving the main body 30 in this way, the final harvesting direction may be determined to be closer to the ideal harvesting direction. In this case, the harvesting device 1 can efficiently harvest without interfering with obstacles with improved accuracy.

[0092] Next, in step S14, the control unit 60 moves the end effector 200 in accordance with the new harvesting direction determined in step S13. As a result, as shown in Fig. 14, the end effector 200 can be moved to an appropriate position for harvesting while avoiding the obstacle OB. Note that if it is determined in step S10 that there is no interference, the control unit 60 moves the end effector 200 in accordance with the harvesting direction determined in step S6. Even in this case, the end effector 200 can be moved to an appropriate position for harvesting without causing interference.

[0093] In step S15, the control unit 60 controls the end effector 200 to harvest the harvest object.

[0094] [Harvesting method using end effector 200] The method for harvesting the harvest target by the end effector 200 will be described below.

[0095] FIG. 15 is a perspective view of the end effector 200. As shown in FIG. 15, the end effector 200 includes a retracting guide 201, a harvesting ring 202, a harvested fruit guide 203, a harvested fruit sheet 204, and a base 205. The harvesting ring 202 is configured to be separable into two parts, an upper part and an lower part. Under the control of the control unit 60, the upper part of the harvesting ring 202 can move relative to the lower part. In the following description, the vertical direction of the harvesting target object and the end effector 200 will be described with the retracting guide 201 side of the end effector 200 as the top and the base 205 side as the bottom, as shown in FIG. 15.

[0096] In this embodiment, cherry tomatoes are harvested not by harvesting the whole bunch, but by picking each fruit from the bunch. This is because the fruits on the bunch vary in maturity, so it is preferable to harvest only the fully matured fruits. Furthermore, cherry tomatoes are generally eaten raw, so they must be harvested with the calyx intact without damaging the surface of the fruit.

[0097] 16A to 16C are diagrams illustrating the operation of the end effector 200 during harvesting. Fig. 16A shows the operation of pulling in a fruit 404 as a harvesting target. Fig. 16B shows the operation of inserting the harvesting ring 202. Fig. 16C shows the operation of separating and harvesting the fruit 404.

[0098] As shown in Figure 16A, first, pulling guide 201, which is made up of a substantially arc-shaped belt member, is moved toward fruit 404 of the harvest target (arrow A1 in Figure 16A), and fruit 404 is placed inside pulling guide 201 through a slit provided in part of pulling guide 201. In this state, pulling guide 201 is used to pull fruit 404 from below, creating a gap between it and other cherry tomatoes (not shown).

[0099] Next, as shown in Figure 16B, the entire end effector 200 is moved upward (arrow A2 in Figure 16B) to insert the harvesting ring 202 into the gap. At this time, the pedicel 401 slips through the slit in the retracting guide 201, and the fruit 404 leaves the inside of the retracting guide 201.

[0100] Then, as shown in Figure 16C, the upper part of the harvesting ring 202 is moved relative to the lower part (arrow A3 in Figure 16C), pulling in the fruit 404 and pushing out the stalk 405. This stretches the stalk 401 and allows it to be separated at the abscission layer 402. As a result, the stalk 401 is separated at the abscission layer 402, allowing the harvest of fruit 404 that is undamaged (or has little damage).

[0101] Thereafter, the harvested fruits 404 pass through the harvested fruit guide 203 and the harvested fruit sheet 204 and are stored in a harvesting container (not shown) or the like provided below, thereby completing the harvesting operation in step S15.

[0102] When the harvesting of the harvest objects is completed in step S15, the control unit 60 controls the main body moving unit 10 to move the main body 30 to a position directly opposite a new harvest object, and restarts the harvesting operation of the harvesting device 1 from step S1. By repeating this process, multiple harvest objects can be harvested successfully.

[0103] As described above, according to the harvesting device 1 of the embodiment of the present disclosure, when the end effector 200 for automatically harvesting the harvest target is moved to the appropriate position, the position of the work arm 20 or the main body 30 is moved so that the end effector 200 or the work arm 20 does not interfere with obstacles, thereby enabling efficient and stable automatic harvesting.

[0104] <Modification> In the above-described embodiment, the end effector 200 is described as an example of a harvesting unit of the present disclosure, and the working arm 20 is described as an example of a harvesting unit moving unit. However, the harvesting unit of the present disclosure may have a configuration completely different from the end effector 200 shown in FIG. 15 and the like, as long as it has a configuration that allows harvesting at the desquamation layer without damaging the harvested object. Furthermore, the harvesting unit moving unit of the present disclosure may have a configuration completely different from the working arm 20 shown in FIG. 3 and the like, as long as it has a configuration that allows the harvesting unit to be freely moved, such as a wire that can be bent at any position. [Industrial Applicability]

[0105] The harvesting device of the present disclosure is useful for automatically harvesting harvesting objects such as fruit. [Explanation of symbols]

[0106] 1. Harvesting equipment 10 Main body moving part 12 Control Unit 20 Working Arm 20R1 Working arm first region 20R2 Second region of the working arm 21 Link 1 21C 1st center line 21R1 First link, first area 21R2 2nd area of ​​1st link 22 Second Link 22C 2nd center line 22R1 1st area of ​​2nd link 22R2 Second area of ​​second link 30 Main Unit 40 Imaging device 60 Control Unit 200 End Effector 201 Guide 202 Harvest Ring 203 Harvest Guide 204 Harvested Fruit Sheet 205 Base

Claims

1. A harvesting unit that harvests the harvest target; a harvesting unit moving unit that moves the harvesting unit to an appropriate position for harvesting the harvest object; a main body provided with the harvesting unit and the harvesting unit moving unit; a main body moving unit that moves the main body; a control unit, The control unit a first step of determining whether interference between the harvesting unit and an obstacle occurs on a path when the harvesting unit is moved to the appropriate position; a second step of determining whether interference will occur between the harvesting unit moving unit and the obstacle on a path when the harvesting unit is moved to the appropriate position, if it is determined in the first step that interference will not occur; a third step of controlling the harvesting unit to perform harvesting when it is determined in the second step that no interference will occur; To perform the harvesting equipment.

2. When it is determined that interference will occur in the second step, the control unit further performs a fourth step of controlling the main body moving unit to move the main body so as to avoid interference between the harvesting unit moving unit and an obstacle interfering with the harvesting unit moving unit.

10. The harvesting device of claim 1.

3. further comprising an image capturing unit for capturing an image; The control unit further performs a fifth step of identifying a position of an obstacle interfering with the harvesting unit moving unit based on the image, and determining at least one of a moving direction and a moving amount of the main body required for the harvesting unit moving unit to avoid interference with the identified obstacle based on the position of the identified obstacle and the position of the harvesting unit moving unit. Harvesting device according to claim 1 or 2.

4. In the second step, the control unit generates coordinate data of the harvesting unit moving unit on the route and coordinate data of the identified obstacle. Harvesting device according to claim 3.

5. The coordinate data of the harvesting unit moving unit and the coordinate data of the identified obstacle are composed of a plurality of first axes along a first direction in which the main body can move and a plurality of second axes along a second direction which is the direction of the harvest object as seen from the main body.

5. Harvesting device according to claim 4.

6. The coordinate data of the harvesting unit moving unit and the coordinate data of the identified obstacle are composed of the first axis, the second axis, and a plurality of third axes along the height direction of the harvesting device, Harvesting apparatus according to claim 5.

7. In the fifth step, the control unit calculates a degree of overlap between the area indicating the position of the harvesting unit moving unit and the area indicating the position of the identified obstacle when the coordinate data of the harvesting unit moving unit and the coordinate data of the identified obstacle are superimposed, and determines the direction in the first direction from the area with a large degree of overlap to the area with a small degree of overlap as the movement direction of the main body. Harvesting device according to claim 5 or 6.

8. In the fifth step, the control unit measures the size of an overlapping area between the area indicating the position of the harvesting unit moving unit and the area indicating the position of the identified obstacle when the coordinate data of the harvesting unit moving unit and the coordinate data of the identified obstacle are superimposed, and determines the movement amount of the main body based on the size. Harvesting device according to any one of claims 5 to 7.

9. 1. A harvesting method implemented by a computer controlling a harvesting device, comprising: a first step of determining whether interference between the harvesting unit and an obstacle will occur on a path when the harvesting unit that harvests the harvest object is moved to an appropriate position for harvesting the harvest object; a second step of determining whether interference will occur between a harvesting unit moving unit that moves the harvesting unit to the appropriate position and the obstacle on a path when it is determined in the first step that interference will not occur; a third step of controlling the harvesting unit to perform harvesting when it is determined in the second step that no interference will occur; A harvesting method comprising:

10. A program executed by a computer that controls a harvesting device, a first step of determining whether interference between the harvesting unit and an obstacle will occur on a path when the harvesting unit that harvests the harvest object is moved to an appropriate position for harvesting the harvest object; a second step of determining whether interference will occur between a harvesting unit moving unit that moves the harvesting unit to the appropriate position and the obstacle on a path when it is determined that no interference will occur in the first step; a third step of controlling the harvesting unit to perform harvesting when it is determined in the second step that no interference will occur; A program that causes the computer to execute the above.

11. The control unit and further performing a sixth step of assuming that the harvesting unit moves to the correct position along a predetermined third direction; In the first step, when the harvesting unit moves in the third direction, it is determined whether interference between the harvesting unit and the obstacle occurs.

10. The harvesting device of claim 1.

12. The control unit: In the first step, when it is determined that interference between the harvesting unit and the obstacle occurs, the sixth step is executed again, and the movement direction of the harvesting unit is determined to be a fourth direction deviated from the third direction; When it is determined in the first step that no interference occurs between the harvesting unit and the obstacle, the movement direction of the harvesting unit is determined to be the third direction. Harvesting device according to claim 11.

Images