A tomato robot capable of picking tomatoes from obstructed areas and its control method

Abstract

The present invention discloses a tomato robot capable of harvesting tomatoes from obstructed areas and a control method thereof. The tomato robot comprises a chassis, running wheels, an electrical box, a harvesting mechanism, and a branch and leaf processing mechanism. The harvesting mechanism comprises a harvesting assembly and a lifting and rotating adjustment mechanism. A camera is mounted on the harvesting assembly and is signal-connected to the electrical box. Branch and leaf processing mechanisms are provided on both sides of the harvesting mechanism. The branch and leaf processing mechanisms comprise a second lifting mechanism, a second horizontal rotating mechanism, a second telescopic mechanism, and a lever and cutting blade located at the end of the second telescopic mechanism. The electrical box houses a tomato image dataset, an image processing module, and a control module. Based on the degree of obstruction of tomatoes by branches and leaves, the present invention utilizes the branch and leaf processing mechanism to remove or cut branches and leaves that obstruct tomatoes and hinder harvesting, thereby reducing misjudgment of ripe tomatoes, improving the accuracy and harvesting rate of ripe tomatoes, and minimizing damage to the tomatoes during the harvesting process.

Description

Technical Field

[0001] The present invention relates to the technical field of tomato picking, in particular to a tomato robot capable of picking tomatoes from obstructed areas and a control method thereof. Background Art

[0002] my country is a major producer of fruits and vegetables. Among all fruits and vegetables, tomato production accounts for a significant proportion and continues to grow annually. Currently, tomato harvesting in my country is largely done manually. Automated harvesting is crucial for ensuring a secure supply and efficient production of tomatoes. This requires a tomato robot that can harvest even obscured areas.

[0003] The background of tomato greenhouse cultivation is complex, and the growth forms of tomato fruits are different. There are problems such as overlapping and adhesion between fruits. Since the growth states of tomato leaves and branches are not the same, when picking tomatoes, the tomato leaves and branches will cause varying degrees of obstruction to the tomatoes. The existing tomato picking robots are inconvenient to avoid or clean the leaves and branches around the tomatoes, which is not conducive to human use. Therefore, the tomato robot needs to be redesigned and modified to effectively prevent the phenomenon of poor picking effect. Summary of the Invention

[0004] To address the problem that existing tomato-picking robots have difficulty avoiding or clearing branches and leaves around tomatoes, the present invention provides a tomato-picking robot capable of picking tomatoes even when obstructed by them, and a control method thereof. Based on the obstruction of tomatoes by branches and leaves, the present invention utilizes a branch and leaf handling mechanism to remove or cut branches and leaves that obstruct tomatoes and hinder picking. This reduces misjudgment of ripe tomatoes, improves the accuracy and rate of picking ripe tomatoes, and minimizes damage to the tomatoes during the picking process.

[0005] The technical solution adopted by the present invention is:

[0006] A tomato robot capable of picking tomatoes from obstructed areas, characterized by comprising a chassis, running wheels mounted below the chassis, an electrical box fixed to the chassis, a picking mechanism, and a branch and leaf processing mechanism;

[0007] The picking mechanism includes a picking assembly for picking tomatoes, a lifting and rotating adjustment mechanism connecting the chassis and the picking assembly, and the lifting and rotating adjustment mechanism is used to adjust the position of the picking assembly; a camera is provided on the picking assembly, and the camera is connected to the electrical box signal;

[0008] Both sides of the picking mechanism are provided with a branch and leaf processing mechanism, which includes a second lifting mechanism fixed to the chassis, a second horizontal rotation mechanism connected to the end of the second lifting mechanism, a second telescopic mechanism connected to the second horizontal rotation mechanism, and a lever and a cutting blade located at the end of the second telescopic mechanism; the lever and the cutting blade are fixed to the turntable, and the extension direction of the lever and the cutting blade is parallel to the same diameter of the turntable; the rotation of the turntable is driven by a fifth motor to adjust the direction of the lever and the cutting blade;

[0009] The electrical box is equipped with a tomato image data set, an image processing module, and a control module. The image processing module is used to determine the maturity of tomatoes and the occlusion of tomatoes by branches and leaves based on the images taken in real time by the camera. f , and determine the position of the tomato stalk or the centroid of the tomato, and the position of the branches and stems that constitute obstacles; the tomato image data set is used for the image processing module to determine the occlusion area of ​​the tomato by the branches and leaves, estimate the size of the tomato being occluded and determine the occlusion ratio; the control module estimates the occlusion ratio according to the occlusion ratio f , the position of the branches, leaves and stems that constitute obstacles, send control instructions to the branch and leaf processing mechanism to separate and / or cut the branches, leaves and stems that constitute obstacles, and send control instructions to the picking mechanism to pick the ripe tomatoes according to the position of the tomato stalk or the centroid position of the tomato.

[0010] Furthermore, the lifting and rotating adjustment mechanism includes a first lifting mechanism fixed on the chassis, a first horizontal rotating mechanism connected to the end of the first lifting mechanism, a first telescopic mechanism connected to the first horizontal rotating mechanism, and a vertical rotating mechanism connected to the end of the first telescopic mechanism, which respectively control the picking component to lift in the vertical direction, rotate in the horizontal direction, move in the radial direction along the horizontal rotation circle, and rotate in the vertical direction, and the vertical rotation axis is parallel to the movement direction of the picking component controlled by the first telescopic mechanism; the picking component includes a second drive box, a sixth motor fixed on the second drive box, a positive and negative threaded rod installed in the second drive box, a picking blade, and a storage box installed below the second drive box, the positive and negative threaded rod is driven to rotate by the sixth motor, the threads on both sides of the positive and negative threaded rod have opposite rotation directions, and the two sides are connected to the picking blade through a movable block, so that the picking blades on both sides approach / move away from each other when the positive and negative threaded rod rotates.

[0011] Furthermore, the first lifting mechanism includes a first drive box fixed on the chassis, a first motor fixed on the first drive box, a threaded rod installed on the first drive box, and a movable frame. The threaded rod is driven to rotate by the first motor, so that the movable frame threadedly connected to the threaded rod moves up and down; the first horizontal rotation mechanism includes a second motor and a movable column. The movable column is vertically arranged to the output shaft of the second motor and fixed on the output shaft of the second motor. The second motor is fixed on the movable frame; the first telescopic mechanism is a first electric cylinder. The first electric cylinder is fixedly installed on the movable column. The output end of the first electric cylinder is fixed to the telescopic column, driving the telescopic column to slide in the movable column; the vertical rotation mechanism is a third motor. The third motor is fixedly installed on the telescopic column. The second drive box is fixed to the output shaft end of the third motor.

[0012] Furthermore, the second lifting mechanism is a second electric cylinder, which is fixed to the chassis through a fixed block, and the free end of the second electric cylinder is equipped with a movable plate; a vertical limit rod is provided on the fixed block to limit the lifting and lowering of the movable plate, and a limit plate is provided on the top of the limit rod; the second horizontal rotation mechanism includes a fourth motor and a mounting frame, and the mounting frame is arranged perpendicular to the output shaft of the fourth motor and fixed on the output shaft of the fourth motor; the fourth motor is fixed to the movable plate through a connecting frame; the second telescopic mechanism is a third electric cylinder, and the third electric cylinder is fixedly mounted on the mounting frame, and the fifth motor is fixed to the output end of the third electric cylinder.

[0013] Furthermore, a through opening is provided on one side of the storage box, and the side of the storage box where the through opening is provided is slidably connected to a baffle through a slide groove, and when the baffle slides to the bottom of the slide groove, the through opening is blocked by the baffle, and when the baffle slides to the top of the slide groove, the through opening is exposed.

[0014] Furthermore, a rubber plate is provided at the bottom of the storage box, a first silicone buffer plate is installed above the inner wall of the storage box on the side close to the opening, and a second silicone buffer plate is installed below the inner wall of the storage box on the side away from the opening. The first silicone buffer plate and the second silicone buffer plate are both arranged to be inclined downward relative to the horizontal plane and the inclination directions of the two are opposite.

[0015] Furthermore, a weight sensor is provided at the bottom of the storage box, and the weight sensor is connected to the electrical box signal.

[0016] Furthermore, a protective plate is installed on the front side of the chassis in the forward direction.

[0017] Furthermore, the running wheel is a Mecanum wheel; and an antenna is provided on the electrical box.

[0018] The control method for the tomato robot capable of picking tomatoes from obstructed areas includes the following steps:

[0019] S1. The camera captures images in real time, and the image processing module processes the images in real time, selecting images with ripe tomato color. If the tomatoes in the image are not obscured by branches or leaves, and the area of ​​ripe tomato color accounts for at least 0.75 of the total tomato area, the control module issues a drive instruction to the picking mechanism based on the position of the tomato stem or the tomato centroid, and the ripe tomatoes in the image are picked. If the tomatoes in the image are obscured by branches or leaves, step S2 is performed to determine the obscuration ratio. f ;

[0020] S2. The image processing module performs threshold segmentation on the image to determine the outline of the obscured tomato, the outline of the branches and leaves, and the location of the branches and leaves, and to determine the occlusion boundary. The module searches for a tomato image in the tomato image dataset that is similar in shape and size to the exposed portion of the obscured tomato in the image, and replaces the obscured tomato image with the similar tomato image in the orientation of the obscured tomato. The module then determines the location or centroid of the obscured tomato's stem, as well as the occlusion ratio of the obscured tomato. f ;

[0021] When the occlusion ratio f <15%, and the ratio of the area of ​​ripe tomato color in the exposed part of the blocked tomato to the area of ​​the entire tomato is ≥0.75, the image processing module further determines whether the tomato stalk is blocked; if the stalk is not blocked, a picking instruction is issued to the picking mechanism to pick the tomato; if the stalk is blocked, the control module issues a control instruction to the branch and leaf processing mechanism according to the position of the branches and stems that constitute the obstacle, controls the lever to move toward the branches and stems and to the position of the branches and stems, and pushes away the branches and stems that constitute the obstacle; until the stalk is no longer blocked, the control module issues a control instruction to the picking mechanism according to the position of the tomato stalk or the position of the tomato centroid to pick the tomato;

[0022] When the occlusion ratio is 15% ≤ f When the occlusion ratio is less than 50%, the control module sends a control instruction to the branch processing mechanism according to the position of the branches and stems that constitute the obstacle, controls the lever to move toward the branches and stems and move to the position of the branches and stems, and pushes away the branches and stems that constitute the obstacle until the occlusion ratio is f <15% and the fruit stem is not blocked; the image processing module further identifies whether the tomatoes in the image after removing the branches, leaves and stems that constitute obstacles are ripe tomatoes. If the tomatoes in the image are determined to be ripe tomatoes, the control module sends a control instruction to the picking mechanism to pick the ripe tomatoes based on the position of the tomato fruit stem or the position of the tomato centroid;

[0023] When the occlusion ratio fWhen the occlusion ratio is greater than 50%, the control module sends a control instruction to the branch processing mechanism according to the position of the branches and stems that constitute the obstacle, controls the cutting blade to move toward the branches and stems and to the position of the branches and stems, and cuts the branches and stems that constitute the obstacle until the occlusion ratio is greater than 50%. f When the tomato is less than 15% and the stalk is not blocked, the image processing module further identifies whether the tomato in the image after cutting off the branches, leaves and stems that constitute the obstacle is a ripe tomato. If the tomato in the image is determined to be a ripe tomato, the control module sends a control instruction to the picking mechanism to pick the ripe tomatoes according to the position of the tomato stalk or the position of the tomato centroid.

[0024] Furthermore, the ripe tomato color is: 200≤R≤255, 20≤G≤150, 0≤B≤120.

[0025] The beneficial effects of the present invention are:

[0026] The present invention uses the branch and leaf processing mechanisms on both sides of the picking mechanism to separate or cut the branches and leaves that block the tomatoes and hinder picking, thereby reducing misjudgment of ripe tomatoes, improving the accuracy and picking rate of ripe tomatoes, and reducing damage to tomatoes during the picking process. Specifically, the image processing module judges the ripeness of tomatoes, the obstruction of tomatoes by branches and leaves, and the obstruction ratio based on the images captured in real time by the camera. f , and determine the position of the tomato stem or the centroid of the tomato, and the position of the branches and stems that constitute obstacles; the control module is based on the occlusion ratio f , the position of the branches, leaves and stalks that constitute an obstacle, sends a control instruction to the branch and leaf processing mechanism to push aside and / or cut the branches, leaves and stalks that constitute an obstacle, and sends a control instruction to the picking mechanism to pick the ripe tomatoes according to the position of the tomato stalk or the centroid position of the tomato. When the branch and leaf processing mechanism is executing the pushing aside and / or cutting of the branches, leaves and stalks that constitute an obstacle, it controls the second lifting mechanism, the second horizontal rotation mechanism, and the second telescopic mechanism to respectively control the fifth motor, the shifting rod and the cutting blade to the appropriate position, so that the shifting rod or the cutting blade is close to the branches, leaves and stalks that constitute an obstacle, and then pushes aside and / or cuts the branches, leaves and stalks that constitute an obstacle through the second lifting mechanism, the second horizontal rotation mechanism, and the second telescopic mechanism. When it is determined that the tomato is ripe, a control instruction is sent to the picking mechanism to pick the ripe tomato. The present invention is convenient for picking tomatoes, has a high degree of automation, and the device has the advantage of good picking effect.

[0027] The present invention can also control the picking component to rise and fall in the vertical direction, rotate in the horizontal direction, move in the radial direction of the horizontal rotation circle, and rotate in the vertical direction by controlling the first lifting mechanism, the first horizontal rotation mechanism, the first telescopic mechanism, and the vertical rotation mechanism, and the vertical rotation axis is parallel to the movement direction of the picking component controlled by the first telescopic mechanism, so as to control the picking component to the appropriate angle and position, thereby realizing multi-angle picking of tomatoes.

[0028] The present invention provides a baffle, a chute and a through-hole. When the baffle slides to the bottom of the chute, the through-hole is blocked by the baffle. At this time, tomatoes that fall into the storage box are blocked inside the storage box by the baffle. When the baffle slides to the top of the chute, the through-hole is exposed. At this time, the tomatoes can be taken out from the through-hole. Since the picking blade is located above the storage box, if the operator takes out the tomatoes from the top of the storage box, the operator's hands are easily scratched by the picking blade. The provision of the baffle, the chute and the through-hole makes it easy for the operator to take out the tomatoes from the storage box. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Schematic diagram of the robot structure of the present invention.

[0030] Figure 2 This is a schematic diagram of the travel wheel structure of the present invention.

[0031] Figure 3 This is a schematic diagram of the structure of the third motor and picking blade of the present invention.

[0032] Figure 4 It is a front cross-sectional schematic diagram of the storage box structure of the present invention.

[0033] Figure 5 This is a schematic cross-sectional view of the storage box of the present invention from another perspective with the baffle hidden.

[0034] Figure 6 This is a schematic diagram of the turntable and the fifth motor structure of the present invention.

[0035] Figure 7 It is a structural schematic diagram of the movable plate and the connecting frame of the present invention.

[0036] Figure 8 It is a schematic diagram of the mounting frame structure of the present invention.

[0037] Figure 9 This is a schematic diagram of the protective plate structure of the present invention.

[0038] Figure 10 Schematic diagrams of tomatoes with different degrees of occlusion, where (a) is the diagram of an unobstructed tomato and (b) is the occlusion ratio. f Schematic diagram of tomatoes with an occlusion ratio of 15% or less. (c) f Schematic diagram of tomatoes with a size of ≤50%, (d) is the occlusion ratio fDiagram of tomatoes with >50% yield.

[0039] In the figure, 1. chassis, 2. walking wheel, 3. electrical box, 4. antenna, 5. first drive box, 6. movable frame, 7. first motor, 8. second motor, 9. movable column, 10. telescopic column, 11. first electric cylinder, 12. third motor, 13. second drive box, 14. positive and negative threaded rod, 15. movable block, 16. picking blade, 17. storage box, 18. fixed block, 19. limit rod, 20. second electric cylinder, 21. movable plate, 22. connecting frame, 23. fourth motor, 24. mounting frame, 25. third electric cylinder, 26. fifth motor, 27. turntable, 28. lever, 29. cutting blade, 30. sixth motor, 31. first silicone buffer plate, 32. second silicone buffer plate, 33. rubber plate, 34. baffle, 35. limit plate, 36. protective plate, 37. camera, 38. through port. DETAILED DESCRIPTION

[0040] The present invention will be further described in detail below with reference to the specific embodiments of the drawings, but the protection scope of the present invention is not limited thereto.

[0041] Figure 1 The figure shows a specific embodiment of the tomato robot capable of picking tomatoes from obstructed areas according to the present invention, comprising a chassis 1, running wheels 2 mounted below the chassis 1, an electrical box 3 fixed to the chassis 1, a picking mechanism, and a branch and leaf processing mechanism. Figure 2 As shown, the running wheels 2 are Mecanum wheels. By controlling the rotation of each Mecanum wheel, the device can be quickly moved left and right, forward and backward, and rotated on the spot, making the device more flexible and allowing it to be flexibly moved in a small space. An antenna 4 is provided above the electrical box 3 to facilitate the electrical box 3 to receive and send signals.

[0042] The picking mechanism includes a picking assembly for picking tomatoes, a lifting and rotating adjustment mechanism connecting the chassis 1 and the picking assembly, and the lifting and rotating adjustment mechanism is used to adjust the position of the picking assembly. The lifting and rotating adjustment mechanism includes a first lifting mechanism fixed to the chassis 1, a first horizontal rotating mechanism connected to the end of the first lifting mechanism, a first telescopic mechanism connected to the first horizontal rotating mechanism, and a vertical rotating mechanism connected to the end of the first telescopic mechanism. These mechanisms respectively control the picking assembly to lift vertically, rotate horizontally, move radially along a horizontal rotation circle, and rotate vertically. The vertical rotation axis is parallel to the direction of movement of the picking assembly controlled by the first telescopic mechanism, thereby achieving multi-angle picking of tomatoes.

[0043] The first lifting mechanism includes a first drive box 5 fixed on the chassis 1, a first motor 7 fixed on the first drive box 5, a threaded rod installed on the first drive box 5, and a movable frame 6. The threaded rod is driven to rotate by the first motor 7, so that the movable frame 6 threadedly connected to the threaded rod moves up and down. The threaded rod is driven to rotate by the first motor 7, thereby driving the movable frame 6 to move up and down. The first drive box 5 is rotatably connected to the threaded rod through bearings arranged at both ends of the threaded rod. The arrangement of the bearings can reduce the friction between the first drive box 5 and the threaded rod, thereby facilitating the rotation of the threaded rod. The first horizontal rotation mechanism includes a second motor 8 and a movable column 9. The movable column 9 is arranged perpendicular to the output shaft of the second motor 8 and is fixed on the output shaft of the second motor 8. The second motor 8 is fixed on the movable frame 6, and the movable column 9 is driven by the second motor 8 to rotate relative to the movable frame 6. The first telescopic mechanism is a first electric cylinder 11, which is fixedly mounted on the movable column 9. The movable column 9 has a cavity for the telescopic column 10 to slide. The output end of the first electric cylinder 11 is fixed to the telescopic column 10, driving the telescopic column 10 to slide within the movable column 9. The first electric cylinder 11 drives the telescopic column 10 to move relative to the movable column 9. The vertical rotation mechanism is a third motor 12, which is fixedly mounted on the telescopic column 10. The second drive box 13 is fixed to the output shaft end of the third motor 12. Figure 3 As shown, the driving box 13 is driven to rotate by the third motor 12 , and the rotation axis is parallel to the movable direction of the telescopic column 10 .

[0044] like Figure 3 、 Figure 4 、 Figure 5 As shown, the picking assembly includes a second drive box 13, a sixth motor 30 fixed to the second drive box 13, a reciprocating threaded rod 14 mounted within the second drive box 13, a picking blade 16, and a storage box 17 mounted below the second drive box 13. The reciprocating threaded rod 14 is driven to rotate by the sixth motor 30. The threads on both sides of the reciprocating threaded rod 14 rotate in opposite directions, and the two sides are connected to the picking blades 16 via movable blocks 15. This allows the picking blades 16 on both sides to move closer or further apart when the reciprocating threaded rod 14 rotates. The picking blades 16 are located above the tomato inlet of the storage box 17. The sixth motor 30 drives the reciprocating threaded rod 14 to rotate. Since the threads on both sides of the reciprocating threaded rod 14 rotate in opposite directions, the rotation of the reciprocating threaded rod 14 drives the movable blocks 15 on both sides and the picking blades 16 on the movable blocks 15 toward or away from each other. When picking tomatoes, the picking blades 16 are controlled to move closer together until the stems are severed, and the tomatoes fall into the storage box 17, thereby completing the picking process.

[0045] The storage box 17 has an opening 38 on one side. A baffle 34 is slidably connected to the side of the storage box 17 where the opening 38 is located, via a chute. When the baffle 34 slides to the bottom of the chute, the opening 38 is blocked by the baffle 34. When the baffle 34 slides to the top of the chute, the opening 38 is exposed, allowing the tomatoes to be removed from the opening 38. Since the picking blade 16 is located above the storage box 17, the operator's hand could be scratched by the picking blade 16 when removing tomatoes from the top of the storage box 17. The baffle 34, chute, and opening 38 facilitate the operator's removal of tomatoes from the storage box 17. A rubber sheet 33 is provided at the bottom of the storage box 17. A first silicone cushioning plate 31 is mounted above the inner wall of the storage box 17 on the side near the opening 38, and a second silicone cushioning plate 32 is mounted below the inner wall of the storage box 17 on the side away from the opening 38. Both the first and second silicone cushioning plates 31 and 32 are tilted downward relative to the horizontal plane, and their tilt directions are opposite. By configuring the rubber sheet 33, the first silicone cushioning sheet 31, and the second silicone cushioning sheet 32, tomatoes falling into the storage bin 17 are first cushioned by the first silicone cushioning sheet 31, then fall onto the surface of the second silicone cushioning sheet 32, where they are cushioned for a second time by the second silicone cushioning sheet 32, and finally by the rubber sheet 33 for a third time. This prevents the tomatoes from being damaged when they fall into the storage bin 17, thereby protecting the tomatoes. A weight sensor is located at the bottom of the storage bin 17 and is connected to the electrical box 3. Before the robot begins operation, a weight threshold is manually set based on the capacity of the storage bin 17. During the picking process, when the control module detects that the weight sensor reading reaches this threshold, it determines that the storage bin 17 is full of tomatoes, stops moving and operating, and sounds an alarm to alert staff to clear the tomatoes from the storage bin 17.

[0046] Both sides of the picking mechanism are provided with a branch and leaf processing mechanism for processing the branches and leaves on both sides that hinder the picking of tomatoes and block the tomatoes. The branch and leaf processing mechanism includes a second lifting mechanism fixed on the chassis 1, a second horizontal rotation mechanism connected to the end of the second lifting mechanism, a second telescopic mechanism connected to the second horizontal rotation mechanism, and a lever 28 and a cutting blade 29 located at the end of the second telescopic mechanism. The second lifting mechanism, the second horizontal rotation mechanism, and the second telescopic mechanism respectively control the fifth motor 26 to lift in the vertical direction, rotate in the horizontal direction, and move in the radial direction of the horizontal rotation circle. Figure 6As shown, the shift rod 28 and the cutting blade 29 are fixed on the turntable 27, and the extension direction of the shift rod 28 and the cutting blade 29 is parallel to the same diameter of the turntable 27. The rotation of the turntable 27 is driven by the fifth motor 26 to adjust the orientation of the shift rod 28 and the cutting blade 29. The second lifting mechanism is a second electric cylinder 20, which is fixed to the chassis 1 through a fixed block 18. The free end of the second electric cylinder 20 is equipped with a movable plate 21; a vertical limit rod 19 is provided on the fixed block 18 to limit the lifting and lowering of the movable plate 21, and a limit plate 35 is provided on the top of the limit rod 19 to limit the sliding of the movable plate 21 to prevent the movable plate 21 from sliding outside the limit rod 19 and then separating from the limit rod 19. The movable plate 21 can be driven up and down by controlling the second electric cylinder 20. As shown Figure 7 and Figure 8 As shown, the second horizontal rotation mechanism includes a fourth motor 23 and a mounting bracket 24. The mounting bracket 24 is arranged perpendicular to and fixed to the output shaft of the fourth motor 23. The fourth motor 23 is fixed to the movable plate 21 via a connecting bracket 22, and the fourth motor 23 drives the mounting bracket 24 to rotate relative to the connecting bracket 22. The second telescopic mechanism is a third electric cylinder 25, which is fixedly mounted on the mounting bracket 24. The fifth motor 26 is fixed to the output end of the third electric cylinder 25. The third electric cylinder 25 drives the fifth motor 26 to move, thereby driving the lever 28 and cutting blade 29 to move, thereby adjusting the lever 28 and cutting blade 29 to the desired position.

[0047] A protective plate 36 is installed on the front side of the chassis 1 in the forward direction. When moving, it can block obstacles in front of the device, thereby playing the role of protecting the device. The second motor 8, the third motor 12, the fourth motor 23 and the fifth motor 26 are all servo motors. The servo motor can efficiently control the rotation angle, thereby improving the control effect of the present invention. The first motor 7 and the sixth motor 30 are both DC motors. The DC motor has a larger torque. The first motor 7 can better control the up and down movement of the movable frame 6, and the sixth motor 30 can better control the opening and closing of the two picking blades 16. The first motor 7, the second motor 8, the third motor 12, the fourth motor 23, the fifth motor 26, the sixth motor 30, the first electric cylinder 11, the second electric cylinder 20, and the third electric cylinder 25 are all connected to the electrical box 3 signal.

[0048] Combine Figure 10The diagram shows tomatoes with different degrees of obscuration. In an unstructured facility environment, tomatoes grown on the ground are usually obscured by branches and leaves, and the fruits obscure each other. When a tomato picking robot encounters severely obscured tomatoes during operation, target recognition and positioning become extremely difficult. Inaccurate recognition and positioning may cause changes in the robot's motion trajectory and damage to other fruits. According to the degree of obscuration by branches and leaves, tomatoes are divided into no obscuration, light obscuration, moderate obscuration and heavy obscuration. Figure 10 (a) is a schematic diagram of unobstructed tomatoes. At this time, the tomatoes are not blocked and can be picked directly. Figure 10 Middle (b) is the occlusion ratio f Diagram of a tomato with a occlusion ratio of <15%. f If the stalk is not blocked, the tomatoes can be picked directly. If the stalk is blocked, the lever 28 is used to push away the branches and leaves that constitute the obstacle, and the tomatoes are picked after being determined to be ripe. Figure 10 Middle (c) means the occlusion ratio is 15% or less f Schematic diagram of a tomato with an occlusion ratio of 15% or less f ≤50% is marked as moderate occlusion. In this case, the branches, leaves and stems that constitute obstacles are pushed aside using the lever 28, and the tomatoes are picked after being determined to be ripe. Figure 10 Middle (d) is the occlusion ratio f >50% tomato diagram, occlusion ratio f If the occlusion rate exceeds 50%, it is marked as severe occlusion. At this point, the cutting blade 29 is used to cut the obstructing branches, leaves, and stems, and the tomatoes are picked after they are determined to be ripe. By classifying occlusions, the tomato picking robot's vision is more accurate, achieving efficient and intelligent picking.

[0049] The picking assembly is provided with a camera 37, which is connected to the electrical box 3. The electrical box 3 is provided with an image processing module and a control module. The image processing module is used to judge the ripeness of tomatoes and the occlusion of tomatoes by branches and leaves and the occlusion ratio based on the images taken in real time by the camera 37. f , and determine the position of the tomato stalk or the centroid of the tomato, and the position of the branches and stems that constitute obstacles; the tomato image data set is used for the image processing module to determine the occlusion area of ​​the tomato by the branches and leaves, estimate the size of the tomato being occluded and determine the occlusion ratio; the control module estimates the occlusion ratio according to the occlusion ratio f, the position of the branches, leaves and stems that constitute an obstacle, and send a control instruction to the branch and leaf processing mechanism to push away and / or cut the branches, leaves and stems that constitute an obstacle, and send a control instruction to the picking mechanism according to the position of the tomato stalk or the position of the tomato centroid to pick the ripe tomatoes. Specifically, the image processing module processes the image taken by the camera 37 in real time, and filters out the image with the color of ripe tomatoes, and the ripe tomato color is: 200≤R≤255, 20≤G≤150, 0≤B≤120. If the tomatoes in the image are not blocked by branches and leaves, and the area of ​​the ripe tomato color accounts for more than 0.75 of the entire tomato area, the control module sends a drive instruction to the picking mechanism according to the position of the tomato stalk or the position of the tomato centroid to pick the ripe tomatoes in the image. If the tomatoes in the image are blocked by branches and leaves, the image processing module performs threshold segmentation on the image to determine the outline of the blocked tomato, the outline of the branches and leaves, and the position of the branches and leaves and stems, and determines the blocking boundary. Search for a tomato image in the tomato image dataset that is similar in shape and size to the exposed part of the occluded tomato in the image, and replace the occluded tomato image with the similar tomato image in the orientation of the occluded tomato, and determine the position of the stalk or centroid of the occluded tomato, as well as the occlusion ratio of the occluded tomato f , where the occlusion ratio of the occluded tomato is f =The area of ​​the tomato blocked by the branches and leaves / the area of ​​the tomato in the similar tomato image.

[0050] When the occlusion ratio f <15%, and the ratio of the area of ​​ripe tomato color in the exposed part of the blocked tomato to the area of ​​the entire tomato is ≥0.75, the image processing module further determines whether the tomato stalk is blocked; if the stalk is not blocked, a picking instruction is issued to the picking mechanism to pick the tomato; if the stalk is blocked, the control module issues a control instruction to the branch and leaf processing mechanism according to the position of the branches and stems that constitute the obstacle, controls the lever 28 to move toward the branches and stems and to the position of the branches and stems, and pushes away the branches and stems that constitute the obstacle; until the stalk is not blocked, the control module issues a control instruction to the picking mechanism according to the position of the tomato stalk or the position of the tomato centroid to pick the tomato; when the occlusion ratio is 15%≤ f When the occlusion ratio is less than 50%, the control module sends a control instruction to the branch processing mechanism according to the position of the branches and stems that constitute the obstacle, controls the lever 28 to move toward the branches and stems and to the position of the branches and stems, and pushes away the branches and stems that constitute the obstacle until the occlusion ratio is less than 50%. f <15% and the fruit stem is not blocked; the image processing module further identifies whether the tomato in the image after removing the branches, leaves and stems that constitute obstacles is a ripe tomato. If the tomato in the image is determined to be a ripe tomato, the control module sends a control instruction to the picking mechanism to pick the ripe tomato according to the position of the tomato fruit stem or the position of the tomato centroid; when the occlusion ratio fWhen the occlusion ratio is greater than 50%, the control module sends a control instruction to the branch processing mechanism according to the position of the branches and stems that constitute the obstacle, controls the cutting blade 29 to move toward the branches and stems and to the position of the branches and stems, and cuts the branches and stems that constitute the obstacle until the occlusion ratio is greater than 50%. f When the tomato is less than 15% and the stalk is not blocked, the image processing module further identifies whether the tomato in the image after cutting off the branches, leaves and stems that constitute the obstacle is a ripe tomato. If the tomato in the image is determined to be a ripe tomato, the control module sends a control instruction to the picking mechanism to pick the ripe tomatoes according to the position of the tomato stalk or the position of the tomato centroid.

[0051] The electrical box 3 can also be connected to the signal of an external remote control. The remote control is provided with a display screen. The camera 37 takes real-time images and transmits them to the display screen through the electrical box 3, so that the staff can check the working status of the tomato picking robot in real time. If necessary, the robot can be remotely started and stopped by the remote control.

[0052] During operation, the control module issues a control instruction to drive the threaded rod to rotate by controlling the first motor 7, thereby driving the movable frame 6, the storage box 17 and the picking blade 16 to move up or down; drives the movable column 9 to rotate by controlling the second motor 8, thereby driving the telescopic column 10, the storage box 17 and the picking blade 16 to rotate; controls the first electric cylinder 11 to extend or contract, thereby driving the second drive box 13, the storage box 17 and the picking blade 16 to move; drives the second drive box 13 to rotate by controlling the third motor 12, thereby driving the storage box 17 and the picking blade 16 to rotate, and finally makes the storage box 17 and the picking blade 16 at a suitable angle and position. When picking tomatoes, the storage box 17 is covered on the outside of the tomatoes by adjusting the operation of the first motor 7, the second motor 8, the first electric cylinder 11 and the third motor 12, and then the picking blades 16 are moved to both sides of the stem between the tomatoes and the tomato tree. Then, the sixth motor 30 is controlled to rotate to drive the forward and reverse threaded rods 14 to rotate, thereby driving the two movable blocks 15 to approach each other, and then driving the two picking blades 16 to approach each other, until the stems are cut off and the tomatoes fall into the storage box 17, and the picking of the tomatoes is completed.

[0053] When the image processing module determines that the surrounding branches and leaves will hinder the picking of tomatoes and block the tomatoes, making it inconvenient for the picking blade 16 to pick the tomatoes, the control module issues a control instruction to control the second electric cylinder 20 to extend or contract, thereby driving the movable plate 21 to move up or down, and similarly driving the lever 28 or cutting blade 29 to move up or down. The control module can adjust the fourth motor 23 to rotate and drive the mounting frame 24, the third electric cylinder 25, the fifth motor 26 and the turntable 27 to rotate. The control module can adjust the third electric cylinder 25 to extend or contract, thereby driving the fifth motor 26 and the turntable 27 to move left and right, so that the lever 28 or cutting blade 29 is close to the branches and leaves that constitute an obstacle. The control module can control the movement of the lever 28 by controlling the second electric cylinder 20, the fourth motor 23 and the third electric cylinder 25, thereby moving the branches and leaves that constitute an obstacle away or cutting. The control module can adjust the movement of the lever 28 according to the occlusion ratio. f By controlling the fifth motor 26, the shifting rod 28 or the cutting blade 29 is controlled to approach the branches, leaves and stems that constitute obstacles, and the branches, leaves and stems that constitute obstacles are shifted or cut.

[0054] The control method for the tomato robot capable of picking tomatoes from obstructed areas includes the following steps:

[0055] S1. Camera 37 captures an image in real time. The image processing module processes the image captured by camera 37 in real time, selecting images with a ripe tomato color. The ripe tomato color is: 200 ≤ R ≤ 255, 20 ≤ G ≤ 150, and 0 ≤ B ≤ 120. If the tomatoes in the image are not obscured by branches or leaves, and the area of ​​the ripe tomato color accounts for at least 0.75 of the total tomato area, the control module issues a drive instruction to the picking mechanism based on the position of the tomato stem or the tomato centroid, and the ripe tomatoes in the image are picked. If the tomatoes in the image are obscured by branches or leaves, step S2 is performed to determine the obscuration ratio. f .

[0056] S2. The image processing module performs threshold segmentation on the image to determine the outline of the obscured tomato, the outline of the branches and leaves, and the location of the branches and leaves, and also determines the occlusion boundary. The module searches for a tomato image in the tomato image dataset that is similar in shape and size to the exposed portion of the obscured tomato in the image. The module then replaces the obscured tomato image with the similar tomato image in the same orientation as the obscured tomato, and determines the location or centroid of the obscured tomato's stem, as well as the occlusion ratio of the obscured tomato. f ;

[0057] When the occlusion ratio f<15%, and the ratio of the area of ​​ripe tomato color in the exposed part of the blocked tomato to the area of ​​the entire tomato is ≥0.75, the image processing module further determines whether the tomato stalk is blocked; if the stalk is not blocked, a picking instruction is issued to the picking mechanism to pick the tomatoes; if the stalk is blocked, the control module issues a control instruction to the branch and leaf processing mechanism according to the position of the branches and stems that constitute the obstacle, controls the lever 28 to move toward the branches and stems and to the position of the branches and stems, and pushes away the branches and stems that constitute the obstacle; until the stalk is no longer blocked, the control module issues a control instruction to the picking mechanism according to the position of the tomato stalk or the position of the tomato centroid to pick the tomatoes;

[0058] When the occlusion ratio is 15% ≤ f When the occlusion ratio is less than 50%, the control module sends a control instruction to the branch processing mechanism according to the position of the branches and stems that constitute the obstacle, controls the lever 28 to move toward the branches and stems and to the position of the branches and stems, and pushes away the branches and stems that constitute the obstacle until the occlusion ratio is less than 50%. f <15% and the fruit stem is not blocked; the image processing module further identifies whether the tomatoes in the image after removing the branches, leaves and stems that constitute obstacles are ripe tomatoes. If the tomatoes in the image are determined to be ripe tomatoes, the control module sends a control instruction to the picking mechanism to pick the ripe tomatoes based on the position of the tomato fruit stem or the position of the tomato centroid;

[0059] When the occlusion ratio f When the occlusion ratio is greater than 50%, the control module sends a control instruction to the branch processing mechanism according to the position of the branches and stems that constitute the obstacle, controls the cutting blade 29 to move toward the branches and stems and to the position of the branches and stems, and cuts the branches and stems that constitute the obstacle until the occlusion ratio is greater than 50%. f When the tomato is less than 15% and the stalk is not blocked, the image processing module further identifies whether the tomato in the image after cutting off the branches, leaves and stems that constitute the obstacle is a ripe tomato. If the tomato in the image is determined to be a ripe tomato, the control module sends a control instruction to the picking mechanism to pick the ripe tomatoes according to the position of the tomato stalk or the position of the tomato centroid.

[0060] The examples are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments. Any obvious improvements, substitutions or modifications that can be made by those skilled in the art without departing from the essence of the present invention are within the scope of protection of the present invention.

Claims

1. A tomato robot capable of harvesting tomatoes from obscured areas, characterized by: It comprises a chassis (1), a running wheel (2) mounted below the chassis (1), an electrical box (3) fixed on the chassis (1), a picking mechanism, and a branch and leaf processing mechanism; The picking mechanism comprises a picking assembly for picking tomatoes, a lifting and rotating adjustment mechanism connecting the chassis (1) and the picking assembly, and the lifting and rotating adjustment mechanism is used to adjust the position of the picking assembly; a camera (37) is provided on the picking assembly, and the camera (37) is connected to the electrical box (3) for signal communication; Both sides of the picking mechanism are provided with a branch and leaf processing mechanism, and the branch and leaf processing mechanism includes a second lifting mechanism fixed on the chassis (1), a second horizontal rotation mechanism connected to the end of the second lifting mechanism, a second telescopic mechanism connected to the second horizontal rotation mechanism, and a lever (28) and a cutting blade (29) located at the end of the second telescopic mechanism; the lever (28) and the cutting blade (29) are fixed on the turntable (27), and the extension direction of the lever (28) and the cutting blade (29) is parallel to the same diameter of the turntable (27); the rotation of the turntable (27) is driven by a fifth motor (26) to adjust the direction of the lever (28) and the cutting blade (29); The electrical box (3) is provided with a tomato image data set, an image processing module, and a control module. The image processing module is used to judge the maturity of the tomato and the occlusion of the tomato by the branches and leaves and the occlusion ratio based on the image taken in real time by the camera (37). f , and determine the position of the tomato stalk or the centroid of the tomato, and the position of the branches and stems that constitute obstacles; the tomato image data set is used for the image processing module to determine the occlusion area of ​​the tomato by the branches and leaves, estimate the size of the tomato being occluded and determine the occlusion ratio; the control module estimates the occlusion ratio according to the occlusion ratio f , the position of the branches, leaves and stems that constitute obstacles, sending control instructions to the branch and leaf processing mechanism to push away and / or cut the branches, leaves and stems that constitute obstacles, and sending control instructions to the picking mechanism to pick the ripe tomatoes according to the position of the tomato stalk or the position of the tomato centroid; Specifically, occlusion ratio f The determination method is as follows: the image processing module performs threshold segmentation on the image to determine the outline of the obscured tomato, the outline of the branches and leaves, and the position of the branches and leaves, and determines the obscuration boundary; searches for a tomato image in the tomato image data set that is similar in shape and size to the exposed part of the obscured tomato in the image, and replaces the obscured tomato image with the similar tomato image in the orientation of the obscured tomato, and determines the fruit stem position or centroid position of the obscured tomato, as well as the obscuration ratio of the obscured tomato. f .

2. The tomato robot capable of picking tomatoes from obstructed areas according to claim 1, characterized in that: The lifting and rotating adjustment mechanism comprises a first lifting mechanism fixed on the chassis (1), a first horizontal rotating mechanism connected to the end of the first lifting mechanism, a first telescopic mechanism connected to the first horizontal rotating mechanism, and a vertical rotating mechanism connected to the end of the first telescopic mechanism, which respectively control the picking component to lift in the vertical direction, rotate in the horizontal direction, move in the radial direction of the horizontal rotation circle, and rotate in the vertical direction, and the vertical rotation axis is parallel to the movement direction of the picking component controlled by the first telescopic mechanism; The picking assembly comprises a second drive box (13), a sixth motor (30) fixed on the second drive box (13), a forward and reverse threaded rod (14) installed in the second drive box (13), a picking blade (16), and a storage box (17) installed below the second drive box (13). The forward and reverse threaded rod (14) is driven to rotate by the sixth motor (30). The threads on both sides of the forward and reverse threaded rod (14) rotate in opposite directions, and both sides are connected to the picking blade (16) through a movable block (15), so that the picking blades (16) on both sides approach / move away from each other when the forward and reverse threaded rod (14) rotates.

3. The tomato robot capable of picking tomatoes from obstructed areas according to claim 2, characterized in that: The first lifting mechanism comprises a first drive box (5) fixed on the chassis (1), a first motor (7) fixed on the first drive box (5), a threaded rod installed on the first drive box (5), and a movable frame (6), wherein the threaded rod is driven to rotate by the first motor (7), so that the movable frame (6) threadedly connected to the threaded rod moves up and down; The first horizontal rotation mechanism comprises a second motor (8) and a movable column (9), wherein the movable column (9) is arranged perpendicular to the output shaft of the second motor (8) and is fixed on the output shaft of the second motor (8), and the second motor (8) is fixed on the movable frame (6); The first telescopic mechanism is a first electric cylinder (11), which is fixedly mounted on the movable column (9). The output end of the first electric cylinder (11) is fixed to the telescopic column (10), driving the telescopic column (10) to slide inside the movable column (9); The vertical rotation mechanism is a third motor (12), the third motor (12) is fixedly mounted on the telescopic column (10), and the second drive box (13) is fixed to the end of the output shaft of the third motor (12).

4. The tomato robot capable of picking tomatoes from obstructed areas according to claim 1, characterized in that: The second lifting mechanism is a second electric cylinder (20), which is fixed to the chassis (1) via a fixing block (18), and a movable plate (21) is mounted on the free end of the second electric cylinder (20); a vertical limiting rod (19) is provided on the fixing block (18) to limit the lifting of the movable plate (21), and a limiting plate (35) is provided on the top of the limiting rod (19); The second horizontal rotation mechanism comprises a fourth motor (23) and a mounting frame (24), wherein the mounting frame (24) is arranged perpendicular to the output shaft of the fourth motor (23) and is fixed on the output shaft of the fourth motor (23); the fourth motor (23) is fixed on the movable plate (21) via a connecting frame (22); The second telescopic mechanism is a third electric cylinder (25), the third electric cylinder (25) is fixedly mounted on the mounting frame (24), and the fifth motor (26) is fixed to the output end of the third electric cylinder (25).

5. The tomato robot capable of picking tomatoes from obstructed areas according to claim 2, characterized in that: A through opening (38) is provided on one side of the storage box (17), and a baffle (34) is slidably connected to the side of the storage box (17) where the through opening (38) is provided through a chute, and when the baffle (34) slides to the bottom of the chute, the through opening (38) is blocked by the baffle (34), and when the baffle (34) slides to the top of the chute, the through opening (38) is exposed.

6. The tomato robot capable of picking tomatoes from obstructed areas according to claim 5, characterized in that: A rubber plate (33) is provided at the bottom of the storage box (17), a first silicone buffer plate (31) is installed above the inner wall of the storage box (17) on the side close to the opening (38), and a second silicone buffer plate (32) is installed below the inner wall of the storage box (17) on the side away from the opening (38), the first silicone buffer plate (31) and the second silicone buffer plate (32) are both arranged to be inclined downward relative to the horizontal plane and the inclination directions of the two are opposite.

7. The tomato robot capable of picking tomatoes from obstructed areas according to claim 2, characterized in that: A weight sensor is provided at the bottom of the storage box (17), and the weight sensor is connected to the electrical box (3) for signal transmission.

8. The tomato robot capable of picking tomatoes from obstructed areas according to claim 1, characterized in that: A protective plate (36) is installed on the front side of the chassis (1) in the forward direction.

9. The tomato robot capable of picking tomatoes from obstructed areas according to claim 1, characterized in that: The traveling wheel (2) is a Mecanum wheel; an antenna (4) is provided on the electrical box (3).

10. The control method for a tomato robot capable of picking tomatoes from obstructed areas according to any one of claims 1 to 9, characterized in that: The following steps are involved: S1. The camera (37) captures an image in real time, and the image processing module processes the image captured by the camera (37) in real time, and selects an image with a ripe tomato color; if the tomatoes in the image are not blocked by branches and leaves, and the area of ​​the ripe tomato color accounts for more than 0.75 of the entire tomato area, the control module sends a driving instruction to the picking mechanism according to the position of the tomato stem or the position of the tomato centroid, and picks the ripe tomatoes in the image; if the tomatoes in the image are blocked by branches and leaves, step S2 is performed to determine the blocking ratio f ; S2. The image processing module performs threshold segmentation on the image to determine the outline of the obscured tomato, the outline of the branches and leaves, and the location of the branches and leaves, and to determine the occlusion boundary. The module searches for a tomato image in the tomato image dataset that is similar in shape and size to the exposed portion of the obscured tomato in the image, and replaces the obscured tomato image with the similar tomato image in the orientation of the obscured tomato. The module then determines the location or centroid of the obscured tomato's stem, as well as the occlusion ratio of the obscured tomato. f ; When the occlusion ratio f <15%, and the ratio of the area of ​​the ripe tomato color in the exposed part of the blocked tomato to the area of ​​the entire tomato is ≥0.75, the image processing module further determines whether the tomato stem is blocked; if the tomato stem is not blocked, a picking instruction is sent to the picking mechanism to pick the tomato; if the tomato stem is blocked, the control module sends a control instruction to the branch and leaf processing mechanism according to the position of the branches and leaves that constitute the obstacle, controls the lever (28) to move toward the branches and leaves and move to the position of the branches and leaves, and pushes away the branches and leaves that constitute the obstacle; until the tomato stem is not blocked, the control module sends a control instruction to the picking mechanism according to the position of the tomato stem or the position of the tomato centroid to pick the tomato; When the occlusion ratio is 15% ≤ f When the occlusion ratio is less than 50%, the control module sends a control instruction to the branch and leaf processing mechanism according to the position of the branches and leaves constituting the obstacle, controls the lever (28) to move toward the branches and leaves and move to the position of the branches and leaves, and pushes away the branches and leaves constituting the obstacle until the occlusion ratio is less than 50%. f <15% and the fruit stem is not blocked; the image processing module further identifies whether the tomatoes in the image after removing the branches, leaves and stems that constitute obstacles are ripe tomatoes. If the tomatoes in the image are determined to be ripe tomatoes, the control module sends a control instruction to the picking mechanism to pick the ripe tomatoes based on the position of the tomato fruit stem or the position of the tomato centroid; When the occlusion ratio f When the occlusion ratio is greater than 50%, the control module sends a control instruction to the branch and leaf processing mechanism according to the position of the branches and leaves that constitute the obstacle, controls the cutting blade (29) to move toward the branches and leaves and move to the position of the branches and leaves and cut the branches and leaves that constitute the obstacle until the occlusion ratio is greater than 50%. f When the tomato is less than 15% and the stalk is not blocked, the image processing module further identifies whether the tomato in the image after cutting off the branches, leaves and stems that constitute the obstacle is a ripe tomato. If the tomato in the image is determined to be a ripe tomato, the control module sends a control instruction to the picking mechanism to pick the ripe tomatoes according to the position of the tomato stalk or the position of the tomato centroid.

11. The control method for a tomato robot capable of picking tomatoes from obstructed areas according to claim 10, characterized in that: The ripe tomato color is: 200≤R≤255, 20≤G≤150, 0≤B≤120.

Images