Finger plate air suction and deflation type approach triggering variable-structure flexible picking manipulator and control method thereof

Abstract

The invention relates to the technical field of agricultural fruit automatic picking robots, in particular to a finger plate air suction and deflation type approaching triggering variable structure flexible picking manipulator and a control method thereof.The finger plate air suction and deflation type approaching triggering variable structure flexible picking manipulator comprises a support, one end of the support is connected with a mechanical arm through a grabbing driving system, and the other end of the support is connected with a mechanical arm; image acquisition equipment is mounted on the mechanical arm, a variable structure mechanism is mounted in the middle of the support, the support is connected with the finger plate suction and release type grabbing device through the variable structure mechanism, and an approach triggering system is mounted at the other end of the support; the approach triggering system and the image acquisition equipment are in communication connection with the variable structure mechanism and the finger plate suction and release type grabbing device through the grabbing driving system so as to realize self-adaptive automatic picking of fruits; the device has the beneficial effects that finger plate air suction and deflation type grabbing and passive grabbing force control are achieved, proximity type grabbing triggering is achieved, effective grabbing is achieved by changing the grabbing structure according to fruits of different shapes, the fruit picking accuracy is improved, automatic picking is achieved, operation is convenient, and the popularization performance is high.

Description

Technical Field

[0001] The present invention belongs to the technical field of agricultural fruit automatic picking robots, and particularly relates to a finger plate air-sucking, air-releasing, proximity-triggered variable-structure flexible picking manipulator and a control method thereof. Technical Background

[0002] As a way to realize the low-loss and non-destructive picking actuator of fruits in the agricultural field, flexible manipulators have developed rapidly with the rise of intelligent picking robots. However, the existing technical system still has significant limitations. First, the grasping actions of existing pneumatic flexible manipulators are driven by a centralized air source, which is large in volume and high in cost, and requires a high-precision air pressure and air flow regulation system. Second, the grasping behavior and force are completely actively controlled, and often require precise force feedback to achieve low-loss grasping, further increasing the control cost. Third, the current mainstream pneumatic picking devices generally adopt a homogenized structure, and the actuating mechanism lacks the ability of active deformation. It can only realize the grasping operation of a single variety through a preset shape, and cannot realize the non-destructive picking of fruits with various different shapes.

[0003] The Chinese invention patent "Picking Robot Suitable for Picking Multiple Kinds of Fruits and Its Picking Method" (CN112840862B) can realize the pneumatic picking of various fruits with different sizes by adjusting the grasping range of the soft fingers. However, it still needs to be driven by an external air compressor. After starting picking, the internal cavity air pressure of its soft fingers cannot be automatically adjusted, making it difficult to achieve force-adaptive non-destructive picking of different fruits. And because of its simple structure and inability to deform, it cannot accurately grasp fruits with complex shapes such as cylindrical and square shapes, and has poor versatility. The Chinese invention patent "Fruit and Vegetable Picking Actuator Based on Flexible Grasping and Integrated Clamping and Cutting and Its Picking Method" (CN110432000B) can realize the picking of fruits and vegetables with different shapes by changing the structure. However, its soft fingers can only bend and deform according to the preset air pressure, and adopt an active control grasping method, and cannot adaptively adjust the clamping force according to the specific size of the target fruit and vegetable.

[0004] By searching the existing technology, there is currently no finger plate air-sucking, air-releasing, proximity-triggered, and variable-structure self-adaptive flexible picking manipulator. To meet the development needs of modern agricultural intelligent picking technology, an invention provides a finger plate air-sucking, air-releasing, proximity-triggered variable-structure flexible picking manipulator and a control method thereof to realize the automatic non-destructive harvesting of different fruits and promote the further development of picking robot technology in China. Summary of the Invention

[0005] Aiming at the current situation in the field of agricultural intelligent picking in China that lacks a general-purpose low-loss harvesting end effector, the present invention provides a finger plate air-sucking, air-releasing, proximity-triggered variable-structure flexible picking manipulator and a control method thereof, which realize finger plate air-sucking and air-releasing grasping and passive grasping force control, realize proximity grasping triggering, and change the grasping structure for effective grasping of fruits with different shapes.

[0006] To achieve the above object, the technical solution adopted by the present invention to solve its technical problems is as follows: A finger plate air suction and exhaust type proximity trigger variable structure flexible picking manipulator, which includes a bracket connected to a robotic arm equipped with an image acquisition device. One end of the bracket is equipped with a grasping drive system, a variable structure mechanism is installed in the middle of the bracket and is connected to a finger plate air suction and exhaust type grasping device through the variable structure mechanism, and a proximity trigger system is installed at the other end of the bracket; the proximity trigger system and the image acquisition device are communicatively connected to the variable structure mechanism and the finger plate air suction and exhaust type grasping device through the grasping drive system to achieve adaptive automatic picking of fruits.

[0007] The bracket includes a fixed base, a connection base, and a hardware fixing plate. One end of the fixed base is equipped with the grasping drive system, the other end of the fixed base is connected to the hardware fixing plate through the connection base, the variable structure mechanism is installed on the connection base, and the proximity trigger system is installed on the hardware fixing plate.

[0008] One end of the fixed base is provided with an opening. The connection base includes a connecting rod fixing plate, a support column, and a mounting plate. The connecting rod fixing plate is installed at the opening end of the fixed base. The connecting rod fixing plate is connected to the mounting plate through multiple support columns. The variable structure mechanism is installed between the mounting plate and the connecting rod fixing plate, and a hardware fixing plate is installed on the side of the mounting plate away from the support column.

[0009] The variable structure mechanism includes a servo motor installed on the mounting plate and a transmission mechanism installed on the connecting rod fixing plate. The servo motor is connected to the grasping drive system through the transmission mechanism;

[0010] The transmission mechanism includes a double-blade transmission rod, a slider, a connecting rod mechanism, a transmission shaft, and a transmission block. The double-blade transmission rod is installed at the driving end of the servo motor. Sliders are slidably connected to both ends of the double-blade transmission rod. Each slider is connected to a corresponding transmission shaft through multiple groups of connecting rod mechanisms. Multiple transmission shafts are connected to the finger plate air suction and exhaust type grasping device through corresponding transmission blocks.

[0011] The middle of the double-blade transmission rod is fixedly connected to the driving end of the servo motor. Arc-shaped sliding grooves arranged in central symmetry are respectively provided at both ends of the double-blade transmission rod. A connecting column is slidably connected in each arc-shaped sliding groove and is connected to the slider through the connecting column; A strip-shaped sliding groove for slidably connecting the two sliders is provided in the middle of the connecting rod fixing plate; The connecting rod mechanism includes a driving rod rotatably connected to the slider and a driven rod rotatably connected to the driving rod. The driven rod is fixedly connected to the corresponding transmission shaft. The connecting rod mechanism and the transmission shaft are respectively installed on both sides of the connecting rod fixing plate.

[0012] The finger plate suction and release type grasping device includes finger fixing parts, soft fingers, a micro air pump and air pipes. A plurality of the finger fixing parts are provided and are respectively installed on corresponding transmission blocks. Each finger fixing part is connected to a soft finger. The soft finger is connected to the micro air pump through an air pipe to realize the adaptive grasping of fruits by the soft finger.

[0013] The finger fixing part includes a constraint base fixed on the transmission block and a clamping block clamped and fixed therein. The soft finger is nested and fixed in a clamping space between the constraint base and the clamping block.

[0014] The soft finger includes a strain layer, a restraint layer and flexible suction cups. The strain layer and the restraint layer are fixedly connected. The strain layer is arranged on the side away from the fruit, and the restraint layer is fixedly connected to the side of the strain layer close to the fruit in a fitting manner. A plurality of flat arc-shaped structures are arranged at intervals along the length direction on the side of the strain layer away from the restraint layer. A plurality of arc-shaped air channels are arranged conformally inside the strain layer. The plurality of arc-shaped air channels are communicated to form a sealed air channel. One end of the sealed air channel is connected to an air port of the micro air pump through an air pipe Ⅰ. A plurality of connected tubular air channels are arranged in the restraint layer. A plurality of flexible suction cups are installed on the side of the restraint layer away from the strain layer. The plurality of flexible suction cups are communicated with the corresponding tubular air channels. After the plurality of tubular air channels converge, they are connected to another air port of the micro air pump through an air pipe Ⅱ.

[0015] The proximity trigger system includes a color sensor and a ranging sensor installed on the side facing the fruit on the hardware fixing plate.

[0016] The grasping drive system includes a motor driver, an embedded picking controller and a power supply installed at one end of the fixed base away from the connection base. The motor driver includes a motor driver Ⅰ for driving the variable structure mechanism to act and a motor driver Ⅱ for driving the finger plate suction and release type grasping device to act. The proximity trigger system and the image acquisition device are communicatively connected to the motor driver and the robotic arm through the embedded picking controller.

[0017] The control method of the finger plate suction and release air type proximity trigger variable structure flexible picking manipulator includes the following steps:

[0018] 1) System initialization, calibrating the image acquisition device and the robotic arm.

[0019] 2) The image acquisition device collects information on the fruit image within the field of view driven by the robotic arm.

[0020] 3) The collected fruit image is uploaded to the grasping drive system. The grasping drive system controls the deformation of the variable structure mechanism according to the shape and size of the fruit.

[0021] 4) Determine the spatial position coordinates and grasping poses of the fruits;

[0022] 5) Plan the fruit picking path;

[0023] 6) Plan the approaching and triggering finger plate suction and release type grasping actions;

[0024] 7) The robotic arm drives the picking manipulator to grasp the target fruits in sequence according to the methods of step 4), step 5), and step 6) above;

[0025] 8) Determine whether the picking manipulator has grasped the target fruits. If not, continue to execute the instructions sequentially starting from step 3); If the target fruits have been grasped, the robotic arm drives the picking manipulator to pick the fruits by the pulling method and place the fruits in the storage area;

[0026] 9) Determine whether all the fruits have been completely picked. If not, continue to execute the instructions sequentially starting from step 3); If all the fruits have been completely picked, determine whether a sufficient amount of fruits has been obtained. If not, continue to execute the instructions sequentially starting from step 2). If a sufficient amount of fruits has been obtained, end the action and the robotic arm returns to the initial position.

[0027] Compared with the current technology, the present invention has the following advantages:

[0028] 1) The present invention provides a finger plate suction and release pneumatic approaching and triggering variable structure flexible picking manipulator. The end of the manipulator realizes adaptive grasping through the suction and release of air by the flexible finger plate, thereby realizing passive grasping force control and reducing the damage to the fruits during the picking process.

[0029] 2) By designing a variable structure mechanism and an approaching and triggering system, the present invention changes the jaw structure for effective grasping for fruits of different shapes, and realizes approaching type grasping trigger by detecting the color and the target distance, and proposes a grasping control of variable structure first and then approaching and triggering, improving the accuracy of fruit picking.

[0030] 3) By designing an integrated grasping drive system for fruit recognition, positioning, variable grasping structure, approaching type trigger, and finger plate suction and release type grasping, the present invention realizes the function of automatic picking, with high automation degree, convenient operation, and strong popularization. Description of the Drawings

[0031] Figure 1 It is a schematic structural diagram of the finger plate suction and release pneumatic approaching and triggering variable structure flexible picking manipulator of the present invention installed on the robotic arm;

[0032] Figure 2 It is a schematic overall structural diagram of the finger plate suction and release pneumatic approaching and triggering variable structure flexible picking manipulator of the present invention;

[0033] Figure 3 This is a schematic structural diagram of the cylindrical fruit grasping of the finger plate air-sucking and air-releasing proximity-triggered variable-structure flexible picking manipulator of the present invention;

[0034] Figure 4 This is a schematic structural diagram of the spherical fruit grasping of the finger plate air-sucking and air-releasing proximity-triggered variable-structure flexible picking manipulator of the present invention;

[0035] Figure 5 This is a schematic structural diagram of the variable-structure mechanism of the present invention;

[0036] Figure 6 This is a schematic structural diagram of the transmission mechanism in the variable-structure mechanism of the present invention;

[0037] Figure 7 is Figure 5 a schematic structural diagram of the double-blade transmission rod in;

[0038] Figure 8 This is an exploded view of the soft finger in the finger plate air-sucking and air-releasing grasping device of the present invention;

[0039] Figure 9 This is a full sectional view of the soft finger in the finger plate air-sucking and air-releasing grasping device of the present invention;

[0040] Figure 10 This is a schematic installation structure diagram of the micro air pump in the finger plate air-sucking and air-releasing grasping device of the present invention;

[0041] Figure 11 This is a schematic installation structure diagram of the proximity-triggering system of the present invention;

[0042] Figure 12 This is a schematic installation structure diagram of the grasping drive system of the present invention;

[0043] Figure 13 This is a flow chart of the control method of the finger plate air-sucking and air-releasing proximity-triggered variable-structure flexible picking manipulator of the present invention;

[0044] The markings in the above figures are as follows: 1. Support, 1-1. Fixed base, 1-2. Connecting base, 1-21. Link fixing plate, 1-211. Strip-shaped sliding groove, 1-22. Support column, 1-23. Mounting plate, 1-3. Hardware fixing plate, 2. Gripping drive system, 2-1. Motor driver, 2-2. Embedded picking controller, 2-3. Power supply, 3. Robot arm, 4. Image acquisition device, 5. Variable structure mechanism, 5-1. Servo, 5-2. Transmission mechanism, 5-21. Double-blade transmission rod, 5-211. Arc-shaped sliding groove, 5-22. Slide block, 5-23. Link mechanism, 5-24. Transmission shaft, 5-25. Transmission block, 6. Finger plate suction and release gripping device, 6-1. Finger fixing part, 6-11. Constraint base, 6-12. Clamping block, 6-2. Soft finger, 6-21. Strain layer, 6-211. Flat arc-shaped structure, 6-212. Arc-shaped air duct, 6-22. Limiting layer, 6-221. Tubular air duct, 6-23. Flexible suction cup, 6-3. Micro air pump, 6-4. Air pipe Ⅰ, 6-5. Air pipe Ⅱ, 7. Proximity trigger system, 7-1. Color sensor, 7-2. Range sensor. Detailed implementation manners

[0045] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. The following embodiments are used to illustrate the present invention but are not intended to limit the scope of the present invention.

[0046] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "upper", "lower", "front", "rear", "left", "right", "vertical", "inner", "outer", etc. is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention.

[0047] In the description of the present invention, it should be noted that unless otherwise clearly specified and limited, the terms "installation", "connection", and "connection" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0048] The present invention will be further described below with reference to the accompanying drawings. As Figures 1-4As shown in the figure, the present invention provides a finger plate air-suction and air-release type proximity-triggered variable structure flexible picking manipulator. The picking manipulator includes a bracket 1 connected to a robotic arm 3 equipped with an image acquisition device 4. One end of the bracket 1 is installed with a grasping drive system 2. The image acquisition device 4 therein can be set as a depth camera for acquiring images in the field of view to find the position of fruits. A variable structure mechanism 5 is installed in the middle of the bracket 1 and is connected to a finger plate air-suction and air-release type grasping device 6 through the variable structure mechanism 5. The deformation of the variable structure mechanism 5 can drive the finger plate air-suction and air-release type grasping device 6 to complete the adaptive grasping of fruits with different shapes and sizes, realizing passive grasping force control and reducing the damage to fruits during the picking process. The other end of the bracket 1 is installed with a proximity trigger system 7, which realizes proximity grasping trigger by detecting some signals (such as detecting fruit color and target distance) to grasp and pick ripe fruits, improving the accuracy of fruit picking. The proximity trigger system 7 and the image acquisition device 4 are communicatively connected to the variable structure mechanism 5 and the finger plate air-suction and air-release type grasping device 6 through the grasping drive system 2 to realize the adaptive automatic picking of fruits, with high automation degree, convenient operation and strong popularization.

[0049] Specifically, as Figure 2 shown, the bracket 1 includes a fixed base 1-1, a connecting base 1-2 and a hardware fixing plate 1-3. One end of the fixed base 1-1 is installed with a grasping drive system 2, and the grasping drive system 2 is connected to the robotic arm 3 through an end flange. The robotic arm 3 can be set as a six-degree-of-freedom robotic arm 3. The other end of the fixed base 1-1 is connected to the hardware fixing plate 1-3 through the connecting base 1-2. The variable structure mechanism 5 is installed on the connecting base 1-2, and the proximity trigger system 7 is installed on the hardware fixing plate 1-3. The above settings make the installation of the proximity trigger system 7, the grasping drive system 2 and the variable structure mechanism 5 more reasonable, the structure layout more compact, reducing the volume of the entire manipulator and making the operation more flexible.

[0050] One end of the fixed base 1-1 is open and set as a box structure, providing an installation space for other components and making the structure design more compact. The connecting base 1-2 includes a connecting rod fixing plate 1-21, a support column 1-22 and a mounting plate 1-23. The connecting rod fixing plate 1-21 is installed at the open end of the fixed base 1-1. The circumferential direction of the connecting rod fixing plate 1-21 is connected to the mounting plate 1-23 through a plurality of support columns 1-22. An installation space is formed between the mounting plate 1-23 and the connecting rod fixing plate 1-2. The variable structure mechanism 5 is installed in this installation space. A hardware fixing plate 1-3 is installed on the side of the mounting plate 1-23 away from the support column 1-22. The hardware fixing plate 1-3 is connected to the variable structure mechanism 5 through a universal joint and is indirectly fixed on the connecting base, and is not affected by the movement of the variable structure mechanism 5, and is used to fix the proximity trigger system 7.

[0051] Specifically, as Figure 2, Figures 5-7 As shown in Figures 5-7 , the variable structure mechanism 5 therein includes a servo 5-1 mounted on the mounting plate 1-23 and a transmission mechanism 5-2 mounted on the link fixing plate 1-2. The servo 5-1 is connected to the finger plate suction and release grasping device 6 through the transmission mechanism 5-2. The rotational movement of the driving shaft of the servo 5-1 is converted into the adaptive grasping change of the finger plate suction and release grasping device 6 through the transmission of the transmission mechanism 5-2, so as to achieve the adaptive grasping of fruits with different shapes and sizes.

[0052] The transmission mechanism 5-2 therein includes a double-blade transmission rod 5-21, a slider 5-22, a linkage mechanism 5-23, a transmission shaft 5-24, and a transmission block 5-25. The double-blade transmission rod 5-21 is installed at the driving end of the steering gear 5-1. Sliders 5-22 are slidably connected to both ends of the double-blade transmission rod 5-21. Each slider 5-22 is connected to the corresponding transmission shaft 5-24 through multiple groups of linkage mechanisms 5-23. Two groups of linkage mechanisms 5-23 are shown in the figure. The two groups of linkage mechanisms 5-23 connected to each slider 5-22 are symmetrically arranged with the sliding direction of the slider 5-22 as the axis of symmetry. The linkage mechanisms 5-23 on the two sliders 5-22 are symmetrically arranged with the direction perpendicular to the sliding direction of the slider 5-22 as the axis of symmetry. Of course, more than two groups of the above-mentioned linkage mechanisms 5-23 can also be set as required. Each group of linkage mechanisms 5-23 is connected to a transmission shaft 5-24. Each transmission shaft 5-24 is connected to the finger plate suction and release grasping device 6 through a transmission block 5-25. The middle of the double-blade transmission rod 5-21 is fixedly connected to the driving end of the steering gear 5-1. Arc-shaped sliding grooves 5-211 arranged in central symmetry are respectively provided at both ends of the double-blade transmission rod 5-21. If there are 4 groups of linkage mechanisms 5-23, two groups of arc-shaped sliding grooves 5-211 arranged in central symmetry are arranged on the double-blade transmission rod 5-21. A connecting column is slidably connected in each arc-shaped sliding groove 5-211 and is connected to the slider 5-22 through the connecting column. A strip-shaped sliding groove 1-211 for slidably connecting with the two sliders 5-22 is provided in the middle of the connecting rod fixing plate 1-2. A rectangular hole is provided in the middle of the connecting rod fixing plate 1-2. Strip-shaped plates are respectively fixed on the inner and outer sides of the two pairs of sides of the rectangular hole. A strip-shaped sliding groove 1-211 for slidingly cooperating with the slider 5-22 is formed between the two strip-shaped plates. The linkage mechanism 5-23 therein includes a driving rod rotatably connected to the slider 5-22 and a driven rod rotatably connected to the driving rod. The driven rod is fixedly connected to the corresponding transmission shaft 5-24. The linkage mechanism 5-23 and the transmission shaft 5-24 are respectively installed on both sides of the connecting rod fixing plate 1-2, and the transmission shaft 5-24 is vertically installed between the connecting rod fixing plate 1-2 and the mounting plate 1-23.During the process of the steering gear 5-1 driving the drive shaft to rotate and driving the double-blade transmission rod 5-21 to rotate, since two sets of arc-shaped chutes 5-211 arranged in central symmetry are arranged on the double-blade transmission rod 5-21, the two sliders 5-22 will slide relative to or towards each other along the strip-shaped chute 1-211 under the constraint of the strip-shaped chute 1-211, thereby driving the link mechanisms 5-23 on the two sliders 5-22 to rotate in opposite directions, and thus driving the transmission shafts 5-24 on the sides of the two sliders 5-22 to rotate in opposite directions around their central axes. Since the transmission block 5-25 is fixed at the connection between the transmission shaft 5-24 and the mounting plate 1-23, the transmission block 5-25 fixedly connected to the transmission shaft 5-24 is driven to rotate, which is used for the final motion output of the variable structure mechanism 5, and the finger plate suction and release type grasping device 6 connected to the transmission block 5-25 can be expanded outwards or contracted inwards, and the adaptive grasping of fruits with different shapes and sizes can be realized.

[0053] Specifically, as Figure 2 , Figures 8-10 shown, the finger plate suction and release type grasping device 6 therein includes finger fixing members 6-1, soft fingers 6-2, a micro air pump 6-3 and air pipes. There are multiple finger fixing members 6-1, which are respectively installed on the corresponding transmission blocks 5-25. Each finger fixing member 6-1 is connected to a soft finger 6-2, and the soft finger 6-2 is connected to the micro air pump 6-3 through an air pipe to realize the adaptive grasping of fruits by the soft finger.

[0054] The finger fixing member 6-1 therein includes a constraint base 6-11 fixed on the transmission block 5-25 and a clamping block 6-12 clamped and fixed therein. The soft finger 6-2 is nested and fixed in the clamping space between the constraint base 6-11 and the clamping block 6-12, realizing the reliable positioning and installation of the soft finger 6-2.

[0055] The soft finger 6-2 therein includes a strain layer 6-21, a limiting layer 6-22 and a flexible suction cup 6-23. The strain layer 6-21 and the limiting layer 6-22 are fixedly connected. The strain layer 6-21 is arranged on the side away from the fruit, and the limiting layer 6-22 is fixedly connected to the side of the strain layer 6-21 close to the fruit. On the side of the strain layer 6-21 away from the limiting layer 6-22, a plurality of flat arc-shaped structures 6-211 are arranged at intervals along its length direction. An included angle is provided at the top of the flat arc-shaped structure 6-211, and the upper half of the air cavity is removed. The overall height is only half of that of a common air cavity, but still meets the bending performance requirements of the soft finger 6-2 and saves production materials. A plurality of arc-shaped air channels 6-212 are arranged conformally inside the strain layer 6-21. The plurality of arc-shaped air channels 6-212 are connected to form a sealed air channel. One end of the sealed air channel is connected to an air port of a micro air pump 6-3 through an air pipe I 6-4. A plurality of connected tubular air channels 6-221 are arranged inside the limiting layer 6-22. A plurality of flexible suction cups 6-23 are installed on the side of the limiting layer 6-22 away from the strain layer 6-21. The plurality of flexible suction cups 6-23 are connected to the corresponding tubular air channels 6-221. After the plurality of tubular air channels 6-221 converge, they are connected to another air port of the micro air pump 6-3 through an air pipe II 6-5. The air pipe I 6-4 and the air pipe II 6-5 are rubber hoses. The two ports of the air pipe I 6-4 are respectively connected to the air inlet (or air outlet) of the micro air pump 6-3 and the air port of the limiting layer 6-22, and the two ports of the air pipe II 6-5 are respectively connected to the air outlet (or air inlet) of the micro air pump 6-3 and the air port of the strain layer 6-21 for transmitting gas. In the present invention, 4 soft fingers 6-2 are provided. Of course, more than 4 can be provided according to needs to ensure the stability of fruit grasping.

[0056] Driven by the micro air pump 6-3, the flexible suction cup 6-23 on the limiting layer 6-22 sucks air from the outside. Through the air pipe I 6-4, the gas enters the air inlet of the micro air pump 6-3 and is discharged from the air outlet of the micro air pump 6-3. Then it enters the sealed air channel of the strain layer 6-21 through the air pipe II 6-5, so that the soft finger 6-2 bends, and vice versa. The strain layer 6-21 is located on the dorsal side of the soft finger 6-2, and the limiting layer 6-22 is located on the ventral side of the soft finger 6-2. By inflating the strain layer 6-21 through the air port, the arc-shaped air channel 6-212 expands, so that the soft finger 6-2 can bend towards the ventral side. By sucking air from the air port of the strain layer 6-21, the arc-shaped air cavity is compressed, so that the soft finger 6-2 bends towards the dorsal side. Moreover, the air channel of the flexible suction cup 6-23 is communicated with the outside, and the flexible suction cup 6-23 is communicated with the air port of the limiting layer 6-22 through the air channel, which is used to exchange gas with the outside to make the strain layer 6-21 bend. The flexible suction cup 6-23 can be adaptively attached according to the types of fruits to be picked. In this article, a flat suction cup is selected. When the flexible suction cup 6-23 is fully attached to the fruit surface, relying on the micro air pump 6-3, the air cavities of the strain layer 6-21 and the limiting layer 6-22 of the soft finger 6-2 achieve dynamic air pressure balance, so as to realize the adaptive suction and clamping picking of different fruits by the air suction and release of the flexible suction cup 6-23. In addition, the micro air pump 6-3 is installed in the fixed base 1-1. A plurality of through grooves for the corresponding air pipe I 6-4 and air pipe II 6-5 to penetrate are arranged circumferentially on the fixed base 1-1. One or more micro air pumps 6-3 can be set. When one micro air pump 6-3 is set, the air inlet and air outlet of the micro air pump 6-3 can be respectively connected to the corresponding air pipes through a plurality of three-way pipes; of course, multiple micro air pumps 6-3 can also be set, and each micro air pump 6-3 controls one soft finger 6-2.

[0057] Specifically, as Figure 2 and Figure 11 shown, the proximity trigger system 7 includes a color sensor 7-1 and a ranging sensor 7-2 installed on the side of the hardware fixing plate 1-3 facing the fruit. The color sensor 7-1 is installed in the square through groove of the hardware fixing plate 1-3. Its sensing head penetrates and does not exceed the square through groove and faces the fruit, realizes the detection of the maturity of the fruit through color recognition, and can supplement light to the fruits within the viewing angle range to facilitate triggering picking; the laser ranging sensor 7-2 is installed in the trapezoidal through groove of the hardware fixing plate 1-3. Its lens penetrates and does not exceed the trapezoidal through groove and faces the fruit, and can measure the distance from the center of the mechanical palm to the fruit by using the time-of-flight method (TOF). Linked with the color sensor 7-1, it can realize the proximity trigger picking of ripe fruits.

[0058] Specifically, as Figure 2 and Figure 12As shown in the figure, the grasping drive system 2 includes a motor driver 2-1, an embedded picking controller 2-2, and a power supply 2-3 installed at one end of the fixed base 1-1 away from the connecting base. The motor driver 2-1 includes a motor driver 2-1Ⅰ for driving the servo 5-1 in the variable structure mechanism 5 to act and a motor driver 2-1Ⅱ for driving the micro air pump 6-3 in the finger plate suction and release grasping device 6 to act. The embedded picking controller 2-2 can be implemented based on a single-chip microcomputer system, a microcontroller system, or an embedded computer system. The power supply 2-3 supplies power to the servo 5-1, the micro air pump 6-3, the depth camera, the color sensor 7-1, the ranging sensor 7-2, and the embedded picking controller 2-2. The proximity trigger system 7 (color sensor 7-1, ranging sensor 7-2) and the image acquisition device 4 are communicatively connected to the motor driver 2-1 and the robotic arm 3 through the embedded picking controller 2-2, and can realize the integrated control of the identification, positioning, ranging, variable grasping structure, and picking of different fruits.

[0059] The installation method of the picking manipulator of the present invention is as follows: The picking manipulator is installed on the robotic arm 3 with the image acquisition device 4 through its end flange, and then the embedded picking controller 2-2 is connected to the color sensor 7-1, the ranging sensor 7-2, the depth camera, the motor driver 2-1, and the robotic arm 3 through signal lines, so as to realize the integrated control of fruit identification, positioning, variable grasping structure, proximity triggering, and finger plate suction and release gas grasping.

[0060] As Figure 13 shown, the control method of the above-mentioned finger plate suction and release gas proximity trigger variable structure flexible picking manipulator includes the following steps:

[0061] 1) System initialization, calibrate the image acquisition device 4 and the robotic arm 3. Calibrate the depth camera on the robotic arm 3 based on the Zhang Zhengyou calibration method, and then perform hand-eye calibration on the robotic arm 3 and the depth camera based on the hand-eye calibration method with the eye outside the hand.

[0062] 2) The image acquisition device 4 collects information on the fruit image within the field of view driven by the robotic arm 3. The embedded picking controller 2-2 controls the operation of the robotic arm 3 to drive the picking manipulator to move, so that the image acquisition device 4 can clearly capture the fruits within the viewing angle range. Then, the embedded picking controller 2-2 controls the image acquisition device 4 to collect fruit image information, and uploads the fruit image information to the embedded picking controller 2-2 after the collection is completed.

[0063] 3) The collected fruit images are uploaded to the grasping drive system 2, and the grasping drive system 2 controls the deformation of the variable structure mechanism 5 according to the shape and size of the fruit. After receiving the image information uploaded by the image acquisition device 4, the embedded picking controller 2-2 calls the database information to identify and determine the specific shape of the target fruit. Then, the embedded picking controller 2-2 controls the motor driver 2-1Ⅰ to drive the servo 5-1 in the variable structure mechanism 5 to rotate at a fixed angle. The variable structure mechanism 5 operates with the rotation of the servo 5-1, so that the finger fixing part 6-1 at the end of the transmission block 5-25 installed in the variable structure mechanism 5 drives the soft finger 6-2 to rotate at a fixed angle, so that the bottom surface of the flexible suction cup 6-23 of the soft finger 6-2 is parallel to the surface contour of the target fruit to the greatest extent, completing the grasping structure deformation operation.

[0064] 4) Determine the spatial position coordinates and grasping pose of the fruit. The embedded picking controller 2-2 starts to detect the characterization feature information (such as perimeter, area, attitude information, etc.) and position information of the fruit.

[0065] The specific method is as follows: First, preprocess the fruit image information through the neural network algorithm to obtain the image point cloud data;

[0066] Then, obtain the detection frame and mask information of the fruit through the object detection and instance segmentation algorithms, then segment the image according to the detection frame, and then detect the centroid of the fruit through the centroid algorithm based on the mask information in the area after the image is segmented. Then, introduce the centroid matching algorithm to classify the centroid of the fruit, and determine the attitude of the fruit through the position relationship and belonging relationship between the centroids, realizing the determination of the two-dimensional centroid coordinates and attitude of the fruit;

[0067] Then, the embedded picking controller 2-2 calculates and converts the two-dimensional centroid coordinates of the fruit in the image acquisition device 4 into the three-dimensional centroid coordinates relative to the optical center of the image acquisition device 4 (depth camera), and then converts them into the world coordinates relative to the base of the robotic arm 3 to obtain the three-dimensional spatial position coordinates of the fruit;

[0068] a. The conversion process of converting two-dimensional centroid coordinates to three-dimensional centroid coordinates is as follows:

[0069] Assume that the internal parameter matrix of the image acquisition device (depth camera) is

[0070]

[0071] In the formula, f x 、f y are the focal lengths of the depth camera in the x and y directions respectively, and (c x , c y ) represents the principal point coordinates of the image.

[0072] Convert the two-dimensional centroid coordinates (u, v) and the corresponding depth value d into the three-dimensional centroid coordinates (X c , Y c , Z c ) in the depth camera coordinate system. The conversion formula is as follows:

[0073]

[0074] In matrix form, it is expressed as:

[0075]

[0076] b. The conversion process of the three-dimensional centroid coordinates into world coordinates is as follows:

[0077] Assume that the pose of the depth camera relative to the base of the robotic arm can be represented by a 4×4 homogeneous transformation matrix T:

[0078]

[0079] In the formula, R is a 3×3 rotation matrix representing the rotation of the camera coordinate system relative to the base coordinate system of the robotic arm; t is a 3×1 translation vector representing the translation of the origin of the camera coordinate system relative to the origin of the base coordinate system of the robotic arm.

[0080] Convert the three-dimensional centroid coordinates (X c , Y c , Z c ) in the depth camera coordinate system into world coordinates (X, Y, Z):

[0081]

[0082] After expansion, it is obtained:

[0083]

[0084] Finally, according to the determined fruit pose, calculate the poses of the robotic arm 3 and the picking manipulator when grasping the fruit, and realize the determination of the grasping pose of the fruit.

[0085] 5) Plan the fruit picking path. Combining the spatial position and grasping pose of the fruit determined by the above operations, the embedded picking controller 2-2 performs inverse kinematics solution based on the current coordinate values and grasping coordinate values of the robotic arm 3 and the picking manipulator, and interpolates to obtain the motion trajectories of the robotic arm 3 and the picking manipulator, and obtains the fruit picking path, completing the picking path planning of the target fruit.

[0086] 6) Plan to approach and trigger the finger plate suction and release grasping action. When picking fruits according to the planned path, during the process of the embedded picking controller 2-2 controlling the robotic arm 3 to drive the picking manipulator to approach the target fruit directly and slowly, the following operations are carried out quickly. First, the embedded picking controller 2-2 controls the color sensor 7-1 to detect the color of the target fruit, and uploads the detected color information of the target fruit to the embedded picking controller 2-2 for fruit maturity detection; second, if it is detected that the target fruit is not ripe, stop and pick the next fruit. If the target fruit is ripe, the embedded picking controller 2-2 controls the distance sensor 7-2 to measure the minimum distance from the center of the picking manipulator's palm to the target fruit; finally, upload the distance information to the embedded picking controller 2-2. When the appropriate picking distance is reached, the picking manipulator stops moving, and the embedded picking controller 2-2 controls the micro air pump 6-3 to suck and release air, causing the soft finger 6-2 to bend. When the flexible suction cup 6-23 is fully attached to the surface of the target fruit, the micro air pump 6-3 is in an idle state, completing the approach trigger finger plate suction and release air grasping of the target fruit.

[0087] 7) The robotic arm 3 drives the picking manipulator to grasp the target fruit in turn according to the methods in steps 4), 5), and 6) above. Specifically, the embedded picking controller 2-2 automatically issues communication commands. After the overall control plan is completed, a fruit is randomly selected. The embedded picking controller 2-2 sends the various information obtained to each device in the form of operation commands according to the steps. Specifically, after receiving the operation command of the embedded picking controller 2-2, the variable structure mechanism 5 rotates the servo 5-1 at a fixed angle to deform it into the corresponding grasping structure for the fruit, and performs the following operations according to the operation command for the fruit: First, the embedded picking controller 2-2 controls the robotic arm 3 to drive the picking manipulator to move along the picking path, so that the picking manipulator moves to the front of the centroid of the fruit, with a certain distance from the fruit; then, the embedded picking controller 2-2 controls the end of the robotic arm 3 to rotate, driving the picking manipulator to rotate to the grasping pose of the fruit; second, the embedded picking controller 2-2 controls the robotic arm 3 to drive the picking manipulator to approach the fruit directly and slowly. After detecting that the fruit is ripe, slowly approach until the suction and clamping action of the fruit is triggered and then stop moving; finally, when the grasping of the fruit tends to be stable, the embedded picking controller 2-2 controls the robotic arm 3 to drive the picking manipulator to move along the picking path to grasp the target fruit.

[0088] 8) Judge whether the picking manipulator has grasped the target fruit. If not, continue to execute the instructions sequentially from step 3); if the target fruit has been grasped, the robotic arm 3 drives the picking manipulator to pick the fruit by the pulling method and place the fruit in the storage area.

[0089] 9) Determine whether all the fruits have been completely picked. If not, continue to sequentially execute the instructions starting from step 3). If all the fruits have been completely picked, then determine whether a sufficient amount of fruits has been obtained. If not, continue to sequentially execute the instructions starting from step 2). If a sufficient amount of fruits has been obtained, end the operation and the robotic arm 3 returns to its initial position. The above determination method can be manually operated; or the required number of pickings can be pre-entered into the embedded picking controller 2-2. Each time the picking manipulator completes a picking, a count is made until the set number is reached, indicating that all picking work is completed; or it can be determined whether the picking is completed based on whether there are still ripe fruits in the field of view.

[0090] In summary, the finger plate air suction and release type proximity-triggered variable structure flexible picking manipulator and its control method provided by the present invention achieve finger plate air suction and release type grasping and passive grasping force control, achieve proximity grasping trigger, and change the grasping structure for different shaped fruits to achieve effective grasping, improve the accuracy of fruit picking, realize automated picking, are easy to operate, and have strong popularization.

[0091] As described above, only some principles of the present invention are illustrated by diagrams. This specification is not intended to limit the invention to the specific structures and applicable scopes shown and described. Therefore, all possible corresponding modifications and equivalents that can be utilized belong to the scope of the patent applied for by the invention.

Claims

1. A finger plate suction and deflation type proximity trigger variable structure flexible picking manipulator, characterized in that: It includes a bracket connected to a robotic arm equipped with an image acquisition device. One end of the bracket is installed with a grasping drive system. A variable structure mechanism is installed in the middle of the bracket and is connected to a finger plate suction and release grasping device through the variable structure mechanism. The other end of the bracket is installed with a proximity trigger system. The proximity trigger system and the image acquisition device are communicatively connected through the grasping drive system to the variable structure mechanism and the finger plate suction and release grasping device to achieve adaptive automated picking of fruits.

2. The finger plate air suction and exhaust type proximity trigger variable structure flexible picking manipulator according to claim 1, characterized in that: The bracket includes a fixed base. One end of the fixed base is installed with the grasping drive system. The other end of the fixed base is connected to a hardware fixing plate through a connecting base. The variable structure mechanism is installed on the connecting base. The proximity trigger system is installed on the hardware fixing plate.

3. The finger plate air suction and exhaust type proximity trigger variable structure flexible picking manipulator according to claim 2, wherein: One end of the fixed base is open. The connecting base includes a connecting rod fixing plate installed at the open end of the fixed base. The connecting rod fixing plate is connected to a mounting plate through multiple support columns. The variable structure mechanism is installed between the mounting plate and the connecting rod fixing plate. A hardware fixing plate is installed on the side of the mounting plate away from the support columns.

4. The finger plate air suction and release type proximity trigger variable structure flexible picking manipulator according to claim 3, wherein: The variable structure mechanism includes a servo motor installed on the mounting plate and a transmission mechanism installed on the connecting rod fixing plate. The servo motor is connected to the finger plate suction and release grasping device through the transmission mechanism. The transmission mechanism includes a double-blade transmission rod installed at the driving end of the servo motor. Sliders are slidably connected to both ends of the double-blade transmission rod. Each slider is connected to a corresponding transmission shaft through multiple sets of connecting rod mechanisms. Multiple transmission shafts are connected to the finger plate suction and release grasping device through corresponding transmission blocks.

5. The finger plate air suction and exhaust type proximity trigger variable structure flexible picking manipulator according to claim 4, characterized in that: The middle of the double-blade transmission rod is fixedly connected to the driving end of the servo motor. Arc-shaped chutes arranged in central symmetry are respectively provided at both ends of the double-blade transmission rod. A connecting column is slidably connected in each arc-shaped chute and is connected to the slider through the connecting column. A strip-shaped chute for slidably connecting the two sliders is provided in the middle of the connecting rod fixing plate. The connecting rod mechanism includes a driving rod rotatably connected to the slider and a driven rod rotatably connected to the driving rod. The driven rod is fixedly connected to the corresponding transmission shaft. The connecting rod mechanism and the transmission shaft are respectively installed on both sides of the connecting rod fixing plate.

6. The finger plate air suction and exhaust type proximity trigger variable structure flexible picking manipulator according to claim 4, characterized in that: The finger plate suction and release grasping device includes finger fixing parts installed on each transmission block and soft fingers connected to each finger fixing part. The soft fingers are connected to a micro air pump through air pipes to achieve adaptive grasping of fruits by the soft fingers.

7. The finger plate air suction and exhaust type proximity trigger variable structure flexible picking manipulator according to claim 6, characterized in that: The finger fixing part includes a constraint base fixed on the transmission block and a clamping block clamped and fixed therein. The soft finger is nested and fixed in the clamping space between the constraint base and the clamping block.

8. The finger plate air suction and exhaust type proximity trigger variable structure flexible picking manipulator according to claim 7, characterized in that: The soft finger includes a strain layer and a limiting layer fixedly connected. The strain layer is arranged on the side away from the fruit. The limiting layer is fixedly connected to the strain layer on the side close to the fruit. On one side of the strain layer away from the confinement layer, a plurality of flat circular arc structures are arranged at intervals along its length direction. A plurality of circular arc air channels are arranged conformally inside the strain layer, and the plurality of circular arc air channels are connected to form a sealed air channel. One end of the sealed air channel is connected to an air port of the micro air pump through a trachea I; a plurality of connected tubular air channels are arranged inside the confinement layer. A plurality of flexible suction cups are installed on one side of the confinement layer away from the strain layer, and the plurality of flexible suction cups are connected to the corresponding tubular air channels. After the plurality of tubular air channels converge, they are connected to another air port of the micro air pump through a trachea II.

9. The finger plate air suction and exhaust type proximity trigger variable structure flexible picking manipulator according to claim 2, wherein: The proximity trigger system includes a color sensor and a ranging sensor installed on the side facing the fruit on the hardware fixing plate; The grasping drive system includes a motor driver, an embedded picking controller, and a power supply installed at one end of the fixed base away from the connection base. The motor driver includes a motor driver I for driving the variable structure mechanism to act and a motor driver II for driving the finger plate suction and release grasping device to act. The proximity trigger system and the image acquisition device are communicatively connected to the motor driver and the robotic arm through the embedded picking controller, and can realize the integrated control of the identification, positioning, ranging, variable grasping structure, and picking of different fruits.

10. A control method for a finger plate air suction and exhaust type proximity trigger variable structure flexible picking manipulator as described in any one of claims 1-9, characterized in that, It includes the following steps: 1) System initialization, calibrating the image acquisition device and the robotic arm; 2) The image acquisition device collects information on the fruit images within the field of view driven by the robotic arm; 3) The collected fruit images are uploaded to the grasping drive system, and the grasping drive system controls the deformation of the variable structure mechanism according to the shape and size of the fruit; 4) Determine the spatial position coordinates and grasping poses of the fruit; 5) Plan the fruit picking path; 6) Plan the proximity trigger finger plate suction and release grasping action; 7) The robotic arm drives the picking manipulator to grasp the target fruit in sequence according to the methods in steps 4), 5), and 6) above; 8) Judge whether the picking manipulator has grasped the target fruit. If not, continue to execute the instructions sequentially from step 3); if the target fruit has been grasped, the robotic arm drives the picking manipulator to pick the fruit by the pulling method and place the fruit in the storage area; 9) Judge whether all the fruits have been completely picked. If not, continue to execute the instructions sequentially from step 3); if all the fruits have been completely picked, judge whether a sufficient amount of fruits have been obtained. If not, continue to execute the instructions sequentially from step 2). If a sufficient amount of fruits have been obtained, end the action and the robotic arm returns to the initial position.

Images