Apple picking clamping jaw integrating visual identification and independent driving mechanism

Abstract

The invention belongs to the technical field of agricultural robots, and particularly relates to an apple picking clamping jaw integrating visual recognition and an independent driving mechanism, which comprises a plurality of picking fingers distributed on one side of a shell in a circumferential array, and a pressure detection element arranged on each picking finger, the pressure detection element is used for acquiring the contact pressure between the picking finger and the apple in real time; the air blowing assembly is used for blowing away leaves around the apples; the plurality of depth cameras are uniformly distributed on the shell and are used for shooting the apples and each picking finger when the leaves are blown away so as to obtain position pictures of the apples and each picking finger; and the control module is used for acquiring distance information between the apples and each picking finger according to the position pictures, determining the initial moving stroke of each picking finger through the distance information, and adjusting the secondary moving stroke of the picking fingers according to the pressure information acquired by each pressure detection element in real time. Therefore, the clamping force of each picking finger to the apple is balanced.

Description

Technical Field

[0001] This invention belongs to the field of agricultural robot technology, specifically relating to an apple picking gripper that integrates visual recognition and a separate drive mechanism. Background Technology

[0002] As the world's largest producer and consumer of fruit, my country's total fruit production surged from 46.528 million tons in 1996 to 327 million tons in 2021, a 6.03-fold increase. However, the fresh fruit industry is a typical labor-intensive industry, with labor costs accounting for over 50% of total costs in apple harvesting. Currently, population aging is leading to a continuous decrease in the working-age population, while labor costs are rising year by year. Traditional manual harvesting is not only inefficient but also prone to damaging the fruit due to improper handling, resulting in economic losses. Therefore, with the development of technology, harvesting robots have emerged and been widely used, providing convenience and safety for apple harvesting.

[0003] However, the development of apple harvesting robots in my country still faces many challenges. As a key component of harvesting robots, the apple-picking gripper struggles to maintain a balanced gripping force. For example, Chinese invention patent CN107455088B discloses an apple-picking gripper with a movable first slider at the lower part of its operating rod, and a symmetrical robotic arm hinged to the upper end to form a receiving cavity. An internal pull rod is connected to the robotic arm via a connecting rod and linked to the first slider via a pull line to achieve gripping. The robotic arm has a cutting mechanism that operates after gripping to complete the harvesting, featuring a simple structure and strong practicality. Another example is Chinese invention patent CN114012768A, which discloses an apple-picking gripper including a three-finger gripping and stem-cutting mechanism. During harvesting, the three fingers simultaneously grip the apple inwards, and two motors drive the arc-shaped spherical blades to rotate in opposite directions to cut the stem. Afterwards, the blades reset, and the fingers open to collect the apple. The curved blade design allows for rapid cutting in various postures. When encountering thick stems, the blade grips and, in conjunction with the robotic arm's rotation, severs the stem, enabling harvesting with the stem intact and ensuring fruit storage. Both of these apple-picking grippers can efficiently replace manual labor for apple harvesting. However, during the apple-grabbing process, all grippers move simultaneously. Since apples are not perfectly round, and most apple surfaces have irregular curvatures, some grippers may have difficulty making contact with the apple surface or apply insufficient force when all grippers are grasping simultaneously to ensure the apple's integrity. This requires harvesters to adjust the gripper positions as much as possible to improve the uniformity of the grip, especially for taller apples or those far from the harvester. Due to foliage, harvesters cannot see the entire apple and cannot adjust the gripper's contact position according to the apple's shape. This results in uneven gripping force from each gripper, affecting the stability of the grasp. Summary of the Invention

[0004] In order to solve the problems existing in the prior art, the purpose of this invention is to provide an apple picking gripper that integrates visual recognition and a separate driving mechanism, which can improve the uniformity of the gripping force of each picking finger on the apple during the apple picking process and make the apple gripping more secure.

[0005] The technical solution of this invention is: An apple-picking gripper integrating visual recognition and a separate drive mechanism includes a housing for fixing to a picking robotic arm, and further includes: The picking assembly includes multiple picking fingers, which are arranged in a circumferential array on one side of the housing. Each picking finger is provided with a pressure detection element, which is used to acquire the contact pressure between the picking finger and the apple in real time. An air blowing assembly for blowing away leaves around an apple tree, the air blowing assembly having an air blowing nozzle disposed on the housing; Multiple depth cameras are evenly distributed on the housing to photograph the apple and each picking finger when the leaves are blown away, and to obtain positional images of the apple and each picking finger. The control module is used to obtain distance information between the apple and each picking finger based on the position image of the apple and each picking finger, determine the initial movement stroke of each picking finger based on the distance information, and adjust the secondary movement stroke of the picking finger based on the pressure information obtained in real time by each pressure detection element, so as to achieve a balanced clamping force of each picking finger on the apple.

[0006] Preferably, the air blowing assembly further includes an air pump and an adjusting cylinder. The adjusting cylinder is fixed inside the housing. A cylinder connecting rod is fixed to the output end of the adjusting cylinder. The end of the cylinder connecting rod away from the adjusting cylinder extends out of the housing and is fixedly connected to the air blowing nozzle. The cylinder connecting rod has a hollow structure. The air outlet end of the air pump is connected to the air blowing nozzle through the cylinder connecting rod. Both the air pump and the adjusting cylinder are electrically connected to the control module.

[0007] Preferably, the picking assembly further includes a connecting plate corresponding to each picking finger and a finger driving cylinder. The connecting plate is fixedly connected to the lower end of the picking finger, and the finger driving cylinder is fixed inside the housing. Its telescopic end extends out of the housing and is hinged to one end of the corresponding connecting plate, and the other end of the connecting plate is hinged to the housing.

[0008] Preferably, a suction cup is fixedly connected to one end of the cylinder connecting rod at the air nozzle. The suction cup has a cavity communicating with the air nozzle, and a plurality of air holes facing different directions are opened on the side of the suction cup away from the housing.

[0009] Preferably, it also includes an air intake pump, the air intake port of which is connected to the cylinder connecting rod, and the air intake pump is electrically connected to the control module.

[0010] Preferably, the inner side of the picking finger is arc-shaped, and an elastic pad is fixed on the arc-shaped sidewall of the picking finger.

[0011] Preferably, the pressure detection element includes at least one pressure sensor, which is fixed between the elastic pad and the arcuate sidewall of the picking finger, and the pressure sensor is electrically connected to the control module.

[0012] Preferably, a triangular through hole is provided on the side wall of the picking finger near the end of the connecting plate.

[0013] Preferably, at least three depth cameras are provided, and the three depth cameras are arranged in a circular array around the periphery of the housing and fixed to the housing.

[0014] Preferably, a hinge assembly is provided between the connecting plate and the housing and the finger-driven cylinder. The hinge assembly includes a connecting block, a first movable piece and a second movable piece. The connecting block is fixed to the housing. One end of the first movable piece is fixed to the connecting block and the other end is hinged to the connecting plate. One end of the second movable piece is hinged to the connecting plate and the other end is hinged to the telescopic end of the finger-driven cylinder.

[0015] Compared with the prior art, the apple picking gripper of the present invention, which integrates visual recognition and a separate driving mechanism, has the following advantages: During apple harvesting, this device uses an air-blowing component to disperse the leaves around the apple. Multiple depth cameras then capture real-time images of the apple from multiple angles, providing omnidirectional visual coverage and obtaining real-time positional images of the apple and each harvesting finger. These images are transmitted to the control module in real time. The control module uses these images to determine the distance between the apple and each harvesting finger, and uses this distance information to determine the initial movement distance of each finger. Upon completion of the initial movement distance adjustment, a secondary movement distance adjustment is performed based on pressure information obtained in real-time from the corresponding pressure detection element. This allows for precise adjustment of the gripping force of each finger on the apple, ensuring better contact between the fingers and the apple surface while precisely controlling the balance of gripping force. This facilitates the harvesting of apples with different shapes and contours. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the overall structure in an embodiment of the present invention; Figure 2 This is a cross-sectional view of the overall structure in an embodiment of the present invention; Figure 3 This is a schematic diagram of the hinge assembly in an embodiment of the present invention; Figure 4 This is a schematic diagram of picking up an apple with fingers in an embodiment of the present invention; Figure 5 This is a flowchart illustrating the harvesting process in an embodiment of the present invention.

[0017] Explanation of reference numerals in the attached figures: 1. Housing; 2. Picking finger; 3. Connecting plate; 4. Finger driving cylinder; 5. Depth camera; 6. Air nozzle; 7. Adjusting cylinder; 8. Cylinder connecting rod; 9. Suction cup; 10. Elastic pad; 11. Through hole; 12. Connecting block; 13. First movable piece; 14. Second movable piece; 15. Connecting flange; 16. Bracket. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0019] Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this invention.

[0020] Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0021] See Figures 1 to 5As shown, in order to improve the uniformity of the gripping force on apples during apple picking and to ensure a more secure grasp, this embodiment provides an apple picking gripper that integrates visual recognition and a separate drive mechanism, including a housing 1, a picking assembly, and a target positioning unit. The housing 1 is fixedly connected to a picking robotic arm via a connecting flange 15. The picking assembly includes multiple sets of picking fingers 2 and finger-driven cylinders 4, each arranged in a corresponding manner. The number of picking fingers 2 can be 3 to 8, with a minimum of three; this invention uses three picking fingers 2 as an example. The three picking fingers 2 are distributed in a 120° circumferential array on the side wall of the housing 1 away from the picking robotic arm. The inner wall of the picking fingers 2 has a certain curvature design to better adapt to the outer contour of the apple. Each picking finger 2 has a connecting plate 3 fixed to its lower end, with one end of the connecting plate 3 hinged to the housing 1. Furthermore, each of the three picking fingers 2 is equipped with an independent finger-drive cylinder 4. The finger-drive cylinder 4 is fixed inside the housing 1, with its telescopic end extending out of the housing 1 and hinged to the other end of the connecting plate 3 at the lower end of the corresponding picking finger 2. The telescopic movement of the finger-drive cylinder 4 drives the picking finger 2 to rotate around the hinge axis between the connecting plate 3 and the housing 1, thus adjusting the gripping force of the picking finger 2. Moreover, the three picking fingers 2, each with its own independent finger-drive cylinder 4, can adjust their gripping force independently without interference. Each picking finger 2 is also equipped with a pressure detection element to obtain the contact pressure between the picking finger 2 and the apple in real time. The target positioning unit includes an air blowing assembly, multiple depth cameras 5, and a control module. The air blowing assembly has an air nozzle 6, which is mounted on the housing 1 and positioned between the three picking fingers 2. The air blowing assembly is used to blow away the leaves around the apple when the picking fingers 2 are picking the apple. The multiple depth cameras 5 are evenly distributed on the housing 1 and fixed to the housing 1 by a bracket 16. They are used to take multi-directional pictures of the apple when the leaves are blown away, to obtain position images of the apple and each picking finger 2, and transmit them to the control module. The control module is electrically connected to the picking robotic arm and the multiple finger drive cylinders 4, respectively, and obtains the distance between the apple and each picking finger 2 based on the position images of the apple and each picking finger 2. Distance information is used to determine the movement distance of each picking finger 2. Then, each finger-driven cylinder 4 controls each picking finger 2 separately. After the picking finger 2 contacts the apple, the corresponding pressure detection element transmits the acquired contact pressure information to the control module in real time. The control module processes the detected pressure information and the preset pressure information, and then controls the finger-driven cylinder 4 to adjust the secondary movement stroke of the picking finger 2, so that the picking finger 2 reaches the specified clamping force. In this way, while adapting to the basic shape of the apple, the clamping force of each picking finger 2 is adjusted evenly, so as to achieve precise and firm grasping of the apple and reduce damage.

[0022] See Figure 1 and Figure 3As shown, the picking finger 2 is connected to the housing 1 and the finger-driven cylinder 4 via a hinge assembly. The hinge assembly includes a connecting block 12, a first movable plate 13, and a second movable plate 14. The connecting block 12 is fixed to the housing 1. The first movable plate uses a three-hole flat angle bracket and has two pieces. The two first movable plates are symmetrically arranged about the connecting block 12. One end of the first movable plate is fixed to the connecting block 12, and the other end is hinged to the connecting plate 3. The second movable plate 14 uses a two-hole flat angle bracket; one end is hinged to the connecting plate 3, and the other end is hinged to the end of the finger-driven cylinder 4 that protrudes from the housing 1. Therefore, during use, the extension and retraction of the finger-driven cylinder 4 can individually control the rotation of the corresponding picking finger 2 around the hinge axis between the first movable block and the connecting plate 3, thereby adjusting the apple-gripping force.

[0023] See Figure 1 and Figure 2 As shown, the air blowing assembly further includes an air pump and an adjusting cylinder 7. The adjusting cylinder 7 is fixed inside the housing 1. A cylinder connecting rod 8 is fixed to the output end of the adjusting cylinder 7. The end of the cylinder connecting rod 8 away from the adjusting cylinder 7 extends out of the housing 1 and is fixedly connected to the air nozzle 6. The cylinder connecting rod 8 is a hollow structure. The air outlet of the air pump is connected to the air nozzle 6 through the cylinder connecting rod 8, and the air inlet extends out of the housing 1 to draw in outside air. Both the air pump and the adjusting cylinder 7 are electrically connected to the control module. Preferably, the air pump is also fixed inside the housing 1. In use, the air pump delivers gas at a certain pressure to the air nozzle 6 through the cylinder connecting rod 8. The air blown out by the air nozzle 6 disperses the leaves around the apple to be picked, making it easier for the depth camera 5 to locate the apple and capture its external contour. This facilitates the control module's precise control of the picking robotic arm and the picking fingers 2, allowing the picking fingers 2 to reach a precise picking position and achieve stable grasping of the apple.

[0024] See Figure 1 and Figure 2As shown, further, a suction cup 9 is fixedly connected to one end of the cylinder connecting rod 8 at the air nozzle 6. The suction cup 9 has a cavity communicating with the air nozzle 6, and several air holes facing different directions are opened on the side of the suction cup 9 away from the housing 1. First, through the air holes facing different directions, multiple air columns in different directions can be generated around the apple during the blowing stage, which makes the blowing effect on the leaves around the apple better, facilitating the accurate shooting of the depth camera 5. Furthermore, an air pump is also provided in conjunction with the suction cup 9. The air intake of the air pump is connected to the cylinder connecting rod 8, and the air pump is electrically connected to the control module. After the depth camera 5 completes positioning and the picking fingers 2 retract and grasp the apple, the cylinder connecting rod 8 continues to rise until it abuts against the tail of the apple. At this time, the air pump draws air to form a vacuum suction, which, together with the picking fingers 2, constitutes a multi-directional fixed constraint on the apple, ensuring the stability of the fruit grasping state. After the apples are picked and separated, during the transfer process by the picking robotic arm, when the picking finger 2 is released, the suction cup 9 continuously supports the apple through vacuum adsorption, preventing the fruit from shaking or falling due to inertia; after the picking finger 2 is adjusted to the designated position, the suction cup 9 releases the vacuum, completing the stable placement of the fruit.

[0025] See Figure 1 As shown, furthermore, to ensure a more reliable grip on the apple and provide sufficient protection, an elastic pad 10 is fixed to the arc-shaped sidewall on the inner side of the picking finger 2. The elastic deformation of the elastic pad 10 ensures full contact with the apple when the picking finger 2 is driven to retract, improving the grip's stability. Simultaneously, the elastic pad 10 avoids rigid contact between the picking finger 2 and the apple, enhancing the protection. The elastic pad 10 is preferably made of rubber or sponge.

[0026] Furthermore, the pressure detection unit includes at least one pressure sensor. The pressure sensor on each picking finger 2 is fixed between the elastic pad 10 and the arc-shaped sidewall of the picking finger 2. All pressure sensors are electrically connected to the control module. By setting pressure thresholds for the pressure sensors, the clamping force of the picking fingers 2 can be more effectively controlled during the closing process, ensuring uniform clamping force across all picking fingers 2. This balanced clamping force guarantees the safety of the apple and a secure grip.

[0027] See Figure 1 and Figure 2 As shown, a triangular through hole 11 is further provided on the side wall of the picking finger 2 near the connecting plate 3. The design of the triangular through hole 11 utilizes the stabilizing effect of the triangle to ensure the stability of the picking finger 2. This reduces deformation of the picking finger 2 caused by changes in clamping force, ensuring the stability of apple picking while also extending the service life of the picking finger 2.

[0028] See Figure 1 and Figure 2 As shown, at least three depth cameras 5 are configured, and the three depth cameras 5 are arranged in a circular array around the perimeter of the housing 1 and fixed to the housing 1. The depth cameras 5 integrate five modules: an infrared projection module, a binocular infrared vision module, a vision processing module, a color imaging module, and an inertial measurement module. The infrared projection module emits a specifically coded infrared light pattern, and the depth and contour of the object are calculated by combining the reflected signals; the binocular infrared vision module uses dual cameras to simultaneously acquire images, and generates sub-millimeter precision 3D point clouds through parallax analysis; the vision processing module uses a dedicated processor to calculate depth images in real time, adapting to complex lighting conditions such as strong light and shade; the color imaging module fuses RGB camera data with depth data to generate RGBD images, supporting maturity recognition and interference detection; and the inertial measurement module (IMU) provides attitude information, compensating for mechanical vibrations to ensure dynamic positioning accuracy. Thus, the three depth cameras 5, arranged in a tri-lens configuration, achieve 360° omnidirectional visual coverage. The precise positioning and image acquisition of the apple are achieved through the multi-mode block collaboration within the three depth cameras 5, which indirectly provides reliable visual feedback for the operation of the picking finger 2, enabling accurate and stable apple grasping and picking.

[0029] The specific usage method of this device is as follows: See Figures 1 to 5 As shown Pre-positioning and obstacle removal: After the depth camera 5 identifies the apple's position, the cylinder connecting rod 8 moves upward, and the air pump is activated to cause the suction cup 9 to perform a blowing action, dispersing the leaves around the apple and clearing the field of vision for the depth camera 5 to accurately shoot and position, ensuring the accuracy of fruit spatial position recognition.

[0030] Multi-directional assisted positioning: After the depth camera 5 completes positioning and the picking finger 2 retracts to grasp the apple, the cylinder connecting rod 8 continues to rise until it abuts against the tail of the apple. At this time, the suction pump draws in air to form a vacuum adsorption, which together with the picking finger 2 constitutes a multi-directional fixed constraint on the apple, ensuring the stability of the fruit grasping state.

[0031] Non-destructive transport support: After the apples are picked and separated, during the transport process of the picking robotic arm, when the picking finger 2 is released, the suction cup 9 continuously supports the apples through vacuum adsorption, preventing the fruit from shaking or falling due to inertia; after the picking finger 2 is adjusted to the designated position, the suction cup 9 releases the vacuum, completing the stable placement of the fruit.

[0032] See Figure 5As shown, the above functions are achieved through closed-loop coordination via the timing logic of the control system: depth camera 5 identifies → suction cup 9 blows air → picking finger 2 grasps → suction cup 9 adheres → picking finger 2 moves into position → picking finger 2 releases → suction cup 9 supports → suction cup 9 releases → apple falls into the frame → reset. This design effectively solves the problems of low positioning accuracy, poor grasping stability, and easy damage during transport in traditional picking devices under complex fruit and leaf environments, significantly improving the intelligence level of apple picking operations and the ability to ensure fruit quality.

[0033] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. An apple-picking gripper integrating visual recognition and a separate drive mechanism, comprising a housing (1) for fixing to a picking robotic arm, characterized in that, Also includes: The picking assembly includes multiple picking fingers (2), and the multiple picking fingers (2) are arranged in a circumferential array on one side of the housing (1). Each picking finger (2) is provided with a pressure detection element, which is used to obtain the contact pressure between the picking finger (2) and the apple in real time. An air blowing assembly for blowing away leaves around an apple tree, the air blowing assembly having an air blowing nozzle (6) disposed on the housing (1); Multiple depth cameras (5) are evenly distributed on the housing (1) to take pictures of the apple and each of the picking fingers (2) when the leaves are blown away, and to obtain positional images of the apple and each picking finger (2); The control module is used to obtain the distance information between the apple and each picking finger (2) based on the position image of the apple and each picking finger (2), determine the initial movement stroke of each picking finger (2) based on the distance information, and adjust the secondary movement stroke of the picking finger (2) based on the pressure information obtained in real time by each pressure detection element, so as to achieve a balanced force of each picking finger (2) clamping the apple.

2. The apple picking gripper integrating visual recognition and a separate driving mechanism according to claim 1, characterized in that, The air blowing assembly also includes an air pump and an adjusting cylinder (7). The adjusting cylinder (7) is fixed inside the housing (1). A cylinder connecting rod (8) is fixed to the output end of the adjusting cylinder (7). The end of the cylinder connecting rod (8) away from the adjusting cylinder (7) passes through the housing (1) and is fixedly connected to the air blowing nozzle (6). The cylinder connecting rod (8) is a hollow structure. The air outlet of the air pump is connected to the air blowing nozzle (6) through the cylinder connecting rod (8). Both the air pump and the adjusting cylinder (7) are electrically connected to the control module.

3. The apple picking gripper integrating visual recognition and a separate driving mechanism according to claim 1, characterized in that, The picking assembly also includes a connecting plate (3) corresponding to each picking finger (2) and a finger driving cylinder (4). The connecting plate (3) is fixed to the lower end of the picking finger (2), and the finger driving cylinder (4) is fixed inside the housing (1). Its telescopic end protrudes from the housing (1) and is hinged to one end of the corresponding connecting plate (3). The other end of the connecting plate (3) is hinged to the housing (1).

4. The apple picking gripper integrating visual recognition and a separate driving mechanism according to claim 2, characterized in that, The cylinder connecting rod (8) is fixedly connected to a suction cup (9) at one end of the air nozzle (6). The suction cup (9) has a cavity that communicates with the air nozzle (6). The suction cup (9) has several air holes facing different directions on the side away from the housing (1).

5. An apple picking gripper integrating visual recognition and a separate driving mechanism according to claim 4, characterized in that, It also includes an air intake pump, the air intake port of which is connected to the cylinder connecting rod (8), and the air intake pump is electrically connected to the control module.

6. The apple picking gripper integrating visual recognition and a separate driving mechanism according to claim 1, characterized in that, The inner side of the picking finger (2) is arc-shaped, and an elastic pad (10) is fixed on the arc-shaped sidewall of the picking finger (2).

7. An apple picking gripper integrating visual recognition and a separate driving mechanism according to claim 6, characterized in that, The pressure detection element includes at least one pressure sensor, which is fixed between the elastic pad (10) and the arcuate sidewall of the picking finger (2), and the pressure sensor is electrically connected to the control module.

8. An apple picking gripper integrating visual recognition and a separate driving mechanism according to claim 3, characterized in that, A triangular through hole (11) is provided on the side wall of the picking finger (2) near the end of the connecting plate (3).

9. An apple picking gripper integrating visual recognition and a separate driving mechanism according to claim 1, characterized in that, At least three depth cameras (5) are provided, and the three depth cameras (5) are arranged in a circular array around the housing (1) and fixed to the housing (1).

10. An apple picking gripper integrating visual recognition and a separate driving mechanism according to claim 3, characterized in that, A hinge assembly is provided between the connecting plate (3) and the housing (1) and the finger-driven cylinder (4). The hinge assembly includes a connecting block (12), a first movable piece (13), and a second movable piece (14). The connecting block (12) is fixed to the housing (1). One end of the first movable piece is fixed to the connecting block (12), and the other end is hinged to the connecting plate (3). One end of the second movable piece (14) is hinged to the connecting plate (3), and the other end is hinged to the telescopic end of the finger-driven cylinder (4).

Images