Double-arm citrus picking robot and picking method

Abstract

The invention discloses a double-arm citrus picking robot and a picking method, and belongs to the technical field of agricultural automation equipment. The robot comprises a crawler-type moving chassis, a symmetrically-arranged double-mechanical-arm system, a machine vision system, a citrus collecting bin and a control system. The double-mechanical-arm system adopts a main and auxiliary collaborative framework, the tail end of a main mechanical arm is integrated with a clamping jaw and vacuum suction disc parallel mechanism, an auxiliary mechanical arm is provided with a rotary circular saw blade, and lossless clamping of fruits and precise cutting of fruit stems are achieved. The machine vision system realizes high-precision citrus identification under complex illumination by improving a YOLOv5 network and fusing an HSV dynamic threshold value and Focal Loss optimization; graded path planning is combined with an A * algorithm and an RRT * algorithm, and motion safety of the mechanical arm is ensured through bounding box collision detection. The picking method comprises the steps of three-dimensional semantic map construction, collaborative grabbing and cutting, lossless transfer and the like. The problems that a single-arm robot is low in efficiency, fruits are prone to being damaged, fruit stems are not thoroughly treated and the like are solved, and the robot is suitable for automatic orchard harvesting in hilly areas.

Description

Technical Field

[0001] This invention belongs to the field of agricultural automation equipment technology, specifically relating to a citrus harvesting robot based on the collaborative operation of two robotic arms, which is particularly suitable for automated fruit harvesting operations in orchards in hilly areas. Background Technology

[0002] Citrus harvesting is one of the most labor-intensive parts of orchard management, and current mainstream technologies have the following technical shortcomings: Limitations of a single robotic arm: In dense fruit tree environments, the robotic arm's movement is prone to interference, resulting in low work efficiency, sometimes even less than manual harvesting. End effector damage problem: Traditional grippers directly hold the fruit, and the pressure sensor is not accurate enough, resulting in a high rate of fruit peel damage; Visual positioning error: Existing binocular vision systems have large positioning errors in scenarios where branches and leaves are obstructed, leading to an increased harvesting failure rate; Missing fruit stem processing: Most patents do not include a dedicated fruit stem cutting device, and manual recutting increases operating costs. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a dual-arm citrus harvesting robot. Through the collaborative operation of two robotic arms, multi-sensor fusion positioning, and a non-destructive harvesting mechanism, it solves the problems of low efficiency, easy damage to fruit, and incomplete removal of fruit stems in existing single-arm robots, thereby achieving efficient, precise, and automated harvesting in orchard environments.

[0004] Another object of the present invention is to provide a harvesting method for the above-mentioned harvesting robot.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A dual-arm citrus harvesting robot includes a tracked mobile chassis (1), a dual robotic arm system (2), a machine vision system (3), a citrus collection bin (4), and a control system; wherein the tracked mobile chassis (1) adopts a wide rubber track design and has good terrain adaptability; two identical robotic arm systems (2) are symmetrically arranged at the front of the chassis, and the citrus collection bin (4) is integrated at the rear.

[0007] The dual robotic arm system (2) adopts a main-auxiliary collaborative architecture. Each robotic arm unit includes a rotatable gimbal (21), a lifting mechanism (22), a four-degree-of-freedom main robotic arm (23), a main arm end effector (24), and a two-degree-of-freedom auxiliary robotic arm (25). The rotatable gimbal (21) achieves 360° continuous rotation through a high-precision slewing bearing. The lifting mechanism (22) set on its top uses a stepper motor (221) to drive a synchronous belt (222), and cooperates with a double-row ball bearing guide structure to achieve precise lifting and adjustment.

[0008] The main robotic arm (23) adopts a four-degree-of-freedom serial structure, which includes three sequentially hinged links (231, 232, 233). Each joint is equipped with a servo motor with a high-precision absolute encoder. The end effector (24) of the main arm integrated at its end innovatively adopts a parallel structure of gripper (241) and vacuum suction plate (242). The gripper (241) is a bionic two-finger design with a pressure sensor embedded at the fingertip. The vacuum suction plate (242) is connected to a negative pressure generator through a fast-response solenoid valve, and a silicone flexible sealing ring is set on the edge of the suction plate.

[0009] The auxiliary robotic arm (25) is innovatively integrated on the third link (233) of the main robotic arm, and is equipped with a rotatable circular saw blade (251) at the end. The auxiliary arm and the end effector (24) of the main arm form a collaborative operation mechanism: when the main arm holds the citrus, the auxiliary arm automatically starts the saw blade to cut the fruit stem. The actions of the two are coordinated in millisecond time through the control system.

[0010] The machine vision system (3) adopts a "global-local" dual-level architecture: the global vision module (31) is composed of a high-resolution RGB-D camera and is used to construct a three-dimensional map of the orchard; the local vision module (32) uses two sets of binocular cameras, which are installed at the end of the robotic arm to achieve precise positioning of the picking target; all visual data are transmitted to the control system in real time via industrial Ethernet.

[0011] The citrus collection bin (4) is equipped with a multi-layer elastic net buffer structure, which is connected to the working area of ​​the robotic arm through an inclined slide, which can effectively reduce the damage caused by fruit falling. The control system adopts a distributed architecture, with the main control unit responsible for path planning and task scheduling, and each robotic arm equipped with an independent motion controller to achieve precise trajectory tracking.

[0012] A method for harvesting citrus fruits using a dual-arm citrus harvesting robot includes the following steps:

[0013] Step S1: Construction of 3D Semantic Map

[0014] The global vision module (31) scans the work area and uses an improved YOLOv5 instance segmentation network for citrus recognition: the network sets a dynamic recognition threshold in the HSV color space: hue component H∈[25,40], saturation component S>0.45σ. s The lightness component V > 0.35σ v , where σ s σ v The brightness is adaptively adjusted based on the median brightness of the current frame image to enhance illumination robustness; the classification loss function is optimized using Focal Loss. .

[0015] The parameters were set to α=0.8 and γ=2.5 to balance the ratio of positive to negative samples to 1:3, which effectively alleviated the problem of missed detection in dense fruit scenes. By fusing RGB-D depth data and point cloud segmentation results, a semantic map containing the three-dimensional structure of fruit trees and the pose information of citrus fruits was constructed.

[0016] Step S2: Hierarchical Path Planning

[0017] Employing a global-local two-level programming strategy:

[0018] Global planning: Based on semantic map point cloud data, the movement path of the robotic arm base is calculated using the A* algorithm to achieve coarse localization;

[0019] Local planning: Combining real-time data from binocular cameras (32), the motion trajectory of the end effector is optimized using the RRT* algorithm, and the interference risk between robotic arms and between robotic arms and fruit trees is avoided by using a continuous collision detection algorithm based on bounding boxes (OBB).

[0020] Step S3: Robotic arm pose initialization

[0021] The lifting mechanism (22) adjusts the working reference plane of the main robotic arm (23) according to the target height, and the stepper motor (221) drives the lifting through the synchronous belt (222). The gimbal (21) rotates to the target position.

[0022] Step S4: Precise Fruit Grabbing

[0023] The local vision module (32) guides the end effector (24) of the main arm to position itself on the surface of the citrus fruit. The vacuum suction plate (242) preferentially adsorbs the fruit, and the flexible sealing ring at the edge of the suction plate adapts to the fruit surface. The gripper (241) then closes, and the bionic two-finger structure adjusts the gripping force based on the feedback of the pressure sensor to ensure that the fruit skin is undamaged.

[0024] Step S5: Cut the fruit stalk in synergy

[0025] When the vacuum adsorption pressure reaches the preset safe adsorption threshold and the gripper contact force is stable within the stable gripping range, the control system triggers the auxiliary robotic arm (25) to move, and the micro servo motor drives the rotating circular saw blade (251) to cut the fruit stem.

[0026] Step S6: Harmless transfer of fruit

[0027] The robotic arm transfers the fruit to the top of the collection bin (4), the gripper (241) releases first, the vacuum suction plate (242) closes the negative pressure after a certain delay, and the fruit falls into the elastic net bag along the inclined slide by gravity.

[0028] The beneficial effects of this invention are:

[0029] The end effector of the main arm adopts a parallel design of gripper and vacuum suction plate. The biomimetic two-finger structure of the gripper dynamically adjusts the gripping force through a pressure sensor, and the flexible sealing ring of vacuum adsorption adapts to the fruit surface. The two work together to effectively reduce the fruit peel damage rate.

[0030] The auxiliary robotic arm integrates a rotating circular saw blade, which triggers the cutting action through the linkage of dual safety thresholds of adsorption pressure and clamping force, so as to completely cut off the fruit stem, avoid the manual recutting process, and ensure the fruit is harvested intact.

[0031] The YOLOv5 network was improved by fusing HSV color space dynamic threshold and Focal Loss optimization strategy. The recognition robustness was enhanced by adaptive lighting adjustment. The recognition accuracy and positioning precision of citrus fruits were significantly improved in scenarios with foliage occlusion and sudden changes in lighting.

[0032] Hierarchical path planning combines global coarse localization using the A algorithm with local trajectory optimization using the RRT algorithm, and real-time obstacle avoidance using OBB collision detection, significantly improving the safety of the robotic arm's motion trajectory and ensuring stable operation in dense orchard environments. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the structure of the present invention.

[0034] Figure 2 This is the left view of the structure of the present invention.

[0035] Figure 3 This is a schematic diagram of the robotic arm in this invention.

[0036] Figure 4 This is a top view of the robotic arm in this invention.

[0037] Figure 5 This is a flowchart illustrating the harvesting method in this invention.

[0038] Among them, 1. Tracked mobile chassis; 21. Rotatable gimbal; 221. Stepper motor; 222. Synchronous belt; 231, 232, 233. Main robotic arm linkage; 24. Main arm end effector; 241. Gripper; 242. Vacuum suction plate; 25. Auxiliary robotic arm; 251. Rotary circular saw blade; 31. Global vision module; 32. Local vision module; 4. Collection bin. Detailed Implementation

[0039] The following embodiments will help those skilled in the art to further understand the present invention. However, they are not intended to limit the present invention in any way. It should be noted that those skilled in the art can make several modifications and improvements to the present invention without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. In order to facilitate understanding of the present invention, the present invention will be described in detail below with reference to specific embodiments.

[0040] Example 1

[0041] like Figures 1 to 4 As shown, the dual-arm citrus harvesting robot of the present invention includes the following core components:

[0042] Tracked mobile chassis (1): It adopts a wide rubber track design with a track width of 200mm and a ground pressure of ≤15kPa, which is suitable for the rugged terrain in hilly areas; a dual mechanical arm system (2) is symmetrically installed at the front of the chassis, and a citrus collection bin (4) is integrated at the rear.

[0043] Dual robotic arm system (2): includes two sets of robotic arm units with the same structure. Each set of units consists of a rotatable gimbal (21), a lifting mechanism (22), a four-degree-of-freedom main robotic arm (23), a main arm end effector (24), and a two-degree-of-freedom auxiliary robotic arm (25).

[0044] Machine vision system (3): It consists of a global vision module (31) and a local vision module (32). The global module uses an Orbbec Astra Pro RGB-D camera, and the local module uses a Hikvision MV-CA013-21GC binocular camera.

[0045] Citrus collection bin (4): The inside is equipped with three layers of elastic net bags (the material is polyurethane elastomer with a Shore hardness of 60A). The net bags are connected to the working area of ​​the robotic arm through an inclined slide with an inclination angle of 30°.

[0046] Combination Figure 5 The process shown below, with specific implementation steps as follows:

[0047] Step S1: 3D semantic map construction. The global vision module (31) scans the orchard, extracts citrus targets through the improved YOLOv5 instance segmentation network, and generates a 3D semantic map by fusing depth data (point cloud density is 5cm / point). The classification loss function is optimized using Focal Loss, with parameters set to α=0.8 and γ=2.5. The training dataset contains 100,000 orchard scene images (positive and negative sample ratio 1:3).

[0048] Step S2: Hierarchical path planning. Global path planning: Calculate the movement path of the robotic arm base based on the A* algorithm, with a grid map resolution of 10cm. Local path planning: Optimize the end-point trajectory using the RRT* algorithm, combined with the OBB collision detection algorithm (bounding box expansion coefficient 1.2) for real-time obstacle avoidance, with a planning frequency of 100Hz.

[0049] Step S3: Initialize the pose of the robotic arm, adjust the main robotic arm (23) to the target height (error ±1mm) by the lifting mechanism (22), and rotate the gimbal (21) to the target orientation (error ±0.5°).

[0050] Step S4: The fruit is precisely grasped. The local vision module (32) guides the gripper (241) to be positioned on the surface of the citrus fruit. The vacuum suction plate (242) preferentially adsorbs the fruit (adsorption pressure ≥ -50kPa). When the gripper closes, the pressure sensor feedback adjusts the clamping force to 3-5N to ensure that the peel is undamaged.

[0051] Step S5: Collaborative stem cutting. When the adsorption pressure is ≥-50kPa and the gripper contact force is stable in the range of 3-5N (lasting for 200ms), the auxiliary robotic arm (25) starts the saw blade (251), and the cutting time is ≤0.5s.

[0052] Step S6: Fruit transfer without damage. The main robotic arm (23) moves the fruit directly above the collection chamber (4). The gripper (241) releases first. The vacuum suction plate (242) closes the negative pressure after a delay of 300ms. The fruit falls freely into the elastic net along the slide, with a drop height of ≤20cm.

[0053] The above description is merely an embodiment of the present invention, but the implementation of the present invention is not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention are equivalent substitutions and are included within the protection scope of the present invention.

Claims

1. A dual-arm citrus harvesting robot, characterized in that, The system includes a tracked mobile chassis (1), a dual robotic arm system (2) symmetrically arranged at the front of the chassis, a machine vision system (3), a citrus collection bin (4), and a control system. The dual robotic arm system (2) comprises two sets of robotic arm units with identical structures. Each set of robotic arm units includes: A rotatable gimbal (21) is mounted on a chassis (1) via a slewing bearing; The lifting mechanism (22) is vertically mounted on the gimbal (21); The four-degree-of-freedom main robotic arm (23) has its base hinged to the top of the lifting mechanism (22) and includes three sequentially hinged links (231, 232, 233), each joint of which is driven by a servo motor. The end effector (24) of the main boom is located at the end of the third link (233); A two-degree-of-freedom auxiliary robotic arm (25) is mounted on the third link (233) and is arranged in parallel with the end effector (24) of the main arm. It is driven independently by a micro servo motor.

2. The dual-arm citrus harvesting robot according to claim 1, characterized in that, The lifting mechanism (22) is driven by a stepper motor (221), a synchronous belt (222) and a double-row ball bearing guide structure to achieve lifting motion. The lifting speed is adjusted by the pulse frequency of the stepper motor (221). The main arm end actuator (24) consists of a gripper (241) and a vacuum suction plate (242). The gripper (241) and the vacuum suction plate (242) are installed side by side at the end of the third link (233). The gripper (241) adopts a bionic two-finger structure with a pressure sensor at the fingertip. The vacuum suction plate (242) is connected to a negative pressure generator through a solenoid valve. A flexible sealing ring is set on the edge of the suction plate.

3. The dual-arm citrus harvesting robot according to claim 1, characterized in that, The two-degree-of-freedom auxiliary robotic arm (25) is equipped with a rotating circular saw blade (251) at its end. The start and stop actions of the rotating circular saw blade (251) are linked with the clamping action of the main arm end effector (24) through the control system.

4. The dual-arm citrus harvesting robot according to claim 1, characterized in that, The machine vision system (3) includes a global vision module (31) and a local vision module (32). The global vision module (31) is composed of an RGB-D camera installed at the front of the chassis (1). The local vision module (32) is composed of a set of binocular cameras installed at the front end of the third link (233) of each robotic arm. The global vision module (31) and the local vision module (32) are connected to the control system via an industrial Ethernet.

5. The dual-arm citrus harvesting robot according to claim 1, characterized in that, The citrus collection bin (4) is installed at the rear of the chassis (1) and has an elastic net bag inside.

6. A method for harvesting citrus fruits based on the dual-arm citrus harvesting robot according to any one of claims 1-5, characterized in that, Includes the following steps: S1. Scan the work area through the global vision module (31), identify the point cloud of fruit tree branches based on the improved YOLOv5 instance segmentation network, and integrate the depth data to construct a semantic map containing three-dimensional structure to locate the target pose of the citrus that can be picked. S2. Based on the target pose, the robot arm operation path is planned, and a continuous collision detection algorithm based on bounding box (OBB) is used to avoid interference between robot arms and between robot arms and fruit trees; S3. Control the lifting mechanism (22) to adjust the initial working height of the main robotic arm (23) and drive the gimbal (21) to rotate to the target position; S4. The local vision module (32) guides the end effector (24) of the main arm to be precisely positioned on the surface of the citrus fruit. First, it controls the vacuum suction plate (242) to adsorb the surface of the fruit, and then controls the gripper (241) to close and wrap the fruit. S5. Start the two-degree-of-freedom auxiliary robotic arm (25) and drive the rotating circular saw blade (251) to cut the fruit stalk; S6. The robotic arm transfers the fruit to the collection bin (4), releases the gripper (241), and resets after closing the negative pressure. This process is repeated until the harvesting of the area is completed.

7. The harvesting method according to claim 6, characterized in that, In step S1, the improved YOLOv5 network sets a citrus recognition threshold in the HSV color space that satisfies the following: (1) in , To adapt the illumination intensity parameter, it is dynamically adjusted based on the median brightness of the current frame image; the improved YOLOv5 network uses Focal Loss to improve the classification loss. (2) With α=0.8 and γ=2.5, the ratio of positive to negative samples is balanced to 1:

3.

8. The harvesting method according to claim 6, characterized in that, Step S2 employs a hierarchical path planning strategy, specifically including: Global path planning stage: Based on RGB-D camera point cloud data, a coarse positioning path for the robotic arm base is generated using the A* algorithm; Local path planning stage: Using data from binocular cameras, the RRT* algorithm is used to optimize the motion trajectory of the end effector.

9. The harvesting method according to claim 6, characterized in that, In step S5, the starting time of the rotating circular saw blade (251) is determined by the following logic: when the adsorption pressure of the vacuum suction plate (242) reaches the preset safe adsorption threshold, and the fingertip pressure sensor of the gripper (241) detects that the contact force is in the stable clamping range, the saw blade rotation cutting action is triggered.

10. The harvesting method according to claim 6, characterized in that, The fruit transfer process in step S6 is as follows: after the robotic arm moves the fruit directly above the collection bin (4), the gripper (241) releases first, the vacuum suction plate (242) closes the negative pressure after a certain delay, and the fruit falls freely into the elastic net bag by gravity.

Images