Visual guidance multi-branch vacuum adsorption fruit picking manipulator and picking strategy method
By using a vision-guided multi-branch vacuum adsorption fruit-picking robot and picking strategy, stable posture perception and multi-branch adsorption coordination are achieved in complex orchard environments, improving the success rate and real-time performance of picking and reducing the risk of fruit damage.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHEJIANG FORESTRY UNIVERSITY
- Filing Date
- 2026-03-26
- Publication Date
- 2026-05-12
AI Technical Summary
Existing fruit-picking robots struggle to achieve stable posture perception and multi-path adsorption coordination in complex orchard environments, resulting in low picking success rates, poor real-time performance, and easy damage to the fruit.
A vision-guided multi-branch vacuum adsorption fruit-picking robot is adopted, which combines a vision camera and a multi-branch vacuum adsorption mechanism. It achieves adaptive on/off control through a logic valve, combined with flexible gripping by three claws, and uses a visual posture detection model to recognize fruit posture and optimize picking strategies.
It improves the success rate and real-time performance of harvesting, reduces the risk of fruit damage, and is suitable for efficient automated harvesting operations in orchards.
Smart Images
Figure CN122004047A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of agricultural robot technology, specifically a vision-guided multi-branch vacuum adsorption fruit-picking robot and a picking strategy method. Background Technology
[0002] Against the backdrop of large-scale orchard production and rising labor costs, fruit-picking robots are gradually becoming an important piece of equipment in smart orchards. Because harvesting operations take place in natural lighting and unstructured environments, fruits often face challenges such as backlighting, foliage obstruction, overlapping and dense distribution, size differences, and frequent changes in posture. For the system to achieve stable and low-damage harvesting, in addition to the end effector possessing flexible adaptability and reliable grasping capabilities, it also requires visual perception to provide accurate position and posture information, and to form a harvesting strategy coordinated with the execution actions to ensure stability throughout the entire process of "approach—alignment—adhesion—grasping—separation."
[0003] Existing end effectors mainly fall into two categories: finger clamp type and vacuum suction type. Finger clamp type actuators offer high gripping stability, but their adaptability to targets of different sizes and postures is limited. They are prone to over-clamping, causing skin damage, or under-clamping, leading to detachment. Vacuum suction type actuators can reduce compression damage, but under conditions such as changes in fruit surface curvature, localized depressions, and interference from branches and leaves, it is difficult to establish a stable suction seal. Especially in multi-suction cup structures, when some suction cups are not properly attached, branch leakage can easily occur, causing a decrease in total negative pressure, resulting in weak suction and detachment during handling, thus affecting the harvesting success rate and efficiency.
[0004] In terms of visual perception, existing systems mainly rely on two-dimensional bounding boxes or center point positioning, which are insufficient in expressing the constraint information of the main axis orientation of the fruit and vulnerable parts such as the pedicel and calyx. Under conditions of backlighting, occlusion and overlap, local feature loss is prone to occur, resulting in unstable pose estimation. While fine annotation of multiple key points can enhance pose expression, it significantly increases annotation costs and is prone to introducing consistency noise, which restricts engineering applications.
[0005] Therefore, there is an urgent need for a technical solution that can provide stable attitude perception information in complex orchard environments and work in deep coordination with end effectors and harvesting strategies to solve problems such as insufficient visual guidance, easy air leakage from multi-branch adsorption, and unreliable alignment, thereby improving harvesting success rate, real-time performance, and low damage level. Summary of the Invention
[0006] The purpose of this invention is to overcome the shortcomings of the above-mentioned background technology and provide a vision-guided multi-branch vacuum adsorption fruit picking robot and a picking strategy method to improve the success rate and real-time performance of picking and reduce damage.
[0007] The technical solution of this invention is: A vision-guided multi-branch vacuum adsorption fruit-picking robot includes a robotic arm, an actuator located at the front end of the robotic arm, a vision camera, and a controller; the actuator includes a main support, a motor, a gripper mechanism for grasping fruit and driven by the motor, and an adsorption mechanism for adsorbing fruit.
[0008] The gripper mechanism includes a fixed seat at the front end of the main support, a movable seat that is axially slidably positioned on the main support, a lead screw that is rotatably positioned on the main support and coaxially connected to the motor shaft, a nut that meshes with the lead screw and is fixed to the movable seat, and three grippers arranged around the main support and connecting the movable seat and the fixed seat.
[0009] The motor is fixed to the rear end of the main support; the movable seat is disposed between the motor and the fixed seat; the middle part of the gripper is pivotally positioned on the fixed seat, and the rear end of the gripper is slidably positioned on the movable seat.
[0010] The adsorption mechanism includes a vacuum generator, several suction cups disposed on the gripping surface of the gripper and the front of the fixing plate, a first distributor, and a second distributor; the vacuum generator is connected to the first distributor through a main path, the first distributor is connected to four second distributors through four branch paths, and each second distributor is connected to a group of suction cups through several branch paths; the branch pipes are also equipped with logic valves.
[0011] The controller is electrically connected to the robotic arm, vision camera, motor, vacuum generator, first distributor, second distributor, and logic valve.
[0012] A vision-guided multi-branch vacuum adsorption fruit picking strategy method, employing the aforementioned vision-guided multi-branch vacuum adsorption fruit picking robot, includes the following steps: Step S1: Use a vision camera to acquire RGB-D images of the fruit; input the RGB images into a visual pose detection model to detect the fruit pose, combine the depth image index to obtain the three-dimensional position of the fruit, and determine the feasibility of picking the fruit. Step S2: Sort the harvestable fruits according to the distance rule and generate a harvesting priority queue. Step S3: Obtain the three-dimensional coordinates of the target fruit at the first priority in the picking queue. The robotic arm first aligns the actuator axis with the target fruit, the actuator controls the gripper mechanism to open, and the robotic arm then moves the actuator to the preset position for alignment, so that the three grippers surround the target fruit. Step S4: The vacuum generator generates negative pressure, which is distributed to all branches through the first and second distributors. The logic valves of each branch automatically switch on and off according to the adhesion between the suction cup and the target fruit to achieve air leakage prevention control. Step S5: Once the adsorption force reaches a stable threshold, the actuator controls the gripper mechanism to close and perform a gripping action to accurately grasp the fruit. Step S6: The robotic arm controls the actuator to perform twisting and separating actions according to the direction of the fruit's main axis, so that the target fruit is separated from the fruit tree and transported to an external collection device.
[0013] The visual pose detection model includes: embedding an MSCA attention module between the Neck network and Head network of the YOLOv12-Pose model.
[0014] The feasibility determination includes: if the key point sequence contains first and last key points and the confidence level is higher than the threshold, proceed to the next step; otherwise, leave it for manual processing or secondary observation processing.
[0015] The air leakage prevention control includes: when the suction cup adheres to the fruit surface and forms a seal, the negative pressure of the branch rises rapidly, and the logic valve opens the branch to establish a stable adsorption force; when the suction cup does not adhere to the target fruit or there is a tendency to leak air, the negative pressure of the branch remains unchanged or rises slowly, and the logic valve automatically throttles / closes the branch, leaving only a small amount of air passage in the branch.
[0016] The beneficial effects of this invention are: 1. This invention is based on a visual pose detection model, which combines key point interpolation and multi-scale channel attention mechanism. It can still stably obtain the main axis direction and spatial pose information of the fruit under conditions of backlight, occlusion and fruit overlap. Compared with the existing methods that only use two-dimensional positioning, the pose estimation stability is significantly improved and the manual annotation cost is lower.
[0017] 2. By setting an independent logic valve for each suction cup, adaptive on / off control of the adsorption branch is achieved, so that the overall negative pressure stability is not affected when some suction cups are not attached, thus overcoming the technical defect of adsorption failure caused by single-point air leakage in the existing multi-suction cup structure.
[0018] 3. The fruit is picked by a combination of vacuum adsorption and flexible gripping with three claws, and grasped along the fruit's posture under visual guidance. This significantly reduces the risk of fruit skin damage while ensuring gripping stability, making it suitable for continuous, efficient and automated picking operations in orchards. Attached Figure Description
[0019] The following describes some specific embodiments of the invention in a detailed manner by way of example and not limitation, with reference to the accompanying drawings. The same reference numerals in the drawings denote the same or similar parts or components.
[0020] Figure 1 This is a three-dimensional structural diagram of the actuator of the present invention.
[0021] Figure 2 This is a three-dimensional structural diagram of the gripper of the present invention.
[0022] Figure 3This is a three-dimensional structural diagram of the connecting plate and the motor of the present invention.
[0023] Figure 4 This is a three-dimensional structural diagram of the main support and motor of the present invention.
[0024] Figure 5 This is a three-dimensional structural diagram of the main support and movable base of the present invention.
[0025] Figure 6 This is a flowchart of the actuator of the present invention.
[0026] Figure 7 This is an architecture diagram of the visual pose detection model of the present invention.
[0027] Figure 8 This is an architecture diagram of the MSCA attention module of the present invention.
[0028] Figure 9 This is a schematic diagram of manual annotation and grouping in this invention.
[0029] Figure label: 1. Robotic arm; 2. Connecting plate; 3. Vacuum generator; 4. Main support; 5-1 main path; 5-2 branch path; 5-3 sub-path; 6. Deep groove ball bearing; 7. Gripper; 8. Suction cup; 9. Logic valve; 10. First distributor; 11. Motor; 12. Second distributor; 13. Moving seat; 13-1 slide; 14. Fixed seat; 15. Lead screw. Detailed Implementation
[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0031] I. Robotic Arm like Figure 1 As shown, the present invention provides a vision-guided multi-branch vacuum adsorption fruit picking robot, including a robotic arm 1, an actuator, a vision camera, and a controller disposed at the front end of the robotic arm; The robotic arm is used to drive the actuator to pick the fruit. The actuator is used to grasp the fruit. The vision camera is used to capture images of the fruit. The controller determines the fruit's posture based on the images and controls the robotic arm and actuator to perform the picking action of "aligning-approaching-clamping-separating".
[0032] The robotic arm is used for positioning the actuator. The robotic arm is existing technology and will not be described in detail in this invention.
[0033] The vision camera is a depth camera used to acquire RGB-D images of the fruit. The RGB-D image includes both an RGB image and a depth image. The vision camera is mounted on the actuator, but is not shown in the figure.
[0034] The actuator is located at the front end of the robotic arm and includes a main support 4, a motor 11, a gripper mechanism, and an adsorption mechanism.
[0035] The gripper mechanism includes a fixed base 14, a movable base 13, a lead screw 15, a nut (omitted in the figure), and grippers 7. The fixed base is located at the front end of the main support, the motor is fixed at the rear end of the main support, the movable base is located between the motor and the fixed base, and the movable base is axially (along the length direction of the main support) and slidably positioned on the main support. The lead screw is coaxially connected to the motor shaft, parallel to the sliding direction of the movable base, and rotatably positioned on the main support. The nut meshes with the lead screw and is fixed to the movable base. Three grippers are arranged around the main support, and the grippers connect both the movable base and the fixed base. When the motor drives the movable base to move back and forth, the three grippers open or close simultaneously to achieve adaptive gripping of fruits of different sizes and millimeter-level fine adjustment of the suction cups on the grippers. Fine adjustment at the end of the actuator ensures that the actuator grasps the fruit, compensating for positioning errors at the end of the robotic arm.
[0036] The middle part of the gripper is pivotally positioned on the fixed base, and the rear end of the gripper is slidably positioned on the movable base. The movable base is provided with a radially extending groove 13-1, and the gripper is slidably positioned in the groove by a deep groove ball bearing 6. The end of the gripper is provided with a flexible arc-shaped rounded corner to improve the fitting and cushioning capabilities.
[0037] The adsorption mechanism includes a vacuum generator 3, a suction cup 8, a first diverter 10, and a second diverter 12. The suction cup is positioned on the gripping surface of the jaws and the front side of the fixing plate. There is one first diverter and four second diverters. Figure 1 and Figure 2 As shown, the fixed plate is equipped with a set of suction cups (the number of suction cups in this set is 4), and each gripper is also equipped with a set of suction cups (the number of suction cups in each set is 4), for a total of 4 sets of suction cups, and a total of 12 suction cups.
[0038] The vacuum generator is connected to the first distributor via the main path 5-1. The first distributor is connected to four second distributors via four branch paths 5-2. Each second distributor is connected to a group of suction cups via several branch paths 5-3.
[0039] The vacuum generator is connected to 0.4–0.6 MPa compressed air and generates negative pressure using the Venturi effect. The negative pressure is then distributed to the suction cups of each branch via the first and second distributors to ensure that the negative pressure supply to all suction cups is consistent.
[0040] A logic valve 9 is also connected in series on the branch. The logic valve is an adaptive on / off element, and it has a flow orifice, a throttling orifice, and a spring: when the suction cup adheres to the fruit to form a seal, the logic valve is open to quickly establish negative pressure; when the suction cup does not adhere or there is a tendency to leak air, the logic valve automatically throttles / closes the branch, leaving only a small amount of airflow, thereby avoiding the decay of the total negative pressure. Through the adaptive control of multiple branches by the logic valve, the problem of adsorption failure caused by air leakage in multiple branches is completely solved.
[0041] The controller is electrically connected to the robotic arm, vision camera, motor, vacuum generator, first distributor, second distributor, and logic valve.
[0042] The robotic arm is fixed to the actuator via a connecting plate 2, and the connecting plate is fixed to the main support or motor of the actuator.
[0043] The vision camera and vacuum generator are also fixed to the connecting plate. The connecting plate has reserved positioning pin holes and fastening holes (omitted in the figure) to ensure the consistency of the assembly reference between the vision camera and the actuator, and between the vacuum generator and the actuator.
[0044] II. Harvesting Strategies and Methods Based on the attitude detection of the end effector and vision camera mentioned above, this invention provides a vision-guided multi-branch vacuum adsorption fruit picking strategy method. This method uses the three-dimensional pose of the fruit output by the M-YOLOv12-Pose model, and through the coordinated cooperation of "robotic arm positioning + end effector fine adjustment + multi-branch adaptive control of logic valve", it achieves stable execution of the entire process of "approach-alignment-adsorption-clamping-separation", enabling the actuator to accurately grasp and pick the fruit, avoiding negative pressure attenuation caused by air leakage from multiple suction cups, and reducing the risk of damage to vulnerable parts such as fruit stems and calyxes due to force.
[0045] like Figure 1 As shown, this invention provides a vision-guided multi-branch vacuum adsorption fruit picking strategy method, employing the aforementioned vision-guided multi-branch vacuum adsorption fruit picking robot, including the following steps: Step S1: Target selection and harvesting feasibility assessment 1. Use a vision camera to capture real-time RGB-D images of apples in the orchard. The vision camera is an Intel RealSense D435i depth camera with an RGB image resolution of 1280×720.
[0046] 2. Input the RGB image into the visual pose detection model (M-YOLOv12-Pose model) to detect the fruit pose, obtain the bounding box, 10 key point sequence and confidence of each fruit in the RGB image, and obtain the three-dimensional position of the fruit by combining the depth image index.
[0047] The visual pose detection model first identifies all fruits in the RGB image and generates a bounding box for each fruit. Then, it detects the position of 10 key points for each identified fruit. Based on the predicted confidence level, the bounding boxes and key points are filtered, retaining only the bounding boxes and key points with a confidence level higher than a preset threshold. At the same time, it combines the depth map and obtains the three-dimensional pose of the fruit by indexing the depth information.
[0048] 3. Filtering based on key point sequence completeness and confidence: When the keypoint sequence contains both the first and last keypoints and the confidence level is higher than the threshold, it indicates that the keypoint sequence can stably fit the principal axis. The fruit is then determined to be a "mechanizable harvestable target" and is considered a target fruit that has passed the screening. Proceed to the next step.
[0049] Otherwise (when the first and last key points are missing or the confidence level is low due to occlusion), it indicates that the key point sequence is severely missing or the attitude is unstable, and it is judged as "the target will not be picked for the time being" and left for manual processing or secondary observation processing.
[0050] Step S2: Harvesting sequence generation and robotic arm alignment For all target fruits that pass the screening in the RGB image, the picking priority queue is generated by sorting them according to the three-dimensional position of the target fruit, the distance to the current pose of the robotic arm, and the fruit's accessibility, based on the rules of distance.
[0051] The specific steps include: For the target fruit that passes the screening, firstly, its two-dimensional detection center is obtained based on the RGB image. Then, the robust depth value of the corresponding region is extracted from the depth map. The pixel coordinates are back-projected to the robot base coordinate system by combining the camera intrinsic parameters and the hand-eye calibration extrinsic parameters, thereby obtaining the three-dimensional position of the fruit. Secondly, the Euclidean distance between the current actuator and the target fruit is calculated based on the current actuator pose as a quantitative indicator of "closeness". Fruit accessibility is determined by whether the fruit is within the picking range of the robotic arm within the workspace (existing method). Finally, the harvesters are sorted according to their distance, with those closest to the source being harvested first, to generate a harvesting priority queue, thus achieving rule-driven quantitative sorting decisions.
[0052] Step S3: Align the robotic arm The robotic arm completes the alignment under visual guidance: it acquires the three-dimensional coordinates of the target fruit at the first priority of the picking queue, and first aligns the actuator axis with the target fruit so that the actuator's orientation is approximately consistent with the main axis direction of the fruit (the direction of the line connecting the first and last key points). Then, the actuator controls the gripper mechanism to open, and the robotic arm moves the actuator to the preset position for alignment. The actuator maintains a preset safe distance (e.g., 200–250 mm) from the target fruit, and the three grippers surround the target fruit, providing the initial angle conditions for the subsequent adsorption array alignment.
[0053] Step S4: Vacuum adsorption and logic valve to prevent air leakage Once the actuator approaches the target position, it activates the suction, automatically adjusting the target fruit's position even with slight deviations in actuator positioning, thus improving grasping accuracy. The specific process is as follows: 1. The vacuum generator produces negative pressure, which is then evenly distributed to all branches via the first and second distributors; 2. The logic valves of each branch circuit adaptively open and close based on the adhesion state between the suction cup and the target fruit to achieve air leakage prevention control: When the suction cup adheres to the surface of the target fruit and forms a seal, the negative pressure of the branch circuit rises rapidly, and the logic valve opens the branch circuit to establish a stable adsorption force; when the suction cup does not adhere to the target fruit or there is a tendency to leak, the negative pressure of the branch circuit remains unchanged or rises slowly, and the logic valve automatically throttles / closes the branch circuit, leaving only a small amount of airflow in the branch circuit, thereby avoiding leakage in the branch circuit that would cause a decrease in the total negative pressure and ensuring the adsorption stability of other adhered suction cups.
[0054] Step S5: Flexible gripping and stable grasping by the grippers Once the adsorption force reaches a stable threshold, the actuator controls the gripper mechanism to close and perform a clamping action, accurately grasping the target fruit. The flexible silicone pads of the three grippers and the flexible structure of the suction cup work together to buffer contact impact, reducing skin pressure and abrasion. By sharing the load through the clamping force and adsorption force, the risk of detachment is significantly reduced during the handling and separation stages.
[0055] Step S6: Separate harvesting and recycling After a stable grasp is completed, the robotic arm controls the actuator to perform twisting and separation actions according to the main axis direction of the fruit, so that the target fruit is separated from the fruit tree. After separation, the robotic arm transports the target fruit to the external collection device, the gripper mechanism opens and releases the vacuum adsorption of the suction cup, and the target fruit is placed. Then it returns to step S1 and enters the cycle of picking the next target fruit.
[0056] III. Visual Pose Detection Model To address the challenges of high manual annotation costs, unstable pose calculation, and large errors in backlighting and occlusion scenarios during apple fruit pose recognition in complex natural orchard environments, this invention also provides a training method for a visual pose detection model. Through a process of "two-point annotation—interpolation expansion—attention enhancement—pose calculation," it achieves rapid and accurate perception of fruit pose, providing reliable pose information support for the alignment and planning of the automatic harvesting end effector.
[0057] A method for training a visual pose detection model includes the following steps: Step S1: Obtain and label the fruit image dataset. The vision camera used was an Intel RealSense D435i depth camera with an RGB image resolution of 1280×720. The vision camera was positioned at three heights: 100 cm, 150 cm, and 200 cm, maintaining a horizontal distance of approximately 1.5 m from the target fruit. Data acquisition covered morning, afternoon, and evening to obtain orchard fruit (apple) image samples under various typical natural lighting conditions, including front lighting, backlighting, and side lighting.
[0058] The collected fruit images were labeled to obtain the dataset: 1. Draw a rectangular border for each fruit; then mark two key points on the border: the first key point is the starting point, which is where the fruit stem connects to the fruit, and the second key point is the ending point, which is the calyx indentation; for fruits that are severely obscured, blurry, or whose main axis direction cannot be determined, only the border is retained without marking the key points to reduce the noise of the posture label.
[0059] 2. Group the border and the first and last points into the same group and establish an "instance-attitude point" binding relationship.
[0060] like Figure 9 As shown, the rectangle is the target detection box for the apple, where "0" represents the key point at the head of the apple and "9" represents the key point at the tail of the apple.
[0061] Step S2: Construct a visual pose detection model like Figure 7 As shown, an MSCA attention module is introduced into the YOLOv12-Pose model to obtain a visual pose detection model (M-YOLOv12-Pose model). The MSCA attention module is embedded between the Neck network and the Head network to achieve cross-scale information interaction and channel attention weighting, thereby enhancing the response capability to small targets, local areas of the fruit stalk, and occlusion edges.
[0062] The posture detection model structure includes a backbone network, a neck network, and a head network.
[0063] The backbone network consists of an input layer, Conv, C3k2 module, and A2C2f module.
[0064] The Neck Network includes the Upsample module, Concat module, A2C2f module, and Conv module.
[0065] like Figure 8As shown, the MSCA attention module extracts features through multi-scale convolutions, including dw5x5, dw1x7, dw7x1, dw1x11, dw11x1, dw1x21, and dw21x1 convolutions. Effective features are then enhanced through channel fusion and convolutional attention mechanisms. This structure improves detection and keypoint localization capabilities in complex scenes while maintaining lightweight design, and works in conjunction with interpolation keypoint strategies to enhance pose stability.
[0066] The YOLOv12-Pose model and the MSCA attention module are both existing technologies.
[0067] Step S3: Dataset interpolation, expansion, and enhancement processing Based on the annotation of the two key points at the beginning and end, a linear equidistant interpolation method is used to perform interpolation at the starting point (x0, y0) and the ending point (x0, y0). n ,y n Eight intermediate points are generated between the points to form a sequence of 10 key points. This method can automatically complete the key point structure with only two manual annotations, significantly reducing annotation costs. At the same time, it improves the continuity and anti-occlusion ability of the attitude calculation through "multi-point geometric constraints". Even if some points are shifted due to backlighting, occlusion or sepal interference, the calculation of the fruit principal axis remains stable.
[0068] Data augmentation strategies such as Mosaic, random flipping, and color jitter are employed to improve the robustness and generalization ability of the model and adapt it to the complex and ever-changing environmental conditions of the orchard.
[0069] Step S4: Train the visual pose detection model The processed dataset is converted into a YOLOv12-Pose model multi-task training format and input into the visual pose detection model for training to optimize the model's hyperparameters.
[0070] Training environment: The training platform is Ubuntu 20.04.6 operating system, equipped with 128 GB of memory, NVIDIA RTX 4090 (24 GB) graphics card, and AMD Ryzen 99950X processor. The deep learning framework is PyTorch 1.11.0.
[0071] Network training parameter settings: The deep learning framework is PyTorch 1.11.0, the input image size is 640×640, the number of iterations (epochs) is 500, and the batch size is 8; the optimizer is SGD, the initial learning rate is 0.0003 with cosine annealing decay, the momentum is 0.937, and the weight decay coefficient is 0.0005; the keypoint regression weights in the multi-task loss are set to 12.0 to enhance pose learning.
[0072] Step S5: Model Evaluation Let the coordinates of the starting point of the fruit stalk be... The endpoint coordinates are Then the principal axis direction vector of the fruit can be expressed as: Its corresponding slope is: Based on this slope information, the rotation angle of the fruit in the two-dimensional plane can be determined. Specifically, the main axis direction vector of the fruit is determined by connecting the beginning and end of the fruit, and the angle between the predicted direction and the true direction is calculated: In the formula This represents the actual direction vector of the fruit stalk (obtained by connecting the first and last key points of the label). The predicted pedicel direction vector (obtained by fitting a straight line based on 10 detected keypoints) is ultimately calculated as the mean angle error across all samples. MAE This indicator serves as the overall evaluation standard. The smaller the value of this indicator, the closer the attitude estimation result is to the actual situation.
[0073] The method for calculating the average angle difference is as follows: calculate the angle difference for each fruit. and Find the angle between the fruits, and then average the angles of all the fruits.
[0074] Step S6: Model Deployment; The trained visual pose detection model is deployed to output apple bounding boxes and key point sequences in real time from the input image. The three-dimensional pose of the fruit is obtained by combining the key point sequence with the depth map.
[0075] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.
Claims
1. A vision-guided multi-branch vacuum adsorption fruit-picking robot, characterized in that: It includes a robotic arm (1), an actuator, a vision camera, and a controller located at the front end of the robotic arm; the actuator includes a main support (4), a motor (11), a gripper mechanism for grasping fruit and driven by the motor, and an adsorption mechanism for adsorbing fruit.
2. The vision-guided multi-branch vacuum adsorption fruit-picking robot according to claim 1, characterized in that: The gripper mechanism includes a fixed seat (14) located at the front end of the main support, a movable seat (13) axially slidably positioned on the main support, a lead screw (15) rotatably positioned on the main support and coaxially connected to the motor shaft, a nut meshing with the lead screw and fixed to the movable seat, and three grippers (7) arranged around the main support and connecting the movable seat and the fixed seat.
3. The vision-guided multi-branch vacuum adsorption fruit-picking robot according to claim 1, characterized in that: The motor is fixed to the rear end of the main support; the movable seat is disposed between the motor and the fixed seat; the middle part of the gripper is pivotally positioned on the fixed seat, and the rear end of the gripper is slidably positioned on the movable seat.
4. The vision-guided multi-branch vacuum adsorption fruit-picking robot according to claim 1, characterized in that: The adsorption mechanism includes a vacuum generator (3), several suction cups (8) disposed on the clamping surface of the gripper and the front of the fixing plate, a first distributor (10), and a second distributor (12); the vacuum generator is connected to the first distributor through the main path (5-1), the first distributor is connected to four second distributors through four branches (5-2), and each second distributor is connected to a group of suction cups through several branches (5-3); the branch pipe is also provided with a logic valve (9).
5. The vision-guided multi-branch vacuum adsorption fruit-picking robot according to claim 1, characterized in that: The controller is electrically connected to the robotic arm, vision camera, motor, vacuum generator, first distributor, second distributor, and logic valve.
6. A vision-guided multi-branch vacuum adsorption fruit picking strategy method, employing the vision-guided multi-branch vacuum adsorption fruit picking robot as described in any one of claims 1-5, comprising the following steps: Step S1: Use a vision camera to acquire RGB-D images of the fruit; input the RGB images into a visual pose detection model to detect the fruit pose, combine the depth image index to obtain the three-dimensional position of the fruit, and determine the feasibility of picking the fruit. Step S2: For the harvestable fruits, sort them according to the rule of "closest distance - minimal occlusion - stable posture" to generate a harvesting priority queue; Step S3: Obtain the three-dimensional coordinates of the target fruit at the first priority in the picking queue. The robotic arm first aligns the actuator axis with the target fruit, the actuator controls the gripper mechanism to open, and the robotic arm then moves the actuator to the preset position for alignment, so that the three grippers surround the target fruit. Step S4: The vacuum generator generates negative pressure, which is distributed to all branches through the first and second distributors. The logic valves of each branch automatically switch on and off according to the adhesion between the suction cup and the target fruit to achieve air leakage prevention control. Step S5: Once the adsorption force reaches a stable threshold, the actuator controls the gripper mechanism to close and perform a gripping action to accurately grasp the fruit. Step S6: The robotic arm controls the actuator to perform twisting and separating actions according to the direction of the fruit's main axis, so that the target fruit is separated from the fruit tree and transported to an external collection device.
7. The vision-guided multi-branch vacuum adsorption fruit-picking robot according to claim 6, characterized in that: The visual pose detection model includes: embedding an MSCA attention module between the Neck network and Head network of the YOLOv12-Pose model.
8. The vision-guided multi-branch vacuum adsorption fruit-picking robot according to claim 7, characterized in that: The feasibility determination includes: if the key point sequence contains first and last key points and the confidence level is higher than the threshold, proceed to the next step; otherwise, leave it for manual processing or secondary observation processing.
9. The vision-guided multi-branch vacuum adsorption fruit-picking robot according to claim 8, characterized in that: The air leakage prevention control includes: when the suction cup adheres to the fruit surface and forms a seal, the negative pressure of the branch rises rapidly, and the logic valve opens the branch to establish a stable adsorption force; when the suction cup does not adhere to the target fruit or there is a tendency to leak air, the negative pressure of the branch remains unchanged or rises slowly, and the logic valve automatically throttles / closes the branch, leaving only a small amount of air passage in the branch.