Kiwi bionic bud-thinning claw and control method thereof

Abstract

This invention provides a biomimetic bud-removing claw for kiwifruit and its control method, comprising an inverted T-shaped frame, a clamping claw, a left suction claw, a right suction claw, an image detection system, and a control system. This biomimetic bud-removing claw dynamically monitors the pose of buds and pedicels using deep learning and stereoscopic vision. The clamping claw grips the main pedicel, while the biomimetic threads of the suction claw's fingertips grip the lateral buds. Based on pressure feedback, the rotation or contraction amplitude of the suction claw is adjusted to remove the lateral buds, achieving an effect similar to human hand-eye coordination during harvesting. This protects the main bud and pedicel from damage while achieving precise removal of lateral buds. The control method employs a deep fusion model of YOLOv8 and Mask RCNN to extract the main pedicel, left and right lateral buds, and their angle and orientation features, enabling the detection of kiwifruit pedicels and buds, and controlling the clamping and suction claws. Its effect is equivalent to a human's simultaneous observation, clamping, and harvesting control function.

Description

Technical Field

[0001] This invention relates to the field of kiwifruit bud thinning machinery, specifically to a kiwifruit biomimetic bud thinning claw and its control method. Background Technology

[0002] The quality of management at every stage of kiwifruit cultivation affects the economic value of the fruit. Among these, bud thinning is the most direct factor influencing kiwifruit quality. Bud thinning involves removing excess buds during the bud stage to reduce nutrient consumption by the tree, ensuring sufficient nutrients for the main bud's growth, thus preventing energy waste and maximizing yield. Bud thinning is a preliminary step; kiwifruit has a short flowering period but a long budding period, so usually only buds are thinned, not flowers. Only by doing a good job of bud thinning can kiwifruit yield be increased.

[0003] Currently, kiwifruit bud thinning is done manually. The distribution of flower stalks and buds on each branch is observed visually, and unwanted lateral buds and diseased buds are identified and broken off by twisting or pulling with the fingers. Typically, there is one main bud and two lateral buds on each main flower stalk, ultimately only the central main bud is retained. Kiwifruit buds are small, delicate, and easily damaged, increasing the difficulty of the process. At the same time, with rising labor costs and the large workload of manual bud thinning, the problem of kiwifruit bud thinning is becoming increasingly prominent and is a real issue in the development of the kiwifruit industry. Summary of the Invention

[0004] In order to overcome the above-mentioned defects and deficiencies in the prior art, the present invention provides a kiwifruit biomimetic bud-reducing claw and its control method.

[0005] To solve the above-mentioned technical problems, the present invention provides a kiwifruit biomimetic bud-reducing claw, comprising an inverted T-shaped frame, a clamping claw, a left suction claw, a right suction claw, an image detection system, and a motion control system. The clamping claw is located in the middle of the inverted T-shaped frame and connected to it; the left and right suction claws are symmetrically located on the left and right sides of the inverted T-shaped frame and connected to it. The inverted T-shaped frame includes a horizontal frame support, a vertical frame support, a rotary motor, and a fixed base. The rotary motor is located in the middle of the horizontal frame support and is mounted on the fixed base. A [missing information - likely a device or mechanism] is located on the upper side of the middle position of the horizontal frame support. The frame has a vertical support with a clamping claw on one side. The horizontal support has a left suction claw and a right suction claw at each end. A main RGBD camera is located in the middle of the horizontal support, and each of the left and right suction claws has an RGBD camera. The main RGBD camera and the two RGBD cameras are connected to an image detection system. The image detection system acquires images of flower buds and flower stalks and performs image detection using deep learning. It calculates the pose of the flower stalks and flower buds in real time. The control system controls the rotation of the inverted T-shaped frame, the clamping and locking of the main flower stalk by the clamping claws, and the bud thinning operation of the left and right suction claws.

[0006] Preferably, the clamping claw includes a clamping claw auxiliary motor, a main electric telescopic rod, a support base plate, a spherical joint, a middle support rod, a middle main electric push rod, side connecting rods, and an arc-shaped finger-face clamping plate; one end of the main electric telescopic rod is connected to the vertical support of the frame through the clamping claw auxiliary motor, and the other end is connected to the support base plate; the support base plate is provided with a middle support rod and two spherical joints, which are respectively connected to a side connecting rod; a middle main electric push rod is provided in the middle of the two side connecting rods, and the middle main electric push rod drives the side connecting rods to open or close around the spherical joints to both sides; the ends of the two side connecting rods are respectively connected to an arc-shaped finger-face clamping plate, and a flexible sensor is provided on the middle surface of the arc-shaped finger-face clamping plate to measure the pressure in real time; pneumatic flexible locking pouches are provided at both ends of the arc-shaped finger-face clamping plate for fixing the main flower stem.

[0007] Preferably, the left and right suction claws have the same structure. The left and right suction claws each include a side motor, a main electric push rod, a hinge, a slave electric push rod, a joint rotary motor, a finger adjustment base plate, a finger adjustment base, a finger adjustment push rod, and three fingers. One end of the main electric push rod is connected to the horizontal support of the frame through the side motor, and the other end is connected to the slave electric push rod through the hinge. The joint rotary motor is set in the hinge. The telescopic end of the slave electric push rod is equipped with a rotary motor. The rotary motor is equipped with a finger adjustment base plate. The finger adjustment base plate is equipped with a finger adjustment base. The finger adjustment base is equipped with a sliding groove. The finger adjustment push rod is slidably connected in the sliding groove through a limiting pin. One end of the finger adjustment push rod is equipped with a finger, which includes a finger root and a fingertip. One end of the finger root is connected to the finger adjustment push rod, and the other end of the finger root is connected to the fingertip.

[0008] Preferably, the fingertip is provided with a bionic nail, one side of the bionic nail is provided with a bionic muscle, and one side of the bionic muscle is provided with a bionic thread. The bionic thread includes a positive electrode, a negative electrode and a PDMS matrix. The PDMS matrix is ​​provided with a positive electrode and a negative electrode. The positive electrode and the negative electrode are combined to form a self-capacitance proximity sensor, which is used to measure the distance between the side bud and the bionic thread and the magnitude of the contact force on the side bud surface.

[0009] Preferably, a method for controlling the biomimetic bud-reducing claw of a kiwifruit includes the following steps:

[0010] Step S1: The kiwi-inspired bud-opening claw moves to the front of the kiwi flower bud;

[0011] Step S2: Perform angle visual servo control on the inverted T-shaped frame to make the straight line direction of the main flower stem consistent with the vertical direction of the image;

[0012] Step S3: Control the gripper; the gripper uses simple translation control, without too much fine control. The main RGBD camera detects the distance between the main flower stem and the gripper, and moves the gripper directly so that the main flower stem is in front of the gripper, so that the gripper can clamp the main flower stem tightly and prevent the main flower stem from shaking.

[0013] Step S4: Control the left and right suction claws. The control methods for the left and right suction claws are the same, that is, first perform visual servo control of the bud distance, and then perform visual servo control of the bud orientation. The visual servo control of the bud distance adjusts the distance between the left and right suction claws and the bud, and the visual servo control of the bud orientation ensures that the bud target is located in the center position of the left and right suction claws. When the left and right suction claws move to the position of the bud, they clamp the left and right buds. If the left and right suction claws do not move to the appropriate position, they return to the position before the distance visual servo control, and then repeat the adjustment of the distance and orientation with the bud until the optimal position is met.

[0014] Step S5: The clamping claws grip the main flower stalk, the left suction claw grips the left bud, and the right suction claw grips the right bud.

[0015] Step S6: The clamping claw, the left suction claw, and the right suction claw work together or individually to remove the left and right buds.

[0016] Preferably, the angle visual servo control steps of the inverted T-shaped frame are as follows: First, the image detection system obtains the straight position of the main flower stem, subtracts it from the vertical position in the image to obtain the angle deviation e(k), the deviation signal is sent to the fractional-order PID controller, and then the voltage control signal u(k) is obtained to drive the PWM pulse generator and drive the rotary motor to rotate a certain angle. The image detection system dynamically monitors the straight position of the main flower stem, so that the gripper tracks the main flower stem and remains perpendicular to the main flower stem, which facilitates accurate gripping of the main flower stem.

[0017] Preferably, the visual servo control steps for the lateral bud distance are as follows: based on the reference distance R... S The distance error is calculated from the actual distance fed back by the main RGBD displacement estimator. The distance error signal is input to the transfer function G2(S) related to the main controller. The displacement error is calculated based on the reference displacement value and the actual displacement fed back from the RGBD displacement estimator. The displacement error signal is input to the transfer function G1(S) related to the slave controller. Then, the transfer function L1(s) related to the electric actuator is calculated based on the slave controller voltage control signal U1(S). Finally, the final distance output signal Y2(S) is calculated based on U2(S), the load disturbance D(s), and the transfer function L2(s) of the main electric actuator. Y2(s) = U2(s) * L2(s) + D(s), where U2(S) is the inner loop output voltage or the outer loop input voltage of the main electric actuator, and * is convolution.

[0018] Preferably, the lateral bud orientation visual servo control steps are as follows: the set reference center coordinates are compared with the actual lateral bud center coordinates obtained by the RGBD sensor, and the resulting error is sent to the vision controller for adjustment. Then, the set reference angular displacement is compared with the actual angular displacement detected by the joint angular displacement sensor, and the calculated displacement error is input to the joint controller. The joint controller controls the joint rotation motor to adjust the orientation amplitude.

[0019] Preferably, in the visual servo control of the inverted T-shaped frame angle, the visual servo control of the distance between the left and right buds, and the visual servo control of the bud orientation, a target detection and information fusion method based on YOLOv8 and Mask RCNN is adopted to achieve deep learning-based target detection and feature extraction. Specific steps include: first, acquiring a certain number of RGB images and performing image augmentation processing, followed by image annotation; then constructing a database, including a training database, a validation database, and a test database; selecting a target detection model, and using the training database to perform visual servo control on YOLOv8 and Mask RCNN respectively. The R-CNN model is trained and evaluated using performance metrics. If the training fails to meet the requirements, the model is rebuilt, parameters are selected, and training continues until the requirements are met. If the training succeeds, the model is validated on a validation dataset, and the validation results are evaluated. If the validation metrics fail to meet the requirements, the model and parameters are modified, and training continues. After training, validation continues until the validation meets the requirements. Once the model meets the requirements, it is tested on a test dataset. During testing, the YOLOv8 model is first used to obtain object detection and instance segmentation maps of the main flower stalk, main flower bud, left bud, and right bud, as well as the tilt angle of the main flower stalk and the center coordinates of the left and right buds. Then, the regions of the main flower stalk, main flower bud, left bud, and right bud are used as initial proposed regions, and then a Mask is applied. The R-CNN model is tested on the image, obtaining new object detection and instance segmentation maps of the main flower stalk, main flower bud, left bud, and right bud, as well as the tilt angle of the main flower stalk and the center coordinates of the left and right buds. The tilt angle of the main flower stalk and the center coordinates of the left and right buds obtained from the YOLOv8 model and the Mask R-CNN model are fused using the following formula:

[0020] θ f =wy olo *θ yolo +w mask *θ mask , tc f =wc yolo *tc yolo ,+wc mask *tc mask ,

[0021] Where, θ yolo With θ maskThe main flower stalk tilt angles obtained from the YOLOv8 model and the Mask RCNN model are w, respectively. yolo with w mask For fusion weights, θ f The angle of inclination of the main flower stalk after fusion, tc yolo With tc mask The center coordinates of the left and right buds, obtained from the YOLOv8 model and the Mask RCNN model, are wc. yolo with wc mask To integrate weights, tc f The coordinates of the center of the left and right buds are obtained after fusion; finally, the fused parameters are transmitted to the visual servo controller.

[0022] Preferably, the Mask RCNN model consists of two stages. The first stage traverses the entire image and generates proposed regions. The second stage performs object classification on the proposed regions to obtain bounding boxes and masks.

[0023] The beneficial technical effects achieved by this invention are as follows: In use, the kiwifruit biomimetic bud-thinning claw utilizes an image detection and control system to detect, locate, and identify the main flower stalk, main bud, and side buds. The flexible gripping claw, through an arc-shaped fingertip clamp, adaptively senses the dynamic force and drives the clamp to tighten the flower stalk. The left and right suction claws simultaneously grasp the side buds on both sides and use their fingertip threads to adhere and fix them. The side buds are removed through rotation or backward movement. The effect is equivalent to the human harvesting process; buds are thinned while observing, one hand holding the flower stalk, and the other hand removing the buds. The adaptive dynamic adjustment of finger force protects the delicate buds from damage while removing excess buds. The control method of the kiwifruit bud-thinning claw provided by this invention includes visual servo control of the inverted T-shaped frame angle, gripping claw control, and left and right suction claw control. The inverted T-shaped frame's overall angle control ensures the vertical support of the frame is parallel to the main flower stalk, facilitating direct translation and clamping of the gripping claws. Left and right suction claw control includes distance and orientation visual servo control, respectively controlling the extension and retraction of the dual electric push rods and the rotation of the joint motors. Visual feature detection employs a deep fusion model of YOLOv8 and Mask RCNN to extract the main flower stalk, left and right buds, and their angle and orientation features. The controller uses a fractional-order PID model, optimized for its fractional order using a Tyrannosaurus Rex hunting algorithm. This control method, through deep learning image detection and instance segmentation algorithms, detects the kiwi flower stalk and buds, and controls the gripping and suction claws, achieving an effect equivalent to a human's "see, grab, and pick simultaneously" control function. Attached Figure Description

[0024] Figure 1 This is a three-dimensional schematic diagram of a kiwifruit biomimetic bud-opening claw;

[0025] Figure 2This is a left view of a kiwifruit-inspired bud-reducing claw.

[0026] Figure 3 This is a top view of a kiwifruit-inspired bud-like claw.

[0027] Figure 4 yes Figure 2 A magnified view of part A;

[0028] Figure 5 yes Figure 2 A magnified view of part B;

[0029] Figure 6 yes Figure 3 A magnified view of a portion of C;

[0030] Figure 7 This is a schematic diagram of a biomimetic thread structure;

[0031] Figure 8 This is a flowchart of the control method for the biomimetic bud-reducing claw of kiwifruit;

[0032] Figure 9 This is the flowchart for the visual servo control of the inverted T-shaped frame angle;

[0033] Figure 10 This is the flowchart of the lateral bud visual distance servo control.

[0034] Figure 11 This is the flowchart of the visual servo control for the side bud orientation;

[0035] Figure 12 This is a flowchart of the target detection fusion process;

[0036] Figure 13 It is an RGB image;

[0037] Figure 14 It involves augmenting the image. Figure 14 (a) is the image rotated 90 degrees to the left. Figure 14 (b) is a 90-degree right rotation. Figure 14 (c) is a vertically flipped image. Figure 14 (d) is a horizontally flipped image.

[0038] Figure 15 It is image annotation;

[0039] Figure 16 These are the coordinates of the center of the lateral bud. Detailed Implementation

[0040] The present invention will be further described below with reference to specific embodiments. These embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0041] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0042] like Figures 1-7 As shown, a kiwifruit biomimetic bud-reducing claw includes an inverted T-shaped frame, a clamping claw, a left suction claw, a right suction claw, an image detection system, and a control system. The clamping claw is located in the middle of the inverted T-shaped frame and connected to it. The left and right suction claws are symmetrically located on the left and right sides of the inverted T-shaped frame and connected to it. The inverted T-shaped frame includes a horizontal frame support 3, a vertical frame support 4, a rotary motor 2, and a fixed base 1. The rotary motor 2 is located in the middle of the horizontal frame support 3 and is mounted on the fixed base 1. The vertical frame support 4 is located on the upper side of the middle of the horizontal frame support 3, and the clamping claw is located on one side of the vertical frame support 4. The horizontal support frame 3 has a left suction claw and a right suction claw at each end. The image detection system includes a main RGBD camera 5 and two RGBD cameras. A main RGBD camera 5 is set on one side of the middle position of the horizontal support frame. An RGBD camera 16 is set on each of the left and right suction claws. The main RGBD camera 5 and the two RGBD cameras are all connected to the image detection system. The image detection system acquires images of flower buds and flower stalks and performs image detection using deep learning. It calculates the pose of flower stalks and flower buds in real time. The control system controls the rotation of the inverted T-shaped frame, the clamping and locking of the main flower stalk by the clamping claws, and the bud thinning operation of the left and right suction claws.

[0043] The clamping claw includes a clamping claw auxiliary motor 6, a main electric telescopic rod 7, a support base plate 8, a spherical joint 9, a middle support rod 10, a middle main electric push rod 11, a side connecting rod 12, and an arc-shaped finger-face clamping plate 13. One end of the main electric telescopic rod 7 is connected to the vertical bracket 4 of the inverted T-shaped frame through the clamping claw auxiliary motor 6, and the other end is connected to the support base plate 8. The support base plate 8 is provided with a middle support rod 10 and two spherical joints 9. The two spherical joints 9 are respectively connected to a side connecting rod. A middle main electric push rod 11 is provided in the middle position of the two side connecting rods, and the middle main electric push rod drives the side connecting rods to open or close around the spherical joints. The ends of the two side connecting rods are respectively connected to an arc-shaped finger-face clamping plate 13. A flexible sensor 32 is provided on the middle surface of the arc-shaped finger-face clamping plate 13, which measures the pressure in real time and provides feedback to drive the arc-shaped finger-face clamping plate to clamp the main flower stem. Pneumatic flexible locking pouches 31 are provided at both ends of the arc-shaped finger-face clamping plate for fixing the main flower stem.

[0044] The left and right suction claws have the same structure, each including a side motor 24, a main electric push rod 23, a hinge 22, a driven electric push rod 20, a joint rotary motor 21, a finger adjustment base plate 19, a finger adjustment base 18, a finger adjustment push rod 17, and three fingers. One end of the main electric push rod 23 is connected to the horizontal support of the frame via the side motor 24, and the other end is connected to the driven electric push rod 20 via the hinge 22. The joint rotary motor 21 is located in the hinge. The telescopic end of the rod 20 is equipped with a rotary motor 25, on which a finger adjustment base plate 19 is mounted. The finger adjustment base plate 19 has a finger adjustment seat 18, which has a sliding groove 27. A finger adjustment push rod 17 is slidably connected to the sliding groove 27 via a limiting pin 26. One end of the finger adjustment push rod 17 has a finger, which includes a finger root 15 and a fingertip 14. One end of the finger root 15 is connected to the finger adjustment push rod 17, and the other end is connected to the fingertip 14. The fingertip has a bionic nail 28, and one side of the bionic nail 28 has a bionic muscle 29. One side of the bionic muscle 29 has a bionic thread 30, which has an electrostatic adsorption function for adsorbing side buds. The biomimetic thread 30 includes a positive electrode 34, a negative electrode 35, and a PDMS matrix 33. The PDMS matrix 33 is provided with a positive electrode 34 and a negative electrode 35. The positive electrode and the negative electrode are combined to form a self-capacitance proximity sensor, which is used to measure the distance between the side bud and the biomimetic thread and to measure the magnitude of the contact force on the side bud surface.

[0045] The rotary motor 2, the gripper auxiliary motor 6, the main electric telescopic rod 7, the intermediate main electric push rod 11, the finger adjustment push rod 17, the slave electric push rod 20, the joint rotary motor 21, the main electric push rod 23, the side motor 24, the rotary motor 25, the pneumatic flexible locking bladder 31, the flexible sensor 32, the positive electrode 34, and the negative electrode 35 are all connected to the motion control system.

[0046] The kiwifruit biomimetic bud-removing claw moves to the front of the main flower stalk. The main RGBD camera 5 and RGBD camera 16 detect and identify the main flower stalk, left flower stalk, right flower stalk, left bud, right bud, and main bud in real time, dynamically adjusting the position and posture of the flexible gripping claw and the left and right suction claws. The arc-shaped finger-surface clamp 13 moves to the main flower stalk and clamps it. The flexible sensor 32 measures the surface pressure of the arc-shaped finger-surface clamp 13. The image detection system and control system adjust the optimal distance between the two arc-shaped finger-surface clamps 13 based on the pressure feedback to ensure that the main flower stalk is not damaged. The pneumatic flexible locking bladder 31 locks the main flower stalk to prevent it from sliding. The left and right suction claws adjust the relative position of the adjustment push rods 17 of each finger so that the biomimetic thread 30 approaches and contacts the side bud. The motion control system can measure the distance between the side bud and the biomimetic thread and the magnitude of the surface contact force to ensure that the fingertip pressure on the side bud is controllable. The multiple fingers of the left and right suction claws clamp the left and right buds respectively, and apply an electric field to the positive electrode 34 and negative electrode 35 of the biomimetic thread 30 to attract the lateral buds, ensuring that the buds on both sides neither fall off nor are crushed. Slightly move the suction claws to separate them from the main flower stalk by a small distance, which facilitates the subsequent bud thinning process and avoids affecting the main bud.

[0047] The biomimetic thread 30 at the fingertip has functions of lateral bud distance sensing, touch force sensing, and electrostatic adsorption. The anode 34 and cathode 35 constitute a self-capacitance complementary proximity sensor. When the anode 34 and cathode 35 operate in independent "self-capacitance" mode, they can measure the distance between the lateral bud and the biomimetic thread to determine whether the fingertip biomimetic thread is in contact with the buds on both sides. When the fingertip biomimetic thread contacts the surface of the lateral bud, the anode 34 and cathode 35 operate in "mutual capacitance" mode to measure the touch force on the lateral bud surface, ensuring that the fingers of the adsorption claw can grasp the lateral bud without crushing it. The anode 34 and cathode 35 form a spiral shape and are semi-embedded in the PDMS matrix 33. When the biomimetic thread is used in the bud-picking mode, the electrodes form an electrostatic adsorption biomimetic thread, with the thread surface at zero degrees to the lateral bud contact surface, and the adsorption force on the working surface is...

[0048] F = TS + Rw

[0049] Where τ is the frictional stress, R is the effective adsorption area, R is the adsorption surface energy, and w is the effective adsorption width. Both τ and R are related to the applied voltage V. The magnitude of the voltage V is determined by the motion control system, which in turn controls the magnitude of F to realize the electrostatic adsorption function of the fingertip bionic thread, complete the suction of the side bud, and ensure that the side bud is not easily detached or crushed when the adsorption claw rotates or moves backward.

[0050] There are two methods for thinning flower buds: rotation and retraction. Taking right-side thinning as an example, the right suction claw grips the right-side bud, moves a short distance, and then the control system drives a rotary motor to rotate, separating the right-side bud from the right flower stalk, thus achieving right-side thinning. This is equivalent to a person grasping a flower bud with their fingers and rotating their wrist to remove it. Taking left-side thinning as an example, the left suction claw grips the left-side bud, moves a short distance, and then the control system drives the suction claw's electric push rod to retract, pulling the left-side bud away from the left flower stalk, thus achieving left-side thinning. This is equivalent to a person grasping a flower bud with their fingers and moving their fingers to pull it apart.

[0051] The movement of the kiwifruit biomimetic bud-opening claw is mainly controlled by an image detection system and a control system. Image detection is a prerequisite for accurate control. Through deep learning-based image detection of flower stalks and buds, precise control of the motor and push rod is achieved.

[0052] like Figure 8 As shown, the biomimetic bud-sparser claw is controlled by the robot body to precisely control the bud thinning process in front of the kiwi flower buds. First, the inverted T-shaped frame is subjected to angle visual servo control to ensure that the straight direction of the main flower stalk is aligned with the vertical direction of the image. The specific control process is as follows: Figure 9 As shown; then, the left suction claw, clamping claw, and right suction claw are controlled separately. The clamping claw uses simple translation control, requiring no excessive fine control. As long as the main RGBD camera detects the distance between the main flower stem and the clamping claw, the clamping claw can be moved directly. Image detection ensures that the main flower stem is directly in front of the clamping claw, facilitating the clamping claw to firmly grip the main flower stem and prevent it from wobbling. The left and right suction claws are symmetrically set and controlled in the same way. The control of the left and right suction claws includes two parts: distance visual servo control and side bud orientation visual servo control. The distance visual servo control is as follows: Figure 10 As shown, the main control system uses two electric actuators, one master and one slave, to adjust the distance between the tip of the bionic finger and the lateral bud. The lateral bud's orientation is controlled by visual servo control as follows: Figure 11 As shown, the main control involves two joint rotary motors to position the lateral bud target at the center of the bionic finger at the end, facilitating adsorption and bud thinning. When the bionic thread of the end finger moves to the position of the lateral bud, it can tighten the bionic thread at the fingertip of the three fingers to clamp the left and right lateral buds. If the bionic thread of the end finger does not move to the appropriate position, the program returns to the position before the visual servo control, and repeats the adjustment of the distance and orientation to the lateral bud until the optimal position is achieved. Finally, after the gripping claw clamps the main flower stem and the left and right adsorption claws clamp the left and right lateral buds, the three claws work together to remove the left and right lateral buds, completing the function of the bionic bud thinning claw. It should be noted that the left and right adsorption claws can work independently or together. The visual image detection and segmentation of this invention adopts a deep learning method, with the models being YOLOv8 and Mask RCNN. Its target detection and information fusion are as follows: Figure 12 As shown.

[0053] The flowchart of the inverted T-shaped frame angle visual servo control is as follows: Figure 9 As shown, the angle control of the biomimetic bud-sparse claw of the kiwifruit is achieved by first obtaining the straight-line position of the main flower stalk through the image detection system, and then subtracting it from the vertical position in the image to obtain the angle deviation e(k). Figure 9 The deviation signal (θ) is sent to the fractional-order PID controller to obtain the control signal u(k), which drives the PWM pulse generator and drives the rotary motor to rotate a certain angle. The image detection system dynamically monitors the straight position of the main flower stem, so that the flexible clamping hand tracks the main flower stem and keeps it perpendicular to the main flower stem, which facilitates accurate clamping of the main flower stem.

[0054] The angle controller uses a fractional-order PID controller, whose expression is G. FPID =K p +K I s -λ +K D s μ K p ,K I ,K D λ and μ are the proportional gain, integral gain, differential gain, exponent of the integral term, and exponent of the differential term, respectively, and s is the complex frequency.

[0055] Let the image of the straight position of the main flower stalk detect the transfer function. Image detection gain is K s The time constant is T s To obtain the optimal fractional order, the Tyrannosaurus Rex hunting algorithm can be used for optimization. First, define the objective function:

[0056] J obj =(1-e -β )*(w o *M p +E ss )+e -β (T set -T r (1)

[0057] In the formula w o M represents the weighting coefficient. p For overshoot, E ss For steady-state error, T set The stationary time is T, β is the weighting factor, and T is the stationary time. r The rising time.

[0058] Then, N initial prey location solutions are generated according to formula (2).

[0059] X i =rand(np,di)*(ub-lb)+lb (2)

[0060] In the formula X i = [x1, x2, x3...x n [] represents the location of the prey, np represents the population size, n represents the dimension, di represents the dimension of the search space, ub and lb represent the upper and lower bounds respectively, and rand is the function for generating random numbers.

[0061] When a Tyrannosaurus Rex sees the nearest prey, it will attempt to hunt. Sometimes this is to protect itself from being hunted, or it may try to escape. Tyrannosaurus Rex hunting involves both chasing and capturing prey, so when a Tyrannosaurus Rex hunts, it hunts randomly, and the formula for finding a new hunting location is:

[0062]

[0063] In the formula, Er is the estimated distance to the dispersed prey, rand() is a random function, and Random is a random number. That is, when the Tyrannosaurus Rex begins hunting, the prey begins to disperse, and the dinosaur updates the prey's position to hunt it.

[0064] x new =x+rand()*sr*(tpos*tr-targ*pr) (4)

[0065] In the formula, sr is the hunting success rate between [0.1, 1], tpos is the Tyrannosaurus Rex position, and x represents the original hunting position. If the success rate is 0, it means the prey escaped, the hunt failed, and the prey position must be updated accordingly. targ is the minimum position from the prey to the Tyrannosaurus Rex. tr is the Tyrannosaurus Rex's running speed, which is considered to be between [0.067, 0.3]. pr is the prey's running speed, which is between [0, 1], and the prey's running speed should be less than the Tyrannosaurus Rex's speed. The selection process depends on the prey's position, i.e., the target prey's current position and previous position. If the Tyrannosaurus Rex fails to hunt, or if the prey escapes or protects itself from being hunted, the prey's position becomes zero. This is achieved by comparing the fitness function.

[0066]

[0067] In the formula, f(X) is the fitness function of the initial random prey position, f(X) new ) is the fitness function for updating the prey position. The optimal orders λ and μ are obtained through iterative calculations based on the objective function, the hunting position, and the minimum distance.

[0068] The flowchart of the visual distance servo control for the left and right buds is as follows: Figure 10 As shown, each suction claw of the biomimetic bud-opening claw can be equivalent to a two-link robotic arm, including a master electric actuator and a slave electric actuator, with the end suction claw located on the slave electric actuator. The distance control of the suction claw is as follows: Figure 10 As shown. The electric actuator provides fine micro-distance control over the end suction claw, while the main electric actuator provides coarse distance control over the main body of the suction claw. The two work together to control the distance between the suction claw and the left and right side buds.

[0069] G1(s) and G2(s) represent the transfer functions associated with the slave controller and master controller, respectively. Furthermore, L1(s) and L2(s) are the transfer functions associated with the objects in the inner and outer loops, respectively. The final distance Y2(s) of the system is affected by the load disturbance D(s), calculated as follows:

[0070] Y2(s)=U2(s)*L2(s)+D(s) (6)

[0071] In the formula, U2(s) is the inner loop output or the outer loop input. U2(S) controls Y(s) to track the signal R(s). U1(s) is the control signal from the controller voltage. Similarly, the inner loop output Y1(s) can be obtained in the following way:

[0072] Y1(s)=U2(s)=U1(s)*L1(s) (7)

[0073] The control system utilizes two fractional-order controllers to construct a cascaded system. The FPI controller is cascaded with the TDμ controller, where the FPI controller constitutes the master controller G2(s) and the TDμ controller slave G1(s).

[0074]

[0075]

[0076] Therefore, the closed-loop transfer function of the cascaded system is expressed as follows:

[0077]

[0078] The order of the two fractional-order coupled controllers was also calculated using the Tyrannosaurus Rex hunting algorithm.

[0079] The flowchart of the visual servo control for the adsorption claw side bud orientation is as follows: Figure 11As shown, when the end effector of the bionic bud-removing claw approaches the target lateral bud, precise control of the claw's orientation is required. Orientation detection directly affects the effectiveness of lateral bud removal; the orientation image detection of the lateral bud and the control accuracy of the claw are prerequisites for successful lateral bud removal. The orientation control of the claw mainly includes a vision controller and a joint controller. The vision controller compares the lateral bud center coordinates obtained by the RGBD sensor with the set reference center coordinates, and the resulting error is sent to the vision controller for adjustment. The joint controller uses a PD control strategy to drive the joint rotation motor for drive control, and its stability is illustrated using a Lyapunov function. The overall system achieves "hand-eye coordination," with the vision controller observing the orientation error and then instructing the joint controller to adjust the orientation range.

[0080] The vision controller's task is to track the center coordinates of the lateral target using an RGBD camera on the end effector, establishing a relationship between changes in image features and joint angles. The information represented by the image is first processed, then converted to a position relative to the camera based on an ideal pinhole camera model, and further converted to coordinates relative to a base frame using the relationship between the object and the camera. Thus, the coordinates (X, Y, Z) of the object point are represented as the corresponding image point (u, v) as follows:

[0081]

[0082] in f is the RGBD camera intrinsic matrix, representing the relationship between camera frames and image frames, which can be obtained by measurement or calculation given a field of view. x and f y ρ is the effective focal length of the camera, expressed in pixels along the xc and yc axes, and ρ is the camera tilt factor. and This indicates the difference between the camera center and the image center. This indicates the relationship between object frames and camera frames. Defined as a rotation matrix, t is the translational displacement from the camera to the object, which can be determined by the equivalent angular axis constructed from the polar angle and the azimuth angle.

[0083] The design of the vision controller and the selection of the associated control gain need to be examined in the image Jacobian matrix, which correlates the feature velocity with the camera velocity in image coordinates. Definition and These represent the linear velocity and angular velocity of the camera relative to the image frame, respectively. A point P(X, Y, Z) in the camera frame and a point P(u, v) in the corresponding projected image space can be linked through the image Jacobian matrix J.

[0084]

[0085] in Feedback control of the adsorption claw requires image frame error. If the desired image lateral bud center position is defined as (u... d , v d The desired center is O(0,0), where (u0, v0) = (u0, v0). We can set u0 = 0 and v0 = 0 to allow the RGBD camera of the suction claw to constantly detect and track the side bud, ensuring the suction claw aligns with the center of the side bud for easy grasping. Since the angle and distance are driven by other controllers, this vision controller only considers the side bud orientation error. The orientation error is defined as:

[0086] (e1, e2) = (u - u0, v - v0) (13)

[0087] Where e1 is the horizontal error term, e2 is the vertical error term, u0 is the horizontal coordinate value of the image center (set to 0), and v0 is the coordinate value of the image center (set to 0). During the alignment process between the lateral bud center and the center of the image plane, the vision controller employs a PD control strategy.

[0088]

[0089] Where (v x ,v y ) is the translation speed relative to the current camera frame; k pi ,k di Let i = 1, 2, and be the positive gain. Taking the derivative of its equation, the error dynamics approach 0 from the image Jacobian matrix J and its equation, along with the controller equation.

[0090]

[0091] in, The meaning is that f is the effective focal length of the camera, and Zc is the depth distance.

[0092] The joint controller does not consider friction and gravity effects; the equation for the adsorption claw is:

[0093]

[0094] In the formula, D is the positive definite inertia matrix, and C represents the centrifugal and Coriolis force terms. For joint acceleration, To determine the joint speed, the joint controller also selects the PD control rate.

[0095]

[0096] Where the tracking error e = q d -q, when using fixed-point control, q d Since it is a constant, The adsorption claw equation is:

[0097]

[0098] Where K d K represents the differential coefficient of the controller. p For the proportional term coefficient of the controller,

[0099] Take the Lyapunov function as

[0100]

[0101] Due to D, K p If V is positive definite, then V is also globally positive definite.

[0102]

[0103] in If obliquely symmetrical, then

[0104]

[0105] because It is negative semi-definite and K d If the value is positive definite, then the adsorption claw is controlled and globally asymptotically stable.

[0106] Visual object detection flowchart as follows Figure 12 As shown. In the visual servo control of the inverted T-shaped frame angle, the visual servo control of the distance between the left and right buds, and the visual servo control of the orientation of the left and right adsorption claw buds, feature extraction and target detection of the image are required, including the main flower stalk, main flower bud, left and right buds, the offset angle between the straight line direction of the main flower stalk and the perpendicular direction of the image, the RGBD distance value, and the center coordinates of the left and right buds. Therefore, it is necessary to adopt a target detection and information fusion method based on YOLOv8 and Mask RCNN to realize deep learning target detection and feature extraction, and provide visual sensing feedback signals for the control algorithm. First, a certain number of RGB images (such as...) are acquired. Figure 13 As shown), and the image is augmented (e.g. Figure 14 (Including but not limited to 90-degree leftward rotation, 90-degree rightward rotation, vertical flip, horizontal flip, etc.), and then image annotation is performed to obtain the main flower stalk, main flower bud, and left and right lateral buds (see...). Figure 15 (and the coordinates of the center of the left and right lateral buds. See the diagram illustrating how the coordinates of the center of the left and right lateral buds were obtained.) Figure 16 With the image center O(0,0) as the origin, it is possible to, for example, t 1左 t 1右 t 2左 t 2右Four coordinate values ​​are used. Using the Y-axis as the primary direction, the offset angles between the straight line of the main flower stalk and the perpendicular direction of the image are obtained, such as θ1 and θ2. Then, a database is constructed, including a training database, a validation database, and a test database. An object detection model is selected, and the YOLOv8 and Mask R-CNN models are trained using the training database respectively. The models are evaluated using performance metrics (Formulas 13-16). If the training fails to meet the requirements, the model is rebuilt and parameters are selected for another round of training until the training meets the requirements. If the training meets the requirements, the model is validated on the validation dataset, and the validation effect is evaluated. If the validation metrics fail to meet the requirements, the model and parameters are modified in the new model section, and the next round of training is conducted. After training is completed, validation continues until the validation meets the metric requirements. Once the model meets the requirements, it is tested on the test dataset. The testing phase included using both the YOLOv8 and Mask R-CNN models. First, the YOLOv8 model was used to test the images, obtaining object detection and instance segmentation maps for the main flower stalk, main flower bud, left bud, and right bud, as well as the tilt angle of the main flower stalk and the center coordinates of the left and right buds. Then, these regions were used as initial "suggested regions," and the Mask R-CNN model was used to test the images, obtaining new object detection and instance segmentation maps for the main flower stalk, main flower bud, left bud, and right bud, along with the tilt angle of the main flower stalk and the center coordinates of the left and right buds. Next, the tilt angle of the main flower stalk and the center coordinates of the left and right buds obtained from the YOLOv8 and Mask R-CNN models were fused using the following formula:

[0107] θ f =w yolo *θ yolo +w mask *θ mask (twenty two)

[0108] tc f =wc yolo *tc yolo +wc mask *tc mask (twenty three)

[0109] Where θ yolo With θ mask The main flower stalk tilt angles obtained from the YOLOv8 model and the Mask RCNN model are w, respectively. yolo with w mask For fusion weights, θ f The angle of inclination of the main flower stalk after fusion. tc yolo With tc mask The coordinates of the center of the left and right buds obtained by the YOLOv8 model and the Mask R-CNN model are respectively, wc yolo with wc maskTo integrate weights, tc f These are the coordinates of the center of the left and right buds after fusion. Finally, the fused parameters are transmitted to the visual servo controller.

[0110] Leveraging the dual capabilities of YOLOv8 for object detection and instance segmentation, real-time detection is achieved by transforming the detection problem into a regression problem. For the biomimetic bud-sparse claw, four object classes are considered: main flower stalk, main flower bud, left bud, and right bud. Each bounding box includes a category attribute, bounding box coordinates, confidence score, main flower stalk tilt angle θ, and center coordinates t of the left and right buds.

[0111] The loss function measures the distance between the predicted information and the expected information (label) of the neural network. The closer the predicted information is to the expected information, the smaller the loss function value. The YOLO loss function L is defined as follows: yolo It includes five parts, namely, the category loss error E class Rectangular frame coordinate error E corrd Regression loss IOU error E iou Error E of the tilt angle of the main flower stalk θ , the coordinate error of the center of the left and right buds E center .

[0112] L yolo =w1*E class +w2*E corrd +w3*E iou +w4*E θ +w5*E center (twenty four)

[0113] In the formula, w1, w2, w3, w4, and w5 are the weights of the five types of errors, which can be set according to their importance, and satisfy w1+w2+w3+w4+w5=1.

[0114]

[0115]

[0116]

[0117]

[0118]

[0119] In the formula, L is the size of the segmented grid, N is the number of classification targets, and t′ x ,t′ y ,t′ w ,t′ h ,c′,p′(c),θ′ x ,t′ cx ,t′cy The parameters in the label are: the horizontal and vertical coordinates of the regression boxes for the four object classes, their width and height, the confidence score of the regression boxes, the probability of the target class, the tilt angle of the main flower stalk, and the center coordinates of the left and right buds. The corresponding network prediction result is t. x , t y , t w , t h c, p(x), θ x , t cx , t cy . λ corrd λ represents the weight of the coordinate error coefficients. no For non-target regression loss error weights, For the j-th predicted bounding box in the i-th grid, the label information corresponding to its value of 1 is assigned, where p′ i (c) represents the target class probability of the i-th grid, c′ i Let p be the confidence score of the regression box for the i-th grid. i (c) represents the predicted probability value of the target category in the i-th grid, c i is the predicted confidence value of the regression box in the i-th grid.

[0120] Mask R-CNN is a two-stage framework. The first stage traverses the entire image and generates "proposal regions." The second stage performs object classification on the proposed regions, obtaining bounding boxes and masks. The "proposal regions" are obtained by YOLOv8 detection of the main flower stalk, main flower bud, left bud, and right bud. The second stage performs pixel classification and detection, obtaining instance segmentation results, masks, and bounding boxes. It can also obtain the tilt angle θ of the main flower stalk and the center coordinates t of the left and right buds. A corresponding loss function can be defined for training, and the multi-task loss function L... mask-rcnn It consists of three parts: category loss L cls The enclosure box is missing L box And prediction mask loss L mask L cls It is the difference between the predicted and actual values ​​for the main flower stalk, main flower bud, and lateral bud categories; L box This represents the distance between the predicted and actual location parameters (origin, width and height, center coordinate t, tilt θ) for each instance; and L mask This represents the model confidence in a binary classification of each pixel in the foreground object (main flower stalk and buds, etc.) and the background. It is the binary cross-entropy used for pixel classification.

[0121] L mask-rcnn =L cls +L box +L mask (29)

[0122]

[0123]

[0124]

[0125] In the formula L cls For category loss; L box The enclosure is missing; L mask To predict the mask loss; p i and Let N be the predicted probability and the true value of anchor point i; reg t is the number of pixels in the feature map; i and t represents the bounding box coordinates of the predicted and actual values ​​for the flower stalk and flower bud, respectively; c and These are the center coordinates of the predicted and actual values ​​of the left and right buds, respectively; t θ and These are the predicted and actual values ​​of the main flower stalk tilt angle, respectively; R(·) is the smoothing L1 function. For the mask branch, each ROI generates m... 2 The dimensions are output; therefore, This represents the true coordinates (i,j) in the m×m region, and y ij This indicates the prediction result.

[0126] The performance evaluation metrics for YOLOv8 and Mask R-CNN network models consist of average accuracy (AP), average recall, F1 score, and frames per second (FPS). The first three parameters are used to evaluate the accuracy of flower stalk and bud location detection and segmentation, while frames per second are used to evaluate the algorithm's speed. The accuracy and recall mentioned above are represented as the proportion of correctly predicted positive samples out of the total predicted positive samples, and the proportion of correctly predicted positive samples out of the total positive samples, respectively. The calculation methods for these evaluation metrics are shown in equations (13-16).

[0127]

[0128]

[0129]

[0130]

[0131] In the formula, TP represents true positives; FP represents false positives; FN represents false negatives; NmF represents the total training inference images; and TT represents the total time required for training inference.

[0132] The present invention has been disclosed above with reference to preferred embodiments, but it is not intended to limit the scope of the invention. All technical solutions obtained by adopting equivalent substitutions or equivalent transformations fall within the protection scope of the present invention.

Claims

1. A kiwifruit biomimetic bud-reducing claw, comprising an inverted T-shaped frame, a clamping claw, a left suction claw, a right suction claw, an image detection system, and a control system, characterized in that: The clamping claw is located in the middle of the inverted T-shaped frame and connected to it. The left and right suction claws are symmetrically located on the left and right sides of the inverted T-shaped frame and connected to it. The inverted T-shaped frame includes a horizontal frame support, a vertical frame support, a rotary motor, and a fixed base. The rotary motor is located in the middle of the horizontal frame support and is mounted on the fixed base. The vertical frame support is located above the middle of the horizontal frame support, and a clamping claw is located on one side of the vertical frame support. A left suction claw and a right suction claw are located at opposite ends of the horizontal frame support. A main suction claw is located on one side of the middle of the horizontal frame support. An RGBD camera system is used, with one RGBD camera on each of the left and right suction claws. The main RGBD camera and the two RGBD cameras are connected to an image detection system. This system acquires images of flower buds and stems and performs image detection using deep learning technology, calculating the pose of the flower buds and stems in real time. The control system controls the rotation of the inverted T-shaped frame, the clamping and locking of the main flower bud by the gripping claws, and the bud thinning operations of the left and right suction claws. In the visual servo control of the inverted T-shaped frame angle, the visual servo control of the distance between the left and right lateral buds, and the visual servo control of the lateral bud orientation, a target detection and information fusion method based on YOLOv8 and Mask RCNN is adopted to achieve deep learning-based target detection and feature extraction. Specific steps include: first, acquiring a certain number of RGB images and performing image augmentation and annotation; then constructing a database, including a training database, a validation database, and a test database; selecting a target detection model, and using the training database to test YOLOv8 and Mask RCNN respectively. The R-CNN model is trained and evaluated using performance metrics. If the training fails to meet the requirements, the model is rebuilt and parameters are selected for another round of training until the requirements are met. If the training succeeds, the model is validated on a validation dataset, and the validation results are evaluated. If the validation metrics fail to meet the requirements, the model and parameters are modified and the next round of training is conducted. After training, validation continues until the validation meets the requirements. Once the model meets the requirements, it is tested on a test dataset. During testing, the YOLOv8 model is first used to obtain object detection and instance segmentation maps of the main flower stalk, main flower bud, left bud, and right bud, as well as the tilt angle of the main flower stalk and the center coordinates of the left and right buds. These regions are used as initial proposed regions. Then, the Mask R-CNN model is used to test the images, obtaining new object detection and instance segmentation maps of the main flower stalk, main flower bud, left bud, and right bud, as well as the tilt angle of the main flower stalk and the center coordinates of the left and right buds. The YOLOv8 model and Mask R-CNN model are then compared. The main flower stalk tilt angle and the center coordinates of the left and right buds obtained from the RCNN model are fused using the following formula: i f =w yolo *i yolo +w mask *i mask ,tc f =wc yolo *tc yolo +wc mask *tc mask , Where, θ yolo With θ mask The main flower stalk tilt angles obtained from the YOLOv8 model and the Mask RCNN model are w, respectively. yolo with w mask For fusion weights, θ f The angle of inclination of the main flower stalk after fusion, tc yolo With tc mask The coordinates of the center of the left and right buds obtained by the YOLOv8 model and the Mask R-CNN model are respectively, wc yolo with wc mask To integrate weights, tc f The coordinates of the center of the left and right buds are obtained after fusion; finally, the fused parameters are transmitted to the visual servo controller.

2. The kiwifruit biomimetic bud-reducing claw according to claim 1, characterized in that: The clamping claw includes a clamping claw auxiliary motor, a main electric telescopic rod, a support base plate, spherical joints, a middle support rod, a middle main electric push rod, side connecting rods, and arc-shaped finger-face clamping plates. One end of the main electric telescopic rod is connected to the vertical support of the frame via the clamping claw auxiliary motor, and the other end is connected to the support base plate. The support base plate is equipped with a middle support rod and two spherical joints, which are respectively connected to a side connecting rod. A middle main electric push rod is located in the middle of the two side connecting rods, and is driven by the middle main electric push rod to open or close the side connecting rods around the spherical joints. The ends of the two side connecting rods are respectively connected to an arc-shaped finger-face clamping plate. The middle surface of the arc-shaped finger-face clamping plate is equipped with a flexible sensor to measure the pressure in real time. The two ends of the arc-shaped finger-face clamping plate are equipped with pneumatic flexible locking pouches for fixing the main flower stem.

3. The kiwifruit biomimetic bud-reducing claw according to claim 1, characterized in that: The left and right suction claws have the same structure. Each suction claw includes a side motor, a main electric push rod, a hinge, a slave electric push rod, a joint rotary motor, a finger adjustment base plate, a finger adjustment base, a finger adjustment push rod, and three fingers. One end of the main electric push rod is connected to the horizontal support of the frame via the side motor, and the other end is connected to the slave electric push rod via the hinge. The joint rotary motor is located in the hinge. The telescopic end of the slave electric push rod is equipped with a rotary motor. The rotary motor is equipped with a finger adjustment base plate, and the finger adjustment base plate is equipped with a finger adjustment base. The finger adjustment base plate is equipped with a sliding groove, and the finger adjustment push rod is slidably connected in the sliding groove via a limiting pin. One end of the finger adjustment push rod is equipped with a finger, which includes a finger root and a fingertip. One end of the finger root is connected to the finger adjustment push rod, and the other end of the finger root is connected to the fingertip.

4. The kiwifruit biomimetic bud-reducing claw according to claim 3, characterized in that: The fingertip is provided with a bionic nail, one side of which is provided with a bionic muscle, and the other side of which is provided with a bionic thread. The bionic thread includes a positive electrode, a negative electrode and a PDMS matrix. The PDMS matrix is ​​provided with a positive electrode and a negative electrode. The positive electrode and the negative electrode are combined to form a self-capacitance proximity sensor, which is used to measure the distance between the side bud and the bionic thread and the magnitude of the contact force on the side bud surface.

5. A method for controlling the biomimetic bud-reducing claw of kiwifruit as described in any one of claims 1-4, comprising the following steps: Step S1: The kiwi-inspired bud-opening claw moves to the front of the kiwi flower bud; Step S2: Perform angle visual servo control on the inverted T-shaped frame to make the straight line direction of the main flower stem consistent with the vertical direction of the image; Step S3: Control the gripper; The gripper uses simple translation control, which does not require too much fine control. The main RGBD camera detects the distance between the main flower stem and the gripper, and moves the gripper directly to make the main flower stem directly in front of the gripper, so that the gripper can clamp the main flower stem tightly and prevent the main flower stem from shaking. Step S4: Control the left and right suction claws. The control methods for the left and right suction claws are the same. First, perform visual servo control of the bud distance, and then perform visual servo control of the bud orientation. The visual servo control of the bud distance adjusts the distance between the left and right suction claws and the bud. The visual servo control of the bud orientation ensures that the bud target is located at the center position of the left and right suction claws. When the left and right suction claws move to the position of the bud, they clamp the left and right buds. If the left and right suction claws do not move to the appropriate position, return to the distance vision servo control and repeat the adjustment of the distance and orientation with the side bud until the optimal position is met. Step S5: The clamping claws grip the main flower stalk, the left suction claw grips the left bud, and the right suction claw grips the right bud. Step S6: The clamping claw, the left suction claw, and the right suction claw work together or individually to remove the left and right buds.

6. The control method according to claim 5, characterized in that: The angle visual servo control steps of the inverted T-shaped frame are as follows: The image detection system obtains the straight position of the main flower stem, subtracts it from the vertical position in the image to obtain the angle deviation e(k), the deviation signal is sent to the fractional-order PID controller, and then the voltage control signal u(k) is obtained to drive the PWM pulse generator and drive the rotary motor to rotate a certain angle. The image detection system dynamically monitors the straight position of the main flower stem, so that the gripper tracks the main flower stem and keeps it perpendicular to the main flower stem, which facilitates accurate gripping of the main flower stem.

7. The control method according to claim 5, characterized in that: The steps of the side-mounted distance visual servo control are as follows: Calculate the distance error based on the reference distance RS and the actual distance fed back by the main RGBD displacement estimator. Input the distance error signal into the transfer function G2(S) of the main controller. Calculate the displacement error based on the reference displacement value and the actual displacement fed back from the RGBD displacement estimator. Input the displacement error signal into the transfer function G1(S) of the slave controller. Calculate the transfer function L1(s) of the slave electric actuator based on the slave controller voltage control signal U1(S). Calculate the final distance output signal Y2(S) based on U2(S), the load disturbance D(s), and the transfer function L2(s) of the main electric actuator. Y2(s) = U2(s) * L2(s) + D(s), where U2(S) is the inner loop output voltage or the outer loop input voltage of the main electric actuator.

8. The control method according to claim 5, characterized in that: The steps of the lateral bud orientation visual servo control are as follows: The set reference center coordinates are compared with the actual lateral bud center coordinates obtained by the RGBD sensor, and the error is sent to the vision controller for adjustment. Then, the set reference angular displacement is compared with the actual angular displacement detected by the joint angular displacement sensor, and the calculated displacement error is input to the joint controller. The joint controller controls the joint rotation motor to adjust the orientation amplitude.

9. The control method according to claim 5, characterized in that: The Mask RCNN model consists of two stages. The first stage traverses the entire image and generates proposed regions. The second stage classifies the proposed regions to obtain bounding boxes and masks.

Images