A multi-modal based pinch-and-cut citrus picking method, end effector and application

Abstract

The application discloses a multi-modal-based clamp-cutting type citrus picking method, an end effector and application. The method constructs a visual subnetwork and a tactile subnetwork, performs multi-modal feature weighting, obtains a neural network model, further combines a deep deterministic policy gradient algorithm, designs a fault-tolerant mechanism, optimizes the neural network model, and enables the neural network model to adaptively adjust a picking strategy, so that the position of the citrus can be accurately positioned, and picking efficiency is improved. The end effector comprises a visual sensing system, a tactile sensing system, a clamping part, a cutting part and a driving part. Through the design of the clamping part and the cutting part, damage to the fruit is reduced, and picking quality is ensured.

Description

Technical Field

[0001] This invention belongs to the field of intelligent agricultural machinery, specifically relating to a multimodal clamping method for citrus harvesting, an end effector, and its application. Background Technology

[0002] Traditional citrus harvesting methods rely primarily on manual labor, resulting in low efficiency, high labor intensity, and high costs. In recent years, automated citrus harvesting technology has gradually attracted attention and research, including the use of machine vision, machine learning, and robotics to achieve the perception, localization, and harvesting of citrus fruits. However, existing automated citrus harvesting technologies still have some limitations. For example, traditional visual perception systems may be affected by factors such as lighting and occlusion, leading to low accuracy in citrus location. Single tactile perception systems are limited by the surface features and deformation of citrus fruits, making it difficult to adapt to harvesting citrus fruits of different shapes. Researchers have attempted to implement visual-tactile fusion, such as the method for classifying grasped objects based on visual-tactile fusion disclosed in patent CN202311778555.4, which attempts to extract and splice together visual and tactile features through a residual network; however, this method has failed to achieve efficient information integration and is unable to achieve high-precision localization of citrus fruit positions. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a multimodal clamping method for citrus harvesting, an end effector and its application. By constructing a visual subnetwork and a tactile subnetwork and performing multimodal feature weighting, the location of the citrus can be accurately positioned, thereby improving harvesting efficiency.

[0004] The objective of this invention is achieved through the following technical solution:

[0005] A multimodal citrus harvesting method includes a visual subnetwork, a tactile subnetwork, a multimodal fusion module, and an end-to-end learning module;

[0006] (1) Perform self-supervised pre-training of the visual subnetwork and the tactile subnetwork, and extract visual and tactile features using unlabeled data;

[0007] (2) The visual and tactile features are effectively fused using a multimodal fusion module to generate a comprehensive feature vector and obtain a neural network model;

[0008] (3) Constructing an end-to-end learning module: First, collect citrus picking data containing multimodal information (including visual and tactile information), and use labeled data for supervised learning of the neural network model; then, use the Deep Deterministic Policy Gradient (DDPG) algorithm to optimize the performance of the neural network model in the picking task, design a fault tolerance mechanism, and analyze the impact of end effector changes and environmental interference on positioning accuracy by measuring positioning errors in dynamic environments, so that the neural network model can adaptively adjust the picking strategy and obtain the optimized neural network model;

[0009] (4) Input the visual and tactile data of the real-time location of the citrus to be picked, and perform network inference through the optimized neural network model to accurately obtain the location information of the citrus to be picked.

[0010] The visual sub-network is pre-trained using the SimCLR (Simple Framework for Contrastive Learning of Visual Representations) algorithm. The SimCLR algorithm performs two random transformations on the input image, such as cropping and color jittering, and treats the transformed image pairs as positive samples. The model is trained to maximize the similarity between positive samples, thereby extracting effective visual features from the image.

[0011] The tactile sub-network is self-supervised pre-trained using the CPC (Contrastive Predictive Coding) algorithm based on temporal prediction. The CPC algorithm uses the data sequences collected by tactile sensors to train the model by predicting future tactile data, thereby extracting tactile features.

[0012] The multimodal fusion module employs a weighted feature fusion algorithm combined with a multi-head self-attention mechanism for feature weighting and fusion. The multi-head self-attention mechanism divides the input features into multiple heads, each independently calculating its self-attention weights. These weights are then applied to the input features to obtain the weighted feature representation. The formula for the weighted feature fusion algorithm is as follows:

[0013]

[0014] Among them, F combined F represents the comprehensive feature vector. i Let w represent the i-th eigenvector. iThe corresponding weights are represented by the values. By dynamically adjusting the weights of visual and tactile features, the optimal combination of features is achieved. The PCA algorithm is then used for dimensionality reduction to generate a comprehensive feature vector set.

[0015] In step (3), the optimization objective function of the DDPG algorithm is:

[0016]

[0017] Here, θ is the parameter of the policy network, Q is the value function, s represents the state, and π represents the policy. The policy is optimized by maximizing the expected value.

[0018] In step (3), the fault-tolerance mechanism is achieved by constructing a comprehensive test platform integrating an end effector, a visual positioning system, and vibration simulation functions to simulate varying harvesting conditions, measure and analyze positioning errors; then, based on error analysis, a fault-tolerance design strategy is proposed, and the design of the end effector is optimized using tolerance limits and safety margins; wherein, the error compensation formula is:

[0019] E corrected =E measured +K·(E predicted -E meeasured )

[0020] Among them, E corrected E represents the corrected error. measured E represents the measurement error. predicted The value represents the prediction error, and K is the Kalman gain. By introducing the Kalman filter algorithm, error compensation and clamping posture correction are performed when positioning the picking point, thereby effectively dealing with random errors and ensuring stable performance in complex environments.

[0021] A clamping and cutting end effector for picking citrus fruits includes a visual sensing system, a tactile sensing system, a clamping component, a cutting component, and a driving component; the visual sensing system includes a Kinect sensor. The v2 depth camera 3 and RealSense depth camera 7 are used to capture visual information of citrus fruits. The tactile sensing system includes a tactile sensor, a microcontroller, and a solenoid valve. The tactile sensor 22 is mounted on the tactile gripping arm of the clamping component to detect the citrus fruit stem in the clamping area. The clamping component includes a silicone gripping arm and a tactile gripping arm, which cooperate to clamp the citrus fruit stem. The cutting component includes a U-shaped connecting plate 11, a blade 12, and a blade holder 23. The cutting component is mounted on a thin cylinder support, and the blade is mounted on the blade holder. One end of the U-shaped connecting plate 11 is connected to the blade, and the other end is fixed to the piston push rod of the thin cylinder 10. The driving component includes an air pump, a thin cylinder, and a slide cylinder. Motion control is achieved by connecting the cylinder through an air pipe. The thin cylinder is used to control the movement of the cutting component, and the slide cylinder is used to control the movement of the clamping component.

[0022] In the visual sensing system, the Kinect v2 depth camera 3 is mounted on the side of the robotic arm, and the RealSense depth camera 7 is mounted above the end effector.

[0023] The tactile sensing system includes a microcontroller, a tactile sensor, a solenoid valve, a relay, a transformer, and a power supply. The main control system of the tactile sensing system is the microcontroller, which is used to detect whether the clamping component is gripping the fruit stem and to control the changes in the solenoid valve channel. The solenoid valve is connected to the cylinder through an air pipe and is used to control the rapid advance and rapid retraction of the cylinder. The power supply is stepped down by the transformer and supplies power to the microcontroller, the tactile sensor, and the relay respectively. The microcontroller receives signals through a program and controls the state of the relay by changing the high and low levels of the port, thereby controlling the actions of the clamping component and the cutting component.

[0024] A tactile sensor 22 is attached to the side of the tactile gripping arm, and a silicone strip 21 is provided on the side of the silicone gripping arm for clamping the citrus bunch to be cut; a slide cylinder support 25 is fixedly connected to the rear end of the tactile gripping arm, and a slide cylinder 24 is fixed on the slide cylinder support 25; the silicone gripping arm is fixed on the slide cylinder 24, and the silicone gripping arm moves left and right under the action of the slide cylinder 24.

[0025] The cutting component includes a U-shaped connecting plate 11, a blade 12, and a blade holder 23; the blade holder 23 is mounted on a thin cylinder support; one end of the blade 12 is mounted on the blade holder 23, and the other end is mounted on the slot of the blade holder 23; one end of the U-shaped connecting plate 11 is connected to the blade 12, and the other end is fixed to the piston rod of the thin cylinder 10.

[0026] The driving components include an air pump, a thin cylinder 10, and a slide cylinder 24. The air pump controls the outward and inward movements of the cylinder via an air pipe. The thin cylinder 10 is mounted on a thin cylinder support, and its piston rod is connected to a U-shaped connecting plate 11. When the piston rod of the thin cylinder 10 pushes outward, the blade 12 rotates outward relative to the tool holder 23, i.e., it is in the open state. When the piston rod pulls inward, the blade 12 rotates inward relative to the tool holder 23, i.e., it is in the closed state. One opening... The opening and closing action enables the end effector to cut the citrus fruit stem. The slide cylinder 24 is mounted on the slide cylinder support 25, and the silicone clamping arm is mounted on the slide cylinder 24. When the piston rod of the slide cylinder 24 pushes outward, the silicone clamping arm moves in the same direction as the piston rod, that is, clamping. When the piston rod pulls inward, the silicone clamping arm moves in the same direction as the piston rod, that is, releasing the clamp. One opening and closing action can realize the end effector's clamping and opening function for the citrus fruit stem.

[0027] The end effector also includes a check component, which is disposed on the clamping component. The check component includes a check block 20 and a spring 26. The check block 20 is rotatably connected to the tactile clamping arm. The end of the tactile clamping arm is provided with a protruding structure. One end of the spring 26 is mounted on the protruding structure of the tactile clamping arm, and the other end is mounted on the check block to realize the function of resetting the check block 20.

[0028] The end effector also includes a user interface, which is designed with a graphical interface to display the status information of the end effector, the data collected, and to provide user control options.

[0029] The end effector's communication module uses the MQTT (Message Queuing Telemetry Transport) protocol to remotely transmit the end effector's status information and harvesting data to a designated server or user device.

[0030] The application of the clamping-type citrus harvesting end effector employs the aforementioned multimodal citrus harvesting method, including the following steps:

[0031] (1) The visual sensing system and tactile sensing system of the end effector collect visual data and tactile data of the real-time location of the citrus fruit, respectively. The optimized neural network model is used for network inference to accurately obtain the location information of the citrus fruit to be picked.

[0032] (2) Based on the location information of the citrus to be picked, start the sliding cylinder and clamp the citrus stem with the clamping component; then start the thin cylinder and cut the citrus stem with the cutting component to complete the citrus picking.

[0033] Compared with the prior art, the present invention has the following advantages and effects:

[0034] (1) By efficiently combining visual and tactile information, this invention can achieve accurate identification and positioning of citrus fruits, thereby improving the accuracy of harvesting and operational efficiency.

[0035] (2) This invention uses a neural network combined with the Deep Deterministic Strategy Gradient (DDPG) algorithm to design a fault-tolerant mechanism. By measuring the positioning error in a dynamic environment, it deeply analyzes the impact of changes in the actuator and environmental interference on the positioning accuracy. It also analyzes the impact of changes in the end effector and environmental interference on the positioning accuracy during the picking process. This enables the neural network model to adaptively adjust the picking strategy to improve the stability of the system and the success rate of picking, so that it can maintain stable picking performance under different environments and fruit characteristics.

[0036] (3) The present invention reduces damage to the fruit and ensures harvesting quality through the design of clamping and cutting components. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the overall structure of the end effector of the present invention.

[0038] Figure 2 This is a schematic diagram of the overall structure of the harvesting robot.

[0039] Figure 3 This is a schematic diagram of the shearing component.

[0040] Figure 4 This is a schematic diagram of the clamping component.

[0041] Figure 5 This is a schematic diagram of a silicone strip.

[0042] Figure 6 This is a schematic diagram of a tactile sensor.

[0043] Figure 7 This is a schematic diagram of a slide cylinder.

[0044] Figure 8 A schematic diagram of the stop-return component.

[0045] Figure 9 This is a flowchart illustrating the process of driving the components.

[0046] The labels in the diagram are as follows:

[0047] 1. Tracked chassis 2. tripod 3. Kinect v2 Depth Camera 4. Six-axis robotic arm 5. Left cylinder support 6. Cylinder support 7. RealSense Depth Camera 8. Camera mount 9. Right cylinder support 10. Thin cylinder 11. U-shaped connecting plate 12. blade 13. gasket 14. Right silicone gripper arm 15. Right tactile gripping arm 16. Medium silicone gripper arm 17. Mid-tactile gripper arm 18. Left silicone gripper arm 19. Left tactile gripping arm 20. Check block 21. silicone strip 22. tactile sensor 23. knife holder 24. Slide cylinder 25. Slide Cylinder Support 26. spring Detailed Implementation

[0048] To facilitate understanding of the present invention, specific embodiments will be described in detail below. These embodiments will help those skilled in the art to further understand the present invention; however, they are not intended to limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements to the present invention without departing from its conceptual framework, and these modifications and improvements all fall within the scope of protection of the present invention.

[0049] Example 1

[0050] like Figure 1 As shown, the clamping and cutting citrus harvesting end effector of this embodiment is mounted on a six-axis robotic arm and includes a vision sensing system, a tactile sensing system, a drive component, a cutting component, a clamping component, and a backlash prevention component.

[0051] The visual sensing system consists of a Kinect v2 depth camera and a RealSense depth camera. These two cameras collect RGB images while simultaneously using infrared technology to acquire depth images of the citrus fruit. For example... Figure 2 As shown, the Kinect v2 depth camera 3 is mounted on tripod 2, which is placed on the plane of the tracked chassis 1. The ReaSense depth camera 7 is mounted on camera mount 8. In addition, the ReaSense depth camera 7 has two recesses and a vertical mounting plate at the rear for securely fixing the camera position.

[0052] like Figure 9 As shown, the tactile sensing system includes an STM32 microcontroller, a tactile sensor, a solenoid valve, a relay, a transformer, and a 12V power supply. The tactile sensor is mounted on the tactile gripping arm of the clamping component to detect the presence of citrus stems in the gripping area. The main control system of the tactile sensing system is an STM32 microcontroller, used to detect whether the clamping component is gripping the stem and control the changes in the solenoid valve channel. The solenoid valve is connected to the cylinder of the end effector via an air tube to control the cylinder's rapid advance and retraction. The entire system is powered by a 12V power supply, with a main power switch that is turned on only after the corresponding modules are connected. The transformer's step-down module reduces the 12V voltage to 5V. The 12V power is used to drive the solenoid valve, while the reduced 5V voltage supplies power to the microcontroller, tactile sensor, and relays. The microcontroller receives signals and controls the relay state by changing the high and low levels of its ports through a program, thereby controlling the actions of the clamping and cutting components. In practice, the microcontroller receives data from the tactile sensor via a program and determines whether to clamp the fruit stem based on a set pressure threshold. It then controls the corresponding relay to open the switch, energizing the solenoid valve and air pump. The air pump delivers gas through an air pipe to drive the cylinder, which in turn drives the end effector to perform the clamping or cutting action.

[0053] In practice, the cutting component can be one or more as needed. For example... Figure 3 As shown, the cutting component in this embodiment includes three sub-components: left, middle, and right, all with identical structures. For example, there are three blade holders 23, which are respectively bolted to the left cylinder support 5, the middle cylinder support 6, and the right cylinder support 9. The following description uses the middle sub-component as an example: the blade 12 is made of Sk5 steel with a cutting edge angle of 20° and a thickness of 3mm; the blade holder 23 is made of Cr12MoV material with a cutting edge angle of 20° and a thickness of 3mm; the blade holder 23 is fixed to the middle cylinder support 6 with bolts; one end of the blade 12 is bolted to the blade holder as a rotating shaft, and the other end is bolted to the slot of the blade holder 23; two gaskets 13 are bolted to the slot at the other end of the blade 12, respectively installed above the blade 12 and below the blade holder 23, to prevent the blade from shaking during cutting; the U-shaped connecting plate 11 is bolted to the other end of the blade 12 and fixed to the piston rod of the thin cylinder 10.

[0054] Similarly, in practical applications, the clamping component can be one or more as needed, and it must cooperate with the cutting component. For example... Figure 4 As shown, the clamping component in this embodiment includes three sub-components: left, middle, and right, all with identical structures. Specifically, it includes a right silicone clamping arm 14, a right-middle silicone clamping arm 16, a left silicone clamping arm 18, a right tactile clamping arm 15, a middle tactile clamping arm 17, and a left tactile clamping arm 19. Figure 6 As shown, the middle tactile gripping arm 17 has horizontal, non-through convex grooves at both ends. The left end of the right tactile gripping arm 15 and the right end of the left tactile gripping arm 19 have convex structures, which are inserted into the convex grooves of the middle tactile gripping arm 17 to form tactile gripping arms. Figure 5 As shown, elongated tactile sensors 22, measuring 110mm × 15mm, are attached to the right side of the left, middle, and right tactile gripping arms. The rear end of the middle tactile gripping arm 17 has two square slots into which a slide cylinder support 25 is inserted and fixed with bolts. The slide cylinder support 25 has three through holes for fixing the slide cylinder (MXS6-30) 24. The left and right ends of the middle silicone gripping arm 16 have vertical non-through convex slots. The right end of the left silicone gripping arm 18 and the left end of the right silicone gripping arm 14 have convex structures, which are inserted into the convex slots of the middle silicone gripping arm 16 to form the silicone gripping arm. The upper part of the middle silicone gripping arm 16 has four countersunk holes, which are used to fix the middle silicone gripping arm 16 to the slide cylinder (MXS6-30) 24 with bolts. Figure 7As shown, the slide cylinder (MXS6-30) 24 is fixed to the slide cylinder support 25 by bolts. The silicone clamping arms move left and right under the action of the slide cylinder (MXS6-30) 24. The left side of the left, middle, and right silicone clamping arms has a vertical, non-through L-shaped groove, and the silicone strips 21 have an L-shaped structure, which are inserted into the corresponding L-shaped grooves. The silicone strips 21 are detachable and used to clamp the citrus fruit bunches cut by the blade, while ensuring that different thicknesses of fruit stems are clamped simultaneously in different cutting and clamping areas. The clamping component, composed of the left, middle, and right tactile clamping arms and the left, middle, and right silicone clamping arms, completes the clamping of the citrus fruit stems. The tips of the left, middle, and right silicone clamping arms and the left, middle, and right tactile clamping arms have a guiding function, guiding the citrus fruit stems into multiple identical cutting and clamping areas.

[0055] The drive components include a 550W-8L air compressor, a thin cylinder 10, and a sliding cylinder 24. The air compressor controls the outward and inward movements of the two types of cylinders via air pipes. Specifically, in this embodiment, the three thin cylinders 10 are bolted to the left cylinder support 5, the middle cylinder support 6, and the right cylinder support 9, respectively. The piston rod of the thin cylinder 10 is bolted to the U-shaped connecting plate 11. When the piston rod of the thin cylinder 10 pushes outward, the blade 12 rotates outward relative to the blade holder 23, i.e., in the open state; when the piston rod pulls inward, the blade 12 rotates inward relative to the blade holder 23, i.e., in the closed state. This opening and closing action enables the end effector to cut the citrus stem. The sliding cylinder 24 is bolted to the sliding cylinder support 25, and the middle silicone clamping arm 16 is bolted to the sliding cylinder 24. When the piston rod of the slide cylinder 24 pushes outward, the middle silicone clamping arm 16 moves in the same direction as the piston rod, i.e., clamping; when the piston rod pulls inward, the middle silicone clamping arm 16 moves in the same direction as the piston rod, i.e., releasing the clamp. Similarly, a single opening and closing action can realize the clamping and opening function of the end effector on the citrus fruit stem.

[0056] like Figure 8 As shown, the anti-return component includes an anti-return block 20 and a spring 26. Small protruding structures are provided at the ends of the left, middle, and right tactile gripping arms. Through holes are provided next to these protruding structures. The anti-return block 20 is rotatably connected to the tactile gripping arm via bolts. Circular grooves are provided inside the protruding structures and the anti-return block 20, allowing the spring 26 to be installed. One end of the spring 26 is mounted on the protruding structure, and the other end is mounted on the anti-return block, enabling the anti-return block 20 to reset. The cooperation of the protruding structures on the gripping arms, the anti-return block 20, and the spring 26 prevents the citrus stem from detaching from the cutting component during the clamping and cutting operation. This design ensures the stability and accuracy of the gripping and cutting components during operation.

[0057] During operation, the clamping and cutting citrus harvesting end effector employs the aforementioned multimodal citrus harvesting method. First, the end effector's visual and tactile sensing systems collect real-time visual and tactile data of the citrus fruit's location, respectively. Through optimized neural network models, network inference is performed to accurately obtain the location information of the citrus fruit to be harvested. Then, based on the location information of the citrus fruit to be harvested, the sliding cylinder is activated, and the clamping component clamps the citrus fruit stem. Finally, the thin cylinder is activated, and the cutting component cuts the citrus fruit stem, completing the citrus harvesting.

[0058] It is understood that the above specific description of the present invention is only for illustrating the present invention and is not limited to the technical solutions described in the embodiments of the present invention. Those skilled in the art should understand that local modifications or equivalent substitutions can still be made to the present invention to achieve the same technical effect; as long as the use needs are met, they are all within the protection scope of the present invention.

Claims

1. A multimodal citrus harvesting method, characterized in that: It includes a visual subnetwork, a tactile subnetwork, a multimodal fusion module, and an end-to-end learning module; and harvesting is carried out through a clamp-type citrus harvesting end effector; The clamping and cutting citrus harvesting end effector includes a visual sensing system, a tactile sensing system, a clamping component, a cutting component, and a driving component; the visual sensing system includes Kinect. The v2 depth camera and RealSense depth camera are used to capture visual information about citrus fruits. The tactile sensing system includes a tactile sensor, a microcontroller, and a solenoid valve. The tactile sensor is mounted on the tactile gripping arm of the clamping component to detect the citrus fruit stem in the clamping area. The clamping component includes a silicone gripping arm and a tactile gripping arm, which cooperate to clamp the citrus fruit stem. The cutting component includes a U-shaped connecting plate, a blade, and a blade holder. The cutting component is mounted on a thin cylinder support, and the blade is mounted on the blade holder. One end of the U-shaped connecting plate is connected to the blade, and the other end is fixed to the piston rod of the thin cylinder. The driving component includes an air pump, a thin cylinder, and a sliding cylinder. Motion control is achieved by connecting the cylinder through an air pipe. The thin cylinder controls the movement of the cutting component, and the sliding cylinder controls the movement of the clamping component. The harvesting process using a clamping citrus harvesting end effector includes the following steps: (1) Perform self-supervised pre-training of the visual subnetwork and the tactile subnetwork, and extract visual and tactile features using unlabeled data; (2) The visual and tactile features are effectively fused using a multimodal fusion module to generate a comprehensive feature vector and obtain a neural network model; The multimodal fusion module employs a weighted feature fusion algorithm, combined with a multi-head self-attention mechanism for feature weighting and fusion. The multi-head self-attention mechanism divides the input features into multiple heads, each independently calculating its self-attention weights. These weights are then applied to the input features to obtain the weighted feature representation. The formula for the weighted feature fusion algorithm is as follows: in, Represents the comprehensive feature vector. This represents the i-th eigenvector. The corresponding weights are represented by the values. By dynamically adjusting the weights of visual and tactile features, the optimal combination of features is achieved. The PCA algorithm is then used for dimensionality reduction to generate a comprehensive feature vector set. (3) Constructing an end-to-end learning module: First, collect citrus picking data containing multimodal information and use labeled data for supervised learning of the neural network model; then, use the deep deterministic strategy gradient (DDPG) algorithm to optimize the performance of the neural network model in the picking task, design a fault tolerance mechanism, and analyze the impact of end effector changes and environmental interference on the positioning accuracy by measuring the positioning error in the dynamic environment, so that the neural network model can adaptively adjust the picking strategy and obtain the optimized neural network model. The objective function of the DDPG algorithm is: Where θ is the parameter of the policy network, Q is the value function, s represents the state, and π represents the policy. The policy is optimized by maximizing the expected value. (4) The visual sensing system and tactile sensing system of the clamp-type citrus picking end effector collect visual data and tactile data of the real-time location of the citrus, respectively. The optimized neural network model is used for network inference to accurately obtain the location information of the citrus to be picked. (5) Based on the location information of the citrus to be picked, start the sliding cylinder and clamp the citrus stem with the clamping component; then start the thin cylinder and cut the citrus stem with the cutting component to complete the citrus picking.

2. The multimodal citrus harvesting method according to claim 1, characterized in that: In step (3), the fault-tolerance mechanism is achieved by constructing a comprehensive test platform integrating an end effector, a visual positioning system, and vibration simulation functions to simulate varying harvesting conditions, measure and analyze positioning errors; then, based on error analysis, a fault-tolerance design strategy is proposed, and the design of the end effector is optimized using tolerance limits and safety margins; wherein, the error compensation formula is: in, This indicates the corrected error. Indicates measurement error. The value represents the prediction error, and K is the Kalman gain. By introducing the Kalman filter algorithm, error compensation and clamping posture correction are performed when positioning the picking point, thereby effectively dealing with random errors and ensuring stable performance in complex environments.

3. The multimodal citrus harvesting method according to claim 1, characterized in that: A tactile sensor is attached to the side of the tactile gripping arm, and a silicone strip is provided on the side of the silicone gripping arm for clamping the citrus bunch to be cut; a slide cylinder support is fixedly connected to the rear end of the tactile gripping arm, and a slide cylinder is fixed on the slide cylinder support; the silicone gripping arm is fixed on the slide cylinder, and the silicone gripping arm moves left and right under the action of the slide cylinder.

4. The multimodal citrus harvesting method according to claim 1, characterized in that: The cutting component includes a U-shaped connecting plate, a blade, and a blade holder; the blade holder is mounted on a thin cylinder support; one end of the blade is mounted on the blade holder, and the other end is mounted on the slot of the blade holder; one end of the U-shaped connecting plate is connected to the blade, and the other end is fixed to the piston rod of the thin cylinder.

5. The multimodal citrus harvesting method according to claim 1, characterized in that: The drive components include an air pump, a thin cylinder, and a sliding cylinder. The air pump controls the outward and inward movements of the cylinder through an air pipe. The thin cylinder is mounted on a thin cylinder support, and its piston rod is connected to a U-shaped connecting plate. When the piston rod pushes outward, the blade rotates outward relative to the blade holder, i.e., in an open state. When the piston rod pulls inward, the blade rotates inward relative to the blade holder, i.e., in a closed state. One opening and closing action can realize the end effector's function of cutting citrus stems. The sliding cylinder is mounted on a sliding cylinder support, and a silicone clamping arm is mounted on the sliding cylinder. When the piston rod pushes outward, the silicone clamping arm moves in the same direction as the piston rod, i.e., clamping. When the piston rod pulls inward, the silicone clamping arm moves in the same direction as the piston rod, i.e., releasing the clamp. One opening and closing action can realize the end effector's function of clamping and opening citrus stems.

6. The multimodal citrus harvesting method according to claim 1, characterized in that: The end effector includes a check component, which is disposed on the clamping component. The check component includes a check block and a spring. The check block is rotatably connected to the tactile clamping arm. The end of the tactile clamping arm is provided with a protruding structure. One end of the spring is mounted on the protruding structure of the tactile clamping arm, and the other end is mounted on the check block to realize the function of resetting the check block.

Images