Self-propelled red pepper picking system

Abstract

The invention relates to the technical field of zanthoxylum bungeanum maxim picking, in particular to a self-propelled zanthoxylum bungeanum maxim picking system which comprises a zanthoxylum bungeanum maxim picking device and a central processing unit arranged on the zanthoxylum bungeanum maxim picking device. The red pepper picking device comprises a self-propelled chassis, a first mechanical arm, a second mechanical arm, an image acquisition module, a flexible end effector, a control box and a branch shifting module, wherein the first mechanical arm, the second mechanical arm and the control box are all arranged on the self-propelled chassis. According to the method, the picking path plan is determined through the central processing unit according to the actual current position of the flexible end effector and the actual target position of the red pepper cluster, the flexible end effector is moved to the position of the red pepper cluster to be picked according to the picking path plan, and then the red pepper cluster is picked through the flexible end effector. Therefore, automatic picking of the red peppers from near to far in sequence is achieved, dependence of manual picking is reduced, and the labor intensity of picking personnel is relieved.

Description

Technical Field

[0001] This invention relates to the field of Sichuan pepper harvesting technology, specifically a self-propelled Sichuan pepper harvesting system. Background Technology

[0002] Sichuan pepper is a unique spice and medicinal herb in my country, widely cultivated in Sichuan, Chongqing, and Northwest China. However, the harvesting of Sichuan pepper mainly relies on manual labor, which is not only labor-intensive and inefficient, but also results in the harsh working environment and high labor costs due to the sharp thorns covering the branches. Although hand-held harvesting tools such as serrated, scissor, and rotary harvesters can be used to assist in harvesting, they still require manual operation and cannot effectively alleviate the labor intensity of the harvesters. Summary of the Invention

[0003] The purpose of this invention is to provide a self-propelled red pepper harvesting system to solve the problems mentioned in the background art.

[0004] To achieve the above objectives, the present invention provides the following technical solution: The self-propelled Sichuan pepper harvesting system includes a Sichuan pepper harvesting device and a central processing unit, with the central processing unit located on the Sichuan pepper harvesting device. The red pepper harvesting device includes a self-propelled chassis, robotic arm one, robotic arm two, an image acquisition module, a flexible end effector, a control box, and a branch-picking module. Robotic arm one, robotic arm two, and the control box are all mounted on the self-propelled chassis. The image acquisition module includes a depth camera, which is mounted on the output end of robotic arm one. The flexible end effector is mounted on the output end of robotic arm two. The central processing unit is located inside the control box. A collection frame is fixedly mounted on the self-propelled chassis. The image acquisition module can acquire image information of Sichuan peppercorns and transmit it to the central processing unit (CPU). The CPU can analyze the image information, identify clusters of Sichuan peppercorns in the image, and reconstruct a 3D model of the clusters of Sichuan peppercorns to be harvested based on the position information acquired by the depth camera. It can also determine the actual target position of the clusters of Sichuan peppercorns to be harvested in the actual coordinate system based on the image. The CPU can determine the execution action of the second robotic arm and the harvesting path planning of the flexible end effector based on the current position of the flexible end effector in the actual coordinate system, the actual target position of the Sichuan peppercorn clusters, and the motion model of the second robotic arm. After the second robotic arm moves the flexible end effector to the actual target position of the clusters of Sichuan peppercorns to be harvested, the flexible end effector performs the harvesting action.

[0005] Furthermore, the red pepper harvesting device also includes an air pump, which provides an air source for the flexible end effector. The control box contains an air pump controller, which is connected to the flexible end effector via an air pipe. The air pump controller can adjust the driving pressure of the flexible end effector to achieve various action modes such as opening, clamping, and releasing.

[0006] Furthermore, when determining the actual target position of the red pepper cluster to be harvested in the actual coordinate system based on the image information, the actual target position is determined by using a preset transformation model and a virtual target position. The transformation model is the transformation relationship between the virtual target position of the red pepper cluster in the image and the actual position in the actual coordinate system. The virtual target position is the cluster center position of the red pepper cluster to be harvested in the image. The actual target position includes the planar position and the height position.

[0007] Furthermore, when identifying clusters of Sichuan pepper in an image, for image information that cannot be completely identified as clusters of Sichuan pepper, i.e., image information of uncertain clusters of Sichuan pepper whose feature parameters do not meet the threshold, coordinate marking is performed in the image to trigger an abandonment of harvesting instruction. The marked real-time image data is transmitted wirelessly to a handheld control tablet. The tablet interface simultaneously displays the location markers, morphological parameters, and surrounding branch environment of the fruit cluster. Users can then manually issue instructions to pick the fruit or confirm that they are abandoning the picking based on their actual planting needs. When a picking instruction is manually issued, the central processing unit (CPU) automatically determines the actual target location of the Sichuan pepper cluster based on the image information. Then, the CPU automatically plans the picking path. Alternatively, based on the three-dimensional spatial model of the Sichuan pepper plant, the user can manually click on the three-dimensional coordinate point corresponding to the target fruit cluster in the model to manually select the picking location. The CPU then automatically plans the picking path based on the picking location.

[0008] Furthermore, the image acquisition module also includes a mounting plate, the depth camera is fixedly mounted on the mounting plate, the mounting plate is fixedly mounted on the output end of the robotic arm, and there are two branch-removing modules, both of which are mounted on the mounting plate. The branch-pulling module includes a base, a drive shaft, and a lever; The base is fixedly mounted on the mounting plate, and an adjustment motor is installed inside the base. The drive shaft is mounted on the base, and the motor output is connected to the drive shaft for transmission. The lever is fixedly mounted on the drive shaft.

[0009] Preferably, when the central processing unit identifies the red pepper clusters in the image, it inputs the collected image information into the YOLO neural network that has completed preset training and outputs the recognition result. The recognition result includes the regional information of the red pepper clusters in the image, and divides the fruits of the red pepper clusters to be harvested into scattered clusters and clustered clusters. According to the type of red pepper clusters, scattered clusters and clustered clusters, the opening size of the flexible end effector is adjusted.

[0010] Furthermore, the pre-trained neural network is trained using a training sample set to obtain a trained YOLO neural network. First, an initial image sample set is obtained, which includes multiple sample images, all of which contain clusters of Sichuan peppercorns. The sample images in the initial image sample set are labeled, the regions of Sichuan peppercorn clusters in the sample images are selected, and the type of Sichuan peppercorn clusters is labeled to obtain a training sample set. Then, the training sample set is input into the pre-trained neural network for training. After training, a trained YOLO neural network is formed.

[0011] Preferably, when the robotic arm 2 moves the flexible end effector to perform the picking action, collision detection between the fruit and the branch is performed. First, based on the current joint angle, link length and connection method of the robotic arm 2, the real-time posture of each joint and link of the robotic arm 2 in three-dimensional space is obtained through forward kinematics calculation. Then, based on the structural parameters of the robotic arm 2, an overall model including the joint sphere, the link envelope and the shape of the end effector is constructed. Meanwhile, the dynamic environmental point cloud acquired and updated by the image acquisition module, together with the electronic fence boundary set according to the work area, are fused to generate a unified environmental grid representation. The central processing unit uses the spatial minimum distance calculation method to periodically solve the distance between the surface of any link of the robotic arm and the surface of all obstacles in the environment, and synchronously detects possible self-collisions between the links. When any minimum distance is lower than the set safety threshold, it is determined that there is a potential collision risk in the current state.

[0012] Furthermore, during the collision detection process, the theoretical torque of each joint is calculated in real time based on the dynamic model of the second robotic arm, and compared with the actual current signal fed back by the drive motor. When an abnormal force situation occurs that is inconsistent with the normal motion trend, the system regards the situation as an early sign of contact collision, and uses it to perform secondary verification of the geometric detection results.

[0013] Furthermore, when either the geometric distance judgment or the force verification meets the collision risk conditions, the system enters a restriction mode. This mode prevents the robotic arm from continuing to move towards the high-risk area by adjusting the joint speed, restricting the direction of movement, or pausing the trajectory execution. If necessary, the system also triggers the host planner to replan the current path to ensure that the robotic arm can achieve continuous and reliable collision prevention capabilities in complex and dynamic environments.

[0014] Compared with the prior art, the beneficial effects of the present invention are: The central processing unit determines the harvesting path based on the actual current position of the flexible end effector and the actual target position of the Sichuan pepper cluster. Then, according to the harvesting path plan, the robotic arm moves the flexible end effector to the position of the Sichuan pepper cluster to be harvested. The flexible end effector then harvests the Sichuan pepper cluster, thus realizing the automatic harvesting of Sichuan peppers from near to far. This helps to reduce reliance on manual harvesting, alleviate the labor intensity of harvesters, and ensure the quality of harvesting and avoid damage to the Sichuan pepper fruits as much as possible by using a flexible end effector. In addition, the path planning can avoid collisions with next year's young buds, thus achieving low-loss harvesting.

[0015] By marking the image information of uncertain red pepper clusters during the identification process and transmitting the image information to the tablet for manual confirmation, the risk of over-harvesting is avoided by avoiding the picking of immature fruit clusters or miscellaneous branches. During the harvesting process, the harvesting position in the red pepper model can also be manually selected as needed. Manual intervention during the harvesting process can supplement automatic harvesting, which helps to improve the harvesting accuracy and avoid over-harvesting and erroneous harvesting as much as possible. By setting a depth camera on the output end of robotic arm one, when the flexible end effector is picking the nearest cluster of Sichuan pepper, the image acquisition module can be moved by robotic arm one to take pictures of the Sichuan pepper plant from multiple angles, which further improves the accuracy of the 3D model of the Sichuan pepper cluster. When encountering an uncertain cluster of Sichuan pepper, the cluster can be identified from multiple angles, and the image acquisition module can be extended into the plant to further identify the obscured clusters, which helps to improve the accuracy of identification. Attached Figure Description

[0016] Figure 1 This is a system block diagram of the present invention; Figure 2 This is a schematic diagram of the red pepper harvesting device in this invention; Figure 3 This is a schematic diagram of the branching module structure in this invention; Figure 4 This is a flowchart of the harvesting path planning in this invention; Figure 5 This is a flowchart of the harvesting process in this invention; Figure 6 This is a structural diagram of the YOLO model in this invention.

[0017] In the diagram: 100, Sichuan pepper harvesting device; 110, self-propelled chassis; 111, collection frame; 120, robotic arm one; 130, robotic arm two; 140, image acquisition module; 141, mounting plate; 150, flexible end effector; 160, air pump; 170, control box; 180, branch-picking module; 181, base; 182, drive shaft; 183, lever. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Please see Figure 1-5 In this embodiment of the invention, the self-propelled Sichuan pepper harvesting system includes a Sichuan pepper harvesting device 100 and a central processing unit, wherein the central processing unit is mounted on the Sichuan pepper harvesting device 100. The Sichuan pepper harvesting device 100 includes a self-propelled chassis 110, a robotic arm 120, a robotic arm 2 130, an image acquisition module 140, a flexible end effector 150, a control box 170, and a branch-picking module 180. The robotic arms 120, 230, and control box 170 are all mounted on the self-propelled chassis 110. Both robotic arms 120 and 230 are six-degree-of-freedom articulated structures and are driven by motor servo drives. The robotic arms can provide real-time feedback of their end-effector pose to the central processing unit. The image acquisition module 140 includes a depth camera. The central processing unit can control the start and stop of the image acquisition module 140. The depth camera is located at the output end of the robotic arm 120. The flexible end effector 150 is located at the output end of the robotic arm 2 130. The central processing unit is located inside the control box 170. A collection frame 111 is fixedly mounted on the self-propelled chassis 110. The self-propelled chassis 110 has a built-in battery pack, motor drive module and obstacle avoidance sensor. It can communicate with the control system via wireless signal to realize remote or autonomous navigation and movement. It can be configured with a wireless communication module. The self-propelled chassis 110 can transmit driving information and its own status information to the central processor. The central processor can transmit the information to the user's handheld tablet through the wireless communication module, so that the user can monitor the driving status of the self-propelled chassis 110. The Sichuan pepper harvesting device 100 also includes an air pump 160, which provides an air source for the flexible end effector 150. The control box 170 contains an air pump controller, which is connected to the flexible end effector 150 via an air pipe. The air pump controller can adjust the driving pressure of the flexible end effector 150. The flexible end effector 150 is a soft manipulator with two pneumatic flexible fingers. Each pneumatic flexible finger is made of high-elasticity silicone material and has an internal cavity structure. The cavity is connected to the air pump controller via an air pipe, and the air pump controller is connected to the air pump 160. The air pump controller controls the driving pressure of each pneumatic flexible finger, thereby realizing multiple action modes such as opening, clamping, and releasing of the two pneumatic flexible fingers. The image acquisition module 140 can acquire image information of Sichuan peppercorns and transmit the image information to the central processing unit. The central processing unit can analyze the image information, identify the clusters of Sichuan peppercorns in the image, and reconstruct the three-dimensional model of the clusters of Sichuan peppercorns to be harvested based on the position information acquired by the depth camera. It can also determine the actual target position of the clusters of Sichuan peppercorns to be harvested in the actual coordinate system based on the image. The central processing unit can determine the execution action of the robotic arm 130 and the harvesting path planning of the flexible end effector 150 based on the actual current position of the flexible end effector 150 in the actual coordinate system, the actual target position of the Sichuan peppercorn clusters, and the motion model of the robotic arm 130. After the robotic arm 130 moves the flexible end effector 150 to the actual target position of the clusters of Sichuan peppercorns to be harvested, the flexible end effector 150 performs the harvesting action. Specifically, when determining the actual target position of the red pepper cluster to be harvested in the actual coordinate system based on image information, the actual target position is determined by using a preset transformation model and a virtual target position. The transformation model is the transformation relationship between the virtual target position of the red pepper cluster in the image and the actual position in the actual coordinate system. The virtual target position is the cluster center position of the red pepper cluster to be harvested in the image, and the actual target position includes the planar position and the height position.

[0020] Specifically, during harvesting, the self-propelled chassis 110 first travels along the working path. The radar on the self-propelled chassis 110 acquires road condition information and transmits the information to the central processor. The central processor then issues the relevant movement direction and distance and transmits the movement information of the self-propelled chassis 110 to the tablet terminal, where the user monitors the movement position. The user can also manually plan the working path on the tablet terminal according to the working terrain. The tablet terminal can transmit the working path to the central processor via wireless communication, and the central processor then controls the movement of the self-propelled chassis 110. During the movement of the self-propelled chassis 110, the Sichuan pepper plant can be identified through the image acquisition module 140. After the self-propelled chassis 110 moves to the target Sichuan pepper plant position, the self-propelled chassis 110 stops moving and the Sichuan pepper cluster is identified through the depth camera of the image acquisition module 140. The depth camera of the image acquisition module 140 can transmit image information to the central processing unit. The central processing unit analyzes the image and establishes an actual coordinate system with the base of robotic arm 120 as the origin. Based on the distance between the base of robotic arm 120 and robotic arm 2130, and based on the posture of robotic arm 2130, the actual current position of the flexible end effector 150 in the actual coordinate system is determined. Based on the posture information of robotic arm 120, the position information of the depth camera is obtained. After the Sichuan pepper cluster is identified, based on the position information of the depth camera, the actual target position is determined by using the virtual target position of the Sichuan pepper cluster to be harvested in the image and the preset transformation model, and the planar position and height position information of the Sichuan pepper cluster to be harvested are obtained. The central processing unit can determine the execution action of the robotic arm 130 and the picking path planning of the flexible end effector 150 based on the three-dimensional model of the red pepper clusters collected by the depth camera, the actual current position of the flexible end effector 150, the actual target position of the red pepper clusters, and the motion model of the robotic arm 130, so that the flexible end effector 150 can pick multiple red pepper clusters from near to far. After the flexible end effector 150 is moved to the side of the red pepper cluster to be harvested, the driving gas pressure of the flexible end effector 150 is adjusted by the air pump controller to control the flexible end effector 150 to perform the harvesting action of the red pepper cluster. After one harvest is completed, the position of the flexible end effector 150 is adjusted by the second robotic arm 130 to move the red pepper cluster to the side of the collection box 111. Then the flexible end effector 150 releases the red pepper cluster and collects the red pepper cluster in the collection box 111. Then the second robotic arm 130 moves the flexible end effector 150 to the position of the next red pepper cluster for the next harvest. After multiple red pepper clusters on a red pepper plant are harvested, the self-propelled chassis 110 moves to the side of the next red pepper plant until the operation is completed. The flexible end effector 150 enables flexible and automatic harvesting of Sichuan pepper clusters, which helps to reduce reliance on manual harvesting, alleviate the labor intensity of harvesters, ensure the quality of harvesting as much as possible, and avoid damage to the Sichuan pepper fruits. In addition, the path planning can avoid the emergence of young buds in the following year, thus achieving low-loss harvesting.

[0021] Example 1 like Figure 1As shown in this embodiment, when identifying clusters of Sichuan pepper in an image, for image information that cannot be completely identified as clusters of Sichuan pepper, i.e., image information of uncertain clusters of Sichuan pepper whose feature parameters do not meet the threshold, coordinate marking is performed in the image to trigger an abandonment of harvesting command. The marked real-time image data is transmitted wirelessly to a handheld control tablet. The tablet interface simultaneously displays the location markers, morphological parameters, and surrounding branch environment of the fruit cluster. Users can then manually issue instructions to pick the fruit or confirm that they are abandoning the picking based on their actual planting needs. When a picking instruction is manually issued, the central processing unit (CPU) automatically determines the actual target location of the Sichuan pepper cluster based on the image information. Then, the CPU automatically plans the picking path. Alternatively, based on the three-dimensional spatial model of the Sichuan pepper plant, the user can manually click on the three-dimensional coordinate point corresponding to the target fruit cluster in the model to manually select the picking location. The CPU then automatically plans the picking path based on the picking location.

[0022] In practice, uncertain Sichuan pepper clusters are marked, such as semi-ripe Sichuan pepper clusters, suspected Sichuan pepper clusters overlapping with miscellaneous branches and leaves, and scattered Sichuan pepper clusters with too small a particle size. The image information is transmitted to the tablet via wireless communication. Suspected Sichuan pepper clusters that are obscured by branches and leaves and scattered Sichuan pepper clusters with too small a particle size can be manually identified. After manual identification, a picking command can be issued manually. For semi-ripe Sichuan pepper clusters, a picking command or a confirmation to abandon picking can be issued manually as needed. By manually verifying and confirming, the risk of over-harvesting is avoided by preventing the picking of immature fruit clusters or miscellaneous branches. During the harvesting process, the harvesting position in the Sichuan pepper model can also be manually selected as needed. Manual intervention during the harvesting process can supplement automatic harvesting, which helps to improve harvesting accuracy and avoid over-harvesting and erroneous harvesting as much as possible.

[0023] like Figure 2 and Figure 3 As shown, in this embodiment, the image acquisition module 140 also includes a mounting plate 141, the depth camera is fixedly mounted on the mounting plate 141, the mounting plate 141 is fixedly mounted on the output end of the robotic arm 120, and there are two branch-removing modules 180, both of which are mounted on the mounting plate 141. The branch-pulling module 180 includes a base 181, a drive shaft 182, and a lever 183; The base 181 is fixedly mounted on the mounting plate 141. An adjustment motor is installed inside the base 181. The base 181 can protect the adjustment motor. The transmission shaft 182 is rotatably mounted on the base 181. The output end of the adjustment motor is connected to the transmission shaft 182. The lever 183 is fixedly mounted on the transmission shaft 182.

[0024] In practice, after the nearest cluster of Sichuan peppercorns is identified, the second robotic arm 130 drives the flexible end effector 150 to perform the picking action. Simultaneously, the first robotic arm 120 can drive the image acquisition module 140 to move. The depth camera of the image acquisition module 140 can perform multi-angle identification of the remaining part of the Sichuan peppercorn plant, identify the plant, the identified clusters of Sichuan peppercorns and the uncertain clusters of Sichuan peppercorns, verify the information of the identified clusters of Sichuan peppercorns, and change the position of the image acquisition module 140 to avoid the obstruction of branches and leaves to the uncertain clusters of Sichuan peppercorns, so as to further confirm the uncertain clusters of Sichuan peppercorns. This can realize the simultaneous picking and identification, which is conducive to improving the accuracy of identification. When encountering an uncertain cluster of Sichuan peppercorns located inside the plant, the image acquisition module 140 can be inserted into the plant. At this time, the drive shaft 182 can be rotated by adjusting the motor, and the actuating rod 183 can pass over the surface of the image acquisition module 140 to push aside the branches and leaves on the front side of the image acquisition module 140. The two actuating rods 183 can be rotated in opposite directions first to clamp the branches and leaves, and then rotated in the same direction to push the clamped branches and leaves towards one side of the image acquisition module 140, so as to avoid the branches and leaves blocking the image acquisition module 140 and to ensure the recognition effect. During the harvesting process, the position of the image acquisition module 140 can be moved by the robotic arm 120 to bypass the obstruction of branches and leaves on the flexible end effector 150, and the image acquisition module 140 can be used to check whether the flexible end effector 150 has moved into place.

[0025] Example 2 Based on Example 1, such as Figure 4-6 As shown, in this embodiment, when the central processing unit identifies the red pepper clusters in the image, it inputs the collected image information into the YOLO neural network that has completed preset training and outputs the recognition result. The recognition result includes the region information of the red pepper clusters in the image, and divides the fruits of the red pepper clusters to be picked into scattered clusters and clustered clusters. According to the type of red pepper clusters, scattered clusters and clustered clusters, the opening size of the flexible end effector 150 is adjusted. The pre-trained neural network is trained using a training sample set to obtain a trained YOLO neural network. First, an initial image sample set is obtained, which includes multiple sample images, all of which contain clusters of Sichuan pepper. The sample images in the initial image sample set are labeled, the regions of Sichuan pepper clusters in the sample images are selected, and the type of Sichuan pepper clusters is labeled to obtain a training sample set. The training sample set is then input into the pre-trained neural network for training. After training, a trained YOLO neural network is formed.

[0026] In practice, the YOLO neural network is a training model based on modifications to YOLOv8. Image recognition uses YOLOv8n (Red-YOLO) as the basic module, improves the head layer and backbone layer, and adds the CBAM attention mechanism to the head layer to enhance important information related to the red pepper cluster before feature fusion; The CBAM attention mechanism first models the channel dimension, as shown in Equations 1 and 2. Its purpose is to weight the model to focus on key channels related to the Sichuan pepper clusters given the input feature map. The channel attention weights are obtained by global average pooling and global max pooling: Formula 1: in, and This represents a shared, fully connected layer. The Sigmoid function is used to multiply the input features to obtain the enhanced features: Formula 2: in, This indicates element-wise multiplication; Subsequently, as shown in Equations 3 and 4, CBAM allocates attention in the spatial dimension, highlighting the key area where the Sichuan pepper clusters are located, and the feature map after channel compression. Aggregate channels using max pooling and average pooling, then concatenate them before performing convolution: Formula 3: in, This represents a 7×7 convolution operation; The final output is: Formula 4: Features processed by the CBAM module The network is optimized in both channel and spatial dimensions, which enables it to significantly enhance its focus on Sichuan pepper clusters while maintaining a lightweight design. CBAM can effectively suppress complex background interference and highlight the fruit area, thus achieving higher accuracy and robustness in Sichuan pepper detection. By adding a model size reduction module to the backbone layer, GSConv's efficient feature extraction capability enables the model to quickly capture fine-grained features of Sichuan pepper under limited computing resources, while VoV-GSCSP enhances the robustness of features through cross-layer information interaction, enabling the model to maintain stable detection accuracy in complex natural scenes.

[0027] like Figure 1As shown, in this embodiment, when the robotic arm 2 130 moves the flexible end effector 150 to perform the picking action, collision detection between the fruit and the branch is performed. First, based on the current joint angle, link length and connection method of the robotic arm 2 130, the real-time posture of each joint and link of the robotic arm 2 130 in three-dimensional space is obtained through forward kinematics calculation. Then, based on the structural parameters of the robotic arm 2 130, an overall model including the joint sphere, the link envelope and the shape of the end effector is constructed. Meanwhile, the dynamic environmental point cloud acquired and updated by the image acquisition module 140, together with the electronic fence boundary set according to the work area, is fused to generate a unified environmental grid representation. The central processing unit uses the spatial minimum distance calculation method to periodically solve the distance between any link surface of the robotic arm 130 and the surfaces of all obstacles in the environment, and synchronously detects possible self-collisions between links. When any minimum distance is lower than the set safety threshold, it is determined that there is a potential collision risk in the current state.

[0028] In practice, RED-YOLO is used to identify pepper clusters from the acquired RGB image, and bounding boxes are output. Based on the corresponding bounding boxes, corresponding points are taken in the depth image to generate a cluster point cloud. Euclidean clustering is used to extract a single pepper cluster point set. The centroid, principal direction and surface normal of the cluster point cloud are extracted to generate several candidate boxes. The candidate box scores are calculated, and all candidate capture poses are output to grasp_planner_node. The inverse kinematics (IK) solution is calculated for each candidate pose, unreachable poses are filtered out, and MoveIt performs collision detection to eliminate trajectories that may collide with tree branches. For successful candidates, three motion trajectories are generated: pre-grasp, approach, and retract. The system comprehensively scores the IK success rate, reachability, occlusion, and grasping success rate estimates, selects the optimal solution, and outputs the robotic arm motion plan to moveit_interface_node; MoveIt receives the trajectory and sends JointTrajectory to the joint controller. During execution, it monitors the joint position, current, and torque in real time. If a large torque, suspected collision, or joint deviation occurs, it stops and retracts immediately. After reaching the gripping point, it pauses briefly to stabilize. After the robotic arm reaches the gripping position, it sends a gripper_command(close, p_set) to start the air pump 160, which raises the internal air pressure of the flexible end effector 150 to the target pressure. The robotic arm 130 is then rotated to grip the cluster of Sichuan peppercorns and harvest them. After harvesting, the harvested cluster of Sichuan peppercorns is placed in the collection box 111 to complete the harvesting process. By detecting collisions during the harvesting process, collisions with branches and leaves can be minimized to avoid causing significant damage to the Sichuan peppercorn plant, thus achieving low-damage harvesting.

[0029] like Figure 1 As shown, in this embodiment, during the collision detection process, the theoretical torque of each joint is calculated in real time based on the dynamic model of the robotic arm 2 130 and compared with the actual current signal fed back by the drive motor. When an abnormal force situation occurs that is inconsistent with the normal movement trend, the system regards the situation as an early sign of contact collision and uses it to perform secondary verification of the geometric detection results. When either the geometric distance judgment or the force verification meets the collision risk conditions, the system enters the restriction mode. By adjusting the joint speed, restricting the movement direction, or pausing the trajectory execution, the robotic arm 2 130 is prevented from continuing to move towards the high-risk area. If necessary, the system triggers the upper-level planner to replan the current path to ensure that the robotic arm can achieve continuous and reliable collision prevention capabilities in complex and dynamic environments.

[0030] In practice, by analyzing the current signal of the robotic arm 130, abnormal force conditions can be obtained, and secondary verification can be performed, which helps to improve the accuracy of detection. Furthermore, by limiting the movement of the robotic arm 130, collisions between the robotic arm 130 and branches and leaves can be avoided as much as possible, reducing damage to the plants during the harvesting process.

[0031] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0032] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A self-propelled Sichuan pepper harvesting system, characterized in that, It includes a red pepper harvesting device (100) and a central processing unit, wherein the central processing unit is installed on the red pepper harvesting device (100); The red pepper harvesting device (100) includes a self-propelled chassis (110), a first robotic arm (120), a second robotic arm (130), an image acquisition module (140), a flexible end effector (150), a control box (170), and a branch-picking module (180). The first robotic arm (120), the second robotic arm (130), and the control box (170) are all mounted on the self-propelled chassis (110). The image acquisition module (140) includes a depth camera, which is mounted on the output end of the first robotic arm (120). The flexible end effector (150) is mounted on the output end of the second robotic arm (130). The image acquisition module (140) can acquire Sichuan pepper image information. The image acquisition module (140) can transmit the Sichuan pepper image information to the central processing unit. The central processing unit can analyze the image information and identify the Sichuan pepper clusters in the image. The central processing unit can reconstruct the three-dimensional model of the Sichuan pepper cluster to be picked based on the position information acquired by the depth camera, and determine the actual target position of the Sichuan pepper cluster to be picked in the actual coordinate system based on the image. The central processing unit can determine the execution action of the robotic arm (130) and the picking path planning of the flexible end effector (150) based on the actual current position of the flexible end effector (150) in the actual coordinate system, the actual target position of the Sichuan pepper cluster, and the action model of the robotic arm (130).

2. The self-propelled Sichuan pepper harvesting system according to claim 1, characterized in that, The red pepper harvesting device (100) also includes an air pump (160), which is used to provide an air source to the flexible end effector (150). The control box (170) is equipped with an air pump controller, which is connected to the flexible end effector (150) through an air pipe. The air pump controller can adjust the driving pressure of the flexible end effector (150).

3. The self-propelled Sichuan pepper harvesting system according to claim 1, characterized in that, When determining the actual target position of the red pepper cluster to be harvested in the actual coordinate system based on image information, the actual target position is determined by using a preset transformation model and the virtual target position. The transformation model is the transformation relationship between the virtual target position of the red pepper cluster in the image and the actual position in the actual coordinate system.

4. The self-propelled Sichuan pepper harvesting system according to claim 1, characterized in that, The image acquisition module (140) also includes a mounting plate (141), the depth camera is fixedly mounted on the mounting plate (141), and the mounting plate (141) is fixedly mounted on the output end of the robotic arm (120); The branching module (180) includes: The base (181) is fixedly mounted on the mounting plate (141), and an adjustment motor is provided inside the base (181); A drive shaft (182) is rotatably mounted on a base (181), and the output end of the regulating motor is connected to the drive shaft (182) in a transmission connection. The lever (183) is fixedly mounted on the drive shaft (182).

5. The self-propelled Sichuan pepper harvesting system according to claim 1, characterized in that, When identifying clusters of Sichuan pepper in an image, for image information that cannot be completely identified as clusters of Sichuan pepper, i.e., image information of uncertain clusters of Sichuan pepper whose feature parameters do not meet the threshold, coordinate marking is performed in the image to trigger an abandonment of harvesting command. The marked real-time image data is transmitted wirelessly to a handheld control tablet. The tablet interface simultaneously displays the location markers, morphological parameters, and surrounding branch environment of the fruit cluster. Users can manually issue harvesting or confirmation to abandon harvesting based on actual planting needs. According to the three-dimensional spatial model of the pepper plant, users can manually click on the three-dimensional coordinate point corresponding to the target fruit cluster in the model to manually select the harvesting location. The central processor then automatically plans the harvesting path based on the harvesting location.

6. The self-propelled Sichuan pepper harvesting system according to any one of claims 1-5, characterized in that, When the central processing unit identifies clusters of Sichuan pepper in an image, it inputs the collected image information into a YOLO neural network that has been pre-trained and outputs the recognition results. The recognition results include the regional information of the Sichuan pepper clusters in the image and divide the fruits of the Sichuan pepper clusters to be harvested into scattered clusters and clustered clusters.

7. The self-propelled Sichuan pepper harvesting system according to claim 6, characterized in that, The pre-trained neural network is trained using a training sample set to obtain a trained YOLO neural network. First, an initial image sample set is obtained, which includes multiple sample images, all of which contain clusters of Sichuan pepper. The sample images in the initial image sample set are labeled, the regions of Sichuan pepper clusters in the sample images are selected, and the type of Sichuan pepper clusters is labeled to obtain a training sample set. The training sample set is then input into the pre-trained neural network for training. After training, a trained YOLO neural network is formed.

8. The self-propelled Sichuan pepper harvesting system according to any one of claims 1-5, characterized in that, When the robotic arm 2 (130) drives the flexible end effector (150) to move and perform picking actions, collision detection between the fruit and the branch is performed. First, based on the current joint angle, link length and connection method of the robotic arm 2 (130), the real-time posture of each joint and link of the robotic arm 2 (130) in three-dimensional space is obtained through forward kinematics calculation. Based on the structural parameters of the robotic arm 2 (130), an overall model including the joint sphere, link envelope and end effector shape is constructed. Meanwhile, the dynamic environmental point cloud acquired and updated by the image acquisition module (140) and the electronic fence boundary set according to the work area are fused together to generate a unified environmental grid representation. The central processing unit uses the spatial minimum distance calculation method to periodically solve the distance between any link surface of the robotic arm (130) and the surface of all obstacles in the environment, and synchronously detects the possible self-collisions between the links. When any minimum distance is lower than the set safety threshold, it is determined that there is a potential collision risk in the current state.

9. The self-propelled Sichuan pepper harvesting system according to claim 8, characterized in that, During the collision detection process, the theoretical torque of each joint is calculated in real time based on the dynamic model of the second robotic arm (130), and compared with the actual current signal fed back by the drive motor. When an abnormal force situation occurs that is inconsistent with the normal motion trend, the system regards the situation as an early sign of contact collision and uses it to perform secondary verification of the geometric detection results.

10. The self-propelled Sichuan pepper harvesting system according to claim 9, characterized in that, When either the geometric distance judgment or the force check meets the collision risk conditions, the system enters the restriction mode and prevents the robotic arm 2 (130) from continuing to move into the high-risk area by adjusting the joint speed, restricting the direction of movement, or pausing the trajectory execution.

Images