A cluster tomato picking system of human-like harvesting mode and a control method thereof

Abstract

The application discloses a cluster-shaped tomato picking system of a human-like harvesting mode and a control method thereof, and belongs to the field of agricultural machinery. The application solves the compliance requirement, the accurate picking requirement and the high-efficiency picking requirement of the cluster-shaped tomato picking system, is suitable for the cluster-shaped tomato picking, has a simple structure, reliable action and simple control, and greatly improves the efficiency and the non-damage rate of the cluster-shaped tomato picking.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of agricultural picking robots, in particular to a cluster tomato picking system in a human-like harvesting mode and a control method thereof. BACKGROUND

[0002] Tomatoes are rich in nutrients, China is a major producer and consumer of fresh tomatoes, the market demand for fresh tomatoes is increasing, and the scale of greenhouse tomato cultivation will further expand, and the problem of how to efficiently pick tomatoes will follow. At present, the harvesting of tomatoes in China basically relies on manual picking, and the labor occupied by manual picking accounts for 30% to 50% of the total labor in the process, which is high in labor intensity, low in efficiency and low in mechanization level. Tomato picking is one of the core links of vegetable harvesting, and it is very important to improve the mechanized harvesting.

[0003] In the current research of tomato picking manipulator, the shear clamping fruit stem picking type and the shear clamping / adsorption integrated manipulator are easy to cause damage to the tomato fruit body, the structure control design is relatively complex, and the dependence on the pose of the fruit body is high. The flexible manipulator is good for the protection of the tomato fruit body, but it has the defects of slow speed, small clamping force, large volume and easy interference with non-target fruits. The common rigid manipulator has strong practicability for tomato picking, but the current developed manipulator is insufficient for fruit protection, and is mainly used for picking single tomato fruit, without specific structure design for the growth mode of cluster tomato plants.

[0004] In addition, in the current research of precise tomato picking, due to the random fruit setting of greenhouse tomatoes, there are situations such as mutual occlusion of fruits and occlusion of fruits by stems and leaves, and there is less research on the recognition of tomato fruit setting posture, resulting in the lack of rationality of the end posture of the manipulator when picking tomatoes. The harvesting efficiency and overall success rate of the picking robot are low, and it cannot cope with the complex working conditions in the actual scene, so it is urgent to combine the picking manipulator control strategy with the diversified characteristics of tomato fruit setting, develop a tomato vine picking strategy, combine a compliant picking manipulator with a high-precision vision system, and realize efficient tomato picking. SUMMARY

[0005] Technical purpose: In view of the deficiencies of existing tomato picking robots, such as easy damage to the fruit body, imperfect positioning information, and long picking time, the present application discloses a cluster tomato picking system in a human-like harvesting mode and a control method thereof, which can realize non-damaging picking, precise picking and efficient picking of tomatoes.

[0006] Technical scheme: In order to achieve the above technical purpose, the present application adopts the following technical scheme:

[0007] A cluster tomato picking system of a human-like harvesting mode comprises a vision system, a mechanical hand, a mechanical arm and a connecting flange group, the mechanical hand is connected with the mechanical arm through the connecting flange group, a part of the connecting flange group extends outward to form an extending end, and the vision system is fixedly installed on the extending end of the connecting flange group.

[0008] The mechanical hand comprises a driving mechanism, a fixed component and a plurality of mechanical fingers, the driving mechanism is arranged in the fixed component, one end of the mechanical finger is located in the fixed component and connected with the driving mechanism, and the other end of the mechanical finger penetrates through one end of the fixed component, and the other end of the fixed component is connected with the connecting flange group.

[0009] The application further provides a control method of the cluster tomato picking system of the human-like harvesting mode, which is applied to any one of the cluster tomato picking systems of the human-like harvesting mode and comprises the following three stages.

[0010] In stage one, a tomato vine full-view sensing stage, the picking system reaches a current picking point, hand-eye calibration is performed in an eye-in-hand mode, the vision system scans the tomato vine full view from top to bottom with (0, 0, Z), YOLOv5s algorithm is used for tomato maturity discrimination training of the vision system, then it is judged whether there is a mature tomato on the tomato vine, if there is no mature tomato, the picking system goes to the next picking point, if there is a mature tomato, positioning information of n tomato clusters in the output image is output, and stage three is entered.

[0011] In stage two, a tomato cluster focusing sensing stage, the mth tomato cluster is positioned at a close distance in the positioning information of the n tomato clusters in the output image, the initial value of m is 1, it is judged whether there is a mature tomato in the mth tomato cluster, if there is no mature tomato, it is judged whether m is greater than or equal to n, if m is greater than or equal to n, the current tomato vine picking is completed, and the picking system goes to the next picking point in stage one, if m is less than n, the (m+1)th tomato cluster is positioned at a close distance, if there is a harvestable mature tomato, the six-dimensional pose information of the mature tomato with a low tomato height and an outer tomato depth is output through weight The six-dimensional pose information of the mature tomato with a low tomato height and an outer tomato depth is output, and stage three is entered, wherein Z represents tomato depth information, X represents tomato height information, p is a tomato depth information weight, q is a tomato height information weight, and p+q=1.

[0012] In stage three, a picking and recycling stage, the mechanical arm reaches the picking point through path planning, the mechanical hand performs picking work through I / O control, then the mechanical arm reaches the recycling point, and the picked tomato is placed on the recycling point, after the picking and recycling stage is completed, the picking system returns to the scanning tomato vine full view process in the tomato cluster focusing sensing stage, and the next picking cycle is performed.

[0013] Beneficial effects: the cluster tomato picking system of the human-like harvesting mode provided by the application has the following beneficial effects:

[0014] The picking system of the present application cooperates with the mechanical arm, the connecting flange group, the mechanical hand and the vision system, the connecting flange group is provided with an extension end for placing the vision system, which avoids the mechanical hand from blocking the vision system when the vision system is placed at the rear end of the mechanical hand, can effectively collect the ripening information of tomatoes, and the mechanical hand is composed of a driving assembly, a fixing assembly and a plurality of mechanical fingers, the driving assembly drives the mechanical fingers to move for picking tomatoes, and the fixing assembly can stabilize the mechanical hand to ensure stable operation of the mechanical hand, through the cooperation of the system structure, the tomatoes can be picked quickly and efficiently.

[0015] The mechanical hand of the present application adopts an underactuated structure, the up-and-down movement of the screw nut drives the rotation of the finger transmission rod and the motor transmission rod, thereby driving the opening and closing movement of the mechanical fingers, so that the structure is smaller and more compact, the control is simple, the action is reliable, the power requirement is reduced, the weight is reduced, and the production cost is reduced, and the present application adopts a non-destructive picking structure, a roller is arranged on the fingertip, which plays a rolling friction effect with the surface of the tomato during the coating process without affecting the installation, reduces the skin damage caused by sliding friction, and enhances the protection of the mechanical hand to the tomato.

[0016] The finger pulp and the fingertip of the present application are covered with foam from top to bottom with uniform thickness, the concave arc curvature of the flexible foam covering the clamping surface is close to the convex arc curvature of the tomato, the contact area with the tomato is increased, slipping during picking tomatoes is avoided, the instantaneous contact pressure is reduced, the burden of the mechanical hand is reduced, the torsional spring is embedded on the cylindrical pin between the finger root and the finger transmission rod, two torsional arms act on the isolation columns of the finger root and the finger transmission rod respectively, the torsional force requirement of the mechanical fingers is met, and the function of continuous buffer pressure is played.

[0017] The present application provides a tomato vine picking control method, including a tomato vine overall visual perception stage, a tomato cluster focusing perception stage and a picking and recycling stage, and a control strategy based on imitating hand picking is designed for the mechanical hand picking link, the picking mode of the working process of adjusting the pose, gripping the tomato, pulling down the fruit, rotating and twisting off and transporting the tomato is adopted, the requirements of cluster tomato picking flexibility, accurate picking and efficient picking are solved.

[0018] The present application matches the region of interest (ROI) of the tomato and its corresponding fruit stem according to the matching algorithm, calculates the matching success probability of the tomato and the fruit stem, determines the two-dimensional coordinates of the tomato and the fruit stem by using a two-dimensional coordinate positioning algorithm, and then obtains the three-dimensional coordinates of the corresponding fruit stem of the tomato by quickly judging the points in the closed area and traversing the points in the irregular area, finally calculates and generates a vector from the tomato to the fruit stem as the reference posture of the mechanical arm entering during picking, and realizes accurate positioning and posture estimation of the tomato. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows.

[0020] Figure 1 Structure isometric view of a cluster tomato picking system of the present application in a human-like harvesting mode;

[0021] Figure 2 Structure isometric view of a mechanical hand of a cluster tomato picking system of the present application in a human-like harvesting mode;

[0022] Figure 3 Structure isometric view of a connecting flange group of the present application;

[0023] Figure 4 Structure isometric view of a mechanical finger and its lower mechanical structure of the present application;

[0024] Figure 5 Flow chart of a visual system identification of the present application;

[0025] Figure 6 Flow chart of a picking and recycling stage of the present application;

[0026] Figure 7 Diagram of a picking system perceiving the overall appearance of a tomato vine of the present application;

[0027] Figure 8 Diagram of a picking system focusing on perceiving a tomato cluster of the present application;

[0028] Figure 9 Diagram of a picking system picking and recycling a tomato of the present application;

[0029] Figure 10 Diagram of a tomato picking control flow of the present application.

[0030] In the drawings: 1, visual system; 2, mechanical hand; 3, mechanical arm; 4, connecting flange group; 5, roller; 6, flexible foam; 7, finger pulp; 8, finger root; 9, torsion spring; 10, finger transmission rod; 11, stepping motor; 12, bottom flange; 13, supporting copper column; 14, motor transmission rod; 15, screw nut; 16, top flange; 17, finger tip; 18, mechanical hand connecting flange; 18, mechanical arm connecting flange. DETAILED DESCRIPTION

[0031] The present application will be described in more detail by a preferred embodiment and in conjunction with the drawings, but the present application is not limited in the scope of the described embodiments.

[0032] As Figure 1As shown, a cluster tomato picking system in a human-like harvesting mode comprises a vision system 1, a mechanical hand 2, a mechanical arm 3 and a connecting flange group 4, the mechanical hand 2 is connected with the mechanical arm 3 through the connecting flange group 4, a part of the connecting flange group 4 extends outward to form an extension end, the vision system 1 is fixedly installed on the extension end of the connecting flange group 4, the vision system 1 contains a depth camera, and the flange extension end is used for installing the vision system 1, while avoiding that the depth camera is placed at the rear end of the mechanical hand to block the field of view of the depth camera when the mechanical hand fingers open and close.

[0033] The mechanical hand 2 comprises a driving mechanism, a fixed component and a plurality of mechanical fingers, the driving mechanism is arranged inside the fixed component, one end of the mechanical finger is located inside the fixed component and connected with the driving mechanism, the other end penetrates one end of the fixed component, and the other end of the fixed component is connected with the connecting flange group 4.

[0034] As shown in Figure 2 and Figure 4 The mechanical finger comprises a roller 5, a flexible foam 6, a finger pulp 7, a finger root 8 and a finger tip 17, the roller 5 has a height of 5 mm and a diameter of 8 mm, which does not affect the installation and plays a rolling friction effect with the surface of the tomato in the coating process, reducing the skin damage caused by sliding friction, the roller 5 is connected with the finger tip 17 through a cylindrical pin and is located at the center position and slightly higher than the finger tip 17, the finger tip 17 is fixedly arranged at one end of the finger pulp 7, the other end of the finger pulp 7 is connected with the finger root 8 through a cylindrical pin, and the flexible foam 6 is arranged on the finger pulp 7 and the finger tip 17, the thickness of the flexible foam at the finger tip 17 is 1 mm, and the average thickness of the flexible foam at the finger pulp 7 is 2 mm, the thickness of the flexible foam increases uniformly from top to bottom, and the concave arc surface curvature of the flexible foam covering the clamping surface is close to the convex arc surface curvature of the tomato, which meets the curvature requirement in the coating process, and the finger root 8 is connected with the driving mechanism through a cylindrical pin.

[0035] As shown in Figure 2As shown, the driving mechanism includes torsion spring 9, finger drive rod 10, stepper motor 11, motor drive rod 14 and screw nut 15, the finger drive rod 10 is connected with the finger root 8 through a cylindrical pin and the cylindrical pin is sleeved with a torsion spring 9, the finger root 8 and the finger drive rod 10 are both provided with an isolation column, the two torsion arms of the torsion spring 9 act on the isolation columns of the finger root 8 and the finger drive rod 10 respectively, the torsion angle is 90°, the wire diameter is 2mm, the effective number of spring turns is 3 turns, and the torsion arm length is 20mm, which meets the requirement of the mechanical finger on the torsion force, the motor drive rod 14 is rotatably connected with the finger drive rod 10 through a cylindrical pin, realizes the driving of the mechanical hand, supports the realization of tomato picking, and the screw nut 15 and the motor drive rod 14 are rotatably connected with the stepper motor 11 after being connected with the cylindrical pin, and the stepper motor 11 is fixed in the inside of the fixed assembly.

[0036] As shown in Figure 4 The fixed assembly includes a bottom flange 12, a support copper column 13 and a top flange 16, the support copper column 13 is connected with the top flange 16 and the bottom flange 12 through threads respectively, the stepper motor 11 is located between the bottom flange 12 and the top flange 16 and is fixed on the bottom flange 12, the bottom flange 12 is fixedly connected on the connecting flange group 4, the top flange 16 is provided with three limiting holes with a length of 38mm and a width of 16mm for isolating the working part and the driving part of the mechanical finger, and simultaneously assisting in limiting the opening and closing amplitude of the mechanical finger to about 45°, and flexibly picking various sizes of tomatoes.

[0037] As shown in Figure 3 The connecting flange group 4 includes a mechanical hand connecting flange 18 and a mechanical arm connecting flange 19, the mechanical hand connecting flange 18 is embedded and fixed in the mechanical arm connecting flange 19 and is connected by threads, the mechanical hand flange 18 is fixedly connected with the bottom flange 12, the mechanical arm connecting flange 19 is fixedly connected with the mechanical arm 3, the mechanical arm 3 is connected with the mechanical hand 2 through the four threaded holes of the mechanical hand connecting flange 18 of the connecting flange group 4, and the mechanical arm connecting flange 19 is provided with the protruding end, the depth camera is installed on the protruding end of the mechanical arm connecting flange 19, which avoids the blocking of the camera field of view when the fingers are opened and closed, saves the component assembly space, and prevents the mechanical hand from being entangled by tomato vines and leaves.

[0038] As shown in Figure 10 A control method of a cluster tomato picking system in a human-like harvesting mode, any one of the cluster tomato picking systems in a human-like harvesting mode, includes the following three stages:

[0039] As shown in Figure 7As shown, stage one, tomato vine overall perception stage: the picking system reaches the current picking point, and the hand-eye calibration is performed in the eye-in-hand manner. The vision system based on the ROS operating system framework controls the stage control. The vision system scans the tomato vine overall from top to bottom with (0, 0, Z). The vision system uses the YOLOv5s algorithm to train the tomato maturity discrimination. Then, the maturity of the tomatoes on the tomato vine is judged. If there is no mature tomato, the next picking point is reached. If there is a mature tomato, the positioning information of n tomato clusters in the image is output, and the second stage is entered.

[0040] As shown in Figure 8 Stage two, tomato cluster focusing perception stage: the mth tomato cluster is positioned at a close distance, and m is initially 1. It is judged whether there is a mature tomato that can be picked. If there is no mature tomato that can be picked, it is judged whether m is greater than or equal to n. If m is greater than or equal to n, the current tomato vine picking is ended, and the next picking point in stage one is reached. If m is less than n, the m+1th tomato cluster is positioned at a close distance. If there is a mature tomato that can be picked, the six-dimensional pose information of the mature tomato with a low height and an outer depth is output through the weights of the tomato depth information and the tomato height information, so that the mechanical hand is positioned more accurately, and the third stage is entered. In the third stage, Z represents the tomato depth information, X represents the tomato height information, p represents the tomato depth information weight, q represents the tomato height information weight, p+q=1, and the values of p and q are determined according to actual conditions. In the initial stage, p=0.7 and q=0.3 can be taken.

[0041] As shown in Figure 9 Stage three, picking and recycling stage: the total control box of the mechanical arm communicates with the computer through a network port to control the path planning of the mechanical arm and the opening and closing of the mechanical hand. The mechanical hand is controlled by a single-chip microcomputer and can be automatically opened and closed according to image information and current feedback. The mechanical hand communicates with the mechanical arm control box through I / O signals.

[0042] The mechanical arm reaches the picking point through path planning, and the mechanical hand performs picking work through I / O signals. Then, the mechanical arm reaches the recycling point to place the picked tomatoes at the recycling point. After the picking and recycling stage is completed, the scanning process of the tomato vine overall in the tomato cluster focusing perception stage is returned, and the next picking cycle is performed until the vision system does not identify tomatoes, and the picking cycle is ended.

[0043] As shown in Figure 5 The specific steps of using the YOLOv5s algorithm to train the tomato maturity discrimination of the vision system include the following steps:

[0044] S11, image acquisition: the vision system acquires tomato images under different distances, angles and light conditions, and pre-processes the tomato images.

[0045] Collect tomato images under different distances, angles and light conditions, and focus on the main factors that affect the recognition effect: 1) no occlusion between tomatoes; 2) slight overlap between tomatoes; 3) main stem occludes tomatoes; 4) leaf occludes tomatoes; 5) multiple clusters of tomatoes appear.

[0046] S12, labeling and training: the preprocessed tomato image data samples are manually labeled, the red ripe period tomato sample is labeled as "red ripening", the hard ripe period tomato is labeled as "hard ripening", the green ripe period tomato is labeled as "green ripening", and the target fruit stem sample is labeled as "stem". Then, the model is trained based on the YOLOv5s algorithm.

[0047] The positioning information of the n tomato clusters in the output image specifically includes the following steps:

[0048] S21, tomatoes and fruit stems are paired according to the matching algorithm: the tomato and fruit stem matching algorithm is used to correspond the tomato with its corresponding fruit stem ROI region, the pairing score of each fruit stem to each tomato is calculated, and the higher score is taken for pairing to obtain the tomato and corresponding fruit stem ROI.

[0049] S22, two-dimensional coordinate positioning algorithm is used to determine the two-dimensional coordinates of the tomato and the fruit stem: after obtaining the ROI region of the tomato and the corresponding fruit stem, the R enhancement GB suppression of the ROI region of the tomato is performed to obtain a gray image , the G enhancement RB suppression of the ROI region of the fruit stem is performed to obtain a gray image , the accurate contours of the tomato and the fruit stem are determined respectively, and the center of the minimum circumscribed rectangle of the contour is taken as the two-dimensional coordinates of each.

[0050] That is, adaptive threshold segmentation and contour finding are performed, the size of the minimum circumscribed rectangle of the fruit stem contour is taken as 1 / 2, that is, the longer side is shortened to half of the original, and the maximum gray region that can be framed in the ROI region is found, and the center of the rectangular frame is the two-dimensional coordinates (x, y) of the fruit stem. The two-dimensional physical coordinates (x, y) of the tomato and the fruit stem relative to the depth camera can be calculated by combining the screen space coordinates (x, y) with the depth camera internal parameters.

[0051] S23, depth information acquisition: based on the two-dimensional coordinates in the tomato and fruit stem gray images obtained in step S22, the depth map (Depth Map) of the tomato and the fruit stem is obtained by solving the depth in the Z-axis direction, the points in the closed region in the Depth Map are quickly distinguished, and the points in the irregular edge region in the closed region of the Depth Map are traversed, to obtain the three-dimensional coordinates of the tomato and the fruit stem. The closed region in the Depth Map refers to the region containing valid data in the Depth Map.

[0052] That is, the Z-axis depth of the RGBD image is provided by the D channel, and a method of averaging irregular edge regions in the D channel to obtain the overall Z-axis depth of the object is proposed to optimize the effective data area of the measurement.

[0053] Quickly determine the point in the closed area: by taking the gradient, thresholding, and contour detection, the closed contour of the fruit stem and tomato in the picture can be obtained, which is composed of multiple line segments and stored in an array in the form of continuous line segment endpoint coordinates. In two-dimensional space, a point in the closed contour emits an arbitrary ray, and the intersection of the ray and the contour must have an odd number of points. In order to simplify the concept, the point emits a ray in the x direction, and the intersection of the ray and the contour is calculated. In this way, it can be determined whether the point is in the closed area. After partial compression and fitting of the contour line segment, this method has high efficiency.

[0054] Traverse the points in the irregular region: as described above, the irregular region contour is composed of multiple line segments and stored in an array in the form of continuous line segment endpoint coordinates. The present application proposes a method for traversing the irregular region. The non-rotating minimum circumscribed rectangle of the irregular region is taken to obtain the upper and lower boundaries of the region. Traverse from the upper boundary y_1 to the lower boundary y_2. For each y, take all the intersection points of the line segment at the corresponding height, arrange them in ascending order of x, and take all the odd-numbered x as the lower boundary of x, and all the even-numbered x as the upper boundary. The points between the lower boundary and the upper boundary are the points that need to be traversed.

[0055] S24, precise pose information acquisition of the mechanical arm: after obtaining the three-dimensional coordinates of the tomato and the corresponding fruit stem, a vector pointing from the tomato to the fruit stem is calculated. The generated vector is moved in the opposite direction of the pointing direction until the starting point of the vector falls on the edge of the tomato, and the pose that the mechanical arm should enter when picking is obtained.

[0056] As shown in Figure 6 the picking and recycling stage specifically includes the following steps:

[0057] S31, mechanical hand opening control: according to the diameter of the tomato output by the vision system, the mechanical hand is opened, the step motor step number is controlled, and the mechanical hand is opened to 70% of the diameter of the tomato;

[0058] S32, mechanical arm pose adjustment: through the acquired six-dimensional pose information, path planning is performed, the mechanical arm is controlled to reach the set picking point, and the pose of the mechanical arm is adjusted. There are two ways to reach the tomato when adjusting the pose of the mechanical arm, one is the bottom gripping method, that is, the gripping action is completed by deepening the bottom of the tomato, and the other is the lateral gripping method, that is, the gripping action is completed by wrapping the tomato from the side;

[0059] S33, mechanical arm clamping control: after reaching the picking point and adjusting the mechanical arm pose, the current during motor rotation is detected to realize clamping control when the fingers are closed. In the case of bottom gripping and lateral gripping of the mechanical arm, the actual force of the mechanical hand type is the thumb, index finger and ring finger;

[0060] S34, picking control: after the mechanical arm clamps the tomato, the mechanical arm is controlled to rotate a certain angle, so that the connection between the fruit stem and the tomato is torn, and the fruit stem separation is completed. In the case of bottom gripping of the mechanical arm, the mechanical arm rotates to rotate the mechanical hand 540° around the fruit stem axis, so that the tomato is broken and falls off from the separation layer. In the case of lateral gripping of the mechanical arm, the mechanical arm rotates to rotate the mechanical hand 180° around the axis perpendicular to the fruit stem, so that the tomato is separated from the fruit stem to realize picking.

[0061] S35, recycling and transportation control: the mechanical arm is controlled to transport the successfully picked tomato to the fixed position of the designated collection basket. The mechanical hand opens the fingers, and the tomato falls freely. The free-fall height is about 40cm, which ensures that the tomato will not be damaged when falling.

[0062] The tomato and the fruit stem are matched according to a matching algorithm, which specifically includes: first, a two-dimensional vector pointing from the tomato to the fruit stem is generated, and the included angle between the two-dimensional vector and the y-axis is calculated , unit radian Then, the ratio k of the size of the overlapping area of the detection box of the tomato and the fruit stem to the size of the detection box of the fruit stem is calculated, and finally the matching score , wherein M represents the pose weight, N represents the overlap weight, and M+N=1 is always true. M and N need to be adjusted according to the actual situation. Initially, M=0.6 and N=0.4 can be taken. The matching scores of all tomatoes and fruit stems are calculated, and the tomato with the higher matching score is matched with the fruit stem.

[0063] The above only describes the preferred embodiments of the present application. It should be noted that for ordinary skilled persons in the art, without departing from the principles of the present application, several improvements and refinements can be made, which should also be considered as the protection scope of the present application.

Claims

1. A control method of a cluster tomato picking system in a human-like harvesting mode, characterized in that, the cluster tomato picking system comprises a vision system (1), a mechanical hand (2), a mechanical arm (3), and a connecting flange group (4), the mechanical hand (2) is connected with the mechanical arm (3) through the connecting flange group (4), a part of the connecting flange group (4) extends outward to form an extension end, and the vision system (1) is fixedly installed on the extension end of the connecting flange group (4); the mechanical hand (2) comprises a driving mechanism, a fixed component, and a plurality of mechanical fingers, the driving mechanism is arranged in the fixed component, one end of the mechanical finger is located in the fixed component and connected with the driving mechanism, the other end of the mechanical finger penetrates one end of the fixed component, and the other end of the fixed component is connected with the connecting flange group (4); the mechanical finger comprises a roller (5), flexible foam (6), a finger pulp (7), a finger root (8), and a finger tip (17), the roller (5) is connected with the finger tip (17) through a cylindrical pin, the finger tip (17) is fixedly arranged at one end of the finger pulp (7), the other end of the finger pulp (7) is connected with the finger root (8) through a cylindrical pin, and the flexible foam (6) is arranged on the finger pulp (7) and the finger tip (17), and the finger root (8) is connected with the driving mechanism through a cylindrical pin; the driving mechanism comprises a torsional spring (9), a finger transmission rod (10), a stepping motor (11), a motor transmission rod (14), and a screw nut (15), the finger transmission rod (10) is connected with the finger root (8) through a cylindrical pin, and a torsional spring (9) is sleeved on the cylindrical pin, the finger root (8) and the finger transmission rod (10) are both provided with an isolation column, two torsional arms of the torsional spring (9) act on the isolation column of the finger root (8) and the isolation column of the finger transmission rod (10) respectively, the motor transmission rod (14) is rotationally connected with the finger transmission rod (10) through a cylindrical pin, and the screw nut (15) and the motor transmission rod (14) are rotationally connected with the stepping motor (11) after being connected through a cylindrical pin, and the stepping motor (11) is fixedly arranged in the fixed component; the control method comprises the following three stages: Stage 1: whole tomato vine perception stage: the picking system reaches the current picking point, performs hand-eye calibration in an eye-in-hand mode, the vision system scans the whole tomato vine from top to bottom with (0, 0, Z), uses a YOLOv5s algorithm on the vision system to perform tomato maturity discrimination training, then judges the maturity of each tomato cluster on the tomato vine, if there is no mature tomato, goes to the next picking point, if there is a mature tomato, outputs the positioning information of n tomato clusters in the image, and enters stage 2. Stage two, tomato cluster focus perception stage: in the output image n tomato cluster positioning information in the close range positioning of the m tomato cluster, m initial value is 1, judge whether there is ripe tomato in the m tomato cluster, if there is no ripe tomato, judge whether m is greater than or equal to n, if m is greater than or equal to n, the current tomato vine picking is finished, enter stage one to go to the next picking point, if m is less than n, the m+1 tomato cluster close range positioning is carried out, if there is mature tomato that can be picked, the weight Output the six-dimensional pose information of the ripe tomato with the height of the tomato at the lower end and the depth of the tomato at the outer end, enter stage three, wherein Z represents the depth information of the tomato, X represents the height information of the tomato, p is the weight of the depth information of the tomato, q is the weight of the height information of the tomato, and p+q=1; in stage two, the positioning information of n tomato clusters in the output image includes pairing the tomato and the stem according to the matching algorithm, that is, using the tomato and stem matching algorithm to correspond the tomato and its corresponding stem ROI region, calculating the pairing score of each stem to each tomato, and pairing the higher score, obtaining the tomato and the corresponding stem ROI region; further processing to obtain the positioning information of n tomato clusters; Stage three, picking and recycling stage: through path planning, the mechanical arm reaches the picking point, the mechanical hand performs picking work through I / O signal control, then the mechanical arm reaches the recycling point, and the picked tomatoes are placed in the recycling point. After the picking and recycling stage is completed, it returns to the tomato cluster focusing perception stage, and the next picking cycle is performed until the vision system does not identify tomatoes, and the picking cycle is ended.

2. The control method of the cluster tomato picking system of the human-like harvesting method according to claim 1, characterized in that, The specific steps of using the YOLOv5s algorithm to distinguish and train the maturity of tomatoes for the vision system include the following steps: S11, image acquisition: the vision system collects tomato images under different distances, angles and light conditions, and pre-processes the tomato images; S12, labeling and training: the pre-processed tomato image data samples are manually labeled, the red ripe tomato samples are labeled as "red ripening", the hard ripe tomato samples are labeled as "hard ripening", the green ripe tomato samples are labeled as "green ripening", and the stem samples are labeled as "stem". Then, the model is trained based on the YOLOv5s algorithm.

3. The control method of the cluster tomato picking system of the human-like harvesting method according to claim 1, characterized in that, The positioning information of the n tomato clusters in the output image further includes the following steps after the tomatoes and stems are paired according to the matching algorithm: S22, determine the two-dimensional coordinates of the tomato and the fruit stalk by using a two-dimensional coordinate positioning algorithm: after obtaining the ROI region of the tomato and the corresponding fruit stalk, enhance the R enhancement GB suppression of the ROI region of the tomato to obtain a gray image , enhance the G enhancement RB suppression of the ROI region of the fruit stalk to obtain a gray image , respectively determine the accurate contour of the tomato and the fruit stalk, and take the center of the minimum circumscribed rectangle of the contour as the respective two-dimensional coordinates; S23, depth information acquisition: based on the two-dimensional coordinates in the gray-scale images of the tomatoes and stems obtained in step S22, the Depth Map of the tomatoes and stems is obtained by solving the depth in the Z-axis direction. The points in the closed area in the Depth Map are quickly identified, and the points in the irregular edge area in the closed area of the Depth Map are traversed to obtain the three-dimensional coordinates of the tomatoes and stems; S24, precise pose information acquisition of the mechanical arm: after obtaining the three-dimensional coordinates of the tomatoes and corresponding stems, a vector pointing from the tomato to the stem is calculated and generated. The generated vector is moved in the opposite direction of the pointing direction until the starting point of the vector falls on the edge of the tomato, and the pose that the mechanical arm should enter when picking is obtained.

4. The control method of the cluster tomato picking system of the human-like harvesting method according to claim 1, characterized in that, The picking and recycling stage specifically includes the following steps: S31, mechanical hand opening control: the mechanical hand is opened according to the diameter of the tomato output by the vision system, and the step motor step number is controlled to realize the opening of the mechanical hand to 70% of the diameter of the tomato; S32, mechanical arm pose adjustment: through the acquired six-dimensional pose information, path planning is performed to control the mechanical arm to reach the set picking point and adjust the pose of the mechanical arm. There are two ways to reach the tomato when adjusting the pose of the mechanical arm, one is the bottom gripping method, that is, the gripping action is completed by deepening the bottom of the tomato, and the other is the lateral gripping method, that is, the gripping action is completed by wrapping the tomato from the side; S33, mechanical hand clamping control: after reaching the picking point and adjusting the pose of the mechanical arm, the current during motor rotation is detected to realize clamping control when the fingers are closed. When the mechanical arm is in the bottom gripping mode and the lateral gripping mode, the actual force fingers of the mechanical hand shape are the thumb, index finger and ring finger. S34, picking control: after the mechanical hand holds the tomato, the mechanical arm is controlled to rotate a certain angle, so that the connection between the fruit stem and the tomato is torn, and the fruit stem separation is completed. When the mechanical arm is a bottom gripping method, the mechanical arm rotates to drive the mechanical hand to rotate 540° around the fruit stem axis, so that the tomato is broken and falls off from the separation layer. When the mechanical arm is a lateral gripping method, the mechanical arm rotates to drive the mechanical hand to rotate 180° around the axis perpendicular to the fruit stem, so that the tomato is separated from the fruit stem to realize picking; S35, recycling and transportation control: control the mechanical arm to transport the successfully picked tomatoes to the fixed position of the designated collection basket to complete the tomato recycling.

5. The control method of the cluster tomato picking system of the human-like harvesting method according to claim 1, characterized in that, The pairing of the tomatoes and the peduncles according to the matching algorithm specifically comprises: first generating a two-dimensional vector pointing from the tomato to the peduncle, calculating an included angle between the two-dimensional vector and a y-axis , then calculating a ratio k of a size of an overlapping region of a detection box of the tomato and the peduncle to a size of a detection box of the peduncle, and finally calculating a pairing score , wherein M represents a posture weight, N represents an overlapping weight, M+N=1, the pairing scores of all the tomatoes and the peduncles are calculated respectively, and the tomato and the peduncle with a higher pairing score are paired.

6. The control method of the cluster tomato picking system of the human-like harvesting method according to claim 1, characterized in that, The fixed assembly includes a bottom flange (12), a support copper column (13), and a top flange (16). The support copper column (13) is connected with the top flange (16) and the bottom flange (12) through threads. The stepper motor (11) is located between the bottom flange (12) and the top flange (16) and is fixed on the bottom flange (12). The bottom flange (12) is fixedly connected to the connecting flange group (4).

7. The control method of a cluster tomato picking system of a human-like harvesting method according to claim 6, characterized in that, The connecting flange group (4) includes a mechanical hand connecting flange (18) and a mechanical arm connecting flange (19). The mechanical hand connecting flange (18) is embedded and fixed in the mechanical arm connecting flange (19). The mechanical hand connecting flange (18) is fixedly connected with the bottom flange (12). The mechanical arm connecting flange (19) is fixedly connected with the mechanical arm (3). The mechanical arm connecting flange (19) is provided with the extension end. The vision system (1) is fixedly installed on the extension end of the mechanical arm connecting flange (19).

8. The control method of the cluster tomato picking system of the human-like harvesting method according to claim 1, characterized in that, The flexible foam (6) at the fingertip (17) is 1mm thick. The flexible foam (6) at the finger pulp (7) is 2mm thick on average. The flexible foam (6) at the finger pulp (7) increases in thickness uniformly from top to bottom, and the flexible foam (6) covering the clamping surface is a concave circular arc surface.

Images