A method of harvesting mature fruit

Abstract

The present application relates to a kind of mature fruit picking method, comprising the following steps: according to the growth characteristics of fruit, preset the pre-picking position of mechanical arm and the pre-picking posture of end effector;Get the image of fruit to be picked, analyze and detect the position information and posture information of mature fruit stem in camera coordinate system;The position information of mature fruit stem in depth camera coordinate system is transformed into the position information in the base coordinate system of mechanical arm, path planning is carried out and the intermediate point of auxiliary picking, cutting position and auxiliary retreat intermediate point are determined;Mechanical arm moves from pre-picking position to auxiliary picking intermediate point along the planned path, adjusts the angle of end effector and then moves linearly to cutting position, cuts and clamps mature fruit through end effector and linearly retreats to auxiliary retreat intermediate point along original path;Move to the release position set in advance to release the picked mature fruit, complete picking.Compared with prior art, the present application improves picking success rate and efficiency.

Description

Technical Field

[0001] This invention relates to the field of fruit harvesting technology, and in particular to a method for harvesting mature fruits. Background Technology

[0002] Fruit harvesting is the most time-consuming and labor-intensive stage of fruit production. Labor required during harvest accounts for approximately 50% to 70% of the entire planting process. To reduce labor input during harvesting, existing technologies utilize agricultural robots for fruit harvesting. However, fruit harvesting places high demands on agricultural robots, requiring high harvesting precision to ensure quality. The quality of harvesting directly affects the storage, processing, and sale of the fruit, ultimately influencing market prices and economic benefits. Due to the diverse and often irregular growth patterns of fruit stalks in the growing environment, fruit harvesting is highly complex, resulting in a still low level of automation in fruit harvesting. For example, patent application CN116034732A discloses a method for fuzzy fruit harvesting. By acquiring an image of the fruit to be harvested, the method identifies and analyzes the image to calculate the direction of the fruit cluster and the coordinates of the harvesting point on the fruit stalk. The direction of the fruit cluster is divided into forward growth, left growth, and right growth. Based on the direction of the fruit cluster and the coordinates of the harvesting point on the fruit stalk, the method determines the harvesting posture and cutting position of the end effector. The end effector moves to the cutting position to complete the harvesting of the fruit. However, in the actual growing environment, the direction of the fruit cluster is often chaotic. Simply classifying the direction of the fruit cluster into forward growth, left growth, and right growth often leads to unsatisfactory harvesting results. For example, patent application CN114846998A discloses a fruit picking method and system for a binocular robot based on the YOLOv4 algorithm. This patent describes a method for dealing with fruits that are blocked by fruit leaves. However, under the condition of cultivating fruits in greenhouses using cultivation racks, when the fruits are ripe, there are rarely cases where the fruits are blocked. Adding classification logic for blocked fruits will lead to an increase in fruit picking costs.

[0003] Current technologies do not take into account the characteristics of fruit stems in the planting environment, such as disordered growth direction and different shapes, nor do they consider the collision and pulling problems between the robotic arm and surrounding crops when performing harvesting tasks. At the same time, there are problems such as long harvesting cycle and low efficiency. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a method for harvesting mature fruits. It takes into account the problem of collision and pulling between the robotic arm and the surrounding crops when performing the harvesting task, thereby improving the fruit harvesting efficiency and shortening the harvesting cycle.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] A method for harvesting ripe fruit includes the following steps:

[0007] Based on the growth characteristics of the fruit, the pre-harvesting position of the robotic arm and the pre-harvesting posture of the end effector are preset;

[0008] Acquire images of fruits to be picked, perform image analysis and detection on the images of fruits to be picked, and obtain the position and orientation information of the mature fruit stems in the camera coordinate system;

[0009] The position information of the mature fruit stem in the depth camera coordinate system is transformed to the position information of the mature fruit stem in the robot arm base coordinate system. Based on the position information of the mature fruit stem in the robot arm base coordinate system and the structural parameters of the end effector, path planning is performed and the auxiliary picking intermediate point, cutting position and auxiliary retraction intermediate point are determined.

[0010] The robotic arm is moved from the pre-harvesting position to the midpoint of the auxiliary harvesting along the planned path, and the end effector is adjusted to a suitable angle according to the posture information of the mature fruit stem.

[0011] After adjustment, the robotic arm is moved in a straight line from the auxiliary picking midpoint to the cutting position, and the mature fruit is cut and clamped by the end effector. Then, it is moved back in a straight line along the original path to the auxiliary return midpoint.

[0012] The robotic arm moves from the auxiliary retraction midpoint to a pre-set release position to release the harvested ripe fruit, thus completing the harvest.

[0013] Furthermore, the principle for setting the pre-harvesting position of the robotic arm is that the depth camera on the robotic arm acquires the image of the fruit to be harvested at the pre-harvesting position, which contains the largest number of fruits.

[0014] Furthermore, the pre-harvesting posture of the end effector is specifically such that the end effector is facing the fruit planting row and the angle between the shearing plane of the end effector and the horizontal plane is 0°.

[0015] Furthermore, the step of obtaining the posture information of the mature fruit stem includes: identifying the mature fruit in the image of the fruit to be picked using the YOLO algorithm, obtaining the mature fruit stem by fitting a straight line, and taking the angle between the mature fruit stem and the y-axis of the camera coordinate system as the posture information of the mature fruit stem.

[0016] The included angle and the angle at which the end effector adjusts at the midpoint of the assisted picking are matched.

[0017] Furthermore, the specific steps for transforming the position information of the mature fruit stem in the depth camera coordinate system to the position information of the mature fruit stem in the robot arm base coordinate system include:

[0018] The homogeneous transformation matrix between the depth camera coordinate system and the robotic arm end effector coordinate system is obtained through a visual hand-eye calibration method.

[0019] The homogeneous transformation matrix between the end effector coordinate system and the base coordinate system of the robotic arm is obtained by using the kinematics method of the robotic arm.

[0020] The homogeneous transformation matrix between the depth camera coordinate system and the robot arm base coordinate system is obtained from the above two homogeneous transformation matrices;

[0021] The position information of the mature fruit stem in the depth camera coordinate system is transformed into the position information of the mature fruit stem in the robot arm base coordinate system using the homogeneous transformation matrix between the depth camera coordinate system and the robot arm base coordinate system.

[0022] Furthermore, the position information of the mature fruit stem in the depth camera coordinate system is as follows:

[0023] (P x ,P y ,P z )

[0024] In the formula, P x P represents the horizontal position information of the mature fruit stem in the depth camera coordinate system. y P represents the vertical position information of the mature fruit stem in the depth camera coordinate system. z This provides the positional information of the mature fruit stem along the depth direction in the depth camera coordinate system.

[0025] The position information of the mature fruit stem in the coordinate system of the robotic arm base is as follows:

[0026] (P_t x ,P_t y ,P_t z )

[0027] In the formula, P_t x P_t represents the position information of the mature fruit stem on the x-axis in the coordinate system of the robot arm base. y P_t represents the position information of the mature fruit stem on the y-axis in the coordinate system of the robot arm base. z This provides the position information of the mature fruit stem on the z-axis in the coordinate system of the robotic arm base.

[0028] Furthermore, the coordinates of the auxiliary harvesting midpoint are:

[0029] (P_m1x ,P_m1 y ,P_m1 z )

[0030] P_m1 x =P_t x -(L1+L2)*cos(α)

[0031] P_m1 y =P_t y +(L1+L2)*sin(α)

[0032] P_m1 z =P_t z +Z1

[0033] In the formula, P_m1 x To assist in determining the position of the picking center point on the x-axis, P_m1 y To assist in determining the position of the picking center point on the y-axis, P_m1 z To assist in determining the position of the picking center point on the z-axis, L1 is the axial length of the end effector, L2 is the compensation value caused by the installation position of the end effector, α is the angle between the line connecting the projection point of the mature fruit stem position coordinates on the horizontal plane and the origin of the coordinate system in the depth camera coordinate system and the z-axis of the camera coordinate system, and Z1 is the compensation value in the z-axis direction in the robot arm base coordinate system.

[0034] Furthermore, the selection rule for the compensation value in the z-axis direction under the coordinate system of the robotic arm base is as follows:

[0035]

[0036] In the formula, Z0 is the height threshold, and L z To compensate for the height.

[0037] Furthermore, the position coordinates of the shearing location are:

[0038] (P_p x , P_p y , P_p z )

[0039] P_p x =P_m1 x +L3*cos(α)

[0040] P_p y =P_m1 y -L3*sin(α)

[0041] P_p z =P_m1 z -Z2

[0042]

[0043] In the formula, P_p x P_p represents the position information of the shearing point on the x-axis. y To provide the position information of the shearing point on the y-axis, P_p z L3 represents the position information of the cutting position on the z-axis, L2 is the straight-line distance the robotic arm travels from the midpoint of the assisted picking to the cutting position, and Z2 is the compensation value provided when selecting the cutting position. z2 To determine the compensation height at the shearing position.

[0044] Furthermore, the coordinates of the auxiliary retraction midpoint are:

[0045] (P_m2 x ,P_m2 y ,P_m2 z )

[0046] P_m2 x =P_p x -L4*cos(α)

[0047] P_m2 y =P_p y +L4*sin(α)

[0048] P_m2 z =P_p z +Z3

[0049]

[0050] In the formula, P_m2 x To assist in determining the position of the retraction center point on the x-axis, P_m2 y To assist in determining the position of the retraction center point on the y-axis, P_m2 z To provide the position information of the auxiliary retraction center point on the z-axis, L4 is the straight-line distance of the end effector retracting from the shear position to the auxiliary retraction center point, and Z3 is the compensation value provided when selecting the auxiliary retraction center point. z3 To determine the compensation height when the auxiliary retraction midpoint is reached.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] 1. This invention acquires images of fruits to be harvested, performs image analysis and detection on these images, and obtains the position and orientation information of the mature fruit stem in the camera coordinate system. The stem orientation information is used to determine the final harvesting orientation of the end effector. Based on this orientation, the end effector is guided to adjust to a suitable angle, allowing it to harvest the fruit almost perpendicular to the stem. This significantly improves the harvesting success rate, shortens harvesting time, and enhances harvesting efficiency.

[0053] 2. This invention determines the intermediate point for assisted harvesting by using the stem pose information of mature fruit. Based on the fruit height, the fruit harvesting method is divided into horizontal harvesting and oblique harvesting. Different intermediate point selection strategies for assisted harvesting are adopted according to different fruit heights, which effectively improves the success rate of robotic arm path planning and reduces collisions between the robotic arm and surrounding fruit plants during the movement of the robotic arm. Attached Figure Description

[0054] Figure 1 This is a flowchart illustrating the mature fruit harvesting method proposed in this invention.

[0055] Figure 2 This is a flowchart illustrating the coordinate transformation process.

[0056] Figure 3 This is a schematic diagram of the preset release point for mature fruit. Detailed Implementation

[0057] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0058] The following English abbreviations are involved:

[0059] Coordinate transformation: TransForm, TF

[0060] Example 1

[0061] This embodiment provides a method for harvesting mature fruits, such as... Figure 1 As shown, it includes the following steps:

[0062] S1. The robotic arm moves to the designated position to wait for the fruit picking instruction. When the robotic arm receives the picking instruction, it pre-sets the pre-picking position and posture of the end effector of the robotic arm according to the growth characteristics of the fruit; otherwise, the robotic arm continues to wait for the picking instruction at the current position.

[0063] The robotic arm is equipped with an end effector and a depth camera. The principle for setting the pre-harvesting position of the robotic arm is to maximize the number of fruits in the image of the fruit to be harvested obtained by the depth camera at the pre-harvesting position. Specifically, the pre-harvesting posture of the end effector is that the end effector is directly facing the fruit planting row and the angle between the end effector's cutting plane and the horizontal plane is 0°.

[0064] S2. Take pictures of the fruit crop using a depth camera installed on the end effector to obtain images of the fruit to be harvested in the planting environment at the current moment.

[0065] S3. The recognition model in the image recognition and detection system analyzes the image of the fruit to be picked. If a mature fruit is detected in the current image, the recognition model analyzes and determines the growth position and posture of the mature fruit stem in the image; otherwise, the robotic arm moves to the next working position and continues to wait for the picking instruction.

[0066] In this embodiment, the YOLOv5 recognition model is used to identify mature fruits in the image of fruits to be picked, and to obtain the position and posture information of the mature fruit stems. The steps for obtaining the posture information of the mature fruit stems include: identifying mature fruits in the image of fruits to be picked using the YOLO algorithm, obtaining the mature fruit stems through straight line fitting, and using the angle between the mature fruit stems and the y-axis of the camera coordinate system as the posture information of the mature fruit stems.

[0067] S4. Transform the position information of the mature fruit stem in the camera coordinate system to the position information of the mature fruit in the coordinate system of the robot arm base. Based on the position information of the fruit root in the coordinate system of the robot arm base and the structural parameters of the end effector, perform robot arm path planning and determine the auxiliary picking midpoint, cutting position and auxiliary retraction midpoint. The end effector moves from the pre-picking pose to the auxiliary picking midpoint.

[0068] The position information of the mature fruit stem in the depth camera coordinate system is as follows:

[0069] (P x ,P y ,P z )

[0070] In the formula, P x P represents the horizontal position information of the mature fruit stem in the depth camera coordinate system. y P represents the vertical position information of the mature fruit stem in the depth camera coordinate system. z This provides the positional information of the mature fruit stem in the depth direction within the depth camera coordinate system.

[0071] The position information of the mature fruit stem in the coordinate system of the robotic arm base is as follows:

[0072] (P_tx ,P_t y ,P_t z )

[0073] In the formula, P_t x P_t represents the position information of the mature fruit stem on the x-axis in the coordinate system of the robot arm base. y P_t represents the position information of the mature fruit stem on the y-axis in the coordinate system of the robot arm base. z This provides the position information of the mature fruit stem on the z-axis in the coordinate system of the robotic arm base.

[0074] The RTT path planning algorithm is used to plan the path of the robotic arm from the pre-harvesting position to the auxiliary harvesting intermediate point. The TF transformation is then used to transform the coordinates from the depth camera coordinate system to the robotic arm base coordinate system, obtaining the homogeneous transformation matrix from the robotic arm base coordinate system to the depth camera coordinate system. In the base coordinate system, the robotic arm path planning and the determination of the coordinates of the auxiliary intermediate point in the harvesting path are completed. The TF coordinate transformation method can be divided into obtaining the homogeneous transformation matrix between the depth camera coordinate system and the robotic arm end-effector coordinate system, and obtaining the homogeneous transformation matrix between the robotic arm base coordinate system and the robotic arm end-effector coordinate system.

[0075] The specific steps for transforming the position information of the mature fruit stem in the depth camera coordinate system to the position information of the mature fruit stem in the robot arm base coordinate system are as follows: Figure 2 As shown, it includes:

[0076] S1. Obtain the homogeneous transformation matrix between the depth camera coordinate system and the robotic arm end effector coordinate system through the visual hand-eye calibration method;

[0077] S2. Obtain the homogeneous transformation matrix between the end-effector coordinate system and the base coordinate system of the robotic arm using the kinematics method of the robotic arm;

[0078] S3. Obtain the homogeneous transformation matrix between the depth camera coordinate system and the robot arm base coordinate system from the above two homogeneous transformation matrices;

[0079] S4. Based on the homogeneous transformation matrix between the depth camera coordinate system and the robot arm base coordinate system, transform the position information of the mature fruit stem in the depth camera coordinate system to the position information of the mature fruit stem in the robot arm base coordinate system.

[0080] The location of the assisted harvesting midpoint is determined by the position information of the mature fruit and the installation method and size of the end effector. The coordinates of the assisted harvesting midpoint are:

[0081] (P_m1 x ,P_m1 y ,P_m1 z )

[0082] P_m1 x =P_t x -(L1+L2)*cos(α)

[0083] P_m1 y =P_t y +(L1+L2)*sin(α)

[0084] P_m1 z =P_t z +Z1

[0085] In the formula, P_m1 x To assist in determining the position of the picking center point on the x-axis, P_m1 y To assist in determining the position of the picking center point on the y-axis, P_m1 z To assist in determining the position of the picking center point on the z-axis, L1 is the axial length of the end effector, L2 is the compensation value caused by the installation position of the end effector, α is the angle between the line connecting the projection point of the mature fruit stem position coordinates on the horizontal plane and the origin of the coordinate system in the depth camera coordinate system and the z-axis of the camera coordinate system, and Z1 is the compensation value in the z-axis direction in the robot arm base coordinate system.

[0086] The selection rule for the compensation value in the z-axis direction in the coordinate system of the robot arm base is as follows:

[0087]

[0088] In the formula, Z0 is the height threshold, and L z To compensate for height. In this embodiment, the height threshold Z0 is set to 0, and the compensation height L is... z The value is 0.15 meters.

[0089] In the coordinate system of the robotic arm base, when the position information P_t of the mature fruit stem on the z-axis... z When the height threshold Z0 is greater than the height threshold, the compensation value Z1 is 0, meaning the auxiliary intermediate point and the coordinates of the mature fruit stem are on the same horizontal plane. The position information P_t of the mature fruit stem on the z-axis... z When the height is less than the height threshold Z0, the coordinates of the auxiliary picking midpoint and the mature fruit stem are not on the same horizontal plane. Adding a compensation value can reasonably select the coordinates of the auxiliary picking midpoint. Different auxiliary picking midpoint selection strategies are adopted according to different fruit heights, which can effectively improve the success rate of robotic arm path planning.

[0090] S5. At the midpoint of the assisted harvesting, adjust the end effector to a suitable angle based on the fruit stem posture information.

[0091] At the midpoint of the assisted harvesting process, the end effector is adjusted to a suitable posture for shearing the fruit stem. The adjusted angle represents the posture information of the mature fruit stem, which is the angle between the fruit stem (after linear fitting by an image recognition and detection system) and the y-axis of the depth camera coordinate system.

[0092] S6. After moving a certain distance in a straight line from the midpoint of the assisted picking after the posture adjustment, the end effector reaches the cutting position of the end effector. At this cutting position, the end effector completes the cutting and clamping of the mature fruit.

[0093] The coordinates of the shearing position are:

[0094] (P_p x , P_p y , P_p z )

[0095] P_p x =P_m1 x +L3*cos(α)

[0096] P_p y =P_m1 y -L3*sin(α)

[0097] P_p z =P_m1 z -Z2

[0098] In the formula, P_p x P_p represents the position information of the shearing point on the x-axis. y To provide the position information of the shearing point on the y-axis, P_p z L3 represents the position information of the cutting position on the z-axis, and L3 is the straight-line distance of the robotic arm from the midpoint of the assisted picking to the cutting position. The straight-line motion can effectively avoid collisions with the surrounding planting environment of the fruit.

[0099] The selection rules for Z2 are as follows:

[0100]

[0101] Z2 is the compensation value provided when selecting the shearing position, L z2 To determine the compensation height at the shearing position, a compensation value Z2 is added to reasonably select the shearing position coordinates. Different shearing position selection strategies are adopted according to different fruit heights, which effectively improves the success rate of robotic arm path planning.

[0102] In this embodiment, the straight-line segment movement distance L3 is taken as 0.1 meters, and the compensation height L... z2 The value is 0.05 meters.

[0103] S7. After completing the cutting and clamping work, the end effector clamps the fruit and moves back a distance along the original straight path to reach the auxiliary retraction midpoint.

[0104] The coordinates of the auxiliary rollback center point are:

[0105] (P_m2 x ,P_m2 y ,P_m2 z )

[0106] P_m2 x =P_p x -L4*cos(α)

[0107] P_m2 y =P_p y +L4*sin(α)

[0108] P_m2 z =P_p z +Z3

[0109] In the formula, P_m2 x To assist in determining the position of the retraction center point on the x-axis, P_m2 y To assist in determining the position of the retraction center point on the y-axis, P_m2 z To provide information on the position of the auxiliary retraction center point on the z-axis, L4 is the straight-line distance from the end effector retracting from the shear position to the auxiliary retraction center point.

[0110] The selection rules for Z3 are as follows:

[0111]

[0112] Z3 is the compensation value provided when selecting the intermediate point of the auxiliary backoff, L z3 To determine the compensation height when the auxiliary retraction midpoint is reached, the compensation value Z3 is added to reasonably select the coordinates of the auxiliary retraction midpoint. Different auxiliary retraction midpoint selection strategies are adopted according to different fruit heights, which effectively improves the success rate of the robotic arm's path planning.

[0113] In this embodiment, the linear motion distance L4 from the shear position to the auxiliary retraction midpoint is 0.12 meters, and the compensation height L is set when determining the auxiliary retraction midpoint. z3 The value is 0.05 meters.

[0114] S8. The end effector moves from the auxiliary retraction midpoint to the pre-set release pose to release the harvested ripe fruit. After releasing the ripe fruit, the robotic arm returns to the working position to wait for the harvesting command.

[0115] In this embodiment, to prevent mature fruits from piling up in a "mountain" shape within the fruit frame and causing them to fall out, three release positions are provided to ensure that the mature fruits are evenly distributed throughout the fruit frame. A top-view view of the fruit release positions is shown below. Figure 3 As shown.

[0116] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for harvesting mature fruits, characterized in that, Includes the following steps: Based on the growth characteristics of the fruit, the pre-harvesting position of the robotic arm and the pre-harvesting posture of the end effector are preset; Acquire images of fruits to be picked, perform image analysis and detection on the images of fruits to be picked, and obtain the position and orientation information of the mature fruit stems in the depth camera coordinate system; The position information of the mature fruit stem in the depth camera coordinate system is transformed to the position information of the mature fruit stem in the robot arm base coordinate system. Based on the position information of the mature fruit stem in the robot arm base coordinate system and the structural parameters of the end effector, path planning is performed and the auxiliary picking midpoint, cutting position and auxiliary retraction midpoint are determined. The robotic arm is moved from the pre-harvesting position to the midpoint of the auxiliary harvesting along the planned path, and the end effector is adjusted to a suitable angle according to the posture information of the mature fruit stem. After adjustment, the robotic arm is moved in a straight line from the auxiliary picking midpoint to the cutting position, and the mature fruit is cut and clamped by the end effector. Then, it is moved back in a straight line along the original path to the auxiliary return midpoint. The robotic arm moves from the auxiliary retraction midpoint to the pre-set release position to release the harvested ripe fruit, thus completing the harvest; The position information of the mature fruit stem in the depth camera coordinate system is as follows: In the formula, This provides the horizontal position information of the mature fruit stem in the depth camera coordinate system. This provides the vertical position information of the mature fruit stem in the depth camera coordinate system. This provides the positional information of the mature fruit stem along the depth direction in the depth camera coordinate system. The position information of the mature fruit stem in the coordinate system of the robotic arm base is as follows: In the formula, This provides the position information of the mature fruit stem on the x-axis in the coordinate system of the robotic arm base. This provides the position information of the mature fruit stem on the y-axis in the coordinate system of the robotic arm base. This provides the position information of the mature fruit stem on the z-axis in the coordinate system of the robotic arm base. The coordinates of the intermediate point for the auxiliary harvesting are: In the formula, To assist in determining the position of the harvesting center point on the x-axis, To assist in determining the position of the harvesting center point on the y-axis, To assist in determining the position of the harvesting center point on the z-axis, The axial length of the end effector. This is the compensation value caused by the installation position of the end effector. Let be the angle between the line connecting the projection point of the mature fruit stem's position coordinates on the horizontal plane in the depth camera coordinate system and the origin, and the z-axis of the camera coordinate system. This is the compensation value in the z-axis direction in the coordinate system of the robot arm base; The selection rule for the compensation value in the z-axis direction under the coordinate system of the robotic arm base is as follows: In the formula, For height threshold, To compensate for height; The coordinates of the shearing position are: In the formula, This provides the positional information of the shearing location on the x-axis. This provides the position information of the shearing point on the y-axis. This provides the position information of the shearing point on the z-axis. This represents the straight-line distance the robotic arm travels from the midpoint of the assisted harvesting point to the cutting position. This is the compensation value provided when selecting the shearing position. To determine the compensation height at the shearing position.

2. The method for harvesting mature fruits according to claim 1, characterized in that, The principle for setting the pre-harvesting position of the robotic arm is that the depth camera on the robotic arm obtains the image of the fruit to be harvested at the pre-harvesting position, which contains the largest number of fruits.

3. The method for harvesting mature fruits according to claim 1, characterized in that, The pre-harvesting posture of the end effector is specifically that the end effector is facing the fruit planting row and the angle between the shearing plane of the end effector and the horizontal plane is 0°.

4. The method for harvesting mature fruits according to claim 1, characterized in that, The steps for obtaining the posture information of the mature fruit stem include: identifying the mature fruit in the image of the fruit to be picked using the YOLO algorithm, obtaining the mature fruit stem by fitting a straight line, and taking the angle between the mature fruit stem and the y-axis of the camera coordinate system as the posture information of the mature fruit stem. The included angle and the angle at which the end effector adjusts at the midpoint of the assisted picking are matched.

5. The method for harvesting mature fruits according to claim 1, characterized in that, The specific steps for transforming the position information of the mature fruit stem in the depth camera coordinate system to the position information of the mature fruit stem in the robot arm base coordinate system include: The homogeneous transformation matrix between the depth camera coordinate system and the robotic arm end effector coordinate system is obtained through a visual hand-eye calibration method. The homogeneous transformation matrix between the end effector coordinate system and the base coordinate system of the robotic arm is obtained by using the kinematics method of the robotic arm. The homogeneous transformation matrix between the depth camera coordinate system and the robot arm base coordinate system is obtained from the above two homogeneous transformation matrices; The position information of the mature fruit stem in the depth camera coordinate system is transformed to the position information of the mature fruit stem in the robot arm base coordinate system using the homogeneous transformation matrix between the depth camera coordinate system and the robot arm base coordinate system.

6. The method for harvesting mature fruits according to claim 1, characterized in that, The coordinates of the auxiliary rollback midpoint are: In the formula, To assist in reverting to the position information of the center point on the x-axis, To assist in reverting to the position information of the center point on the y-axis, To assist in reverting to the position information of the center point on the z-axis, The straight-line distance from the shear position to the midpoint of the auxiliary retraction of the end effector. This is the compensation value provided when selecting the intermediate point for the auxiliary backoff. To determine the compensation height when the auxiliary retraction midpoint is reached.

Images