A deep learning-based cluster fruit recognition and picking method

Abstract

The application discloses a kind of based on deep learning's stringed fruit recognition picking method, comprising the following steps: using RGB-D camera obtains the color chart and depth chart of stringed fruit to be picked;Based on deep learning algorithm, fruit, fruit peduncle and three kinds of target of main stem in color chart are identified;According to the communication relationship of fruit and fruit peduncle, fruit peduncle and main stem, the object that can be picked is filtered out;The mask of fruit peduncle and main stem identified is handled into fruit peduncle skeleton map and main stem skeleton map respectively using morphological refinement operation, and the picking point is extracted using fruit peduncle skeleton map, and the main stem point is extracted using main stem skeleton map;The spatial three-dimensional coordinates of picking point and main stem point are calculated in combination with depth chart, and the vector pointed to picking point by main stem point is constructed, as the growth direction of predicted stringed fruit relative to main stem;Mechanical arm controls end effector to carry out nondestructive picking with suitable posture according to the growth direction of predicted stringed fruit.

Description

Technical Field

[0001] This invention relates to the field of image recognition and fruit-picking robot technology, specifically to a method for picking fruit clusters based on deep learning. Background Technology

[0002] The large-scale cultivation of bunch-fruited crops places higher demands on harvesting efficiency and operational quality. These crops, such as bunch tomatoes, grapes, and raisins, are characterized by dense bunches, slender stalks, and colors similar to the stems and vines. Under natural growing conditions, the spatial orientation of bunches of fruit is complex and variable, posing a challenge to the adaptive adjustment of the harvesting posture of the end effector of the robotic arm.

[0003] Among crops that produce clusters of fruit, manual harvesting of bunch tomatoes is highly seasonal and labor-intensive. As the planting area of ​​tomatoes expands, labor costs are increasing year by year. Intelligent harvesting of bunch tomatoes by robots instead of manual labor is the future development direction.

[0004] Existing technologies for identifying and harvesting bunch tomatoes face challenges due to the thin stems that blend into the background, increasing the difficulty of identifying and locating the harvesting point. Furthermore, the free growth of the tomato plant's main stem creates an unstructured environment, further complicating the safe and damage-free harvesting process for robotic arms. For example, a fuzzy harvesting method for bunch tomatoes, authorized in CN116034732B, first presets three end effector postures, then fuzzily predicts the direction of the bunch based on two target categories: fruit and stem, guiding the robotic arm to complete the harvesting task. When describing the posture of a bunch of tomatoes, the identification of the main stem can be introduced to more accurately depict the growth direction of the bunch of tomatoes relative to the main stem. As a result, the picking posture of the end effector can be expanded from three fixed postures to several postures that are solved in real time based on the predicted growth direction of the bunch of tomatoes relative to the main stem, enhancing the adaptability and flexibility of the picking robot. Furthermore, the identification of the main stem enriches the environmental perception information of the picking robot, which is more conducive to obstacle avoidance and damage-free picking by the robotic arm. The method of introducing the YOLOv8-seg segmentation algorithm to identify the mask of the fruit stem and refine the extraction of the picking point helps to improve the accuracy of the picking point positioning and increase the success rate of robot picking. Summary of the Invention

[0005] The purpose of this invention is to provide a method for identifying and picking clustered fruits based on deep learning, which combines deep learning and robotics to overcome the shortcomings of existing technologies.

[0006] A deep learning-based method for identifying and harvesting clusters of fruits includes the following steps:

[0007] (1) Use an RGB-D camera as a visual sensor to acquire color and depth images of plants with clusters of fruit;

[0008] (2) Based on deep learning algorithms, three types of targets in the captured color images are identified: fruit, fruit stalk, and main stem.

[0009] (3) Based on the connection between the fruit and the pedicel, and between the pedicel and the main stem, select clusters of fruits that intersect with the pedicel and also with the main stem as harvestable objects.

[0010] (4) The masks of the two categories of fruit stalks and main stems in the identified and screened pluckable objects are processed into fruit stalk skeleton maps and main stem skeleton maps through morphological refinement operation, and the fruit stalk skeleton map is used to extract the picking point and the main stem skeleton map is used to extract the main stem point.

[0011] (5) Combine the depth map to calculate the spatial three-dimensional coordinates of the picking point and the main stem point, and construct a vector from the main stem point to the picking point as the predicted growth direction of the bunch of fruits relative to the main stem.

[0012] (6) The robotic arm controls the end effector to harvest the fruit in a suitable posture based on the predicted growth direction of the cluster of fruits.

[0013] Preferably, in step (2), the deep learning algorithm for recognizing the color image adopts the YOLOv8-seg segmentation algorithm.

[0014] Preferably, in step (3), the connectivity relationship is determined by IoU calculation. The IoU of the detection frames of the fruit and the fruit stalk is calculated, and the result is recorded as IoU. tomato_stem The IoU between the fruit stalk and the main stem is calculated, and the result is recorded as IoU. stem_trunk When IoU tomato_stem >0 and IoU stem_trunk When the value is greater than 0, select fruits and fruit stalks of the same genus located on a main stem of a plant.

[0015] Preferably, in step (4), extracting the picking point and main stem point from the fruit stalk and main stem cover specifically includes:

[0016] S1. The masks for the two categories of fruit stalks and main stems in the identified and screened harvestable objects are processed into fruit stalk skeleton maps and main stem skeleton maps using the Zhang-Suen refinement algorithm;

[0017] S2. Draw a horizontal center line on the fruit stalk skeleton diagram, and take the intersection of the horizontal center line and the refined fruit stalk skeleton as the picking point.

[0018] S3. After extracting the picking point, draw a horizontal line on the color map that passes through the picking point and intersects with the main stem bone after x. The intersection point is taken as the main stem point.

[0019] Preferably, in step S1, the width and height pixel size of the fruit stalk skeleton map are consistent with the width and height pixel size of the detection box of the fruit stalk identified by the deep learning algorithm, and the width and height pixel size of the main stem skeleton map are consistent with the width and height pixel size of the detection box of the main stem identified by the deep learning algorithm.

[0020] Preferably, in step S2, the two-dimensional coordinates P of the picking point on the captured color image are... pick (u pick ,v pick The coordinates (u1, v1) of the intersection point on the fruit stalk skeleton diagram can be read from the picking point, combined with the relative position of the fruit stalk skeleton diagram on the color image (u...). left1 ,v left1 The location information of the fruit stem identified by the deep learning algorithm is the coordinates of the center point of the detection box of the fruit stem relative to the upper left corner of the color image (u). center1 ,v center1 This requires using the pixel dimensions (w1, h1) of the detection bounding box of the fruit stalk to convert it into the coordinates of the top-left corner of the detection bounding box relative to the top-left corner of the color image, i.e., the relative position of the fruit stalk skeleton image on the color image (u). left1 ,v left1 The detailed formula is as follows:

[0021]

[0022] Among them, (u left1 ,v left1 (u) represents the coordinates of the top-left corner of the detection box for the fruit stem identified by the deep learning algorithm relative to the top-left corner of the color image. center1 ,v center1 (w1, h1) represents the coordinates of the center point of the detection box for the fruit stem identified by the deep learning algorithm relative to the top-left corner of the color image, and (w1, h1) represents the width and height of the detection box for the fruit stem identified by the deep learning algorithm in pixels. pick ,v pick (u1, v1) represents the coordinates of the picking point relative to the top left corner of the original color image, and (u1, v1) represents the coordinates of the picking point extracted from the fruit stalk skeleton image relative to the top left corner of the fruit stalk skeleton image itself. These coordinates can be directly read when the horizontal center line intersects with the refined fruit stalk skeleton.

[0023] Preferably, in step S3, the two-dimensional coordinates P of the main stem point on the color image are... trunk (u trunk ,v trunk v in ) trunk With v pick Equal, and u trunk It can be obtained from the following formula:

[0024]

[0025] Among them, u left2 u represents the horizontal coordinates of the top-left corner of the detection box for the main stem identified by the deep learning algorithm relative to the top-left corner of the color image. center2 w1 represents the horizontal coordinate of the center point of the detection box for the main stem identified by the deep learning algorithm relative to the top-left corner of the color image, w2 represents the width (in pixels) of the detection box for the main stem identified by the deep learning algorithm, and u represents the width (in pixels). trunk u1 is the horizontal coordinate of the main stem point relative to the top left corner of the original color image, and u2 is the horizontal coordinate of the main stem point extracted from the main stem skeleton image relative to the top left corner of the main stem skeleton image itself. It can be read directly when the horizontal line intersects with the refined main stem skeleton.

[0026] Preferably, in step (5), the three-dimensional coordinates of the picking point and the main stem point in the camera coordinate system can be obtained by combining the depth value on the captured depth map with the camera intrinsic parameters. Then, by combining the transformation matrix between the camera and the robotic arm calibration, the three-dimensional coordinates of the picking point and the main stem point in the robotic arm coordinate system can be obtained, denoted as P. pick (x pick ,y pick ,z pick ) and P trunk (x trunk ,y trunk ,z trunk Construct a vector pointing from the main stem point to the harvest point. This is used to predict the growth direction of clusters of fruits relative to the main stem.

[0027] Preferably, in step (6), controlling the end effector to perform non-destructive harvesting in a suitable posture means harvesting in a posture where the axis of the end effector is perpendicular to the predicted growth direction of the bunch of fruits relative to the main stem, so as to avoid interference between the end effector and the main stem.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] 1. When describing the posture of a bunch of fruits, this invention introduces the identification of two key targets, the fruit stalk and the main stem, which can more accurately depict the growth direction of the bunch of fruits relative to the main stem.

[0030] 2. This invention calculates the appropriate picking posture of the end effector in real time based on the predicted growth direction of the cluster of fruits relative to the main stem, which enhances the adaptability and flexibility of the picking robot. Furthermore, the identified main stem enriches the environmental perception information of the picking robot, making it easier for the robotic arm to avoid obstacles and pick without damage.

[0031] 3. This invention introduces the YOLOv8-seg segmentation algorithm from deep learning algorithms to identify and segment the mask of the fruit stalk and refine the extraction of the picking point. Compared with the detection box that only identifies the fruit stalk, it is more conducive to improving the positioning accuracy of the picking point and increasing the success rate of robot picking. Attached Figure Description

[0032] Figure 1 This is a flowchart of a deep learning-based method for identifying and picking clustered fruits, as shown in the embodiment.

[0033] Figure 2 This is a schematic diagram illustrating the extraction of the picking point and main stem point in the embodiment;

[0034] Figure 3 This is a schematic diagram illustrating how the end effector in the embodiment harvests fruits based on the predicted growth direction of the cluster of fruits relative to the main stem. Detailed Implementation

[0035] To enable those skilled in the art to fully understand the technical solutions of this invention, the invention will be further described below with reference to embodiments represented by bunch tomatoes and accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, not all embodiments. This invention is also applicable to the identification and harvesting of bunches of fruits such as grapes and raisins. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0036] like Figure 1 As shown, a deep learning-based method for identifying and harvesting clustered fruits includes the following steps:

[0037] (1) Use an RGB-D camera as a visual sensor to acquire color and depth images of the tomato plants;

[0038] (2) Based on deep learning algorithms, three types of targets in the captured color images are identified: fruit, fruit stalk, and main stem.

[0039] (3) Based on the connection between the fruit and the pedicel, and between the pedicel and the main stem, select cluster tomatoes with overlapping fruits and pedicels and overlapping pedicels and main stems as harvestable targets.

[0040] (4) The masks of the two categories of fruit stalks and main stems in the identified and screened pluckable objects are processed into fruit stalk skeleton maps and main stem skeleton maps through morphological refinement operation, and the fruit stalk skeleton map is used to extract the picking point and the main stem skeleton map is used to extract the main stem point.

[0041] (5) Combine the depth map to calculate the spatial three-dimensional coordinates of the picking point and the main stem point, and construct a vector from the main stem point to the picking point as the predicted growth direction of the bunch tomatoes relative to the main stem.

[0042] (6) The robotic arm controls the end effector to harvest the tomatoes in a suitable posture based on the predicted growth direction of the bunch. The end effector in this embodiment can refer to the harvesting device disclosed in CN111937580B.

[0043] In some embodiments, a deep learning-based method for identifying and picking tomatoes follows the following steps:

[0044] Step 1: Use an RGB-D camera as a visual sensor to acquire color and depth images of the tomato plants.

[0045] Step 2: Identify three types of targets in the captured color image: fruit, stem, and main stem, using a deep learning algorithm. It should be noted that the deep learning algorithm needs to be able to segment the target mask; in some embodiments, the YOLOv8-seg segmentation algorithm is used for identification. The YOLOv8-seg segmentation algorithm is trained using a dataset containing three categories of tomatoes: fruit, stem, and main stem.

[0046] Step 3: The three target categories identified by the deep learning algorithm—fruit, fruit stalk, and main stem—are independent instances, and the connection relationships between these instances have not yet been established. In some embodiments, IoU calculation is used to determine the connectivity relationships of each category. The IoU of the detection boxes for the fruit and fruit stalk is calculated, and the result is recorded as IoU. tomato_stem The IoU between the fruit stalk and the main stem is calculated, and the result is recorded as IoU. stem_trunk When IoU tomato_stem >0 and IoU stem_trunk When the value is greater than 0, select fruits and fruit stalks of the same genus located on a main stem of a plant.

[0047] See Figure 2 To select a group of three targets—fruits, stalks, and main stem—that belong to the same cluster on a single main stem, only then can the robot consider them as harvestable objects for further refinement and extraction of harvesting and main stem points. If a fruit is identified but no stalk intersects with it, it means that the stalks belonging to the same cluster may be obscured and cannot be harvested. If a stalk is identified but no fruit intersects with it, it means that the fruit belonging to the same cluster may be immature or obscured and cannot be harvested. If a stalk is identified but no main stem intersects with it, the main stem point cannot be extracted, and a vector from the main stem point to the harvesting point cannot be constructed to describe the growth direction of the tomato cluster relative to the main stem. This is not conducive to the end effector's obstacle avoidance of the main stem, and harvesting cannot be performed from the perspective of safe and damage-free harvesting.

[0048] Step 4: Extract the picking point and main stem point from the fruit stalk and main stem mask of the identified and screened harvestable objects. Specific details are as follows:

[0049] S1. The masks for the two categories of fruit stalks and main stems are processed into fruit stalk skeleton maps and main stem skeleton maps using the Zhang-Suen thinning algorithm. The width and height pixel size of the fruit stalk skeleton map are consistent with the width and height pixel size of the detection box of the fruit stalk identified by the deep learning algorithm, and the width and height pixel size of the main stem skeleton map are consistent with the width and height pixel size of the detection box of the main stem identified by the deep learning algorithm.

[0050] S2. In some embodiments, a horizontal center line is drawn on the fruit stalk skeleton diagram, and the intersection of the horizontal center line and the refined fruit stalk skeleton is taken as the picking point. The two-dimensional coordinates P of the picking point on the captured color image are... pick (u pick ,v pick The coordinates (u1, v1) of the intersection point on the fruit stalk skeleton diagram can be read from the picking point, combined with the relative position of the fruit stalk skeleton diagram on the color image (u...). left1 ,v left1 The location information of the fruit stem identified by the deep learning algorithm is the coordinates of the center point of the detection box of the fruit stem relative to the upper left corner of the color image (u). center1 ,v center1 This requires using the pixel dimensions (w1, h1) of the detection bounding box of the fruit stalk to convert it into the coordinates of the top-left corner of the detection bounding box relative to the top-left corner of the color image, i.e., the relative position of the fruit stalk skeleton image on the color image (u). left1 ,v left1 The detailed formula is as follows:

[0051]

[0052] Among them, (u left1 ,v left1 (u) represents the coordinates of the top-left corner of the detection box for the fruit stem identified by the deep learning algorithm relative to the top-left corner of the color image. center1 ,v center1 (w1, h1) represents the coordinates of the center point of the detection box for the fruit stem identified by the deep learning algorithm relative to the top-left corner of the color image, and (w1, h1) represents the width and height of the detection box for the fruit stem identified by the deep learning algorithm in pixels. pick ,v pick (u1, v1) represents the coordinates of the picking point relative to the top left corner of the original color image, and (u1, v1) represents the coordinates of the picking point extracted from the fruit stalk skeleton image relative to the top left corner of the fruit stalk skeleton image itself. These coordinates can be read directly when the horizontal center line intersects with the refined fruit stalk skeleton.

[0053] S3. After extracting the picking point, draw a horizontal line on the color map passing through the picking point and intersecting it with the main stem skeleton after x. The intersection point is taken as the main stem point, and the two-dimensional coordinates of the main stem point on the color map are P. trunk (u trunk ,v trunk v in ) trunk With v pick Equal, and u trunk It can be obtained from the following formula:

[0054]

[0055] Among them, u left2 u represents the horizontal coordinates of the top-left corner of the detection box for the main stem identified by the deep learning algorithm relative to the top-left corner of the color image. center2 w1 represents the horizontal coordinate of the center point of the detection box for the main stem identified by the deep learning algorithm relative to the top-left corner of the color image, w2 represents the width (in pixels) of the detection box for the main stem identified by the deep learning algorithm, and u represents the width (in pixels). trunk u1 is the horizontal coordinate of the main stem point relative to the top left corner of the original color image, and u2 is the horizontal coordinate of the main stem point extracted from the main stem skeleton image relative to the top left corner of the main stem skeleton image itself. It can be read directly when the horizontal line intersects with the refined main stem skeleton.

[0056] Step 5: Combining the depth values ​​on the captured depth map with the camera's intrinsic parameters, the three-dimensional coordinates of the picking point and the main stem point in the camera coordinate system can be obtained. Then, combining this with the transformation matrix between the camera and the robotic arm calibration, the three-dimensional coordinates of the picking point and the main stem point in the robotic arm coordinate system can be obtained, denoted as P. pick (x pick ,y pick ,z pick ) and P trunk (x trunk ,y trunk ,z trunk Construct a vector pointing from the main stem point to the harvest point. This is used to predict the growth direction of the cluster tomatoes relative to the main stem.

[0057] Step 6: Control the robotic arm to harvest the tomatoes in an orientation where the end effector axis is perpendicular to the predicted growth direction of the bunch of tomatoes relative to the main stem. This helps to avoid interference between the end effector and the main stem.

[0058] In summary, this invention uses the YOLOv8-seg deep learning algorithm to locate the picking point of a bunch of tomatoes. The proposed method of extracting picking points by screening masks and combining them with refinement operations is more accurate and reliable than the traditional clustering method. This invention constructs a vector from the main stem point to the picking point to describe the growth direction of the bunch of tomatoes relative to the main stem, and then calculates the optimal picking posture of the end effector in real time, which enhances the adaptability and flexibility of the picking robot and makes it easier for the robotic arm to avoid obstacles and pick without damage.

[0059] In some embodiments of the present invention, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the methods described in the embodiments above.

[0060] In some embodiments of the present invention, a computer-readable storage medium is also provided, on which a computer program / instructions are stored, characterized in that the computer program / instructions are executed by a processor to implement the steps of the method described in the foregoing embodiments.

[0061] The specific values ​​of the parameters in the foregoing embodiments are merely specific examples and do not constitute a limitation on the scope of protection.

[0062] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for identifying and harvesting clusters of fruits based on deep learning, characterized in that, Includes the following steps: (1) Use an RGB-D camera as a visual sensor to acquire color and depth images of plants with clusters of fruit; (2) Based on deep learning algorithms, three types of targets in the captured color images are identified: fruit, fruit stalk, and main stem. (3) Based on the connection between the fruit and the pedicel, and between the pedicel and the main stem, select clusters of fruits that intersect with the pedicel and also with the main stem as harvestable objects. (4) The masks of the two categories of fruit stalks and main stems in the identified and screened pluckable objects are processed into fruit stalk skeleton maps and main stem skeleton maps through morphological refinement operation, and the fruit stalk skeleton map is used to extract the picking point and the main stem skeleton map is used to extract the main stem point. The specific methods for extracting the picking point and main stem point from the fruit stalk and main stem cover include: S1. The masks for the two categories of fruit stalks and main stems in the identified and screened harvestable objects are processed into fruit stalk skeleton maps and main stem skeleton maps using the Zhang-Suen refinement algorithm; S2. Draw a horizontal center line on the fruit stalk skeleton diagram, and take the intersection of the horizontal center line and the refined fruit stalk skeleton as the picking point. S3. After extracting the picking point, draw a horizontal line on the color map that passes through the picking point and intersects with the refined main stem skeleton. The intersection point is taken as the main stem point. (5) Calculate the spatial three-dimensional coordinates of the picking point and the main stem point using the depth map, and construct a vector from the main stem point to the picking point as the predicted growth direction of the cluster of fruits relative to the main stem; combine the depth values ​​on the captured depth map with the camera intrinsic parameters to obtain the three-dimensional coordinates of the picking point and the main stem point in the camera coordinate system; then, combine the transformation matrix between the camera and the robotic arm calibration to obtain the three-dimensional coordinates of the picking point and the main stem point in the robotic arm coordinate system, denoted as follows: and Construct a vector pointing from the main stem point to the picking point. As a prediction of the growth direction of clusters of fruits relative to the main stem; (6) The robotic arm controls the end effector to harvest the fruit without damage based on the predicted growth direction of the bunches of fruit.

2. The method for identifying and harvesting clustered fruits based on deep learning according to claim 1, characterized in that, In step (2), the deep learning algorithm used to identify the color image is the YOLOv8-seg segmentation algorithm.

3. The method for identifying and harvesting clustered fruits based on deep learning according to claim 1, characterized in that, In step (3), the connectivity relationship is determined by intersection-over-union (IoU) calculation. The IoU of the detection frames of the fruit and the fruit stalk is calculated, and the result is recorded as follows. The IoU between the fruit stalk and the main stem is calculated, and the result is recorded as follows. ,when and At that time, select fruits and fruit stalks of the same genus located on the main stem of a plant.

4. The method for identifying and harvesting clustered fruits based on deep learning according to claim 1, characterized in that, In step S1, the width and height pixel size of the fruit stalk skeleton map are consistent with the width and height pixel size of the detection box of the fruit stalk identified by the deep learning algorithm, and the width and height pixel size of the main stem skeleton map are consistent with the width and height pixel size of the detection box of the main stem identified by the deep learning algorithm.

5. The method for identifying and harvesting clustered fruits based on deep learning according to claim 1, characterized in that, In step S2, the two-dimensional coordinates of the picking point on the captured color image are determined. The coordinates of the intersection point can be read from the picking point on the fruit stalk skeleton diagram. Combine the relative positions of the fruit stalk skeleton diagram on the color image The location information recorded by the fruit stem identified by the deep learning algorithm is the coordinates of the center point of the fruit stem's detection box relative to the top left corner of the color image. It is necessary to use the width and height pixel size of the detection box of the fruit stem. This is converted to the coordinates of the top-left corner of the detection box of the fruit stalk relative to the top-left corner of the color image, i.e., the relative position of the fruit stalk skeleton map on the color image. The detailed formula is as follows: in, The coordinates of the top-left corner of the detection box for the fruit stem identified by the deep learning algorithm relative to the top-left corner of the color image. These are the coordinates of the center point of the detection box for the fruit stem identified by the deep learning algorithm relative to the top-left corner of the color image. The width and height in pixels are the detection bounding boxes for the fruit stems identified by the deep learning algorithm. These are the coordinates of the picking point relative to the top left corner of the original color map. The coordinates of the picking point extracted from the fruit stalk skeleton relative to the upper left corner of the fruit stalk skeleton itself can be read directly when the horizontal center line intersects with the refined fruit stalk skeleton.

6. The method for identifying and harvesting clustered fruits based on deep learning according to claim 1, characterized in that, In step S3, the two-dimensional coordinates of the main stem point on the color map are... In and Equal, and It can be obtained from the following formula: in, This represents the horizontal coordinates of the top-left corner of the detection box for the main stem identified by the deep learning algorithm, relative to the top-left corner of the color image. This represents the lateral coordinates of the center point of the detection box for the main stem identified by the deep learning algorithm relative to the top-left corner of the color image. This refers to the width (in pixels) of the detection bounding box for the main stem identified by the deep learning algorithm. The horizontal coordinates of the main stem point relative to the top left corner of the original color image. The horizontal coordinates of the main stem point extracted from the main stem skeleton relative to the top left corner of the main stem skeleton itself can be read directly when the horizontal line intersects with the refined main stem skeleton.

7. The method for identifying and harvesting clustered fruits based on deep learning according to claim 1, characterized in that, In step (6), controlling the end effector to perform non-destructive harvesting in a suitable posture means harvesting in a posture where the axis of the end effector is perpendicular to the predicted growth direction of the cluster of fruits relative to the main stem.

8. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method according to any one of claims 1-7.

Images