Honeysuckle recognition and optimal picking point positioning method based on improved Yolov8 target detection

Abstract

The invention discloses a honeysuckle identification and optimal picking point positioning method based on improved Yov8 target detection, and the method comprises the steps: collecting the image data of a to-be-picked part containing honeysuckle, inputting the image data into an improved Yov8 target detection model, carrying out the feature extraction and multi-scale feature fusion through the improved Yov8 target detection model, obtaining a detection target, and carrying out the detection of the detection target. The detection targets are divided into unsheltered honeysuckle buds and sheltered honeysuckle buds; for the situation that the detected target outputs unshielded honeysuckle buds, central point extraction is conducted on the detection frame, and the extracted central point serves as a honeysuckle bud picking point; the three-dimensional coordinates of the honeysuckle bud picking points are obtained by combining the depth information of the camera; for the fact that a detected target outputs a sheltered honeysuckle bud, depth image alignment, point cloud data preprocessing, point cloud segmentation and 3D point cloud reconstruction need to be carried out to obtain an actual picking point. The method is more accurate and efficient in honeysuckle bud recognition and optimal picking point positioning.

Description

Technical Field

[0001] The present invention relates to the field of intelligent picking technology, and in particular to a method for honeysuckle identification and optimal picking point positioning based on improved Yolov8 target detection. Background Art

[0002] With the development of modern agricultural technology, more and more intelligent picking technologies have been proposed. However, due to the complex natural growth environment of crops, the complexity of the growth environment will inevitably lead to the obstruction of some parts to be picked, making it impossible to accurately locate the picking points.

[0003] Take honeysuckle as an example. Honeysuckle, also known as honeysuckle, is a common Chinese medicinal material. Its buds and flowers are used as medicine. It has many medicinal effects such as clearing heat and detoxification, anti-inflammatory and antibacterial. The buds of honeysuckle are round or oval, curved on the outside, and relatively smooth on the edges. The buds are green at the beginning, and turn light yellow or white when mature. The buds usually grow upward or tilted in clusters and are located at the top of new branches. The leaves of honeysuckle are oval or oval. The growth direction of the leaves of mature honeysuckle plants is diverse, some are upward, and some are horizontally spread. Under natural growth conditions, part of the buds will be blocked by the leaves, forming a local shadow area, affecting the picking of the buds. According to the growth characteristics of honeysuckle, honeysuckle is usually picked before the buds open. Since the buds are usually gathered in clusters, most of them are picked individually to ensure the quality of picking and subsequent medicinal value. However, the biggest problem currently faced by intelligent honeysuckle picking is that the complex growing environment causes some parts to be picked to be blocked, resulting in the robotic arm operating to the wrong position and picking incorrectly, which affects both the accuracy and efficiency of picking.

[0004] Although the prior art has also proposed solutions to the occlusion problem in intelligent picking, such as a method for identifying young tea shoots and locating picking points disclosed in CN116958823A, which obtains depth information of the picking part through point cloud completion and then performs three-dimensional picking point positioning, this method has large storage and calculation volume, easy loss of geometric structure, and complex post-processing method. Summary of the invention

[0005] In order to address the deficiencies in the prior art, the present application proposes a method for identifying honeysuckle and locating the optimal picking point based on improved Yolov8 target detection; the improved Yolov8 target detection model can be used to accurately and quickly determine whether there is an occlusion in the honeysuckle buds; and when there is occlusion, point cloud segmentation is performed first, and then point cloud reconstruction is performed, thereby effectively determining the actual picking point location, and the information on these locations is accurate, reliable and has practical application value.

[0006] The technical solution adopted by the present invention is as follows:

[0007] A method for honeysuckle identification and optimal picking point positioning based on improved Yolov8 target detection includes the following steps:

[0008] Step 1: Collect image data containing honeysuckle at the place to be picked, input the image data into the improved Yolov8 target detection model, and perform feature extraction and multi-scale feature fusion on the improved Yolov8 target detection model to obtain detection targets. The detection targets are divided into unobstructed honeysuckle buds and obstructed honeysuckle buds;

[0009] Step 2: The detection target is output as an unobstructed honeysuckle bud, and the center point of the detection frame is extracted. The extracted center point is used as the picking point of the honeysuckle bud; the three-dimensional coordinates of the picking point of the honeysuckle bud are obtained in combination with the depth information of the camera;

[0010] Step 3: The output of the detection target is the occluded honeysuckle bud, which requires deep image alignment, point cloud data preprocessing, point cloud segmentation and 3D point cloud reconstruction to obtain the actual picking point.

[0011] Further, the improved Yolov8 target detection model includes a backbone network, a neck network and a detection head; wherein the backbone network includes a first Conv module, a first C2f module, a second Conv module, a second C2f module, a MultiSEAM attention mechanism module, a third Conv module, a third C2f module, a SEAM attention mechanism module, a fourth Conv module, a fourth C2f module, and an SPPF module connected in sequence; the MultiSEAM attention mechanism module outputs a first feature layer, the SEAM attention mechanism module outputs a second feature layer, and the SPPF module outputs a third feature layer;

[0012] The neck network includes a first upsampling module, a first Concat module, a fifth C2f module, a second upsampling module, a second Concat module, a sixth C2f module, a fifth Conv module, a third Concat module, a seventh C2f module, a sixth Conv module, a fourth Concat module, and an eighth C2f module connected in sequence; wherein the inputs of the first upsampling module and the fourth Concat module are features output by the SPPF module, the input of the first Concat module is features output by the SEAM attention mechanism module, and the input of the second Concat module is features output by the MultiSEAM attention mechanism module;

[0013] The detection head receives the features output by the sixth C2f module, the seventh C2f module, and the eighth C2f module respectively, and generates the final detection frame and classification result.

[0014] Furthermore, the honeysuckle images were used to train the improved Yolov8 target detection model. The training process was as follows:

[0015] Step 1.1: Collect images of honeysuckle buds and pre-process the collected images to obtain training data;

[0016] Step 1.2: Use the training data prepared in step 1.1 to conduct preliminary training on the improved Yolov8 target detection model, test the trained weights, and test the recognition under different lighting conditions and angles; perform additional incremental training on the unrecognized image data, crop and record the misrecognized image data, and put the cropped positive and negative samples into the classification network for training to complete "recognition + reclassification".

[0017] Furthermore, the image preprocessing in step 1.1 includes: image enhancement, image format conversion, segmentation and cropping.

[0018] Furthermore, the process of determining the “actual picking point” in step 3 is as follows:

[0019] Step 3.1: When using an RGB-D camera to capture images, ensure that the depth information is aligned with the RGB image and converted into 3D point cloud data;

[0020] Step 3.2: Smooth the generated 3D point cloud data to remove noise and isolated points in the point cloud data;

[0021] Step 3.3: Use PointCNN to segment the occluded honeysuckle bud point cloud data in the 3D point cloud data;

[0022] Step 3.4: After the segmentation is completed, the ProxyFormer network structure is used to reconstruct the point cloud 3D. The picking point is determined based on the point cloud reconstruction result and the geometric center corresponding to the unobstructed part of the honeysuckle bud.

[0023] Furthermore, in step 3.1, the depth image is calculated into point cloud data through coordinate transformation, and the image coordinates are transformed into the world coordinate system according to the camera intrinsic parameters, i.e., the constraints of the coordinate transformation.

[0024] Furthermore, in step 3.2, the generated 3D point cloud data is smoothed by a Gaussian filter.

[0025] Furthermore, in step 3.3, the process of using PointCNN to segment the obscured honeysuckle bud point cloud data in the 3D point cloud data is as follows:

[0026] Step 3.3.1: Construct a 3D point cloud dataset of honeysuckle buds and use the 3D point cloud dataset to train the PointCNN model;

[0027] Step 3.3.2: Apply the trained PointCNN model to the newly acquired honeysuckle bud point cloud data segmentation and output the 3D point cloud segmentation result.

[0028] Further, the process of determining the picking point in step 3.4 is as follows:

[0029] Step 3.4.1: Select proxy points from the segmented point cloud data based on the geometric features of the point cloud data;

[0030] Step 3.4.2: Input the proxy point into the multi-head self-attention mechanism, merge the multiple outputs of the multi-head self-attention mechanism to form a global feature representation;

[0031] Step 3.4.3: Use the occlusion-aware module in ProxyFormer to analyze the proxy points and global features to infer the geometric structure of the occluded area;

[0032] Step 3.4.4: Combine the geometric structure with the original point cloud to generate a complete 3D reconstruction model of the honeysuckle bud;

[0033] Step 3.4.5: Use Gaussian filtering technology to smooth the 3D reconstructed model;

[0034] Step 3.4.6: The 3D reconstruction model is converted into a 3D point cloud of the honeysuckle bud through the ProxyFormer model, and the geometric center point of the 3D point cloud of the honeysuckle bud is used as the theoretical picking point (Xc, Y C , Z c );

[0035] Step 3.4.7: Considering the actual conditions, the picking point offset (ΔX, ΔY, ΔZ) is introduced to obtain the actual picking point, which is expressed as (X p , Y p , Z P ).

[0036] Furthermore, the basis for selecting the proxy point is: by extracting the normal vector or density of the point cloud data, a point with a large normal vector change or a high density is selected as the proxy point.

[0037] Beneficial effects of the present invention:

[0038] (1) The SEAM and MultiSEAM modules are used to optimize the Yolov8 target detection network. The optimized Yolov8 target detection network has increased feature extraction capabilities, improved the model's detection accuracy for occluders, and improved the model's computational efficiency and generalization ability.

[0039] (2) For images with occlusion, the PointCNN point cloud segmentation network is first used to perform point cloud segmentation of honeysuckle buds, and then the Proxformer point cloud reconstruction network structure is used to perform point cloud reconstruction of honeysuckle buds. The advantages of this method are the permutation invariance and clear segmentation edges of PointCNN, which effectively identify the fine structure of honeysuckle buds; and Proxformer can sense the position of missing points and predict the point cloud information of the occluded part through its innovative missing part sensitive transformer and proxy alignment mechanism, so as to achieve a more complete reconstruction. This processing method not only improves the quality of point cloud segmentation and reconstruction, but also enhances the model's ability to handle occlusion and incomplete data, making the identification of honeysuckle buds and the positioning of the optimal picking point more accurate and efficient. Compared with the identification and picking point positioning methods disclosed in the prior art, the processing method of the present invention can handle occlusion problems more efficiently and accurately, improve segmentation accuracy, retain more local features, and improve point cloud quality. At the same time, it can generate a three-dimensional point cloud with more complete structure and details through cross-modal information fusion, and show better performance in honeysuckle bud identification and actual picking point positioning.

[0040] (3) Compared with the prior art, the present invention has the advantage that it can accurately identify honeysuckle according to the specific morphology of a single flower bud under occlusion and obtain the optimal picking point. The improved Yolov8 target detection algorithm model of the present invention has a certain generalization and can be applied to the accurate identification of similar drugs under occlusion and the positioning of the optimal picking point. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is the overall operation logic diagram of the method of the present invention.

[0042] Figure 2 It is a network structure diagram of the improved YOLOv8 algorithm described in the method of the present invention.

[0043] Figure 3 It is the improved YOLOv8 method network structure described in the present invention, wherein a is the SEAM network structure diagram and b is the CSMM module network structure diagram.

[0044] Figure 4 It is the PointCNN point cloud segmentation network structure used in the present invention, where a is a simple PointCNN with two X-Conv layers and b is a classification task.

[0045] Figure 5 The hierarchical convolution structure introduced in the PointCNN point cloud segmentation network structure used in the present invention.

[0046] Figure 6 The Proxformer point cloud reconstruction network structure used in the present invention.

[0047] Figure 7 The point cloud attention module in the Proxformer point cloud reconstruction network structure used in the invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0049] Combined with Figure 1 The present invention proposes a method for honeysuckle identification and optimal picking point positioning based on improved Yolov8 target detection, comprising the following steps:

[0050] Step 1: Collect image data containing honeysuckle at the place to be picked, input the image data into the improved Yolov8 target detection model, and perform feature extraction and multi-scale feature fusion on the improved Yolov8 target detection model to obtain detection targets. The detection targets are divided into unobstructed honeysuckle buds and obstructed honeysuckle buds;

[0051] The network structure of the improved Yolov8 target detection model in step 1 above is as follows Figure 2 As shown, it includes a backbone network, a neck network and a detection head.

[0052] The backbone network includes a first Conv module, a first C2f module, a second Conv module, a second C2f module, a MultiSEAM attention mechanism module, a third Conv module, a third C2f module, a SEAM attention mechanism module, a fourth Conv module, a fourth C2f module, and an SPPF module connected in sequence; the MultiSEAM attention mechanism module outputs the first feature layer, the SEAM attention mechanism module outputs the second feature layer, and the SPPF module outputs the third feature layer. The present invention introduces SEAM and MultiSEAM attention mechanisms in the backbone network, and the MultiSEAM attention mechanism can process multi-level feature maps to capture target information of different scales. The SEAM mechanism helps the model pay more attention to the unobstructed honeysuckle buds by emphasizing important areas in the feature map, learning adaptive weights, enhancing target-related areas in the feature map, and suppressing background noise. The present invention allows the model to apply attention mechanisms on different feature layers to improve the detection ability of small or obscured targets.

[0053] The neck network uses a feature pyramid to fuse feature maps of different scales. The specific network structure includes: a first upsampling module, a first Concat module, a fifth C2f module, a second upsampling module, a second Concat module, a sixth C2f module, a fifth Conv module, a third Concat module, a seventh C2f module, a sixth Conv module, a fourth Concat module, and an eighth C2f module connected in sequence; wherein, the inputs of the first upsampling module and the fourth Concat module are features output by the SPPF module, the input of the first Concat module is features output by the SEAM attention mechanism module, and the input of the second Concat module is features output by the MultiSEAM attention mechanism module.

[0054] The detection head receives the features output by the sixth, seventh, and eighth C2f modules respectively, and is responsible for generating the final detection box and classification results, and integrating the RepulsionLoss into the loss calculation module as part of the bounding box regression loss. When calculating the loss, the detection head not only considers the accuracy of the bounding box, but also increases the repulsion between occluded objects, encouraging the model to separate the targets, thereby better locating the occluded honeysuckle buds.

[0055] The coordinate information of the detection box of each object to be picked is (center_x, center_y, width, height), where center_x and center_y are the midpoint coordinates of the detection box, and width and height are the width and height of the detection box.

[0056] After completing the construction of the above improved Yolov8 target detection model framework, it is necessary to use honeysuckle images to train the improved Yolov8 target detection model. The specific training process is as follows:

[0057] Step 1.1: Use an RGB-D camera to capture images of honeysuckle buds, ensure that there is sufficient training data under different lighting conditions, angles, and occlusions, and preprocess the captured images, such as image enhancement, image format conversion, segmentation, and cropping.

[0058] Step 1.2: Use the training data prepared in step 1.1 to conduct preliminary training on the improved Yolov8 target detection model, test the trained weights, test the recognition under different lighting conditions and angles, supplement the unrecognized image data with incremental training, and crop the misrecognized image data and record them. Put the cropped positive and negative samples into the classification network for training to complete the overall task of "recognition + reclassification". Finally, conduct multiple experiments to continuously supplement data and optimize model weights.

[0059] Step 2: The detection target output is an unobstructed honeysuckle bud, and the center point of the detection frame is extracted. The extracted center point is used as the picking point of the honeysuckle bud; the three-dimensional coordinates of the picking point of the honeysuckle bud are obtained in combination with the depth information of the camera.

[0060] Step 3: If the detection target output is an obstructed honeysuckle bud, it is necessary to perform deep image alignment, point cloud data preprocessing, point cloud segmentation and 3D point cloud reconstruction. Through analysis, multiple possible picking points are obtained, and the size of the obstruction, the grasping path, and the accessibility of the robot arm are considered to finally select the optimal "actual picking point". The specific process is as follows:

[0061] Step 3.1: When using an RGB-D camera to capture images, ensure that the depth information is aligned with the RGB image and converted into 3D point cloud data.

[0062] For the occluded honeysuckle buds, it is necessary to fuse depth images from multiple perspectives to obtain more complete 3D information.

[0063] The depth image can be calculated as point cloud data after coordinate transformation, and the image coordinates can be converted into the world coordinate system based on the camera intrinsic parameters, that is, the constraints of the coordinate transformation.

[0064] The point cloud conversion formula is:

[0065]

[0066] Among them, x, y, z are the point cloud coordinate system, x′, y′ are the image coordinate system, D is the depth value, and f x is the focal length of the camera internal reference in the horizontal direction, f y is the focal length of the camera intrinsic parameter in the vertical direction.

[0067] Step 3.2: Smoothing the generated 3D point cloud data with a Gaussian filter can remove noise and isolated points in the point cloud data. The overall shape and features of the point cloud can be retained during smoothing, avoiding information loss caused by over-smoothing, and improving the point cloud quality and the robustness of the model.

[0068] The formula for the Gaussian filter is:

[0069]

[0070] When processing point clouds, they can be expanded into three dimensions:

[0071]

[0072] Where G(x,y) is the value of the Gaussian filter, (x,y) is the coordinate from the center point, and σ is the standard deviation, which controls the width of the filter.

[0073] Step 3.3: In a complex growth environment, the 3D point cloud data of honeysuckle buds are often blocked, making the foreground segmentation task more difficult; PointCNN is highly efficient, can learn local features efficiently, and can quickly integrate global features, so it can flexibly handle different point cloud processing tasks, such as classification and segmentation; PointCNN is used to segment the blocked point cloud data of honeysuckle buds in the 3D point cloud data. Figure 4 , 5 The process is as follows:

[0074] Step 3.3.1: Construct a 3D point cloud dataset of honeysuckle buds, use the 3D point cloud dataset to train the PointCNN model, set an appropriate learning rate and batch size to prevent overfitting during training. Ensure that the model can effectively identify occluded and unoccluded bud parts, and use evaluation indicators (such as accuracy, recall, and F1 score) to judge the segmentation effect of the model on occluded and unoccluded honeysuckle buds.

[0075] Step 3.3.2: Apply the trained PointCNN model to the newly acquired honeysuckle bud point cloud data segmentation and output the 3D point cloud segmentation result.

[0076] Step 3.4: After the segmentation is completed, the ProxyFormer network structure is used to reconstruct the point cloud 3D, such as Figure 6 , 7 The network infers the missing geometric structure by extracting proxy points and global features, especially processing the information of occluded parts.

[0077] Finally, according to the reconstruction results of the above point cloud and the geometric center corresponding to the unobstructed part of the honeysuckle bud, multiple possible picking points are obtained, and the above picking points are defined as “theoretical picking points” and “actual picking points”.

[0078] Step 3.4.1: Select proxy points from the segmented point cloud data. These points can represent important geometric features or regions and can effectively capture the overall shape and structure information. The basis for selecting proxy points is: by extracting the geometric features of the point cloud data (such as normal vector, density, etc.), points with large normal vector changes or high density can be selected as proxy points.

[0079] Step 3.4.2: Input the proxy point into the multi-head self-attention mechanism, merge the multiple outputs of the multi-head self-attention mechanism to form a global feature representation, which integrates various information of the point cloud and can fully consider the geometric structure under occlusion.

[0080] Step 3.4.3: Use the Occlusion-Aware Module in ProxyFormer to analyze the proxy points and global features to infer the geometric structure of the occluded area. The Occlusion-Aware Module uses the feature interaction mechanism to enhance the understanding of the missing parts and generate more accurate geometric structures.

[0081] Step 3.4.4: Combine the geometric structure with the original point cloud to generate a complete 3D reconstruction model of the honeysuckle bud. During the reconstruction process, use techniques such as interpolation or generative adversarial networks (GAN) to optimize the reconstruction results and improve details and authenticity.

[0082] Step 3.4.5: Use Gaussian filtering technology to smooth the 3D reconstructed model, remove noise and incoherent parts, and ensure the quality of the 3D reconstructed model.

[0083] Step 3.4.6: The 3D reconstruction model is converted into a 3D point cloud of the honeysuckle bud through the ProxyFormer model, and the geometric center point of the 3D point cloud of the honeysuckle bud is used as the theoretical picking point (Xc, Y C , Z c );

[0084] Step 3.4.7: Considering the actual conditions (such as occlusion, inaccessible areas, etc.), the actual picking point is obtained based on the theoretical picking point, expressed as (X p , Y p , Z P );

[0085] The specific calculation is as follows:

[0086] X p =Xc+ΔX

[0087] Y p =Y C +ΔY

[0088] Z P =Z c +ΔZ

[0089] Among them, (Xc, Y C , Z c ) is the coordinate of the theoretical picking point, and the picking point offset is: (ΔX, ΔY, ΔZ).

[0090] The above embodiments are only used to illustrate the design ideas and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.

Claims

1. A method for honeysuckle identification and optimal picking point positioning based on improved Yolov8 target detection, characterized in that: The steps include: Step 1: Collect image data containing honeysuckle at the place to be picked, input the image data into the improved Yolov8 target detection model, and perform feature extraction and multi-scale feature fusion on the improved Yolov8 target detection model to obtain detection targets. The detection targets are divided into unobstructed honeysuckle buds and obstructed honeysuckle buds; Step 2: The detection target is output as an unobstructed honeysuckle bud, and the center point of the detection frame is extracted. The extracted center point is used as the picking point of the honeysuckle bud; the three-dimensional coordinates of the picking point of the honeysuckle bud are obtained in combination with the depth information of the camera; Step 3: The output of the detection target is the occluded honeysuckle bud, which requires deep image alignment, point cloud data preprocessing, point cloud segmentation and 3D point cloud reconstruction to obtain the actual picking point.

2. According to claim 1, a method for identifying and locating the optimal picking point of honeysuckle based on improved Yolov8 target detection is characterized in that: The improved Yolov8 target detection model includes a backbone network, a neck network and a detection head; wherein the backbone network includes a first Conv module, a first C2f module, a second Conv module, a second C2f module, a MultiSEAM attention mechanism module, a third Conv module, a third C2f module, a SEAM attention mechanism module, a fourth Conv module, a fourth C2f module, and an SPPF module connected in sequence; the MultiSEAM attention mechanism module outputs a first feature layer, the SEAM attention mechanism module outputs a second feature layer, and the SPPF module outputs a third feature layer; The neck network includes a first upsampling module, a first Concat module, a fifth C2f module, a second upsampling module, a second Concat module, a sixth C2f module, a fifth Conv module, a third Concat module, a seventh C2f module, a sixth Conv module, a fourth Concat module, and an eighth C2f module connected in sequence; wherein the inputs of the first upsampling module and the fourth Concat module are features output by the SPPF module, the input of the first Concat module is features output by the SEAM attention mechanism module, and the input of the second Concat module is features output by the MultiSEAM attention mechanism module; The detection head receives the features output by the sixth C2f module, the seventh C2f module, and the eighth C2f module respectively, and generates the final detection frame and classification result.

3. The method for identifying and locating the optimal picking point of honeysuckle based on improved Yolov8 target detection according to claim 2 is characterized in that: The improved Yolov8 target detection model is trained using honeysuckle images. The training process is as follows: Step 1.1: Collect images of honeysuckle buds and pre-process the collected images to obtain training data; Step 1.2: Use the training data prepared in step 1.1 to conduct preliminary training on the improved Yolov8 target detection model, test the trained weights, and test the recognition under different lighting conditions and angles; perform additional incremental training on the unrecognized image data, crop and record the misrecognized image data, and put the cropped positive and negative samples into the classification network for training to complete "recognition + reclassification".

4. The method for identifying and locating the optimal picking point of honeysuckle based on improved Yolov8 target detection according to claim 3 is characterized in that: The image preprocessing in step 1.1 includes: image enhancement, image format conversion, segmentation and cropping.

5. The method for identifying and locating the optimal picking point of honeysuckle based on improved Yolov8 target detection according to claim 1 is characterized in that: The process of determining the "actual picking point" in step 3 is as follows: Step 3.1: When using an RGB-D camera to capture images, ensure that the depth information is aligned with the RGB image and converted into 3D point cloud data; Step 3.2: Smooth the generated 3D point cloud data to remove noise and isolated points in the point cloud data; Step 3.3: Use PointCNN to segment the occluded honeysuckle bud point cloud data in the 3D point cloud data; Step 3.4: After the segmentation is completed, the ProxyFormer network structure is used to reconstruct the point cloud 3D; the picking point is determined based on the point cloud reconstruction result and the geometric center corresponding to the unobstructed part of the honeysuckle bud.

6. The method for identifying and locating the optimal picking point of honeysuckle based on improved Yolov8 target detection according to claim 5 is characterized in that: In step 3.1, the depth image is calculated into point cloud data through coordinate transformation, and the image coordinates are transformed into the world coordinate system according to the camera intrinsic parameters, i.e., the constraints of the coordinate transformation.

7. The method for identifying and locating the optimal picking point of honeysuckle based on improved Yolov8 target detection according to claim 5 is characterized in that: In step 3.2, the generated 3D point cloud data is smoothed using a Gaussian filter.

8. The method for identifying and locating the optimal picking point of honeysuckle based on improved Yolov8 target detection according to claim 5 is characterized in that: Step 3.3 The process of using PointCNN to segment the obscured honeysuckle bud point cloud data in the 3D point cloud data is as follows: Step 3.3.1: Construct a 3D point cloud dataset of honeysuckle buds and use the 3D point cloud dataset to train the PointCNN model; Step 3.3.2: Apply the trained PointCNN model to the newly acquired honeysuckle bud point cloud data segmentation and output the 3D point cloud segmentation result.

9. The method for identifying and locating the optimal picking point of honeysuckle based on improved Yolov8 target detection according to claim 5, characterized in that: Step 3.4 The process of determining the picking point is as follows: Step 3.4.1: Select proxy points from the segmented point cloud data based on the geometric features of the point cloud data; Step 3.4.2: Input the proxy point into the multi-head self-attention mechanism, merge the multiple outputs of the multi-head self-attention mechanism to form a global feature representation; Step 3.4.3: Use the occlusion-aware module in ProxyFormer to analyze the proxy points and global features to infer the geometric structure of the occluded area; Step 3.4.4: Combine the geometric structure with the original point cloud to generate a complete 3D reconstruction model of the honeysuckle bud; Step 3.4.5: Use Gaussian filtering technology to smooth the 3D reconstructed model; Step 3.4.6: The 3D reconstruction model is converted into a 3D point cloud of the honeysuckle bud through the ProxyFormer model, and the geometric center point of the 3D point cloud of the honeysuckle bud is used as the theoretical picking point (Xc, Y C , Z c ); Step 3.4.7: Considering the actual conditions, the picking point offset (ΔX, ΔY, ΔZ) is introduced to obtain the actual picking point, which is expressed as (X p , Y p , Z P ).

10. The method for identifying honeysuckle and locating the optimal picking point based on improved Yolov8 target detection according to claim 9, characterized in that: The basis for selecting proxy points is: by extracting the normal vector or density of the point cloud data, points with large normal vector changes or high density are selected as proxy points.

Images