A honeysuckle recognition and optimal picking point positioning method based on improved yolov8 target detection

Abstract

The application discloses a honeysuckle recognition and optimal picking point positioning method based on improved Yolov8 target detection, collects image data containing honeysuckle at a picking position, inputs the image data into an improved Yolov8 target detection model, performs feature extraction and multi-scale feature fusion on the improved Yolov8 target detection model, obtains a detection target, and divides the detection target into unshielded honeysuckle flower buds and shielded honeysuckle flower buds; the unshielded honeysuckle flower buds are output for the detection target, a center point of a detection frame is extracted, and the extracted center point is used as a honeysuckle flower bud picking point; three-dimensional coordinates of the honeysuckle flower bud picking point are obtained in combination with depth information of a camera; the shielded honeysuckle flower buds are output for the detection target, and depth image alignment, point cloud data preprocessing, point cloud segmentation and 3D point cloud reconstruction are required to obtain an actual picking point. The honeysuckle flower bud recognition and optimal picking point positioning are more accurate and efficient.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of intelligent picking, and in particular to a honeysuckle recognition and optimal picking point positioning method based on improved Yolov8 target detection. BACKGROUND

[0002] With the development of modern agricultural technology, more and more intelligent picking technologies have been proposed. However, due to the complexity of the natural growth environment of crops, the complexity of the growth environment inevitably leads to the phenomenon of partial occlusion of the picking part, and it is difficult to accurately position the picking point.

[0003] Taking honeysuckle as an example, honeysuckle is a common traditional Chinese medicine, and its flower buds and flowers are used as medicine, which has the effects of clearing heat and detoxifying, anti-inflammatory and antibacterial. The flower bud of honeysuckle is round or oval in shape, and the outer part is curved with a smooth edge. The flower bud is green in the early stage and turns yellow or white in the mature stage. The flower bud usually grows in a cluster and is located at the top of the new branch. The leaf of honeysuckle is oval or ovate in shape. The growth direction of the mature honeysuckle plant leaf is diverse, some grow upward and some spread horizontally. Under natural growth conditions, part of the flower bud is blocked by the leaf, forming a local shadow area, which affects the pickability of the flower bud. According to the growth characteristics of honeysuckle, the picking of honeysuckle is usually carried out before the flower bud opens. Since the flower buds are usually clustered, single picking is usually adopted to ensure the quality and subsequent medicinal value. However, the biggest problem in the current intelligent picking of honeysuckle is that the complex growth environment leads to the phenomenon of partial occlusion of the picking part, which causes the mechanical arm to operate to the wrong position and pick incorrectly, affecting the accuracy and efficiency of picking.

[0004] Although the prior art also proposes a solution to the occlusion problem in intelligent picking, such as the tea tender shoot recognition and picking point positioning method disclosed in CN116958823A, which obtains the depth information of the picking part through point cloud completion and then performs three-dimensional picking point positioning. However, this method has large storage and calculation amount, easy loss of geometric structure, and complex post-processing method. SUMMARY

[0005] In order to solve the problems existing in the prior art, the application provides a honeysuckle recognition and optimal picking point positioning method based on improved Yolov8 target detection. The improved Yolov8 target detection model can accurately and quickly determine whether the honeysuckle flower bud is occluded. For the case of occlusion, point cloud segmentation is performed first, and then point cloud reconstruction is performed, so that the actual picking point position can be effectively determined. The information of these positions is accurate, reliable and has practical application value.

[0006] The technical scheme adopted by the application is as follows:

[0007] A honeysuckle recognition and optimal picking point positioning method based on improved Yolov8 target detection, comprising the following steps:

[0008] Step 1: Collect image data containing honeysuckle at the picking site, input the image data into the improved Yolov8 target detection model, and the improved Yolov8 target detection model performs feature extraction and multi-scale feature fusion to obtain a detection target. The detection target is divided into unobstructed honeysuckle buds and obstructed honeysuckle buds;

[0009] Step 2: For the unobstructed honeysuckle buds output by the detection target, the center point of the detection frame is extracted, and the extracted center point is used as the honeysuckle bud picking point. Combined with the depth information of the camera, the three-dimensional coordinates of the honeysuckle bud picking point are obtained.

[0010] Step 3: For the obstructed honeysuckle buds output by the detection target, depth image alignment, point cloud data preprocessing, point cloud segmentation and 3D point cloud reconstruction are needed to obtain the actual picking point.

[0011] Further, the improved Yolov8 target detection model includes a backbone network, a neck network and a detection head; wherein the backbone network includes a first Conv module, a first C2f module, a second Conv module, a second C2f module, a MultiSEAM attention mechanism module, a third Conv module, a third C2f module, a SEAM attention mechanism module, a fourth Conv module, a fourth C2f module and a SPPF module connected in sequence; the first feature layer is output by the MultiSEAM attention mechanism module, the second feature layer is output by the SEAM attention mechanism module, and the third feature layer is output by the SPPF module;

[0012] The neck network includes a first up-sampling module, a first Concat module, a fifth C2f module, a second up-sampling module, a second Concat module, a sixth C2f module, a fifth Conv module, a third Concat module, a seventh C2f module, a sixth Conv module, a fourth Concat module and an eighth C2f module connected in sequence; wherein the input of the first up-sampling module and the fourth Concat module is the feature output by the SPPF module, the input of the first Concat module is the feature output by the SEAM attention mechanism module, and the input of the second Concat module is the feature output by the MultiSEAM attention mechanism module;

[0013] The detection head receives the features output by the sixth C2f module, the seventh C2f module and the eighth C2f module respectively, and generates the final detection frame and classification result.

[0014] Further, the improved Yolov8 target detection model is trained using honeysuckle image pairs, and the training process is as follows:

[0015] Step 1.1: Image acquisition is performed on the honeysuckle flower buds, and the collected images are preprocessed to obtain training data;

[0016] Step 1.2: The improved Yolov8 target detection model is preliminarily trained using the training data prepared in step 1.1, and the trained weights are tested to identify the recognition situation under different light conditions and angles; the unrecognizable picture data is supplemented for incremental training, and the misrecognized picture data is cropped and recorded, and the cropped positive samples and negative samples are put into the classification network for training to complete the "recognition + reclassification".

[0017] Further, the image preprocessing in step 1.1 includes image enhancement, image format conversion, segmentation and cropping.

[0018] Further, the process of determining the "actual picking point" in step 3 is as follows:

[0019] Step 3.1: When using an RGB-D camera to collect images, ensure that the depth information is aligned with the RGB image, and convert it into 3D point cloud data;

[0020] Step 3.2: Smooth the generated 3D point cloud data to remove noise and isolated points in the point cloud data;

[0021] Step 3.3: Use PointCNN to segment the occluded honeysuckle flower bud point cloud data in the 3D point cloud data;

[0022] Step 3.4: After segmentation, apply the ProxyFormer network structure for point cloud three-dimensional reconstruction; determine the picking point according to the point cloud reconstruction result and the geometric center corresponding to the unoccluded part of the honeysuckle flower bud.

[0023] Further, in step 3.1, the depth image is converted into point cloud data through coordinate conversion, and the image coordinates are converted into the world coordinate system according to the constraints of the camera intrinsic parameters, i.e., coordinate transformation.

[0024] Further, in step 3.2, the generated 3D point cloud data is smoothed by a Gaussian filter.

[0025] Further, the process of using PointCNN to segment the occluded honeysuckle flower bud point cloud data in the 3D point cloud data in step 3.3 is as follows:

[0026] Step 3.3.1: Construct a 3D point cloud dataset of honeysuckle flower buds, and train the PointCNN model using the 3D point cloud dataset;

[0027] Step 3.3.2: Apply the trained PointCNN model to the newly acquired honeysuckle flower bud point cloud data segmentation, and output the 3D point cloud segmentation result.

[0028] Further, the process of determining the picking point in step 3.4 is as follows:

[0029] Step 3.4.1: Based on the geometric features of the point cloud data, select the proxy points from the segmented point cloud data;

[0030] Step 3.4.2: Input the proxy points into the multi-head self-attention mechanism, combine the multiple outputs of the multi-head self-attention mechanism to form a global feature representation;

[0031] Step 3.4.3: Analyze the proxy points and global features using the Occlusion-Aware Module in ProxyFormer to infer the geometric structure of the occluded area;

[0032] Step 3.4.4: Combine the geometric structure with the original point cloud to generate a complete three-dimensional reconstruction model of the honeysuckle flower bud;

[0033] Step 3.4.5: Smooth the three-dimensional reconstruction model using Gaussian filtering technology;

[0034] Step 3.4.6: Convert the three-dimensional reconstruction model to honeysuckle flower bud 3D point cloud through the ProxyFormer model, and take the geometric center point of the honeysuckle flower bud 3D point cloud as the theoretical picking point (Xc, Y C , Z c );

[0035] Step 3.4.7: Considering the real conditions, introduce the picking point offset (ΔX, ΔY, ΔZ) to obtain the actual picking point, represented as (X p , Y p , Z P ).

[0036] Further, the basis for selecting proxy points is: by extracting the normal vector or density of the point cloud data, select the points with large normal vector changes or high density as proxy points.

[0037] The beneficial effects of the present application are:

[0038] (1) The SEAM and MultiSEAM modules are used to optimize the Yolov8 target detection network, and the optimized Yolov8 target detection network has increased feature extraction capability, improved detection accuracy of occlusions, improved model calculation efficiency, and improved model generalization ability.

[0039] (2) For the image with occlusion, first use the PointCNN point cloud segmentation network to perform honeysuckle bud point cloud segmentation processing, and then use the Proxformer point cloud reconstruction network structure to perform honeysuckle bud point cloud reconstruction processing. The advantage of this method is that the permutation invariance of PointCNN and the clear segmentation edge make the fine structure of honeysuckle bud be effectively recognized. And the Proxformer can perceive the position of missing points and predict the point cloud information of the occluded part through its innovative missing part sensitive transformer and proxy alignment mechanism, so as to realize more complete reconstruction. This processing method not only improves the quality of point cloud segmentation and reconstruction, but also enhances the ability of the model to process occlusion and incomplete data, so that the recognition and optimal picking point positioning of honeysuckle bud are more accurate and efficient. Compared with the recognition and picking point positioning method disclosed in the prior art, the processing method of the present application can more efficiently and accurately process occlusion, improve segmentation accuracy, retain more local features, and improve point cloud quality. At the same time, it can cross-modal information fusion to generate three-dimensional point cloud with more complete structure and details, and show better performance in honeysuckle bud recognition and actual picking point positioning.

[0040] (3) Compared with the prior art, the advantage of the present application is that the specific morphology of the single honeysuckle bud can accurately recognize honeysuckle in the case of occlusion, and the optimal picking point can be obtained. The improved Yolov8 target detection algorithm model of the present application has a certain generalization and can be applied to the accurate recognition and optimal picking point positioning of similar drugs in the case of occlusion. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 is the overall operation logic diagram of the method of the present application.

[0042] Figure 2 is the improved YOLOv8 algorithm network structure diagram of the method of the present application.

[0043] Figure 3 is the improved YOLOv8 algorithm network structure of the present application, wherein a is the SEAM network structure diagram, and b is the CSMM module network structure diagram.

[0044] Figure 4 is the PointCNN point cloud segmentation network structure used in the present application, wherein a is a simple PointCNN of two X-Conv layers, and b is a classification task.

[0045] Figure 5 The hierarchical convolution structure introduced in the PointCNN point cloud segmentation network structure used in the present application.

[0046] Figure 6 The Proxformer point cloud reconstruction network structure used in the present application.

[0047] Figure 7 The point cloud attention module in the Proxformer point cloud reconstruction network structure used in the application. DETAILED DESCRIPTION

[0048] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0049] In combination with the drawings Figure 1 , the present application proposes a honeysuckle recognition and optimal picking point positioning method based on improved Yolov8 target detection, including the following steps:

[0050] Step 1: collect image data containing honeysuckle at the picking site, input the image data into the improved Yolov8 target detection model, and the improved Yolov8 target detection model performs feature extraction and multi-scale feature fusion to obtain a detection target, the detection target being divided into unobstructed honeysuckle buds and obstructed honeysuckle buds;

[0051] The network structure of the improved Yolov8 target detection model in the above step 1 is as shown in Figure 2 , including a backbone network, a neck network and a detection head.

[0052] The backbone network includes a first Conv module, a first C2f module, a second Conv module, a second C2f module, a MultiSEAM attention mechanism module, a third Conv module, a third C2f module, a SEAM attention mechanism module, a fourth Conv module, a fourth C2f module and a SPPF module connected in sequence; the first feature layer is output by the MultiSEAM attention mechanism module, the second feature layer is output by the SEAM attention mechanism module, and the third feature layer is output by the SPPF module. The present application introduces SEAM and MultiSEAM attention mechanisms in the backbone network, and the MultiSEAM attention mechanism can process multi-level feature maps, thereby capturing target information of different scales. The SEAM mechanism helps the model pay more attention to unobstructed honeysuckle buds by emphasizing important regions in the feature map, learning adaptive weights, enhancing regions related to the target in the feature map and simultaneously suppressing background noise. The present application allows the model to apply attention mechanisms on different feature layers to improve the detection capability for small or obstructed targets.

[0053] The neck network fuses feature maps of different scales using a feature pyramid, and the specific network structure comprises: a first up-sampling module, a first Concat module, a fifth C2f module, a second up-sampling module, a second Concat module, a sixth C2f module, a fifth Conv module, a third Concat module, a seventh C2f module, a sixth Conv module, a fourth Concat module, and an eighth C2f module connected in sequence; wherein the input of the first up-sampling module and the fourth Concat module is the feature output by the SPPF module, the input of the first Concat module is the feature output by the SEAM attention mechanism module, and the input of the second Concat module is the feature output by the MultiSEAM attention mechanism module.

[0054] The detection head receives the features output by the sixth C2f module, the seventh C2f module and the eighth C2f module respectively, is responsible for generating the final detection frame and classification result, and integrates RepulsionLoss into the loss calculation module as part of the bounding box regression loss. When calculating the loss, the detection head not only considers the accuracy of the bounding box, but also increases the repulsion force between the occluded objects, encouraging the model to separate the target, so as to better locate the honeysuckle flower bud.

[0055] The coordinate information of the detection frame of each target to be picked is (center_x, center_y, width, height), wherein center_x and center_y are the midpoint coordinates of the detection frame, and width and height are the width and height of the detection frame.

[0056] After completing the construction of the improved Yolov8 target detection model framework, the honeysuckle image needs to be used to train the improved Yolov8 target detection model, and the specific training process is as follows:

[0057] Step 1.1: Use an RGB-D camera to collect images of honeysuckle flower buds, ensure that there are enough rich training data under different lighting conditions, angles and occlusion conditions, and pre-process the collected images, such as image enhancement, image format conversion, segmentation and cropping, etc.

[0058] Step 1.2: Use the training data prepared in step 1.1 to preliminarily train the improved Yolov8 target detection model, test the trained weights, test the recognition under different lighting conditions and angles, supplement the increment training of the pictures not recognized, and record the pictures misrecognized by cropping, put the positive samples and negative samples obtained by cropping into the classification network for training, complete the overall task of "recognition + reclassification", and finally perform multiple experiments to continuously supplement data and optimize model weights.

[0059] Step 2: For the detection target output is no shelter of honeysuckle flower bud, the center point of the detection box is extracted, and the extracted center point is used as the honeysuckle flower bud picking point; combined with the depth information of the camera, the three-dimensional coordinates of the honeysuckle flower bud picking point are obtained.

[0060] Step 3: For the detection target output is sheltered honeysuckle flower bud, depth image alignment, point cloud data preprocessing, point cloud segmentation and 3D point cloud reconstruction are needed, through analysis, multiple possible picking points are obtained, considering the size of the shelter, the grabbing path, the accessibility of the mechanical arm, and finally the optimal "actual picking point" is selected. The specific process is as follows:

[0061] Step 3.1: When using an RGB-D camera to collect images, ensure that the depth information is aligned with the RGB image, and convert it into 3D point cloud data.

[0062] For sheltered honeysuckle flower buds, multiple perspective depth images need to be combined to obtain more complete 3D information.

[0063] Depth image can be calculated as point cloud data after coordinate conversion. According to the constraint condition of camera internal parameter, image coordinates can be converted to world coordinate system.

[0064] The point cloud conversion formula is:

[0065]

[0066] Where x, y, z are the point cloud coordinate system, x', y' are the image coordinate system, D is the depth value, f x is the focal length of the camera internal parameter in the horizontal direction, f y is the focal length of the camera internal parameter in the vertical direction.

[0067] Step 3.2: The generated 3D point cloud data is smoothed by a Gaussian filter, which can remove noise and isolated points in the point cloud data. During smoothing, the overall shape and features of the point cloud can be preserved, avoiding excessive smoothing that may cause information loss, improving the quality of the point cloud and the robustness of the model.

[0068] The formula of Gaussian filter is:

[0069]

[0070] When processing point cloud, it can be extended to three-dimensional form:

[0071]

[0072] Where G(x, y) is the value of the Gaussian filter, (x, y) is the coordinate distance from the center point, and σ is the standard deviation, which controls the width of the filter.

[0073] Step 3.3: In a complex growth environment, the 3D point cloud data of honeysuckle flower buds is often obscured, making the foreground segmentation task more difficult; due to the characteristics of PointCNN, such as efficient learning of local features, rapid integration of global features, it can flexibly handle different point cloud processing tasks, such as classification, segmentation, etc.; therefore, PointCNN is used to segment the obscured honeysuckle flower bud point cloud data in the 3D point cloud data. Reference Figure 4 、 5 The process is as follows:

[0074] Step 3.3.1: Construct a 3D point cloud dataset of honeysuckle flower buds, use the 3D point cloud dataset to train the PointCNN model, set appropriate learning rate and batch size to prevent overfitting during training. Ensure that the model can effectively identify obscured and unobscured flower bud parts, and use evaluation metrics such as accuracy, recall, and F1 score to judge the segmentation effect of the model on obscured and unobscured honeysuckle flower buds.

[0075] Step 3.3.2: Apply the trained PointCNN model to the new honeysuckle flower bud point cloud data segmentation, and output the 3D point cloud segmentation results.

[0076] Step 3.4: After segmentation, apply the ProxyFormer network structure for point cloud three-dimensional reconstruction, such as Figure 6 、 7 The network extracts proxy points and global features, especially processes information about obscured parts, and infers missing geometric structures.

[0077] Finally, according to the above point cloud reconstruction results and the geometric center of the unobscured part of the honeysuckle flower bud, multiple possible picking points are obtained, which are defined as "theoretical picking points" and "actual picking points".

[0078] Step 3.4.1: Select proxy points from the segmented point cloud data, these points can represent important geometric features or regions, and can effectively capture overall shape and structure information. The basis for selecting proxy points is: by extracting the geometric features of point cloud data (such as normal vector, density, etc.), points with large changes in normal vector or high density can be selected as proxy points.

[0079] Step 3.4.2: Input the proxy points into the multi-head self-attention mechanism, combine the multiple outputs of the multi-head self-attention mechanism to form a global feature representation, which integrates multiple information of the point cloud and can fully consider the geometric structure under the condition of occlusion.

[0080] Step 3.4.3: Analyze the proxy points and global features using the Occlusion-Aware Module in ProxyFormer to infer the geometry of the occluded regions. The Occlusion-Aware Module utilizes feature interaction mechanisms to enhance understanding of missing parts and generate more accurate geometry.

[0081] Step 3.4.4: Combine the geometry with the original point cloud to generate a complete 3D reconstruction model of the honeysuckle bud. During the reconstruction process, techniques such as interpolation or Generative Adversarial Networks (GAN) are used to optimize the reconstruction results and improve details and realism.

[0082] Step 3.4.5: Smooth the 3D reconstruction model using Gaussian filtering techniques to remove noise and discontinuous parts, ensuring the quality of the 3D reconstruction model.

[0083] Step 3.4.6: Convert the 3D reconstruction model to a honeysuckle bud 3D point cloud using the ProxyFormer model, and take the geometric center point of the honeysuckle bud 3D point cloud as the theoretical picking point (Xc, Y C , Z c );

[0084] Step 3.4.7: Considering real-world conditions such as occlusions and inaccessible regions, obtain the actual picking point based on the theoretical picking point, represented as (X p , Y p , Z P );

[0085] The specific calculation is as follows:

[0086] X p = Xc + ΔX

[0087] Y p = Y C + ΔY

[0088] Z P = Z c + ΔZ

[0089] Where (Xc, Y C , Z c ) are the coordinates of the theoretical picking point, and (ΔX, ΔY, ΔZ) are the picking point offsets.

[0090] The above examples are only used to illustrate the design ideas and characteristics of the present application, and the purpose is to enable those skilled in the art to understand the content of the present application and implement it, and the protection scope of the present application is not limited to the above examples. Therefore, any equivalent changes or modifications made in accordance with the principles and design ideas disclosed by the present application are within the scope of protection of the present application.

Claims

1. A honeysuckle recognition and optimal picking point positioning method based on improved Yolov8 target detection, characterized in that, Comprising the following steps: Step 1: Collecting image data containing honeysuckle at the picking site, inputting the image data into the improved Yolov8 target detection model, and performing feature extraction and multi-scale feature fusion by the improved Yolov8 target detection model to obtain a detection target, wherein the detection target is divided into unobstructed honeysuckle flower buds and obstructed honeysuckle flower buds; the improved Yolov8 target detection model comprises a backbone network, a neck network and a detection head; wherein the backbone network comprises a first Conv module, a first C2f module, a second Conv module, a second C2f module, a MultiSEAM attention mechanism module, a third Conv module, a third C2f module, a SEAM attention mechanism module, a fourth Conv module and a fourth C2f module connected in sequence, and an SPPF module; the first feature layer is output by the MultiSEAM attention mechanism module, the second feature layer is output by the SEAM attention mechanism module, and the third feature layer is output by the SPPF module; The neck network comprises a first upsampling module, a first Concat module, a fifth C2f module, a second upsampling module, a second Concat module, a sixth C2f module, a fifth Conv module, a third Concat module, a seventh C2f module, a sixth Conv module, a fourth Concat module and an eighth C2f module connected in sequence; wherein the input of the first upsampling module and the fourth Concat module is the feature output by the SPPF module, the input of the first Concat module is the feature output by the SEAM attention mechanism module, and the input of the second Concat module is the feature output by the MultiSEAM attention mechanism module; The detection head receives the features output by the sixth C2f module, the seventh C2f module and the eighth C2f module respectively, and generates a final detection frame and a classification result; Step 2: For the detection target outputting unobstructed honeysuckle flower buds, the center point of the detection frame is extracted, and the extracted center point is used as the honeysuckle flower bud picking point; then, the three-dimensional coordinates of the honeysuckle flower bud picking point are obtained in combination with the depth information of the camera; Step 3: For the detection target outputting obstructed honeysuckle flower buds, depth image alignment, point cloud data preprocessing, point cloud segmentation and 3D point cloud reconstruction are needed to obtain the actual picking point; the process of determining the actual picking point in step 3 is as follows: Step 3.1: When collecting images using an RGB-D camera, ensure that the depth information is aligned with the RGB image, and convert it into 3D point cloud data; Step 3.2: Smooth the generated 3D point cloud data to remove noise and isolated points in the point cloud data; Step 3.3: Use PointCNN to segment the obstructed honeysuckle flower bud point cloud data in the 3D point cloud data; Step 3.4: After segmentation, apply the ProxyFormer network structure for 3D point cloud reconstruction; determine the picking point according to the point cloud reconstruction result and the geometric center corresponding to the unobstructed honeysuckle flower bud; the process of determining the picking point in step 3.4 is as follows: Step 3.4.1: Select proxy points from the segmented point cloud data based on geometric features of the point cloud data; Step 3.4.2: Input the proxy points into a multi-head self-attention mechanism, combine the multiple outputs of the multi-head self-attention mechanism to form a global feature representation; Step 3.4.3: Analyze the proxy points and global features using the Occlusion-Aware Module in ProxyFormer to infer the geometric structure of the occluded region; Step 3.4.4: Combine the geometric structure with the original point cloud to generate a complete 3D reconstruction model of the honeysuckle bud; Step 3.4.5: Smooth the 3D reconstruction model using Gaussian filtering technology; Step 3.4.6: Convert the 3D reconstructed model into a 3D point cloud of honeysuckle buds using the ProxyFormer model, and use the geometric center point of the 3D point cloud of honeysuckle buds as the theoretical picking point (Xc, Y). C Z c ); Step 3.4.7: Considering the real conditions, the picking point offset (ΔX, ΔY, ΔZ) is introduced to get the actual picking point, denoted as (X p , Y p , Z P ).

2. The improved Yolov8 target detection-based honeysuckle recognition and optimal picking point positioning method according to claim 1, characterized in that, The improved Yolov8 target detection model is trained using honeysuckle image data. The training process is as follows: Step 1.1: Collect honeysuckle bud images and preprocess the collected images to obtain training data; Step 1.2: Use the training data prepared in step 1.1 to preliminarily train the improved Yolov8 target detection model, test the trained weights, and test the recognition under different light conditions and angles; supplement the increment training of the unrecognized picture data, and record the misrecognized picture data for cutting, put the positive and negative samples obtained by cutting into the classification network for training, and complete the "recognition + reclassification".

3. The improved Yolov8 target detection-based honeysuckle recognition and optimal picking point positioning method according to claim 2, characterized in that, The image preprocessing in step 1.1 includes image enhancement, image format conversion, segmentation, and cropping.

4. The improved Yolov8 target detection-based honeysuckle recognition and optimal picking point positioning method according to claim 1, characterized in that, In step 3.1, the depth image is converted to point cloud data through coordinate conversion. According to the constraint condition of camera internal parameter, i.e. coordinate transformation, the image coordinates are converted to world coordinates.

5. The improved Yolov8 target detection-based honeysuckle recognition and optimal picking point positioning method according to claim 1, characterized in that, In step 3.2, the generated 3D point cloud data is smoothed by a Gaussian filter.

6. The improved Yolov8 target detection-based honeysuckle recognition and optimal picking point positioning method according to claim 1, characterized in that, In step 3.3, the process of segmenting the occluded honeysuckle bud point cloud data in the 3D point cloud data using PointCNN is as follows: Step 3.3.1: Construct a 3D point cloud dataset of honeysuckle buds, and use the 3D point cloud dataset to train the PointCNN model; Step 3.3.2: Apply the trained PointCNN model to the newly acquired honeysuckle bud point cloud data segmentation to output 3D point cloud segmentation results.

7. The improved Yolov8 target detection-based honeysuckle recognition and optimal picking point positioning method according to claim 1, characterized in that, The basis for selecting proxy points is to select points with large changes in normal vectors or high density as proxy points by extracting the normal vectors or density of the point cloud data.

Images