A method and system for seedling grasping and directional delivery in a directional decoupling and skeleton-sensing grasping field
By employing directional decoupling and skeleton-aware grasping field methods, and utilizing a multi-task model and encoder dynamic compensation, the positioning error and gripping instability issues in the grasping and directional delivery of Chinese medicinal seedlings were resolved, achieving efficient seedling grasping and directional delivery.
Patent Information
- Application Number
- CN202611496114.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-24
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies for grasping and directional delivery of Chinese medicinal seedlings suffer from problems such as large positioning errors at the grasping point, unstable gripping, separation of vision methods from robot motion control, and difficulty in achieving 0-360° end-oriented command output.
By employing a method of orientation decoupling and skeleton perception of the grasping field, the robot's pose is generated through dynamic compensation of encoder position, based on the prediction of the rotating target box, end angle, and grasping field using a multi-task model, achieving directional delivery in a realistic end orientation of 0-360°.
It improves the positioning accuracy and stability of seedling grasping, reduces the amount of computation, enhances the joint perception stability under complex shapes and multi-target interference, and realizes efficient targeted seedling delivery.
Smart Images

Figure CN123118770A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of agricultural robot visual perception and intelligent grasping technology, and in particular to a method and system for grasping and directionally delivering seedlings in a directional decoupling and skeleton perception grasping field. Background Technology
[0002] Root and rhizome medicinal herbs such as Codonopsis pilosula and Astragalus membranaceus are commonly transplanted from seedlings to the field. The main production areas are mostly located in hilly and mountainous regions, where plots are scattered and slopes and land preparation quality vary significantly, limiting the operational organization and adaptability of large-scale general-purpose agricultural machinery. Existing transplanters mostly employ semi-mechanized operations, requiring operators to perform seedling sorting, separation, and placement in the field, resulting in low efficiency. Combining robotics technology to automatically identify, grasp, and orient medicinal herb seedlings in a controlled environment, weaving them into rolls for placement in the field, can move the labor-intensive placement step forward, providing an effective way to improve the efficiency and quality of medicinal herb transplanting. During robotic operation, bent and forked seedlings enter the visual area with random orientations; the orienting of seedlings is crucial for the continuous and stable operation of the seedling roll preparation. Furthermore, the length, thickness, bending, and forking positions of medicinal herb root seedlings vary significantly, and the seedling head and root tip cannot be interchanged. The grasping point is determined by the stability of the gripper, which does not correspond to a uniform biological characteristic point.
[0003] Existing technologies, such as "A Sweet Potato Grasping Point Localization Method Based on Saliency Target Detection Technology" and "A Study on the Optimal Grasping Angle Algorithm for Plug Seedlings Based on MachineVision," disclose methods for grasping point localization based on contour, geometric features, or multimodal information. Song et al.'s 2022 publication, "A novel vision-based multi-task robotic grasp detection method for multi-object scenes," proposed a decoupled single-stage multi-task grasping detection method based on Faster R-CNN. Ma et al.'s 2025 publication, "STRAW-YOLO: A detection method for strawberry fruits targets and key points," uses YOLOv8-pose to jointly detect key points for strawberry fruit and stem picking, and combines depth data for 3D localization, disclosing a method for multi-task visual grasping and agricultural key point localization. Existing technology, "A BIM and Visual Observation Information Fusion Method and System for Wall-Masonry Robot Grasping and Recognition," discloses a method for rotating the frame's main axis and the robot's grasping posture.
[0004] However, existing technologies have the following drawbacks: (1) Standard OBB can only express undirected geometric axes that are 180° apart. When the end effector needs to determine the “head end” and “tail end” of the target and adjust the gripper opening direction or delivery direction accordingly, the OBB angle alone cannot form a complete 0-360° execution command.
[0005] (2) Representing the grab point as the local normalized coordinates of the OBB, and then predicting the center, scale and angle of the OBB and transforming it to the image plane, will cause the detection error of the OBB to be transmitted again and amplify the grab point positioning error, which is especially unfavorable for slender and curved targets.
[0006] (3) Single-point coordinate regression only gives a numerical position, which is difficult to express both the preferred clamping point selected by human and the clampable structure extending along the main trunk near that point; if the entire skeleton is directly used as a hard constraint, it is easy to pull the probability centroid away from the human verification point.
[0007] (4) Conventional isotropic convolution lacks explicit modeling of the main axis continuity, end differences and local bending of slender targets. Under conditions of bending, bifurcation, connection and adjacent interference, it is easy to miss detection, end error or grab point offset.
[0008] (5) Existing seedling vision methods and robot motion control are often designed separately. The vision results lack definite instance attribution and position compensation at the grasping moment, making it difficult to directly convert them into robot grasping pose and directional delivery actions on the motion conveyor belt. Summary of the Invention
[0009] The purpose of this invention is to provide a method and system for seedling grasping and directional deployment in a directional decoupling and skeleton-aware grasping field. While maintaining the ability of the OBB (On-Board Component) to compactly describe slender seedling heads, it eliminates the 180° directional ambiguity of the OBB axis, outputting a 0-360° true end orientation that can be used for end-effector rotation and directional deployment. It avoids the dependence of the grasping point on the predicted OBB geometric inverse transformation, allowing the grasping point to be directly predicted in the image coordinate system and maintaining a one-to-one association with the corresponding seedling head instance. It unifies the accuracy of manually verified grasping points and the grippable structures near the seedling's main skeleton into a single learnable target, while suppressing field centroid drift caused by the entire skeleton. It enhances the main axis, end, and local bending characteristics of slender seedlings with lower additional computational cost, improving joint perception stability under complex morphology and multi-target interference. It combines visual output, camera calibration, encoder displacement, and robot arrival time to generate the grasping position, gripper direction, and directional release command for the moving target.
[0010] To achieve the above objectives, the present invention provides the following solution: A method for seedling grasping and directional delivery in a directional decoupling and skeleton-sensing grasping field includes: S1. Obtain an RGB image containing a flexible and slender target, establish a unique instance index for each target instance and label it, and construct a training image, wherein the flexible and slender target is a bare-root seedling of a root-type Chinese medicinal herb; S2. Extract the target contour for each target instance in the training image, calculate the peak field, skeleton soft region field and context gating field of the manually verified grab point, and fuse and normalize them to obtain the target field probability. If the target instance cannot form an effective longest main skeleton, the target field probability is determined by the peak field of the manually verified grab point. S3. Input the training image and the target field probability into a multi-task model of rotating target box, apical angle and grasping field, and train the multi-task model, wherein the multi-task model extracts and fuses features through a shared backbone network and multi-scale neck; S4. Simultaneously acquire the RGB image of the flexible and slender target to be grasped and the encoder position, and after preprocessing, input it into the trained multi-task model to predict the rotation target box, end angle and grasping field of the corresponding relationship of the same instance. S5. Based on the rotation target box, apical angle and grab field corresponding to the same instance, perform rotation box suppression and executability candidate screening. If there is an executable candidate, proceed to S6 to grab the current candidate decoding image point. If no executable candidate is obtained, record the reason for no candidate and return to S4 to enter the next acquisition cycle. S6. Perform coordinate calibration transformation and encoder dynamic compensation on the decoded grasping points to obtain the compensated robot base coordinate position. If the target exceeds the grasping window after compensation, return to S5 to select alternative candidates or return to S4 to re-acquire. S7. Generate the robot pose based on the compensated robot base coordinate position and the predicted end angle, and control the robot to complete the grasping, transfer and directional release of the flexible and slender target to be grasped.
[0011] This embodiment also discloses a seedling grasping and directional delivery system for a directional decoupling and skeleton sensing grasping field, used to implement the seedling grasping and directional delivery method for the aforementioned directional decoupling and skeleton sensing grasping field, including: The module includes a data acquisition and synchronization module, a shared feature extraction and multi-scale fusion module, an image coordinate skeleton perception and grasping field module, an instance association and candidate selection module, a coordinate transformation and dynamic compensation module for transportation, and a robot grasping and directional delivery module.
[0012] The beneficial effects of this invention are as follows: (1) This invention separates the undirected geometric axis of the OBB from the directed end direction of the target in modeling: the major axis of the OBB continues to retain 180° equivalence, and the independent directional branches output signed unit vectors with endpoint semantics. Through "signed direction supervision + collinear constraint that is insensitive to the sign of the undirected major axis", the problem of "correct geometric box but wrong head and tail directions" is solved without destroying the geometric stability of the revolved box, and the true end direction of 0-360° can be obtained directly.
[0013] (2) This invention predicts the probabilistic grasping field in the image coordinate system and decodes the continuous grasping points through soft-argmax. The grasping point localization does not depend on the center, scale and angle inverse transformation of the predicted OBB, thus cutting off the secondary propagation link of the rotation box error to the grasping point.
[0014] (3) The skeleton-aware grasping field retains the precise peak value of the manually verified points and the local grippable structure near the longest main skeleton, and limits the scope of the skeleton prior with the context gating of the manually verified points to avoid the probabilistic mass on the whole skeleton causing the field centroid to drift along the main trunk. The field target formed by "point peak value + main skeleton soft region + context gating" takes into account positioning accuracy, structural tolerance and executability, and is more suitable for flexible and slender targets than single point coordinate regression or hard skeleton region constraint.
[0015] (4) Orientation decoupling and skeleton-aware grasping field are not isolated outputs: they share features with OBB and maintain the same candidate and instance allocation relationship, so that "which position to grasp" and "what is the true orientation of the target" are closed on the same instance, reducing the risk of grasping point, target category and orientation mismatch in multi-target scenarios. This combination also retains the possibility that the orientation decoupling sub-algorithm and the grasping field sub-algorithm can be implemented independently.
[0016] (5) The contour, skeleton and manual verification points are only used to construct the supervised target during the training period. Only RGB images are needed during the inference stage to output OBB, real end orientation and image coordinate capture points. No online skeleton extraction and point cloud processing links are added, which is convenient for deployment on industrial control computers or edge computing devices.
[0017] (6) As an enhancement structure, AAFEM uses horizontal, vertical and dilated depth convolutions to form lightweight orientation-sensitive basis functions, and enhances the slender main axis, end and local bending features through spatial gating and residual connection. It can be combined with different main trunks and multi-scale necks without changing the core representation relationship of the aforementioned orientation decoupling and grasping field.
[0018] (7) In the application layer of the algorithm output, the present invention can unify the image coordinate grasping point, the real end direction, the encoder cumulative displacement, the camera to robot calibration relationship and the expected grasping time, and transform the static vision result into the dynamic robot pose on the motion conveyor belt, and complete the directional release or fixed distance placement; this execution link does not constitute a necessary limitation on the independent protection scope of the aforementioned vision algorithm. Attached Figure Description
[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0020] Figure 1 This is a block diagram of the dynamic capture and targeted delivery system according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating the training and online execution of a seedling grasping and directional delivery method in a directional decoupling and skeleton perception grasping field according to an embodiment of the present invention. Detailed Implementation
[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0023] Example 1: This embodiment proposes a method for seedling grasping and directional delivery in a directional decoupling and skeleton-sensing grasping field, including: S1. Obtain an RGB image containing a flexible and slender target, establish a unique instance index for each target instance and label it, and construct a training image, wherein the flexible and slender target is a bare-root seedling of a root-type Chinese medicinal herb, and the bare-root seedling of the root-type Chinese medicinal herb is at least one of Codonopsis pilosula, Astragalus membranaceus and Angelica sinensis. S2. Extract the target contour for each target instance in the training image, calculate the peak field, skeleton soft region field and context gating field of the manually verified grab point, and fuse and normalize them to obtain the target field probability. If the target instance cannot form an effective longest main skeleton, the target field probability is determined by the peak field of the manually verified grab point. S3. Input the training image and the target field probability into a multi-task model of rotating target box, apical angle and grasping field, and train the multi-task model. The multi-task model extracts and fuses features through a shared backbone network and multi-scale neck. The multi-scale feature fusion includes an axis-aware feature enhancement module, which includes a horizontal depthwise convolution branch, a vertical depthwise convolution branch and a dilated depthwise convolution branch. After feature fusion of each branch, gated weights are generated by channel pooling and spatial convolution, and enhanced features are output through residual connections. S4. Simultaneously acquire the RGB image of the flexible and slender target to be grasped and the encoder position, and after preprocessing, input it into the trained multi-task model to predict the rotation target box, end angle and grasping field of the corresponding relationship of the same instance. S5. Based on the rotation target box, apical angle and grab field corresponding to the same instance, perform rotation box suppression and executability candidate screening. If there is an executable candidate, proceed to S6 to grab the current candidate decoding image point. If no executable candidate is obtained, record the reason for no candidate and return to S4 to enter the next acquisition cycle. S6. Perform coordinate calibration transformation and encoder dynamic compensation on the decoded grasping points to obtain the compensated robot base coordinate position. If the target exceeds the grasping window after compensation, return to S5 to select alternative candidates or return to S4 to re-acquire. S7. Generate the robot pose based on the compensated robot base coordinate position and the predicted end angle, and control the robot to complete the grasping, transfer and directional release of the flexible and slender target to be grasped.
[0024] Specifically, the multi-task model extracts shared features from an RGB image containing a flexible, slender target, which has a main body extending along its length and distinguishable first and second ends. Based on the shared features, it predicts a rotated bounding box with 180° equivalence and a two-dimensional unit direction vector representing the true end orientation of the flexible, slender target. During the training phase, signed direction supervision is applied to the two-dimensional unit direction vector, and a sign-insensitive collinearity constraint is applied between the two-dimensional unit direction vector and the major axis of the rotated bounding box. During the inference phase, the undirected geometric meaning of the major axis of the rotated bounding box is preserved, and the first and second ends of the flexible, slender target are distinguished by the two-dimensional unit direction vector. The two-dimensional unit direction vector is normalized and then subjected to arctangent operation to obtain the true end orientation angle of 0-360°.
[0025] Furthermore, the annotation includes: annotating the undirected rotation target bounding box, category label, real end-direction unit vector that can distinguish the first end and the second end, and manually verified image capture points. The real end-direction is directed from one end to the other according to a preset endpoint semantic and is normalized so that the undirected rotation target bounding box, end-direction and manually verified capture points of the same instance are kept in a one-to-one correspondence.
[0026] Furthermore, extracting the target contour for each target instance in the training image includes: For each target instance, a binary foreground mask is obtained by using HSV threshold segmentation, grayscale threshold segmentation, or a learned segmentation network within the corresponding undirected rotating target bounding box region. Background fragments are then removed by connected component preservation, hole filling, and morphological opening and closing operations to obtain the target contour.
[0027] Furthermore, the calculation of the peak field, skeleton soft region field, and context-gated field of the manually verified capture points includes: For multi-scale candidate anchor points that match the target instance, a regular sampling grid with an odd side length is set within a preset range of the multi-scale candidate anchor points, and the sampling position of each image is determined by the anchor point, grid offset, sampling interval and feature layer step size. Based on the manually verified image capture point, the peak radius of the manually verified point, and the image sampling position, calculate the peak field centered on the manually verified capture point; The soft region field of the skeleton is calculated using the minimum Euclidean distance from the image sampling position to each line segment of the longest main skeleton polyline; The context gating field is calculated based on the manually verified image capture point, the context gating radius of the manually verified point, and the image sampling position.
[0028] Specifically, for a flexible, slender target instance in the training image, manually verified grasping points and the longest main skeleton extracted from the target contour are obtained. The flexible, slender target has a main body extending along its length. A point peak field is constructed based on the manually verified grasping points, a main skeleton soft region field is constructed based on the longest main skeleton, and a context-gated field centered on the manually verified grasping points is constructed. The point peak field and the main skeleton soft region field constrained by the context-gated field are fused to obtain the target grasping field. The grasping field prediction network is trained using the target grasping field. During the inference phase, the grasping field prediction network outputs field logits corresponding to multiple image sampling positions based on the RGB image. The field logits are normalized to a grasping field probability distribution, and the multiple image sampling positions are probability-weighted to obtain continuous image coordinate grasping points, wherein the image coordinate grasping points do not depend on the geometric inverse transformation of the predicted rotating target box.
[0029] Furthermore, during the training of the multi-task model, a total loss consisting of the sum of rotated bounding box regression loss, classification loss, distribution regression loss, signed orientation loss, weak alignment loss of the rotated bounding box OBB axis, cross-entropy between the target field and the prediction field, and smooth regression loss of the decoding point is used for training. The signed orientation supervision includes at least orientation cosine loss, and the sign-insensitive collinearity constraint includes at least an axis alignment loss constructed based on the absolute value of the dot product of the two-dimensional unit orientation vector and the major axis of the rotated bounding box.
[0030] Specifically, the multi-task model for decoupling the orientation of flexible, slender targets includes: extracting shared features from an RGB image containing the flexible, slender target, which has a main body extending along its length and distinguishable first and second ends; predicting, based on the shared features, a rotated bounding box with 180° equivalence and a two-dimensional unit orientation vector representing the true orientation of the flexible, slender target; during the training phase, applying signed orientation supervision to the two-dimensional unit orientation vector and applying a sign-insensitive collinearity constraint to the two-dimensional unit orientation vector and the major axis of the rotated bounding box; and during the inference phase, preserving the undirected geometric meaning of the major axis of the rotated bounding box and distinguishing the first and second ends of the flexible, slender target by the two-dimensional unit orientation vector.
[0031] Furthermore, preprocessing of the RGB image and encoder position of the target to be grasped includes: The correspondence between the RGB image and the encoder position is established by hardware triggering and unified clock or timestamp registration. The controller simultaneously loads the calibration matrix from the RGB image plane to the transport plane, the rigid body transformation from the transport coordinate system to the robot base coordinate system, the unit vector of the transport direction, and the end-effector mounting offset, and performs scaling, normalization, and necessary color preprocessing on the RGB image.
[0032] Furthermore, the decoded capture points undergo coordinate calibration transformation and encoder dynamic compensation, including: The decoded grab points are transformed into homogeneous coordinates, multiplied with the homography matrix from the RGB image plane to the transport plane, and the result of the multiplication is homogenized and embedded into the mapping of the three-dimensional transport coordinate system to obtain the three-dimensional coordinates of the grab points in the transport coordinate system. The controller estimates the time of arrival at the grasping point based on robot scheduling and motion planning, and calculates the cumulative encoder displacement increment from the image acquisition time to the expected grasping time; Based on the three-dimensional coordinates of the gripping point in the transport coordinate system and the cumulative displacement increment of the encoder, the target position is compensated to the gripping moment by a unit vector along the transport direction, and transformed to the robot base coordinate system to obtain the compensated robot base coordinate position. If the compensated robot base coordinate position has exceeded the gripping window, return to S5 to select an alternative candidate or return to S4 to re-acquire.
[0033] Furthermore, generating the robot pose based on the compensated robot base coordinate position and the predicted end-angle includes: Using the compensated robot base coordinate position as the gripper target position, the predicted end angle is added to the end mount offset to obtain the target rotation angle; The controller normalizes the target rotation angle to an equivalent angle and selects the same-direction equivalent angle based on the gripper's mechanical limit. The motion planner generates the robot pose by combining the safety height, gripper opening, and target local main direction. If the pose is unreachable or a collision occurs, the system switches to the next candidate.
[0034] Furthermore, controlling the robot to grasp, transfer, and release the flexible, slender target includes: If the grasping, transfer, and directional release all meet the preset success conditions, then the instance identifier, grasping position, target orientation, encoder compensation amount, robot arrival time, and deployment result of the successful target are recorded. The occupied robot execution window is released, and the scheduling queue is updated according to the expected arrival order of subsequent candidates on the conveyor belt. Then, the process returns to S4 to process the next acquisition cycle. If a false grab, slippage, fall, attitude deviation, or release failure is detected, the failure type and the confidence level, grab point, end orientation, encoder compensation amount, and execution status of the corresponding candidate are recorded. The current candidate is temporarily downgraded, skipped, or added to the review queue. If there are still candidate candidates that meet the time window in the current frame, return to S5 to reorder; otherwise, return to S4 to enter the next acquisition cycle.
[0035] Example 2: Figure 2 S101-S104 represent the offline training phase, where the contour and longest main skeleton are only involved in the construction of the supervised target during this phase. S201-S210 represent the online inference and robot execution phase, where the inference input includes only the RGB image, encoder state, model parameters, and pre-calibrated coordinate relationships. Specifically, this includes: S101 Collect and label the OBB, end directions, and manual grasping points. Collect training images containing flexible, slender targets, establish a unique instance index for each target instance, and label the undirected rotated target box (OBB), category label, real end direction unit vector that distinguishes the first and second ends, and manual verification image grasping points. The real end direction points from one end to the other according to the preset endpoint semantics and is normalized, so that the OBB, end direction, and manual verification grasping points of the same instance maintain a one-to-one correspondence, providing a supervised basis for the joint output shown in Equation (1).
[0036] For example The combined output is written as: (1); Directional branch first outputs the original two-dimensional data. Normalization, and obtaining the directed angle through the arctangent: (2); (3); The signed direction loss and the OBB axis weak alignment loss are respectively: (4); (5); The absolute value in equation (5) is only used to preserve the 180° equivalence of the OBB axis. The distinction between the shoot tip and the root tip is determined by... Responsible.
[0037] Examples, detections, and direction symbols are shown in Table 1.
[0038] Table 1 S102 Extract the contour and longest main skeleton. For each target instance obtained in S101, a binary foreground mask is obtained in the corresponding OBB region using HSV threshold segmentation, grayscale threshold segmentation, or a learned segmentation network. Background fragments are removed by connected component preservation, hole filling, and morphological opening and closing operations to obtain the target contour. Then, morphological thinning or skeletonization is performed on the foreground mask. The skeleton pixels are used as nodes, and the eight-neighbor connectivity is used as edges. Short branches with a length lower than a preset threshold are deleted and the endpoints are determined. The geodesic path along the skeleton between the candidate endpoints is calculated, and the connected path with the largest geodesic length is selected as the longest main skeleton polyline used in Equation (8). The soft region radius of the skeleton used in Equation (8) is determined according to the local width of the target contour, the short side of the OBB, or a combination of the two. If the target instance cannot form an effective longest main skeleton, a degradation flag is recorded, so that the subsequent target grabbing field degrades to the peak field of the manually verified grabbing point.
[0039] No. Each image sampling location is determined by the anchor point, regular offset, and feature layer stride: (6); The artificial point peak field, the skeleton soft region field, and the context-gated field are as follows: (7); (8); (9); in, Let be the minimum Euclidean distance from the sampling point to each line segment of the longest main skeleton. The target field and its normalized probability are: (10); (11); The model's predicted probability and soft-argmax are the key points: (12); (13); The capture field loss consists of the cross-entropy between the target field and the prediction field, as well as the smooth regression loss at the decoding points. (14); (15); The field symbols are shown in Table 2: Table 2 S103 Constructs point peaks, skeleton soft regions, and context gating. For multi-scale candidate anchor points matching the target instance, a regular sampling grid with an odd side length of K is set near them, and the sampling positions of each image are determined according to Equation (6) by the anchor point, grid offset, sampling interval, and feature layer step size. The point peak field centered on the manually verified grab point is calculated according to Equation (7), the skeleton soft region field is calculated according to Equation (8) using the minimum Euclidean distance from the sampling position to each line segment of the longest main skeleton polyline, and the context gating field that restricts the scope of the skeleton prior is calculated according to Equation (9). Then, the three fields are fused according to Equation (10) to obtain the unnormalized target grab field, and then normalized to the target field probability according to Equation (11). For instances marked as having no effective main skeleton in S102, the skeleton soft region term is set to zero, so that the unnormalized target grab field is determined only by the point peak field.
[0040] S104 trains the OBB, orientation, and grasping field multi-task model. The OBB, orientation, and manually verified grasping point annotations from S101, as well as the target field probability from S103, are input into the training process. The backbone network and multi-scale neck extraction features are shared. The OBB branch generates the detection loss in Equation (16); the orientation branch normalizes the original two-dimensional output according to Equation (2) and obtains the 0-360° directed angle according to Equation (3). The signed orientation loss of Equation (4) is used to distinguish the semantics at both ends, and the absolute dot product loss of Equation (5) is used to keep it collinear with the undirected major axis of OBB; the grasping field branch outputs the field logit of each sampling position, obtains the prediction probability according to Equation (12), and decodes the continuous image grasping points according to Equation (13). Then, the field cross-entropy loss and decoding point loss are calculated by Equations (14) and (15). The various losses are weighted and summed according to Equation (16) and backpropagated. After training, the network parameters are fixed; the contour, skeleton, and manually verified points are not used as online inference input.
[0041] The total loss is: (16); The image capture point is first transformed to the transport plane, then incrementally compensated according to the encoder to the capture time, and finally transformed to the robot's base coordinate system. (17); (18); (19); (20); Equation (19) uses unit vectors Explicitly representing any conveying direction; when the conveying direction coincides with the first axis of the conveying coordinate system. The final target rotation angle is: (twenty one); The controller can The equivalent angle is normalized, and the same-direction equivalent angle is selected according to the mechanical limit of the gripper.
[0042] The total loss, coordinate transformation, and dynamic compensation symbols are shown in Table 3.
[0043] Table 3 The S201 synchronously acquires RGB images and encoder positions. During online execution, the industrial camera acquires RGB images at the moment of image acquisition, and simultaneously reads and latches the cumulative displacement of the conveyor encoder at that moment. The correspondence between the image and the conveyor position is established through hardware triggering, unified clock, or timestamp registration. The controller simultaneously loads the calibration matrix from the image plane to the conveyor plane, the rigid body transformation from the conveyor coordinate system to the robot base coordinate system, the unit vector of the conveyor direction, and the end effector mounting offset. After scaling, normalizing, and performing necessary color preprocessing on the RGB image, it is sent to the vision model.
[0044] S202 extracts multi-scale features with axis enhancement. A shared backbone network extracts semantic features at different resolutions from RGB images, fusing deep semantic information with shallow spatial details at the multi-scale neck. An axis-aware feature enhancement module located in one or more fusion layers first compresses channels, then extracts the slender main axis, ends, and local bending responses using horizontal depth convolution, vertical depth convolution, and dilated depth convolution, respectively. After fusion of each branch, gating weights are generated through channel average pooling, max pooling, and spatial convolution, and multi-scale features are enhanced through residual connections for sharing by the subsequent three task branches.
[0045] S203 jointly predicts the OBB, 0-360° orientation, and grasping field. Under the same multi-scale candidate location and the same instance assignment relationship, the OBB branch outputs the class confidence and rotation box parameters, the orientation branch outputs the original two-dimensional orientation vector, and the grasping field branch outputs the field logits corresponding to K² sampling locations. The orientation vector is normalized according to Equation (2) and converted into 0-360° orientation angles with endpoint semantics according to Equation (3). The three branches share the candidate index and instance index, so the predicted box, orientation, and grasping field naturally correspond to the same target, and together with the grasping point decoding result of S205, they form the joint output shown in Equation (1); this step does not extract the online contour or skeleton.
[0046] S204 performs bounding box suppression and candidate filtering. First, based on the rotation IoU, non-maximum suppression of the bounding box is performed on overlapping OBBs, and the endocentric and grasping field logits are simultaneously retained in the same instance record to avoid mismatch of the three types of outputs. Then, visual filtering is performed based on class confidence, OBB length and width range, endocentric stability, target occlusion degree, and adjacent backbone interference degree. Feasibility filtering and ranking are further combined with the grasping point's supported area, robot reachability domain, expected target arrival time, and safety window. Candidates that are outside the workspace, have left the grasping window by the expected arrival time, or pose a significant collision risk are downgraded or eliminated.
[0047] Determine if there is an executable candidate in D1. If no executable candidate is found in S204, record the reason for no candidate and return to S201 to enter the next acquisition cycle; if there are one or more executable candidates, select the current candidate according to the sorting result and enter S205, while retaining the instance correspondence between its OBB, end direction and grab field.
[0048] S205 decodes the image capture points using soft-argmax. For the current candidate, K² image sampling positions are recovered according to equation (6). The field logits output by S203 are converted into predicted probabilities using temperature softmax according to equation (12). Then, the sampling positions are weighted and summed according to equation (13) to obtain continuous capture points in the coordinate system of the entire input image. This decoding process does not use the predicted OBB center, scale, and angle for geometric inverse transformation, thereby avoiding the secondary propagation of OBB prediction errors to the capture points. If the probability is invalid or the decoded point exceeds the supported area, the current candidate is skipped and the next candidate is tried.
[0049] S206 completes coordinate calibration transformation and encoder dynamic compensation. The image capture points obtained in S205 are written as homogeneous coordinates, and the position of the transport plane at the acquisition time is obtained by using the homography matrix from the image plane to the transport plane and dehomogeneous mapping according to Equation (17). The controller estimates the time of arrival at the capture point based on robot scheduling and motion planning, calculates the cumulative displacement increment of the encoder from the image acquisition time to the expected capture time according to Equation (18), compensates the target position to the capture time by unit vector along the transport direction according to Equation (19), and finally transforms it to the robot base coordinate system according to Equation (20). If the target has exceeded the capture window after compensation, return to S204 to select alternative candidates or return to S201 to re-acquire. The target position refers to the position of the Chinese medicinal herb seedling that the robot is about to capture.
[0050] S207 generates the gripping position and end effector rotation angle. The compensated robot base coordinate position is used as the target position of the gripper. The predicted end angle is added to the end effector mounting offset according to Equation (21) to obtain the target rotation angle. The equivalent angle is normalized while keeping the end effector semantics unchanged. An executable equivalent angle in the same direction is selected based on the end effector mechanical limit. The motion planner generates the approach pose, gripping pose, and departure pose by combining the safety height, gripper opening, and target local main direction. If the pose is unreachable or a collision occurs, the next candidate is switched.
[0051] S208 Descending, gripping, lifting, transferring, and directional release. The robot descends to the target position according to the trajectory generated in S207, allowing the flexible gripper to grip the target trunk near the decoding gripping point. After gripping, it is lifted and transferred to the delivery area. During the transfer or after reaching the delivery area, the end effector rotates to the target rotation angle, and the controller releases the target according to the preset seedling orientation, plant spacing, or process spacing. Successful gripping and delivery can be determined by at least one of the following: gripper position, gripping force, vacuum pressure, or downstream visual confirmation.
[0052] Determine whether D2 execution was successful. If the grab, transfer, and directional release all meet the preset success conditions, proceed to S209; if an empty grab, slippage, fall, attitude deviation, or release failure is detected, proceed to S210.
[0053] S209 Record results and update the schedule. Record the instance identifier, grasping position, target orientation, encoder compensation amount, robot arrival time and deployment result of the successful target, release the occupied robot execution window, and update the scheduling queue according to the expected arrival order of subsequent candidates on the conveyor belt. Then return to S201 to process the next acquisition cycle.
[0054] S210 marks empty captures or drops and adjusts the candidate strategy. It records the failure type and the corresponding candidate's confidence level, capture point, orientation, dynamic compensation amount, and execution status. The current candidate is temporarily downgraded, skipped, or added to the review queue. If there are still candidate samples that meet the time window requirements in the current frame, it returns to S204 for reordering; otherwise, it returns to S201 to enter the next acquisition cycle. Failure records can be used for subsequent threshold calibration, hard sample supplementation, and offline model retraining, but are not used for contour or skeleton extraction in this online execution.
[0055] Thus, S101-S104 form an offline algorithm of "instance annotation - contour and longest main skeleton extraction - target grasping field construction - multi-task model training", while S201-S210 form an online execution closed-loop process of "synchronous acquisition - joint prediction - candidate screening - grasping point decoding - dynamic compensation - robot grasping and directional deployment".
[0056] The preferred range of parameters in this embodiment is shown in Table 4.
[0057] Table 4 Example 3: This embodiment uses bare-root seedlings of root-type Chinese medicinal herbs as the object; in an optional embodiment, the seedlings are at least one of Codonopsis pilosula, Astragalus membranaceus, and Angelica sinensis. 1536 RGB images of the conveyor belt scene were collected, and 2219 seedling heads (OBBs), their corresponding 0-360° true orientations, and manually verified capture points were labeled. The training set, validation set, and test set contained 1075, 231, and 230 images, respectively. Color thresholding, morphological processing, and skeletonization were performed on each target contour, and the longest main skeleton was extracted; 2213 out of the 2219 instances formed effective main skeleton regions, while instances that could not form effective skeletons were only processed using manually generated peak fields.
[0058] This embodiment uses a 640×640 input. , , Pixels Pixels , , , and To maintain the geometric correspondence between OBB, end orientation, manually grabbed points, and skeleton polylines, unverified random rotations, translations, scaling, shearing, perspective transformations, flipping, and image stitching enhancements are disabled. The above settings are merely preferred examples and do not constitute a limitation on the scope of protection.
[0059] Three random seeds were used for actual offline repeated training. The complete configuration, including the skeleton-aware grasping field and AAFEM, achieved a detection coverage of 96.01% ± 0.17% on the independent test set, a mean absolute error of 14.625 ± 0.239 pixels for conditional grasping points, 60.18% ± 0.79% for end-to-end Acc@15, 0.7248 ± 0.0017 for OBB mAP50-95, a mean absolute error of 3.645° ± 0.141° for orientation, and a single-graph inference time of 7.572 ± 0.080 ms. These results demonstrate that the structure of this invention is trainable and inferable, and can generate detection, orientation, and grasping point outputs within the same model; the values can vary under different datasets and hardware.
[0060] The industrial camera is fixed above the conveyor belt and operates at all times. Acquire RGB images, encoder records synchronously After the model outputs candidate instances, the controller discards those with abnormal dimensions, gripping points outside the gripping area, expected arrival times outside the robot's working window, or severe interference with adjacent targets. For the selected instances, the controller will... Transform to the transport plane and estimate based on the robot trajectory. Read or predict The motion displacement is compensated according to equations (17) to (20). The end-rotator mechanism is aligned with the direction of the gripper opening and the target spindle and the predetermined tip direction according to equation (21), and the robot sequentially performs descent, gripping, lifting, transfer and release.
[0061] In a joint debugging session of a complete robotic system, validation was conducted for normal single-target, multi-target adjacent, curved or branching seedlings, and mixed continuous operations. In a mixed continuous operation of 300 seedlings, 281 seedlings were successfully grasped and transferred, achieving an overall success rate of 93.7% and an average cycle time of 1.47 seconds per seedling. These results illustrate the engineering feasibility of the overall process of "visual perception—dynamic compensation—robot execution—directional release."
[0062] The longest main skeleton can be obtained from HSV, grayscale, learned segmentation mask, or depth profile; skeleton soft region radius. The context-gated field can be determined by the short side of the target OBB, the local width of the contour, or the distance transformation statistics. The context-gated field is not limited to the Gaussian form; it can also use truncated Gaussians, generalized Gaussians, cosine windows, or tightly supported radial basis functions, as long as it can preserve the skeleton prior and suppress the probabilistic quality of the distant skeleton within the neighborhood of the manually grasped point.
[0063] The capture field can be constructed on a regular square grid, a polar coordinate sampling grid, or a learnable offset sampling position; the decoding method can be soft-argmax, temperature-adjustable probability centroid, or local peak refinement while maintaining differentiability. Regardless of the equivalent form used, it is preferable to maintain the four technical relationships of "image coordinate representation, accurate peak values of artificial points, soft priors of local structure, and separation of training and inference paths".
[0064] The shared feature network is not limited to a specific version of YOLO and can be replaced with other single-stage detectors, two-stage rotated box detectors, or hybrid networks of convolution and Transformer. AAFEM can be set at all or part of the neck scale, and horizontal and vertical depth convolutions can be extended to multi-angle separable convolutions, as long as they form orientation-sensitive enhancements to slender principal axes, ends, and curved structures.
[0065] The robot can be a Delta parallel robot, a SCARA robot, or a six-axis robotic arm; the gripper can be a flexible two-finger gripper, a three-finger gripper, or a suction-grip composite end effector. Dynamic compensation can be achieved by directly using the encoder to accumulate displacement, or by incorporating acceleration models, timestamp interpolation, or multi-frame state estimation when the conveying speed changes. Targeted delivery can be used for seedlings facing outwards, seedlings facing the same side, arranged at preset plant spacing, entering the seedling rolling mechanism, or entering the subsequent transplanting seedling supply mechanism.
[0066] Example 4: like Figure 1The system illustrates a seedling grasping and directional delivery system for a directional decoupling and skeleton-sensing grasping field, used to implement a seedling grasping and directional delivery method for a directional decoupling and skeleton-sensing grasping field, comprising: The data acquisition and synchronization module records the RGB image and encoder position at the moment of image acquisition, ensuring that the position of the visual target and the movement of the conveyor belt share a common time reference. The camera can be fixedly mounted above the conveyor belt, and the encoder is connected to the conveyor drive mechanism. The synchronization information is used to subsequently calculate the robot's actual grasping position from the acquisition position.
[0067] The shared feature extraction and multi-scale fusion module is used to extract semantic features of different resolutions from RGB images and fuse deep semantic information with shallow spatial details. Specifically, the shared feature extraction module extracts semantic features at different resolutions from RGB images, while the multi-scale neck module fuses deep semantic information with shallow spatial details, providing unified features for small-sized saplings, curved ends, and densely packed multiple targets. The backbone network can employ convolutional networks, visual Transformers, or a combination of both; the number of multi-scale layers and channels can be adjusted based on computing power.
[0068] The Axis-Aware Feature Enhancement (AAFEM) module is positioned at one or more feature fusion locations on the multi-scale neck. This module first compresses channels, then extracts complementary responses through horizontal, vertical, and dilated deep convolutional branches. The horizontal and vertical branches enhance the continuity of the slender principal axis under different poses, while the dilated branch covers local curvature and end neighborhoods. After fusion, the branches are used to generate gated weights through channel average pooling, max pooling, and spatial convolution, which are then residually superimposed with the input features. The module's role is not to restrict the target to only horizontal or vertical orientations, but rather to combine responses under different principal axis poses using lightweight direction-sensitive basis functions.
[0069] The OBB detection branch predicts the center, length, width, and undirected rotation axis of the target, used to compactly describe the gripping area of the seed head. The ground truth apical branch independently predicts a two-dimensional unit direction vector, used to express the directed semantics from near the gripping point to the seed head or from the root to the seed head. During training, the direction branch receives signed apical supervision; simultaneously, the absolute dot product constraint ensures that the direction is collinear with the OBB major axis, making the 180° equivalence of the OBB axis a weak geometric prior, rather than replacing the apical supervision. Thus, the same target simultaneously possesses two complementary representations: an "undirected geometric box" and a "directed apical axis."
[0070] The image coordinate skeleton perception grasping field module is used to extract the target contour for each target instance in the training image, calculate the peak field, skeleton soft region field and context gating field of the manually verified grasping point, and fuse and normalize them to obtain the target field probability. If the target instance cannot form an effective longest main skeleton, the target field probability is determined by the peak field of the manually verified grasping point. Specifically, the image coordinate skeleton-aware grasping field module establishes a discrete sampling grid with odd-numbered side lengths near each multi-scale candidate anchor point and predicts a field logit for each sampling location. The training target consists of three parts: a point peak field centered on the manually verified grasping point, a skeleton soft region representing the distance from the sampling point to the longest main skeleton, and a context gating that restricts the scope of the skeleton prior centered on the manually verified point. These three are fused and normalized to form the target probability field. During inference, the grasping field logits are transformed into a probability distribution through temperature softmax, and then soft-argmax is performed on the probability weighted sum of each sampling location to directly obtain continuous grasping points in the coordinate system of the entire input image. These grasping points do not undergo the geometric inverse transformation of the predicted OBB, and therefore do not inherit the center, scale, and angle errors of the predicted OBB. The manually created points, contours, and skeletons are only used for constructing the training target and do not enter the inference path.
[0071] The instance association and candidate filtering module is used to perform rotation box suppression and executability candidate filtering based on the rotation target box, end angle and grab field of the same instance correspondence. If there is an executable candidate, the grab point of the current candidate decoded image is obtained. If no executable candidate is obtained, the reason for no candidate is recorded and the next acquisition cycle is entered. Specifically, the OBB, true end-point orientation, and grasping field are generated from the same multi-scale candidate positions and shared instance allocation relationships, thus naturally corresponding to the same target. The candidate selection module first performs non-maximum suppression of the bounding box, and then combines category confidence, OBB size, whether the grasping point is located in the gripping area, robot reachability, candidate arrival time, and occlusion or adjacent backbone interference between targets for selection and sorting. For candidates that are outside the workspace, have left the window by the expected arrival time, have unstable end-point orientation, or whose grasping point is not executable, their execution priority is reduced or the current frame is skipped.
[0072] The coordinate transformation and dynamic compensation module is used to perform coordinate calibration transformation and encoder dynamic compensation on the decoded grasping points to obtain the robot's base coordinate position after compensation. If the target exceeds the grasping window after compensation, it returns to the instance association and candidate filtering module to select alternative candidates or enters the data acquisition and synchronization module to re-acquire data. Specifically, the coordinate transformation module uses homography transformation from the camera imaging plane to the conveyor plane or a camera calibration model to transform the image capture points to the conveyor belt plane coordinates, and then uses rigid body transformation from the conveyor belt coordinate system to the robot's base coordinate system to obtain the robot's target position. The dynamic compensation module corrects the target position along the conveying direction based on the encoder displacement increment between the image acquisition time and the robot's expected arrival time. Compensation can use the encoder's accumulated displacement, or it can be predicted by combining the conveyor speed, acceleration, and the robot's expected motion time.
[0073] The robot grasping and directional delivery module is used to generate the robot pose based on the compensated robot base coordinate position and the predicted end angle, and control the robot to complete the grasping, transfer and directional release of the flexible and slender target to be grasped.
[0074] Specifically, the robot's grasping and directional delivery module generates descent, gripping, lifting, and transfer trajectories based on the compensated position. The end effector rotation mechanism adjusts the gripper opening direction or delivery angle according to the actual end orientation, and the flexible gripper grips the target trunk near the grasping point. Upon reaching the delivery area, the system releases the target according to a preset orientation, plant spacing, or process spacing, ensuring the seedling head faces the designated direction. Damage to flexible seedlings can be reduced through elastic connections, force limiting, or gripper opening restrictions.
[0075] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.
Claims
1. A method for seedling grasping and directional delivery in a directional decoupling and skeleton-sensing grasping field, characterized in that, include: S1. Obtain an RGB image containing a flexible and slender target, establish a unique instance index for each target instance and label it, and construct a training image, wherein the flexible and slender target is a bare-root seedling of a root-type Chinese medicinal herb; S2. Extract the target contour for each target instance in the training image, calculate the peak field, skeleton soft region field and context gating field of the manually verified grab point, and fuse and normalize them to obtain the target field probability. If the target instance cannot form an effective longest main skeleton, the target field probability is determined by the peak field of the manually verified grab point. S3. Input the training image and the target field probability into a multi-task model of rotating target box, apical angle and grasping field, and train the multi-task model, wherein the multi-task model extracts and fuses features through a shared backbone network and multi-scale neck; S4. Simultaneously acquire the RGB image of the flexible and slender target to be grasped and the encoder position, and after preprocessing, input it into the trained multi-task model to predict the rotation target box, end angle and grasping field of the corresponding relationship of the same instance. S5. Based on the rotation target box, apical angle and grab field corresponding to the same instance, perform rotation box suppression and executability candidate screening. If there is an executable candidate, proceed to S6 to grab the current candidate decoding image point. If no executable candidate is obtained, record the reason for no candidate and return to S4 to enter the next acquisition cycle. S6. Perform coordinate calibration transformation and encoder dynamic compensation on the decoded grasping points to obtain the compensated robot base coordinate position. If the target exceeds the grasping window after compensation, return to S5 to select alternative candidates or return to S4 to re-acquire. S7. Generate the robot pose based on the compensated robot base coordinate position and the predicted end angle, and control the robot to complete the grasping, transfer and directional release of the flexible and slender target to be grasped.
2. The seedling grasping and directional delivery method in the orientation decoupling and skeleton sensing grasping field according to claim 1, characterized in that, The annotation process includes: annotating the undirected rotating target bounding box, category labels, real end-direction unit vectors that can distinguish the first and second ends, and manually verified image capture points. The real end-direction points are directed from one end to the other according to preset endpoint semantics and are normalized to ensure that the undirected rotating target bounding box, end-direction, and manually verified capture points of the same instance maintain a one-to-one correspondence.
3. The seedling grasping and directional delivery method in the orientation decoupling and skeleton sensing grasping field according to claim 1, characterized in that, Extracting the target contour for each target instance in the training image includes: For each target instance, a binary foreground mask is obtained by using HSV threshold segmentation, grayscale threshold segmentation, or a learned segmentation network within the corresponding undirected rotating target bounding box region. Background fragments are then removed by connected component preservation, hole filling, and morphological opening and closing operations to obtain the target contour.
4. The seedling grasping and directional delivery method in the orientation decoupling and skeleton sensing grasping field according to claim 2, characterized in that, The calculation of the peak field, skeleton soft region field, and context-gated field of manually verified capture points includes: For multi-scale candidate anchor points that match the target instance, a regular sampling grid with an odd side length is set within a preset range of the multi-scale candidate anchor points, and the sampling position of each image is determined by the anchor point, grid offset, sampling interval and feature layer step size. Based on the manually verified image capture point, the peak radius of the manually verified point, and the image sampling position, calculate the peak field centered on the manually verified capture point; The soft region field of the skeleton is calculated using the minimum Euclidean distance from the image sampling position to each line segment of the longest main skeleton polyline; The context gating field is calculated based on the manually verified image capture point, the context gating radius of the manually verified point, and the image sampling position.
5. The seedling grasping and directional delivery method in the orientation decoupling and skeleton sensing grasping field according to claim 1, characterized in that, During the training of the multi-task model, a total loss consisting of the sum of the rotated target box regression loss, classification loss, distribution regression loss, signed orientation loss, weak alignment loss of the rotated target box OBB axis, cross-entropy between the target field and the prediction field, and smooth regression loss of the decoding point is used for training.
6. The seedling grasping and directional delivery method in the orientation decoupling and skeleton sensing grasping field according to claim 1, characterized in that, Preprocessing of the RGB image and encoder position for grasping flexible, slender targets includes: The correspondence between the RGB image and the encoder position is established by hardware triggering and unified clock or timestamp registration. The controller simultaneously loads the calibration matrix from the RGB image plane to the transport plane, the rigid body transformation from the transport coordinate system to the robot base coordinate system, the unit vector of the transport direction, and the end-effector mounting offset, and performs scaling, normalization, and necessary color preprocessing on the RGB image.
7. The seedling grasping and directional delivery method in the orientation decoupling and skeleton sensing grasping field according to claim 1, characterized in that, The process of performing coordinate calibration transformation and encoder dynamic compensation on the decoded capture points includes: The decoded grab points are transformed into homogeneous coordinates, multiplied with the homography matrix from the RGB image plane to the transport plane, and the result of the multiplication is homogenized and embedded into the mapping of the three-dimensional transport coordinate system to obtain the three-dimensional coordinates of the grab points in the transport coordinate system. The controller estimates the time of arrival at the grasping point based on robot scheduling and motion planning, and calculates the cumulative encoder displacement increment from the image acquisition time to the expected grasping time; Based on the three-dimensional coordinates of the gripping point in the transport coordinate system and the cumulative displacement increment of the encoder, the target position is compensated to the gripping moment by a unit vector along the transport direction, and transformed to the robot base coordinate system to obtain the compensated robot base coordinate position. If the compensated robot base coordinate position has exceeded the gripping window, return to S5 to select an alternative candidate or return to S4 to re-acquire.
8. The seedling grasping and directional delivery method in the orientation decoupling and skeleton sensing grasping field according to claim 1, characterized in that, Generating the robot pose based on the compensated robot base coordinate position and the predicted end angle includes: Using the compensated robot base coordinate position as the gripper target position, the predicted end angle is added to the end mount offset to obtain the target rotation angle; The controller normalizes the target rotation angle to an equivalent angle and selects the same-direction equivalent angle based on the gripper's mechanical limit. The motion planner generates the robot pose by combining the safety height, gripper opening, and target local main direction. If the pose is unreachable or a collision occurs, the system switches to the next candidate.
9. The seedling grasping and directional delivery method in the orientation decoupling and skeleton sensing grasping field according to claim 1, characterized in that, Controlling the robot to grasp, transfer, and release the flexible, slender target includes: If the grasping, transfer, and directional release all meet the preset success conditions, then the instance identifier, grasping position, target orientation, encoder compensation amount, robot arrival time, and deployment result of the successful target are recorded. The occupied robot execution window is released, and the scheduling queue is updated according to the expected arrival order of subsequent candidates on the conveyor belt. Then, the process returns to S4 to process the next acquisition cycle. If a false grab, slippage, fall, attitude deviation, or release failure is detected, the failure type and the confidence level, grab point, end orientation, encoder compensation amount, and execution status of the corresponding candidate are recorded. The current candidate is temporarily downgraded, skipped, or added to the review queue. If there are still candidate candidates that meet the time window in the current frame, return to S5 to reorder; otherwise, return to S4 to enter the next acquisition cycle.
10. A seedling grasping and directional delivery system for a directional decoupling and skeleton sensing grasping field, used to implement the seedling grasping and directional delivery method for a directional decoupling and skeleton sensing grasping field as described in any one of claims 1-9, characterized in that, include: The data acquisition and synchronization module is used to simultaneously acquire the RGB image of the flexible and slender target to be grasped and the encoder position; The shared feature extraction and multi-scale fusion module is used to extract semantic features of different resolutions from RGB images and fuse deep semantic information with shallow spatial details. The image coordinate skeleton perception grasping field module is used to extract the target contour for each target instance in the training image, calculate the peak field, skeleton soft region field and context gating field of the manually verified grasping point, and fuse and normalize them to obtain the target field probability. If the target instance cannot form an effective longest main skeleton, the target field probability is determined by the peak field of the manually verified grasping point. The instance association and candidate filtering module is used to perform rotation box suppression and executability candidate filtering based on the rotation target box, end angle and grab field of the same instance correspondence. If there is an executable candidate, the grab point of the current candidate decoded image is obtained. If no executable candidate is obtained, the reason for no candidate is recorded and the next acquisition cycle is entered. The coordinate transformation and dynamic compensation module is used to perform coordinate calibration transformation and encoder dynamic compensation on the decoded grasping points to obtain the robot's base coordinate position after compensation. If the target exceeds the grasping window after compensation, it returns to the instance association and candidate filtering module to select alternative candidates or enters the data acquisition and synchronization module to re-acquire data. The robot grasping and directional delivery module is used to generate the robot pose based on the compensated robot base coordinate position and the predicted end angle, and control the robot to complete the grasping, transfer and directional release of the flexible and slender target to be grasped.
