A yarn bobbin gripping pose detection method
By introducing a hybrid physical metric evaluation index and a multi-scale feature attention fusion module, the yarn bobbin grasping pose detection method is improved, which solves the problem of low automation in yarn bobbin grasping in textile enterprises, achieves higher accuracy and robustness, and adapts to the diversity of yarn bobbin shapes in complex industrial scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- DONGHUA UNIV
- Filing Date
- 2025-04-22
- Publication Date
- 2026-05-29
AI Technical Summary
Existing yarn bobbin grasping methods in textile enterprises have low automation, poor scene adaptability, and insufficient accuracy. They are unable to cope with the diversity of yarn bobbin shapes and random stacking postures. Furthermore, existing grasping methods have weak generalization ability and cannot handle scenarios with missing or occluded point cloud data.
By employing a hybrid physical metric evaluation index and a multi-scale feature attention fusion module, combined with the GraspNet-baseline network, the accuracy and generalization of yarn tube grasping pose detection are improved through point cloud preprocessing, feature extraction, multi-scale feature attention fusion, and physical metric evaluation.
It improves the accuracy and robustness of yarn bobbin gripping posture detection, can adapt to the diversity of yarn bobbin shapes in complex industrial scenarios, and enhances the comprehensiveness and stability of gripping quality assessment.
Smart Images
Figure CN122115550A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of robot grasping technology, specifically a method for detecting the grasping posture of a yarn tube. Background Technology
[0002] Currently, doffing robots in textile production lines typically use a teaching method when grasping yarn bobbins, resulting in a low level of automation. When the shape, size, or placement environment of the yarn bobbins changes, re-teaching is required. This method is not only inefficient but also significantly increases operational time and labor costs, limiting the overall automation level of textile production lines.
[0003] Besides the teaching method, traditional grasping methods for yarn tube grasping mainly rely on 3D modeling and rule algorithms, which have the following drawbacks: poor scene adaptability, requiring the pre-construction of a complete 3D model, and unable to handle scenes with missing point cloud data or occlusion; weak generalization ability, relying on manually designed grasping rules, making it difficult to cope with the diversity of yarn tube shapes and random stacking postures; and insufficient accuracy, with existing grasping quality evaluation indicators being singular and not integrating physical constraints and geometric features, resulting in poor grasping stability.
[0004] With the rapid innovation of depth sensor technology and the rapid development of deep learning algorithms, deep learning-based object grasping pose detection methods have made significant progress in academic research and industrial applications. One commonly used method is direct regression, which uses a deep learning model to directly predict the grasping pose of the target object. This method outputs grasping pose parameters directly from perceptual data (such as RGB images or point clouds) in an end-to-end manner, simplifying the candidate sampling and evaluation process in traditional grasping systems. Its advantages lie in its fast model inference speed, making it suitable for real-time applications. Its disadvantages include weak generalization ability and unstable output, which may generate infeasible grasping poses.
[0005] Existing regression-based grasping methods (GraspNet-baseline network) can directly regress grasping poses, but they lack the ability to perceive local details in incomplete point clouds and lack quantitative evaluation of the physical feasibility of grasping. Therefore, there is an urgent need for a yarn spool grasping pose detection method that can adapt to complex industrial scenarios. Summary of the Invention
[0006] To address the shortcomings of existing technologies, this invention provides a method for detecting the pose of yarn bobbin grasping. By introducing a hybrid physical metric evaluation index and a multi-scale feature attention fusion module, the performance of the GraspNet-baseline network is further improved, thereby enhancing the accuracy and generalization of yarn bobbin grasping pose prediction.
[0007] To achieve the above objectives, the present invention provides a method for detecting the gripping posture of a yarn bobbin, the method comprising the following steps: The input point cloud data is preprocessed by using Farthest Point Sampling (FPS) to effectively reduce the amount of data while preserving the geometric features of the point cloud.
[0008] Furthermore, the preprocessed point cloud data is fed into a feature extraction network (PointNet++) to extract key features.
[0009] Furthermore, the point cloud data after feature extraction is fed into the ApproachNet module to estimate seed points and proximity vectors.
[0010] Furthermore, the point cloud data is input into the multi-scale feature attention fusion module to obtain key geometric feature information of the yarn tube.
[0011] Furthermore, based on the features generated by the multi-scale feature attention fusion module, the grabbing candidate information is sent to the hybrid physical metric evaluation index module for comprehensive evaluation to help the model more accurately determine which grabbing poses are reasonable and feasible. These key indicators include force closure conditions, flatness, center of gravity position, and collision perturbation.
[0012] Furthermore, the point cloud data is input into the OperationNet module, which ultimately outputs the gripper configuration parameters for grasping the yarn bobbin. These parameters include: gripper rotation angle, gripper opening width, proximity to the object, and confidence level of the grasping operation.
[0013] Preferably, the implementation steps of the multi-scale feature attention fusion module include: a. Perform multi-scale cylindrical neighborhood division on the point cloud features output by the feature extraction network; b. Map local features to a high-dimensional space using a multilayer perceptron (MLP) and calculate feature importance weights using a local attention mechanism; c. Concatenate the features processed at each scale along the channel dimension to generate a comprehensive feature representation containing multi-scale information.
[0014] Preferably, the calculation method of the hybrid physical metric evaluation index includes: a. Force closure score: calculated based on the coefficient of friction; b. Smoothness score: Evaluated by the cosine distance between the contact point normal vector and the normal vectors of adjacent points, as well as the consistency of the reverse vectors; c. Based on the Euclidean distance between the point of contact and the object's center of gravity; d. Collision disturbance score: based on the minimum Euclidean distance between the gripper endpoint and the point of contact with the object.
[0015] Preferably, the gripper configuration parameters are determined in the following manner: a. Rotation angle (R): Determines the rotation angle of the gripper about the approach axis; b. Opening width (W): The gripper width is calculated based on the yarn bobbin diameter and the distance between contact points; c. Approach distance (D): The distance from the object's grasping point to the origin of the gripper coordinate system; d. Confidence: A comprehensive evaluation result combining capture score, rotation classification, proximity distance classification, and width regression.
[0016] Preferably, the loss function of the method consists of three parts: objectivity loss, proximity vector loss, and grasping loss, wherein: a. The object-oriented loss function is used; b. The proximity vector loss uses the mean squared error (MSE) loss function; c. The capture loss includes a weighted combination of capture score regression, rotation classification, proximity distance classification, and width regression.
[0017] Compared with the prior art, the technical solution of this application has the following beneficial effects: By employing multi-scale feature attention fusion, the model's ability to represent features of yarn bobbins of different sizes is enhanced, enabling it to better adapt to the diversity of yarn bobbin shapes and sizes. By integrating multi-dimensional indicators such as force closure and flatness, the comprehensiveness of grasping quality assessment is improved. Even with low-quality point cloud data, more reasonable grasping poses can be generated, significantly improving the accuracy and robustness of yarn bobbin grasping pose prediction. This makes it suitable for grasping diverse yarn bobbin shapes in complex industrial scenarios. Attached Figure Description
[0018] Figure 1 This is a simplified flowchart of the detection method of the present invention. Figure 2 This is a detailed flowchart illustrating the detection method of the present invention; Figure 3 This is a schematic diagram of the multi-scale feature attention fusion module structure of the present invention; Figure 4 This is a schematic diagram of the force-sealing index of the present invention; Figure 5 This is a schematic diagram illustrating the capture of configuration parameters in this invention; Figure 6 This is a schematic diagram illustrating the optimal detection effect of the present invention on cylindrical yarn tubes.
[0019] Figure 7 This is a schematic diagram illustrating the optimal detection effect of the present invention on conical yarn bobbins. Detailed Implementation
[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Example
[0021] like Figure 2 As shown, this embodiment provides a method for detecting the gripping posture of a yarn bobbin, which includes the following steps: Step 1: Preprocess the point cloud data by using farthest point sampling (FPS) to reduce the amount of data while preserving geometric features; Step 2: Extract local and global features of the preprocessed point cloud using the PointNet++ network; Step 3: Use the ApproachNet module to estimate the seed point and proximity vector, and generate the objectivity score and optimal viewpoint orientation; Step 4: Input the point cloud data into the multi-scale feature attention fusion module, and enhance the expression of key geometric features of the yarn tube through cylindrical region query, local attention mechanism and multi-scale feature stitching. Step 5: Evaluate the feasibility of the grasping pose based on hybrid physical metrics (force closure conditions, flatness, center of gravity position, collision disturbance); Step 6: Output the gripper configuration parameters through the OperationNet module, including rotation angle, opening width, approach distance, and confidence level.
[0022] The farthest point sampling (FPS) method is used to preprocess the input point cloud data. The specific implementation of the farthest point sampling method is as follows: Randomly select a point as the initial sampling point, calculate the minimum distance from the remaining points to the selected sampling point, select the point with the largest minimum distance as the new sampling point, and repeat the process until the preset sampling point is reached.
[0023] Furthermore, the preprocessed point cloud data is fed into the feature extraction network PointNet++, which can better capture the local and global features of the point cloud. The specific process is as follows: The coordinates of each point are mapped using a multilayer perceptron (MLP) to convert them into a high-dimensional feature vector. Use Ball Query or K-Nearest Neighbors (KNN) methods to determine the local neighborhood of each point; The local feature aggregation module aggregates features within a local neighborhood to obtain higher-level local features; Repeat the above steps to perform multi-layer feature extraction and obtain a global feature representation containing rich geometric information.
[0024] Furthermore, the point cloud data after feature extraction is fed into the ApproachNet module. The design of ApproachNet combines multi-layer one-dimensional convolution and batch normalization to generate two key pieces of information through a series of feature extraction and aggregation operations: Objectness Score and Best Viewpoint Direction. The Objectness Score is used to evaluate the probability that a seed point belongs to an object, while the Best Viewpoint Direction refers to the optimal grabbing angle or direction around each seed point.
[0025] Furthermore, the point cloud data is input into a multi-scale feature attention fusion module, such as... Figure 3 As shown, the design of this module includes several key steps: cylindrical region query and grouping, MLP feature mapping, local attention mechanism, and multi-scale feature concatenation. Cylindrical region query and grouping: The point cloud features output by the feature extraction network are divided into multi-scale cylindrical neighborhoods, with each scale corresponding to a local region of different radii; Feature enhancement and local attention mechanism: Local features are mapped to a high-dimensional space through MLP, and feature importance weights are calculated by combining local attention mechanism to enhance the features of key regions; Multi-scale feature concatenation: The features processed at each scale are concatenated along the channel dimension to generate a comprehensive feature representation containing multi-scale information.
[0026] Furthermore, the captured candidate information is fed into the hybrid physics metrics evaluation module for comprehensive evaluation, helping the model to more accurately determine which grasping poses are reasonable and feasible. The metrics include force closure conditions, flatness, center of gravity position, and collision perturbation. like Figure 4 As shown, the force closure condition is an important physical constraint for judging the stability of the gripping action. The necessary and sufficient condition for satisfying the force closure condition is that the line connecting the resultant forces on any two contact points on the target object must be located inside the friction cone of the two contact points. The friction cone is determined by the friction coefficient and normal force of the contact points, reflecting the force situation at the contact points and the gripper's ability to constrain the object. The force closure score is calculated as follows: (1) This represents the coefficient of friction; the smaller the coefficient of friction, the higher the score. For the gripping task of parallel grippers, the flatter the contact area, the easier it is to meet the stability and safety requirements of gripping. Therefore, a flatness index is introduced to quantify the flatness of the contact area. The specific calculation of the flatness score is as follows: (2) in V q It is the normal vector of the contact point. V n It is the normal vector of its adjacent points. K It is the number of adjacent points. V a It is the corresponding vector; First, the flatness score of a point is measured by comparing the similarity of local normal vectors within the contact area, denoted as S. f1 The score is calculated using the cosine distance between the contact point normal vector and the normal vectors of adjacent points. Then, the consistency between the reverse vector and the contact point normal vector is calculated using the cosine distance, and this consistency score S is used. f2 The final score S f It is obtained through the multiplication of these two scores; The center of gravity position is another key factor affecting the rationality of the gripping posture. Ideally, the gripping point of the gripper should be as close as possible to the center of gravity of the object to reduce the influence of the gripping force on the object. The Euclidean distance between the point of contact and the center of gravity is used as the center of gravity score: (3) in P cl , P cr , P gc These are the coordinates of the left contact point, the right contact point, and the center of gravity, respectively. Collision disturbance assessment is used to detect potential collision risks during the gripping process, and the minimum Euclidean distance between the gripper tip and the object contact point is expressed as the collision disturbance score; The collision disturbance score is: (4) P el , P er These are the left and right endpoints of the gripper, and the final score is calculated as follows: (5) Furthermore, the OperationNet module outputs grasping configuration parameters to help the robot make accurate grasping decisions. This module accepts multidimensional features extracted from the point cloud and processes them through several convolutional layers and batch normalization layers to output multiple grasping-related gripper operation parameters: rotation angle, width, approach distance, and grasping quality confidence.
[0027] like Figure 5As shown in the figure, V represents the approach vector, D represents the distance from the object gripping point to the origin of the gripper coordinate system, i.e., the approach distance, R: rotation around the approach axis, and W: gripper width. The proposed method belongs to an end-to-end multi-stage network. The loss function mainly consists of three parts: object property calculation loss, proximity vector loss, and grasping loss. Calculating object property belongs to a classification problem, so the cross-entropy loss function can be used. Its calculation formula is as follows: (6) Proximity vector prediction is a regression problem, and its loss function is calculated as follows: (7) In the above formula N Indicates the number of seed points N =1024, V Indicates the number of near vector categories V =300, m i It is a binary label; if the point belongs to an object, it is 1. m̂ i Indicates the predicted value. s ij This represents the view label value with the highest score in each proximity direction. ŝ ij Indicates the predicted value; The calculation of crawling loss is divided into four parts: crawling score regression, rotation classification, proximity distance classification, and width regression. The loss calculation formula is as follows: (8) In the above formula A =12 indicates 12 possible rotation angles. D =4 indicates 4 possible approach distances. s ij This represents 48 candidate score labels. r i This represents the index of the highest score among the 12 rotation angles. d i This represents the index of the highest score among the four proximity distances. w i express r i Directional grab width, s ij , r ij , d ij , w ij These represent the corresponding predicted values, and are set accordingly. α =0.2, β=0.3. The total loss function is as follows: (9) In summary, this invention proposes an improved grasping pose detection method, which enhances the accuracy and robustness of yarn bobbin grasping by combining multi-scale feature fusion and hybrid physical metric evaluation with existing grasping pose detection methods.
[0028] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.
[0029] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A method for detecting the gripping posture of a yarn bobbin, characterized in that, Includes the following steps: Step 1: Preprocess the point cloud data by using farthest point sampling (FPS) to reduce the amount of data while preserving geometric features; Step 2: Extract local and global features of the preprocessed point cloud using the PointNet++ network; Step 3: Use the ApproachNet module to estimate the seed point and proximity vector, and generate the objectivity score and optimal viewpoint orientation; Step 4: Input the point cloud data into the multi-scale feature attention fusion module, and enhance the expression of key geometric features of the yarn tube through cylindrical region query, local attention mechanism and multi-scale feature stitching. Step 5: Evaluate the feasibility of the grasping pose based on hybrid physical metrics (force closure conditions, flatness, center of gravity position, collision disturbance); Step 6: Output the gripper configuration parameters through the OperationNet module, including rotation angle, opening width, approach distance, and confidence level.
2. The detection method according to claim 1, characterized in that, The implementation steps of the multi-scale feature attention fusion module include: a. Perform multi-scale cylindrical neighborhood division on the point cloud features output by the feature extraction network; b. Map local features to a high-dimensional space using a multilayer perceptron (MLP) and calculate feature importance weights using a local attention mechanism; c. Concatenate the features processed at each scale along the channel dimension to generate a comprehensive feature representation containing multi-scale information.
3. The detection method according to claim 1, characterized in that, The calculation methods for the hybrid physical metric evaluation index include: a. Force closure score: calculated based on the coefficient of friction; b. Smoothness score: Evaluated by the cosine distance between the contact point normal vector and the normal vectors of adjacent points, as well as the consistency of the reverse vectors; c. Based on the Euclidean distance between the point of contact and the object's center of gravity; d. Collision disturbance score: based on the minimum Euclidean distance between the gripper endpoint and the point of contact with the object.
4. The detection method according to claim 1, characterized in that, The gripper configuration parameters are determined in the following way: a. Rotation angle (R): Determines the rotation angle of the gripper about the approach axis; b. Opening width (W): The gripper width is calculated based on the yarn bobbin diameter and the distance between contact points; c. Approach distance (D): The distance from the object's grasping point to the origin of the gripper coordinate system; d. Confidence: A comprehensive evaluation result combining capture score, rotation classification, proximity distance classification, and width regression.
5. The detection method according to claim 1, characterized in that, The loss function of the method consists of three parts: objectivity loss, proximity vector loss, and grasping loss, where: a. The object-oriented loss function is used; b. The proximity vector loss uses the mean squared error (MSE) loss function; c. The capture loss includes a weighted combination of capture score regression, rotation classification, proximity distance classification, and width regression.