Grabbing and sampling method based on simple semantic point cloud
By using the simple semantic point cloud method, using RGB and depth images to construct semantic point clouds, the problem of unknown object models and pose collision in robot grasping is solved, and high-quality pose generation and stability of grabbing in complex environments is achieved promote.
Patent Information
- Application Number
- CN202411166660.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-12-27
- Estimated Expiration
- 2044-08-23
AI Technical Summary
In robot crawling tasks, traditional methods require known CAD models, making it difficult to obtain accurate object models in actual scenarios, resulting in unknown objects and large amount of calculations, and insufficient consideration of the influence of surrounding point clouds, which easily leads to the crawling pose collision.
Using a simple semantic point cloud capture sampling method, a semantic point cloud is constructed through RGB images and depth images, and a PCA algorithm is used to solve the normal vector and the main axis vector, establish a grab coordinate system, define constraints to filter candidate poses, and use translation and sum Rotate to generate multiple poses, and finally select the best pose through the confidence function.
Generate high-quality grab sampling poses in unstructured and complex environments, improve the stability of the robotic arm's visual grasp, avoid grab pose collisions, and reduce the calculation amount.
Smart Images

Figure CN119188728A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of robot target object grasping, and in particular to a grasping sampling method based on simple semantic point cloud and a system thereof. Background Art
[0002] With the development of science and technology, robots, as important participants in the field of artificial intelligence, play an important role. Among them, the autonomous grasping operation of robots is a crucial technology, which effectively improves work efficiency and frees human hands. Compared with the grasping of fixed objects and the grasping of fixed positions, it is of great significance for robots to grasp specified objects autonomously and accurately. To complete the robot's autonomous grasping task, the key issue is to know the objects in the task scene and how to detect the grasping posture under different visual information. Therefore, grasping posture estimation is one of the research hotspots in the field of robotics.
[0003] For six-DOF grasping posture detection, the traditional grasping posture estimation method generates grasping posture based on the object posture of the known CAD model. It relies on the three-dimensional model of the object to be grasped, but in actual task scenarios, it is difficult to obtain an accurate CAD model of the target object. Model-free grasping detection does not need to obtain the category and posture of the object. It uses a large amount of data training to extract the characteristics of the object and grasping information, and then directly generates the grasping posture. Among them, the two-stage grasping method, as the mainstream grasping method, includes two steps: grasping sampling and grasping classification. Therefore, improving the candidate posture of the sampled grasp is a prerequisite for ensuring the generation of stable grasps. Object detection is the basis for improving the candidate posture of the sampled grasp. Instance segmentation, as an important part of object detection, naturally needs to be considered. Summary of the invention
[0004] In order to solve the above technical problems, the present invention mainly solves the following technical problems: on the one hand, during grasping detection, all point clouds in the scene will be grasped and sampled, resulting in unknown grasped objects and excessive calculations; on the other hand, when grasping and sampling point clouds, the influence of surrounding point clouds is not fully considered, which easily causes grasping postures to collide. In response to these problems, a grasping sampling method based on simple semantic point clouds is proposed, and the grasping classification network is improved to improve the network evaluation capability. The present invention provides a grasping sampling method based on simple semantic point clouds, which makes full use of the target information provided by RGB images and depth images, and can generate high-quality grasping sampling postures for specified target objects in unstructured complex environments, which helps to improve the stability of visual grasping of robotic arms.
[0005] The first object of the present invention is to provide a grabbing sampling method based on a simple semantic point cloud, comprising: Obtain the RGB image and depth image of the target object, and obtain the mask image containing the target object; Construct a target-centered semantic point cloud based on the RGB image, depth image, and mask image; The semantic point cloud of the target is uniformly sampled, and the PCA algorithm is used to solve the normal vector and principal axis vector of the point cloud at each sampling point, and the grasping coordinate system is established; Define constraints and preliminarily select the grabbing coordinate systems that meet the constraints; Perform local grid search on the initially selected grasping coordinate system to generate candidate grasping postures, and then generate multiple candidate grasping postures through translation and rotation; A grasping confidence function is established and the candidate grasping postures are evaluated and ranked, and the candidate posture with the highest score is selected as the best grasping posture.
[0006] Preferably, the mask image containing the target object is obtained according to the following steps: The YOLACT algorithm is used to divide instance segmentation into two parallel subtasks. The prototype mask of the current image and the mask coefficient of each bounding box instance are predicted in parallel, and then the prototype mask is combined with the mask coefficient to generate a mask image containing the target object.
[0007] Preferably, the target-centered semantic point cloud is obtained according to the following steps: The RGB image, depth image and mask image are traversed simultaneously at a certain sampling interval. When the mask value is found to be the target object, the minimum enclosing rectangle of the target object is recorded. Traverse inside the minimum circumscribed rectangle until the mask value corresponding to the pixel coordinate is the target object, then traverse a certain pixel range outside the rectangle, and use the point cloud generated during the traversal process as the surrounding point cloud of the target point cloud to generate a semantic point cloud.
[0008] Preferably, the process of obtaining the normal vector and the principal axis vector of the point cloud includes: For each sampling point, a neighbor search algorithm is used to obtain multiple neighboring points around it to form a local point cloud centered at each sampling point; Obtain the centroid of each sampling point based on the local point cloud centered at each sampling point; Obtain the covariance matrix of each sampling point based on the local point cloud centered at each sampling point and the corresponding centroid; The covariance matrix of each sampling point is decomposed using eigenvalue to obtain multiple eigenvalues and corresponding eigenvectors; Two of the eigenvectors are used as estimates of the normal vector and principal axis vector of the local point cloud, and then the normal vector and principal axis vector of each sampling point are obtained by negating the normal vector according to the camera position when the image is acquired, so as to obtain the normal vector and principal axis vector of the point cloud.
[0009] Preferably, the grabbing coordinate system is established based on the normal vector and the principal axis vector of the point cloud.
[0010] Preferably, the constraint conditions are defined, including: the gripping posture of the robotic arm gripper cannot collide with the point cloud of the target object and the point cloud of the surrounding environment; and the closed area of the robotic arm gripper must include the point cloud of the target object.
[0011] Preferably, the plurality of candidate grasping postures are obtained according to the following steps: Based on the currently generated candidate grasping posture, the grasping coordinates are processed with additional translation along the y-axis and additional rotation around the z-axis, and then the grasping position is updated by translating along the x-axis until the gripper just touches the point cloud. Finally, the grasping posture is added to the grasping candidate set.
[0012] Preferably, the grasping confidence function is as follows:
[0013] In the formula, To capture confidence; , and are the weighting coefficients of the grasping point distance to the centroid, the grasping angle, and the grasping point clustering, respectively; The maximum distance between the candidate points and the object point cloud centroid; It is the sum of the absolute values of the distances between the current grasping point and multiple grasping points around it; is the number of all captured sampling points; is the number of clustered grasping points centered on the current grasping point; It is the angle between the normal vector of the surrounding point cloud and the grab approach vector.
[0014] The second object of the present invention is to provide a grabbing sampling system based on a simple semantic point cloud, comprising: The data acquisition module is used to obtain the RGB image and depth image of the target object, and obtain the mask image containing the target object; construct a semantic point cloud centered on the target based on the RGB image, depth image and mask image; The coordinate system establishment module is used to uniformly sample the semantic point cloud of the target, and use the PCA algorithm to solve the normal vector and principal axis vector of the point cloud at each sampling point, and establish the grasping coordinate system; The grasping posture acquisition module is used to define the constraint conditions and preliminarily screen the grasping coordinate system that meets the constraint conditions; perform local grid search on the preliminarily screened grasping coordinate system to generate candidate grasping postures, and then generate multiple candidate grasping postures through translation and rotation; The grasping posture selection module is used to establish a grasping confidence function and evaluate and sort the candidate grasping postures, and select the candidate posture with the highest score as the best grasping posture.
[0015] The present invention has at least the following beneficial effects: The present invention provides a grasping sampling method based on a simple semantic point cloud. The present invention obtains RGB images and depth images through a depth camera; then performs instance segmentation through YOLACT to obtain a mask image; uses the color image, the depth image and the mask image to construct a semantic point cloud centered on a target; uniformly samples the target point cloud, and uses the PCA algorithm to solve the normal vector and the principal axis vector of the point cloud at each sampling point, and thereby establishes a grasping coordinate system for the target object; according to constraint conditions, preliminarily screens grasping coordinate systems that meet the conditions; performs a local grid search on the grasping coordinate system to generate candidate grasping postures, and then generates more grasping postures through translation and rotation.
[0016] At present, grasping detection technology is mainly divided into model-driven grasping detection and data-driven grasping detection. Compared with the traditional model-driven grasping detection technology, the present invention does not need to obtain the object category and posture, but uses a large amount of data to train the features and grasping information that can extract the object, and then directly generates the grasping posture or classifies and detects the feasibility of the grasping posture generated by sampling. Most of the traditional data-driven grasping detection technologies do not consider the category of the object, so they cannot complete the grasping task of the specified target. The present invention solves the problem of the grasping task of the specified target while avoiding the possibility of collision of the grasping posture.
[0017] The present invention makes full use of the target information provided by RGB images and depth images, and can generate high-quality grasping sampling postures of designated target objects in unstructured complex environments, which helps to improve the stability of visual grasping of the robotic arm. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 A flowchart of a grabbing and sampling method based on a simple semantic point cloud provided by the present invention; Figure 2 Construct a schematic diagram for the semantic point cloud; Figure 3Schematic diagram of the grabbing coordinate system for calculating the target point cloud using the PCA algorithm; Figure 4 Schematic diagram of the grabbing coordinate system for sampling points; Figure 5 This is a capture sampling flow chart provided by the present invention. DETAILED DESCRIPTION
[0019] In order to illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the following is a detailed description in conjunction with embodiments.
[0020] The present invention is aimed at the traditional grasping posture estimation method that generates grasping posture based on the object posture of the known CAD model. It relies on the three-dimensional model of the object to be grasped, but in the actual task scenario, it is difficult to obtain an accurate CAD model of the target object. The YOLACT algorithm is an innovative single-stage instance segmentation model, which achieves a good balance between real-time performance and segmentation accuracy by combining multi-scale feature maps, prototype masks and fast NMS algorithms. Because the YOLACT algorithm has significant speed, the YOLACT algorithm can assist in fast grasping sampling, further reduce the computational complexity of subsequent grasping classification, and improve grasping efficiency. At the same time, it can also screen out high-quality grasping postures that can be directly used for simple objects without the participation of network models.
[0021] The implementation of the present invention requires a depth camera or a binocular camera and a GPU. In the specific implementation process, a laptop with a Geforce 3060 GPU and a RealSense D435i depth camera are used.
[0022] To achieve the above purpose, see Figure 1 As shown, the present invention provides a grabbing sampling method based on a simple semantic point cloud, comprising: S1. Obtain the RGB image and depth image of the target object, and obtain the mask image containing the target object; The mask image containing the target object is obtained according to the following steps: The YOLACT algorithm is used to divide instance segmentation into two parallel subtasks. The prototype mask of the current image and the mask coefficient of each bounding box instance are predicted in parallel, and then the prototype mask is combined with the mask coefficient to generate a mask image containing the target object.
[0023] It should be noted that the instance of instance segmentation generally refers to a specific target object in an image. The instance segmentation task aims to identify each object in the image and perform pixel-level segmentation on each object instance. In this embodiment, the YOLACT algorithm is used to segment instances in RGB images.
[0024] In this embodiment, when acquiring the RGB image and depth image of the target object, the camera used is a depth camera or a binocular camera, which can be fixed at the end of the robotic arm and move with the robotic arm, or fixed outside the robotic arm and remain stationary.
[0025] S2, constructing a semantic point cloud centered on the target based on the RGB image, depth image and mask image; The target-centered semantic point cloud is obtained by following the steps below: The RGB image, depth image and mask image are traversed simultaneously at a certain sampling interval. When the mask value is found to be the target object, the minimum enclosing rectangle of the target object is recorded. Traverse inside the minimum circumscribed rectangle until the mask value corresponding to the pixel coordinate is the target object, then traverse a certain pixel range outside the rectangle, and use the point cloud generated during the traversal process as the surrounding point cloud of the target point cloud to generate a semantic point cloud.
[0026] In this embodiment, see Figure 2 As shown in the figure, the depth camera is first registered, and the semantic point cloud centered on the target is constructed using the color image, depth image and mask image. In the process of acquiring the point cloud, the RGB image, depth image and mask image are traversed at the same time at a certain sampling interval. When the mask value is found to be a certain target object, the minimum circumscribed rectangle of the target object is recorded. First, traverse inside the rectangle until the mask value corresponding to the pixel coordinate is the target object, and then traverse a certain pixel range outside the rectangle. The point cloud generated during the traversal process is used as the surrounding point cloud of the target point cloud to generate the semantic point cloud. Figure 2 The middle (a) figure shows the target detection of the image. Figure 2 (b) is the target point cloud segmented by the YOLACT algorithm. Figure 2 (c) is the surrounding point cloud segmented by the YOLACT algorithm. Figure 2 (d) is the semantic point cloud centered on the target.
[0027] S3, uniformly sample the semantic point cloud of the target, and use the PCA algorithm to solve the normal vector and principal axis vector of the point cloud at each sampling point, and establish a grasping coordinate system; The process of obtaining the normal vector and the principal axis vector of the point cloud includes: For each sampling point, a neighbor search algorithm is used to obtain multiple neighboring points around it to form a local point cloud centered at each sampling point; Obtain the centroid of each sampling point based on the local point cloud centered at each sampling point; Obtain the covariance matrix of each sampling point based on the local point cloud centered at each sampling point and the corresponding centroid; The covariance matrix of each sampling point is decomposed using eigenvalue to obtain multiple eigenvalues and corresponding eigenvectors; Two of the eigenvectors are used as estimates of the normal vector and principal axis vector of the local point cloud, and then the normal vector and principal axis vector of each sampling point are obtained by negating the normal vector according to the camera position when the image is acquired, so as to obtain the normal vector and principal axis vector of the point cloud.
[0028] The grabbing coordinate system is established according to the normal vector and the principal axis vector of the point cloud.
[0029] In this embodiment, the centroid of the target point cloud is calculated, and then the covariance matrix of the target point cloud is calculated using the PCA algorithm. The direction with the smallest variance is used as the grabbing approach direction of the target point cloud. At the same time, a reference coordinate system of the target point cloud is established as follows: Figure 3 As shown in the figure. Since the normal vector calculated by this method may point to the inside or outside of the object, the normal vector is negated so that the normal vector direction points to the direction opposite to the camera's viewing angle, so as to establish the grabbing coordinate system of the point, as shown in the figure. Figure 4 shown.
[0030] Specifically, assuming Indicates the position of the grasp candidate. For each sampling point , use the nearest neighbor search algorithm to obtain the k nearest neighbor points around it, so as to form the point The local point cloud centered : Request local point cloud The centroid ,as follows
[0031] Calculate point cloud The covariance matrix of is as follows
[0032] Where x, y, and z represent the coordinate values of k points in the local point cloud on the x, y, and z axes, respectively. The three eigenvalues of the covariance matrix can be obtained by eigenvalue decomposition: (Assumption ) and the corresponding eigenvector , then the eigenvector and The sampling points Estimation of surface normal and principal axis vectors of a local point cloud.
[0033] Since the normal vector calculated by this method may point to the inside or outside of the object, this problem can be solved by combining the camera perspective. Assume that the camera position is , by negating the normal vector to satisfy the following conditions:
[0034] In the formula, Indicates the camera position; Indicates the sampling point.
[0035] like Figure 4 As shown in , the normal vector direction is finally made to point in the direction opposite to the camera view, thereby establishing the grabbing coordinate system of the sampling point. In the grabbing coordinate system, the X axis is established with the principal axis vector, and The vector direction establishes the Y axis, and the normal vector direction establishes the Z axis.
[0036] S4, define constraint conditions and preliminarily select the grasping coordinate system that meets the constraint conditions; The constraint conditions are defined, including: the grasping posture of the robot arm gripper cannot collide with the point cloud of the target object and the point cloud of the surrounding environment; the closed area of the robot arm gripper must include the point cloud of the target object.
[0037] In this embodiment, the conditions for initially screening candidate grasping postures are: (1) The gripping posture of the robot arm gripper cannot collide with the point cloud of the target object or the point cloud of the surrounding environment.
[0038] (2) The closed area of the robot gripper must include the point cloud of the target object.
[0039] These two constraints can ensure that the generated grasping posture will not collide with the surrounding environment and that the grasping area contains the target object. According to the above two conditions, the grasping candidates are screened, invalid grasping candidates are filtered out, and a grasping candidate set is generated.
[0040] S5, performing a local grid search on the initially screened grasping coordinate system to generate candidate grasping postures, and then generating multiple candidate grasping postures through translation and rotation; The multiple candidate grasping postures are obtained according to the following steps: Based on the currently generated candidate grasping posture, the grasping coordinates are processed with additional translation along the y-axis and additional rotation around the z-axis, and then the grasping position is updated by translating along the x-axis until the gripper just touches the point cloud. Finally, the grasping posture is added to the grasping candidate set.
[0041] In this embodiment, in order to generate more candidate grasping postures, the current grasping coordinate system is processed with additional translation along the y-axis and additional rotation around the z-axis, and then the grasping position is updated by translating along the x-axis until the gripper just touches the point cloud, and finally the grasping posture is added to the grasping candidate set.
[0042] S6. Establish a grasping confidence function and evaluate and sort the candidate grasping postures, and select the candidate posture with the highest score as the best grasping posture.
[0043] The grasp confidence function is as follows:
[0044] In the formula, To capture confidence; , and are the weighting coefficients of the grasping point distance to the centroid, the grasping angle, and the grasping point clustering, respectively; The maximum distance between the candidate points and the object point cloud centroid; It is the sum of the absolute values of the distances between the current grasping point and multiple grasping points around it; is the number of all captured sampling points; is the number of clustered grasping points centered on the current grasping point; It is the angle between the normal vector of the surrounding point cloud and the grab approach vector.
[0045] Among them, the normal vector of the surrounding point cloud and grab approach vector The angle is calculated as follows:
[0046] The angle Constrained to ,Right now
[0047] When the target object has a regular shape, the When the surrounding environment is relatively flat, it can also be appropriately increased. According to the above confidence function, the grasping candidate postures are scored and sorted, and then the top n grasping candidate postures are selected as high-quality grasping sampling postures.
[0048] See also Figure 5As shown, the present invention collects the RGB image and depth image of the target object through the camera, and then segments the target point cloud and the surrounding environment point cloud through examples, and then generates a simple semantic point cloud centered on the target. The simple semantic point cloud of the target is processed, and the normal vector and the principal axis vector of the point cloud are solved at each sampling point to establish a grasping coordinate system. The grasping coordinate systems that do not meet the constraints are screened, and the selected grasping coordinate systems are subjected to local grid search to generate candidate grasping postures, and then multiple candidate grasping postures are generated through translation and rotation, and finally the candidate posture with the highest confidence function score is selected as the best grasping posture.
[0049] The present invention also provides a grabbing and sampling system based on a simple semantic point cloud, comprising: The data acquisition module is used to obtain the RGB image and depth image of the target object, and obtain the mask image containing the target object; construct a semantic point cloud centered on the target based on the RGB image, depth image and mask image; The coordinate system establishment module is used to uniformly sample the semantic point cloud of the target, and use the PCA algorithm to solve the normal vector and principal axis vector of the point cloud at each sampling point, and establish the grasping coordinate system; The grasping posture acquisition module is used to define the constraint conditions and preliminarily screen the grasping coordinate system that meets the constraint conditions; perform local grid search on the preliminarily screened grasping coordinate system to generate candidate grasping postures, and then generate multiple candidate grasping postures through translation and rotation; The grasping posture selection module is used to establish a grasping confidence function and evaluate and sort the candidate grasping postures, and select the candidate posture with the highest score as the best grasping posture.
[0050] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A grabbing and sampling method based on a simple semantic point cloud, characterized in that: include: Obtain the RGB image and depth image of the target object, and obtain the mask image containing the target object; Construct a target-centered semantic point cloud based on the RGB image, depth image, and mask image; The semantic point cloud of the target is uniformly sampled, and the PCA algorithm is used to solve the normal vector and principal axis vector of the point cloud at each sampling point, and the grasping coordinate system is established; Define constraints and preliminarily select the grabbing coordinate systems that meet the constraints; Perform local grid search on the initially selected grasping coordinate system to generate candidate grasping postures, and then generate multiple candidate grasping postures through translation and rotation; A grasping confidence function is established and the candidate grasping postures are evaluated and ranked, and the candidate posture with the highest score is selected as the best grasping posture.
2. The method for grabbing and sampling based on simple semantic point cloud according to claim 1, characterized in that: The mask image containing the target object is obtained according to the following steps: The YOLACT algorithm is used to divide instance segmentation into two parallel subtasks. The prototype mask of the current image and the mask coefficient of each bounding box instance are predicted in parallel, and then the prototype mask is combined with the mask coefficient to generate a mask image containing the target object.
3. The method for grabbing and sampling based on simple semantic point cloud according to claim 1, characterized in that: The target-centered semantic point cloud is obtained by following the steps below: The RGB image, depth image and mask image are traversed simultaneously at a certain sampling interval. When the mask value is found to be the target object, the minimum enclosing rectangle of the target object is recorded. Traverse inside the minimum circumscribed rectangle until the mask value corresponding to the pixel coordinate is the target object, then traverse a certain pixel range outside the rectangle, and use the point cloud generated during the traversal process as the surrounding point cloud of the target point cloud to generate a semantic point cloud.
4. The method for grabbing and sampling based on simple semantic point cloud according to claim 1, characterized in that: The process of obtaining the normal vector and the principal axis vector of the point cloud includes: For each sampling point, a neighbor search algorithm is used to obtain multiple neighboring points around it to form a local point cloud centered at each sampling point; Obtain the centroid of each sampling point based on the local point cloud centered at each sampling point; Obtain the covariance matrix of each sampling point based on the local point cloud centered at each sampling point and the corresponding centroid; The covariance matrix of each sampling point is decomposed using eigenvalue to obtain multiple eigenvalues and corresponding eigenvectors; Two of the eigenvectors are used as estimates of the normal vector and principal axis vector of the local point cloud, and then the normal vector and principal axis vector of each sampling point are obtained by negating the normal vector according to the camera position when the image is acquired, so as to obtain the normal vector and principal axis vector of the point cloud.
5. The method for grabbing and sampling based on simple semantic point cloud according to claim 4 is characterized in that: The grabbing coordinate system is established according to the normal vector and the principal axis vector of the point cloud.
6. The method for grabbing and sampling based on simple semantic point cloud according to claim 1, characterized in that: The constraint conditions are defined, including: the grasping posture of the robot arm gripper cannot collide with the point cloud of the target object and the point cloud of the surrounding environment; the closed area of the robot arm gripper must include the point cloud of the target object.
7. The method for grabbing and sampling based on simple semantic point cloud according to claim 1, characterized in that: The multiple candidate grasping postures are obtained according to the following steps: Based on the currently generated candidate grasping posture, the grasping coordinates are processed with additional translation along the y-axis and additional rotation around the z-axis, and then the grasping position is updated by translating along the x-axis until the gripper just touches the point cloud. Finally, the grasping posture is added to the grasping candidate set.
8. The method for grabbing and sampling based on simple semantic point cloud according to claim 1, characterized in that: The grasp confidence function is as follows: In the formula, To capture confidence; , and are the weighting coefficients of the grasping point distance to the centroid, the grasping angle, and the grasping point clustering, respectively; The maximum distance between the candidate points and the object point cloud centroid; It is the sum of the absolute values of the distances between the current grasping point and multiple grasping points around it; is the number of all captured sampling points; is the number of clustered grasping points centered on the current grasping point; It is the angle between the normal vector of the surrounding point cloud and the grab approach vector.
9. A grabbing and sampling system based on simple semantic point cloud, characterized in that: include: A data acquisition module is used to obtain an RGB image and a depth image of the target object, and obtain a mask image containing the target object; Construct a target-centered semantic point cloud based on the RGB image, depth image, and mask image; The coordinate system establishment module is used to uniformly sample the semantic point cloud of the target, and use the PCA algorithm to solve the normal vector and principal axis vector of the point cloud at each sampling point, and establish the grasping coordinate system; The grasping posture acquisition module is used to define the constraint conditions and preliminarily screen the grasping coordinate system that meets the constraint conditions; perform local grid search on the preliminarily screened grasping coordinate system to generate candidate grasping postures, and then generate multiple candidate grasping postures through translation and rotation; The grasping posture selection module is used to establish a grasping confidence function and evaluate and sort the candidate grasping postures, and select the candidate posture with the highest score as the best grasping posture.
Citation Information
Patent Citations
Robot grabbing detection method based on instance segmentation under single-view-angle point cloud
CN110363815A
Robot grabbing pose detection method based on domain migration under single-view-angle point cloud
CN112489117A
Seven-degree-of-freedom grabbing posture detection method based on RGB image and depth image
CN114140418A
Robot grabbing detection method based on multi-mode visual information fusion
CN115861999A
Autonomous mobile grabbing method for mechanical arm based on visual-haptic fusion under complex illumination condition
US20230042756A1
Cited By
Object grabbing method and device, terminal equipment and storage medium
CN120244970A
Humanoid robot control method based on image segmentation
CN120307300A
Data labeling method and system, storage medium, computer equipment and program product
CN120431581A
Grabbing attitude generation method and system based on multi-modal large model
CN120588235A
Mechanical arm grabbing method based on reference segmentation and spatial perception
CN120901933A
