A 6D pose estimation robot arm grasping method and system based on global context modulation

Abstract

The application provides a 6D pose estimation mechanical arm grabbing method and system based on global context modulation, acquires point cloud data of a scene, pre-processes, extracts point-by-point features of the point cloud by using a backbone network, inputs the point-by-point features into a 6D pose estimation network, extracts global context features associated with the scene from the point-by-point features, dynamically modulates local geometric features of a candidate grabbing point, generates grabbing features carrying global structure information, outputs an optimal 6D grabbing pose, converts to a mechanical arm base coordinate system, obtains a target execution pose, and controls the mechanical arm to complete a grabbing task; the system comprises an information acquisition and preprocessing module, a 6D pose estimation module, and a coordinate transformation and control module. The application can utilize overall scene information to assist decision-making, significantly improves the robustness of grabbing pose prediction, has good engineering compatibility and migratability, improves overall grabbing success rate, has good generalization ability for unknown category objects, and is convenient for rapid deployment and application.

Description

Technical Field

[0001] This invention relates to the technical field of image or video recognition or understanding, and in particular to a 6D pose estimation robotic arm grasping method and system based on global context modulation. Background Technology

[0002] With the rapid development of intelligent manufacturing and intelligent service robots, robotic arms are increasingly being used in scenarios such as parts sorting, logistics depalletizing, and home services. In practical applications, target objects are often randomly stacked and obstructed, posing a significant challenge to the autonomous grasping capabilities of robotic arms.

[0003] Existing robotic arm grasping methods can be mainly divided into two categories. The first category is based on traditional image processing or manually designed features, which estimates the grasping position by detecting edges and key points in two-dimensional images. This type of method has difficulty accurately obtaining the three-dimensional spatial pose of the object and has a low grasping success rate when there is severe occlusion. The second category is 6D pose estimation methods based on deep learning. These methods acquire scene point clouds through depth cameras and use neural networks to directly regress or classify to obtain the grasping pose, achieving better results.

[0004] In recent years, researchers have proposed various point cloud-based grasping pose estimation networks. For example, GSNet predicts grasping pose by learning from point cloud features, while GraspFast decouples the grasping direction into a classification problem and employs a multi-scale cylindrical region reasoning (CRR) module to extract local geometric features. However, these methods primarily focus on local geometric information near the grasping point, neglecting the overall structure of the target object and scene-level contextual information, such as the relative positions and support relationships between objects. When there is severe occlusion in the scene or local missing points in the target point cloud, relying solely on local features can easily lead to incorrect grasping pose predictions, limiting the network's robustness and generalization ability.

[0005] Furthermore, existing methods implicitly assume that similar local geometries correspond to similar grasping feasibility. However, in real-world scenarios, the grasping feasibility of the same or similar local geometries varies significantly across different object contexts. For example, a slender columnar structure might serve as a good grasping point if it belongs to an independent object, such as a test tube. However, if this structure is part of a larger object, such as the neck of a giraffe model, its connection to the overall object and complex center of gravity distribution make grasping this part risky, leading to the object swaying, slipping, or the gripper failing to fully enclose it, drastically reducing grasping feasibility. Existing methods, relying solely on local geometric features, cannot distinguish between these two scenarios, frequently resulting in misjudgments in complex scenes.

[0006] Therefore, how to effectively utilize global scene context information to assist in grasping decisions under conditions of incomplete local information is a key technical problem that urgently needs to be solved in this field. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention provides a 6D pose estimation robotic arm grasping method and system based on global context modulation. This solves the problem that existing grasping pose estimation methods rely solely on local geometric features and cannot distinguish between different grasping semantics corresponding to the same local geometry in different object contexts, resulting in a high misjudgment rate and poor robustness in grasping feasibility.

[0008] The technical solution adopted in this invention is a 6D pose estimation robotic arm grasping method based on global context modulation, comprising the following steps:

[0009] S1 acquires point cloud data of the scene, preprocesses it, and uses the backbone network to extract point-by-point features of the point cloud.

[0010] S2 inputs the point-by-point features into the 6D pose estimation network to predict the 6D grasping pose;

[0011] The 6D pose estimation network includes a global context modulation module, which extracts global context features associated with the scene from point-by-point features, and dynamically modulates the local geometric features of candidate grasping points based on the global context features to generate grasping features carrying global structural information, and outputs the optimal 6D grasping pose based on the grasping features.

[0012] S3 transforms the optimal 6D grasping pose into the robot arm's base coordinate system to obtain the target execution pose and controls the robot arm to complete the grasping task.

[0013] Preferably, in S1, the preprocessing includes at least one of workspace clipping, outlier removal, and voxelization downsampling.

[0014] Preferably, in S2, the global context modulation module performs global pooling on the point-by-point features output by the backbone network to obtain a global feature vector representing the scene context information, and generates a modulation weight vector with the same dimension as the local geometric features through a gating network.

[0015] Enhanced grasping features are obtained based on the modulation weight vector and the local geometric feature vectors of the candidate grasping points.

[0016] Preferably, in S2, the 6D pose estimation network predicts the grasping approximation direction by using a classification method based on predefined direction templates when predicting the 6D grasping pose; several candidate direction templates are pre-constructed on a unit sphere, the probability of each candidate direction is predicted by classification, and the direction with the highest probability is selected as the optimal grasping approximation direction.

[0017] Preferably, in S2, local geometric features are extracted using a multi-scale cylindrical region reasoning method; multiple cylindrical local neighborhoods of different scales are established with the candidate grasping point as the center and the predicted optimal grasping approximation direction as the axis, and the point cloud geometric features in each cylindrical neighborhood are extracted respectively, and the multi-scale features are fused.

[0018] Preferably, the 6D pose estimation network also outputs the rotation angle around the grasping approximation direction and the grasping quality score corresponding to each candidate grasping pose, and selects the grasping pose with the highest score as the optimal 6D grasping pose.

[0019] Preferably, a loss function is constructed to train the 6D pose estimation network, and the loss function is associated with the grasping direction classification loss, rotation angle regression loss, grasping quality score loss, and grasping width classification loss.

[0020] Preferably, in S3, the spatial transformation relationship between the camera coordinate system and the robot arm base coordinate system is obtained through hand-eye calibration, and the optimal 6D grasping pose is transformed into the robot arm base coordinate system using homogeneous coordinate transformation.

[0021] Preferably, in S3, a smooth interpolation method is used to plan the motion trajectory of the robotic arm's end effector, and the robotic arm is controlled to move along the motion trajectory to complete the grasping task.

[0022] A 6D pose estimation robotic arm grasping system based on global context modulation includes:

[0023] The information acquisition and preprocessing module is used to acquire point cloud data of the scene, perform preprocessing, and extract point-by-point features of the point cloud using the backbone network.

[0024] The 6D pose estimation module is configured with a 6D pose estimation network including a global context modulation module. It is used to extract global context features associated with the scene from point-by-point features, and dynamically modulate the local geometric features of candidate grasping points based on the global context features to generate grasping features carrying global structural information and output the optimal 6D grasping pose.

[0025] The coordinate transformation and control module is used to transform the optimal 6D grasping pose to the robot arm's base coordinate system and control the robot arm to complete the grasping task.

[0026] This invention provides a 6D pose estimation robotic arm grasping method and system based on global context modulation. The method involves acquiring point cloud data of a scene, preprocessing it, and extracting point-by-point features from the point cloud using a backbone network. These point-by-point features are then input into a 6D pose estimation network. The global context modulation module extracts global context features associated with the scene from the point-by-point features, dynamically modulates the local geometric features of candidate grasping points, generates grasping features carrying global structural information, outputs the optimal 6D grasping pose, transforms it to the robotic arm's base coordinate system, obtains the target execution pose, and controls the robotic arm to complete the grasping task. The system includes an information acquisition and preprocessing module, a 6D pose estimation module, and a coordinate transformation and control module.

[0027] The beneficial effects of this invention are as follows:

[0028] (1) By extracting global context features representing the overall structure and relative positional relationship of objects from the entire scene through the global context modulation module, the local geometric features are dynamically modulated. The global context modulation mechanism is introduced into the grasping pose estimation, enabling the network to identify different grasping semantics corresponding to the same local geometric structure in different object contexts, breaking through the limitations of local geometric features and effectively correcting the grasping feasibility misjudgment caused by relying solely on local geometric features. When faced with severe occlusion and missing local point clouds, the model can use the overall scene information to assist decision-making, significantly improving the robustness of grasping pose prediction.

[0029] (2) The global context modulation module contains only three core computational paths: global pooling, fully connected gating network and residual connection. It has few parameters and low computational cost, and can be easily embedded into various existing point cloud-based grasping pose estimation networks, such as GSNet and GraspFast. Performance improvement can be achieved without significantly modifying the original network architecture, and it has good engineering compatibility and portability.

[0030] (3) Experiments show that after introducing the global context modulation module proposed in this invention into the GSNet baseline network, on the Novel Test set of the public dataset GraspNet-1Billion, with the GSNet network as the baseline and using the same training parameters and evaluation metrics, the comprehensive evaluation metric AP is improved by 2.83 percentage points; in the real robotic arm grasping experiment, the overall grasping success rate is improved from 60% to 75%, and it has good generalization ability for unknown categories of objects;

[0031] (iv) The entire system can be trained end-to-end from the original point cloud input to the grasping pose output, without the need for manual feature design, which facilitates rapid deployment and application on actual robot platforms. Attached Figure Description

[0032] Figure 1 This is a flowchart of the method of the present invention;

[0033] Figure 2 This is a schematic diagram of the structure of the 6D pose estimation network in an embodiment of the present invention;

[0034] Figure 3 This is a flowchart illustrating the workflow of the global context modulation module in an embodiment of the present invention.

[0035] Figure 4 This is a visualization result of the grasping pose on the GraspNet-1 Billion dataset in an embodiment of the present invention;

[0036] Figure 5 This is a visualization of the grasping process and pose in a real, cluttered scene, as described in this embodiment of the invention.

[0037] Figure 6 This is a schematic diagram of the system structure of the present invention. Detailed Implementation

[0038] The present invention will be further described in detail below with reference to embodiments, but the scope of protection of the present invention is not limited thereto.

[0039] This invention relates to a 6D pose estimation robotic arm grasping method based on global context modulation, comprising the following steps:

[0040] S1 acquires point cloud data of the scene, preprocesses it, and uses the backbone network to extract point-by-point features of the point cloud.

[0041] S2 inputs the point-by-point features into the 6D pose estimation network to predict the 6D grasping pose;

[0042] The 6D pose estimation network includes a global context modulation module, which extracts global context features associated with the scene from point-by-point features, and dynamically modulates the local geometric features of candidate grasping points based on the global context features to generate grasping features carrying global structural information, and outputs the optimal 6D grasping pose based on the grasping features.

[0043] S3 transforms the optimal 6D grasping pose into the robot arm's base coordinate system to obtain the target execution pose and controls the robot arm to complete the grasping task.

[0044] The method will be described below with reference to specific embodiments.

[0045] (1) Acquire point cloud data of the scene, preprocess it, and use the backbone network to extract point-by-point features of the point cloud.

[0046] RGB and depth images of cluttered scenes are acquired by a depth camera installed at the end of a robotic arm, and the depth images are reconstructed into 3D point cloud data based on the camera's intrinsic parameters.

[0047] This embodiment uses a Rellman collaborative robotic arm as the actuator and an Intel RealSense D435i depth camera as the vision sensor, mounted at the end of the robotic arm, forming an "eye-in-hand" configuration. The host computer configuration is: AMD epyc 7302, NVIDIA GeForce RTX 2080Ti GPU, Ubuntu 20.04 operating system, and ROS Noetic middleware. First, hand-eye calibration is performed to obtain the homogeneous transformation matrix from the camera coordinate system to the robotic arm's base coordinate system.

[0048] Preprocessing includes at least one of workspace clipping, outlier removal, and voxelization downsampling.

[0049] In this embodiment, ResUNet is used as the backbone for point cloud feature extraction. The input is the preprocessed scene point cloud, and the output point-by-point features (graph) are represented as follows:

[0050]

[0051] Where N is the number of points and C is the number of feature channels.

[0052] (2) Input the point-by-point features into the 6D pose estimation network to predict the 6D grasping pose;

[0053] The 6D pose estimation network includes a global context modulation module, which extracts global context features associated with the scene from point-by-point features, and dynamically modulates the local geometric features of candidate grasping points based on the global context features to generate grasping features carrying global structural information, and outputs the optimal 6D grasping pose based on the grasping features.

[0054] In this invention, 6D pose is decoupled into two subtasks: grasping approximation direction prediction and rotation angle regression around the approximation direction.

[0055] (2-1) Grab Direction Prediction Branch

[0056] The 6D pose estimation network predicts the grasping approximation direction by using a classification method based on predefined orientation templates when predicting the 6D grasping pose. Several candidate orientation templates are pre-constructed on a unit sphere, and the probability of each candidate orientation is predicted by classification. The orientation with the highest probability is selected as the optimal grasping approximation direction.

[0057] Specifically, 300 directions are uniformly sampled on a unit sphere using the Fibonacci spherical sampling algorithm as candidate direction templates. For each point, the network outputs a 300-dimensional probability vector, and the direction corresponding to the highest probability is selected as the approximation direction for that candidate point.

[0058] (2-2) Multi-scale local geometric feature extraction

[0059] Local geometric features are extracted using a multi-scale cylindrical region reasoning method. Multiple cylindrical local neighborhoods of different scales are established with the candidate grab point as the center and the predicted optimal grab approximation direction as the axis. Point cloud geometric features in each cylindrical neighborhood are extracted and the multi-scale features are then fused.

[0060] Specifically, a multi-scale cylindrical region reasoning method is used to extract local geometric features; for candidate grab points The grabbing direction predicted in (2-1) Set the radius for each axis. Four cylindrical neighborhoods are identified, and local point cloud features are extracted within these neighborhoods. These features are then encoded to obtain local geometric features. .

[0061] (2-3) Global Context Modulation Module

[0062] Because the feasibility of grasping the same or similar local geometric structures varies across different object contexts. For example, a slender columnar structure can serve as a good grasping point if it belongs to an independent object, such as a test tube; however, when the structure is part of a connecting section of a large object, such as the neck of a giraffe model, grasping this part will cause the object to sway or slip due to its connection to the overall object and complex center of gravity distribution, significantly reducing the feasibility of grasping. The global context modulation module utilizes the overall structural information of the scene to identify the object context in which candidate grasping points are located, thereby re-evaluating the grasping feasibility represented by local geometric features.

[0063] The input to the global context modulation module is global features. Local geometric features of candidate grab points .

[0064] The global context modulation module performs global pooling on the point-by-point features output by the backbone network to obtain a global feature vector representing the scene context information. A modulation weight vector with the same dimension as the local geometric features is generated through a gating network.

[0065] The enhanced grasping features are obtained based on the modulation weight vector and the local geometric feature vectors of the candidate grasping points. The internal calculation process of the module is as follows:

[0066] a) Global pooling

[0067]

[0068] in, It is a global vector. Represents a coarse-scale global vector. Represents a fine-scale global vector. This represents the concatenated global context vector. It is the number of feature vectors. Indicates the first 1 eigenvector This represents the concatenation operation of features; coarse-scale and fine-scale pooling capture scene structure information at different granularities.

[0069] b) Generation of modulation weights and gating coefficients

[0070]

[0071] in, This represents a multilayer perceptron. The weight matrix represents the concatenated features. Indicates modulation weights, This represents the gating coefficient.

[0072] c) Modulation and Residual Connection

[0073]

[0074] in, Indicates output features, Indicates original features, This indicates element-wise multiplication.

[0075] Through residual connections, the network incorporates the modulation effects of the global context while preserving the original local geometric information.

[0076] (2-4) Capture quality score and pose regression

[0077] The 6D pose estimation network also outputs the rotation angle around the grasping approximation direction and the grasping quality score corresponding to each candidate grasping pose, and selects the grasping pose with the highest score as the optimal 6D grasping pose.

[0078] The enhanced features are fed into a fully connected network, and the two output heads output the grasping quality score and the rotation angle around the grasping direction, respectively. For each candidate point, the network outputs a grasping pose and a score, and the system selects the pose with the highest score as the optimal grasping pose.

[0079] (2-5) Network Training

[0080] The 6D pose estimation network is trained by constructing a loss function, which is associated with the grasping direction classification loss, rotation angle regression loss, grasping quality score loss, and grasping width classification loss.

[0081] The GraspNet-1 Billion dataset was used for training. The loss function included:

[0082] The classification loss for grasping direction is achieved using cross-entropy loss. ,

[0083]

[0084] Where N is the number of sample points. This represents the true value of the i-th point in the k-th direction. The corresponding predicted value;

[0085] Rotation angle regression loss, using smoothed L1 loss. ,

[0086]

[0087] in, Let be the actual rotation angle value of the i-th candidate point. The predicted rotation angle value is given by SmoothL1, which is the smoothing L1 loss function.

[0088] The capture quality score loss is obtained using binary cross-entropy loss. ,

[0089]

[0090] in, Let i be the capture quality score of the i-th candidate point. This corresponds to the predicted capture quality score.

[0091] The capture width classification loss is adopted, and the cross-entropy loss is used. ,

[0092]

[0093] in, For the i-th point and the k-th width category, This represents the predicted width category probability.

[0094] The total loss function satisfies The training parameters are batch size=8, initial learning rate 0.001, Adam optimizer, and training for 10 epochs.

[0095] (3) Transform the optimal 6D grasping pose into the robot arm base coordinate system to obtain the target execution pose and control the robot arm to complete the grasping task.

[0096] The spatial transformation relationship between the camera coordinate system and the robot arm base coordinate system is obtained through hand-eye calibration. The optimal 6D grasping pose is then transformed into the robot arm base coordinate system using homogeneous coordinate transformation, and used as the target execution pose.

[0097] Furthermore, a smooth interpolation method is used to plan the motion trajectory of the robotic arm's end effector, and the robotic arm is controlled to move along the motion trajectory to complete the grasping task.

[0098] Specifically, after the robotic arm's grasping action is initiated, the following steps are performed:

[0099] Initialize the depth camera, robotic arm, and ROS nodes, and load the weights of the trained network model; the depth camera acquires RGB-D images of the current scene and generates the original point cloud;

[0100] Point cloud preprocessing: The working space (x: -0.5~0.5m, y: -0.5~0.5m, z: 0~0.8m) is clipped by pass-through filtering, downsampled by voxel filter (leaf size=0.005m), and outliers are removed by radius filtering (radius 0.01m, neighbor number threshold 10).

[0101] The preprocessed point cloud is input into a 6D pose estimation network to obtain the optimal grasping pose in the camera coordinate system. ; Using hand-eye calibration matrix Calculate the grasping pose in the base coordinate system.

[0102]

[0103] in, It is the optimal grasping pose in the base coordinate system;

[0104] The robotic arm uses fifth-order polynomial interpolation to plan a smooth trajectory from its current position to the gripping point. The robotic arm moves along the trajectory and, upon reaching the gripping point, controls the gripper to close and grasp the target object.

[0105] After the capture is complete, the system determines whether there are any uncaptured targets. If so, it returns; otherwise, it terminates the task.

[0106] The network was tested on the Novel Test set of the GraspNet-1 Billion dataset. After introducing the global context modulation module of this invention, the AP metric improved from 23.98 to 26.81 (an improvement of 2.83 percentage points) compared to the GSNet baseline; the AP_0.8 metric improved from 29.93 to 33.39 (an improvement of 3.46 percentage points). This demonstrates that the present invention effectively improves the prediction accuracy of grasping pose, especially for high-precision grasping requirements. The visualization effect of the simulation experiment is shown in the figure.

[0107] To further verify the effectiveness of the method of the present invention, a comparative experiment was conducted on the Novel Test set of the GraspNet-1 Billion dataset, comparing the method of the present invention with a variety of existing methods. The results are shown in Table 1.

[0108] Table 1. Comparison of capture pose estimation accuracy of different methods on the GraspNet-1 Billion Novel Test set.

[0109] GraspNet 3.98 11.25 10.55 Scale-balanced grasp 12.08 28.53 22.63 GSNet 14.05 29.93 23.98 Ours 14.81 33.39 26.81

[0110] Experimental results show that the method of this invention outperforms all comparative methods in both AP_0.8 and AP. Specifically, in the AP_0.8 metric (reflecting the stability of grasping, with higher numbers indicating more stable grasping), this invention achieves 33.39, an improvement of 3.46 percentage points compared to the baseline GSNet and 4.86 percentage points compared to Scale-Balanced Grasp; in the comprehensive evaluation metric AP, this invention achieves 26.81, an improvement of 2.83 percentage points compared to GSNet and 4.18 percentage points compared to Scale-Balanced Grasp. In the AP_0.4 metric, this invention achieves 14.81, also outperforming all comparative methods. These comparative results fully verify that the global context modulation module proposed in this invention can effectively improve the grasping pose estimation accuracy under different evaluation thresholds.

[0111] In a real-world robotic arm experiment, three different shaped objects—a non-rigid cuboid, a rigid cuboid, and a non-rigid irregular object—were stacked in a real, cluttered scene. The GraspNet-Baseline and the proposed method were run 20 times each. Results showed that the proposed method achieved a 75% success rate in grasping, higher than the baseline method's 60%; the average grasping time was 69.7 seconds, slightly better than the baseline method's 71.3 seconds. The data acquisition results and prediction visualization results for the real-world scene are shown in the figure. Grasping failures were mainly due to the target exceeding the reach of the robotic arm, with very few due to pose prediction errors. The experimental results validate the effectiveness and engineering feasibility of this invention in real-world cluttered scenes.

[0112] This invention also relates to a 6D pose estimation robotic arm grasping system based on global context modulation, comprising:

[0113] The information acquisition and preprocessing module is used to acquire point cloud data of the scene, perform preprocessing, and extract point-by-point features of the point cloud using the backbone network.

[0114] The 6D pose estimation module is configured with a 6D pose estimation network including a global context modulation module. It is used to extract global context features associated with the scene from point-by-point features, and dynamically modulate the local geometric features of candidate grasping points based on the global context features to generate grasping features carrying global structural information and output the optimal 6D grasping pose.

[0115] The coordinate transformation and control module is used to transform the optimal 6D grasping pose to the robot arm's base coordinate system and control the robot arm to complete the grasping task.

[0116] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0117] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0118] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0119] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0120] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0121] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A 6D pose estimation robotic arm grasping method based on global context modulation, characterized in that, Includes the following steps: S1 acquires point cloud data of the scene, preprocesses it, and uses the backbone network to extract point-by-point features of the point cloud. S2 Input the point-by-point features into the 6D pose estimation network to predict the 6D grasping pose; The 6D pose estimation network includes a global context modulation module, which extracts global context features associated with the scene from point-by-point features, and dynamically modulates the local geometric features of candidate grasping points based on the global context features to generate grasping features carrying global structural information, and outputs the optimal 6D grasping pose based on the grasping features. S3 transforms the optimal 6D grasping pose into the robot arm's base coordinate system to obtain the target execution pose and controls the robot arm to complete the grasping task.

2. The method according to claim 1, characterized in that, In S1, preprocessing includes at least one of workspace clipping, outlier removal, and voxelization downsampling.

3. The method according to claim 1, characterized in that, In S2, the global context modulation module performs global pooling on the point-by-point features output by the backbone network to obtain a global feature vector representing the scene context information. The modulation weight vector with the same dimension as the local geometric features is generated through the gating network. Enhanced grasping features are obtained based on the modulation weight vector and the local geometric feature vectors of the candidate grasping points.

4. The method according to claim 1, characterized in that, In S2, the 6D pose estimation network predicts the grasping approximation direction by using a classification method based on predefined direction templates when predicting the 6D grasping pose. Several candidate direction templates are pre-constructed on a unit sphere, and the probability of each candidate direction is predicted by classification. The direction with the highest probability is selected as the optimal grasping approximation direction.

5. The method according to claim 5, characterized in that, In S2, local geometric features are extracted using a multi-scale cylindrical region reasoning method. Multiple cylindrical local neighborhoods of different scales are established with the candidate grab point as the center and the predicted optimal grab approximation direction as the axis. Point cloud geometric features in each cylindrical neighborhood are extracted, and the multi-scale features are fused.

6. The method according to claim 5, characterized in that, The 6D pose estimation network also outputs the rotation angle around the grasping approximation direction and the grasping quality score corresponding to each candidate grasping pose, and selects the grasping pose with the highest score as the optimal 6D grasping pose.

7. The method according to claim 1, characterized in that, The 6D pose estimation network is trained by constructing a loss function, which is associated with the grasping direction classification loss, rotation angle regression loss, grasping quality score loss, and grasping width classification loss.

8. The method according to claim 1, characterized in that, In S3, the spatial transformation relationship between the camera coordinate system and the robot arm base coordinate system is obtained through hand-eye calibration, and the optimal 6D grasping pose is transformed into the robot arm base coordinate system using homogeneous coordinate transformation.

9. The method according to claim 1, characterized in that, In S3, a smooth interpolation method is used to plan the motion trajectory of the robotic arm's end effector, and the robotic arm is controlled to move along the motion trajectory to complete the grasping task.

10. A 6D pose estimation robotic arm grasping system based on global context modulation, characterized in that, include: The information acquisition and preprocessing module is used to acquire point cloud data of the scene, perform preprocessing, and extract point-by-point features of the point cloud using the backbone network. The 6D pose estimation module is configured with a 6D pose estimation network including a global context modulation module. It is used to extract global context features associated with the scene from point-by-point features, and dynamically modulate the local geometric features of candidate grasping points based on the global context features to generate grasping features carrying global structural information and output the optimal 6D grasping pose. The coordinate transformation and control module is used to transform the optimal 6D grasping pose to the robot arm's base coordinate system and control the robot arm to complete the grasping task.

Images