Robot grasping pose selection method and device, equipment, storage medium and program product

Abstract

The application relates to a robot grasping posture selection method, device, equipment, storage medium and program product. The method comprises the following steps: acquiring scene point cloud data and a real-time scene image of the robot when the robot performs a grasping task; segmenting the scene point cloud data based on the real-time scene image to obtain environment point cloud data and target object point cloud data; inputting the target object point cloud data and the environment point cloud data into a preset collision probability prediction model; predicting the environment collision risk of the robot under each candidate grasping posture through the preset collision probability prediction model to obtain a collision probability prediction value of the robot under each candidate grasping posture; and determining a target grasping posture of the robot from all candidate grasping postures according to all collision probability prediction values. The method improves the grasping posture selection accuracy of the robot.

Description

Technical Field

[0001] This application relates to the field of robot grasping technology, and in particular to a method, apparatus, computer device, computer-readable storage medium, and computer program product for selecting a robot's grasping posture. Background Technology

[0002] With the continuous development of technology, robot grasping technology is also iterating step by step. By selecting the optimal grasping posture, robots can achieve high-precision grasping operations in complex environments. However, when robots perform grasping tasks, how to quickly and accurately assess the grasping posture of the target object in a dynamic environment, ensure the stability of grasping and avoid collisions, is an urgent problem to be solved.

[0003] Currently, robots typically rely on depth cameras or RGB images for object detection when selecting a grasping posture. However, since the position, shape, and posture of objects in dynamic environments are not fixed, direct grasping can lead to significant errors in the grasping posture. Therefore, the current accuracy of robot grasping posture selection is poor. Summary of the Invention

[0004] Therefore, it is necessary to provide a robot grasping posture selection method, device, computer equipment, computer-readable storage medium, and computer program product that improves the accuracy of robot grasping posture selection in order to address the above-mentioned technical problems.

[0005] Firstly, this application provides a method for selecting a robot's grasping posture, the method comprising:

[0006] Acquire scene point cloud data and real-time scene images of the robot when it performs the grasping task;

[0007] The scene point cloud data is segmented based on the real-time scene image to obtain environmental point cloud data and target object point cloud data;

[0008] The target object point cloud data and the environment point cloud data are input into a preset collision probability prediction model. The preset collision probability prediction model is used to predict the environmental collision risk of the robot in each candidate grasping posture, and the collision probability prediction value of the robot in each candidate grasping posture is obtained.

[0009] The target grasping posture of the robot is determined from all candidate grasping postures based on all collision probability predictions.

[0010] In one embodiment, the step of segmenting the scene point cloud data based on the real-time scene image to obtain environmental point cloud data and target object point cloud data includes:

[0011] Based on the real-time scene image, generate a mask image of the area where the target object is located;

[0012] Perform a back projection operation on the mask image to convert the pixels corresponding to the region where the target object is located into point cloud data of the target object;

[0013] The scene point cloud data is filtered based on the mask image to remove the point cloud data corresponding to the area where the non-target object is located, thus obtaining the environment point cloud data.

[0014] In one embodiment, the step of inputting the target object point cloud data and the environment point cloud data into a preset collision probability prediction model, and using the preset collision probability prediction model to predict the environmental collision risk of the robot in each candidate grasping posture, to obtain the collision probability prediction value of the robot in each candidate grasping posture, includes:

[0015] The robot's grasping posture is sampled to obtain multiple candidate grasping postures of the robot;

[0016] Based on the target object point cloud data, generate target point cloud data for each candidate grasping posture of the robot;

[0017] Based on the target point cloud data of the grasping posture and the environmental point cloud data, the collision probability prediction value of the robot under each candidate grasping posture is determined.

[0018] In one embodiment, generating target point cloud data for each candidate grasping posture of the robot based on the target object point cloud data includes:

[0019] Based on the rotation matrix corresponding to each candidate grasping posture, the three-dimensional points in the target object point cloud data are rotated to obtain the rotated target object point cloud data.

[0020] Based on the translation vector corresponding to each candidate grasping posture, the rotated target object point cloud data is transformed by translation to obtain the grasping posture target point cloud data corresponding to each candidate grasping posture.

[0021] In one embodiment, determining the predicted collision probability value of the robot in each candidate grasping posture based on the target point cloud data of the grasping posture and the environmental point cloud data includes:

[0022] Calculate the minimum point cloud distance between the target point cloud data in the grasping posture and the environmental point cloud data;

[0023] Based on the comparison between the minimum distance of the point cloud and the preset collision threshold, the collision risk value of the corresponding candidate grasping posture is determined.

[0024] The collision risk value is mapped to the corresponding collision probability prediction value of the candidate grabbing posture.

[0025] In one embodiment, determining the robot's target grasping posture from all candidate grasping postures based on all collision probability predictions includes:

[0026] The collision probability prediction value corresponding to each of the candidate grasping postures is converted into a stability probability value, and the grasping quality score corresponding to each of the candidate grasping postures is obtained.

[0027] The stability probability value and the corresponding grasping quality score of each candidate grasping posture are weighted and fused to obtain the comprehensive quality score of each candidate grasping posture.

[0028] The target grasping posture of the robot is determined from all candidate grasping postures based on all comprehensive quality scores.

[0029] Secondly, this application also provides a robot grasping posture selection device, comprising:

[0030] The acquisition module is used to acquire scene point cloud data and real-time scene images when the robot performs the grasping task;

[0031] The segmentation module is used to segment the scene point cloud data based on the real-time scene image to obtain environmental point cloud data and target object point cloud data.

[0032] The prediction module is used to input the point cloud data of the target object and the point cloud data of the environment into a preset collision probability prediction model, and to predict the environmental collision risk of the robot in each candidate grasping posture through the preset collision probability prediction model, so as to obtain the collision probability prediction value of the robot in each candidate grasping posture.

[0033] The determination module is used to determine the target grasping posture of the robot from all candidate grasping postures based on all collision probability prediction values.

[0034] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0035] Acquire scene point cloud data and real-time scene images of the robot when performing a grasping task; segment the scene point cloud data based on the real-time scene images to obtain environmental point cloud data and target object point cloud data; input the target object point cloud data and the environmental point cloud data into a preset collision probability prediction model, and predict the environmental collision risk of the robot in each candidate grasping posture through the preset collision probability prediction model to obtain the collision probability prediction value of the robot in each candidate grasping posture; determine the target grasping posture of the robot from all candidate grasping postures based on all collision probability prediction values.

[0036] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0037] Acquire scene point cloud data and real-time scene images of the robot when performing a grasping task; segment the scene point cloud data based on the real-time scene images to obtain environmental point cloud data and target object point cloud data; input the target object point cloud data and the environmental point cloud data into a preset collision probability prediction model, and predict the environmental collision risk of the robot in each candidate grasping posture through the preset collision probability prediction model to obtain the collision probability prediction value of the robot in each candidate grasping posture; determine the target grasping posture of the robot from all candidate grasping postures based on all collision probability prediction values.

[0038] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0039] Acquire scene point cloud data and real-time scene images of the robot when performing a grasping task; segment the scene point cloud data based on the real-time scene images to obtain environmental point cloud data and target object point cloud data; input the target object point cloud data and the environmental point cloud data into a preset collision probability prediction model, and predict the environmental collision risk of the robot in each candidate grasping posture through the preset collision probability prediction model to obtain the collision probability prediction value of the robot in each candidate grasping posture; determine the target grasping posture of the robot from all candidate grasping postures based on all collision probability prediction values.

[0040] The aforementioned robot grasping posture selection method, device, computer equipment, computer-readable storage medium, and computer program product first acquire scene point cloud data and real-time scene images during the robot's grasping task; then, based on the real-time scene images, the scene point cloud data is segmented to obtain environmental point cloud data and target object point cloud data; the target object point cloud data and environmental point cloud data are then input together into a preset collision probability prediction model, which predicts the environmental collision risk of the robot under each candidate grasping posture, obtaining the predicted collision probability value for each candidate grasping posture; finally, based on all the predicted collision probability values, the target grasping posture of the robot is determined from all candidate grasping postures, that is, the target is accurately identified during the robot's grasping task. Furthermore, by leveraging a preset collision probability prediction model, the system can automatically generate collision risk quantification results for each candidate grasping posture. This achieves the goal of automatically selecting the optimal grasping posture for the robot along the collision risk assessment process after completing scene point cloud segmentation. It is applicable to various complex grasping scenarios and can autonomously complete collision risk assessment and posture optimization. Therefore, it overcomes the technical shortcomings of traditional grasping posture selection, which relies solely on pure 3D point cloud segmentation, resulting in insufficient accuracy in distinguishing between the target and background and a tendency for missegmentation. At the same time, posture selection relies solely on a single collision index as a judgment criterion, making it difficult to balance safety and grasping success rate and failing to match actual operational needs. Thus, it improves the accuracy of robot grasping posture selection. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart illustrating the robot's grasping posture selection method in one embodiment;

[0043] Figure 2 This is a flowchart illustrating the robot's grasping posture selection method in another embodiment;

[0044] Figure 3 This is a structural block diagram of a robot's grasping posture selection device in one embodiment;

[0045] Figure 4 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0047] First, it should be understood that current robots primarily rely on deep learning and physical simulation methods for grasping posture selection and collision detection. Specifically, deep learning-based technologies typically identify grasping points using sensor data (such as RGB-D images or point clouds) and predict grasping actions based on trained models. However, this approach faces several problems: 1) High computational resource consumption: Deep learning models require significant computational resources during training and inference, limiting their application in low-power devices or real-time tasks. Especially in complex dynamic environments, the processing speed of deep learning models often struggles to meet real-time grasping requirements, leading to delays and inefficiencies in grasping actions. 2) Deep learning methods typically exhibit poor adaptability to environmental changes, failing to dynamically adjust quality standards to adapt to variations in environmental factors such as light and temperature, resulting in instability and loss of grasping accuracy under different environmental conditions. On the one hand, physical simulation-based methods simulate the entire grasping process by constructing physical models of objects and their environment for collision detection. However, these methods have high computational complexity. Physical simulation requires handling numerous interactions between objects and their environment, especially in three-dimensional space, which significantly increases the computational burden and leads to excessively long computation times, making it difficult to guarantee real-time response speed. Furthermore, because physical simulation methods involve a large amount of physical modeling and simulation, their real-time performance is poor, and they cannot quickly respond to changes in dynamic environments. Especially in complex dynamic environments, physical simulation methods cannot flexibly cope with changes in the states of objects and the environment, resulting in low execution efficiency and a high failure rate for grasping tasks. Therefore, existing physical simulation methods cannot meet the requirements of real-time performance, accuracy, and stability in modern industrial applications. Thus, there is an urgent need for a robot grasping posture selection method that can improve the accuracy of robot posture selection.

[0048] In one embodiment, such as Figure 1As shown, a method for selecting a robot's grasping posture is provided. This embodiment takes the application of this method to a robot grasping posture selection system as an example. The robot's grasping posture selection system is communicatively connected to the robot. It can be understood that the robot's grasping posture selection system is deployed on a computer device. Through the robot's grasping posture selection system, the welding robot can be controlled to perform grasping posture selection tasks. This embodiment takes the application of this method to a terminal as an example. The terminal includes, but is not limited to, a teach pendant, a personal computer, a laptop, a smartphone, and a tablet computer. The terminal includes an acquisition module, a segmentation module, a prediction module, and a determination module. The acquisition module is used to acquire scene point cloud data and real-time scene images when the robot is performing a grasping task. The segmentation module is used to segment the scene point cloud data based on the real-time scene images to obtain environmental point cloud data and target object point cloud data. The prediction module is used to input the target object point cloud data and environmental point cloud data into a preset collision probability prediction model. The preset collision probability prediction model is used to determine the robot's position in each collision. The collision risk of the environment under candidate grasping postures is predicted to obtain the collision probability prediction value of the robot under each candidate grasping posture. The determination module is used to determine the target grasping posture of the robot from all candidate grasping postures based on all collision probability prediction values. That is, through the information interaction between the acquisition module, segmentation module, prediction module and determination module, the target object and surrounding environmental structure are accurately identified during the robot's grasping task. Furthermore, it can automatically generate the collision risk quantification result corresponding to each candidate grasping posture with the help of a preset collision probability prediction model. Thus, after completing the scene point cloud segmentation process, the robot can automatically complete the selection of the optimal grasping posture along the collision risk assessment process. It can be applied to different complex grasping scenarios and can autonomously complete collision risk assessment and posture optimization. Therefore, it improves the accuracy of robot grasping posture selection. It can be understood that this method can also be applied to servers and systems including terminals and servers, and can be implemented through the interaction between terminals and servers. In this embodiment, the method includes the following steps 202 to 208. Wherein:

[0049] Step 202: Obtain scene point cloud data and real-time scene images when the robot is performing the grasping task.

[0050] It should be noted that the robot's grasping task can be triggered automatically by the system or actively by the operator. Active triggering is initiated by the operator's input of grasping task commands into the robot's grasping posture selection system. Automatic triggering is initiated by the robot's grasping posture selection system periodically detecting workstation loading signals and material arrival signals. It is understandable that in the process of performing grasping tasks in complex dynamic scenarios, if posture planning is based solely on single visual data, there will be problems such as incomplete perception of environmental obstacles and target object positioning deviations. For example, in a scenario of scattered stacked industrial workpieces, the three-dimensional spatial position of the object cannot be obtained solely by RGB images, and the target object cannot be accurately distinguished from surrounding clutter solely by point cloud data. Furthermore, in one feasible approach, when there are changes in lighting or partial occlusion of objects in the scene, if multi-source aligned visual data is not acquired simultaneously, the subsequent point cloud segmentation accuracy will decrease, leading to grasping collisions or grasping detachment. Based on the above, the current process of robot grasping posture selection suffers from problems such as insufficient perception data dimensions, unclear distinction between target and environment boundaries, and poor adaptability to dynamic scenarios.

[0051] It should be noted that the scene point cloud data and real-time scene images are synchronously acquired by an RGB-D camera deployed at the robot's end effector or above the workstation. This can include real-time RGB color images and corresponding aligned 3D scene point clouds. The real-time scene images contain 2D texture and region information of the target object and its surrounding environment. The scene point cloud data is a set of points in 3D space, containing the 3D coordinate information of all objects in the scene. For example, in one feasible approach, the acquired data includes RGB image pixel information, 3D point coordinates corresponding to the depth map, camera intrinsic and extrinsic calibration parameters, acquisition timestamps, etc. The real-time scene images are used to generate target segmentation masks, and the scene point cloud data is used for 3D collision detection and attitude calculation. That is, when the robot receives a grasping task instruction, the robot's grasping attitude selection system automatically triggers the RGB-D camera to complete the data acquisition of the current scene and perform preprocessing such as timestamp alignment, distortion correction, and point cloud downsampling to facilitate subsequent point cloud segmentation and collision risk calculation.

[0052] As an example, when the robot detects the arrival signal of materials at the workstation and automatically triggers the grasping task, the robot's grasping posture selection system will control the RGB-D camera to collect the real-time RGB image of the current workstation and the corresponding scene point cloud, and then complete and process the recording of timestamps, camera pose parameters, scene point cloud size and image resolution information, etc.

[0053] Understandably, camera intrinsic parameters are used to perform back-projection calculations from 2D pixels to 3D points, camera extrinsic parameters are used to perform pose transformations from the camera coordinate system to the robot base coordinate system, real-time scene images are used to identify the regional location of target objects, and scene point cloud data is used to characterize the actual positions of objects and obstacles in 3D space. The alignment of their timestamps is used to ensure the spatial correspondence between the 2D segmentation results and the 3D point cloud data, avoiding segmentation deviations caused by data asynchrony.

[0054] Step 204: Segment the scene point cloud data based on the real-time scene image to obtain environmental point cloud data and target object point cloud data.

[0055] It is important to note that point cloud segmentation is the core foundation of grasping posture planning. Segmentation accuracy directly affects the accuracy of subsequent collision detection. Understandably, the required boundary accuracy between the segmented target point cloud and the environment point cloud varies depending on the target object in different scenarios. That is, in compact scenarios where the target object is close to surrounding obstacles, higher precision mask segmentation results are needed to avoid misclassifying the environment point cloud as the target point cloud, which would lead to deviations in collision risk calculation. For example, in one feasible approach, assuming the target object is a randomly stacked set of small precision parts, if the mask boundary exceeds the actual range of the target object, the extracted target point cloud will include points of surrounding obstacles, resulting in a lower calculated collision probability and triggering an actual grasping collision. Conversely, assuming the target object is a large box-shaped material, if the mask boundary is smaller than the actual range of the target object, the target point cloud will be missing, leading to incomplete sampling range of grasping posture and missing feasible, highly stable grasping postures.

[0056] It should be noted that the mask image is a two-dimensional binary image generated based on real-time scene images. It is used to identify whether a pixel belongs to the target object region. Specifically, it can be a target object region mask generated by an instance segmentation network. In the mask image, the region with a pixel value of 1 corresponds to the location of the target object, and the region with a pixel value of 0 corresponds to the location of the background environment. In addition, the point cloud segmentation process is automatically triggered after the data preprocessing is completed. That is, after the RGB-D camera completes data acquisition and preprocessing, the system automatically triggers the point cloud segmentation process. For example, in one feasible approach, assuming the operator selects the type of target workpiece to be captured on the system interface, the system automatically calls the corresponding instance segmentation model to infer the real-time RGB image and generate the corresponding binary mask image of the target object, thus completing the two-dimensional localization of the target region.

[0057] As an example, step 204 includes: inputting a pre-trained instance segmentation model with a real-time scene image to generate a binary mask image of the region where the target object is located; performing a back projection operation on the target region with a pixel value of 1 in the mask image, and converting the two-dimensional pixels into three-dimensional spatial points in combination with camera intrinsic parameters to obtain the target object point cloud data; filtering the original scene point cloud based on the mask image, removing the point cloud corresponding to the target region with a mask value of 1, and obtaining the environment point cloud data.

[0058] Step 206: Based on the workpiece welding path, control the welding robot to select the robot's gripping posture.

[0059] It should be noted that collision probability prediction is used to assess the risk of collision between the robot and the surrounding environment when the robot grasps the target object under each candidate grasping posture. It is a core link to ensure grasping safety. It can be understood that different candidate grasping postures correspond to different grasping poses of the target object. When the robot grasps the target object in this posture, the spatial pose of the target object will change accordingly. If the target object after the change has spatial overlap with the environmental point cloud or is too close, a collision will occur. For example, in one feasible method, if the distance between the target object corresponding to the candidate grasping posture and the side wall of the workstation is less than the safe distance after rotation, a collision will occur during the robot's lifting and movement. If this risk is not predicted in advance, it will lead to problems such as workpiece collision and damage to the robotic arm structure.

[0060] It should be noted that the candidate grasping postures are a set of poses obtained by sampling 6 degrees of freedom within the graspable space of the target object's point cloud. Each candidate posture is represented by a corresponding rotation matrix and translation vector, which correspond to the posture rotation angle and grasping point spatial position during grasping, respectively. In addition, the collision probability prediction model in this scheme does not require complex full-scene physical simulation. Risk assessment can be completed simply by calculating the spatial distance of the point cloud, which can significantly reduce the consumption of computing resources, improve the inference speed, and meet the response requirements of real-time grasping. For example, in one feasible approach, the system uniformly samples multiple grasping points on the graspable surface of the target object, and generates multiple feasible candidate grasping postures by combining the opening and closing direction and angle constraints of the grasper, and calculates the collision risk under each posture one by one.

[0061] As an example, step 206 includes: firstly, sampling 6-DOF grasping postures in the graspable area of ​​the target object point cloud in conjunction with the grasper parameters to obtain multiple candidate grasping postures of the robot; for each candidate grasping posture, rotating each 3D point in the target object point cloud according to its corresponding rotation matrix to obtain rotated target object point cloud data, and then translating the rotated point cloud according to the corresponding translation vector to obtain grasping posture target point cloud data corresponding to each candidate posture; calculating the minimum point cloud distance between the grasping posture target point cloud data and the environmental point cloud data, determining the collision risk value of the corresponding candidate posture based on the comparison result of the minimum point cloud distance and the preset collision threshold, and then mapping the collision risk value to a collision probability prediction value in the 0-1 interval.

[0062] Understandably, the rotation matrix is ​​used to represent the rotation angle of the target object during grasping, the translation vector is used to represent the actual position of the grasping point in three-dimensional space, and the minimum point cloud distance is the minimum Euclidean distance between all point pairs in the transformed target point cloud and the environment point cloud. If the minimum Euclidean distance is less than the preset collision threshold, it is determined that there is a collision risk in this posture. The smaller the distance, the higher the probability of collision. If the minimum distance is greater than the preset safe distance threshold, it is determined that there is no collision risk in this posture and the collision probability is 0.

[0063] Step 208: Determine the robot's target grasping posture from all candidate grasping postures based on all collision probability predictions.

[0064] It should be noted that selecting a grasping posture solely based on collision probability may result in the posture meeting safety standards but insufficient grasping firmness, leading to object detachment. Conversely, selecting only a grasping quality as a single indicator may select postures with collision risks, causing workpiece impacts or damage to the robotic arm. Therefore, this step employs a two-dimensional fusion approach to achieve a comprehensive assessment of the posture. The collision probability prediction value corresponds inversely to the safety and stability of the grasping process, while the grasping quality score corresponds to the clamping stability of the grasping action itself. The fusion weight of these two dimensions can be dynamically adjusted according to the current working conditions. For example, in scenarios with scattered material boxes and dense obstacles, the weight of the safety dimension can be increased; in scenarios involving grasping heavy workpieces in open workstations, the weight of the clamping quality dimension can be increased. This achieves dynamic adaptation of the quality threshold, addressing the shortcomings of existing technologies where evaluation standards are fixed and cannot adapt to dynamic scenarios.

[0065] As an example, step 208 includes: normalizing and back-mapping the predicted collision probability values ​​corresponding to each candidate grasping posture to convert them into stability probability values ​​ranging from 0 to 1. The lower the collision probability, the higher the corresponding stability probability value, indicating that the risk of the robot colliding with the environment in this posture is lower. At the same time, based on the local geometric features of the target object point cloud, the angle between the normal vectors of the grasping contact points, the contact area between the gripper opening and closing stroke and the object surface, the grasping quality score corresponding to each candidate grasping posture is calculated. The higher the grasping quality score, the higher the firmness of the gripped object in this posture. According to the obstacle distribution density of the current scene and the material type of the workpiece to be grasped, the corresponding fusion weight coefficient is preset, and the stability probability value and the corresponding grasping quality score of each candidate posture are weighted and summed to obtain the comprehensive quality score of each candidate grasping posture. The comprehensive quality scores of all candidate postures are sorted, and the candidate posture with the highest comprehensive quality score is selected as the target grasping posture of the robot. Based on the target posture, the robot inverse kinematics solution is completed, the motion trajectory of the robotic arm is generated, and the robot is controlled to perform the grasping action of the target object.

[0066] The aforementioned method for selecting the robot's grasping posture first acquires scene point cloud data and real-time scene images during the robot's grasping task. Then, it segments the scene point cloud data based on the real-time scene images to obtain environmental point cloud data and target object point cloud data. These target object point cloud data and environmental point cloud data are then input into a preset collision probability prediction model. This model predicts the environmental collision risk for each candidate grasping posture, yielding a predicted collision probability value for each posture. Finally, based on all predicted collision probabilities, the target grasping posture is determined from all candidate postures. In other words, the target object and surrounding environmental structure are accurately identified during the robot's grasping task, further enhancing its grasping capabilities. Furthermore, it can automatically generate collision risk quantification results for each candidate grasping posture using a preset collision probability prediction model. This achieves the goal of automatically selecting the optimal grasping posture of the robot along the collision risk assessment process after completing scene point cloud segmentation. It is applicable to different complex grasping scenarios and can autonomously complete collision risk assessment and posture optimization. Therefore, it overcomes the technical defects of traditional grasping posture selection, which relies solely on pure 3D point cloud segmentation, resulting in insufficient accuracy in distinguishing between the target and background and easy missegmentation. At the same time, posture selection relies solely on a single collision index as the judgment criterion, which makes it difficult to balance safety and grasping success rate and cannot match actual operation requirements. Therefore, it improves the accuracy of robot grasping posture selection.

[0067] In one embodiment, such as Figure 2As shown, scene point cloud data is segmented based on real-time scene images to obtain environmental point cloud data and target object point cloud data, including:

[0068] Step 302: Generate a mask image of the area where the target object is located based on the real-time scene image.

[0069] It should be noted that the mask image is a two-dimensional binary image with the same size as the real-time scene image. It is used to identify whether a pixel belongs to the target object region. Specifically, it can be generated through inference by the instance segmentation network. A mask value of 1 indicates that the corresponding pixel belongs to the target object, and a mask value of 0 indicates that the corresponding pixel belongs to the environmental background. In some complex stacked scenes, since the geometric shape of the target object is highly similar to that of the surrounding workpieces, the target boundary cannot be accurately distinguished by point cloud geometric features alone. In this case, the region is not directly divided based on the original point cloud. Instead, the range of the target region is determined first by the visual semantic information of the real-time scene image, which provides an accurate regional basis for subsequent point cloud extraction.

[0070] As an example, step 302 includes: inputting the real-time scene image after distortion correction into a pre-trained instance segmentation model, and combining it with a pre-selected target object category identifier to infer a pixel-level binary mask image of the region where the target object is located.

[0071] Step 304: Perform a back projection operation on the mask image to convert the pixels corresponding to the area where the target object is located into point cloud data of the target object.

[0072] It should be noted that during the point cloud segmentation process, the system can establish a mapping relationship between two-dimensional pixels and three-dimensional space based on pre-calibrated camera parameters, and convert the pixel area covered by the mask into a three-dimensional point set. The target object point cloud refers to a point set that only contains the three-dimensional coordinates of the object to be captured, which is the core basic data for subsequent capture posture sampling and target pose transformation.

[0073] As an example, step 304 includes: extracting the pre-calibrated RGB-D camera intrinsic parameter matrix and the extrinsic parameter transformation matrix from the camera coordinate system to the robot base coordinate system; combining the pixel coordinates of all target areas with a mask value of 1 in the mask image with the depth values ​​of the corresponding positions; calculating the three-dimensional spatial coordinates corresponding to each pixel through back projection of the camera imaging model; and obtaining the target object point cloud data after removing outliers and downsampling voxels from all three-dimensional coordinate points.

[0074] Step 306: Filter the scene point cloud data based on the mask image, remove the point cloud data corresponding to the area where the non-target object is located, and obtain the environment point cloud data.

[0075] As an example, step 306 includes: traversing all three-dimensional points in the original scene point cloud, projecting the three-dimensional points from the robot base coordinate system back to the camera's two-dimensional image plane through the extrinsic parameter matrix, and removing the point from the original scene point cloud if the projected pixel falls in the target area with a mask value of 1 in the mask image. After the remaining three-dimensional points are filtered for outliers, the environmental point cloud data is obtained.

[0076] In this embodiment, in complex scenarios requiring precise segmentation of scene point clouds, a pixel-level mask image of the target object can be generated first based on the real-time scene image. Then, the target object point cloud can be accurately extracted through back projection. Finally, a clean environmental point cloud can be obtained through mask filtering. This allows for precise division of the target and environment boundaries based on the visual semantic information of the RGB image if the target and environment have similar geometric features and the point cloud boundaries are blurry and difficult to distinguish during point cloud segmentation. This avoids boundary errors caused by pure point cloud segmentation. Therefore, while improving the accuracy of point cloud segmentation, it lays the foundation for improving the accuracy of subsequent collision probability prediction and grasping posture selection.

[0077] In one embodiment, target object point cloud data and environmental point cloud data are jointly input into a preset collision probability prediction model. The preset collision probability prediction model is used to predict the environmental collision risk of the robot in each candidate grasping posture, and the predicted collision probability value of the robot in each candidate grasping posture is obtained, including:

[0078] The robot's grasping posture is sampled to obtain multiple candidate grasping postures; based on the target object point cloud data, the target point cloud data of the grasping posture corresponding to each candidate grasping posture is generated; based on the target point cloud data of the grasping posture and the environmental point cloud data, the predicted collision probability value of the robot under each candidate grasping posture is determined.

[0079] It should be noted that each candidate grasping posture is represented by a corresponding rotation matrix and translation vector. The target point cloud of the grasping posture is a three-dimensional point set of the target object point cloud after corresponding posture transformation. The calculation rules for the collision probability between the target point cloud of the grasping posture and the environment point cloud differ under different threshold conditions. That is, if the minimum distance of the point cloud meets the first preset threshold condition, it can be directly determined that the candidate posture has no collision risk. If the minimum distance of the point cloud meets the second preset threshold condition, the corresponding collision risk value needs to be calculated based on the distance value. The first preset threshold condition and the second preset threshold condition are mutually exclusive judgment conditions. For example, in one feasible method, it is assumed that the first preset threshold condition is that the minimum distance of the point cloud is greater than or equal to the preset safe distance threshold, and the second preset threshold condition is that the minimum distance of the point cloud is less than the preset safe distance threshold. The collision risk value is used to characterize the probability of the transformed target object colliding with the robotic arm and the surrounding environment when the robot grasps the target under the candidate grasping posture.

[0080] As an example, firstly, 6-DOF grasping posture sampling is performed on the graspable surface of the target object point cloud, combined with the gripper opening and closing parameters, to obtain multiple candidate grasping postures for the robot. For each candidate grasping posture, all 3D points in the target object point cloud are rotated according to their corresponding rotation matrix to obtain rotated target object point cloud data. Then, the rotated point cloud is translated according to the translation vector corresponding to the posture to obtain the grasping posture target point cloud data corresponding to the candidate posture. The Euclidean distance between all point pairs between the grasping posture target point cloud and the environment point cloud is calculated, and the minimum value is taken as the minimum point cloud distance. If the minimum point cloud distance is greater than or equal to a preset safety distance threshold, the collision probability prediction value of the candidate grasping posture is directly determined to be 0. If the minimum point cloud distance is less than the preset safety distance threshold, the corresponding collision risk value is calculated based on the difference between the minimum distance and the preset collision threshold, and the collision risk value is normalized and mapped to a collision probability prediction value in the range of 0-1. The smaller the minimum point cloud distance, the higher the collision probability prediction value. This embodiment determines the threshold condition satisfied by the minimum distance of the point cloud and uses corresponding rules to calculate the collision probability prediction value of each candidate posture. This ensures that the collision risk prediction of all candidate grasping postures can be completed quickly and accurately without the need for complex deep learning inference and full-scene physical simulation. This significantly reduces the computational resource consumption of the collision detection process and meets the real-time grasping response requirements in dynamic scenarios. Therefore, it lays a foundation for improving the real-time performance and accuracy of robot grasping posture selection.

[0081] In one embodiment, based on the target object point cloud data, the robot generates target point cloud data for each candidate grasping posture, including:

[0082] Based on the rotation matrix corresponding to each candidate grasping posture, the three-dimensional points in the target object point cloud data are rotated to obtain the rotated target object point cloud data; based on the translation vector corresponding to each candidate grasping posture, the rotated target object point cloud data is translated to obtain the grasping posture target point cloud data corresponding to each candidate grasping posture.

[0083] It should be noted that when performing grasping tasks, the robot will adopt different grasping posture types depending on the shape of the target object and the grasping scenario, such as grasping from the horizontal top surface, grasping from the vertical side, and grasping from an inclined angle. The rotation matrix is ​​used to represent the rotation angle of the target object relative to the initial pose during grasping, and the translation vector is used to represent the offset of the grasping point relative to the centroid of the target object. However, the corresponding rotation angle and the offset of the grasping point differ under different grasping posture types. Therefore, it is necessary to calculate the coordinate transformation of the target point cloud of the grasping posture by using the pose parameters carried by the candidate grasping posture.

[0084] As an example, for candidate poses involving horizontal grasping of the top surface of a target object, the 3D coordinates of the top surface grasping point and the vertically downward orientation of the gripper are extracted from the candidate pose parameters. The corresponding rotation matrix is ​​calculated based on the grasping orientation direction. All 3D points in the target object's point cloud are then rotated to make the top surface grasping plane of the target object parallel to the gripper's opening and closing plane. Next, the corresponding translation vector is calculated based on the coordinates of the top surface grasping point. The position coinciding with the gripper's grasping center is located in the rotated point cloud, and the entire point cloud is translated to this position to obtain the target point of the grasping pose corresponding to this type of candidate pose. For candidate poses involving vertical grasping of the target object's side, the three-dimensional coordinates of the side grasping points and the attitude direction of the gripper's horizontal orientation are extracted from the candidate pose parameters. The corresponding rotation matrix is ​​calculated based on the side grasping attitude direction. All three-dimensional points in the target object's point cloud are rotated to make the side grasping surface of the target object parallel to the gripper's opening and closing plane. Then, the corresponding translation vector is calculated based on the coordinates of the side grasping points. The position that coincides with the gripper's grasping center is located in the rotated point cloud. The entire point cloud is translated to this position to obtain the target point cloud of the grasping pose corresponding to this type of candidate pose.

[0085] In this embodiment, by obtaining pose information of different types of poses from candidate grasping pose parameters and performing rotation and translation coordinate transformation of the point cloud, the actual pose of the target object under different grasping poses can be accurately simulated in three-dimensional space. This achieves the purpose of accurately calculating the target point cloud pose under different grasping types, avoiding collision distance calculation errors caused by point cloud pose transformation deviations. Therefore, it lays the foundation for further improving the accuracy of robot grasping pose collision detection and selection.

[0086] In one embodiment, determining the predicted collision probability of the robot in each candidate grasping posture based on the target point cloud data and the environmental point cloud data includes:

[0087] Calculate the minimum point cloud distance between the target point cloud data and the environment point cloud data in the grasping posture; determine the collision risk value of the corresponding candidate grasping posture based on the comparison result of the minimum point cloud distance and the preset collision threshold; map the collision risk value to the collision probability prediction value of the corresponding candidate grasping posture.

[0088] It should be noted that, in order to ensure the accuracy and real-time performance of collision detection, corresponding preset collision thresholds and distance risk mapping rules can be set for different grasping scenarios. Risk calculation can be completed without complex full-scene physical simulation. Specifically, the preset collision threshold can be 2mm, 3mm, or 5mm, etc., and the safety margin of the distance-risk mapping can be adjusted according to the type of workpiece. For example, a larger safety distance can be set for precision and fragile parts scenarios.

[0089] As an example, the process iterates through all 3D points in the target point cloud data and the environment point cloud data of the grasping posture, calculates the Euclidean distance between point pairs, and takes the minimum value as the minimum point cloud distance. It obtains the preset collision threshold and safety margin parameters of the current grasping scene, and determines the collision risk value of the corresponding candidate posture based on the comparison result between the minimum point cloud distance and the preset collision threshold. The smaller the minimum point cloud distance, the higher the collision risk value. The calculated collision risk value is normalized and mapped to a value in the range of 0-1 as the predicted collision probability value of the candidate grasping posture. The collision probability calculation process is automatically triggered after the target point cloud of the grasping posture is generated.

[0090] In one embodiment, determining the robot's target grasping pose from all candidate grasping poses based on all collision probability predictions includes:

[0091] The collision probability prediction value corresponding to each candidate grasping posture is converted into a stability probability value, and the grasping quality score corresponding to each candidate grasping posture is obtained. The stability probability value and the corresponding grasping quality score of each candidate grasping posture are weighted and fused to obtain the comprehensive quality score of each candidate grasping posture. Based on all comprehensive quality scores, the target grasping posture of the robot is determined from all candidate grasping postures.

[0092] It should be noted that in scenarios where the stability requirements for grasping cannot be met by screening based solely on the collision probability index, or where there is a risk of collision when screening based solely on the grasping quality index, it is necessary to combine the safety and stability indicators to calculate a comprehensive quality score for the candidate grasping postures to determine the optimal posture. The target grasping posture refers to the 6-DOF pose that the robot ultimately adopts when performing the grasping action, which is obtained through the dual-dimensional weighted fusion optimization.

[0093] As an example, the predicted collision probability values ​​of each candidate posture are converted into stability probability values ​​through inverse normalization, and the grasping quality score and scene dynamic fusion weight parameters corresponding to each candidate grasping posture are obtained. The grasping quality score includes the grasping contact area, the angle between the point cloud normal vectors, and the force closure score. According to the scene dynamic fusion weight parameters, the stability probability value and the corresponding grasping quality score are weighted and summed to obtain the comprehensive quality score of each candidate posture. The candidate posture with the highest comprehensive quality score is selected as the robot's target grasping posture.

[0094] In one feasible approach, the robot's grasping posture selection process is as follows: 1) First, input and preparation work is performed. Specifically, the input data includes real-time scene images and scene point cloud data. The target image is an image acquired by an RGB-D camera, which contains visual information about the target object and its surrounding environment. The scene point cloud data is point cloud data acquired by an RGB-D camera, which includes the target object point cloud P and the environmental structure point cloud. Scene point cloud It is a set of points representing objects and their environment in three-dimensional space, represented as: ,in, Let P be the unit coordinate of the Nth point cloud. P is obtained by segmenting the scene point cloud using a depth mask image, while the environment point cloud P is the remaining part of the scene point cloud after removing the target object; 2) Then, the scene point cloud data is segmented, which is performed by the perception module. Specifically, based on the depth mask image generated from the target image... Extract the point cloud of the target object region, assuming It is a two-dimensional binary mask that represents each pixel value corresponding to the RGB image 1. M t arg et ( i , j ) ∈ [ 0 , 1 ] ,in, This indicates that the pixel belongs to the target object. Through back projection, the target area of ​​the mask image is converted into a 3D point cloud: Where P is the target point cloud (carrying target point cloud data), and the environmental point cloud is... (Carrying environmental point cloud data); 3) Then, the collision probability prediction model of the collision detection module makes a prediction. Specifically, the target object point cloud data and the environmental point cloud data are input into the preset collision probability prediction model, and the output is the collision probability prediction value of the robot in each candidate grasping posture. In order to ensure the stability of the grasping action, the geometric information of the target object and the environmental point cloud are used. Collision detection is performed; specifically, the first step is to predict the grab pose for each candidate. The grasping posture is determined by the rotation matrix. Translation vector express:

[0095] T candidate = [ R candidate t candidate 0 1 ]

[0096] Stability prediction is based on the relative geometric relationship between the object and the environment, assuming candidate grasping poses. It will act on every point of the object's point cloud P, and through transformation, obtain the object's point cloud in the new pose. , Next, the collision detection algorithm evaluates... Is it related to environmental point clouds? When a collision occurs, the collision risk between the object and the environment is calculated, and the stability probability p of the grasping posture is output. If the grasping posture can stably grasp the object and avoid collisions, the value of p is higher. It can be understood that the core calculation of collision detection is to examine every point of the target object. Is it related to obstacles in the environment? A collision occurs. A simple form of collision detection is to calculate the distance between points; if the distance is less than a certain threshold... If so, a collision is considered to have occurred, and the collision risk is considered to be high. It can be calculated using the following formula:

[0097]

[0098] Based on the collision detection results, the stability probability p is calculated as follows:

[0099]

[0100] Where N is the number of points in the point cloud of the target object. This refers to the collision risk. The calculated probability p represents the stability of the grasping posture; a higher probability indicates a more stable grasping posture. After the collision detection module completes, the system selects the most feasible grasping posture as the final action based on the stability probability p. Assume there are N candidate grasping postures. , ... Each candidate grasping posture corresponds to a stability probability. The system will select the grasping posture with the largest value of p:

[0101]

[0102] Choose the optimal grasping posture It is a 6-DOF pose, including rotation matrix and displacement vector. Finally, the system will calculate the robot's motion trajectory based on the grasping pose and execute precise grasping actions.

[0103] In other words, during the robot's grasping task, the target object and surrounding environment structure are accurately identified. Furthermore, it can automatically generate collision risk quantification results for each candidate grasping posture using a preset collision probability prediction model. This achieves the goal of automatically selecting the optimal grasping posture of the robot along the collision risk assessment process after completing scene point cloud segmentation. It is applicable to different complex grasping scenarios and can autonomously complete collision risk assessment and posture optimization. Therefore, it overcomes the technical defects of traditional grasping posture selection, which relies solely on pure 3D point cloud segmentation, resulting in insufficient accuracy in distinguishing between the target and background and easy missegmentation. At the same time, posture selection relies solely on a single collision index as a judgment criterion, which makes it difficult to balance safety and grasping success rate and cannot match actual operation requirements. Therefore, it improves the accuracy of robot grasping posture selection.

[0104] Based on the same inventive concept, this application also provides a robot grasping posture selection device for implementing the above-described robot grasping posture selection method. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more robot grasping posture selection device embodiments provided below can be found in the limitations of the robot grasping posture selection method described above, and will not be repeated here.

[0105] In one exemplary embodiment, such as Figure 3 As shown, a robot grasping posture selection device is provided, including: an acquisition module 401, a segmentation module 402, a prediction module 403, and a determination module 404, wherein:

[0106] The acquisition module 401 is used to acquire scene point cloud data and real-time scene images when the robot is performing a grasping task;

[0107] The segmentation module 402 is used to segment scene point cloud data based on real-time scene images to obtain environmental point cloud data and target object point cloud data;

[0108] The prediction module 403 is used to input the target object point cloud data and the environment point cloud data into the preset collision probability prediction model. The preset collision probability prediction model is used to predict the environmental collision risk of the robot under each candidate grasping posture, and obtain the collision probability prediction value of the robot under each candidate grasping posture.

[0109] The determination module 404 is used to extract the pose from all candidate grasping positions based on all collision probability predictions.

[0110] In one embodiment, the segmentation module 402 is further configured to:

[0111] Based on the real-time scene image, a mask image of the area where the target object is located is generated; a back projection operation is performed on the mask image to convert the pixels corresponding to the area where the target object is located into point cloud data of the target object; the scene point cloud data is filtered based on the mask image to remove the point cloud data corresponding to the area where the target object is located, so as to obtain the environment point cloud data.

[0112] In one embodiment, the prediction module 403 is further configured to:

[0113] The robot's grasping posture is sampled to obtain multiple candidate grasping postures; based on the target object point cloud data, the target point cloud data of the grasping posture corresponding to each candidate grasping posture is generated; based on the target point cloud data of the grasping posture and the environmental point cloud data, the predicted collision probability value of the robot under each candidate grasping posture is determined.

[0114] In one embodiment, the prediction module 403 is further configured to:

[0115] Based on the rotation matrix corresponding to each candidate grasping posture, the three-dimensional points in the target object point cloud data are rotated to obtain the rotated target object point cloud data; based on the translation vector corresponding to each candidate grasping posture, the rotated target object point cloud data is translated to obtain the grasping posture target point cloud data corresponding to each candidate grasping posture.

[0116] In one embodiment, the prediction module 403 is further configured to:

[0117] Calculate the minimum point cloud distance between the target point cloud data and the environment point cloud data in the grasping posture; determine the collision risk value of the corresponding candidate grasping posture based on the comparison result of the minimum point cloud distance and the preset collision threshold; map the collision risk value to the collision probability prediction value of the corresponding candidate grasping posture.

[0118] In one embodiment, the determining module 404 is further configured to:

[0119] The collision probability prediction value corresponding to each candidate grasping posture is converted into a stability probability value, and the grasping quality score corresponding to each candidate grasping posture is obtained. The stability probability value and the corresponding grasping quality score of each candidate grasping posture are weighted and fused to obtain the comprehensive quality score of each candidate grasping posture. Based on all comprehensive quality scores, the target grasping posture of the robot is determined from all candidate grasping postures.

[0120] The various modules in the robot's grasping posture selection device described above can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0121] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a method for selecting a robot's grasping posture. Those skilled in the art will understand that... Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0122] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0123] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0124] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0125] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0126] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0127] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for selecting a robot's grasping posture, characterized in that, The method includes: Acquire scene point cloud data and real-time scene images of the robot when it performs the grasping task; The scene point cloud data is segmented based on the real-time scene image to obtain environmental point cloud data and target object point cloud data; The target object point cloud data and the environment point cloud data are input into a preset collision probability prediction model. The preset collision probability prediction model is used to predict the environmental collision risk of the robot in each candidate grasping posture, and the collision probability prediction value of the robot in each candidate grasping posture is obtained. The target grasping posture of the robot is determined from all candidate grasping postures based on all collision probability predictions.

2. The method according to claim 1, characterized in that, The step of segmenting the scene point cloud data based on the real-time scene image to obtain environmental point cloud data and target object point cloud data includes: Based on the real-time scene image, generate a mask image of the area where the target object is located; Perform a back projection operation on the mask image to convert the pixels corresponding to the region where the target object is located into point cloud data of the target object. The scene point cloud data is filtered based on the mask image to remove the point cloud data corresponding to the area where the non-target object is located, thus obtaining the environment point cloud data.

3. The method according to claim 1, characterized in that, The step of inputting the target object point cloud data and the environment point cloud data into a preset collision probability prediction model, and using the preset collision probability prediction model to predict the environmental collision risk of the robot in each candidate grasping posture, to obtain the collision probability prediction value of the robot in each candidate grasping posture, includes: The robot's grasping posture is sampled to obtain multiple candidate grasping postures of the robot; Based on the target object point cloud data, generate target point cloud data for each candidate grasping posture of the robot; Based on the target point cloud data of the grasping posture and the environmental point cloud data, the collision probability prediction value of the robot under each candidate grasping posture is determined.

4. The method according to claim 3, characterized in that, The step of generating target point cloud data for each candidate grasping posture of the robot based on the target object point cloud data includes: Based on the rotation matrix corresponding to each candidate grasping posture, the three-dimensional points in the target object point cloud data are rotated to obtain the rotated target object point cloud data. Based on the translation vector corresponding to each candidate grasping posture, the rotated target object point cloud data is transformed by translation to obtain the grasping posture target point cloud data corresponding to each candidate grasping posture.

5. The method according to claim 3, characterized in that, The step of determining the collision probability prediction value of the robot in each of the candidate grasping postures based on the target point cloud data of the grasping posture and the environmental point cloud data includes: Calculate the minimum point cloud distance between the target point cloud data in the grasping posture and the environmental point cloud data; Based on the comparison between the minimum distance of the point cloud and the preset collision threshold, the collision risk value of the corresponding candidate grasping posture is determined. The collision risk value is mapped to the corresponding collision probability prediction value of the candidate grabbing posture.

6. The method according to claim 1, characterized in that, Determining the robot's target grasping posture from all candidate grasping postures based on all collision probability predictions includes: The collision probability prediction value corresponding to each of the candidate grasping postures is converted into a stability probability value, and the grasping quality score corresponding to each of the candidate grasping postures is obtained. The stability probability value and the corresponding grasping quality score of each candidate grasping posture are weighted and fused to obtain the comprehensive quality score of each candidate grasping posture. The target grasping posture of the robot is determined from all candidate grasping postures based on all comprehensive quality scores.

7. A device for selecting the grasping posture of a robot, characterized in that, The device includes: The acquisition module is used to acquire scene point cloud data and real-time scene images when the robot performs the grasping task; The segmentation module is used to segment the scene point cloud data based on the real-time scene image to obtain environmental point cloud data and target object point cloud data. The prediction module is used to input the target object point cloud data and the environment point cloud data into a preset collision probability prediction model, and to predict the environmental collision risk of the robot in each candidate grasping posture through the preset collision probability prediction model, so as to obtain the collision probability prediction value of the robot in each candidate grasping posture. The determination module is used to determine the target grasping posture of the robot from all candidate grasping postures based on all collision probability prediction values.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images