Workpiece grasping method, device, equipment and storage medium

Abstract

This invention discloses a workpiece gripping method, apparatus, device, and storage medium. The method includes: acquiring a scene point cloud corresponding to a work scene using a depth camera; determining a first workpiece posture and a first material frame posture in the depth camera coordinate system based on a point cloud matching algorithm, according to the scene point cloud, a pre-constructed workpiece model, and a pre-constructed material frame model; determining a target gripping posture of the robotic arm based on a preset correspondence between the depth camera coordinate system and the robotic arm coordinate system, as well as the first workpiece posture, the first material frame posture, and a preset gripping posture; acquiring the adjustment joint angle of the adjustable gripper corresponding to the target gripping posture; and controlling the robotic arm to grip the workpiece in the material frame according to the adjustment joint angle. This invention utilizes the additional degrees of freedom provided by the adjustable gripper to effectively avoid collisions in the scene, reduce the probability of damage to the robotic arm, and improve the gripping success rate when using a high-degree-of-freedom robotic arm, which is more expensive.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and in particular to a workpiece gripping method, apparatus, device, and storage medium. Background Technology

[0002] In the traditional process of feeding track links during production, workers need to manually handle heavy metal track links, which can easily cause physical injury during prolonged labor. Therefore, it is necessary to use robotic arms for automated gripping. However, the geometric model of track links is relatively complex, and they are usually placed in a material frame. When the robotic arm performs gripping, it is easy to collide with the material frame or other workpieces. Existing technology cannot effectively avoid obstacles observed by depth cameras and grip track links in the material frame. Summary of the Invention

[0003] This invention provides a workpiece gripping method, apparatus, device, and storage medium to effectively avoid collisions in a scene, reduce the probability of damage to the robotic arm, and improve the gripping success rate.

[0004] According to one aspect of the present invention, a workpiece gripping method is provided, the method comprising:

[0005] The scene point cloud corresponding to the working scene is obtained by a depth camera. The working scene includes a robotic arm, a material frame and a workpiece in the material frame. The end of the robotic arm is equipped with an adjustable gripper, and the adjustable gripper includes an adjustment joint.

[0006] Based on the point cloud matching algorithm, the first workpiece pose and the first material frame pose in the depth camera coordinate system are determined according to the scene point cloud, the pre-built workpiece model and the pre-built material frame model.

[0007] Based on the correspondence between the preset depth camera coordinate system and the robotic arm coordinate system, as well as the first workpiece posture, the first material frame posture and the preset grasping posture, the target grasping posture of the robotic arm is determined.

[0008] Obtain the adjustment joint angle of the adjustable gripper corresponding to the target grasping posture;

[0009] The robotic arm is controlled to grip the workpiece in the material frame by adjusting the joint angle.

[0010] According to another aspect of the present invention, a workpiece gripping device is provided, the device comprising:

[0011] The first acquisition module is used to acquire scene point cloud corresponding to the work scene through a depth camera. The work scene includes a robotic arm, a material frame and a workpiece in the material frame. The end of the robotic arm is equipped with an adjustable gripper, and the adjustable gripper includes an adjustment joint.

[0012] The first determining module is used to determine the first workpiece pose and the first material frame pose in the depth camera coordinate system based on the point cloud matching algorithm, according to the scene point cloud, the pre-built workpiece model and the pre-built material frame model.

[0013] The second determining module is used to determine the target grasping posture of the robotic arm based on the correspondence between the preset depth camera coordinate system and the robotic arm coordinate system, as well as the first workpiece posture, the first material frame posture and the preset grasping posture.

[0014] The second acquisition module is used to acquire the adjustment joint angle of the adjustable gripper corresponding to the target grasping posture;

[0015] The control module is used to control the robotic arm to grasp the workpiece in the material frame by adjusting the joint angle.

[0016] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0017] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the workpiece gripping method according to any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the workpiece gripping method according to any embodiment of the present invention.

[0019] The technical solution of this invention involves acquiring scene point clouds corresponding to the working scene using a depth camera. Based on a point cloud matching algorithm, and according to the scene point cloud, a pre-constructed workpiece model, and a pre-constructed material frame model, the first workpiece posture and the first material frame posture in the depth camera coordinate system are determined. Based on the preset correspondence between the depth camera coordinate system and the robotic arm coordinate system, and the first workpiece posture, the first material frame posture, and the preset grasping posture, the target grasping posture of the robotic arm is determined. The adjustment joint angle of the adjustable gripper corresponding to the target grasping posture is obtained, and the robotic arm is controlled to grasp the workpiece in the material frame according to the adjusted joint angle. This technical solution, by utilizing the additional degrees of freedom provided by the adjustable gripper, can effectively avoid collisions in the scene, reduce the probability of damage to the robotic arm, and improve the grasping success rate when using a high-degree-of-freedom robotic arm, which is more expensive.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of a workpiece gripping method provided in Embodiment 1 of the present invention;

[0023] Figure 2 This is a schematic diagram of an adjustable gripper according to Embodiment 1 of the present invention;

[0024] Figure 3 This is a schematic diagram of the structure of a workpiece gripping device according to Embodiment 2 of the present invention;

[0025] Figure 4 This is a schematic diagram of the structure of an electronic device that implements the workpiece gripping method of this invention. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first," "target," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] Example 1

[0029] Figure 1 This is a flowchart of a workpiece gripping method according to Embodiment 1 of the present invention. This embodiment is applicable to workpiece gripping situations. The method can be executed by a workpiece gripping device, which can be implemented in hardware and / or software. This workpiece gripping device can be integrated into any electronic device that provides workpiece gripping functionality. Figure 1 As shown, the method includes:

[0030] S101. Obtain the scene point cloud corresponding to the working scene through a depth camera.

[0031] In this embodiment, the depth camera can be any camera capable of acquiring depth (distance) information of the object being photographed. This embodiment does not limit the specific type and parameters of the depth camera.

[0032] The work scenario includes a robotic arm, a material frame, and workpieces in the material frame. The end of the robotic arm is equipped with an adjustable gripper, which includes an adjustable joint.

[0033] Figure 2 This is a schematic diagram of an adjustable gripper according to Embodiment 1 of the present invention. Figure 2 As shown, the adjustable gripper generally consists of the gripper root, the gripper tip, and a one-degree-of-freedom adjustment joint connecting the two.

[0034] Specifically, the end of the robotic arm is connected to the root of the adjustable gripper. The root and tip of the gripper are connected by an adjustable joint, which can rotate at different angles. The tip of the gripper is used to grasp the workpiece.

[0035] Among them, the scene point cloud can be the scene point cloud corresponding to the working scene obtained by the depth camera.

[0036] Specifically, depth images and color / grayscale images corresponding to the work scene are acquired using a depth camera. Distortion correction processing is then performed on these images to ensure alignment at the pixel coordinates corresponding to the depth camera. The work scene includes a robotic arm, a material box, and workpieces within the material box. An adjustable gripper with adjustable joints is mounted at the end of the robotic arm. The scene point cloud corresponding to the work scene is obtained from the depth images acquired from the depth camera.

[0037] S102. Based on the point cloud matching algorithm, determine the first workpiece pose and the first material frame pose in the depth camera coordinate system according to the scene point cloud, the pre-built workpiece model and the pre-built material frame model.

[0038] In this embodiment, the point cloud matching algorithm can be any common point cloud matching algorithm, and this embodiment is not limited to it. For example, the point cloud matching algorithm can be the ICP (Iterative Closest Point) algorithm.

[0039] In actual operation, the workpiece model and the material frame model can be three-dimensional models corresponding to the workpiece and the material frame respectively, which are pre-established based on the workpiece and the material frame in the actual production environment.

[0040] The depth camera coordinate system can be the pixel coordinate system corresponding to the depth camera.

[0041] It should be noted that the first workpiece posture can be the six-degree-of-freedom posture of the workpiece in the depth camera coordinate system, and the first material frame posture can be the six-degree-of-freedom posture of the material frame in the depth camera coordinate system.

[0042] Specifically, common point cloud matching algorithms are used to match the workpiece and the material frame based on the scene point cloud, and the iterative nearest point algorithm based on point to normal vector is used to fine-tune the six-degree-of-freedom pose of the workpiece and the material frame in the final depth camera coordinate system.

[0043] S103. Based on the preset correspondence between the depth camera coordinate system and the robotic arm coordinate system, as well as the first workpiece posture, the first material frame posture, and the preset gripping posture, determine the target gripping posture of the robotic arm.

[0044] The robotic arm coordinate system can be the three-dimensional coordinate system corresponding to the robotic arm base.

[0045] It should be explained that the preset gripping posture can be a six-degree-of-freedom posture for the robotic arm to grip the workpiece, preset according to the actual situation of the workpiece.

[0046] It should be noted that the target grasping posture can be the optimal six-degree-of-freedom posture that ensures the robotic arm does not collide with the material frame or other workpieces when grasping the workpiece.

[0047] Specifically, signed distance fields are established at the root and tip of the adjustable gripper, and the three-dimensional direction and position of the joint's rotation axis relative to the gripper root are adjusted. A predefined dot pattern is printed onto an acrylic plate, which is then mounted on the end effector of a six-axis robotic arm. The end effector of the robotic arm is moved multiple times, and the acrylic plate is photographed using a depth camera. The six-DOF pose of the dot pattern and the corresponding six-DOF pose of the end effector of the robotic arm are identified and recorded using a visual algorithm. After sampling more than five sets of data, the six-DOF pose of the depth camera relative to the robotic arm base is calibrated, obtaining the correspondence between the depth camera coordinate system and the robotic arm coordinate system. Based on the preset correspondence between the depth camera coordinate system and the robotic arm coordinate system, as well as the first workpiece pose, the first material frame pose, and the preset gripping pose, the target gripping pose of the robotic arm is determined.

[0048] S104. Obtain the adjustment joint angle of the adjustable gripper corresponding to the target grasping posture.

[0049] Among them, adjusting the joint angle can be the rotation angle of the adjustable gripper joint corresponding to the target grasping posture.

[0050] Specifically, the joint angles corresponding to the target grasping posture are determined based on the posture data corresponding to the target grasping posture.

[0051] S105. Adjust the joint angle to control the robotic arm to grab the workpiece in the material frame.

[0052] Specifically, the robotic arm is controlled by adjusting the joint angle to grasp the workpiece in the material frame, so as to ensure that the robotic arm does not collide with the material frame or other workpieces when grasping the workpiece.

[0053] The technical solution of this invention involves acquiring scene point clouds corresponding to the working scene using a depth camera. Based on a point cloud matching algorithm, and according to the scene point cloud, a pre-constructed workpiece model, and a pre-constructed material frame model, the first workpiece posture and the first material frame posture in the depth camera coordinate system are determined. Based on the preset correspondence between the depth camera coordinate system and the robotic arm coordinate system, and the first workpiece posture, the first material frame posture, and the preset grasping posture, the target grasping posture of the robotic arm is determined. The adjustment joint angle of the adjustable gripper corresponding to the target grasping posture is obtained, and the robotic arm is controlled to grasp the workpiece in the material frame according to the adjusted joint angle. This technical solution, by utilizing the additional degrees of freedom provided by the adjustable gripper, can effectively avoid collisions in the scene, reduce the probability of damage to the robotic arm, and improve the grasping success rate when using a high-degree-of-freedom robotic arm, which is more expensive.

[0054] Optionally, the robotic arm can be controlled to grip the workpiece in the material frame by adjusting the joint angle, including:

[0055] The posture of the adjustable gripper at the end of the robotic arm is determined based on the posture data corresponding to the target grasping posture and by adjusting the joint angles.

[0056] It should be noted that the attitude data corresponding to the target grasping posture can be the six-degree-of-freedom attitude data corresponding to the target grasping posture.

[0057] In this embodiment, the adjustable gripper posture specifically refers to the six-degree-of-freedom posture of the gripper root corresponding to the target grasping posture.

[0058] Specifically, the attitude of the adjustable gripper at the end of the robotic arm is inferred from the joint angles corresponding to each workpiece in the work scenario and the attitude data corresponding to the target grasping posture, in order to drive the robotic arm to perform workpiece grasping. Optionally, an analytical inverse kinematics method can be used to infer the attitude of the adjustable gripper at the end of the robotic arm by borrowing the joint axis direction and position information of the adjustable gripper.

[0059] The robotic arm is controlled to grasp the workpiece in the material frame based on the adjustable gripper posture at the end of the robotic arm.

[0060] Specifically, first drive the adjustable gripper's adjustment joint to move to the adjustment joint angle, then drive the robotic arm's end effector to the target grasping posture to perform the grasping operation.

[0061] Optionally, based on the preset correspondence between the depth camera coordinate system and the robotic arm coordinate system, as well as the first workpiece posture, the first material frame posture, and the preset grasping posture, the target grasping posture of the robotic arm is determined, including:

[0062] Based on the pre-defined correspondence between the depth camera coordinate system and the robotic arm coordinate system, the scene point cloud, the first workpiece pose, the first material frame pose, and the pre-defined grasping pose are transformed into the robotic arm coordinate system to determine the candidate grasping poses in the robotic arm coordinate system.

[0063] Among them, the candidate grasping posture is the grasping posture that is transformed from the preset grasping posture to the robotic arm coordinate system.

[0064] Specifically, based on the preset correspondence between the depth camera coordinate system and the robotic arm coordinate system, the scene point cloud, the first workpiece pose, the first material frame pose, and the preset grasping pose in the depth camera coordinate system are transformed to the robotic arm coordinate system to obtain the grasping pose in the robotic arm coordinate system.

[0065] The candidate grasping postures are filtered and fine-tuned to obtain the target grasping posture.

[0066] In this embodiment, the fine-tuning process can take the clamping direction of the adjustable gripper's claw tip as the axis and perform rotational fine-tuning of the candidate grasping postures around the axis at different angles. The filtering process can filter out candidate grasping postures that do not meet preset conditions.

[0067] Specifically, the candidate grasping postures are fine-tuned to obtain several grasping postures, and then these several grasping postures are filtered to obtain the target grasping posture.

[0068] Optionally, the candidate grasping poses are filtered and fine-tuned to obtain the target grasping pose, including:

[0069] Obtain at least one finely adjusted first grasping posture.

[0070] The first grasping posture can be the grasping posture generated by fine-tuning the candidate grasping posture.

[0071] Specifically, taking the clamping direction of the adjustable gripper's claw tip as the axis, the candidate gripping posture is rotated and fine-tuned around the axis at different angles to obtain at least one fine-tuned first gripping posture.

[0072] Get the number of collision points corresponding to each first grasping posture.

[0073] In this embodiment, the number of collision points can be understood as the number of point clouds in the scene point cloud when the robotic arm collides with the material frame or other workpieces during the process of grasping the workpiece in the first grasping posture.

[0074] Specifically, obtain the number of collision points corresponding to each first grasping posture.

[0075] The first grasping posture with a number of collision points greater than or equal to a preset threshold is filtered to obtain at least one second grasping posture.

[0076] The preset quantity threshold can be a quantity threshold set by the user according to the actual situation. In this embodiment, the specific value of the preset quantity threshold is not limited.

[0077] It should be noted that the second grasping posture can be a grasping posture with fewer than a preset threshold number of collision points.

[0078] Specifically, the first grasping posture with a collision point count greater than or equal to a preset threshold is filtered out to obtain at least one remaining second grasping posture.

[0079] Obtain the obstacle avoidance score corresponding to each second grasping posture.

[0080] The obstacle avoidance score can be understood as the score of the ability to avoid obstacles when grasping the workpiece in each second grasping posture.

[0081] Specifically, obtain the obstacle avoidance score corresponding to each second grasping posture.

[0082] The target grasping posture is determined based on the obstacle avoidance score corresponding to each second grasping posture.

[0083] Specifically, for each workpiece, the second grasping posture with the highest obstacle avoidance score can be determined as the target grasping posture.

[0084] Optionally, obtain the number of collision points corresponding to each first grasping posture, including:

[0085] Obtain the number of point clouds in the scene point cloud that fall into the region of negative signed distance field.

[0086] As we know, a signed distance field (SDF) can be represented by a scalar field function or a volume map. Simply put, it is a spatial representation that stores the distance from a point in space to the nearest triangle. If it is inside an object, it is a negative value.

[0087] In this embodiment, the negative region of the signed distance field can be the inner region of the claw tip of the signed distance collision model of the adjustable claw tip, wherein the signed distance collision model of the claw tip can be pre-established.

[0088] Specifically, the number of collision points is given by the number of point clouds in the scene point cloud that fall into the negative region of the signed distance field. Specifically, trilinear interpolation can be used to determine whether the point cloud falls into the negative region and obtain the number of point clouds in the scene point cloud that fall into the negative region of the signed distance field.

[0089] The number of collision points corresponding to each first grasping posture is determined based on the number of point clouds falling into the negative region of the signed distance field.

[0090] Specifically, the number of point clouds falling into the negative region of the signed distance field is determined as the number of collision points corresponding to each first grasping posture.

[0091] Optional, adjustable grippers include gripper tips.

[0092] Obtain the obstacle avoidance score for each second grasping posture, including:

[0093] Obtain the Euclidean distance between the point cloud closest to the gripper tip and the gripper tip in all scene point clouds.

[0094] In the implementation process, the obstacle avoidance score consists of a signed distance collision model at the gripper tip and a predetermined nonlinear distance score between the scene point cloud, as well as a regularization term that requires the orientation to be vertically downward.

[0095] Specifically, the distance between all scene point clouds and the gripper tip is calculated. The point cloud closest to the gripper tip is found among all scene point clouds. Then, the Euclidean distance between the closest point cloud and the gripper tip is calculated. For example, the Euclidean distance can be represented by d0.

[0096] Get the preset threshold.

[0097] The preset threshold can be a threshold set by the user according to actual conditions. This embodiment does not limit the specific value of the preset threshold. For example, the preset threshold can be represented by d. m To express.

[0098] Specifically, it retrieves a preset threshold d set by the user. m .

[0099] The cutoff threshold is determined based on the Euclidean distance and a preset threshold.

[0100] For example, the truncation threshold can be d c This is represented. Specifically, it is based on the Euclidean distance d0 and the preset threshold d. m Determine the cutoff threshold d c The calculation method can be expressed as: d c =min(d m ,d0).

[0101] The nonlinear distance score is determined based on the truncation threshold.

[0102] For example, non-linear distance scoring can be used with s d This is represented. Specifically, based on the truncation threshold d... c Determine the nonlinear distance score s d The calculation method can be expressed as:

[0103] The inner product of the first matrix corresponding to each second grasping posture and the second matrix corresponding to the robotic arm coordinate system is determined as the inner product corresponding to each second grasping posture.

[0104] It should be noted that the first matrix can be the matrix corresponding to the z-axis of the rotational portion in the six-DOF pose of each second grasping posture. For example, the first matrix can be expressed as z... p This can be represented as follows. The second matrix can be the matrix corresponding to the z-axis of the rotational portion in the six-degree-of-freedom pose of the robotic arm coordinate system. For example, the second matrix can be represented using z... b To express.

[0105] For example, the inner product can be represented by s z This is represented. Specifically, the first matrix z corresponds to each second grasping posture.p The second matrix z corresponding to the robotic arm coordinate system b inner product s z The calculation method can be expressed as: The inner product of the first matrix corresponding to each second grasping posture and the second matrix corresponding to the robotic arm coordinate system is determined as the inner product corresponding to each second grasping posture.

[0106] The obstacle avoidance score for each second grasping posture is determined based on the first weight, the nonlinear distance score, the second weight, and the inner product corresponding to each second grasping posture.

[0107] The first weight can be a non-linear distance score s pre-set by the user based on the actual situation. d Regarding the corresponding weights, this embodiment does not limit the specific value of the first weight. For example, the first weight can be 0.7.

[0108] The second weight can be the inner product s corresponding to each second grasping posture, which is preset by the user according to the actual situation. z Regarding the weighting, this embodiment does not limit the specific value of the second weight. For example, the second weight can be 0.3.

[0109] Specifically, a weighted average is performed based on the first weight, the nonlinear distance score, the second weight, and the inner product corresponding to each second grasping posture to obtain the obstacle avoidance score corresponding to each second grasping posture.

[0110] Optional, adjustable grippers include gripper roots.

[0111] Obtain the adjustable joint angles of the gripper corresponding to the target grasping posture, including:

[0112] Acquire the fine-tuned gripper root posture data corresponding to at least one target grasping posture.

[0113] The attitude data at the root of the gripper can be the six-degree-of-freedom attitude data of the root of the adjustable gripper.

[0114] Specifically, based on the target gripping posture corresponding to each workpiece, we try to rotate the adjustable joint of the adjustable gripper by different angles to obtain the fine-tuned gripper root posture data corresponding to at least one target gripping posture.

[0115] The obstacle avoidance score corresponding to the attitude data of each fine-tuned gripper root is determined based on the attitude data of each fine-tuned gripper root and the signed distance field corresponding to the gripper root.

[0116] Specifically, based on the six-DOF attitude data of each fine-tuned gripper root and the corresponding signed distance field, the obstacle avoidance score corresponding to the attitude data of each fine-tuned gripper root is calculated. In actual operation, the method and principle for calculating the obstacle avoidance score corresponding to the attitude data of each fine-tuned gripper root are the same as those for calculating the obstacle avoidance score corresponding to each second grasping posture, and will not be repeated here.

[0117] The adjustment joint angle of the adjustable gripper is determined based on the posture data of the gripper root with the highest obstacle avoidance score.

[0118] Specifically, the adjustable joint angle of the adjustable gripper corresponding to the posture data of the gripper root with the highest obstacle avoidance score is determined as the adjustable joint angle of the adjustable gripper corresponding to the target grasping posture.

[0119] The technical solution of this invention uses an adjustable gripper. The additional degree of freedom introduced by the adjustable gripper, combined with a collision detection algorithm, ensures obstacle avoidance and grasping coverage, improves the usability and clearing rate of the grasping algorithm, and avoids the use of expensive high-degree-of-freedom robotic arms, while greatly reducing the probability of the robotic arm being damaged by collisions.

[0120] Example 2

[0121] Figure 3 This is a schematic diagram of a workpiece gripping device according to Embodiment 2 of the present invention. Figure 3 As shown, the device includes: a first acquisition module 201, a first determination module 202, a second determination module 203, a second acquisition module 204, and a control module 205.

[0122] The first acquisition module 201 is used to acquire scene point cloud corresponding to the work scene through a depth camera. The work scene includes a robotic arm, a material frame and a workpiece in the material frame. The end of the robotic arm is equipped with an adjustable gripper, and the adjustable gripper includes an adjustment joint.

[0123] The first determining module 202 is used to determine the first workpiece posture and the first material frame posture in the depth camera coordinate system based on the point cloud matching algorithm, according to the scene point cloud, the pre-built workpiece model and the pre-built material frame model.

[0124] The second determining module 203 is used to determine the target grasping posture of the robotic arm based on the correspondence between the preset depth camera coordinate system and the robotic arm coordinate system, as well as the first workpiece posture, the first material frame posture and the preset grasping posture.

[0125] The second acquisition module 204 is used to acquire the adjustment joint angle of the adjustable gripper corresponding to the target grasping posture;

[0126] The control module 205 is used to control the robotic arm to grasp the workpiece in the material frame by adjusting the joint angle.

[0127] Optionally, the control module 205 includes:

[0128] The first determining unit is used to determine the posture of the adjustable gripper at the end of the robotic arm based on the posture data corresponding to the target grasping posture and the adjustment of the joint angle.

[0129] The control unit is used to control the robotic arm to grasp the workpiece in the material frame based on the adjustable gripper posture at the end of the robotic arm.

[0130] Optionally, the second determining module 203 includes:

[0131] The second determining unit is used to transform the scene point cloud, the first workpiece posture, the first material frame posture, and the preset grasping posture into the robotic arm coordinate system according to the preset correspondence between the depth camera coordinate system and the robotic arm coordinate system, and to determine the candidate grasping posture in the robotic arm coordinate system, wherein the candidate grasping posture is the grasping posture transformed from the preset grasping posture into the robotic arm coordinate system.

[0132] The processing unit is used to filter and fine-tune the candidate grasping postures to obtain the target grasping posture.

[0133] Optionally, the processing unit includes:

[0134] The first acquisition subunit is used to acquire at least one finely adjusted first grasping posture;

[0135] The second acquisition subunit is used to acquire the number of collision points corresponding to each of the first grasping postures;

[0136] The filtering subunit is used to filter out first grasping postures with a number of collision points greater than or equal to a preset threshold, so as to obtain at least one second grasping posture.

[0137] The third acquisition subunit is used to acquire the obstacle avoidance score corresponding to each of the second grasping postures;

[0138] A determination subunit is used to determine the target grasping posture based on the obstacle avoidance score corresponding to each second grasping posture.

[0139] Optionally, the second acquisition subunit is specifically used for:

[0140] Obtain the number of point clouds in the scene point cloud that fall into the region of negative signed distance field;

[0141] The number of collision points corresponding to each of the first grasping postures is determined based on the number of point clouds falling into the negative region of the signed distance field.

[0142] Optionally, the adjustable gripper includes a gripper tip;

[0143] The third acquisition subunit is specifically used for:

[0144] Obtain the Euclidean distance between the point cloud closest to the gripper tip and the gripper tip in all scene point clouds;

[0145] Obtain the preset threshold;

[0146] The truncation threshold is determined based on the Euclidean distance and the preset threshold.

[0147] The nonlinear distance score is determined based on the truncation threshold;

[0148] The inner product of the first matrix corresponding to each second grasping posture and the second matrix corresponding to the robotic arm coordinate system is determined as the inner product corresponding to each second grasping posture;

[0149] The obstacle avoidance score for each second grasping posture is determined based on the first weight, the nonlinear distance score, the second weight, and the inner product corresponding to each second grasping posture.

[0150] Optionally, the adjustable gripper includes a gripper root;

[0151] The second acquisition module 204 includes:

[0152] The acquisition unit is used to acquire at least one finely adjusted gripper root posture data corresponding to the target grasping posture;

[0153] The third determining unit is used to determine the obstacle avoidance score corresponding to the attitude data of each finely adjusted gripper root based on the attitude data of each finely adjusted gripper root and the signed distance field corresponding to the gripper root.

[0154] The fourth determining unit is used to determine the adjustment joint angle of the adjustable gripper corresponding to the target grasping posture based on the posture data of the gripper root with the highest obstacle avoidance score.

[0155] The workpiece gripping device provided in the embodiments of the present invention can execute the workpiece gripping method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0156] Example 3

[0157] Figure 4A schematic diagram of an electronic device 30 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0158] like Figure 4 As shown, the electronic device 30 includes at least one processor 31 and a memory, such as a read-only memory (ROM) 32 or a random access memory (RAM) 33, communicatively connected to the at least one processor 31. The memory stores computer programs executable by the at least one processor. The processor 31 can perform various appropriate actions and processes based on the computer program stored in the ROM 32 or loaded from storage unit 38 into the RAM 33. The RAM 33 can also store various programs and data required for the operation of the electronic device 30. The processor 31, ROM 32, and RAM 33 are interconnected via a bus 34. An input / output (I / O) interface 35 is also connected to the bus 34.

[0159] Multiple components in electronic device 30 are connected to I / O interface 35, including: input unit 36, such as keyboard, mouse, etc.; output unit 37, such as various types of monitors, speakers, etc.; storage unit 38, such as disk, optical disk, etc.; and communication unit 39, such as network card, modem, wireless transceiver, etc. Communication unit 39 allows electronic device 30 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0160] Processor 31 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 31 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 31 performs the various methods and processes described above, such as the workpiece gripping method:

[0161] The scene point cloud corresponding to the working scene is obtained by a depth camera. The working scene includes a robotic arm, a material frame and a workpiece in the material frame. The end of the robotic arm is equipped with an adjustable gripper, and the adjustable gripper includes an adjustment joint.

[0162] Based on the point cloud matching algorithm, the first workpiece pose and the first material frame pose in the depth camera coordinate system are determined according to the scene point cloud, the pre-built workpiece model and the pre-built material frame model.

[0163] Based on the correspondence between the preset depth camera coordinate system and the robotic arm coordinate system, as well as the first workpiece posture, the first material frame posture and the preset grasping posture, the target grasping posture of the robotic arm is determined.

[0164] Obtain the adjustment joint angle of the adjustable gripper corresponding to the target grasping posture;

[0165] The robotic arm is controlled to grip the workpiece in the material frame by adjusting the joint angle.

[0166] In some embodiments, the workpiece gripping method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 38. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 30 via ROM 32 and / or communication unit 39. When the computer program is loaded into RAM 33 and executed by processor 31, one or more steps of the workpiece gripping method described above may be performed. Alternatively, in other embodiments, processor 31 may be configured to perform the workpiece gripping method by any other suitable means (e.g., by means of firmware).

[0167] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0168] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0169] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0170] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0171] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0172] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0173] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0174] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A workpiece gripping method, characterized in that, include: The scene point cloud corresponding to the working scene is acquired by a depth camera. The working scene includes a robotic arm, a material frame and a workpiece in the material frame. The end of the robotic arm is equipped with an adjustable gripper. The adjustable gripper includes a gripper root, a gripper tip and an adjustment joint. The end of the robotic arm is connected to the gripper root of the adjustable gripper. The gripper root and the gripper tip of the adjustable gripper are connected by an adjustment joint. Based on the point cloud matching algorithm, the first workpiece pose and the first material frame pose in the depth camera coordinate system are determined according to the scene point cloud, the pre-built workpiece model and the pre-built material frame model. Based on the correspondence between the preset depth camera coordinate system and the robotic arm coordinate system, as well as the first workpiece posture, the first material frame posture and the preset grasping posture, the target grasping posture of the robotic arm is determined. Acquire the fine-tuned gripper root posture data corresponding to at least one of the target grasping postures; The obstacle avoidance score corresponding to the attitude data of each fine-tuned gripper root is determined based on the attitude data of each fine-tuned gripper root and the signed distance field corresponding to the gripper root. The adjustment joint angle of the adjustable gripper corresponding to the target grasping posture is determined based on the posture data of the gripper root with the highest obstacle avoidance score. The robotic arm is controlled to grip the workpiece in the material frame by adjusting the joint angle.

2. The method according to claim 1, characterized in that, The robotic arm is controlled by adjusting the joint angle to grip the workpiece in the material frame, including: The posture of the adjustable gripper at the end of the robotic arm is determined based on the posture data corresponding to the target grasping posture and the adjustment of the joint angle. The robotic arm is controlled to grasp the workpiece in the material frame based on the adjustable gripper posture at the end of the robotic arm.

3. The method according to claim 1, characterized in that, Based on the preset correspondence between the depth camera coordinate system and the robotic arm coordinate system, and the first workpiece posture, the first material frame posture, and the preset grasping posture, the target grasping posture of the robotic arm is determined, including: Based on the preset correspondence between the depth camera coordinate system and the robotic arm coordinate system, the scene point cloud, the first workpiece posture, the first material frame posture, and the preset grasping posture are transformed into the robotic arm coordinate system to determine the candidate grasping posture in the robotic arm coordinate system. The candidate grasping posture is the grasping posture transformed from the preset grasping posture into the robotic arm coordinate system. The candidate grasping postures are filtered and fine-tuned to obtain the target grasping posture.

4. The method according to claim 3, characterized in that The candidate grasping postures are filtered and fine-tuned to obtain the target grasping posture, including: Obtain at least one finely adjusted first grasping posture; Obtain the number of collision points corresponding to each of the first grasping postures; The first grasping posture with a number of collision points greater than or equal to a preset threshold is filtered to obtain at least one second grasping posture; Obtain the obstacle avoidance score corresponding to each of the second grasping postures; The target grasping posture is determined based on the obstacle avoidance score corresponding to each second grasping posture.

5. The method according to claim 4, characterized in that, Obtain the number of collision points corresponding to each of the first grasping postures, including: Obtain the number of point clouds in the scene point cloud that fall into the region of negative signed distance field; The number of collision points corresponding to each of the first grasping postures is determined based on the number of point clouds falling into the negative region of the signed distance field.

6. The method according to claim 4, characterized in that, Obtain the obstacle avoidance score corresponding to each of the second grasping postures, including: Obtain the Euclidean distance between the point cloud closest to the gripper tip and the gripper tip in all scene point clouds; Obtain the preset threshold; The truncation threshold is determined based on the Euclidean distance and the preset threshold. The nonlinear distance score is determined based on the truncation threshold; The inner product of the first matrix corresponding to each second grasping posture and the second matrix corresponding to the robotic arm coordinate system is determined as the inner product corresponding to each second grasping posture; The obstacle avoidance score for each second grasping posture is determined based on the first weight, the nonlinear distance score, the second weight, and the inner product corresponding to each second grasping posture.

7. A workpiece gripping device, characterized in that, include: The first acquisition module is used to acquire scene point clouds corresponding to the work scene through a depth camera. The work scene includes a robotic arm, a material frame, and a workpiece in the material frame. The end of the robotic arm is equipped with an adjustable gripper. The adjustable gripper includes a gripper base, a gripper tip, and an adjustment joint. The end of the robotic arm is connected to the gripper base of the adjustable gripper, and the gripper base and the gripper tip of the adjustable gripper are connected by an adjustment joint. The first determining module is used to determine the first workpiece pose and the first material frame pose in the depth camera coordinate system based on the point cloud matching algorithm, according to the scene point cloud, the pre-built workpiece model and the pre-built material frame model. The second determining module is used to determine the target grasping posture of the robotic arm based on the correspondence between the preset depth camera coordinate system and the robotic arm coordinate system, as well as the first workpiece posture, the first material frame posture and the preset grasping posture. The second acquisition module is used to acquire at least one fine-tuned gripper root posture data corresponding to the target grasping posture; determine the obstacle avoidance score corresponding to each fine-tuned gripper root posture data based on the posture data of each fine-tuned gripper root and the signed distance field corresponding to the gripper root; and determine the adjustment joint angle of the adjustable gripper corresponding to the target grasping posture based on the posture data of the gripper root with the highest obstacle avoidance score. The control module is used to control the robotic arm to grasp the workpiece in the material frame by adjusting the joint angle.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the workpiece gripping method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the workpiece gripping method according to any one of claims 1-6.

Images