Object grasping method, device, equipment and medium based on object pose

Abstract

The present application provides an object grasping method based on object poses, which comprises: obtaining a first object pose of a first target object, wherein the first target object comprises an object determined from N objects to be grasped; calculating a pose distance between the first object pose and at least one of M second object poses, wherein the M second object poses comprise object poses of M objects that fail to be grasped from the N objects to be grasped, M or N is an integer greater than or equal to 1, and M is less than or equal to N; and stopping grasping the first target object if any of the calculated pose distances is less than a first distance threshold. The present application also provides an object grasping device based on object poses, an electronic device and a storage medium.

Description

Technical Field

[0001] This invention relates to the field of intelligent logistics, and more specifically to a method, apparatus, device, and medium for object grasping based on object pose. Background Technology

[0002] With the development of automation technology, automated machinery is gradually replacing manual labor in some scenarios. For example, a robotic arm can grab an object and move it to the target location, replacing the manual handling method.

[0003] When a robotic arm attempts to grasp an object, it may fail to do so. The control algorithm calculates that the object that failed to grasp has the highest priority, and the planned grasping path was successful. Therefore, the robotic arm may repeatedly attempt to grasp the object after a failure. If the object that failed to grasp has no chance of being successfully grasped, this repeated grasping will waste time and resources, reducing overall grasping efficiency. Summary of the Invention

[0004] In view of the above problems, the present invention provides an object grasping method, apparatus, device and medium based on object pose that can improve the overall grasping efficiency.

[0005] One aspect of this invention provides an object grasping method, comprising: acquiring a first object pose of a first target object, wherein the first target object includes objects determined from N objects to be grasped; calculating a pose distance between the first object pose and at least one of M second object poses, wherein the M second object poses include the object poses of M objects that failed to be grasped from the N objects to be grasped, where M or N is an integer greater than or equal to 1, and M is less than or equal to N; and stopping grasping the first target object if any of the calculated pose distances is less than a first distance threshold.

[0006] According to an embodiment of the present invention, the method further includes: grasping the first target object if any of the calculated pose distances is greater than or equal to a first distance threshold.

[0007] According to an embodiment of the present invention, grasping the first target object includes: grasping the first target object based on a first grasping pose of the first target object, wherein the first grasping pose is the same as or different from the pose of the first object.

[0008] According to an embodiment of the present invention, grasping the first target object based on the first grasping posture of the first target object includes: controlling the suction cup to pick up the first target object in the first grasping posture; or controlling the gripper to clamp the first target object in the first grasping posture.

[0009] According to an embodiment of the present invention, the failure record queue records the M second object poses. Before obtaining the first object pose of the first target object, if the failure record queue does not record the M second object poses, the method further includes: obtaining a third object pose of the second target object and a second grasping pose, wherein the third object pose is the same as or different from the second grasping pose; grasping the second target object based on the second grasping pose; if grasping the second target object fails, the second target object is a grasping failure object, and the third object pose is written as the second object pose into the failure record queue.

[0010] According to an embodiment of the present invention, the method further includes: if the first target object fails to be grasped, the first target object is a failed grasping object, and the pose of the first object is written as the pose of the second object into the failure record queue.

[0011] According to an embodiment of the present invention, the method further includes: setting the maximum number of records in the failure record queue to be less than or equal to N-1; before writing the first object pose as the second object pose into the failure record queue, the method further includes: if M equals N-1, then deleting the earliest recorded second object pose among the M second object poses from the failure record queue.

[0012] According to an embodiment of the present invention, the method further includes determining the first target object, specifically including: determining S target objects from the N objects to be grasped, wherein S is an integer greater than or equal to 1 and S is less than or equal to N; determining the grasping priority of the S target objects; and taking the object with the highest priority among the S target objects as the first target object.

[0013] According to an embodiment of the present invention, the method further includes: writing the object pose of each of the S pending target objects into a priority queue based on the sorting of the grasping priorities, wherein the priority queue is used to determine the grasping order of the S pending target objects.

[0014] According to an embodiment of the present invention, stopping the grasping of the first target object when any of the calculated pose distances is less than a first distance threshold includes: deleting the pose of the first object from the priority queue when any of the calculated pose distances is less than a second distance threshold, wherein the second distance threshold is less than the first distance threshold.

[0015] According to an embodiment of the present invention, stopping the grasping of the first target object when any of the calculated pose distances is less than a first distance threshold further includes: reducing the priority order of the first object pose in the priority queue when any of the calculated pose distances is greater than or equal to a second distance threshold and less than the first distance threshold.

[0016] Another aspect of this invention provides an object grasping device, comprising: a pose acquisition module for acquiring a first object pose of a first target object, wherein the first target object includes objects determined from N objects to be grasped; a pose distance module for calculating a pose distance between the first object pose and at least one of M second object poses, wherein the M second object poses include the object poses of M objects that failed to be grasped from the N objects to be grasped, M or N being an integer greater than or equal to 1, and M less than or equal to N; and an object grasping module for stopping grasping the first target object if any of the calculated pose distances is less than a first distance threshold.

[0017] Another aspect of the present invention provides an electronic device, including: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the method as described above.

[0018] Another aspect of this invention provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the method described above.

[0019] Compared with the method in related technologies where a robotic arm continues to repeatedly attempt to grasp an object after a failure, the embodiments of the present invention can first obtain the first object pose of the first target object, and then use the object poses of M failed grasping objects as references. If the pose distance between the first object pose and at least one of the M second object poses is less than a first distance threshold, it can be determined that the first target object may fail to be grasped, thereby stopping the grasping process. This avoids the waste of time and resources caused by repeated attempts and improves the overall grasping efficiency.

[0020] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0021] The above-described features, other objects, and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0022] Figure 1 A flowchart illustrating an object grasping method according to an embodiment of the present invention is shown schematically.

[0023] Figure 2 This schematically illustrates a flowchart of writing the pose of a third object into a failure record queue according to an embodiment of the present invention;

[0024] Figure 3 This schematically illustrates a flowchart of writing a first object pose into a failure record queue according to an embodiment of the present invention;

[0025] Figure 4 A flowchart illustrating the determination of a first target object according to an embodiment of the present invention is shown schematically;

[0026] Figure 5 A schematic diagram illustrating the priority queue and failure record queue according to an embodiment of the present invention is shown.

[0027] Figure 6 A flowchart illustrating an object grasping method according to another embodiment of the present invention is shown schematically;

[0028] Figure 7 This schematic diagram illustrates the structure of an object grasping device according to an embodiment of the present invention.

[0029] Figure 8 A schematic diagram of the structure of a computing device according to an embodiment of the present invention is shown. Detailed Implementation

[0030] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0031] Figure 1 A flowchart illustrating an object grasping method according to an embodiment of the present invention is shown schematically.

[0032] like Figure 1 As shown, the object grasping method of this embodiment includes operations S110 to S150.

[0033] In operation S110, the first object pose of the first target object is obtained, wherein the first target object includes the object determined from N objects to be grabbed.

[0034] For example, firstly, point cloud data of N objects to be grasped is obtained through photography or LiDAR scanning. Then, object detection algorithms (such as classification or template matching algorithms) are used to process the point cloud data to determine the first target object. The pose of the first object can be obtained from the point cloud data.

[0035] In operation S120, the pose distance between the first object pose and at least one of the M second object poses is calculated, wherein the M second object poses include the object poses of the M failed-to-grab objects among the N objects to be grasped, and M or N is an integer greater than or equal to 1, and M is less than or equal to N.

[0036] For example, an object's pose can be represented by a rotation-translation matrix [x, y, z, α, β, γ], where x, y, and z represent displacements of three degrees of freedom, and α, β, and γ represent spatial rotations of three degrees of freedom. The pose distance can be calculated as the Euclidean distance between the matrix representing the pose of a first object and the matrix passing through the pose of any second object. The first object's pose can include its position and orientation, used to characterize the object's location and orientation.

[0037] Failed grasping refers to objects that were historically attempted to be grasped but were unsuccessful, and remain among the N objects to be grasped. A grasping failure can also mean that the gripper made contact with an object and applied external force, performing a normal grasping operation, but the object did not move or rotate, or only moved or rotated slightly, and did not leave its current position. A grasping failure can also mean that an object was not successfully grasped from its initial grasping position to its target position.

[0038] In operation S130, it is determined whether the calculated distance to any pose is less than a first distance threshold. If yes, operation S140 is executed. If no, operation S150 is executed.

[0039] In operation S140, if any calculated pose distance is less than the first distance threshold, the grasping of the first target object is stopped.

[0040] The first distance threshold D1 can be used to characterize the initial movement range of an object. Different first distance thresholds D1 can be defined for different object types. For example, for small objects, a grasping attempt may be made even with a small movement range. For large objects, a grasping attempt may only be made after a larger movement range has occurred.

[0041] In some embodiments, different first distance thresholds D1 can be preset for different types of objects. For example, when acquiring a first target object, the type of the first target object can be determined, and then the corresponding first distance threshold D1 can be obtained based on the type. Finally, each pose distance is compared with the first distance threshold D1. The object type can be classified according to volume, length, height, etc., or it can be a specific object name, type, or model. It should be noted that the object to be grasped in this invention can be a regular object, that is, an object whose shape can be expressed by a formula, such as a rectangle or a circle. It can also be an irregular object.

[0042] In related technologies, for example, when a robotic arm uses a suction cup to pick up an object, if the surface condition of the area to be picked up is poor, preventing the suction cup from creating a vacuum, the picking may fail. Another example is when a robotic arm uses a gripper to pick up objects, such as heavy metal parts like car crankshafts. Due to the stacking of crankshafts, some may become tightly jammed, allowing the robot to successfully grip the object but not lift it. From a visual calculation perspective, these objects have a higher priority for picking up (e.g., picking up higher objects first, then lower objects), so they are always picked up first. From a motion perspective, the picking plan is successful, but the actual picking fails. When this happens, the robot repeatedly tries to pick up the same object, leading to a decrease in overall picking efficiency and wasting time and resources.

[0043] Compared to the approach in related technologies where a robotic arm repeatedly attempts to grasp an object after a failure, this method first obtains the first object pose of the first target object. Then, using the object poses of M failed grasping objects as references, if the pose distance between the first object pose and at least one of the M second object poses is less than a first distance threshold, it can be determined that the first target object may have failed to be grasped, thus stopping the grasping process. This avoids wasting time and resources due to repeated attempts and improves the overall grasping efficiency.

[0044] In operation S150, if the calculated distance to any pose is greater than or equal to the first distance threshold, the first target object is grasped.

[0045] When the pose distance is greater than or equal to the first distance threshold D1, it can be determined that the first target object has not been attempted to be grasped, or although the grasp failed, the movement range meets the conditions, and the grasp can be attempted again. Therefore, by calculating the pose distance between the first object and each second object, and comparing it with the first distance threshold, on the one hand, objects that have failed to be grasped before and whose poses have not changed can be filtered out. On the other hand, grasping operations can be performed on both objects that have not been grasped before and objects that can be grasped repeatedly, without any omissions. This improves the success rate of single grasping and the overall grasping efficiency.

[0046] According to an embodiment of the present invention, the grasping of the first target object in operation S150 includes: grasping the first target object based on a first grasping pose of the first target object, wherein the first grasping pose is the same as or different from the pose of the first object.

[0047] Object pose and grasping pose are, for example, inherent properties of an object. An object pose can be defined at any location, such as at or outside the object's center, on or off the object. Once defined, the object pose is uniquely determined for that object. In practical applications, the location of the object pose can be set by software (such as Mech-Vision). For example, after obtaining a crankshaft model, Mech-Vision can edit the object pose on the loaded crankshaft.

[0048] The grasping pose can be determined based on factors such as the camera or radar position and the robotic arm's grasping method. There can be one or more grasping poses. Taking a crankshaft as an example, there are typically three gripping locations: the front, middle, and rear. The grasping pose will be defined at these three gripping positions. Since the object pose can be set according to the actual situation, the object pose and the grasping pose can be the same or different. The grasping pose can be acquired simultaneously with the object pose, or it can be acquired before or after the object pose. It should be noted that the front and rear of the crankshaft can be flexibly defined according to the actual coordinate system, with the middle located between the front and rear.

[0049] Therefore, after determining the first target object to be grasped, the robotic arm is controlled to grasp the first target object according to the first grasping posture. For example, when the first target object is a crankshaft, the first grasping posture can be in the middle groove position, so the robotic arm grasps the middle groove of the crankshaft.

[0050] One alternative implementation is that grasping the first target object based on a first grasping pose includes controlling the suction cup to pick up the first target object in the first grasping pose.

[0051] Suction cups can be used to pick up plate-shaped objects, box-shaped objects (such as shipping containers), and lightweight objects (such as small and medium-sized express delivery boxes).

[0052] Another alternative implementation is that grasping the first target object based on the first grasping pose of the first target object includes controlling the gripper to grasp the first target object in the first grasping pose.

[0053] For regular or irregular objects with grooves suitable for gripping, matching grippers can be used for clamping.

[0054] In some embodiments, the end effector of the robotic arm can be equipped with multiple types of grippers, such as suction cups and jaws. The specific type of object can be determined using a classification algorithm, and then a suitable gripper can be selected based on that type. Alternatively, image recognition can be performed on the object's surface to determine which areas are suitable for a particular gripper. For example, target detection can be performed on grooves in a crankshaft; once the grooves are detected, it can be determined that a jaw is suitable for use.

[0055] Figure 2 The flowchart illustrating the process of writing a third object pose into a failure record queue according to an embodiment of the present invention is shown. The failure record queue records M second object poses.

[0056] like Figure 2 As shown, before performing operation S110, for example, before obtaining the first object pose of the first target object, if there are no M second object poses recorded in the failure record queue, operations S210 to S230 can be performed.

[0057] In operation S210, the third object pose of the second target object and the second grasping pose are obtained, wherein the third object pose is the same as or different from the second grasping pose.

[0058] In operation S220, the second target object is grasped based on the second grasping pose.

[0059] In operation S230, if the capture of the second target object fails, the second target object is considered a failed capture object, and the pose of the third object is written into the failure record queue as the pose of the second object.

[0060] Here, "queue" refers to a data structure, a special type of linear list. The failure record queue is used to store the object poses of objects that failed to be grasped. For example, if there is no data recorded in the failure record queue, calculating the pose distance is meaningless, so the third object pose and the second grasping pose of the second target object are directly obtained. If the grasp is successful, the next target object is grasped directly. If the grasp fails, the third object pose is written to the failure record queue. When grasping the next target object, the pose distance is calculated and compared with a first distance threshold.

[0061] Figure 3 The flowchart illustrating the writing of a first object pose into a failure record queue according to an embodiment of the present invention is shown schematically.

[0062] After performing operation S150, as Figure 3 As shown, operations S310 to S340 can also be performed.

[0063] In operation S310, determine whether the first target object has been successfully captured. If yes, end the current operation. If no, proceed to operation S320.

[0064] In operation S320, determine whether the current number M of records in the failure record queue is equal to N-1. If yes, proceed to operation S330. If no, proceed to operation S340.

[0065] According to an embodiment of the present invention, before performing operation S320, the maximum number of records in the failure record queue can be set to less than or equal to N-1.

[0066] Taking crankshafts as an example again, N crankshafts to be grasped are packed in a logistics box. After one is grasped, the next logistics box is used for crankshaft grasping. If there is no limit to the maximum number of records in the failure record queue, then when the robotic arm runs continuously, the object poses of crankshafts from other logistics boxes that failed to be grasped may still be recorded in the queue. In fact, when grasping the crankshaft in the current logistics box, the object poses of crankshafts in other logistics boxes are meaningless. In other words, the comparison result between the calculated pose distance and the first distance threshold is meaningless. Therefore, the maximum number of records can be set to be less than or equal to N.

[0067] In some embodiments, the record data in the failure record queue can be cleared based on the completion of the grabbing of N objects, without setting a maximum number of records.

[0068] In operation S330, if M equals N-1, the earliest recorded second object pose among the M second object poses is deleted from the failure record queue.

[0069] Taking N=3 as an example, there are 3 objects to be grasped. If the maximum number of records is 3, and all 3 objects fail to be grasped, the pose distance between the pose of each object to be grasped and the pose of the objects in the failure record queue may be less than the first distance threshold. In this case, the robotic arm may stop grasping any one of the objects to be grasped, thus making the grasping work impossible.

[0070] If the maximum number of records is 2, one object can be removed from the queue if all three objects to be grasped fail to be grasped. Specifically, if three objects A, B, and C are to be grasped, and each attempt to grasp them is successful but fails, setting the maximum number of records to 2 will have the following effect: After failing to grasp A, the object pose of A is recorded. After failing to grasp B, the object poses of A and B are recorded. After failing to grasp C, A is removed from the queue, and the object poses of B and C are recorded. At this point, A can attempt to grasp. Thus, by continuously applying external force to the objects through the robotic arm, the grasping success rate can be improved by changing the objects' position or orientation.

[0071] When operating S340, if the capture of the first target object fails, the first target object is considered a failed capture object, and the pose of the first object is written as the pose of the second object into the failure record queue.

[0072] According to an embodiment of the present invention, when an object grasping fails, its pose can be written into a failure record queue as a reference for subsequent object grasping. For example, if the first target object is a failed grasping object, then before the next grasping plan, the pose distance is calculated and compared with a first distance threshold to determine whether it can be grasped. This avoids the situation where the first target object has not moved, causing the second grasping to fail again.

[0073] Figure 4 The flowchart illustrating the determination of a first target object according to an embodiment of the present invention is shown schematically.

[0074] like Figure 4 As shown, determining the first target object in this embodiment may include operations S410 to S430.

[0075] In operation S410, S target objects are determined from N objects to be grabbed, where S is an integer greater than or equal to 1 and less than or equal to N.

[0076] In operation S420, the grasping priority of S pending target objects is determined.

[0077] In operation S430, the object with the highest priority among the S pending target objects is selected as the first target object.

[0078] For example, when grabbing N crankshafts from a logistics box, the crankshafts are stacked. Taking pictures of the N crankshafts yields point cloud data of the topmost crankshafts. First, target detection is performed on the topmost crankshafts, and they are designated as S potential target objects, where S is less than or equal to the number of topmost crankshafts. Then, further calculations are performed to determine one of these objects as the primary target object. For example, the object with the highest grabbing success rate can be set as the highest priority primary target object, and priorities can be arranged according to grabbing success rate (this is just an example).

[0079] According to an embodiment of the present invention, based on the sorting of grasping priorities, the object pose of each of the S pending target objects is written into a priority queue, wherein the priority queue is used to determine the grasping order of the S pending target objects.

[0080] One option is to write the object pose of each of the S target objects to a corresponding grasping pose into a priority queue. The object pose is used to determine the object, and the grasping pose is used to grasp it. If a grasping attempt fails, the object pose is written to a failure record queue. The priority order of the object poses in the priority queue determines the grasping order. The robotic arm can then be controlled to grasp objects according to the grasping poses corresponding to their respective object poses, based on the grasping order.

[0081] Figure 5 The diagram illustrates a priority queue and a failure record queue according to an embodiment of the present invention.

[0082] like Figure 5 As shown, the priority queue can include the first object pose and S-1 pending target object poses. The failure record queue includes the second object poses 1 to M. The robotic arm follows... Figure 5 The priority order for grasping is used. Since the first object's pose has the highest priority, it is grasped first. Before planning the grasping process, the first object's pose can be compared with the M second object poses in the failed record queue one by one. The following section combines... Figure 6 Further explanation.

[0083] Figure 6 A flowchart illustrating an object grasping method according to another embodiment of the present invention is shown schematically.

[0084] like Figure 6 As shown, the object grasping method of this embodiment may include operations S110 and S150, and may also include operations S610 to S660.

[0085] In operation S110, the first object pose of the first target object is obtained.

[0086] In operation S610, the pose distance between the pose of the first object and the pose of the i-th object is calculated. Here, the initial value of i is 1, and i is less than or equal to M.

[0087] In operation S620, it is determined whether the pose distance between the first object pose and the i-th object pose is less than a second distance threshold. If yes, operation S630 is executed. If no, operation S640 is executed. Here, the second distance threshold D2 is less than the first distance threshold D1.

[0088] The second distance threshold D2 can be used to characterize the second movement amplitude of the object. Different second distance thresholds D2 can be defined for different object types. In some embodiments, different second distance thresholds D2 can be preset to correspond to different object types. For example, when acquiring a second target object, the type of the second target object can be determined, and then the corresponding second distance threshold D2 can be obtained based on the type. Finally, each pose distance is compared with the second distance threshold D2.

[0089] In operation S630, if the calculated distance to any pose is less than the second distance threshold, the first object pose is removed from the priority queue.

[0090] Reference Figure 5 The first object pose is removed from the priority queue, meaning the priority of the pending target object pose 1 becomes the highest, and the priority queue still records S-1 object poses. Since S pending target objects may be identified before each grasping plan, and grasping is performed sequentially. The second distance threshold D2 is used to determine whether attempting to grasp the first target object is feasible. If the movement range of the first target object is less than the second movement range, the grasping success rate is considered very low, and it should not be assigned grasping priority; the grasping attempt should be abandoned.

[0091] In operation S640, it is determined whether the pose distance between the first object pose and the i-th object pose is less than the first distance threshold D1. If yes, operation S650 is executed. If no, operation S660 is executed.

[0092] In operation S650, if the calculated distance of any pose is greater than or equal to the second distance threshold and less than the first distance threshold, the priority order of the first object pose in the priority queue is reduced.

[0093] If the pose distance is greater than or equal to the second distance threshold and less than the first distance threshold, it is assumed that the first object's pose may have previously failed to be grasped, but it has undergone some movement. In this case, the first target object may be affected by other objects, such as objects pressing on top of it. After grasping other objects, attempting to grasp the first target object again will have a higher success rate. (Refer to...) Figure 5 The first object pose can be inserted between any two of the target object poses 1 to S-1. For example, the first object pose can be inserted between the target object pose 1 and the target object pose 2, which means that the priority order is downgraded.

[0094] The priority level to be lowered for the first object's pose can be determined based on actual operational conditions. Taking crankshafts as an example, observe approximately how many crankshafts need to be grasped before the crankshaft in the logistics box becomes loose. Setting this priority level is appropriate. This is because crankshafts that are prioritized are generally easier to grasp; their grasping priority is lowered because they are too tightly gripped to be effectively grasped. However, once the crankshaft becomes loose (i.e., the object's pose changes), it can still be grasped first.

[0095] In some embodiments, refer to Figure 5 If the poses of the target objects from 1 to S-1 have all been removed from the priority queue due to successful or failed capture, even if the priority of the first object pose is reduced to the target object pose S-1, and the pose distance to the i-th second object pose is still greater than or equal to the second distance threshold and less than the first distance threshold when only the first object pose is in the priority queue, then it can be set to delete the first object pose if its priority cannot be reduced.

[0096] In operation S660, determine if i is less than M. If yes, set i = i + 1 and execute operation S610. If no, execute operation S150.

[0097] Reference Figure 5 The first object pose is compared with the poses of each object in the second set of objects (1 to M) to obtain the pose distance, thus determining whether the first object pose can be grasped. After determining that it can be grasped, the grasping pose, path, angle, and fixture selection are planned. For example, obtaining the first object pose first, and then determining the grasping pose, grasping path, angle, and fixture after confirming that it can be grasped, can save computational resources.

[0098] Operating S150, grab the first target object.

[0099] According to embodiments of the present invention, stopping the grasping of the first target object can be implemented in two ways: deleting it from the priority queue or reducing its priority in the priority queue. This allows for flexible handling of scenarios where crankshafts are firmly stuck and will not loosen under any circumstances; in such cases, attempts to grasp the object are made but not successful, and no further attempts are made. Alternatively, in other scenarios, as the crankshaft is grasped, any ungrabbed crankshafts may become loose due to the grasping process. Therefore, for crankshafts that have been attempted but not successfully grasped, their priority is simply reduced, and the grasping attempt can be restarted after a certain period. This adapts to the complex and changing working conditions during the grasping process, improving overall grasping efficiency.

[0100] Based on the above object grasping method, the present invention also provides an object grasping device. The following will be combined with... Figure 7The device is described in detail.

[0101] Figure 7 A schematic block diagram of an object grasping device 700 according to an embodiment of the present invention is shown.

[0102] like Figure 7 As shown, the object grasping device 700 of this embodiment includes a pose acquisition module 710, a pose distance module 720, and an object grasping module 730.

[0103] The pose acquisition module 710 may, for example, perform operation S110 to acquire the first object pose of the first target object, wherein the first target object includes an object determined from N objects to be captured.

[0104] The pose distance module 720 can, for example, perform operation S120 to calculate the pose distance between the first object pose and at least one of the M second object poses, wherein the M second object poses include the object poses of the M failed-to-grab objects among the N objects to be grasped, and M or N is an integer greater than or equal to 1, and M is less than or equal to N.

[0105] The object grasping module 730 may, for example, perform operation S140 to stop grasping the first target object if any calculated pose distance is less than a first distance threshold.

[0106] The object grasping module 730 can also perform operation S150, which is used to grasp a first target object when the calculated distance to any pose is greater than or equal to a first distance threshold. Grasping the first target object includes: grasping the first target object based on a first grasping pose, wherein the first grasping pose may be the same as or different from the first object pose.

[0107] The object grasping module 730 can also be used to control the suction cup to pick up the first target object in the first grasping posture, or to control the gripper to grasp the first target object in the first grasping posture.

[0108] The object grasping device 700 may further include a failure record queue module, which records M second object poses. Before acquiring the first object pose of the first target object, if the failure record queue does not contain M second object poses, the failure record queue module is used to: acquire the third object pose of the second target object and the second grasping pose, wherein the third object pose may be the same as or different from the second grasping pose. The second target object is then grasped based on the second grasping pose. If grasping the second target object fails, the second target object is considered a failed grasp object, and the third object pose is written into the failure record queue as the second object pose.

[0109] The object grasping device 700 may further include a first writing module, configured to write the first object pose as a second object pose to a failure record queue in the event that grasping the first target object fails (the first target object is considered a failed grasping object). It is also configured to set the maximum number of records in the failure record queue to be less than or equal to N-1. Before writing the first object pose as a second object pose to the failure record queue, the module further includes: if M equals N-1, deleting the earliest recorded second object pose from the M second object poses from the failure record queue.

[0110] The object grasping device 700 may further include a target determination module, used to determine S potential target objects from N objects to be grasped, where S is an integer greater than or equal to 1 and less than or equal to N. The grasping priority of the S potential target objects is determined. The object with the highest priority among the S potential target objects is designated as the first target object.

[0111] The object grasping device 700 may further include a second writing module, used to write the object pose of each of the S pending target objects into a priority queue based on grasping priority, wherein the priority queue is used to determine the grasping order of the S pending target objects. It is also used to delete the first object pose from the priority queue if the calculated distance of any pose is less than a second distance threshold, wherein the second distance threshold is less than a first distance threshold. Furthermore, it is used to reduce the priority order of the first object pose in the priority queue if the calculated distance of any pose is greater than or equal to the second distance threshold and less than the first distance threshold.

[0112] Compared with the related art method where a robotic arm continues to repeatedly attempt to grasp an object after failing to do so, the object grasping device 700 of the embodiment of the present invention can first obtain the first object pose of the first target object, and then use the object poses of M failed grasping objects as references. If the pose distance between the first object pose and at least one of the M second object poses is less than a first distance threshold, it can be determined that the first target object may fail to be grasped, thereby stopping the grasping process. This avoids the waste of time and resources caused by repeated attempts and improves the overall grasping efficiency.

[0113] Figure 8 The diagram illustrates the structure of a computing device according to an embodiment of the present invention. The specific embodiments of the present invention do not limit the specific implementation of the computing device.

[0114] like Figure 8 As shown, the computing device may include: a processor 802, a communications interface 804, a memory 806, and a communications bus 808.

[0115] in:

[0116] The processor 802, communication interface 804, and memory 806 communicate with each other through the communication bus 808.

[0117] The communication interface 804 is used to communicate with other network elements such as clients or other servers.

[0118] The processor 802 is used to execute program 810, which can specifically execute the relevant steps in the above-described object grasping method embodiment.

[0119] Specifically, program 810 may include program code that includes computer operation instructions.

[0120] Processor 802 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The computing device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0121] Memory 806 is used to store program 810. Memory 806 may include high-speed RAM memory, and may also include nonvolatile memory, such as at least one disk storage device.

[0122] Specifically, program 810 can be used to cause processor 802 to execute the object grasping method in any of the above method embodiments. The specific implementation of each step in program 810 can be found in the corresponding descriptions of the steps and units in the above object grasping embodiments, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0123] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0124] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device.

[0125] Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is readily apparent from the above description. Furthermore, the embodiments of the present invention are not directed to any particular programming language. It should be understood that the content of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0126] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0127] Similarly, it should be understood that, in order to simplify the invention and aid in understanding one or more of the various inventive aspects, features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the above description of exemplary embodiments of the invention. However, this approach should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of the single embodiment disclosed above.

[0128] Therefore, the claims following the specific embodiments are hereby expressly incorporated into the specific embodiments, wherein each claim itself is a separate embodiment of the invention.

[0129] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0130] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0131] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components according to the embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0132] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

Claims

1. A method for object grasping based on object pose, comprising: Obtain the first object pose of the first target object, wherein the first target object includes an object determined from N objects to be grabbed; Calculate the pose distance between the first object pose and at least one of the M second object poses, wherein the M second object poses include the object poses of the M failed-to-grab objects among the N objects to be grasped, and M or N is an integer greater than or equal to 1, and M is less than or equal to N. If any of the calculated pose distances is less than a first distance threshold, the grasping of the first target object is stopped.

2. The method according to claim 1, wherein, The method further includes: If any of the calculated pose distances is greater than or equal to a first distance threshold, the first target object is grasped.

3. The method according to claim 2, wherein, The grasping of the first target object includes: The first target object is grasped based on a first grasping pose, wherein the first grasping pose is the same as or different from the pose of the first object.

4. The method according to claim 3, wherein, The grasping of the first target object based on the first grasping pose of the first target object includes: Control the suction cup to pick up the first target object in the first grasping pose; or Control the gripper to grasp the first target object in the first grasping position.

5. The method according to claim 2, wherein, The failure record queue records the M second object poses. Before acquiring the first object pose of the first target object, if the failure record queue does not record the M second object poses, the method further includes: Obtain the third object pose of the second target object and the second grasping pose, wherein the third object pose is the same as or different from the second grasping pose; The second target object is grasped based on the second grasping pose; If the capture of the second target object fails, the second target object is considered a failed capture object, and the pose of the third object is written into the failure record queue as the pose of the second object.

6. The method according to claim 5, wherein, The method further includes: If the capture of the first target object fails, the first target object is a failed capture object, and the pose of the first object is written into the failure record queue as the pose of the second object.

7. The method according to claim 6, wherein, The method further includes: If the maximum number of records in the failure record queue is set to be less than the number of objects to be captured. Before writing the first object pose as the second object pose into the failure record queue, the method further includes: If all objects to be grabbed fail to be grabbed, the earliest recorded second object pose is deleted from the failure record queue.

8. The method according to claim 1, wherein, The method further includes determining the first target object, specifically including: S target objects are determined from the N objects to be grabbed, where S is an integer greater than or equal to 1 and less than or equal to N. Determine the grasping priority of S pending target objects; The object with the highest priority among the S undetermined target objects is selected as the first target object.

9. The method according to claim 8, wherein, The method further includes: Based on the sorting of the grasping priorities, the object pose of each of the S pending target objects is written into a priority queue, wherein the priority queue is used to determine the grasping order of the S pending target objects.

10. The method according to claim 9, wherein, The step of stopping the grasping of the first target object when any of the calculated pose distances is less than the first distance threshold includes: If any of the calculated pose distances is less than a second distance threshold, the first object pose is removed from the priority queue, wherein the second distance threshold is less than the first distance threshold.

11. The method according to claim 10, wherein, The step of stopping the grasping of the first target object when any of the calculated pose distances is less than the first distance threshold further includes: If any of the calculated pose distances is greater than or equal to the second distance threshold and less than the first distance threshold, the priority order of the first object pose in the priority queue is reduced.

12. An object grasping device based on object pose, comprising: The pose acquisition module is used to acquire the first object pose of the first target object, wherein the first target object includes an object determined from N objects to be captured. The pose distance module is used to calculate the pose distance between the first object pose and at least one of the M second object poses, wherein the M second object poses include the object poses of the M failed-to-grab objects among the N objects to be grasped, and M or N is an integer greater than or equal to 1, and M is less than or equal to N. The object grasping module is used to stop grasping the first target object if any of the calculated pose distances is less than a first distance threshold.

13. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors perform the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 11.

Images