Object grabbing method and device, electronic equipment and storage medium

Abstract

The invention provides an object grabbing method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining an image containing a to-be-grabbed target object; obtaining a first drop point area around the target object for a clamping jaw to drop in the image; segmenting the first drop point area according to a first threshold value to obtain a second drop point area comprising a plurality of slice areas; obtaining depth data of the second drop point area; determining a minimum grabbable area in the second drop point area according to the depth data; determining a grabbing point in the minimum grabbable area; and the clamping jaw is controlled to move to the grabbing point to grab the target object. By means of the object grabbing method and device, the object grabbing efficiency and the object grabbing accuracy can be improved, interference caused by interferents around the object is avoided, the object positioning accuracy can be improved, and objects of different shapes can be grabbed easily and stably.

Description

Technical Field

[0001] The present application relates to the field of robotic arm grasping technology, and more specifically, to a method, device, electronic device and storage medium for grasping an object. Background Art

[0002] There are three steps in the grasping of the visual robot arm: obtaining the target to be grasped through image recognition, analyzing the graspable position or area of ​​the target to be grasped, and identifying the best grasping point for grasping.

[0003] The existing technology finds the best grasping point by combining algorithms with the specifications and postures of the target to be grasped, hardware limitations (such as the structure and clamping force of the gripper; the range of motion and load of the robotic arm, etc.), environmental requirements, etc. Common algorithm logics include those based on the principle of balance, the principle of morphology, and the principle of orientation. However, there are many disadvantages in the implementation of the above methods. For example, the positioning is easily disturbed by interference objects around the grasped object, resulting in damage to the gripper or the robotic arm; it is only suitable for the positioning of stationary target objects, not for the positioning of moving target objects; for grasping objects with varying specifications and postures, it is easy to miss or be unstable. Summary of the invention

[0004] The purpose of the present application is to provide a method, device, electronic device and storage medium for grasping an object, which can improve the efficiency and accuracy of grasping an object, will not be disturbed by interference objects around the object, can improve the accuracy of object positioning, and can also achieve easy and stable grasping of objects of various shapes.

[0005] In a first aspect, an embodiment of the present application provides a method for grasping an object, the method comprising:

[0006] Acquire an image containing a target object to be grasped;

[0007] Obtaining a first landing point area around the target object in the image where the gripper can fall;

[0008] Segmenting the first landing area according to a first threshold to obtain a second landing area including a plurality of slice areas;

[0009] Obtaining depth data of the second landing point area;

[0010] Determine a minimum graspable area in the second landing area according to the depth data;

[0011] determining a grasping point in the minimum graspable area;

[0012] The gripper is controlled to move to the gripping point to grip the target object.

[0013] In the above implementation process, the target object can be accurately and stably grasped by segmenting the area around the target object and performing graspability analysis based on the depth data. This can improve the efficiency and accuracy of object grasping, avoid being disturbed by interference objects around the object, improve the accuracy of object positioning, and achieve easy and stable grasping of objects of various shapes.

[0014] Furthermore, the step of determining the minimum graspable area in the second landing area according to the depth data comprises:

[0015] Acquire depth data of each slice area in the second landing area;

[0016] Performing depth evaluation on the depth data of each slice area respectively to obtain multiple depth evaluation results;

[0017] comparing the plurality of depth assessment results with a second threshold respectively;

[0018] If the depth evaluation result is greater than or equal to the second threshold, determining the slice area corresponding to the depth evaluation result as a candidate graspable area;

[0019] If there is no depth evaluation result greater than or equal to the second threshold, it is determined that there is no graspable area of ​​the target object, and the target object is grasped a second time;

[0020] The minimum graspable area among the candidate graspable areas is determined.

[0021] In the above implementation process, the depth data of each slice area after segmentation is evaluated, and the candidate graspable area is determined when the depth evaluation result meets the requirements. The graspable area can be located quickly and accurately to provide support for the grasping of the target object.

[0022] Furthermore, the step of performing depth evaluation on the depth data of each slice area to obtain multiple depth evaluation results includes:

[0023] Acquire depth data of all pixels in each slice area;

[0024] Arrange the depth data of all pixels in each slice area in ascending order to obtain a plurality of first sequences;

[0025] Selecting depth data in the plurality of first sequences respectively according to a third threshold value to obtain a plurality of second sequences;

[0026] performing arithmetic averaging on the second sequences respectively to obtain multiple average values;

[0027] The plurality of average values ​​are determined as the plurality of depth evaluation results.

[0028] In the above implementation process, the depth data of the pixels in each slice area are sorted to obtain the corresponding sequence, and then the average value of each sequence is calculated as the depth evaluation result. The depth evaluation can be performed on each point in the slice area to ensure the safety and stability of the grasping action.

[0029] Furthermore, the step of determining the minimum graspable area in the candidate graspable areas includes:

[0030] Merging a preset number of adjacent slice areas in the candidate graspable area to obtain a plurality of candidate minimum graspable areas;

[0031] The average depth, length and center of gravity distance of multiple candidate minimum graspable areas are weighted averaged to obtain a weighted score;

[0032] The candidate minimum graspable area with a high weight score is determined as the minimum graspable area.

[0033] In the above implementation process, some slice areas in the candidate graspable area are merged, and the merged candidate minimum graspable area is evaluated, so as to accurately find the most suitable position for grasping and reduce the possible errors in the grasping process.

[0034] Furthermore, the step of performing secondary grasping on the target object comprises:

[0035] Acquire a preset area around the target object;

[0036] Dividing the preset area according to a preset method to obtain a first area to be cleared;

[0037] Determining whether to clear the first area to be cleared according to the depth data of the first area to be cleared;

[0038] If so, the gripper is controlled to clear the first area to be cleared and then the target object is grasped for a second time.

[0039] In the above implementation process, the first area to be cleared is judged to be clearable, and then the gripper is controlled to clear it and then perform a second grip. The target object can be gripped in a complex and changeable environment without being disturbed by the surrounding environment, thereby improving the gripping efficiency.

[0040] Furthermore, the step of determining whether to clear the first area to be cleared according to the depth data of the first area to be cleared includes:

[0041] Obtaining a mean depth value of the target object;

[0042] Obtaining a depth average of a first removable sub-region and a depth average of a second removable sub-region in the first removable sub-region according to the depth data of the first removable sub-region;

[0043] Obtaining a difference between a depth mean of the target object and a depth mean of a first clearable sub-region in the first to-be-cleared region, and determining the difference as a first difference;

[0044] Obtaining a difference between a depth mean of the target object and a depth mean of a second clearable sub-region in the first to-be-cleared region, and determining the difference as a second difference;

[0045] Whether to clear the first area to be cleared is determined according to the first difference and the second difference.

[0046] In the above implementation process, the first area to be cleared is determined to be cleared according to the difference in the depth mean, thereby reducing the error of the gripper during the gripping process, so that target objects at different positions can be gripped, and the situation of unstable gripping can be reduced.

[0047] Further, the step of judging whether to clear the first area to be cleared according to the first difference and the second difference includes:

[0048] Obtaining a difference between an absolute value of the first difference and an absolute value of the second difference, and determining the difference to be a third difference;

[0049] Determining whether the third difference is greater than a fourth threshold;

[0050] If so, clear the first area to be cleared.

[0051] In the above implementation process, the third difference is compared with the fourth threshold to determine whether the first area to be cleared can be cleared, thereby increasing the probability of the first area to be cleared being cleared, reducing the situation where the target object cannot be grasped, and improving the grasping speed.

[0052] In a second aspect, an embodiment of the present application further provides a device for grasping an object, the device comprising:

[0053] An acquisition module, used for acquiring an image containing a target object to be grasped;

[0054] A data acquisition module, for obtaining a first landing point area around the target object in the image where the gripper can land; for obtaining depth data of the second landing point area; and for determining a minimum graspable area in the second landing point area according to the depth data;

[0055] A segmentation module, configured to segment the first landing area according to a first threshold value to obtain a second landing area including a plurality of slicing areas;

[0056] A grasping point determination module, used to determine a grasping point in the minimum graspable area;

[0057] The grasping module is used to control the gripper to grasp the target object at the grasping point.

[0058] In the above implementation process, the target object can be accurately and stably grasped by segmenting the area around the target object and performing graspability analysis based on the depth data. This can improve the efficiency and accuracy of object grasping, avoid being disturbed by interference objects around the object, improve the accuracy of object positioning, and achieve easy and stable grasping of objects of various shapes.

[0059] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any one of the first aspects when executing the computer program.

[0060] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which instructions are stored. When the instructions are executed on a computer, the computer executes the method as described in any one of the first aspects.

[0061] Other features and advantages of the present disclosure will be set forth in the following description, or some features and advantages may be inferred or unambiguously determined from the description, or may be learned by implementing the above-mentioned technology of the present disclosure.

[0062] And it can be implemented according to the contents of the specification. The following is a detailed description of the preferred embodiments of the present application with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the range values. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0064] Figure 1 A schematic diagram of a process for grasping an object is provided for an embodiment of the present application;

[0065] Figure 2 A schematic diagram of the structure of a grasping device for an object provided in an embodiment of the present application;

[0066] Figure 3A schematic diagram of the structural composition of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0067] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0068] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0069] The specific implementation methods of the present application are further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present application, but are not used to limit the scope of the present application.

[0070] Embodiment 1

[0071] Figure 1 is a flow chart of a method for grasping an object provided in an embodiment of the present application, such as Figure 1 As shown, the method includes:

[0072] S1, acquiring an image containing a target object to be grasped;

[0073] S2, obtaining a first landing point area around the target object in the image where the gripper can fall;

[0074] S3, segmenting the first landing area according to the first threshold value to obtain a second landing area including a plurality of slice areas;

[0075] S4, obtaining depth data of the second landing point area;

[0076] S5, determining the minimum graspable area in the second landing area according to the depth data;

[0077] S6, determining a grasping point in the minimum graspable area;

[0078] S7, controls the gripper to the grasping point to grasp the target object.

[0079] In the above implementation process, the target object can be accurately and stably grasped by segmenting the area around the target object and performing graspability analysis based on the depth data. This can improve the efficiency and accuracy of object grasping, avoid being disturbed by interference objects around the object, improve the accuracy of object positioning, and achieve easy and stable grasping of objects of various shapes.

[0080] In S1, the embodiment of the present application first identifies the target object before grasping it. The target recognition adopts target detection of a deep learning framework + a rotation detection frame with angle information. Therefore, in the image containing the target object to be grasped, the target object has a detection frame, and the detection frame is a rectangular frame.

[0081] In S2, since the target object may have an irregular shape and the gripper itself has physical limitations, for convenience, the embodiment of the present application will treat the target object as a rectangle. If the target object is uneven in thickness, it will be divided into several rectangular frames, and then the length and width will be determined by segment.

[0082] Therefore, the first landing area is the periphery of the narrower sides of the rectangular frame of the target object, that is, the two sides with smaller width of the rectangular frame are the narrower sides of the target object, which are also rectangular, with a length equal to the longer side of the target object and a width twice the width of the clamping fingers of the gripper. Being twice the width of the clamping fingers is to increase the tolerance of the landing point.

[0083] In S3, the first landing area is a rectangle, and there are two of them, one on each side of the target object. The two first landing areas are sliced, and the width of the clamp is n, and the length of the longer side of the rectangular frame is M. Then each first landing area is a rectangle of M*2n, and the M side length is segmented from the center to both ends with a length of n (first threshold). If the segmentation is not an integer multiple of n at the end, the segmentation is stopped. At this time, the M side can be divided into (M / n rounded) slice areas. After segmentation, there will be an equal number of symmetrical sliced ​​areas (slice areas) on the left and right sides of the rectangular frame, and each slice area is a 2n*n rectangle.

[0084] Furthermore, S5 includes:

[0085] Acquire depth data of each slice area in the second landing area;

[0086] Performing depth evaluation on the depth data of each slice area respectively to obtain multiple depth evaluation results;

[0087] comparing the plurality of depth evaluation results with a second threshold respectively;

[0088] If the depth evaluation result is greater than or equal to the second threshold, determining the slice area corresponding to the depth evaluation result as a candidate graspable area;

[0089] If there is no depth assessment result greater than or equal to the second threshold, it is determined that there is no graspable area of ​​the target object, and the target object is grasped a second time;

[0090] Determine the minimum graspable area among the candidate graspable areas.

[0091] In the above implementation process, the depth data of each slice area after segmentation is evaluated, and the candidate graspable area is determined when the depth evaluation result meets the requirements. The graspable area can be located quickly and accurately to provide support for the grasping of the target object.

[0092] Perform an in-depth assessment of each slicing area to ensure that the gripper falls within a threshold range, i.e., the minimum graspable area, when grasping. The minimum graspable area meets the requirements of the gripper's structure, gripping force, etc.

[0093] The depth data is based on the height value of each pixel, that is, the actual distance from the camera lens to a point in the image. The value of this depth data is set to z.

[0094] Suppose the depth data of all pixels in a certain slice area is {Zi} (where i = 1, 2, 3, ..., 2n*n), and assume that the distance between the target object and the camera is h1. According to the physical structure of the gripper, as long as the gripper can reach a depth of h2 on both sides of the target object, it can clamp the target object.

[0095] If each value in {zi} is greater than or equal to h1+h2 (the second threshold), the gripper can definitely reach the bottom. However, in reality, there may be a small amount of abnormal depth data, and there may be a small amount of data that does not meet the requirements at the edge of the slicing area, but it does not actually affect the gripper's reach.

[0096] Further, the step of performing depth evaluation on the depth data of each slice area to obtain multiple depth evaluation results includes:

[0097] Get the depth data of all pixels in each slice area;

[0098] Arrange the depth data of all pixels in each slice area in ascending order to obtain a plurality of first sequences;

[0099] Selecting depth data in the plurality of first sequences respectively according to a third threshold value to obtain a plurality of second sequences;

[0100] Performing arithmetic averaging on the second sequence respectively to obtain multiple average values;

[0101] Multiple average values ​​are determined as the multiple depth evaluation results.

[0102] In the above implementation process, the depth data of the pixels in each slice area are sorted to obtain the corresponding sequence, and then the average value of each sequence is calculated as the depth evaluation result. The depth evaluation can be performed on each point in the slice area to ensure the safety and stability of the grasping action.

[0103] Based on redundancy considerations, the embodiment of the present application sorts {Zi} in ascending order and selects the first two-thirds (third threshold) of the data for arithmetic average. If the average value is greater than or equal to h1+h2, the depth assessment requirement is met, that is, the slice area is the minimum graspable area.

[0104] The definition of h2 is related to the weight of the target object and the structure of the gripper. For example, if the target object is heavy, or the gripper fingers are narrow, or the friction of the gripper fingers is small, or the gripping force is small, a larger h2 will be obtained.

[0105] Furthermore, the step of determining the minimum graspable area among the candidate graspable areas includes:

[0106] Merging a preset number of adjacent slice areas in the candidate graspable area to obtain multiple candidate minimum graspable areas;

[0107] The average depth, length and center of gravity distance of multiple candidate minimum graspable areas are weighted averaged to obtain a weighted score;

[0108] The candidate minimum graspable area with a high weight score is determined as the minimum graspable area.

[0109] In the above implementation process, some slice areas in the candidate graspable area are merged, and the merged candidate minimum graspable area is evaluated, so as to accurately find the most suitable position for grasping and reduce the possible errors in the grasping process.

[0110] After depth evaluation of each slice area, the minimum graspable area is found among the candidate graspable areas.

[0111] For example, the side of the target object with a length of M can be divided into 22 slices on the left and right. Among these slices, assuming that m2, m3, m4, m5, m8, m9, m14, m15, and m16 all meet the minimum graspable area determination, they can be merged into three (preset number) candidate minimum graspable areas according to the method of merging adjacent areas, namely [m2, m3, m4, m5], [m8, m9], and [m14, m15, m16]. Suppose these three candidate minimum graspable areas correspond to the parts above the target object, namely M1, corresponding to [m2, m3, m4, m5]; M2, corresponding to [m8, m9]; and M3, corresponding to [m14, m15, m16].

[0112] The depth data of the three candidate minimum graspable areas are Z1, Z2, and Z3 respectively. Let Zmax be the maximum value among them, then Zi / Zmax is the average depth (where i=1, 2, 3).

[0113] Let the length of M1 be L1, the length of M2 be L2, and the length of M3 be L3. Let Lmax be the maximum value, then Li / Lmax is the length (where i=1, 2, 3).

[0114] The target length is M (assuming the two end points are AB). If the thickness is uneven, it will be simply divided into two segments according to the thickness. The thick end is one segment (assuming it is segment AB), and the thin end is another segment (assuming it is segment BC). After the division, each segment is close to or equivalent to uniform thickness. Assuming the center of gravity of the entire target is point E, the AE length formula is (W2*V2+W1*V1) / 2W1, where W1 is the width of AB and V1 is the length of AB; W2 (W1>W2) is the width of BC and V2 is the length of BC. Assuming the center distance of the three candidate minimum graspable areas to point A is AE1, AE2, and AE3, and the distance from the center of gravity of the target object is 1-(|AEi-AE| / AE) (where i=1, 2, 3).

[0115] The weighted average calculation is used for the average depth, length and center of gravity distance. The formula is as follows:

[0116] 0.3*(Zi / Zmax)+0.4*(Li / Lmax)+0.3*(1-(|AEi-AE| / AE))

[0117] According to the formula, the one with the highest weight score is selected as the minimum graspable area (the minimum graspable area is on the left and right sides of the target object, and is two areas of the same size and symmetrical).

[0118] Furthermore, the step of performing secondary grasping on the target object includes:

[0119] Get the preset area around the target object;

[0120] Dividing the preset area according to a preset method to obtain a first area to be cleared;

[0121] Determining whether to clear the first area to be cleared according to the depth data of the first area to be cleared;

[0122] If so, the gripper is controlled to clean the first area to be cleaned and then the target object is grasped for the second time.

[0123] In the above implementation process, the first area to be cleared is judged to be clearable, and then the gripper is controlled to clear it and then perform a second grip. The target object can be gripped in a complex and changeable environment without being disturbed by the surrounding environment, thereby improving the gripping efficiency.

[0124] If there is no minimum graspable area for the gripper to grasp, the area around the target object is cleared and then a secondary grasp is performed to obtain a preset area around the target object. The preset area is set according to the depth data around the target object. For example, the depth distribution of a certain area has a large difference, and the closer to the target object, the smaller the depth, and the farther away from the target object, the deeper the depth, and the area of ​​the area with a large depth far away from the target object meets a threshold (50% of the size of the target object), it is defined as the preset area.

[0125] Get the depth distribution of the preset area, and divide it into 2 to 10 equal parts (preset method) according to the longest side of the target object. The specific equal parts are set in a linear relationship with the target length. The embodiment of this application takes the division into 3 equal parts as an example. The first area to be cleared is obtained by segmentation. For the narrower sides of the target object, three square areas are drawn on each side (the side length is one-third of the longest side of the target object), and three squares of the same size are drawn on the periphery of each. Take the square close to the left side of the target object as an example, set it as Lai, and the square adjacent to Lai but far away from the target object is Lbi (where i = 1, 2, 3).

[0126] Furthermore, the step of determining whether to clear the first area to be cleared according to the depth data of the first area to be cleared includes:

[0127] Get the mean depth of the target object;

[0128] Obtaining a depth average of a first removable sub-region and a depth average of a second removable sub-region in the first removable region according to the depth data of the first removable region;

[0129] Obtaining a difference between a depth mean of the target object and a depth mean of a first clearable sub-region in the first to-be-cleared region, and determining the difference as a first difference;

[0130] Obtaining a difference between a depth mean of the target object and a depth mean of a second clearable sub-region in the first to-be-cleared region, and determining the difference as a second difference;

[0131] It is determined whether to clear the first area to be cleared according to the first difference and the second difference.

[0132] In the above implementation process, the first area to be cleared is determined to be cleared according to the difference in the depth mean, thereby reducing the error of the gripper during the gripping process, so that target objects at different positions can be gripped, and the situation of unstable gripping can be reduced.

[0133] Further, the step of judging whether to clear the first area to be cleared according to the first difference and the second difference includes:

[0134] Obtaining a difference between an absolute value of the first difference and an absolute value of the second difference, and determining the difference to be a third difference;

[0135] Determining whether the third difference is greater than a fourth threshold;

[0136] If so, clear the first area to be cleared.

[0137] In the above implementation process, the third difference is compared with the fourth threshold to determine whether the first area to be cleared can be cleared, thereby increasing the probability of the first area to be cleared being cleared, reducing the situation where the target object cannot be grasped, and improving the grasping speed.

[0138] According to the depth data of the first area to be cleared, it is determined whether to clear the first area to be cleared. The depth mean of the target object is set to Mz. The depth mean of Lai and Lbi areas are calculated and set to avg{Lai} and avg{Lbi} respectively. The judgment is made according to the following formula:

[0139] |avg{Lbi}-Mz|-|avg{Lai}-Mz|>2*h2

[0140] Among them, h2 is the minimum gripping depth that the gripper can reach on both sides of the target object.

[0141] Select the object that meets the above conditions and the symmetrical position on the right side of the target object also meets similar conditions, that is,

[0142] |avg{Rbi}-Mz|-|avg{Rai}-Mz|>2*h2

[0143] Among them, Rai is the square close to the right side of the target, Rbi is the square adjacent to Rai but far away from the target object, and the fourth threshold is 2*h2.

[0144] Both sides of this section of the target object satisfy that the depth of the area close to the target is smaller, and the depth of the area on the other side far from the target object is larger, that is, it can be cleared, and the first area to be cleared is cleared.

[0145] In S6, after the minimum graspable area is selected and determined, a position at three quarters of the length and close to the center of gravity is selected as the grasping point, and the center of gravity is point E. Let the minimum graspable area be M1. If an end point D of M1 is closest to point E, then point D is the position close to the center of gravity. Then the position at one quarter of the distance from point D is the grasping point.

[0146] After clearing, a point close to the target object is set around the target object (such as the center point of the Rai and Lai areas), the robot arm is controlled to move down to the position, and then moves toward the first clearable area, with a moving distance of the Lai side length to achieve the clearance of the interference.

[0147] The embodiments of the present application effectively improve the grasping method that can locate the target but cannot grasp it, or the grasping method drops midway due to unstable center of gravity, insufficient depth (insufficient contact area between the clamp and the target), etc., so as to achieve efficient grasping and avoid the occurrence of the above-mentioned situations.

[0148] Embodiment 2

[0149] In order to execute the method corresponding to the above embodiment 1 to achieve the corresponding functions and technical effects, a device for grasping an object is provided below, such as Figure 3 As shown, the device comprises:

[0150] An acquisition module 1 is used to acquire an image containing a target object to be captured;

[0151] Data acquisition module 2, used to obtain a first landing point area around the target object in the image where the gripper can land; used to obtain depth data of a second landing point area; used to determine a minimum graspable area in the second landing point area according to the depth data;

[0152] A segmentation module 3 is used to segment the first landing area according to the first threshold value to obtain a second landing area including a plurality of slice areas;

[0153] A grasping point determination module 4 is used to determine a grasping point in a minimum graspable area;

[0154] The grasping module 5 is used to control the gripper to grasp the target object at the grasping point.

[0155] In the above implementation process, the target object can be accurately and stably grasped by segmenting the area around the target object and performing graspability analysis based on the depth data. This can improve the efficiency and accuracy of object grasping, avoid being disturbed by interference objects around the object, improve the accuracy of object positioning, and achieve easy and stable grasping of objects of various shapes.

[0156] Furthermore, the data acquisition module 2 is also used for:

[0157] Acquire depth data of each slice area in the second landing area;

[0158] Performing depth evaluation on the depth data of each slice area respectively to obtain multiple depth evaluation results;

[0159] comparing the plurality of depth evaluation results with a second threshold respectively;

[0160] If the depth evaluation result is greater than or equal to the second threshold, determining the slice area corresponding to the depth evaluation result as a candidate graspable area;

[0161] If there is no depth assessment result greater than or equal to the second threshold, it is determined that there is no graspable area of ​​the target object, and the target object is grasped a second time;

[0162] Determine the minimum graspable area among the candidate graspable areas.

[0163] In the above implementation process, the depth data of each slice area after segmentation is evaluated, and the candidate graspable area is determined when the depth evaluation result meets the requirements. The graspable area can be located quickly and accurately to provide support for the grasping of the target object.

[0164] Furthermore, the data acquisition module 2 is also used for:

[0165] Get the depth data of all pixels in each slice area;

[0166] Arrange the depth data of all pixels in each slice area in ascending order to obtain a plurality of first sequences;

[0167] Selecting depth data in the plurality of first sequences respectively according to a third threshold value to obtain a plurality of second sequences;

[0168] Performing arithmetic averaging on the second sequence respectively to obtain multiple average values;

[0169] Multiple average values ​​are determined as the multiple depth evaluation results.

[0170] In the above implementation process, the depth data of the pixels in each slice area are sorted to obtain the corresponding sequence, and then the average value of each sequence is calculated as the depth evaluation result. The depth evaluation can be performed on each point in the slice area to ensure the safety and stability of the grasping action.

[0171] Furthermore, the data acquisition module 2 is also used for:

[0172] Merging a preset number of adjacent slice areas in the candidate graspable area to obtain multiple candidate minimum graspable areas;

[0173] The average depth, length and center of gravity distance of multiple candidate minimum graspable areas are weighted averaged to obtain a weighted score;

[0174] The candidate minimum graspable area with a high weight score is determined as the minimum graspable area.

[0175] In the above implementation process, some slice areas in the candidate graspable area are merged, and the merged candidate minimum graspable area is evaluated, so as to accurately find the most suitable position for grasping and reduce the possible errors in the grasping process.

[0176] Furthermore, the data acquisition module 2 is also used for:

[0177] Get the preset area around the target object;

[0178] Dividing the preset area according to a preset method to obtain a first area to be cleared;

[0179] Determining whether to clear the first area to be cleared according to the depth data of the first area to be cleared;

[0180] If so, the gripper is controlled to clear the first area to be cleared and then the target object is grasped for the second time.

[0181] In the above implementation process, the first area to be cleared is judged to be clearable, and then the gripper is controlled to clear it and then perform a second grip. The target object can be gripped in a complex and changeable environment without being disturbed by the surrounding environment, thereby improving the gripping efficiency.

[0182] Furthermore, the data acquisition module 2 is also used for:

[0183] Get the mean depth of the target object;

[0184] Obtaining a depth average of a first removable sub-region and a depth average of a second removable sub-region in the first removable region according to the depth data of the first removable region;

[0185] Obtaining a difference between a depth mean of the target object and a depth mean of a first clearable sub-region in the first to-be-cleared region, and determining the difference as a first difference;

[0186] Obtaining a difference between a depth mean of the target object and a depth mean of a second clearable sub-region in the first to-be-cleared region, and determining the difference as a second difference;

[0187] It is determined whether to clear the first area to be cleared according to the first difference and the second difference.

[0188] In the above implementation process, the first area to be cleared is determined to be cleared according to the difference in the depth mean, thereby reducing the error of the gripper during the gripping process, so that target objects at different positions can be gripped, and the situation of unstable gripping can be reduced.

[0189] Furthermore, the data acquisition module 2 is also used for:

[0190] Obtaining a difference between an absolute value of the first difference and an absolute value of the second difference, and determining the difference to be a third difference;

[0191] Determining whether the third difference is greater than a fourth threshold;

[0192] If so, clear the first area to be cleared.

[0193] In the above implementation process, the third difference is compared with the fourth threshold to determine whether the first area to be cleared can be cleared, thereby increasing the probability of the first area to be cleared being cleared, reducing the situation where the target object cannot be grasped, and improving the grasping speed.

[0194] The above-mentioned object grabbing device can implement the method of the above-mentioned embodiment 1. The options in the above-mentioned embodiment 1 are also applicable to this embodiment and will not be described in detail here.

[0195] The rest of the contents of the embodiments of the present application can refer to the contents of the above-mentioned embodiment 1, and will not be repeated in this embodiment.

[0196] Embodiment 3

[0197] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the object grasping method of the first embodiment.

[0198] Optionally, the above-mentioned electronic device may be a server.

[0199] See also Figure 3 , Figure 3 The schematic diagram of the structure of the electronic device provided in the embodiment of the present application. The electronic device may include a processor 31, a communication interface 32, a memory 33 and at least one communication bus 34. The communication bus 34 is used to realize direct connection and communication between these components.

[0200] Optionally, the electronic device may further include a storage controller and an input / output unit. The memory 33, the storage controller, the processor 31, the peripheral interface, and the input / output unit are electrically connected to each other directly or indirectly to achieve data transmission or interaction.

[0201] The input and output unit is used to provide users with the task creation and to create an optional time period or preset execution time for the task to enable interaction between the user and the server. The input and output unit can be, but is not limited to, a mouse and a keyboard.

[0202] Understandably, Figure 3 The structure shown is for illustration only. The electronic device may also include Figure 3 More or fewer components as shown, or with Figure 3 Different configurations are shown. Figure 3 The components shown in the figure can be implemented by hardware, vehicle-mounted software or a combination thereof.

[0203] In addition, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the object grabbing method of the first embodiment.

[0204] The embodiment of the present application also provides a computer program product, which, when running on a computer, enables the computer to execute the method described in the method embodiment.

[0205] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application. It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0206] The above is only a specific implementation of the present application, but the protection range of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical range disclosed in the present application, which should be included in the protection range of the present application. Therefore, the protection range of the present application should be based on the protection range of the claims.

Claims

1. A method for grasping an object, characterized in that: The method comprises: Acquire an image containing a target object to be grasped; Obtaining a first landing point area around the target object in the image where the gripper can fall; Segmenting the first landing area according to a first threshold to obtain a second landing area including a plurality of slice areas; Obtaining depth data of the second landing point area; Determine a minimum graspable area in the second landing area according to the depth data; determining a grasping point in the minimum graspable area; The gripper is controlled to move to the gripping point to grip the target object.

2. The method for grasping an object according to claim 1, characterized in that: The step of determining the minimum graspable area in the second landing area according to the depth data comprises: Acquire depth data of each slice area in the second landing point area; Performing depth evaluation on the depth data of each slice area respectively to obtain multiple depth evaluation results; comparing the plurality of depth assessment results with a second threshold respectively; If the depth evaluation result is greater than or equal to the second threshold, determining the slice area corresponding to the depth evaluation result as a candidate graspable area; If there is no depth evaluation result greater than or equal to the second threshold, it is determined that there is no graspable area of ​​the target object, and the target object is grasped a second time; The minimum graspable area among the candidate graspable areas is determined.

3. The method for grasping an object according to claim 2, characterized in that: The step of performing depth evaluation on the depth data of each slice area to obtain multiple depth evaluation results comprises: Acquire depth data of all pixels in each slice area; Arrange the depth data of all pixels in each slice area in ascending order to obtain a plurality of first sequences; Selecting depth data in the plurality of first sequences respectively according to a third threshold value to obtain a plurality of second sequences; performing arithmetic averaging on the second sequences respectively to obtain multiple average values; The plurality of average values ​​are determined as the plurality of depth evaluation results.

4. The method for grasping an object according to claim 2, characterized in that: The step of determining the minimum graspable area among the candidate graspable areas comprises: Merging a preset number of adjacent slice areas in the candidate graspable area to obtain a plurality of candidate minimum graspable areas; The average depth, length and center of gravity distance of multiple candidate minimum graspable areas are weighted averaged to obtain a weighted score; The candidate minimum graspable area with a high weight score is determined as the minimum graspable area.

5. The method for grasping an object according to claim 2, characterized in that: The step of performing secondary grasping on the target object comprises: Acquire a preset area around the target object; Dividing the preset area according to a preset method to obtain a first area to be cleared; Determining whether to clear the first area to be cleared according to the depth data of the first area to be cleared; If so, the gripper is controlled to clear the first area to be cleared and then the target object is grasped for a second time.

6. The method for grasping an object according to claim 5, characterized in that: The step of judging whether to clear the first area to be cleared according to the depth data of the first area to be cleared comprises: Obtaining a mean depth value of the target object; Obtaining a depth average of a first removable sub-region and a depth average of a second removable sub-region in the first removable sub-region according to the depth data of the first removable sub-region; Obtaining a difference between a depth mean of the target object and a depth mean of a first clearable sub-region in the first to-be-cleared region, and determining the difference as a first difference; Obtaining a difference between a depth mean of the target object and a depth mean of a second clearable sub-region in the first to-be-cleared region, and determining the difference as a second difference; Whether to clear the first area to be cleared is determined according to the first difference and the second difference.

7. The method for grasping an object according to claim 6, characterized in that: The step of judging whether to clear the first area to be cleared according to the first difference and the second difference includes: Obtaining a difference between an absolute value of the first difference and an absolute value of the second difference, and determining the difference to be a third difference; Determining whether the third difference is greater than a fourth threshold; If so, clear the first area to be cleared.

8. A device for grasping an object, characterized in that: The device comprises: An acquisition module, used for acquiring an image containing a target object to be grasped; A data acquisition module, for obtaining a first landing point area around the target object in the image where the gripper can land; for obtaining depth data of the second landing point area; and for determining a minimum graspable area in the second landing point area according to the depth data; A segmentation module, configured to segment the first landing area according to a first threshold value to obtain a second landing area including a plurality of slicing areas; A grasping point determination module, used to determine a grasping point in the minimum graspable area; The grasping module is used to control the gripper to grasp the target object at the grasping point.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the object grasping method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The device stores a computer program, which, when executed by a processor, implements the method for grasping an object as claimed in any one of claims 1 to 7.

Images