Object grasping method, device, electronic device and storage medium

Abstract

The present application provides a method, device, electronic device and storage medium for grasping an object, wherein the method includes: acquiring an image containing a target object to be grasped; obtaining a first landing area around the target object in the image for the gripper to fall; segmenting the first landing area according to a first threshold value to obtain a second landing area containing multiple slice areas; obtaining depth data of the second landing area; determining the minimum graspable area in the second landing area according to the depth data; determining a grasping point in the minimum graspable area; and controlling the gripper to grasp the target object at the grasping point. The implementation of the present application can improve the efficiency and accuracy of object grasping, will not be interfered with by interference objects around the object, can improve the accuracy of object positioning, and can also achieve easy and stable grasping of objects of various shapes.

Description

Technical Field

[0001] The present application relates to the field of robotic arm grasping technology, and more specifically, to a method, device, electronic device, and storage medium for grasping an object. Background Art

[0002] There are three steps in the grasping of a visual robotic arm: obtaining the target to be grasped through image recognition, analyzing the graspable position or area of ​​the target to be grasped, and determining the best grasping point for grasping.

[0003] Existing technologies use algorithms to find the optimal grasping point based on the specifications and posture of the target to be grasped, hardware limitations (such as the structure and clamping force of the gripper; the range of motion and load of the robotic arm), and environmental requirements. Common algorithm logics include those based on the principles of balance, morphology, and orientation. However, these methods have many drawbacks during implementation. For example, positioning can be easily disrupted by objects around the grasped object, causing damage to the gripper or robotic arm; they are only suitable for positioning stationary targets, not moving ones; and they are prone to missing or unstable grasps of objects with varying specifications and postures. Summary of the Invention

[0004] The purpose of this application is to provide a method, device, electronic device and storage medium for grasping an object, which can improve the efficiency and accuracy of object grasping, will not be interfered with by interference objects around the object, can improve the accuracy of object positioning, and can also achieve easy and stable grasping of objects of various shapes.

[0005] In a first aspect, an embodiment of the present application provides a method for grasping an object, the method comprising:

[0006] Acquire an image containing a target object to be grasped;

[0007] Obtaining a first landing area around the target object in the image, where the gripper can land;

[0008] Segmenting the first landing area according to a first threshold to obtain a second landing area including a plurality of slice areas;

[0009] Obtaining depth data of the second landing area;

[0010] determining a minimum graspable area in the second landing area according to the depth data;

[0011] determining a grasping point within the minimum graspable area;

[0012] The gripper is controlled to move to the gripping point to grip the target object.

[0013] In the above implementation process, by segmenting the area around the target object and performing graspability analysis based on the depth data, the target object can be accurately and stably grasped, which can improve the efficiency and accuracy of object grasping, and will not be disturbed by interference objects around the object. The accuracy of object positioning can be improved, and objects of various shapes can also be grasped easily and stably.

[0014] Furthermore, the step of determining the minimum graspable area in the second landing area according to the depth data includes:

[0015] Acquire depth data of each slice area in the second landing area;

[0016] Performing depth evaluation on the depth data of each slice area respectively to obtain multiple depth evaluation results;

[0017] comparing the plurality of depth assessment results with a second threshold respectively;

[0018] If the depth evaluation result is greater than or equal to the second threshold, determining the slice area corresponding to the depth evaluation result as a candidate graspable area;

[0019] If there is no depth assessment result greater than or equal to the second threshold, determining that the target object does not have a graspable area, and performing a secondary grasp on the target object;

[0020] The minimum graspable area among the candidate graspable areas is determined.

[0021] In the above implementation process, the depth data of each slice area after segmentation is evaluated, and the candidate graspable area is determined when the depth evaluation result meets the requirements. The graspable area can be located quickly and accurately to provide support for grasping the target object.

[0022] Furthermore, the step of performing depth evaluation on the depth data of each slice area to obtain multiple depth evaluation results includes:

[0023] Obtaining depth data of all pixels in each slice area;

[0024] Arrange the depth data of all pixels in each slice area in ascending order to obtain multiple first sequences;

[0025] Selecting depth data from the plurality of first sequences respectively according to a third threshold value to obtain a plurality of second sequences;

[0026] performing arithmetic averaging on the second sequences respectively to obtain multiple average values;

[0027] The plurality of average values ​​are determined as the plurality of depth evaluation results.

[0028] In the above implementation process, the depth data of the pixels in each slice area are sorted to obtain the corresponding sequence, and then the average value of each sequence is calculated as the depth evaluation result. Depth evaluation can be performed on each point in the slice area to ensure the safety and stability of the grasping action.

[0029] Furthermore, the step of determining the minimum graspable area among the candidate graspable areas includes:

[0030] Merging a preset number of adjacent slice areas in the candidate graspable area to obtain multiple candidate minimum graspable areas;

[0031] The average depth, length and center of gravity distance of multiple candidate minimum graspable areas are weighted averaged to obtain a weighted score;

[0032] The candidate minimum graspable area with the highest weight score is determined as the minimum graspable area.

[0033] In the above implementation process, some slice areas in the candidate graspable area are merged, and the merged candidate minimum graspable area is evaluated, so as to accurately find the most suitable position for grasping and reduce possible errors in the grasping process.

[0034] Furthermore, the step of performing a secondary grasping of the target object includes:

[0035] Acquire a preset area around the target object;

[0036] Divide the preset area according to a preset method to obtain a first area to be cleared;

[0037] Determining whether to clear the first area to be cleared according to the depth data of the first area to be cleared;

[0038] If so, the gripper is controlled to clear the first area to be cleared and then the target object is grasped for a second time.

[0039] In the above implementation process, the first area to be cleared is judged to be clearable, and then the gripper is controlled to clear it and then perform a second grip. This can clamp the target object in a complex and changeable environment without being disturbed by the surrounding environment, thereby improving the gripping efficiency.

[0040] Furthermore, the step of determining whether to clear the first area to be cleared based on the depth data of the first area to be cleared includes:

[0041] Obtaining a mean depth of the target object;

[0042] Obtaining, according to the depth data of the first area to be cleared, a depth average of a first clearable sub-area and a depth average of a second clearable sub-area in the first area to be cleared;

[0043] Obtaining a difference between a depth mean of the target object and a depth mean of a first clearable sub-region in the first to-be-cleared region, and determining the difference as a first difference;

[0044] Obtaining a difference between a depth mean of the target object and a depth mean of a second clearable sub-region in the first to-be-cleared region, and determining the difference as a second difference;

[0045] Whether to clear the first area to be cleared is determined according to the first difference and the second difference.

[0046] In the above implementation process, the first area to be cleared is determined to be cleared based on the difference in the depth mean, which reduces the error of the gripper during the gripping process, so that target objects at different positions can be gripped, reducing the situation of unstable gripping.

[0047] Furthermore, the step of determining whether to clear the first area to be cleared based on the first difference and the second difference includes:

[0048] Obtaining a difference between an absolute value of the first difference and an absolute value of the second difference, and determining the difference to be a third difference;

[0049] Determining whether the third difference is greater than a fourth threshold;

[0050] If so, clear the first area to be cleared.

[0051] In the above implementation process, the third difference is compared with the fourth threshold to determine whether the first area to be cleared can be cleared, thereby increasing the probability of the first area to be cleared being cleared, reducing the situation where the target object cannot be grasped, and improving the grasping speed.

[0052] In a second aspect, an embodiment of the present application further provides a device for grasping an object, the device comprising:

[0053] An acquisition module, used to acquire an image containing a target object to be grasped;

[0054] a data acquisition module, configured to obtain, in the image, a first landing area around the target object where the gripper can land; to obtain depth data of the second landing area; and to determine a minimum graspable area in the second landing area based on the depth data;

[0055] a segmentation module, configured to segment the first landing area according to a first threshold value to obtain a second landing area including a plurality of slice areas;

[0056] a grasping point determination module, configured to determine a grasping point in the minimum graspable area;

[0057] The gripping module is used to control the gripper to move to the gripping point to grip the target object.

[0058] In the above implementation process, by segmenting the area around the target object and performing graspability analysis based on the depth data, the target object can be accurately and stably grasped, which can improve the efficiency and accuracy of object grasping, and will not be disturbed by interference objects around the object. The accuracy of object positioning can be improved, and objects of various shapes can also be grasped easily and stably.

[0059] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any one of the first aspects when executing the computer program.

[0060] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which instructions are stored. When the instructions are executed on a computer, the computer executes the method as described in any one of the first aspects.

[0061] Other features and advantages of the present disclosure will be set forth in the following description, or some features and advantages may be inferred or unambiguously determined from the description, or may be learned by practicing the above-mentioned technology of the present disclosure.

[0062] It can be implemented according to the contents of the specification. The following is a detailed description of the preferred embodiments of the present application with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the range values. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0064] Figure 1 A schematic diagram of a process for grasping an object is provided for an embodiment of the present application;

[0065] Figure 2 A schematic diagram of the structure of a gripping device for an object provided in an embodiment of the present application;

[0066] Figure 3A schematic diagram of the structural composition of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0067] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0068] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0069] The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0070] Example 1

[0071] Figure 1 This is a flow chart of a method for grasping an object provided in an embodiment of the present application. Figure 1 As shown, the method includes:

[0072] S1, acquiring an image containing the target object to be grasped;

[0073] S2, obtaining a first landing area around the target object in the image where the gripper can land;

[0074] S3, segmenting the first landing area according to the first threshold to obtain a second landing area including a plurality of slice areas;

[0075] S4, obtaining depth data of the second landing area;

[0076] S5, determining a minimum graspable area in the second landing area according to the depth data;

[0077] S6, determining the grasping point in the minimum graspable area;

[0078] S7, controls the gripper to the grasping point to grasp the target object.

[0079] In the above implementation process, by segmenting the area around the target object and performing graspability analysis based on the depth data, the target object can be accurately and stably grasped, which can improve the efficiency and accuracy of object grasping, and will not be disturbed by interference objects around the object. The accuracy of object positioning can be improved, and objects of various shapes can also be grasped easily and stably.

[0080] In S1, the embodiment of the present application first identifies the target object before grasping it. The target recognition adopts target detection of the deep learning framework + rotation detection frame with angle information. Therefore, in the image containing the target object to be grasped, the target object has a detection frame, and the detection frame is a rectangular frame.

[0081] In S2, since the target object may have an irregular shape and the gripper itself has physical limitations, for convenience, the embodiment of the present application will treat the target as a rectangle. If the target object is uneven in thickness, it will be divided into several rectangular frames, and then its length and width will be determined in segments.

[0082] Therefore, the first landing area is the outer periphery of the narrower sides of the rectangular frame of the target object, that is, the two sides with smaller width of the rectangular frame are the narrower sides of the target object, which are also rectangular, with a length equal to the length of the longer side of the target object and a width twice the size of the clamping fingers of the gripper. It is twice as wide as the clamping fingers in order to increase the tolerance of the landing point.

[0083] In S3, there are two first landing areas, one on each side of the target object. Slice the two first landing areas. Assume the finger width is n and the length of the longer side of the rectangle is M. Each first landing area is an M*2n rectangle. Segments of length n (the first threshold) are performed on the M sides from the center to the ends. If the segmentation is not an integer multiple of n at the end, the segmentation stops. At this point, the M sides can be divided into (M / n rounded up) slice areas. After segmentation, there will be an equal number of symmetrical sliced ​​areas (slice areas) on the left and right sides of the rectangle, and each slice area is a 2n*n rectangle.

[0084] Furthermore, S5 includes:

[0085] Acquire depth data of each slice area in the second landing area;

[0086] Performing depth evaluation on the depth data of each slice area respectively to obtain multiple depth evaluation results;

[0087] comparing the plurality of depth evaluation results with a second threshold respectively;

[0088] If the depth evaluation result is greater than or equal to the second threshold, determining the slice area corresponding to the depth evaluation result as a candidate graspable area;

[0089] If there is no depth assessment result greater than or equal to the second threshold, it is determined that there is no graspable area of ​​the target object, and the target object is grasped a second time;

[0090] Determine the minimum graspable area among the candidate graspable areas.

[0091] In the above implementation process, the depth data of each slice area after segmentation is evaluated, and the candidate graspable area is determined when the depth evaluation result meets the requirements. The graspable area can be located quickly and accurately to provide support for grasping the target object.

[0092] A depth assessment is performed on each slicing area to ensure that the gripper falls within a threshold range, i.e., the minimum graspable area, when grasping. The minimum graspable area meets the requirements of the gripper's structure, gripping force, etc.

[0093] The depth data is based on the height value of each pixel, that is, the actual distance from the camera lens to a point in the image. The value of this depth data is set to z.

[0094] Suppose the depth data of all pixels in a certain slice area is {Zi} (where i = 1, 2, 3, ..., 2n*n), and assume that the distance between the target object and the camera is h1. According to the physical structure of the gripper, as long as the gripper can reach the depth of h2 on both sides of the target object, it can clamp the target object.

[0095] If every value in {zi} is greater than or equal to h1+h2 (the second threshold), the gripper can definitely reach its full depth. However, in reality, there may be some anomalies in the depth data, and there may be some non-compliant data at the edge of the slicing area, but this does not actually affect the gripper's ability to reach its full depth.

[0096] Furthermore, the step of performing depth evaluation on the depth data of each slice area to obtain multiple depth evaluation results includes:

[0097] Get the depth data of all pixels in each slice area;

[0098] Arrange the depth data of all pixels in each slice area in ascending order to obtain multiple first sequences;

[0099] Selecting depth data from the plurality of first sequences according to a third threshold value to obtain a plurality of second sequences;

[0100] Perform arithmetic averaging on the second sequence respectively to obtain multiple average values;

[0101] Multiple average values ​​are determined as the multiple depth evaluation results.

[0102] In the above implementation process, the depth data of the pixels in each slice area are sorted to obtain the corresponding sequence, and then the average value of each sequence is calculated as the depth evaluation result. Depth evaluation can be performed on each point in the slice area to ensure the safety and stability of the grasping action.

[0103] Based on redundancy considerations, this embodiment of the application sorts {Zi} from smallest to largest and selects the first two-thirds (the third threshold) of the data for arithmetic average. If this average is greater than or equal to h1 + h2, the depth assessment requirement is met, meaning that the slice area is the minimum graspable area.

[0104] The definition of h2 is related to the weight of the target object and the structure of the gripper. For example, if the target object is heavier, or the gripper fingers are narrow, or the friction of the gripper fingers is small, or the gripping force is small, then a larger h2 will be obtained.

[0105] Furthermore, the step of determining the minimum graspable area among the candidate graspable areas includes:

[0106] Merging a preset number of adjacent slice areas in the candidate graspable area to obtain multiple candidate minimum graspable areas;

[0107] The average depth, length and center of gravity distance of multiple candidate minimum graspable areas are weighted averaged to obtain a weighted score;

[0108] The candidate minimum graspable area with the highest weight score is determined as the minimum graspable area.

[0109] In the above implementation process, some slice areas in the candidate graspable area are merged, and the merged candidate minimum graspable area is evaluated, so as to accurately find the most suitable position for grasping and reduce possible errors in the grasping process.

[0110] After depth evaluation of each slice area, the minimum graspable area is found among the candidate graspable areas.

[0111] For example, if the length of a target object is M, the left and right sides can be divided into 22 slices. Among these slices, assume that m2, m3, m4, m5, m8, m9, m14, m15, and m16 all meet the minimum graspable area criteria. Based on the method of merging adjacent regions, they can be merged into three (predetermined number) candidate minimum graspable regions, namely [m2, m3, m4, m5], [m8, m9], and [m14, m15, m16]. Suppose these three candidate minimum graspable regions correspond to the upper part of the target object, namely M1, corresponding to [m2, m3, m4, m5]; M2, corresponding to [m8, m9]; and M3, corresponding to [m14, m15, m16].

[0112] The depth data of the three candidate minimum graspable areas are Z1, Z2, and Z3 respectively. Let Zmax be the maximum value among them, and then Zi / Zmax is the average depth (where i=1, 2, 3).

[0113] Let the length of M1 be L1, the length of M2 be L2, and the length of M3 be L3. Let Lmax be the maximum value, then Li / Lmax is the length (where i = 1, 2, 3).

[0114] The target length is M (assuming the two endpoints are AB). If the thickness is uneven, it will be simply divided into two segments according to the thickness. The thick end is one segment (set as AB segment), and the thin end is the other segment (set as BC segment). After the division, each segment is close to or equivalent to uniform thickness. Suppose the center of gravity of the entire target is point E, then the AE length formula is (W2*V2+W1*V1) / 2W1, where W1 is the width of AB and V1 is the length of AB; W2 (W1>W2) is the width of BC and V2 is the length of BC. Let the center distance of the three candidate minimum graspable areas be point A, namely AE1, AE2, and AE3, and the distance from the center of gravity of the target object is 1-(|AEi-AE| / AE) (where i=1, 2, 3).

[0115] The weighted average calculation is used for the three dimensions of average depth, length and center of gravity distance. The formula is as follows:

[0116] 0.3*(Zi / Zmax)+0.4*(Li / Lmax)+0.3*(1-(|AEi-AE| / AE))

[0117] According to this formula, the one with the highest weight score is selected as the minimum graspable area (the minimum graspable area is on the left and right sides of the target object, and is two symmetrical areas of the same size).

[0118] Furthermore, the step of performing secondary grasping on the target object includes:

[0119] Get the preset area around the target object;

[0120] Divide the preset area according to a preset method to obtain a first area to be cleared;

[0121] Determining whether to clear the first area to be cleared according to the depth data of the first area to be cleared;

[0122] If so, the gripper is controlled to clear the first area to be cleared and then the target object is grasped for the second time.

[0123] In the above implementation process, the first area to be cleared is judged to be clearable, and then the gripper is controlled to clear it and then perform a second grip. This can clamp the target object in a complex and changeable environment without being disturbed by the surrounding environment, thereby improving the gripping efficiency.

[0124] If there is no minimum graspable area for the gripper to grasp, the area around the target object is cleared and then a second grasp is performed to obtain a preset area around the target object. The preset area is set according to the depth data around the target object. For example, the depth distribution of a certain area has a large difference, and the closer to the target object, the smaller the depth, and the farther away from the target object, the deeper the depth, and the area of ​​the area with a large depth far away from the target object meets a threshold (50% of the size of the target object), which is defined as the preset area.

[0125] Obtain the depth distribution of a preset area. Divide the area into 2 to 10 equal parts (preset method) based on the longest side of the target object. The specific amount of equal parts is linearly related to the target length. In this embodiment of the application, the area to be cleared is divided into three equal parts. For the narrower sides of the target object, three square areas are drawn on each side (with a side length of one-third of the longest side of the target object). Three squares of the same size are drawn on the periphery of each. For example, the square close to the left side of the target object is set as Lai, and the squares adjacent to Lai but far from the target object are Lbi (where i = 1, 2, 3).

[0126] Furthermore, the step of determining whether to clear the first area to be cleared according to the depth data of the first area to be cleared includes:

[0127] Get the mean depth of the target object;

[0128] Obtaining a depth average of a first clearable sub-region and a depth average of a second clearable sub-region in the first clearable region according to the depth data of the first clearable region;

[0129] Obtaining a difference between a depth mean of the target object and a depth mean of a first clearable sub-region in the first to-be-cleared region, and determining the difference as a first difference;

[0130] Obtaining a difference between a depth mean of the target object and a depth mean of a second clearable sub-region in the first to-be-cleared region, and determining the difference as a second difference;

[0131] It is determined whether to clear the first area to be cleared according to the first difference and the second difference.

[0132] In the above implementation process, the first area to be cleared is determined to be cleared based on the difference in the depth mean, which reduces the error of the gripper during the gripping process, so that target objects at different positions can be gripped, reducing the situation of unstable gripping.

[0133] Furthermore, the step of determining whether to clear the first area to be cleared according to the first difference and the second difference includes:

[0134] Obtaining a difference between an absolute value of the first difference and an absolute value of the second difference, and determining the difference as a third difference;

[0135] determining whether the third difference is greater than a fourth threshold;

[0136] If so, clear the first area to be cleared.

[0137] In the above implementation process, the third difference is compared with the fourth threshold to determine whether the first area to be cleared can be cleared, thereby increasing the probability of the first area to be cleared being cleared, reducing the situation where the target object cannot be grasped, and improving the grasping speed.

[0138] Determine whether to clear the first area to be cleared based on the depth data of the first area to be cleared. Set the depth mean of the target object to Mz. Calculate the depth mean of the Lai and Lbi areas and set them as avg{Lai} and avg{Lbi} respectively. Make the judgment according to the following formula:

[0139] |avg{Lbi}-Mz|-|avg{Lai}-Mz|>2*h2

[0140] Among them, h2 is the minimum gripping depth that the gripper can reach on both sides of the target object.

[0141] Select the one that meets the above conditions and the symmetrical position on the right side of the target object also meets similar conditions, that is,

[0142] |avg{Rbi}-Mz|-|avg{Rai}-Mz|>2*h2

[0143] Here, Rai is the square close to the right side of the target, Rbi is the square adjacent to Rai but far away from the target object, and the fourth threshold is 2*h2.

[0144] On both sides of this section of the target object, the depth of the area close to the target is smaller, and the depth of the area on the other side away from the target object is larger, which means that it can be cleared, and the first area to be cleared is cleared.

[0145] In S6, after the minimum graspable area is determined, a position three-quarters of the length, close to the center of gravity, is selected as the grasping point. The center of gravity is point E. Let the minimum graspable area be M1. If an endpoint D of M1 is closest to point E, then point D is located close to the center of gravity. Then, the position one-quarter of the distance from point D is the grasping point.

[0146] After clearing, set a point close to the target object around the target object (such as the center point of the Rai and Lai areas), control the robotic arm to move down to this position, and then move toward the first clearable area, moving a distance equal to the length of the Lai side to clear the interference object.

[0147] The embodiments of the present application effectively improve the grasping method that can locate the target but cannot grasp it, or the grasping method drops midway due to unstable center of gravity, insufficient depth (insufficient contact area between the clamping claw and the target), etc., so as to achieve efficient grasping and avoid the occurrence of the above-mentioned situations.

[0148] Example 2

[0149] In order to execute the method corresponding to the above embodiment 1 to achieve the corresponding functions and technical effects, a device for grasping an object is provided below, such as Figure 3 As shown, the device includes:

[0150] Acquisition module 1, used to acquire an image containing a target object to be grasped;

[0151] Data acquisition module 2 is used to obtain a first landing area around the target object in the image where the gripper can land; to obtain depth data of a second landing area; and to determine a minimum graspable area in the second landing area based on the depth data;

[0152] A segmentation module 3 is configured to segment the first landing area according to a first threshold value to obtain a second landing area including a plurality of slice areas;

[0153] A grasping point determination module 4 is used to determine the grasping point in the minimum graspable area;

[0154] The gripping module 5 is used to control the gripper to move to the gripping point to grip the target object.

[0155] In the above implementation process, by segmenting the area around the target object and performing graspability analysis based on the depth data, the target object can be accurately and stably grasped, which can improve the efficiency and accuracy of object grasping, and will not be disturbed by interference objects around the object. The accuracy of object positioning can be improved, and objects of various shapes can also be grasped easily and stably.

[0156] Furthermore, the data acquisition module 2 is also used for:

[0157] Acquire depth data of each slice area in the second landing area;

[0158] Performing depth evaluation on the depth data of each slice area respectively to obtain multiple depth evaluation results;

[0159] comparing the plurality of depth evaluation results with a second threshold respectively;

[0160] If the depth evaluation result is greater than or equal to the second threshold, determining the slice area corresponding to the depth evaluation result as a candidate graspable area;

[0161] If there is no depth assessment result greater than or equal to the second threshold, it is determined that there is no graspable area of ​​the target object, and the target object is grasped a second time;

[0162] Determine the minimum graspable area among the candidate graspable areas.

[0163] In the above implementation process, the depth data of each slice area after segmentation is evaluated, and the candidate graspable area is determined when the depth evaluation result meets the requirements. The graspable area can be located quickly and accurately to provide support for grasping the target object.

[0164] Furthermore, the data acquisition module 2 is also used for:

[0165] Get the depth data of all pixels in each slice area;

[0166] Arrange the depth data of all pixels in each slice area in ascending order to obtain multiple first sequences;

[0167] Selecting depth data from the plurality of first sequences according to a third threshold value to obtain a plurality of second sequences;

[0168] Perform arithmetic averaging on the second sequence respectively to obtain multiple average values;

[0169] Multiple average values ​​are determined as the multiple depth evaluation results.

[0170] In the above implementation process, the depth data of the pixels in each slice area are sorted to obtain the corresponding sequence, and then the average value of each sequence is calculated as the depth evaluation result. Depth evaluation can be performed on each point in the slice area to ensure the safety and stability of the grasping action.

[0171] Furthermore, the data acquisition module 2 is also used for:

[0172] Merging a preset number of adjacent slice areas in the candidate graspable area to obtain multiple candidate minimum graspable areas;

[0173] The average depth, length and center of gravity distance of multiple candidate minimum graspable areas are weighted averaged to obtain a weighted score;

[0174] The candidate minimum graspable area with the highest weight score is determined as the minimum graspable area.

[0175] In the above implementation process, some slice areas in the candidate graspable area are merged, and the merged candidate minimum graspable area is evaluated, so as to accurately find the most suitable position for grasping and reduce possible errors in the grasping process.

[0176] Furthermore, the data acquisition module 2 is also used for:

[0177] Get the preset area around the target object;

[0178] Divide the preset area according to a preset method to obtain a first area to be cleared;

[0179] Determining whether to clear the first area to be cleared according to the depth data of the first area to be cleared;

[0180] If so, the gripper is controlled to clear the first area to be cleared and then the target object is grasped for the second time.

[0181] In the above implementation process, the first area to be cleared is judged to be clearable, and then the gripper is controlled to clear it and then perform a second grip. This can clamp the target object in a complex and changeable environment without being disturbed by the surrounding environment, thereby improving the gripping efficiency.

[0182] Furthermore, the data acquisition module 2 is also used for:

[0183] Get the mean depth of the target object;

[0184] Obtaining a depth average of a first clearable sub-region and a depth average of a second clearable sub-region in the first clearable region according to the depth data of the first clearable region;

[0185] Obtaining a difference between a depth mean of the target object and a depth mean of a first clearable sub-region in the first to-be-cleared region, and determining the difference as a first difference;

[0186] Obtaining a difference between a depth mean of the target object and a depth mean of a second clearable sub-region in the first to-be-cleared region, and determining the difference as a second difference;

[0187] It is determined whether to clear the first area to be cleared according to the first difference and the second difference.

[0188] In the above implementation process, the first area to be cleared is determined to be cleared based on the difference in the depth mean, which reduces the error of the gripper during the gripping process, so that target objects at different positions can be gripped, reducing the situation of unstable gripping.

[0189] Furthermore, the data acquisition module 2 is also used for:

[0190] Obtaining a difference between an absolute value of the first difference and an absolute value of the second difference, and determining the difference as a third difference;

[0191] determining whether the third difference is greater than a fourth threshold;

[0192] If so, clear the first area to be cleared.

[0193] In the above implementation process, the third difference is compared with the fourth threshold to determine whether the first area to be cleared can be cleared, thereby increasing the probability of the first area to be cleared being cleared, reducing the situation where the target object cannot be grasped, and improving the grasping speed.

[0194] The object grabbing device can implement the method of the above embodiment 1. The options in the above embodiment 1 are also applicable to this embodiment and will not be described in detail here.

[0195] The rest of the contents of the embodiments of this application can refer to the contents of the above-mentioned embodiment 1, and will not be repeated in this embodiment.

[0196] Example 3

[0197] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the object grasping method of the first embodiment.

[0198] Optionally, the above-mentioned electronic device may be a server.

[0199] See Figure 3 , Figure 3 Schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include a processor 31, a communication interface 32, a memory 33, and at least one communication bus 34. The communication bus 34 is used to enable direct communication between these components.

[0200] Optionally, the electronic device may further include a storage controller and an input / output unit. The memory 33, storage controller, processor 31, peripheral interface, and input / output unit are electrically connected to each other directly or indirectly to achieve data transmission or interaction.

[0201] The input and output unit is used to provide users with the ability to create tasks and to create optional start time periods or preset execution times for the tasks to enable interaction between the user and the server. The input and output unit can be, but is not limited to, a mouse and keyboard.

[0202] I understand. Figure 3 The structure shown is only for illustration, and the electronic device may also include Figure 3 More or fewer components than shown, or with Figure 3 Different configurations shown. Figure 3 The components shown in the figure can be implemented using hardware, vehicle-mounted software, or a combination thereof.

[0203] In addition, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the object grasping method of the first embodiment.

[0204] An embodiment of the present application further provides a computer program product, which, when running on a computer, enables the computer to execute the method described in the method embodiment.

[0205] The foregoing is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included within the scope of protection of the present application. It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined or explained in subsequent figures.

[0206] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included within the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for grasping an object, characterized in that: The method comprises: Acquire an image containing a target object to be grasped; Obtaining a first landing area around the target object in the image, where the gripper can land; Segmenting the first landing area according to a first threshold to obtain a second landing area including a plurality of slice areas; Obtaining depth data of the second landing area; determining a minimum graspable area in the second landing area according to the depth data; determining a grasping point within the minimum graspable area; The gripper is controlled to move to the gripping point to grip the target object.

2. The method for grasping an object according to claim 1, wherein: The step of determining the minimum graspable area in the second landing area according to the depth data comprises: Acquire depth data of each slice area in the second landing area; Performing depth evaluation on the depth data of each slice area respectively to obtain multiple depth evaluation results; comparing the plurality of depth assessment results with a second threshold respectively; If the depth evaluation result is greater than or equal to the second threshold, determining the slice area corresponding to the depth evaluation result as a candidate graspable area; If there is no depth assessment result greater than or equal to the second threshold, determining that the target object does not have a graspable area, and performing a secondary grasp on the target object; The minimum graspable area among the candidate graspable areas is determined.

3. The object grasping method according to claim 2, characterized in that: The step of performing depth evaluation on the depth data of each slice area to obtain multiple depth evaluation results includes: Obtaining depth data of all pixels in each slice area; Arrange the depth data of all pixels in each slice area in ascending order to obtain multiple first sequences; Selecting depth data from the plurality of first sequences respectively according to a third threshold value to obtain a plurality of second sequences; performing arithmetic averaging on the second sequences respectively to obtain multiple average values; The plurality of average values ​​are determined as the plurality of depth evaluation results.

4. The method for grasping an object according to claim 2, wherein: The step of determining the minimum graspable area among the candidate graspable areas comprises: Merging a preset number of adjacent slice areas in the candidate graspable area to obtain multiple candidate minimum graspable areas; The average depth, length and center of gravity distance of multiple candidate minimum graspable areas are weighted averaged to obtain a weighted score; The candidate minimum graspable area with the highest weight score is determined as the minimum graspable area.

5. The object grasping method according to claim 2, characterized in that: The step of performing secondary grasping on the target object comprises: Acquire a preset area around the target object; Divide the preset area according to a preset method to obtain a first area to be cleared; Determining whether to clear the first area to be cleared according to the depth data of the first area to be cleared; If so, the gripper is controlled to clear the first area to be cleared and then the target object is grasped for a second time.

6. The method for grasping an object according to claim 5, characterized in that: The step of determining whether to clear the first area to be cleared based on the depth data of the first area to be cleared includes: Obtaining a mean depth of the target object; Obtaining, according to the depth data of the first area to be cleared, a depth average of a first clearable sub-area and a depth average of a second clearable sub-area in the first area to be cleared; Obtaining a difference between a depth mean of the target object and a depth mean of a first clearable sub-region in the first to-be-cleared region, and determining the difference as a first difference; Obtaining a difference between a depth mean of the target object and a depth mean of a second clearable sub-region in the first to-be-cleared region, and determining the difference as a second difference; Whether to clear the first area to be cleared is determined according to the first difference and the second difference.

7. The object grasping method according to claim 6, characterized in that: The step of determining whether to clear the first area to be cleared according to the first difference and the second difference includes: Obtaining a difference between an absolute value of the first difference and an absolute value of the second difference, and determining the difference to be a third difference; Determining whether the third difference is greater than a fourth threshold; If so, clear the first area to be cleared.

8. A device for grasping an object, characterized in that: The device comprises: An acquisition module, used to acquire an image containing a target object to be grasped; a data acquisition module, configured to obtain, in the image, a first landing area around the target object where the gripper can land; to obtain depth data of a second landing area; and to determine a minimum graspable area in the second landing area based on the depth data; a segmentation module, configured to segment the first landing area according to a first threshold value to obtain a second landing area including a plurality of slice areas; a grasping point determination module, configured to determine a grasping point in the minimum graspable area; The gripping module is used to control the gripper to move to the gripping point to grip the target object.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the object grasping method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The device stores a computer program, which, when executed by a processor, implements the object grasping method according to any one of claims 1 to 7.

Images