A method, apparatus and robot for grasping

Abstract

The present disclosure relates to a grabbing method, device and robot, wherein the method comprises: in response to receiving a grabbing instruction, instructing a camera module of the robot to capture an environment image of the surrounding environment, the grabbing instruction including a grabbing target; identifying the environment image to determine that the grabbing target exists in the environment image; determining that the grabbing target is within a preset grabbable range; determining a matching grabbing point from the coordinate point of the grabbing target according to a set of preset grabbable points within the preset grabbable range; determining a preset path corresponding to the matching grabbing point as a grabbing path according to a preset path of each grabbable point in the set of preset grabbable points; and moving a mechanical arm according to the grabbing path to grab the grabbing target. The present disclosure solves the problem of grabbing objects in a non-fixed workspace, enabling the robot to grab objects in combination with its own movement, and the way of determining the grabbing point and the grabbing path is practical and efficient.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of robots, and in particular, to a grabbing method, device and robot. BACKGROUND

[0002] At present, industrial robot arms are relatively mature in object grabbing. The industrial robot arm generally extends from the body. When grabbing an object, the body is stationary, and the robot arm performs object grabbing. Therefore, it is only applicable to the grabbing of objects in a fixed workspace. SUMMARY

[0003] To overcome the problems in the related art, embodiments of the present disclosure provide a grabbing method, device and robot. The technical solutions are as follows.

[0004] According to a first aspect of an embodiment of the present disclosure, a grabbing method is provided, and the method comprises:

[0005] In response to receiving a grabbing instruction, instructing a camera module of a robot to capture an environment image of a surrounding environment, the grabbing instruction including a grabbing target;

[0006] Identifying the environment image to determine that the grabbing target exists in the environment image;

[0007] Determining that the grabbing target is within a preset grabbable range;

[0008] According to a set of grabbable points within the preset grabbable range, determining a matching grabbing point from the coordinate point of the grabbing target;

[0009] According to a preset path of each grabbable point in the set of preset grabbable points, determining the preset path corresponding to the matching grabbing point as a grabbing path;

[0010] Moving the robot arm according to the grabbing path to grab the grabbing target.

[0011] In one embodiment, the method further comprises:

[0012] Obtaining a set of grabbable points by randomly sampling a preset number of points in a grabbable three-dimensional space within the grabbable range of the robot;

[0013] For each grabbable point in the set of grabbable points, generating at least one path from the grabbing starting point of the robot arm to the grabbable point as a preset path of the grabbable point;

[0014] Storing each grabbable point and the corresponding at least one preset path to generate a corresponding relationship.

[0015] In one embodiment, the moving the robot arm according to the grabbing path to grab the grabbing target comprises:

[0016] determining a robot arm grabbing starting point of the grabbing path;

[0017] fine-tuning a form of the robot to make a robot arm end located at the robot arm grabbing starting point;

[0018] moving the robot arm to grab the grabbing target according to the grabbing path.

[0019] In one embodiment, the determination of the preset path corresponding to the matching grabbing point as the grabbing path comprises:

[0020] determining whether the preset path corresponding to the matching grabbing point exists an obstacle;

[0021] determining the preset path without the obstacle as the grabbing path.

[0022] In one embodiment, the method further comprises:

[0023] if the preset paths corresponding to the matching grabbing points all exist the obstacle, re-determining the matching grabbing point of the grabbing target.

[0024] In one embodiment, the determination of the grabbing target in a preset grabbable range comprises:

[0025] determining whether the grabbing target is in the preset grabbable range according to depth information of the grabbing target or a distance from the grabbing target; wherein the environment image comprises the depth information;

[0026] if the grabbing target is not in the preset grabbable range, moving a robot body to make the grabbing target in the preset grabbable range.

[0027] In one embodiment, the identification of the environment image to determine that the environment image exists the grabbing target comprises:

[0028] identifying the environment image according to a deep learning model to determine that the environment image exists the grabbing target; wherein the deep learning model is obtained by pre-training a picture of the grabbing target through a deep learning network.

[0029] According to a second aspect of the embodiments of the present disclosure, a grabbing device is provided, the device comprising:

[0030] a first acquisition module, configured to instruct a camera module of a robot to capture an environment image of a surrounding environment in response to receiving a grabbing instruction, the grabbing instruction comprising a grabbing target;

[0031] The first determining module is configured to identify the environment image and determine that the grabbing target exists in the environment image.

[0032] The second determining module is configured to determine that the grabbing target is within a preset grabbable range.

[0033] The third determining module is configured to determine a matching grabbing point from the coordinate point of the grabbing target according to a set of grabbable points within the preset grabbable range.

[0034] The fourth determining module is configured to determine a preset path corresponding to the matching grabbing point as a grabbing path according to a preset path of each grabbable point in the set of grabbable points.

[0035] The grabbing module is configured to move the robot arm to grab the grabbing target according to the grabbing path.

[0036] In an embodiment, the device further includes:

[0037] The second obtaining module is configured to obtain the set of grabbable points by randomly sampling a preset number of points in a grabbable three-dimensional space within the robot grabbable range.

[0038] The generating module is configured to generate at least one path from the robot arm grabbing starting point to each grabbable point in the set of grabbable points as a preset path of the grabbable point.

[0039] The storage module is configured to store each grabbable point and the corresponding at least one preset path to generate a corresponding relationship.

[0040] In an embodiment, the grabbing module includes:

[0041] The determining unit is configured to determine a robot arm grabbing starting point of the grabbing path.

[0042] The adjusting unit is configured to finely adjust the form of the robot to enable the robot arm end to be located at the robot arm grabbing starting point.

[0043] The grabbing unit is configured to move the robot arm to grab the grabbing target according to the grabbing path.

[0044] According to a third aspect of the embodiments of the present disclosure, a grabbing device is provided, including:

[0045] A processor;

[0046] A memory for storing processor-executable instructions;

[0047] The processor is configured to:

[0048] In response to receiving the grabbing instruction, instructing the camera module of the robot to capture an environment image of the surrounding environment, the grabbing instruction including the grabbing target;

[0049] Identifying the environment image to determine that the grabbing target exists in the environment image;

[0050] Determining that the grabbing target is within a preset grabbable range;

[0051] From a set of preset grabbable points within the preset grabbable range, determining a matching grabbable point from the coordinate point of the grabbing target;

[0052] According to the preset path of each preset grabbable point in the set of preset grabbable points, determining the preset path corresponding to the matching grabbable point as a grabbing path;

[0053] According to the grabbing path, moving the mechanical arm to grab the grabbing target.

[0054] According to a fourth aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, and the computer readable storage medium stores computer instructions, and the instructions are executed by a processor to implement the steps of the method according to any one of the preceding aspects.

[0055] According to a fifth aspect of the embodiments of the present disclosure, a robot is provided, and the robot comprises the grabbing device.

[0056] The technical scheme provided in the present application has the following beneficial effects: for a movable robot, first, determine whether there is a grabbing target by capturing an image and determine that the grabbing target is within a preset grabbable range, then determine a grabbing point by matching the coordinate point of the grabbing target, and then obtain a corresponding preset path as a grabbing path. The problem of grabbing objects in a non-fixed workspace is solved, so that the robot can grab objects in combination with its own movement, and the way of determining the grabbing point and the grabbing path is practical and efficient.

[0057] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0058] The accompanying drawings, which are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure.

[0059] Figure 1a is a schematic diagram of a robot according to an exemplary embodiment.

[0060] Figure 1b is a schematic diagram of a robot according to an exemplary embodiment.

[0061] Figure 2 is a flowchart of a grasping method according to an example embodiment.

[0062] Figure 3 is a flowchart of a grasping method according to an example embodiment.

[0063] Figure 4 is a structural diagram of a grasping device according to an example embodiment.

[0064] Figure 5 is a structural diagram of a grasping device according to an example embodiment. DETAILED DESCRIPTION

[0065] The example embodiments will be described in detail herein with reference to the drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following example embodiments do not represent all implementations consistent with the present disclosure. Instead, they are merely examples of apparatuses and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0066] The technical solution of the present application is mainly applied to a robot with a mechanical arm that can walk, such as a humanoid robot. Compared with the fixed mechanical arm grasping, the humanoid robot can grasp based on walking, which has the advantages of non-fixed working space and more flexible grasping range, but it also brings greater difficulty to the accurate grasping of objects.

[0067] Figure 1a and Figure 1b is a schematic diagram of a robot according to an example embodiment, as Figure 1a and Figure 1b As shown in FIG. 1, the robot includes a robot body 101, a mechanical arm 102; wherein the robot body 101 can include a biped, realizing the walking of the robot, and being more humanized. In other embodiments of the present application, the robot body 101 can include a mobile chassis, realizing the walking of the robot. The mechanical arm 102 is connected with the body 101 and extends outward from the body 101. The mechanical arm can be one mechanical arm, or can be a left and right mechanical arm simulating a human. Alternatively, in other embodiments of the present application, the number of mechanical arms can be more. The present application does not limit the number and specific position of the mechanical arm.

[0068] Figure 2 is a flowchart of a grasping object method according to an example embodiment, the execution subject of the method can be a robot, and further, the execution subject of the method can be a processor arranged in the robot; as shown in FIG. 1, the method includes the following steps 201-206:

[0069] In step 201, in response to receiving the grabbing instruction, the camera module of the robot is instructed to take pictures of the surrounding environment to obtain an environment image, and the grabbing target is included in the grabbing instruction.

[0070] For example, the robot can be equipped with a receiving module for receiving various types of grabbing instructions, such as receiving a grabbing instruction sent by a user to the robot through a mobile phone APP, receiving a grabbing instruction issued by a user in a voice, receiving a grabbing instruction issued by a user in a gesture, etc. In an embodiment, the robot providing an interface for TCP / IP protocol communication can also receive a grabbing instruction sent by a host computer.

[0071] The robot can also be equipped with a camera module, for example, if it is a humanoid robot, a camera module can be arranged at the head position or body position of the robot for taking pictures of the surrounding environment of the robot. When taking pictures, the robot can take pictures of the surrounding environment of, for example, 360 degrees, or can be limited to taking pictures of the surrounding environment of a preset angle, for example, 180 degrees in front of the robot. For example, the camera module can be, for example, an RGBD (RED, Green, Blue, Depth) depth camera, a binocular camera, etc. Among them, the RGBD depth camera has the feature that it can directly measure the distance of each pixel in the image from the camera through infrared structured light or TOF (Time of flight) principle; the binocular camera can indirectly obtain the depth information of the common view pixels of the two cameras through parallax.

[0072] In order to make the movement of the robot more accurate, the robot can also be equipped with an IMU (Inertial Measurement Unit), which is a sensor mainly used for detecting and measuring acceleration and rotational motion, and can provide the instantaneous motion state of the robot.

[0073] In step 202, the environment image is identified to determine whether there is a grabbing target in the environment image.

[0074] After taking the environment image, the robot identifies the environment image to identify whether there is a grabbing target in the environment image. For example, the grabbing instruction received in step 201 is "grab the cup", and the robot identifies whether there is a cup in the environment image.

[0075] In an embodiment of the present application, the environment image can be identified according to a deep learning model to determine whether the picking target exists in the environment image; wherein the deep learning model is obtained by pre-training the pictures of the picking target through a deep learning network. Through the deep learning model, not only can the picking target be identified in the image, but also the coordinate points of the picking target in the environment image can be obtained, which can be multiple, such as along the outer contour of the picking target, and a coordinate point can be obtained every predetermined distance, so as to obtain multiple coordinate points of the picking target in the environment image. In the present application, any suitable deep learning network can be used to implement the target detection algorithm, such as yolo (a target detector using deep convolutional neural network learning features to detect objects), faster-rcnn (fast region convolutional neural network), and the specific algorithm of target detection is not described here.

[0076] In step 203, it is determined that the picking target is in the preset pickable range.

[0077] The preset pickable range refers to the pickable range of the robot arm when the robot body does not move and only the robot arm moves. For example, a preset range with the initial end of the robot arm as the center and a radius of M is the preset pickable range. A preset range with the end of the robot arm as the starting point and a distance of N is the preset pickable range. Or, a preset range within a horizontal distance of S from the robot is the preset pickable range.

[0078] In step 204, a matching picking point is determined from the coordinate points of the picking target according to the set of pickable points in the preset pickable range.

[0079] The set of pickable points is predetermined, which refers to a number of points in the preset pickable range. For example, the coordinate points of the picking target can be obtained when it is determined that the picking target exists in the environment image during the identification of the environment image. In this step, it is determined whether there is a coordinate point in the set of pickable points among the multiple coordinate points of the picking target, and if so, the coordinate point is determined as the matching picking point.

[0080] In step 205, a preset path corresponding to the matching picking point is determined as the picking path according to the preset path of each pickable point in the preset pickable point set.

[0081] The preset path of each pickable point in the set of pickable points can be predetermined, that is, the picking path of the robot arm is planned in advance for each pickable point in the preset pickable range. In this way, as long as the pickable point is determined, the corresponding picking path can be quickly obtained.

[0082] In step 206, the mechanical arm is moved according to the grabbing path to grab the grabbing target.

[0083] In the technical solution provided in the present application, for a movable robot, it is first determined that the grabbing target is in a preset grabbable range, then a grabbing point is determined by matching the coordinate points of the grabbing target, and a corresponding preset path is obtained as a grabbing path. The problem of grabbing objects in a non-fixed workspace is solved, so that the robot can grab objects in combination with its movement, and the way of determining the grabbing point and the grabbing path is practical and efficient.

[0084] In an embodiment of the present application, on the basis of the above-mentioned embodiment, the method for grabbing an object disclosed in the present application further comprises steps A1-A3 for pre-planning the grabbing path of the mechanical arm:

[0085] In step A1, a set of grabbable points is obtained by randomly sampling a preset number of points in the grabbable three-dimensional space within the grabbable range of the robot.

[0086] For example, when the mechanical arm of the robot is installed, the grabbable range of the robot can be basically determined, and the grabbable range is related to the length of the mechanical arm, the rotation angle of the mechanical arm, etc. As described above, for example, a preset range with the initial end of the mechanical arm of the robot as the center and a radius of M is the preset grabbable range; or a preset range with a distance of N or less from the robot is the preset grabbable range. In addition, when the robot is configured with multiple mechanical arms, the grabbable range is the superposition of the grabbable ranges of the multiple mechanical arms.

[0087] Randomly sampling points in the grabbable three-dimensional space within the grabbable range as grabbable points. For example, 100 grabbable points P1(x1, y1, z1), P2(x2, y2, z2), … P 100 (x 100 , y 100 , z 100 ) are obtained in the grabbable three-dimensional space as a set of grabbable points; wherein x, y and z represent three-dimensional coordinates in the grabbable three-dimensional space of the robot.

[0088] In step A2, for each grabbable point in the set of grabbable points, at least one path from the grabbing starting point of the mechanical arm to the grabbing point is generated as the preset path of the grabbable point.

[0089] For P1(x1, y1, z1) in the set of graspable points, at least one path S1 from the robot arm grasp starting point to the grasp point P1 is generated as the preset path of P1, for P2(x2, y2, z2) in the set of graspable points, at least one path S2 from the robot arm grasp starting point to the grasp point P2 is generated as the preset path of P2, and so on. Specifically, the preset path from the starting point to the grasp point can be generated by a three-dimensional A* algorithm. The A*(A-Star) algorithm is a direct search method for solving the shortest path in a static road network, and the specific algorithm is not described here.

[0090] The grasp starting point of the robot arm is, for example, the position of the end of the robot arm when the robot is upright and the robot arm is in an initial state. It can also be the position of the end of the robot arm after the robot is slightly bent down, bent over, and turned around. The starting points of different preset paths can be the same or different.

[0091] In an embodiment of the present application, in order to ensure that a usable grasp path can be selected from the preset paths during actual grasping, more than one preset path can be generated for each graspable point. Preferably, four different preset paths can be generated for each graspable point. In this way, when one of the preset paths fails to grasp, other preset paths can be selected for grasping again.

[0092] In step A3, each grasp point is associated with at least one preset path to generate a corresponding relationship.

[0093] For example, as shown in the following table, the corresponding relationship generated by associating each grasp point with at least one preset path is shown. Each graspable point corresponds to four different preset paths.

[0094]

[0095] In an embodiment of the present application, on the basis of the above-mentioned embodiment, the step 205 of determining the preset path corresponding to the matching grasp point as the grasp path according to the preset path of each graspable point in the set of graspable points can include the following steps B1-B2.

[0096] In step B1, it is determined whether the preset path corresponding to the matching grasp point exists an obstacle.

[0097] In the embodiment of the present application, the robot can check whether there is an obstacle on the preset path through the camera vision of the set camera module. Specifically, the obstacle detection technology based on the depth camera can be used: the parallax map and three-dimensional coordinates are obtained through the camera, the ground interference area and the area outside the travel path are removed using OpenCV first; then the processed parallax map is binarized and segmented using OpenCV, and the obstacles are roughly extracted; finally, the convex hull of all obstacles is calculated and the area of the convex hull is calculated, and when the area is less than a certain threshold, it is not considered, and the convex hull coordinates of the obstacle are finally output. After obtaining the obstacle coordinates, it can be judged whether the preset path corresponding to the matching grabbing point exists.

[0098] In an embodiment of the present application, when there are multiple preset paths corresponding to the matching grabbing point, it can be determined whether one or more of the multiple preset paths exist obstacles.

[0099] In step B2, the preset path without obstacles is taken as the grabbing path.

[0100] In an example, the preset path without obstacles in the multiple preset paths is taken as the grabbing path. For example, according to the set of grabbable points in the preset grabbable range, a first coordinate point is determined from the multiple coordinate points of the grabbing target in the set of grabbable points, the first coordinate point is taken as the matching grabbing point, the matching grabbing point corresponds to four preset paths, at this time, it can be determined whether any one of the four preset paths exists obstacles, if there are obstacles, another preset path in the four preset paths can be selected to determine whether there are obstacles, and thus the preset path without obstacles is taken as the grabbing path.

[0101] In an embodiment of the present application, there is also a case that each preset path of the matching grabbing point exists obstacles, at this time, the method can further include step B3:

[0102] In step B3, if the preset paths corresponding to the matching grabbing point all exist obstacles, the matching grabbing point of the grabbing target is re-determined.

[0103] For example, the first coordinate point is determined as the matching grabbing point from the multiple coordinate points of the grabbing target according to the set of grabbable points in the preset grabbable range, but the four preset paths of the matching grabbing point all exist obstacles. At this time, the second coordinate point can be determined as the matching grabbing point from the multiple coordinate points of the grabbing target according to the set of grabbable points in the preset grabbable range again, and the path without obstacles is selected as the grabbing path in the preset path corresponding to the second coordinate point.

[0104] In an embodiment of the present application, when determining the matching grabbing point, the matching grabbing point closest to the robot can be selected from the coordinate points of the grabbing target, so that the motion stroke of the robot arm is shorter and the grabbing task is completed faster.

[0105] In an embodiment of the present application, based on the above embodiment, the step 206 of moving the robot arm according to the grabbing path to grab the grabbing target can further include steps C1-C3.

[0106] In step C1, the robot arm grabbing starting point of the grabbing path is determined.

[0107] In step C2, the posture of the robot is fine-tuned so that the robot arm end is located at the robot arm grabbing starting point.

[0108] In step C3, the robot arm is moved according to the grabbing path to grab the grabbing target.

[0109] In an embodiment of the present application, since the generated preset path is from the robot arm grabbing starting point to the grabbable point, in addition to the matching of the grabbing point, the robot arm grabbing starting point also needs to be matched. For example, when the grabbing starting point of the preset path is that the robot is upright and the robot arm is in the initial state, the posture of the robot is fine-tuned so that the robot arm is in the initial state. For another example, if the grabbing starting point of the preset path is the position of the robot arm end after the robot slightly crouches, bends over, and turns around, the posture of the robot needs to be fine-tuned in this embodiment, including slightly crouching, bending over, and rotating the body, so that the robot arm end is at the robot arm grabbing starting point of the grabbing path. When the robot is a robot with a movable base, fine-tuning the posture of the robot can include rotating, advancing, and / or retreating, etc.

[0110] In an embodiment of the present application, the step 203 of determining that the grabbing target is in the preset grabbable range can include steps D1-D4.

[0111] In step D1, whether the grabbing target is in the preset grabbable range is determined according to the depth information of the grabbing target or the distance from the grabbing target; wherein the environment image includes the depth information.

[0112] The depth information of the grabbing target can be obtained from the environment image captured by the robot. As described above, the environment information captured by the RGBD depth camera and the binocular camera equipped by the robot can include the distance from each point in the image to the plane where the camera is located. In addition, the distance between the robot and the grabbing target can also be obtained by other distance measurement methods, such as using infrared rays to measure the distance.

[0113] In step D2, if the grasping target is not within the preset graspable range, the robot body is moved to place the grasping target within the preset graspable range.

[0114] In this embodiment, when the grasping target is too far away, the robot body needs to be moved so that the grasping target is within the preset graspable range before the matching of the grasping point is performed. In this way, it can be ensured that the graspable point can be matched.

[0115] Figure 3 is a flowchart of a grasping method according to an exemplary embodiment. The execution subject of the method can be a robot, which is equipped with a camera that can measure depth information (RGBD or binocular camera) and an IMU.

[0116] First, the robot needs to perform preliminary work on the objects to be grasped: determine the preset graspable range, obtain the graspable point set within the preset graspable range, obtain the preset path of each graspable point in the preset graspable point set, and train a deep learning model for identifying the grasping target.

[0117] Determine the preset graspable range, obtain the graspable point set within the preset graspable range, and obtain the preset path of each graspable point in the preset graspable point set. In this application, since three-dimensional robot path planning is very power-consuming, a pre-planned path is adopted, that is, random sampling of graspable three-dimensional space within the preset graspable range of the robot, then path planning, and generation of the corresponding list. Each grasping point corresponds to 4 different paths. In application, a table lookup form can be used to obtain the path more quickly. Deep learning model training.

[0118] The image of the graspable object is trained through the deep learning network. Through this model, a specific grasping target can be found in the image, and the pixel coordinate point of the grasping target in the image (the coordinate point is the grasping point) can be obtained.

[0119] Next, the robot can perform the grasping operation. Figure 3 The grasping method shown includes the following steps:

[0120] Step 301: Obtain a grasping instruction, which includes a grasping target.

[0121] Step 302: Take an environmental image and identify the grasping target.

[0122] Step 303: Measure the distance between the robot and the grasping target through the ranging camera.

[0123] Step 304: Determine whether the distance exceeds the preset graspable range. If yes, go to step 305; if no, go to step 306.

[0124] Step 305: After moving a preset distance towards the grabbing target, step 306 is performed.

[0125] Step 306: According to the set of graspable points within the preset graspable range, a matching graspable point is determined from the coordinate point of the grabbing target.

[0126] Step 307: According to the preset path of each graspable point in the set of preset graspable points, the preset path of the matching graspable point is obtained as the grabbing path.

[0127] Step 308: Check if the grabbing path will collide with obstacles. If yes, go to step 306; if no, go to step 309.

[0128] Step 309: Fine-tune the robot form to make the end of the robotic arm at the robotic arm grabbing starting point of the grabbing path.

[0129] Fine-tune the humanoid robot (e.g., slightly squat, rotate the body, etc.) to make the starting point of the robotic arm grabbing algorithm at the starting point of the grabbing path.

[0130] Step 310: According to the grabbing path, move the robotic arm to grab the grabbing target.

[0131] The following is an embodiment of the device of the present disclosure, which can be used to execute the method embodiments of the present disclosure.

[0132] Figure 4 is a structural schematic diagram of a grabbing device according to an exemplary embodiment; the device can be implemented in various ways, for example, implementing all components of the device in a robot, or implementing components in the device in a coupled manner on the side of the robot; the device can implement the above-mentioned methods related to the present disclosure through software, hardware, or a combination of both, such as Figure 4 As shown, the grabbing device includes:

[0133] The first acquisition module 401 is configured to, in response to receiving a grabbing instruction, instruct the camera module of the robot to capture an environment image of the surrounding environment, and the grabbing instruction includes a grabbing target.

[0134] The first determination module 402 is configured to identify the environment image and determine that the grabbing target exists in the environment image.

[0135] The second determination module 403 is configured to determine that the grabbing target is within a preset graspable range.

[0136] The third determination module 404 is configured to determine a matching graspable point from the coordinate point of the grabbing target according to a set of graspable points within the preset graspable range.

[0137] The fourth determination module 405 is configured to determine, according to a preset path of each of the preset set of graspable points, the preset path corresponding to the matched graspable point as the grasp path.

[0138] The grasping module 406 is configured to move the robot arm to grasp the grasp target according to the grasp path.

[0139] The apparatus provided by the embodiments of the present disclosure can be used for executing Figure 2 The technical solutions of the illustrated embodiments have similar implementation manners and beneficial effects, which will not be repeated here.

[0140] In a possible implementation, the apparatus further includes:

[0141] The second acquisition module is configured to acquire a set of graspable points by performing random sampling in a graspable three-dimensional space within a graspable range of the robot.

[0142] The generation module is configured to generate, for each of the set of graspable points, at least one path from a robot arm grasping starting point to the graspable point as a preset path of the graspable point.

[0143] The storage module is configured to store each graspable point and the corresponding at least one preset path to generate a corresponding relationship.

[0144] In a possible implementation, the grasping module 406 includes:

[0145] The determination unit is configured to determine a robot arm grasping starting point of the grasp path.

[0146] The adjustment unit is configured to finely adjust the form of the robot so that the end of the robot arm is located at the robot arm grasping starting point.

[0147] The grasping unit is configured to move the robot arm to grasp the grasp target according to the grasp path.

[0148] In a possible implementation, the fourth determination module 405 is configured to:

[0149] Determine whether the preset path corresponding to the matched graspable point has an obstacle.

[0150] Determine the preset path without an obstacle as the grasp path.

[0151] In a possible implementation, the third determination module 404 is configured to re-determine the matched graspable point of the grasp target if the preset paths corresponding to the matched graspable points all have obstacles.

[0152] In a possible implementation, the second determination module 403 is configured to:

[0153] determine whether the grasping target is within a preset graspable range according to the depth information of the grasping target or the distance from the grasping target; wherein the environment image comprises depth information;

[0154] If the grasping target is not within the preset graspable range, the robot body is moved to place the grasping target within the preset graspable range.

[0155] In a possible implementation, the first determination module 402 is configured to:

[0156] According to the depth learning model, the environment image is identified to determine that the grasping target exists in the environment image; wherein the depth learning model is obtained by pre-training the pictures of the grasping target through the depth learning network.

[0157] Figure 5 is a block diagram of a grasping device 50 according to an exemplary embodiment. The grasping device 50 can be implemented in various ways, for example, all components of the device are implemented in a robot, or components of the device are implemented in a coupled manner on the side of the robot; see Figure 5 The grasping device 50 comprises:

[0158] a processor 501;

[0159] a memory 502 for storing processor-executable instructions;

[0160] The processor 501 is configured to:

[0161] in response to receiving a grasping instruction, instructing the camera module of the robot to capture the surrounding environment to obtain an environment image, wherein the grasping instruction comprises a grasping target;

[0162] identifying the environment image to determine that the grasping target exists in the environment image;

[0163] determine that the grasping target is within a preset graspable range;

[0164] According to a set of graspable points within the preset graspable range, determine a matching grasping point from the coordinate point of the grasping target;

[0165] According to a preset path of each graspable point in the set of preset graspable points, determine the preset path corresponding to the matching grasping point as a grasping path;

[0166] According to the grasping path, move the mechanical arm to grasp the grasping target.

[0167] With regard to the apparatus in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be described in detail here.

[0168] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a robot or a processor of the robot, enable the robot to perform a grasping method, the method comprising:

[0169] In response to receiving a grasping instruction, instructing a camera module of the robot to take an environment image of the surrounding environment, the grasping instruction including a grasping target;

[0170] Identifying the environment image to determine that the grasping target exists in the environment image;

[0171] Determining that the grasping target is within a preset graspable range;

[0172] From a set of graspable points within the preset graspable range, determining a matching graspable point from the coordinate point of the grasping target;

[0173] According to the preset path of each graspable point in the preset set of graspable points, determining the preset path corresponding to the matching graspable point as a grasping path;

[0174] According to the grasping path, moving a mechanical arm to grasp the grasping target.

[0175] The present application also proposes a robot, comprising at least a grasping device, the grasping device comprising:

[0176] A first acquisition module for, in response to receiving a grasping instruction, instructing a camera module of the robot to take an environment image of the surrounding environment, the grasping instruction including a grasping target;

[0177] A first determination module for identifying the environment image to determine that the grasping target exists in the environment image;

[0178] A second determination module for determining that the grasping target is within a preset graspable range;

[0179] A third determination module for, from a set of graspable points within the preset graspable range, determining a matching graspable point from the coordinate point of the grasping target;

[0180] A fourth determination module for, according to the preset path of each graspable point in the preset set of graspable points, determining the preset path corresponding to the matching graspable point as a grasping path;

[0181] A grasping module for, according to the grasping path, moving a mechanical arm to grasp the grasping target.

[0182] The grabbing device further comprises:

[0183] The second acquisition module is configured to acquire a set of grabbable points by randomly sampling a preset number of points in a grabbable three-dimensional space within a robot-grabbing range;

[0184] The generation module is configured to generate, for each of the set of grabbable points, at least one path from a robot-grabbing starting point to the grabbable point as a preset path of the grabbable point;

[0185] The storage module is configured to store each of the grabbable points in association with the corresponding at least one preset path to generate a corresponding relationship.

[0186] In a possible implementation, the grabbing module comprises:

[0187] The determination unit is configured to determine a robot-grabbing starting point of the grabbing path;

[0188] The adjustment unit is configured to fine-tune the form of the robot so that the end of the robot arm is located at the robot-grabbing starting point;

[0189] The grabbing unit is configured to move the robot arm according to the grabbing path to grab the grabbing target.

[0190] In a possible implementation, the fourth determination module is configured to:

[0191] Determine whether the preset path corresponding to the matching grabbable point has an obstacle;

[0192] Take the preset path without the obstacle as the grabbing path.

[0193] In a possible implementation, the third determination module is configured to, if the preset paths corresponding to the matching grabbable points all have obstacles, re-determine the matching grabbable point of the grabbing target.

[0194] In a possible implementation, the second determination module is configured to:

[0195] Determine, according to the depth information of the grabbing target or the distance from the grabbing target, whether the grabbing target is within a preset grabbable range; wherein the environment image comprises depth information;

[0196] If the grabbing target is not within the preset grabbable range, move the robot body so that the grabbing target is within the preset grabbable range.

[0197] In a possible implementation, the first determination module is configured to:

[0198] The environment image is recognized according to a deep learning model to determine that a grabbing target exists in the environment image; wherein the deep learning model is obtained by pre-training pictures of the grabbing target through a deep learning network.

[0199] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.

[0200] It should be understood that the present disclosure is not limited to the precise structures as herein described and illustrated in the drawings, and that various modifications and changes can be made without departing from its scope. The scope of the present disclosure is limited only by the claims that follow.

Claims

1. A method of grasping, characterized by, The method is applied to the grabbing of a walkable robot in a non-fixed workspace, and the method comprises the following steps: In response to receiving a grabbing instruction, instructing a camera module of the robot to capture an environment image of the surrounding environment, the grabbing instruction comprising a grabbing target; Identifying the environment image to determine that the grabbing target exists in the environment image; Determining that the grabbing target is within a preset grabbable range; According to a set of grabbable points in the preset grabbable range, determining a matching grabbing point from the coordinate point of the grabbing target; According to a preset path of each grabbable point in the set of preset grabbable points, determining the preset path corresponding to the matching grabbing point as a grabbing path; Moving a mechanical arm according to the grabbing path to grab the grabbing target; The method further comprises the following steps: Obtaining a set of grabbable points by randomly sampling a preset number of points in a grabbable three-dimensional space within the grabbable range of the robot; For each grabbable point in the set of grabbable points, generating at least one path from the grabbing starting point of the mechanical arm to the grabbable point as the preset path of the grabbable point; Storing each grabbable point and the corresponding at least one preset path to generate a corresponding relationship.

2. The method of claim 1, wherein, The step of moving the mechanical arm according to the grabbing path to grab the grabbing target comprises the following steps: Determining the mechanical arm grabbing starting point of the grabbing path; Fine-tuning the form of the robot to make the end of the mechanical arm located at the mechanical arm grabbing starting point; Moving the mechanical arm according to the grabbing path to grab the grabbing target.

3. The method of claim 1, wherein, The step of determining the preset path corresponding to the matching grabbing point as the grabbing path comprises the following steps: Determining whether the preset path corresponding to the matching grabbing point exists obstacles; Taking the preset path without obstacles as the grabbing path.

4. The method of claim 3, wherein, The method further comprises the following steps: If the preset paths corresponding to the matching grabbing points all exist obstacles, re-determining the matching grabbing point of the grabbing target.

5. The method of claim 1, wherein, The step of determining that the grabbing target is within a preset grabbable range comprises the following steps: According to the depth information of the grabbing target or the distance from the grabbing target, determining whether the grabbing target is within a preset grabbable range; wherein the environment image comprises depth information; If the grabbing target is not within the preset grabbable range, moving the robot body to make the grabbing target within the preset grabbable range.

6. The method of claim 1, wherein, The step of identifying the environment image to determine that the grabbing target exists in the environment image comprises the following steps: According to a deep learning model, identifying the environment image to determine that the grabbing target exists in the environment image; wherein the deep learning model is obtained by pre-training a deep learning network on pictures of the grabbing target.

7. A gripping device, characterized in that The device is applied to the grabbing of a walkable robot in a non-fixed workspace, and the device comprises the following steps: A first obtaining module is configured to, in response to receiving a grabbing instruction, instruct a camera module of the robot to capture an environment image of the surrounding environment, the grabbing instruction comprising a grabbing target; A first determining module is configured to identify the environment image to determine that the grabbing target exists in the environment image; A second determining module is configured to determine that the grabbing target is within a preset grabbable range; a third determining module, configured to determine a matching grasp point from the coordinate points of the grasp target according to a set of grasp points in a preset graspable range; a fourth determining module, configured to determine a preset path corresponding to the matching grasp point as a grasp path according to a preset path of each grasp point in the set of grasp points; a grasping module, configured to move the robot arm to grasp the grasp target according to the grasp path; The device further comprises: a second obtaining module, configured to obtain a set of grasp points by randomly sampling a preset number of points in a graspable three-dimensional space within the graspable range of the robot; a generating module, configured to generate at least one path from a robot arm grasping starting point to each grasp point in the set of grasp points as a preset path of the grasp point; a storage module, configured to store each grasp point and the corresponding at least one preset path to generate a corresponding relationship.

8. The apparatus of claim 7, wherein, The grasping module comprises: a determining unit, configured to determine a robot arm grasping starting point of the grasp path; an adjusting unit, configured to finely adjust the shape of the robot so that the end of the robot arm is located at the robot arm grasping starting point; a grasping unit, configured to move the robot arm to grasp the grasp target according to the grasp path.

9. A gripping device, characterized in that comprises: a processor; a memory for storing processor-executable instructions, wherein the instructions, when executed by the processor, implement the steps of the method of any one of claims 1-6.

10. A computer readable storage medium having stored thereon computer instructions, wherein, The instructions, when executed by the processor, implement the steps of the method of any one of claims 1-6.

11. A robot, characterized in that comprises the grasping device of any one of claims 7-8.

Images