Vision-guided grasping method and grasping device

Abstract

A kind of visual guidance-based grabbing method for grabbing second object on first object, it includes the following steps: photographing the second object on the first object to obtain point cloud image;The point cloud image is similarity matched with standard image, to judge whether the point cloud image is in line with standard;When the point cloud image is in line with standard, the position matching of the point cloud image and the standard image is carried out;The point cloud image after position matching is parsed, to obtain the second object on the point cloud image on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object on the second object

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machining, in particular to a grabbing method and device based on visual guidance. BACKGROUND

[0002] For most modern industrial automatic production lines, such as cleaning lines, a first object, such as a logistics vehicle, is generally positioned manually, a plurality of second objects, such as material frames, are regularly placed in the first object, a plurality of workpieces are placed in each material frame, and then a grabbing device is used to grab the second objects in the first object and place them on a cleaning machine so that the cleaning machine can clean the material frames and the workpieces in the material frames.

[0003] However, the first object placement point is generally a fixed area. Since manual positioning of the first object has deviations, and the second objects have a certain amount of deformation after being used for a long time, and the grabbing point of the grabbing device is generally fixed, this makes it difficult for the grabbing device to grab the second objects and increases the risk of the second objects falling during the grabbing process. SUMMARY

[0004] In view of the above, it is necessary to provide a grabbing method and device based on visual guidance to solve the technical problems that the existing grabbing device cannot accurately grab the second objects and the second objects have a risk of falling.

[0005] The embodiment of the present application provides a grabbing method based on visual guidance, which is used for grabbing a second object on a first object and includes the following steps: photographing the second object on the first object to obtain a point cloud image; performing similarity matching on the point cloud image and a standard image to determine whether the point cloud image meets a standard; when the point cloud image meets the standard, performing position matching on the point cloud image and the standard image; analyzing the position-matched point cloud image to obtain a grabbing point on the point cloud image; and moving and carrying a device to grab the second object according to the grabbing point.

[0006] The grabbing method described above first obtains a point cloud image of a second object on a first object by photographing, then performs similarity matching on the point cloud image and a standard image to determine whether the point cloud image meets a standard, and when the point cloud image meets the standard, performs position matching on the point cloud image and the standard image; then analyzes the position-matched point cloud image to obtain a grabbing point of the second object on the point cloud image; and finally makes a moving and carrying device grab the second object according to the grabbing point, which can accurately obtain the grabbing point of the second object, so that the moving and carrying device can accurately grab the second object, avoid the risk of the second object falling during the grabbing process, and has high grabbing efficiency.

[0007] In some embodiments, the step of capturing the second object on the first object to obtain a first point cloud image comprises: fixing a visual camera at a preset height; moving the first object with the second object to a field of view of the visual camera; and capturing the first object by the visual camera to obtain the point cloud image, wherein the point cloud image comprises a point cloud image of the second object on the first object.

[0008] In some embodiments, before the step of position matching the point cloud image with the standard image, the method further comprises: performing noise reduction processing on the standard-compliant point cloud image to remove abnormal images in the point cloud image; and performing segmentation extraction on the point cloud image after removing the abnormal images to obtain images corresponding to a plurality of second objects of each layer.

[0009] In some embodiments, before the step of analyzing the position-matched point cloud image, the method further comprises: performing interference processing on the position-matched point cloud image to retain the second object on the uppermost layer.

[0010] In some embodiments, before the step of capturing the second object on the first object, the method further comprises: calibrating positions of the visual camera and the transfer device to enable the transfer device to accurately grasp the second object at the grasping point.

[0011] In some embodiments, the transfer device comprises a mechanical arm, a flange, and a calibration plate, the mechanical arm comprises a fixed end on one side of the visual camera and a driving end, the driving end is connected with the flange, the flange is connected with the calibration plate, the step of calibrating the positions of the visual camera and the transfer device comprises: establishing a fixed coordinate system with the fixed end as the center, establishing a flange coordinate system with the center of the flange as the center, establishing a calibration coordinate system with the center of the calibration plate as the center, and establishing a camera coordinate system with the center of the visual camera as the center; moving the flange and the calibration plate in different poses in a field of view of the visual camera by the driving end; capturing the calibration plate in a corresponding pose by the visual camera to obtain a calibration point cloud image; obtaining a first center coordinate of the calibration plate after rotation and translation conversion of the camera coordinate system and the fixed coordinate system, and obtaining a second center coordinate of the calibration plate after rotation and translation conversion of the flange coordinate system and the fixed coordinate system according to the calibration point cloud images in different poses; and determining whether the calibration is completed according to whether a difference between the first center coordinate and the second center coordinate is within a preset range.

[0012] In some embodiments, the first center coordinate satisfies the following relationship: Pb=X*B*Pg; wherein Pb is the first center coordinate, Pg is the coordinate of the calibration plate in the calibration coordinate system, X is the coordinate relationship between the center of the vision camera and the fixed end in the calibration coordinate system, and B is the coordinate relationship between the center of the calibration plate and the center of the vision camera in the calibration coordinate system; the second center coordinate satisfies the following relationship: Pb'=A*C*Pg; wherein Pb' is the second center coordinate, A is the coordinate relationship between the center of the flange and the fixed end in the calibration coordinate system, and C is the coordinate relationship between the center of the calibration plate and the center of the flange in the calibration coordinate system.

[0013] In some embodiments, the calibration plate is provided with a plurality of regularly arranged calibration holes, and the steps of obtaining the first center coordinate of the calibration plate after rotation and translation conversion by the camera coordinate system and the fixed coordinate system, and obtaining the second center coordinate of the calibration plate after rotation and translation conversion by the flange coordinate system and the fixed coordinate system according to the calibration point cloud images in different poses include: obtaining the center coordinates of each calibration hole on the calibration plate in the calibration coordinate system according to the calibration point cloud images in different poses; obtaining the center coordinates of the calibration plate in the calibration coordinate system according to the center coordinates of each calibration hole; and performing rotation and translation conversion on the center coordinates of the calibration plate in the calibration coordinate system by the camera coordinate system and the fixed coordinate system to obtain the first center coordinate, and performing rotation and translation conversion by the flange coordinate system and the fixed coordinate system to obtain the second center coordinate.

[0014] In some embodiments, when the calibration is completed, the method further comprises: disassembling the calibration plate connected to the flange, and installing a clamping jaw for clamping the grasping point to the flange.

[0015] Embodiments of the present application also propose a vision-guided grasping device for grasping a second object on a first object, comprising a transfer device, a vision camera, and a control device; the transfer device is used for grasping the second object; the vision camera is used for photographing the second object on the first object to obtain a point cloud image; the control device is electrically connected with the vision camera and the transfer device respectively, and is used for: performing similarity matching on the point cloud image and a standard image to determine whether the point cloud image meets the standard; when the point cloud image meets the standard, performing position matching on the point cloud image and the standard image; analyzing the position-matched point cloud image to obtain a grasping point on the first object on the point cloud image; and making the transfer device grasp the second object according to the grasping point.

[0016] The grabbing device of the application first obtains the second object on the first object through a visual camera to obtain a point cloud image, and then controls the device to match the point cloud image with a standard image for similarity to determine whether the point cloud image meets the standard. When the point cloud image meets the standard, the point cloud image and the standard image are positionally matched. Then the positionally matched point cloud image is analyzed to obtain a grabbing point on the point cloud image. Finally, the grabbing device controls the transfer device to grab the second object on the first object according to the grabbing point, which can accurately obtain the grabbing point on the second object, so that the transfer device accurately grabs the second object, avoids the risk of falling during grabbing, and has high grabbing efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 is a schematic diagram of the three-dimensional structure of a grabbing device according to an embodiment of the application.

[0018] Figure 2 is Figure 1 is a schematic diagram of the three-dimensional structure of a grabbing device according to an embodiment of the application.

[0019] Figure 3 is a flowchart of a grabbing method based on visual guidance according to an embodiment of the application.

[0020] Figure 4 is Figure 3 is a flowchart of obtaining a first point cloud image by shooting a second object on a first object in

[0021] Figure 5 is Figure 4 is a schematic diagram of a point cloud image obtained by shooting all second objects on a first object by a visual camera in

[0022] Figure 6 is Figure 3 is a schematic diagram of a point cloud image after similarity matching in

[0023] Figure 7 is Figure 3 is a schematic diagram of a point cloud image meeting the standard after position matching in

[0024] Figure 8 is Figure 3 is a method flowchart before position matching of a point cloud image and a standard image in

[0025] Figure 9 is Figure 8 is a schematic diagram of a point cloud image after removing abnormal images in

[0026] Figure 10 is Figure 3 is a schematic diagram of a positionally matched point cloud image after interference processing in

[0027] Figure 11 is Figure 3 the flow chart before the step of acquiring the second object on the first object in

[0028] Figure 12 is Figure 11 the flow chart of acquiring the first center coordinate and the second center coordinate in

[0029] Main element symbol explanation

[0030] Grabbing device 1

[0031] Transfer device 10

[0032] Mechanical arm 11

[0033] Fixed end 111

[0034] Driving end 112

[0035] Flange 12

[0036] Calibration plate 13

[0037] Calibration hole 131

[0038] Clamping jaw 14

[0039] Visual camera 20

[0040] Control device 30

[0041] First object 200

[0042] Second object 300 DETAILED DESCRIPTION

[0043] Embodiments of the present application are described below in detail, examples of which are shown in the accompanying drawings, wherein the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary only, and are used only for the purpose of explaining the present application, and cannot be understood as limiting the present application.

[0044] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "connecting", "connection" should be understood in a broad sense, for example, can be fixedly connected, can be detachably connected, or integrally connected; can be mechanically connected, or electrically connected or can communicate with each other; can be directly connected, or indirectly connected through an intermediate medium, can be the internal communication of two elements or the interaction relationship between two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0045] The following disclosure provides many different embodiments, or examples, for implementing different structures of the present application. For the purpose of simplifying the present application, the components and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present application. In addition, the present application can repeatedly refer to numbers and / or letters in different examples, and such repetition is for the purpose of simplification and clarity, which does not indicate the relationship between the various embodiments and / or settings discussed. In addition, the present application provides examples of various specific processes and materials, but those of ordinary skill in the art can realize the application of other processes and / or the use of other materials.

[0046] See Figure 1 , the embodiment of the present application provides a visual guidance based grabbing device 1 for grabbing a second object 300 on a first object 200. In the embodiment, the first object 200 is a logistics vehicle, and the second object 300 is a frame. The logistics vehicle is placed with multiple layers of frames, each layer is placed with multiple frames, and each frame is placed with multiple workpieces. The grabbing device 1 comprises a transfer device 10, a visual camera 20 and a control device 30.

[0047] The transfer device 10 is used to grab the second object 300, and the transfer device 10 can be a mechanical hand. See Figure 2 , the transfer device 10 comprises a mechanical arm 11, a flange 12, a calibration plate 13 and a gripper 14. The mechanical arm 11 comprises a fixed end 111 on one side of the visual camera 20 and a driving end 112. The driving end 112 is connected with the flange 12. The flange 12 can be selectively connected with the calibration plate 13 and the gripper 14. When the flange 12 is connected with the calibration plate 13, the position of the transfer device 10 and the visual camera 20 can be calibrated by shooting the calibration plate 13 through the visual camera 20. After calibration, the calibration plate 13 on the flange 12 is disassembled and the gripper 14 is installed, so that the gripper 14 can accurately grab the second object 300. In the embodiment, the mechanical arm 11 is a six-axis mechanical arm.

[0048] The visual camera 20 is used to shoot the second object 300 on the first object 200 to obtain a point cloud image. The visual camera 20 can be a depth camera.

[0049] The control device 30 is electrically connected with the visual camera 20 and the transfer device 10 respectively. The control device 30 can be a microcomputer, a computer and the like, but is not limited thereto. The control device 30 is used for: similarity matching the point cloud image with a standard image to determine whether the point cloud image conforms to the standard; position matching the point cloud image with the standard image when the point cloud image conforms to the standard; analyzing the position-matched point cloud image to obtain a grabbing point position on the first object 200 on the point cloud image; and making the transfer device 10 grab the second object 300 according to the grabbing point position.

[0050] The grabbing device 1 of the embodiment of the application first captures the second object 300 on the first object 200 by the visual camera 20 to obtain a point cloud image, then the control device 30 performs similarity matching on the point cloud image and a standard image to determine whether the point cloud image conforms to the standard, and when the point cloud image conforms to the standard, the control device 30 performs position matching on the point cloud image and the standard image; then the point cloud image after the position matching is analyzed to obtain a grabbing point position on the point cloud image; finally, the control device 30 controls the transfer device 10 to grab the second object 300 on the first object 200 according to the grabbing point position, so that the transfer device 10 can accurately grab the second object 300, and the risk of falling during the grabbing process is avoided, and the grabbing efficiency is high.

[0051] Please refer to Figure 3 The embodiment of the application also provides a grabbing method based on visual guidance, which is used for grabbing the second object 300 on the first object 200. Specifically, the method comprises the following steps:

[0052] S110, capturing the second object 300 on the first object 200 to obtain a point cloud image.

[0053] Please refer to Figure 4 capturing the second object 300 on the first object 200 to obtain a first point cloud image, and the specific steps comprise:

[0054] S111, fixing the visual camera 20 at a preset height.

[0055] The preset height can be set according to actual needs, as long as it can capture all the second objects 300 on the first object 200.

[0056] S112, moving the first object 200 provided with the second object 300 to the visual field range of the visual camera 20.

[0057] The size of the first object 200 is 1200mm*1100mm*1000mm or 1200mm*1000mm*950mm. The depth of field of the visual camera 20 is 1.5m, the visual field range of the near field of the visual camera 20 is 1.5m*1.2m, the visual field range of the far field of the visual camera 20 is 3m*2.4m, and the height of the visual camera 20 from the ground is 3m. In this way, the height of the visual camera 20 from the top of the first object 200 is greater than 1.5m, and through the above setting, the first object 200 can be in the visual field range of the visual camera 20.

[0058] S113, making the visual camera 20 capture the first object 200 to obtain a point cloud image, wherein the point cloud image comprises the point cloud images of all the second objects 300 on the first object 200.

[0059] Wherein, the visual camera 20 shoots all the second articles 300 on the first article 200, and obtains a point cloud image as shown in Figure 5 The second article 300 is four layers, and each layer has six second articles 300 arranged in an array. The visual camera 20 can shoot the point cloud image of the four layers of second articles 300. It can be understood that in other embodiments, the second articles 300 on the first article 200 are three layers or six layers, but not limited thereto.

[0060] S120, the point cloud image is matched with the standard image in similarity to determine whether the point cloud image meets the standard.

[0061] Wherein, the standard image is a template image of the second article 300 meeting the standard.

[0062] Wherein, the similarity matching is to compare each frame of pixels in the point cloud image and the standard image to obtain a similarity value. In this embodiment, the similarity threshold is set to be greater than 0.85. When the similarity of the point cloud image and the standard image is within the threshold range, it means that the shot point cloud image meets the standard. For example, the similarity value after comparison of the second article 300 in Figure 6 It can be understood that in other embodiments, the similarity threshold can be set to be greater than 0.4, but not limited thereto. When the similarity of the point cloud image and the standard image is not within the threshold range, it means that the shot point cloud image does not meet the standard. At this time, the position of the visual camera 20 needs to be adjusted to be re-shot until it meets the standard.

[0063] S130, when the point cloud image meets the standard, the position matching of the point cloud image and the standard image is performed.

[0064] Wherein, the position matching can obtain a point cloud image as shown in Figure 7 The position matching is to match the size of each layer of second articles 300 in the point cloud image with the size of each layer of second articles 300 in the standard image, so as to prepare for the calculation of the grabbing point position on the second article 300 in the matched point cloud image.

[0065] S140, the position-matched point cloud image is analyzed to obtain the grabbing point position on the second article 300 in the point cloud image.

[0066] Wherein, after the position matching of each second article 300, the grabbing point position can be selected according to the size of the second article 300. For example, the grabbing point position can be set as the midpoint of one edge of the second article 300, wherein the edge is the edge close to the transfer device 10.

[0067] S150, the transfer device 10 grabs the second article 300 according to the grabbing point position.

[0068] Wherein, since the second objects 300 of each layer are multiple, the grabbing points are also multiple, so that the transfer device 10 can grab the grabbing points in sequence.

[0069] Wherein, after the second objects 300 of each layer are grabbed, a lifting mechanism (such as a telescopic rod) can be used to control the visual camera 20 to move downward by the height of the second objects 300 of each layer, so as to ensure the image quality of the visual camera 20.

[0070] In some embodiments, before the position matching of the point cloud image and the standard image, please refer to Figure 8 , the method further comprises:

[0071] S122, the point cloud image meeting the standard is subjected to noise reduction processing to remove abnormal images in the point cloud image;

[0072] S124, the point cloud image after removing the abnormal images is subjected to segmentation extraction to obtain the images corresponding to the multiple second objects of each layer.

[0073] Wherein, please refer to Figure 9 , the point cloud image includes the image of the first object 200 and the image of the second object 300, when the point cloud image meets the standard, in order to more accurately match, it is necessary to remove the abnormal images in the point cloud image, that is, the image of the first object 200 in the point cloud image, so as to obtain the image shown in Figure 7 .

[0074] In some embodiments, before the position matching of the point cloud image and the standard image, please refer to

[0075] The position-matched point cloud image is subjected to interference processing to retain the second object of the uppermost layer.

[0076] Wherein, the purpose of the interference processing is to retain the image of the second object 300 of the uppermost layer, so as to obtain the image as shown in Figure 10 . In this way, the size of the second object in the point cloud image can be accurately calculated, so as to obtain the grabbing point of the second object more accurately.

[0077] In some embodiments, before the step of shooting the second object 300 on the first object 200, the method further comprises:

[0078] The positions of the visual camera 20 and the transfer device 10 are calibrated, so that the transfer device 10 can accurately grab the second object 300 of the grabbing point.

[0079] Wherein, please refer to Figure 11 , the step of calibrating the positions of the visual camera 20 and the transfer device 10 comprises:

[0080] S10, establishing a fixed coordinate system O with the fixed end 111 as the axis b S10, establishing a fixed coordinate system O with the fixed end 111 as the axis f S10, establishing a fixed coordinate system O with the fixed end 111 as the axis g S10, establishing a fixed coordinate system O with the fixed end 111 as the axis c .

[0081] S20, moving the flange 12 and the calibration board 13 in different poses in the field of view of the visual camera 20 by driving the driving end 112.

[0082] S30, capturing the calibration point cloud image of the calibration board 13 in the corresponding pose by the visual camera 20.

[0083] S40, obtaining the first center coordinate of the calibration board 13 after rotation and translation conversion through the camera coordinate system O c and the fixed coordinate system O b and the second center coordinate of the calibration board 13 after rotation and translation conversion through the flange coordinate system O f and the fixed coordinate system O b .

[0084] S50, determining whether the calibration is completed according to whether the difference between the first center coordinate and the second center coordinate is within a preset range.

[0085] In some embodiments, the different poses are at least three, at least one of which is that the calibration board 13 is inclined upward, at least one of which is that the calibration board 13 is placed horizontally, and at least one of which is that the calibration board 13 is inclined downward.

[0086] In some embodiments, the difference between the first center coordinate and the second center coordinate meets the requirements, which can improve the stability of the second object 300.

[0087] In some embodiments, the first center coordinate satisfies the following relationship:

[0088] Pb=X*BPg;

[0089] wherein Pb is the first center coordinate, Pg is the coordinate of the calibration board 13 in the calibration coordinate system O g , X is the coordinate relationship between the center of the visual camera 20 and the fixed end 111 in the calibration coordinate system O g , and B is the coordinate relationship between the center of the calibration board 13 and the center of the visual camera 20 in the calibration coordinate system O gthe coordinate relationship between the center of the vision camera 20 and the fixed end 111 is a constant value, and the coordinate relationship between the center coordinate of the calibration board 13 and the vision camera 20 can be obtained by capturing the calibration point cloud image of the calibration board 13 at the corresponding pose by the vision camera 20 and calculating.

[0090] The second center coordinate satisfies the following relationship:

[0091] Pb' = A * C * Pg;

[0092] wherein Pb' is the second center coordinate, A is the coordinate relationship between the center of the flange 12 and the fixed end 111 in the calibration coordinate system O g , C is the coordinate relationship between the center of the calibration board 13 and the center of the flange 12 in the calibration coordinate system O g ; wherein the relative position of the calibration board 13 and the flange 12 is fixed, so the coordinate relationship between the coordinate of the calibration board 13 in the calibration coordinate system O g and the flange 12 is a constant value; and the coordinate relationship between the flange 12 and the fixed end 111 can be obtained by capturing the calibration point cloud image of the flange 12 at the corresponding pose by the vision camera 20 and calculating.

[0093] In some embodiments, a plurality of calibration holes 131 are arranged on the calibration board 13 in a regular manner. Wherein, referring to Figure 12 , the first center coordinate and the second center coordinate are obtained according to the calibration point cloud images at different poses, which includes:

[0094] S41, obtaining the center coordinates of the calibration board 13 in the calibration coordinate system O g according to the center coordinates of each calibration hole 131;

[0095] S42, obtaining the center coordinates of the calibration board 13 in the calibration coordinate system O g according to the center coordinates of each calibration hole 131;

[0096] S43, rotating and translating the center coordinates of the calibration board 13 in the calibration coordinate system O g to the camera coordinate system O c and the fixed coordinate system O b to obtain the first center coordinate, and rotating and translating the center coordinates of the calibration board 13 in the calibration coordinate system O f to the flange coordinate system O b and the fixed coordinate system O g to obtain the second center coordinate.

[0097] wherein, in this embodiment, the calibration board 13 has three poses, and when the calibration board 13 is in the first pose, the center coordinates of the calibration board 13 in the calibration coordinate system O gthe center coordinates of the calibration plate 13 in the calibration coordinate system O g are obtained, and then the center coordinates of the calibration plate 13 in the calibration coordinate system O g are sequentially converted by the camera coordinate system O c and the fixed coordinate system O b to obtain the first center coordinates, and sequentially converted by the flange coordinate system O f and the fixed coordinate system O b to obtain the second center coordinates, and finally determine whether the difference between the first center coordinates and the second center coordinates meets the requirements.

[0098] Next, the determination of the second and third poses is performed by referring to the above steps until the difference between the first center coordinates and the second center coordinates of all poses meets the requirements. It can be understood that the coordinate values of the difference between the first center coordinates and the second center coordinates in the X, Y, and Z directions are all less than 5 mm.

[0099] In which, please refer to Table 1, Table 1 shows the coordinate values of the grabbing points of the 16 second objects 300 on the first object 200 in different positions.

[0100] Table 1

[0101] Number X (mm) Y (mm) Z (mm) 1 7 10 8 2 6 12 11 3 6 19 7 4 6 17 13 5 5 9 9 6 8 15 7 7 5 15 7 8 8 14 7 9 7 10 10 10 9 15 10 11 8 11 6 12 5 15 7 13 6 10 8 14 10 12 9 15 5 13 2 16 8 10 2

[0102] By grabbing the second objects 300 in different positions by the transfer device 10, the maximum positional difference of the second objects 300 when grabbed can be obtained as (5 mm, 10 mm, 11 mm).

[0103] Please refer to Table 2, Table 2 shows the coordinate values of the grabbing points of a single second object 300 on the first object 200 in different positions.

[0104] Table 2

[0105] Number X (mm) Y (mm) Z (mm) 1 5 19 12 2 6 20 8 3 7 19 10 4 6 20 7 5 6 19 14 6 5 20 7 7 6 20 12 8 6 18 8 9 6 21 12 10 6 20 10

[0106] By grabbing the single second objects 300 in different positions by the transfer device 10, the maximum positional difference of the second objects 300 when grabbed can be obtained as (2 mm, 3 mm, 7 mm).

[0107] In some embodiments, when the calibration is completed, the method further comprises:

[0108] Dismounting the calibration plate 13 connected to the flange 12, and installing the clamping jaw 14 for clamping the grabbing point to the flange 12.

[0109] The above grabbing method first obtains a point cloud image by shooting the second object 300 on the first object 200, then performs similarity matching on the point cloud image and a standard image to determine whether the point cloud image meets the standard, and when the point cloud image meets the standard, performs position matching on the point cloud image and the standard image; then analyzes the position-matched point cloud image to obtain a grabbing point position of the second object 300 on the point cloud image; finally, the transfer device 10 grabs the second object 300 according to the grabbing point position, which can accurately obtain the grabbing point position on the second object 300, so that the transfer device 10 accurately grabs the second object 300, avoids the risk of the second object 300 falling during the grabbing process, and has high grabbing efficiency.

[0110] It is apparent for a person skilled in the art that the application is not limited to the details of the above exemplary embodiments, but can be implemented in other concrete forms without departing from the spirit or essential characteristics of the application. Therefore, the embodiments should be considered as exemplary and non-limiting, and the scope of the application is defined by the appended claims and not by the above description, and all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be embraced in the application. Any reference signs in the claims should not be considered as limiting the claims involved. Furthermore, it is clear that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the application rather than limit the application, and although the application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the application.

Claims

1. A vision-guided grasping method for grasping a second object on a first object, characterized in that, The method comprises the following steps: calibrating positions of a vision camera and a transfer device, the transfer device comprising a mechanical arm, a flange and a calibration plate, the mechanical arm comprising a fixed end and a driving end on one side of the vision camera, the driving end being connected with the flange, the flange being connected with the calibration plate, the calibration of the positions of the vision camera and the transfer device comprising: establishing a fixed coordinate system with the fixed end as the axis, establishing a flange coordinate system with the center of the flange as the axis, and establishing a calibration coordinate system with the center of the calibration plate as the axis; establishing a camera coordinate system with the center of the vision camera as the axis; moving the driving end, the flange and the calibration plate in different poses in the field of view of the vision camera; making the vision camera capture calibration point cloud images of the calibration plate in corresponding poses; obtaining a first center coordinate of the calibration plate after rotation and translation conversion of the camera coordinate system and the fixed coordinate system, and obtaining a second center coordinate of the calibration plate after rotation and translation conversion of the flange coordinate system and the fixed coordinate system; determining whether the calibration is completed according to whether the difference between the first center coordinate and the second center coordinate is within a preset range; capturing the second object on the first object to obtain a point cloud image; performing similarity matching on the point cloud image and a standard image to determine whether the point cloud image meets the standard; when the point cloud image meets the standard, performing position matching on the point cloud image and the standard image; analyzing the position-matched point cloud image to obtain a grasping point on the second object on the point cloud image; the transfer device grasps the second object according to the grasping point; the first center coordinate satisfies the following relationship: ； wherein Pb is the first center coordinate, Pg is the coordinate of the calibration plate in the calibration coordinate system, X is the coordinate relationship between the center of the vision camera and the fixed end in the calibration coordinate system, and B is the coordinate relationship between the center of the calibration plate and the center of the vision camera in the calibration coordinate system; the second center coordinate Pb' satisfies the following relationship: ； wherein Pb' is the second center coordinate, A is the coordinate relationship between the center of the flange and the fixed end in the calibration coordinate system, and C is the coordinate relationship between the center of the calibration plate and the center of the flange in the calibration coordinate system.

2. The vision-guided grasping method of claim 1, wherein, The step of capturing the second object on the first object to obtain a point cloud image comprises: fixing the vision camera at a preset height; moving the first object provided with the second object into the field of view of the vision camera; making the vision camera capture the first object to obtain the point cloud image, wherein the point cloud image comprises a point cloud image of the second object on the first object.

3. The vision-guided grasping method of claim 1, wherein, Before the position matching of the point cloud image and the standard image, the method further comprises: performing noise reduction processing on the point cloud image meeting the standard to remove abnormal images in the point cloud image. The point cloud image after removing the abnormal image is segmented and extracted to obtain an image corresponding to the second object of each layer.

4. The vision-guided grasping method of claim 3, wherein, Before the position-matched point cloud image is analyzed, the method further comprises: The position-matched point cloud image is subjected to interference processing to retain the second object of the uppermost layer.

5. The vision-guided grasping method of claim 1, wherein, The calibration plate is provided with a plurality of regularly arranged calibration holes, and the first center coordinate of the calibration plate after rotation and translation conversion through the camera coordinate system and the fixed coordinate system and the second center coordinate of the calibration plate after rotation and translation conversion through the flange coordinate system and the fixed coordinate system are obtained according to the calibration point cloud images of different poses. The center coordinates of each calibration hole on the calibration plate in the calibration coordinate system are obtained according to the calibration point cloud images of different poses. The center coordinates of the calibration plate in the calibration coordinate system are obtained according to the center coordinates of each calibration hole. The center coordinates of the calibration plate in the calibration coordinate system are converted through the camera coordinate system and the fixed coordinate system to obtain the first center coordinate, and converted through the flange coordinate system and the fixed coordinate system to obtain the second center coordinate.

6. The vision-guided grasping method of claim 1, wherein, When the calibration is completed, the method further comprises: The calibration plate connected to the flange is disassembled, and a clamping jaw for clamping the grabbing point is installed on the flange.

7. A vision-guided grasping device applied to the vision-guided grasping method of claim 1, for grasping a second object on a first object, characterized in that, It comprises a transfer device, a visual camera and a control device; The transfer device is used to grab the second object; The visual camera is used to shoot the second object on the first object to obtain a point cloud image; The control device is electrically connected with the visual camera and the transfer device respectively, and is used to: Match the point cloud image with a standard image for similarity to determine whether the point cloud image meets the standard; When the point cloud image meets the standard, the point cloud image and the standard image are position-matched; The position-matched point cloud image is analyzed to obtain the grabbing point on the second object on the point cloud image; According to the grabbing point, the transfer device grabs the second object.

Images