Picking method, picking system and picking control device

Abstract

A picking method, picking system, and picking control device capable of improving work efficiency. In the picking method, a picking action is repeatedly performed, the picking action comprising: a first step of acquiring an image containing multiple objects using a sensor; a second step of calculating the pose of the objects based on the image; a third step of determining the gripping pose of the objects whose poses are calculated when being gripped by a robot; a fourth step of gripping the objects in the determined gripping pose; a fifth step of positioning the gripped objects within the field of view of the sensor and acquiring an image containing multiple objects not gripped by the robot and the object gripped by the robot; and a sixth step of calculating the pose of the objects gripped by the robot based on the image, wherein the fifth step in the nth picking action also serves as the first step in the n+1th picking action, where n is an integer greater than or equal to 1.

Description

Technical Field

[0001] The present invention relates to a picking method, a picking system and a picking control device. Background Art

[0002] Patent document 1 describes a method for obtaining the posture of a robot hand when holding a workpiece, obtaining the posture of the workpiece held by the robot hand based on an image obtained by a camera, and then obtaining the relative posture of the robot hand and the workpiece based on the posture of the robot hand and the posture of the workpiece.

[0003] Patent Document 1: Japanese Patent Application Laid-Open No. 2015-199155

[0004] This method acquires the position of the workpiece held by the robot hand, thereby accurately determining the relative position of the robot hand and the workpiece. However, there is no detailed description of the camera's shooting timing, making it difficult to shorten the operation time based on this shooting timing. Summary of the Invention

[0005] The picking method of the present invention repeatedly performs a picking action, and the picking action includes: a first step of using a sensor to obtain an image including multiple objects not held by the robot; a second step of calculating the posture of at least one of the objects based on the image obtained in the first step; a third step of determining the holding posture of the robot when at least one of the objects whose posture is calculated in the second step is held by the robot; a fourth step of causing the robot to hold the object in the holding posture determined by the third step; a fifth step of causing the object held by the robot in the fourth step to be located within the field of view of the sensor, and using the sensor to obtain an image including multiple objects not held by the robot and the object held by the robot; and a sixth step of calculating the posture of the object held by the robot based on the image obtained in the fifth step, the fifth step in the nth picking action also serving as the first step in the n+1th picking action, wherein n is an integer greater than 1.

[0006] The picking system of the present invention has a robot and a sensor, and the picking system repeatedly performs a picking action, and the picking action includes: a first step of using the sensor to obtain an image including multiple objects not held by the robot; a second step of calculating the posture of at least one of the objects based on the image obtained in the first step; a third step of determining the holding posture of the robot when at least one of the objects whose posture is calculated in the second step is held by the robot; a fourth step of causing the robot to hold the object in the holding posture determined by the third step; a fifth step of causing the object held by the robot in the fourth step to be located within the field of view of the sensor, and using the sensor to obtain an image including multiple objects not held by the robot and the object held by the robot; and a sixth step of calculating the posture of the object held by the robot based on the image obtained in the fifth step, the fifth step in the nth picking action also serving as the first step in the n+1th picking action, wherein n is an integer greater than 1.

[0007] The picking control device of the present invention repeatedly performs a picking action, and the picking action includes: a first step of using a sensor to obtain an image including multiple objects not held by the robot; a second step of calculating the posture of at least one of the objects based on the image obtained in the first step; a third step of determining the holding posture of the robot when at least one of the objects whose posture is calculated in the second step is held by the robot; a fourth step of causing the robot to hold the object in the holding posture determined by the third step; a fifth step of causing the object held by the robot in the fourth step to be within the field of view of the sensor, and using the sensor to obtain an image including multiple objects not held by the robot and the object held by the robot; and a sixth step of calculating the posture of the object held by the robot based on the image obtained in the fifth step, the fifth step in the nth picking action also serving as the first step in the n+1th picking action, wherein n is an integer greater than 1. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 This is a diagram showing the overall structure of the picking system.

[0009] Figure 2 It is a flowchart showing the picking method.

[0010] Figure 3 A diagram showing a first image.

[0011] Figure 4A diagram showing a first image.

[0012] Figure 5 This is a diagram for explaining the holding method.

[0013] Figure 6 This is a diagram for explaining the holding method.

[0014] Figure 7 This is a diagram for explaining the holding method.

[0015] Figure 8 A diagram showing the second image.

[0016] Figure 9 3 is a diagram showing the positional relationship when the second image is captured.

[0017] Figure 10 This is a diagram showing a state where the calculated posture is compared with the correct posture.

[0018] Description of Reference Numerals

[0019] 100…picking system; 200…carrying platform; 300…sensor; 310…3D camera; 500…manipulator; 501…claw; 502…claw; 600…robot; 610…base; 620…robot arm; 621…arm; 622…arm; 623…arm; 624…arm; 625…arm; 626…arm; 640…robot control device; E…encoder; G1…first image; G2…second image; J1…joint; J2…joint; J3…joint; J4…joint ; J5…joint; J6…joint; M…motor; P0…correct posture; P1…posture; Q…loading area; S1…first step; S2…second step; S3…third step; S4…fourth step; S5…fifth step; S6…sixth step; S7…seventh step; S8…eighth step; S9…ninth step; Sp…picking action; Sp(n)…picking action; Sp(n+1)…picking action; T1…area; T2…area; W…workpiece; Wa…workpiece; Wb…workpiece. DETAILED DESCRIPTION

[0020] Hereinafter, the picking method, picking system, and picking control device of the present invention will be described in detail based on the embodiments shown in the drawings.

[0021] Figure 1 This is a diagram showing the overall structure of the picking system. Figure 2 It is a flowchart showing the picking method. Figure 3 and Figure 4 Each of them is a diagram showing the first image. Figures 5 to 7 This is a diagram for explaining the holding method. Figure 8 A diagram showing the second image. Figure 93 is a diagram showing the positional relationship when the second image is captured. Figure 10 This is a diagram showing a state where the calculated posture is compared with the correct posture.

[0022] Figure 1 The illustrated picking system 100 includes a sensor 300 that measures a plurality of workpieces W as objects placed in bulk on a mounting table 200 , and a robot 600 that includes a manipulator 500 that picks a specific workpiece W from the mounting table 200 .

[0023] The robot 600 is a six-axis robot with six drive axes. The robot 600 has a base 610 fixed to the ground and a robotic arm 620 connected to the base 610. In addition, the robotic arm 620 is a robotic arm formed by a plurality of arms 621, 622, 623, 624, 625, and 626 being rotatably connected, and has six joints J1 to J6. Among them, the joints J2, J3, and J5 are bending joints, and the joints J1, J4, and J6 are torsion joints. In addition, a motor M as a drive source and an encoder E for detecting the rotation amount of the motor M (the rotation angle of the arm) are respectively provided on the joints J1, J2, J3, J4, J5, and J6.

[0024] The robot 500 is connected to the front end of the arm 626. The robot 500 is not particularly limited and can be appropriately set according to the target work. However, in this embodiment, the robot 500 is configured to grip the workpiece W with a pair of claws 501 and 502.

[0025] The robot 600 also includes a robot control unit 640, which serves as a picking control device. This unit controls the joints J1 to J6 and the drive of the manipulator 500 to pick workpieces W. The robot control unit 640 is comprised of, for example, a computer and includes a processor (CPU) for processing information, a memory communicably connected to the processor, and an external interface. The memory stores various programs executable by the processor, and the processor can read and execute the various programs stored in the memory.

[0026] The above briefly describes the robot 600. However, the structure of the robot 600 is not particularly limited. For example, in addition to a six-axis robot, the robot 600 may also be a horizontal multi-joint robot (SCARA robot), a dual-arm robot having two robotic arms 620 as described above, or the like. Furthermore, the robot 600 may not be fixed to the ground; for example, it may be fixed to an unmanned transport vehicle such as an AMR (Autonomous Mobile Robot) or an AGV (Automatic Guided Vehicle).

[0027] The sensor 300 is positioned above the stage 200 so as not to interfere with the movement of the robot 600. The sensor 300 is not particularly limited as long as it can acquire the measurement information necessary to identify the position of the workpiece W on the stage 200. In this embodiment, a 3D camera 310 (stereo camera) is used as a 3D sensor that captures a range image with depth information (depth information) for each pixel. Alternatively, a measuring device that measures three-dimensional shapes using a phase shift method, for example, can be used as the sensor 300.

[0028] The overall configuration of the picking system 100 has been described above. Next, a picking method for picking workpieces W using the picking system 100 will be described. The picking method is implemented by the robot control device 640 controlling each unit.

[0029] like Figure 2 As shown, the picking method repeatedly executes the picking action Sp including the first step S1 to the ninth step S9 as one cycle.

[0030] First step S1

[0031] like Figure 3 As shown, in the first step S1, the 3D camera 310 is used to acquire a first image G1 including a plurality of workpieces W arranged on the mounting table 200. The first image G1 is a range image, and each pixel included in the first image G1 has depth information.

[0032] Second step S2

[0033] like Figure 4 As shown, in the second step S2, at least one workpiece W is extracted from the first image G1 obtained in the first step S1, and the posture of the extracted workpiece W is calculated. In this embodiment, multiple workpieces W are extracted from the first image G1, and the posture of each extracted workpiece W is calculated. In this way, by calculating the postures of multiple workpieces W, an option is added in the subsequent fourth step S4, which enables smoother and more accurate operation. It should be noted that in Figure 4 , the workpiece W whose posture is calculated is colored and illustrated. In addition, for convenience of explanation, the workpiece W whose posture is calculated is also referred to as "workpiece Wa" below.

[0034] The method for calculating the pose of the workpiece W is not particularly limited; for example, template matching can be used. In this method, a 3D model of the workpiece W is first created using CAD. The created 3D model is then set to various poses (positions and postures) across 360 degrees, and the pose of the workpiece W in the contour shape at that time is estimated, and a learning model is generated that associates them. The pose of the workpiece W is then calculated by comparing the workpiece W extracted from the first image G1 with the learning model. This method allows the pose of the workpiece W to be calculated with high precision.

[0035] However, the second step S2 is not limited thereto, and one workpiece W may be extracted from the first image G1. This shortens the time required for extraction and improves the efficiency of the picking operation Sp.

[0036] Third step S3

[0037] In the third step S3, the gripping method and gripping posture of robot 600 are determined when at least one of the workpieces Wa whose posture was calculated in the second step S2 is being gripped by robot 600. In this embodiment, the gripping method and gripping posture of robot 600 are determined for all workpieces Wa. By calculating the postures of multiple workpieces W, additional options are provided in the subsequent fourth step S4, enabling smoother and more accurate work.

[0038] First, the gripping method of the robot 600 is described. In this embodiment, the workpiece W is formed into a flat ring shape, and the robot 500 has a pair of claws 501 and 502. Therefore, there is Figure 5 The first holding method shown is to hold the workpiece W by clamping it laterally with a pair of claws 501 and 502. Figure 6 The second holding method shown in FIG. 1 is to hold the workpiece W longitudinally by a pair of claws 501 and 502, and the second holding method shown in FIG. Figure 7 The third gripping method shown is to grip the workpiece Wa by inserting a pair of claws 501 and 502 into a hole and pressing against the inner circumference. In the third step S3, it is determined which of the first, second, and third gripping methods is suitable for each workpiece Wa and which gripping method can easily and firmly grip the workpiece Wa. For example, the gripping method is determined taking into account the posture of the workpiece Wa and the postures of surrounding workpieces W.

[0039] However, depending on the shape of the workpiece W, when there is actually only one holding method, or when there are multiple holding methods but only a specific holding method is used for holding, there is no option for the holding method, and the determination of the holding method may be omitted.

[0040] Next, the gripping posture of robot 600 will be described. The gripping posture refers to the posture of robot 600 approaching workpiece Wa, that is, the relative posture of workpiece W and robot arm 500. For example, the gripping posture is determined by considering the posture of workpiece Wa, the postures of surrounding workpieces W, the movable range of robot arm 620, and other factors.

[0041] However, the third step S3 is not limited thereto, and the gripping method and gripping posture of the robot 600 may be determined when the robot 600 grips one of the workpieces Wa. This shortens the time required to determine the gripping method and gripping posture, and improves the efficiency of the picking operation Sp.

[0042] The method for selecting a workpiece Wa from multiple workpieces Wa is not particularly limited; for example, a method based on the confidence level of template matching can be used. Since a higher confidence level indicates a higher probability that the workpiece Wa's position calculated in step S2 will match its actual position, selecting the workpiece Wa with a higher confidence level allows for a highly accurate picking operation Sp. In this case, the workpiece Wa with the highest confidence level can be selected from the multiple workpieces Wa, or a workpiece Wa can be randomly selected from multiple workpieces Wa with a confidence level greater than a specified value.

[0043] Fourth step S4

[0044] In the fourth step S4, a workpiece Wa to be gripped by the robot 600 is first selected from among multiple workpieces Wa. The selection method is not particularly limited; for example, it can be determined based on considerations such as the degree of freedom in template matching, the ease of access to the workpiece Wa, and the ability to smoothly transition to the post-gripping motion. Next, the robot arm 620 and the manipulator 500 are controlled so that the robot 600 grips the workpiece Wa using the gripping method and gripping posture determined in the third step S3. It should be noted that, for ease of explanation, the workpiece Wa gripped by the robot 600 will be referred to as "workpiece Wb" below.

[0045] Fifth step S5

[0046] In the fifth step S5, first, the driving of the robot arm 620 is controlled so that the workpiece Wb held by the robot 600 in the fourth step S4 is located at a predetermined position within the field of view of the 3D camera 310. Figure 8 As shown, by imaging with the 3D camera 310 , a second image G2 including the plurality of workpieces W placed on the mounting table 200 and the workpiece Wb held by the robot 600 is acquired.

[0047] Here, the position of the workpiece Wb within the field of view of the 3D camera 310 will be described. In this embodiment, the workpiece Wb is positioned within the field of view of the 3D camera 310 so as to minimize overlap with the multiple workpieces W placed on the loading platform 200. This position will be referred to as the "shooting position" below. As will be described later, since the second image G2 is used as the first image G1 in the next picking operation Sp, it is preferable to include more workpieces W in the second image G2 in order to increase the options for workpieces W to be grasped.

[0048] In particular, Figure 8 As shown, in this embodiment, based on the posture calculation results in the second step S2, the placement area Q for the workpieces W bulk loaded on the loading table 200 is set. The position of the workpiece Wb within the field of view of the 3D camera 310 is determined so that the overlap area (amount of overlap) with the set placement area Q is minimized, preferably to zero. This determination method utilizes the first image G1 and the calculation results in the second step S2, eliminating the need to re-detect the workpieces W on the loading table 200. The position of the workpiece Wb within the field of view of the 3D camera 310 can be easily and appropriately determined. It should be noted that, for example, workpieces W with a posture confidence level exceeding a certain value can be extracted, and the placement area Q can be set based on the positions of these extracted workpieces W. This method reduces overlap with workpieces W with high confidence levels, allowing it to be effectively used as the first image G1 in the next picking operation Sp.

[0049] In addition, if Figure 9 As shown, in this embodiment, the workpiece Wb is positioned closer to the 3D camera 310 than the multiple workpieces W placed on the stage 200. That is, the workpiece W on the stage 200 and the workpiece Wb are staggered in the depth direction (optical axis direction) of the 3D camera 310. It should be noted that the second image G2 is a distance image in which each pixel has depth information. Therefore, as described above, by staggering the workpiece W on the stage 200 and the workpiece Wb in the depth direction of the 3D camera 310, and based on the depth information, it is possible to Figure 8 As shown, the region T1 including the workpiece W placed on the mounting table 200 and the region T2 including the workpiece Wb held by the robot 600 are distinguished and extracted from the second image G2 .

[0050] Sixth step S6

[0051] In step S6, based on the second image G2 acquired in step S5, and in particular the region T2 extracted from the second image G2, the posture P1 of the workpiece Wb held by the robot 600 is calculated. In other words, the relative positional relationship between the robot hand 500 and the workpiece Wb (the position and posture of the workpiece Wb held relative to the robot hand 500) is calculated.

[0052] Step S7

[0053] like Figure 10 As shown, in the seventh step S7, for example, the posture P1 of the workpiece Wb calculated in the sixth step S6 is compared with the correct posture P0, which is the ideal posture of the workpiece Wb, and the error is calculated. It should be noted that the correct posture P0 can be set in advance through teaching or other learning based on the shape of the workpiece W, the structure of the manipulator 500, the content of the work performed by the robot 600, etc.

[0054] However, the method for setting the correct posture P0 is not particularly limited. For example, the correct posture P0 can also be calculated by detecting the posture of the manipulator 500 based on the second image G2, and calculating the posture of the workpiece W relative to the posture of the manipulator 500 according to the gripping method determined in the third step S3. According to such a method, since no prior learning is required, the time spent on preparation is shortened. In addition, in this embodiment, in order to compare with the correct posture P0, it is preferred to make the posture of the manipulator 500 when the second image G2 is taken consistent with the correct posture P0. In this regard, according to the method of calculating the correct posture P0 based on the second image G2, the posture of the manipulator 500 when shooting can be arbitrarily set. Therefore, it becomes easier to select a shooting position with a small overlapping area.

[0055] Alternatively, the position and posture of the manipulator 500 may be calculated not based on the second image G2 but based on the relative positional relationship between the robot 600 and the 3D camera 310 and the posture of the manipulator arm 620 calculated from the outputs of the encoders E at the joints J1 to J6. This method can also achieve the same effect as calculating the correct position and posture P0 based on the second image G2.

[0056] Eighth step S8

[0057] Here, the motion command generated based on the workpiece Wb being in the correct posture P0 is input to the robot control device 640 from the host computer (not shown). Figure 10As shown in FIG, if the actual posture P1 deviates from the correct posture P0, the motion required by the motion command may not be performed smoothly, and this failure may lead to a decrease in yield. Therefore, in the eighth step S8, the motion command is corrected based on the error between the posture P1 calculated in the seventh step S7 and the correct posture P0, and a corrected motion command is generated.

[0058] Ninth step S9

[0059] In the ninth step S9 , the driving of the robot 600 is controlled based on the correction operation command generated in the eighth step S8 , so that the robot 600 performs a predetermined operation. The operation is not particularly limited.

[0060] The above describes the picking action Sp. In the picking method, such picking action Sp is repeated. Then, as shown in FIG. Figure 2 As shown, the fifth step S5 in the n-th (where n is an integer greater than or equal to 1) picking operation Sp(n) is also used as the first step S1 in the subsequent n+1-th picking operation Sp(n+1). That is, the second image G2 acquired in the fifth step S5 in the n-th picking operation Sp(n), more specifically, the area T1 in the second image G2, is used as the first image G1 acquired in the first step S1 in the n+1-th picking operation Sp(n+1). According to such a method, since it is not necessary to acquire the first image G1 again in the n+1-th picking operation Sp(n+1), the number of steps in the n+1-th picking operation Sp(n+1) is reduced, and the time required for the operation can be shortened accordingly.

[0061] It should be noted that in the first picking operation Sp, there is no second image G2 that can be used as the first image G1, so the first image G1 must be acquired. However, if, for example, the arrangement of the workpieces W on the loading platform 200 has not changed between the last stop and the current restart of the robot 600, the second image G2 acquired before the last stop of the robot 600 may be used as the first image G1 in the current first picking operation Sp.

[0062] Furthermore, for example, in step S5 of the picking operation Sp(n+1), when the second image G2 acquired by the picking operation Sp(n) is used to determine the imaging position, the bulk state of the workpieces W on the mounting table 200 in area T2 is unclear (obscured by the workpieces Wb and invisible). Therefore, the mounting area Q cannot be set, and the overlapping area cannot be calculated. Therefore, in step S5 of the picking operation Sp(n+1), the overlapping area at the same imaging position as that of the picking operation Sp(n) is estimated to be equal to the overlapping area calculated by the picking operation Sp(n). This allows the calculation of an imaging position with a smaller overlapping area.

[0063] The above describes the picking method, the picking system 100, and the robot control device 640 as a picking control device. As described above, such a picking method repeatedly performs a picking action Sp, and the picking action Sp includes: a first step S1, using the sensor 300 to obtain a first image G1 including a plurality of workpieces W as objects that are not held by the robot 600; a second step S2, based on the first image G1 obtained in the first step S1, calculating the posture of at least one workpiece W; a third step S3, determining the gripping posture of the robot 600 when at least one of the workpieces Wa whose posture is calculated in the second step S2 is gripped by the robot 600; and a third step S4, determining the gripping posture of the robot 600 when at least one of the workpieces Wa whose posture is calculated in the second step S2 is gripped by the robot 600. In step S4, the robot 600 grasps the workpiece Wa in the grasping posture determined in step S3. In step S5, the workpiece Wb grasped by the robot 600 in step S4 is positioned within the field of view of the sensor 300, and the sensor 300 is used to acquire a second image G2 that includes the workpieces W not grasped by the robot 600 and the workpiece Wb grasped by the robot 600. Furthermore, in step S6, the posture of the workpiece Wb grasped by the robot 600 is calculated based on the second image G2 acquired in step S5. Step S5 in the nth (where n is an integer greater than or equal to 1) picking operation Sp(n) also serves as step S1 in the n+1th picking operation Sp(n+1). In other words, the second image G2 acquired in the picking operation Sp(n) serves as the first image G1 in the picking operation Sp(n+1). Therefore, according to such a method, the time (takt time) required for the picking operation Sp(n+1) can be shortened, and the efficiency of the picking operation can be improved.

[0064] As described above, the picking operation Sp includes the seventh step S7, which calculates the error between the posture P1 calculated in the sixth step S6 and the correct posture P0, which is the ideal posture of the workpiece Wb grasped by the robot 600. This error can be reflected in the control of the robot 600, making the operation of the robot 600 smoother.

[0065] As described above, in the fifth step S5, the position of the workpiece Wb held by the robot 600 within the field of view of the sensor 300 is determined based on the first image G1 acquired in the first step S1. Thus, by using the first image G1, the position of the workpiece Wb within the field of view of the sensor 300 can be easily determined.

[0066] Furthermore, as described above, in step S5, the position of the workpiece Wb held by the robot 600 is determined within the field of view of the sensor 300 so that the overlapping area with the workpiece W whose posture was calculated in step S2 is minimized. This allows the option of the workpiece Wb to be increased in the next picking operation Sp.

[0067] As described above, the sensor 300 is a 3D camera 310 serving as a 3D sensor. Based on the depth information obtained from the 3D camera 310, the second image G2 acquired in step S5 is used to distinguish between the region T1 including the plurality of workpieces W not held by the robot 600 and the region T2 including the workpiece Wb held by the robot 600. This method facilitates the distinction between the regions T1 and T2.

[0068] In addition, as described above, the picking system 100 includes a robot 600 and a sensor 300, and repeatedly performs a picking action Sp, which includes: a first step S1, using the sensor 300 to obtain a first image G1 of a plurality of workpieces W as objects that are not held by the robot 600; a second step S2, based on the first image G1 obtained in the first step S1, calculating the posture of at least one workpiece W; a third step S3, determining the position of the robot 600 when at least one of the workpieces Wa whose posture is calculated in the second step S2 is held by the robot 600 In step S4, the robot 600 grasps the workpiece W in the grasping posture determined in step S3. In step S5, the workpiece Wb grasped by the robot 600 in step S4 is positioned within the field of view of the sensor 300, and the sensor 300 is used to acquire a second image G2 including the workpieces W not grasped by the robot 600 and the workpiece Wb grasped by the robot 600. Furthermore, in step S6, the posture of the workpiece Wb grasped by the robot 600 is calculated based on the second image G2 acquired in step S5. The fifth step S5 in the nth (where n is an integer greater than or equal to 1) picking operation Sp(n) also serves as the first step S1 in the n+1th picking operation Sp(n+1). In other words, the second image G2 acquired in the picking operation Sp(n) serves as the first image G1 in the picking operation Sp(n+1). Therefore, according to such a method, the time required for the picking operation Sp(n+1) can be shortened, and the efficiency of the picking operation can be improved.

[0069] In addition, as described above, the robot control device 640 as the picking control device repeatedly performs the picking action Sp, which includes: a first step S1, using the sensor 300 to obtain a first image G1 including a plurality of workpieces W as objects that are not held by the robot 600; a second step S2, based on the first image G1 obtained in the first step S1, calculating the posture of at least one workpiece W; a third step S3, determining the position of the robot 600 when at least one of the workpieces Wa whose posture is calculated in the second step S2 is held by the robot 600. 0; a fourth step S4 causes the robot 600 to grasp the workpiece W in the grasping posture determined in the third step S3; a fifth step S5 places the workpiece Wb grasped by the robot 600 in the fourth step S4 within the field of view of the sensor 300, and uses the sensor 300 to acquire a second image G2 that includes multiple workpieces W not grasped by the robot 600 and the workpiece Wb grasped by the robot 600; and a sixth step S6 calculates the posture of the workpiece Wb grasped by the robot 600 based on the second image G2 acquired in the fifth step S5. The fifth step S5 in the nth (where n is an integer greater than or equal to 1) picking operation Sp(n) also serves as the first step S1 in the n+1th picking operation Sp(n+1). In other words, the second image G2 acquired in the picking operation Sp(n) serves as the first image G1 in the picking operation Sp(n+1). Therefore, according to such a method, the time required for the picking operation Sp(n+1) can be shortened, and the efficiency of the picking operation can be improved.

[0070] The above description of the picking method, picking system, and picking control device of the present invention is based on the illustrated embodiments, but the present invention is not limited thereto, and the structure of each part can be replaced with any structure having the same function. In addition, any other structure can also be added.

Claims

1. A picking method, characterized in that: Repeatedly perform a picking action, wherein the picking action includes: In a first step, a sensor is used to acquire an image including a plurality of objects not held by the robot; A second step is to calculate the posture of at least one of the objects based on the image acquired in the first step; A third step is to determine a gripping posture of the robot when at least one of the objects whose postures are calculated in the second step is gripped by the robot; A fourth step is to cause the robot to hold the object in the holding posture determined in the third step; A fifth step is to place the object held by the robot in the fourth step within the field of view of the sensor, and to use the sensor to acquire an image including a plurality of objects not held by the robot and the object held by the robot; as well as The sixth step is to calculate the posture of the object held by the robot based on the image acquired in the fifth step. The fifth step in the nth picking operation is also used as the first step in the n+1th picking operation, where n is an integer greater than or equal to 1.

2. The picking method according to claim 1, wherein: The picking operation includes a seventh step of calculating an error between the posture calculated in the sixth step and a correct posture that is an ideal posture of the object grasped by the robot.

3. The picking method according to claim 1 or 2, wherein: In the fifth step, the position of the object held by the robot within the field of view of the sensor is determined based on the image acquired in the first step.

4. The picking method according to claim 3, wherein: The position of the object held by the robot within the field of view of the sensor is determined so that the overlapping area with the object whose posture is calculated in the second step is as small as possible.

5. The picking method according to claim 1 or 2, wherein: The sensor is a 3D sensor, Based on the depth information obtained from the 3D sensor, a region including the plurality of objects not held by the robot and a region including the object held by the robot are distinguished from the image acquired in the fifth step.

6. A picking system, characterized in that: The picking system comprises a robot and a sensor, and repeatedly performs a picking action, wherein the picking action includes: In a first step, the sensor is used to acquire an image including a plurality of objects not held by the robot; A second step is to calculate the posture of at least one of the objects based on the image acquired in the first step; A third step is to determine a gripping posture of the robot when at least one of the objects whose postures are calculated in the second step is gripped by the robot; A fourth step is to cause the robot to hold the object in the holding posture determined in the third step; A fifth step is to place the object held by the robot in the fourth step within the field of view of the sensor, and to use the sensor to acquire an image including a plurality of objects not held by the robot and the object held by the robot; as well as The sixth step is to calculate the posture of the object held by the robot based on the image acquired in the fifth step. The fifth step in the nth picking operation is also used as the first step in the n+1th picking operation, where n is an integer greater than or equal to 1.

7. A picking control device, characterized in that: Repeatedly perform a picking action, wherein the picking action includes: In a first step, a sensor is used to acquire an image including a plurality of objects not held by the robot; A second step is to calculate the posture of at least one of the objects based on the image acquired in the first step; A third step is to determine a gripping posture of the robot when at least one of the objects whose postures are calculated in the second step is gripped by the robot; A fourth step is to cause the robot to hold the object in the holding posture determined in the third step; A fifth step is to place the object held by the robot in the fourth step within the field of view of the sensor, and to use the sensor to acquire an image including a plurality of objects not held by the robot and the object held by the robot; as well as The sixth step is to calculate the posture of the object held by the robot based on the image acquired in the fifth step. The fifth step in the nth picking operation is also used as the first step in the n+1th picking operation, where n is an integer greater than or equal to 1.

Images