Vision system and vision detection method

Abstract

The present invention accurately obtains the removal position where a robot is to remove a workpiece, on the basis of detection results based on image vision detection.　A vision system equipped with an acquisition unit for acquiring a first image which depicts a region where a plurality of workpieces are present, a detection unit for outputting detection results about the plurality of workpieces, and a robot for executing an operation for changing the position of one or more of the plurality of workpieces, wherein: said robot executes a first operation for changing the position of the plurality of workpieces; the acquisition unit acquires a second image which depicts the region where the plurality of workpieces are present after the first operation is executed; and the detection unit subjects the first and second images to detection, and outputs one or more elements from among the positions of the plurality of workpieces, the orientations thereof and the outer shape information thereof as the detection results, on the basis of at least the amount of change between the first image data, which includes detection results, and the second image data, which includes detection results.

Description

Vision system and vision detection method

[0001] The present invention relates to a vision system and a vision detection method.

[0002] There are applications in which vision detection is performed on images captured by a 2D / 3D camera, and a robot picks up a workpiece based on information such as the detected position, orientation, and outer shape of the workpiece contained in the detection results. For example, Patent Document 1 proposes a technology in which a robot hand is moved to induce a load of workpieces to collapse, and then, after the load collapses, an image is taken again to find a new workpiece that can be grasped. Patent Document 2 proposes a technology in which a contact operation is performed on a workpiece, the force applied to the robot during contact and the positions of each joint of the robot are measured, and the position of the tip of the end effector is calculated by performing a forward transformation from the joint positions to update the workpiece surface shape, and a new gripping position is found to grip the workpiece.

[0003] JP 2019-198949 A JP 2017-136677 A

[0004] In Patent Document 1, the force inducing the workpiece load to collapse can cause two workpieces to stick together. In this case, the two stuck workpieces may be erroneously detected as a single large workpiece, potentially leading to inaccurate detection of the workpiece's position, orientation, and outer size. In Patent Document 2, the contact operation is stopped when the force measured during contact reaches an upper limit, but the force information is not used to update the map information (i.e., vision detection) for recalculating the workpiece surface shape. Patent Document 2 also calculates the end effector tip position using measured values ​​of each robot joint position to update the map information, which is 3D point cloud information. However, the calculated end effector tip position may be inaccurate due to adverse effects such as backlash in the reduction gear within the robot body and manufacturing errors and deflection of the robot arm and end effector / hand. Therefore, Patent Document 2 has the problem of being unable to create accurate map information, resulting in an inability to accurately capture the workpiece's surface shape and therefore inaccurate detection results. Furthermore, when performing vision detection on a captured image, tape or labels attached to the cardboard boxes serving as the workpiece may cause the position and outer size of the workpiece to be erroneously detected, resulting in an erroneous vision detection result being output. For example, the vision may be unable to detect gaps between multiple densely stacked cardboard boxes, resulting in the erroneous detection of a location including the gap as the detected position, or it may be unable to detect small grooves, steps, holes, etc., resulting in the erroneous detection of a location including these as the detected position. In these cases, when a robot hand equipped with a suction pad attempts to pick up a workpiece such as a cardboard box from the detected detection position, air leaks out, resulting in the pick-up operation failing.

[0005] Therefore, it is desirable to accurately determine the pick-up position of the workpiece to be picked up by the robot based on the detection results obtained by image vision detection.

[0006] One aspect of the vision system disclosed herein includes an acquisition unit that acquires a first image in which the areas where multiple workpieces are present are captured, a detection unit that outputs detection results for the multiple workpieces, and a robot that performs an operation to change the position of at least one of the multiple workpieces, wherein the robot performs a first operation to change the positions of the multiple workpieces, the acquisition unit acquires a second image in which the areas where the multiple workpieces are present are captured after the first operation is performed, and the detection unit performs detection on the first image and the second image, and outputs at least one of the position, posture, and external shape information of the multiple workpieces as the detection result based at least on the amount of change between first image data including the detection result and second image data including the detection result.

[0007] One aspect of the vision detection method disclosed herein comprises an acquisition step of acquiring a first image in which the presence areas of multiple workpieces are captured, a detection step of outputting detection results of the multiple workpieces, and an execution step of having a robot perform an operation to change the position of at least one of the multiple workpieces, wherein the execution step performs a first operation to have the robot change the positions of the multiple workpieces, the acquisition step acquires a second image in which the presence areas of the multiple workpieces are captured after the first operation has been performed, and the detection step performs detection on the first image and the second image, and outputs at least one of the position, posture, and external shape information of the multiple workpieces as the detection result based at least on the amount of change between first image data including the detection result and second image data including the detection result.

[0008] According to one aspect, the pick-up position of the workpiece to be picked up by the robot can be determined with high accuracy based on the detection result obtained by vision detection of the image.

[0009] FIG. 1 is a diagram showing an example of the configuration of a vision system according to the first embodiment. FIG. 2 is a functional block diagram showing an example of the functional configuration of a vision detection device. FIG. 3 is a diagram showing an example of an image of an area where multiple cardboard boxes are present before the position of the cardboard box is changed. FIG. 4 is a diagram showing an example of an image captured after the position of the cardboard box in FIG. 3 is changed. FIG. 5 is a diagram showing an example of an image captured after the position of the cardboard box in FIG. 4 is changed. FIG. 6 is a diagram showing an example of a final detection result. FIG. 7 is a flowchart explaining the detection processing of the vision system. FIG. 1 is a diagram showing an example of the configuration of a vision system according to the second embodiment. FIG. 2 is a functional block diagram showing an example of the functional configuration of a vision detection device. FIG. 3 is a flowchart explaining the detection processing of the vision system. FIG. 4 is a diagram showing an example of the configuration of a vision system according to the third embodiment. FIG. 5 is a functional block diagram showing an example of the functional configuration of a vision detection device. FIG. 6 is a diagram showing an example of the waveform of a contact force due to contact with the surface of a cardboard box, measured by a tactile / force sensor. FIG. 7 is a flowchart explaining the detection processing of the vision system.

[0010] First to third embodiments will be described in detail with reference to the drawings. Each embodiment has a common configuration in which a robot performs a first operation on a workpiece to obtain at least one of the workpiece's position, orientation, or outer shape information by performing vision detection on a first image capturing an area where the workpiece is present, and outputs at least one of the workpiece's position, orientation, or outer shape information as a detection result. However, in the first embodiment, in a state where multiple workpieces are densely stacked, the robot performs a first operation to change the positions of the multiple workpieces, obtains a second image capturing the area where the multiple workpieces are present after the first operation, performs vision detection on the first image and the second image, and outputs the detection results for the multiple workpieces based on the amount of change between the first image data including the detection results and the second image data including the detection results. In contrast, in the second embodiment, if there is an area in the first image where the 3D position of the workpiece cannot be obtained by vision detection (e.g., if there is an area where halation occurs or if the workpiece is transparent), the robot executes a first operation to touch the workpiece in the failed area, acquires contact information indicating the 3D position of the area, and corrects the detection result based on the acquired contact information. Furthermore, in the third embodiment, if the surface of a single workpiece is not uniform but has features that separate the surface areas (e.g., different textures or materials in different surface areas), resulting in an erroneous determination that multiple workpieces exist, the robot executes a first operation to touch the surface of the workpiece, identifies changes in the texture on the workpiece, distinguishes the area, and corrects the detection result for the workpiece. Below, the first embodiment will be described in detail, and then the second and third embodiments will be described, particularly focusing on differences from the first embodiment.

[0011] First Embodiment FIG. 1 is a diagram illustrating an example of the configuration of a vision system according to a first embodiment. Here, the example illustrates a case where multiple cardboard boxes are densely stacked as multiple workpieces. The present invention can be applied to jigs and other workpieces other than cardboard boxes, and can also be applied to workpieces with shapes such as triangles and hexagons. As shown in FIG. 1 , the vision system 1 includes a vision detection device 10, a robot control device 20, a robot 30, an imaging device 40, multiple cardboard boxes 50, and a pallet 60. The vision detection device 10, the robot control device 20, the robot 30, and the imaging device 40 may be directly connected to each other via a connection interface (not shown). The vision detection device 10, the robot control device 20, the robot 30, and the imaging device 40 may be connected to each other via a network (not shown), such as a local area network (LAN) or the Internet. In this case, the vision detection device 10, the robot control device 20, the robot 30, and the imaging device 40 each include a communication unit (not shown) for communicating with each other via such a connection. 1 depicts the vision detection device 10 and the robot control device 20 as separate entities, and in this case the vision detection device 10 may be configured, for example, by a computer. However, the invention is not limited to this configuration, and for example, the vision detection device 10 may be implemented inside the robot control device 20 and integrated with it, as will be described later.

[0012] The robot control device 20 is a device known to those skilled in the art for controlling the operation of the robot 30. In FIG. 1 , a teaching pendant 25 that teaches the robot 30 how to operate is connected to the robot control device 20. The robot control device 20 also has a display unit 21 such as a liquid crystal display. The robot control device 20 operates the robot 30 based on, for example, detection results output from a vision detection device 10 (described later) to change the position of at least one of the densely stacked cardboard boxes 50. The robot control device 20 receives the detection results of the cardboard boxes 50 detected by the vision detection device 10 (described later) using an image of the areas where the multiple cardboard boxes 50 exist captured by the imaging device 40 before the position change and an image of the areas where the multiple cardboard boxes 50 exist after the change. The robot control device 20 displays the received detection results on the display unit 21 of the robot control device 20 together with the image captured by the imaging device 40. The robot control device 20 selects a cardboard box 50 to be removed based on the received detection results or a user's selection instruction via the teaching pendant 25. The robot control device 20 generates a control signal for controlling the operation of the robot 30 to move the hand 31 of the robot 30 to a position for picking up the selected cardboard box 50. The robot control device 20 then outputs the generated control signal to the robot 30. The display unit 21 may be disposed on the teaching operation panel 25. The robot control device 20 may also include a vision detection device 10, as will be described later.

[0013] The robot 30 operates under the control of the robot control device 20. The robot 30 includes a base for rotating around a vertical axis, a movable and rotating arm, and a hand 31 attached to the arm for holding the cardboard boxes 50. While FIG. 1 shows an air suction-type pick-up hand attached to the hand 31 of the robot 30, a grip-type pick-up hand may also be attached. In response to a control signal output from the robot control device 20, the robot 30 drives the arm and hand 31 to move the hand 31 to the pick-up position for the selected cardboard box 50, and then picks the selected cardboard box 50 from the pallet 60. The destination of the picked cardboard box 50 is not shown. The specific configuration of the robot 30 is well known to those skilled in the art, so a detailed description will be omitted.

[0014] Furthermore, the vision detection device 10 and the robot control device 20 are assumed to have been calibrated in advance to correspond the machine coordinate system for controlling the robot 30 to the camera coordinate system of the detection results indicating the position, posture, and external shape information of the cardboard box 50.

[0015] The imaging device 40 is a digital camera or the like, and captures a two-dimensional image of the area where multiple cardboard boxes 50 densely stacked on the pallet 60 are present, projected onto a plane perpendicular to the optical axis of the imaging device 40. The image captured by the imaging device 40 may be a visible light image such as an RGB color image, a grayscale image, or a depth image. The imaging device 40 may also be configured to include an infrared sensor to capture thermal images, or may be configured to include an ultraviolet sensor to capture ultraviolet images for inspecting scratches, spots, etc. on the surface of an object. The imaging device 40 may also be configured to include an X-ray camera sensor to capture X-ray images, or may be configured to include an ultrasonic sensor to capture ultrasonic images. The imaging device 40 may also be a three-dimensional measuring device such as a stereo camera.

[0016] The cardboard boxes 50 are placed in a densely stacked state on a pallet 60. As described above, the workpiece is not limited to the cardboard boxes 50, but may be anything that can be held by the hand 31 attached to the arm of the robot 30, and the shape and the like of the workpiece are not particularly limited.

[0017] <Vision detection device 10> Fig. 2 is a functional block diagram showing an example of the functional configuration of the vision detection device 10. The vision detection device 10 is a computer known to those skilled in the art, and as shown in Fig. 2, has a control unit 100 and a storage unit 200. The control unit 100 also has an acquisition unit 110 and a detection unit 120.

[0018] <Storage Unit 200> The storage unit 200 is a solid state drive (SSD), a hard disk drive (HDD), etc. The storage unit 200 stores an operating system and application programs executed by the control unit 100, as well as images captured by the imaging device 40 and acquired by the acquisition unit 110 (described later).

[0019] <Control Unit 100> The control unit 100 includes a CPU (Central Processing Unit), ROM, RAM (Random Access Memory), CMOS (Complementary Metal-Oxide-Semiconductor) memory, and other components that are configured to communicate with each other via a bus and are well known to those skilled in the art. The CPU is a processor that controls the entire vision detection device 10. The CPU reads system programs and application programs stored in the ROM via the bus and controls the entire vision detection device 10 in accordance with the system programs and application programs. As a result, as shown in FIG. 2, the control unit 100 is configured to realize the functions of the acquisition unit 110 and the detection unit 120. The RAM stores various data, such as temporary calculation data and display data. The CMOS memory is backed up by a battery (not shown) and is configured as a non-volatile memory that retains its stored state even when the vision detection device 10 is turned off.

[0020] <Acquisition unit 110> The acquisition unit 110 acquires, for example, images of areas where multiple cardboard boxes 50 are present from the imaging device 40. The acquisition unit 110 stores the acquired images in the storage unit 200. Note that although the acquisition unit 110 acquires images from the imaging device 40, it may also acquire three-dimensional point cloud data, range image data, or the like.

[0021] <Detection Unit 120> The detection unit 120 detects the presence of the multiple cardboard boxes 50 based on, for example, an image (first image) of the presence area of ​​the multiple cardboard boxes 50 captured by the imaging device 40 before the robot control device 20 operates the robot 30 to change the position of the cardboard boxes 50, and an image (second image) of the presence area of ​​the multiple cardboard boxes 50 captured by the imaging device 40 after the position change. The detection unit 120 outputs the detection results, including position, posture, and outer shape information of the multiple cardboard boxes 50, to the robot control device 20 based at least on the amount of change between the first image data including the detection results and the second image data including the detection results. Specifically, the detection unit 120 reads, for example, from the storage unit 200, the image (first image) of the presence area of ​​the multiple cardboard boxes 50 captured by the imaging device 40 before the robot control device 20 operates the robot 30 to change the position of the cardboard boxes 50, and performs vision detection on the read image. FIG. 3 is a diagram showing an example of the image of the presence area of ​​the multiple cardboard boxes 50 before the position of the cardboard boxes 50 is changed. The image in Fig. 3 is an image of a plurality of cardboard boxes 50 taken from above, for example, from the Z-axis direction, by the imaging device 40. The detection unit 120 detects three cardboard boxes 50A1, 50A2, and 50A3 (as viewed from the top surface of the cardboard boxes 50) that are closely stacked together and are indicated by thick rectangles, by vision detection of the image in Fig. 3, and calculates the positions (positions of plus marks) X of the detected cardboard boxes 50A1, 50A2, and 50A3. Ａ１ , X Ａ２ , X Ａ３ And posture P Ａ１ , P Ａ２ , P Ａ３ and the outer shape information (for example, the width W of the thick rectangle) Ａ１ , W Ａ２ , W Ａ３ and depth D Ａ１ , D Ａ２ , D Ａ３ ) and get.

[0022] Next, the detection unit 120 executes an operation (first operation, auxiliary operation) in which the robot control device 20 uses the hand 31 attached to the robot 30 to apply force to the three cardboard boxes 50A1, 50A2, and 50A3 in the direction of the arrows (Y-axis direction) shown in FIG. 3 , reads from the storage unit 200 an image (second image) captured by the imaging device 40 after the positions of the multiple cardboard boxes 50 have been changed, and performs vision detection on the read image. FIG. 4 is a diagram showing an example of an image captured after the position of the cardboard box 50 in FIG. 3 has been changed. As shown in FIG. 4 , the detection unit 120 detects the four cardboard boxes 50B1, 50B2, 50B3, and 50B4 indicated by thick rectangles through vision detection on the image, and calculates the positions (positions of the plus marks) X of the detected cardboard boxes 50B1, 50B2, 50B3, and 50B4. Ｂ１ , X Ｂ２ , X Ｂ３ , X Ｂ４ And posture P Ｂ１ , P Ｂ２ , P Ｂ３ , P Ｂ４ and the outer shape information (width W Ｂ１ , W Ｂ２ , W Ｂ３ , W Ｂ４ and depth D Ｂ１ , D Ｂ２ , D Ｂ３ , D Ｂ４ 3 as one cardboard box, but as a result of adding the auxiliary operation of the robot 30, the detection unit 120 was able to detect the gap between the two cardboard boxes 50B1 and 50B2, and thus was able to correctly detect the cardboard boxes 50B1 and 50B2.

[0023] Furthermore, the detection unit 120 executes an auxiliary operation (second operation) in which the robot control device 20 uses the hand 31 attached to the robot 30 to apply force to the four cardboard boxes 50B1, 50B2, 50B3, and 50B4 in the direction of the arrows (X-axis direction) shown in Figure 4, reads from the storage unit 200 an image (third image) captured by the imaging device 40 after the positions of the multiple cardboard boxes 50 have been changed, and performs vision detection on the read image. Figure 5 is a diagram showing an example of an image captured after the position of the cardboard box 50 in Figure 4 has been changed. The detection unit 120 detects the three cardboard boxes 50C1, 50C2, and 50C3 indicated by the thick rectangles through vision detection on the image in Figure 5, and calculates the positions (positions of the plus signs) X of the detected cardboard boxes 50C1, 50C2, and 50C3. Ｃ１ , X Ｃ２ , X Ｃ３ And posture P Ｃ１ , P Ｃ２ , P Ｃ３ and the outer shape information (width W Ｃ１ , W Ｃ２ , W Ｃ３ and depth D Ｃ１ , D Ｃ２ , D Ｃ３ ) and are acquired. Note that the detection unit 120 detected the cardboard boxes 50B3 and 50B4 in FIG. 4 (the cardboard boxes 50A2 and 50A3 in FIG. 3) as two cardboard boxes, but because the auxiliary operation by the robot 30 made it impossible to detect the gap between the cardboard boxes 50B3 and 50B4, it erroneously detected the cardboard boxes 50B3 and 50B4 as one cardboard box 50C3. In this way, adding an auxiliary operation can have the opposite effect of causing an erroneous detection of something that was originally correctly detected by the vision.

[0024] The detection unit 120 integrates the first image data, the second image data, and the third image data, including the vision detection results before and after the application of the multiple auxiliary operations shown in Figures 3 to 5. For example, the detection unit 120 calculates the area from the width and depth of the detected outline of the cardboard box 50. By calculating and comparing the difference in area between the vision detection results shown in Figures 3 and 4, it can determine that two cardboard boxes 50B1 and 50B2 were mistakenly detected as one cardboard box 50A1. As a result, the detection unit 120 can output the correct final detection results, including the position, orientation, and outline information of each cardboard box 50D1 to 50D4, on an image, as shown in Figure 6, and store them as a file in the memory unit 200. The detection unit 120 outputs the detection results to the robot control device 20.

[0025] The above describes outputting the final detection results for multiple workpieces based on the amount of change in the position, posture, and outer shape information of the workpieces as detection results, but this is not limited to this. For example, by calculating the amount of change (difference) between only two images, as shown in Figures 3 and 4, captured before and after the first operation, it can be seen that a large difference in pixel values ​​occurs in the area of ​​cardboard boxes 50B1 and 50B2 in the post-operation image, which corresponds to the area of ​​cardboard box 50A1 in the pre-operation image, due to the cardboard box being moved. If cardboard box 50A1 is a single cardboard box rather than two cardboard boxes stuck together, even if an auxiliary operation is performed, the difference in the image of cardboard box 50A1 before and after the operation should not be large. The large difference indicates that the addition of the auxiliary operation separated the two cardboard boxes from their stuck state. This indicates that the two stuck cardboard boxes were mistakenly detected as one.

[0026] <Detection Process of Vision System 1> Next, the flow of the detection process of the vision system 1 will be described with reference to Fig. 7. Fig. 7 is a flowchart explaining the detection process of the vision system 1. The flow shown here is repeatedly executed each time multiple cardboard boxes 50 are imaged by the imaging device 40.

[0027] In step S11, the imaging device 40 captures an image of the area where the multiple cardboard boxes 50 are present.

[0028] In step S12 , the vision detection device 10 (acquisition unit 110 ) acquires the image captured in step S11 and stores the acquired image in the storage unit 200 .

[0029] In step S13, the robot control device 20 determines whether the robot 30 has performed a predetermined number of assisting operations (e.g., twice) using the hand 31 on the multiple cardboard boxes 50. If the robot 30 has performed the predetermined number of assisting operations using the hand 31, the process proceeds to step S16. On the other hand, if the robot 30 has not performed the predetermined number of assisting operations using the hand 31, the process proceeds to step S14.

[0030] In step S14, the robot control device 20 causes the robot 30 to perform one assist movement in a predetermined direction shown in FIG. 3 or FIG.

[0031] In step S15, the robot control device 20 determines whether the single assist operation executed in step S14 has been completed. If the single assist operation has been completed, the process returns to step S11. On the other hand, if the single assist operation has not been completed, the process waits until the single assist operation has been completed.

[0032] In step S16, the vision detection device 10 (detection unit 120) reads out the image captured in step S11 from the storage unit 200, and performs vision detection on the read out image.

[0033] In step S17, the vision detection device 10 (detection unit 120) calculates the difference in position, posture, or shape information between image data including the detection results of the vision detection, and calculates the position, posture, and shape information of each cardboard box 50 based on the calculated difference (amount of change).

[0034] In step S18, the vision detection device 10 (detection unit 120) calculates a final detection result based on the calculation result in step S17, and outputs the result to the robot control device 20.

[0035] In step S19, the robot control device 20 controls the robot 30 to take out one cardboard box 50 based on the detection result in step S18.

[0036] As described above, the vision system 1 according to the first embodiment allows the robot 30 to perform an auxiliary operation on multiple closely stacked cardboard boxes 50, thereby accurately detecting the position, posture, and outer shape information of each individual cardboard box 50 from an image, even when the multiple cardboard boxes 50 are densely stacked. This allows the vision system 1 to accurately determine the pick-up position of the workpiece to be picked up by the robot based on the detection results of the vision detection of the image. Furthermore, by capturing and integrating multiple images before and after the robot 30 performs an auxiliary operation that brings the workpiece into contact with the cardboard box 50, the vision system 1 can calculate the difference between the detected position and outer size of the workpiece in the images before and after the auxiliary operation, thereby calculating the correct position and outer size of the workpiece. The first embodiment has been described above.

[0037] Second Embodiment Next, the second embodiment will be described. As described above, in the first embodiment, when multiple workpieces are densely stacked, the robot is caused to perform a first operation to change the positions of the multiple workpieces, a second image capturing the area where the multiple workpieces are present after the first operation is performed is acquired, vision detection is performed on the first image and the second image, and detection results for the multiple workpieces are output based on the amount of change between the first image data containing the detection results and the second image data containing the detection results. In contrast, in the second embodiment, if there is an area in the first image where the three-dimensional position of the first image cannot be obtained by vision detection (for example, when halation occurs or the workpiece is transparent), the robot is caused to perform a first operation to touch the workpiece in the failed area, contact information indicating the three-dimensional position of the failed area is acquired, and the three-dimensional position of the failed area is complemented based on the acquired contact information to correct the detection result. This differs from the first embodiment in that the vision system 1 can accurately determine the pick-up position of the workpiece to be picked up by the robot based on the detection results of the image by vision detection. The second embodiment will be described below.

[0038] FIG. 8 is a diagram illustrating an example of the configuration of a vision system according to the second embodiment. Elements having similar functions to those of the vision system 1 in FIG. 1 are designated by the same reference numerals, and detailed descriptions thereof are omitted. Here, examples of workpieces include a glossy workpiece made of stainless steel or a transparent workpiece made of plastic. The present invention can also be applied to any workpiece that generates halation, other than glossy or transparent workpieces. As shown in FIG. 8, the vision system 1 includes a vision detection device 10a, a robot control device 20, a robot 30, an imaging device 40, one workpiece 50a, and a pallet 60. The robot control device 20, the robot 30, and the pallet 60 have functions equivalent to those of the robot control device 20, the robot 30, and the pallet 60 in the first embodiment. A force sensor 32 serving as a force measurement unit is disposed at the end of the robot 30. The workpiece 50a is a glossy or transparent workpiece and is placed on the pallet 60.

[0039] <Vision detection device 10a> Figure 9 is a functional block diagram showing an example of the functional configuration of the vision detection device 10a. Elements having the same functions as those of the vision detection device 10 in Figure 2 are given the same reference numerals, and detailed description thereof will be omitted. Like the vision detection device 10 according to the first embodiment, the vision detection device 10a has a control unit 100a and a storage unit 200. The control unit 100a also has an acquisition unit 110 and a detection unit 120a. The storage unit 200 has the same functions as the storage unit 200 in the first embodiment.

[0040] <Control Unit 100a> The control unit 100a includes a CPU, ROM, RAM, CMOS memory, etc., which are configured to be able to communicate with each other via a bus and are well known to those skilled in the art. The CPU is a processor that controls the entire vision detection device 10a. The CPU reads system programs and application programs stored in the ROM via the bus and controls the entire vision detection device 10a in accordance with the system programs and application programs. As a result, as shown in FIG. 9, the control unit 100a is configured to realize the functions of the acquisition unit 110 and the detection unit 120a. The acquisition unit 110 has functions equivalent to those of the acquisition unit 110 in the first embodiment.

[0041] <Detection Unit 120a> Similar to the detection unit 120 of the first embodiment, the detection unit 120a performs vision detection on an image of the area where the workpiece 50a is located, captured by the imaging device 40, calculates the position, posture, and external shape information of the workpiece 50a based on the amount of change between the image data including the detection results, and outputs the detection results to the robot control device 20. However, if the workpiece 50a is made of stainless steel or other material that generates halation due to its gloss, or if the workpiece 50a is made of transparent material such as plastic, the detection unit 120a may fail to acquire three-dimensional data on the workpiece's position, posture, and external shape information. In this case, the detection unit 120a outputs, for example, a signal indicating failure to acquire the three-dimensional data to the robot control device 20. The robot control device 20 performs an auxiliary operation (first operation) in which the hand 31 of the robot 30 touches and contacts the workpiece 50a, causing the force sensor 32 mounted at the end of the hand of the robot 30 to detect contact with the workpiece 50a. The detection unit 120a calculates and complements the shape of the workpiece 50a based on contact information (force data) measured by the force sensor 32 for the entire workpiece 50a or for an area where the measurement results of the three-dimensional position of the workpiece 50a are missing due to halation, and corrects the three-dimensional position of the workpiece 50a. Note that the contact information preferably includes information on the presence or absence of contact and information on the contact position. The detection unit 120a then outputs the corrected detection results to the robot control device 20. Note that the detection unit 120a may also correct the images (first image, second image) captured by the imaging device 40 based on the contact information.

[0042] <Detection Process of Vision System 1> Next, the flow of the detection process of the vision system 1 will be described with reference to Fig. 10. Fig. 10 is a flowchart illustrating the detection process of the vision system 1. The flow shown here is repeatedly executed each time the image of the workpiece 50a is captured by the imaging device 40. Note that the processes of steps S31, S32, S39, and S40 are similar to the processes of steps S11, S12, S18, and S19 in the first embodiment, and therefore will not be described again.

[0043] In step S33, the vision detection device 10a (detection unit 120a) performs vision detection on the image captured in step S11.

[0044] In step S34, the vision detection device 10a (detection unit 120a) determines whether the image detection result is insufficient (for example, halation occurs or the workpiece 50a is transparent). If the image detection result is insufficient (for example, halation occurs or the workpiece 50a is transparent), the detection unit 120a outputs a signal indicating that acquisition of three-dimensional data has failed to the robot control device 20, and the process proceeds to step S35. On the other hand, if the image detection result is sufficient (for example, halation does not occur and the workpiece 50a is not transparent), the process proceeds to step S39.

[0045] In step S35, the robot control device 20 executes an auxiliary operation for the robot 30 to bring the hand 31 into contact with the workpiece 50a.

[0046] In step S36, the robot control device 20 determines whether the hand 31 has come into contact with the workpiece 50a based on the force data from the force sensor 32. If the hand 31 has come into contact with the workpiece 50a, the process proceeds to step S37. On the other hand, if the hand 31 has not come into contact with the workpiece 50a, the process returns to step S35 and waits until the hand 31 comes into contact with the workpiece 50a.

[0047] In step S37, the force sensor 32 measures contact information (force data) of the entire workpiece 50a or an area where the measurement results of the three-dimensional position of the workpiece 50a are missing due to halation in response to the auxiliary operation in step S35.

[0048] In step S38, the vision detection device 10a (detection unit 120a) corrects the detection result of the workpiece 50a based on the contact information measured by the force sensor 32 for the entire workpiece 50a or for areas where the measurement results of the three-dimensional position of the workpiece 50a are missing due to halation.

[0049] As described above, when the image captured by the imaging device 40 contains an area of ​​the first image where the acquisition of the three-dimensional position by vision detection fails (for example, when halation occurs or the workpiece 50a is transparent), the vision system 1 according to the second embodiment executes an auxiliary operation of touching the hand 31 of the robot 30 to make contact with the workpiece 50a, causes the force sensor 32 mounted on the hand to measure the entire workpiece 50a or an area where the measurement results of the three-dimensional position of the workpiece 50a are missing due to halation, calculates and complements the shape of the workpiece 50a based on the measured contact information, and corrects the detection results of the workpiece 50a. This allows the vision system 1 to accurately determine the pick-up position of the workpiece to be picked up by the robot based on the detection results of the vision detection of the image. That is, the vision system 1 calculates the change in force level from contact information including the contact force or contact moment when the robot 30 comes into contact with the workpiece 50a, extracts features, and feeds back the features reflecting the force information at the contact position where the hand 31 actually comes into contact with the workpiece 50a to the vision detection result, thereby adding information that cannot be obtained by vision and correcting detection results that are often erroneous with vision, thereby obtaining highly accurate detection results.

[0050] <Modification of Second Embodiment> In the second embodiment, one workpiece 50a is placed on the pallet 60, but this is not limiting. For example, multiple workpieces 50a may be densely stacked, similar to the case of the cardboard boxes 50 in the first embodiment. In this case, the vision system 1 may not be able to determine which workpiece is located on top and which workpiece is located on the bottom through vision detection. Therefore, the robot control device 20 may execute an auxiliary operation to have the robot 30 touch the workpiece 50a with the hand 31, and have the force sensor 32 measure contact information. The vision detection device 10a may determine the relative positional relationship of the workpiece 50a based on the measured contact information and correct the detection result of the vision detection.

[0051] Third Embodiment Next, a third embodiment will be described. As described above, in the first embodiment, when multiple workpieces are densely stacked, the robot is caused to perform a first operation to change the positions of the multiple workpieces, a second image capturing an area where the multiple workpieces are present after the first operation is performed is acquired, vision detection is performed on the first image and the second image, and detection results for the multiple workpieces are output based on the amount of change between the first image data including the detection results and the second image data including the detection results. Furthermore, in the second embodiment, if there is an area in the first image where the three-dimensional position of the workpiece cannot be obtained by vision detection (for example, if there is an area where halation occurs or if the workpiece is transparent), the robot is caused to perform a first operation to touch the workpiece in the failed area, contact information indicating the three-dimensional position of the area is acquired, and the three-dimensional position of the failed area is complemented based on the acquired contact information to correct the detection result. This differs from the first embodiment in that: Furthermore, the third embodiment differs from the first and second embodiments in that, in cases where the surface of a single workpiece is not uniform but has features that separate the surface into regions (e.g., different textures or materials in different surface regions), resulting in an erroneous determination that multiple workpieces exist, the robot executes a first operation of touching the surface of the workpiece, identifies the changes in the texture on the workpiece, determines the regions, and corrects the detection result for the workpiece. This allows the vision system 1 to accurately determine the pick-up position of the workpiece to be picked by the robot based on the detection results from vision detection of the image. The third embodiment will now be described.

[0052] FIG. 11 is a diagram illustrating an example of the configuration of a vision system according to the third embodiment. Elements having similar functions to those of the vision system 1 in FIG. 1 are denoted by the same reference numerals, and detailed descriptions thereof are omitted. Here, the workpiece is illustrated as a cardboard box to which tape and labels such as address labels are attached. The present invention can also be applied to any workpiece other than cardboard box to which tape and labels are attached. As shown in FIG. 11 , the vision system 1 includes a vision detection device 10b, a robot control device 20, a robot 30, an imaging device 40, one workpiece 50b, and a pallet 60. The robot control device 20, the robot 30, and the pallet 60 have functions equivalent to those of the robot control device 20, the robot 30, and the pallet 60 in the first embodiment. A tactile / force sensor 33 serving as a force measurement unit is disposed at the tip of the hand 31. The workpiece 50b is a cardboard box to which tape and labels such as address labels are attached, and is placed on the pallet 60. Hereinafter, unless otherwise specified, it will also be referred to as the cardboard box 50b.

[0053] <Vision detection device 10b> Figure 12 is a functional block diagram showing an example of the functional configuration of the vision detection device 10b. Elements having the same functions as those of the vision detection device 10 in Figure 2 are given the same reference numerals, and detailed description thereof will be omitted. Like the vision detection device 10 according to the first embodiment, the vision detection device 10b includes a control unit 100b and a storage unit 200. The control unit 100b also includes an acquisition unit 110 and a detection unit 120b. The storage unit 200 has the same functions as the storage unit 200 in the first embodiment.

[0054] <Control Unit 100b> The control unit 100b includes a CPU, ROM, RAM, CMOS memory, etc., which are configured to be able to communicate with each other via a bus and are well known to those skilled in the art. The CPU is a processor that controls the entire vision detection device 10b. The CPU reads system programs and application programs stored in the ROM via the bus and controls the entire vision detection device 10b in accordance with the system programs and application programs. As a result, as shown in FIG. 12, the control unit 100b is configured to realize the functions of the acquisition unit 110 and the detection unit 120b. The acquisition unit 110 has functions equivalent to those of the acquisition unit 110 in the first embodiment.

[0055] <Detection Unit 120b> Similar to the detection unit 120 of the first embodiment, the detection unit 120b performs vision detection on an image of the area where one cardboard box 50b is located, captured by the imaging device 40, calculates the position, posture, and shape information of the cardboard box 50b based on the amount of change in image data including the detection results, and outputs the detection results to the robot control device 20. However, because the cardboard box 50b has tape and labels such as address labels attached, the detection unit 120b may erroneously detect the tape area (hereinafter also referred to as the "tape area"), the label area (hereinafter also referred to as the "label area"), and the cardboard area (hereinafter also referred to as the "plain area") as three workpieces with different textures. In this case, the detection unit 120b outputs a signal to the robot control device 20 requesting confirmation that the three different textures belong to the same workpiece. The robot control device 20 executes an auxiliary operation (first operation) in which the hand 31 of the robot 30 touches and makes contact with the cardboard box 50b and moves the hand 31 along the surface of the cardboard box 50b while maintaining contact with the cardboard box 50b with a constant force (e.g., 5 N). At this time, the tactile / force sensor 33 mounted on the proximal end of the hand 31 measures the waveform of the contact force due to contact with the surface of the cardboard box 50b. FIG. 13 shows an example of the waveform of the contact force due to contact with the surface of the cardboard box 50b measured by the tactile / force sensor 33. The top part of FIG. 13 shows an example of the tape area, label area, and plain area on the surface of the cardboard box 50b. The middle part of FIG. 13 shows an example of an auxiliary operation. The bottom part of FIG. 13 shows an example of a graph of the contact force waveform measured by the tactile / force sensor 33. As shown in FIG. 13 , even when the same predetermined force is applied, the pattern of change in the contact force (amplitude, frequency, etc.) differs depending on the texture of the material of the tape area, label area, and plain area. The detection unit 120b calculates the amplitude and frequency of the waveform from the measured contact force waveform (contact information) and identifies the tape area, label area, and plain area. The detection unit 120b corrects the detection result by removing the adverse effects of the tape and label, and can obtain a correct vision detection result for a single cardboard box 50b with tape and labels attached.The detection unit 120b then outputs the corrected detection result to the robot control device 20. Note that the contact information may include information on the presence or absence of contact, information on the contact position, information on the magnitude and direction of the contact moment, and information on slippage, in addition to a force waveform as information on the magnitude, direction, and amount of change in distribution of the contact force.

[0056] <Detection Process of Vision System 1> Next, the flow of the detection process of the vision system 1 will be described with reference to Fig. 14. Fig. 14 is a flowchart illustrating the detection process of the vision system 1. The flow shown here is repeatedly executed each time an image of the cardboard box 50b is captured by the imaging device 40. Note that the processes of steps S51 to S53, steps S55 to S57, and step S61 are similar to the processes of steps S31 to S33, steps S35 to S37, and step S40 in the second embodiment, and therefore will not be described again.

[0057] In step S54, the vision detection device 10b (detection unit 120b) determines whether multiple different textures were detected by the vision detection in step S53. If multiple different textures were detected, the vision detection device 10b (detection unit 120b) outputs a signal to the robot control device 20 requesting confirmation of whether the multiple different textures belong to the same workpiece, and the process proceeds to step S55. On the other hand, if multiple different textures were not detected, the process proceeds to step S60.

[0058] In step S58, the vision detection device 10b (detection unit 120b) calculates the texture characteristics of the top surface of the cardboard box 50b based on the force waveform (contact information) measured by the tactile / force sensor 33 when the hand 31 of the robot 30 contacts the top surface of the cardboard box 50b with a constant force, and identifies the contact area as a tape area, a label area, and a plain area, respectively.

[0059] In step S59, the vision detection device 10b (detection unit 120b) calculates the final detection result for the cardboard box 50b as one workpiece, based on the contact area identified in step S58.

[0060] In step S60, the vision detection device 10b (detection unit 120b) outputs the final detection result to the robot control device 20.

[0061] As described above, in the case of a cardboard box 50b to which labels such as address labels are attached together with tape, the vision system 1 according to the third embodiment executes an auxiliary operation of touching the hand 31 of the robot 30 to the cardboard box 50b with a certain force, causes the tactile / force sensor 33 mounted at the tip of the hand 31 to measure the force waveform on the top surface of the cardboard box 50b, calculates the texture characteristics of the cardboard box 50b based on the contact information of the measured force waveform, and identifies the contact area as a tape area, a label area, and a plain area, thereby correcting the detection result of the cardboard box 50b as a single workpiece. This allows the vision system 1 to accurately determine the pick-up position of the workpiece to be picked up by the robot based on the detection results of the image vision detection. That is, the vision system 1 applies an auxiliary motion to make the robot 30 come into contact with the cardboard box 50b, calculates a force change pattern from the measurement information of the tactile / force sensor 33 in the contact area when the robot 30 comes into contact with the cardboard box 50b, extracts texture features, and divides the contact area into a tape area, a label area, and a plain area based on the extracted features. This allows the vision system 1 to obtain accurate vision detection results that eliminate the adverse effects of tape and labels. The third embodiment has been described above.

[0062] <Modification of the Third Embodiment> In the third embodiment, the workpiece is a cardboard box 50b to which a label such as an address label is attached along with tape. However, this is not limiting. For example, the workpiece may have a groove or the like formed on its upper surface. In this case, the detection unit 120b may also cause the robot control device 20 to execute an auxiliary operation (first operation) in which the hand 31 of the robot 30 touches and contacts the grooved upper surface of the workpiece and moves the hand 31 along the upper surface while maintaining contact with the workpiece with a constant force (e.g., 5 N). At this time, the tactile / force sensor 33 mounted on the tip of the hand 31 measures a force waveform resulting from contact with the upper surface of the workpiece. FIG. 15 is a diagram illustrating an example of a force waveform resulting from contact with the upper surface of the workpiece measured by the tactile / force sensor 33. The upper part of FIG. 15 illustrates an example of an auxiliary operation. The lower part of FIG. 15 illustrates an example of a force waveform measured by the tactile / force sensor 33. As shown in Figure 15, when the hand 31 of the robot 30 moves toward a groove in the workpiece, the magnitude of the force measured by the tactile / force sensor 33 drops, and then returns to normal after passing through the groove. The detection unit 120b can determine from the force change point that a groove exists at a specific position on the workpiece surface based on the contact information of the force waveform in Figure 15. That is, the vision system 1 calculates multiple force change points from the contact information, extracts features such as grooves, edges, steps, holes, and protrusions, detects gaps between the workpieces, and calculates detection positions that do not include features that may cause air leaks, thereby obtaining accurate detection results.

[0063] The first, second, and third embodiments have been described above, but the vision system 1 is not limited to the above-mentioned embodiments and includes modifications, improvements, etc. within the scope that can achieve the purpose.

[0064] <Modifications> In the first, second, and third embodiments, the vision detection device 10 is a device separate from the robot control device 20, but this is not limiting. For example, the vision detection device 10 may be implemented inside the robot control device 20 and integrated with the robot control device 20.

[0065] Note that the functions included in the vision system 1 in the first, second, and third embodiments can be realized by hardware, software, or a combination of these. Here, "realized by software" means that the functions are realized by a computer reading and executing a program.

[0066] The program can be stored and supplied to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs). The program may be provided to the computer by various types of transient computer-readable media. Examples of transient computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transient computer-readable media can provide the program to the computer via a wired communication path such as an electrical wire or optical fiber, or via a wireless communication path.

[0067] In addition, the steps of writing a program to be recorded on a recording medium include not only processes that are performed chronologically in accordance with the order, but also processes that are not necessarily performed chronologically but are performed in parallel or individually.

[0068] In other words, the vision system and vision detection method of the present disclosure can take various forms having the following configurations.

[0069] (1) The vision system 1 of the present disclosure includes an acquisition unit 110 that acquires a first image capturing an area where multiple workpieces 50 are present, a detection unit 120 that outputs a detection result for the multiple workpieces 50, and a robot 30 that executes an operation to change the position of at least one of the multiple workpieces 50, wherein the robot 30 executes the first operation to change the position of the multiple workpieces 50, the acquisition unit 110 acquires a second image capturing an area where multiple workpieces 50 are present after the first operation is executed, and the detection unit 120 performs detection on the first image and the second image, and outputs at least one of position, posture, and outer shape information of the multiple workpieces 50 as a detection result based at least on the amount of change between the first image data including the detection result and the second image data including the detection result. According to this vision system 1, the pick-up position of a workpiece to be picked up by the robot can be accurately determined based on the detection result obtained by vision detection of the image.

[0070] (2) In the vision system 1 described in (1), the detection unit 120 may output detection results for the multiple workpieces 50 based on the difference in at least one of the position, posture, and external shape information of the multiple workpieces 50 contained in the first image data and the second image data.

[0071] (3) In the vision system 1 described in (1) or (2), the robot 30 may perform a second operation different from the first operation of changing the positions of the plurality of workpieces 50, the acquisition unit 110 may acquire a third image of the plurality of workpieces 50 after the second operation has been performed, and the detection unit 120 may perform detection on the third image and output the detection result based on the amount of change between at least two of the first image data, the second image data, and the third image data including the detection result.

[0072] (4) In the vision system 1 described in any one of (1) to (3), the robot 30 is provided with a force sensor 32 (tactile / force sensor 33) that measures contact information when it comes into contact with multiple workpieces 50, and the robot 30 performs a contact operation with the surfaces of the multiple workpieces 50 based on the detection results of at least one of the first image and the second image, and the force sensor 32 (tactile / force sensor 33) may measure the contact information of the surface.

[0073] (5) In the vision system 1 described in (4), the robot 30 may perform at least a first action on the plurality of workpieces 50a based on the contact information.

[0074] (6) In the vision system 1 described in (4) or (5), the contact information may include at least one of information regarding the presence or absence of contact, information regarding the contact position, information regarding the magnitude, direction, and distribution of the contact force, information regarding the magnitude and direction of the contact moment, and information regarding slippage.

[0075] (7) In the vision system 1 according to any one of (4) to (6), the detectors 120a and 120b may correct the detection results based on at least information about the contact position among the contact information.

[0076] (8) In the vision system 1 described in (6), the detection unit 120a may correct at least one of the first image, the second image, or the detection result based on information regarding the magnitude and direction or amount of change in distribution of the contact force.

[0077] (9) The vision system 1 according to any one of (1) to (8) may further include a display unit 21 that displays the detection results.

[0078] (10) In the vision system 1 described in (9), the display unit 21 may display the first image or the second image acquired by the acquisition unit 110 together with the detection result.

[0079] (11) The display unit may be a display that the user can operate using, for example, a touch panel. A user interface may be displayed on the display unit, and the user may specify the position, magnitude and direction of the force, number of times, and conditions for the robot to perform the first or second motion based on the first image or the second image displayed on the user interface. The user may also specify, via the user interface, the position for the robot to perform the contact motion, the magnitude and direction of the contact force, the path along which the robot will move while making contact, or the area to be contacted. The user interface may include a setting screen on which parameters related to the robot's motion conditions or the imaging conditions of the imaging device can be set. The user interface may also display the first image and the second image (and the third image) superimposed or side-by-side. The user interface may also display detection results together with the first image, etc. The user interface may also display detection information detected by the force measurement unit when the robot is performing the contact motion.

[0080] (12) The vision detection method of the present disclosure includes an acquisition step of acquiring a first image in which an area where multiple workpieces 50 are present, a detection step of outputting a detection result of the multiple workpieces 50, and an execution step of having the robot 30 perform an operation to change the position of at least one of the multiple workpieces 50, wherein the execution step executes the first operation to change the position of the multiple workpieces 50, the acquisition step acquires a second image in which an area where multiple workpieces 50 are present after the first operation is performed, and the detection step performs detection on the first image and the second image, and outputs at least one of position, posture, and outer shape information of the multiple workpieces 50 as the detection result based at least on an amount of change between the first image data including the detection result and the second image data including the detection result. This vision detection method can achieve the same effect as (1).

[0081] REFERENCE SIGNS LIST 1 Vision system 10, 10a, 10b Vision detection device 100, 100a, 100b Control unit 110 Acquisition unit 120, 120a, 120b Detection unit 200 Memory unit 20 Robot control device 21 Display unit 25 Teaching operation panel 30 Robot 31 Pick-up hand 32 Force sensor 33 Tactile / force sensor 40 Imaging device 50 Cardboard 60 Pallet

Claims

1. An acquisition unit that acquires a first image in which an area where a plurality of workpieces are present is captured; A detection unit that outputs detection results of the plurality of workpieces; a robot that performs an operation of changing the position of at least one of the plurality of workpieces; The robot performs a first operation of changing positions of the plurality of workpieces; The acquisition unit acquires a second image in which an area in which the plurality of workpieces are present is imaged after the first operation is performed, The detection unit performs detection on the first image and the second image, and outputs at least one of position, posture, and outer shape information of the plurality of workpieces as the detection result based at least on an amount of change between first image data including a detection result and second image data including a detection result. Vision system.

2. The vision system of claim 1 , wherein the detection unit outputs the detection results of the multiple workpieces based on a difference in at least one of position, posture, and external shape information of the multiple workpieces contained in the first image data and the second image data.

3. The robot performs a second operation different from the first operation to change positions of the plurality of workpieces; The acquisition unit acquires a third image in which a plurality of workpieces are imaged after the second operation is performed, The vision system of claim 1 or claim 2, wherein the detection unit performs detection on the third image and outputs the detection result based on the amount of change between at least any two of the first image data, the second image data, and the third image data including the detection result.

4. The robot includes a force measuring unit that measures contact information when the robot comes into contact with the plurality of workpieces, The robot performs a contact operation with surfaces of the plurality of workpieces based on the detection result of at least one of the first image and the second image; The vision system according to claim 1 , wherein the force measuring unit measures the contact information of the surface.

5. The vision system of claim 4 , wherein the robot performs at least the first action on the plurality of workpieces based on the contact information.

6. The vision system of claim 4 , wherein the contact information includes at least one of information regarding the presence or absence of contact, information regarding the contact position, information regarding the magnitude, direction, and distribution of the contact force, information regarding the magnitude and direction of the contact moment, and information regarding slippage.

7. The vision system according to claim 4 , wherein the detection unit corrects the detection result based on at least information related to a contact position among the contact information.

8. The vision system of claim 6 , wherein the detection unit corrects at least one of the first image, the second image, or the detection result based on information regarding a change in magnitude and direction or distribution of the contact force.

9. The vision system according to claim 1 , further comprising a display unit that displays the detection result.

10. The vision system according to claim 9 , wherein the display unit displays the first image or the second image acquired by the acquisition unit together with the detection result.

11. An acquisition step of acquiring a first image in which an area where a plurality of workpieces are present is captured; A detection step of outputting detection results of the plurality of workpieces; and an execution step of causing the robot to execute an operation of changing the position of at least one of the plurality of workpieces, The execution step executes a first operation for causing the robot to change positions of the plurality of workpieces; The acquiring step acquires a second image in which an area in which the plurality of workpieces are present is imaged after the first operation is executed, The detection step performs detection on the first image and the second image, and outputs at least one of position, posture, and outer shape information of the plurality of workpieces as the detection result based at least on an amount of change between first image data including a detection result and second image data including a detection result. Vision detection methods.