Control device, three-dimensional position measuring system and program

Abstract

This control device includes: a combination generation unit that generates a plurality of combinations in which three or more detection targets are selected from three or more detection targets that are located on a workpiece and have a known positional relationship to each other and that are detected based on an image captured by a visual sensor; a selection unit that selects one or more combinations from the plurality of combinations based on an index representing a positional deviation of the detected positions of the three or more detection targets from an ideal position, the positional deviation being calculated for each combination of the generated plurality of combinations; and a three-dimensional position determination unit that determines the three-dimensional position of the workpiece from the selected one or more combinations.

Description

Technical area

[0001] The present invention relates to a control device, a three-dimensional position measuring system and a program. Related prior art

[0002] Various measurement systems for measuring the three-dimensional position of a three-dimensional object using a visual sensor have been proposed. For example, Patent References 1 and 2 describe a method for detecting three detection targets that have a known mutual positional relationship on a three-dimensional object using three cameras, and measuring the three-dimensional position of the three-dimensional object based on the detected positions of the three detection targets.

[0003] Regarding three-dimensional position measurement, Patent 3 describes a method for creating a three-dimensional model used for three-dimensional recognition processing using a stereo camera. Patent 4 describes an example of a method for performing three-dimensional measurement of the position and posture of an article transported by a conveyor. [Citation list][Patent literature] [PTL 1] Unexamined Japanese Patent Publication (Kokai) No. H07-13613 A [PTL 2] Unexamined Japanese Patent Publication (Kokai) No. S62-54115 A [PTL 3] Unexamined Japanese Patent Publication (Kokai) No. 2010-121999 A [PTL 4] Unexamined Japanese Patent Publication (Kokai) No. 2019-128274 A Summary[Technical problem]

[0004] In the case of a configuration for detecting three detection targets on a three-dimensional object and obtaining a three-dimensional position of the three-dimensional object from the detected positions of the three detection targets, as described in PTLs 1 and 2, a detection result may include an error in a position of the detection target itself and a measurement error in a detected position of the detection target. Thus, sufficient accuracy may not be obtained from the measurement of the three-dimensional object by the detected positions of the three detection targets. In addition, if the error of a part of the three detection targets is large, the total error, that is, the result of measuring the three-dimensional position of the three-dimensional object, may be equally large due to the error of the part of the three detection targets.

[0005] A technique is desired that can reduce the influence of an error that may be included in a detected position of a detection target, and thus enables improved accuracy of measuring a three-dimensional position of a three-dimensional object. (Solution to the problem)

[0006] One embodiment of the present disclosure is a control device comprising: a combination generation unit configured to generate a plurality of combinations, each of which is obtained by selecting three or more detection targets from detection targets detected based on an image in which three or more detection targets present on a workpiece and having a known positional relationship to each other are captured by a visual sensor; a selection unit configured to select one or more combinations from the plurality of combinations based on an indicator representing a positional deviation of the detected positions of the three or more detection targets from an ideal position, the indicator being calculated for each of the plurality of generated combinations;and a three-dimensional position determining unit configured to determine a three-dimensional position of the workpiece from the one or more selected combinations;

[0007] The objects, features and advantages, as well as other objects, features and advantages, will become more apparent from the detailed description of typical embodiments of the present invention illustrated in the accompanying drawings. Brief description of the drawings

[0008] They show: Fig. 1 is a diagram for describing a configuration of a robot system including a robot control device according to an embodiment; Fig. 2 is a diagram showing a car body and a detection target as a concrete example of a workpiece; Fig. 3 is a diagram showing a vision coordinate system and a sensor coordinate system assigned to each reference point located at a zero-deviation location on a workpiece; Fig. 4 is a diagram showing a projection of the sensor coordinate system and a reference point onto an image plane; Fig. 5 is a functional block diagram of the robot control device and an image processing device; and Fig. 6 a flowchart describing a basic process for processing three-dimensional position measurements. Description of implementation examples

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. A similar configured portion or a similar functional portion is denoted by the same reference numeral throughout the drawings. To facilitate understanding, the scale of the drawings is changed accordingly. An embodiment illustrated in the drawing is an example of the implementation of the present invention, but the present invention is not limited to the illustrated embodiment.

[0010] Fig. 1 is a diagram showing a configuration of the robot system 100 including the robot control device 50 according to an embodiment. As shown in Fig. 1, the robot system 100 includes the robot 10, the visual sensor 70 mounted on a portion of the tip of the robot 10, the robot controller 50 that controls the robot 10, the teach pendant 40, and the image processing device 20. The teach pendant 40 and the image processing device 20 are connected to the robot controller 50. The visual sensor 70 is connected to the image processing device 20. The robot system 100 according to the present embodiment is configured as a three-dimensional position measuring system capable of highly accurately measuring a three-dimensional position of a workpiece W, which is a three-dimensional object placed on the table 1 (e.g., a carriage on a conveyor and a rack), by detecting three or more detection targets on the workpiece W.

[0011] Assume that robot 10 is a vertical articulated-arm robot. Note that other robot types, such as a horizontal articulated-arm robot, a parallel-joint robot, or a dual-arm robot, can also be used as robot 10 depending on the work objective. Robot 10 can perform the desired work with an end effector attached to a wrist portion. The end effector is an external device that can be replaced depending on the use, e.g., a hand, a welding gun, a tool, and the like. Fig. 1 shows an example in which the hand 33 is used as an end effector.

[0012] The robot control device 50 controls a movement of the robot 10 according to a movement program or a command of the teaching pendant 40. The robot control device 50 may have a hardware configuration as a general computer, including a processor 51 ( Fig. 5), a memory (e.g. ROM, RAM and non-volatile memory), a storage unit, an operation unit, an input / output interface, a network interface and the like.

[0013] The image processing device 20 has a function of controlling the visual sensor 70 and a function of executing image processing, including target object detection processing, and the like. The image processing device 20 may have a hardware configuration as a general computer, including a processor, a memory (such as ROM, RAM, and non-volatile memory), a storage unit, an operation unit, a display unit, an input / output interface, a network interface, and the like.

[0014] It should be noted that Fig. 1 shows a configuration example in which the image processing device having the functions of controlling the visual sensor 70 and image processing is provided as an independent device in the robot system 100, but the functions of the image processing device 20 may be integrated into the robot control device 50.

[0015] The teaching pendant 40 serves as an operating terminal for teaching and various settings of the robot 10. A teaching device formed from a tablet computer or the like can be used as the teaching pendant 40. The teaching pendant 40 may have a hardware configuration as a general computer, including a processor, a memory (such as ROM, RAM, and non-volatile memory), a storage unit, an operating unit, a display unit 41 ( Fig. 5), an input / output interface, a network interface, and the like.

[0016] The workpiece W, which serves as the target of the three-dimensional position measurement, is, for example, a car body, as shown in Fig. 2. The workpiece W is provided with three or more detection targets (e.g., a circular hole M) located at positions with a known positional relationship to each other. The detection target is arranged, for example, on the underside of a car body. The robot system 100 calculates a three-dimensional position of the entire workpiece W by detecting the positions of the three or more detection targets through the vision sensor 70. The robot system 100 can obtain a three-dimensional position of the workpiece W and perform various types of work on the workpiece W.

[0017] Fig. 1 shows a configuration example for a case where the visual sensor 70 is mounted on the tip of the robot 10. In this configuration, the visual sensor 70 is moved by the robot 10, and the visual sensor 70 is positioned at each pickup position to pick up the detection target (circular hole M), and detects and picks up the detection target. A pickup position in which each detection target of a workpiece W located at a reference position can be picked up can be taught to the robot 10 in advance.

[0018] Instead of the configuration in which the visual sensor is mounted on the robot 10, a configuration may be adopted in which a detection target is detected by one or more visual sensors fixedly arranged in a work area. In this case, a plurality of visual sensors, each of which detects a plurality of detection targets on a workpiece, may be arranged. Alternatively, a single visual sensor may be arranged at a position where the visual sensor detects two or more detection targets. In the latter case, the number of arranged visual sensors may be smaller than the total number of detection targets.

[0019] A recording position (pose) of a detection target by the visual sensor 70 follows the constraint that the image planes should not be equally planar to each other in each recording position (pose) of the visual sensor. It should be noted that normal vectors also preferably form a significant angle with respect to any image plane.

[0020] The robot system 100 (robot controller 50) detects the positions of three or more detection targets on the workpiece W and obtains a three-dimensional position of the workpiece W based on the detected positions. Next, a technique for detecting the positions of three detection targets on a workpiece will be described, which is a basic detection technique, and then a technique for extending the technique to four or more detection targets. Next, the determination of the three-dimensional position of a three-dimensional object based on the detected positions of the three or more detection targets will be described.

[0021] The technique for detecting three detection targets on the workpiece W and obtaining a three-dimensional position of the workpiece is described below. A "position detection function" for detecting positions of three detection targets on the workpiece W can be implemented as a function of the image processing unit (detection unit) 121 ( Fig. 5) of the image processing device 20. As in Fig. 3, the workpiece W can be considered as a rigid body having three known points (detection targets). When the workpiece W is located at a location with zero deviation, that is, at an ideal target position, a vision coordinate system (hereinafter also referred to as VCS) is considered, which is a local coordinate system located on the workpiece W or the origin near the workpiece W. Three orthogonal vectors are provided perpendicular to each other with a point as the starting point associated with each detection target (herein also described as reference points), which are located at locations providing zero deviation, a magnitude of the vectors is a unit length, and their direction is parallel to a direction of three vectors in the vision coordinate system VCS.Small coordinate systems, generated at each point by three unit vectors, are called sensor coordinate systems 1, 2, and 3 (also referred to as SCS1, SCS2, and SCS3). The transformation of the three sensor coordinate systems is invariant.

[0022] Assume that the visual coordinate system VCS has a fixed relationship to a pickup position (pose) of the visual sensor 70. A coordinate system fixed on the workpiece W is called a workpiece coordinate system (also referred to as BCS). When the workpiece W is located at a zero-deviation location, each reference point corresponds exactly to the origin of the three sensor coordinate systems.

[0023] When the workpiece W moves from the zero-deviation position, the rigid body motion experienced by the workpiece W is completely determined by the transformation [T] relating the VCS to the BCS. The transformation is a transformation defined with respect to the VCS, and the position and azimuthal angle of the BCS, and thus the position of the workpiece W, are completely determined by the transformation.

[0024] When a zero-deviation position coordinate of a reference point in the VCS and a position coordinate assumed when the reference point is shifted are provided, the zero-deviation position coordinate and the shifted position coordinate are directly related to each other by the transformation [T]. Being able to determine the transformation [T] by detecting a reference point in a field of view of the visual sensor 70 at each shooting position is one purpose of the three-dimensional positioning function described below.

[0025] When the workpiece W is located at a location that has a slight deviation, a reference point on an image plane shifts to a position away from the origin of the SCS coordinate system. Fig. Figure 4 shows a projection of the SCS1 coordinate system and a reference point P1 onto the image plane in this case. In general, by combining three or more projections onto the image plane with calibration data, six degrees of freedom in the deviation from a nominal position of a three-dimensional object can be determined. Assuming that point P1 is located on an XY plane of the SCS1 coordinate system, the position of point P1 can be obtained from independent images taken at any camera position. The vectors A, B, and P are defined as follows: u is a horizontal axis on the image plane, and v is a vertical axis on the image plane. u and v with roof are unit vectors in a horizontal axis direction and a vertical axis direction of the image plane, respectively. [Mathematical Expression 1] P=P1−O1=(uP1−UO1)u^+(vP1−vO1)v^A=A1−O1=(uA1−UO1)u^+(vA1−vO1)v^B=B1−O1=(uB1−UO1)u^+(vB1−vO1)v^ P=(uP1O1)u^+(vP1O1)v^A=(uA1O1)u^+(vA1O1)v^B=(uB1O1)u^+(vB1O1)v^ O1=(uO1)u^+(vO1)v^P1=(uP1)u^+(vP1)v^A1=(uA1)u^+(vA1)v^B1=(uB1)u^+(vB1)v^

[0026] Vector A and vector B are projections of a unit vector in the X direction and a unit vector in the Y direction in the SCS1 coordinate system onto the image plane. An X coordinate and a Y coordinate (i.e., x1 and y1) of point P1 are given by equations (1) to (4) described below. [Mathematical Expression 2] P=x1A+y1B P=(uP1O1)u^+(vP1O1)v^ P=x1[(uA1O1)u^+(vP1O1)v^] +y1[(uB1O1)u^+(vB1O1)v^] uP1O1=x1uA1O1+y1uB1O1vP1O1=x1vA1O1+y1vB1O1} x1=|uP1O1uB1O1vP1O1vB1O1||uA1O1uB1O1vA1O1vB1O1| y1=|uA1O1uP1O1vA1O1vP1O1||uA1O1uB1O1vA1O1vB1O1|

[0027] With reference to Fig. 4, a more general case is represented by equations (5) to (9). In this case, it is assumed that z can take any value. [Mathematical Expression 3] O1'=O1+z1(C1−O1) =(uO+z1uco)u^+(vO+z1vCO)v^ P'=P1−O1 =(uP−uO−z1uCO)u^ +(vP−vO−z1vCO)v^ A=A1−O1=(uAO)u^+(vAO)v^ B=B1−O1=(uBO)u^+(vBO)v^ P'=x1A+y1B

[0028] From equations (5) to (9), equations (10) to (11) are obtained, and a solution is obtained in which x1 and y1 are represented by z1. [Mathematical Expression 4] (uP−uO−z1uCO)u^+(vP−vO−z1vCO)v^ =x1[(uAO)u^+(vAO)v^] +y1[(uBO)u^+(vBO)v^] uPO−z1uCO=x1uAO+y1uBOuPO−z1vCO=x1vAO+y1vBO x1=|(uPO−z1uCO)uBO(vPO−z1vCO)vBO||uAOuBOvAOvBO| y1=|uAO(uPO−z1uCO)vAO(vPO−z1vCO)||uAOuBOvAOvBO|

[0029] x1 and y1 described above are transformed as follows. [Mathematical Expression 5] x1=α1z1+β1y1=γ1z1+δ1}

[0030] Where α1, β1, γ1 and δ1 are constants resulting from the following equations. [Mathematical Expression 6] α1=−|uCOuBOvCOvBO||uAOuBOvAOvBO|β1=−|uPOuBOvPOvBO||uAOuBOvAOvBO|γ1=−|uAOuCOvAOvCO||uAOuBOvAOvBO|δ1=−|uAOuPOvAOvPO||uAOuBOvAOvBO|

[0031] Equation (12) states that both x1 and y1 are linear functions of z1. A similar equation is derived for the other two image planes. A complete set of equations is provided by equations (13) to (15). Various constants appearing in equations (13) to (15) can be obtained by calibration with a calibration template. For example, a cube with ridge lines and scales corresponding to mutually orthogonal coordinate axes in the SCS coordinate system is positioned such that the three ridge lines are parallel to the mutually orthogonal coordinate axes in the SCS coordinate system.Then, the cube is captured by the visual sensor 70 in a position and posture in which the reference point (SCS coordinate system) is captured, and information (calibration data) can be obtained about what type of vector the unit vectors of the X, Y, and Z axes in the SCS coordinate system correspond to in an image by using information about the actual dimensions of the cube. Such calibration data is stored in advance in the storage unit 122 (. Fig. 5) the image processing device 20, etc. [Mathematical Expression 7] x1=α1z1+β1y1=γ1z1+δ1 x2=α2z2+β2y2=γ2z2+δ2 x3=α3z3+β3y3=γ3z3+δ3

[0032] Equations (13) to (15) are six linear equations with nine unknowns. To solve the equations, the fact that a workpiece is a rigid body is considered as an additional boundary condition. In other words, a condition is used here that a distance between the reference points on a workpiece is fixed. The origins in each SCS coordinate system are denoted as (X O1 , Y O1 , Z O1 ), (X O2 , Y O2 , Z O2 ) and (X O3 , Y O3 , Z O3 ), and the origins of each reference point after translation are represented as P1(X1, Y1, Z1), P2(X2, Y2, Z2), and P3(X3, Y3, Z3). A distance between the origins in the three SCS coordinate systems is represented as follows, and a distance between the reference points after translation is given as Equation (16). [Mathematical Expression 8] d12=|O1O2¯|=|P1P2¯|d23=|O2O3¯|=|P2P3¯|d31=|O3O1¯|=|P3P1¯| d12=[(xO1−xO2)2+(yO1−yO2)2+(zO1−zO2)2]12d12=[xO122+yO122+zO122]12d23=[xO232+yO232+zO232]12d31=[xO312+yO312+zO312]12 [Mathematical Expression 9] d122=(xO12+x1−x2)2+(yO12+y1−y2)2 +(zO12+z1−z2)2d232=(xO23+x2−x3)2+(yO23+y2−y3)2 +(zO23+z2−z3)2d312=(xO31+x3−x1)2+(yO31+y3−y1)2 +(zO31+z3−z1)2

[0033] When equations (13) to (15) are replaced by equation (16), a first group (equation (17)) of the following equations is obtained. Furthermore, a second group (equation (18)) of equations is obtained by rearranging the equations. k, l, and m in the equations are constants. [Mathematical Expression 10] (xO12+α1z1+β1−α2z2−β2)2 +(yO12+γ1z1+δ1−γ2z2−δ2)2 +(zO12+z1−z2)2=d122(xO23+α2z2+β2−α3z3−β3)2 +(yO23+γ2z2+δ2−γ3z3−δ3)2 +(zO23+z2−z3)2=d232(xO31+α3z3+β3−α1z1−β1)2 +(yO31+γ3z3+δ3−γ1z1−δ1)2 +(zO31+z3−z1)2=d312 [Mathematical Expression 11] k11z12+k10z1+k20z2+k12z1z2+k22z22+k00=0l22z22+l20z2+l30z3+l23z2z3+l33z32+l00=0m33z32+m30z3+m10z1+m31z3z1+m11z12+m00=0

[0034] The second group (equation (18)) of the equations described above is solved, for example, using an iterative method according to Newton's method. Obtaining the values ​​of x1, x2, x3, and y1, y2, y3 is obtained by substituting the values ​​for equations (13) to (15). (x1, y1, z1), (x2, y2, z2), and (x3, y3, z3) thus obtained are positions on each of the SCS coordinate systems after the displacement of each reference point. The positions can be transformed into values ​​on the VSC. In this way, the transformation [T] relating the VCS to the BCS can be obtained. In other words, a three-dimensional position of the workpiece W after the displacement is obtained.

[0035] It should be noted that the technique described above assumes an orthogonal projection of three reference points onto an image plane. If the actual projection is assumed to be similar to a perspective projection, processing can be performed to compensate for an error in a mapping relationship. To compensate for the error, a mapping relationship is established between an actual coordinate axis and each projection axis of the actual coordinate axis, which is described by the equation below. The mapping relationship with respect to each axis can be obtained by measuring three or more points on each coordinate axis at the time of calibration and using an interpolation method to obtain a necessary relationship. [Mathematical Expression 12] x1'=f11(x1),y1'=f12(y1),z1'=f13(z1)x2'=f21(x2),y2'=f22(y2),z2'=f23(z2)x3'=f31(x3),y3'=f32(y3),z3'=f33(z3) λ1=x1x1'μ1=y1y1'η1=z1z1'λ2=x2x2'μ2=y2y2'η2=z2z2'λ3=x3x3'μ3=y3y3'η3=z3z3'

[0036] Das Ergebnis der Berechnung eines neuen Skalierungsfaktor ist nachstehend beschrieben. [Mathematical expression 13] α1'=(η1λ1)α1, β1'=(1λ1)β1, γ1'=(η1λ1)γ1, δ1'=(1μ1)δ1α2'=(η2λ2)α2, β2'=(1λ2)β2, γ2'=(η2λ2)γ2, δ2'=(1μ2)δ2α3'=(η3λ3)α3, β3'=(1λ3)β3, γ3'=(η3λ3)γ3, δ3'=(1μ3)δ3

[0037] The calculation technique described above calculates a positional deviation of three points using a least-squares method. At the end of the least-squares calculation, the nonlinearity is compensated by multiplying a projection of a coordinate axis by a new scaling factor. Using the new scaling factor provided by three sets of equations, each of the constants α, β, γ, and δ described above is recalculated for each image plane. Then, the least-squares calculation is performed again.

[0038] The following describes a method in which the above-described calculation method is extended to a case where four or more reference points are used. Here, a case where four reference points are used is described. The position of each of the four reference points is obtained by establishing the above-described equations (1) to (4) with respect to the four reference points, assuming that each of the reference points is located in the XY plane of the SCS coordinate system as described above.

[0039] In a more general case, similar to the case where equations (13) to (15) are obtained for the three reference points, the x,y coordinates of the four reference points can be represented as linear functions of z, as in equations (19) to (22) described below. [Mathematical Expression 14] x1=α1z1+β1y1=γ1z1+δ1 x2=α2z2+β2y2=γ2z2+δ2 x3=α3z3+β3y3=γ3z3+δ3 x4=α4z4+β4y4=γ4z4+δ4

[0040] Next, the following equations are obtained regarding a distance between the origins in the four SCS coordinate systems and a distance between four measurement points (reference points) based on the fact that the workpiece W is a rigid body and the fact that the distance between the origins in the four SCS coordinate systems and the distance between the four reference points are the same.It should be noted that a solution to equations (19) to (22) is obtained by setting up equations in terms of d12, d23, d34 and d41 as the distance between the origins in the four SCS coordinate systems (distance between the four reference points), but a solution to equations (19) to (22) can also be obtained by setting up further equations in terms of d13 and d24 as the distance between the origins in the four SCS coordinate systems (distance between the four reference points) and by considering these equations. [Mathematical Expression 15] d12=[xO122+yO122+zO122]12d23=[xO232+yO232+zO232]12d34=[xO342+yO342+zO342]12d41=[xO412+yO412+zO412]12 [Mathematical Expression 16] d122=(xO12+x1−x2)2+(yO12+y1−y2)2 +(zO12+z1−z2)2d232=(xO23+x2−x3)2+(yO23+y2−y3)2 +(zO23+z2−z3)2d342=(xO34+x3−x4)2+(yO34+y3−y4)2 +(zO34+z3−z4)2d412=(xO41+x4−x1)2+(yO41+y4−y1)2 +(zO41+z4−z1)2

[0041] By replacing equations (19) to (22) with an equation (23), equations (24) and (25) are obtained, which extend the above-described equations (17) and (18) for the four reference points as follows. [Mathematical Expression 17] (xO12+α1z1+β1−α2z2−β2)2 +(yO12+γ1z1+δ1−γ2z2−δ2)2 +(zO12+z1−z2)2=d122(xO23+α2z2+β2−α3z3−β3)2 +(yO23+γ2z2+δ2−γ3z3−δ3)2 +(zO23+z2−z3)2=d232(xO34+α3z3+β3−α4z4−β4)2 +(yO34+γ3z3+δ3−γ4z4−δ4)2 +(zO34+z3−z4)2=d342(xO41+α4z4+β4−α1z1−β1)2 +(yO41+γ4z4+δ4−γ1z1−δ1)2 +(zO41+z4−z1)2=d412 [Mathematical Expression 18] k11z12+k10z1+k20z2+k12z1z2+k22z22+k00=0l22z22+l20z2+l30z3+l23z2z3+l33z32+l00=0 l33z32+m30z3+m40z4+m34z3z4+m44z42+m00=0n44z42+n40z4+n10z1+n41z4z1+n11z12+n00=0

[0042] This equation is solved by an iterative method similar to the technique described above, and x1, x2, x3, x4, and y1, y2, y3, y4, the positions of the four reference points after translation, can be obtained. Then, a three-dimensional position of the workpiece W is obtained by combining the detected positions of the four reference points. In other words, the transformation [T] that relates the VCS to the BCS is obtained from the detected positions.

[0043] It is also apparent that the calculation method described above can be similarly extended to a case where the measurement is performed for a further increased number of detection targets (reference points).

[0044] It should be noted that various techniques can be used to determine a three-dimensional position of the workpiece W from the detected positions of three or more detection targets (reference points). As an example, various techniques such as the following can be applied. It should be noted that the techniques listed below as examples maintain a condition regarding the arrangement of detection targets (reference points), if any. (1) A technique for obtaining the parameters of the transformation [T] described above (parameters representing translation and rotation) by solving simultaneous equations. (2) A technique for determining a position and a posture of a workpiece by applying a polygon having a known shape (polygon connecting reference points located at a zero deviation position) to a line of sight of a camera with respect to a detected position of each reference point, as described in PTL 4 (Japanese Unexamined Patent Publication (Kokai) No. JP 2019-128274 A). (3) A technique for recognizing a coordinate system on a workpiece by specifying a plane (e.g., an XY plane) of the coordinate system from the positions of three or more reference points on the workpiece. In this case, the coordinate system is recognized, for example, assuming that a first reference point is the origin, a second reference point is a position in the X-axis direction, and a third reference point (and fourth or subsequent reference points) is a position on the XY plane. A calculation unit for obtaining a three-dimensional position of the workpiece W from the detected positions of three or more detection targets (reference points) may be implemented as a function in the selecting unit 153 or the three-dimensional position determining unit 154 in the robot control device 50.

[0045] Fig. 5 shows a functional block diagram of the robot control device 50 and the image processing device 20. As in Fig. As shown in Figure 5, the robot control device 50 includes the motion control unit 151, the combination generation unit 152, the selection unit 153, and the three-dimensional position determination unit 154. Note that the functional blocks can be implemented by the processor 51 of the robot control device 50 executing a program. Furthermore, the robot control device 50 includes a storage unit 155.

[0046] The storage unit 155 is formed, for example, from a non-volatile memory, a hard disk device, or the like. The storage unit 155 stores a motion program that controls the robot 10, a program (vision program) that performs image processing, such as detecting a workpiece based on an image captured by the vision sensor 70, various types of setting information, and the like.

[0047] The motion control unit 151 controls a movement of a robot according to a motion program of the robot. The robot control device 50 includes a servo control unit (not shown) that performs servo control of a servo motor of each axis according to a command for each axis generated by the motion control unit 151. The motion control unit 151 has the function of moving the visual sensor 70 and positioning the position sensor 70 to a pickup position to pick up each detection target.

[0048] The combination generation unit 152 provides a function of generating a plurality of combinations, each of which is obtained by selecting three or more detection targets from the detection targets detected on the workpiece W.

[0049] The selection unit 153 enables a function to select one or more combinations from the plurality of combinations based on a “deviation amount” calculated from each of the plurality of generated combinations.

[0050] The three-dimensional position determination unit 154 determines the three-dimensional position of the workpiece W from one or more combinations of detection targets selected by the selection unit 153. Details of the functions of the combination generation unit 152, the selection unit 153, and the three-dimensional position determination unit 154 will be described below.

[0051] The image processing device 20 includes an image processing unit 121 and a storage unit 122. The storage unit 122 is a storage device formed, for example, from a non-volatile memory. The storage unit 122 stores various types of data required for image processing, such as shape data about a detection target and calibration data. The image processing unit 121 performs various types of image processing, such as detection processing of a workpiece. In other words, the image processing unit 121 has a function as a detection unit that detects a detection target on an image captured by the visual sensor 70 within a detection range including the detection target.

[0052] A three-dimensional measuring function of the workpiece W by the robot control device 50 is described. Fig. 6 shows a flowchart describing a basic process of three-dimensional position measurement controlled by the robot control device 50 (the processor 51).

[0053] First, the image processing unit (detection unit) 121 detects a detection target based on an image in which the detection target was captured by the visual sensor 70 (step S1). At this time, the visual sensor 70 is positioned in a capture position to detect each detection target by the robot 10 and captures an image including the detection target. The image processing unit (detection unit) 121 obtains a position (x, y) of each of the three or more detection targets through the position detection function described above.

[0054] Next, the combination generation unit 152 generates a plurality of combinations, each of which is obtained by selecting three or more detection targets from the detected detection targets (step S2). For example, the combination generation unit 152 may generate all possible combinations from the three or more detected detection targets. In this case, if the number of detected detection targets is five, for example, the number of possible combinations is a total of the number of combinations using all five detection targets, the number of combinations using four of the five detection targets, and the number of combinations using three of the five detection targets.

[0055] Alternatively, the combination generation unit 152 may generate a combination of detection targets according to the following rules. (Rule 1) A capture target to be excluded is selected from the three or more captured capture targets. However, at least three capture targets must remain. (Rule 2) A maximum number of targets to be excluded may be specified. (Rule 3) A minimum number of targets to be retained may be specified.

[0056] When selecting an excluded detection target, a combination of multiple detection targets can be generated by changing one excluded detection target. For example, if eight detection targets are detected and the maximum number of excluded targets is two, (1 + 8 + 8 × 7 ÷ 2 = 37) combinations of detection results can be generated.

[0057] The combination generation unit 152 can be configured to accept an input for "selecting a detection target to be excluded," a "maximum number of detection targets to be excluded," or a "minimum number of detection targets to be left" (an input from an external device or a user input). A user interface for accepting user input can be displayed on the display unit 41 of the teach pendant 40. User input can be provided via the operation unit of the teach pendant 40. The combination generation unit 152 can generate the combinations using set values ​​preset in the robot controller 50 for "selecting a detection target to be excluded," a "maximum number of detection targets to be excluded," or a "minimum number of detection targets to be left."

[0058] In this way, by incorporating a larger number of detection targets into the calculation of a three-dimensional position of the entire object (three-dimensional position of the workpiece W), the influence of an error that may be included in each detection target can be reduced overall and the accuracy of measuring a three-dimensional position can be increased.

[0059] Next, for each of the generated combinations of detection targets, the selection unit 153 calculates a three-dimensional position of the entire object (three-dimensional position of the workpiece W) and an indicator representing a positional deviation of the detected positions of three or more detection targets included in the combination from an ideal position (hereinafter, the indicator is referred to as "positional deviation"). Subsequently, the selection unit 153 selects one or more combinations based on the "positional deviation" (step S3).

[0060] As an example, the selection unit 153 calculates the "positional deviation" as follows. Suppose that a three-dimensional position of the entire object is obtained as position A with respect to a certain combination. By using a design position Pi of an i-th detection target on the workpiece W, an ideal position of the detection target when the three-dimensional position of the workpiece W is position A is obtained as A Pi. Let the number of detection targets in this combination be n. For example, the selection unit 153 may calculate a positional deviation D based on a difference Ki between A Pi and a position P'i after the displacement of the i-th detection target (reference point), which is obtained in the above-described equation (25). For example, the selection unit 153 may obtain the positional deviation D as the average value ΣKi / n of Ki.In this case, the positional deviation D is an indicator of a specific combination that indicates the degree of deviation of the detected position of a detection target included in the combination from an ideal position. Alternatively, the selection unit 153 may calculate the positional deviation D based on a distance Di between a line of sight Li to a detected actual position of the i-th detection target and A·Pi. For example, the selection unit 153 may determine the positional deviation D as an average ΣDi / n of Di. Likewise, in this case, the positional deviation D is an indicator of a specific combination that indicates the degree of deviation of a detected position of a detection target included in the combination from an ideal position.

[0061] The selection unit 153 may select one or more combinations based on the positional deviation D calculated for each of the generated combinations. In this case, the selection unit 153 may select a combination using a criterion that (r1) the smaller the positional deviation D, the higher the accuracy. Therefore, for example, the selection unit 153 may select a predetermined number of combinations having a small value of the positional deviation D, or it may select a combination having the smallest value of the positional deviation D.

[0062] In this way, with the configuration for selecting a combination used for calculating a three-dimensional position of the entire object (three-dimensional position of the workpiece W) based on the positional deviation D, a combination having a high probability of having a large error can be excluded, and the accuracy of measuring a three-dimensional position can be increased.

[0063] Next, the three-dimensional position determination unit 154 determines a final three-dimensional position of the workpiece W from the one or more combinations selected by the selection unit 153 (step S4). If the combination selected by the selection unit 153 is a single combination, the three-dimensional position determination unit 154 may determine the position A of the workpiece W obtained by the combination as the final three-dimensional position of the workpiece W.

[0064] When the plurality of combinations are selected by the selection unit 153, the three-dimensional position determination unit 154 may determine a final three-dimensional position of the workpiece W based on statistics related to a three-dimensional position of the workpiece W, which are again obtained for each of the plurality of combinations. For example, the three-dimensional position determination unit 154 may determine, as the final three-dimensional position of the workpiece W, an average value or a median value of the three-dimensional position of the workpiece W obtained for each of the plurality of selected combinations.

[0065] In this way, according to the embodiment of the three-dimensional position measurement processing, the influence of an error can be reduced and the accuracy of measuring a three-dimensional position of a three-dimensional object can be improved.

[0066] When a combination is selected in step S3 of the three-dimensional position measurement processing described above, the selection unit 153 may also consider the number of detection targets in each of the generated combinations. In this case, the selection unit 153 may make the selection based on selection criteria such that (r1) the accuracy is higher, the smaller the position deviation D is, and (r2) The greater the number of detection targets in a combination, the higher the accuracy. Note that the selection criterion (r2) in this case is based on the fact that with a larger number of detection targets, any error that may be present in each of the detection targets can be rounded off and the overall position measurement accuracy can be increased.

[0067] As an example, assume a situation where there are a plurality of selection candidates for a combination having an excellent (relatively small) positional deviation D. In this case, the selection unit 153 may select one or a plurality of combinations having a large number of detection targets from among the plurality of selection candidates.

[0068] When a combination is generated in step S2 of the three-dimensional position measurement processing described above, the combination generation unit 152 may output, as the generated combination, a specific combination selected from combinations that can be generated from detected detection targets. For example, consider a situation where the number of detection targets detected in step S1 is large. In this case, the number of combinations that can be generated is extremely high. In such a situation, the combination generation unit 152 may output combinations randomly selected from all possible combinations that can be generated. In this case, a combination can be balancedly selected and used from a large number of combination candidates.

[0069] In step S3 of the three-dimensional position measurement processing described above, a situation is considered in which a large number of combinations are selected based on the position deviation D or based on the position deviation D and the number of detection targets. In this case, the number of selected combinations can be limited by repeating the processing in steps S2 to S3 one or more times for the selected combinations. In this case: (1) newly generating a plurality of combinations (a plurality of second combinations) by the combination generating unit 152, each of which is obtained by selecting three or more detection targets based on detection targets included in one or more combinations selected by the selecting unit 153, and (2) Reselecting one or more combinations from the plurality of second combinations by the selecting unit 153 based on an indicator (positional deviation) calculated for each of the plurality of second combinations is performed one or more times.

[0070] For example, if the number of detected detection targets is 20 and a combination is generated according to a rule stating that "the minimum number of detection targets to be left is 10" when the combination generation unit 152 initially generates a combination, it is assumed that the number of combinations selected by the selection unit 153 is a substantial number. In this case, the combination generation unit 152 may, for example, apply the rule "the minimum number of detection targets to be left is 15" to the detection targets included in the combinations selected by the selection unit 153 and generate a plurality of second combinations.In this case, however, the plurality of second combinations are generated by setting the combinations preselected by the selection unit 153 as a population and selecting from the population a combination that satisfies the rule that "a minimum number of detection targets to be left is 15." The selection unit 153 may select one combination from the plurality of second combinations based on the above-described selection criterion (r1) or the above-described selection criteria (r1) and (r2).

[0071] The restriction of the selection by repeatedly generating a combination by the combination generating unit 152 and the selection by the selecting unit 153 can be carried out as follows. The combination generating unit 152 newly generates a plurality of combinations with three or more detection targets by deleting, from one or more combinations selected by the selecting unit 153, one or more detected positions that satisfy the criterion that an indicator (e.g.Ki or Di, described above) representing a positional deviation calculated for a particular detected position is greater than an indicator representing a positional deviation calculated for another detected position, and the combination generation unit 152 performs regeneration one or more times until an indicator representing a positional deviation calculated for a detection target in each of the newly generated combinations has satisfied a predetermined condition. In this case, the predetermined condition may be a condition that an average value of indicators representing positional deviations for detection targets in each of the newly generated combinations or a value of the indicator is less than or equal to a predetermined value.

[0072] In detail, the process can be carried out as follows. (b1) The combination generation unit 152 performs one or more times an operation for regenerating a plurality of combinations including three or more detection targets by deleting, from one or more combinations selected by the selection unit 153, one or more detected positions that satisfy the criterion that “the difference Ki calculated for a certain detected position is greater than the difference Ki calculated for another detected position”, (b2) such that ΣKi / n or Ki for the generated combination is less than or equal to a predetermined value. In (b1) described above, for example, deletion of a predetermined number of detection targets having a large difference Ki with respect to detection targets included in a certain combination may be carried out.

[0073] Alternatively, the selection may be performed by repeatedly generating a combination by the combination generating unit 152 and selecting by the selecting unit 153 as follows. (c1) The combination generation unit 152 performs one or more times an operation for newly generating a plurality of combinations including three or more detection targets by deleting, from one or more combinations selected by the selection unit 153, one or more detected positions that satisfy a criterion that “the distance Di calculated for a certain detected position is greater than the distance Di calculated for another detected position”, (c2) so that ΣDi / n or Di for the generated combination is less than or equal to a predetermined value.

[0074] In (c1) described above, for example, the processing of deleting a predetermined number of detection targets having a large distance Di with respect to detection targets included in a certain combination may be executed.

[0075] With the configuration for repeated selection in this way, appropriate narrowing down of the selection candidates can be performed at high speed, especially in a situation where the number of detection targets is large.

[0076] As described above, according to the present embodiment, the influence of an error that may be included in a detected position of a detection target can be reduced, and the accuracy of measuring a three-dimensional position of a three-dimensional object can thus be improved.

[0077] The functional arrangement in the function block diagram in Fig. 3 is an example, and various modification examples for function distribution in the robot system 100 are possible. For example, a configuration example is possible in which some of the functions in the robot controller 50 are arranged on the teach pendant 40 side.

[0078] The overall function of the teaching pendant 40 and the robot control device 50 can also be referred to as a robot control device.

[0079] The configuration of the robot control device (including the case where the function of the image processing device is integrated) in the above-described embodiment can be applied to control devices of various industrial machines.

[0080] The Fig.The functional blocks of the robot control device and the image processing device shown in Fig. 5 can be realized by executing various types of software stored in a storage device by the processor of the robot control device and the image processing device, or by a configuration in which hardware such as an application-specific integrated circuit (ASIC) is a main component.

[0081] The program for executing various types of processing such as the three-dimensional position measurement in the above-described embodiment can be recorded in various computer-readable recording media (e.g., a ROM, an EEPROM, a semiconductor memory such as a flash memory, a magnetic recording medium, and an optical disk such as a CD-ROM and a DVD-ROM).

[0082] Although the present disclosure has been described in detail above, it is not limited to the individual embodiments described above. Various types of additions, replacements, modifications, partial deletions, and the like can be made to the embodiments without affecting the purpose of the present disclosure or the content described in the claims and the scope of the present disclosure derived from their equivalents. Furthermore, the embodiments can also be implemented in combination. In the embodiments described above, for example, an order of operations and an order of processing steps are indicated, but are not limited thereto. The same applies to the case where a numerical value or a numerical expression is used in the description of the embodiments described above.

[0083] With regard to the embodiments and modification examples described above, additional comments are given below. (Supplementary Note 1)

[0084] A control device (50) comprising: a combination generation unit (152) configured to generate a plurality of combinations, each of which is obtained by selecting three or more detection targets from detection targets detected based on an image in which three or more detection targets present on a workpiece and having a known positional relationship to each other are captured by a visual sensor (70); a selection unit (153) configured to select one or more combinations from the plurality of combinations based on an indicator representing a positional deviation of the detected positions of the three or more detection targets from an ideal position, the indicator being calculated for each of the plurality of generated combinations; and a three-dimensional position determining unit (154) configured to determine a three-dimensional position of the workpiece from the one or more selected combinations. (Supplementary Note 2)

[0085] The control device (50) according to supplementary note 1, wherein the combination generating unit (152) generates all possible combinations from the detected detection targets. (Supplementary Note 3)

[0086] The control device (50) according to supplementary note 1, wherein the combination generating unit (152) generates the plurality of combinations by excluding or selecting a predetermined number of detection targets from the detected detection targets. (Supplementary Note 4)

[0087] The control device (50) according to supplementary note 1, wherein the combination generating unit (152) generates the plurality of combinations by randomly selecting combinations from the combinations that can be generated from the detected detection targets. (Supplementary Note 5)

[0088] The control device (50) according to any one of supplementary notes 1 to 4, wherein, for each of the plurality of generated combinations, the selection unit (153) (1) as A Pi, an ideal position of an i-th detection target on the workpiece is obtained when a three-dimensional position of the workpiece obtained from a single combination is a position A and a design position of the i-th detection target on the workpiece is Pi, and (2) for each detection target in the single combination, a difference Ki between a detected position P'i of the i-th detection target in the single combination and A Pi is obtained, and the indicator is obtained based on the obtained difference Ki. (Supplementary Note 6)

[0089] The control device (50) according to supplementary note 5, wherein the selection unit (153) obtains as the indicator ΣKi / n, which is an average value of the difference Ki when the number of detection targets in the single combination is n. (Supplementary Note 7)

[0090] The control device (50) according to any one of supplementary notes 1 to 4, wherein, for each of the plurality of generated combinations, the selection unit (153) (1) as A Pi, an ideal position of an i-th detection target on the workpiece is obtained when a three-dimensional position of the workpiece obtained from a single combination is a position A and a design position of the i-th detection target on the workpiece is Pi, and (2) for each detection target in the single combination, obtain a distance Di between a line of sight Li to a detected position of the i-th detection target from the visual sensor in the single combination and A Pi, and obtain the indicator based on the obtained distance Di. (Supplementary Note 8)

[0091] The control device (50) according to supplementary note 7, wherein the selection unit (153) obtains as an indicator ΣDi / n, which is an average value of the distance Di when the number of detection targets in the single combination is n. (Supplementary Note 9)

[0092] The control device (50) according to any one of supplementary notes 1 to 8, wherein the selection unit (153) performs the selection of the one or more combinations using a selection criterion in which the accuracy is higher the smaller the size of the indicator is. (Supplementary Note 10)

[0093] The control device (50) according to any one of supplementary notes 1 to 8, wherein the selecting unit (153) selects one or more combinations from the plurality of combinations based on the indicator calculated for each of the plurality of combinations and the number of detection targets in each of the plurality of combinations. (Supplementary Note 11)

[0094] The control device (50) according to supplementary note 10, wherein the selection unit (153) performs the selection of the one or more combinations for each of the plurality of combinations using selection criteria, that (1) the accuracy is higher the smaller the size of the indicator, and (2) the accuracy is higher the larger the number of targets detected in a combination. (Supplementary Note 12)

[0095] The control device (50) according to any one of supplementary notes 1 to 11, wherein the three-dimensional position determining unit (154) determines a three-dimensional position of the workpiece based on statistics of a three-dimensional position of the workpiece obtained from each of the one or more selected combinations. (Supplementary Note 13)

[0096] The control device (50) according to supplementary note 12, wherein the three-dimensional position determining unit (154) determines, as a three-dimensional position of the three-dimensional object, an average value or a median of a three-dimensional position of the workpiece obtained from each of the one or more selected combinations. (Supplementary Note 14)

[0097] The control device (50) according to any one of supplementary notes 1 to 13, wherein the following processes regenerating, by the combination generating unit (152), a plurality of combinations, each of which is obtained by selecting three or more detection targets, based on detection targets included in the one or more combinations selected by the selecting unit (153), and Reselecting one or more combinations from the plurality of newly generated combinations by the selection unit (153) based on the indicator calculated for each of the plurality of newly generated combinations be carried out one or more times. (Supplementary Note 15)

[0098] The control device (50) according to any one of supplementary notes 1 to 4, wherein the combination generation unit (152) newly generates a plurality of combinations, each of which includes three or more detection targets, by deleting, from the one or more combinations selected by the selection unit (153), one or more detected positions that satisfy a criterion that an indicator representing the positional deviation calculated for a particular detected position is greater than an indicator representing the positional deviation calculated for another detected position, and the combination generation unit performs the regeneration one or more times until an indicator representing the positional deviation calculated for the detection target in each of the newly generated combinations satisfies a predetermined condition. (Supplementary Note 16)

[0099] The control device (50) according to supplementary note 15, wherein the predetermined condition is a condition that an average value of indicators representing the positional deviations for the detection targets in each of the newly generated combinations or a value of the indicator is less than or equal to a predetermined value. (Supplementary Note 17)

[0100] A three-dimensional position measuring system (100) comprising: a visual sensor (70); a detection unit (121) configured to detect three or more detection targets located on a workpiece and having a known positional relationship to each other based on an image captured by the visual sensor; a combination generating unit (152) configured to generate a plurality of combinations, each of which is obtained by selecting three or more detection targets from the detected targets; a selection unit (153) configured to select one or more combinations from the plurality of combinations based on an indicator representing a positional deviation of the detected positions of the three or more detection targets from an ideal position, the indicator being calculated for each of the plurality of generated combinations; and a three-dimensional position determining unit (154) configured to determine a three-dimensional position of the workpiece from the one or more selected combinations. (Supplementary Note 18)

[0101] The three-dimensional position measuring system (100) according to Supplementary Note 17, further comprising: a robot (10) to which the visual sensor (70) is attached; and a motion control unit (151) configured to control the robot (10) and to position the visual sensor (70) in a pickup position to detect each of the three or more detection targets. (Supplementary Note 19)

[0102] A program that causes a computer processor to execute: a step of detecting three or more detection targets present on a workpiece and having a known positional relationship to each other based on an image picked up by a visual sensor (70); a step of generating a plurality of combinations each of which is obtained by selecting three or more detection targets from the detected detection targets; a step of selecting one or more combinations from the plurality of combinations based on an indicator representing a positional deviation of the detected positions of the three or more detection targets from an ideal position, the indicator being calculated for each of the plurality of generated combinations; and a step of determining a three-dimensional position of the workpiece from the one or more selected combinations. List of reference symbols 1 table 10 robots 20 Image processing device 33 hands 40 Programming pendant 41 Display unit 50 Robot control device 51 processor 70 Visual Sensor 100 robot system 121 Image processing unit 122 storage unit 151 Motion control unit 152 Combination Generation Unit 153 Selection Unit 154 Determination unit for a three-dimensional position 155 storage unit QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] JP 2010-121999 A

[0003] JP 2019-128274 A [0003, 0044]

Claims

[1] Control device comprising: a combination generation unit configured to generate a plurality of combinations, each of which is obtained by selecting three or more detection targets from the detection targets detected based on an image in which three or more detection targets present on a workpiece and having a known positional relationship to each other are captured by a visual sensor; a selection unit configured to select one or more combinations from the plurality of combinations based on an indicator representing a positional deviation of the detected positions of the three or more detection targets from an ideal position, the indicator being calculated for each of the plurality of generated combinations; and a three-dimensional position determining unit configured to determine a three-dimensional position of the workpiece from the one or more selected combinations. [2] The control device according to claim 1, wherein the combination generating unit generates all possible combinations from the detected detection targets. [3] The control device according to claim 1, wherein the combination generating unit generates the plurality of combinations by excluding or selecting a predetermined number of detection targets from the detected detection targets. [4] The control device according to claim 1, wherein the combination generating unit generates the plurality of combinations by randomly selecting combinations from the combinations that can be generated from the detected detection targets. [5] Control device according to one of claims 1 to 4, wherein, for each of the plurality of generated combinations, the selection unit (1) An ideal position of an i-th detection target on the workpiece is obtained as A Pi when a three-dimensional position of the workpiece obtained from a single combination is a position A and a design position of the i-th detection target on the workpiece is Pi, and (2) for each detection target in the single combination, a difference Ki between a detected position P'i of the i-th detection target in the single combination and A Pi is obtained, and the indicator is obtained based on the obtained difference Ki. [6] The control device according to claim 5, wherein the selecting unit obtains, as the indicator, ΣKi / n which is an average value of the difference Ki when the number of detection targets in the single combination is n. [7] Control device according to one of claims 1 to 4, wherein, for each of the plurality of generated combinations, the selection unit (1) as A Pi, an ideal position of an i-th detection target on the workpiece is obtained when a three-dimensional position of the workpiece obtained from a single combination is a position A and a design position of the i-th detection target on the workpiece is Pi, and (2) for each detection target in the single combination, obtain a distance Di between a line of sight Li to a detected position of the i-th detection target from the visual sensor in the single combination and A Pi, and obtain the indicator based on the obtained distance Di. [8] The control device according to claim 7, wherein the selecting unit obtains, as an indicator, ΣDi / n which is an average value of the distance Di when the number of detection targets in the single combination is n. [9] Control device according to one of claims 1 to 8, wherein the selection unit performs the selection of the one or more combinations using a selection criterion that the smaller the size of the indicator, the higher the accuracy. [10] The control device according to any one of claims 1 to 8, wherein the selecting unit selects one or more combinations from the plurality of combinations based on the indicator calculated for each of the plurality of combinations and the number of detection targets in each of the plurality of combinations. [11] The control device according to claim 10, wherein the selection unit, for each of the plurality of combinations, performs the selection of the one or more combinations using selection criteria that (1) the accuracy is higher the smaller the size of the indicator, and (2) the accuracy is higher the larger the number of targets detected in a combination. [12] The control device according to any one of claims 1 to 11, wherein the three-dimensional position determining unit determines a three-dimensional position of the workpiece based on statistics of a three-dimensional position of the workpiece obtained from each of the one or more selected combinations. [13] The control device according to claim 12, wherein the three-dimensional position determining unit determines, as the three-dimensional position of the workpiece, an average or a median of a three-dimensional position of the workpiece obtained from each of the one or more selected combinations. [14] Control device according to one of claims 1 to 13, wherein operations for Re-generating, by the combination generating unit, a plurality of combinations, each of which is obtained by selecting three or more detection targets, based on detection targets included in the one or more combinations selected by the selecting unit, and Reselecting one or more combinations from the plurality of newly generated combinations by the selection unit based on the indicator calculated for each of the plurality of newly generated combinations, be carried out one or more times. [15] The control device according to any one of claims 1 to 4, wherein the combination generation unit newly generates a plurality of combinations each including three or more detection targets by deleting, from the one or more combinations selected by the selection unit, one or more detected positions that satisfy a criterion that an indicator representing the positional deviation calculated for a certain detected position is larger than an indicator representing the positional deviation calculated for another detected position, and the combination generation unit performs the regeneration one or more times until an indicator representing the positional deviation calculated for the detection target satisfies a predetermined condition in each of the newly generated combinations. [16] The control device according to claim 15, wherein the predetermined condition is a condition that an average value of indicators representing the positional deviations for the detection targets in each of the newly generated combinations or a value of the indicator is less than or equal to a predetermined value. [17] Three-dimensional position measuring system comprising: a visual sensor; a detection unit configured to detect three or more detection targets present on a workpiece and having a known positional relationship to each other based on an image captured by the visual sensor; a combination generating unit configured to generate a plurality of combinations, each of which is obtained by selecting three or more detection targets from the detected detection targets; a selection unit configured to select one or more combinations from the plurality of combinations based on an indicator representing a positional deviation of the detected positions of the three or more detection targets from an ideal position, the indicator being calculated for each of the plurality of generated combinations; and a three-dimensional position determining unit configured to determine a three-dimensional position of the workpiece from the one or more selected combinations. [18] A three-dimensional position measuring system according to claim 17, further comprising: a robot to which the visual sensor is attached; and a motion control unit configured to control the robot and position the visual sensor in a pickup position to detect each of the three or more detection targets. [19] Program that causes a computer processor to execute: a step of detecting three or more detection targets present on a workpiece and having a known positional relationship to each other based on an image captured by a visual sensor; a step of generating a plurality of combinations each of which is obtained by selecting three or more detection targets from the detected detection targets; a step of selecting one or more combinations from the plurality of combinations based on an indicator representing a positional deviation of the detected positions of the three or more detection targets from an ideal position, the indicator being calculated for each of the plurality of generated combinations; and a step of determining a three-dimensional position of the workpiece from the one or more selected combinations.

Images