Device, robot control device, robot system, and method

Abstract

In the past, there have been cases in which a workpiece model is incorrectly matched to detection data from a shape detection sensor, thereby causing incorrect detection of position data.　This device comprises: a model acquisition unit that acquires a plurality of workpiece models resulting from modeling a workpiece in a plurality of orientations, the workpiece models including a feature model corresponding to a visual feature of the workpiece; and an information acquisition unit that, with respect to a first workpiece model in one orientation, acquires non-similar information for identifying a first feature model that is in a non-similar relationship with the features of the workpiece in other orientations.

Description

[Technical field]

[0001] The present disclosure relates to an apparatus, a robot controller, a robot system, and a method for obtaining dissimilarity information in a workpiece model. [Background technology]

[0002] There is known an apparatus that acquires position data of a workpiece by matching a workpiece model that is a model of the workpiece with shape data of the workpiece detected by a shape detection sensor (e.g., a three-dimensional visual sensor) (e.g., Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-102529 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventionally, erroneous matching of a workpiece model with detection data from a shape detection sensor may result in erroneous detection of position data. [Means for solving the problem]

[0005] In one aspect of the present disclosure, an apparatus includes a model acquisition unit that acquires a plurality of workpiece models that respectively model a workpiece in a plurality of postures, the workpiece models including feature models corresponding to visual features of the workpiece, and an information acquisition unit that acquires dissimilarity information for identifying a first feature model in a first workpiece model in one posture that is dissimilar to features of the workpiece in other postures.

[0006] In another aspect of the present disclosure, a method includes a processor obtaining a plurality of workpiece models each modeling a workpiece in a plurality of poses, each including a feature model corresponding to a visual feature of the workpiece, and obtaining dissimilarity information for identifying a first feature model in a first workpiece model in one pose that is in a dissimilar relationship to features of the workpiece in other poses. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a schematic diagram of a robot system according to an embodiment. [Diagram 2] FIG. 2 is a block diagram of the robot system shown in FIG. [Diagram 3] 1 shows a workpiece and a workpiece model arranged in a first orientation. [Figure 4] 1 shows a workpiece and a workpiece model arranged in a second orientation. [Diagram 5] 13 shows a workpiece and a workpiece model arranged in a third posture. [Figure 6] 4 shows an example of detection data detected by a shape detection sensor. [Figure 7] FIG. 13 is a diagram for explaining matching between a workpiece model and shape data, showing an example in which the workpiece model is appropriately matched to the shape data. [Figure 8] FIG. 13 is a diagram for explaining matching between a workpiece model and shape data, showing an example in which the workpiece model is inappropriately matched to the shape data. [Figure 9] FIG. 13 is a diagram for explaining matching between a workpiece model and shape data, showing an example in which the workpiece model is appropriately matched to the shape data. [Figure 10] FIG. 13 is a diagram for explaining matching between a workpiece model and shape data, showing an example in which the workpiece model is inappropriately matched to the shape data. [Figure 11] FIG. 11 is a block diagram showing other functions of the robot system. [Figure 12] 12 is a flowchart showing an example of an operation flow of the robot system shown in FIG. 11. [Figure 13] FIG. 11 is a block diagram showing further functions of the robot system. [Figure 14] 14 is a flowchart showing an example of an operation flow of the robot system shown in FIG. 13. [Figure 15] FIG. 11 is a block diagram showing further functions of the robot system. [Figure 16] 16 is a flowchart showing an example of an operation flow of the robot system shown in FIG. 15. [Figure 17] 17 is a flowchart showing an example of the process of step S22 in FIG. 16. [Figure 18] This shows the state in which the workpiece model in the first orientation OR is properly matched to the shape data. [Figure 19] 17 is a flowchart showing an example of the process of step S23 in FIG. 16. [Figure 20] FIG. 11 is a block diagram showing further functions of the robot system. [Figure 21] 21 is a flowchart showing an example of an operation flow of the robot system shown in FIG. 20. [Figure 22] FIG. 13 is a diagram for explaining matching between a workpiece model and shape data, showing an example in which the workpiece model is inappropriately matched to the shape data. [Figure 23] FIG. 13 is a diagram for explaining matching between a workpiece model and shape data, showing an example in which the workpiece model is inappropriately matched to the shape data. [Figure 24] 21 is a flowchart showing an example of the process of step S23 in FIG. 16, which is executed by the robot system shown in FIG. 20. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of the present disclosure will be described in detail with reference to the drawings. In various embodiments described below, similar elements are given the same reference numerals, and duplicated explanations will be omitted. First, a robot system 10 according to one embodiment will be described with reference to Figs. 1 and 2. The robot system 10 includes a robot 12, a shape detection sensor 14, and a control device 16.

[0009] In this embodiment, the robot 12 is a vertical articulated robot and includes a robot base 18, a rotating body 20, a lower arm 22, an upper arm 24, a wrist 26, and an end effector 28. The robot base 18 is fixed onto the floor of a work cell. The rotating body 20 is mounted on the robot base 18 so as to be rotatable around a vertical axis.

[0010] The base end of the lower arm 22 is rotatably attached to the rotating body 20 about a horizontal axis, and the base end of the upper arm 24 is rotatably attached to the tip of the lower arm 22. The wrist 26 has a wrist base 26a attached to the tip of the upper arm 24 so as to be rotatable about two axes that are perpendicular to each other, and a wrist flange 26b attached to the wrist base 26a so as to be rotatable about the wrist axis A1.

[0011] The end effector 28 is detachably attached to the wrist flange 26b. The end effector 28 is, for example, a robot hand capable of gripping the workpiece 100, a welding torch for welding the workpiece 100, or a laser processing head for laser processing the workpiece 100, and performs a predetermined operation (workpiece handling, welding, laser processing, etc.) on the workpiece 100.

[0012] Each component of the robot 12 (robot base 18, rotating body 20, lower arm 22, upper arm 24, wrist 26) is provided with a servo motor 30 (FIG. 2). These servo motors 30 rotate each movable element of the robot 12 (rotating body 20, lower arm 22, upper arm 24, wrist base 26a, wrist flange 26b) in response to a command from the control device 16. As a result, the robot 12 can move the end effector 28 and place it in any position.

[0013] The shape detection sensor 14 detects the shape of the workpiece 100. In this embodiment, the shape detection sensor 14 is a three-dimensional visual sensor having an image sensor (CMOS, CCD, etc.) and an optical lens (collimator lens, focus lens, etc.) that guides a subject image to the image sensor, and is fixed to the end effector 28 (or wrist flange 26b).

[0014] The shape detection sensor 14 is configured to capture an image of a subject along an optical axis A2 and measure a distance d to the subject. The shape detection sensor 14 may be fixed to the end effector 28 so that the optical axis A2 and the wrist axis A1 are parallel to each other (or perpendicular to each other). The shape detection sensor 14 supplies the detected detection data DD to the control device 16.

[0015] 1, a robot coordinate system C1 and a tool coordinate system C2 are set for the robot 12. The robot coordinate system C1 is a control coordinate system C for controlling the operation of each movable element of the robot 12. In this embodiment, the robot coordinate system C1 is fixed to the robot base 18 such that its origin is located at the center of the robot base 18 and its z axis is parallel to (specifically, coincides with) the rotation axis of the rotating body 20.

[0016] On the other hand, the tool coordinate system C2 is a control coordinate system C that defines the position and posture of the end effector 28 in the robot coordinate system C1. In this embodiment, the tool coordinate system C2 is set with respect to the end effector 28 so that its origin (so-called TCP) is located at a work position (for example, a workpiece gripping position, a welding position, or a laser light emission port) of the end effector 28 and its z axis is parallel to (specifically, coincides with) the wrist axis A1.

[0017] When moving the end effector 28, the control device 16 sets a tool coordinate system C2 in the robot coordinate system C1, and generates commands to the servo motors 30 of the robot 12 to place the end effector 28 in the position and orientation represented by the set tool coordinate system C2. In this way, the control device 16 can position the end effector 28 in any position and orientation in the robot coordinate system C1.

[0018] On the other hand, a sensor coordinate system C3 is set for the shape detection sensor 14. The sensor coordinate system C3 is a control coordinate system C that defines the position and posture of the shape detection sensor 14 in the robot coordinate system C1 (i.e., the position and direction of the optical axis A2). In this embodiment, the sensor coordinate system C3 is set for the shape detection sensor 14 so that its origin is located at the center of the image sensor of the shape detection sensor 14 and its z axis is parallel to (specifically, coincides with) the optical axis A2. The sensor coordinate system C3 defines the coordinates of each pixel of the detection data DD (or the image sensor) detected by the shape detection sensor 14.

[0019] The positional relationship between the sensor coordinate system C3 and the tool coordinate system C2 is known by calibration, and therefore the coordinates of the sensor coordinate system C3 and the coordinates of the tool coordinate system C2 can be mutually converted via a known conversion matrix (e.g., a homogeneous conversion matrix). In addition, the positional relationship between the tool coordinate system C2 and the robot coordinate system C1 is known, and therefore the coordinates of the sensor coordinate system C3 and the coordinates of the robot coordinate system C1 can be mutually converted via the tool coordinate system C2. In other words, the position and orientation of the shape detection sensor 14 in the robot coordinate system C1 (specifically, the coordinates of the sensor coordinate system C3) are known.

[0020] The control device 16 controls the operation of the robot 12 to execute a predetermined task (workpiece handling, welding, or laser processing) on ​​the workpiece 100. Specifically, as shown in Fig. 2, the control device 16 is a computer having a processor 32, a memory 34, and an I / O interface 36. The processor 32 has a CPU or a GPU, etc., and is communicatively connected to the memory 34 and the I / O interface 36 via a bus 38, and performs arithmetic processing to realize various functions described later while communicating with these components.

[0021] The memory 34 has a RAM or a ROM, etc., and temporarily or permanently stores various data. The memory 34 can be composed of a storage medium such as a volatile memory, a non-volatile memory, a magnetic storage medium, or an optical storage medium. The I / O interface 36 has, for example, an Ethernet (registered trademark) port, a USB port, an optical fiber connector, or an HDMI (registered trademark) terminal, and communicates data with an external device by wire or wirelessly under instructions from the processor 32. Each servo motor 30 of the robot 12 and the shape detection sensor 14 are communicatively connected to the I / O interface 36.

[0022] The control device 16 is also provided with a display device 40 and an input device 42. The display device 40 and the input device 42 are communicatively connected to the I / O interface 36. The display device 40 has a liquid crystal display, an organic EL display, or the like, and visibly displays various data under instructions from the processor 32.

[0023] The input device 42 has a push button, a switch, a keyboard, a mouse, a touch panel, or the like, and receives data input from an operator. The display device 40 and the input device 42 may be integrated into the housing of the control device 16, or may be externally attached to the housing of the control device 16 as a separate computer (such as a PC).

[0024] In this embodiment, the processor 32 acquires position data PD of the workpieces 100 in the robot coordinate system C1 based on detection data DD of the shape detection sensor 14 that detects the shapes of a plurality of workpieces 100 randomly piled in the container B. Hereinafter, an example of the workpieces 100 will be described with reference to FIGS.

[0025] In this embodiment, the workpiece 100 has a main body portion 102, a shaft portion 104, and a protruding portion 106. The main body portion 102 has an outer shape of a rectangular prism. The shaft portion 104 is cylindrical and protrudes from one side surface of the main body portion 102. The shaft portion 104 has a cylindrical side surface 108 and a ring-shaped end surface 110. The protruding portion 106 is rectangular prism-shaped and protrudes from the top surface of the main body portion 102.

[0026] The main body portion 102 (specifically, the edges and surfaces that define the main body portion 102), the shaft portion 104 (specifically, the side surface 108, the end surface 110, and their edges), and the convex portion 106 (specifically, the edges and surfaces that define the convex portion 106) constitute the visual features 102, 104, 106, 108, 110 of the workpiece 100.

[0027] 3 and 4 show three different postures OR when the workpiece 100 is viewed from three different viewpoints. n(n=1, 2, 3) of the workpiece 100. Specifically, Fig. 3 shows the workpiece 100 in a first orientation OR1, Fig. 4 shows the workpiece 100 in a second orientation OR2, and Fig. 5 shows the workpiece 100 in a third orientation OR3.

[0028] In this embodiment, the processor 32 selects a plurality of postures OR as shown in FIGS. n Multiple workpiece models 100M, each of which is a model of the workpiece 100 n Get the work model 100M n For example, CAD model 100Mc n And the CAD model 100Mc n A point cloud model of 100Mp that represents the model components (edges, faces, etc.) as points (or normals) n and

[0029] CAD Model 100Mc n is a three-dimensional CAD model, which is created in advance by an operator using a CAD device (not shown). The processor 32 receives the CAD model 100M from the CAD device. n and then image-generating the CAD model 100Mc according to a predetermined image generation algorithm. n By adding a point cloud to the model component, the point cloud model can be created with a size of 100Mp. n In addition, the work model 100M n CAD Model 100Mc n and point cloud model 100Mp n The model is not limited to the above, and may be any type of model representing the shape of the workpiece 100.

[0030] The processor 32 obtains a workpiece model 100M1 in the first orientation OR1, which is a model of the workpiece 100 in the first orientation OR1 shown in Fig. 3. In the following description, a model corresponding to a feature XXX (e.g., an end face 110) of the workpiece 100 is referred to as a feature model XXXM (e.g., an end face model 110M). The workpiece model 100M1 in the first orientation OR1 includes a main body model 102M, a shaft model 104M (specifically, a side face model 108M and an end face model 110M), and a convex part model 106M.

[0031] The processor 32 also acquires a workpiece model 100M2 in the second orientation OR2, which is a model of the workpiece 100 in the second orientation OR2 shown in Fig. 4. The workpiece model 100M2 in the second orientation OR2 includes a main body model 102M, a shaft model 104M, and a convex portion model 106M. However, in the workpiece model 100M2, the shaft model 104M has a side model 108M, but does not have an end face model 110M (i.e., it is not visible from the viewpoint of Fig. 4).

[0032] The processor 32 also acquires a workpiece model 100M3 in the third orientation OR3, which is a model of the workpiece 100 in the third orientation OR3 shown in Fig. 5. The workpiece model 100M3 in the third orientation OR3 includes a main body model 102M and a shaft model 104M (specifically, a side model 108M and an end face model 110M), but does not include a convex portion model 106M (i.e., it is not visible from the viewpoint of Fig. 5).

[0033] Work Model 100M n A workpiece coordinate system C4 is set for each of the workpiece models 100M in the control coordinate system C (specifically, the robot coordinate system C). nIn this embodiment, the workpiece coordinate system C4 is a coordinate system for expressing the position and orientation of the workpiece model 100M. In this embodiment, the workpiece coordinate system C4 is arranged such that its origin is located at one vertex O of the main body model 102M, its x-axis is parallel to the width direction of the main body model 102M, its y-axis is parallel to the length direction of the main body model 102M (or the axis of the shaft model 104M), and its z-axis is parallel to the height direction of the main body model 102M. n is set for.

[0034] The processor 32 detects various postures OR as shown in FIGS. n Multiple workpiece models of 100M n In the example shown in FIG. 3 to FIG. 5, three postures OR n (n=1,2,3) work model 100M n However, the processor 32 may be configured to select three or more postures OR n (e.g., n=1~500) 100M work models n In this manner, in the present embodiment, the processor 32 may acquire a plurality of postures OR n A plurality of workpiece models 100M each of which is a model of the workpiece 100 n The model acquisition unit 44 (FIG. 2) acquires the model.

[0035] In addition, the work model 100M n may have only the model data of the front side visible from the corresponding viewpoint, and may not have the model data of the back side invisible from the viewpoint. For example, in the case of the work model 100M2 in Fig. 4, it may have only the model data of the front side of the paper in Fig. 4 (model data of the main body model 102M, the side model 108M, and the convex part model 106M) visible when viewed from the viewpoint in Fig. 4, but may not have the model data of the back side of the paper in Fig. 4 (model data of the end face model 110M) invisible from the viewpoint in Fig. 4.

[0036] More specifically, when the processor 32 generates the workpiece model 100M2 in Fig. 4 as the point cloud model 100Mp2, it generates point cloud model data of the model components on the front side of the paper that are visible in Fig. 4, but does not have to generate point cloud model data of the model components on the back side of the paper that are not visible (i.e., edges and faces on the back side when viewed from the viewpoint of Fig. 4). The same applies to the workpiece model 100M1 in Fig. 3 and the workpiece model 100M3 in Fig. 5. With this configuration, the workpiece model 100M to be acquired n This can reduce the amount of data.

[0037] The processor 32 inputs the work model 100M through the input device 42. n The processor 32 may receive an input from the operator to set the workpiece coordinate system C4 to the workpiece model 100M. n is stored in the memory 34 together with the setting information of the work coordinate system C4.

[0038] The processor 32 outputs the obtained workpiece models 100M n Each of the above is matched with detection data DD detected by the shape detection sensor 14 by imaging the workpiece 100 in the container B, thereby acquiring position data PD of the workpiece 100 in the robot coordinate system C1. FIG. 6 shows an example of the imaged detection data DD.

[0039] In this embodiment, the detection data DD is three-dimensional point cloud image data, and includes shape data SD of the workpiece 100 in various orientations detected by the shape detection sensor 14. i (i=1, 2, 3, . . .). For ease of understanding, FIG. 6 shows an example in which shape data SD1 of the workpiece 100 in a first orientation OR1, shape data SD2 of the workpiece 100 in a second orientation OR2, and shape data SD3 of the workpiece 100 in a third orientation OR3 are shown. However, when the shape detection sensor 14 actually captures an image of the workpieces 100 randomly piled up in the container B, the detection data DD includes three or more shape data SD i Please understand that this may be captured.

[0040] Each shape data SD i has a point cloud that indicates the visual features of the workpiece 100 (i.e., the edges or faces of the main body 102, the edges or faces of the shaft 104 (side surface 108, end surface 110), and the edges or faces of the protrusions 106), and each point that constitutes the point cloud has information on the distance d described above. i Each point constituting the point cloud is a three-dimensional coordinate (X S ,Y S ,Z S )

[0041] The processor 32 calculates the shape data SD of each object reflected in the detection data DD in the sensor coordinate system C3. i 100M of multiple workpiece models n For example, as shown in Fig. 7, the processor 32 matches the workpiece model 100M2 (Fig. 7(a)) in the second orientation OR2 with the shape data SD2 (Fig. 7(b)) captured in the detection data DD.

[0042] In this case, as shown in FIG. 7C, all the feature models of the workpiece model 100M2 (main body model 102M, side model 108M, and convex part model 106M) match the features of the shape data SD2 (shape data of the main body 102, side 108, and convex part 106 of the workpiece 100). i The workpiece coordinate system C4 set in the workpiece model 100M2 at this time accurately indicates the position and posture of the workpiece 100 detected as the shape data SD2.

[0043] On the other hand, as shown in Fig. 8, the processor 32 also matches the workpiece model 100M1 (Fig. 8(a)) in the first orientation OR1 to the shape data SD2 (Fig. 8(b)) in the detection data DD. In this case, as shown in Fig. 8(c), the main body model 102M and the convex portion model 106M of the workpiece model 100M1 match the features of the main body 102 and the convex portion 106 in the shape data SD2.

[0044] On the other hand, the shaft part model 104M (specifically, the side model 108M and the end face model 110M) of the workpiece model 100M1 is inconsistent with the features of the shaft part 104 shown in the shape data SD2. That is, in this case, the workpiece model 100M1 is inappropriately matched to the shape data SD2, and the workpiece coordinate system C4 set in the workpiece model 100M1 at this time does not accurately indicate the position and posture of the workpiece 100 detected as the shape data SD2.

[0045] Such an inappropriate matching may occur because the main body model 102M and the convex part model 106M of the workpiece model 100M1 are similar to the features of the main body 102 and the convex part 106 shown in the shape data SD2. On the other hand, the end face model 110M of the workpiece model 100M1 does not exist in the shape data SD2, and is therefore not similar to any features shown in the shape data SD2. As a result, the end face model 110M, which is dissimilar to the features of the shape data SD2, does not match any features of the shape data SD2 in the inappropriate matching as shown in FIG. 8(c).

[0046] In addition, the processor 32 adds a plurality of workpiece models 100M to the shape data SD3 shown in the detection data DD of FIG. n For example, as shown in Fig. 9, the processor 32 matches the workpiece model 100M3 in the third orientation OR3 (Fig. 9(a)) with the shape data SD3 shown in the detection data DD (Fig. 9(b)). As a result, the workpiece model 100M3 and the shape data SD3 are appropriately matched (Fig. 9(c)).

[0047] On the other hand, as shown in Fig. 10, the processor 32 also matches the workpiece model 100M1 (Fig. 10(a)) in the first orientation OR1 to the shape data SD3 (Fig. 10(b)) shown in the detection data DD. In this case, as shown in Fig. 10(c), the convex part model 106M of the workpiece model 100M1 does not match any feature of the shape data SD3, and therefore the workpiece model 100M1 is inappropriately matched to the shape data SD3.

[0048] Such inappropriate matching may occur because the main body model 102M and the shaft model 104M of the workpiece model 100M1 are similar to the features of the main body 102 and the shaft 104 depicted in the shape data SD3. On the other hand, the convex part model 106M of the workpiece model 100M1 does not exist in the shape data SD3, and is therefore not similar to any features depicted in the shape data SD3. As a result, when inappropriate matching such as that shown in FIG. 10(c) is performed, the convex part model 106M, which is dissimilar to the features of the shape data SD3, does not match any features of the shape data SD3.

[0049] As mentioned above, the nth posture OR n Work Model 100M n mth posture OR m Shape data SD for (m ≠ n) m When matched with the work model 100M n Among the feature models included in the shape data SD m (That is, the mth posture OR m By focusing on a feature model (e.g., end face model 110M in FIG. 8(c) or convex portion model 106M in FIG. 10(c)) that is dissimilar to the features of the workpiece 100, it is possible to determine whether the matching is inappropriate.

[0050] Therefore, in this embodiment, the nth posture OR n Work Model 100M n In the mth position, OR mDissimilarity information NI is obtained to identify a feature model that is dissimilar to the feature of the workpiece 100 (that is, the end face model 110M in FIG. 8(c) or the convex portion model 106M in FIG. 10(c)).

[0051] For example, the processor 32 may generate dissimilarity information NI for identifying the end face model 110M as a feature model that is dissimilar to the feature of the workpiece 100 in the second orientation OR2 (i.e., the shape data SD2) in the workpiece model 100M1 in the first orientation OR1 shown in FIG. 1_2 As an example, the operator visually checks the features of the workpiece model 100M1 (e.g., the CAD model 100Mc1) displayed on the display device 40, and operates the input device 42 to obtain dissimilarity information NI for identifying the end face model 110M. 1_2 may be manually entered into the processor 32.

[0052] As another example, the processor 32 attempts to match the workpiece model 100M1 to the detection data DD obtained by capturing an image of the workpiece 100 by the shape detection sensor 14 in the real space or the virtual space, thereby obtaining dissimilarity information NI for identifying the end face model 110M. 1_2 It is also possible to obtain dissimilar information NI automatically by performing matching. 1_2 The function of obtaining the above information will be described in detail later.

[0053] Similarly, the processor 32 generates dissimilarity information NI for identifying the convex part model 106M in the workpiece model 100M1 in the first orientation OR1 shown in FIG. 10 as a feature model that is dissimilar to the feature of the workpiece 100 in the third orientation OR3. 1_3 In this manner, in the present embodiment, the processor 32 may obtain dissimilarity information NI for identifying the feature model 110M or 106M in the workpiece model 100M1 in one orientation OR1 that is dissimilar to the feature of the workpiece 100 in the other orientation OR2 or OR3. 1_2 Or NI 1_3 Thus, the processor 32 obtains the dissimilarity information NI 1_2 , N.I. 1_3The information acquisition unit 46 (FIG. 2) acquires the above information.

[0054] The dissimilarity information NI may be an identification code (character string, symbol, etc.) for identifying the feature model (end face model 110M, convex portion model 106M) or a flag. The processor 32 transfers the acquired dissimilarity information NI to the workpiece model 100M. n or the work model 100M n The model data is stored in the memory 34 in association with the model data.

[0055] As described above, in this embodiment, the processor 32 functions as the model acquisition unit 44 and the information acquisition unit 46 to acquire the workpiece model 100M. n Thus, the model acquiring unit 44 and the information acquiring unit 46 constitute an apparatus 80 (FIG. 2) for acquiring the dissimilarity information NI.

[0056] In this apparatus 80, the model acquisition unit 44 acquires a plurality of postures OR n A plurality of workpiece models 100M each of which is a model of the workpiece 100 n The information acquisition unit 46 acquires dissimilarity information NI for identifying a feature model 110M or 106M that is dissimilar to the feature of the workpiece 100 in one posture OR1 and the feature model 110M or 106M that is dissimilar to the feature of the workpiece 100 in another posture OR2 or OR3. 1_2 Or NI 1_3 Get the.

[0057] According to this system 80, the processor 32 selects a work model 100M that is of interest for detecting improper matches as described above. n As a result, it becomes possible to detect the inappropriate matching with high accuracy, and therefore it is possible to reduce erroneous detection of the position data PD of the workpiece 100.

[0058] Next, other functions of the robot system 10 will be described with reference to Fig. 11. The robot system 10 shown in Fig. 11 acquires the flow shown in Fig. 12. The flow shown in Fig. 12 is a flow for acquiring the above-mentioned dissimilarity information NI. The flow in Fig. 12 is started when the processor 32 receives an operation start command from an operator, a higher-level controller, or a computer program PG.

[0059] In step S1, the processor 32 functions as the model acquisition unit 44 to acquire a plurality of workpiece models 100M. n Specifically, the processor 32 acquires a plurality of postures OR such as those illustrated in FIGS. n (e.g., n=1~500) multiple workpiece models 100M n The processor 32 acquires the work model 100M. n ,CAD model 100Mc n Alternatively, it may be acquired as a point cloud model.

[0060] In step S2, the processor 32 performs the following steps on each of the workpiece models 100M acquired in step S1: n From the work model 100M n and assigns an identification code ID to each of the extracted feature models. For example, the processor 32 extracts the main body model 102M, the side model 108M, the end face model 110M, and the convex part model 106M as feature models from the workpiece model 100M1 in the first orientation OR1 shown in FIG.

[0061] Then, the processor 32 assigns a unique identification code ID to each of the extracted main body model 102M, side model 108M, end face model 110M, and convex model 106M. For example, the identification code ID is a character string code (e.g., "Feature group A", "Feature group B", "Feature group C", etc.). This identification code ID enables the processor 32 to individually identify the feature models 102M, 108M, 110M, and 106M included in the workpiece model 100M1. Thus, in this embodiment, the processor 32 assigns a unique identification code ID to each of the workpiece models 100M n and functions as a feature extraction unit 48 (FIG. 11) that extracts a feature model from the feature data and assigns an identification code ID to the feature model.

[0062] The processor 32 is a work model 100M n may be extracted as one feature model. For example, the processor 32 may extract the side model 108M and the end face model 110M of the workpiece model 100M3 in the third orientation OR3 shown in Fig. 5 as one feature model 104M, and assign an identification code ID (a character string code such as "Feature group C") to the one feature model 104M. The same applies to the main body model 102M and the convex part model 106M, which are composed of model components of multiple faces and edges.

[0063] In addition, the main body model 102M, the side model 108M, the end face model 110M, and the convex part model 106M are not limited to being extracted as one feature model each, but for example, a combination of the main body model 102M and the convex part model 106M may be extracted as one feature model, or a plurality of faces or edges defining the main body model 102M may be extracted as one feature model each. n It may be arbitrarily determined by an operator whether to extract the model components included in the feature model.

[0064] In step S3, the processor 32 calculates the shape data SD i As an example, the processor 32 acquires detection data DD (FIG. 6) obtained by actually capturing an image of the workpieces 100 randomly piled up in the container B by the shape detection sensor 14 of the real machine arranged in the real space.

[0065] As another example, the processor 32 acquires detection data DD obtained by executing a simulation SL in which a virtual shape detection sensor 14 arranged in a virtual space simulates capturing images of workpieces 100 piled randomly in a container B (e.g., a workpiece model 100M arranged in a model of the container B).

[0066] The detection data DD acquired in this way includes various postures OR i Shape data SD of multiple workpieces 100 i (i=1, 2, 3, . . . ). Next, in step S4, the processor 32 n Posture OR n Set the identifying number "n" to "1".

[0067] In step S5, the processor 32 receives the shape data SD i , the nth posture OR n Work Model 100M n For example, when n=1 is set at the start of step S5, the processor 32 matches the workpiece model 100M1 (FIG. 3) in the first orientation OR1 acquired in step S1 with one shape data SD captured in the detection data DD (FIG. 6) acquired in step S3 according to a predetermined matching algorithm. i (For example, the shape data SD1, SD2, or SD3). In this manner, in the present embodiment, the processor 32 matches the shape data SD i Work Model 100M nThe model matching execution unit 50 (FIG. 11) performs matching.

[0068] In step S6, the processor 32 calculates the shape data SD i Work Model 100M n When matching, the work model is 100M n The feature model included in the shape data SD i For example, assume that in the most recent step S5, the processor 32 matches the workpiece model 100M1 in the first orientation OR1 with the shape data SD1 captured in the detection data DD shown in FIG.

[0069] In this case, all feature models included in the workpiece model 100M1, such as the main body model 102M, the shaft model 104M (side model 108M and end face model 110M), and the convex portion model 106M, will be consistent with the features depicted in the shape data SD1 (i.e., the features of the main body 102, shaft 104, and convex portion 106 of the workpiece 100).

[0070] For example, the processor 32 may generate feature points Fm constituting a feature model of a workpiece model 100M1 to which an identification code ID has been assigned, and shape data SD i Then, a degree of agreement β between the feature point Fm and the feature point Fs that constitutes the feature captured in the image is calculated. This degree of agreement β includes, for example, an error in the distance between the feature point Fm and the feature point Fs that corresponds to the feature point Fm. In this case, the higher the degree of agreement between the feature point Fm and the feature point Fs, the smaller the degree of agreement β becomes.

[0071] Alternatively, the degree of agreement β includes a degree of similarity representing the similarity between the feature point Fm and the feature point Fs corresponding to the feature point Fm. In this case, the higher the degree of agreement between the feature point Fm and the feature point Fs, the higher the degree of agreement β. The processor 32 calculates the degree of agreement β by comparing the calculated degree of agreement β with a predetermined threshold value β. th1By comparing these, it is possible to determine whether each feature model of the work model 100M1 (main body model 102M, side model 108M, end face model 110M, convex portion model 106M) is consistent with the features (main body 102, side face 108, end face 110, convex portion 106) depicted in the shape data SD1.

[0072] For example, the processor 32 may determine whether the degree of match β is greater than or equal to a threshold value β th1 (β>β th1 , or β<β th1 ), the processor 32 determines that the feature models of the workpiece model 100M1 match the features of the shape data SD1. As a result of this determination, the processor 32 can detect that all feature models of the workpiece model 100M1 match the features of the shape data SD1. The processor 32 then increments the frequency α by "1" (α=α+1) for each of the main body model 102M, the side model 108M, the end face model 110M, and the convex portion model 106M.

[0073] That is, in this embodiment, the frequency α is n Shape data SD i When matched with the work model 100M n The feature model is the shape data SD i In step S6, the processor 32 calculates the number of times that the workpiece model 100M matches the feature captured in the workpiece model 100M. n The frequency α counted for each feature model is associated with an identification code ID assigned to the feature model and stored in the memory 34. Note that the frequency α is not limited to the number of matches described above, and may be calculated, for example, by applying a weighting calculation to the number of matches according to the matched feature model or the degree of coincidence β, or may be calculated by any method.

[0074] On the other hand, suppose that the processor 32 has matched the workpiece model 100M1 in the first orientation OR1 to the shape data SD2 shown in Fig. 7 in the most recent step S5. In this case, as shown in Fig. 8(c), feature models such as the main body model 102M and the convex part model 106M included in the workpiece model 100M1 match features such as the main body part 102 and the convex part 106 of the workpiece 100 shown in the shape data SD2. Therefore, in this case, the processor 32 increments the frequency α by "1" (α=α+1) for the main body part model 102M and the convex part model 106M included in the workpiece model 100M1.

[0075] On the other hand, the shaft part model 104M (i.e., the side face model 108M and the end face model 110M) of the workpiece model 100M1 does not match the features of the workpiece 100 depicted in the shape data SD2. Therefore, in this case, the processor 32 does not increment the frequency α for the side face model 108M and the end face model 110M of the workpiece model 100M1.

[0076] Thus, in the most recent step S5, the processor 32 i Work Model 100M n When matching, the work model is 100M n The feature model included in the shape data SD i Therefore, the processor 32 functions as a frequency calculation unit 52 (FIG. 11) that calculates the frequency α.

[0077] In step S7, the processor 32 extracts all the shape data SD i Work Model 100M n (If n=1 is set, it is determined whether or not the workpiece model 100M1 in the first orientation OR1 has been matched.)

[0078] If the processor 32 judges the result as YES, the process proceeds to step S8, whereas if the processor 32 judges the result as NO, the process returns to step S5. In this manner, the processor 32 repeatedly executes the loop of steps S5 to S7 until the processor 32 judges the result as YES in step S7, and obtains the shape data SD i Work Model 100M n Each time you match, 100M of the work model n The frequency α is calculated for each feature model, and the frequency α stored in the memory 34 is updated in association with the identification code ID assigned to the feature model.

[0079] In step S8, the processor 32 n Based on the frequency α obtained for the multiple feature models included in i For example, when n=1 is set at the start of step S8, the processor 32 calculates the frequency α of each feature model (main body model 102M, side model 108M, end face model 110M, and convex part model 106M) of the workpiece model 100M1 each time step S6 is executed, by multiplying the frequency α by a predetermined threshold value α. th1 Compare with.

[0080] Then, the processor 32 checks whether the frequency α of one feature model (e.g., the end face model 110M) is equal to or exceeds a threshold α th1 Below (α≦α th1 ), the one feature model is the shape data SD i In this manner, in this embodiment, the processor 32 determines that the feature model is in a dissimilar relationship with the feature of the shape data SD based on the frequency α. i The image functions as a similarity determination unit 54 (FIG. 11) that determines whether the image has a dissimilar relationship with the features shown in the image.

[0081] If processor 32 determines that at least one feature model included in work model 100M1 is in a dissimilar relationship (i.e., YES), it proceeds to step S9, whereas if processor 32 determines that none of the feature models included in work model 100M1 are in a dissimilar relationship (i.e., NO), it proceeds to step S10.

[0082] In step S9, the processor 32 functions as the information acquisition unit 46 to acquire the workpiece model 100M. n Among the multiple feature models included in the first orientation OR1, dissimilarity information NI is obtained for feature models that are determined to be in a dissimilar relationship in the immediately preceding step S8. For example, assume that n=1 is set at the start of this step S9, and as a result of step S8, the end face model 110M (FIG. 8(c)) and the convex part model 106M (FIG. 10(c)) included in the workpiece model 100M1 in the first orientation OR1 are determined to be in a dissimilar relationship with the feature of the workpiece 100 in the second orientation OR2 and the feature of the workpiece 100 in the third orientation OR3, respectively.

[0083] In this case, the processor 32 uses dissimilarity information NI for identifying the end face model 110M among the multiple feature models (the main body model 102M, the side face model 108M, the end face model 110M, and the convex part model 106M) included in the workpiece model 100M1 in the first orientation OR1. 1_2 and dissimilarity information NI for identifying the convex part model 106M 1_3 You will get the following:

[0084] The processor 32 receives the dissimilarity information NI of the end face model 110M. 1_2 is acquired in association with the identification code ID given to the end face model 110M, and the dissimilarity information NI 1_3 is acquired in association with the identification code ID given to the convex portion model 106M.

[0085] Then, the processor 32 calculates the obtained dissimilarity information NI 1_2 and N.I. 1_3are stored in the memory 34 in association with the respective identification code IDs. As a result, the identification code IDs of the feature models of the workpiece model 100M1 and the dissimilarity information NI of the feature models are 1_2 , N.I. 1_3 are stored in the memory 34 in association with each other.

[0086] In step S10, the processor 32 n Posture OR n The number "n" that identifies the maximum value of n MAX (n=n MAX ) is checked. MAX is the work model 100M obtained in step S1. n The total number of postures (i.e., posture OR n (for example, n MAX 12. If the processor 32 determines that the answer is YES, the process proceeds to step S11.

[0087] In step S11, the processor 32 n Posture OR n The processor 32 increments the number "n" that identifies the number by "1" (n=n+1). Then, the processor 32 returns to step S5. In this way, the processor 32 returns to step S10 if the answer is YES. The loop of steps S5 to S11 is repeatedly executed until it is determined that all the workpiece models 100M obtained in step S1 are n For each of the above, a determination of a dissimilarity relationship is made (step S8), and dissimilarity information NI is obtained (step S9).

[0088] As described above, in this embodiment, the processor 32 functions as the model acquisition unit 44, the information acquisition unit 46, the feature extraction unit 48, the model matching execution unit 50, the frequency calculation unit 52, and the similarity determination unit 54 to generate the work model 100M. nThus, the model acquisition unit 44, the information acquisition unit 46, the feature extraction unit 48, the model matching execution unit 50, the frequency calculation unit 52, and the similarity determination unit 54 constitute a device 82 (FIG. 11) for acquiring dissimilarity information NI.

[0089] In this device 82, the feature extraction unit 48 extracts the feature of each workpiece model 100M. n From the work model 100M n Then, the information acquiring unit 46 extracts a plurality of feature models included in the feature model 110M and assigns an identification code ID to each of the extracted feature models (step S2). Then, the information acquiring unit 46 associates the identification code ID assigned to the feature model (for example, the end face model 110M or the convex portion model 106M) with the dissimilarity information NI 1_2 Or NI 1_3 (Step S9).

[0090] According to this configuration, the processor 32 is a work model 100M n It is possible to quickly recognize which of the multiple feature models included in the database are in a dissimilar relationship from the mutually associated identification code ID and dissimilarity information NI. In addition, the dissimilarity information NI can be accumulated for each feature model classified by the identification code ID.

[0091] In the device 82, the model matching execution unit 50 receives the shape data SD i Work Model 100M n (Step S5). The frequency calculation unit 52 also performs matching on the model matching execution unit 50 by using a plurality of postures OR i Shape data SD i (For example, shape data SD1, SD2, SD3 in Fig. 6) work model 100M n (For example, work model 100M1) is matched with the work model 100M n The feature model included in each of the shape data SD i The frequency α of matching with the features captured in the image is calculated (step S6).

[0092] In addition, the similarity determination unit 54 determines whether the frequency calculation unit 52 is a work model 100M. n Based on the frequency α obtained for the plurality of feature models included in the workpiece model 100M, the information acquisition unit 46 determines whether or not the feature models are in a dissimilar relationship (step S8). n Among the multiple feature models included in the workpiece model 100M, dissimilarity information is acquired for a feature model (for example, the end face model 110M or the convex portion model 106M) that is determined to be in a dissimilar relationship by the similarity determination unit 54 (step S9). n Since it is possible to determine with high accuracy whether or not the feature model is in a dissimilar relationship with the feature of the workpiece 100 based on the frequency α, it is possible to acquire dissimilarity information NI for each feature model with high accuracy.

[0093] In the device 82, the similarity determination unit 54 determines whether the frequency α calculated by the frequency calculation unit 52 is equal to or smaller than a predetermined threshold value α th1 If the following is true, the workpiece model is 100M. n The feature model of the workpiece 100 (specifically, the shape data SD i ) is determined to be in a dissimilar relationship with the feature of the feature. With this configuration, the determination of the dissimilar relationship can be performed with a relatively simple algorithm and with high accuracy.

[0094] In addition, in one example of the device 82, the model matching execution unit 50 generates shape data SD i Using this, work model 100M n According to this configuration, the shape data SD detected by the shape detection sensor 14 of the actual machine is matched (step S2). i As a result, dissimilarity information NI can be obtained with high accuracy.

[0095] On the other hand, in another example of the device 82, the model matching execution unit 50 uses the shape data SD acquired by a simulation SL in which the shape detection sensor 14 captures an image of the workpiece 100 in a virtual space. i(Step S2) This configuration makes it possible to easily try various types of matching, thereby simplifying the task of obtaining dissimilarity information NI.

[0096] In the above-mentioned step S6, the processor 32 uses the workpiece model 100M matched in the immediately preceding step S5. n All feature models of the shape data SD i If it is detected that the feature matches the feature of all the feature models, it is not necessary to increment the frequency α of each of the feature models.

[0097] For example, when the processor 32 matches the workpiece model 100M1 in the first orientation OR1 with the shape data SD1 shown in the detection data DD shown in Fig. 6 in the immediately preceding step S5, it can detect that all feature models of the workpiece model 100M1 match the features of the shape data SD1, as described above. In this case, the processor 32 does not increment the frequency α for each of the main body model 102M, side model 108M, end face model 110M, and convex portion model 106M of the workpiece model 100M1.

[0098] That is, in this case, the processor 32 n In the same posture OR n Shape data SD i If the shape data SD i The feature models that match the features of are excluded from the calculation of the frequency α. This makes it possible to obtain the dissimilarity information NI for detecting inappropriate matching more efficiently.

[0099] Next, still another function of the robot system 10 will be described with reference to Fig. 13. The robot system 10 shown in Fig. 13 executes a flow shown in Fig. 14. The flow shown in Fig. 14 is a flow for acquiring dissimilarity information NI like the flow in Fig. 12, but differs from the flow in Fig. 12 in step S12.

[0100] Specifically, after step S5, in step S12, the processor 32 determines whether the workpiece model 100M was created in the immediately preceding step S5. n When matching is performed, the work model is 100M n At least one feature model in and shape data SD i It is then determined whether there is a mismatch between the features captured in the image and the features captured in the image.

[0101] For example, assume that in the immediately preceding step S5, the workpiece model 100M1 in the first orientation OR1 is matched with the shape data SD2 shown in the detection data DD in Fig. 6. In this case, the processor 32 determines, for example, the degree of agreement β between the feature points Fm constituting each feature model (main body model 102M, side face model 108M, end face model 110M, and convex part model 106M) of the workpiece model 100M1 to which the identification code ID has been assigned, and the feature points Fs constituting the features shown in the shape data SD2.

[0102] Then, the processor 32 calculates the degree of match β and the threshold value β for each feature model to which the identification code ID is assigned. th1 The degree of match β is compared with the threshold value β th1 8(c), the processor 32 determines that there is a mismatch between the feature model of the workpiece model 100M1 and the features of the shape data SD2 (i.e., YES). As a result of this determination, the processor 32 determines that there is a mismatch between the side model 108M and the end face model 110M (i.e., the shaft part model 104M) of the workpiece model 100M1 and the features of the shape data SD2, as shown in FIG.

[0103] If the result of the determination is YES, the processor 32 proceeds to step S6, and checks whether the feature models (the main body model 102M and the convex part model 106M in FIG. 8C) included in the workpiece model 100M1 match the shape data SD i On the other hand, if the processor 32 determines that the result is NO, the process proceeds to step S7.

[0104] Thus, in this embodiment, the processor 32 performs the following in step S5: n When matching is performed using the work model 100M n The feature model and shape data SD i The image sensor 54 functions as an inconsistency determining unit 56 (FIG. 13) that determines whether or not an inconsistency occurs between the image and the features captured in the image.

[0105] As described above, in this embodiment, the processor 32 functions as the model acquisition unit 44, the information acquisition unit 46, the feature extraction unit 48, the model matching execution unit 50, the frequency calculation unit 52, the similarity determination unit 54, and the inconsistency determination unit 56 to generate the work model 100M. n Thus, the model acquiring unit 44, the information acquiring unit 46, the feature extracting unit 48, the model matching executing unit 50, the frequency calculating unit 52, the similarity determining unit 54, and the inconsistency determining unit 56 constitute a device 84 (FIG. 11) for acquiring dissimilarity information NI.

[0106] In this device 84, the mismatch determination unit 56 determines whether the model matching execution unit 50 is a work model 100M. n When matching is performed using (for example, workpiece model 100M1) (step S5), n At least one feature model (for example, a side model 108M and an end face model 110M) included in the shape data SD i It is determined whether or not there is a mismatch between the feature corresponding to the at least one feature model and the feature captured in the shape data SD2 (for example) (step S12).

[0107] Then, when the inconsistency determination unit 56 determines that an inconsistency has occurred (i.e., YES in step S12), the frequency calculation unit S6 determines whether a feature model other than the at least one feature model (for example, the main body portion 102 and the convex portion 106) is included in the shape data SD i The frequency α of matching with the features of the shape data SD2, for example, is calculated (step S6).

[0108] According to this configuration, the work model 100M n and shape data SD i When a mismatch occurs in the matching, the shape data SD i The frequency α can be calculated only for feature models that match the features of the feature models. Here, the feature models that are determined to be inconsistent in step S12 are likely to have the above-mentioned dissimilar relationship. Therefore, by collecting the frequency α calculated when it is determined that an inconsistency has occurred in step S12, the dissimilarity information NI can be efficiently obtained in step S9.

[0109] Next, still another function of the robot system 10 will be described with reference to Fig. 15. The robot system 10 shown in Fig. 15 executes a flow shown in Fig. 16. The flow in Fig. 16 is a flow for causing the robot 12 to execute a predetermined task on the workpieces 100 piled up in a container B, and is executed after the flow in Fig. 12 or Fig. 14 is executed (i.e., after the dissimilarity information NI is acquired). After the flow in Fig. 12 or Fig. 14 is executed, the processor 32 starts the flow in Fig. 16 when it receives a task start command from an operator, a higher-level controller, or a computer program PG.

[0110] In step S21, the processor 32 acquires the detection data DD detected by the shape detection sensor 14. Specifically, the processor 32 operates the robot 12 to position the shape detection sensor 14 at an imaging position where the workpieces 100 piled up in the container B are within the detection range of the shape detection sensor 14. Then, the processor 32 operates the shape detection sensor 14 to capture an image of the workpieces 100 in the container B, thereby detecting the detection data DD as exemplified in FIG. 6. The processor 32 acquires the detected detection data DD from the shape detection sensor 14.

[0111] In step S22, the processor 32 calculates the shape data SD i Work Model 100M nBy matching these, position data PD of the workpiece 100 in the control coordinate system C is acquired. That is, in this embodiment, the processor 32 functions as a position data acquisition unit 58 (FIG. 15) that acquires the position data PD of the workpiece 100. This step S22 will be described with reference to FIG. 17. The processor 32 functions as the position data acquisition unit 58 and executes the flow of step S22 shown in FIG. 17.

[0112] After the start of step S22, in step S31, the processor 32 executes pre-processing PP on the detection data DD acquired in the most recent step S21. For example, as the pre-processing PP, the processor 32 may execute a process of deleting from the detection data DD a point group to be invalid (for example, a point group that exists outside the container B, or a point group that is significantly deviated from a point group of the shape data SDi of the workpiece 100 by more than a predetermined distance) among the point groups included in the detection data DD.

[0113] In step S32, the processor 32 executes a rough search RS in accordance with a predetermined matching algorithm. Specifically, as the rough search RS, the processor 32 performs a rough search RS by mapping the plurality of workpiece models 100M acquired in the above-mentioned step S1 in a virtual space defined by the sensor coordinate system C3 of the detection data DD. n The work model 100M n Each shape data SD included in the detection data DD i That is, in this embodiment, the processor 32 matches one piece of shape data SD i For multiple workpiece models 100M n Match each of them.

[0114] At this time, the processor 32 calculates the workpiece model 100M located in the sensor coordinate system C3. n The processor 32 then repeatedly displaces the position of the work model 100M by a predetermined amount of displacement. n Each time the position of the work model 100M is displaced, nThe degree of coincidence β between the feature points Fm of the shape data SDi and the feature points Fs of the shape data SDi is calculated.

[0115] Then, the processor 32 calculates the obtained degree of match β and a threshold value β th2 and the degree of match β is compared with the threshold value β th2 (i.e., β>β th2 , or β<β th2 ), the workpiece model is 100M in the sensor coordinate system C3. n and shape data SD i It is determined that the two match approximately. th2 is the above threshold value β th1 It may be the same value as or a different value.

[0116] 18 shows a state where the workpiece model 100M1 in the first orientation OR1 is matched to the shape data SD1. The processor 32 calculates the coordinate Q1 of the sensor coordinate system C3 of the workpiece coordinate system C4 set in the workpiece model 100M1 matched to the shape data SD1 as a rough search RS. S (X1 S ,Y1 S ,Z1 S ,W1 S ,P1 S ,R1 S ) to the sensor coordinate system position data PD 0_1 Obtain as.

[0117] This sensor coordinate system position data PD 0_1 (That is, coordinate Q1 S ), coordinates (X1 S ,Y1 S ,Z1 S ) indicates the origin position of the workpiece coordinate system C4 in the sensor coordinate system C3, and the coordinate (W1 S ,P1 S ,R1 S ) indicate the attitude (so-called yaw, pitch, and roll) of the workpiece coordinate system C4 in the sensor coordinate system C3.

[0118] Then, the processor 32 calculates the sensor coordinate system position data PD 0_1The coordinate Q1 in the robot coordinate system C1 R (X1 R ,Y1 R ,Z1 R ,W1 R ,P1 R ,R1 R ) and convert the coordinate Q1 R The initial position data PD 1_1 This initial position data PD 1_1 represents approximate values ​​of the position and orientation in the robot coordinate system C1 of the workpiece 100 detected by the shape detection sensor 14 as the shape data SD1.

[0119] In the same manner, the processor 32 extracts other shape data SD i Sensor coordinate system position data PD 0_i , and the initial position data PD 1_i In this way, the processor 32 sequentially acquires the work model 100M in the rough search RS. n Shape data SD i By matching with the position data PD 0_i , P.D. 1_i has been obtained.

[0120] In step S33, the processor 32 executes a fine search PS. Specifically, the processor 32 searches for shape data SD i Regarding the initial position data PD obtained in step S32 1_i Based on this, the work model 100M is calculated in the sensor coordinate system C3 according to a predetermined matching algorithm (for example, a mathematical optimization algorithm such as ICP: Iterative Closest Point). n Shape data SD i Search for a location that matches closely with

[0121] For example, the processor 32 may calculate the initial position data PD 1_i Work Model 100M n Point cloud model 100Mp arranged as nThe point cloud and the shape data SD included in the detection data DD i For example, the degree of agreement γ is calculated for a point cloud model of 100 Mp n Point cloud and shape data SD i Distance error with 3D point cloud, or point cloud model 100Mp n Point cloud and shape data SD i Includes similarity to the 3D point cloud.

[0122] Then, the processor 32 compares the obtained degree of coincidence γ with a predetermined threshold value γ th and the degree of agreement γ is compared with a threshold γ th When γ exceeds γ th , or γ<γ th ), the workpiece model is 100M in the sensor coordinate system C3. n (For example, a point cloud model of 100 Mp n ) and shape data SD i On the other hand, if the degree of match γ is greater than or equal to the threshold value γ th If it does not exceed the threshold, the processor 32 calculates the workpiece model 100M n The position of the workpiece model 100M is displaced by a specified amount. n Each time the position of is shifted, the degree of match γ is calculated and the threshold value γ th Compare with.

[0123] The processor 32 determines whether the degree of match γ is greater than or equal to a threshold γ th When it exceeds this limit, the shape data SD is used as the dense search PS. i Highly matched work model 100M n Coordinate Q2 in the sensor coordinate system C3 of the work coordinate system C4 set as S (X2 S ,Y2 S ,Z2 S ,W2 S ,P2 S ,R2 S ) to the sensor coordinate system position data PD 2_i Obtain as.

[0124] Then, the processor 32 calculates the sensor coordinate system position data PD 2_i The coordinate Q2 in the robot coordinate system C1 R (X2 R ,Y2 R ,Z2 R ,W2 R ,P2 R ,R2 R ) and convert the coordinates Q2 R , shape data SD i Position data PD of workpiece 100 detected as 3_i This position data PD 3_i represents the highly accurate position and orientation of the workpiece 100 in the robot coordinate system C1. In this way, the processor 32 performs the fine search PS by n Shape data SD i By matching with the position data PD 2_i , P.D. 3_i has been obtained.

[0125] As a result of step S33, the processor 32 obtains position data PD of the plurality of workpieces 100 captured in the detection data DD. 3_i The processor 32 acquires the acquired position data PD 3_i are stored as detection result data DT in the memory 34. An example of the data structure of the detection result data DT is shown in Table 1 below.

[0126] [Table 1]

[0127] The detection result data DT shown in Table 1 includes the position data PD of the workpiece 100. 3_i (Specifically, coordinate Q2 R (X2 R ,Y2 R ,Z2 R ,W2 R ,P2 R ,R2 R )) and the agreement γ i , plane proportion δ i , and exposure rate εi The detection result parameters PM are stored in the form of position data PD. 3_i The shape data SD of workpiece 100 executed when i and Work Model 100M n Specifically, the degree of match γ i In step S33, the dense search PS finds 100M work models. n Shape data SD i This is the degree of match when it is determined that there is a high degree of matching.

[0128] Plane ratio δ i For example, the shape data SD i Work model 100M matched to n In this case, the work model 100M n The largest plane model included in the n ) but the work model 100M n The entire point cloud model (specifically, the entire point cloud model is 100Mp n Alternatively, the plane fraction δ i , dense search PS work model 100M n The shape data SD i The point group representing the largest plane included in the shape data SD i It may be a ratio of the total (specifically, the total point cloud).

[0129] Exposure rate ε i For example, the shape data SD i Work model 100M matched to n The entire (specifically, the point cloud model 100Mp n For all points constituting the shape data SD i Matching work model 100M n (i.e., the shape data SD i A 100Mp point cloud model that matches the point cloud of n The percentage of points in the cloud is shown.

[0130] The processor 32 receives the position data PD 3_i At the same time, the detection result parameter PM (matching degree γ i , plane proportion δ i , and exposure rate ε i ) and obtain the location data PD 3_i The detection result parameter PM is stored in the memory 34 as the detection result data DT in association with the degree of match γ i , plane proportion δ i , and exposure rate ε i The parameter is not limited to the above, and may have any other parameters.

[0131] In addition, the detection result data DT shown in Table 1 includes the position data PD 3_i The shape data SD i Information and acquired location data PD 3_i The number j (j=1, 2, 3, . . .) assigned to each of the 3_i The detection result parameter PM is stored in association with the detection result parameter PM.

[0132] In step S34, the processor 32 executes post-processing OP on the detection result data DT. For example, the processor 32 performs the post-processing OP on the position data PD whose detection result parameters PM do not satisfy a predetermined criterion. 3_i from the detection result data DT.

[0133] Specifically, the processor 32 calculates the degree of match γ i , plane proportion δ i , or exposure rate ε i does not exceed the predetermined reference value, it is determined that the detection result parameter PM does not satisfy the reference, and the position data PD associated with the detection result parameter PM 3_i may be deleted from the detection result data DT. As described above, the processor 32 functions as the position data acquisition unit 58, executes step S22 shown in FIG. 17, and acquires the position data PD 0_i , P.D. 1_i , P.D. 2_i , P.D.3_i Get the.

[0134] 16 again, in step S23, the processor 32 verifies whether the position data PD acquired in step S22 is erroneously detected. Here, in the above-mentioned steps S32 and S33, the processor 32 verifies whether the position data PD acquired in step S22 is erroneously detected. n Shape data SD i As a result of improper matching to the location data PD 3_i The location data PD obtained as a result of such an inappropriate match may be 3_i This results in erroneous detection data that does not accurately represent the position and orientation of the workpiece 100.

[0135] Therefore, in this embodiment, the processor 32 in step S23 calculates the position data PD 3_i Step S23 will be described with reference to Fig. 19. After starting step S23, in step S41, the processor 32 checks whether the position data PD 3_i Set the identifying number "j" to "1".

[0136] In step S42, the processor 32 calculates the j-th position data PD 3_i The work model 100M used for matching to obtain n At this point, the number "j" of the position data PD is set to j=2, and the second (i.e., number j=2) position data PD stored in the detection result data DT in Table 1 is 3_2 However, as shown in FIG. 8(c), a case will be described in which this is obtained as a result of inappropriate matching of the workpiece model 100M1 in the first orientation OR1 with the shape data SD2.

[0137] In this case, the processor 32 calculates the second position data PD 3_2The workpiece model 100M1 used for matching when acquiring the workpiece model 100M1 is identified, and dissimilarity information NI for identifying the end face model 110M and the convex portion model 106M in the above-mentioned step S9 is acquired for the workpiece model 100M1. 1_2 and N.I. 1_3 It can be recognized that the above has been acquired. Therefore, in this case, the processor 32 determines YES in this step S42. If the processor 32 determines YES, the process proceeds to step S43, whereas if the processor 32 determines NO, the process proceeds to step S45.

[0138] In step S43, the processor 32 calculates the j-th position data PD 3_i The work model 100M used to obtain n Among the feature models included in the j-th position data PD, the feature model identified by the dissimilarity information NI is 3_i The shape data SD i It is then determined whether the features match those captured in the image.

[0139] Specifically, the processor 32 receives the second position data PD 3_2 In the workpiece model 100M1 in the first posture OR1 used to acquire the dissimilarity information NI 1_2 It is determined whether the end face model 110M identified by is consistent with the features depicted in the shape data SD2.

[0140] For example, the processor 32 detects the second position data PD 3_2 The matching data obtained when the degree of match γ2 stored in association with the above is obtained is analyzed, and the degree of match γ2' between the end face model 110M matched in the above-mentioned step S33 (fine search PS) and the features of the shape data SD2 is obtained.

[0141] Then, the processor 32 checks whether the degree of match γ2′ is equal to or smaller than a predetermined threshold γ th If it exceeds γ2'>γ th ', or γ2'<γ th ') and non-similar information NI1_2 It is determined that the end face model 110M identified by is consistent with the features depicted in the shape data SD2 (i.e., YES).

[0142] In addition, this threshold γ th ' is the above threshold γ th In this embodiment, as shown in FIG. 8C, the end face model 110M of the workpiece model 100M1 is inconsistent with the features of the shape data SD2, so the degree of agreement γ2′ is less than the threshold value γ th ' does not exceed.

[0143] Therefore, in this case, the processor 32 calculates the dissimilarity information NI 1_2 The processor 32 determines that the end face model 110M identified by is inconsistent with the features depicted in the shape data SD2 (i.e., NO). If the processor 32 determines NO, the process proceeds to step S44, whereas if the processor 32 determines YES, the process proceeds to step S45.

[0144] Thus, in this embodiment, the processor 32 i Work Model 100M n By matching the position data PD 3_i When the feature model identified by the dissimilarity information NI is obtained, the shape data SD i The image functions as a consistency determining unit 62 (FIG. 15) that determines whether the image is consistent with the features shown in the image.

[0145] In step S43, the processor 32 calculates dissimilarity information NI for the workpiece 100 in the third orientation OR3 (i.e., the shape data SD3 in FIG. 10C) in the workpiece model 100M1 in the first orientation OR1. 1_3 It may be determined whether the convex part model 106M identified by is consistent with the features depicted in the shape data SD2.

[0146] In this case, the processor 32 functions as a consistency determination unit 62 and determines whether the convex part model 106M is consistent with the features of the shape data SD2. As shown in FIG. 8C, the convex part model 106M of the workpiece model 100M1 is consistent with the features (convex part 106) of the shape data SD2. Therefore, the processor 32 determines the dissimilarity information NI 1_3 The convex part model 106M identified by is determined to be consistent with the features of the shape data SD2.

[0147] That is, in step S43, the processor 32 n When there are multiple pieces of dissimilarity information NI in the shape data SD i The processor 32 then determines whether the at least one feature model matches the features of the shape data SD i If there is a mismatch with the characteristics of the above, the answer is NO.

[0148] In step S44, the processor 32 invalidates the position data PD of the workpiece 100 acquired in the above-mentioned step S22. Here, as described above, in this embodiment, the second position data PD 3_2 8(c) is obtained as a result of inappropriate matching. In this case, if the processor 32 determines in step S25 described later that the position data PD 3_2 If the robot 12 is caused to perform a task on the workpiece 100 using this, the task cannot be performed with high accuracy.

[0149] In this embodiment, the processor 32 in step S44 uses the position data PD 3_2 In order to avoid using the position data PD 3_2 As an example, the processor 32 invalidates the position data PD from the detection result data DT created in the above-mentioned step S22. 3_2 Delete.

[0150] As another example, the processor 32 may convert the position data PD 3_2 In this case, when executing step S25 described later, the processor 32 assigns an invalid flag FL to the position data PD 3_2 When the position data PD is read out, 3_2 The position data PD is 3_2 As a result, the processor 32 cancels the operation on the workpiece 100 detected as the shape data SD2 in step S25.

[0151] In this manner, in step S44, the processor 32 calculates the position data PD 3_i In this way, the position data PD obtained as a result of the inconsistency is invalidated (for example, deleted or given an invalid flag FL) in the control of step S25 described later. 3_i can be avoided.

[0152] Thus, in this embodiment, the processor 32 uses the position data PD 3_i To obtain the shape data SD i Work model 100M matched to n and the shape data SD i (i.e., if the result of step S43 is NO), the position data PD 3_i The position data canceling unit 60 (FIG. 15) cancels the position data.

[0153] In step S45, the processor 32 checks whether the number “j” of the position data PD set at this time is j=j MAX Determine whether or not this j MAX is the position data PD acquired in step S22 3_iIf the processor 32 determines that the answer is YES, the processor 32 ends the flow of FIG. 19, whereas if the processor 32 determines that the answer is NO, the processor 32 proceeds to step S46.

[0154] In step S46, the processor 32 receives the position data PD 3_i The processor 32 increments the number "j" of the detection result data DT by "1" (j=j+1). Then, the processor 32 returns to step S42. In this manner, the processor 32 repeatedly executes the loop of steps S42 to S46 until the determination in step S45 is YES, and increments the number "j" of the position data PD 3_i Among them, the position data PD obtained as a result of the inconsistency 3_i (For example, location data PD 3_2 ) to disable

[0155] Referring again to FIG. 16, in step S24, the processor 32 3_i Specifically, the processor 32 determines the detection result data DT after executing the above-mentioned step S23 as formal control data to be used for control in step S25 described later, and stores it in the memory 34 (for example, RAM).

[0156] In step S25, the processor 32 refers to the detection result data DT determined in step S24, and operates the robot 12 to perform an operation (workpiece handling, welding, laser processing, or the like) on the workpiece 100. Assuming that in the above-mentioned step S44, the first position data PD 3_1 The second position data PD was not invalidated. 3_2 is invalidated.

[0157] In this case, the processor 32 calculates the first position data PD stored in the detection result data DT. 3_1 , and the robot 12 is operated to move the end effector 28 (i.e., the tool coordinate system C2) according to the position data PD 3_1 (i.e., coordinate Q2 in the robot coordinate system C1) R (X2 R ,Y2R ,Z2 R ,W2 R ,P2 R ,R2 R ) and executes an operation on the workpiece 100 detected as the shape data SD1 by the end effector 28. Meanwhile, the processor 32 determines the second position data PD 3_2 The operation on the workpiece 100 detected as the acquired shape data SD2 is canceled.

[0158] In step S26, the processor 32 determines whether or not the work on all the workpieces 100 in the container B has been completed. If the processor 32 determines YES, it ends the flow shown in Fig. 16, whereas if the processor 32 determines NO, it returns to step S21. In this manner, the processor 32 repeatedly executes the loop of steps S21 to S26 until it determines YES in step S26, obtains detection data DD newly detected by the shape detection sensor 14 in step S21, and executes steps S22 to S25 based on the new detection data DD.

[0159] As described above, in this embodiment, the processor 32 functions as the model acquisition unit 44, the information acquisition unit 46, the feature extraction unit 48, the model matching execution unit 50, the frequency calculation unit 52, the similarity determination unit 54, the inconsistency determination unit 56, the position data acquisition unit 58, the position data cancellation unit 60, and the consistency determination unit 62 to generate the workpiece model 100M. n Obtain the dissimilarity information NI of

[0160] Therefore, the model acquisition unit 44, the information acquisition unit 46, the feature extraction unit 48, the model matching execution unit 50, the frequency calculation unit 52, the similarity determination unit 54, the inconsistency determination unit 56, the position data acquisition unit 58, the position data cancellation unit 60, and the consistency determination unit 62 constitute an apparatus 86 (Figure 15) for acquiring dissimilarity information NI.

[0161] In this device 86, the position data acquisition unit 58 acquires shape data SD of the workpiece 100 detected by the shape detection sensor 14 arranged at a known position in the control coordinate system C (robot coordinate system C1).i Work Model 100M n By matching, the position data PD of the workpiece 100 in the control coordinate system C (specifically, the position data PD 0_i , P.D. 1_i , P.D. 2_i , P.D. 3_i ) is obtained (steps S32 and S33).

[0162] The position data canceling unit 60 then cancels the shape data SD in order for the position data acquiring unit 58 to acquire the position data PD. i Work model 100M matched to n and the shape data SD i (that is, if NO is determined in step S43), the position data PD acquired by the position data acquisition unit 58 is invalidated (step S44).

[0163] According to this configuration, the position data PD obtained as a result of inappropriate matching, such as that shown in FIG. 8(c) or FIG. 10(c), 3_i It is possible to avoid having the robot 12 perform the task on the workpiece 100 (step S25) using the above-mentioned. This makes it possible to improve the accuracy of the task performed by the robot 12 in step S25.

[0164] In the device 86, the matching determination unit 62 determines whether the position data acquisition unit 58 has received the shape data SD i (For example, shape data SD2) with work model 100M n (For example, work model 100M1) is matched to obtain position data PD (PD 3_2 ) is obtained, the dissimilarity information NI (NI 1_2 ) is identified by the feature model (end face model 110M) in the shape data SD i It is then determined whether the features match those shown in the image (step S43).

[0165] Then, the position data canceling unit 60 checks whether the feature model (110M) matches the shape data SD iIf it is determined that the characteristics of the location are inconsistent with those of the location data acquired by the location data acquisition unit 58 (PD 3_2 ) is invalidated. With this configuration, by focusing on the feature model identified by the dissimilarity information NI, it is possible to verify whether or not inappropriate matching occurred when the position data PD was acquired. This makes it possible to find erroneous detection of the position data PD with high accuracy.

[0166] Next, further functions of the robot system 10 will be described with reference to Fig. 20. The robot system 10 shown in Fig. 20 executes a flow shown in Fig. 21. In the flow shown in Fig. 21, the same processes as those in the flow of Fig. 14 are given the same step numbers, and duplicated descriptions will be omitted.

[0167] In the flow of FIG. 21, if the determination in step S7 is YES, in step S13, the processor 32 n Based on the frequency α obtained for the multiple feature models included in i It is then determined whether the feature is similar to the feature captured in the image.

[0168] Specifically, the processor 32 performs the above-mentioned step S6 by n The frequency α obtained for each of the feature models (for example, the main body model 102M, the side model 108M, the end face model 110M, and the convex part model 106M) is set to a predetermined threshold value α th2 Compare with.

[0169] Then, the processor 32 determines whether the frequency α of one feature model is greater than or equal to the threshold α th2 or more (α≧α th2 ), the one feature model is the shape data SD i This threshold α th2 is the above-mentioned threshold α th1 The processor 32 sets the work model 100M nAt least one feature model in the shape data SD i If the feature is similar to that of the work model 100M, the determination is YES, and the process proceeds to step S14. n All feature models included in the shape data SD i If the feature is not similar to the feature of the feature, the result is determined as NO, and the process proceeds to step S14.

[0170] In step S14, the processor 32 functions as the information acquisition unit 46 to acquire the workpiece model 100M. n Among the multiple feature models included in the above, similarity information SI is obtained for feature models that have been determined to have a similar relationship in the immediately preceding step S13. For example, suppose that in the immediately preceding step S13, for the workpiece model 100M2 in the second orientation OR2 (in other words, the number "n"=2) shown in Fig. 22(a), the main body model 102M of the workpiece model 100M2 has been determined to have a similar relationship to the shape data SD1 depicting the main body 102 of the workpiece 100 in the first orientation OR1 shown in Fig. 22(b).

[0171] In this case, the processor 32 generates similarity information SI 2_1 In this embodiment, the similarity information SI 2_1 The linking information I links the main body model 102M of the workpiece model 100M2 in the second orientation OR2 with the main body model 102M of the workpiece model 100M1 in the first orientation OR1 as a similarity relationship. 2_1 Includes.

[0172] As an example, the operator visually checks the matching result between the workpiece model 100M2 and the shape data SD1 in step S5 displayed on the display device 40, and recognizes that the main body model 102M of the workpiece model 100M2 in the second orientation OR2 is in a similar relationship with the main body model 102M of the workpiece model 100M1 in the first orientation OR1. Then, the operator operates the input device 42 to input the above-mentioned linking information I 2_1 may be manually entered into the processor 32.

[0173] As another example, when the most recent step S5 is executed based on the detection data DD obtained by executing the above-mentioned simulation SL in step S3, the processor 32 may obtain the association information I from the matching result in step S5. 2_1 In the case of the simulation SL, since the position and posture of the workpiece 100 (for example, the workpiece model 100M) placed in the virtual space are known, the processor 32 can automatically recognize that there is a similarity relationship between the main body model 102M of the workpiece model 100M2 in the second posture OR2 and the main body model 102M of the workpiece model 100M1 in the first posture OR1.

[0174] On the other hand, in the immediately preceding step S13, for the work model 100M4 in the fourth orientation OR4 (in other words, the number "n" = 4) shown in Figure 23(a), it is determined that the main body model 102M of the work model 100M4 is in a similar relationship to the shape data SD1 that depicts the main body 102 of the workpiece 100 in the first orientation OR1 shown in Figure 23(b).

[0175] In this case, the processor 32 generates similarity information SI that identifies that the main body model 102M included in the workpiece model 100M4 in the fourth orientation OR4 is in a similar relationship with the main body 102 of the workpiece 100 in the first orientation OR1. 4_1 Get similar information SI 4_1 The linking information I links the main body model 102M of the workpiece model 100M4 in the fourth orientation OR4 with the main body model 102M of the workpiece model 100M1 in the first orientation OR1 as a similarity relationship.4_1 After step S14, the processor 32 proceeds to step S8.

[0176] After executing the flow of FIG. 21 (i.e., after acquiring the similar information SI and the non-similar information NI), the processor 32 executes the flow shown in FIG. 16. Here, the flow of FIG. 16 executed in this embodiment differs from the above-mentioned embodiment in step S23. Step S23 executed in this embodiment will be described below with reference to FIG. 24. Note that in the flow shown in FIG. 24, the same step numbers are assigned to processes similar to those in the flow of FIG. 19, and duplicated descriptions will be omitted.

[0177] 24, if the processor 32 determines NO in step S42 or YES in step S43, the process proceeds to step S47. In step S47, the processor 32 3_i 100M work model used for matching to obtain n It is determined whether similar information SI has been acquired for the

[0178] As an example, the number “j” of the position data PD at this time is set to j=1, and the first (i.e., number j=1) position data PD stored in the detection result data DT in Table 1 is 3_1 23(c), assume that the workpiece model 100M4 in the fourth orientation OR4 has been obtained as a result of inappropriate matching with the shape data SD1. Since no dissimilar shape NI has been obtained for this workpiece model 100M4, the processor 32 determines NO in step S42 and executes step S47.

[0179] In this case, the processor 32 receives the first position data PD 3_1 For the workpiece model 100M4 used to acquire the similarity information SI 4_1 Therefore, in this case, the processor 32 determines that the answer is YES.

[0180] As another example, j=1 is set at this point, and the first position data PD 3_1 However, as shown in Fig. 22(c), it is assumed that the workpiece model 100M2 in the second orientation OR2 is obtained as a result of inappropriate matching with the shape data SD1. In addition, dissimilarity information NI for identifying that the convex part model 106M of this workpiece model 100M2 is in a dissimilar relationship with the feature of the workpiece 100 in the third orientation OR3 (Fig. 10(b)) is obtained. 2_3 Assume that the following has been obtained.

[0181] In this case, the processor 32 determines YES in step S42. Also, as shown in FIG. 2_3 The convex portion model 106M of the workpiece model 100M2 identified by is consistent with the feature (i.e., the convex portion 106) depicted in the shape data SD1 of the workpiece 100 in the first orientation OR1. Therefore, the processor 32 determines YES in step S43.

[0182] In this case, in step S47, the processor 32 calculates the first position data PD 3_1 Regarding the workpiece model 100M2 used to acquire the similarity information SI 2_1 Therefore, in this case, the processor 32 determines that the answer is YES.

[0183] If the result of the determination is YES, the processor 32 proceeds to step S48. 3_i The work model 100M used to obtain n If similarity information SI has not been acquired for, the result is NO, and the process proceeds to step S45.

[0184] In step S48, the processor 32 calculates the j-th position data PD 3_i The work model 100M used to obtainn A workpiece model 100M different from the workpiece model 100M obtained in step S1 described above is selected.

[0185] For example, as described above, the first position data PD 3_1 When the inappropriate matching shown in FIG. 22C occurs during acquisition of the similarity information SI 2_1 The processor 32 refers to the similarity information SI 2_1 From the first position data PD 3_1 It can be recognized that the main body model 102M of the workpiece model 100M2 used when acquiring the first orientation OR1 is in a similar relationship to the main body 102 of the workpiece 100 in the first orientation OR1.

[0186] The processor 32 then processes this similarity information SI 2_1 Linking information included in I 2_1 , and the workpiece model 100M2 in the second posture OR2 includes a feature model that is in a similar relationship to the main body model 102M of the workpiece model 100M2 in the second posture OR2. n As multiple workpiece models 100M n From among these, the workpiece model 100M1 in the first orientation OR1 is selected.

[0187] Or, as described above, the first position data PD 3_1 When the inappropriate matching shown in FIG. 23C occurs during acquisition of the fourth posture OR4, the processor 32 uses the similarity information SI 4_1 Then, the processor 32 refers to this similarity information SI 4_1 Linking information included in I 4_1 , and the workpiece model 100M4 in the fourth posture OR4 includes a feature model that is similar to the main body model 102M of the workpiece model 100M4. n As multiple workpiece models 100M n From among these, the workpiece model 100M1 in the first orientation OR1 is selected.

[0188] In this way, the processor 32 derives the position data PD by matching the workpiece model 100M2 or 100M4 to the shape data SD1. 3_1 When you obtain similar information SI 2_1 or SI 4_1 The workpiece model 100M1 including the feature model 102M corresponding to the feature (main body portion 102) that is identified to have a similar relationship with the feature model 102M by the above is selected as a plurality of workpiece models 100M n Therefore, the processor 32 selects from among the multiple work models 100M. n It functions as a model selection unit 64 (FIG. 20) that selects the workpiece model 100M1 from among the above.

[0189] In step S49, the processor 32 functions as the position data acquisition unit 58, and acquires the workpiece model 100M selected in the previous step S48 in accordance with a predetermined matching algorithm. n The jth position data PD 3_i The shape data SD i For example, in the case of the above-described example of Fig. 22(c) or Fig. 23(c), the processor 32 matches the workpiece model 100M1 selected in the previous step S48 with the shape data SD1.

[0190] In step S50, the processor 32 functions as the consistency determination unit 62 and determines whether the work model 100M that was matched in the previous step S49 is a matched work model 100M. n Among the feature models included in, the feature model identified by the dissimilarity information NI is the j-th position data PD 3_i The shape data SD i It is then determined whether the features match those captured in the image.

[0191] For example, when the workpiece model 100M1 is matched with the shape data SD1 in the immediately preceding step S49, the processor 32 generates dissimilarity information NI 1_2 It is determined whether the end face model 110M identified by is consistent with the features depicted in the shape data SD1.

[0192] Specifically, the processor 32 obtains a degree of match γ″ between the features of the end face model 110M and the shape data SD1, and determines whether the degree of match γ″ is greater than or equal to a predetermined threshold value γ th If γ exceeds γ th ", or γ"<γ th " and non-similar information NI 1_2 It is determined that the end face model 110M identified by is consistent with the features depicted in the shape data SD1 (i.e., YES).

[0193] When the workpiece model 100M1 matches the shape data SD1 (for example, see FIG. 18), the processor 32 determines YES in this step S50. When the processor 32 determines YES, the process proceeds to step S44, where it functions as the position data cancellation unit 60 and cancels the first position data PD acquired in the above-mentioned step S22. 3_1 Disable.

[0194] If the result of the determination in step S50 is YES, the first position data PD acquired in step S22 3_1 This means that the matching performed when obtaining the image was inappropriate (FIG. 22(c) or FIG. 23(c)), whereas the matching performed in step S49 was appropriate.

[0195] Therefore, when the determination in step S50 is YES, the processor 32 executes step S44 to obtain the first position data PD 3_1 On the other hand, in step S50, the processor 32 determines whether the feature model identified by the dissimilarity information NI is invalidated in the shape data SD i If the feature does not match the feature shown in the image, the result is determined to be NO, and the process proceeds to step S45.

[0196] As described above, in this embodiment, the processor 32 functions as a model acquisition unit 44, an information acquisition unit 46, a feature extraction unit 48, a model matching execution unit 50, a frequency calculation unit 52, a similarity determination unit 54, an inconsistency determination unit 56, a position data acquisition unit 58, a position data cancellation unit 60, a consistency determination unit 62, and a model selection unit 64 to generate the workpiece model 100M. n Obtain the dissimilarity information NI of

[0197] Therefore, the model acquisition unit 44, the information acquisition unit 46, the feature extraction unit 48, the model matching execution unit 50, the frequency calculation unit 52, the similarity determination unit 54, the inconsistency determination unit 56, the position data acquisition unit 58, the position data cancellation unit 60, the consistency determination unit 62, and the model selection unit 64 constitute an apparatus 88 (Figure 20) for acquiring dissimilarity information NI.

[0198] In this device 88, the information acquisition unit 46 acquires similarity information SI for identifying a feature model (main body part model 102M) that is similar to the feature (main body part 102) of the workpiece 100 in the first orientation OR1 in the workpiece model 100M2 (or 100M4) in the second orientation OR2 (or the fourth orientation OR4). 2_1 (or SI 4_1 ) is obtained (step S14).

[0199] Then, the model selection unit 64 obtains the position data PD by the position data acquisition unit 58 matching the workpiece model 100M2 (or 100M4) to the shape data SD1. 3_1 When you obtain similar information SI 2_1 (or SI 4_1 ) to identify the feature model 102M as being in a similar relationship with the feature model 102M (main body portion 102), the workpiece model 100M1 in the first posture OR1 is compared with the plurality of workpiece models 100M. n (Step S48).

[0200] In addition, when the position data acquisition unit 58 matches the workpiece model 100M1 selected by the model selection unit 64 with the shape data SD1 again (step S49), the matching determination unit 62 obtains the dissimilarity information NI1_2 It is determined whether the feature model (end face model 110M) identified by the above is consistent with the feature (end face 110) depicted in the shape data SD1 (step S50).

[0201] Then, when the consistency determination unit 62 determines that the feature model 110M is consistent with the feature 110 (i.e., YES in step S50), the position data cancellation unit 60 cancels the position data PD 3_1 According to this configuration, if the position data PD 3_i When the position data PD is acquired, 3_i can be avoided from being used in the control of step S25 described above.

[0202] When the processor 32 determines that the result of the determination in step S50 is YES, the processor 32 functions as a position data acquisition unit 58 to acquire the position data PD 3_1 Then, in a subsequent step S44, the processor 32 obtains the original position data PD 3_1 While disabling the newly acquired location data PD 3_1 ' may be added to the detection result data DT.

[0203] As a result of step S22, one shape data SD i For example, in step S22, the processor 32 matches the workpiece model 100M1 (FIG. 3) in the first orientation OR1 with the shape data SD1 shown in FIG. 22(b) to obtain multiple pieces of position data PD 3_1_1 Get the.

[0204] At the same time, in step S22, the processor 32 derives position data PD3_1_2 can be obtained. Thus, for one shape data SD1, a plurality of position data PD 3_1_1 and PD 3_1_2 An example of the data structure of the detection result data DT when obtained is shown in Table 2 below.

[0205]

Table 2

[0206] In this case, the total number i of the shape data SD i appearing in the detection data DD and the total number j of the position data PD 3_i obtained in step S22 may be different (i < j). In this case, the processor 32 executes the flow of FIG. 24 to invalidate the position data PD 3_1_2 obtained as a result of the inappropriate matching shown in FIG. 22(c). On the other hand, the processor 32 will maintain the position data PD 3_1_1 obtained as a result of the appropriate matching.

[0207] Note that the processor 32 may execute the flows shown in FIGS. 12, 14, 16, 17, 19, 21, or 24 according to a computer program PG stored in advance in the memory 34. Also, the functions of the devices 80, 82, 84, 86, or 88 (that is, the model acquisition unit 44, the information acquisition unit 46, the feature extraction unit 48, the model matching execution unit 50, the frequency calculation unit 52, the similarity determination unit 54, the inconsistency determination unit 56, the position data acquisition unit 58, the position data cancellation unit 60, the consistency determination unit 62, the model selection unit 64) executed by the processor 32 may be functional modules realized by the computer program PG.

[0208] Also, various changes can be made to the flows shown in FIGS. 12, 14, 16, 17, 19, 21, or 24. For example, in the above-described embodiment, in the flow of FIG. 16, the case where the processor 32 executes step S23 after step S22 (more specifically, step S34) has been described.

[0209] However, the present invention is not limited to this, and the processor 32 may execute step S23 after step S32 or S33 in Fig. 17. For example, when executing step S23 after step S32, the processor 32 1_i 19 or steps S41 to S50 in FIG. 24 are executed based on the above. Then, when the result of the determination in step S43 is NO (or the result of the determination in S50 is YES), the initial position data PD 1_i Disable.

[0210] Furthermore, the coarse search RS of step S32 and the fine search PS of step S33 described above are examples of methods for acquiring the position data PD of the work 100, and any modifications may be made to the coarse search RS or the fine search PR, or the position data PD of the work 100 may be acquired by any other method that does not perform the coarse search RS and the fine search PR.

[0211] 12, 14 or 21. In this case, the feature extraction unit 48 can be omitted from the device 82, 84, 86 or 88. Further, step S12 can be omitted from the flow of Fig. 21. The inconsistency determination unit 56 can be omitted from the device 88.

[0212] 12, 14, or 21, the processor 32 may execute model matching similar to the above-mentioned step S32 (coarse search RS) or step S33 (fine search PS). Also, in step S6 in FIG. 12, 14, or 21, the processor 32 may execute a matching degree γ i , plane proportion δ i , and exposure rate ε i For example, in step S6, the processor 32 determines the frequency α by taking into account the detection result parameter PM such as the threshold value β th1 and the detection result parameter PM satisfies a predetermined standard (for example, exceeds a standard value), the frequency α may be calculated.

[0213] 12, 14, or 21, the shape detection sensor 14 does not necessarily have to be disposed at a known position in the control coordinate system C (specifically, the robot coordinate system C1). Then, the processor 32 detects whether the shape detection sensor 14 disposed at an arbitrary position has various postures OR n The detection data DD may be obtained by repeatedly imaging the workpieces 100 one by one, and the matching in step S5 may be performed for each detection data DD. As described above, various modifications may be made to the flow shown in FIG. 12, FIG. 14, FIG. 16, FIG. 17, FIG. 19, FIG. 21, or FIG. 24.

[0214] In addition, in the above-mentioned embodiment, the cases have been described in which the functions of devices 80, 82, 84, 86 and 88 (i.e., model acquisition unit 44, information acquisition unit 46, feature extraction unit 48, model matching execution unit 50, frequency calculation unit 52, similarity determination unit 54, inconsistency determination unit 56, position data acquisition unit 58, position data cancellation unit 60, consistency determination unit 62, model selection unit 64) are implemented in the control device 16.

[0215] However, the present invention is not limited thereto, and the functions of the devices 80, 82, 84, 86, or 88 may be implemented in, for example, a teaching device (such as a teaching pendant or a tablet-type mobile terminal) that teaches the robot 12 the operation for the task of step S25, or any other computer such as a PC. In this case, a processor of the teaching device or a computer such as a PC functions as the devices 80, 82, 84, 86, or 88.

[0216] In the above embodiment, the shape detection sensor 14 is fixed to the end effector 28 (or the wrist flange 26b) and moved by the robot 12. However, this is not limiting, and the shape detection sensor 14 may be fixed to a known position in the control coordinate system C (robot coordinate system C1) using, for example, a support structure.

[0217] Furthermore, the shape detection sensor 14 is not limited to a three-dimensional visual sensor, and may be a laser scanner capable of detecting the three-dimensional shape of an object. Alternatively, the shape detection sensor 14 may have a two-dimensional camera and a distance measuring sensor capable of measuring the distance d to a subject. Furthermore, the detection data DD detected by the shape detection sensor 14 is not limited to the three-dimensional point cloud image data as shown in Fig. 6, and may include a data set of two-dimensional image data captured by a two-dimensional camera and the distance d measured by a distance measuring sensor, or may be any other type of image data (for example, distance image data).

[0218] In addition, the robot 12 is not limited to a vertical articulated robot, and may be any type of robot, such as a horizontal articulated robot, a parallel link robot, etc. Although the present disclosure has been described above through the embodiments, the above-described embodiments do not limit the invention according to the claims.

[0219] As described above, the present disclosure describes the following aspects. (Embodiment 1) An apparatus comprising: a model acquisition unit that acquires multiple work models that respectively model workpieces in multiple postures, the workpiece models including feature models corresponding to visual features of the workpiece; and an information acquisition unit that acquires dissimilarity information for identifying a first feature model in a first workpiece model in one posture that is dissimilar to the features of the workpiece in other postures. (Aspect 2) The device described in aspect 1 further includes a feature extraction unit that extracts multiple feature models contained in each work model from the work model and assigns an identification code to each of the extracted feature models, and the information acquisition unit acquires dissimilarity information in association with the identification code assigned to the first feature model. (Aspect 3) The device described in aspect 1 or 2 further includes a model matching execution unit that matches a work model to shape data of the work detected by the shape detection sensor, a frequency calculation unit that determines the frequency at which a feature model included in the first work model matches features depicted in each of the shape data when the model matching execution unit matches the first work model to each of the shape data of a plurality of postures, and a similarity determination unit that determines whether each of the feature models included in the first work model is in a dissimilar relationship based on the frequency determined for the plurality of feature models included in the first work model by the frequency calculation unit, and the information acquisition unit acquires dissimilarity information for a first feature model among the plurality of feature models included in the first work model that is determined to be in a dissimilar relationship by the similarity determination unit. (Aspect 4) In the device according to aspect 3, the similarity determination unit determines that the feature models are in a dissimilar relationship when the frequency calculated by the frequency calculation unit is equal to or less than a predetermined threshold value. (Aspect 5) The device described in aspect 3 or 4, further comprising an inconsistency determination unit that determines whether or not an inconsistency occurs between at least one feature model included in the first work model and a feature reflected in the shape data when the model matching execution unit performs matching using the first work model, and the frequency calculation unit, when the inconsistency determination unit determines that an inconsistency occurs, calculates the frequency at which a feature model other than the at least one feature model matches the feature. (Aspect 6) An apparatus described in any of aspects 3 to 5, wherein the model matching execution unit performs matching using shape data actually detected by a shape detection sensor in real space to detect the shape of the workpiece, or performs matching using shape data obtained by a simulation in which a shape detection sensor simulates the detection of the shape of the workpiece in virtual space. (Aspect 7) An apparatus described in any of aspects 1 to 6, further comprising a position data acquisition unit that acquires position data of the workpiece in the control coordinate system by matching a workpiece model to shape data of the workpiece detected by a shape detection sensor arranged at a known position in the control coordinate system, and a position data cancellation unit that invalidates the position data acquired by the position data acquisition unit when an inconsistency occurs between the shape data and the workpiece model matched to the shape data by the position data acquisition unit to acquire the position data. (Aspect 8) The device described in aspect 7, further comprising a consistency determination unit that determines whether the first feature model identified by the dissimilarity information is consistent with the features depicted in the shape data when the position data acquisition unit acquires first position data by matching the first work model to the shape data, and the position data cancellation unit invalidates the first position data acquired by the position data acquisition unit when the consistency determination unit determines that the first feature model is inconsistent with the features. (Aspect 9) The information acquisition unit further acquires similarity information for identifying a second feature model in a second work model of another posture that has a similar relationship to the features of the work in one posture, and the device further includes a model selection unit that selects from the multiple work models a first work model including a feature model corresponding to the features for which the second feature model is identified as having a similar relationship by the similarity information when the position data acquisition unit acquires second position data by matching the second work model to the shape data, and a consistency determination unit that determines whether the first feature model identified by the dissimilarity information is consistent with the features appearing in the shape data when the position data acquisition unit again matches the first work model selected by the model selection unit to the shape data, and the position data cancellation unit invalidates the second position data acquired by the position data acquisition unit when the consistency determination unit determines that the first feature model is consistent with the features. (Aspect 10) A control device including the device according to any one of aspects 1 to 9, for controlling the operation of a robot that performs a predetermined task on a workpiece. (Aspect 11) A robot system including a robot that performs a predetermined task on a workpiece, a shape detection sensor that detects the shape of the workpiece, and the device according to any one of aspects 1 to 9. (Embodiment 12) A method in which a processor acquires a plurality of work models, each of which models a work in a plurality of postures, and each of which includes a feature model corresponding to a visual feature of the work, and acquires dissimilarity information for identifying a first feature model in a first work model in one posture that is in a dissimilar relationship to the features of the work in other postures. [Explanation of symbols]

[0220] 10 Robot System 12. Robot 14 Shape detection sensor 16 Control device 32 processors 44 Model Acquisition Department 46 Information Acquisition Department 48 Feature Extraction Unit 50 Model matching execution unit 52 Frequency calculation section 54 Similarity determination section 56 Inconsistency judgment part 58 Location data acquisition unit 60 Location data cancellation unit 62 Consistency judgment section 64 Model Selection Section 100 Work 100M Work Model

Claims

1. A model acquisition unit that acquires a plurality of workpiece models each of which is modeled with a plurality of workpiece postures, the workpiece models including a feature model corresponding to a visual feature of the workpiece; An apparatus comprising: an information acquisition unit that acquires dissimilarity information for identifying a first feature model in a first work model in one of the postures, the first feature model being in a dissimilar relationship to the features of the work in another of the postures.

2. A feature extraction unit is further provided which extracts a plurality of feature models included in each of the workpiece models from the workpiece models and assigns an identification code to each of the extracted feature models; The device according to claim 1 , wherein the information acquisition unit acquires the dissimilarity information in association with the identification code assigned to the first feature model.

3. a model matching execution unit that matches the workpiece model to shape data of the workpiece detected by a shape detection sensor; a frequency calculation unit that calculates a frequency at which the feature model included in the first work model matches the features depicted in each of the shape data when the model matching execution unit matches the first work model to each of the shape data of a plurality of postures; The frequency calculation unit further includes a similarity determination unit that determines whether or not each of the feature models is in the dissimilar relationship based on the frequency calculated for the feature models included in the first workpiece model, The device according to claim 1 , wherein the information acquisition unit acquires the dissimilarity information for the first feature model among the plurality of feature models included in the first work model that is determined to be in the dissimilar relationship by the similarity determination unit.

4. The device according to claim 3 , wherein the similarity determination unit determines that the feature models are in the dissimilar relationship when the frequency calculated by the frequency calculation unit is equal to or smaller than a predetermined threshold value.

5. and a mismatch determination unit that determines whether or not a mismatch occurs between at least one of the feature models included in the first workpiece model and the feature depicted in the shape data when the model matching execution unit executes the matching using the first workpiece model, The device according to claim 3 , wherein the frequency calculation unit calculates the frequency at which the feature model other than the at least one feature model matches the feature when the inconsistency determination unit determines that the inconsistency occurs.

6. The model matching execution unit is The matching is performed using the shape data obtained by actually detecting the shape of the workpiece in real space using the shape detection sensor, or The apparatus according to claim 3 , wherein the matching is performed using the shape data acquired by a simulation in which the shape detection sensor detects the shape of the workpiece in a virtual space.

7. a position data acquisition unit that acquires position data of the workpiece in a control coordinate system by matching the workpiece model to shape data of the workpiece detected by a shape detection sensor arranged at a known position in the control coordinate system; The apparatus of claim 1 , further comprising a position data cancellation unit that invalidates the position data acquired by the position data acquisition unit when an inconsistency occurs between the shape data and the work model that the position data acquisition unit has matched to the shape data to acquire the position data.

8. and a consistency determination unit that determines whether or not the first feature model identified by the dissimilarity information is consistent with the features depicted in the shape data when the position data acquisition unit acquires the first position data by matching the first workpiece model to the shape data, The device according to claim 7 , wherein the position data cancellation unit invalidates the first position data acquired by the position data acquisition unit when the consistency determination unit determines that the first feature model is inconsistent with the feature.

9. The information acquisition unit further acquires similarity information for identifying a second feature model in a second workpiece model in the other posture that is in a similar relationship to the feature of the workpiece in the one posture, The apparatus comprises: a model selection unit that selects, from among the plurality of work models, the first work model including the feature model corresponding to the feature that is identified as being in the similar relationship with the second feature model by the similarity information when the position data acquisition unit acquires the second position data by matching the second work model to the shape data; a consistency determination unit that determines whether or not the first feature model identified by the dissimilarity information is consistent with the features depicted in the shape data when the position data acquisition unit matches the first workpiece model selected by the model selection unit to the shape data again, The device according to claim 7 , wherein the position data cancellation unit invalidates the second position data acquired by the position data acquisition unit when the consistency determination unit determines that the first feature model is consistent with the feature.

10. A control device comprising the device according to claim 1 and controlling an operation of a robot that performs a predetermined task on the workpiece.

11. A robot that performs a predetermined task on a workpiece; A shape detection sensor for detecting a shape of the workpiece; A robot system comprising the apparatus according to claim 1.

12. The processor: Acquire a plurality of workpiece models each of which is a model of a workpiece having a plurality of postures, the plurality of workpiece models each including a feature model corresponding to a visual feature of the workpiece; The method includes obtaining dissimilarity information for identifying a first feature model in a first workpiece model in one of the poses that is in a dissimilar relationship to the features of the workpiece in another of the poses.

Images