Apparatus, robot control device, robot system, method

Abstract

In the past, there have been cases in which a workpiece model is incorrectly matched to detection data from a shape detection sensor, thereby causing incorrect detection of position data.　This device comprises: a model acquisition unit that acquires a plurality of workpiece models resulting from modeling a workpiece in a plurality of orientations, the workpiece models including a feature model corresponding to a visual feature of the workpiece; and an information acquisition unit that, with respect to a first workpiece model in one orientation, acquires non-similar information for identifying a first feature model that is in a non-similar relationship with the features of the workpiece in other orientations.

Description

[Technical Field]

[0001] This disclosure relates to an apparatus, a robot control device, a robot system, and a method for acquiring dissimilar information in a work model. [Background technology]

[0002] A device is known that acquires position data of a workpiece by matching a workpiece model, which is a model of the workpiece, with shape data of the workpiece detected by a shape detection sensor (for example, a 3D vision sensor) (for example, Patent Document 1). [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2017-102529 [Overview of the project] [Problems that the invention aims to solve]

[0004] Conventionally, incorrect positional data detection sometimes occurred due to the mismatching of the work model with the detection data from the shape detection sensor. [Means for solving the problem]

[0005] In one embodiment of the present disclosure, the apparatus includes a model acquisition unit that acquires a plurality of work models, each modeling a work in a plurality of orientations, wherein the work model includes a feature model corresponding to the visual features of the work, and an information acquisition unit that acquires dissimilarity information for identifying a first feature model in a first work model of one orientation that is dissimilar to the features of a work in another orientation.

[0006] In another embodiment of the present disclosure, the method involves a processor acquiring a plurality of work models, each modeling a work in a plurality of orientations, each including a feature model corresponding to the visual features of the work; and acquiring dissimilarity information for identifying a first feature model in a first work model of one orientation that is dissimilar to the features of a work in another orientation. [Brief explanation of the drawing]

[0007] [Figure 1] This is a schematic diagram of a robot system according to one embodiment. [Figure 2] Figure 1 is a block diagram of the robot system shown. [Figure 3] The workpiece and workpiece model are shown positioned in the first orientation. [Figure 4] The workpiece and workpiece model are shown in the second orientation. [Figure 5] The workpiece and workpiece model are shown in the third orientation. [Figure 6] An example of detection data detected by a shape detection sensor is shown. [Figure 7] This diagram illustrates the matching of a work model with geometric data, showing an example where the work model is properly matched to the geometric data. [Figure 8] This diagram illustrates the matching of a work model with geometric data, and shows an example where the work model does not match the geometric data properly. [Figure 9] This diagram illustrates the matching of a work model with geometric data, showing an example where the work model is properly matched to the geometric data. [Figure 10] This diagram illustrates the matching of a work model with geometric data, and shows an example where the work model does not match the geometric data properly. [Figure 11] This is a block diagram showing other functions of the robot system. [Figure 12] Figure 11 is a flowchart showing an example of the operation flow of a robot system. [Figure 13] This block diagram shows other features of the robot system. [Figure 14] Figure 13 is a flowchart showing an example of the operation flow of a robot system. [Figure 15] This block diagram shows other features of the robot system. [Figure 16] Figure 15 is a flowchart showing an example of the operation flow of a robot system. [Figure 17] This is a flowchart showing an example of the flow of step S22 in Figure 16. [Figure 18] This shows the work model of the first orientation OR appropriately matched to the shape data. [Figure 19] This is a flowchart showing an example of the flow of step S23 in Figure 16. [Figure 20] This block diagram shows other features of the robot system. [Figure 21] Figure 20 is a flowchart showing an example of the operation flow of a robot system. [Figure 22] This diagram illustrates the matching of a work model with geometric data, and shows an example where the work model does not match the geometric data properly. [Figure 23] This diagram illustrates the matching of a work model with geometric data, and shows an example where the work model does not match the geometric data properly. [Figure 24] Figure 20 is a flowchart showing an example of the flow of step S23 in Figure 16, which is performed by the robot system shown in Figure 20. [Modes for carrying out the invention]

[0008] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the various embodiments described below, similar elements will be denoted by the same reference numerals, and redundant descriptions will be omitted. First, a robot system 10 according to one embodiment will be described with reference to Figures 1 and 2. The robot system 10 comprises a robot 12, a shape detection sensor 14, and a control device 16.

[0009] In this embodiment, the robot 12 is a vertical articulated robot having a robot base 18, a swivel torso 20, a forearm 22, an upper arm 24, a wrist 26, and an end effector 28. The robot base 18 is fixed to the floor of the work cell. The swivel torso 20 is mounted on the robot base 18 so as to be able to swivel around a vertical axis.

[0010] The lower arm portion 22 is mounted on the swivel body 20 so as to be rotatable around a horizontal axis at its base end, and the upper arm portion 24 is mounted on the tip of the lower arm portion 22 so as to be rotatable at its base end. The wrist portion 26 has a wrist base 26a mounted on the tip of the upper arm portion 24 so as to be rotatable around two mutually orthogonal axes, and a wrist flange 26b mounted on the wrist base 26a so as to be rotatable around a wrist axis A1.

[0011] The end effector 28 is detachably attached to the wrist flange 26b. The end effector 28 is, for example, a robot hand capable of gripping the workpiece 100, a welding torch for welding the workpiece 100, or a laser processing head for laser processing the workpiece 100, and performs a predetermined operation (workpiece handling, welding, or laser processing, etc.) on the workpiece 100.

[0012] Each component of the robot 12 (robot base 18, swivel torso 20, forearm 22, upper arm 24, wrist 26) is equipped with a servo motor 30 (Figure 2). These servo motors 30 rotate each movable element of the robot 12 (swivel torso 20, forearm 22, upper arm 24, wrist base 26a, wrist flange 26b) in response to commands from the control device 16. As a result, the robot 12 can move the end effector 28 to any desired position.

[0013] The shape detection sensor 14 detects the shape of the workpiece 100. In this embodiment, the shape detection sensor 14 is a three-dimensional vision sensor having an imaging sensor (CMOS, CCD, etc.) and an optical lens (collimating lens, focusing lens, etc.) that guides the image of the subject to the imaging sensor, and is fixed to the end effector 28 (or wrist flange 26b).

[0014] The shape detection sensor 14 is configured to image the subject along the optical axis A2 and to measure the distance d to the subject. The shape detection sensor 14 may be fixed to the end effector 28 so that the optical axis A2 and the wrist axis A1 are parallel (or perpendicular) to each other. The shape detection sensor 14 supplies the detected detection data DD to the control device 16.

[0015] As shown in Figure 1, the robot 12 is configured with a robot coordinate system C1 and a tool coordinate system C2. The robot coordinate system C1 is a control coordinate system C for controlling the movement of each movable element of the robot 12. In this embodiment, the robot coordinate system C1 is fixed to the robot base 18 such that its origin is located at the center of the robot base 18 and its z-axis is parallel to (specifically coincides with) the rotation axis of the swivel body 20.

[0016] On the other hand, the tool coordinate system C2 is a control coordinate system C that defines the position and orientation of the end effector 28 in the robot coordinate system C1. In this embodiment, the tool coordinate system C2 is set relative to the end effector 28 such that its origin (so-called TCP) is located at the working position of the end effector 28 (for example, the workpiece gripping position, welding position, or laser beam output port), and its z-axis is parallel to (specifically coincides with) the wrist axis A1.

[0017] When moving the end effector 28, the control device 16 sets a tool coordinate system C2 in the robot coordinate system C1 and generates commands to each servo motor 30 of the robot 12 to position the end effector 28 at the position and orientation represented by the set tool coordinate system C2. In this way, the control device 16 can position the end effector 28 at any position and orientation in the robot coordinate system C1.

[0018] On the other hand, the shape detection sensor 14 is configured with a sensor coordinate system C3. Sensor coordinate system C3 is a control coordinate system C that defines the position and orientation of the shape detection sensor 14 in the robot coordinate system C1 (i.e., the position and direction of the optical axis A2). In this embodiment, the sensor coordinate system C3 is configured for the shape detection sensor 14 such that its origin is located at the center of the image sensor of the shape detection sensor 14, and its z-axis is parallel to (specifically coincides with) the optical axis A2. Sensor coordinate system C3 defines the coordinates of each pixel of the detection data DD (or image sensor) detected by the shape detection sensor 14.

[0019] The positional relationship between the sensor coordinate system C3 and the tool coordinate system C2 is known through calibration; therefore, the coordinates of the sensor coordinate system C3 and the coordinates of the tool coordinate system C2 are mutually convertible via a known transformation matrix (e.g., a homogeneous transformation matrix). Furthermore, since the positional relationship between the tool coordinate system C2 and the robot coordinate system C1 is known, the coordinates of the sensor coordinate system C3 and the coordinates of the robot coordinate system C1 are mutually convertible via the tool coordinate system C2. In other words, the position and orientation of the shape detection sensor 14 in the robot coordinate system C1 (specifically, the coordinates of the sensor coordinate system C3) are known.

[0020] The control device 16 controls the movement of the robot 12 to perform a predetermined operation (work handling, welding, or laser processing) on ​​the workpiece 100. Specifically, as shown in Figure 2, the control device 16 is a computer having a processor 32, memory 34, and I / O interface 36. The processor 32 has a CPU or GPU, and is connected to the memory 34 and I / O interface 36 via a bus 38 so as to be able to communicate with these components, and performs calculation processing to realize various functions described later.

[0021] Memory 34 has RAM or ROM, etc., and stores various data temporarily or permanently. Memory 34 can be composed of a storage medium such as volatile memory, non-volatile memory, magnetic storage medium, or optical storage medium. I / O interface 36 has, for example, an Ethernet® port, a USB port, an optical fiber connector, or an HDMI® terminal, and communicates data with external devices via wired or wireless connection under commands from processor 32. Each servo motor 30 and shape detection sensor 14 of robot 12 is communicated to I / O interface 36.

[0022] Furthermore, the control device 16 is equipped with a display device 40 and an input device 42. The display device 40 and the input device 42 are communicatively connected to the I / O interface 36. The display device 40 has a liquid crystal display or an organic EL display, and displays various data in a visible manner under commands from the processor 32.

[0023] The input device 42 has push buttons, switches, a keyboard, a mouse, or a touch panel, and accepts data input from the operator. The display device 40 and the input device 42 may be integrated into the housing of the control device 16, or they may be attached to the housing of the control device 16 as a separate computer (PC, etc.).

[0024] In this embodiment, the processor 32 acquires position data PD of the workpieces 100 in the robot coordinate system C1 based on the detection data DD of the shape detection sensor 14 that detects the shapes of multiple workpieces 100 that are loosely stacked in container B. An example of the workpieces 100 will be described below with reference to Figures 3 to 5.

[0025] In this embodiment, the workpiece 100 has a main body portion 102, a shaft portion 104, and a protrusion portion 106. The main body portion 102 has a rectangular prism shape. The shaft portion 104 is cylindrical and protrudes from one side of the main body portion 102. The shaft portion 104 has a cylindrical side surface 108 and a ring-shaped end surface 110. The protrusion portion 106 is rectangular prism shape and protrudes from the top surface of the main body portion 102.

[0026] These main body portion 102 (specifically, the edges and surfaces defining the main body portion 102), shaft portion 104 (specifically, the side surface 108, end surface 110, and their edges), and protrusion 106 (specifically, the edges and surfaces defining the protrusion 106) constitute the visual features 102, 104, 106, 108, and 110 of the workpiece 100.

[0027] Figures 3 and 4 show three different orientations of workpiece 100 from three different viewpoints. nThe workpieces 100 for (n = 1, 2, 3) are shown. Specifically, FIG. 3 shows the workpiece 100 in the first posture OR1, FIG. 4 shows the workpiece 100 in the second posture OR2, and FIG. 5 shows the workpiece 100 in the third posture OR3.

[0028] In the present embodiment, the processor 32 models a plurality of workpieces 100 in a plurality of postures OR as shown in FIGS. 3 to 5. n to obtain a plurality of workpiece models 100M. The workpiece model 100M n has, for example, a CAD model 100Mc of the workpiece 100 n and a point cloud model 100Mp that represents model components (edges, surfaces, etc.) of the CAD model 100Mc n in point clouds (or normal vectors). n n

[0029] The CAD model 100Mc n is a three-dimensional CAD model and is created in advance by an operator using a CAD device (not shown). The processor 32 acquires the CAD model 100Mc from the CAD device n and may generate the point cloud model 100Mp by assigning point clouds to the model components of the CAD model 100Mc according to a predetermined image generation algorithm. Note that the workpiece model 100M n n is not limited to the CAD model 100Mc and the point cloud model 100Mp, and may be any type of model that represents the shape of the workpiece 100. n n n

[0030] The processor 32 obtains a work model 100M1 of the first orientation OR1, which models the work 100 in the first orientation OR1 shown in Figure 3. In the following description, a model corresponding to a feature XXX (e.g., end face 110) of the work 100 will be referred to as a feature model XXXM (e.g., end face model 110M). The work model 100M1 of the first orientation OR1 includes a main body model 102M, a shaft model 104M (specifically, a side model 108M and an end face model 110M), and a convex part model 106M.

[0031] Furthermore, the processor 32 acquires a work model 100M2 of the second orientation OR2, which models the workpiece 100 in the second orientation OR2 shown in Figure 4. The work model 100M2 of the second orientation OR2 includes a main body model 102M, a shaft model 104M, and a convex part model 106M. However, in the work model 100M2, the shaft model 104M has a side model 108M, but does not have an end face model 110M (i.e., it is not visible from the viewpoint in Figure 4).

[0032] Furthermore, the processor 32 acquires a work model 100M3 of the workpiece 100 in the third orientation OR3 shown in Figure 5. The work model 100M3 of the workpiece 100 in the third orientation OR3 includes the main body model 102M and the shaft model 104M (specifically, the side model 108M and the end face model 110M), but does not include the convex part model 106M (i.e., it is not visible from the viewpoint in Figure 5).

[0033] Work Model 100M n Each of these is assigned a work coordinate system C4. This work coordinate system C4 is the work model 100M in the control coordinate system C (specifically, the robot coordinate system C). nThis is a coordinate system for representing the position and orientation of the workpiece model 102M. In this embodiment, the workpiece coordinate system C4 has its origin at one vertex O of the main body model 102M, its x-axis is parallel to the width direction of the main body model 102M, its y-axis is parallel to the length direction of the main body model 102M (or the axis of the shaft model 104M), and its z-axis is parallel to the height direction of the main body model 102M, with respect to the workpiece model 100M. n It is set for.

[0034] The processor 32 can perform various orientations OR as illustrated in Figures 3 to 5. n Multiple work models 100M n These are acquired individually as separate model data. Note that in the example shown in Figures 3 to 5, three poses OR n Work model 100M for (n=1,2,3) n Although this is shown as an example, the processor 32 has 3 or more orientations OR n Work model 100M (for example, n=1 to 500) n Each of these may be obtained. Thus, in this embodiment, the processor 32 obtains multiple attitude OR n Multiple work models 100M, each modeling a different work 100. n It functions as a model acquisition unit 44 (Figure 2) that obtains the data.

[0035] Note: Work model 100M n The model may have only the front-side model data that is visible from the corresponding viewpoint, and may not have the back-side model data that is not visible from that viewpoint. For example, in the case of the work model 100M2 in Figure 4, it may have only the model data for the front side of the paper in Figure 4 (model data for the main body model 102M, the side model 108M, and the protruding part model 106M) that is visible from the viewpoint in Figure 4, while it may not have the model data for the back side of the paper in Figure 4 (model data for the end face model 110M) that is not visible from the viewpoint in Figure 4.

[0036] More specifically, when the processor 32 generates the work model 100M2 in Figure 4 as a point cloud model 100Mp2, it generates point cloud model data for the model components on the front side of the paper that are visible in Figure 4, but does not need to generate point cloud model data for the model components on the back side of the paper that are not visible (i.e., edges and faces on the back side as seen from the viewpoint in Figure 4). The same applies to the work model 100M1 in Figure 3 and the work model 100M3 in Figure 5. With this configuration, the acquired work model 100M n This can reduce the amount of data.

[0037] Furthermore, the processor 32 receives the work model 100M through the input device 42. n The processor 32 may accept input from the operator to set the work coordinate system C4. n This information, along with the settings for the work coordinate system C4, is stored in memory 34.

[0038] Processor 32 acquired multiple work models 100M n Each of these is matched with the detection data DD detected by the shape detection sensor 14 when it images the workpiece 100 inside container B, thereby obtaining the position data PD of the workpiece 100 in the robot coordinate system C1. Figure 6 schematically shows an example of the imaged detection data DD.

[0039] In this embodiment, the detected data DD is a 3D point cloud image data, and the shape data SD is the shape data of the workpiece 100 in various orientations detected by the shape detection sensor 14. i (i=1,2,3,...) is included. Figure 6 shows an example where the shape data SD1 of the workpiece 100 in the first orientation OR1, the shape data SD2 of the workpiece 100 in the second orientation OR2, and the shape data SD3 of the workpiece 100 in the third orientation OR3 are captured, for ease of understanding. However, when the shape detection sensor 14 actually images the workpiece 100 that are loosely stacked in container B, the detected data DD will contain 3 or more shape data SDs. i Please understand that this may be captured in the image.

[0040] Each shape data SD i The data has a point cloud that represents the visual features of the workpiece 100 (i.e., the edges or faces of the main body 102, the edges or faces of the shaft portion 104 (side 108, end face 110), and the edges or faces of the protrusion 106), and each point constituting the point cloud has the distance d information described above. Therefore, the shape data SD i Each point that makes up the point cloud is in the 3D coordinate system (X) of the sensor coordinate system C3. S ,Y S ,Z S It can now be expressed as ).

[0041] The processor 32 processes each shape data SD that is reflected in the detected data DD in the sensor coordinate system C3. i Multiple work models 100M n The two are matched accordingly. For example, as shown in Figure 7, the processor 32 matches the work model 100M2 of the second orientation OR2 (Figure 7(a)) with the shape data SD2 (Figure 7(b)) reflected in the detected data DD.

[0042] In this case, as shown in Figure 7(c), all feature models of the work model 100M2 (body model 102M, side model 108M, and protrusion model 106M) match the features of the shape data SD2 (shape data of the body 102, side 108, and protrusion 106 of the work 100). That is, in this case, the work model 100M2 matches the shape data SD2. i When properly matched, the work coordinate system C4 set for the work model 100M2 at this time will accurately indicate the position and orientation of the work 100 detected as shape data SD2.

[0043] On the other hand, as shown in Figure 8, the processor 32 also matches the work model 100M1 in the first orientation OR1 (Figure 8(a)) to the shape data SD2 (Figure 8(b)) reflected in the detection data DD. In this case, as shown in Figure 8(c), the main body model 102M and the convex part model 106M of the work model 100M1 are matched to the features of the main body 102 and convex part 106 reflected in the shape data SD2.

[0044] On the other hand, the shaft model 104M of the workpiece model 100M1 (specifically, the side model 108M and the end face model 110M) is inconsistent with the characteristics of the shaft 104 as reflected in the shape data SD2. In other words, in this case, the workpiece model 100M1 is not properly matched to the shape data SD2, and the workpiece coordinate system C4 set for the workpiece model 100M1 at this time does not accurately represent the position and orientation of the workpiece 100 detected as shape data SD2.

[0045] Such an inappropriate matching can occur because the main body model 102M and the protrusion model 106M of the work model 100M1 are similar to the features of the main body 102 and protrusion 106 reflected in the shape data SD2. On the other hand, the end face model 110M of the work model 100M1 does not exist in the shape data SD2, and therefore is not similar to any of the features reflected in the shape data SD2. As a result, the end face model 110M, which is dissimilar to the features of the shape data SD2, does not match any of the features of the shape data SD2 in the inappropriate matching shown in Figure 8(c).

[0046] Furthermore, the processor 32 processes the shape data SD3, which is reflected in the detection data DD in Figure 6, into multiple work models 100M. n The two are matched accordingly. For example, as shown in Figure 9, the processor 32 matches the work model 100M3 in the third orientation OR3 (Figure 9(a)) with the shape data SD3 reflected in the detected data DD (Figure 9(b)). As a result, the work model 100M3 and the shape data SD3 are appropriately matched (Figure 9(c)).

[0047] On the other hand, as shown in Figure 10, the processor 32 also matches the work model 100M1 in the first orientation OR1 (Figure 10(a)) to the shape data SD3 (Figure 10(b)) reflected in the detection data DD. In this case, as shown in Figure 10(c), the convex part model 106M of the work model 100M1 does not match any feature of the shape data SD3, and therefore the work model 100M1 is not properly matched to the shape data SD3.

[0048] Such an inappropriate matching can occur because the main body model 102M and shaft model 104M of the work model 100M1 are similar to the features of the main body 102 and shaft 104 as depicted in the shape data SD3. On the other hand, the convex part model 106M of the work model 100M1 does not exist in the shape data SD3, and therefore is not similar to any of the features depicted in the shape data SD3. As a result, when an inappropriate matching is performed as shown in Figure 10(c), the convex part model 106M, which is dissimilar to the features of the shape data SD3, is not consistent with any of the features of the shape data SD3.

[0049] As mentioned above, the nth posture OR n Work model 100M n The mth position OR m (m≠n) Shape data SD m When matched, the work model 100M n Among the feature models included, shape data SD m (That is, the mth position OR m By focusing on feature models that are dissimilar to the features of workpiece 100 (for example, the end face model 110M in Figure 8(c), or the convex part model 106M in Figure 10(c)), it is possible to determine whether the matching is inappropriate or not.

[0050] Therefore, in this embodiment, the nth posture OR n Work model 100M n In this case, the mth posture OR mDissimilarity information NI is obtained to identify feature models that are dissimilar to the features of workpiece 100 (i.e., the end face model 110M in Figure 8(c), or the convex part model 106M in Figure 10(c)).

[0051] For example, the processor 32 identifies the end face model 110M as a feature model that is dissimilar to the features of the work model 100M1 in the first orientation OR1 shown in Figure 8 (i.e., the shape data SD2) in the second orientation OR2, by providing dissimilar information NI. 1_2 For example, the operator visually inspects the work model 100M1 (e.g., CAD model 100Mc1) displayed on the display device 40 to verify its characteristics and operates the input device 42 to obtain dissimilar information NI for identifying the end face model 110M. 1_2 You may also manually input this into processor 32.

[0052] As another example, the processor 32 attempts to match the workpiece model 100M1 to the detection data DD obtained when the shape detection sensor 14 images the workpiece 100 in real or virtual space, thereby identifying dissimilar information NI for identifying the end face model 110M. 1_2 It may be possible to obtain this automatically. Furthermore, by attempting matching, dissimilar information (NI) can be obtained. 1_2 Details of the function for obtaining this information will be described later.

[0053] Similarly, the processor 32 identifies the convex model 106M as a feature model that is dissimilar to the features of the workpiece 100 in the third orientation OR3, in the workpiece model 100M1 in the first orientation OR1 shown in Figure 10, as dissimilar information NI. 1_3 The processor 32 may obtain dissimilar information NI for identifying feature models 110M or 106M that are dissimilar to the features of work models 100M1 of one orientation OR1 compared to the features of work models 100 of other orientations OR2 or OR3. 1_2 or NI 1_3 The processor 32 obtains the dissimilar information NI. 1_2 NI 1_3It functions as an information acquisition unit 46 (Figure 2) that obtains information.

[0054] The dissimilar information NI may be an identification code (string, symbol, etc.) or flag that identifies the feature models (end face model 110M, convex part model 106M). The processor 32 processes the acquired dissimilar information NI into the work model 100M. n Either attach it to the model data, or add it to the work model 100M n It is associated with the model data and stored in memory 34.

[0055] As described above, in this embodiment, the processor 32 functions as a model acquisition unit 44 and an information acquisition unit 46, and acquires the work model 100M n Dissimilar information NI is obtained. Therefore, the model acquisition unit 44 and the information acquisition unit 46 constitute a device 80 (Figure 2) for acquiring dissimilar information NI.

[0056] In this device 80, the model acquisition unit 44 performs multiple orientation OR n Multiple work models 100M, each modeling a different work 100. n The information acquisition unit 46 acquires the information NI for identifying dissimilar information NI that a feature model 110M or 106M in a work model 100M1 in one orientation OR1 is dissimilar to the features of a work 100 in another orientation OR2 or OR3. 1_2 or NI 1_3 Obtain it.

[0057] According to this device 80, the processor 32 is responsible for detecting the work model 100M in order to detect the inappropriate matching described above. n This makes it possible to identify feature models 110M or 106M. As a result, the inappropriate matching can be detected with high accuracy, thereby reducing the false detection of the workpiece 100's position data PD.

[0058] Next, other functions of the robot system 10 will be described with reference to Figure 11. The robot system 10 shown in Figure 11 acquires the flow shown in Figure 12. The flow shown in Figure 12 is for acquiring the dissimilar information NI described above. The flow in Figure 12 is started when the processor 32 receives an operation start command from an operator, a higher-level controller, or a computer program PG.

[0059] In step S1, the processor 32 functions as a model acquisition unit 44 and acquires multiple work models 100M n To obtain, the processor 32 obtains multiple attitude ORs as illustrated in Figures 3 to 5. n Multiple work models (for example, n=1 to 500) 100M n These are obtained respectively. Processor 32 is for work model 100M n CAD model 100Mc n Alternatively, it may be obtained as a point cloud model.

[0060] In step S2, the processor 32 processes each work model 100M acquired in step S1. n Therefore, the work model 100M n The processor extracts multiple feature models contained within and assigns an identification code ID to each extracted feature model. For example, the processor 32 extracts the main body model 102M, the side model 108M, the end face model 110M, and the convex part model 106M as feature models from the work model 100M1 in the first orientation OR1 shown in Figure 3.

[0061] The processor 32 then assigns a unique identification code ID to each of the extracted main body model 102M, side model 108M, end face model 110M, and convex part model 106M. For example, the identification code ID is a string code (e.g., "Feature group A", "Feature group B", "Feature group C", etc.). This identification code ID allows the processor 32 to individually identify the feature models 102M, 108M, 110M, and 106M included in the work model 100M1. Thus, in this embodiment, the processor 32 identifies each of the work model 100M n It functions as a feature extraction unit 48 (Figure 11) that extracts a feature model and assigns an identification code ID to the feature model.

[0062] Note that processor 32 is for work model 100M n Multiple model components included in the above may be extracted as a single feature model. For example, the processor 32 may extract the side model 108M and end face model 110M of the work model 100M3 in the third orientation OR3 shown in Figure 5 as a single feature model 104M, and assign an identification code ID (a string code such as "Feature group C") to the single feature model 104M. The same applies to the main body model 102M and the convex part model 106M, which are composed of multiple face and edge model components.

[0063] Furthermore, the main body model 102M, side model 108M, end face model 110M, and convex part model 106M are not each extracted as a single feature model. For example, the combination of the main body model 102M and the convex part model 106M may be extracted as a single feature model, or multiple faces or edges defining the main body model 102M may each be extracted as a single feature model. Work model 100M n Whether the model components included in the model are extracted as feature models can be arbitrarily determined by the operator.

[0064] In step S3, the processor 32 processes the shape data SD of the workpiece 100. i For example, the processor 32 acquires detection data DD (Figure 6) obtained when the shape detection sensor 14 of the actual machine placed in real space actually images the workpieces 100 that are piled up loosely inside container B.

[0065] As another example, the processor 32 obtains detection data DD obtained by executing a simulation SL in which a virtual shape detection sensor 14 placed in a virtual space simulates imaging of workpieces 100 piled up in container B (for example, a workpiece model 100M placed in a model of container B).

[0066] The detection data DD obtained in this way includes various postures OR, for example, as shown in Figure 6. i Shape data SD of multiple workpieces 100 i (i=1,2,3,...) will be included. Next, in step S4, the processor 32 will use the work model 100M obtained in step S1. n posture OR n Set the identifying number "n" to "1".

[0067] In step S5, the processor 32 processes the shape data SD detected by the shape detection sensor 14. i The nth posture OR n Work model 100M n The matching is performed. For example, if n is set to 1 at the start of step S5, the processor 32, according to a predetermined matching algorithm, maps the work model 100M1 (Figure 3) of the first orientation OR1 acquired in step S1 to one shape data SD that is projected onto the detection data DD (Figure 6) acquired in step S3. i (For example, shape data SD1, SD2, or SD3) is matched. Thus, in this embodiment, the processor 32 matches the shape data SD of the workpiece 100 detected by the shape detection sensor 14. i Work model 100M nfunctions as a model matching execution unit 50 (FIG. 11) for matching.

[0068] In step S6, the processor 32, when matching the shape data SD i with the work model 100M n in the most recent step S5, determines the frequency α with which the feature model included in the work model 100M n matches the features shown in the shape data SD i For example, assume that in the most recent step S5, the processor 32 matches the work model 100M1 in the first posture OR1 with the shape data SD1 shown in the detection data DD in FIG. 6.

[0069] In this case, all the feature models such as the main body model 102M, the shaft model 104M (side model 108M and end face model 110M), and the convex model 106M included in the work model 100M1 will match the features shown in the shape data SD1 (that is, the features of the main body 102, the shaft 104, and the convex 106 of the work 100).

[0070] For example, the processor 32 determines the degree of coincidence β between the feature points Fm that constitute the feature model of the work model 100M1 to which the identification code ID is assigned and the feature points Fs that constitute the features shown in the shape data SD i This degree of coincidence β includes, for example, the error in the distance between the feature point Fm and the feature point Fs corresponding to the feature point Fm. In this case, the closer the feature point Fm and the feature point Fs are highly aligned, the smaller the value of the degree of coincidence β.

[0071] Alternatively, the degree of coincidence β includes a similarity representing the similarity between the feature point Fm and the feature point Fs corresponding to the feature point Fm. In this case, the closer the feature point Fm and the feature point Fs are highly aligned, the larger the value of the degree of coincidence β. The processor 32 compares the obtained degree of coincidence β with a threshold β th1By comparing them, it can be determined whether each feature model (main body model 102M, side model 108M, end face model 110M, convex part model 106M) of the work model 100M1 matches the features (main body part 102, side face 108, end face 110, convex part 106) shown in the shape data SD1.

[0072] For example, when the degree of coincidence β exceeds the threshold value β th1 (β > β th1 , or β < β th1 ), it is determined that the feature model of the work model 100M1 matches the features of the shape data SD1. As a result of this determination, the processor 32 can detect that all the feature models of the work model 100M1 match the features of the shape data SD1. Then, for each of the main body model 102M, side model 108M, end face model 110M, and convex part model 106M, the processor 32 increments the frequency α by only "1" (α = α + 1).

[0073] That is, in this embodiment, the frequency α indicates the number of times the feature model of the work model 100M n matches the shape data SD i when the work model 100M n is matched. The processor 32 associates the frequency α counted for each feature model of the work model 100M i with the identification code ID assigned to the feature model and stores it in the memory 34. Note that the frequency α is not limited to the above-mentioned number of matches. For example, it may be obtained by adding a weighting operation according to the matching feature model or the degree of coincidence β to the number of matches, or may be obtained by any method. n

[0074] ​On the other hand, suppose that in the most recent step S5, the processor 32 matches the work model 100M1 in the first orientation OR1 to the shape data SD2 shown in Figure 7. In this case, as shown in Figure 8(c), feature models such as the main body model 102M and the convex part model 106M included in the work model 100M1 will match the features of the work 100, such as the main body 102 and the convex part 106, as reflected in the shape data SD2. Therefore, in this case, the processor 32 increments the frequency α by "1" (α = α + 1) for the main body model 102M and the convex part model 106M included in the work model 100M1.

[0075] On the other hand, the shaft model 104M of the workpiece model 100M1 (i.e., the side model 108M and the end face model 110M) does not match the features of the workpiece 100 as reflected in the shape data SD2. Therefore, in this case, the processor 32 does not increment the frequency α for the side model 108M and the end face model 110M of the workpiece model 100M1.

[0076] Thus, in the most recent step S5, the processor 32 processes the shape data SD i Work model 100M n When matching, the work model 100M n The feature model included in the shape data SD i The processor 32 is looking for a frequency α that matches the features that are projected. Therefore, the processor 32 functions as a frequency calculation unit 52 (Figure 11) that calculates the frequency α.

[0077] In step S7, the processor 32 processes all shape data SD that is reflected in the detection data DD acquired in step S3. i Work model 100M n If n is set to 1, it determines whether or not the work model 100M1 in the first orientation OR1 has been matched.

[0078] If the processor 32 determines that the result is YES, it proceeds to step S8; otherwise, it returns to step S5. In this way, the processor 32 repeatedly executes the loop of steps S5 to S7 until it determines that the result is YES in step S7, and the shape data SD is reflected in the detected data DD. i Work model 100M n Each time a match is made, the work model 100M n The frequency α is calculated for each feature model, and the frequency α stored in memory 34 is updated in association with the identification code ID assigned to the feature model.

[0079] In step S8, processor 32 controls work model 100M n Based on the frequency α obtained for each of the multiple feature models included, each of the feature models matches the shape data SD obtained in step S5. i It is determined whether the features that are reflected are in a dissimilar relationship. For example, if n is set to 1 at the start of step S8, the processor 32 determines the frequency α obtained for each feature model of the work model 100M1 (main body model 102M, side model 108M, end face model 110M, and convex part model 106M) each time step S6 is executed, and sets the frequency α to a predetermined threshold α th1 Compare it to this.

[0080] Then, the processor 32 determines that the frequency α of one feature model (for example, the end face model 110M) is a threshold α. th1 (α≦α) th1 ) If the one feature model is shape data SD i It is determined that the features are in a dissimilar relationship. Thus, in this embodiment, the processor 32 determines that the feature model is in a dissimilar relationship to the shape data SD based on frequency α. i It functions as a similarity determination unit 54 (Figure 11) that determines whether or not the features captured are dissimilar.

[0081] If the processor 32 determines that at least one feature model included in the work model 100M1 is dissimilar (i.e., YES), it proceeds to step S9. However, if it determines that none of the feature models included in the work model 100M1 are dissimilar (i.e., NO), it proceeds to step S10.

[0082] In step S9, the processor 32 functions as an information acquisition unit 46 and acquires the work model 100M n Among the multiple feature models included, dissimilarity information NI is obtained for the feature models that were determined to be dissimilar in the previous step S8. For example, suppose that n=1 is set at the start of this step S9, and as a result of step S8, the end face model 110M (Figure 8(c)) and the convex part model 106M (Figure 10(c)) included in the work model 100M1 of the first orientation OR1 are determined to be dissimilar to the features of the work 100 of the second orientation OR2 and the features of the work 100 of the third orientation OR3, respectively.

[0083] In this case, the processor 32 identifies the end face model 110M from among the multiple feature models (body model 102M, side model 108M, end face model 110M, and convex model 106M) included in the work model 100M1 of the first orientation OR1, using dissimilar information NI. 1_2 And, dissimilar information NI for identifying the convex model 106M 1_3 This will result in obtaining [the desired outcome].

[0084] Processor 32 receives dissimilar information NI for end face model 110M. 1_2 This is obtained in association with the identification code ID assigned to the end face model 110M, and also the dissimilar information NI of the convex part model 106M. 1_3 This is obtained in association with the identification code ID assigned to the protruding part model 106M.

[0085] Then, the processor 32 retrieves the dissimilar information NI 1_2 and NI 1_3These are associated with their respective identification code IDs and stored in memory 34. As a result, the identification code ID of the feature model of work model 100M1 and the dissimilar information NI of the feature model are stored. 1_2 NI 1_3 These are associated with each other and stored in memory 34.

[0086] In step S10, the processor 32 controls the work model 100M n posture OR n The number "n" that identifies the maximum value n MAX (n=n MAX Determine whether or not this maximum value n MAX This is the work model 100M obtained in step S1. n The total number (i.e., posture OR n (Total number) (For example, n MAX (=500). If the processor 32 determines that the result is YES, it terminates the flow shown in Figure 12; otherwise, it proceeds to step S11.

[0087] In step S11, the processor 32 controls the work model 100M n posture OR n The identifying number "n" is incremented by "1" (n = n + 1). Then, the processor 32 returns to step S5. Thus, the processor 32 YES in step S10. The loop from steps S5 to S11 is repeatedly executed until a determination is made, and all 100M work models obtained in step S1 are processed. n Regarding this, the determination of dissimilarity (step S8) and the acquisition of dissimilarity information (NI) (step S9) are performed.

[0088] As described above, in this embodiment, the processor 32 functions as a model acquisition unit 44, an information acquisition unit 46, a feature extraction unit 48, a model matching execution unit 50, a frequency calculation unit 52, and a similarity determination unit 54, to acquire the work model 100M nDissimilarity information NI is obtained. Therefore, the model acquisition unit 44, information acquisition unit 46, feature extraction unit 48, model matching execution unit 50, frequency calculation unit 52, and similarity determination unit 54 constitute a device 82 (Figure 11) for acquiring dissimilarity information NI.

[0089] In this device 82, the feature extraction unit 48 extracts each work model 100M n Therefore, the work model 100M n Step S2 extracts multiple feature models contained in and assigns an identification code ID to each of the extracted feature models. Then, the information acquisition unit 46 associates the identification code ID assigned to the feature model (for example, the end face model 110M or the convex part model 106M) with the dissimilar information NI. 1_2 or NI 1_3 Obtain (step S9).

[0090] According to this configuration, processor 32 is for work model 100M n Among the multiple feature models included, it is possible to quickly recognize which feature models are dissimilar based on their associated identifier IDs and dissimilarity information (NI). Furthermore, dissimilarity information (NI) can be accumulated for each feature model classified by its identifier ID.

[0091] Furthermore, in the device 82, the model matching execution unit 50 receives shape data SD of the workpiece 100 detected by the shape detection sensor 14. i Work model 100M n Matching (Step S5). Also, the frequency calculation unit 52 and the model matching execution unit 50 perform multiple posture OR i Shape data SD i (For example, shape data SD1, SD2, SD3 in Figure 6) Work model 100M n When matching each work model (for example, work model 100M1), the work model 100M n The feature models included in each shape data SD i We find the frequency α that matches the features that are captured (Step S6).

[0092] Furthermore, the similarity determination unit 54 has a frequency calculation unit 52 that calculates the work model 100M. n Based on the frequency α obtained for each of the multiple feature models included, it is determined whether or not the respective feature models are dissimilar (step S8). Then, the information acquisition unit 46 retrieves the work model 100M n Among the multiple feature models included, the feature model determined to be dissimilar by the similarity determination unit 54 (for example, the end face model 110M or the convex part model 106M) is for which dissimilar information is obtained (step S9). With this configuration, the work model 100M n Since it is possible to determine with high accuracy whether the feature model is dissimilar to the features of workpiece 100 based on frequency α, dissimilarity information NI can be obtained with high accuracy for each feature model.

[0093] Furthermore, in the device 82, the similarity determination unit 54 determines that the frequency α obtained by the frequency calculation unit 52 is a predetermined threshold α th1 Work model 100M if the following conditions apply n The characteristic model is workpiece 100 (specifically, shape data SD i It determines that the features of the elements are dissimilar. With this configuration, the determination of dissimilar relationships can be performed with relatively simple algorithms and with high accuracy.

[0094] Furthermore, in one example of the device 82, the model matching execution unit 50 generates shape data SD obtained by the shape detection sensor 14 from the workpiece 100 in real space. i Using work model 100M n Matching is performed (step S2). According to this configuration, the shape data SD detected by the actual shape detection sensor 14 is used. i By attempting matching through this method, dissimilar information (NI) can be obtained with high accuracy.

[0095] On the other hand, in another example of the device 82, the model matching execution unit 50 acquires shape data SD obtained by a simulation SL in which the shape detection sensor 14 simulates imaging the workpiece 100 in a virtual space. iExecute matching using this (step S2). According to this configuration, various matchings can be easily tried, so the work of obtaining the non-similarity information NI can be simplified.

[0096] In addition, in step S6 described above, the processor 32 uses the work model 100M that was matched in the immediately preceding step S5 n All the feature models of which are detected to be in agreement with the features of the shape data SD i When this is detected, it may not be necessary to increment the frequency α of each of all these feature models.

[0097] For example, when the processor 32 matches the work model 100M1 in the first posture OR1 in the immediately preceding step S5 to the shape data SD1 shown in the detection data DD shown in FIG. 6, as described above, it can be detected that all the feature models of the work model 100M1 are in agreement with the features of the shape data SD1. In this case, the processor 32 does not increment the frequency α for each of the main body model 102M, the side surface model 108M, the end surface model 110M, and the convex portion model 106M of the work model 100M1.

[0098] That is, in this case, when the processor 32 matches the work model 100M n to the shape data SD n in the same posture OR i (that is, an appropriate matching), for the feature models that are in agreement with the features of the shape data SD i they are excluded from the calculation target of the frequency α. Thereby, the non-similarity information NI for detecting an inappropriate matching can be obtained more efficiently.

[0099] Next, referring to FIG. 13, still other functions of the robot system 10 will be described. The robot system 10 shown in FIG. 13 executes the flow shown in FIG. 14. The flow shown in FIG. 14 is a flow for obtaining the non-similarity information NI similar to the flow of FIG. 12, but is different from the flow of FIG. 12 in step S12.

[0100] Specifically, after step S5, in step S12, the processor 32 determines whether there is an inconsistency between at least one feature model included in the work model 100M n when the matching of the work model 100M n is executed in the most recent step S5, and the features shown in the shape data SD i .

[0101] For example, assume that in the immediately preceding step S5, the work model 100M1 in the first posture OR1 is matched with the shape data SD2 shown in the detection data DD of FIG. 6. In this case, the processor 32, for example, obtains the degree of coincidence β between the feature points Fm constituting each feature model (the main body model 102M, the side surface model 108M, the end surface model 110M, and the convex portion model 106M) of the work model 100M1 to which the identification code ID is assigned, and the feature points Fs constituting the features shown in the shape data SD2.

[0102] Then, the processor 32 compares the degree of coincidence β obtained for each feature model to which the identification code ID is assigned with the threshold value β th1 , and when the degree of coincidence β does not exceed the threshold value β th1 , it is determined that there is an inconsistency (that is, YES) between the feature model of the work model 100M1 and the features of the shape data SD2. As a result of this determination, the processor 32 determines that, as shown in FIG. 8(c), the side surface model 108M and the end surface model 110M (that is, the shaft portion model 104M) of the work model 100M1 are inconsistent with the features of the shape data SD2.

[0103] If the processor 32 determines YES, it proceeds to step S6 and obtains the frequency α with which the feature models (the main body model 102M and the convex portion model 106M in FIG. 8(c)) included in the work model 100M1 match the features (the features of the main body portion 102 and the convex portion 106) shown in the shape data SD i . On the other hand, if the processor 32 determines NO, it proceeds to step S7.

[0104] Thus, in this embodiment, the processor 32 processes the work model 100M in step S5. n When matching is performed using the work model 100M n The feature model included and the shape data SD i It functions as a mismatch determination unit 56 (Figure 13) that determines whether or not there is a mismatch between the features captured and the image.

[0105] As described above, in this embodiment, the processor 32 functions as a model acquisition unit 44, an information acquisition unit 46, a feature extraction unit 48, a model matching execution unit 50, a frequency calculation unit 52, a similarity determination unit 54, and a mismatch determination unit 56, to acquire the work model 100M n Dissimilar information NI is obtained. Therefore, the model acquisition unit 44, information acquisition unit 46, feature extraction unit 48, model matching execution unit 50, frequency calculation unit 52, similarity determination unit 54, and inconsistency determination unit 56 constitute a device 84 (Figure 11) for acquiring dissimilar information NI.

[0106] In this device 84, the mismatch determination unit 56 determines that the model matching execution unit 50 determines the work model 100M n When matching is performed using (for example, work model 100M1) (step S5), the work model 100M n At least one feature model included (e.g., side model 108M and end model 110M) and shape data SD i (For example, step S12) determines whether there is an inconsistency between the features corresponding to the at least one feature model that is reflected in the shape data SD2.

[0107] Then, if the frequency calculation unit S6 determines that an inconsistency has occurred by the inconsistency determination unit 56 (i.e., YES in step S12), then another feature model (for example, the main body 102 and the convex part 106) is used in the shape data SD i (For example, determine the frequency α that matches the features of the shape data SD2) (step S6).

[0108] According to this configuration, the work model 100M n and shape data SD i If a mismatch occurs in the matching with the shape data SD, i Frequency α can only be determined for feature models that are consistent with the features. Here, feature models that are determined to be inconsistent in step S12 are likely to fall under the dissimilar relationship described above. Therefore, by collecting the frequency α obtained when inconsistency is determined to have occurred in step S12, dissimilar information NI can be efficiently obtained in step S9.

[0109] Next, with reference to Figure 15, further functions of the robot system 10 will be described. The robot system 10 shown in Figure 15 executes the flow shown in Figure 16. The flow in Figure 16 is a flow for causing the robot 12 to perform a predetermined task on workpieces 100 that are loosely packed in container B, and is executed after the execution of the flow in Figure 12 or Figure 14 (i.e., after the acquisition of dissimilar information NI). The processor 32 starts the flow in Figure 16 when it receives a work start command from the operator, higher-level controller, or computer program PG after the execution of the flow in Figure 12 or Figure 14.

[0110] In step S21, the processor 32 acquires the detection data DD detected by the shape detection sensor 14. Specifically, the processor 32 operates the robot 12 to position the shape detection sensor 14 at an imaging position that places the workpieces 100, which are loosely stacked in container B, within the detection range of the shape detection sensor 14. Then, the processor 32 operates the shape detection sensor 14 to image the workpieces 100 in container B, thereby detecting the detection data DD as illustrated in Figure 6. The processor 32 acquires the detected detection data DD from the shape detection sensor 14.

[0111] In step S22, the processor 32 processes the shape data SD that is reflected in the detection data DD detected by the shape detection sensor 14. i Work model 100M nBy matching these, the position data PD of the workpiece 100 in the control coordinate system C is obtained. That is, in this embodiment, the processor 32 functions as a position data acquisition unit 58 (Figure 15) that acquires the position data PD of the workpiece 100. Step S22 will be explained with reference to Figure 17. The processor 32 functions as a position data acquisition unit 58 and executes the flow of step S22 shown in Figure 17.

[0112] After the start of step S22, in step S31, the processor 32 performs preprocessing PP on the detection data DD acquired in the most recent step S21. For example, as preprocessing PP, the processor 32 may perform a process to delete from the detection data DD any point clouds that are to be invalidated (for example, point clouds that exist outside the container B, or point clouds that are far beyond a predetermined distance from the point cloud of the shape data SDi of the workpiece 100).

[0113] In step S32, the processor 32 performs a coarse search RS according to a predetermined matching algorithm. Specifically, as a coarse search RS, the processor 32 places multiple work models 100M acquired in step S1 into a virtual space defined by the sensor coordinate system C3 of the detected data DD. n Arrange them in order, and the work model 100M n Each shape data SD included in the detected data DD i Matching to that. In other words, in this embodiment, the processor 32 matches one shape data SD included in the detected data DD. i In contrast, multiple work models 100M n Match each of them.

[0114] At this time, the processor 32 processes the work model 100M placed in the sensor coordinate system C3. n The position is repeatedly displaced by a predetermined amount. Then, the processor 32 processes the work model 100M n Each time the position of the workpiece model 100M is displaced, nThe degree of agreement β between the feature points Fm and the feature points Fs of the shape data SDi is calculated.

[0115] Then, the processor 32 calculates the degree of agreement β and the threshold β. th2 The degree of agreement β is compared with the threshold β. th2 When it exceeds (i.e., β > β) th2 , or β<β th2 ), in sensor coordinate system C3, work model 100M n and shape data SD i It is determined that there is a general match. Note that this threshold β th2 This is the threshold β mentioned above. th1 It can be the same value or a different value.

[0116] Figure 18 shows the state in which the work model 100M1 in the first orientation OR1 is matched to the shape data SD1. The processor 32 sets the coordinate Q1 of the sensor coordinate system C3 in the work coordinate system C4 set for the work model 100M1 which has been matched to the shape data SD1 as a rough search RS. S (X1 S ,Y1 S ,Z1 S ,W1 S ,P1 S ,R1 S ) to sensor coordinate system position data PD 0_1 It will be acquired as follows.

[0117] This sensor coordinate system position data PD 0_1 (That is, coordinate Q1 S ) of which, coordinate (X1 S ,Y1 S ,Z1 S ) indicates the origin position of the work coordinate system C4 in the sensor coordinate system C3, and coordinate (W1 S ,P1 S ,R1 S ) indicates the orientation (so-called yaw, pitch, and roll) of the workpiece coordinate system C4 in the sensor coordinate system C3.

[0118] Then, the processor 32 processes the sensor coordinate system position data PD. 0_1coordinate Q1 in robot coordinate system C1 R (X1 R ,Y1 R ,Z1 R ,W1 R ,P1 R ,R1 R Convert to the coordinate Q1 R to initial position data PD 1_1 This is obtained as initial position data PD. 1_1 This represents the approximate position and orientation of the workpiece 100, as detected by the shape detection sensor 14, in the robot coordinate system C1, as shape data SD1.

[0119] Similarly, the processor 32 processes other shape data SD that is reflected in the detected data DD. i Sensor coordinate system position data PD 0_i , and initial position data PD 1_i The data is acquired sequentially. In this way, the processor 32, in the coarse search RS, obtains the work model 100M n Shape data SD i By matching, the position data PD of workpiece 100 is obtained. 0_i , PD 1_i They have obtained it.

[0120] In step S33, the processor 32 performs a dense search PS. Specifically, the processor 32 searches for shape data SD that is reflected in the detected data DD. i Regarding this, the initial position data PD acquired in step S32 1_i Based on this, and according to a predetermined matching algorithm (for example, a mathematical optimization algorithm such as ICP: Iterative Closest Point), the work model 100M in sensor coordinate system C3 is determined. n Shape data SD i We search for a position that matches highly.

[0121] For example, the processor 32 receives the initial position data PD of the sensor coordinate system C3. 1_i Work model 100M n Point cloud model 100Mp nThe point cloud and the shape data SD included in the detection data DD i Obtain the degree of coincidence γ with the 3D point cloud. For example, this degree of coincidence γ is the point cloud model 100Mp n The point cloud and the shape data SD i The error in the distance between the 3D point clouds, or the similarity between the point cloud model 100Mp n The point cloud and the shape data SD i The 3D point cloud.

[0122] Then, the processor 32 compares the obtained degree of coincidence γ with a predetermined threshold γ th for the degree of coincidence γ, and when the degree of coincidence γ exceeds the threshold γ th (for example, γ>γ th , or γ<γ th ), it is determined that the work model 100M n (for example, the point cloud model 100Mp n ) and the shape data SD i are highly matched. On the other hand, when the degree of coincidence γ does not exceed the threshold γ th , the processor 32 displaces the position of the work model 100M arranged in the sensor coordinate system C3 by a predetermined displacement amount, and obtains the degree of coincidence γ each time the position of the work model 100M n is displaced, and compares it with the threshold γ<( n . th

[0123] When the degree of coincidence γ exceeds the threshold γ th , the processor 32 uses the coordinates Q2 i of the work coordinate system C4 set in the work model 100M that highly matches the shape data SD as the dense search PS n [[ID=Z46]]in the sensor coordinate system C3 S (X2 S , Y2 S , Z2 S , W2 S , P2 S , R2 S ) and obtains it as the sensor coordinate system position data PD 2_i .

[0124] ​Then, the processor 32 processes the sensor coordinate system position data PD. 2_i coordinates Q2 in robot coordinate system C1 R (X2 R ,Y2 R ,Z2 R ,W2 R ,P2 R ,R2 R Convert to the coordinate Q2 R shape data SD i The position data PD of the workpiece 100 was detected as follows: 3_i This location data PD is obtained as follows. 3_i This represents the high-precision position and orientation of the workpiece 100 in the robot coordinate system C1. In this way, the processor 32 performs a dense search PS using the workpiece model 100M. n Shape data SD i By matching, the position data PD of workpiece 100 is obtained. 2_i , PD 3_i They have obtained it.

[0125] As a result of step S33, the processor 32 generates position data PD of multiple workpieces 100 reflected in the detected data DD. 3_i The processor 32 obtains the acquired position data PD. 3_i This is stored in memory 34 as detection result data DT. An example of the data structure of detection result data DT is schematically shown in Table 1 below.

[0126] [Table 1]

[0127] The detection result data DT shown in Table 1 includes the position data PD of the workpiece 100. 3_i (Specifically, coordinate Q2 R (X2 R ,Y2 R ,Z2 R ,W2 R ,P2 R ,R2 R )) along with the degree of agreement γ i , plane proportion δ i , and exposure rate εi Detection result parameters PM are stored. These detection result parameters PM are stored in the location data PD. 3_i Shape data SD of workpiece 100 obtained during the acquisition i and work model 100M n This parameter represents the result of the matching with [the specified variable]. Specifically, it is the degree of agreement γ. i In step S33, the dense search PS is performed on work model 100M. n Shape data SD i This is the degree of agreement when it is determined that a high level of matching has been achieved.

[0128] Plane ratio δ i For example, shape data SD in dense search PS i Work model 100M matched to n In the work model 100M n The largest planar model included (specifically, the 100Mp planar point cloud model) n ) However, the work model 100M n The whole (specifically, the entire point cloud model 100Mp) n This is the proportion of ) to the total. Alternatively, the plane ratio δ i This involves dense search PS for a 100M work model. n Matched shape data SD i The point cloud representing the largest plane contained within the shape data SD i It may also be the proportion of the whole (specifically, the entire point cloud).

[0129] Exposure rate ε i For example, shape data SD in dense search PS i Work model 100M matched to n The whole (specifically, the point cloud model 100Mp) n The shape data SD for all the point clouds that make up the shape i Work model 100M that matches n The region (i.e., shape data SD) i The point cloud model 100Mp is matched with the point cloud. n This shows the proportion of the point clusters.

[0130] The processor 32 receives location data PD 3_i Along with the detection result parameter PM (agreement γ i , plane proportion δ i , and exposure rate ε i ) obtains location data PD 3_i The detection result data DT is stored in memory 34 in association with this. The detection result parameter PM is the degree of agreement γ. i , plane proportion δ i , and exposure rate ε i This is not limited to this; it may have any other parameters.

[0131] Furthermore, the detection result data DT shown in Table 1 includes location data PD. 3_i Shape data SD obtained i Information and acquired location data PD 3_i The sequentially assigned numbers j (j=1,2,3,...) and the location data PD 3_i It is also stored together with the detection result parameter PM, in association with it.

[0132] In step S34, the processor 32 performs post-processing OP on the detection result data DT. For example, as post-processing OP, the processor 32 processes position data PD where the detection result parameter PM does not meet a predetermined criterion. 3_i You may also perform a process to remove it from the detection result data DT.

[0133] Specifically, processor 32 determines the degree of agreement γ. i , plane proportion δ i , or exposure rate ε i If the detection result parameter PM does not exceed a predetermined standard value, it is determined that the detection result parameter PM does not meet the standard, and the location data PD associated with the detection result parameter PM is determined to be missing. 3_i The detected result data DT may be deleted. As described above, the processor 32 functions as a position data acquisition unit 58 and executes step S22 shown in Figure 17, and the position data PD of the workpiece 100 is obtained. 0_i , PD 1_i , PD 2_i , PD3_i Obtain it.

[0134] Referring again to Figure 16, in step S23, the processor 32 verifies the false detection of the position data PD acquired in step S22. Here, in steps S32 and S33 described above, the processor 32 processes the work model 100M as illustrated in Figure 8(c) or Figure 10(c). n Shape data SD i As a result of inappropriate matching, location data PD 3_i There is a possibility of obtaining location data PD as a result of such inappropriate matching. 3_i This data represents a false detection that does not accurately represent the position and orientation of workpiece 100.

[0135] Therefore, in this embodiment, in step S23, the processor 32 stores the position data PD in the detection result data DT. 3_i The false detection is verified. This step S23 will be explained with reference to Figure 19. After the start of step S23, in step S41, the processor 32 stores the position data PD in the detection result data DT. 3_i Set the identifying number "j" to "1".

[0136] In step S42, the processor 32 processes the j-th position data PD 3_i Work model 100M was used for matching to obtain n We determine whether dissimilar information NI has been obtained. At this point, the position data PD number "j" is set to j=2, and the second position data PD (i.e., number j=2) stored in the detection result data DT in Table 1 is... 3_2 However, as shown in Figure 8(c), we will now explain the case where the work model 100M1 in the first posture OR1 was obtained as a result of inappropriately matching it to the shape data SD2.

[0137] In this case, the processor 32 processes the second position data PD 3_2When obtaining the data, the work model 100M1 used for matching is identified, and for the work model 100M1, dissimilar information NI is used to identify the end face model 110M and the convex part model 106M in step S9 described above. 1_2 and NI 1_3 It can be recognized that the data has been acquired. Therefore, in this case, the processor 32 will determine YES in step S42. If the processor 32 determines YES, it proceeds to step S43, and if it determines NO, it proceeds to step S45.

[0138] In step S43, the processor 32 processes the j-th position data PD 3_i Work model 100M used to obtain n Among the feature models included, the feature model identified by the dissimilar information NI is the j-th position data PD. 3_i Shape data SD obtained i Determine whether the features captured in the image are consistent with the actual features.

[0139] Specifically, the processor 32 processes the second position data PD 3_2 In the work model 100M1 of the first orientation OR1 used to obtain the data, the dissimilar information NI for the work 100 of the second orientation OR2 1_2 It is determined whether the end face model 110M identified by this method matches the features reflected in the shape data SD2.

[0140] For example, the processor 32 processes the second position data PD in the detection result data DT. 3_2 The matching data obtained when the matching degree γ2 stored in association with is acquired is analyzed, and the matching degree γ2' between the end face model 110M matched in step S33 (dense search PS) described above and the features of the shape data SD2 is obtained.

[0141] Then, the processor 32 determines that the degree of agreement γ2' is a predetermined threshold γ th If it exceeds ' (γ2'>γ th ', or γ2'<γ th ') in, dissimilar information NI1_2 It is determined that the end face model 110M identified by this is consistent with the features reflected in the shape data SD2 (i.e., YES).

[0142] Note that this threshold γ th ' is the threshold γ mentioned above. th It may be the same value or a different value. In this embodiment, as shown in Figure 8(c), the end face model 110M of the work model 100M1 is inconsistent with the features of the shape data SD2, so the degree of agreement γ2' is the threshold γ th It does not exceed '.

[0143] Therefore, in this case, the processor 32 processes the dissimilar information NI 1_2 The end face model 110M identified by this will be determined to be inconsistent with the features reflected in the shape data SD2 (i.e., NO). If the processor 32 determines it to be NO, it proceeds to step S44; if it determines it to be YES, it proceeds to step S45.

[0144] Thus, in this embodiment, the processor 32 processes shape data SD i Work model 100M n By matching, location data PD 3_i When obtained, the feature model identified by the dissimilar information NI is the shape data SD i It functions as a consistency determination unit 62 (Figure 15) that determines whether or not the features captured are consistent with the image.

[0145] Furthermore, in step S43, the processor 32 provides dissimilar information NI for the workpiece 100 in the third orientation OR3 (i.e., the shape data SD3 in Figure 10(c)) in the workpiece model 100M1 in the first orientation OR1. 1_3 It may be determined whether the convex model 106M identified by this method is consistent with the features reflected in the shape data SD2.

[0146] In this case, the processor 32 functions as a consistency determination unit 62 to determine whether the convex model 106M is consistent with the features of the shape data SD2. As shown in Figure 8(c), the convex model 106M of the work model 100M1 is consistent with the features (convex 106) of the shape data SD2. Therefore, the processor 32 determines that the dissimilar information NI 1_3 The convex model 106M identified by this method will be determined to be consistent with the features of the shape data SD2.

[0147] In other words, in step S43, the processor 32 processes the work model 100M n If there are multiple dissimilar information NIs, then for each of the multiple feature models identified by these dissimilar information NIs, the shape data SD i The processor 32 then determines whether or not the feature model matches the shape data SD. i If the characteristics are inconsistent, the answer is NO.

[0148] In step S44, the processor 32 invalidates the position data PD of the workpiece 100 acquired in step S22 described above. Here, as described above, in this embodiment, the second position data PD stored in the detection result data DT of Table 1 3_2 However, this is obtained as a result of inappropriate matching as shown in Figure 8(c). In this case, if the processor 32, in step S25 described later, obtains the position data PD obtained as a result of the mismatch, 3_2 If the robot 12 is made to perform an operation on the workpiece 100 using this method, it will not be able to perform the operation with high precision.

[0149] Therefore, in this embodiment, in step S44, the processor 32 controls the position data PD in step S25, which will be described later. 3_2 To avoid using the position data PD 3_2 Disable it. For example, the processor 32 obtains position data PD from the detection result data DT created in step S22 above. 3_2 Delete.

[0150] As another example, the processor 32 stores the position data PD of the workpiece 100 in the detection result data DT. 3_2 The invalid flag FL is assigned to it. In this case, when the processor 32 executes step S25 described later, the position data PD of the detection result data DT is assigned. 3_2 When the data is read, the position data PD 3_2 Referencing the invalid flag FL assigned to the position data PD, 3_2 This is ignored. As a result, in step S25, the processor 32 cancels the operation on the workpiece 100 that was detected as shape data SD2.

[0151] In this way, in step S44, the processor 32 processes the position data PD stored in the detection result data DT. 3_i This is disabled (for example, by deleting it or by assigning the disabled flag FL), thereby preventing the control of step S25 described below from receiving the position data PD obtained as a result of the inconsistency. 3_i This allows us to avoid using it.

[0152] Thus, in this embodiment, the processor 32 processes the position data PD 3_i To obtain shape data SD i Work model 100M matched to n and the shape data SD i If an inconsistency occurs between the two (i.e., determined to be NO in step S43), the position data PD acquired in step S22 is used. 3_i It functions as a position data cancellation unit 60 (Figure 15) that disables the data.

[0153] In step S45, the processor 32 determines that the position data PD number "j" set at this point is j = j MAX Determine whether or not this j MAX This is the position data PD acquired in step S22. 3_iThis is the total number. If the processor 32 determines that the result is YES, it terminates the flow shown in Figure 19; otherwise, it proceeds to step S46.

[0154] In step S46, the processor 32 generates position data PD 3_i The number "j" is incremented by "1" (j = j + 1). Then, the processor 32 returns to step S42. In this way, the processor 32 repeatedly executes the loop of steps S42 to S46 until it determines YES in step S45, and the position data PD stored in the detection result data DT is processed. 3_i Of these, the location data PD obtained as a result of the inconsistency 3_i (For example, location data PD) 3_2 ) disable.

[0155] Referring again to Figure 16, in step S24, the processor 32 generates position data PD 3_i Specifically, the processor 32 determines the detection result data DT after executing step S23 as formal control data to be used for control in step S25 described later, and stores it in memory 34 (for example, RAM).

[0156] In step S25, the processor 32 refers to the detection result data DT determined in step S24 and operates the robot 12 to perform work on the workpiece 100 (workpiece handling, welding, or laser processing, etc.). If, in step S44 described above, the first position data PD 3_1 While the first one was not invalidated, the second location data PD 3_2 Let's assume it has been invalidated.

[0157] In this case, the processor 32 uses the first position data PD stored in the detection result data DT. 3_1 Refer to the above and operate the robot 12 to move the end effector 28 (i.e., tool coordinate system C2) to the position data PD 3_1 The position and orientation shown (i.e., coordinates Q2 in robot coordinate system C1) R (X2 R ,Y2R ,Z2 R ,W2 R ,P2 R ,R2 R The end effector 28 positions the workpiece 100 as shape data SD1 and performs operations on it. Meanwhile, the processor 32 processes the second position data PD 3_2 The operation on workpiece 100, which was detected as shape data SD2, is canceled.

[0158] In step S26, the processor 32 determines whether it has completed operations on all workpieces 100 in container B. If the processor 32 determines that it is YES, it terminates the flow shown in Figure 16; otherwise, it returns to step S21. In this way, the processor 32 repeatedly executes the loop of steps S21 to S26 until it determines that it is YES in step S26. It then acquires the detection data DD newly detected by the shape detection sensor 14 in step S21, and executes steps S22 to S25 based on the new detection data DD.

[0159] As described above, in this embodiment, the processor 32 functions as a model acquisition unit 44, an information acquisition unit 46, a feature extraction unit 48, a model matching execution unit 50, a frequency calculation unit 52, a similarity determination unit 54, a mismatch determination unit 56, a position data acquisition unit 58, a position data cancellation unit 60, and a consistency determination unit 62, to acquire the work model 100M n Obtain dissimilar information (NI).

[0160] Therefore, the model acquisition unit 44, information acquisition unit 46, feature extraction unit 48, model matching execution unit 50, frequency calculation unit 52, similarity determination unit 54, mismatch determination unit 56, position data acquisition unit 58, position data cancellation unit 60, and consistency determination unit 62 constitute a device 86 (Figure 15) for acquiring dissimilar information NI.

[0161] In this device 86, the position data acquisition unit 58 acquires shape data SD of the workpiece 100 detected by the shape detection sensor 14, which is positioned at a known location in the control coordinate system C (robot coordinate system C1).i Work model 100M n By matching the position data PD of the workpiece 100 in the control coordinate system C (specifically, the position data PD 0_i , PD 1_i , PD 2_i , PD 3_i ) obtain (steps S32, S33).

[0162] Then, the position data cancellation unit 60 receives shape data SD so that the position data acquisition unit 58 can acquire the position data PD. i Work model 100M matched to n and the shape data SD i If an inconsistency occurs between the two (i.e., determined to be NO in step S43), the position data PD acquired by the position data acquisition unit 58 is invalidated (step S44).

[0163] According to this configuration, the position data PD obtained as a result of improper matching, such as shown in Figure 8(c) or Figure 10(c), 3_i This allows us to avoid having the robot 12 perform the operation on the workpiece 100 (step S25). This improves the accuracy of the operation performed by the robot 12 in step S25.

[0164] Furthermore, in the device 86, the matching determination unit 62 acquires shape data SD from the position data acquisition unit 58. i (For example, shape data SD2) with a 100M work model n (For example, by matching work model 100M1) the position data PD (PD 3_2 When obtaining ) dissimilar information NI(NI 1_2 The feature model (end face model 110M) identified by the shape data SD i It is determined whether or not the features captured in the image are consistent (step S43).

[0165] Then, the position data cancellation unit 60 determines that the feature model (110M) is shape data SD by the consistency determination unit 62. iIf it is determined that there is a mismatch with the characteristics (NO in step S43), the position data acquisition unit 58 acquires the position data PD (PD 3_2 This disables the feature identified by the dissimilar information (NI). This configuration allows us to focus on the feature model identified by the dissimilar information (NI) and verify whether or not an inappropriate match occurred when acquiring the location data (PD). This enables high-accuracy detection of false positives in the location data (PD).

[0166] Next, with reference to Figure 20, further functions of the robot system 10 will be described. The robot system 10 shown in Figure 20 executes the flow shown in Figure 21. In the flow shown in Figure 21, the same step numbers are used for processes that are the same as those in the flow of Figure 14, and redundant explanations are omitted.

[0167] In the flow shown in Figure 21, if the result in step S7 is YES, then in step S13, the processor 32 will determine the work model 100M n Based on the frequency α obtained for each of the multiple feature models included, each of the feature models matches the shape data SD obtained in step S5. i Determine whether the features captured in the image are similar to those in the image.

[0168] Specifically, each time the processor 32 executes step S6 described above, it processes 100M of work models. n The frequency α obtained for each feature model (for example, the main body model 102M, the side model 108M, the end face model 110M, and the convex part model 106M) is set to a predetermined threshold α. th2 Compare it to this.

[0169] Then, the processor 32 determines that the frequency α of one feature model is threshold α th2 (α≧α) th2 ) If the one feature model is shape data SD i It is determined that the characteristics are similar to those of the other characteristics. This threshold α th2 This is the threshold α mentioned above. th1 It is set to a value greater than . Processor 32 is for work model 100M nAt least one feature model included is shape data SD i If the characteristics are similar to those of the previous step, the answer is determined to be YES, and the process proceeds to step S14. Meanwhile, the processor 32 processes the work model 100M n All feature models included are shape data SD i If there is no similar relationship with the characteristics, the result is determined to be NO, and the process proceeds to step S14.

[0170] In step S14, the processor 32 functions as an information acquisition unit 46 and acquires the work model 100M n Among the multiple feature models included, similarity information SI is obtained for the feature models that were determined to be similar in the previous step S13. For example, suppose in the previous step S13, for the work model 100M2 in the second orientation OR2 (in other words, number "n" = 2) shown in Figure 22(a), it was determined that the main body model 102M of the work model 100M2 is similar to the shape data SD1 that represents the main body 102 of the work 100 in the first orientation OR1 shown in Figure 22(b).

[0171] In this case, the processor 32 identifies similarity information SI that the main body model 102M included in the work model 100M2 of the second orientation OR2 is similar to the main body 102 of the work 100 of the first orientation OR1. 2_1 Obtain this similar information SI. 2_1 This is a linking information I that establishes a similar relationship between the main body model 102M of the work model 100M2 in the second orientation OR2 and the main body model 102M of the work model 100M1 in the first orientation OR1. 2_1 Includes.

[0172] For example, the operator visually confirms the matching result between the work model 100M2 and the shape data SD1 in step S5, as displayed on the display device 40, and recognizes that the main body model 102M of the work model 100M2 in the second orientation OR2 is similar to the main body model 102M of the work model 100M1 in the first orientation OR1. Then, the operator operates the input device 42 to input the aforementioned linked information I 2_1 This can also be manually entered into processor 32.

[0173] As another example, if the most recent step S5 is executed based on the detection data DD obtained by performing the simulation SL described above in step S3, the processor 32 will obtain the associated information I from the matching results in step S5. 2_1 It is also possible to acquire this automatically. In the case of simulation SL, the position and orientation of the workpiece 100 (for example, workpiece model 100M) placed in the virtual space are known, so the processor 32 can automatically recognize that the main body model 102M of the workpiece model 100M2 in the second orientation OR2 and the main body model 102M of the workpiece model 100M1 in the first orientation OR1 are similar.

[0174] On the other hand, in the preceding step S13, it was determined that the main body model 102M of the work model 100M4 in the fourth orientation OR4 (in other words, number "n" = 4) shown in Figure 23(a) is similar in relationship to the shape data SD1 that represents the main body 102 of the work 100 in the first orientation OR1 shown in Figure 23(b).

[0175] In this case, the processor 32 identifies similarity information SI that the main body model 102M included in the work model 100M4 of the fourth orientation OR4 is similar to the main body 102 of the work 100 of the first orientation OR1. 4_1 Obtain this similar information SI. 4_1 This is a linking information I that establishes a similar relationship between the main body model 102M of the work model 100M4 in the fourth orientation OR4 and the main body model 102M of the work model 100M1 in the first orientation OR1.4_1 This includes step S14. After step S14, the processor 32 proceeds to step S8.

[0176] After executing the flow shown in Figure 21 (i.e., after obtaining similar information SI and dissimilar information NI), the processor 32 executes the flow shown in Figure 16. Hereinafter, the flow shown in Figure 16 executed in this embodiment differs from the embodiment described above in step S23. Step S23 executed in this embodiment will be described below with reference to Figure 24. In the flow shown in Figure 24, the same step numbers are used for processes similar to those in the flow of Figure 19, and redundant explanations are omitted.

[0177] In the flow shown in Figure 24, if the processor 32 determines NO in step S42, or YES in step S43, it proceeds to step S47. In step S47, the processor 32 processes the j-th position data PD. 3_i Work model 100M used for matching to obtain n Determine whether similar information (SI) has been obtained for that purpose.

[0178] As an example, at this point, the location data PD number "j" is set to j=1, and the first location data PD (i.e., number j=1) stored in the detection result data DT in Table 1 is... 3_1 However, as shown in Figure 23(c), suppose the work model 100M4 in the fourth orientation OR4 was obtained as a result of inappropriate matching to the shape data SD1. Since no dissimilar shape NI has been obtained for this work model 100M4, the processor 32 determines NO in step S42 and executes step S47.

[0179] In this case, the processor 32 processes the first position data PD 3_1 Regarding the work model 100M4 used to obtain the above, similar information SI identifies the main body model 102M in step S14. 4_1 It can be recognized that the data has been acquired. Therefore, in this case, processor 32 determines that the result is YES.

[0180] As another example, at this point j=1, and the first position data PD 3_1 However, as shown in Figure 22(c), the workpiece model 100M2 in the second orientation OR2 was obtained as a result of inappropriate matching to the shape data SD1. Furthermore, dissimilarity information NI is used to identify that the convex part model 106M of this workpiece model 100M2 is dissimilar to the features of the workpiece 100 in the third orientation OR3 (Figure 10(b)). 2_3 Let's assume that it was acquired.

[0181] In this case, the processor 32 will determine YES in step S42. Also, as shown in Figure 22(c), dissimilar information NI 2_3 The convex model 106M of the work model 100M2 identified by matches the feature (i.e., convex 106) reflected in the shape data SD1 of the work 100 in the first orientation OR1. Therefore, the processor 32 determines YES in step S43.

[0182] In this case, in step S47, the processor 32 receives the first position data PD 3_1 Regarding the work model 100M2 used to obtain the above, similar information SI identifies the main body model 102M in step S14. 2_1 It can be recognized that the data has been acquired. Therefore, in this case, processor 32 determines that the result is YES.

[0183] If the processor 32 determines that the answer is YES, it proceeds to step S48. Meanwhile, the processor 32 processes the j-th position data PD. 3_i Work model 100M used to obtain n If no similar information SI is obtained, the result is determined to be NO, and the process proceeds to step S45.

[0184] In step S48, the processor 32 determines the j-th position data PD based on the similar information SI. 3_i Work model 100M used to obtainn A different work model 100M is selected from among the multiple work models 100M obtained in step S1 described above.

[0185] For example, as mentioned above, the first position data PD 3_1 If an inappropriate matching occurs during acquisition as shown in Figure 22(c), the processor 32 will use the similar information SI obtained for the main body model 102M of the work model 100M2 of the second orientation OR2. 2_1 See below. Processor 32 uses this similar information SI 2_1 From there, the first position data PD 3_1 It can be recognized that the main body model 102M of the work model 100M2 used when acquiring the data is similar to the main body 102 of the work 100 in the first orientation OR1.

[0186] Then, the processor 32 processes this similar information SI 2_1 Linking information I included 2_1 Refer to the second posture OR2, and work model 100M which includes a feature model that is similar to the main body model 102M of work model 100M2. n As such, multiple work models 100M n From these, select work model 100M1 in the first posture OR1.

[0187] Alternatively, as described above, the first position data PD 3_1 If an inappropriate matching occurs during acquisition as shown in Figure 23(c), the processor 32 will use the similar information SI obtained for the main body model 102M of the work model 100M4 of the fourth orientation OR4. 4_1 See also. Then, processor 32 processes this similar information SI 4_1 Linking information I included 4_1 Refer to the following and work model 100M which includes a feature model that is similar to the main body model 102M of work model 100M4 in the fourth posture OR4. n As such, multiple work models 100M n From these, select work model 100M1 in the first posture OR1.

[0188] In this way, the processor 32 matches the shape data SD1 with the work model 100M2 or 100M4 to obtain position data PD 3_1 When obtaining similar information SI 2_1 or SI 4_1 A work model 100M1 containing a feature model 102M that corresponds to a feature (main body 102) identified as having a similar relationship to the feature model 102M is provided for multiple work models 100M n Select from among them. Therefore, processor 32 will select from multiple work models 100M n It functions as a model selection unit 64 (Figure 20) that selects work model 100M1 from among them.

[0189] In step S49, the processor 32 functions as a position data acquisition unit 58 and, according to a predetermined matching algorithm, acquires the work model 100M selected in the previous step S48. n The j-th position data PD 3_i Shape data SD obtained i Then, matching is performed again. For example, in the example of Figure 22(c) or Figure 23(c) described above, the processor 32 matches the work model 100M1 selected in the previous step S48 to the shape data SD1.

[0190] In step S50, the processor 32 functions as a matching determination unit 62 and determines the work model 100M that was matched in the preceding step S49. n Among the feature models included, the feature model identified by the dissimilar information NI is the j-th position data PD. 3_i Shape data SD obtained i Determine whether the features captured in the image are consistent with the actual features.

[0191] For example, if the work model 100M1 was matched to the shape data SD1 in the previous step S49, the processor 32 will determine the dissimilar information NI in the work model 100M1. 1_2 It is determined whether the end face model 110M identified by this method matches the features reflected in the shape data SD1.

[0192] Specifically, the processor 32 obtains the degree of agreement γ'' between the end face model 110M and the features of the shape data SD1, and when the degree of agreement γ'' is set to a predetermined threshold γ th If it exceeds (γ) > γ th ", or, γ" < γ th "), dissimilar information NI 1_2 It is determined that the end face model 110M identified by this is consistent with the features reflected in the shape data SD1 (i.e., YES).

[0193] When the work model 100M1 is matched with the shape data SD1 (see, for example, Figure 18), the processor 32 will determine YES in step S50. If the processor 32 determines YES, it proceeds to step S44, where it functions as a position data cancellation unit 60 and takes the first position data PD acquired in step S22 described above. 3_1 Disable.

[0194] If the result in step S50 is YES, then the first position data PD obtained in step S22 3_1 This means that the matching performed when acquiring the data was inappropriate (Figure 22(c) or Figure 23(c)), while the matching performed in step S49 was appropriate.

[0195] Therefore, if the processor 32 determines YES in step S50, it executes step S44 and the first position data PD obtained in step S22 3_1 Disable it. Meanwhile, in step S50, the processor 32 determines that the feature model identified by the dissimilar information NI is the shape data SD i If the features captured do not match, the result is determined to be NO, and the process proceeds to step S45.

[0196] Thus, in this embodiment, the processor 32 functions as a model acquisition unit 44, an information acquisition unit 46, a feature extraction unit 48, a model matching execution unit 50, a frequency calculation unit 52, a similarity determination unit 54, a mismatch determination unit 56, a position data acquisition unit 58, a position data cancellation unit 60, a consistency determination unit 62, and a model selection unit 64, to acquire the work model 100M n Obtain dissimilar information (NI).

[0197] Therefore, the model acquisition unit 44, information acquisition unit 46, feature extraction unit 48, model matching execution unit 50, frequency calculation unit 52, similarity determination unit 54, inconsistency determination unit 56, position data acquisition unit 58, position data cancellation unit 60, consistency determination unit 62, and model selection unit 64 constitute a device 88 (Figure 20) for acquiring dissimilar information NI.

[0198] In this device 88, the information acquisition unit 46 acquires similar information SI to identify feature models (main body model 102M) that are similar in relationship to the features (main body 102) of the workpiece 100 in the first orientation OR1 in the workpiece model 100M2 (or 100M4) in the second orientation OR2 (or fourth orientation OR4). 2_1 (or SI 4_1 Obtain (step S14).

[0199] Then, the model selection unit 64 matches the work model 100M2 (or 100M4) with the shape data SD1 using the position data acquisition unit 58, thereby generating position data PD 3_1 When obtaining similar information SI 2_1 (or SI 4_1 A work model 100M1 in a first orientation OR1 that includes a feature model 102M corresponding to a feature (main body 102) identified as having a similar relationship to the feature model 102M, and multiple work models 100M n Select from the options (Step S48).

[0200] Furthermore, when the position data acquisition unit 58 matches the work model 100M1 selected by the model selection unit 64 to the shape data SD1 again (step S49), the matching determination unit 62 determines the dissimilar information NI1_2 It is determined whether the feature model identified by (end face model 110M) matches the feature (end face 110) reflected in the shape data SD1 (step S50).

[0201] Then, if the position data cancellation unit 60 determines that the feature model 110M is consistent with the feature 110 (i.e., YES in step S50), the position data PD acquired by the position data acquisition unit 58 is canceled by the position data cancellation unit 60. 3_1 Disable (step S44). With this configuration, if in step S22 above the position data PD is incorrectly matched as shown in Figure 22(c) or Figure 23(c), 3_i If obtained, the position data PD 3_i This avoids the use of the above-mentioned step S25 for control.

[0202] Furthermore, when the processor 32 determines YES in step S50, it functions as a position data acquisition unit 58 and rematches the position data PD of the work model 100M1 with the shape data SD1. 3_1 ' may be obtained. Then, in the subsequent step S44, the processor 32 obtains the original position data PD 3_1 While disabling the newly acquired location data PD 3_1 ' may be added to the detection result data DT.

[0203] Furthermore, as a result of step S22, one shape data SD is obtained. i In this case, multiple position data PD may be acquired. For example, in step S22, the processor 32 matches the work model 100M1 (Figure 3) in the first orientation OR1 with the shape data SD1 shown in Figure 22(b), thereby acquiring the position data PD 3_1_1 Obtain it.

[0204] In addition, in step S22, the processor 32 matches the shape data SD1 shown in Figure 22(b) with the work model 100M2 of the second orientation OR2 shown in Figure 22(a), thereby generating the position data PD3_1_2 can be obtained. Thus, for one shape data SD1, a plurality of position data PD 3_1_1 and PD 3_1_2 An example of the data structure of the detection result data DT when obtained is shown in Table 2 below.

[0205]

Table 2

[0206] In this case, the total number i of the shape data SD i shown in the detection data DD and the total number j of the position data PD 3_i obtained in step S22 may be different (i < j). In this case, the processor 32 executes the flow of FIG. 24 to invalidate the position data PD 3_1_2 obtained as a result of inappropriate matching shown in FIG. 22(c). On the other hand, the processor 32 maintains the position data PD 3_1_1 obtained as a result of appropriate matching.

[0207] Note that the processor 32 may execute the flows shown in FIGS. 12, 14, 16, 17, 19, 21 or 24 according to a computer program PG stored in advance in the memory 34. Also, the functions of the devices 80, 82, 84, 86 or 88 (that is, the model acquisition unit 44, the information acquisition unit 46, the feature extraction unit 48, the model matching execution unit 50, the frequency calculation unit 52, the similarity determination unit 54, the inconsistency determination unit 56, the position data acquisition unit 58, the position data cancellation unit 60, the consistency determination unit 62, the model selection unit 64) executed by the processor 32 may be functional modules realized by the computer program PG.

[0208] Also, various changes can be made to the flows shown in FIGS. 12, 14, 16, 17, 19, 21 or 24. For example, in the above-described embodiment, in the flow of FIG. 16, the case where the processor 32 executes step S23 after step S22 (more specifically, step S34) has been described.

[0209] However, the processor 32 may also execute step S23 after step S32 or S33 in Figure 17. For example, if step S23 is executed after step S32, the processor 32 will use the initial position data PD acquired in step S32. 1_i Based on this, steps S41-S45 in Figure 19, or steps S41-S50 in Figure 24 are executed. Then, when NO is determined in step S43 (or YES in S50), the initial position data PD is obtained in step S44. 1_i Disable.

[0210] Furthermore, the coarse search RS in step S32 and the dense search PS in step S33 described above are examples of methods for obtaining the position data PD of the workpiece 100, and any modifications may be made to the coarse search RS or dense search PR, or the position data PD of the workpiece 100 may be obtained by any other method that does not involve performing the coarse search RS and dense search PR.

[0211] Furthermore, step S2 may be omitted from the flowchart in Figure 12, Figure 14, or Figure 21. In this case, the feature extraction unit 48 can be omitted from devices 82, 84, 86, or 88. Also, step S12 may be omitted from the flowchart in Figure 21. The mismatch determination unit 56 can be omitted from device 88.

[0212] Furthermore, in step S5 of Figure 12, Figure 14, or Figure 21, the processor 32 may perform model matching similar to step S32 (coarse search RS) or step S33 (dense search PS) described above. Also, in step S6 of Figure 12, Figure 14, or Figure 21, the processor 32 performs matching γ in addition to matching β. i , plane proportion δ i , and exposure rate ε i The frequency α may be determined by taking into account the detection result parameters PM, such as the above. For example, in step S6, the processor 32 determines that the degree of agreement β is the threshold β th1 The frequency α may be calculated if the detection result parameter PM exceeds a certain threshold and also meets a predetermined standard (for example, exceeds the standard value).

[0213] Furthermore, in step S3 of Figure 12, Figure 14, or Figure 21, the shape detection sensor 14 does not necessarily have to be placed at a known position in the control coordinate system C (specifically, the robot coordinate system C1). Then, the processor 32 determines that the shape detection sensor 14, which is placed at an arbitrary position, is capable of various orientations OR n Alternatively, detection data DD may be obtained by repeatedly imaging each workpiece 100 one by one, and matching in step S5 may be performed for each detection data DD. As described above, various modifications can be made to the flows shown in Figures 12, 14, 16, 17, 19, 21, or 24.

[0214] Furthermore, the above-described embodiment mentions the case where the functions of devices 80, 82, 84, 86, and 88 (i.e., the model acquisition unit 44, the information acquisition unit 46, the feature extraction unit 48, the model matching execution unit 50, the frequency calculation unit 52, the similarity determination unit 54, the mismatch determination unit 56, the position data acquisition unit 58, the position data cancellation unit 60, the consistency determination unit 62, and the model selection unit 64) are implemented in the control device 16.

[0215] However, the functions of devices 80, 82, 84, 86, or 88 are not limited to these, and may be implemented in, for example, a teaching device (teaching pendant, tablet-type mobile terminal, etc.) that teaches the robot 12 the actions for the task of step S25, or in any other computer such as a PC. In this case, the processor of the teaching device or the computer such as a PC functions as device 80, 82, 84, 86, or 88.

[0216] In the above-described embodiment, the shape detection sensor 14 is fixed to the end effector 28 (or wrist flange 26b) and moved by the robot 12. However, the shape detection sensor 14 is not limited to this and may be fixed at a known position in the control coordinate system C (robot coordinate system C1), for example, using a support structure.

[0217] Furthermore, the shape detection sensor 14 is not limited to a 3D vision sensor, but may also be a laser scanner capable of detecting the 3D shape of an object. Alternatively, the shape detection sensor 14 may have a 2D camera and a distance measuring sensor capable of measuring the distance d to the subject. Also, the detection data DD detected by the shape detection sensor 14 is not limited to 3D point cloud image data as shown in Figure 6, but may include a dataset of 2D image data captured by the 2D camera and the distance d measured by the distance measuring sensor, or it may be any other type of image data (for example, distance image data).

[0218] Furthermore, the robot 12 is not limited to a vertical articulated robot, but may be any type of robot, such as a horizontal articulated robot or a parallel link robot. The present disclosure has been explained through the embodiments described above, but the embodiments described above do not limit the invention to the scope of the claims.

[0219] As described above, this disclosure describes the following aspects. (Aspect 1) A model acquisition unit for acquiring multiple work models, each modeling a work in a multiple posture, wherein the work model includes a feature model corresponding to the visual features of the work; and an information acquisition unit for acquiring dissimilarity information for identifying a first feature model in a first work model of one posture that is dissimilar to the features of a work in another posture. (Aspect 2) The apparatus according to aspect 1, further comprising a feature extraction unit that extracts a plurality of feature models contained in each work model and assigns an identification code to each of the extracted feature models, and an information acquisition unit that acquires dissimilar information in association with the identification code assigned to the first feature model. (Aspect 3) The apparatus according to aspect 1 or 2, further comprising: a model matching execution unit that matches a work model to shape data of a work detected by a shape detection sensor; a frequency calculation unit that determines the frequency with which a feature model included in the first work model matches a feature reflected in each of the shape data when the model matching execution unit matches the first work model to shape data of a plurality of orientations; and a similarity determination unit that determines whether each of the feature models is in a dissimilar relationship based on the frequencies obtained by the frequency calculation unit for the plurality of feature models included in the first work model, wherein the information acquisition unit acquires dissimilar information for the first feature model that has been determined to be in a dissimilar relationship by the similarity determination unit among the plurality of feature models included in the first work model. (Aspect 4) The apparatus according to aspect 3, wherein the similarity determination unit determines that the feature models are dissimilar when the frequency calculated by the frequency calculation unit is below a predetermined threshold. (Aspect 5) The apparatus according to aspect 3 or 4, wherein when a model matching execution unit performs matching using a first work model, the apparatus further comprises a mismatch determination unit that determines whether or not a mismatch occurs between at least one feature model included in the first work model and the features reflected in the shape data, and the frequency calculation unit determines the frequency at which a feature model other than the at least one feature model matches the features when the mismatch determination unit determines that a mismatch has occurred. (Aspect 6) The apparatus according to any one of aspects 3 to 5, wherein the model matching execution unit performs matching using shape data obtained by actually detecting the shape of the workpiece in real space using a shape detection sensor, or performs matching using shape data obtained by a simulation in which the shape detection sensor simulates detecting the shape of the workpiece in a virtual space. (Aspect 7) The apparatus according to any one of aspects 1 to 6, further comprising: a position data acquisition unit that acquires position data of a workpiece in a control coordinate system by matching a workpiece model to shape data of a workpiece detected by a shape detection sensor positioned at a known position in the control coordinate system; and a position data cancellation unit that invalidates the position data acquired by the position data acquisition unit when an inconsistency occurs between the shape data and the workpiece model that the position data acquisition unit has matched to the shape data in order to acquire the position data. (Aspect 8) The apparatus according to aspect 7, wherein when a position data acquisition unit acquires first position data by matching a first work model to shape data, the apparatus further comprises a consistency determination unit that determines whether or not a first feature model identified by dissimilar information is consistent with the features reflected in the shape data, and the position data cancellation unit invalidates the first position data acquired by the position data acquisition unit when the consistency determination unit determines that the first feature model is inconsistent with the features. (Aspect 9) The apparatus according to aspect 7, wherein the information acquisition unit further acquires similarity information to identify a second feature model that is similar in relationship to the features of a work in one orientation in a second work model of a different orientation, and the apparatus further includes a model selection unit that selects from a plurality of work models a first work model that includes a feature model corresponding to a feature identified by the similarity information as being similar in relationship to the features of the second feature model when the position data acquisition unit acquires second position data by matching the second work model to shape data, and a consistency determination unit that determines whether the first feature model identified by the dissimilarity information is consistent with the features reflected in the shape data when the position data acquisition unit matches the first work model selected by the model selection unit to shape data again, and the position data cancellation unit invalidates the second position data acquired by the position data acquisition unit when the consistency determination unit determines that the first feature model is consistent with the features. (Aspect 10) A control device that controls the operation of a robot that performs a predetermined operation on a workpiece, comprising the apparatus described in any of aspects 1 to 9. (Aspect 11) A robot system comprising a robot that performs a predetermined operation on a workpiece, a shape detection sensor that detects the shape of the workpiece, and the apparatus described in any one of aspects 1 to 9. (Aspect 12) A method for a processor to acquire a plurality of work models, each of which models a work in a plurality of orientations, and each of which includes a feature model corresponding to the visual features of the work, and to acquire dissimilarity information for identifying a first feature model in a first work model of one orientation that is dissimilar to the features of a work in another orientation. [Explanation of symbols]

[0220] 10 Robot Systems 12 Robots 14 Shape detection sensors 16 Control device 32 processors 44 Model Acquisition Section 46 Information Acquisition Department 48 Feature Extraction Unit 50 Model Matching Execution Unit 52 Frequency Calculation Unit 54 Similarity determination section 56 Inconsistency judgment section 58 Location data acquisition unit 60 Location data cancellation unit 62 Consistency judgment section 64 Model Selection Section 100 Work 100M Work Model

Claims

1. A model acquisition unit that acquires multiple work models, each modeling a work in multiple postures, wherein the work model includes a feature model corresponding to the visual characteristics of the work. An apparatus comprising: an information acquisition unit that acquires dissimilarity information for identifying a first feature model that is dissimilar to the features of the workpiece in one of the aforementioned orientations in the first workpiece model in one of the aforementioned orientations.

2. The system further includes a feature extraction unit that extracts a plurality of feature models contained in each of the aforementioned work models and assigns an identification code to each of the extracted feature models, The apparatus according to claim 1, wherein the information acquisition unit acquires the dissimilar information in association with the identification code assigned to the first feature model.

3. A model matching execution unit that matches the workpiece model to the shape data of the workpiece detected by the shape detection sensor, The model matching execution unit matches the first work model to the shape data of multiple poses, and the frequency calculation unit determines the frequency at which the feature model included in the first work model matches the features reflected in each of the shape data. The frequency calculation unit further comprises a similarity determination unit that determines whether each of the feature models is in a dissimilar relationship based on the frequencies obtained for the plurality of feature models included in the first work model, The apparatus according to claim 1, wherein the information acquisition unit acquires the dissimilarity information for the first feature model that has been determined to be in a dissimilar relationship by the similarity determination unit, among the plurality of feature models included in the first work model.

4. The apparatus according to claim 3, wherein the similarity determination unit determines that the feature models are in a dissimilar relationship when the frequency calculated by the frequency calculation unit is less than or equal to a predetermined threshold.

5. The model matching execution unit further comprises a mismatch determination unit that determines whether or not a mismatch occurs between at least one feature model included in the first work model and the feature reflected in the shape data when the model matching execution unit performs the matching using the first work model. The apparatus according to claim 3, wherein the frequency calculation unit determines, when the mismatch determination unit determines that the mismatch has occurred, the frequency at which a feature model other than the at least one feature model matches the feature.

6. The aforementioned model matching execution unit, In real space, the shape detection sensor actually detects the shape of the workpiece and performs the matching using the shape data obtained, or, The apparatus according to claim 3, wherein the matching is performed using the shape data obtained by a simulation in which the shape detection sensor simulates detecting the shape of the workpiece in a virtual space.

7. A position data acquisition unit acquires position data of the workpiece in the control coordinate system by matching the workpiece model with the shape data of the workpiece detected by a shape detection sensor placed at a known position in the control coordinate system. The apparatus according to claim 1, further comprising: a position data cancellation unit that invalidates the position data acquired by the position data acquisition unit when an inconsistency occurs between the work model matched to the shape data by the position data acquisition unit in order to acquire the position data and the shape data.

8. When the position data acquisition unit acquires the first position data by matching the first work model to the shape data, the system further includes a consistency determination unit that determines whether the first feature model identified by the dissimilar information is consistent with the features reflected in the shape data. The apparatus according to claim 7, wherein the position data cancellation unit invalidates the first position data acquired by the position data acquisition unit when the consistency determination unit determines that the first feature model is inconsistent with the features.

9. The information acquisition unit further acquires similar information to identify a second feature model in the second work model of the other orientation that is similar in relationship to the features of the work in the one orientation, The aforementioned device is When the position data acquisition unit acquires the second position data by matching the second work model with the shape data, the model selection unit selects from the plurality of work models the first work model which includes the feature model corresponding to the feature identified by the similarity information as being in a similar relationship with the second feature model, The position data acquisition unit further comprises a consistency determination unit that determines whether the first feature model identified by the dissimilar information matches the features reflected in the shape data when the first work model selected by the model selection unit is matched again with the shape data. The apparatus according to claim 7, wherein the position data cancellation unit invalidates the second position data acquired by the position data acquisition unit when the consistency determination unit determines that the first feature model is consistent with the features.

10. A control device that controls the operation of a robot that performs a predetermined operation on a workpiece, comprising the apparatus described in claim 1.

11. A robot that performs a predetermined task on a workpiece, A shape detection sensor for detecting the shape of the workpiece, A robot system comprising the apparatus described in claim 1.

12. The processor, Multiple work models are obtained, each modeling a workpiece in multiple postures, and each of these work models includes a feature model corresponding to the visual characteristics of the workpiece. A method for obtaining dissimilarity information to identify a first feature model that is dissimilar to the features of a workpiece in one of the aforementioned orientations.

Images