Device and method for acquiring position and orientation of workpiece, device and method for designating feature on workpiece, and computer program

Abstract

There have been cases in which, when a workpiece having a feature that serves as a reference for orientation is detected, the orientation of the workpiece cannot be detected accurately.　A device 80 for acquiring the position and the orientation of a workpiece 100, 120 comprises: a position acquisition unit 84 that acquires the position and the orientation by matching a workpiece model 100M, 120M, which models the workpiece 100, 120 and in which a feature model region FTe, FTe' corresponding to a feature FT is designated, to shape data SDi for the workpiece 100, 120 detected by a shape detection sensor 14; and a feature assessment unit 88 that assesses whether or not the feature is included in the shape data SDi on the basis of the data volume δ of the shape data SDi that overlaps with the feature model region FTe, FTe' designated in the workpiece model 100M, 120M matched to the shape data SDi.

Description

Apparatus and method for acquiring the position and orientation of a workpiece, apparatus and method for specifying characteristics of a workpiece, and computer program

[0001] The present disclosure relates to an apparatus and method for acquiring the position and orientation of a workpiece, an apparatus and method for specifying features of a workpiece, and a computer program.

[0002] There is known a technique for detecting a workpiece having a characteristic that serves as a reference for its posture, taking into consideration the characteristic (for example, Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2021-86432

[0004] When detecting a workpiece having characteristics that serve as a reference for its posture, there are cases where the posture cannot be detected accurately.

[0005] In one aspect of the present disclosure, an apparatus for acquiring the position and posture of a workpiece having features that serve as references for determining its posture includes a position acquisition unit that acquires the position and posture by matching a workpiece model, which is a model of the workpiece and has a feature model area designated corresponding to the features, to shape data of the workpiece detected by a shape detection sensor, and a feature determination unit that determines whether the shape data includes features based on the amount of data in the shape data that overlaps with the feature model area designated in the workpiece model matched to the shape data.

[0006] In another aspect of the present disclosure, an apparatus for acquiring the position and posture of a workpiece having a feature that serves as a reference for determining its posture includes a model generation unit that generates, based on a drawing model that represents the overall shape of the workpiece, a plurality of search models that each represent a partial shape of the drawing model as viewed from a plurality of viewpoints; a feature designation unit that designates a feature model area that corresponds to a feature in a search model that includes the feature among the plurality of search models generated by the model generation unit; a position acquisition unit that acquires the position and posture by matching each search model to shape data of the workpiece detected by a shape detection sensor; and a feature determination unit that determines that the feature is included in the shape data when the search model that matched the shape data when the position and posture was acquired contains the designated feature model area.

[0007] In yet another aspect of the present disclosure, an apparatus for specifying a feature in a workpiece having a reference feature for determining its posture includes an image generation unit that generates image data displaying a workpiece model that models the workpiece, an input reception unit that receives input specifying a feature model area corresponding to the feature in the workpiece model displayed in the image data, and a position data generation unit that generates position data indicating the position in the workpiece model of the feature model area specified by the input received by the input reception unit.

[0008] In yet another aspect of the present disclosure, an apparatus for specifying a feature in a workpiece having a reference feature for determining its posture includes a model placement unit that places a workpiece model that models the workpiece in a virtual space; a quantization execution unit that quantizes the virtual space in which the workpiece model is placed by the model placement unit into minimum image units; a simulation unit that executes a simulation in which the workpiece model is rotated by a predetermined angle around an axis of symmetry in the virtual space quantized by the quantization execution unit; an image unit identification unit that identifies the minimum image unit whose image value has changed before and after the simulation; and a feature designation unit that designates a feature model area in the workpiece model that corresponds to the feature based on the minimum image unit identified by the image unit identification unit.

[0009] In yet another aspect of the present disclosure, a method for acquiring the position and posture of a workpiece having features that serve as a reference for determining its posture includes: matching a workpiece model, which is a model of the workpiece, with a feature model area designated to correspond to the features, to shape data of the workpiece detected by a shape detection sensor to acquire the position and posture; and determining whether the shape data includes the features based on the amount of data in the shape data that overlaps with the feature model area designated in the workpiece model matched to the shape data.

[0010] In yet another aspect of the present disclosure, a method for acquiring the position and posture of a workpiece having a feature that serves as a reference for determining its posture includes generating, based on a drawing model that represents the overall shape of the workpiece, a plurality of search models that each represent a partial shape of the drawing model as viewed from a plurality of viewpoints, specifying a feature model area corresponding to the feature in a search model that includes the feature among the plurality of search models generated, matching each search model to shape data of the workpiece detected by a shape detection sensor to acquire the position and posture, and determining that the feature is included in the shape data if the specified feature model area is present in the search model that matched the shape data when the position and posture were acquired.

[0011] In yet another aspect of the present disclosure, a method for specifying a feature in a workpiece having a reference feature for determining its posture includes generating image data displaying a workpiece model that models the workpiece, accepting input in the workpiece model displayed in the image data that specifies a feature model area corresponding to the feature, and generating position data indicating the position in the workpiece model of the feature model area specified by the accepted input.

[0012] In yet another aspect of the present disclosure, a method for specifying a feature in a workpiece having a reference feature for determining its posture includes placing a workpiece model that models the workpiece in a virtual space, quantizing the virtual space in which the workpiece model is placed into minimum image units, performing a simulation in which the workpiece model is rotated by a predetermined angle around an axis of symmetry in the quantized virtual space, identifying the minimum image unit whose image value has changed before and after the simulation, and specifying a feature model area in the workpiece model that corresponds to the feature based on the identified minimum image unit.

[0013] 1. A schematic diagram of a robot system according to one embodiment. A block diagram of the robot system shown in FIG. 1. A top view of a workpiece and drawing model according to one embodiment. A perspective view of the workpiece and drawing model shown in FIG. 3. A side view of the workpiece and drawing model shown in FIG. 3. A top view of a workpiece and drawing model according to another embodiment. A perspective view of the workpiece and drawing model shown in FIG. 6. A side view of the workpiece and drawing model shown in FIG. 6. A GUI is shown when specifying characteristics for a workpiece model. A block diagram showing other functions of the robot system shown in FIG. 1. A flowchart showing a method for specifying characteristics for a workpiece in the robot system shown in FIG. 10. A diagram schematically showing a state in which a virtual space in which a workpiece model is arranged is quantized. A diagram showing a state after a simulation in which a workpiece model is moved in the virtual space, corresponding to FIG. 12. A diagram showing the smallest image unit in which image values ​​change before and after the simulation. A diagram showing a workpiece model moved in the virtual space before and after the simulation. A diagram schematically showing a state in which a virtual space in which a workpiece model is arranged is quantized. A diagram showing a state after a simulation in which a workpiece model is moved in the virtual space, corresponding to FIG. 16. A diagram showing a workpiece model moved in the virtual space before and after the simulation. A block diagram showing further other functions of the robot system shown in FIG. 1. 28. A flowchart showing the flow of work on a workpiece in the robot system shown in FIG. 19. An example of image data captured by a shape detection sensor is shown. A search model of the workpiece shown in FIG. 3 is shown. A search model of the workpiece shown in FIG. 4 is shown. A search model of the workpiece shown in FIG. 5 is shown. A search model of the workpiece shown in FIG. 3 when viewed from the back side is shown. A state in which the search model is matched to image data captured by the shape detection sensor is shown. An example of detection result data is shown. A flowchart showing the flow of step S13 in FIG. 20. A state in which a search model is matched to shape data in step S22 in FIG. 28. Quantized image data in step S24 in FIG. 28 is shown, showing a state in which another search model is matched to shape data in step S22 in FIG. 28. A flowchart showing the flow of step S14 in FIG. 20. A search model of the workpiece shown in FIG. 6 is shown. A search model of the workpiece shown in FIG. 7 is shown.20 shows a search model for the workpiece shown in Fig. 8. FIG. 21 schematically shows image data quantized in step S24 in Fig. 28. FIG. 22 is a block diagram showing still another function of the robot system shown in Fig. 1. FIG. 23 is a flowchart showing another flow of step S13 in Fig. 20.

[0014] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In various embodiments described below, like elements will be designated by like reference numerals, and duplicated descriptions will be omitted. First, a robot system 10 according to one embodiment will be described with reference to FIGS. 1 and 2. The robot system 10 includes a robot 12, a shape detection sensor 14, and a control device 16.

[0015] In this embodiment, the robot 12 is a vertical articulated robot and includes a robot base 18, a rotating body 20, a lower arm 22, an upper arm 24, a wrist 26, and an end effector 28. The robot base 18 is fixed to the floor of a work cell or to an automated guided vehicle (AVG). The rotating body 20 is attached to the robot base 18 so as to be rotatable about a vertical axis. The lower arm 22 is attached to the rotating body 20 so as to be rotatable about a horizontal axis, and the upper arm 24 is rotatably attached to the distal end of the lower arm 22. The wrist 26 is attached to the distal end of the upper arm 24 so as to be rotatable about two axes that are perpendicular to each other.

[0016] The end effector 28 is detachably attached to the wrist 26. The end effector 28 is, for example, a robot hand capable of gripping a workpiece, a welding torch for welding a workpiece, or a laser processing head for laser processing a workpiece, and performs a predetermined operation on the workpiece (workpiece handling, welding, or laser processing).

[0017] Each component of the robot 12 (robot base 18, rotating body 20, lower arm 22, upper arm 24, wrist 26) is provided with a servo motor 30 ( FIG. 2 ). These servo motors 30 rotate each movable element of the robot 12 (rotating body 20, lower arm 22, upper arm 24, wrist 26, end effector 28) in response to a command from the control device 16. As a result, the robot 12 moves the end effector 28 to a desired position.

[0018] The shape detection sensor 14 detects the shape of an object such as a workpiece. In this embodiment, the shape detection sensor 14 is a three-dimensional visual sensor including a pair of image sensors (CMOS, CCD, etc.), a pair of optical lenses (collimating lenses, focusing lenses, etc.) that respectively guide an object image to the pair of image sensors, and an image processing processor. The shape detection sensor 14 may be fixed at a predetermined imaging position Pi. The shape detection sensor 14 is configured to capture an image of the object along an optical axis A1 and measure the distance to the object.

[0019] 1, a robot coordinate system C1 and a tool coordinate system C2 are set for the robot 12. The robot coordinate system C1 is a coordinate system for automatically controlling the operation of each movable element of the robot 12. In this embodiment, the robot coordinate system C1 is set with respect to the robot base 18 so that its origin is located at the center of the robot base 18 and its z axis is parallel to (specifically, coincides with) the rotation axis of the rotating body 20.

[0020] On the other hand, the tool coordinate system C2 is a coordinate system that defines the position and posture of the end effector 28 in the robot coordinate system C1. In this embodiment, the tool coordinate system C2 is set with respect to the end effector 28 so that its origin (so-called TCP) is located at a work position of the end effector 28 (for example, a workpiece gripping position, a welding position, or a laser beam emission port).

[0021] When moving the end effector 28, the control device 16 sets a tool coordinate system C2 in the robot coordinate system C1 and generates commands to the servo motors 30 of the robot 12 so as to place the end effector 28 at the position and orientation represented by the set tool coordinate system C2. In this way, the control device 16 can position the end effector 28 at any position and orientation in the robot coordinate system C1.

[0022] Meanwhile, a sensor coordinate system C3 is set for the shape detection sensor 14. The sensor coordinate system C3 is a coordinate system that defines the position and orientation of the shape detection sensor 14 in the robot coordinate system C1 (i.e., the position and direction of the optical axis A1). In this embodiment, the sensor coordinate system C3 is set for the shape detection sensor 14 so that its origin is located at the center of the image sensor of the shape detection sensor 14 and its z-axis is parallel to (specifically, coincides with) the optical axis A1.

[0023] The positional relationships among the robot coordinate system C1, the tool coordinate system C2, and the sensor coordinate system C3 are known by calibration, and therefore the coordinates of the robot coordinate system C1, the tool coordinate system C2, and the sensor coordinate system C3 can be mutually converted via a known transformation matrix (e.g., a homogeneous transformation matrix).

[0024] The control device 16 controls the operations of the robot 12 and the shape detection sensor 14. Specifically, as shown in Fig. 2, the control device 16 is a computer having a processor 32, a memory 34, an I / O interface 36, a display device 38, and an input device 40. The processor 32 has a CPU or a GPU, etc., and is communicatively connected to the memory 34, the I / O interface 36, the display device 38, and the input device 40 via a bus 42, and performs arithmetic processing to realize various functions described below while communicating with these components.

[0025] The memory 34 includes RAM, ROM, or the like, and temporarily or permanently stores various data. The memory 34 may be configured from a computer-readable non-transitory storage medium, such as a volatile memory, a non-volatile memory, a magnetic storage medium, or an optical storage medium. The I / O interface 36 includes, for example, an Ethernet (registered trademark) port, a USB port, an optical fiber connector, or an HDMI (registered trademark) terminal, and communicates data with external devices via wired or wireless communication under instructions from the processor 32. The servo motors 30 of the robot 12 and the shape detection sensor 14 are communicatively connected to the I / O interface 36.

[0026] The display device 38 has a liquid crystal display, an organic EL display, or the like, and visibly displays various data under instructions from the processor 32. The input device 40 has a push button, a switch, a keyboard, a mouse, a touch panel, or the like, and receives data input from an operator. The display device 38 and the input device 40 may be integrally incorporated into the housing of the control device 16, or may be connected to the I / O interface 36 as a single computer (PC, etc.) separate from the housing of the control device 16.

[0027] In this embodiment, the processor 32 performs an operation on workpieces stacked in a container E (not shown). FIGS. 3 to 5 show a workpiece 100 according to one embodiment. The workpiece 100 has a cylindrical main body 102 having a central axis A2 and a cylindrical protrusion 104 protruding from the main body 102. The protrusion 104 is disposed at a position spaced apart from the central axis A2 of the main body 102. Specifically, the protrusion 104 has a cylindrical outer peripheral surface 108 extending from an upper surface 106 of the main body 102 and an end surface 110 connected to the edge of the outer peripheral surface 108. The main body 102 is symmetrical with respect to the central axis A2. Therefore, the central axis A2 constitutes the axis of symmetry of the workpiece 100.

[0028] 6 to 8 show a workpiece 120 according to another embodiment. The workpiece 120 has a cylindrical main body 122 having a central axis A3 and two holes 124 penetrating the main body 122. Each hole 124 is defined by a cylindrical inner circumferential surface 126. An edge 130 of the hole 124 is defined on an upper surface 128 of the main body 122, while an edge 132 of the hole 124 is defined on a lower surface 134 of the main body 122. The main body 122 is symmetrical with respect to the central axis A3. Therefore, the central axis A3 constitutes the axis of symmetry of the workpiece 120.

[0029] A workpiece coordinate system C4 is set for the workpieces 100 and 120. The workpiece coordinate system C4 is a coordinate system that represents the position and orientation of the workpieces 100 and 120 in the robot coordinate system C1. In the workpiece 100 shown in FIGS. 3 to 5 , the workpiece coordinate system C4 is set with respect to the workpiece 100 so that its origin is located at the center of the main body 102, its y-axis positive direction points toward the central axis of the protrusion 104, and its z-axis positive direction points toward the top surface 106. The protrusion 104 (specifically, the outer peripheral surface 108 and the end surface 110) constitutes a feature FT that serves as a reference for determining the orientation of the workpiece 100, as shown in the workpiece coordinate system C4 in FIGS. 3 to 5 .

[0030] 6 to 8, the workpiece coordinate system C4 is set with respect to the workpiece 120 so that its origin is located at the center of the main body 122, its y-axis positive direction points in the direction of the central axis of one hole 124, its x-axis positive direction points in the direction of the central axis of the other hole 124, and its z-axis positive direction points in the direction of the top surface 128. The hole 124 (specifically, the inner peripheral surface 126, and the edges 130 and 132) constitutes a feature FT that serves as a reference for determining the posture of the workpiece 120, as shown in the workpiece coordinate system C4 in FIGS.

[0031] In this embodiment, the processor 32 of the control device 16 specifies features FT that serve as a reference for the posture of the workpiece 100 or 120. A method for specifying the features FT for the workpiece 100 will be described below. In this embodiment, the processor 32 first acquires a workpiece model 100M that models the workpiece 100 to be worked on. In this embodiment, the workpiece model 100M includes a drawing model 100D and a search model 100S.

[0032] The drawing model 100D is, for example, a three-dimensional CAD model, and represents the overall shape of the workpiece 100. The drawing model 100D is created by an operator using a design support system (a so-called CAD / CAM system) or the like, and is downloaded to the control device 16. The processor 32 acquires the drawing model 100D through the I / O interface 36 and stores it in the memory 34.

[0033] As shown in Figures 3 to 5, the drawing model 100D has a main body model 102D and a protrusion model 104D (outer peripheral surface model 108D, end surface model 110D). The above-mentioned workpiece coordinate system C4 is set in the drawing model 100D, and the model components (faces, edges, vertices, etc.) of the drawing model 100D are expressed as coordinates of the workpiece coordinate system C4. On the other hand, the search model 100S is used to search for the workpiece 100 from image data of the workpiece 100 captured by the shape detection sensor 14 for work. The search model 100S will be described later.

[0034] In this embodiment, the processor 32 generates image data 200 that displays the drawing model 100D together with the workpiece coordinate system C4 in the three-dimensional virtual space VS, and displays the image data 200 on the display device 38. Therefore, the processor 32 functions as an image generation unit 52 ( FIG. 2 ) that generates image data 200 that displays the workpiece model 100M (specifically, the drawing model 100D) that is a model of the workpiece 100.

[0035] An example of the generated image data 200 is shown in Fig. 9. While visually viewing the image data 200, the operator operates the input device 40 to specify a feature model area FTe corresponding to the feature FT in the drawing model 100D. As an example, the processor 32 displays a designation box image 202 in the virtual space VS, as shown in Fig. 9. By operating the input device 40, the operator can move the designation box image 202 to any position in the virtual space VS and change the size (length, width, and height) of the designation box image 202.

[0036] The operator can specify a desired model component of the drawing model 100D as the feature model region FTe by manipulating the input device 40 to surround it with a designation box image 202. For example, the operator provides an input IP1 through the input device 40 to surround the protrusion model 104D with the designation box image 202, as shown in Fig. 9. The processor 32 then receives the input IP1, identifies the protrusion model 104D (or the outer peripheral surface model 108D or the end surface model 110D) that is within the designation box image 202, and specifies it as the feature model region FTe.

[0037] 9 , the processor 32 displays a cursor image 204 that moves within the virtual space VS in response to an operation on the input device 40. The operator can operate the input device 40 to select a desired model component of the drawing model 100D using the cursor image 204. For example, the operator provides an input IP2 that clicks to select at least one (e.g., both) of the outer peripheral surface model 108D and the end surface model 110D that constitute the protrusion model 104D in the drawing model 100D (or specifies an area for selecting the model by a drag-and-drop operation).

[0038] The processor 32 then receives the input IP2 and designates the outer circumferential surface model 108D and the end surface model 110D selected by the cursor image 204 as the feature model area FTe. In this manner, in the present embodiment, the processor 32 functions as the input receiving unit 54 ( FIG. 2 ) that receives the input IP1 or IP2 that designates the feature model area FTe in the drawing model 100D displayed in the image data 200.

[0039] Next, the processor 32 generates position data Pf indicating the position in the drawing model 100D of the protrusion model 104D (or the outer peripheral surface model 108D and the end surface model 110D) specified in the feature model area FTe by the above-mentioned operation. Specifically, the processor 32 obtains the coordinates in the workpiece coordinate system C4 of the exclusive area of ​​the specified protrusion model 104D, outer peripheral surface model 108D, or end surface model 110D, and generates new position data Pf for the feature model area FTe.

[0040] In this way, the processor 32 functions as a position data generating unit 56 ( FIG. 2 ) that generates position data Pf indicating the position of the feature model region FTe specified by the input IP1 or IP2 in the drawing model 100D (specifically, the workpiece coordinate system C4). The processor 32 stores the generated position data Pf in association with the drawing model 100D. For example, the processor 32 may store the position data Pf in the memory 34 as data separate from the drawing model 100D in association with the drawing model 100D. Alternatively, the processor 32 may associate the generated position data Pf with the drawing model 100D by storing it within the data of the drawing model 100D.

[0041] The processor 32 can similarly specify the feature model region FTe for the workpiece 120 shown in Figures 6 to 8. Specifically, the processor 32 acquires a workpiece model 120M that models the workpiece 120. The workpiece model 120M includes a drawing model 120D and a search model 120S. The search model 120S will be described later. As shown in Figures 6 to 8, the drawing model 120D (three-dimensional CAD model) represents the overall shape of the workpiece 120. Specifically, the drawing model 120D includes a main body model 122D and a hole model 124D (inner peripheral surface model 126D, and edge models 130D and 132D).

[0042] The processor 32 then functions as the image generation unit 52 to generate image data 200 that displays the drawing model 120D and the workpiece coordinate system C4. The processor 32 also displays a designation box image 202 or a cursor image 204 on the image data 200, and functions as the input reception unit 54 to receive an input IP1 or IP2 that designates the feature model area FTe using the designation box image 202 or the cursor image 204.

[0043] For example, the operator specifies two hole models 124D (or the inner peripheral surface model 126D, and the edge models 130D and 132D) as the feature model region FTe. Then, the processor 32 functions as the position data generator 54 to generate position data Pf indicating the position in the workpiece coordinate system C4 of the regions occupied by the hole model 124D, the inner peripheral surface model 126D, or the edge models 130D and 132D.

[0044] As described above, in this embodiment, the processor 32 functions as the image generation unit 52, the input reception unit 54, and the position data generation unit 56 to specify the feature FT (feature model area FTe) in the workpiece 100 or 120 (specifically, the drawing model 100D or 120D). Therefore, the image generation unit 52, the input reception unit 54, and the position data generation unit 56 constitute a device 50 ( FIG. 2 ) that specifies the feature FT in the workpiece 100 or 120.

[0045] In this device 50, an image generation unit 52 generates image data 200 that displays a workpiece model 100M or 120M (specifically, a drawing model 100D or 120D) that is a model of a workpiece 100 or 120. An input reception unit 54 receives an input IP1 or IP2 that specifies a feature model region FTe in the workpiece model 100M or 120M displayed in the image data 200.

[0046] Then, the position data generation unit 56 generates position data Pf indicating the position in the workpiece model 100M or 120M (specifically, the coordinates in the workpiece coordinate system C4) of the feature model area FTe specified by the input IP1 or IP2 received by the input receiving unit 54. With this configuration, the operator can easily and appropriately specify the feature model area FTe in the workpiece model 100M or 120M through the image data 200 displaying the workpiece model 100M or 120M.

[0047] The processor 32 may execute the method of assigning the feature FT to the work 100 or 120 in the device 50 in accordance with a computer program PG2 pre-stored in the memory 34. The function of the device 50 executed by the processor 32 may be a functional module realized by the computer program PG2.

[0048] Next, other functions of the robot system 10 will be described with reference to FIG. 10. In this embodiment, the processor 32 of the control device 16 automatically specifies the above-mentioned features FT for the workpiece 100 or 120. A method for specifying the features FT for the workpiece 100 will be described below with reference to FIG. 11. In step S1, the processor 32 places the workpiece model 100M in the virtual space VS. In this embodiment, the processor 32 places the drawing model 100D shown in FIGS. 3 to 5 in the virtual space VS together with the workpiece coordinate system C4.

[0049] As described above, in this embodiment, the processor 32 functions as a model placement unit 62 (FIG. 10) that places the workpiece model 100M (drawing model 100D) in the virtual space VS. The processor 32 may also function as the image generation unit 52 described above, generate image data 200 in which the drawing model 100D and the workpiece coordinate system C4 are placed in the virtual space VS as shown in FIG. 9, and display the image data 200 on the display device 38.

[0050] In step S2, the processor 32 quantizes the virtual space VS in which the workpiece model 100M is placed into the smallest image unit IU. In this embodiment, the processor 32 quantizes the virtual space VS into voxels 206, which are the smallest image units IU. Figure 12 shows an enlarged view of a portion of the drawing model 100D shown in Figure 3 when the virtual space VS in which the drawing model 100D shown in Figure 3 is placed has been quantized into voxels 206.

[0051] 12 shows the boundary between the outer peripheral surface model 108D of the protrusion model 104D of the drawing model 100D and the hollow region of the virtual space VS, with the group of voxels 206 shown in gray indicating the region occupied by the protrusion model 104D (i.e., the solid region) and the group of voxels 206 shown in white indicating the hollow region. In this embodiment, the processor 32 sets a reference coordinate system C5, which is the basis for quantization, at an arbitrary position in the virtual space VS.

[0052] For example, the processor 32 may analyze the drawing model 100D and determine its center of gravity (or the center of the main body model 102D) and the central axis A2 as a symmetry axis. Then, the processor 32 may place the origin of the reference coordinate system C5 at the determined center of gravity and automatically set its z-axis to be parallel to the central axis A2. Alternatively, the processor 32 may set the position of the reference coordinate system C5 and the direction of each axis in response to an input operation by the operator via the input device 40, or may use any coordinate system, such as the coordinate system set in the drawing model 100D, as the reference coordinate system C5.

[0053] Next, the processor 32 quantizes the virtual space VS in which the drawing model 100D is placed into a plurality of voxels 206 arranged in a grid pattern along the x-, y-, and z-axis directions of the set reference coordinate system C5 in accordance with the image quantization program PG1 previously stored in the memory 34. Each voxel 206 has a unit volume Vu and is expressed as coordinates (x, y, z) in the reference coordinate system C5. The unit volume Vu may be defined by the image quantization program PG1. Alternatively, the operator may operate the input device 40 to arbitrarily set the size of the unit volume Vu.

[0054] Each voxel 206 has an image value Vi. In this embodiment, the image value Vi is numerical information such as "0" or "1." For example, in FIG. 12 , the image value Vi of the voxels 206 shown in gray (i.e., the region occupied by the protrusion model 104D) is "1," while the image value Vi of the voxels 206 shown in white (i.e., the hollow region) is "0." In this way, in this embodiment, the processor 32 functions as a quantization execution unit 64 ( FIG. 10 ) that quantizes the virtual space VS in which the workpiece model 100M (drawing model 100D) is arranged into the minimum image unit IU (voxel 206).

[0055] In step S3, the processor 32 executes a simulation SL in which the workpiece model 100M is rotated by a predetermined angle θ around the axis of symmetry in the quantized virtual space VS. Specifically, as the simulation SL, the processor 32 simulates rotating the drawing model 100D around the central axis A2 ( FIG. 2 ) serving as the axis of symmetry from the initial position P0 at the time of quantizing the virtual space VS in the voxels 206 by a predetermined angle θ (e.g., θ = 0.1°).

[0056] Figure 13 shows a voxel 206 after execution of the simulation SL. The voxel 206 shown in Figure 13 corresponds to Figure 12. As shown in Figure 13, the simulation SL causes the protrusion model 104D to move from the initial position P0 shown in Figure 12 relative to the reference coordinate system C5 in the virtual space VS. Meanwhile, the reference coordinate system C5 and the voxel 206 remain stationary. In this way, in this embodiment, the processor 32 functions as the simulating unit 66 (Figure 2) that executes the simulation SL.

[0057] In step S4, the processor 32 identifies the smallest image units IU (voxels 206) whose image values ​​Vi have changed between before and after the simulation SL in the immediately preceding step S3. As shown in Figures 12 and 13, as a result of executing step S3, the image values ​​Vi of some voxels 206 have changed from "1" to "0." In Figure 14, voxels 206A whose image values ​​Vi have changed from "1" to "0" are shown in gray.

[0058] The processor 32 monitors the image value Vi of each voxel 206 before and after step S3, and identifies a voxel 206A whose image value Vi has changed. The processor 32 then acquires the coordinates of the identified voxel 206A in the reference coordinate system C5. In this manner, in this embodiment, the processor 32 functions as an image unit identification section 68 ( FIG. 10 ) that identifies the smallest image unit IU (voxel 206) whose image value Vi has changed before and after the simulation SL.

[0059] In step S5, the processor 32 determines whether or not to end the simulation SL. Here, while the processor 32 determines NO in step S5, the processor 32 repeatedly executes the loop of steps S3 to S5. As a result of repeatedly executing step S3, the processor 32 determines whether or not the total angle θs (=Σθ) obtained by rotating the drawing model 100D around the central axis A2 from the initial position P0 in the virtual space VS has reached a predetermined threshold value θth.

[0060] For example, if the workpiece to be worked on has rotational symmetry with a symmetry angle θa, the threshold value θth may be set to θth = θa. On the other hand, in the case of the workpiece 100 shown in Figures 3 to 5, the main body 102 has infinite rotational symmetry. In this case, the threshold value θth may be set to any angle equal to or less than 360° (for example, θth = 90°). The threshold value θth is determined by the operator, taking into consideration the symmetry of the workpiece 100 to be worked on.

[0061] In this way, the processor 32 determines whether to end the simulation SL based on the total angle θs, and if the total angle θs has reached the threshold value θth, the determination is YES and the process proceeds to step S6. On the other hand, if the total angle θs has not reached the threshold value θth, the processor 32 determines NO, returns to step S3, and executes steps S3 and S4 again.

[0062] In step S6, the processor 32 specifies a feature model area FTe corresponding to the feature FT in the workpiece model 100M based on the minimum image unit IU identified in step S4. Here, by repeatedly executing step S3, the voxel 206 (gray voxel 206 in FIG. 12) that represents the protrusion model 104D of the drawing model 100D, which was located at the initial position P0, changes to a cavity area in the virtual space VS. This state is shown in FIG. 15.

[0063] 15 indicates the protrusion model 104D of the drawing model 100D that was placed at the initial position P0, while the solid line 104D indicates the protrusion model 104D after it has been moved by the simulation SL in step S3. The voxel 206A (FIG. 14) identified by repeating step S4 matches the gray region 104D' shown in FIG. 15, and therefore matches the protrusion model 104D as the feature FT.

[0064] In this way, the processor 32 can identify the protrusion model 104D based on the voxel 206A identified in step S4 and designate the protrusion model 104D as the feature model region FTe. In this way, the processor 32 functions as a feature designation unit 70 ( FIG. 10 ) that designates the feature model region FTe based on the minimum image unit IU (voxel 206A) identified in step S4.

[0065] In step S7, the processor 32 functions as the above-mentioned position data generator 56 to generate position data Pf (coordinates in the workpiece coordinate system C4) of the feature model area FTe specified in step S6. In this way, the processor 32 can automatically specify the feature FT (feature model area FTe) in the workpiece 100 (drawing model 100D).

[0066] By executing the flow of Fig. 11, the processor 32 can automatically specify the feature FT (feature model area FTe) also in the workpiece 120 (drawing model 120D) shown in Fig. 6 to Fig. 8. Specifically, in step S1, the processor 32 functions as the model placement unit 62 and places the drawing model 120D at the initial position P0 in the virtual space VS.

[0067] In step S2, the processor 32 functions as the quantization execution unit 64 to quantize the virtual space VS in which the drawing model 120D is arranged into voxels 206. Fig. 16 shows an enlarged view of the inner peripheral surface model 126D that defines the hole model 124D when the virtual space VS in which the drawing model 120D shown in Fig. 6 is arranged is quantized. In Fig. 16, the group of voxels 206 shown in white indicates the hole model 124D (i.e., the cavity region), while the group of voxels 206 shown in gray indicates the region occupied by the main body model 122D.

[0068] In step S3, the processor 32 functions as the simulator 66 and executes a simulation SL in which the drawing model 120D is rotated by an angle θ around the central axis A3, which serves as the axis of symmetry, in the quantized virtual space VS. FIG. 17 shows a voxel 206 after execution of the simulation SL. As shown in FIG. 17, the main body model 122D and the hole model 124D move relative to the reference coordinate system C5 within the virtual space VS as a result of the simulation SL.

[0069] In step S4, the processor 32 functions as the image unit identification unit 68 to identify voxels 206 whose image values ​​Vi have changed between before and after the immediately preceding step S3. Here, in the simulation SL of the drawing model 120D, the image values ​​Vi of some voxels 206 have changed from "0" to "1," as shown in Figures 16 and 17. Figure 14 shows a voxel 206A whose image value Vi has changed from "0" to "1." The processor 32 identifies the voxel 206A whose image value Vi has changed, and acquires the coordinates of the voxel 206A in the reference coordinate system C5.

[0070] Thereafter, if the determination in step S5 is YES, in step S6, the processor 32 functions as the feature designation unit 70 and designates a feature model region FTe in the drawing model 120D based on the voxel 206A identified in step S4. When step S3 is repeatedly executed in the drawing model 120D, the voxel 206 (white voxel 206 in FIG. 16 ) that represents the hole model 124D of the drawing model 120D and that was located at the initial position P0 changes to an exclusive region (i.e., a solid region) of the main body model 122D. This state is shown in FIG. 18 .

[0071] 18 indicates the hole model 124D of the drawing model 120D that was placed at the initial position P0, while the solid line 124D indicates the hole model 124D after it has been moved by the simulation SL in step S3. The voxel 206A (FIG. 14) identified in step S4 matches the gray region 124D' shown in FIG. 18, and therefore matches the hole model 124D as the feature FT.

[0072] The processor 32 identifies the hole model 124D (or the inner surface model 126D, the edge models 130D and 132D) based on the voxel 206A identified in step S4, and designates the hole model 124D as the feature model area FTe. Then, in step S7, the processor 32 functions as the position data generator 56 to generate position data Pf (coordinates in the workpiece coordinate system C4) for each of the hole models 124D designated in the feature model area FTe.

[0073] As described above, in this embodiment, the processor 32 functions as the model placement unit 62, quantization execution unit 64, simulating unit 66, image unit identification unit 68, feature designation unit 70, and position data generation unit 56 to designate features FT (feature model areas FTe) in the workpieces 100 and 120 (drawing models 100D and 120D). Therefore, the model placement unit 62, quantization execution unit 64, simulating unit 66, image unit identification unit 68, feature designation unit 70, and position data generation unit 56 constitute a device 60 ( FIG. 10 ) that designates features FT in the workpieces 100 and 120.

[0074] In this device 60, the model placement unit 62 places the work models 100M, 120M (drawing models 100D, 120D) in the virtual space VS (step S1), and the quantization execution unit 64 quantizes the virtual space VS in which the model placement unit 62 has placed the work models 100M, 120M into the smallest image unit IU (voxel 206) (step S2).

[0075] The simulating unit 66 executes a simulation SL (step S3) in which the workpiece models 100M, 120M (drawing models 100D, 120D) are rotated by a predetermined angle θ around the symmetry axes A2, A3 in the virtual space VS quantized by the quantization executing unit 64. The image unit identifying unit 68 identifies the smallest image unit IU (voxel 206A) whose image value Vi has changed before and after the simulation SL (step S4).

[0076] Then, the feature designation unit 70 designates a feature model area FTe (protrusion model 104D, hole model 124D) corresponding to the feature FT in the workpiece model 100M, 120M based on the minimum image unit IU (voxel 206A) identified by the image unit designation unit 68 (step S6). With this configuration, the feature model area FTe can be automatically searched for in the workpiece model 100M or 120M, and the feature model area FTe can be automatically designated in the workpiece model 100M or 120M. This significantly reduces the burden on the operator in designating the feature model area FTe.

[0077] Furthermore, in the device 60, the position data generation unit 56 generates position data Pf indicating the position of the feature model region FTe specified by the feature designation unit 70 on the workpiece models 100M, 120M. With this configuration, the position data Pf of the specified feature model region FTe can be automatically generated and stored. Note that the position data generation unit 56 may be omitted from the device 60. In this case, an operator may confirm the feature model region FTe (protrusion model 104D, hole model 124D) automatically specified by the processor 32 and manually generate the position data Pf of the feature model region FTe.

[0078] In the simulation SL of step S3 described above, the processor 32 may determine the angle θ based on the size (specifically, the unit volume Vu) of the smallest image unit IU (voxel 206) for the workpiece model 100M or 120M in the quantized virtual space VS. For example, in the case of the drawing model 100D shown in Figures 3 to 5, if the angle θ is excessively small, even if the processor 32 executes the simulation SL, the movement of the protrusion model 104D shown in Figures 12 and 13 within the virtual space VS will be extremely small. In this case, the number of voxels 206A shown in Figure 14 will be extremely small, which may result in a decrease in the efficiency of step S4 and an increase in the number of times the loop of steps S3 to S5 is repeatedly executed.

[0079] Conversely, if the angle θ is excessively large, it may be impossible to identify the feature FT (protrusion model 104D) with high accuracy. Therefore, the processor 32 determines the angle θ based on the size of the unit volume Vu of the voxel 206 with respect to the drawing model 100D placed in the virtual space VS quantized in step S2. Specifically, if the unit volume Vu of the voxel 206 with respect to the drawing model 100D when quantized in step S2 is large, the processor 32 sets the angle θ to a relatively large value.

[0080] On the other hand, if the unit volume Vu of the voxel 206 relative to the drawing model 100D after quantization is small, the processor 32 sets the angle θ to a relatively small value. As an example, the processor 32 calculates the ratio Rv=Vu / Vm of the unit volume Vu of the voxel 206 to the volume Vm of the occupied region of the drawing model 100D. The processor 32 then calculates the ratio Rv such that Rv<Rv ｔｈ１ If so, set the angle θ=θ1, and Rv ｔｈ１ ≦Rv<Rv ｔｈ２ If so, set θ=θ2 (<θ1), and Rv ｔｈ２ If Rv≦Rv, set θ=θ3 (<θ2). ｔｈ１ and Rv ｔｈ２ can be defined by the operator.

[0081] In this way, the processor 32 can automatically determine the angle θ based on the size (unit volume Vu) of the quantized minimum image unit IU (voxel 206). Therefore, the processor 32 functions as the angle determination unit 72 ( FIG. 2 ) that determines the angle θ. Note that the processor 32 can also automatically determine the angle θ for the drawing model 120D based on the unit volume Vu of the voxel 206 relative to the drawing model 120D.

[0082] The angle determination unit 72, together with the model placement unit 62, the quantization execution unit 64, the simulation unit 66, the image unit identification unit 68, the feature specification unit 70, and the position data generation unit 56, constitutes the device 60. With this configuration, the optimal angle θ for the simulation SL can be automatically determined, thereby improving the efficiency of step S4 in Fig. 11 and optimizing the number of times the loop of steps S3 to S5 is repeatedly executed.

[0083] In the above-described embodiment, the image value Vi is numerical information such as "0" or "1." However, the present invention is not limited to this, and the image value Vi may be information on brightness (such as luminance) in multiple levels (for example, 0 to 255 levels). Even in this case, the processor 32 monitors changes in the image value Vi in step S4 described above, and can identify the voxel 206A in accordance with the changes.

[0084] In the above-described embodiment, the processor 32 may accept an input IP3 for editing a designated feature model region FTe (e.g., the protrusion model 104D or the hole model 124D). For example, if the protrusion model 104D is designated as the feature model region FTe as a result of the flow of Fig. 11 , the operator operates the input device 40 to provide an input IP3 for excluding, from the feature model region FTe, part of the regions occupied by the outer circumferential surface model 108D and the end face model 110D that define the protrusion model 104D.

[0085] The processor 32 functions as the above-mentioned input accepting unit 54 to accept the input IP3 and edits the feature model region FTe so as to exclude from the feature model region FTe a portion of the outer circumferential surface model 108D and the end face model 110D specified by the input IP3. That is, in this case, the device 60 further includes an input accepting unit 54 that accepts the input IP3. With this configuration, the operator can freely edit the feature model region FTe, thereby enabling the operator to specify the feature model region FTe that is optimal for the workpiece model 100M or 120M.

[0086] 11 in accordance with a computer program PG3 pre-stored in the memory 34. The functions of the device 60 executed by the processor 32 may be functional modules realized by the computer program PG3.

[0087] Next, further functions of the robot system 10 will be described with reference to Figures 19 to 22. In this embodiment, the processor 32 of the control device 16 acquires the position and orientation of the workpiece 100 or 120, and executes a predetermined task on the workpiece 100 or 120. Below, with reference to Figure 20, a case will be described in which the processor 32 performs workpiece handling in which the workpiece 100, which is piled up in a container E, is removed from the container E and placed on a jig J in a predetermined orientation Ow.

[0088] When the processor 32 receives a work start command from an operator, a higher-level controller, or the computer program CP2, the processor 32 starts the flow of Fig. 20. Note that in this embodiment, the feature model area FTe (protrusion model 104D) is specified in advance in the drawing model 100D by the above-mentioned device 50 or 60, and the above-mentioned position data Pf is acquired in advance.

[0089] In step S11, the processor 32 captures an image of the workpiece 100. Specifically, the processor 32 activates the shape detection sensor 14, and causes the shape detection sensor 14 to capture an image of the workpieces 100 piled up in the container E. FIG. 21 shows an example of image data 210 of the workpieces 100 captured by the shape detection sensor 14. In this embodiment, the image data 210 is three-dimensional point cloud image data, and the image data 210 includes shape data SD representing the visual features (faces, edges, vertices, etc.) of a plurality of workpieces 100 in various orientations. １ , S.D. ２ , S.D. ３ and S.D. ４ is shown in the photo.

[0090] Shape data SD ｉ Each of the points (i=1, 2, 3, 4) is composed of a group of points distributed in the sensor coordinate system C3, and each point of the group of points is expressed as a coordinate in the sensor coordinate system C3. ｉ However, it should be understood that in reality, five or more workpieces may be captured. The processor 32 acquires image data 210 captured by the shape detection sensor 14.

[0091] In step S12, the processor 32 calculates the shape data SD of the workpiece 100 detected by the shape detection sensor 14. ｉ Specifically, the processor 32 obtains the position and orientation of the workpiece 100 by matching the workpiece model 100M with the drawing model 100D of the workpiece 100. Specifically, the processor 32 obtains a plurality of search models 100S, each of which represents a partial shape of the drawing model 100D as viewed from a plurality of viewpoints VP, based on the drawing model 100D of the workpiece 100. ｍ This search model 100S ｍ As described above, these constitute the workpiece model 100M.

[0092] 22 to 25 show the search model 100S. ｍ The search model 100S shown in FIGS. １ , 100S ２ , 100S ３ and 100S ４Each of the search models 100S is composed of a point cloud, and the processor 32 generates search models 100S in various poses by attaching the point clouds to the model components of the drawing model 100D viewed from different viewpoints VP. １ , 100S ２ , 100S ３ and 100S ４ Generate.

[0093] In this embodiment, the processor 32 generates a search model 100S. ｍ When generating (m=1, 2, 3, 4), a point cloud is generated for the front-side model components that can be seen from the viewpoint VP, but a point cloud is not generated for the back-side model components that cannot be seen from the viewpoint VP (for example, the edges and faces on the back side of the paper of the model components of drawing model 100D in Figure 3).

[0094] Generated search model 100S ｍ In this way, the processor 32 generates search models 100S of various orientations based on the drawing model 100D. ｍ Thus, processor 32 generates search model 100S ｍ The model generator 82 (FIG. 19) generates the model.

[0095] As described above, the position data Pf of the feature model region FTe is previously acquired in association with the drawing model 100D. The processor 32 references the position data Pf and generates the search model 100S. ｍ The processor 32 can identify the feature model area FTe in the work coordinate system C4 set as follows. ｍ The point group is extracted as the feature model region FTe'.

[0096] For example, the search model 100S in FIGS. １ , 100S ２ and 100S ３Since the feature model region FTe exists in the search model 100S shown in FIG. ４ Since the feature model region FTe does not exist (in other words, it is not visible) in the search model 100S, ４ Thus, the processor 32 determines whether the feature model region FTe′ exists in the search model 100S. ｍ and generate the search model 100S ｍ It can be recognized whether or not the object is in the feature model region FTe'.

[0097] The processor 32 then generates search models 100S for various poses. ｍ The shape data SD shown in the image data 210 captured in the immediately preceding step S11 ｉ The processor 32 matches the search model 100S with various poses. ｍ Shape data SD ｉ Each time the search model 100S is matched with the search model 100S, a score indicating the degree of match between the two is calculated, and the search model 100S is adjusted so that the score is greater than a predetermined threshold value. ｍ Shape data SD that matches ｉ Explore.

[0098] In FIG. 26, the shape data SD shown in the image data 210 in the sensor coordinate system C3 is ｉ Search Model 100S ｍ In the example shown in FIG. 26, the shape data SD １ Search Model 100S １ is matched, and the shape data SD ２ Search Model 100S ４ is matched, and the shape data SD ３ Search Model 100S ２ are matched.

[0099] The processor 32 generates the shape data SD as shown in FIG. １ Search model 100S that matches １ The coordinate Qs in the sensor coordinate system C3 of the workpiece coordinate system C4 set as １(x, y, z, w, p, r) is obtained. １ is the shape data SD １ represents the position and orientation of the workpiece 100 detected in the container E in the sensor coordinate system C3.

[0100] Coordinate Qs １ Of the coordinates (x, y, z, w, p, r), the coordinates (x, y, z) indicate the position of the workpiece 100 in the sensor coordinate system C3, and the coordinates (w, p, r) indicate the orientation of the workpiece 100 in the sensor coordinate system C3 (so-called yaw, pitch, roll). Similarly, the processor 32 calculates the shape data SD ２ Search model 100S that matches ４ Coordinate Qs in the sensor coordinate system C3 of the workpiece coordinate system C4 ２ and shape data SD ３ Search model 100S that matches ２ Coordinate Qs in the sensor coordinate system C3 of the workpiece coordinate system C4 ３ Get.

[0101] Then, the processor 32 calculates the three detected coordinates Qs ｎ (n=1, 2, 3) detection result data 212 is created. An example of the data structure of the detection result data 212 is shown in FIG. 27. In FIG. 27, a column 214 contains the detected coordinates Qs ｎ The column 216 shows the coordinate Qs of the identification number "n". ｎ (x, y, z, w, p, r). The processor 32 detects the coordinates Qs ｎ The identification number "n" may be assigned in the order of the z coordinate value (i.e., the height of the detected workpiece 100).

[0102] On the other hand, column 218 shows the search model 100S. ｍ The matching shape data SD ｉ The flag FL indicates whether the feature FT is included in the search model 100S. The flag FL will be described later. In this way, the processor 32 ｍ Shape data SD ｉ By matching the position and posture of the workpiece 100 (coordinates Qs ｎTherefore, the processor 32 functions as a position acquisition unit 84 (FIG. 19) that acquires the position and orientation of the workpiece 100.

[0103] In step S13, the processor 32 executes a feature determination process. This step S13 will be described with reference to Fig. 28. In step S21, the processor 32 executes a feature determination process using the coordinate Qs stored in the detection result data 212 created in the immediately preceding step S12. ｎ The identification number "n" of the device is set to "1" (n=1).

[0104] In step S22, the processor 32 calculates the shape data SD shown in the image data 210 (FIG. 21) captured in the most recent step S11. ｉ , the search model 100S used for matching in the immediately preceding step S12 ｍ Specifically, the processor 32 matches the coordinate Qs identified by the identification number "n" set at this point in the sensor coordinate system C3 that defines the image data 210. ｎ Search Model 100S ｍ Place the

[0105] For example, assume that the identification number "n" is set to 1 at this point. In this case, the processor 32 calculates the coordinate Qs of the sensor coordinate system C3 that defines the image data 210. １ , the coordinate Qs １ When the shape data SD is acquired, １ Search model 100S that matches １ is simulated (i.e., the search model 100S is placed in the sensor coordinate system C3). ｍ (The point cloud is projected onto the sensor coordinate system C3.) This state is shown in FIG. 29. As shown in FIG. 29, the search model 100S is placed (or projected) onto the sensor coordinate system C3. １ As in FIG. 26, the shape data SD １ This search model 100S will match. １ A feature model region FTe' is specified in .

[0106] On the other hand, assume that the identification number "n" is set to n=2 at this point. In this case, the processor 32 calculates the coordinate Qs of the sensor coordinate system C3 that defines the image data 210. ２ , the coordinate Qs ２ When the shape data SD is acquired, ２ Search model 100S that matches ４ (Fig. 26) is placed in a simulated manner. This state is shown in Fig. 30. As shown in Fig. 30, the search model 100S placed in the sensor coordinate system C3 is ４ As in FIG. 26, the shape data SD ２ This search model 100S will match. ４ There is no feature model region FTe′ in the region.

[0107] Referring again to FIG. 28, in step S23, processor 32 selects search model 100S matched in the immediately preceding step S22. ｍ For example, in step S22, a search model 100S is created as shown in FIG. １ is placed, the feature model area FTe' is specified, so the processor 32 determines YES and proceeds to step S24. ４ is placed, the feature model area FTe′ does not exist, so the processor 32 determines NO and proceeds to step S26.

[0108] In step S24, the processor 32 functions as the quantization execution unit 64 to perform the quantization of the search model 100S in the previous step S22. ｍ The matched image data 210 is quantized into the minimum image unit IU. Specifically, the processor 32 quantizes the image data 210 into voxels 206 based on the sensor coordinate system C3 in accordance with the image quantization program PG1.

[0109] As a result, the image data 210 is quantized into a plurality of voxels 206 arranged in a grid pattern in the x-axis, y-axis, and z-axis directions of the sensor coordinate system C3, and each voxel 206 is expressed as coordinates (x, y, z) in the sensor coordinate system C3. Note that the processor 32 may set the above-mentioned reference coordinate system C5 at an arbitrary position in the sensor coordinate system C3 and quantize the image data 210 based on the reference coordinate system C5.

[0110] In step S25, the processor 32 selects the search model 100S that was matched in the most recent step S22. ｍ The shape data SD overlapping with the feature model area FTe' specified in ｉ Based on the data amount δ of the shape data SD ｉ Specifically, the processor 32 determines whether the matched search model 100S is included in the image data 210 quantized in the immediately preceding step S24. ｍ The feature model area FTe' and the shape data SD ｉ 29. When the image data 210 shown in FIG. 29 is quantized in step S24, the voxel 206 corresponding to the region XXX in FIG. 29 is shown in FIG.

[0111] In FIG. 31, a group of voxels 206 shown in white indicates a cavity region, while a group of voxels 206B shown in a shaded region indicates a cavity region. １ A group of voxels 206C shown in a dotted area indicates voxels 206 that only show points in the feature model area FTe'. On the other hand, a group of voxels 206D shown in gray indicates voxels 206 that only show points in the shape data SD １ In this voxel 206D, the shape data SD １ The points of the feature model region FTe′ overlap with the points of the feature model region FTe′.

[0112] The image value Vi of the voxel 206 shown in white is "0". The image value Vi of the voxel 206B shown in the hatched area and the image value Vi of the voxel 206B shown in the dotted area are "1". The image value Vi of the voxel 206D shown in gray is "0". １ The sum of the image value Vi of the point "1" and the image value Vi of the point of the feature model region FTe' is "2".

[0113] The processor 32 calculates the image value Vi of each voxel 206 of the quantized image data 210, and calculates the number N1 of voxels 206D where the image value Vi is "2." This number N1 is calculated based on the shape data SD as shown in FIG. １ Search Model 100S １ When matching is performed, the shape data SD overlapping with the feature model area FTe′ is １ In this way, the processor 32 calculates the data amount δ by dividing the feature model region FTe′ and the shape data SD in the quantized image data 210. １ Therefore, the processor 32 functions as the image-based calculation unit 86 (FIG. 19) that calculates the number N1.

[0114] Next, the processor 32 calculates a ratio R1=N1 / N2 between the calculated number N1 and the number N2 of the smallest image units IU (voxels 206) in the quantized image data 210 that correspond to the feature model region FTe'. Specifically, the processor 32 identifies the voxels 206 (i.e., equivalent to the sum of voxels 206C and 206D) that represent the points in the feature model region FTe' in the image data 210 that has been quantized into voxels 206, and calculates the number N2 of these voxels. The processor 32 then calculates the ratio R1=N1 / N2. In this manner, in this embodiment, the processor 32 constitutes a ratio calculation unit 87 (FIG. 19) that calculates the ratio R1. The processor 32 uses the shape data SD as the number N2. １ The total number of voxels 206 that represent the point cloud may be calculated.

[0115] Then, the processor 32 determines whether the calculated ratio R1 is equal to or smaller than a predetermined threshold value Rth １ or more (R1≧Rth１ ), and in step S22, the search model 100S １ The matching shape data SD １ It is determined that the feature FT is included in R1 (i.e., YES). １ In this case, the shape data SD overlapping the feature model area FTe′ in FIG. １ The amount of data δ (in this embodiment, the number N1) is large, so the shape data SD １ includes data corresponding to the feature FT (in this case, the protrusion 104, the outer peripheral surface 108, or the end surface 110) (in other words, the shape data SD １ This means that most of the feature FT is visible.

[0116] Therefore, in this case, in step S12, the shape data SD in which the feature FT exists is １ Search Model 100S １ This means that the shape data SD including the feature FT is matched. １ Search model 100S that matches １ Coordinates Qs obtained from １ (w, p, r) is the shape data SD １ It can be considered that the posture of the workpiece 100 detected as above is accurately represented.

[0117] Conversely, R1<Rth １ If so, the shape data SD overlapping the feature model area FTe′ １ The amount of data δ is small, so the shape data SD １ does not contain data corresponding to the feature FT (in other words, the shape data SD １ For example, when the shape detection sensor 14 captures an image of the workpiece 100 in step S11, the shape data SD is not visible due to the reflected light from the workpiece 100, the scattered light around it, or halation. １ In this case, the amount of data δ is reduced, and as a result, R1<Rth １ In such a case, the search model 100S matched in step S12 １Coordinates Qs obtained from １ (w, p, r) may not accurately represent the orientation of the workpiece 100.

[0118] If the processor 32 determines YES in step S25, the process proceeds to step S27, whereas if the processor 32 determines NO, the process proceeds to step S26. １ It functions as a feature determining unit 88 (FIG. 19) that determines whether or not a feature FT is included in the

[0119] In step S26, the processor 32 calculates the position and orientation (coordinates Qs ｎ ) for the shape data SD ｉ Specifically, the processor 32 sets a flag FL1 indicating that the feature FT is not included in the coordinate Qs of the identification number "n" set at this point in the detection result data 212 (FIG. 27) created in the most recent step S12. ｎ In response, the flag FL in the column 218 is set to "FL1."

[0120] If n=2 is set at this point, the processor 32 calculates the coordinate Qs of the identification number "2". ２ As described above, the flag FL of the shape data SD shown in FIG. ２ Search model 100S that matches ４ In this case, the processor 32 determines "NO" in step S23 and executes step S26. ４ Coordinates Qs obtained from ２ The flag FL is set to "FL1" as shown in FIG.

[0121] In step S27, the processor 32 calculates the position and orientation (coordinates Qs ｎ ) for the shape data SD ｉIf n=1 is set at this point, the processor 32 sets a flag FL2 indicating that the feature FT is included in the coordinate Qs of the identification number "1" in the detection result data 212. １ The flag FL of the shape data SD is set to "FL2". １ Search model 100S that matches １ If a feature model region FTe′ exists in the search model 100S (that is, if the result of step S23 is YES), and if the result of step S25 is YES, the search model 100S １ Coordinates Qs obtained from １ The flag FL is set to "FL2" as shown in FIG.

[0122] In step S28, the processor 32 checks whether the identification number "n" set at this point is the same as the coordinate Qs detected in step S12. ｎ Number of: n ＭＡＸ If the processor 32 determines YES, it ends step S13 shown in Fig. 28 and proceeds to step S14 in Fig. 20. On the other hand, if the processor 32 determines NO, it proceeds to step S29.

[0123] In step S29, the processor 32 increments the identification number "n" by "1" (n=n+1). After that, the processor 32 returns to step S22. In this way, the processor 32 repeatedly executes the loop of steps S21 to S29 until the determination in step S28 is YES, and the plurality of coordinates Qs stored in the detection result data 212 shown in FIG. ｎ The flag FL is set to "FL1" or "FL2".

[0124] Referring again to FIG. 20, in step S14, the processor 32 executes the task. This step S14 will be described with reference to FIG. 32. In step S31, the processor 32 sets the identification number "n" to "1" (n=1), similarly to step S21 described above. In step S32, the processor 32 calculates the coordinate Qs of the identification number "n" set at this point. ｎSearch model 100S to obtain ｍ The matching shape data SD ｉ It is determined whether the feature FT is included in the

[0125] Specifically, the processor 32 refers to the detection result data 212 created in the most recent step S11, and determines whether the flag FL for the identification number “n” that is set at this time is set to “FL1” or “FL2.” If “FL1” is set, the processor 32 determines NO and proceeds to step S33, whereas if “FL2” is set, the processor 32 determines YES and proceeds to step S33.

[0126] In step S33, the processor 32 grips the workpiece 100 and moves it to a predetermined position Pw. Assume that the identification number "n" is set to "2" at this point. In this case, the processor 32 obtains the coordinate Qs of the identification number "2" from the detection result data 212. ２ and obtain the coordinate Qs ２ is expressed as the coordinate Qr of the robot coordinate system C1. ２ Convert to.

[0127] The processor 32 then calculates the coordinates Qr ２ Then, the processor 32 operates the end effector 28 to move the workpiece 100 (specifically, the shape data SD in FIG. 21 ) with the end effector 28. ２ The robot 12 grasps the workpiece 100 (detected as a workpiece 100), removes it from the container E, and positions it at a position Pw defined in the robot coordinate system C1. This position Pw is determined in advance as a position where the feature FT (protrusion 104) of the workpiece 100 grasped by the robot 12 falls within the field of view of the shape detection sensor 14.

[0128] In step S34, the processor 32 activates the shape detection sensor 14 to again capture an image of the workpiece 100 gripped by the robot 12. The processor 32 acquires the image data ID (three-dimensional point cloud image data) captured by the shape detection sensor 14. The image data ID will include the feature FT (protrusion 104) of the workpiece 100.

[0129] In step S35, the processor 32 acquires the posture of the workpiece 100 gripped by the robot 12 based on the image data ID acquired in the immediately preceding step S34. Specifically, similar to step S12 described above, the processor 32 calculates a search model 100S based on the shape data SD of the workpiece 100 shown in the image data ID. ｍ By matching these, the coordinates Qs' of the position and orientation of the workpiece 100 (workpiece coordinate system C4) in the sensor coordinate system C3 are obtained.

[0130] Since the image data ID includes the feature FT (protrusion 104), the coordinate Qs' accurately represents the posture of the workpiece 100. Then, the processor 32 uses the acquired coordinate Qs' and the known positional relationship between the sensor coordinate system C3 and the tool coordinate system C2 to acquire the coordinate Qt(w, p, r) that indicates the posture of the workpiece 100 (workpiece coordinate system C4) in the tool coordinate system C2.

[0131] In step S36, the processor 32 places the workpiece 100 on the jig J in a predetermined orientation Ow. If step S36 is executed after step S35, the processor 32 places the workpiece 100 to be gripped by the robot 12 on the jig J in the orientation Ow based on the coordinate Qt acquired in the immediately preceding step S35. As described above, the coordinate Qt allows the processor 32 to recognize the accurate orientation of the workpiece 100 gripped by the robot 12 in the tool coordinate system C2. Therefore, the processor 32 can accurately place the workpiece 100 on the jig J in the orientation Ow.

[0132] On the other hand, when step S36 is executed after determining YES in step S32, the processor 32 determines the coordinate Qs of the identification number "n" set at this time. ｎBased on this, the workpiece 100 is taken out of the container E and placed on the jig J. Assume that the identification number "n" is set to "1" at this point. In this case, the processor 32 obtains the coordinates Qs of the identification number "1" from the detection result data 212. １ and obtain the coordinate Qs １ is expressed as the coordinate Qr of the robot coordinate system C1. １ Convert to.

[0133] The processor 32 then calculates the coordinates Qr １ 21. The workpiece 100 (specifically, the shape data SD in FIG. 21) is positioned at a position and posture corresponding to the end effector 28. １ The workpiece 100 detected as the coordinate Qr is grasped. １ is the shape data SD in which the feature FT is captured. １ Search Model 100S １ Since the coordinates are obtained by matching the workpiece coordinate system C4, the accurate orientation of the workpiece 100 (workpiece coordinate system C4) held by the robot 12 in the tool coordinate system C2 at this time is known. Therefore, the processor 32 can accurately place the held workpiece 100 on the jig J in the predetermined orientation Ow.

[0134] In step S37, the processor 32 determines whether the identification number "n" is equal to the number: n ＭＡＸ If the processor 32 determines YES, it ends step S14 shown in FIG. 32 and proceeds to step S15 in FIG. 20, whereas if the processor 32 determines NO, it proceeds to step S38. In step S38, the processor 32 increments the identification number "n" by "1" (n=n+1). Thereafter, the processor 32 returns to step S32.

[0135] 20 again, in step S15, the processor 32 determines whether or not work has been completed on all of the workpieces 100 in the container E. If the processor 32 determines YES, it ends the flow of FIG. 20 , whereas if the processor 32 determines NO, it returns to step S11. In this way, the processor 32 performs workpiece handling, which involves removing the workpieces 100 piled up in the container E and placing them on the jig J, based on the image data 210 captured by the shape detection sensor 14.

[0136] 6 to 8, the processor 32 can also handle the workpiece 120 according to the flow in Fig. 20. Below, differences between the flow in Fig. 20 executed for work on the workpiece 120 and the work flow for the workpiece 100 described above will be mainly explained. After acquiring image data 210' (not shown) of the workpiece 120 in the container E in step S11, the processor 32 uses the search model 120S in step S12 to acquire the position and orientation of the workpiece 120. ｍ 33 to 35 show the search model 120S. ｍ An example where m=1, 2, 3 is shown below.

[0137] The processor 32 generates a search model 120S based on the previously acquired position data Pf. ｍ In this case, the feature model area FTe corresponds to the hole model 124D (i.e., the cavity area). The processor 32 also searches for a search model 120S that overlaps with the identified feature model area FTe. ｍ The point cloud may be extracted as the feature model region FTe'. In this case, the feature model region FTe' corresponds to the region occupied by the inner peripheral surface model 126D and the edge models 130D and 132D (that is, the solid region).

[0138] Search model 120S of FIGS. 33 and 34 １ and 120S ２ On the other hand, the feature model area FTe or FTe′ exists in the search model 120S shown in FIG. ３The processor 32 generates search models 120S for various poses. ｍ The shape data SD shown in the image data 210′ ​​captured in step S11 ｉ By matching with the coordinate Qs ｎ and creates the detection result data 212 similar to that shown in FIG.

[0139] In step S23 of FIG. 28, the processor 32 selects the search model 120S that matched the image data 210′ ​​in the immediately preceding step S22. ｍ For example, in step S22, the search model 120S shown in FIG. 33 or 34 is determined. １ or 120S ２ is placed, the feature model area FTe or FTe' is specified, so the processor 32 determines YES and proceeds to step S24. ３ is placed, the feature model area FTe or FTe′ does not exist, so the processor 32 determines NO and proceeds to step S26.

[0140] In step S25, the processor 32 functions as the feature determining unit 88 and determines the shape data SD based on the data amount δ. ｉ Specifically, the processor 32 determines whether the matched search model 120S is included in the image data 210′ ​​quantized in the immediately preceding step S24. ｍ The feature model area FTe or FTe' specified in the ｉ The number N of voxels where the two overlap is calculated.

[0141] 36 shows voxels 206 obtained by quantizing the image data 210'. In FIG. 36, a group of voxels 206 shown in white are voxels corresponding only to the feature model region FTe (i.e., corresponding to the cavity region of the hole model 124D). A group of voxels 206E shown in gray are voxels corresponding only to the shape data SD ｉ and the Search Model 120S ｍAmong these, the voxels 206 are those that map points representing the main body model 122D. Furthermore, a group of voxels 206C indicated by a dotted region indicates voxels 206 that map only points in the feature model region FTe′ (i.e., corresponding to the region occupied by the inner periphery model 126D, the edge model 130D, or 132D).

[0142] On the other hand, a group of voxels 206F indicated by the hatched area is a feature model area FTe (hole model 124D) and shape data SD ｉ In this voxel 206F, the shape data SD ｉ The points overlap with the feature model region FTe as a cavity region. Also, a group of voxels 206D shown in dark gray are the shape data SD ｉ and a point in the feature model region FTe′ (the inner peripheral surface model 126D, the edge model 130D or 132D). ｉ The points of the feature model region FTe′ overlap with the points of the feature model region FTe′.

[0143] The image value Vi of the voxel 206 shown in white is "0." The image value Vi of the voxel 206F shown in the hatched area and the image value Vi of the voxel 206C shown in the dotted area are "1." The image value Vi of the voxel 206E shown in gray and the image value Vi of the voxel 206D shown in dark gray are "2."

[0144] As an example, the processor 32 functions as the image-based calculation unit 86 to calculate the feature model region FTe and the shape data SD in the quantized image data 210′. ｉ The number N3 of voxels 206F that overlap with the shape data SD is calculated. ｉ Search Model 120S ｍ When matching is performed, the shape data SD overlapping with the feature model region FTe, which is a cavity region, ｉNext, the processor 32 functions as a ratio calculation unit 87, and calculates a ratio R2 (=N3 / N4) between the calculated number N3 and the number N4 of voxels 206 (corresponding to the sum of the white voxels 206 and the hatched voxels 206F in FIG. 36 ) that correspond to the feature model region FTe in the quantized image data 210′.

[0145] Then, the processor 32 determines whether the calculated ratio R2 is equal to or smaller than a predetermined threshold value Rth ２ If R2≦Rth ２ ), and in step S22, the search model 120S ｍ The matching shape data SD ｉ Here, the larger the value of the ratio R2 (i.e., the number N3), the more the feature model region FTe, which is a hollow region, contains the shape data SD representing a solid region. ｉ This means that there are more overlapping points. ｍ The feature model area FTe specified in the shape data SD ｉ This means that the

[0146] Conversely, R2≦Rth ２ When the ratio R2 is small, the feature model area FTe is ｉ Therefore, the shape data SD ｉ The feature FT (in this case, the hole 124) is included in the shape data SD ｉ The processor 32 functions as the characteristic determining unit 88 and determines whether R2≦Rth ２ If R2>Rth, the determination is YES and the process proceeds to step S27. ２ If so, the determination is NO and the process proceeds to step S26. ｉ The total number of voxels 206 that represent the point cloud may be calculated.

[0147] As another example, the processor 32 may function as the image-based calculation unit 86 to calculate the shape data SD in the quantized image data 210′. ｉThe number N1 of voxels 206D where points of the feature model area FTe′ (inner surface model 126D, edge model 130D or 132D) overlap is calculated. This number N1 is calculated based on the shape data SD ｉ Search Model 120S ｍ When matching is performed, the shape data SD overlapping with the feature model area FTe′ is ｉ corresponds to the amount of data δ.

[0148] The processor 32 then functions as a ratio calculation unit 87, and calculates a ratio R1 (=N1 / N2) between the calculated number N1 and the number N2 (corresponding to the sum of the voxels 206C and 206D) of the voxels 206 corresponding to the feature model region FTe' in the quantized image data 210'. The processor 32 then functions as a feature determination unit 88, and calculates a ratio R1 (=N1 / N2) between the calculated number N1 and the number N2 of the voxels 206 corresponding to the feature model region FTe' in the quantized image data 210'. １ If R1<Rth, the determination is YES and the process proceeds to step S27. １ 28 for the workpiece 120, and executes step S14 according to the flag FL set in step S26 or S27. ｉ The total number of voxels 206 that represent the point cloud may be calculated.

[0149] As described above, in this embodiment, the processor 32 functions as the quantization execution unit 64, model generation unit 82, position acquisition unit 84, image-based calculation unit 86, ratio calculation unit 87, and feature determination unit 88 to acquire the position and orientation (coordinates Qs, Qr) of the workpiece 100 or 120 and perform work based on this position and orientation. Therefore, the model generation unit 82, position acquisition unit 84, image-based calculation unit 86, ratio calculation unit 87, and feature determination unit 88 constitute a device 80 ( FIG. 19 ) that acquires the position and orientation of the workpiece 100 or 120.

[0150] In this device 80, the position acquisition unit 84 acquires the shape data SD of the workpieces 100 and 120M (search models 100S and 120S) detected by the shape detection sensor 14, based on the feature model areas FTe and FTe′ of the workpieces 100M and 120M (search models 100S and 120S).ｉ By matching with the above, the positions and orientations (coordinates Qs, Qr) of the workpieces 100, 120 are obtained (step S12).

[0151] Then, the feature determination unit 88 calculates the shape data SD ｉ The shape data SD overlapping with the feature model areas FTe and FTe' designated in the workpiece models 100M and 120M matched to ｉ Based on the data amount δ of the shape data SD ｉ As described above, when the position and orientation of the workpiece 100 or 120 are acquired, it is determined whether the feature FT is included in the shape data SD ｉ If the feature FT (for example, the protrusion 104 or the hole 124) is not included in the workpiece 100 or 120, the orientation of the workpiece 100 or 120 cannot be detected accurately.

[0152] According to the device 80, the shape data SD ｉ Since it is possible to automatically determine whether the feature FT is included in the robot 12, it is possible to recognize the possibility that an inaccurate posture will be detected. As a result, it is possible to cause the robot 12 to perform different actions depending on the determination result of the presence or absence of the feature FT, such as a first action of NO in step S32 and steps S33 to S36 in FIG. 32, and a second action of YES in step S32 and step S36.

[0153] 32, when the processor 32 determines NO in step S32, it may generate an alarm signal instead of steps S33 to S35. Then, the processor 32 may end the work without executing step S36. Even in this case, the operator can recognize that an inaccurate posture may have been detected, and can take measures such as changing the design.

[0154] In the device 80, the shape detection sensor 14 receives the shape data SD ｉ The position acquisition unit 84 has a visual sensor that captures image data 210, 210' in which the workpiece models 100M, 120M are captured as shape data SD in the image data 210, 210'. ｉThe quantization execution unit 64 quantizes the image data 210, 210' into minimum image units IU (voxels 206) (step S24).

[0155] The image unit calculation unit 86 calculates the data amount δ by dividing the feature model regions FTe and FTe′ in the quantized image data 210 and 210′ ​​into the shape data SD ｉ The number N1, N3 of minimum image units IU that overlap with the feature FT is calculated. The feature determination unit 88 then performs determination based on the number N1, N3. This configuration allows the determination of the presence or absence of the feature FT to be performed quickly by calculating the number N1 or N3 of minimum image units IU.

[0156] Furthermore, in the device 80, a ratio calculation unit 87 calculates ratios R1 and R2 between the numbers N1 and N3 calculated by the image unit calculation unit 86 and the numbers N2 and N4 of minimum image units IU corresponding to the feature model regions FTe and FTe' in the quantized image data 210 and 210'. The feature determination unit 88 then makes a determination based on the ratios R1 and R2. This configuration enables the determination of the presence or absence of a feature FT to be made quickly, with higher accuracy, and using a relatively simple algorithm.

[0157] The processor 32 may make the determination in step S25 without calculating the ratios R1 and R2. For example, the processor 32 may make a determination of YES in step S25 when the number N1 is equal to or greater than a predetermined threshold (or the number N3 is equal to or less than a predetermined threshold). In other words, in this case, the ratio calculation unit 87 can be omitted from the device 80. The determination method in step S25 described above is an example, and the processor 32 may use any other method to calculate the shape data SD based on the data amount δ. ｉ It may be determined whether the feature FT is included in the

[0158] In step S12, the processor 32 detects the coordinate Qs of the sensor coordinate system C3. ｎ is the coordinate Qr of the robot coordinate system C1 ｎ and convert the coordinate Qr ｎ Furthermore, steps S22 to S25 may be performed using the coordinates Qr in the robot coordinate system C1.ｎ The processor 32 may use the shape data SD shown in the image data 210 or 210′. ｉ The coordinates of the sensor coordinate system C1 of the point cloud are converted into the robot coordinate system C1, and the shape data SD ｉ can be expressed in the robot coordinate system C1.

[0159] Then, in step S22, the shape data SD ｉ Search Model 100S ｍ or 120S ｍ and quantize the image data 210 or 210′ ​​into minimum image units IU (voxels 206) based on the robot coordinate system C1 in step S24. In this way, the processor 32 can execute steps S22 to S25 based on the robot coordinate system C1.

[0160] In the above embodiment, the case where step S12 is executed to generate the detection result data 212 shown in Fig. 27 and then step S13 shown in Fig. 28 is executed is described. However, this is not limiting, and steps S12 and S13 may be combined to obtain the coordinate Qs in step S12. ｎ It is also possible to execute steps S23 to S27 in FIG. 28 each time the value of the parameter is acquired.

[0161] Specifically, in step S12, the processor 32 calculates the shape data SD shown in the image data 210 or 210′. ｉ Search Model 100S ｍ or 120S ｍ Matching coordinates Qs ｎ When the coordinate Qs is acquired, steps S23 to S27 are executed. ｎ To obtain the shape data SD ｉ Search model 100S that matches ｍ or 120S ｍ Steps S23 to S27 are executed based on the above.

[0162] At this time, the processor 32 calculates the acquired coordinate Qs ｎ is stored in the detection result data 212, and the coordinate Qs ｎIn response to this, the flag FL in the column 218 is set to "FL1" or "FL2" in step S26 or S27. As a result, in step S12, the detection result data 212 in which the flag FL is set can be created as shown in FIG.

[0163] In step S24, the processor 32 may quantize the minimum image unit IU into a plurality of pixels aligned in a grid pattern along the x-axis and y-axis directions of the sensor coordinate system C3, instead of the voxel 206. Then, the processor 32 may execute step S25 based on the image value Vi′ of the pixel.

[0164] 28. In step S25, the processor 32 extracts the shape data SD that overlaps with the feature model region FTe or FTe′. ｉ As the data amount δ, the shape data SD having the same coordinates (or within a predetermined distance) as the coordinates of the points of the feature model area FTe or FTe′ in the sensor coordinate system C3 ｉ The number N5 of points in the image may be acquired. The processor 32 may then determine YES if the number N5 is equal to or greater than a predetermined threshold value. In other words, in this case, the quantization execution unit 64, the image-based calculation unit 86, and the ratio calculation unit 87 can be omitted from the device 80.

[0165] Also, in step S12 described above, the search model 100S ｍ or 120S ｍ Instead of the drawing model 100D or 120D, or any other type of workpiece model 100M or 120M, the shape data SD ｉ In this case, the model generating unit 82 can be omitted from the device 80.

[0166] Next, further functions of the robot system 10 will be described with reference to Figures 37 to 39. In this embodiment, the processor 32 of the control device 16, similar to the above-described embodiment, acquires the position and orientation of the workpiece 100 or 120 and performs workpiece handling on the workpiece 100 or 120. In this embodiment, the processor 32 performs work on the workpiece 100 or 120 by executing the flow shown in Figure 20.

[0167] Specifically, after step S11, in step S12, the processor 32 functions as the model generation unit 82 (FIG. 37) to generate a search model 100S. ｍ or 120S ｍ The processor 32 also functions as a feature specifying unit 70 (FIG. 37) and generates the plurality of search models 100S. ｍ or 120S ｍ Among them, search model 100S including feature FT ｍ or 120S ｍ For example, the processor 32 may designate a feature model area FTe or FTe' corresponding to the feature FT based on the position data Pf generated in advance. １ ~100S ３ In the above, the feature model region FTe' (the protrusion model 104D, the outer peripheral surface model 108D, or the end surface model 110D) is designated.

[0168] Alternatively, the processor 32 may generate a search model 120S shown in FIGS. 33 and 34 based on the previously generated position data Pf. １ and 120S ２ 37, the processor 32 specifies the feature model region FTe (hole model 124D) or the feature model region FTe' (inner peripheral surface model 126D, edge models 130D and 132D). Then, the processor 32 functions as the position acquisition unit 84 (FIG. 37) and ｍ or 120S ｍ The shape data SD shown in the image data 210 or 210′ ​​captured in step S11 ｉto obtain the position and orientation (coordinates Qs) of the workpiece 100 or 120. Then, the processor 32 creates detection result data 212 as shown in FIG.

[0169] In this embodiment, the processor 32 executes the flow shown in Fig. 38 as step S13. In the flow shown in Fig. 38, the same processes as those in Fig. 28 are assigned the same step numbers, and duplicated explanations will be omitted. After starting step S13, the processor 32 executes the above-mentioned step S21.

[0170] In step S41, the processor 32 functions as the feature determining unit 88 (FIG. 37) to obtain the shape data SD ｉ Specifically, the processor 32 determines whether the feature FT is included in the shape data SD captured in the image data 210 or 210′ ​​in the most recent step S12. ｉ Search model 100S that matches ｍ or 120S ｍ 27 is created in the most recent step S12, and the identification number "n" is set to "2" at this point.

[0171] In this case, the processor 32 calculates the shape data SD shown in the image data 210 in the most recent step S12. ２ (Fig. 26) Search Model 100S ２ This search model 100S ２ Therefore, the processor 32 does not include the feature model region FTe′ in the shape data SD ｉ does not include the feature FT (i.e., NO), and the process proceeds to step S27.

[0172] On the other hand, in the most recent step S12, the search model 100S １ , 100S ２ or 100S ３ are matched, a feature model area FTe' is specified for them. In this case, the processor 32 ｉThe processor 32 then determines that the feature FT is included in the image (i.e., YES), and proceeds to step S26. Thereafter, the processor 32 sequentially executes steps S26 to S19 and steps S14 and S15 in FIG. 20, similar to the above-described embodiment.

[0173] As described above, in this embodiment, the processor 32 functions as the model generation unit 82, feature designation unit 70, position acquisition unit 84, and feature determination unit 88 to acquire the position and orientation (coordinates Qs, Qr) of the workpiece 100 or 120 and perform work based on the position and orientation. Therefore, the model generation unit 82, feature designation unit 70, position acquisition unit 84, and feature determination unit 88 constitute a device 90 ( FIG. 37 ) that acquires the position and orientation of the workpiece 100 or 120.

[0174] In this device 90, the feature designation unit 70 selects a plurality of search models 100S generated by the model generation unit 82. ｍ or 120S ｍ Among them, search model 100S including feature FT １ , 100S ２ or 100S ３ , or 120S １ or 120S ２ In the step , the feature model region FTe or FTe' corresponding to the feature FT is designated.

[0175] Then, when the position acquisition unit 84 acquires the position and orientation, the feature determination unit 88 acquires the shape data SD ｉ Search model 100S that matches ｍ or 120S ｍ If the specified feature model area FTe or FTe' exists in the ｉ According to this configuration, it is determined that the search model 100S used for matching to acquire the position and orientation of the workpiece 100 or 120 includes the feature FT (step S41). ｍ or 120S ｍ Therefore, the presence or absence of the feature FT can be determined quickly. Furthermore, the amount of calculation processing can be reduced compared to the flow of Fig. 28, thereby achieving a reduction in cycle time.

[0176] The device 90 may further include the functions of the device 50 described above (the image generation unit 52, the input reception unit 54, and the position data generation unit 56). For example, the processor 32 functions as the model generation unit 82 to generate the search model 100S. ｍ or 120S ｍ When the search model 100S is generated, the image generating unit 52 functions as the image generating unit 52. ｍ or 120S ｍ The image data 200' representing the above is generated and displayed on the display device 38.

[0177] In this image data 200′, the processor 32 may display the above-mentioned designation box image 202 or cursor image 204. Next, the processor 32 functions as the input receiving unit 54, and receives, via the input device 40, the search model 100S displayed in the image data 200′. ｍ or 120S ｍ In step 100, an input IP1 or IP2 specifying the feature model region FTe or FTe' is received.

[0178] For example, the operator operates the input device 40 to select the search model 100S in the designation box image 202. ｍ or 120S ｍ An input IP1 surrounding the point cloud or an input IP2 for selecting the point cloud with a cursor image 204 (or specifying an area for selecting the point cloud by a drag-and-drop operation) may be provided to the processor 32.

[0179] Next, the processor 32 functions as the position data generator 56 to generate a search model S ｍ or 120S ｍ In this way, the device 50 generates position data Pf (coordinates in the workpiece coordinate system C4) indicating the position in the search model S ｍ or 120S ｍ 38 based on the feature model region FTe or FTe′ designated by the function of the device 50.

[0180] 20 in accordance with a computer program PG4 pre-stored in the memory 34. The functions of the device 80 or 90 executed by the processor 32 may be functional modules realized by the computer program PG4.

[0181] The shape detection sensor 14 is not limited to a visual sensor, and may be any type of sensor capable of detecting the shapes of the workpieces 100, 120, such as a laser scanner. The workpieces 100 and 120 are merely examples, and may be any shape of workpieces having features FT that serve as a reference for posture. For example, the workpiece 100 may have multiple protrusions 104, or the workpiece 120 may have any number of holes 124.

[0182] Furthermore, the shape detection sensor 14 is not limited to being fixed at a fixed point, but may be attached to the robot 12 and positioned at the imaging position Pi by the robot 12. In this case, a second shape detection sensor 14' may be fixed at a fixed point in the work cell separately from the shape detection sensor 14, and in the above-mentioned step S34, the processor 32 may cause the second shape detection sensor 14' to capture an image of the workpiece 100 or 120 gripped by the robot 12.

[0183] Alternatively, all of the functions of the devices 50, 60, 80, and 90 may be implemented in the control device 16. In this case, the processor 32 has the functions of an image generation unit 52, an input reception unit 54, a position data generation unit 56, a model placement unit 62, a quantization execution unit 64, a simulation unit 66, an image unit identification unit 68, a feature designation unit 70, a model generation unit 82, a position acquisition unit 84, an image unit calculation unit 86, a ratio calculation unit 87, and a feature determination unit 88.

[0184] In the above-described embodiment, the functions of the devices 50, 60, 80, and 90 are implemented in the control device 16. However, at least some of the functions of the devices 50, 60, 80, or 90 may be implemented in any type of computer separate from the control device 16, such as a teaching device that teaches the robot 12 how to operate, or a PC for the shape detection sensor 14. The robot 12 is not limited to a vertical articulated robot, but may be any type of robot, such as a horizontal articulated robot or a parallel link robot. The robot 12 may be configured to perform any task, such as welding or laser processing, in addition to the above-described workpiece handling.

[0185] Although the present disclosure has been described in detail above, the present disclosure is not limited to the individual embodiments described above. Various additions, substitutions, modifications, partial deletions, etc. are possible in these embodiments without departing from the gist of the present disclosure or the spirit of the present disclosure derived from the content of the claims and their equivalents. These embodiments can also be implemented in combination. For example, in the above-described embodiments, the order of each operation and the order of each process are shown as examples and are not limited to these. The same applies when numerical values ​​or mathematical expressions are used in the description of the above-described embodiments.

[0186] The present disclosure describes the following aspects: (Aspect 1) An apparatus 80 for acquiring the position and orientation of a workpiece 100, 120 having a feature FT that serves as a reference for determining the orientation, the apparatus 80 comprising: a workpiece model 100M, 120M modeled from the workpiece 100, 120, in which a feature model area FTe, FTe′ corresponding to the feature FT is designated; and a shape detection sensor 14 detects the workpiece 100, 120 and outputs shape data SD of the workpiece 100, 120. ｉ a position acquisition unit 84 that acquires the position and orientation by matching with the shape data SD ｉ The shape data SD overlapping with the feature model areas FTe and FTe' designated in the workpiece models 100M and 120M matched to ｉ Based on the data amount δ of the shape data SD ｉand a feature determining unit 88 that determines whether or not a feature is included in the shape data SD. ｉ The position acquisition unit 84 has a visual sensor that captures image data 210, 210' in which the workpiece models 100M, 120M are captured as shape data SD in the image data 210, 210'. ｉ The device 80 includes a quantization execution unit 64 that quantizes the image data 210, 210′ ​​into the minimum image unit IU, and a data amount δ that is used to calculate the feature model regions FTe, FTe′ and the shape data SD in the quantized image data 210, 210′. ｉ and an image unit calculation unit 86 that calculates numbers N1, N3 of smallest image units IU that overlap with the feature model areas FTe, FTe' in the quantized image data 210, 210', and the feature determination unit 88 makes a determination based on the numbers N1, N3. (Aspect 3) The device 80 according to claim 2, further comprising a ratio calculation unit 87 that calculates ratios R1, R2 of the numbers N1, N3 calculated by the image unit calculation unit 86 to the numbers N2, N4 of smallest image units IU that correspond to the feature model areas FTe, FTe' in the quantized image data 210, 210', and the feature determination unit 88 makes a determination based on the ratios R1, R2. (Aspect 4) The device 90 acquires the position and orientation of a workpiece 100, 120 having a feature FT that serves as a reference for determining its orientation, the device 90 comprising: a drawing model 100D, 120D that represents the overall shape of the workpiece 100, 120; ｍ , 120S ｍ and a model generation unit 82 that generates a plurality of search models 100S generated by the model generation unit 82. ｍ , 120S ｍ Among them, search model 100S including feature FT ｍ , 120S ｍ a feature designation unit 70 for designating feature model regions FTe and FTe' corresponding to the feature FT in each search model 100S; ｍ , 120S ｍ The shape data SD of the workpieces 100 and 120 detected by the shape detection sensor 14 ｉa position acquisition unit 84 that acquires the position and orientation by matching the position and orientation with the shape data SD ｉ Search model 100S that matches ｍ , 120S ｍ If the specified feature model areas FTe and FTe' exist in the ｉand a feature determination unit 88 that determines that the feature FT is included in the workpiece 100, 120. (Aspect 5) An apparatus 50 that specifies a feature FT in a workpiece 100, 120 having the feature FT that serves as a reference for determining the posture, the apparatus 50 including: an image generation unit 52 that generates image data 200, 200' that display workpiece models 100M, 120M that model the workpieces 100, 120; an input reception unit 54 that receives inputs IP1, IP2 that specify feature model areas FTe, FTe' corresponding to the feature FT in the workpiece models 100M, 120M displayed in the image data 200; and a position data generation unit 56 that generates position data Pf that indicates the positions, in the workpiece models 100M, 120M, of the feature model areas FTe, FTe' specified by the inputs IP1, IP2 received by the input reception unit 54. (Aspect 6) An apparatus 60 for specifying a feature FT in a workpiece 100, 120 having the feature FT as a reference for determining the posture, the apparatus 60 comprising: a model placement unit 62 that places workpiece models 100M, 120M that are models of the workpieces 100, 120 in a virtual space VS; a quantization execution unit 64 that quantizes the virtual space VS in which the workpiece model 100M is placed by the model placement unit 62 into a minimum image unit IU; a simulation unit 66 that executes a simulation SL that rotates the workpiece models 100M, 120M by a predetermined angle θ around symmetry axes A2, A3 in the virtual space VS quantized by the quantization execution unit 64; an image unit identification unit 68 that identifies the minimum image unit Vi whose image value Vi has changed before and after the simulation SL; and a feature designation unit 70 that designates feature model areas FTe, FTe' in the workpiece models 100M, 120M that correspond to the feature FT based on the minimum image unit IU identified by the image unit identification unit 68. (Aspect 7) The device 60 according to aspect 6, further comprising a position data generation unit 56 that generates position data Pf indicating the position in the workpiece models 100M, 120M of the feature model regions FTe, FTe' specified by the feature specification unit 70. (Aspect 8) The device 60 according to aspect 6 or 7, further comprising an angle determination unit 72 that determines a predetermined angle θ based on the size of the minimum image unit IU relative to the workpiece models 100M, 120M in the quantized virtual space VS.(Aspect 9) A method for acquiring the position and posture of workpieces 100, 120 having features FT that serve as a reference for determining posture, wherein the workpieces 100, 120 are modeled as workpiece models 100M, 120M, and feature model areas FTe, FTe' corresponding to the features FT are specified for the workpiece models 100M, 120M, and shape data SD of the workpieces 100, 120 is detected by a shape detection sensor 14. ｉ By matching with the shape data SD, the position and orientation are acquired. ｉ The shape data SD overlapping with the feature model areas FTe and FTe' designated in the workpiece models 100M and 120M matched to ｉ Based on the data amount δ of the shape data SD ｉ (Aspect 10) A method for acquiring the position and orientation of a workpiece 100, 120 having a feature FT that serves as a reference for determining the orientation, the method comprising: based on drawing models 100D, 120D that represent the overall shapes of the workpieces 100, 120, generating a plurality of search models 100S that represent partial shapes of the drawing models 100D, 120D as viewed from a plurality of viewpoints VP; ｍ , 120S ｍ and generating a plurality of search models 100S. ｍ , 120S ｍ Among them, search model 100S including feature FT ｍ , 120S ｍ In the search model 100S, the feature model areas FTe and FTe' corresponding to the feature FT are specified. ｍ , 120S ｍ The shape data SD of the workpieces 100 and 120 detected by the shape detection sensor 14 ｉ When the position and orientation are acquired, the shape data SD ｉ Search model 100S that matches ｍ , 120S ｍ If the specified feature model areas FTe and FTe' exist in the ｉ(Aspect 11) A method for specifying a feature FT in a workpiece (100, 120) having the feature FT that serves as a reference for determining the posture, the method comprising: generating image data (200, 200') displaying workpiece models (100M, 120M) that model the workpieces (100, 120), receiving inputs IP1, IP2 that specify feature model areas FTe, FTe corresponding to the feature FT in the workpiece models (100M, 120M) displayed in the image data (200), and generating position data Pf that indicates the positions in the workpiece models (100M, 120M) of the feature model areas FTe, FTe specified by the received inputs IP1, IP2. (Aspect 12) A method for specifying a feature FT in a workpiece 100, 120 having the feature FT that serves as a reference for determining its posture, the method comprising: arranging workpiece models 100M, 120M that are models of the workpieces 100, 120 in a virtual space VS; quantizing the virtual space VS in which the workpiece models 100M, 120M are arranged into minimum image units IU; executing a simulation SL in which the workpiece models 100M, 120M are rotated by a predetermined angle θ around the symmetry axes A2, A3 in the quantized virtual space VS; identifying the minimum image unit IU whose image value Vi has changed before and after the simulation SL; and specifying feature model areas FTe, FTe' corresponding to the feature FT in the workpiece models 100M, 120M based on the identified minimum image unit IU. (Aspect 13) Computer programs PG2, PG3, and PG4 that cause a processor 32 to execute the method described in any of Aspects 9 to 12.

[0187] REFERENCE SIGNS LIST 10 Robot system 12 Robot 14 Shape detection sensor 16 Control device 52 Image generation unit 54 Input reception unit 56 Position data generation unit 62 Model placement unit 64 Quantization execution unit 66 Simulating unit 68 Image unit identification unit 70 Feature designation unit 82 Model generation unit 84 Position acquisition unit 86 Image unit calculation unit 87 Ratio calculation unit 88 Feature determination unit 100, 120 Workpiece 100M, 120M Workpiece model 200, 210 Image data

Claims

1. An apparatus for acquiring the position and posture of a workpiece having features serving as a reference for determining the posture, comprising: a workpiece model obtained by modeling the workpiece, in which a feature model region corresponding to the features is specified; a position acquisition unit that acquires the position and the posture by matching the workpiece model to the shape data of the workpiece detected by a shape detection sensor; and a feature determination unit that determines whether or not the features are included in the shape data based on the data amount of the shape data that overlaps with the feature model region specified in the workpiece model matched to the shape data.

2. The shape detection sensor includes a vision sensor that captures image data in which the shape data appears. The position acquisition unit matches the workpiece model to the shape data in the image data. The apparatus further comprises: a quantization execution unit that quantizes the image data into minimum image units; and an image unit calculation unit that obtains, as the data amount, the number of the minimum image units in which the feature model region and the shape data overlap in the quantized image data. The feature determination unit makes the determination based on the number. The apparatus according to claim 1.

3. The apparatus according to claim 2, further comprising a ratio calculation unit that calculates a ratio between the number obtained by the image unit calculation unit and the number of the minimum image units corresponding to the feature model region in the quantized image data. The feature determination unit makes the determination based on the ratio.

4. An apparatus for acquiring the position and posture of a workpiece having a feature serving as a reference for determining the posture, comprising: a model generation unit that generates a plurality of search models each representing a partial shape of a drawing model viewed from a plurality of viewpoints based on a drawing model representing the overall shape of the workpiece; a feature designation unit that designates a feature model region corresponding to the feature in the search model including the feature among the plurality of search models generated by the model generation unit; a position acquisition unit that acquires the position and the posture by matching each of the search models with shape data of the workpiece detected by a shape detection sensor; and a feature determination unit that determines that the feature is included in the shape data when the designated feature model region exists in the search model that has been matched with the shape data when the position and the posture are acquired.

5. An apparatus for designating a feature in a workpiece having a feature serving as a reference for determining the posture, comprising: an image generation unit that generates image data displaying a workpiece model obtained by modeling the workpiece; an input reception unit that receives an input for designating a feature model region corresponding to the feature in the workpiece model displayed in the image data; and a position data generation unit that generates position data indicating the position of the feature model region designated by the input received by the input reception unit in the workpiece model.

6. An apparatus for designating a feature in a workpiece having a feature serving as a reference for determining the posture, comprising: a model arrangement unit that arranges a workpiece model obtained by modeling the workpiece in a virtual space; a quantization execution unit that quantizes the virtual space in which the model arrangement unit has arranged the workpiece model into minimum image units; a simulation unit that executes a simulation of rotating the workpiece model by a predetermined angle around a symmetry axis in the virtual space quantized by the quantization execution unit; an image unit specifying unit that specifies the minimum image unit in which the image value has changed before and after the simulation; and a feature designation unit that designates a feature model region corresponding to the feature in the workpiece model based on the minimum image unit specified by the image unit specifying unit.

7. The apparatus according to claim 6, further comprising a position data generation unit that generates position data indicating a position of the feature model area specified by the feature specification unit in the work model.

8. The apparatus according to claim 6, further comprising an angle determination unit that determines the predetermined angle based on a size of the minimum image unit with respect to the work model in the quantized virtual space.

9. A method for acquiring a position and an attitude of a work having a feature serving as a reference for determining the attitude, the method including: matching a work model, which is a model of the work and in which a feature model area corresponding to the feature is specified, with shape data of the work detected by a shape detection sensor to acquire the position and the attitude; and determining whether the feature is included in the shape data based on an amount of the shape data overlapping with the feature model area specified in the work model matched with the shape data.

10. A method for acquiring a position and an attitude of a work having a feature serving as a reference for determining the attitude, the method including: generating a plurality of search models each representing a partial shape of a drawing model of the work viewed from a plurality of viewpoints based on the drawing model representing an overall shape of the work; specifying a feature model area corresponding to the feature in the search model including the feature among the generated plurality of search models; matching each of the search models with the shape data of the work detected by a shape detection sensor to acquire the position and the attitude; and determining that the feature is included in the shape data when the specified feature model area exists in the search model matched with the shape data when the position and the attitude are acquired.

11. A method for specifying a feature in a work having a feature serving as a reference for determining the attitude, the method including: generating image data displaying a work model obtained by modeling the work; receiving an input for specifying a feature model area corresponding to the feature in the work model displayed in the image data; and generating position data indicating a position of the feature model area specified by the received input in the work model.

12. A method for specifying a feature serving as a reference for determining a posture in a workpiece having the feature, the method comprising: placing a workpiece model obtained by modeling the workpiece in a virtual space; quantizing the virtual space in which the workpiece model is placed into minimum image units; performing a simulation of rotating the workpiece model by a predetermined angle around a symmetry axis in the quantized virtual space; identifying the minimum image units in which the image values have changed before and after the simulation; and designating a feature model region corresponding to the feature in the workpiece model based on the identified minimum image units.

13. A computer program that causes a processor to execute the method according to any one of claims 9 to 12.

Images