Device, control device, robot system, method, and computer program for acquiring workpiece position data

Abstract

In the past, there have been cases in which a workpiece model is erroneously matched to detection data from a shape detection sensor. In such cases, accurate position data pertaining to a workpiece cannot be acquired, and it can become impossible to accurately execute work on the workpiece.　This device 50 that acquires position data pertaining to a workpiece on the basis of detection data from a shape detection sensor 14 comprises: a position data acquisition unit 44 that matches a workpiece model to shape data pertaining to the workpiece included in the detection data, the workpiece model modeling the workpiece and being such that a confirmation region corresponding to a region that cannot be entered is established therein, to thereby acquire the position data; and a position data cancellation unit 46 that invalidates the position data acquired by the position data acquisition unit 44 if shape data pertaining to an object included in the detection data is present in the confirmation region established in the workpiece model that is matched to the shape data pertaining to the workpiece by the position data acquisition unit 44.

Description

[Technical Field]

[0001] This disclosure relates to an apparatus, control device, robot system, method, and computer program for acquiring workpiece position data. [Background technology]

[0002] A device is known that acquires position data of a workpiece by matching detection data from a shape detection sensor that detects the shape of the workpiece with a workpiece model that models the workpiece (for example, Patent Document 1). [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2017-102529 [Overview of the project] [Problems that the invention aims to solve]

[0004] Previously, there were cases where the workpiece model was incorrectly matched to the detection data from the shape detection sensor. In such cases, accurate workpiece position data could not be obtained, potentially making it impossible to perform operations on the workpiece accurately. [Means for solving the problem]

[0005] In one embodiment of the present disclosure, an apparatus for acquiring position data of a workpiece based on detection data from a shape detection sensor that detects the shape of an object and a workpiece adjacent to the object and having an area inaccessible to the object includes a position data acquisition unit that acquires position data by matching a workpiece model, which is a model of the workpiece and has a defined verification area corresponding to an inaccessible area, with the shape data of the workpiece included in the detection data, and a position data cancellation unit that invalidates the position data acquired by the position data acquisition unit when the shape data of an object included in the detection data exists in the verification area defined in the workpiece model matched by the position data acquisition unit to the shape data of the workpiece.

[0006] In another aspect of the present disclosure, a method for obtaining position data of a workpiece based on detection data from a shape detection sensor that detects the shape of an object and a workpiece adjacent to the object and having an area inaccessible to the object, includes a processor that obtains position data by matching a workpiece model that models the workpiece, in which a verification area corresponding to an inaccessible area is defined, with the shape data of the workpiece included in the detection data, and invalidating the obtained position data if the shape data of an object included in the detection data exists in the verification area defined in the workpiece model matched to the shape data of the workpiece. [Effects of the Invention]

[0007] According to this disclosure, it is possible to avoid having a robot perform operations on a workpiece using position data obtained as a result of inconsistencies between the workpiece model and the shape data, which occur when the workpiece model is incorrectly matched to the shape data. As a result, the accuracy of the robot's operations can be improved. [Brief explanation of the drawing]

[0008] [Figure 1] This is a schematic diagram of a robot system according to one embodiment. [Figure 2] Figure 1 is a block diagram of the robot system shown. [Figure 3]This is a perspective view of a workpiece according to one embodiment. [Figure 4] This is a flowchart illustrating an example of the operation flow of a robot system. [Figure 5] This is an example of detection data detected by a shape detection sensor. [Figure 6] This is a flowchart showing an example of the flow of step S2 in Figure 4. [Figure 7] Figure 3 shows a perspective view of the workpiece model, which is a model of the workpiece shown in Figure 3. [Figure 8] Figure 8 shows a top view of the work model. [Figure 9] Figure 5 shows the state after matching the work model with the detection data shown. [Figure 10] Figure 5 shows the result of incorrectly matching the work model to the detection data. [Figure 11] This is a flowchart showing an example of the flow of step S3 in Figure 4. [Figure 12] This flowchart shows another example of the flow in step S3 in Figure 4. [Figure 13] Figure 5 shows the detection data with one shape data point removed. [Figure 14] Figure 2 is a block diagram showing other functions of the robot system. [Figure 15] This flowchart shows an example of how to set a verification area in a work model. [Figure 16] This shows a work model with multiple verification areas set. [Figure 17] Figure 2 is a block diagram showing other functions of the robot system. [Figure 18] This flowchart shows other examples of how to set up verification areas in a work model. [Figure 19] This shows the work model quantized into unit voxels. [Figure 20] This figure illustrates the simulation of step S51 in Figure 18. [Modes for carrying out the invention]

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the various embodiments described below, similar elements will be denoted by the same reference numerals, and redundant descriptions will be omitted. First, a robot system 10 according to one embodiment will be described with reference to Figures 1 and 2. The robot system 10 comprises a robot 12, a shape detection sensor 14, and a control device 16.

[0010] In this embodiment, the robot 12 is a vertical articulated robot having a robot base 18, a swivel torso 20, a forearm 22, an upper arm 24, a wrist 26, and an end effector 28. The robot base 18 is fixed to the floor of the work cell. The swivel torso 20 is mounted on the robot base 18 so as to be able to swivel around a vertical axis.

[0011] The lower arm portion 22 is mounted on the rotating torso 20 so as to be rotatable around a horizontal axis, and the upper arm portion 24 is mounted on the tip of the lower arm portion 22 so as to be rotatable. The wrist portion 26 has a wrist base 26a mounted on the tip of the upper arm portion 24 so as to be rotatable around two mutually orthogonal axes, and a wrist flange 26b mounted on the wrist base 26a so as to be rotatable around a wrist axis A1.

[0012] The end effector 28 is detachably attached to the wrist flange 26b. The end effector 28 is, for example, a robot hand capable of gripping the workpiece W, a welding torch for welding the workpiece W, or a laser processing head for laser processing the workpiece W, and performs a predetermined operation (workpiece handling, welding, or laser processing) on ​​the workpiece W.

[0013] Each component of the robot 12 (robot base 18, swivel torso 20, forearm 22, upper arm 24, wrist 26) is equipped with a servo motor 30 (Figure 2). These servo motors 30 rotate each movable element of the robot 12 (swivel torso 20, forearm 22, upper arm 24, wrist 26, wrist flange 26b) in response to commands from the control device 16. As a result, the robot 12 can move the end effector 28 to any desired position.

[0014] The shape detection sensor 14 detects the shape of an object such as a workpiece W. In this embodiment, the shape detection sensor 14 is a three-dimensional vision sensor having an imaging sensor (CMOS, CCD, etc.) and an optical lens (collimating lens, focusing lens, etc.) that guides the image of the subject to the imaging sensor, and is fixed to the end effector 28 (or wrist flange 26b).

[0015] The shape detection sensor 14 is configured to capture an image of a subject along the optical axis A2 and to measure the distance d to the subject image. The shape detection sensor 14 may be fixed to the end effector 28 so that the optical axis A2 and the wrist axis A1 are parallel (or perpendicular) to each other. The shape detection sensor 14 supplies the detected detection data DD to the control device 16.

[0016] As shown in Figure 1, the robot 12 is configured with a robot coordinate system C1 and a tool coordinate system C2. The robot coordinate system C1 is a control coordinate system for controlling the movement of each movable element of the robot 12. In this embodiment, the robot coordinate system C1 is fixed to the robot base 18 such that its origin is located at the center of the robot base 18 and its z-axis is parallel to (specifically coincides with) the rotation axis of the swivel body 20.

[0017] On the other hand, the tool coordinate system C2 is a control coordinate system that defines the position and orientation of the end effector 28 in the robot coordinate system C1. In this embodiment, the tool coordinate system C2 is set relative to the end effector 28 such that its origin (so-called TCP) is located at the working position of the end effector 28 (for example, the workpiece gripping position, welding position, or laser beam output port), and its z-axis is parallel to (specifically coincides with) the wrist axis A1.

[0018] When moving the end effector 28, the control device 16 sets a tool coordinate system C2 in the robot coordinate system C1 and generates commands to each servo motor 30 of the robot 12 to position the end effector 28 at the position and orientation represented by the set tool coordinate system C2. In this way, the control device 16 can position the end effector 28 at any position and orientation in the robot coordinate system C1.

[0019] On the other hand, the shape detection sensor 14 is configured with a sensor coordinate system C3. Sensor coordinate system C3 is a control coordinate system that defines the position and orientation of the shape detection sensor 14 in the robot coordinate system C1 (i.e., the position and direction of the optical axis A2). In this embodiment, sensor coordinate system C3 is configured for the shape detection sensor 14 such that its origin is located at the center of the image sensor of the shape detection sensor 14, and its z-axis is parallel to (specifically coincides with) the optical axis A2. Sensor coordinate system C3 defines the coordinates of each pixel of the detection data DD (or image sensor) detected by the shape detection sensor 14.

[0020] The positional relationship between the sensor coordinate system C3 and the tool coordinate system C2 is known through calibration; therefore, the coordinates of the sensor coordinate system C3 and the coordinates of the tool coordinate system C2 are mutually convertible via a known transformation matrix (e.g., a homogeneous transformation matrix). Furthermore, since the positional relationship between the tool coordinate system C2 and the robot coordinate system C1 is known, the coordinates of the sensor coordinate system C3 and the coordinates of the robot coordinate system C1 are mutually convertible via the tool coordinate system C2. In other words, the position and orientation of the shape detection sensor 14 in the robot coordinate system C1 (specifically, the coordinates of the sensor coordinate system C3) are known.

[0021] The control device 16 controls the movement of the robot 12. Specifically, as shown in Figure 2, the control device 16 is a computer having a processor 32, memory 34, and I / O interface 36. The processor 32 has a CPU or GPU, and is communicated with the memory 34 and I / O interface 36 via a bus 38. While communicating with these components, it performs calculation processing to realize various functions described later.

[0022] Memory 34 has RAM or ROM, etc., and stores various data temporarily or permanently. Memory 34 can be composed of a storage medium such as volatile memory, non-volatile memory, magnetic storage medium, or optical storage medium. I / O interface 36 has, for example, an Ethernet® port, a USB port, an optical fiber connector, or an HDMI® terminal, and communicates data with external devices via wired or wireless connection under commands from processor 32. Each servo motor 30 and shape detection sensor 14 of the robot 12 are communicated to I / O interface 36.

[0023] Furthermore, the control device 16 is equipped with a display device 40 and an input device 42. The display device 40 and the input device 42 are communicatively connected to the I / O interface 36. The display device 40 has a liquid crystal display or an organic EL display, and displays various data in a visible manner under commands from the processor 32.

[0024] The input device 42 has push buttons, switches, a keyboard, a mouse, or a touch panel, and accepts data input from the operator. The display device 40 and the input device 42 may be integrated into the housing of the control device 16, or they may be attached to the housing of the control device 16 as a separate computer (PC, etc.).

[0025] In this embodiment, the processor 32 acquires position data of the workpieces W in the robot coordinate system C1 based on the detection data DD of the shape detection sensor 14 that detects the shape of multiple workpieces W that are piled up loosely in the container B. As shown in Figure 3, in this embodiment, each workpiece W has a shaft portion S, a flange portion F fixed to one end of the shaft portion S, and a hole H that penetrates the shaft portion S and the flange portion F in the axial direction.

[0026] When the workpieces are stacked loosely in container B, one workpiece W1 will come into contact with another workpiece W2 adjacent to it, but the components of the other workpiece W2 (i.e., the shaft portion S and flange portion F) cannot enter the hole H of workpiece W1. In other words, in this embodiment, each workpiece W has a hole H that is an area into which other workpieces W (objects) cannot enter.

[0027] Next, the functions of the robot system 10 will be described with reference to Figure 4. The flow shown in Figure 4 is started when the processor 32 receives a work start command from the operator, the higher-level controller, or the computer program PG. In step S1, the processor 32 acquires the detection data DD detected by the shape detection sensor 14.

[0028] Specifically, the processor 32 operates the robot 12 to position the shape detection sensor 14 at an imaging position that places the workpieces W, which are loosely stacked in container B, within the detection range of the shape detection sensor 14. Then, the processor 32 operates the shape detection sensor 14 to image the workpieces W inside container B, thereby detecting detection data DD that shows the workpieces W.

[0029] Figure 5 shows an example of an image of the detection data DD. Note that, for ease of understanding, the detection data DD in Figure 5 only shows two adjacent workpieces W1 and W2 within container B. However, in reality, the detection data DD may show three or more loosely stacked workpieces W.

[0030] In this embodiment, the detected data DD is a 3D point cloud image data, and consists of multiple workpieces W n Shape data SD n (Includes n=1,2,3,...) (In Figure 5, shape data SD1 for workpiece W1, shape data SD2 for workpiece W2). Shape data SD n , Work W n It has a point cloud that shows the visual features (edges, faces, etc.), and each point constituting the point cloud has the distance d information described above. Therefore, the shape data SD n Each point that makes up the point cloud is in the 3D coordinate system (X) of the sensor coordinate system C3. S ,Y S ,Z S It can be expressed as ). The processor 32 acquires the detection data DD detected by the shape detection sensor 14 from the shape detection sensor 14.

[0031] In step S2, the processor 32 acquires the position data PD of the workpiece W. That is, in this embodiment, the processor 32 functions as a position data acquisition unit 44 (Figure 14) that acquires the position data PD of the workpiece W. Step S2 will be explained with reference to Figure 6. The processor 32, functioning as a position data acquisition unit 44, executes the flow of step S2 shown in Figure 6.

[0032] After the start of step S2, in step S11, the processor 32 obtains a work model WM that models the work W. An example of the work model WM is shown in FIG. 7. The work model WM includes a shaft portion model SM that models the shaft portion S, a flange portion model FM that models the flange portion F, and a hole model HM that models the hole H.

[0033] The work model WM is, for example, a CAD model WM of the work W C and a point cloud model WM that represents the model components (edges, faces, etc.) of the CAD model WM C by point clouds (or normal vectors). The CAD model WM P is a three-dimensional CAD model and is created in advance by an operator using a CAD device (not shown). The processor 32 obtains the CAD model WM C from the CAD device and generates the point cloud model WM C by assigning point clouds to the model components of the CAD model WM C according to a predetermined image generation algorithm.

[0034] In the present embodiment, in step S11, the processor 32 generates work models WM (for example, point cloud models WM P ) in a plurality of poses as viewed from different virtual line-of-sight directions VL. For example, FIG. 7 shows the work model WM in the first pose when the work model WM is viewed from an oblique virtual line-of-sight direction VL.

[0035] On the other hand, FIG. 8 shows the work model WM in the second pose when the work model WM is viewed from a virtual line-of-sight direction VL parallel to the central axis of the work model WM. The processor 32 generates work models WM in various poses when the work model WM is viewed from various virtual line-of-sight directions VL as shown in FIGS. 7 and 8.

[0036] ​​Furthermore, the generated work model WMs for multiple orientations may only contain model data for the front side that is visible from the virtual line of sight direction VL, and may not contain model data for the back side that is not visible from the virtual line of sight direction VL. For example, the processor 32 generates a point cloud model WM for the first orientation shown in Figure 7. P When generating the model, the system generates point cloud model data for the model components on the front side of the paper that are visible in Figure 7, while not generating point cloud model data for the model components on the back side of the paper that are not visible (i.e., edges and surfaces on the back side as viewed from the virtual line of sight direction VL in Figure 7). This configuration reduces the amount of data generated for the work model WM.

[0037] Each generated work model WM is assigned a work coordinate system C4. This work coordinate system C4 is a control coordinate system that defines the position and orientation of the work model WM. In the examples shown in Figures 7 and 8, the work coordinate system C4 is set relative to the work model WM such that its origin is located at the opening center of the hole model HM that opens at the end face of the shaft model SM, and its z-axis coincides with the central axis of the hole model HM (or work model WM).

[0038] The processor 32 may also accept input from the operator via the input device 42 to set the work coordinate system C4 for the work model WM. The processor 32 stores the generated work model WMs for multiple orientations, along with the work coordinate system C4 setting information, in the memory 34.

[0039] Here, a verification region VE is predetermined for each of the generated work model WMs in multiple orientations. In this embodiment, the hole model HM of the work model WM (i.e., the area on the model corresponding to the area of ​​the work W that other work Ws cannot enter) is defined as the verification region VE. The method for setting the verification region VE in the work model WM will be described later.

[0040] In step S12, the processor 32 performs preprocessing PP on the detection data DD acquired in step S1. For example, as preprocessing PP, the processor 32 selects point clouds from the point clouds included in the detection data DD that should be invalidated (for example, point clouds located outside container B, or workpiece W n Shape data SD n You may also perform a process to remove points from the detection data DD that are significantly separated from the point cloud by a predetermined distance.

[0041] In step S13, the processor 32 performs a rough search RS. Specifically, as a rough search RS, the processor 32 sequentially places the work model WM with multiple poses generated in step S11 into a virtual space defined by the sensor coordinate system C3 of the detected data DD, and places the work model WM into the shape data SD included in the detected data DD. n Match it.

[0042] At this time, the processor 32 repeatedly displaces the position of the work model WM, which is placed in the sensor coordinate system C3, by a predetermined amount. Each time the position of the work model WM is displaced, the processor 32 processes the feature points Fm of the work model WM and the shape data SD. n We calculate the degree of agreement α with the feature point Fs.

[0043] This degree of agreement α includes, for example, the error in the distance between a feature point Fm and the corresponding feature point Fs. In this case, the smaller the value of the degree of agreement α, the more the feature point Fm and the feature point Fs coincide in the sensor coordinate system C3. Alternatively, the degree of agreement α includes a similarity value representing the similarity between a feature point Fm and the corresponding feature point Fs. In this case, the larger the value of the degree of agreement α, the more the feature point Fm and the feature point Fs coincide in the sensor coordinate system C3.

[0044] Then, the processor 32 takes the calculated degree of agreement α and a predetermined threshold α for the degree of agreement α. th The degree of agreement α is compared with the threshold α. th When it exceeds (that is, α ≤ α) th , or α≧αth ), in sensor coordinate system C3, work model WM and shape data SD n It is determined that the two are a good match.

[0045] Figure 9 shows the state in which the work model WM is matched to the shape data SD1 of the work W1. The processor 32 sets the coordinate Q1 of the sensor coordinate system C3 in the work coordinate system C4, which is set in the work model WM that has been matched to the shape data SD1 as a rough search RS. S (X1 S ,Y1 S ,Z1 S ,W1 S ,P1 S ,R1 S ) obtains this coordinate Q1 S (X1 S ,Y1 S ,Z1 S ) indicates the origin position of the work coordinate system C4 in the sensor coordinate system C3, and (W1 S ,P1 S ,R1 S ) indicates the orientation (so-called yaw, pitch, and roll) of the workpiece coordinate system C4 in the sensor coordinate system C3.

[0046] Then, the processor 32 processes the coordinate Q1 of the sensor coordinate system C3. S coordinate Q1 in robot coordinate system C1 R (X1 R ,Y1 R ,Z1 R ,W1 R ,P1 R ,R1 R Convert to the coordinate Q1 R to initial position data PD 1_1 This is obtained as initial position data PD. 1_1 This represents the approximate position and orientation of the workpiece W1 in the robot coordinate system C1.

[0047] Similarly, the processor 32 detects other work W that are reflected in the detected data DD. n Initial position data PD 1_nTo obtain the following: For example, in the detected data DD, the shape data SD1 of workpiece W1, the shape data SD2 of workpiece W2, ...workpiece W n Shape data SD n If (n is a positive integer) is included, the processor 32 will input the initial position data PD of work W1. 1_1 Initial position data PD of workpiece W2 1_2 ,...Work W n Initial position data PD 1_n The processor 32 obtains the shape data SD in the coarse search RS using the work model WM. n By matching with WorkW n Location data PD 1_n They have obtained it.

[0048] In step S14, the processor 32 performs a dense search PR. Specifically, the processor 32 searches for the work W reflected in the detected data DD. n Regarding this, the initial position data PD obtained in step S13 1_n Based on this, and according to a predetermined matching algorithm (for example, a mathematical optimization algorithm such as ICP: Iterative Closest Point), the work model WM in the sensor coordinate system C3 corresponds to the shape data SD. n We search for a position that matches highly.

[0049] For example, the processor 32 receives the initial position data PD of the sensor coordinate system C3. 1_n Point cloud model WM placed as work model WM P The point cloud and the shape data SD included in the detected data DD. n We calculate the degree of agreement β with the 3D point cloud. For example, this degree of agreement β is calculated using the point cloud model WM. P Point cloud and shape data SD n The error in distance to the 3D point cloud, or the point cloud model WM P Point cloud and shape data SD n Includes similarity to the 3D point cloud.

[0050] Then, the processor 32 takes the calculated degree of agreement β and a predetermined threshold β for the degree of agreement β.th Compare it with, and when the degree of coincidence β exceeds the threshold value β th (for example, β ≤ β th or β ≥ β th ), it is determined that the work model WM (for example, the point cloud model WM P ) and the shape data SD n are highly matched. On the other hand, when the degree of coincidence β does not exceed the threshold value β th , the processor 32 displaces the position of the work model WM arranged in the sensor coordinate system C3 by a predetermined displacement amount, calculates the degree of coincidence β every time the position of the work model WM is displaced, and compares it with the threshold value β th .

[0051] When the degree of coincidence β exceeds the threshold value β th , the processor 32 obtains the coordinates Q2 n in the sensor coordinate system C3 of the work coordinate system C4 set in the work model WM that is highly matched with the shape data SD S (X2 S , Y2 S , Z2 S , W2 S , P2 S , R2 S ) in the sensor coordinate system C3 of the work coordinate system C4 set in the work model WM that is highly matched with the shape data SD

[0052] Then, the processor 32 converts the coordinates Q2 S in the sensor coordinate system C3 to the coordinates Q2 R (X2 R , Y2 R , Z2 R , W2 R , P2 R , R2 R ) in the robot coordinate system C1, and obtains the coordinates Q2 R as the position data PD n of the work W 2_n . This position data PD 2_n represents the high-precision position and orientation of the work W n in the robot coordinate system C1.

[0053] As a result of this step S14, the processor 32 has a plurality of works W shown in the detection data DDn Location data PD 2_n The processor 32 obtains the acquired position data PD. 2_n This is stored in memory 34 as detection result data DT. An example of the data structure of detection result data DT is schematically shown in Table 1 below.

[0054] [Table 1]

[0055] The detection result data DT shown in Table 1 includes workpiece W n Location data PD 2_n (Specifically, coordinate Q2 S (X2 R ,Y2 R ,Z2 R ,W2 R ,P2 R ,R2 R )) along with the degree of agreement β n , plane proportion γ n , and exposure rate δ n Detection result parameters PM are stored. These detection result parameters PM are stored in the location data PD. 2_n Work W executed when acquiring n Shape data SD n This parameter represents the result of matching with the work model WM. Specifically, it is the degree of agreement β. n In step S14, the dense search PR converts the work model WM to shape data SD. n This is the degree of agreement when it is determined that a high level of matching has been achieved.

[0056] Plane ratio γ n For example, shape data SD in dense search PR n In the work model WM matched to the work model WM, the largest planar model included in the work model WM (specifically, a point cloud model WM of a plane) P ) refers to the entire work model WM (specifically, the entire point cloud model WM) P This is the proportion of ) to the total. Alternatively, the plane ratio γ nThis is shape data SD, which matches the work model WM using dense search PR. n The point cloud representing the largest plane contained within the shape data SD n It may also be the proportion of the whole (specifically, the entire point cloud).

[0057] Exposure rate δ n For example, shape data SD in dense search PR n The entire work model WM that has been matched (specifically, the point cloud model WM) P The shape data SD for all the point clouds that make up the shape n The area of ​​the work model WM that matches (i.e., shape data SD) n The point cloud model WM is matched with the point cloud. P This shows the proportion of the point clusters.

[0058] The processor 32 receives location data PD 2_n Along with the detection result parameter PM (degree of agreement β), n , plane proportion γ n , and exposure rate δ n ) obtains location data PD 2_n The detection result data DT is stored in memory 34 in association with this. The detection result parameter PM is the degree of agreement β. n , plane proportion γ n , and exposure rate δ n This is not limited to this; it may have any other parameters.

[0059] In step S15, the processor 32 performs post-processing OP on the detection result data DT. For example, as post-processing OP, the processor 32 processes position data PD where the detection result parameter PM does not meet a predetermined criterion. 2_n You may also perform a process to remove it from the detection result data DT.

[0060] Specifically, processor 32 calculates the degree of agreement β. n , plane proportion γ n , or exposure rate δ nIf the detection result parameter PM does not exceed a predetermined standard value, it is determined that the detection result parameter PM does not meet the standard, and the location data PD associated with the detection result parameter PM is determined to be missing. 2_n The detected result data DT may be deleted. As described above, the processor 32 functions as a position data acquisition unit 44 and executes step S2 shown in Figure 6, and the workpiece W n Location data PD 1_n , PD 2_n Obtain it.

[0061] Referring again to Figure 4, in step S3, the processor 32 receives the work model WM and shape data SD. n The inconsistency is verified. Here, in steps S13 and S14 described above, the work model WM is converted to shape data SD. n When matched, the processor 32 converts the work model WM to the shape data SD. n Incorrect matching results in the matched work model WM and shape data SD. n Inconsistencies may arise between these two points.

[0062] Figure 10 shows an example of a state in which such a mismatch occurs. In the example shown in Figure 10, when the processor 32 matches the workpiece model WM to the shape data SD1 of the workpiece W1 reflected in the detected data DD, the flange portion model FM of the workpiece model WM matches to the shape data SD1 of the flange portion F of the workpiece W1.

[0063] On the other hand, the shape data SD1 of the shaft portion S of workpiece W1 does not match the shaft portion model SM of workpiece model WM. As a result, a portion of the point cloud of the shape data SD2 of another workpiece W2 exists within the hole model HM of workpiece model WM, which is set as the verification area VE mentioned above.

[0064] In this embodiment, in step S3, the processor 32 verifies whether an inconsistency as shown in Figure 10 has occurred based on the shape data SD2 of the workpiece W2 located in the verification area VE. If an inconsistency has occurred, the processor 32 obtains the position data PD as the detection result data DT. 2_n This disables the position data PD. In other words, in this embodiment, the processor 32 disables the position data PD. 2_n It functions as a position data cancellation unit 46 (Figure 2) that disables the data.

[0065] Step S3 will now be described with reference to Figure 11. The processor 32 functions as a position data cancellation unit 46 and executes the flow of step S3 shown in Figure 11. After the start of step S3, in step S21, the processor 32 processes the work W stored in the detection result data DT. n Location data PD 2_n Set the identifying number "n" to "1".

[0066] In step S22, the processor 32 controls the work W n Location data PD 2_n Shape data SD within the verification area VE defined in the matched work model WM. m Data size DA (for example, m ≠ n) n The following is calculated. If, for example, the number "n" mentioned above is set to n=1 at the start of step S22 (i.e., the first execution of step S22), the processor 32 calculates the position data PD of the workpiece W1. 2_1 Shape data SD within the verification area VE defined in the matched work model WM. m Calculate the data volume DA1.

[0067] For example, in the case shown in Figure 10, as described above, the point cloud of shape data SD2 for work W2, which is an object adjacent to work W1, exists within the verification region VE of the work model WM, which has been incorrectly matched to the shape data SD1 of work W1. Therefore, in this case, the processor 32 calculates the data amount DA1 of the point cloud of shape data SD2 within the verification region VE defined in the work model WM.

[0068] Specifically, the processor 32 calculates the number of points of shape data SD2 located in the verification area VE in the sensor coordinate system C3 as the data amount DA1. In this way, in step S22, the processor 32 processes the workpiece W n Location data PD 2_n Shape data SD within the verification area VE defined in the matched work model WM. m Data volume DA n Calculate.

[0069] In step S23, the processor 32 calculates the amount of data DA calculated in the previous step S22. n Based on, Work W n Location data PD 2_n When acquiring the matching work model WM, the shape data SD is stored in the verification area VE of the work model WM. m The processor determines whether or not the data amount DA calculated in the previous step S22 is present. n And the data volume DA of the work model WM w By substituting and into the following equation 1, the inconsistency parameter ε n Calculate. ε n =DA n / DA w ...(Formula 1)

[0070] The mismatch parameter ε obtained by the calculation in Equation 1 n This involves the work model WM and shape data SD. nThis parameter quantitatively represents the degree of inconsistency. As described above, the verification region VE is defined in the work model WM so as to correspond to an area that other objects (e.g., workpiece W2) cannot enter. Therefore, as shown in Figure 9 for example, if the work model WM is properly matched to the shape data SD1 of workpiece W1, then the point cloud of the shape data SD2 of workpiece W2, which is another object adjacent to workpiece W1, will not substantially exist in the verification region VE of the work model WM.

[0071] On the other hand, as shown in Figure 10, if the work model WM is incorrectly matched to the shape data SD1, then a point cloud of shape data SD2 will exist in the verification region VE of the work model WM. Therefore, by determining the mismatch parameter ε1 from the data amount DA1 of shape data SD2 in the verification region VE and the above-mentioned equation 1, the mismatch between the work model WM and the shape data SD1 of the work W1, as shown in Figure 10, can be quantitatively verified.

[0072] In step S23, the processor 32 uses the mismatch parameter ε obtained as described above. n However, a predetermined threshold ε th That is all (i.e., ε n ≧ε th Determine whether or not ε n ≧ε th If so, shape data SD in the verification area VE m It is determined that it exists (i.e., YES). On the other hand, processor 32 determines ε n <ε th If that is the case, the answer is NO.

[0073] Note that Equation 1 above is ε n =DA n / DA w The formula can also be modified to multiply by 100. Furthermore, the calculation is not limited to the above formula 1, but also applies to the work model WM and shape data SD. n The inconsistency parameter, which quantitatively represents the degree of inconsistency with the data volume DA, is used to express the degree of inconsistency. n It may be obtained by any operation using [the specified method].

[0074] Alternatively, processor 32 uses the mismatch parameter ε n Without calculating it, the amount of data DA calculated in the previous step S22 n However, the predetermined threshold DA th That is all (i.e., DA n ≥DA th The processor 32 may determine YES if the condition is met. If the condition is met, the processor 32 proceeds to step S24, and if the condition is met, it proceeds to step S25.

[0075] In step S24, the processor 32 functions as a position data cancellation unit 46 and invalidates the position data PD of the workpiece W acquired in step S2 described above. The following describes the case where the number "n" described above is set to n=1 at the start of step S24.

[0076] As shown in Figure 10, if the work model WM is incorrectly matched to the shape data SD1, the work coordinate system C4 set in the work model WM does not accurately represent the position and orientation of the work W1 as reflected in the detected data DD. Therefore, if the processor 32, in step S5 described later, acquires position data PD based on such an inconsistent work coordinate system C4, 2_1 When the robot 12 is made to perform an operation on the workpiece W1 using this method, it becomes impossible to perform the operation with high precision.

[0077] Therefore, in this embodiment, the processor 32 functions as a position data cancellation unit 46 in step S24, and controls the position data PD acquired as a result of the inconsistency in step S5 described later. 2_1 To avoid using the position data PD 2_1 Disable.

[0078] As an example, the processor 32 obtains the position data PD of the workpiece W1 from the detection result data DT created in step S2 described above. 2_1Delete the following. As another example, the processor 32 deletes the position data PD of the workpiece W1 stored in the detection result data DT. 2_1 The invalid flag FL is assigned to it.

[0079] In this case, when the processor 32 executes step S5 described later, the position data PD of the detection result data DT 2_1 When the data is read, the position data PD 2_1 Referencing the invalid flag FL assigned to the position data PD, 2_1 This is ignored. As a result, processor 32 cancels the work on workpiece W1 in step S5.

[0080] In this way, in step S24, the processor 32 processes the position data PD stored in the detection result data DT. 2_n This is disabled (for example, by deleting it or by assigning the disabled flag FL), thereby preventing the control of step S5 described below from receiving the position data PD obtained as a result of the inconsistency. 2_1 This allows us to avoid using it.

[0081] In step S25, the processor 32 determines that the number "n" set at this point is n = n MAX Determine whether or not this n MAX This is the work W obtained in step S2. n Location data PD 2_n The total number (or the work W reflected in the detection data DD acquired in the most recent step S1) n This is the total number of ( ). If the processor 32 determines that the result is YES, it terminates the flow shown in Figure 11, while if it determines that the result is NO, it proceeds to step S26.

[0082] In step S26, the processor 32 increments the aforementioned number "n" by "1" (n = n + 1). Then, the processor 32 returns to step S22. In this way, the processor 32 repeatedly executes the loop of steps S22 to S26 until it determines YES in step S25, and the position data PD stored in the detection result data DT is recovered.2_n Of these, the location data PD obtained as a result of the inconsistency 2_n Disable.

[0083] Referring again to Figure 4, in step S4, the processor 32 generates position data PD 2_n Specifically, the processor 32 determines the detection result data DT after executing step S3 as formal control data to be used for control in step S5 described later, and stores it in memory 34 (for example, RAM).

[0084] In step S5, the processor 32 refers to the detection result data DT determined in step S4 and operates the robot 12 to work on the workpiece W n Perform operations on the workpiece (work handling, welding, or laser processing, etc.). For example, in step S24 above, the position data PD of the workpiece W1 is obtained. 2_1 While this is disabled, the position data PD of work W2 is disabled. 2_2 Let's assume it wasn't invalidated.

[0085] In this case, the processor 32 cancels the operation on workpiece W1, while the position data PD of workpiece W2 stored in the detection result data DT is canceled. 2_2 Refer to the above and operate the robot 12 to move the end effector 28 (i.e., the origin of the tool coordinate system C2) to the position data PD. 2_2 The position and orientation shown (i.e., coordinates Q2 in robot coordinate system C1) R (X2 R ,Y2 R ,Z2 R ,W2 R ,P2 R ,R2 R The end effector 28 is then configured to perform the operation on the workpiece W2.

[0086] In step S6, the processor 32 determines whether it has completed the work on all the workpieces W in container B. If the processor 32 determines it is YES, it terminates the flow shown in Figure 4; otherwise, it returns to step S1. In this way, the processor 32 repeatedly executes the loop of steps S1 to S6 until it determines it is YES in step S6, acquires the newly detected detection data DD from the shape detection sensor 14 in step S1, and executes steps S2 to S5 based on the new detection data DD.

[0087] As described above, in this embodiment, the processor 32 functions as a position data acquisition unit 44 and a position data cancellation unit 46, and based on the detection data DD of the shape detection sensor 14, the workpiece W n Location data PD 1_n , PD 2_n Therefore, the position data acquisition unit 44 and the position data cancellation unit 46 acquire the workpiece W based on the detection data DD of the shape detection sensor 14. n Location data PD 1_n , PD 2_n The device 50 (Figure 2) for acquiring the data is configured.

[0088] In this device 50, the position data acquisition unit 44 determines a work model WM in which a verification area VE corresponding to an area inaccessible to an object (for example, another work W2) is defined, and the work W included in the detected data DD n Shape data SD n By matching with the workpiece W n Location data PD 1_n , PD 2_n Obtain (steps S13, S14).

[0089] Then, the position data acquisition unit 44 is located at the workpiece W n Shape data SD n The shape data SD of the object included in the detected data DD is entered into the verification area VE defined in the work model that matches the data. mIf (for example, the shape data SD2 of workpiece W2 in Figure 10) exists (if YES is determined in step S23), the position data acquisition unit 44 acquires the position data PD. 2_n Disable (step S24).

[0090] According to this configuration, the position data PD obtained as a result of the inconsistency shown in Figure 10 2_n Using work W n This avoids having the robot 12 perform the task (step S5). As a result, the accuracy of the work performed by the robot 12 can be improved.

[0091] Furthermore, in the device 50, the position data cancellation unit 46 controls the shape data SD n Shape data SD of objects within the verification area VE defined in the work model WM that matches m Data volume DA n The calculated data amount DA is calculated (step S22). n Based on this, the shape data SD is used in the verification region VE. m It is determined whether or not it exists (step S23). With this configuration, when appropriate matching is performed, the amount of data DA that cannot exist in the verification area VE is determined. n This allows for the creation of a work model (WM) and shape data (SD). n This allows for quantitative verification of inconsistencies. As a result, the presence or absence of such inconsistencies can be effectively detected.

[0092] Furthermore, in the device 50, the position data cancellation unit 46 calculates the data amount DA in step S22. n And the data volume DA of the work model WM w By performing a predetermined calculation (specifically, the calculation in Equation 1 above) using the above, the shape data SD of the object is stored in the verification region VE. m Determine whether or not it exists (step S23).

[0093] According to this configuration, the work model WM and shape data SD are generated by a predetermined calculation (the calculation in Equation 1). nA parameter that quantitatively represents the degree of inconsistency (for example, the inconsistency parameter ε) n , or data volume DA n The inconsistency can be verified using the parameter, thereby enabling more accurate detection of the presence or absence of the inconsistency.

[0094] Next, with reference to Figure 12, another form of step S3 in Figure 4 will be described. In step S3 shown in Figure 12, after step S21 or S26, in step S31, the processor 32 processes the workpiece W n Location data PD 2_n Regarding this, in step S22, the data volume DA n Determine whether or not to calculate it.

[0095] For example, processor 32 is a worker W n Location data PD 2_n Based on the associated detection result parameter PM, in step S22, the data volume DA n Determine whether or not it is necessary to calculate the position data PD. 2_n The detection result parameter PM obtained is correlated with the probability of the aforementioned inconsistency occurring. Specifically, the degree of agreement β n , plane proportion γ n , or exposure rate δ n If the β is high, the probability of mismatch may be low, while the degree of agreement β n , plane proportion γ n , or exposure rate δ n If the value is significantly low, the probability of inconsistency may increase.

[0096] Therefore, in step S31, the processor 32 processes the position data PD 2_n The degree of agreement β is stored in the detection result data DT in association with it. n , plane proportion γ n , or exposure rate δ n If the amount exceeds a predetermined threshold, the following step S22 is executed to calculate the data amount DA n It is determined that there is no need to calculate (i.e., NO). On the other hand, the processor 32 determines the degree of agreement βn , plane proportion γ n , or exposure rate δ n If the amount does not exceed a predetermined threshold, the following step S22 is executed to obtain the data amount DA n It is determined that it is necessary to calculate (i.e., YES).

[0097] As another example, in step S31, the processor 32 controls the work W n Location data PD 2_n Based on at least one of the position and orientation of the work model WM matched during acquisition, the amount of data DA is calculated in step S22. n It is determined whether or not it is necessary to calculate the workpiece W that is reflected in the detected data DD. n Shape data SD n The probability of the aforementioned mismatch occurring may differ depending on the position or orientation of the work model WM that is matched to it.

[0098] For example, in the detection data DD, the shape data SD located in close proximity to the shape data of container A (i.e., the point cloud representing the visual features of container A) n When matching the work model WM, the probability of mismatch may increase. Alternatively, in the detected data DD, the shape data SD located at a position further away in the positive z-axis direction of the sensor coordinate system C3 (or the negative z-axis direction of the robot coordinate system C1) (i.e., closer to the bottom surface of container A) may be affected. n When matching the work model (WM) to this, the probability of inconsistencies occurring may increase.

[0099] Therefore, in this embodiment, the processor 32 processes the position data PD 2_n (That is, coordinate Q2 R (X2 R ,Y2 R ,Z2 R ,W2 R ,P2 R ,R2 R If the coordinates are within the predetermined coordinate range QR, then step S22 is executed to obtain the data amount DA. nIt is determined that it is necessary to calculate (i.e., YES). As an example, this coordinate range QR can be defined as a range of a predetermined distance from the coordinates of the shape data of container A included in the detected data DD in the robot coordinate system C1.

[0100] As another example, the coordinate range QR is obtained from the acquired location data PD. 2_n : Coordinate Q2 R (X2 R ,Y2 R ,Z2 R ,W2 R ,P2 R ,R2 R ) of which, z coordinate: Z2 R In contrast, Z2 R ≤Z th The range may be defined as the threshold Z that defines this coordinate range QR. th For example, work W included in the detection data DD n Shape data SD n The coordinates of the points in the point cloud may be defined as points located at the end of the robot coordinate system C1 in the positive z-axis direction, and then separated by a predetermined distance in the negative z-axis direction of the robot coordinate system C1.

[0101] The processor 32 receives location data PD 2_n Coordinates Q2 R (X2 R ,Y2 R ,Z2 R ,W2 R ,P2 R ,R2 R If the coordinate range QR is within the coordinate range, then the following step S22 is performed to obtain the data amount DA. n It is determined that it is necessary to calculate (i.e., YES).

[0102] Furthermore, when matching the work model WM of the first orientation shown in Figure 7, the probability of mismatch may be reduced, while when matching the work model WM of the second orientation shown in Figure 8, the probability of mismatch may be increased. Therefore, in this embodiment, the processor 32 processes the position data PD 2_nIf the orientation of the work model WM matched during acquisition is a predetermined reference orientation (for example, the second orientation in Figure 8), then step S22 is executed to obtain the data amount DA. n It is determined that it is necessary to calculate (i.e., YES).

[0103] On the other hand, if the orientation of the matched work model WM is not the reference orientation, the processor 32 executes the following step S22 to obtain the data amount DA n It is determined that there is no need to calculate (i.e., NO). Thus, in step S31, the processor 32 determines the data amount DA based on the detection result parameter PM indicating the matching result, or the orientation of the matched work model WM. n Determine whether or not to calculate it.

[0104] If the processor 32 determines that the result is YES, it proceeds to step S22; otherwise, it proceeds to step S32. In step S31, the processor 50 processes the position data PD. 2_n Coordinates Q2 R The processor 50 may determine whether the coordinate Q2 is within the coordinate range QR, and whether the orientation of the matched work model WM is the reference orientation. Then, the processor 50 determines the coordinate Q2 R The result may be determined as YES if the coordinate range QR is within the coordinate range and the orientation of the work model WM is the reference orientation.

[0105] If NO is determined in step S31 or S23, in step S32, the processor 32 will determine the work W n Shape data SD n This is deleted from the detection data DD. For example, if the above number "n" is set to n=1, the position data PD of workpiece W1 is deleted. 2_1 Suppose the result is NO in step S31 or S23. In this case, the processor 32 deletes the point cloud of the shape data SD1 of the workpiece W1 from the detection data DD shown in Figure 9, which was acquired in the most recent step S1. The detection data DD from which the shape data SD1 has been deleted is schematically shown in Figure 13.

[0106] Subsequently, in step S26, the processor 32 increments the number "n" to "2" and then generates the position data PD of the next workpiece W2. 2_2 Steps S31, S22-S24 are executed for this. For example, the position data PD of workpiece W2. 2_2 Assume that the result in step S31 is YES, and steps S22 and S23 are executed.

[0107] In this case, the processor 32 executes step S22 based on the detection data DD, from which the shape data SD1 has been erased, as shown in Figure 13, and the position data PD of the workpiece W2. 2_2 When acquiring the work model WM that was matched, another work W adjacent to the work W2 within the verification area VE defined in the work model WM m Shape data SD (not shown) m The amount of data DA2 is calculated. Then, in step S23, the processor 32 calculates the position data PD of the workpiece W2 based on the amount of data DA2. 2_2 When obtaining the matching work model WM, the verification area VE is set to work W m Shape data SD m Determine whether or not it exists.

[0108] As described above, in the configuration shown in Figure 12, the position data cancellation unit 46 cancels the data amount DA based on the detected result parameter PM or the orientation of the matched work model WM. n It is determined whether or not to calculate (step S31). According to this configuration, the location data PD is estimated to have a low probability of the above-mentioned inconsistency occurring. 2_n Regarding this, steps S22 to S24 can be omitted, thus reducing the cycle time of step S3.

[0109] Furthermore, in the configuration shown in Figure 12, the position data acquisition unit 44 matches the work model WM with the shape data SD1 of a work W1 (a third work) that is different from work W2 and is included in the detected data DD, thereby obtaining the position data PD of the work W1. 1_1 , PD 2_1The position data cancellation unit 46 acquires the position data PD of the workpiece W1 from the position data acquisition unit 44 (step S1), and the position data cancellation unit 46 acquires the position data PD of the workpiece W1 from the position data acquisition unit 44. 1_1 , PD 2_1 After obtaining the data, the shape data SD1 of the workpiece W1 is deleted from the detection data DD (step S32).

[0110] Then, the position data cancellation unit 46 deletes objects included in the detection data DD (for example, other work W) in the confirmation area VE defined in the work model WM that matches the shape data SD2 of work W2, where the shape data SD1 of work W1 has been erased. m ) Shape data SD m Determine whether or not it exists (step S23).

[0111] With this configuration, the shape data SD1, which does not require verification of inconsistencies (i.e., steps S22-S24), is deleted, and then the position data PD of the next workpiece W2 is processed. 2_2 During acquisition, it is possible to verify the inconsistency between the matched work model WM and the shape data SD2 of the workpiece W2. Therefore, the verification of this inconsistency can be performed with higher accuracy.

[0112] Alternatively, step S31 may be omitted from the flow shown in Figure 12, and the processor 32 may execute step S32 when it determines NO in step S23. Alternatively, step S32 may be omitted from the flow shown in Figure 12, and the processor 32 may proceed to step S25 when it determines NO in step S31 or S23.

[0113] Next, other functions of the robot system 10 will be described with reference to Figures 14 and 15. In this embodiment, the processor 32 executes the flow shown in Figure 15 in order to set the verification area VE described above to the work model WM before executing the flow in Figure 4. The flow shown in Figure 15 is started when the processor 32 receives a verification area setting command from the operator, the higher-level controller, or the computer program PG. In step S41, the processor 32 sets the work model WM (CAD model WM) C, or point cloud model WM P The data is obtained and displayed on the display device 40.

[0114] In step S42, the processor 32 determines whether or not it has received input IP1 specifying a verification area VE in the work model WM. Specifically, the operator determines whether or not the work model WM (CAD model WM) displayed on the display device 40 is the correct input. C , or point cloud model WM P While visualizing the output, the input device 42 is operated to provide the processor 32 with input IP1, which specifies the verification area VE in the work model WM.

[0115] As described above, each workpiece W stacked loosely in container B has an area (in this embodiment, a hole H) that other workpieces W (objects) cannot enter, and the verification area VE is set in the workpiece model WM as the area corresponding to the area that cannot be entered. The operator manually specifies the area of ​​the workpiece model WM that corresponds to the area that cannot be entered.

[0116] In this embodiment, the operator operates the input device 42 to provide the processor 32 with an input IP1 that designates the hole model HM of the work model WM as the inspection area VE. At this time, the processor 32 may change the orientation of the work model WM displayed on the display device 40 (i.e., the virtual line of sight direction VL from which the work model WM is viewed) in response to the input operation of the input device 42 by the operator.

[0117] If the processor 32 receives input IP1 through the input device 42, it determines it is YES and proceeds to step S43; otherwise, it proceeds to step S44. Thus, in this embodiment, it functions as a first input receiving unit 52 (Figure 14) that receives input IP1 specifying the confirmation area VE in the work model WM.

[0118] In step S43, the processor 32 sets a verification area VE in the work model WM according to the input IP1 received in the previous step S42. For example, if the input IP1 received in the previous step S42 specifies a hole model HM of the work model WM, the processor 32 sets the entire area of ​​the hole model HM as the verification area VE according to the input IP1, as shown in Figure 7.

[0119] The processor 32 stores the data of the set verification area VE in memory 34, as coordinate data of the work coordinate system C4, for example, and associates it with the model data of the work model WM. Thus, in this embodiment, the processor 32 functions as a verification area setting unit 54 (Figure 14) that sets the verification area VE in the work model WM according to the input IP1.

[0120] Furthermore, the processor 32 may generate multiple work model WMs for different orientations, as shown in Figures 7 and 8, based on the work model WM acquired in step S41, and set a verification area VE for each of the generated work model WMs for different orientations, according to the input IP1. As a result, as shown in Figures 7 and 8, a verification area VE will be set for each work model WM for each orientation.

[0121] The processor 32 stores the data of the verification area VE set for each posture of the work model WM in memory 34, associating it with the work model WM for each posture. Then, when step S2 in Figure 6 is executed, the processor 32 functions as a position data acquisition unit 44 and, in step S11, refers to the data of the verification area VE set for each posture of the work model WM and generates work model WMs for multiple postures, each with a verification area VE set. Then, in steps S13 and S14, the processor 32 uses the generated work model WMs for multiple postures to represent the work W in the detection data DD. n Shape data SD n Match each one to the others.

[0122] Again, referring to FIG. 15, in step S44, the processor 32 determines whether it has received an input IP2 that specifies the threshold value ε th (or the threshold value DA th ) for the confirmation area VE set in the work model WM at this time. Specifically, the operator operates the input device 42 to input the threshold value ε n (or the data amount DA n ) that is to be compared with the confirmation area VE set in the work model WM at this time, and that is used in step S23 above, to the processor 32. th (or the threshold value DA th ).

[0123] The processor 32 functions as a first input reception unit 52 and receives the input IP2 from the operator through the input device 42. If the processor 32 has received the input IP2, it determines YES and proceeds to step S45; if it determines NO, it proceeds to step S46.

[0124] In step S45, the processor 32 sets the threshold value ε th (or DA th ) for the confirmation area VE set in the work model WM according to the input IP2 received in the immediately previous step S44. The processor 32 stores the data of the set threshold value ε th (or DA th ) in the memory 34 in association with the model data of the work model WM and the data of the confirmation area VE.

[0125] In step S46, the processor 32 determines whether it has received an input IP3 for modifying or deleting the confirmation area VE set in the work model WM at this time. Specifically, the processor 32 displays the work model WM and the confirmation area VE set in the work model WM at this time on the display device 40.

[0126] The operator, while visually inspecting the work model WM and verification area VE displayed on the display device 40, operates the input device 42 to provide the processor 32 with an input IP3 to modify the verification area VE. At this time, the operator may, as the input IP3 to modify the verification area VE, operate the input device 42 to input the coordinate values ​​of the verification area VE in the work coordinate system C4, or may, by dragging and dropping the verification area VE in the work coordinate system C4, displace, expand, or shrink the verification area VE. Alternatively, the operator, while visually inspecting the work model WM and verification area VE displayed on the display device 40, operates the input device 42 to provide the processor 32 with an input IP3 to delete the set verification area VE.

[0127] If the processor 32 receives input IP3 through the input device 42, it determines that it is YES and proceeds to step S47; if it determines that it is NO, it proceeds to step S48. Thus, in this embodiment, the processor 32 functions as a second input receiving unit 56 (Figure 14) that receives input IP3 to modify or delete the verification area VE.

[0128] In step S47, the processor 32 updates the data of the verification area VE stored in memory 34 (for example, the coordinate data of the work coordinate system C4) by modifying or deleting the verification area VE that is set in the work model WM at this point, according to the input IP3 received in the previous step S46. Then, the processor 32 returns to step S42.

[0129] In step S48, the processor 32 determines whether it has received an input IP4 to approve the verification area VE set in the work model WM at this point. Specifically, the processor 32 displays the work model WM and the verification area VE set in the work model WM at this point, along with an approval button image for approving the verification area VE, on the display device 40.

[0130] The operator, while visually inspecting the work model WM and confirmation area VE displayed on the display device 40, operates the input device 42 to provide the processor 32 with input IP4 to operate the approval button image. The processor 32 functions as a second input receiving unit 56 (Figure 14) and receives input IP4 through the input device 42.

[0131] If the processor 32 accepts input IP4, it determines it to be YES and terminates the flow shown in Figure 15. If it determines it to be NO, it returns to step S42. In this way, the processor 32 repeatedly executes steps S42 to S47 until it determines it to be YES in step S48, setting, modifying, or deleting the verification area VE and the threshold ε. th (or DA th The settings and actions of the following will be executed sequentially.

[0132] According to the flow chart in Figure 15, the operator checks the verification area VE and threshold ε in the work model WM. th (or DA th ) can be set arbitrarily. After the flow in Figure 15, the processor 32 executes the flow in Figure 4 in response to the above-mentioned work start command. Then, the processor 32 executes steps S2 and S3 above based on the work model WM in which the verification area VE is set by the flow in Figure 15.

[0133] As described above, in this embodiment, the processor 32 functions as a position data acquisition unit 44, a position data cancellation unit 46, a first input reception unit 52, a confirmation area setting unit 54, and a second input reception unit 56. In the flow shown in Figure 15, the confirmation area VE is set on the work model WM, and in the flow shown in Figure 4, the work W is determined based on the detection data DD of the shape detection sensor 14. n Location data PD 1_n , PD 2_n They have obtained it.

[0134] Therefore, the position data acquisition unit 44, the position data cancellation unit 46, the first input reception unit 52, the confirmation area setting unit 54, and the second input reception unit 56 determine the workpiece W based on the detection data DD of the shape detection sensor 14. nLocation data PD 1_n , PD 2_n The device 60 (Figure 14) for acquiring the data is configured.

[0135] In this device 60, the first input receiving unit 52 receives an input IP1 that specifies a verification area VE in the work model WM (step S42), and the verification area setting unit 54 sets the verification area VE in the work model WM according to the input IP1 received by the first input receiving unit 52 (step S43).

[0136] With this configuration, the operator can consider areas in the workpiece W that other objects (e.g., other workpieces W) cannot enter, and then arbitrarily set a verification area VE corresponding to these inaccessible areas in the workpiece model WM. As a result, the verification area VE that is the focus of the verification of inconsistencies in step S3 described above can be set appropriately.

[0137] Furthermore, in the apparatus 60, when step S2 described above is executed, the position data acquisition unit 44 generates multiple work model WMs for different orientations in order to perform matching (step S11), and the verification area setting unit 54 sets a verification area VE for each work model WM for different orientations (step S43). With this configuration, multiple work model WMs for different orientations are generated to obtain shape data SD. n When matching, the verification area VE can be appropriately set for the work model WM of each posture.

[0138] Furthermore, in the device 60, the second input receiving unit 56 receives an input IP3 to modify or delete the confirmation area VE set by the confirmation area setting unit 54, or an input IP4 to approve the confirmation area VE (step S46 or S48). With this configuration, the operator can modify, delete, or approve the set confirmation area VE as appropriate, thereby easily setting the desired confirmation area VE.

[0139] In step S42, the processor 32 receives multiple verification areas VE from the operator. jThe system may accept input specifying (j=1,2,3,···). For example, as shown in Figure 16, the operator may operate the input device 42 to specify three verification regions VE1, VE2, and VE3 for the hole model HM.

[0140] In the example shown in Figure 16, verification area VE1 is designated in one opening region of the hole model HM, and verification area VE2 is designated in the other opening region of the hole model HM. Verification area VE3 is designated in the central region of the hole model HM between verification area VE1 and verification area VE2. In step S42, the processor 32 may receive an input IP1 specifying these verification areas VE1, VE2, and VE3, and in step S43, set verification areas VE1, VE2, and VE3 in the hole model HM.

[0141] In this case, in step S44, the operator sets different thresholds ε for the verification regions VE1, VE2, and VE3, respectively. th_m (or DA th_m ) may be specified. For example, the operator operates the input device 42 to set a threshold ε for the verification region VE1. th_1 Set the threshold ε for the verification region VE2. th_2 Set the threshold ε for the verification region VE3. th_3 Set the threshold ε. th_3 The threshold ε th_1 and ε th_2 It may be set to a lower value.

[0142] In this case, in step S22 described above, the processor 32 processes the shape data SD in the verification area VE1. m Data volume DA n_1 And, shape data SD within the verification area VE2 m Data volume DA n_2 And, shape data SD within the verification area VE3 m Data volume DA n_3 Calculate and respectively.

[0143] Next, in step S23, the processor 32 calculates the mismatch parameter ε of the verification region VE1 by the calculation of equation 1 described above. n_1 (=DA n_1 / DA w ) and the mismatch parameter ε of the verification region VE2 n_2 (=DA n_2 / DA w ) and the mismatch parameter ε of the verification region VE3 n_3 (=DA n_3 / DA w ) and are determined respectively. Then, the processor 32 calculates the determined mismatch parameter ε n_1 , ε n_2 and ε n_3 And the threshold ε set in step S45 th_1 , ε th_2 and ε th_3 We compare them, and ε n_1 ≧ε th_1 , ε n_2 ≧ε th_2 , or ε n_3 ≧ε th_3 In that case, you may determine the answer to be YES.

[0144] Here, the work model WM with multiple verification regions VE1, VE2, and VE3 as shown in Figure 16 is set to shape data SD n When matched, the verification regions VE1 and VE2, which are close to the opening of the hole model HM, will have other shape data SD. m It is presumed that point clouds are more likely to exist. On the other hand, the verification region VE3 set in the central region of the hole model HM contains shape data SD. m It is presumed that such a point cloud is unlikely to exist.

[0145] Therefore, as described above, the threshold ε th_3 The threshold ε th_1 and ε th_2 By setting it to a lower value, the work model WM and shape data SD n The inconsistency can be verified in detail for each verification region VE1, VE2, and VE3. The threshold ε th_1 and ε th_2 These can be set to the same value.

[0146] In the above-described embodiment, the case where the region where other objects (for example, other workpieces W) cannot enter in the workpiece W is the hole H has been described. However, the present invention is not limited to this, and the solid portion of the workpiece W (that is, the solid portion of the shaft portion S or the flange portion F) can also be a region where other objects (for example, other workpieces W) cannot enter. Therefore, in step S42 described above, the operator may set the solid portion of the shaft portion model SM or the flange portion model FM as the confirmation region VE in the workpiece model WM as the region corresponding to the inaccessible region.

[0147] Alternatively, in step S42, the operator may set the hole model HM of the workpiece model WM as the confirmation region VE1 and set the solid portion of the workpiece model WM as the confirmation region VE2. In this case, in step S44, the operator sets different threshold values ε th_1 and ε th_2 (or the threshold value DA th_1 and DA th_2 ) for the confirmation region VE1 set in the hole model HM and the confirmation region VE2 set in the solid portion of the workpiece model WM, respectively.

[0148] Note that step S42 may be omitted from the flow shown in FIG. 15. In this case, in step S43, the processor 32 generates workpiece models WM in a plurality of poses as shown in FIGS. 7 and 8 based on the workpiece model WM acquired in step S41, and analyzes these workpiece models WM, so that the confirmation region VE may be automatically set for each workpiece model WM in each pose. In this case, the first input receiving unit 52 can be omitted from the apparatus 60.

[0149] Hereinafter, a method for automatically setting the confirmation region VE will be described. Specifically, the processor 50 omits step S42 in FIG. 15, functions as the confirmation region setting unit 54 in step S43, and automatically sets the confirmation region VE for the workpiece model WM based on the workpiece model WM.

[0150] As an example, the processor 50 sets an envelope shape model that covers the work model WM from the outside. The processor 50 may then automatically set a region located inside the work model WM at a predetermined distance from the boundary between the set envelope shape model and the work model WM as the verification region VE.

[0151] As another example, in step S43, the processor 50 analyzes the work model WM and extracts the solid portion of the work model WM (the solid portion of the shaft S or flange F). The processor 50 may then automatically set the extracted solid portion as the verification area VE.

[0152] As yet another example, in step S43, the processor 50 places a virtual surface model (a model of a virtual surface of a plane or curved surface) on the work model WM in a virtual three-dimensional space defined by the work coordinate system C4. The processor 50 then extracts a region (for example, a hole model HM) that is closed off by the virtual surface model and changes into a solid region. The processor 50 may automatically set the extracted region as a verification region VE. As described above, the processor 50 functions as a verification region setting unit 54 and automatically sets the verification region VE on the work model WM based on the work model WM.

[0153] Note that step S44 may be omitted from the flow shown in Figure 15. In this case, in step S45, the processor 32 sets a threshold ε for the verification area VE. th (or threshold DA) th ) may be set automatically, for example, as a predetermined constant. Alternatively, steps S46 to S48 may be omitted from the flow shown in Figure 15. In this case, the second input receiving unit 56 can be omitted from the device 60.

[0154] Next, other functions of the robot system 10 will be described with reference to Figures 17 and 18. In this embodiment, the processor 32 executes the flow shown in Figure 18 instead of the flow shown in Figure 15. In the flow shown in Figure 18, the same step numbers are used for processes similar to those in Figure 15, and redundant explanations are omitted.

[0155] In the flow shown in Figure 18, after step S41, in step S51, the processor 32 performs a simulation SL in which multiple work models WM come into contact with each other. Specifically, the processor 32 first quantizes the work model WM acquired in step S41 into voxels VX, which are cubes of unit volume, as shown in Figure 19. As a result, the solid parts of the work model WM (specifically, the shaft model SM and the flange model FM) are converted into voxels VX. S The quantized work model WM and hole model HM are voxels VX H It is quantized to this extent.

[0156] Next, as shown in Figure 20, the processor 32 repeatedly performs simulation SL in which the quantized work model WM2 is brought into contact with the quantized work model WM1 in various orientations and from various directions. In other words, in this embodiment, the processor 32 functions as a simulation execution unit 58 (Figure 17) that performs simulation SL in which the work model WM2 is brought into contact with the work model WM1.

[0157] In step S52, the processor 32 functions as a verification area setting unit 54 and sets a verification area VE in the work model WM based on the simulation results of step S51. Specifically, when the simulation SL was executed once in step S51, the processor 32 sets a verification area VX for one voxel VX in the hole model HM of the work model WM1. H in the solid portion of the work model WM2, at least one voxel VX S If it enters, the one voxel VX H Increment the score SC by "1".

[0158] This score SC is for the voxel VX of work model WM2 in one simulation SL of step S51. S However, one voxel VX of work model WM1 H This represents the number of times it entered the zone. Therefore, Voxel VX H The initial value of the score SC is SC=0, and in one simulation in step S51, the voxel VX of the work model WM2 S However, one voxel VX of work model WM1 H Upon entering, the one voxel VX H The score SC is incremented by "1", so SC=1.

[0159] Each time the processor 32 runs the simulation SL in step S51, it processes each voxel VX of the work model WM1. H The score SC is calculated. Then, when all simulation SL of step S51 is completed, the processor 32 calculates the score SC for each voxel VX of the work model WM1. H Substitute the score SC into the following equation 2, and for each voxel VX H The score judgment value η is calculated for this. η = SC / i ···(Equation 2)

[0160] Here, i in Equation 2 represents the number of simulation SLs (in other words, the number of work model WM2s in various orientations used in simulation SC). Then, the processor 32 functions as a verification area setting unit 54 and calculates each voxel VX obtained by the calculation in Equation 2. H The score judgment value η is a predetermined threshold η th Determine whether the following is true: η≦η th Voxel VX that meets the requirements H Set this to the verification area VE. As a result, the voxel VX of the hole model HM shown in Figure 19 H At least one (for example, all of them) will be set as the verification area VE.

[0161] Note that equation 2 above can also be modified to the equation η = SC / i × 100. Furthermore, the processor 32 considers the above score SC to be a predetermined threshold SC. th Determine whether the following is true: SC ≤ SC th Voxel VX that meets the requirements H This may be set as the verification area VE. In this way, in step S52, the processor 32 functions as a verification area setting unit 54 and sets the verification area VE in the work model WM based on the simulation results of step S51 (specifically, the score SC and the score judgment value η).

[0162] After step S52, the processor 32 executes steps S42 to S48 described above, and based on the verification region VE set based on the simulation results, it specifies a further verification region VE (step S42), or sets a threshold ε for the verification region VE. th or DA th The process involves setting (step S45), modifying or deleting the verification area VE (step S47), or granting approval to the verification area VE (step S48).

[0163] As described above, in this embodiment, the processor 32 functions as a position data acquisition unit 44, a position data cancellation unit 46, a first input reception unit 52, a confirmation area setting unit 54, a second input reception unit 56, and a simulation execution unit 58. After setting the confirmation area VE in the work model WM in the flow shown in Figure 18, the flow shown in Figure 4 is executed to determine the work W based on the detection data DD of the shape detection sensor 14. n Location data PD 1_n , PD 2_n Obtain it.

[0164] Therefore, the position data acquisition unit 44, the position data cancellation unit 46, the first input reception unit 52, the confirmation area setting unit 54, the second input reception unit 56, and the simulation execution unit 58 determine the workpiece W based on the detection data DD of the shape detection sensor 14. n Location data PD 1_n , PD 2_n The device 70 (Figure 16) is configured to acquire the data.

[0165] In this apparatus 70, the simulation execution unit 58 performs a simulation SL in which the work model WM2 (object model) comes into contact with the work model WM1 (step S51), and the verification area setting unit 54 sets the verification area VE on the work model WM based on the simulation results (score SC, score judgment value η) performed by the simulation execution unit 58. With this configuration, the verification area VE on the work model WM can be set with high accuracy and automatically.

[0166] In this embodiment, the case described is when a simulation SL is performed in step S51 in which two work models WM1 and WM2 are brought into contact with each other. However, the processor 32 is not limited to this, and in step S51, it may perform simulation SL with three or more work models WM1 and WM2. k You can also run a simulation SL where (k=1,2,3,...) are brought into contact with each other.

[0167] Alternatively, in step S51, the processor 32 places multiple work models WM within the container model BM, which models the container B shown in Figure 1. k The materials are stacked randomly to simulate a bulk stacking, and multiple work models (WM) are created from these random stacks. k You may run a simulation SL that repeatedly changes the arrangement and orientation of the objects.

[0168] The processor 32 may execute the flow shown in Figure 4, Figure 15, or Figure 18 according to a computer program PG pre-stored in memory 34. Furthermore, the functions of devices 50, 60, or 70 executed by the processor 32 (i.e., the position data acquisition unit 44, the position data cancellation unit 46, the first input receiving unit 52, the confirmation area setting unit 54, the second input receiving unit 56, and the simulation execution unit 58) may be functional modules realized by the computer program PG. This computer program PG may be provided in the form of a computer-readable non-temporary recording medium, such as a semiconductor memory, magnetic recording medium, or optical recording medium.

[0169] Furthermore, the above-described embodiment mentions the case where the functions of the devices 50, 60, or 70 (i.e., the position data acquisition unit 44, the position data cancellation unit 46, the first input reception unit 52, the confirmation area setting unit 54, the second input reception unit 56, and the simulation execution unit 58) are implemented in the control device 16.

[0170] However, the functions of devices 50, 60, or 70 are not limited to these, and may be implemented in, for example, a teaching device (teaching pendant, tablet-type mobile terminal, etc.) that teaches the robot 12 the actions for the task of step S5, or in any other computer such as a PC. In this case, the processor of the teaching device or the computer such as a PC functions as device 50, 60, or 70.

[0171] In the embodiment described above, the case where the processor 32 executes step S3 after step S2 (more specifically, step S15) in the flow chart of Figure 4 was described. However, the processor 32 may also execute step S3 after step S13 or S14 in Figure 6.

[0172] For example, if step S3 is executed after step S13, the processor 32, in step S22 of step S3 (Figure 11 or Figure 12), the workpiece W n Initial position data PD 1_n Shape data SD within the verification area VE defined in the matched work model WM. m Data volume DA n The processor calculates the work W obtained in step S13. Then, if the processor 32 determines YES in step S23, in step S24, it calculates the work W obtained in step S13. n Initial position data PD 1_n You may disable (for example, delete) it.

[0173] The coarse search RS in step S13 and the dense search PR in step S14 described above are merely examples of methods for obtaining the position data PD of the workpiece W. Any modifications may be made to the coarse search RS or dense search PR, or the position data PD of the workpiece W may be obtained by any other method that does not involve performing the coarse search RS and dense search PR.

[0174] Furthermore, various modifications can be made to the flows shown in Figures 4, 6, 11, 12, 15, or 18. For example, steps S12 or S15 may be omitted from the flow shown in Figure 6. Alternatively, step S22 may be omitted from step S3 (Figures 11 and 12) as described above, and in step S23, the processor 32 will check the verification area VE of the matched work model WM and the shape data SD. m Based on the coordinates of the sensor coordinate system C3 and the point cloud, shape data SD is entered into the verification region VE. m You may also determine whether or not it exists.

[0175] In the above embodiment, we described the case where an object adjacent to one workpiece W (for example, workpiece W1) is a second workpiece W (workpiece W2) that is different from the first workpiece W. However, the invention is not limited to this, and an object adjacent to one workpiece W may be, for example, a jig placed inside a container B, or a protrusion protruding from the inner wall of the container B. In this case, for example, if an object such as a jig or protrusion cannot enter a hole H in the workpiece W, the hole model HM of the workpiece model WM corresponding to the hole H may be defined as the verification area VE described above. Furthermore, the workpiece W shown in Figure 3 is just an example, and may have any other shape.

[0176] In the above-described embodiment, the shape detection sensor 14 is fixed to the end effector 28 (or wrist flange 26b) and moved by the robot 12. However, the shape detection sensor 14 is not limited to this and may be fixed at a known position in the robot coordinate system C1, for example, using a support structure. Furthermore, the shape detection sensor 14 is not limited to a 3D vision sensor, but may also be a laser scanner capable of detecting the 3D shape of an object.

[0177] Alternatively, the shape detection sensor 14 may include a two-dimensional camera and a distance measuring sensor capable of measuring the distance d to an object. Furthermore, the detection data DD detected by the shape detection sensor 14 is not limited to three-dimensional point cloud image data as shown in Figure 5, but may include a dataset of two-dimensional image data captured by the two-dimensional camera and the distance d measured by the distance measuring sensor, or it may be any other type of image data (for example, distance image data).

[0178] Furthermore, the robot 12 is not limited to a vertical articulated robot, but may be any type of robot, such as a horizontal articulated robot or a parallel link robot. The present disclosure has been explained through the embodiments described above, but the embodiments described above do not limit the invention to the scope of the claims. [Explanation of symbols]

[0179] 10 Robot Systems 12 Robots 14 Shape detection sensors 16 Control device 32 processors 44 Location data acquisition unit 46 Location data cancellation unit 50,60,70 equipment 52 First Input Reception Unit 54 Confirmation Area Setting Section 56 Second Input Reception Unit 58 Simulation Execution Unit

Claims

1. A device for acquiring position data of a workpiece based on detection data from a shape detection sensor that detects the shape of an object and a workpiece adjacent to the object and having an area inaccessible to the object, A work model that models the aforementioned work, wherein a confirmation area corresponding to the inaccessible area is defined in the work model, is matched with the shape data of the work included in the detection data to acquire the position data; The apparatus comprises a position data cancellation unit that invalidates the position data acquired by the position data acquisition unit when the position data acquisition unit detects that the shape data of the object included in the detection data exists in the confirmation area defined in the work model which has been matched to the shape data of the workpiece by the position data acquisition unit.

2. The work model includes a first input receiving unit that receives input specifying the verification area, The apparatus according to claim 1, further comprising: a confirmation area setting unit that sets the confirmation area in the work model in accordance with the input received by the first input receiving unit.

3. The apparatus according to claim 1, further comprising a verification area setting unit that automatically sets the verification area in the work model based on the work model.

4. The system further includes a simulation execution unit that performs a simulation in which an object model, which is a model of the aforementioned object, is brought into contact with the work model. The apparatus according to claim 3, wherein the verification area setting unit sets the verification area in the work model based on the simulation results performed by the simulation execution unit.

5. The position data acquisition unit generates the work model in multiple orientations in order to perform the matching, The apparatus according to any one of claims 2 to 4, wherein the confirmation area setting unit sets the confirmation area for each of the work models in the respective orientations.

6. The apparatus according to any one of claims 2 to 4, further comprising a second input receiving unit that receives input for modifying, deleting, or approving the confirmation area set by the confirmation area setting unit.

7. The position data cancellation unit, The amount of shape data of the object within the verification area defined in the matched work model is calculated. The apparatus according to any one of claims 1 to 4, which determines whether or not shape data of the object exists in the confirmation area based on the calculated amount of data.

8. The apparatus according to claim 7, wherein the position data cancellation unit determines whether or not the shape data of the object exists in the confirmation area by performing a predetermined calculation using the calculated data amount and the data amount of the work model.

9. The apparatus according to claim 7, wherein the position data cancellation unit determines whether or not to calculate the amount of data based on the detection result parameter indicating the result of the matching, or the attitude or position of the matched work model.

10. The apparatus according to any one of claims 1 to 4, wherein the object is a second workpiece separate from the workpiece.

11. The position data acquisition unit acquires the position data of a third workpiece by matching the workpiece model with the shape data of a third workpiece, which is different from the workpiece and is included in the detected data. The position data cancellation unit, After the position data acquisition unit acquires the position data of the third workpiece, it deletes the shape data of the third workpiece from the detected data. The apparatus according to any one of claims 1 to 4, which determines whether or not the shape data of the object included in the detection data from which the shape data of the third workpiece has been erased exists in the confirmation area defined in the matched work model.

12. A robot control device comprising the apparatus described in any one of claims 1 to 4.

13. A robot that performs a predetermined task on a workpiece, A shape detection sensor for detecting the shape of the workpiece, A robot system comprising: a control device according to claim 12, which controls the robot to perform the predetermined task based on the detection data of the shape detection sensor.

14. A method for acquiring position data of a workpiece based on detection data from a shape detection sensor that detects the shape of an object and a workpiece adjacent to the object and having an area inaccessible to the object, The processor, The position data is obtained by matching the work model, which is a model of the workpiece and in which a verification area corresponding to the inaccessible area is defined, with the shape data of the workpiece included in the detection data. A method for invalidating acquired position data when the shape data of an object included in the detection data is present in the verification area defined in the work model that matches the shape data of the work.

15. A computer program that causes the processor to execute the method according to claim 14.

Images