Processing apparatus, processing system, and processing method

Abstract

A processing device (20) is provided with: a measurement point group display unit (21) for 3D-displaying a point group including an object on a screen; a region model display unit (22) for 3D-displaying the region model (M) on a screen; a positioning unit (23) that positions the region model by changing the position and orientation of the region model, thereby causing the point group of the object to be at least partially surrounded by the region model; and a setting unit (24) that sets, as a model of the object, a region within the screen surrounded by the positioned region model.

Description

Technical Field

[0001] The present invention relates to a processing device, a processing system and a processing method. Background Art

[0002] The technology of detecting a workpiece as an object using a three-dimensional camera and obtaining the position and posture of the workpiece is widely used. In order to detect the workpiece, it is necessary to teach a model of the workpiece, the position and posture of the workpiece as a reference, etc. in advance to a processing device that performs detection processing.

[0003] Regarding teaching of the model, there is a method of setting the model based on a point group of the workpiece obtained from an image captured by a camera (for example, Japanese Patent Application Laid-Open No. 2021-062416).

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2021-062416 Summary of the invention

[0007] Problems to be solved by the invention

[0008] However, in order to create a model based on the point group of the workpiece, complex calculations are required, which is complicated.

[0009] Therefore, a technology capable of easily creating a model of a workpiece based on a point group of the workpiece is desired.

[0010] Means for solving problems

[0011] According to a first embodiment of the present disclosure, there is provided a processing device comprising: a screen; a measurement point group display unit, which displays a point group including an object measured by a three-dimensional camera in 3D on the screen; a region model display unit, which displays a region model in 3D on the screen; a positioning unit, which changes the position and posture of the region model displayed by the region model display unit to position the region model, thereby causing the point group of the object to be at least partially surrounded by the region model; and a setting unit, which sets the region within the screen surrounded by the region model positioned by the positioning unit as the model of the object.

[0012] Furthermore, a processing system is provided, which includes the processing device according to the first aspect and the three-dimensional camera.

[0013] In addition, a processing method is provided, which makes a 3D point group including an object measured by a three-dimensional camera be displayed on a screen, makes a 3D area model be displayed on the screen, and changes the position and posture of the area model to locate the area model, thereby making the point group of the object at least partially surrounded by the area model, and setting the area within the screen surrounded by the located area model as the model of the object.

[0014] The objects, features and advantages of the present invention will become more apparent from the following description of the embodiments in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram of a system including the processing device in the first embodiment of the present invention.

[0016] Figure 2 It is a block diagram of a system including the processing device in the first embodiment of the present invention.

[0017] Figure 3 This is a flowchart showing the operation of the processing device in the first embodiment of the present invention.

[0018] Figure 4A It is a diagram showing a point group and the like of the workpiece in the first embodiment.

[0019] Figure 4B It is a diagram showing a point group and a region model of a workpiece in the first embodiment.

[0020] Figure 5A This is a schematic diagram of the regional model.

[0021] Figure 5B Yes Figure 5A An enlarged view of the entire regional model is shown.

[0022] Figure 5C Yes Figure 5A The area model shown is after the posture is changed.

[0023] Figure 5D Yes Figure 5A A magnified view of the regional model shown.

[0024] Fig. 6A is a diagram showing other area models.

[0025] Figure 6B This is a diagram showing yet another regional model.

[0026] Figure 6C This is a diagram showing a menu screen for selecting an area model.

[0027] Fig. 7AIt is a diagram showing other workpieces.

[0028] Figure 7B is a diagram showing yet another workpiece.

[0029] Figure 8 It is a diagram showing an area model etc. added in the second embodiment.

[0030] Fig. 9A It is a diagram showing an area model and the like in the third embodiment.

[0031] Fig. 9B Yes means Fig. 9A Figure of the workpiece shown.

[0032] Fig.10 This is a flowchart showing the operation of the processing device in the third embodiment of the present invention. DETAILED DESCRIPTION

[0033] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In all the drawings, corresponding components are denoted by common reference numerals.

[0034] Figure 1 1 is a schematic diagram of a system including a processing device in a first embodiment of the present invention. Figure 1 As shown, the system 1 mainly includes a robot 5 having an end effector 6, such as a multi-joint robot, a three-dimensional camera 9, and a teaching operation panel 30 having an input unit 32. In addition, a workpiece W as an object is placed on a worktable T arranged near the robot 5. The workpiece W has a three-dimensional shape. For example Figure 1 The workpiece W shown is a shape consisting of a central disk portion and a shaft portion penetrating the disk portion. The disk portion and the shaft portion may also be formed integrally. In addition, the shape of the workpiece W is not limited to this.

[0035] exist Figure 1 , a structure is shown in which the end effector 6 of the robot 5 grasps or processes the workpiece W. Alternatively, a structure including a three-dimensional scanner instead of the three-dimensional camera 9 may be used.

[0036] Figure 2 1 is a block diagram of a system including a processing device in the first embodiment of the present invention. The control device 10 of the system 1 is a computer, and includes a storage unit 11, such as ROM, RAM, etc., a CPU (Central Processing Unit) 12, and a bus that interconnects the storage unit 11 and the CPU 12. The storage unit 11 stores, in addition to images captured by the three-dimensional camera 9, the action program of the robot 5, the area model M described later, and various parameters.

[0037] exist Figure 1In the example, the input unit 32 is a part of the teaching operation panel 30, but the input unit 32 may be an external device such as a keyboard, a mouse, or a touch panel. In addition, the screen 31 may be a part of the teaching operation panel 30, or may be an independent screen 31 such as a CRT or a liquid crystal monitor. In addition, the structure including the three-dimensional camera 9, the storage unit 11, the processing device 20, the screen 31, and the input unit 32 is referred to as a processing system 7.

[0038] The CPU 12 functions as a robot control unit 29 for controlling the processing device 20 and the robot 5, which will be described in detail later. The processing device 20 includes a measurement point group display unit 21 for displaying a point group including an object (workpiece) W in a three-dimensional image captured by the three-dimensional camera 9 on a screen 31 in 3D; and a region model display unit 22 for displaying a region model M on the screen 31 in 3D (three-dimensional display). Furthermore, the processing device 20 includes a positioning unit 23 for changing the position and posture of the region model M displayed by the region model display unit 22 to position the region model M so that the point group of the object W is at least partially surrounded by the region model M; and a setting unit 24 for setting the region in the screen 31 surrounded by the region model M positioned by the positioning unit 23 as a model of the object W. Furthermore, the processing device 20 includes a color change unit 25 for changing the color of at least one of the point group of the object W and the region model M according to an input operation from the input unit 32.

[0039] The measurement point group display unit 21, the area model display unit 22, the positioning unit 23, the setting unit 24, and the color change unit 25 are, for example, functional modules implemented by a computer program executed on the processor 12. The computer program for executing the processing of the measurement point group display unit 21 to the color change unit 25 included in the processor 12 may be provided in the form of a computer-readable recording medium such as a semiconductor memory, a magnetic recording medium, or an optical recording medium.

[0040] In addition, the processing device 20 or the robot control unit 29 may also have a shooting control function of the three-dimensional camera 9, a setting function of the model M, MW, detection parameters (score threshold, reference position of the workpiece), etc., a screen display function for teaching the model M, MW to the robot 5, and a workpiece detection function using the taught model M, MW.

[0041] Figure 3 This is a flowchart showing the operation of the processing device in the first embodiment of the present invention. Figure 3 The contents shown are stored in advance in the storage unit 11 of the operation program of the processing device.

[0042] First, in step S1 , the three-dimensional camera 9 captures an image of the workpiece W placed on the table T. The three-dimensional image of the workpiece W is stored in the storage unit 11 . Next, in step S2 , the measurement point group display unit 21 displays a three-dimensional image including the point group of the workpiece W on the screen 31 . Figure 4A is a diagram showing a point group of a workpiece in the first embodiment. Figure 4A In FIG. 1 , the point group of the workpiece W is placed on the point group of the table T.

[0043] Next, in step S3 , the region model display unit 22 displays the region model M3D on the screen 31 in accordance with the operation of the operator. Figure 4B It is a diagram showing a point group and a region model of a workpiece in the first embodiment. Figure 4B The region model M shown is a rectangular parallelepiped frame displayed three-dimensionally. In addition, as described later, the region model M may also be in other shapes, such as a sphere, a capsule, etc.

[0044] The initial values ​​related to the position and posture of the region model M and the initial values ​​related to the size may be arbitrary values ​​determined according to the installation location and size of the workpiece W. Alternatively, the region model display unit 22 may display the region model M on the screen 31 after calculating the initial values ​​related to the position and posture of the region model M and the initial values ​​related to the size so that all point groups measured from the three-dimensional camera are included in the region model M. The following description will be made on the case where the region model M is a rectangular parallelepiped.

[0045] Figure 5A is a schematic diagram of the regional model. Figure 5A In FIG. 1 , each vertex A1 to A8 of the region model M and the center C0 of the region model M are highlighted by white circles. These white circles are referred to as operating points. In addition, the illustration of operating points may be omitted for the purpose of simplicity.

[0046] Next, in step S4 , the operator operates the input unit 32 to position the region model M so that the point group of the workpiece W is at least partially surrounded by the region model M. This operation is performed by the positioning unit 23 .

[0047] Figure 5B to Figure 5D Yes means Figure 5A Figure 1 shows the regional model shown. For example, Figure 5B In the example, the operator operates the input unit 32, and while specifying the operation point A2, performs a drag operation in the diagonal direction of one side of the region model M. As a result, the position of the vertex A2 is changed, thereby expanding the entire region model M. The region model M can also be reduced by the same operation.

[0048] Moreover, in Figure 5CIn the example, the operator operates the input unit 32 to perform a drag operation while specifying the operation point C0 to cause the region model to be translated or rotated. Thus, the region model M is translated or rotated around the center C0 according to the drag operation.

[0049] Regarding the operation point C0, when the operation point C0 is specified, a menu related to whether to translate or rotate the region model M may be displayed on the screen 31. Similarly, when the vertex A2 is specified, a similar menu screen related to whether to enlarge or reduce or rotate the region model M may be displayed on the screen 31. Alternatively, when the vertex A2 is specified, Figure 5C When one of the arrows indicating the XYZ directions is in the direction shown, the region model M is rotated about the designated direction. In this way, the action related to the region model M can be changed according to the designated position of the region model M.

[0050] And, in Figure 5D In FIG. 1 , an operation point C1 is arranged at the center of one surface of the region model M. The operator operates the input unit 32 and, while specifying the operation point C1, performs a drag operation in the depth direction (the normal direction of the surface where the operation point C1 is arranged) of the region model M. Thus, the region model M is partially enlarged or reduced.

[0051] In addition, when the region model M is in other shapes, such as a sphere, Fig. 6A As shown, the operation point C0' located at the center of the sphere can be operated to change the position, or the operation point C1' located at the edge of the sphere can be operated to enlarge or reduce the region model M in the same manner as described above.

[0052] Or, if Figure 6B As shown, when the regional model M is a capsule type (a combination of a cylinder and two hemispheres respectively connected to the end faces of the cylinder), the position and posture can be changed by operating the operating point C0" located at the center of the cylinder, or the operating point C2' located at the center of the end face of the cylinder or the operating point C3' located at the edge of the hemisphere can be operated to enlarge or reduce the regional model M in the same way as described above.

[0053] and, Figure 6C This is a diagram showing a menu screen for selecting an area model. Figure 6C The menu screen 39 shown can also be Figure 3 The screen 31 displays between step S2 and step S3. Then, the operator can select a region model M of a desired shape using the input unit 32. Thus, the region model M most suitable for the shape of the workpiece W can be selected, so that the operation described later can be performed more accurately.

[0054] In addition, numerical values ​​related to the position and posture of the region model M and numerical values ​​related to the size may be separately displayed on the screen 31. Furthermore, the operator may directly input numerical values ​​related to the position and posture of the region model M and numerical values ​​related to the size using the input unit 32 to change the position and posture and size of the region model M.

[0055] Operator through Figure 5B to Figure 5D The operation shown in FIG. 1 is to position the region model M in such a way that the point group of the workpiece W is at least partially surrounded by the region model M. As a result, Figure 4B As shown, the point group of the workpiece W is at least partially surrounded by the region model M. Preferably, the point group of the workpiece W is completely surrounded by the region model M. In addition, the region model M is preferably of a minimum size that at least partially surrounds the workpiece W.

[0056] Next, in step S5 , the region within the screen 31 surrounded by the located region model M is set as the model MW of the workpiece W. Therefore, in the present invention, a simple model MW can be easily created based on the point group of the workpiece W.

[0057] Next, based on the model MW, the position and posture of the workpiece W are taught to the robot 5 by a known method. Then, based on the taught position and posture of the workpiece W, the robot 5 can be operated according to the operation program of the robot 5. Alternatively, the model MW can be used to detect the workpiece W for other images captured by the three-dimensional camera 9.

[0058] in addition, Fig. 7A and Figure 7B It is a diagram showing other workpieces. Fig. 7A The workpiece W1 shown has a shape in which a large rectangular parallelepiped is arranged between a cylinder and a small rectangular parallelepiped. Figure 7B The workpiece W2 shown has a shape in which a large rectangular parallelepiped is arranged between a truncated cone and a small rectangular parallelepiped. In other words, the workpiece W1 and the workpiece W2 have a common portion composed of a small rectangular parallelepiped and a large rectangular parallelepiped.

[0059] In such a case, Fig. 7A as well as Figure 7B As shown, the region model M0 only surrounds the common part of the workpieces W1 and W2. That is, the region model M0 only partially surrounds the workpieces W1 and W2. In this case, the model MW0 of the workpiece W can be set with the common region model M0 for different workpieces W1 and W2. In other words, in other images in which a plurality of workpieces W1 and W2 are captured, the model MW0 of the workpieces W1 and W2 can be set with the common model MW0. Alternatively, the common model MW0 can be used to detect different workpieces W1 and W2 for another image captured by the three-dimensional camera 9.

[0060] Furthermore, the color changing unit 25 not only changes the color of at least one of the area model M and the point group of the object W, but can also change the type of lines constituting the area model M, such as solid lines, dotted lines, and single-point dash lines, according to the operation of the input unit 32 .

[0061] In the second embodiment of the present invention, the same processes S1 to S5 as those in the first embodiment are performed. Figure 3 In step S6, the added region model M' is displayed in 3D on the screen. Then, in step S7, the position and posture of the added region model M' are changed to position the added region model M'. Figure 8 FIG. 2 is a diagram showing an additional area model and the like in the second embodiment. Figure 8 In , the additional region model M' is positioned as a point group that at least partially surrounds the worktable T. In addition, in Figure 8 , the illustration of the region model M for the workpiece W is omitted for the purpose of simplicity.

[0062] Figure 8 The point group of the table T shown is a point group unnecessary for the point group at least partially surrounding the workpiece W by the region model M. In the second embodiment, such a point group of the table T is surrounded by an additional region model M', and then the region surrounded by the additional region model M' is masked.

[0063] In this way, by using the additional regional model M' to mask the point group that is not related to the workpiece W, such as the table T and the background, it is easy to set the model of the workpiece W. In addition, steps S6 and S7 may be performed first, and the unnecessary point group may be surrounded by the regional model M', and then the model WM of the workpiece W may be created by processing steps S1 to S5. It can be seen from this that it becomes easy and accurate to create the model WM of the workpiece W. In addition, multiple regional models M' may be used to mask multiple parts.

[0064] and, Fig.9A is a diagram showing a region model etc. in the third embodiment, Fig. 9B Yes means Fig.9A Figure of the workpiece etc. shown. And, Fig.10 This is a flowchart showing the operation of the processing device in the third embodiment of the present invention. Fig.10 Steps S1 to S3 are the same as those described above, so their further description is omitted.

[0065] Fig. 9AThe workpiece W3 shown is substantially X-shaped and is composed of two elongated portions. When such a workpiece W3 is set as a model, first, in step S4', the position and posture of the region model M1 are changed so that a part of the workpiece W3, for example, one elongated portion, is at least partially surrounded and the region model M1 is positioned.

[0066] Next, in step S5', the added region model M2 is displayed on the screen 31. Then, in step S6', the position and posture of the region model M2 are changed to position the region model M2 so that other parts of the workpiece W3, for example, the other elongated part, are at least partially surrounded.

[0067] Finally, in step S7', the region within the screen 31 surrounded by the positioned region model M1 and the additional region model M2 is set as the model MW3 of the workpiece W3. Of course, two or more region models may be used to surround the workpiece W. Thus, even a workpiece W3 of a complex shape can be easily set to its model MW3.

[0068] And, according to Fig. 9B It can be seen that sometimes foreign matter other than the workpiece is included in the image captured by the three-dimensional camera 9. Thus, even if there is foreign matter around the workpiece W3, by using Fig.10 The processing described above can also surround the workpiece W3 using the region models M1 and M2 so as not to include these foreign objects, thereby accurately setting the model MW3. Therefore, the third embodiment is advantageous for setting the model of a workpiece having a complex shape.

[0069] The embodiments of the present disclosure are described in detail, but the present disclosure is not limited to the above-mentioned embodiments. These embodiments can be variously added, replaced, changed, partially deleted, etc. without departing from the scope of the main purpose of the invention, or without departing from the scope of the invention and the idea and main purpose of the present invention derived from the contents recorded in the scope of the patent protection requested and its equivalents. For example, in the above-mentioned embodiments, the order of each action and the order of each processing are shown as an example, but are not limited to this. In addition, the same is true for the use of numerical values ​​or mathematical formulas in the description of the above-mentioned embodiments. Furthermore, several appropriate combinations of the aforementioned embodiments are included in the scope of the present disclosure.

[0070] Description of Reference Numerals

[0071] 1 System

[0072] 5. Robots

[0073] 7. Processing System

[0074] 10 Control device

[0075] 11 Storage

[0076] 20Processing device

[0077] 21 Measurement point group display

[0078] 22 Area Model Display Unit

[0079] 23 Positioning unit

[0080] 24 Setting Department

[0081] 25 Color Change Department

[0082] 30 Teaching Operation Panel

[0083] 31 pictures

[0084] 32 Input section

[0085] 39Menu screen

[0086] M, M', M0, M1, M2 regional models

[0087] Models for MW, MW0, MW3 artifacts

[0088] W, W1, W2, W3 workpieces.

Claims

1. A processing device, characterized in that: The processing device comprises: picture; a measured point group display unit for displaying a point group including an object measured by a three-dimensional camera in 3D on the screen; A region model display unit, which displays the region model in 3D on the screen; a positioning unit configured to change the position and posture of the region model displayed by the region model display unit to position the region model so that the point group of the object is at least partially surrounded by the region model; as well as A setting unit sets a region within the screen surrounded by the region model positioned by the positioning unit as a model of the object.

2. The processing device according to claim 1, characterized in that The region model is a cuboid, a sphere or a capsule.

3. The processing device according to claim 2, characterized in that The region model has at least one of an operation point for changing a size of the region model and an operation point for changing a position and posture of the region model.

4. The processing device according to any one of claims 1 to 3, characterized in that The processing device includes an input unit for inputting a position, posture, and size of the region model.

5. The processing device according to any one of claims 1 to 4, characterized in that The processing device includes a color changing unit that changes the color of at least one of the point group of the object and the area model.

6. The processing device according to claim 1, characterized in that The region model display unit displays the additional region model in 3D on the screen. The positioning unit changes the position and posture of the additional region model displayed by the region model display unit to position the additional region model, thereby shielding, by the additional region model, a point group that is unnecessary for a point group that at least partially surrounds the object using the region model.

7. The processing device according to claim 1, characterized in that The region model display unit displays the additional region model in 3D on the screen. The positioning unit changes the position and posture of the additional region model displayed by the region model display unit to position the additional region model, thereby at least partially surrounding the point group of the object by the region model and the additional region model.

8. A processing system, characterized in that: The processing system comprises: The processing device according to any one of claims 1 to 7; and The three-dimensional camera.

9. A processing method, characterized in that: Display the point group of the object measured by the 3D camera in 3D on the screen. The regional model is displayed in 3D on the screen. The region model is positioned by changing the position and posture of the region model, so that the point group of the object is at least partially surrounded by the region model. An area within the screen surrounded by the positioned area model is set as a model of the object.

10. The processing method according to claim 9, characterized in that: In the processing method, The additional area model is also displayed in 3D on the screen. The position and orientation of the additional region model are changed to position the additional region model, thereby shielding a point group that is unnecessary for a point group that at least partially surrounds the object using the region model with the additional region model.

11. The processing method according to claim 9, characterized in that: In the processing method, The additional area model is also displayed in 3D on the screen. The position and posture of the additional region model are changed to position the additional region model, thereby at least partially surrounding the point group of the object by the region model and the additional region model.

Images