Programming device

Abstract

A machining line that draws an arbitrary and complex trajectory is generated on a surface of a workpiece, and a program of an industrial machine for performing a machining operation of the trajectory on the basis of the generated machining line is generated. Provided is a programming device in which models of an industrial machine, a tool, a visual sensor, and a workpiece are respectively disposed in a virtual space, a program in which the model of the industrial machine operates on the model of the workpiece through the model of the tool is generated, and a label on which a contour line is drawn is disposed on the surface of the model of the workpiece. A captured image in which a label is captured at a position where a model of a vision sensor is disposed is generated on the basis of an imaging parameter of the model set in the vision sensor, a contour line is detected by performing image processing on the captured image, and a processing line is generated on the basis of the detected contour line. A normal vector of the surface of the model with respect to the workpiece on the generated machining line is calculated, and a program is generated on the basis of the machining line and the normal vector.

Description

Technical Field

[0001] The present disclosure relates to a programming device for generating a program for an industrial machine. Background Art

[0002] The following technology exists: a three-dimensional model of a robot equipped with a tool, a workpiece, and at least one peripheral machine is arranged on a screen and displayed simultaneously, a processing line is specified on the three-dimensional model of the workpiece, the action form, speed, position, and posture of the teaching point generated according to the specified processing line are specified, and an action program for the robot used to perform workpiece processing operations is generated according to the specified processing line and the specified action form, speed, position, and posture.

[0003] In addition, the following technology exists: in a robot system having a robot, a visual sensor, and a workpiece in a working space, a robot model of the robot, a visual sensor model of the visual sensor, and a workpiece model of the workpiece are configured and simulated in a virtual space that three-dimensionally represents the working space. The simulation measures the workpiece model through the visual sensor model, and the robot model performs operations on the workpiece model.

[0004] For example, as a method for specifying a machining line on a three-dimensional workpiece model, the following methods are known: a machining line is specified by detecting features on a shape consisting of contour lines or surfaces, such as basic shapes including circles and polygons, or shapes formed by combining multiple basic shapes, or by detecting portions where three-dimensional workpiece models meet during welding, etc. For example, see Patent Documents 1 and 2.

[0005] Alternatively, the following method is known: a three-dimensional shape containing a curved surface or a three-dimensional shape containing a plurality of continuous planes is filled with a motion pattern consisting of a continuous trajectory representing the periodic motion of a tool, the three-dimensional shape is arranged in a virtual space so that the motion pattern is projected onto at least one surface of a workpiece model, and a machining path for the tool is generated by projecting the motion pattern onto at least one surface of the workpiece model. For example, see Patent Document 3.

[0006] Prior art literature

[0007] Patent Literature

[0008] Patent Document 1: Japanese Patent Application Laid-Open No. 2017-140684

[0009] Patent Document 2: Japanese Patent Application Laid-Open No. 2019-48358

[0010] Patent Document 3: Japanese Patent Application Laid-Open No. 2013-248677 Summary of the Invention

[0011] Problems to be solved by the invention

[0012] However, the teaching of the motion program for the robot to perform a machining operation with a complex trajectory must be performed entirely manually. Therefore, the teaching operation requires a lot of labor and time, and requires the operator's expertise.

[0013] Therefore, it is desirable to generate a processing line that describes an arbitrary and complex trajectory on the surface of a workpiece, for example, and to generate a program for an industrial machine for performing a processing operation along the trajectory according to the generated processing line.

[0014] Means for solving problems

[0015] One embodiment of the programming device disclosed herein is a programming device that respectively configures models of an industrial machine, a tool installed on the industrial machine, a visual sensor, and a workpiece in a virtual space, and generates a program for the model of the industrial machine to operate on the model of the workpiece through the model of the tool. A label depicting a contour line is configured on the surface of the model of the workpiece, and a captured image of the label captured at a position of the model where the visual sensor is configured is generated based on shooting parameters set for the model of the visual sensor. The contour line is detected by performing image processing on the captured image, and a processing line is generated based on the detected contour line. The normal vector of the generated processing line relative to the surface of the model of the workpiece is calculated, and the program is generated based on the processing line and the normal vector.

[0016] One embodiment of the programming device disclosed herein is a programming device that generates a program for the industrial machine to operate on the workpiece through the tool in an industrial machine arranged in a real space, a tool installed on the industrial machine, a visual sensor, and a workpiece, arranges a label depicting a contour line on the surface of the workpiece, obtains an image of the label taken at a position where the visual sensor is arranged from the visual sensor according to shooting parameters set in the visual sensor, detects the contour line by performing image processing on the captured image, generates a processing line according to the detected contour line, calculates a normal vector on the generated processing line relative to the surface of the workpiece, and generates the program according to the processing line and the normal vector. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a functional block diagram showing a functional configuration example of the programming device according to the first embodiment.

[0018] Figure 2 This is a diagram showing an example of an industrial machinery system model in a virtual space.

[0019] Figure 3This is a diagram showing an example in which labels are arranged on the surface of a workpiece model.

[0020] Figure 4 This is a diagram showing an example of the positional relationship between the visual sensor model and the label.

[0021] Figure 5 FIG. 1 is a diagram showing an example of three-dimensional points detected on a contour line.

[0022] Figure 6 This is a diagram showing an example of a generated processing line.

[0023] Figure 7 This is a diagram showing an example of normal vectors with respect to the surface of the workpiece model calculated at each point on the generated machining line.

[0024] Figure 8 This is a diagram showing an example of set teaching points.

[0025] Figure 9 This is a diagram showing an example of simulation of a generated robot program.

[0026] Figure 10 This is a flowchart explaining the program generation processing of the programming device.

[0027] Figure 11 This is a functional block diagram showing a functional configuration example of a programming device according to the second embodiment.

[0028] Figure 12 This is a diagram showing an example of labels of contour lines that intersect at least one point.

[0029] Figure 13 This is a diagram showing an example of the positional relationship between the visual sensor model and the label.

[0030] Figure 14 This is a diagram showing an example of a plurality of line segments dividing a contour line.

[0031] Figure 15 This is a diagram showing an example of the result of combining a plurality of segmented line segments.

[0032] Figure 16 FIG. 1 is a diagram showing an example of three-dimensional points on a detected contour line.

[0033] Figure 17 This is a diagram showing an example of a generated processing line.

[0034] Figure 18 This is a diagram showing an example of normal vectors with respect to the surface of the workpiece model calculated at each point on the generated machining line.

[0035] Figure 19 This is a diagram showing an example of set teaching points.

[0036] Figure 20 This is a flowchart explaining the program generation processing of the programming device.

[0037] Figure 21 This is a functional block diagram showing a functional configuration example of a programming device according to a third embodiment.

[0038] Figure 22 This is a flowchart explaining the program generation processing of the programming device.

[0039] Figure 23 This is a diagram showing an example of a vision sensor that is fixedly arranged separately from a robot. DETAILED DESCRIPTION

[0040] <First embodiment>

[0041] First, let's briefly describe this embodiment. In this embodiment, a programming device places a label depicting a contour line on the surface of the workpiece model in a virtual space containing models of an industrial machine, a tool attached to the industrial machine, a vision sensor, and a workpiece. Based on imaging parameters set for the vision sensor model, the programming device generates an image of the label captured at the location on the model where the vision sensor is located. The programming device performs image processing on the captured image to detect the contour line, generates a machining line based on the detected contour line, calculates the normal vector of the generated machining line relative to the surface of the workpiece model, and generates a program based on the machining line and the normal vector for the industrial machine to operate the workpiece using the tool.

[0042] Thus, according to this embodiment, it is possible to generate a processing line that describes an arbitrary and complex trajectory on the surface of a workpiece, and to generate a program for an industrial machine for performing a processing operation along the trajectory based on the generated processing line.

[0043] The above is an overview of this embodiment.

[0044] Next, the structure of this embodiment will be described in detail using the accompanying drawings. This example illustrates a scenario where models of an industrial machine, including a robot, a tool, a vision sensor, and a workpiece, are arranged in a virtual space, and a robot program is generated for the robot to perform an operation on the workpiece using the tool. Furthermore, the present invention can also be applied to a variety of industrial machines, including machine tools, industrial robots, service robots, forging machines, laser processing machines, and injection molding machines.

[0045] Figure 1 This is a functional block diagram showing a functional configuration example of the programming device according to the first embodiment.

[0046] like Figure 1 As shown, the programming device 1 is a well-known computer and includes a control unit 10, an input unit 11, a display unit 12, and a storage unit 13. The control unit 10 includes a virtual space creation unit 101, a three-dimensional model configuration unit 102, an image configuration unit 103, an image capture unit 104, a contour line detection unit 105, a three-dimensional data conversion unit 106, a processing line generation unit 107, and a robot program generation unit 108.

[0047] Furthermore, the programming device 1 may be connected to a robot control device (not shown) that controls the movement of the robot (not shown) via a network (not shown), such as a LAN (Local Area Network) or the Internet. Alternatively, the programming device 1 may be directly connected to the robot control device (not shown) via a connection interface (not shown).

[0048] <Input Unit 11>

[0049] The input unit 11 is, for example, a keyboard or a touch panel arranged on the display unit 12 described later, and receives input from the user.

[0050] <Display Unit 12>

[0051] The display unit 12 is, for example, a liquid crystal display. As will be described later, the display unit 12 displays, for example, 3D CAD data representing a robot (not shown) in three dimensions (hereinafter also referred to as a "robot model") input (selected) by a user via the input unit 11, along with 3D CAD data representing a tool (not shown) attached to the robot in three dimensions (hereinafter also referred to as a "tool model"), 3D CAD data representing a vision sensor (not shown) in three dimensions (hereinafter also referred to as a "vision sensor model"), and 3D CAD data representing a workpiece (not shown) in three dimensions (hereinafter also referred to as a "workpiece model").

[0052] <Storage Unit 13>

[0053] The storage unit 13 is an SSD (Solid State Drive) or HDD (Hard Disk Drive). The storage unit 13 may also store a program for generating a robot program (not shown) for a robot to perform work on a workpiece using a tool, or a previously generated robot program. The storage unit 13 also includes a model storage unit 131.

[0054] As described above, the model storage unit 131 stores, for example, 3D CAD data (robot model) of a robot (not shown), 3D CAD data (tool model) of a tool (not shown), 3D CAD data (visual sensor model) of a visual sensor (not shown), and 3D CAD data (workpiece model) of a workpiece (not shown), which are input (selected) by the user via the input unit 11 and displayed on the display unit 12.

[0055] <Control Unit 10>

[0056] The control unit 10 has a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a CMOS (Complementary Metal-Oxide-Semiconductor) memory, etc., which are configured to be able to communicate with each other via a bus and are well known to those skilled in the art.

[0057] The CPU is a processor that controls the programming device 1 as a whole. The CPU reads the system program and application program stored in the ROM via the bus, and controls the programming device 1 as a whole according to the system program and application program. Figure 1 As shown, the control unit 10 is configured to implement the functions of a virtual space creation unit 101, a 3D model configuration unit 102, an image configuration unit 103, an image capture unit 104, a contour line detection unit 105, a 3D data conversion unit 106, a processing line generation unit 107, and a robot program generation unit 108. Various data, such as temporary calculation data and display data, are stored in the RAM. Furthermore, the CMOS memory is configured as a non-volatile memory, which is backed up by a battery (not shown) and maintains stored data even when the programming device 1 is powered off.

[0058] The virtual space creation unit 101 generates a virtual space that three-dimensionally represents a work space in which a robot (not shown), a tool (not shown), a visual sensor (not shown), and a workpiece (not shown) are arranged.

[0059] The three-dimensional model configuration unit 102 configures a robot model of a robot (not shown), a tool model of a tool (not shown), a visual sensor model of a visual sensor (not shown), and a workpiece model of a workpiece (not shown) in the three-dimensional virtual space generated by the virtual space production unit 101, for example, based on the input operation of the user's input unit 11.

[0060] Figure 2This is a diagram showing an example of an industrial machinery system model in a virtual space.

[0061] Specifically, if Figure 2 As shown, in order to arrange a robot (not shown) in the virtual space, the three-dimensional model arrangement unit 102 reads the robot model of the robot from the model storage unit 131. The three-dimensional model arrangement unit 102 arranges the read robot model in the virtual space.

[0062] Furthermore, the three-dimensional model placement unit 102 reads a tool model of a tool (not shown) from the model storage unit 131 , and places the read tool model at the tool tip of the robot model of the robot in the virtual space.

[0063] Furthermore, the three-dimensional model placement unit 102 reads a visual sensor model of a visual sensor (not shown) from the model storage unit 131 and places the read visual sensor model of the visual sensor at the tip of the arm of the robot model of the robot in the virtual space.

[0064] In order to arrange the workpiece model of the workpiece (not shown) in the virtual space, the three-dimensional model arrangement unit 102 reads the workpiece model from the model storage unit 131. The three-dimensional model arrangement unit 102 arranges the read workpiece model in the virtual space.

[0065] In addition, Figure 2 In FIG, the workpiece model is arranged on a model represented by a rectangular parallelepiped such as a jig (not shown).

[0066] For example, Figure 3 As shown, the image configuration unit 103 configures a label such as a sticker with a contour line drawn on the surface of the workpiece model. Figure 3 The label shown has the outline of the letter "F" depicted.

[0067] like Figure 4 As shown, the image capture unit 104 moves the robot model in the virtual space, configures the visual sensor model so that a straight line represented by a dot chain line with a plumb bob at the center of the tag is consistent with the optical axis of the visual sensor model, and adjusts the height of the visual sensor model so that the entire tag is within the field of view of the visual sensor model.

[0068] Furthermore, the visual sensor model is set with imaging parameters such as the position of the sensor model coordinate system viewed from the robot model coordinate system, focal length, and image size for performing imaging similar to that performed by the visual sensor in real space.

[0069] The image capturing unit 104 generates a simulated captured image of a label captured by the visual sensor model at a position where the visual sensor model is arranged, based on the capturing parameters set in the visual sensor model. Figure 4 As shown, the outline of the letter "F" is captured in the captured image.

[0070] The contour line detection unit 105 detects a contour line by performing image processing on the simulated image captured by the image capturing unit 104 .

[0071] Specifically, the contour line detection unit 105 detects the black portion of the label in the simulated captured image as a contour line, for example, using a well-known technique (for example, line drawing extraction).

[0072] The three-dimensional data conversion unit 106 converts the contour lines detected by the contour line detection unit 105 into a set of three-dimensional data.

[0073] Specifically, the three-dimensional data conversion unit 106 is as follows. Figure 5 As shown, in the captured image captured by the visual sensor model, the three-dimensional points ( Figure 5 The three-dimensional position coordinates (X, Y, Z) of the black dot (indicated by the figure) are converted into a set of three-dimensional data. The three-dimensional data conversion unit 106 stores the converted three-dimensional data in the storage unit 13.

[0074] The processing line generating unit 107 generates processing lines from a set of three-dimensional data.

[0075] Specifically, the processing line generating unit 107 connects the three-dimensional data of the contour line stored in the storage unit 13, for example, so as to form a Figure 6 As shown, a processing line represented by a dotted line is generated.

[0076] In addition, if Figure 7 As shown, the processing line generation unit 107 calculates the surface of the workpiece model with the contour line labels arranged thereon from the three-dimensional contour line data stored as a set of three-dimensional points, and calculates the normal vector relative to the surface of the workpiece model at each point on the processing line.

[0077] The robot program generation unit 108 generates a program based on the processing line and normal vector generated by the processing line generation unit 107 .

[0078] Specifically, for example Figure 8 As shown, the robot program generation unit 108 calculates the posture of the robot model at each three-dimensional point on the processing line based on the normal vector relative to the surface of the workpiece model at each three-dimensional point on the processing line, and sets the teaching points of the robot program along the processing line. Based on the processing line and the normal vector relative to the surface of the workpiece model on the processing line, the robot program generation unit 108 generates a robot program for the robot model to perform an operation on the workpiece model using a tool model.

[0079] Then, the programming device 1 performs a simulation of the generated robot program, such as Figure 9 As shown, the user can confirm that the robot model moves the tool model along the processing line of the letter "F".

[0080] <Program Generation Processing by Programming Device 1>

[0081] Next, refer to Figure 10 , while explaining the flow of program generation processing of the programming device 1.

[0082] Figure 10 This is a flowchart for explaining the program generation process of the programming device 1. The flow shown here is executed when an instruction to generate a robot program is received.

[0083] In step S1, the three-dimensional model configuration unit 102 configures a robot model of a robot (not shown), a tool model of a tool (not shown), a visual sensor model of a visual sensor (not shown), and a workpiece model of a workpiece (not shown) that constitute the industrial machinery system model in the three-dimensional virtual space generated by the virtual space production unit 101 according to the input operation of the user's input unit 11.

[0084] In step S2 , the image placement unit 103 places a label with a contour line drawn on the surface of the workpiece model.

[0085] In step S3, the image capture unit 104 moves the robot model in virtual space, positions the vision sensor model so that a straight line plumb through the center of the label on the workpiece model aligns with the vision sensor model's optical axis, and adjusts the height of the vision sensor model so that the entire label is within the vision sensor model's field of view. Based on the imaging parameters set for the vision sensor model, the image capture unit 104 generates a simulated image of the label captured by the vision sensor model at the location where the vision sensor model is positioned.

[0086] In step S4 , the contour line detection unit 105 detects a contour line by performing image processing on the simulated image captured in step S3 .

[0087] In step S5 , the three-dimensional data conversion unit 106 converts the contour lines detected in step S4 into a set of three-dimensional data.

[0088] In step S6, the processing line generating unit 107 generates processing lines from a set of three-dimensional data.

[0089] In step S7 , the processing line generating unit 107 calculates the surface of the workpiece model with the contour line label arranged from the three-dimensional contour line data stored as a set of three-dimensional points, and calculates the normal vector relative to the surface of the workpiece model at each point on the processing line.

[0090] In step S8 , the robot program generation unit 108 generates a program based on the processing line and normal vector generated in step S7 .

[0091] Thus, the programming device 1 of the first embodiment places a label depicting a contour line on the surface of a workpiece model located in virtual space and, based on the imaging parameters set for the vision sensor model, generates a captured image of the label at the location where the vision sensor model is located. The programming device 1 detects the contour line by performing image processing on the captured image, generates a machining line based on the detected contour line, calculates the normal vector of the generated machining line relative to the surface of the workpiece model, and generates a program for an industrial machine to perform an operation on the workpiece using a tool based on the machining line and the normal vector. Thus, the programming device 1 can generate a machining line depicting an arbitrary and complex trajectory on the surface of the workpiece and generate a program for the industrial machine to perform machining operations along the trajectory based on the generated machining line.

[0092] Therefore, the programming device 1 can reduce the labor and time of the teaching work without requiring the operator's expertise.

[0093] The first embodiment has been described above.

[0094] <Second embodiment>

[0095] Next, the second embodiment will be described. In the first embodiment, a programming device 1 places a label depicting non-intersecting contours on the surface of a workpiece model. Based on imaging parameters set for the vision sensor model, the programming device 1 generates an image captured at the location on the model where the vision sensor is located. The programming device 1 detects contours by performing image processing on the captured image, generates machining lines based on the detected contours, calculates normal vectors relative to the workpiece surface along the generated machining lines, and generates a program for an industrial machine to perform work on the workpiece using a tool based on the machining lines and normal vectors. In contrast, the programming device 1A in the second embodiment differs from the first embodiment in that a label depicting non-intersecting contours is placed on the surface of the workpiece model. The programming device 1 performs image processing on the captured image capturing the contours to detect the contours and segment them into multiple line segments. At least two consecutive segments of the segmented segments are combined into a single segment, and the order in which the program performs operations is specified for each segment.

[0096] Thus, the programming device 1A of the second embodiment can generate a machining line that describes an arbitrary and complex trajectory on the surface of a workpiece, and generate a program for an industrial machine for performing machining work along the trajectory based on the generated machining line.

[0097] The second embodiment will be described below.

[0098] Figure 11 This is a functional block diagram showing an example of the functional structure of a programming device according to the second embodiment. Figure 1 Elements of the programming device 1 having the same functions are denoted by the same reference numerals and detailed descriptions thereof are omitted.

[0099] like Figure 11 As shown, the programming device 1A of the second embodiment includes a control unit 10a, an input unit 11, a display unit 12, and a storage unit 13. The control unit 10a includes a virtual space creation unit 101, a three-dimensional model configuration unit 102, an image configuration unit 103, an image capture unit 104, a contour line detection unit 105a, a three-dimensional data conversion unit 106a, a processing line generation unit 107a, and a robot program generation unit 108a.

[0100] The input unit 11 , the display unit 12 , and the storage unit 13 have the same functions as those of the input unit 11 , the display unit 12 , and the storage unit 13 in the first embodiment.

[0101] The model storage unit 131 has the same function as the model storage unit 131 of the first embodiment.

[0102] <Control Unit 10a>

[0103] The control unit 10 a includes a CPU, a ROM, a RAM, a CMOS memory, and the like, which are well known to those skilled in the art and are configured to be able to communicate with each other via a bus.

[0104] The CPU is a processor that controls the programming device 1A as a whole. The CPU reads the system program and application program stored in the ROM via the bus and controls the programming device 1A as a whole according to the system program and application program. Figure 11 As shown, the control unit 10a is configured to realize the functions of a virtual space production unit 101, a three-dimensional model configuration unit 102, an image configuration unit 103, an image shooting unit 104, a contour line detection unit 105a, a three-dimensional data conversion unit 106a, a processing line generation unit 107a, and a robot program generation unit 108a.

[0105] The virtual space creation unit 101 , the three-dimensional model configuration unit 102 , the image configuration unit 103 , and the image capture unit 104 have the same functions as those of the first embodiment.

[0106] The contour line detection unit 105 a detects a contour line by performing image processing on the simulated image captured by the image capturing unit 104 .

[0107] In addition, the following are labels for contour lines that intersect each other at least at one point, such as Figure 12 As shown in FIG. 1 , the case where a label having a character “あ” drawn thereon is arranged on the workpiece model will be described as an example.

[0108] For example, Figure 13 As shown, the contour line detection unit 105a obtains a simulated captured image from the image capture unit 104. This simulated captured image is generated by capturing a label at a predetermined position using a visual sensor model according to predetermined capturing parameters. The visual sensor model is configured so that a straight line drawn from the center of the label, indicated by a dotted chain line, coincides with the optical axis of the visual sensor model. Similar to the contour line detection unit 105 of the first embodiment, the contour line detection unit 105a detects the black portion of the label in the simulated captured image as a contour line using known techniques (e.g., line pattern extraction).

[0109] like Figure 14 As shown, the contour line detection unit 105 a divides the detected contour line of the character “ ” into 11 line segments L1 to L11 based on the positions of the intersection points.

[0110] The contour line detection unit 105a combines at least two line segments selected from the line segments L1 to L11 into one line segment, for example, based on the input operation of the user's input unit 11. Specifically, the contour line detection unit 105a combines at least two line segments selected from the line segments L1 to L11 into one line segment. Figure 15 As shown, the line segments L1 and L2 are combined into one line segment C1. In addition, the contour line detection unit 105a combines the line segments L3 to L6 into one line segment C2. In addition, the contour line detection unit 105a combines the line segments L7 to L11 into one line segment C3.

[0111] The contour line detection unit 105 a specifies, based on the user's input operation on the input unit 11 , that the robot program operates, for example, in the order of line segments C1 , C2 , and C3 .

[0112] The three-dimensional data conversion unit 106 a converts the contour lines detected by the contour line detection unit 105 a into a set of three-dimensional data, similarly to the three-dimensional data conversion unit 106 of the first embodiment.

[0113] Specifically, the three-dimensional data conversion unit 106a is as follows. Figure 16 As shown, in the captured image captured by the visual sensor model, the three-dimensional points ( Figure 16 The three-dimensional data conversion unit 106a stores the converted three-dimensional data in the storage unit 13.

[0114] The processing line generating unit 107a generates processing lines from a set of three-dimensional data, similarly to the processing line generating unit 107 of the first embodiment.

[0115] Specifically, the processing line generating unit 107a connects the three-dimensional data of the contour line stored in the storage unit 13, for example, to form a Figure 17 A processing line represented by a dotted line is generated as shown.

[0116] In addition, if Figure 18 As shown, the processing line generation unit 107a calculates the surface of the workpiece model configured with an image from the three-dimensional data of the contour line stored as a set of three-dimensional points, and calculates the normal vector relative to the surface of the workpiece model at each point on the processing line, wherein the image displays the contour line.

[0117] The robot program generation unit 108 a generates a program based on the order of line segments specified by the contour line detection unit 105 a and the processing lines and normal vectors generated by the processing line generation unit 107 a .

[0118] Specifically, for example Figure 19 As shown, the robot program generation unit 108a calculates the posture of the robot model at each three-dimensional point on the processing line based on the normal vector relative to the surface of the workpiece model at each three-dimensional point on the processing line, and sets the robot program teaching points along the processing line. Based on the processing line and the normal vector relative to the surface of the workpiece model on the processing line, the robot program generation unit 108a generates a robot program in which the robot model operates on the workpiece model in the order of specified line segments C1 to C3 using the tool model.

[0119] <Program Generation Processing by Programming Device 1A>

[0120] Next, refer to Figure 20 , while explaining the flow of program generation processing by the programming device 1A.

[0121] Figure 20 This is a flowchart for explaining the program generation process of the programming device 1A. The flowchart shown here is executed when an instruction to generate a robot program is received.

[0122] In addition, the processing of steps S1 to S4 and steps S6 to S8 is the same as Figure 10 The processing of steps S1 to S3 and steps S5 to S7 is the same, so the description is omitted.

[0123] In step S4a, the contour line detection unit 105a detects contour lines by performing image processing on the simulated image captured in step S3. If the detected contour lines have intersections, the contour line detection unit 105a divides the detected contour lines into multiple line segments based on the positions of the intersections. Based on the user's input operation on the input unit 11, the contour line detection unit 105a combines at least two line segments selected from the multiple line segments into a single line segment and specifies the order in which the robot program will operate on each line segment.

[0124] In step S8 a , the robot program generation unit 108 a generates a program based on the order of each line segment specified in step S4 a and the processing line and normal vector generated in step S7 .

[0125] Thus, in a second embodiment, the programming device 1A, when a label is placed on the surface of a workpiece and the label depicts a contour line that intersects at at least one point, detects the contour line by performing image processing on a captured image of the contour line, segmenting it into multiple line segments. At least two of the segmented line segments that are consecutively depicted are combined into a single line segment, and a program specifies the order in which operations are to be performed for each line segment. Thus, the programming device 1A can generate a processing line depicting an arbitrary and complex trajectory on the surface of a workpiece, and generate a program for an industrial machine that performs processing operations along the trajectory according to the generated processing line.

[0126] Therefore, the programming device 1A can reduce the labor and time of the teaching work without requiring the operator's expertise.

[0127] The second embodiment has been described above.

[0128] <Third embodiment>

[0129] Next, the third embodiment will be described. In the first embodiment, a programming device 1 places a label depicting non-intersecting contours on the surface of a workpiece model. Based on imaging parameters set for the vision sensor model, the programming device 1 generates an image captured at the location on the model where the vision sensor is located. The programming device 1 detects contours by performing image processing on the captured image, generates machining lines based on the detected contours, calculates normal vectors relative to the surface of the workpiece model along the generated machining lines, and generates a program for an industrial machine to perform work on the workpiece using a tool based on the machining lines and normal vectors. Furthermore, in the second embodiment, the programming device 1A differs from the first embodiment in that a label depicting non-intersecting contours is placed on the surface of the workpiece model. The programming device 1 performs image processing on the captured image capturing the contours to detect the contours and segment them into multiple line segments. At least two consecutive line segments of the segmented line segments are combined into a single line segment, and the program specifies the order in which the work is to be performed for each line segment. In contrast, in the third embodiment, the programming device 1B differs from the first and second embodiments in that, in an industrial machine arranged in a real space, a tool installed on the industrial machine, a visual sensor, and a workpiece, a label depicting a contour line is arranged on the surface of the workpiece, and based on the shooting parameters set in the visual sensor, the visual sensor obtains a photographic image of the label taken at the position where the visual sensor is arranged.

[0130] Thus, the programming device 1B of the third embodiment can generate a machining line that describes an arbitrary and complex trajectory on the surface of a workpiece, and generate a program for an industrial machine for performing machining work along the trajectory based on the generated machining line.

[0131] The third embodiment will be described below.

[0132] Figure 21 This is a functional block diagram showing an example of the functional structure of a programming device according to the third embodiment. Figure 1 Elements of the programming device 1 having the same functions are denoted by the same reference numerals and detailed descriptions thereof are omitted.

[0133] like Figure 21 As shown, the programming device 1B of the third embodiment includes a control unit 10b, an input unit 11, a display unit 12, and a storage unit 13b. The control unit 10b includes an image configuration unit 103b, an image capture unit 104b, a contour line detection unit 105, a three-dimensional data conversion unit 106, a processing line generation unit 107, and a robot program generation unit 108.

[0134] The input unit 11 and the display unit 12 have the same functions as those of the input unit 11 and the display unit 12 in the first embodiment.

[0135] <Storage Unit 13b>

[0136] Like the storage unit 13 of the first embodiment, the storage unit 13 b is an SSD or HDD, and may store a program for generating a robot program for a robot (not shown) to perform work on a workpiece using a tool, or a previously generated robot program.

[0137] <Control Unit 10b>

[0138] The control unit 10 b includes a CPU, a ROM, a RAM, a CMOS memory, and the like, which are well known to those skilled in the art and are configured to be able to communicate with each other via a bus.

[0139] The CPU is a processor that controls the programming device 1B as a whole. The CPU reads the system program and application program stored in the ROM via the bus and controls the programming device 1B as a whole according to the system program and application program. Figure 21 As shown, the control unit 10 b is configured to realize the functions of an image arrangement unit 103 b , an image capturing unit 104 b , a contour line detection unit 105 , a three-dimensional data conversion unit 106 , a processing line generation unit 107 , and a robot program generation unit 108 .

[0140] In addition, the contour line detection unit 105, the three-dimensional data conversion unit 106, the processing line generation unit 107, and the robot program generation unit 108 have the same functions as the contour line detection unit 105, the three-dimensional data conversion unit 106, the processing line generation unit 107, and the robot program generation unit 108 of the first embodiment.

[0141] The image placement unit 103b may also use a projector (not shown) or the like to place an image on the surface of the workpiece. Figure 3 In the same way, labels with outlines are projected and arranged.

[0142] The image capturing unit 104b causes the robot to move in real space via a robot control device (not shown). Figure 4 Similarly, the visual sensor is configured so that the straight line from the plumb bob at the center of the label, as shown by the dotted chain line, is consistent with the optical axis of the visual sensor, and the height of the visual sensor is adjusted so that the entire label is within the field of view of the visual sensor.

[0143] The image capturing unit 104 b acquires an image of the label at the position where the visual sensor is arranged based on the imaging parameters set in the visual sensor, and outputs the captured image to the contour line detection unit 105 .

[0144] <Program Generation Processing by Programming Device 1B>

[0145] Next, refer to Figure 22 , while explaining the flow of program generation processing by the programming device 1B.

[0146] Figure 22 This is a flowchart for explaining the program generation processing of the programming device 1B. The flowchart shown here is executed when an instruction to generate a robot program is received.

[0147] In addition, the processing of steps S33 to S37 is the same as Figure 10 The processing of steps S4 to S8 is the same, so the description is omitted.

[0148] In step S31 , the image placement unit 103 b projects and places a label having a contour line drawn thereon on the surface of the workpiece using a projector (not shown).

[0149] In step S32, the image capture unit 104b operates the robot in real space, positions the vision sensor so that a straight line drawn from the center of the label on the workpiece aligns with the vision sensor's optical axis, and adjusts the vision sensor's height so that the entire label is within the vision sensor's field of view. Based on the imaging parameters set for the vision sensor, the image capture unit 104b acquires an image of the label at the location where the vision sensor is positioned.

[0150] Thus, the programming device 1B of the third embodiment places a label depicting a contour line on the surface of a workpiece. Based on the imaging parameters set for the vision sensor, the vision sensor acquires an image of the label captured at the location where the vision sensor is located. The programming device 1B detects the contour line by performing image processing on the captured image, generates a machining line based on the detected contour line, calculates the normal vector of the generated machining line relative to the workpiece surface, and generates a program for an industrial machine to perform an operation on the workpiece using a tool based on the machining line and the normal vector. Thus, the programming device 1B can generate a machining line depicting an arbitrary and complex trajectory on the surface of a workpiece, and generate a program for the industrial machine to perform machining operations along the trajectory based on the generated machining line.

[0151] Therefore, the programming device 1B can reduce the labor and time of the teaching work without requiring the operator's expertise.

[0152] The third embodiment has been described above.

[0153] <Modification of the Third Embodiment>

[0154] In the third embodiment, the label placed on the workpiece is a label such as the letter "F" whose contour lines do not intersect with each other, but the present invention is not limited to this. For example, the label may be a label such as the letter "あ" whose contour lines intersect with each other at least at one point. In this case, the programming device 1B may also perform image processing on the captured image of the contour line to detect the contour line and divide it into multiple line segments, similar to the second embodiment, combine at least two line segments that are continuously drawn from the multiple divided line segments into a single line segment, and specify the order of operations to be performed by the program for each line segment.

[0155] As described above in the first to third embodiments, the programming devices 1, 1A, and 1B of the present disclosure can generate processing lines that depict arbitrary and complex trajectories on the surface of a workpiece, and generate programs for industrial machinery that perform processing operations along the trajectories based on the generated processing lines.

[0156] <Variation 1>

[0157] In the first to third embodiments described above, the surface of the workpiece (workpiece model) is set to a plane, but this is not limiting. The surface of the workpiece (workpiece model) may also have any shape, such as concave and convex. In this case, a sticker with a label drawn on it may be attached to the surface of the workpiece (workpiece model) of any shape, or the label may be projected by a projector or the like.

[0158] <Variation 2>

[0159] Furthermore, for example, in the above-described embodiment, the programming devices 1, 1A, and 1B generate robot programs for robots that are industrial machines, but the present invention is not limited to this. For example, the programming devices 1, 1A, and 1B may also generate programs for industrial machines such as machine tools, forging machines, laser processing machines, and injection molding machines.

[0160] <Variation 3>

[0161] In addition, for example, in the above-mentioned embodiment, the visual sensor (visual sensor model) is installed at the front end of the arm of the robot (robot model), but it is not limited to this. Figure 23 As shown, a vision sensor (vision sensor model) is fixed and arranged separately from the robot (robot model) at a position such as a wall or a ceiling where the entire tag arranged on the workpiece is within its field of view.

[0162] Furthermore, each function included in the programming devices 1, 1A, and 1B of the first to third embodiments can be realized by hardware, software, or a combination thereof. Here, realization by software means that a computer reads a program and executes it.

[0163] The program can be stored and provided to the computer using various types of non-transitory computer readable media (Non-transitory computer readable medium). Non-transitory computer readable media include various types of tangible recording media (Tangible storage medium). Examples of non-transitory computer readable media include magnetic recording media (such as floppy disks, magnetic tapes, hard disks), optical magnetic recording media (such as magneto-optical disks), CD-ROM (Read Only Memory), CD-R, CD-R / W, semiconductor memories (such as mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM). In addition, the program can also be supplied to the computer through various types of transient computer readable media (Transitory computer readable medium). Examples of transient computer readable media include electrical signals, optical signals, and electromagnetic waves. Transitory computer readable media can supply the program to the computer via wired communication paths such as wires and optical fibers or wireless communication paths.

[0164] Furthermore, the steps described in the program recorded on the recording medium include processing performed in a time series according to the order in which they are recorded, but they do not necessarily need to be processed in a time series and may also include processing performed in parallel or individually. In addition, the steps described in the program may also be executed by cloud computing.

[0165] Although the present disclosure has been described in detail, the present disclosure is not limited to the above-mentioned embodiments. These embodiments may be subject to various additions, replacements, changes, partial deletions, etc., without departing from the scope of the present disclosure, or without departing from the scope of the present disclosure derived from the contents recorded in the scope of the patent application and its equivalents. In addition, these embodiments may also be combined and implemented. For example, in the above-mentioned embodiments, the order of each action or the order of each processing is shown as an example and is not limited thereto. In addition, the same applies to the case where numerical values or mathematical formulas are used in the description of the above-mentioned embodiments.

[0166] The following supplementary notes are further disclosed regarding the above-mentioned embodiment and modifications.

[0167] (Note 1)

[0168] A programming device (1) is provided, in a virtual space, to respectively configure models of an industrial machine, a tool mounted on the industrial machine, a visual sensor, and a workpiece, and to generate a program for the model of the industrial machine to operate on the model of the workpiece through the model of the tool, wherein a label depicting a contour line is configured on the surface of the model of the workpiece, and a photographic image of the label photographed at the position of the model where the visual sensor is configured is generated based on photographic parameters set in the model of the visual sensor, the contour line is detected by performing image processing on the photographic image, a processing line is generated based on the detected contour line, and a normal vector of the generated processing line relative to the surface of the model of the workpiece is calculated, and a program is generated based on the processing line and the normal vector.

[0169] (Note 2)

[0170] A programming device (1B) generates a program for the industrial machine to operate on a workpiece through a tool, a visual sensor, and a workpiece arranged in a real space, wherein a label depicting a contour line is arranged on the surface of the workpiece, and an image of the label taken at the position where the visual sensor is arranged is obtained from the visual sensor according to shooting parameters set in the visual sensor. The contour line is detected by performing image processing on the captured image, and a processing line is generated based on the detected contour line. The normal vector of the generated processing line relative to the surface of the workpiece is calculated, and the program is generated based on the processing line and the normal vector.

[0171] (Note 3)

[0172] In the programming device (1A) described in Supplementary Note 1 or Supplementary Note 2, a contour line is divided into a plurality of line segments, and each of the plurality of line segments is converted into a set of three-dimensional data.

[0173] (Note 4)

[0174] In the programming device (1A) described in Supplementary Note 3, at least two continuous line segments are selected from a plurality of line segments and combined into one line segment.

[0175] (Note 5)

[0176] In the programming device (1A) described in Supplementary Note 3, the order in which operations are performed on a plurality of line segments is specified by a program.

[0177] (Note 6)

[0178] In the programming device (1) described in Supplementary Note 1, the visual sensor is arranged at a position where the entire tag is within the field of view of the model of the visual sensor.

[0179] (Note 7)

[0180] In the programming device (1B) described in Supplementary Note 2, the visual sensor is arranged at a position where the entire label is within the field of view of the visual sensor.

[0181] Description of Reference Numerals

[0182] 1, 1A, 1B programming device

[0183] 10, 10a, 10b control unit

[0184] 101 Virtual Space Generation Department

[0185] 102 3D Model Configuration Department

[0186] 103, 103b image configuration unit

[0187] 104, 104b image capturing unit

[0188] 105, 105a contour line detection unit

[0189] 106, 106a three-dimensional data conversion unit

[0190] 107, 107a processing line generation unit

[0191] 108, 108a Robot Program Generation Unit

[0192] 11 Input section

[0193] 12 Display unit

[0194] 13, 13b storage unit

[0195] 131 Model storage unit.

Claims

1. A programming device that arranges models of an industrial machine, a tool mounted on the industrial machine, a visual sensor, and a workpiece in a virtual space, and generates a program for the model of the industrial machine to operate on the model of the workpiece through the model of the tool, characterized in that: Arranging a label with a contour line drawn on the surface of the workpiece model, generating a captured image of the tag at a position where the model of the visual sensor is arranged, based on a capturing parameter set in the model of the visual sensor; detecting the contour line by performing image processing on the captured image, generating a processing line according to the detected contour line, and calculating a normal vector on the generated processing line relative to the surface of the model of the workpiece, The program is generated based on the processing line and the normal vector.

2. A programming device for generating a program for an industrial machine, a tool mounted on the industrial machine, a visual sensor, and a workpiece arranged in a real space, for the industrial machine to perform an operation on the workpiece using the tool, wherein: A label with a contour line drawn on the surface of the workpiece is arranged. Acquiring an image of the tag captured at a position where the visual sensor is disposed from the visual sensor according to a photographing parameter set in the visual sensor, detecting the contour line by performing image processing on the captured image, generating a processing line according to the detected contour line, and calculating a normal vector of the generated processing line relative to the surface of the workpiece, The program is generated based on the processing line and the normal vector.

3. The programming device according to claim 1 or 2, characterized in that The contour line is divided into a plurality of line segments, and the plurality of line segments are respectively converted into sets of three-dimensional data.

4. The programming device according to claim 3, characterized in that At least two consecutive line segments are selected from the plurality of line segments and combined into one line segment.

5. The programming device according to claim 3, characterized in that The order in which the operations are to be performed on the plurality of line segments by the program is specified.

6. The programming device according to claim 1, characterized in that The model of the vision sensor is arranged at a position where the entire tag is within the field of view of the vision sensor.

7. The programming device according to claim 2, characterized in that The visual sensor is arranged at a position where the entire tag is within the field of view of the visual sensor.

Images