Terminal device

Abstract

To provide a terminal device capable of creating a work program without lowering an operation rate of a robot.SOLUTION: A terminal device 1 photographs a first image including a work object and a marker for the work object, sets a user coordinate system with the marker for the work object included in the first image as reference, sets an operation path of an industrial robot on the basis of a work spot of the work object included in the first image, creates a work program for causing the industrial robot on the basis of the set operation path of the industrial robot, photographs a second image including the work object and the maker for the work object maintaining a positional relation in the first image and a marker for the robot attached to the industrial robot, calculates a robot coordinate system on the basis of the marker for the robot and position attitude information of the industrial robot included in the second image, and converts the operation path of the industrial robot from the user coordinate system to the robot coordinate system to correct the work program.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a terminal device capable of creating an operation program for a robot. [Background technology]

[0002] Conventionally, there is a technology for creating a program for operating a robot using images captured by a camera. For example, in the technology of Patent Document 1, a program for operating a robot is created using an image including a robot with a marker attached and a welding target captured by a camera. In the technology of Patent Document 2, a program for operating a robot is created by superimposing a virtual image of the robot's effector, etc., on an image of real space including the robot captured by a camera. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7190552 [Patent Document 2] Patent No. 6420229 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the techniques of Patent Documents 1 and 2, when creating a program, it is necessary to take an image including the robot. Therefore, while creating the program, the robot is occupied, and the operating rate of the robot decreases.

[0005] Therefore, an object of the present invention is to provide a terminal device that can create an operation program without reducing the operating rate of a robot. [Means for solving the problem]

[0006] A terminal device according to one embodiment of the present invention includes a first photographing unit that photographs a first image including a work object and a marker for the work object; a user coordinate system setting unit that sets a user coordinate system based on the marker for the work object included in the first image; a movement path setting unit that sets a movement path for the industrial robot in the user coordinate system based on a work location of the work object included in the first image; a program creation unit that creates a work program for operating the industrial robot based on the set movement path of the industrial robot; a second photographing unit that photographs a second image that includes a robot marker attached to the industrial robot in addition to the work object and the marker for the work object that maintain their positional relationship in the first image; a robot coordinate system calculation unit that calculates a robot coordinate system based on the robot marker included in the second image and position and posture information of the industrial robot; and a program correction unit that converts the coordinate system of the movement path of the industrial robot from the user coordinate system to the robot coordinate system and corrects the work program.

[0007] According to this aspect, by capturing a first image including the work object and the work object marker, a user coordinate system based on the work object marker can be set, and a work program for the industrial robot can be created based on the movement path of the industrial robot in the user coordinate system set based on the work location of the work object. Then, by capturing a second image including the work object and the work object marker that maintain the positional relationship in the first image and the robot marker attached to the industrial robot, a robot coordinate system based on the robot marker and the position and orientation information of the industrial robot can be calculated, and the coordinate system of the movement path of the industrial robot can be converted from the user coordinate system to the robot coordinate system to modify the work program.

[0008] This makes it possible to create a work program for the industrial robot in the user coordinate system based on a first image that does not include the industrial robot, and then, based on a second image that is subsequently taken and that includes a robot marker attached to the industrial robot, convert the coordinate system representing the industrial robot's movement path from the user coordinate system to the robot coordinate system, and modify the work program based on the user coordinate system to a work program based on the robot coordinate system.

[0009] In the above aspect, the program correction unit may convert a coordinate system of the motion path of the industrial robot based on a difference between the origin of the user coordinate system and the origin of the robot coordinate system.

[0010] According to this aspect, it is possible to convert the coordinate system of the motion path of the industrial robot based on the difference between the origin of the user coordinate system and the origin of the robot coordinate system.

[0011] In the above aspect, the user coordinate system setting unit may calculate an origin and coordinate axes of the user coordinate system based on the work object marker.

[0012] According to this aspect, it is possible to create a work program for an industrial robot based on the origin and coordinate axes of a user coordinate system calculated with reference to the work object marker.

[0013] In the above aspect, the robot coordinate system calculation unit may calculate an origin and coordinate axes of the robot coordinate system based on the robot marker and position and orientation information of the industrial robot.

[0014] According to this aspect, it is possible to modify the work program of the industrial robot using the origin of the robot coordinate system calculated based on the robot marker and the coordinate axes of the robot coordinate system calculated based on the position and posture information of the industrial robot.

[0015] In the above aspect, the position and orientation information of the industrial robot may include angle information of each axis of the industrial robot acquired from a robot control device that controls the operation of the industrial robot.

[0016] According to this aspect, by using each axis angle information of the industrial robot acquired from the robot control device, it is possible to improve the accuracy of calculating the coordinate axes of the robot coordinate system. Effect of the Invention

[0017] According to the present invention, it is possible to provide a terminal device that can create a work program without reducing the operating rate of a robot. [Brief description of the drawings]

[0018] [Figure 1] FIG. 1 is a diagram illustrating a system configuration of a welding robot system including a terminal device according to an embodiment. [Diagram 2] 2 is a diagram illustrating an example of a physical configuration of the terminal device and the robot control device illustrated in FIG. 1. [Diagram 3] 4 is a schematic diagram for explaining a first image captured by the imaging section. FIG. [Figure 4] 11 is a schematic diagram for explaining a second image captured by the imaging section. FIG. [Diagram 5] 3 is a diagram illustrating an example of a functional configuration of a control unit of the terminal device shown in FIG. 2. [Figure 6] 10 is a flowchart illustrating an example of an operation of the terminal device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] A preferred embodiment of the present invention will be described with reference to the accompanying drawings. In each drawing, the same reference numerals denote the same or similar configurations. Also, since the drawings are schematic, the dimensions and ratios of each component are different from the actual ones.

[0020] FIG. 1 is a diagram illustrating a system configuration of a welding robot system 100 including a terminal device 1 according to an embodiment. The welding robot system 100 includes, for example, a terminal device 1, a robot control device 2, and a manipulator (industrial robot) 3. The terminal device 1 and the robot control device 2, and the robot control device 2 and the manipulator 3 are connected via a network. The terminal device 1 and the robot control device 2 are connected by wireless communication such as WiFi (Wireless Fidelity), and the robot control device 2 and the manipulator 3 are connected via, for example, a communication cable. The network may be wired (including a communication cable) or wireless.

[0021] A teach pendant may be included in the welding robot system 100. The teach pendant is an operating device that can be connected to the robot control device 2 and is used by an operator to teach the manipulator 3 how to operate.

[0022] The manipulator 3 is a welding robot that performs arc welding on a workpiece W, which is a welding target, in accordance with welding conditions set in the robot control device 2. The manipulator 3 has, for example, a multi-joint arm provided on a base member fixed to a factory floor or the like, and a welding torch (tool) connected to the tip of the multi-joint arm.

[0023] A robot marker Mr is attached to the tip of the articulated arm of the manipulator 3, and a workpiece marker Mw is disposed on the workpiece W.

[0024] The robot marker Mr and the work marker Mw may be identifiers that can be recognized by the terminal device 1, but preferably, AR markers are used. By using an AR marker, when an AR marker is recognized, it becomes possible to easily realize a user coordinate system with the AR marker as the origin being superimposed on the actual image and displayed.

[0025] The configurations of the robot control device 2 and the terminal device 1 will be described below in order with reference to FIG.

[0026] The robot control device 2 is a control unit that controls the operation of the manipulator 3, and includes, for example, a control unit 21, a storage unit 22, and a communication unit .

[0027] The control unit 21 is a processor, and controls the manipulator 3 by executing a welding program and the like stored in the storage unit 22 .

[0028] The communication unit 23 is a communication interface, and controls communication with the terminal device 1 and the manipulator 3 connected via a network.

[0029] The robot control device 2 may further include a welding power supply unit. The welding power supply unit supplies a welding current, a welding voltage, and the like to the manipulator 3 according to predetermined welding conditions, for example, to generate an arc between the tip of the welding wire and the workpiece W. The welding power supply unit may be provided separately from the robot control device 2.

[0030] The terminal device 1 is, for example, a portable terminal with a digital camera. Portable terminals include, for example, tablet terminals, smartphones, personal digital assistants (PDAs), notebook PCs (personal computers), and other portable terminals. The terminal device 1 includes, for example, a control unit 11, a storage unit 12, a communication unit 13, an imaging unit 14, a distance measurement unit 15, and a display unit 16.

[0031] The control unit 11 is a processor, and controls each unit of the terminal device 1 by executing a program stored in the storage unit 12 .

[0032] The storage unit 12 is a computer-readable recording medium, and stores programs for implementing various functions of the terminal device 1, various data used by the programs, and the like.

[0033] The communication unit 13 is a communication interface, and controls communication with the robot control device 2 connected via a network.

[0034] The photographing section (first photographing section, second photographing section) 14 is, for example, a 2D camera including a lens and an image sensor (image pickup element), and converts the light of the subject received by the lens into an electrical signal (digital image data).

[0035] In this embodiment, the photographing section 14 photographs two types of images, a first image and a second image, for example. The first image and the second image will be described with reference to Figs.

[0036] 3, the first image captured by the photographing unit 14 includes the workpiece W that is the welding target of the manipulator 3, and a workpiece marker (welding target marker) Mw disposed on the workpiece W. The first image does not need to include the manipulator 3. Therefore, when capturing the first image, the manipulator 3 can perform welding work, for example, in a work location (e.g., a robot cell) that is located in a location different from the location where the workpiece W is present.

[0037] 4, the second image captured by the imaging unit 14 includes the workpiece W and the workpiece marker Mw that maintain the positional relationship in the first image, and the robot marker Mr attached to the manipulator 3. When capturing the second image, the workpiece W and the workpiece marker Mw included in the first image may be moved to the work location of the manipulator 3. Note that the manipulator 3 with the cart may be moved to the work location where the workpiece W and the workpiece marker Mw are present.

[0038] 2 is, for example, a 3D camera equipped with a distance measurement sensor. The distance measurement sensor is a sensor capable of measuring the distance to an object. As the distance measurement sensor, for example, a LiDAR (Light Detection and Ranging) sensor, a millimeter wave sensor, an ultrasonic sensor, etc. can be used.

[0039] The distance measurement unit 15 may be either a 3D camera or a distance measurement sensor. In the case of only a 3D camera, it is preferable to calculate three-dimensional coordinate data corresponding to an object based on a plurality of images of the object taken from a plurality of different positions. In this case, a three-dimensional measurement method using a known stereo method can be used.

[0040] Here, by including a distance measurement sensor in the terminal device 1, the positional relationship between the image sensor and the distance measurement sensor can be fixed, and it becomes possible to synchronize the timing of acquiring data by each sensor. This makes it possible to improve the processing efficiency for setting a specific position of a work marker Mw (described later) on the point cloud data, for example.

[0041] The display unit 16 is, for example, a display having a touch panel, and displays an image of a subject photographed by the photographing unit 14 and accepts input of operation instructions and the like from an operator. The display unit 16 may be provided separately from the terminal device 1, for example, as a display device having a touch panel.

[0042] 5 is a diagram illustrating a functional configuration of the control unit 11 of the terminal device 1 according to the embodiment. The control unit 11 of the terminal device 1 includes, as functional components, for example, a user coordinate system setting unit 111, a coordinate assignment unit 112, a movement path setting unit 113, a program creation unit 114, a robot coordinate system calculation unit 115, and a program correction unit 116.

[0043] The user coordinate system setting unit 111 sets a three-dimensional user coordinate system based on the workpiece marker Mw included in the first image captured by the imaging unit 14. The first image includes the workpiece W and the workpiece marker Mw, as shown in FIG.

[0044] Specifically, the user coordinate system setting unit 111 calculates three-dimensional coordinate axes (orientations) with a specific position of the workpiece marker Mw (for example, a corner of the marker, the center of the marker, or the like) as the origin, and sets the user coordinate system.

[0045] The origin of the user coordinate system is not limited to the specific position of the workpiece marker Mw, and may be, for example, the origin of a provisional robot coordinate system that is determined based on the specific position of the workpiece marker Mw. The provisional robot coordinate system can be set, for example, by placing a virtual model of a manipulator next to the workpiece W and setting the virtual model as the coordinate system of the virtual model. By fixing the position of the workpiece marker Mw with respect to the virtual model of the manipulator, it becomes possible to identify the specific position of the workpiece marker Mw on the provisional robot coordinate system. In this case, the user coordinate system can be treated as the provisional robot coordinate system.

[0046] The position where the virtual model of the manipulator is placed may be arbitrarily specified by the operator by touching the image of the working space, following a guide such as a voice or text message.

[0047] 5 assigns coordinates in a user coordinate system to point cloud data obtained by a distance measurement unit 15 that measures distances to objects included in an image captured by the imaging unit 14. This will be specifically described below.

[0048] The coordinate assignment unit 112 detects a specific position of the workpiece marker Mw based on the first image, and sets the detected specific position of the workpiece marker Mw on the point cloud data obtained by the distance measurement unit 15. The coordinate assignment unit 112 assigns coordinates of a user coordinate system, which has the specific position of the workpiece marker Mw set on the point cloud data as its origin (reference), to the point cloud data.

[0049] This allows the point cloud data to be drawn as data in a user coordinate system. The specific position of the work marker Mw set on the point cloud data may be automatically recognized by data analysis, or may be specified by an operator pointing out the specific position of the work marker Mw on the point cloud data.

[0050] The movement path setting unit 113 sets a movement path of the manipulator 3 in the user coordinate system based on the welding location of the workpiece W included in the first image.

[0051] Illustratively, the motion path setting unit 113 sets a motion path of the virtual model of the manipulator such that, in the first image, a welding torch connected to the tip of the virtual model of the manipulator moves along a welding point of the workpiece W. The motion path may be set by manual selection by a user, for example, or may be set automatically by calculating a distance between the welding point of the workpiece W and the welding torch.

[0052] In addition, the welding points of the workpiece W may be selected by a user from among the detected welding point candidates, or by detecting welding point candidates by subjecting the first image to image processing, and selecting welding points from among the detected welding point candidates. When selecting welding points, the user may select or specify the welding order, welding direction, and the like of the welding points.

[0053] The program creation unit 114 creates a welding program for welding the welding portion of the workpiece W included in the first image, based on the movement path of the manipulator 3 in the user coordinate system set by the movement path setting unit 113.

[0054] The robot coordinate system calculation unit 115 calculates the robot coordinate system based on the position and orientation information of the robot marker Mr and the manipulator 3 included in the second image captured by the imaging unit 14. The second image includes the workpiece W and the workpiece marker Mw that maintain the positional relationship in the first image, and the robot marker Mr attached to the manipulator 3, as shown in FIG.

[0055] Specifically, the robot coordinate system calculation unit 115 calculates the origin of the robot coordinate system based on the robot marker Mr, and calculates the coordinate axes (orientation) of the robot coordinate system based on the position and posture information of the manipulator 3, thereby calculating the robot coordinate system.

[0056] Here, the robot control device 2 holds coordinates in the robot coordinate system corresponding to the attachment position of the robot marker Mr, and position and orientation information of the manipulator 3 controlled by the robot control device 2. The position and orientation information of the manipulator 3 includes, for example, angle information of each axis of the manipulator 3.

[0057] Since the contents of the position and orientation information of the manipulator 3 may vary depending on the type of the manipulator 3, the type of the manipulator 3 may also be acquired when acquiring the position and orientation information of the manipulator 3 from the robot control device 2. In this case, it is preferable that the robot coordinate system calculation unit 115 calculates the coordinate axes (directions) of the robot coordinate system based on the position and orientation information of the manipulator 3 and the type of the manipulator 3.

[0058] Program correction unit 116 in FIG. 5 converts the coordinate system of the movement path of manipulator 3 from the user coordinate system to the robot coordinate system, and corrects the welding program.

[0059] For example, the program correction unit 116 calculates the difference between the origin of the user coordinate system and the origin of the robot coordinate system, and moves the coordinates of the movement path of the manipulator 3 in the user coordinate system based on the calculated difference.

[0060] This allows the coordinates of the movement path to be converted from the user coordinate system to the robot coordinate system. Program correction unit 116 corrects the welding program created by program creation unit 114 so that it is expressed by coordinates in the robot coordinate system after the coordinate conversion.

[0061] The program correction unit 116 transmits the corrected welding program to the robot controller 2 and stores it in the memory unit 22 of the robot controller 2. This enables the control unit 21 of the robot controller 2 to control the manipulator 3 to weld the welding points of the workpiece W included in the second image according to the corrected welding program.

[0062] An example of the operation of the terminal device 1 will be described with reference to FIG.

[0063] First, the photographing unit 14 of the terminal device 1 photographs a first image including the workpiece W that is the welding target of the manipulator 3 and the workpiece marker Mw placed on the workpiece W (step S101).

[0064] Next, the user coordinate system setting unit 111 of the terminal device 1 sets a three-dimensional user coordinate system based on the work marker Mw included in the first image captured in the above step S101 (step S102).

[0065] Next, the movement path setting unit 113 of the terminal device 1 sets a movement path of the manipulator 3 in the user coordinate system based on the welding portion of the workpiece W included in the first image (step S103).

[0066] Next, the program creation unit 114 of the terminal device 1 creates a welding program for welding the welding points of the workpiece W included in the first image based on the movement path of the manipulator 3 in the user coordinate system set in the above step S103 (step S104).

[0067] Next, the photographing unit 14 of the terminal device 1 photographs a second image including the workpiece W and the workpiece marker Mw that maintain the positional relationship in the first image, and the robot marker Mr attached to the manipulator 3 (step S105).

[0068] Next, the robot coordinate system calculation unit 115 of the terminal device 1 calculates a robot coordinate system based on the position and orientation information of the robot marker Mr and the manipulator 3 included in the second image captured in the above step S105 (step S106).

[0069] Next, the program correction unit 116 of the terminal device 1 converts the coordinate system of the movement path of the manipulator 3 set in the above step S103 from the user coordinate system to the robot coordinate system, and corrects the welding program created in the above step S104 (step S107). Then, this operation ends.

[0070] As described above, the terminal device 1 according to the embodiment can capture a first image including the workpiece W and the workpiece marker Mw to set a user coordinate system based on the workpiece marker Mw, and create a welding program for the manipulator 3 based on the movement path of the manipulator 3 in the user coordinate system set based on the welding location of the workpiece W. Then, by capturing a second image including the workpiece W and the workpiece marker Mw, which maintain the positional relationship in the first image, and the robot marker Mr attached to the manipulator 3, a robot coordinate system can be calculated based on the position and orientation information of the robot marker Mr and the manipulator 3, and the coordinate system of the movement path of the manipulator 3 can be converted from the user coordinate system to the robot coordinate system to correct the welding program.

[0071] This makes it possible to create a welding program for the manipulator 3 in the user coordinate system based on a first image that does not include the manipulator 3, and then convert the coordinate system representing the movement path of the manipulator 3 from the user coordinate system to the robot coordinate system based on a second image that is subsequently captured and that includes the robot marker Mr attached to the manipulator 3, and to modify the welding program based on the user coordinate system to a welding program based on the robot coordinate system.

[0072] Therefore, according to the terminal device 1 of the embodiment, the occupancy rate of the manipulator 3 when creating a welding program can be reduced, making it possible to create a welding program without reducing the operating rate of the manipulator 3.

[0073] It should be noted that the present invention is not limited to the above-described embodiment, and can be embodied in various other forms without departing from the spirit and scope of the present invention. Therefore, the above-described embodiment is merely an example, and should not be interpreted as being restrictive.

[0074] For example, in the above-mentioned embodiment, the welding robot is used for explanation, but the present invention is not limited to this. For example, the present invention can be applied to an industrial robot including a handling robot that performs picking and the like. In this case, the welding program, welding target, welding location, and welding operation used in the above-mentioned embodiment can be replaced with a work program, work target, work location, and work, respectively. [Explanation of symbols]

[0075] 1... terminal device, 2... robot control device, 3... manipulator, 11... control unit, 12... memory unit, 13... communication unit, 14... photography unit, 15... distance measurement unit, 16... display unit, 21... control unit, 22... memory unit, 23... communication unit, 100... welding robot system, 111... user coordinate system setting unit, 112... coordinate assignment unit, 113... operation path setting unit, 114... program creation unit, 115... robot coordinate system calculation unit, 116... program correction unit, Mr... robot marker, Mw... work marker, W... work

Claims

1. A first photographing unit that photographs a first image including a work object and a work object marker; a user coordinate system setting unit that sets a user coordinate system based on the work object marker included in the first image; a motion path setting unit that sets a motion path of the industrial robot in the user coordinate system based on a work location of the work object included in the first image; a program creation unit that creates a task program for operating the industrial robot based on a set motion path of the industrial robot; a second photographing unit that photographs a second image including a robot marker attached to the industrial robot in addition to the work object and the work object marker that maintain the positional relationship in the first image; a robot coordinate system calculation unit that calculates a robot coordinate system based on the robot marker and position and orientation information of the industrial robot included in the second image; a program correction unit that converts a coordinate system of a motion path of the industrial robot from the user coordinate system to the robot coordinate system and corrects the work program; A terminal device comprising:

2. the program correction unit converts a coordinate system of a motion path of the industrial robot based on a difference between an origin of the user coordinate system and an origin of the robot coordinate system. The terminal device according to claim 1.

3. The user coordinate system setting unit calculates an origin and coordinate axes of the user coordinate system based on the work object marker. The terminal device according to claim 1.

4. the robot coordinate system calculation unit calculates an origin and coordinate axes of the robot coordinate system based on the robot marker and position and orientation information of the industrial robot; The terminal device according to claim 1.

5. The position and orientation information of the industrial robot includes angle information of each axis of the industrial robot acquired from a robot control device that controls the operation of the industrial robot. The terminal device according to claim 1.

Images