Portable terminal
The teaching program generation device addresses inaccuracies in conventional robot teaching by using detailed robot and tool information to generate precise trajectories, preventing interference and ensuring smooth operation through 3D visualization.
Patent Information
- Application Number
- JP2023090246
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-31
- Publication Date
- 2024-12-12
- Estimated Expiration
- 2043-05-31
AI Technical Summary
Conventional teaching methods for robots fail to account for accurate information about tool shape, size, inertia, initial posture, or position, leading to inaccurate robot trajectories and potential interference when tools or robots are replaced.
A teaching program generation device that utilizes information from a robot control device, including tool shape, size, inertia, initial posture, and position, to generate precise robot trajectories while avoiding interference, using 3D models and point cloud data to visualize and plan robot movements.
Enables the generation of accurate robot trajectories that account for tool and robot changes, preventing interference and ensuring smooth operation by visually confirming replacements through 3D model display.
Smart Images

Figure 2024172490000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a teaching program generating device. [Background technology]
[0002] In the field of factory automation, a process is performed in which a program describing certain operations to be performed by a robot is created in advance, and this process is called teaching. The robot operates according to a program created in advance by teaching, and repeatedly executes certain operations. For example, Patent Document 1 describes a method of generating a trajectory for a robot to operate while avoiding obstacles, as an example of this type of teaching. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] Patent Publication No. 2021-65939 Summary of the Invention [Problem to be solved by the invention]
[0004] However, in conventional teaching, information about the tool attached to the robot (e.g., information about the shape, size, or inertia of the tool) is not used as information for generating a robot trajectory that enables the robot to perform a certain operation while avoiding interference, so when the tool attached to the robot is replaced, it is not possible to generate a robot trajectory that enables the robot to perform a certain operation while avoiding interference, taking into account the shape, size, or inertia of the tool, which differs for each tool.
[0005] Furthermore, in conventional teaching, accurate information about the robot's initial posture or initial position when the robot starts a specified movement is not used as information for generating a robot trajectory that enables the robot to perform a certain movement while avoiding interference, so it was not possible to generate a robot trajectory that enables the robot to perform a certain movement while avoiding interference by taking into account accurate information about the robot's initial posture or initial position.
[0006] In addition, in a conventional method of taking images of a robot and objects in its surrounding environment, measuring the distances to the robot and objects in its surrounding environment, calculating the relative positional relationship between the robot and objects in its surrounding environment based on the taken images and the measured distances, and creating a teaching program for teaching the robot's movements based on the calculation results, an operator had to input initial setting values (for example, the initial position or initial posture when the robot starts to move, or the type of tool attached to the robot, etc.). Therefore, when starting to create a teaching program for teaching a robot a predetermined movement, if the position or posture of the robot has changed or the tool has been replaced, there has been a problem that the initial setting values input by the operator are not necessarily accurate information.
[0007] Therefore, an object of the present invention is to solve such problems and to appropriately generate a robot trajectory that enables the robot to perform a certain operation while avoiding interference. [Means for solving the problem]
[0008] In order to solve the above-mentioned problems, the teaching program generating device according to the present invention is a teaching program generating device that generates a teaching program for teaching a robot a predetermined motion based on information about the robot and its surroundings, and includes an acquisition unit that acquires information used by the robot to perform the predetermined motion while avoiding interference from a robot control device that controls the driving of the robot, and a teaching program creating unit that generates a trajectory along which the robot moves to perform the predetermined motion while avoiding interference based on the above-mentioned information. By acquiring the information used by the robot to perform the predetermined motion while avoiding interference from the robot control device, it is possible to appropriately generate a trajectory for the robot to perform a certain motion while avoiding interference. Note that the information about the robot and its surroundings may be, for example, images of the robot and its surroundings, or point cloud data of the robot and its surroundings.
[0009] Here, the information acquired from the robot control device may include, for example, any of information on the shape, size, or inertia of a tool replaceably attached to the robot. In this way, when the tool attached to the robot is replaced, a trajectory of the robot that allows the robot to perform a certain operation while avoiding interference can be generated, taking into account the shape, size, inertia, etc. of each tool, which differs from one tool to another.
[0010] Furthermore, the information acquired from the robot control device may include, for example, information on the posture or position of the robot when the robot starts a predetermined motion, which allows a trajectory of the robot to perform a certain motion while avoiding interference to be generated, taking into account accurate information on the initial posture or initial position of the robot when the robot starts a predetermined motion.
[0011] The information acquired from the robot control device may also include information indicating the type of the robot, so that when the robot to be taught is replaced, the teaching program generation device can ascertain the correct type of the robot.
[0012] The information acquired from the robot control device may include, for example, information indicating the type of the robot and information indicating the type of a tool replaceably attached to the robot. The teaching program generation device may include a storage unit that stores three-dimensional models of multiple types of robots and three-dimensional models of multiple types of tools replaceably attached to the robot, and a display unit that reads out the three-dimensional model of the robot corresponding to the type of the robot and the three-dimensional model of the tool corresponding to the type of the tool from the storage unit and displays the read three-dimensional model of the robot and the three-dimensional model of the tool. In this way, when the robot or the tool is replaced, the three-dimensional model corresponding to the type of the replaced robot or the type of the tool is displayed, so that the worker can visually confirm that the robot or the tool has been replaced. Effect of the Invention
[0013] According to the present invention, it is possible to appropriately generate a robot trajectory that enables the robot to perform a certain operation while avoiding interference. [Brief description of the drawings]
[0014] [Figure 1] 1 is a diagram illustrating a configuration of a robot system according to an embodiment of the present invention. [Diagram 2] 1 is a diagram illustrating a configuration of a robot control device according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a functional block diagram of a control unit of a terminal device according to an embodiment of the present invention. [Figure 4] 1 is a flowchart showing a process for generating a movement trajectory of a robot according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0015] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Here, the same reference numerals denote the same components, and duplicated explanations will be omitted.
[0016] 1 shows the configuration of a robot system 100 according to an embodiment of the present invention. The robot system 100 includes, for example, a terminal device 1, a robot control device 2, and a robot (manipulator) 3. The terminal device 1 and the robot control device 2, and the robot control device 2 and the robot 3 are connected via a network. The terminal device 1 and the robot control device 2 are connected by wireless communication such as WiFi (Wireless Fidelity), and the robot control device 2 and the robot 3 are connected via, for example, a communication cable. The network may be wired (including a communication cable) or wireless.
[0017] The robot 3 includes a multi-joint arm provided on a base member fixed to the floor surface of a factory or the like, and a tool 31 attached to the tip of the multi-joint arm. The tool 31 can be replaceably attached to the robot 3. By using, for example, a welding torch as the tool 31, the robot 3 can perform arc welding on the workpiece W to be welded, in accordance with a command from the robot control device 2. Also, by using, for example, an end effector as the tool 31, the robot 3 can move the workpiece W while gripping it, in accordance with a command from the robot control device 2. Any one tool selected from a plurality of types of tools can be replaceably attached to the robot 3.
[0018] A marker M is attached to the robot 3. The marker M is an identifier that allows the terminal device 1 to recognize the presence of the marker M in the working space, and is, for example, an AR marker. By using the AR marker, when the terminal device 1 recognizes an AR marker that exists in the working space, it becomes possible to easily realize superimposing and displaying a user coordinate system with the AR marker as the origin on an actual image.
[0019] 2, the robot control device 2 is a control unit that controls the driving of the robot 3. The robot control device 2 includes, for example, a control unit 21, a storage unit 22, and a communication unit 23.
[0020] The control unit 21 is a processor, and controls the robot 3 by executing programs and the like stored in the storage unit 22 .
[0021] The communication unit 23 is a communication interface, and controls communication with the terminal device 1 and the robot 3 connected via a network.
[0022] The terminal device 1 functions as a teaching program generating device that generates a teaching program for teaching a predetermined operation to the robot 3. The terminal device 1 is, for example, a portable terminal with a digital camera. Portable terminals include, for example, tablet terminals, smartphones, personal digital assistants (PDAs), notebook PCs (personal computers), and other portable terminals.
[0023] The terminal device 1 includes, for example, a control unit 11, a storage unit 12, a communication unit 13, an imaging unit 14, a distance measurement unit 15, and a display unit 16.
[0024] The control unit 11 is a processor, and controls each unit of the terminal device 1 by executing a program stored in the storage unit 12 .
[0025] The storage unit 12 is a computer-readable recording medium, and stores programs for implementing various functions of the terminal device 1 and various data used in the programs. The various data stored in the storage unit 12 includes, for example, three-dimensional models of each of the multiple types of robots 3 and three-dimensional models of each of the multiple types of tools 31 that can be attached to the robots 3 in an exchangeable manner.
[0026] The communication unit 13 is a communication interface, and controls communication with the robot control device 2 connected via a network.
[0027] The photographing unit 14 is, for example, a 2D camera including a lens and an image sensor (image pickup element), and converts the light of the subject received by the lens into an electrical signal (digital image data). The photographing unit 14 photographs the robot and its surroundings in response to instructions from the operator.
[0028] The distance measurement unit 15 is, for example, a 3D camera equipped with a distance measurement sensor. The distance measurement sensor is a sensor capable of measuring the distance to an object. For example, a LiDAR (Light Detection and Ranging) sensor, a millimeter wave sensor, an ultrasonic sensor, etc. can be used as the distance measurement sensor. The LiDAR collects three-dimensional information of objects existing around it as point cloud data. The point cloud data has a data format in which each pulse reflected from the surrounding objects detected by the LiDAR is expressed as a point. The distance measurement sensor collects point cloud data of the robot and its surroundings in response to an instruction from an operator.
[0029] The display unit 16 is, for example, a display having a touch panel, and displays the image of the subject photographed by the photographing unit 14 and accepts inputs such as operational instructions from an operator.
[0030] 3 shows a functional block diagram of the control unit 11 of the terminal device 1. The control unit 11 executes a program stored in the storage unit 12, thereby realizing functions as an acquisition unit 111 and a teaching program creation unit 112.
[0031] The acquisition unit 111 acquires the following information (a) to (d) from the robot control device 2, which is used for the robot 3 to perform a predetermined operation while avoiding interference.
[0032] (a) Information indicating the type of robot 3. (b) Information indicating the type of tool 31 replaceably attached to the robot 3. (c) Any information regarding the shape, size, or inertia of the tool 31 that is replaceably attached to the robot 3. (d) Information on the posture or position of the robot 3 when the robot 3 starts a predetermined motion.
[0033] As the information (d), an output value from an encoder that detects the angle of each joint attached to the articulated arm of the robot 3 can be used.
[0034] In addition to the above-mentioned information (a) to (d), version information of the program stored in the memory unit 22 of the robot control device 2 may be used as information used by the robot 3 to perform a predetermined operation while avoiding interference.
[0035] The teaching program creation unit 112 sets a three-dimensional user coordinate system based on the marker M included in the image captured by the imaging unit 14. The image captured by the imaging unit 14 includes, for example, the robot 3 and a workpiece W that exist in a space in which the robot 3 works, as shown in Fig. 1. A marker M is attached to the tip of the robot 3. The teaching program creation unit 112 sets a three-dimensional user coordinate system with a specific position of the marker M (for example, a corner of the marker, the center of the marker, etc.) as the origin.
[0036] The origin of the user coordinate system is not limited to the specific position of the marker M, and may be the origin of the robot coordinate system defined based on the specific position of the marker M. For example, by fixing the position of the marker M with respect to the robot 3, the specific position of the marker M can be identified on the robot coordinate system, and therefore it is possible to identify the origin of the robot coordinate system based on the specific position of the marker M. In this case, the user coordinate system becomes the robot coordinate system.
[0037] The teaching program creation unit 112 assigns coordinates in a user coordinate system to point cloud data obtained by the distance measurement unit 15 that measures the distance to an object included in an image captured by the imaging unit 14. For example, the teaching program creation unit 112 detects a specific position of the marker M based on the image captured by the imaging unit 14, and sets the detected specific position of the marker on the point cloud data obtained by the distance measurement unit 15. The teaching program creation unit 112 assigns coordinates of a user coordinate system to the point cloud data, with the specific position of the marker set on the point cloud data as the origin (reference). As a result, the point cloud data is drawn in the user coordinate system. The specific position of the marker set on the point cloud data may be automatically recognized by data analysis, or may be specified by an operator pointing out the specific position of the marker on the point cloud data.
[0038] The teaching program creation unit 112 reads out from the storage unit 12 a three-dimensional model of the robot 3 corresponding to the type of the robot 3 and a three-dimensional model of the tool 31 corresponding to the type of the tool 31. Using the read out three-dimensional model of the robot 3 and the three-dimensional model of the tool 31, the teaching program creation unit 112 calculates a robot area in the user coordinate system corresponding to the robot 3 and the tool 31 based on the above-mentioned information (a) to (d). The robot area is a virtual three-dimensional area that the robot 3 and the tool 31 occupy in the user coordinate system.
[0039] The initial posture or initial position of the robot 3 when the robot 3 starts a predetermined operation may be, for example, the posture or position of the robot 3 when the image is captured by the image capture unit 14.
[0040] The teaching program creation unit 112 generates a trajectory along which the robot 3 moves to perform a predetermined operation so that the robot area does not interfere with the point cloud data of objects existing around it. Interference means that the robot 3 or the tool 31 comes into contact with an object existing around it, hindering the normal operation of the robot 3.
[0041] In this way, once a trajectory for the robot 3 to move is generated so that the robot area does not interfere with the point cloud data of objects existing around it, the teaching program creation unit 112 converts the movement trajectory of the robot 3 in the user coordinate system into the movement trajectory of the robot 3 in the robot coordinate system, and generates a teaching program for teaching the robot 3 a specified movement.
[0042] The display unit 16 of the terminal device 1 displays a three-dimensional model of the robot 3 corresponding to the type of the robot 3, and a three-dimensional model of the tool 31 corresponding to the type of the tool 31. This allows the worker to understand from the movements of the three-dimensional models what trajectories the robot 3 and the tool 31 will follow and how they will move while avoiding interference, without actually driving the robot 3 and the tool 31.
[0043] The display unit 16 of the terminal device 1 may change a graphical user interface for receiving, from the worker, an input of an operation instruction for causing the robot 3 to execute a predetermined operation, in accordance with the above-mentioned information (a) to (b). For example, when a welding torch is used as the tool 31, the display unit 16 may display a graphical user interface for receiving, from the worker, an input of an instruction for specifying a welding location on the workpiece W. Also, for example, when an end effector is used as the tool 31, the display unit 16 may display a graphical user interface for receiving, from the worker, an input of an instruction for specifying a location to grip the workpiece W.
[0044] FIG. 4 is a flowchart showing the flow of a process for generating a movement trajectory of the robot 3.
[0045] The worker operates the terminal device 1 and captures an image using the image capturing unit 14 (step 401). The control unit 11 of the terminal device 1 sets a three-dimensional user coordinate system based on the marker M included in the image captured by the image capturing unit 14, and assigns coordinates of the user coordinate system to point cloud data obtained by the distance measurement unit 15 that measures the distance to an object included in the image captured by the image capturing unit 14.
[0046] The control unit 11 of the terminal device 1 requests the robot control device 2 to acquire the above-mentioned information (a) to (d) from the robot control device 2 (step 402).
[0047] If the above request is permitted by the robot control device 2 (step 403; YES), the control unit 11 of the terminal device 1 acquires the above information (a) to (d) from the robot control device 2 (step 404).
[0048] If the above request is not permitted by the robot control device 2 (step 403; NO), the control unit 11 of the terminal device 1 reads information preset as default values for the above information (a) to (d) from the memory unit 12 (step 405).
[0049] The control unit 11 of the terminal device 1 calculates a robot area in the user coordinate system corresponding to the robot 3 and the tool 31 using the information (a) to (d) acquired in step 404 or the default value read from the storage unit 12 in step 405, and generates a trajectory for the robot 3 to move so that the robot area does not interfere with the point cloud data of objects existing around it. The control unit 11 of the terminal device 1 converts the movement trajectory of the robot 3 in the user coordinate system into a movement trajectory of the robot 3 in the robot coordinate system, and generates a teaching program for teaching a predetermined operation to the robot 3 (step 406).
[0050] The control unit 11 of the terminal device 1 transmits the teaching program to the robot control device 2 (step 407).
[0051] According to an embodiment of the present invention, the above-mentioned information (b) to (c) is used as information for generating a trajectory of the robot 3 so that the robot 3 to which the tool 31 is attached can perform a certain operation while avoiding interference. Therefore, when the tool 31 attached to the robot 3 is replaced, a trajectory of the robot 3 can be generated so that the robot 3 can perform a certain operation while avoiding interference, taking into account the shape, size, inertia, etc. of the tool 31 which differs for each tool 31.
[0052] Furthermore, according to an embodiment of the present invention, the above-mentioned information (d) is used as information for generating a trajectory of the robot 3 so that the robot 3 performs a certain operation while avoiding interference, so that a trajectory of the robot 3 so that the robot 3 performs a certain operation while avoiding interference can be generated, taking into account accurate information on the initial posture or initial position of the robot 3 when the robot 3 starts a specified operation.
[0053] Furthermore, according to an embodiment of the present invention, when the robot 3 or tool 31 is replaced, a three-dimensional model corresponding to the type of robot 3 or the type of tool 31 after replacement is displayed, allowing the worker to visually confirm that the robot 3 or tool 31 has been replaced.
[0054] The above-described embodiments are provided to facilitate understanding of the present invention, and are not intended to limit the present invention. The present invention may be modified or improved without departing from the spirit of the present invention, and equivalents are also included in the present invention. In other words, even if a person skilled in the art appropriately modifies the design of the embodiments, the present invention will be included in the scope of the present invention as long as the features of the present invention are included. In addition, the elements of the embodiments can be combined to the extent technically possible, and the combination of these elements will be included in the scope of the present invention as long as the features of the present invention are included. [Explanation of symbols]
[0055] 1... Terminal device 11... Control unit 12... Memory unit 13... Communication unit 14... Photographing unit 15... Distance measuring unit 16... Display unit 2... Robot control device 21... Control unit 22... Memory unit 23... Communication unit 3... Robot 31... Tool 111... Acquisition unit 112... Teaching program creation unit
Claims
1. A teaching program generation device that generates a teaching program for teaching a robot a predetermined motion based on information about the robot and its surroundings, an acquisition unit that acquires information used for the robot to perform the predetermined operation while avoiding interference from a robot control device that controls driving of the robot; a teaching program creation unit that generates a trajectory for the robot to move while avoiding interference and perform the predetermined operation based on the information; A teaching program generating device comprising:
2. The teaching program generating device according to claim 1, A teaching program generating device, wherein the information includes any one of information on the shape, size, or inertia of a tool that is replaceably attached to the robot.
3. The teaching program generating device according to claim 1, A teaching program generating device, wherein the information includes information on the posture or position of the robot when the robot starts the specified movement.
4. The teaching program generating device according to claim 1, A teaching program generating device, wherein the information includes information indicating the type of the robot.
5. The teaching program generating device according to claim 1, the information includes information indicating a type of the robot and information indicating a type of a tool replaceably attached to the robot, The teaching program generating device comprises: a storage unit that stores three-dimensional models of a plurality of types of robots and three-dimensional models of a plurality of types of tools that can be interchangeably attached to the robot; a display unit that reads out from the storage unit a three-dimensional model of the robot corresponding to the type of the robot and a three-dimensional model of the tool corresponding to the type of the tool, and displays the read three-dimensional model of the robot and the three-dimensional model of the tool; A teaching program generating device comprising:
Citation Information
Patent Citations
System and method for construction of operating environment model
JP2012135858A
Robot teaching system
JP2023069253A
Robot, control method, and program
WO2021132107A1
Robot teaching method and robot working method
WO2021261411A1
Avoidance trajectory generation device, method and program
JP2021065939A
