control device

Abstract

A control device (40) comprising: a detection result acquisition unit (161) which acquires first information corresponding to a detection result obtained through detecting, by means of a visual sensor, the relative positional relationship between a robot (30) and an object; a coordinate system data output unit (162) which, on the basis of the first information, outputs first coordinate system data as data representing a coordinate system that serve as the basis when operating the robot; and a command generation unit (163) which generates a command for the robot on the basis of the first coordinate system represented by the first coordinate system data.

Description

Technical Field

[0001] The present invention relates to a control device for a robot.

Background Art

[0002] In order to intuitively teach a control program of a robot, a control device that enables programming using icons representing functions constituting the control program of the robot has been proposed (for example, Patent Document 1). In addition, a robot system that performs so-called bulk picking or the like, which images a workpiece with a vision sensor mounted on the robot, grips the appropriate position of the workpiece, and places it at a desired position, is also known (for example, Patent Document 2).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a robot system that performs operations on a workpiece, the relative positional relationship between the workpiece and the robot may change due to the workpiece being displaced from the intended position or the like. In such a case, in order for the user to realize a control program for correcting the position of the robot to appropriately handle the workpiece, the user is required to have advanced knowledge for programming such as the data format of the position data as a detection result and the data format of the coordinate system data. Therefore, generally, it is difficult for the user to handle such programming for realizing the correction of the position of the robot. It is desirable to mount a function on the control device that enables the function of realizing such correction of the position of the robot to be used in a more user-friendly manner. [Means for solving the problem]

[0005] One aspect of the present disclosure includes: a detection result acquisition unit that acquires first information corresponding to a detection result in which a visual sensor detects the relative positional relationship between a robot and an object; a coordinate system data output unit that outputs first coordinate system data as data representing a coordinate system to be followed when operating the robot based on the first information; and a command generation unit that generates a command for the robot in accordance with the first coordinate system represented by the first coordinate system data. An icon control unit provides the functions of the detection result acquisition unit and the coordinate system data output unit as functions implemented in at least one icon representing a function that constitutes the robot's control program, This is a control device equipped with [a specific feature / feature].

[0006] Another aspect of the present disclosure includes a command generation unit that generates commands for a robot using a first coordinate system as the coordinate system to be followed when operating the robot, a detection result acquisition unit that acquires first information corresponding to a detection result of detecting the relative positional relationship between the robot and an object using a visual sensor, and a coordinate system shift unit that shifts the first coordinate system based on the first information. The icon control unit provides the functions of the detection result acquisition unit and the coordinate system shift unit as functions implemented in an icon representing a function that constitutes the robot's control program, This is a control device equipped with [a specific feature / feature]. [Effects of the Invention]

[0007] According to the above configuration, it is possible to provide a function that allows users to use the function of correcting the robot's position based on the detection results from the visual sensor when the relative positional relationship between the robot and the object changes in a more user-friendly manner.

[0008] These and other objects, features, and advantages of the present invention will become even clearer from the detailed description of typical embodiments of the present invention shown in the accompanying drawings. [Brief explanation of the drawing]

[0009] [Figure 1] This diagram shows the overall configuration of the robot system including the control device according to the first embodiment. [Figure 2] This diagram shows an example of the hardware configuration of a robot control device and a teaching / operation device. [Figure 3] This is a block diagram showing the functional configuration of the control device according to the first embodiment. [Figure 4] This figure shows an example of a program creation screen created by the screen display creation unit. [Figure 5] This figure shows an example of a control program in the first embodiment. [Figure 6] This diagram shows the settings screen for configuring the "User Coordinate System Settings" icon. [Figure 7] This diagram shows the settings screen for when you want to teach the "Take / Place" icon. [Figure 8] This diagram shows the settings screen for providing instructions regarding the hand closed icon. [Figure 9] This is a block diagram showing the functional configuration of the control device according to the second embodiment. [Figure 10] This figure shows an example of a control program in the second embodiment. [Figure 11] This diagram shows the settings screen for configuring the detailed settings of the "User Coordinate System Settings (Shift)" icon. [Figure 12] This is a diagram showing the equipment configuration of the robot system according to the third embodiment. [Figure 13] This is a block diagram showing the functional configuration of the control device according to the third embodiment. [Figure 14] This diagram shows the arrangement of markers on a table. [Figure 15] This is a flowchart representing the command generation process. [Figure 16] This is a flowchart illustrating the user coordinate system shift process. [Modes for carrying out the invention]

[0010] Next, embodiments of the present disclosure will be described with reference to the drawings. In the drawings to be referred to, the same components or functional parts are denoted by the same reference numerals. For ease of understanding, the scales of these drawings are appropriately changed. Also, the forms shown in the drawings are examples for implementing the present invention, and the present invention is not limited to the illustrated forms.

[0011] First Embodiment FIG. 1 is a diagram showing the overall configuration of a robot system 100 including a control device 40 according to the first embodiment. In the present embodiment, a control device 40 for controlling the robot 30 is configured by functions provided by the robot control device 50 and the teaching operation device 10. The control device 40 is a device that enables programming using icons representing functions constituting the control program of the robot 30 (that is, representing commands for robot control). As a robot system including such a control device 40, various configuration examples are possible. In the present embodiment, as an example, the robot system 100 shown in FIG. 1 will be described. The robot system 100 includes a robot 30 having a hand 33 mounted at the tip of an arm, a robot control device 50 for controlling the robot 30, a teaching operation device 10 connected to the robot control device 50, a vision sensor 70 attached to the tip of the arm of the robot 30, and a vision sensor control device 20 for controlling the vision sensor 70. The vision sensor 70 is connected to the robot control device 50.

[0012] The robot 30 is a vertically articulated robot in this example, but other types of robots may be used. The teaching operation device 10 is used as a device for performing operation input and screen display for teaching the robot 30 (that is, creating a control program). As the teaching operation device 10, a tablet-type teaching operation device, a teaching operation panel, or an information processing device such as a PC (personal computer) equipped with other teaching functions may be used.

[0013] The vision sensor control device 20 has the function of controlling the vision sensor 70 and the function of performing image processing on the image captured by the vision sensor 70. The vision sensor control device 20 can detect the position of the object (hereinafter also referred to as workpiece) 1 placed on the workbench 2 from the image captured by the vision sensor 70, record the detection result, and provide the position of workpiece 1 as the detection result to the robot control device 50. As a result, the robot 30 (robot control device 50) can correct the taught position and perform operations such as picking up workpiece 1.

[0014] The visual sensor 70 may be a camera that captures grayscale or color images, or a stereo camera or 3D sensor that can acquire depth images or 3D point clouds. Multiple visual sensors may be installed in the robot system 100. The visual sensor control device 20 holds a model pattern of the object and performs image processing to detect the object by matching the image of the object in the captured image with the model pattern. The visual sensor 70 is assumed to be calibrated, and the visual sensor control device 20 is assumed to hold calibration data. Therefore, the relative positional relationship between the robot 30 and the visual sensor 70 is known to the control device 40 (robot control device 50).

[0015] In Figure 1, the vision sensor control device 20 is configured as a separate device from the robot control device 50, but the functions of the vision sensor control device 20 may also be incorporated into the robot control device 50.

[0016] In such a robot system 100, for example, the relative positional relationship between the workpiece 1 and the robot 30 may change if the workpiece 1 deviates from its reference position. The control device 40 uses the detection function of the visual sensor 70 to provide a function that allows the workpiece 1 to be handled appropriately even in such cases, in a form that is easy for the user to use.

[0017] Figure 2 shows an example of the hardware configuration of the robot control device 50 and the teaching and operation device 10. The robot control device 50 may have a configuration similar to a general computer, with a processor 51 connected to memory 52 (ROM, RAM, non-volatile memory, etc.), input / output interface 53, and an operation unit 54 including various operation switches via a bus. The teaching and operation device 10 may have a configuration similar to a general computer, with a processor 11 connected to memory 12 (ROM, RAM, non-volatile memory, etc.), a display unit 13, an operation unit 14 consisting of an input device such as a keyboard (or software keys), and an input / output interface 15 via a bus.

[0018] Figure 3 is a block diagram showing the functional configuration of a control device 40, which consists of a robot control device 50 and a teaching / operation device 10. The functions of the control device 40 shown in Figure 3 may be realized by the processor of the robot control device 50 or the teaching / operation device 10 executing various software stored in a memory device, or they may be realized by a hardware-based configuration such as an ASIC (Application Specific Integrated Circuit).

[0019] As shown in Figure 3, the robot control device 50 includes a robot motion control unit 151, a program creation unit 152, and a storage unit 153.

[0020] The robot motion control unit 151 controls the movement of the robot 30 according to a control program or commands from the teaching device 10. Specifically, the robot motion control unit 151 generates a trajectory plan for a predetermined control part of the robot 30 (e.g., TCP (Tool Center Point)) and generates commands for each axis of the robot 30 through kinematic calculations, according to commands from the control program or teaching device. Then, the robot motion control unit 151 moves the predetermined control part of the robot 30 according to the planned trajectory by executing servo control for each axis according to the commands for each axis.

[0021] The memory unit 153 stores control programs and various setting information related to teaching. The memory unit 153 may be configured as a non-volatile memory within the memory 52, for example. The information related to teaching includes setting information related to the coordinate system and information related to icons (data of the shape (image) of each icon, setting parameters, etc.).

[0022] The program creation unit 152 provides various functions for the user to perform programming using text-based commands or icons via the user interface (display unit 13 and operation unit 14) of the teaching operation device 10. The program creation unit 152 includes an icon control unit 154 and a screen display creation unit 155 as components that provide such functions.

[0023] The screen display creation unit 155 presents various interface screens used for programming and provides a function to accept user input. The various user interface screens may be configured as touch panel operation screens that can be operated by touch.

[0024] Figure 4 shows an example of a program creation screen 400 created by the screen display creation unit 155 and displayed on the display unit 13 of the teaching operation device 10. As shown in Figure 4, the program creation screen 400 includes an icon display area 200 that displays a list of various icons that can be used for programming, and a program creation area 300 for creating a control program by arranging the icons in order. The program creation area 300 is an area where icons are arranged in the order of execution, and is sometimes referred to as a timeline. In the example in Figure 4, the icon display area 200 includes a hand close icon 201 representing a command to close the hand, a hand open icon 202 representing a command to open the hand, a linear movement icon 203, an arc movement icon 204, an add waypoint icon 205, and a rotation icon 206 for rotating the hand.

[0025] Users can select icons, for example, by hovering their cursor over them. Users can also program by selecting desired icons from the icon display area 200 and placing them in the program creation area 300, for example, using drag-and-drop operations.

[0026] On the program creation screen 400, the user selects the Programming tab 261 when programming. The user can open a settings screen for configuring the detailed settings (parameter settings) of an icon by selecting an icon within the program creation area 300 and then selecting the Details tab 262. Furthermore, the user can execute a control program by performing a predetermined operation while an icon is placed in the program creation area 300.

[0027] The icon control unit 154 manages the function settings for icons and various user operations on icons. With the support of the icon control unit 154, users can configure detailed settings for icon functions, select desired icons from a list of icons placed in the icon display area 200, place them in the program creation area 300, and create control programs.

[0028] This section describes the coordinate systems that can be set in the robot's control system. Various coordinate systems can be set in the robot's control system, including the robot's inherent coordinate system and user-configurable coordinate systems. The robot's inherent coordinate system includes the world coordinate system, set at a location whose position does not change with the robot's orientation (e.g., the robot's base), and the faceplate coordinate system, set on the faceplate surface at the arm tip that forms the robot's mechanical interface. User-configurable coordinate systems include the work coordinate system, set on the workpiece to have a specific relationship with its position and orientation. The work coordinate system is set by using the world coordinate system as a reference and adding at least one of a certain translational movement and rotation to the world coordinate system. User-configurable coordinate systems also include the tool coordinate system, which represents the position and orientation of the tool tip relative to the faceplate coordinate system. The user can specify any of these coordinate systems as the coordinate system to follow when operating the robot.

[0029] Furthermore, by setting the user-configurable work coordinate system or tool coordinate system as the coordinate system used by the control device, operations to teach the robot to perform tasks on an object (such as jog operations using the X, Y, and Z axis keys of the teaching operation device) can be carried out smoothly. In this specification, the user-configurable coordinate system may be referred to as the user coordinate system. Below is an example of the data format for user-configurable coordinate system data. This example is a work coordinate system, representing position and orientation relative to the world coordinate system (X, Y, and Z represent 3D position, and W, P, and R represent rotation around the X, Y, and Z axes, respectively). User coordinate system UF1: X=180.000mm Y=0.000mm Z=180.000mm W=180.000deg P=0.000deg R=-90.000deg

[0030] The icon control unit 154 includes a detection result acquisition unit 161, a coordinate system data output unit 162, and a command generation unit 163.

[0031] The detection result acquisition unit 161 acquires information representing the detection result, which is the relative positional relationship between the robot 30 and the workpiece 1 detected by the vision sensor 70. Specifically, the detection result acquisition unit 161 functions as a position data acquisition unit that retrieves position data as a detection result selected by the user from a dedicated storage location. The coordinate system data output unit 162 outputs coordinate system data representing the coordinate system to be followed when operating the robot 30, based on the information acquired by the detection result acquisition unit 161. The command generation unit 163 generates robot operation commands (various operation commands including linear movement, circular movement, workpiece retrieval, etc.) according to the content taught by the user.

[0032] In this embodiment, the icon control unit 154 provides the functions of the detection result acquisition unit 161 and the coordinate system data output unit 162 as functions implemented in icons. As icons related to such functions, the "User Coordinate System Setting" icon 251 and the "User Coordinate System Selection" icon 252 are provided (see Figure 5). The "User Coordinate System Setting" icon 251 is given the functions of the detection result acquisition unit 161 and the coordinate system data output unit 162. The "User Coordinate System Selection" icon 252 provides the function of switching the coordinate system that the robot 30 conforms to to the coordinate system output by the "User Coordinate System Setting" icon 251 (in other words, switching the user coordinate system).

[0033] Figure 5 shows control program 501 as an example of a control program that uses such "User Coordinate System Setting" icon 251 and "User Coordinate System Selection" icon 252. Control program 501 includes a "View" icon 211 corresponding to a command for the detection function by the visual sensor 70, a "User Coordinate System Setting" icon 251, a "User Coordinate System Selection" icon 252, two linear movement icons 212, a "Take / Place" icon 213 for the action of taking or placing a workpiece, and a hand close icon 214 as a function of the workpiece removal operation.

[0034] The "View" icon 211 provides a function that detects a workpiece using a visual sensor and outputs a detection result including the position data of the detected workpiece. The workpiece position data as a detection result represents, for example, the detected position of the workpiece in the coordinate system set within the View icon. This detected position is used together with the coordinate system in the "User Coordinate System Settings" icon 251.

[0035] The linear movement icon 212 provides a function to move a predetermined movable part of the robot (e.g., TCP) in a linear fashion. The "take / place" icon 213 provides a function to pick up a workpiece using the function of the hand close icon 214 which is within its range. The hand close icon 214 corresponds to a command to close the hand and grasp the workpiece.

[0036] Figure 6 shows the settings screen 450 for performing detailed settings (instructions) for the "User Coordinate System Settings" icon 251. This settings screen 450 can be launched, for example, by selecting the "User Coordinate System Settings" icon 251 on the program creation area 300 shown in Figure 5 and then selecting the details tab 262. The settings screen 450 includes a selection field 451 for selecting the "detection result to set in the user coordinate system" and a specification field 452 for specifying the "user coordinate system number to set".

[0037] The detection result selection field 451 is a field for selecting the detection result that the user wishes to use from among the detection results obtained by performing the operation of detecting an object using the vision sensor. By operating the triangular mark in the selection field 451, a drop-down list displaying a list of detection results can be displayed, and the desired detection result data can be selected from this list. The detection results are stored in a specific storage location (for example, the memory of the vision sensor control device 20, the memory of the robot control device 50, external memory, etc.). In this embodiment, the detection results are assumed to be stored in the vision register, which is an internal register of the robot control device 50. Therefore, the user may, for example, specify the register number of the vision register where the work detection result using the "view" icon 211 is stored. Alternatively, the user may specify the detection result by directly specifying the "view" icon 211, etc., which outputs the detection result.

[0038] The User Coordinate System Number specification field 452 is for specifying the user coordinate system number to output the coordinate system data corresponding to the position data extracted from the detection result selected in the detection result selection field 451. By manipulating the triangular mark in specification field 452, a dropdown list displaying a list of user coordinate systems (UF1, UF2, UF3, etc.) can be displayed, and the user coordinate system of the desired number can be selected from this list. As a result, the position data is extracted from the detection result selected in selection field 451 and output as coordinate system data for the user coordinate system specified in specification field 452.

[0039] Let's explain the operation procedure of the function assigned to the "User Coordinate System Settings" icon 251 using an example. Assume that the number of the vision register selected in the detection result selection field 451 is vision register 01, and the user coordinate system number specified in the user coordinate system number specification field 452 is UF1. Assume that vision register 01 indicates the detected position P1 (3D position) of the workpiece obtained from the view icon 251. Assume that the detected position P1 is the position on coordinate system UF1 set in the view icon 251. (Procedure 1) The detection result acquisition unit 161 retrieves the detection position P1 from the vision register 01. (Step 2) The coordinate system data output unit 162 converts the detected position P1 into the robot's position P1' in the world coordinate system using the following formula (1). P1' = UF1·P1 ···(1) Here, UF1 represents the origin position of UF1 as seen from the world coordinate system, and is expressed as a simultaneous transformation matrix. (Step 3) The coordinate system data output unit 162 directly inputs the value of P1' into the user coordinate system specified in the specification field 452. As a result, the user coordinate system specified in the specification field 452 becomes a coordinate system with P1' as the origin, as seen from the world coordinate system.

[0040] Through the above operations, the coordinate system data representing the detected position data selected by the user can be output as a user coordinate system with a number specified by the user.

[0041] The teaching procedure for teaching the control program 501 in this embodiment is as follows. (Procedure T1) Instruct the user to click the "View" icon 211 and the "User Coordinate System Settings" icon 251, respectively. (Procedure T2) By executing the "View" icon 211 and the "User Coordinate System Settings" icon 251 respectively, the coordinate system data values ​​are entered into the user coordinate system specified by the "User Coordinate System Settings" icon 251. (Procedure T3) In that state, execute the "Select User Coordinate System" icon 252 to switch the coordinate system that the robot adheres to to the user coordinate system whose values ​​were entered in Procedure T2. (Procedure T4) Then, teach each action icon (linear movement icon 212, "take / place" icon 213).

[0042] In the above procedure (T4), during the teaching stage, the user coordinate system used by the user is a coordinate system transformed according to the detected position of the workpiece. Therefore, the user can proceed with teaching the robot without having to perform any operations to apply correction values.

[0043] Figures 7 and 8 show the setting screen 470 for teaching the "Pick / Place" icon 213 and the setting screen 480 for teaching the "Hand Close" icon 214 in the above teaching procedure T4. As shown in Figure 7, the setting screen 470 for the "Pick / Place" icon 213 has a specification field 471 for teaching the pick-up or place position, a specification field 472 for specifying the movement to the approach point to the workpiece (axis movement or linear movement), a specification field 473 for specifying the movement speed to the approach point, a specification field 474 for specifying the height between the pick-up or place position and the approach point, and a specification field 475 for specifying the movement speed between the pick-up / place position and the approach point. The user can move the robot 30 to a desired position for gripping the workpiece by jog operation, etc., and then press the memory button 471a to teach the "Pick-up / Place position" for picking up or placing the workpiece. At this stage, the coordinate system that the robot 30 adheres to is one with the workpiece detection position P1' as the origin. Therefore, the user does not need to perform any settings to apply the detection position (correction position) during the teaching stage of the "Pick / Place" icon 213.

[0044] As shown in Figure 8, the setting screen 480 for the hand close icon 214 includes a field 481 for specifying the name of the macro command for closing the hand, a specification field 482 for specifying load information to be switched according to the workpiece, and a specification field 483 for specifying the waiting time during the hand closing operation. The hand closing operation can be executed by pressing the close button 481a on the right.

[0045] Thus, according to this embodiment, the user can output coordinate system data based on the detection results as coordinate system data for the specified user coordinate system simply by performing a few simple operations on the "User Coordinate System Settings" icon 251: selecting the detection result to use and specifying the number of the user coordinate system for which they want to output the coordinate system data. The user does not need to individually apply corrections based on the detection results to each icon (linear movement icon, pick / place icon, etc.).

[0046] According to this embodiment, a coordinate system setting function can be added in a simple manner to various object detection methods provided based on the visual detection function (such as object position detection methods using the "View" icon, or object position detection methods by measuring markers (for example, measuring the relative positional relationship between the robot and the object by measuring the positions of three markers)). Even if a user wishes to set a coordinate system based on the detection results of these existing visual detection functions, they do not need to learn a new method; they only need to add the "User Coordinate System Setting" icon to these existing visual detection functions (such as the "View" icon). Furthermore, according to this embodiment, the method of correcting the robot's position based on the detection results of the visual detection function can be easily switched to a correction method using a coordinate system. That is, normally, it is necessary to perform an operation to correct the motion command icon using the position data (correction amount) output as the detection result of the "View" icon, but as in this embodiment, by activating the "User Coordinate System Setting" icon, it is possible to easily switch to a correction method using a coordinate system.

[0047] Thus, according to this embodiment, it is possible to provide a function that allows users to use the function of correcting the robot's position based on the detection results from the visual sensor when the relative positional relationship between the robot and the object changes in a more user-friendly manner.

[0048] Second Embodiment The control device according to the second embodiment will now be described. The configuration of the robot system including the control device according to the second embodiment, and the hardware configuration, are the same as those of the first embodiment shown in Figures 1 and 2. The control device according to the second embodiment differs from the first embodiment in that it has an icon function for providing coordinate system data based on detection results from a visual sensor, and the robot control device and control device according to the second embodiment will be referred to as robot control device 50A and control device 40A, respectively.

[0049] Figure 9 is a functional block diagram of the control device 40A according to the second embodiment. In Figure 9, functional blocks common to the control device 40 according to the first embodiment are denoted by the same reference numerals. The control device 40A, like the control device 40 according to the first embodiment, is a control device that enables programming using icons. Similar to the control device 40 according to the first embodiment, the control device 40A provides a function that allows the workpiece 1 to be handled appropriately even when the relative positional relationship between the workpiece 1 and the robot 30 changes, in a form that can be used in an easy-to-use manner by the user.

[0050] As shown in Figure 9, the robot control device 50A includes a robot motion control unit 151, a program creation unit 152a, and a storage unit 153. The program creation unit 152a provides various functions for the user to perform programming using text-based commands or icons via the user interface of the teaching operation device 10.

[0051] The program creation unit 152a includes an icon control unit 154a and a screen display creation unit 155. The icon control unit 154a is responsible for setting functions for icons and various user operations on icons.

[0052] The icon control unit 154a comprises a detection result acquisition unit 161a, a coordinate system shift unit 164, and a command generation unit 163a.

[0053] The command generation unit 163a generates commands for the robot using a specific coordinate system (for example, a user-specified user coordinate system) as the coordinate system to be followed when operating the robot. The detection result acquisition unit 161a acquires information representing the detection result, which is the relative positional relationship between the robot 30 and the workpiece 1 detected by the vision sensor 70. Specifically, the detection result acquisition unit 161a functions as a position data acquisition unit that retrieves position data as a detection result selected by the user from a dedicated storage location. The coordinate system shift unit 164 shifts the specific coordinate system used in the command generation unit 163a based on the position data as a detection result.

[0054] In this embodiment, the icon control unit 154a provides the functions of the detection result acquisition unit 161a and the coordinate system shift unit 164 as functions implemented in icons. As icons related to such functions, the "User Coordinate System Setting (Shift)" icon 271 and the "User Coordinate System Selection" icon 272 are provided (see Figure 10). The "User Coordinate System Setting (Shift)" icon 271 is given the functions of the detection result acquisition unit 161a and the coordinate system shift unit 164. The "User Coordinate System Selection" icon 272 provides the function of switching the coordinate system to which the robot 30 conforms to the coordinate system shifted by the "User Coordinate System Setting (Shift)" icon 271 (in other words, switching the user coordinate system).

[0055] Figure 10 shows a control program 502 as an example of a control program created using the "User Coordinate System Setting (Shift)" icon 271 and the "User Coordinate System Selection" icon 272. The control program 502 includes the "View" icon 211, which is a command for the detection function by the visual sensor, the "User Coordinate System Setting (Shift)" icon 271, the "User Coordinate System Selection" icon 272, two linear movement icons 212, the "Pick / Place" icon 213, and the hand close icon 214 as a function of the workpiece removal operation.

[0056] Figure 11 shows the settings screen 460 for performing detailed settings (instructions) for the "User Coordinate System Settings (Shift)" icon 271. This settings screen 460 can be launched, for example, by selecting the "User Coordinate System Settings (Shift)" icon 271 on the program creation area 300 shown in Figure 10 and then selecting the details tab 262. The settings screen 460 includes a selection field 461 for selecting the "detection result to set in the user coordinate system" and a specification field 462 for specifying the "user coordinate number to shift".

[0057] The detection result selection field 461 is a field for selecting the detection result that the user wants to use from among the detection results obtained by performing the action of detecting an object using the vision sensor (the action of the "view" icon). By operating the triangular mark in the selection field 461, a dropdown list displaying a list of detection results can be displayed, and the desired detection result data can be selected from this list. The detection results are stored in a specific storage location. In this embodiment, the detection results are assumed to be stored in the vision register, which is an internal register of the robot control device 50A. Therefore, the user may, for example, specify the register number of the vision register in which the work detection result from the "view" icon 211 is stored. Alternatively, the user may specify the detection result by directly specifying the "view" icon 211, etc., which outputs the detection result.

[0058] The User Coordinate System Specification field 462 is for specifying the number of the user coordinate system to be shifted by the amount of movement corresponding to the correction amount represented by the position data extracted from the detection result selected in the Detection Result Selection field 461. By manipulating the triangular mark in the specification field 462, a dropdown list displaying a list of user coordinate systems (UF1, UF2, UF3...) can be displayed, and the desired user coordinate system number can be selected from this list.

[0059] By activating the "User Coordinate System Settings (Shift)" icon 271 with the above settings, the user coordinate system specified in the user coordinate system specification field 462 can be shifted by an amount corresponding to the correction amount based on the detection results.

[0060] The "Select User Coordinate System" icon 272 switches the coordinate system the robot adheres to to the user coordinate system specified by the number of the "Set User Coordinate System (Shift)" icon.

[0061] Let's use an example to explain the operation procedure of the function assigned to the "User Coordinate System Setting (Shift)" icon 271. Let's assume that the correction amount (the amount of deviation of the workpiece from its reference position) obtained as a detection result by the "View" icon 211 is O1, and that the correction amount O1 is stored in the vision register 01. Let's assume that this correction amount O1 is the correction amount on the coordinate system UF1 set within the "View" icon 211. Let's assume that the detection result selected in the detection result selection field 461 of the "User Coordinate System Setting (Shift)" icon 271 is the vision register 01, and the user coordinate system specified in the user coordinate system specification field 462 is UF2. (Procedure 1) The detection result acquisition unit 161a retrieves the correction amount O1 from the vision register 01. (Step 2) The coordinate system shift unit 164 calculates the coordinate system UF2' obtained by shifting the coordinate system UF2 according to the correction amount O1 using the following formula (2). UF2'=UF1·O1·INV(UF1)·UF2 ···(2) Here, UF1 and UF2 represent the origin positions of their respective coordinate systems as seen from the world coordinate system, and are represented by simultaneous transformation matrices. INV(UF1) is the inverse matrix of UF1. Furthermore, if UF1=UF2 holds true here, equation (2) can be simplified to equation (3) below. UF2' = UF1(UF2)·O1 ···(3) (Step 3) Substitute the calculated value of UF2' directly into the specified user coordinate system UF2.

[0062] Therefore, the user coordinate system UF2 is a coordinate system obtained by shifting the original user coordinate system UF2 on the world coordinate system according to the correction amount O1.

[0063] The teaching procedure for teaching the control program 502 in this embodiment is as follows. (Procedure T21) The user can freely set the coordinate system (UF2). For example, the user may set a work coordinate system that has a specific relationship with the position and orientation of the workpiece in the workspace. (Procedure T22) The user switches the coordinate system the robot adheres to to the coordinate system UF2 set in the above procedure (T21), and teaches the robot (e.g., linear movement icon 212, "take / place" icon 213). (Procedure T23) Instruct the user to click the "View" icon 211 and the "User Coordinate System Settings (Shift)" icon 271, respectively.

[0064] Thus, in this embodiment, the robot is first taught on a user coordinate system that the user freely sets, and then in the subsequent step T23, the user coordinate system is specified, thereby shifting the user coordinate system according to the correction amount. This allows all movements of the robot to be shifted according to the correction amount. Conversely, in the second embodiment, unlike the first embodiment, it is assumed that the user sets the user coordinate system UF2 in advance. Therefore, the user coordinate system before it is overwritten (the value of the user coordinate system UF2 set in step T21) is stored, and the procedure involves performing the calculation for shifting (the above formula (2)) based on that user coordinate system.

[0065] Thus, according to this embodiment, the user coordinate system UF2 used when the user pre-programs the linear movement icon 212 and the "take / place" icon 213 is shifted according to the detection result (correction amount). Therefore, the pre-programmed teaching position becomes a value on the shifted user coordinate system UF2. Consequently, the operation of these icons is performed correctly for the workpiece at the detected position. The user does not need to individually apply corrections based on the detection result to each icon (linear movement icon, take / place icon, etc.).

[0066] Thus, according to this embodiment, it is possible to provide a function that allows users to use the function of correcting the robot's position based on the detection results from the visual sensor when the relative positional relationship between the robot and the object changes in a more user-friendly manner.

[0067] Third Embodiment The third embodiment relates to a robot system 100B configured to perform operations on a workpiece placed on a table using a robot mounted on a trolley. Figure 12 is a diagram of the equipment configuration of the robot system 100B. The hardware configuration of the robot control device 50B and the teaching operation device 10B is the same as in Figure 2. Figure 13 is a functional block diagram focusing on the function of the control device in the robot system 100B. As shown in Figure 12, the robot system 100B includes a robot 30B mounted on a trolley 81, a robot control device 50B that controls the robot 30B, and a teaching operation device 10B connected to the robot control device 50B by wire or wireless. A tool such as a hand (not shown in Figure 12) is attached to the end-effector of the robot 30B. A vision sensor 70 is also attached to the end-effector of the robot 30B. A workpiece W is placed on the table 85, and the robot 30B can perform predetermined operations on the workpiece W, such as picking it up.

[0068] Multiple markers M are attached to the table 85, and by measuring these markers M with a vision sensor 70 mounted on the robot 30B, the relative positional relationship between the table 85 and the trolley 81 (robot 30B) can be determined. Since the workpiece is placed in a specific position on the table 85, the position of the workpiece can be specified in a coordinate system set on the table 85. This coordinate system on the table 85 is called coordinate system UF1. By specifying the position numerically in coordinate system UF1, the robot 30B can be made to perform work on the workpiece W without teaching the robot 30B. However, if the absolute accuracy of the robot 30B is not high, the actual amount of movement of the robot 30B and the amount of movement calculated by the robot 30B may not match with sufficient accuracy. In such cases, even if the robot 30B tries to perform work at the numerically specified position, it may not be able to perform the work with sufficient accuracy. This embodiment makes it possible for the robot 30B to perform work on the workpiece W on the table 85 with good accuracy even in such situations. Furthermore, similar to the control device 40A according to the second embodiment, the control device 40B provides a function that enables appropriate handling of the workpiece W even when the relative positional relationship between the workpiece W and the robot 30B changes, in a form that can be used in a manner that is easy for the user to handle.

[0069] The functions of the control device 40B will be explained with reference to the functional block diagram in Figure 13. In Figure 13, components equivalent to those of the control device 40 in the first embodiment are denoted by the same reference numerals. As shown in Figure 13, the robot control device 50B includes a robot motion control unit 151, a program creation unit 152b, and a storage unit 153. In this embodiment, the robot control device 50B includes a vision sensor control unit 175 that has the functions of the vision sensor control device 20 described above.

[0070] The robot motion control unit 151 controls the movement of the robot 30B. The memory unit 153 stores control programs and various setting information related to teaching. The program creation unit 152b provides various functions for the user to perform programming using icons via the user interface (display unit 13 and operation unit 14) of the teaching operation device 10B. The program creation unit 152b includes an icon control unit 154b and a screen display creation unit 155 as components that provide such functions. The icon control unit 154b controls user operations when the user operates the operation unit 14 of the teaching operation device 10B to perform various operations on icons, tabs, etc. on the program creation screen 400 as exemplified in Figure 4. The screen display creation unit 155 presents various interface screens used for programming using icons and also supports user operations on these interface screens.

[0071] The program creation unit 152b further includes a detection result acquisition unit 171, a touch-up control unit 172, a command generation unit 173, and a coordinate system shift unit 174.

[0072] The detection result acquisition unit 171 includes a marker position measurement unit 171a, which has the function of measuring the three-dimensional position of the marker M using a visual sensor 70. The visual sensor 70 is assumed to be calibrated (i.e., the visual sensor control unit 175 holds calibration data), and the position and orientation of the visual sensor 70 relative to the robot 30B are assumed to be known. As an example, the marker position measurement unit 171a may measure the three-dimensional position of the marker M using the stereo measurement method with the visual sensor 70 acting as a two-dimensional camera. Various markers known in this field for measuring position, also called target marks or visual markers, may be used as the marker M, and various measurement methods known in this field may be used.

[0073] The touch-up control unit 172 controls a so-called touch-up operation, which involves touching a target point with a predetermined position control part (e.g., TCP) of the robot 30B to acquire the position information of the target point.

[0074] The command generation unit 173 provides the function of generating robot movement commands based on position information obtained through touch-up.

[0075] The coordinate system shift unit 174 provides a function to shift the user coordinate system using the detection function of the visual sensor 70. In this embodiment, an example of operation in which the coordinate system (user coordinate system) set by touch-up is shifted using the detection result of the visual sensor 70 will be described.

[0076] The control device 40B according to this embodiment is (F1) A function that uses known position information for predetermined positions on the table to correct the robot's movement command, which is set based on position information obtained by touching a predetermined position on the table. (F2) The function can be provided to shift the user coordinate system using positional information obtained by imaging a predetermined position of the table with a visual sensor before and after a change in the relative position of the robot and the table. By using the command obtained by the above function (F1) in the shifted coordinate system obtained by the above function (F2), the robot can be made to perform work on the workpiece with sufficient precision even after the relative position of the table and the robot has changed.

[0077] Figure 15 shows a flowchart representing the command generation process corresponding to the above function (F1), and Figure 16 shows a flowchart representing the user coordinate system shift process corresponding to the above effect (F2). These processes may be executed as a series of processes. These processes are executed under the control of the processor of the robot control device 50B.

[0078] (Figure 15: Step S1) Multiple points on table 85 with known relative positions are touched up using the TCP set on the end effector of robot 30B. Here, the known positions on table 85 are assumed to be the positions of markers M1 to M3. Figure 14 shows the arrangement of markers M1, M2, and M3. The positions (3D vectors) of the three markers M1 to M3 obtained by this touch-up operation are denoted as PT1 to PT3. The known relative positions of the three markers are denoted as PP1 to PP3. PP1 to PP3 are assumed to be position information with sufficient accuracy. Positions PP1 to PP3 are assumed to be stored, for example, in the memory unit 153. Coordinate system UF1 is assumed to be a coordinate system defined on table 85 based on positions PP1 to PP3.

[0079] (Figure 15: Step S2) The coordinate system UF2 is set using information from PT1 to PT3. Setting a coordinate system using PT1 to PT3 can be done, for example, as follows. Here, it is assumed that PT1 to PT3 (i.e., marker M1 to marker M3) are on the same plane. In this case, the plane defined by PT1 to PT3 is defined as the XY plane, and the axis direction from PT1 (marker M1) to PT2 (marker M2) is defined as the X-axis direction. This allows us to define the Y-axis as an axis perpendicular to the X-axis, and the Z-axis as an axis perpendicular to the XY plane. The origin of coordinate system UF2 and the direction of each coordinate axis can be defined to be consistent with coordinate system UF1. In an ideal state where the absolute accuracy of the robot is high, coordinate system UF1 and coordinate system UF2 coincide. However, as mentioned above, in situations where the absolute accuracy of the robot is not high, the robot's calculated position and the robot's actual position are different, so coordinate system UF1 and coordinate system UF2 do not coincide.

[0080] (Figure 15: Step S3) The ideal position on Table 85 is converted to a position on coordinate system UF2. Here, as an example, this conversion is performed using the following method. For example, marker M1 represents the origin, marker M2 represents the position in the X-axis direction, and marker M3 represents the position in the Y-axis direction. PP1 represents the origin, PP2 represents the position in the X-axis direction, and PP3 represents the position in the Y-axis direction. The x and y coordinates when the ideal position PI on Table 85 is expressed using PP1 to PP3 in the following formula (4) are determined. PI=x(PP2-PP1)+y(PP3-PP1)...(4)

[0081] Next, using PT1 to PT3, the command position P_PT for the robot to move to the ideal position PI is determined by the following equation (5). P_PT=x(PT2-PT1)+y(PT3-PT1)...(5) x and y define an ideal position calculated in a coordinate system using precise coordinate values ​​PP1 to PP3. On the other hand, the positions PT1 to PT3 calculated by robot 30B do not match PP1 to PP3 in terms of coordinate values, but as the actual TCP position of robot 30B, they can match each marker M1 to M3 with good accuracy. Therefore, even if the command to move robot 30B to the command position P_PT obtained by applying x and y to the above equation (2) is a command on coordinate system UF2, it is possible to move robot 30B to the correct position.

[0082] Since P_PT is defined as the coordinate system value from which PT1 to PT3 were measured, this is converted to the position P_UF2 on coordinate system UF2. By moving to coordinate system UF2 (setting UF2 as the coordinate system that robot 30B adheres to) and giving robot 30B a command to move to position P_UF2, the aforementioned accuracy error of robot 30B can be absorbed, allowing robot 30B to move to the ideal position and perform the task.

[0083] Of the operations in steps S1 to S3 described above, steps S1 and S2 can be implemented by the touch-up control unit 172. Furthermore, step S3 can be implemented by the command generation unit 173.

[0084] The process in step S3 can be described as an operation to generate a command for the robot 30B based on position information PP1 to PP3 of a predetermined position on the object (table 85) and position information PT1 to PT3 obtained by touching up the predetermined position on the object (table 85) with the robot 30B. In other words, the process in step S3 is to determine the amount of movement on the coordinate system defined by position information PP1 to PP3 and apply this amount of movement as the amount of movement in coordinate system UF2 (a coordinate system corresponding to the coordinate system defined by position information PT1 to PT3).

[0085] Next, the specific processing details of the above function (F2) will be explained with reference to Figure 16. Here, as an example, in a situation where the trolley 81 moves and the relative positional relationship between the trolley 81 on which the robot 30B is mounted and the table 85 changes, the user coordinate system is shifted using the detection results of the visual sensor 70.

[0086] (Figure 16: Step S11) The positions PV1 to PV3 of markers M1 to M3 as seen from the visual sensor 70 are measured. The position measurement of markers M1 to M3 as seen from the visual sensor 70 can be performed by the marker position measurement unit 171a. Then, the coordinate system UF3 is determined from PV1 to PV3. The above method can be used to set the coordinate system based on the measured positions PV1 to PV3 of markers M1 to M3.

[0087] (Figure 16: Step S12) Now, let's assume that the trolley 81 moves, and the relative positions of the trolley 81 (i.e., the robot 30B) and the table 85 change. At this time, the positions of the markers PV1' to PV3' as seen from the vision sensor 70 are measured. Then, the coordinate system UF3' is determined from PV1' to PV3'.

[0088] (Figure 16: Step S13) The relative displacement O from UF3 to UF3' is, O=UF3'·INV(UF3) This can be calculated using the following formula. Here, INV() represents the calculation of the inverse matrix, and UF3' and UF3 are represented by the simultaneous transformation matrix. The coordinate system UF2', which is the coordinate system UF2 after the movement of the trolley 81, UF2'=O·UF2 This can be determined by the above. By giving the robot 30B a command to move to position P_UF2 on the shifted UF2', the aforementioned accuracy error of the robot 30B can be absorbed, and the robot 30B can be moved to the ideal position and made to perform the task.

[0089] The function provided by steps S11 to S13 described above can be described as a function that allows the robot to continue its operation by shifting the user coordinate system based on the detection results of the visual sensor when the trolley 81 moves. Thus, the function of shifting the user coordinate system can also be implemented as a function represented by a single icon. For example, it may be provided as an icon with an appearance like the "User Coordinate System Setting (Shift)" icon 271 shown in Figure 8. In this case, the detailed settings of the "User Coordinate System Setting (Shift)" icon can be configured to include a selection field for selecting the detection result of the marker by the visual sensor and a specification field for specifying the number of the user coordinate system to shift.

[0090] Thus, according to this embodiment, the function of correcting the robot's position based on detection results from a visual sensor when the relative positional relationship between the robot and the object changes can be used in a way that is even easier for the user to handle.

[0091] Although the present invention has been described above using typical embodiments, those skilled in the art will understand that modifications to the above embodiments and various other modifications, omissions, and additions can be made without departing from the scope of the present invention.

[0092] For example, the distribution of functional blocks shown in the functional block diagrams in each of the embodiments described above is just one example; for instance, one or more functions that are placed in the robot control device may be placed in the teaching and operation device.

[0093] The function of the "User Coordinate System Selection" icon 252 shown in the first embodiment may be integrated into the "User Coordinate System Setting" icon 251. The function of the "User Coordinate System Selection" icon 272 shown in the second embodiment may be integrated into the "User Coordinate System Setting (Shift)" icon 271.

[0094] The program that performs various processes such as command generation processing and user coordinate system shift processing in the above-described embodiment can be recorded on various computer-readable recording media (for example, semiconductor memory such as ROM, EEPROM, and flash memory, magnetic recording media, and optical discs such as CD-ROM and DVD-ROM). [Explanation of Symbols]

[0095] 1. Double job 2 Workbenches 10, 10B Teaching and Operation Device 11 processors 12 memory 13 Display section 14 Control section 15 Input / Output Interfaces 20. Visual sensor control device 30, 30B Robot 33 Hand 40, 40A control device 50, 50A Robot Control Device 51 processors 52 memory 53 Input / Output Interfaces 54 Operation section 70. Vision Sensors 100, 100B Robot System 151 Robot motion control unit 152, 152a, 152b Program creation section 153 Storage section 154, 154a, 154b Icon Control Unit 155 Screen display creation section 161, 161a Detection result acquisition unit 162 Coordinate System Data Output Unit 163, 163a Command generation section 164 Coordinate system shift section 171 Detection result acquisition unit 171a Marker position measurement unit 172 Touch-up control unit 173 Command generation section 174 Coordinate system shift section 200 icon display area 251 "User Coordinate System Settings" icon 252, 272 "User Coordinate System Selection" icon 261 Programming tab 262 Details tab 271 "User Coordinate System Settings (Shift)" icon 300 Program creation area 400 Program creation screen

Claims

1. A detection result acquisition unit that acquires first information corresponding to a detection result in which the relative positional relationship between the robot and the object is detected by a visual sensor, Based on the first information, a coordinate system data output unit outputs first coordinate system data as data representing the coordinate system to be followed when operating the robot, A command generation unit that generates commands for the robot in accordance with the first coordinate system represented by the first coordinate system data, An icon control unit provides the functions of the detection result acquisition unit and the coordinate system data output unit as functions implemented in at least one icon representing a function that constitutes the robot's control program, A control device equipped with the following features.

2. The control device according to claim 1, wherein the detection result acquisition unit acquires first information representing a detection result selected by user input from among one or more detection results stored in a predetermined storage location.

3. The first piece of information represents the detection position of the object on a second coordinate system having a specific relationship with the world coordinate system. The control device according to claim 1 or 2, wherein the coordinate system data output unit outputs the first coordinate system data based on the second coordinate system data representing the second coordinate system and the detected position.

4. A command generation unit generates commands for the robot using a first coordinate system as the coordinate system to be followed when operating the robot, A detection result acquisition unit acquires first information corresponding to the detection result obtained by detecting the relative positional relationship between the robot and the object using a visual sensor, A coordinate system shifting unit that shifts the first coordinate system based on the first information, The icon control unit provides the functions of the detection result acquisition unit and the coordinate system shift unit as functions implemented in an icon representing a function that constitutes the robot's control program, A control device equipped with the following features.

5. The control device according to claim 4, wherein the detection result acquisition unit acquires the first information representing a detection result selected by user input from among one or more detection results stored in a predetermined storage location.

6. The first piece of information represents a correction amount for the position of the object on a second coordinate system having a specific relationship with the world coordinate system. The control device according to claim 4 or 5, wherein the coordinate system shifting unit obtains shifted first coordinate system data as a result of shifting the first coordinate system based on second coordinate system data representing the second coordinate system, the correction amount, and first coordinate system data representing the first coordinate system.

7. A command generation unit generates commands for the robot using a first coordinate system as the coordinate system to be followed when operating the robot, A detection result acquisition unit acquires first information corresponding to the detection result obtained by detecting the relative positional relationship between the robot and the object using a visual sensor, A coordinate system shifting unit that shifts the first coordinate system based on the first information, A storage unit that stores location information of a predetermined position of the object, The system includes a touch-up control unit that touches up a predetermined position on the object with the robot and acquires second information representing the predetermined position of the object, The command generation unit is a control device that determines the amount of movement of the robot to a specific position on a coordinate system defined by the position information stored in the memory unit, and applies the determined amount of movement as the amount of movement in the first coordinate system defined by the second information to generate the command.

8. The position information of a predetermined position of the object stored in the memory unit includes the positions of the first to third markers placed on the object at the origin position of the coordinate system, the position on the X-axis, and the position on the Y-axis, respectively, as defined by the position information. When the positions of the first to third markers on the object are PP1, PP2, and PP3, and the positions of the first to third markers on the object obtained by touching them up with the robot are PT1, PT2, and PT3 respectively as second information, and the specific position is PI, the command generation unit determines the amount of movement (x, y) of the robot to the specific position PI in the coordinate system defined by the position information stored in the storage unit, PI=x(PP2-PP1)+y(PP3-PP1) Calculated by, Next, the command position P_PT in the first coordinate system corresponding to the amount of movement (x, y) is P_PT=x(PT2-PT1)+y(PT3-PT1) Calculated by, The control device according to claim 7, which generates the command according to the command position P_PT.

9. The control device according to claim 7 or 8, wherein the touch-up control unit causes the TCP of the robot to touch up at a predetermined position on the object.

10. The control device according to any one of claims 7 to 9, wherein the detection result acquisition unit obtains the first information as a relative movement amount between the object and the robot by measuring a predetermined position of the object using the vision sensor mounted on the robot before and after the relative positional relationship between the object and the robot changes.

11. The control device according to claim 10, wherein the coordinate system shifting unit shifts the first coordinate system based on the relative displacement amount.

Images