Teaching system, teaching device, robot control device, and program

Abstract

The present invention provides a teaching system for teaching about calibration or position detection, the teaching system comprising: a first device including a first processor and a storage unit storing image data for the calibration or position detection; and a second device including a second processor and a display, wherein the second processor acquires the image data from the first device and displays the image that the image data represents on the display for a visual sensor to capture in performing the calibration or position detection.

Description

Teaching system, teaching device, robot control device, and program

[0001] The present disclosure relates to a teaching system, a teaching device, a robot control device, and a program.

[0002] Various techniques are known in the art for calibrating a camera placed in a robot system using a jig with a specific pattern, such as a dot pattern jig (see, for example, Patent Documents 1 and 2). Also, various techniques are known for measuring or detecting the position of a measurement target by capturing an image of a marker with a camera (see, for example, Patent Documents 3 and 4).

[0003] JP 2014-128845 A JP 2019-42834 A JP 2012-218140 A JP 2005-201824 A

[0004] When performing calibration or position detection using a dot pattern or a marker, an operator generally needs to take out a jig or a printed material on which the dot pattern or the marker is formed from a storage location to perform the work. There is a demand for a technology that can reduce the burden on an operator when performing calibration or position detection using a dot pattern or a marker.

[0005] One aspect of the present disclosure is a teaching system for performing teaching related to calibration or position detection, the teaching system including: a first device having a first processor and a storage unit that stores image data for the calibration or position detection; a second device having a second processor and a display; In this teaching system, the second processor acquires the image data from the first device and displays an image represented by the image data on the display to be captured by a visual sensor in performing the calibration or position detection.

[0006] These and other objects, features and advantages of the present invention will become more apparent from the detailed description of exemplary embodiments of the invention illustrated in the accompanying drawings.

[0007] 3A . FIG. 3B is a system configuration diagram of a teaching system for teaching related to calibration or position detection according to the first embodiment. FIG. 3C is a functional block diagram of the teaching system according to the first embodiment. FIG. 3D is a diagram showing an overall flow of processing related to teaching a predetermined pattern in calibration or position detection according to the first embodiment. FIG. 3E is a flowchart showing the processing content of the second device in step S1 of FIG. 3A. FIG. 3F is a diagram showing a state in which a dot pattern for calibration is displayed on the display of a teaching pendant. FIG. 3G is a diagram showing a state in which a dot pattern is displayed overlapping an operation screen for teaching. FIG. 3H is a diagram showing an example of a registration screen for image data displayed on the display of a teaching pendant. FIG. 3I is a device configuration diagram in which a second device that downloads and displays image data of a predetermined pattern is a tablet terminal separate from the teaching pendant. FIG. 3I is a functional block diagram of the tablet terminal. FIG. 3H is a diagram showing a state in which a marker for position detection is displayed on the display of the tablet terminal. FIG. 3I is a diagram showing a state in which a marker for position detection is displayed on the display of a machine tool. FIG. 3I is a system configuration diagram of a teaching system for teaching related to calibration or position detection according to a second embodiment. FIG. 3I is a functional block diagram of the teaching system according to the second embodiment. 14A is a diagram showing a state in which an image roughly representing a numerical value of a dot spacing as dimensional information is displayed on the display of a teaching operation panel together with a dot pattern. FIG. 14B is a diagram showing a state in which an image of a code is displayed on the display of a teaching operation panel together with a dot pattern as dimensional information. FIG. 14C is a diagram showing an overall flow of processing related to teaching of a predetermined pattern in calibration or position detection in a second embodiment. FIG. 14D is a flowchart showing the processing content of the second device in step S1a of FIG. 14A. FIG. 14B is a system configuration diagram of a teaching system for teaching related to calibration or position detection in a third embodiment. FIG. 14C is a functional block diagram of the teaching system according to the third embodiment. FIG. 17B is a diagram showing an overall flow of processing related to teaching of a predetermined pattern in calibration or position detection in a third embodiment. FIG. 17C is a flowchart showing the processing content of the second device in step S1b of FIG. FIG. 17A is a diagram showing an example of an execution screen of a calibration program.

[0008] Next, embodiments of the present disclosure will be described with reference to the drawings. In the drawings, like components or functional parts are designated by like reference numerals. The scales of these drawings have been changed appropriately to facilitate understanding. Furthermore, the embodiment shown in the drawings is one example for implementing the present invention, and the present invention is not limited to the illustrated embodiment.

[0009] In each of the embodiments described below, a teaching system for teaching calibration and position detection will be described.

[0010] In this specification, the predetermined pattern for calibration may include various dot patterns and markers known in the art that can be used for calibration. The above-mentioned Patent Documents 1 and 2 describe examples of dot patterns and markers that can be used for calibration. Calibration corresponds to obtaining calibration data (transformation matrix) that associates a position on a camera image with a position in three-dimensional space. The calibration data includes internal parameters corresponding to the transformation from the camera coordinate system to the image coordinate system and external parameters corresponding to the transformation (rotation and translation) from the world coordinate system to the camera coordinate system. A dot pattern that can be used for calibration satisfies, for example, the following requirements: (a1) the grid point spacing of the dot pattern is known; (a2) there are a certain number of grid points or more; and (a3) ​​each grid point can be uniquely identified. FIG. 4 illustrates an example of such a dot pattern 301. In dot pattern 301, dots are arranged at equal intervals in a grid pattern, and large dots M (only some of which are marked with symbols) define a coordinate system (X axis, Y axis, and origin O) on the dot pattern, thereby identifying the position of each grid point.

[0011] There are various types of predetermined patterns known in the art, called markers or targets, for detecting the position of a measurement target. Patent Documents 3 and 4, cited above, describe examples of such markers. There are also application examples in which markers are used to correct a coordinate system. For example, in an application example in which a robot mounted on a carriage or an AGV (Automated Guided Vehicle) corrects its positional relationship with a machine tool when the robot moves, markers are affixed to one or more locations on the measurement target (machine tool), and the positional relationship is corrected by detecting the markers using a robot equipped with a calibrated camera. Markers that can be used for position detection include not only the marker 302 illustrated in FIG. 5 and those exemplified in Patent Documents 3 and 4, but also various other markers that can be detected on an image, such as dot patterns, relatively simple figures, symbols, and characters.

[0012] References herein to predetermined patterns for calibration or position detection may include the various types described above that can be used for calibration or for detecting or measuring the position of an object.

[0013] The teaching system of each embodiment described below includes a first device having a first processor and a storage unit that stores image data of a predetermined pattern for calibration or position detection, a second device having a second processor and a display, and the second processor of the second device has a function of acquiring image data of the predetermined pattern from the first device and displaying the image of the predetermined pattern on the display to be captured by a visual sensor in performing calibration or position detection.

[0014] First Embodiment FIG. 1 shows the system configuration of a teaching system 501 according to a first embodiment for teaching calibration or position detection. The teaching system 501 includes a robot system 100. FIG. 2 shows a functional block diagram of the teaching system 501. As shown in FIG. 1, the robot system 100 includes a robot 10, a robot control device 20 that controls the robot 10, a teaching pendant 30, an image processing device 40 connected to the robot control device 20, and a visual sensor 70. The visual sensor 70 is attached to the tip of the arm of the robot 10 and is also connected to the image processing device 40. By calibrating the visual sensor 70, the robot system 100 can detect the position of an object using the visual sensor 70 and handle the object.

[0015] 1 , the robot control device 20 (first device) is responsible for executing calibration or position detection using the visual sensor 70. The teaching pendant 30 (second device) has a function of acquiring image data of a predetermined pattern for calibration or position detection from the robot control device 20 (first device), and displaying an image of the predetermined pattern on a display 32 of the teaching pendant 30 (second device) based on the acquired image data so that the image can be captured by the visual sensor.

[0016] The robot control device 20 controls the operation of the robot 10 in accordance with a robot program or commands from the teaching pendant 30. The robot control device 20 may have a hardware configuration as a general computer having a processor 21, memory (ROM, RAM, non-volatile memory, etc.), a storage unit 22, an operation unit, an input / output interface, a network interface, etc. (see FIG. 2).

[0017] The teaching pendant 30 is connected to the robot control device 20 by wire or wirelessly, and can provide functions for teaching the robot 10 and inputting various settings. A teaching device configured with various portable terminal devices (such as a tablet terminal) may be used as the teaching pendant 30. The teaching pendant 30 may have a hardware configuration as a general computer including a processor 31, memory (such as a ROM, RAM, or non-volatile memory), a storage device, a display 32, an operation unit 33, an input / output interface, a network interface, etc. (see FIG. 2). The operation unit may be configured as a touch operation panel integrated with the display 32.

[0018] Although the robot 10 in FIG. 1 is a vertical articulated robot, various types of robots may be used depending on the work object, such as a horizontal articulated robot, a parallel link robot, or a dual-arm robot. The robot 10 can perform a desired task using an end effector attached to the wrist. The end effector is an external device that can be replaced depending on the application, such as a hand, a welding gun, or a tool. FIG. 1 shows an example in which a hand 11 is used as the end effector.

[0019] The image processing device 40 controls the visual sensor 70 based on commands from the robot control device 20 (operation control unit 121) and performs image processing (detection, determination, etc.) on captured images. The visual sensor 70 may be a camera that captures grayscale or color images, or a stereo camera or three-dimensional sensor that can acquire distance images or three-dimensional point clouds.

[0020] In this embodiment, an example configuration is shown in which the image processing device 40 is placed as a device separate from the robot control device 20, but the functions of the image processing device 40 may also be integrated into the robot control device 20.

[0021] As shown in FIG. 2 , the robot control device 20 includes an operation control unit 121, a dimension information calculation unit 122, and an image data storage control unit 123. These functional blocks may be realized by the processor 21 of the robot control device 20 executing software. FIG. 2 also illustrates a storage unit 22 as a hardware component of the robot control device 20. The storage unit 22 is a storage device formed, for example, of a non-volatile memory or a hard disk drive. The storage unit 22 stores various programs, such as a robot program for controlling the robot 10, a calibration program, and a position detection program (a program for measuring or detecting the position of an object using a predetermined pattern such as a marker), as well as various setting information. The storage unit 22 also stores image data of a predetermined pattern that can be used for calibration or position detection.

[0022] The operation control unit 121 controls the operation of the robot 10 in accordance with the robot program or commands from the teaching pendant 30. The robot control device 20 includes a servo control unit (not shown) that executes servo control of the servo motors of the respective axes in accordance with commands for the respective axes generated by the operation control unit 121.

[0023] As shown in FIG. 2 , the teaching pendant 30 includes an image data acquisition unit 131, an image data display control unit 132, a brightness adjustment unit 133, and an image data registration unit 134. Note that these functional blocks may be implemented by the processor 31 executing software. The image data acquisition unit 131 has a function of acquiring image data of a predetermined pattern for calibration or position detection from the robot control device 20. The image data display control unit 132 has a function of displaying the acquired image of the predetermined pattern on the display 32. With this configuration, an operator can use the teaching pendant 30 as a calibration jig or a jig for detecting the position of an object by displaying an image of a predetermined pattern such as a dot pattern or marker on the teaching pendant 30.

[0024] To perform calibration using a dot pattern, dimensional information about the dot spacing is required. As an example of a configuration for the robot control device 20 that performs calibration to grasp the dimension of the dot spacing of the dot pattern displayed on the teaching pendant 30, in this embodiment, the robot control device 20 calculates the dimensional information based on information related to the display 32 of the teaching pendant 30 and stores the calculated dimensional information in association with image data of the dot pattern.

[0025] The dimensional information calculation unit 122 of the robot control device 20 can calculate the dimension of the dot spacing when the dot pattern is displayed on the display 32, for example, from the number of pixels at the lattice point interval in the image data of the dot pattern and information about the resolution and size of the display 32 of the teaching pendant 30. For example, the dimensional information calculation unit 122 may be configured to control the execution of the following steps: (Step b1) Accept input of information (resolution and size) about the display 32 via the teaching pendant 30. (Step b2) Calculate the dimensional information based on the image data of the dot pattern and information about the display 32. (Step b3) Store the calculated dimensional information in the storage unit 22 in association with the image data of the dot pattern.

[0026] The image data storage control unit 123 has a function of associating image data of dot patterns with dimensional information and storing them in the storage unit 22. When preparing image data of multiple types of dot patterns with different dot spacings, the dot spacing dimensions when displayed on the display 32 for each dot pattern (image data) may be calculated and stored in association with the image data, as shown in Table 1 below. This allows the teaching pendant 30 to be used as multiple types of calibration jigs with different dot spacings.

[0027]

[0028] 3A shows the overall flow of the process for teaching a predetermined pattern for calibration or position detection. First, an operator executes a process for acquiring and displaying image data of a predetermined pattern for calibration or position detection on the teaching pendant 30 (second device) (step 1).

[0029] 3B is a flowchart showing the processing content in step S1 of FIG. 3A. In response to an instruction from an operator, the image data acquisition unit 131 of the teaching pendant 30 (second device) acquires image data of a predetermined pattern for calibration or position detection from the robot control device 20 (first device) (step S11). At this time, if the robot control device 20 holds image data of multiple types of patterns as described above, the image data acquisition unit 131 may be configured to display a list of multiple patterns on the display 32 and accept a user operation to select desired image data from the list.

[0030] Next, the image data display control unit 132 of the teaching pendant 30 displays the image of the predetermined pattern acquired from the robot control device 20 on the display 32 (step S12). This makes the teaching pendant 30 available as a jig for calibration or position detection.

[0031] 3A, the operator then places the teaching pendant 30 (second device) displaying a predetermined pattern at a predetermined position for calibration or position correction, where the visual sensor 70 can capture an image of the teaching pendant 30 displaying the image of the predetermined pattern (step S2). The operator then causes the visual sensor 70 to capture an image of the teaching pendant 30 displaying the image of the predetermined pattern, and causes the robot control device 20 to perform calibration or position detection (step S3). In this case, the robot control device 20 (first device) holds dimensional information for displaying the dot pattern on the teaching pendant 30, and can use that dimensional information when performing calibration.

[0032] FIG. 4 shows a state in which an image of a dot pattern 301 for calibration is displayed on the display 32 of the teaching pendant 30 in step S1. The dot pattern 301 shown in FIG. 4 satisfies the above-mentioned requirements (a1) to (a3) ​​for a dot pattern for performing calibration. The dot spacing d on the display 32 when the dot pattern 301 is displayed as shown in FIG. 4 is known to the robot controller 20, which is responsible for performing calibration. Therefore, the robot controller 20 can appropriately perform calibration using the value of the dot spacing d stored in the robot controller 20. Note that such a dot pattern 301 may also be used to detect the position of an object or to set a coordinate system.

[0033] The operator can perform calibration or position detection simply by operating the teaching pendant 30 to download and display image data of a predetermined pattern from the robot control device 20. Therefore, the operator does not need to perform the time-consuming task of retrieving and setting up jigs for calibration or position detection from a storage location, as was done in the past.

[0034] Furthermore, as described above, by storing image data of dot patterns of various sizes in the robot control device 20, the worker can obtain image data of dot patterns of a size that matches the execution environment from the robot control device 20 at any time, display it on the teaching operation panel 30, and use it for teaching.

[0035] According to this embodiment, the operator can perform calibration or position detection whenever necessary by displaying an image of a predetermined pattern on the teaching pendant 30. As shown in Fig. 5 , while an operation screen 401 for teaching a robot program for the robot 10 is displayed on the teaching pendant 30, the operator can activate the function of the image data acquisition unit 131 whenever necessary to download image data of a predetermined pattern from the robot control device 20 and display the image data overlapping the operation screen 401 for teaching. The example in Fig. 5 shows a case where a dot pattern 301 is displayed overlapping the operation screen 401 for teaching.

[0036] The brightness adjustment unit 133 of the teaching pendant 30 has a function of receiving information regarding the brightness of an image captured by the visual sensor 70 of the teaching pendant 30 displaying a predetermined pattern image from the robot control device 20 (calibration program or position detection program) and adjusting the brightness of the predetermined pattern image displayed on the display 32. For example, if the information from the robot control device 20 indicates that the captured image is too dark, the brightness adjustment unit 133 increases the brightness of the predetermined pattern image on the display 32. Alternatively, if the information from the robot control device 20 indicates that the captured image is too bright, the brightness adjustment unit 133 decreases the brightness of the predetermined pattern image on the display 32. This function makes it possible to automatically adjust the brightness of the predetermined pattern to an appropriate level regardless of the lighting environment in the workspace. The operator does not need to perform time-consuming tasks such as adjusting lighting devices in the workspace in order to adjust the brightness of the image of the predetermined pattern captured by the visual sensor 70.

[0037] The brightness adjustment unit 133 may have a function for manually adjusting the brightness of the image of the predetermined pattern. Even in this case, the operator can adjust the brightness of the image of the predetermined pattern displayed on the teaching pendant 30 to an appropriate state by observing the state of the image captured by the visual sensor 70. The operator does not need to perform time-consuming tasks such as adjusting lighting devices in the work space in order to adjust the brightness of the image of the predetermined pattern captured by the visual sensor 70.

[0038] Since the teaching pendant 30 (second device), which has the function of displaying an image of a predetermined pattern, has the function of appropriately adjusting the brightness of the display of the image of the predetermined pattern, the operator does not need to adjust the brightness of the lighting in the work space, etc. Therefore, the burden on the operator when performing calibration and position detection is reduced, and the operator can perform the work more efficiently.

[0039] The image data registration unit 134 provides a function for newly registering image data of a predetermined pattern for calibration or position detection to the robot control device 20 (first device) that stores image data of a predetermined pattern. The image data registration unit 134 provides a function for registering, for example, any image data among the image data stored in the teaching pendant 30 (or an external storage device connected to the teaching pendant 30) in the robot control device 20 as new image data for calibration or position detection.

[0040] FIG. 6 shows an example of an image data registration screen 402 displayed on the display 32 of the teaching pendant 30 by the function of the image data registration unit 134. The registration screen 402 displays a list of image data stored in the teaching pendant 30 (or an external storage device connected to the teaching pendant 30). The operator selects desired image data from the displayed list of image data and presses the OK button 411 to register the selected image data in the storage unit 22 of the robot controller 20 as image data of a new pattern. The image data storage control unit 123 of the robot controller 20 has a function of accepting a registration request from the image data registration unit 134 and storing the new image data in the storage unit 22. The image data to be newly registered can be any image that can be used for calibration or position detection, such as various marks, figures, symbols, and characters. When accepting registration of image data of a new pattern, the image data registration unit 134 may also be configured to accept registration of information regarding the dimensions of the pattern. The information regarding the dimensions may be, for example, the number of pixels between dots in the dot pattern or the dimension (in millimeters) of the dot spacing on a specified display, as described above. Alternatively, the information about dimensions may be information about the size of the marker (for example, the size (number of dots) of circle C, the size (square millimeters) of circle C when displayed on a specific display, etc.). This allows the robot control device 20 to retain information that associates image data with dimensional information, such as that exemplified in Table 1 above, even for new image data.

[0041] The above-described embodiment is an example of a configuration in which image data of a predetermined pattern held by the robot control device 20 (first device) is downloaded to the teaching pendant 30 (second device) and displayed, but the second device that downloads and displays image data of the predetermined pattern is not limited to the teaching pendant 30. For example, the image data of the predetermined pattern may be downloaded and displayed by a portable terminal device separate from the teaching pendant 30. In this case, the terminal device can be positioned as a teaching device for performing teaching related to calibration or position detection.

[0042] FIG. 7 shows an example of the equipment configuration of such a teaching system 501A. In the teaching system 501A, image data of a predetermined pattern is downloaded and displayed on a tablet terminal 80 separate from the teaching pendant 30. FIG. 8 shows a functional block diagram of the tablet terminal 80 in this system configuration. As shown in FIG. 8, the tablet terminal 80 includes an image data acquisition unit 181, an image data display control unit 182, and an image data registration unit 184. The functions of the image data acquisition unit 181, the image data display control unit 182, and the image data registration unit 184 are equivalent to the functions of the image data acquisition unit 131, the image data display control unit 132, and the image data registration unit 134 of the teaching pendant 30 described above. Note that FIG. 8 also shows a display 82 and an operation unit 83 as hardware components of the tablet terminal 80.

[0043] The tablet terminal 80 may have a hardware configuration as a general computer having a processor 81, memory (ROM, RAM, non-volatile memory, etc.), a storage device, a display 82, an operation unit 83, an input / output interface, a network interface, etc. The operation unit may be configured as a touch operation panel integrated with the display.

[0044] The tablet terminal 80 may be connected to the robot controller 20 so as to be able to communicate with it, or may not be able to communicate with it. If the tablet terminal 80 is not able to communicate with the robot controller 20, for example, the worker may download and store image data of the robot controller 20 in a USB memory. Then, the worker may connect this USB memory to the tablet terminal 80, thereby acquiring and displaying image data of a desired pattern on the tablet terminal 80.

[0045] In this case, the robot control device 20 may store information about the display 82 of the tablet terminal 80 in addition to information about the display 32 of the teaching pendant 30. This allows the robot control device 20 to calculate and store the size of the dot spacing of the dot pattern on the display 82 from the number of pixels of the dot spacing in the image data of the dot pattern and the resolution and size of the display 82 of the tablet terminal 80. This allows the robot control device 20 to store dimensional information about the dot spacing when the dot pattern is displayed on the teaching pendant 30 and dimensional information about the dot spacing when the dot pattern is displayed on the tablet terminal 80, in association with the image data, as shown in Table 2 below.

[0046]

[0047] The information shown in Table 2 can also be expanded to support three or more types of terminal devices. In this case, the dot spacing when the dot pattern is displayed on three or more types of terminal devices can be calculated based on the number of pixels for the dot spacing of the dot pattern and the display information (resolution, size) of the three or more types of terminal devices.

[0048] In this manner, when the robot control device 20 is configured to hold dimensional information for multiple types of displays, the robot control device 20 (processor 21) may be configured to obtain information about which display will be used in performing calibration or position detection through user input, for example via the teaching operation panel 30.

[0049] 7 and 8, the dimensions of the dot spacing when the dot pattern is displayed on the tablet terminal 80 are known to the robot control device 20, and therefore the same advantages as those of the embodiment described above with reference to Figures 1 to 6 can be obtained. Note that the configuration examples shown in Figures 7 and 8 have the advantage that operation of the robot 10 using the teaching pendant 30 and calibration or position detection using the tablet terminal 80 can be performed simultaneously in parallel.

[0050] FIG. 9 illustrates a state in which an image of a marker 302 acquired from the robot control device 20 is displayed on the display 82 of the tablet terminal 80. As an example, the marker 302 is a type of marker that includes mutually perpendicular lines a and b and a circle C of a known size. The marker 302 is used, for example, as follows: When detecting the position of an object using the marker 302, the worker calibrates the visual sensor 70 in advance. Then, the worker operates the tablet terminal 80 to download image data of the marker 302 from the robot control device 20 and display it on the display 82 (step S1). The worker places the tablet terminal 80, on which the marker 302 is displayed, at a predetermined position on the measurement object (e.g., workbench, machine tool) (step S2) and causes the robot control device 20 to execute a position detection program (step S3). The position detection program determines the three-dimensional position of the marker 302 based on the detected position of the marker 302 on the captured image, thereby obtaining the three-dimensional position of the measurement object. Note that a stereo measurement technique using the visual sensor 70 may be used to measure the three-dimensional position of the marker 302. Therefore, even when performing such position detection, the worker does not need to take out a marker jig or a printed copy of the marker stored in a separate location and install it on the target.

[0051] In addition, when detecting the position of an object using a marker of a type such as marker 302, if dimensional information (such as the size of circle C) is required for teaching the marker, the robot control device 20 (first device) may associate the dimensional information with the image data of the marker and store the dimensional information when the marker is displayed on the display 82 of the tablet terminal 80 (second device), and use this dimensional information when detecting the position of the object.

[0052] The second device that displays the image of the predetermined pattern may be a machine tool used together with the robot 10. FIG. 10 shows a schematic diagram of the equipment configuration in this case. For example, this case corresponds to an application example in which the robot 10 is mounted on a carriage or an AGV and placed at a predetermined position relative to the machine tool 90 to carry workpieces into and out of the machine tool 90. The machine tool 90 incorporates a control device (numerical control device) 91 with an integrated display 92. The control device 91 can download and display image data of the predetermined pattern from the robot control device 20, for example, via a network. Alternatively, the image data of the predetermined pattern may be pre-registered in the memory unit of the control device 91.

[0053] 10 shows a state in which the marker 302 is displayed on the display 92 of the control device 91. The display 92 of the control device 91 is mounted at a fixed position on the machine tool 90, so by displaying the marker 302 at a predetermined position on the display 92, the marker 302 can be used as an indicator indicating the position of the machine tool 90. In this case, for example, when performing position detection, an operator operates the control device 91 to display the marker 302 on the display 92. The robot control device 20 measures the positional relationship between the robot 10 and the machine tool 90 by capturing an image of the marker 302 displayed on the display 92 with the visual sensor 70. Note that in this case, by storing information regarding the specifications of the display 92 of the control device 91 (resolution, screen size, etc.), the robot control device 20 can calculate dimensional information when the marker 302 is displayed on the display 92 based on the image data of the marker 302 and use the calculated dimensional information for detection. In this configuration, the processor of the control device 91 can perform the functions of an image data acquisition unit 181, an image data display control unit 182, and an image data registration unit 184, as described above with reference to Figure 8 for the tablet terminal 80.

[0054] Second Embodiment FIG. 11 is a diagram showing the equipment configuration of a teaching system 502 according to a second embodiment. The teaching system 503 includes a robot system 100A. In the teaching system 502 shown in FIG. 11, the second device that displays a predetermined pattern image for calibration or position detection is a teaching pendant 30, and the first device that provides image data to the second device (teaching pendant 30) is an external device separate from the robot control device 20A. In this embodiment, as an example, the external device is another control device (e.g., a numerical control device, a robot control device, etc.) 220 located in the factory where the robot system 100A is installed. It is assumed that the teaching pendant 30 can communicate with the control device 220 via a network within the factory, either wired or wirelessly.

[0055] The control device 220 may have a hardware configuration as a general computer having a processor 224, memory (ROM, RAM, non-volatile memory, etc.), a storage unit 225, a display unit, an operation unit, an input / output interface, a network interface, etc. (see Figure 12).

[0056] In this embodiment, the control device 220 (first device) holds image data of a predetermined pattern for calibration or position detection in the same format as the robot control device 20 in the first embodiment. Therefore, with regard to a dot pattern, the control device 220 holds dimensional information of the dot spacing when the dot pattern is displayed on the display 32 of the teaching pendant 30, in association with the image data of the dot pattern. In this embodiment, to provide the dimensional information of the dot spacing to the robot control device 20A that controls calibration or position detection, an image showing the dimensional information is displayed on the display 32 of the teaching pendant 30 together with the image of the dot pattern.

[0057] FIG. 12 shows a functional block diagram of a robot controller 20A, a teaching pendant 30, and a controller 220 according to the second embodiment. The controller 220 includes a dimension information calculator 221 and an image data storage controller 222. The dimension information calculator 221 and the image data storage controller 222 have functions equivalent to those of the dimension information calculator 122 and the image data storage controller 123 of the robot controller 20 according to the first embodiment. That is, the dimension information calculator 221 and the image data storage controller 222 can execute processes equivalent to those described above (steps b1) to b3) in advance and store image data and dimension information of a predetermined pattern in the storage unit 225. The storage unit 225 is a storage device configured with a nonvolatile memory, a hard disk, or the like, and stores image data and dimension information of a predetermined pattern in addition to programs and various setting information related to machine control in the controller 220.

[0058] The image data acquisition unit 131 of the teaching pendant 30 has a function of downloading image data and dimension information of a predetermined pattern from the storage unit 225 of the control device 220. The image data display control unit 132 and brightness adjustment unit 133 have the same functions as those described above in relation to the first embodiment. The image data registration unit 134 provides the control device 220 with a function of newly registering image data of a predetermined pattern for calibration or position detection.

[0059] The robot control device 20A includes an operation control unit 121 and a dimension recognition unit 124. A robot program, a calibration program, a position detection program, and various other setting information are stored in the memory unit 22. The dimension recognition unit 124 provides a function of recognizing dimensions from an image of dimension information displayed together with an image of a predetermined pattern.

[0060] 13A and 13B show examples of when an image showing dimensional information about dot spacing is displayed on the display 32 of the teaching pendant 30 together with a dot pattern 301. FIG. 13A shows an example in which an image G1 showing the numerical value of the dot spacing is displayed as dimensional information. FIG. 13B shows an example in which an image G2 in which the numerical value of the dot spacing is coded as dimensional information is displayed. The dimensional information calculation unit 221 may have a function to code the calculated dot spacing as in image G2. The control device 220 may store such image G1 or image G2 as dimensional information.

[0061] 14A shows the overall flow of processing related to teaching a predetermined pattern for calibration or position detection. Here, the flow will be described focusing on the processing when a dot pattern is used as the predetermined pattern. First, an operator executes processing on the teaching pendant 30 (second device) to acquire and display image data and dimensional information of the predetermined pattern for calibration or position detection (step S1a).

[0062] 14B is a flowchart showing the processing content in step S1a of FIG. 14A. In response to an instruction from an operator, the image data acquisition unit 131 of the teaching pendant 30 (second device) acquires image data of a predetermined pattern for calibration or position detection and dimensional information from the control device 220 (first device) (step S11a). At this time, if the control device 220 holds image data of multiple types of patterns, the image data acquisition unit 131 may be configured to display a list of multiple patterns on the display 32 and accept a user operation to select desired image data from the list.

[0063] Next, the image data display control unit 132 of the teaching pendant 30 displays the image and dimensional information of the predetermined pattern acquired from the control device 220 on the display 32 (step S12a). This makes the teaching pendant 30 available as a jig for calibration or position detection.

[0064] 14A, the operator then places the teaching pendant 30, on which an image showing dimensional information is displayed together with an image of a dot pattern, at a predetermined position for calibration and position correction, where the visual sensor 70 can capture an image (step S2). Next, the operator causes the robot control device 20A to recognize the dimensional information of the dot spacing (step S2a). In this embodiment, the robot control device 20A can recognize the dot spacing using any of the following methods (c1) to (c3): (c1) The operator directly inputs the information into the setting items of the calibration program; (c2) The robot control device reads and recognizes the numerical values ​​on the image; or (c3) The robot control device reads and recognizes the code information on the image.

[0065] The above-described technique (c1) is effective when an image G1 showing the numerical value of the dot spacing is displayed together with an image of the dot pattern, as shown in FIG. 13A . In this case, the operator can grasp the dot spacing by looking at the image of the dimensional information displayed together with the image of the dot pattern, and directly input the dot spacing into the setting items of the calibration program. FIG. 18 shows an example of an execution screen 450 of the calibration program. The execution screen 450 displays an input field 451 for specifying the dot spacing, along with an image G5 captured by the visual sensor 70 of the teaching pendant 30 placed in a predetermined position. The operator can input the dot spacing by directly entering a numerical value into the input field 451 or by specifying the dot spacing from a menu list of numerical values. The operator can then perform a predetermined operation on the execution screen 450 to execute the calibration and check the results.

[0066] When the above method (c1) is applied as a method for making the robot control device 20A recognize the dot spacing, the robot control device 20A does not need to have the function of the dimension recognition unit 124.

[0067] The above technique (c2) is an effective technique when an image G1 showing the numerical value of the dot spacing is displayed together with an image of a dot pattern, as shown in Fig. 13A. The dimension recognition unit 124 of the robot control device 20A has a function of recognizing the numerical value of the dimensional information from an image captured by the visual sensor 70 of the teaching pendant 30 in which the image G1 of the dimensional information is displayed together with the dot pattern on the display 32, and providing the numerical value to the calibration program. Various character recognition techniques known in the art can be used to recognize the numerical value from the image.

[0068] The above technique (c3) is an effective technique when displaying a code image G2 representing the dot spacing together with an image of a dot pattern, as shown in Fig. 13B. The dimension recognition unit 124 of the robot control device 20A has a function of reading the code from an image captured by the visual sensor 70 of the teaching pendant 30 in which the image G2 of the dimension information is displayed on the display 32 together with the dot pattern, and providing the code to the calibration program. The code may be a one-dimensional code or a two-dimensional code. Various code recognition techniques known in the art can be used to recognize the code.

[0069] Next, the worker causes the visual sensor 70 to capture an image of the dot pattern, and causes the robot control device 20A to perform calibration or position detection (step S3).

[0070] In the equipment configuration of this embodiment, when a marker other than a dot pattern is used as the predetermined pattern displayed on the teaching pendant 30 (second device), in an application example in which the acquisition and display of dimensional information on the teaching pendant 30 (second device) is not required, the processing flow shown in FIGS. 3A-3B in the first embodiment can be applied instead of the processing flow shown in FIGS. 14A-14B. When marker teaching is required in the position detection program, the control device 220 (first device) stores marker configuration information (dimensional information, etc.) along with marker image data. The teaching pendant 30 (second device) may then acquire the configuration information along with the marker image data from the control device 220 (first device) and provide the acquired configuration information to the robot control device 20A so that it can be used for marker teaching. Alternatively, the teaching pendant 30 may display an image indicating the marker's dimensional information on the display 32 along with the marker image. In this case, the robot control device 20A (dimension recognition unit 124) can recognize the dimension information from the captured image in which the visual sensor 70 captures an image showing the marker and the dimension information.

[0071] According to this embodiment, the operator can perform calibration or position detection by displaying an image of a predetermined pattern on the teaching pendant 30 whenever necessary. The operator does not need to perform the time-consuming task of retrieving and installing a jig for calibration or position detection from a storage location, as was done in the past. By displaying an image showing dimensional information together with the image of the predetermined pattern, it is also possible to have the robot control device 20A recognize the dimensional information. Therefore, the operator can perform calibration simply by operating the teaching pendant 30 to download image data of the predetermined pattern from the control device 220 and display it.

[0072] In the example of the equipment configuration shown in FIG. 11 , the second device that downloads and displays image data of a predetermined pattern is the teaching pendant 30. However, a portable terminal device other than the teaching pendant 30 may be used as the second device. In this case, the terminal device is positioned as a teaching device for teaching calibration or position detection. In this case, the control device 220 associates dimensional information of the dot spacing when a dot pattern image is displayed on each terminal device, as shown in Table 2 of the first embodiment, with the image data of the dot pattern and stores the information. When a request for downloading image data is received from a terminal device, the control device 220 (processor 224) may be configured to identify the terminal device that requested the image data and provide the terminal device with dimensional information corresponding to the terminal device along with the image data of the predetermined pattern. In addition, the second device that downloads and displays image data of a predetermined pattern may be a machine tool.

[0073] According to this configuration, an image of a predetermined pattern can be displayed on various terminal devices other than the teaching pendant 30, and can be used for calibration and position detection.

[0074] 11 shows an example in which the first device that provides the image data of the predetermined pattern is the machine control device 220, but the first device that provides the image data of the predetermined pattern may be various external devices. For example, the first device that provides the image data of the predetermined pattern may be the teaching pendant 30 used by the operator, or a computer or cloud that is network-connected to another terminal device.

[0075] In this embodiment, a configuration has been described in which the teaching operation panel 30 displays the dimensional information received from the control device 220 together with an image of a predetermined pattern, but the teaching operation panel 30 may also transmit the dimensional information received from the control device 220 to the robot control device 20A so that the robot control device 20A can use it for calibration and position detection.

[0076] The first and second embodiments described above are configuration examples in which a first device that provides image data of a predetermined pattern has a function for generating dimensional information about the dot spacing when the dot pattern is displayed on a second device, based on information about the display of the second device that displays the image data of the predetermined pattern. In this embodiment, the second device that displays the image data of the predetermined pattern has a function for generating dimensional information about the dot spacing.

[0077] 15 is a diagram showing the device configuration of a teaching system 503 according to the third embodiment. The teaching system 503 includes a robot system 100B. As shown in FIG. 15 , in the teaching system 503, a second device that displays an image of a predetermined pattern for calibration or position detection is a teaching pendant 30B, and a first device that provides image data to the second device (teaching pendant 30B) is an external device 320 that is network-connected to the teaching pendant 30B. In this case, the network may include an in-house network such as a local area network (LAN) or a commercial network such as the Internet.

[0078] The external device 320 may include various devices and computer systems, such as a control device arranged in the same factory as the robot system 100B, a computer, a server connected via a commercial network, a cloud, etc. The external device 320 may have a hardware configuration as a general computer, including a processor 321, memory (ROM, RAM, non-volatile memory, etc.), a storage unit 322, a display unit, an operation unit, an input / output interface, a network interface, etc. (see FIG. 16 ).

[0079] 16 shows a functional block diagram of a robot control device 20A, a teaching pendant 30B, and an external device 320 in a teaching system 503 according to the third embodiment. The external device 320 stores image data of a predetermined pattern for calibration or position detection in a storage unit 322.

[0080] The teaching pendant 30B has functions as an image data acquisition unit 131, an image data display control unit 132, a brightness adjustment unit 133, and an image data registration unit 134. These functions have been described above, so details will be omitted. The image data acquisition unit 131 has a function of acquiring image data of a predetermined pattern from the external device 320. The image data registration unit 134 provides a function of registering new image data that can be used for calibration or position detection in the external device 320. The teaching pendant 30B according to this embodiment further has a dimension information generation unit 135.

[0081] The dimension information generation unit 135 has a function of calculating, based on the resolution and size information of the display 32, the size of the dot spacing of the dot pattern when image data of the dot pattern downloaded from the external device 320 is displayed on the display 32. Specifically, the dimension information generation unit 135 analyzes the image of the dot pattern and determines the number of pixels of the dot spacing. Then, based on the resolution and size of the display 32, the dimension information generation unit 135 calculates the size of the dot spacing on the display 32 and generates dimension information. The dimension information is, for example, a numerical value or a code.

[0082] The image data display control unit 132 can display an image representing the dimensional information together with the image of the dot pattern on the display 32. Examples of the display format in this case include, as shown in Figures 13A and 13B, displaying an image of numerical values ​​representing the dimensional information together with the image of the dot pattern, or displaying an image in which the dimensional information is coded together with the image of the dot pattern.

[0083] As shown in FIG. 16, the robot control device 20A may have the same functional configuration as the robot control device 20A in the second embodiment described above.

[0084] 17A shows the overall flow of processing related to teaching a predetermined pattern for calibration or position detection. Here, the flow will be explained focusing on the processing when a dot pattern is used as the predetermined pattern. First, the operator acquires image data of the predetermined pattern for calibration or position detection on the teaching pendant 30B (second device) and executes processing to display the image data together with dimensional information (step S1b).

[0085] 17B is a flowchart showing the processing content of step S1b in FIG. 17A. In response to an instruction from an operator, the image data acquisition unit 131 of the teaching pendant 30B (second device) acquires image data of a predetermined pattern for calibration or position detection from the external device 320 (first device) (step S11). The processor 321 of the external device 320 (first device) has a function of sending image data stored in the memory unit 322 to the teaching pendant 30B in response to a request from the teaching pendant 30B. At this time, if the external device 320 holds image data of multiple types of patterns, the image data acquisition unit 131 may be configured to display a list of multiple patterns on the display 32 and accept a user operation to select desired image data from the list.

[0086] Next, the dimensional information generating unit 135 of the teaching pendant 30B (second device) calculates dimensional information of the dot spacing when the dot pattern is displayed on the display 32, based on the number of pixels between dots in the acquired image data of the dot pattern and information on the resolution and size of the display 32 (step S11b).Then, the image data display control unit 132 causes the display 32 to display an image representing the dimensional information together with the image of the dot pattern (step S12a).

[0087] 17A, the operator then places the teaching pendant 30B, on which an image showing dimensional information is displayed together with an image of a dot pattern, at a predetermined position for calibration and position correction, where the visual sensor 70 can capture an image (step S2). Next, the operator causes the robot control device 20A to recognize dimensional information about the dot spacing (step S2a). Here, the operator may cause the robot control device 20A to recognize the dot spacing using any of the methods (c1) to (c3) described in the second embodiment.

[0088] Then, the worker causes the visual sensor 70 to capture an image of the dot pattern and performs calibration or position detection (step S3).

[0089] In the equipment configuration of this embodiment, when a marker other than a dot pattern is used as the predetermined pattern to be displayed on the teaching pendant 30B (second device), in an application example in which the generation and display of dimensional information on the teaching pendant 30B (second device) is not required, the process flow shown in FIGS. 3A-3B in the first embodiment can be applied instead of the process flow shown in FIGS. 17A-17B described above. When teaching of a marker is required in the position detection program, the external device 320 (first device) stores configuration information of the marker along with image data of the marker. The teaching pendant 30B (second device) may then acquire the configuration information along with image data of the marker from the external device 320 (first device) and provide the acquired configuration information to the robot control device 20A so that it can be used to teach the marker.

[0090] In this embodiment, the dot spacing of the dot pattern is calculated and displayed on the second device that displays the image of the specified pattern, so there is no need to generate and store dimensional information in advance on the first device that provides the image data of the specified pattern.

[0091] According to this embodiment, the operator can perform calibration or position detection by displaying an image of a predetermined pattern on the teaching pendant 30B whenever necessary. The configuration of this embodiment eliminates the need for the operator to perform the time-consuming task of retrieving and installing jigs for calibration or position detection from a storage location, as in the past. Displaying an image showing dimensional information together with the image of the predetermined pattern also allows the robot control device 20A to recognize the dimensional information. Therefore, the operator can perform calibration simply by operating the teaching pendant 30B to download and display image data of the predetermined pattern from the external device 320.

[0092] 15, the second device that downloads and displays an image of a predetermined pattern from the first device (external device 320) is the teaching pendant 30B, but this is merely an example, and the second device that downloads the image of the predetermined pattern from the first device (external device 320), generates dimensional information, and displays it together with the image of the predetermined pattern may be a portable terminal device separate from the teaching pendant 30B. In this case, the terminal device is positioned as a teaching device for performing teaching related to calibration or position detection.

[0093] In this embodiment, a configuration has been described in which the teaching operation panel 30B displays the calculated dimensional information together with an image of a predetermined pattern, but the teaching operation panel 30B may also transmit the calculated dimensional information to the robot control device 20A so that the robot control device 20A can use it for calibration and position detection.

[0094] As described above, according to each embodiment, the burden on the worker when performing calibration or position detection using a predetermined pattern is reduced, and these operations can be performed efficiently.

[0095] The functional allocation shown in the functional block diagrams of the above-described embodiments is merely an example, and various modifications of the functional allocation are possible. For example, the teaching pendant 30 shown in the first embodiment may not have the functions of the brightness adjustment unit 133 and the image data registration unit 134.

[0096] As a variation of the above-described embodiment, a configuration example is possible in which a first device that performs the function of providing image data also has an application program that displays image data of a predetermined pattern, and provides such an application along with the image data to a second device that performs the function of displaying the predetermined pattern. This application program may have a function of acquiring display information (resolution, size, etc.) of the device that executes this application and calculating dimensional information when the predetermined pattern is displayed. The second device that executes this application may display an image (an image of numerical values ​​or codes) representing the calculated dimensional information together with the image of the pattern, as shown in Figures 13A-13B. Alternatively, the second device that executes this application may provide the calculated dimensional information to the first device.

[0097] This configuration will be described below with reference to the device configuration in FIG. 7 as an example. The robot control device 20, serving as a first device, has image data of a predetermined pattern along with an application program for displaying the image data. The robot control device 20 transmits the application program along with the image data to the tablet terminal 80, serving as a second device. The tablet terminal 80 executes the application program, displays the image of the predetermined pattern, and calculates dimensional information. The tablet terminal 80 may display an image (numerical values ​​or codes) representing the dimensional information along with the image of the predetermined pattern. Alternatively, the tablet terminal 80 may transmit the calculated dimensional information to the robot control device 20. The robot control device 20 can use the dimensional information recognized from the image captured by the visual sensor 70 or provided by the tablet terminal 80 for calibration and position detection.

[0098] In the above-described embodiments, the functional blocks in the functional block diagrams relating to the robot control device, teaching operation panel, tablet terminal, and external device may be realized by one or more processors of these devices executing various software stored in a storage device, or may be realized by a hardware-based configuration such as an ASIC (Application Specific Integrated Circuit).

[0099] The program for executing teaching processing for calibration or position detection in the above-described embodiment can be recorded on various computer-readable recording media (e.g., semiconductor memories such as ROM, EEPROM, and flash memory, magnetic recording media, and optical disks such as CD-ROM and DVD-ROM).

[0100] Although the present disclosure has been described in detail, the present disclosure is not limited to the individual embodiments described above. Various additions, substitutions, modifications, partial deletions, etc. are possible in these embodiments without departing from the gist of the present disclosure or the spirit of the present disclosure derived from the content of the claims and their equivalents. These embodiments can also be implemented in combination. For example, in the above-described embodiments, the order of each operation and the order of each process are shown as examples and are not limited to these. The same applies when numerical values ​​or mathematical expressions are used in the description of the above-described embodiments.

[0101] The following supplementary notes are further provided regarding the above-described embodiment and modified examples: (Supplementary Note 1) A teaching system (501, 501A, 502, 503) for teaching calibration or position detection, comprising: a first device (20, 220, 320) having a first processor and a storage unit that stores image data for the calibration or position detection, and a second device (30, 80, 30B) having a second processor and a display, wherein the second processor of the second device acquires the image data from the first device, and displays an image represented by the image data on the display to be captured by a visual sensor in performing the calibration or position detection. (Supplementary Note 2) The teaching system (501, 501A) according to Supplementary Note 1, wherein the first device is a robot control device (20) that performs the calibration or position detection, the first processor of the first device performs the calibration or position detection based on an image captured by the visual sensor of the image displayed on the display of the second device, and the second device is a teaching operation panel (30) connected to the robot control device (20) by wire or wirelessly, or a device (80) separate from the teaching operation panel. (Supplementary Note 3) The teaching system (501, 501A) according to Supplementary Note 2, wherein the storage unit of the first device (20) stores information relating to dimensions of the image on the display of the second device when the image represented by the image data is displayed on the display, in association with the image data, and the first processor of the first device uses the information relating to the dimensions in performing the calibration or the position detection. (Supplementary Note 4) The teaching system according to Supplementary Note 1, further comprising a robot control device (20A) having a third processor that performs the calibration or the position detection, wherein the first device is a device (220, 320) separate from the robot control device, and the second device is a teaching operation panel (30, 30B) connected to the robot control device (20A) by wire or wirelessly, or a device separate from the teaching operation panel.(Supplementary Note 5) The teaching system (502) according to Supplementary Note 4, wherein the storage unit of the first device (220) holds information on dimensions of an image represented by the image data when the image is displayed on the display of the second device (30), in association with the image data, and the second processor acquires the information on the dimensions together with the image data from the first device (220), and the second processor performs one of the following processes: (1) displaying an image on the display indicating information on the dimensions together with the image represented by the image data, or (2) transmitting the information on the dimensions to the robot control device. (Supplementary Note 6) The teaching system (503) according to Supplementary Note 4, wherein the second processor calculates information on dimensions when the image based on the image data is displayed on the display, based on the acquired image data and information on the resolution and size of the display, and the second processor performs one of the following processes: (1) displaying an image on the display indicating information on the dimensions together with the image represented by the image data, or (2) transmitting the calculated information on the dimensions to the robot control device. (Supplementary Note 7) The teaching system (502, 503) according to Supplementary Note 5 or 6, wherein the other device is a machine control device, an external device, or a cloud. (Supplementary Note 8) The teaching system (502, 503) according to any one of Supplements 5 to 7, wherein the image indicating the dimensional information is an image of numerical values ​​of the dimension or an image in which the dimension is coded. (Supplementary Note 9) The teaching system (502, 503) according to any one of Supplements 5 to 8, wherein the second processor displays an image indicating information about the dimension together with an image represented by the image data on the display, and the third processor of the robot control device (20A) recognizes information about the dimension from an image captured by the visual sensor of the image represented by the image data and an image indicating information about the dimension displayed on the display of the second device (30, 30B), and uses the recognized information about the dimension in performing the calibration or position detection.(Supplementary Note 10) The teaching system (501501A, 502) according to Supplementary Note 3 or 5, wherein the first processor of the first device (20, 220) generates information about the dimensions based on image data of the dot pattern and information about the resolution and size of the display of the second device. (Supplementary Note 11) The teaching system (501, 501A, 502, 503) according to any one of Supplements 1 to 10, wherein the second processor of the second device (30, 80, 30B) registers new image data of a predetermined pattern usable for the calibration or position detection and information about the dimensions of the new image data in the storage unit of the first device based on a user operation. (Supplementary Note 12) The teaching system (501, 501A, 502, 503) according to Supplementary Note 1, wherein the second device is a teaching operation panel (30, 30B) connected to a robot control device, and a second processor of the second device acquires, from the robot control device, information regarding brightness of an image captured by the visual sensor of the second device displaying an image represented by the image data on the display, and adjusts the display brightness of the image based on the information regarding brightness. (Supplementary Note 13) The teaching system (501, 501A, 502) according to Supplementary Note 1, wherein the storage unit of the first device further stores an application program for displaying the image represented by the image data, and the second processor acquires the application program together with the image data from the first device, and the second processor executes the application program to display the image represented by the image data on the display of the first device.(Supplementary Note 14) The application program further has a function of calculating information about dimensions when an image represented by the image data is displayed on a display of a device executing the application, based on information about the display of the device, and the second processor performs one of the following processes: (1) displaying an image indicating information about the dimensions obtained by executing the application together with the image represented by the image data, or (2) transmitting information about the dimensions to the first device. (Supplementary Note 15) A teaching device (30, 80, 30B) used for teaching related to calibration or position detection, comprising: a display (32, 82) and a processor (31, 81), wherein the processor (31, 81) acquires image data for the calibration or position detection from an external device, and displays the image represented by the image data on the display to be captured by a visual sensor in performing the calibration or position detection. (Supplementary Note 16) The teaching device (30) according to Supplementary Note 15, wherein the processor further acquires, from the external device, information regarding the dimensions of the image represented by the image data when the image is displayed on the display, and the processor performs one of the following processes: (1) displaying, on the display, an image indicating information regarding the dimensions together with the image represented by the image data; or (2) transmitting the information regarding the dimensions to a robot control device that controls the calibration or position detection. (Supplementary Note 17) The teaching device (30B) according to Supplementary Note 15, wherein the processor calculates information regarding the dimensions of the image based on the image data when displayed on the display, based on the acquired image data and information regarding a resolution and a size of the display, and the processor performs one of the following processes: (1) displaying, on the display, an image indicating information regarding the dimensions together with the image represented by the image data; or (2) transmitting the calculated information regarding the dimensions to a robot control device that controls the calibration or position detection.(Supplementary Note 18) The teaching device (30, 80, 30B) according to any one of Supplementary Notes 15 to 17, wherein the processor (31, 81) further registers, in a storage unit of the external device, new image data of a predetermined pattern that can be used for the calibration or position detection and information about dimensions of the new image data, based on a user operation. (Supplementary Note 19) The external device is a robot control device (20, 20A) that performs the calibration or position detection, and the processor (31) acquires, from the robot control device, information about brightness of an image captured by the visual sensor of the teaching device displaying an image represented by the image data on the display, and adjusts the display brightness of the image based on the information about brightness. (Supplementary Note 20) A robot control device (20, 20A) comprising: a processor (21); and a storage unit (22) that stores image data for calibration or position detection, wherein the processor (21) transmits the image data to an external device in response to a request from the external device, and performs the calibration or position detection based on an image captured by a visual sensor of an image represented by the image data displayed on a display of the external device. (Supplementary Note 21) The robot control device (20, 20A) according to Supplementary Note 20, wherein the storage unit stores information relating to dimensions of the image on a display of the external device when the image represented by the image data is displayed on the display, in association with the image data, and the processor uses the information relating to the dimensions in performing the calibration or the position detection. (Supplementary Note 22) The robot control device (20) according to Supplementary Note 19, wherein the processor (21) calculates the information relating to the dimensions based on the image data and information relating to the resolution and size of the display of the external device.(Supplementary Note 23) The robot control device (20A) according to Supplementary Note 20, wherein an image represented by the image data is displayed on a display of the external device together with information regarding the dimensions of the image when the image is displayed on the display, and the processor (21) recognizes information regarding the dimensions from an image captured by the visual sensor of the image represented by the image data and an image indicated by the information regarding the dimensions displayed on the display of the external device, and uses the recognized information regarding the dimensions in performing the calibration or position detection. (Supplementary Note 24) A program for causing a computer processor to execute the steps of: storing image data for calibration or position detection in a storage unit; transmitting the image data to the external device in response to a request from the external device; and performing the calibration or position detection based on an image captured by a visual sensor of an image represented by the image data displayed on the display of the external device.

[0102] REFERENCE SIGNS LIST 10 Robot 11 Hand 20, 20A Robot control device 21 Processor 22 Memory unit 30, 30B Teaching operation panel 31 Processor 32 Display 33 Operation unit 40 Image processing device 70 Visual sensor 80 Tablet terminal 81 Processor 82 Display 83 Operation unit 90 Machine tool 91 Control device 92 Display 100, 100A, 100B Robot system 121 Operation control unit 122 Dimension information calculation unit 123 Image data storage control unit 124 Dimension recognition unit 131 Image data acquisition unit 132 Image data display control unit 133 Brightness adjustment unit 134 Image data registration unit 135 Dimension information generation unit 181 Image data acquisition unit 182 Image data display control unit 184 Image data registration unit 220 Control device 221 Dimension information calculation unit 222 Image data storage control unit 224 Processor 225 Storage unit 301 Dot pattern 302 Marker 320 External device 321 Processor 322 Storage unit 402 Registration screen 501, 501A, 502, 503 Teaching system

Claims

1. A teaching system for providing instruction related to calibration or position detection, The first processor and A first apparatus having a storage unit for storing image data for calibration or position detection, The second processor, A second device comprising a display, The second processor is, The image data is acquired from the first device, In the execution of the calibration or position detection, the image represented by the image data is displayed on the display in order to cause the visual sensor to capture an image. Instructional system.

2. The first device is a robot control device that performs the calibration or position detection, The first processor of the first device is The calibration or position detection is performed based on the image captured by the visual sensor, which is the image displayed on the display of the second device. The teaching system according to claim 1, wherein the second device is a teaching control panel connected to the robot control device by wire or wireless connection, or a device separate from the teaching control panel.

3. The storage unit of the first device stores information relating to the dimensions of the image on the display of the second device when the image represented by the image data is displayed on the display of the second device, The teaching system according to claim 2, wherein the first processor of the first device uses information relating to the dimensions in performing the calibration or position detection.

4. The robot control device further comprises a third processor that performs the calibration or position detection, The first device is a separate device from the robot control device, The teaching system according to claim 1, wherein the second device is a teaching control panel connected to the robot control device by wire or wireless connection, or a device separate from the teaching control panel.

5. The storage unit of the first device stores information relating to the image data the dimensions of the image when the image represented by the image data is displayed on the display of the second device, The second processor acquires the image data along with the dimension information from the first device. The second processor is, (1) Display an image on the display that shows information about the dimensions along with the image represented by the image data. (2) Transmit the information relating to the dimensions to the robot control device. The teaching system according to claim 4, which performs either of the following processes.

6. The second processor calculates information regarding the dimensions when the image based on the image data is displayed on the display, based on the acquired image data and information regarding the resolution and size of the display. The second processor is, (1) Display an image on the display that shows information about the dimensions along with the image represented by the image data. (2) Transmit the calculated information regarding the dimensions to the robot control device. The teaching system according to claim 4, which performs either of the following processes.

7. The teaching system according to claim 5 or 6, wherein the other device is a machine control device, an external device, or a cloud.

8. The teaching system according to claim 5 or 6, wherein the image showing information regarding the dimensions is an image of the numerical value of the dimensions, or an image of the dimensions encoded.

9. The second processor displays an image on the display that shows the dimensions along with the image represented by the image data. The third processor of the robot control device is From the image captured by the visual sensor, which shows the image data displayed on the display of the second device and the image showing the information regarding dimensions, the information regarding dimensions is recognized. The teaching system according to claim 5 or 6, wherein the recognized information regarding the dimensions is used in the execution of the calibration or position detection.

10. The teaching system according to claim 3 or 5, wherein the first processor of the first device generates information relating to dimensions based on the image data and information relating to the resolution and size of the display of the second device.

11. The second processor of the second device is A teaching system according to any one of claims 1 to 6, wherein, based on user operation, new image data usable for calibration or position detection and information regarding the dimensions of the new image data are registered in the storage unit of the first device.

12. The second device is a teaching control panel connected to a robot control device. The teaching system according to claim 1, wherein the second processor of the second device, which displays the image represented by the image data on the display, obtains information from the robot control device regarding the brightness of an image captured by the visual sensor, and adjusts the brightness of the display that shows the image based on the brightness information.

13. The storage unit of the first device further stores an application program for displaying the image represented by the image data. The second processor acquires the application program along with the image data from the first device. The teaching system according to claim 1, wherein the second processor displays the image represented by the image data on the display of the first device by executing the application program.

14. The application program further has a function to calculate information regarding the dimensions of the image represented by the image data when it is displayed on the display of the device on which the application is executed, based on information regarding the display of the device on which the application is executed. The second processor is, (1) Display the image showing the dimensions obtained by executing the application together with the image represented by the image data. (2) Transmit the information relating to the dimensions to the first device. The teaching system according to claim 13, which performs either of the following processes.

15. A teaching device used for teaching calibration or position detection, The display and Equipped with a processor, The aforementioned processor, Acquire image data for calibration or position detection from an external device. A teaching device that displays the image represented by the image data on the display in order to cause the visual sensor to capture an image during the execution of the calibration or position detection.

16. The processor further obtains information from the external device regarding the dimensions of the image when the image represented by the image data is displayed on the display. The aforementioned processor, (1) Display an image on the display that shows information about the dimensions along with the image represented by the image data. (2) Transmit the information relating to the dimensions to the robot control device responsible for calibration or position detection. The teaching device according to claim 15, which performs either of the following processes.

17. The processor calculates information regarding the dimensions when the image based on the image data is displayed on the display, based on the acquired image data and information regarding the resolution and size of the display. The aforementioned processor, (1) Display an image on the display that shows information about the dimensions along with the image represented by the image data. (2) Transmit the calculated information regarding the dimensions to the robot control device responsible for calibration or position detection. The teaching device according to claim 15, which performs either of the following processes.

18. The aforementioned processor further, A teaching device according to any one of claims 15 to 17, which registers new image data usable for calibration or position detection and information regarding the dimensions of the new image data in the storage unit of the external device based on user operation.

19. The aforementioned external device is a robot control device that performs the calibration or position detection. The teaching device according to claim 15, wherein the processor obtains information regarding the brightness of an image captured by the visual sensor from the robot control device, and adjusts the brightness of the display on the display that shows the image based on the brightness information.

20. A robot control device, Processor and It comprises a storage unit for storing image data for calibration or position detection, The aforementioned processor, In response to a request from an external device, the image data is transmitted to the external device. A robot control device that performs calibration or position detection based on an image captured by a visual sensor, which represents the image data displayed on the display of the external device.

21. The storage unit stores information relating to the dimensions of the image on the display of the external device when the image represented by the image data is displayed on the display of the external device, and stores this information in association with the image data. The robot control device according to claim 20, wherein the processor uses information regarding the dimensions in performing the calibration or position detection.

22. The aforementioned processor, The robot control device according to claim 21, which calculates information relating to dimensions based on the image data and information relating to the resolution and size of the display of the external device.

23. The display of the external device shows, along with the image represented by the image data, an image of information regarding the dimensions of the image when it is displayed on the display. The aforementioned processor, From the image captured by the visual sensor, the image representing the image data displayed on the display of the external device and the image representing the dimension information, the dimension information is recognized. The robot control device according to claim 20, wherein the recognized information regarding the dimensions is used in the execution of the calibration or position detection.

24. In a computer processor, A procedure for storing image data for calibration or position detection in a storage unit, A procedure for transmitting the image data to an external device in response to a request from the external device, A procedure for performing calibration or position detection based on an image captured by a visual sensor, where the image data displayed on the display of the external device is represented by the image data. A program for performing this procedure.