Teaching system, teaching device, robot control device, and program

Abstract

A teaching system for performing teaching relating to calibration or position detection is provided with: a first device having a first processor and a storage unit for storing image data for the calibration or position detection; and a second device having a second processor and a display, the second processor acquiring the image data from the first device, and during execution of the calibration or position detection, displaying an image represented by the image data on the display in order to cause a vision sensor to photograph.

Description

Technical Field

[0001] This disclosure relates to teaching systems, teaching devices, robot control devices, and programs. Background Technology

[0002] Various methods are known in the art for calibrating cameras configured in robot systems using grippers with specific patterns, such as dot-patterned grippers (e.g., Patent Documents 1-2). Additionally, various techniques are known for measuring or detecting the position of a measurement object by using a camera to capture marks (e.g., Patent Documents 3-4).

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2014-128845

[0006] Patent Document 2: Japanese Patent Application Publication No. 2019-42834

[0007] Patent Document 3: Japanese Patent Application Publication No. 2012-218140

[0008] Patent Document 4: Japanese Patent Application Publication No. 2005-201824 Summary of the Invention

[0009] The problem that the invention aims to solve

[0010] When using dot patterns or markings for calibration or position detection, operators typically need to remove the fixtures or printed materials that form the dot patterns or markings from storage before proceeding. A technology is desired that can reduce the burden on operators when using dot patterns or markings for calibration or position detection.

[0011] Methods for solving problems

[0012] One aspect of this disclosure is a teaching system for performing teaching related to calibration or position detection, comprising: a first device having a first processor and a storage unit for storing image data for the calibration or position detection; and a second device having a second processor and a display. In this teaching system, the second processor acquires the image data from the first device, and during the execution of the calibration or position detection, displays an image represented by the image data on the display to enable a vision sensor to take an image.

[0013] These objects, features, and advantages of the invention will become more apparent from the detailed description of typical embodiments of the invention shown in the accompanying drawings. Attached Figure Description

[0014] Figure 1 This is a system structure diagram of a teaching system for performing teaching related to calibration or position detection according to the first embodiment.

[0015] Figure 2 This is a functional block diagram of the teaching system in the first embodiment.

[0016] Figure 3A This is a diagram illustrating the overall flow of the process related to teaching a predetermined pattern in calibration or position detection in the first embodiment.

[0017] Figure 3B It means Figure 3A The flowchart shows the processing content of the second device in step S1.

[0018] Figure 4 It is a diagram showing the state of the display on the teach pendant panel displaying a dot pattern for calibration.

[0019] Figure 5 It is a diagram that shows the state of dot patterns superimposed on the operation screen used for teaching.

[0020] Figure 6 This is an example diagram showing a registration screen displaying image data on a teach pendant display panel.

[0021] Figure 7 This is a diagram of the device structure when the second device for downloading and displaying image data of a predetermined pattern is a tablet terminal that is different from the teaching operation panel.

[0022] Figure 8 This is a functional block diagram of a tablet terminal.

[0023] Figure 9 This is a diagram showing the state of the tablet terminal's display showing markers used for location detection.

[0024] Figure 10 It is a diagram showing the status of the markers used for position detection displayed on the machine tool's monitor.

[0025] Figure 11 This is a system structure diagram of a teaching system for performing teaching related to calibration or position detection according to the second embodiment.

[0026] Figure 12 This is a functional block diagram of the teaching system in the second embodiment.

[0027] Figure 13A It is a diagram showing the state of a dot pattern and a numerical image representing the dot interval as size information displayed on the monitor of the teaching operation panel.

[0028] Figure 13BIt is a diagram showing the state of a dot pattern and a code as size information displayed together on the display of the teach pendant.

[0029] Figure 14A This is a diagram illustrating the overall flow of the process related to teaching a predetermined pattern in calibration or position detection in the second embodiment.

[0030] Figure 14B It means Figure 14A The flowchart shows the processing content of the second device in step S1a.

[0031] Figure 15 This is a system structure diagram of a teaching system for performing teaching related to calibration or position detection according to the third embodiment.

[0032] Figure 16 This is a functional block diagram of the teaching system in the third embodiment.

[0033] Figure 17A This is a diagram illustrating the overall flow of the process related to teaching a predetermined pattern in calibration or position detection in the third embodiment.

[0034] Figure 17B It means Figure 17A The flowchart of the processing content of the second device in step S1b.

[0035] Figure 18 This is an example of a screen showing the execution of a calibration procedure. Detailed Implementation

[0036] Next, embodiments of the present disclosure will be described with reference to the accompanying drawings. In the drawings, the same reference numerals are used to denote the same structural or functional parts. The scales of these drawings have been appropriately altered for ease of understanding. Furthermore, the embodiments shown in the drawings are examples for carrying out the invention, and the invention is not limited to the illustrated embodiments.

[0037] In the embodiments described below, a teaching system for performing teaching related to calibration and position detection will be described.

[0038] In this specification, the predetermined pattern used for calibration may include various dot patterns and markings known in the art that can be used for calibration. Examples of dot patterns and markings used for calibration are described in Patent Documents 1 and 2 above. Calibration corresponds to determining calibration data (transformation matrix) that associates the position on the camera image with the position in three-dimensional space. The calibration data includes internal parameters corresponding to the transformation from the camera coordinate system to the image coordinate system, and external parameters corresponding to the transformation (rotation, translation) from the world coordinate system to the camera coordinate system. As a dot pattern that can be used for calibration, for example, a dot pattern that satisfies the following conditions: (a1) the grid spacing of the dot pattern is known; (a2) there are a certain number or more grid points; (a3) ​​each grid point can be uniquely identified. Figure 4 The diagram illustrates such a dot pattern 301. In dot pattern 301, dots are arranged in a grid pattern at equal intervals. A coordinate system (X-axis, Y-axis, and origin O) is defined on the dot pattern by a large point M (for only a portion of the annotation symbols). Thus, each grid point is determined by its position.

[0039] Among the predetermined patterns used for detecting the position of a measured object, there are various types of patterns known in the art, referred to as markers or targets. Examples of such markers are described in Patent Documents 3 and 4 mentioned above. There are also application examples for correcting the coordinate system using markers. For example, in an application example of correcting the positional relationship between a robot mounted on a trolley or AGV (Automated Guided Vehicle) and a machine tool, markers are affixed to one or more parts of the measured object (machine tool), and the positional relationship is corrected by detecting the markers using a robot equipped with a calibrated camera. Furthermore, among the markers that can be used for position detection, there are not only... Figure 5 The illustrative mark 302, the mark illustrated in patent documents 3 and 4, may also include various marks that can be detected on an image, such as dot patterns, simpler graphics, symbols, and characters.

[0040] In this specification, when referring to a predetermined pattern for calibration or position detection, it may include the various types of patterns described above that can be used in calibration, detection or measurement of the position of an object.

[0041] The teaching system of each embodiment described below includes: a first device having a first processor and a storage unit for storing image data of a predetermined pattern for calibration or position detection; and a second device having a second processor and a display. In this teaching system, the second processor of the second device has the function of acquiring image data of the predetermined pattern from the first device and displaying the image of the predetermined pattern on the display in order to enable a vision sensor to take a picture during the execution of calibration or position detection.

[0042] First Implementation Method

[0043] Figure 1 The diagram illustrates the system structure of a teaching system 501 according to the first embodiment, used for teaching related to calibration or position detection. This teaching system 501 includes a robot system 100. Figure 2 This represents a functional block diagram of the teaching system 501. For example... Figure 1 As shown, the robot system 100 includes a robot 10, a robot control device 20 for controlling the robot 10, a teaching pendant 30, an image processing device 40 connected to the robot control device 20, and a vision sensor 70. The vision sensor 70 is mounted on the forearm of the robot 10 and connected to the image processing device 40. By calibrating the vision sensor 70, the robot system 100 can perform operations on the object by detecting the position of the object using the vision sensor 70.

[0044] exist Figure 1 In the structure shown, the robot control unit 20 (first unit) is responsible for performing calibration or position detection using the vision sensor 70. The teach pendant 30 (second unit) has the following functions: acquiring image data of a predetermined pattern for calibration or position detection from the robot control unit 20 (first unit), and displaying the image of the predetermined pattern on the display 32 of the teach pendant 30 (second unit) based on the acquired image data, so that the vision sensor can take a picture.

[0045] The robot control device 20 controls the actions of the robot 10 according to the robot program or instructions from the teach pendant 30. The robot control device 20 may also have a general computer hardware structure, including a processor 21, memory (ROM, RAM, non-volatile memory, etc.), storage unit 22, operation unit, input / output interface, network interface, etc. (see reference). Figure 2 ).

[0046] The teach pendant 30 is connected to the robot control device 20 via wired or wireless means, providing functions for teaching the robot 10 and inputting various settings. The teach pendant 30 can also be a teaching device composed of various portable terminal devices (tablet terminals, etc.). The teach pendant 30 can also have a general computer hardware structure, including a processor 31, memory (ROM, RAM, non-volatile memory, etc.), storage device, display 32, operation unit 33, input / output interface, network interface, etc. (see reference). Figure 2 The operation unit can also be configured as a touch operation panel integrated with the display 32.

[0047] Robot 10 in Figure 1While the robot 10 is a vertical articulated robot, it can also be a horizontal articulated robot, a parallel linkage robot, a dual-arm robot, or other types of robots depending on the task at hand. The robot 10 can perform the desired task using an end effector mounted on its wrist. The end effector is an external device that can be replaced depending on the application, such as a robotic arm, welding torch, or tool. Figure 1 An example of using a robotic arm 11 as an end effector is shown.

[0048] The image processing unit 40 is responsible for controlling the vision sensor 70 and performing image processing (detection, judgment, etc.) on the captured images based on instructions from the robot control unit 20 (motion control unit 121). The vision sensor 70 can be a camera that captures grayscale images or color images, or it can be a stereo camera or a 3D sensor that can acquire distance images or 3D point groups.

[0049] Furthermore, in this embodiment, a structural example is shown where the image processing device 40 is configured as a different device from the robot control device 20, but the functions of the image processing device 40 can also be integrated into the robot control device 20.

[0050] like Figure 2 As shown, the robot control device 20 includes a motion control unit 121, a size information calculation unit 122, and an image data storage control unit 123. These functional blocks can also be implemented by executing software through the processor 21 of the robot control device 20. Figure 2 The diagram illustrates the storage unit 22, a hardware component of the robot control device 20. The storage unit 22 is, for example, a storage device comprised of non-volatile memory or a hard disk drive. The storage unit 22 stores various programs, such as robot programs for controlling the robot 10, calibration programs, and position detection programs (programs used to measure or detect the position of an object using predetermined patterns such as markers), as well as various setting information. Furthermore, the storage unit 22 holds image data of predetermined patterns that can be used for calibration or position detection.

[0051] The motion control unit 121 controls the motion of the robot 10 according to the robot program or according to the instructions from the teach pendant 30. The robot control device 20 includes a servo control unit (not shown) that executes servo control of the servo motors for each axis according to the instructions generated by the motion control unit 121 for each axis.

[0052] like Figure 2As shown, the teach pendant 30 includes an image data acquisition unit 131, an image data display control unit 132, a brightness adjustment unit 133, and an image data registration unit 134. Furthermore, these functional blocks can also be implemented by software executed by the processor 31. The image data acquisition unit 131 has the function of acquiring image data of a predetermined pattern for calibration or position detection from the robot control device 20. The image data display control unit 132 has the function of displaying the acquired image of the predetermined pattern on the display 32. According to this structure, by displaying images of predetermined patterns such as dot patterns and marks on the teach pendant 30, the operator can use the teach pendant 30 as a calibration fixture or a fixture for object position detection.

[0053] In order to perform calibration using a dot pattern, the size information of the dot interval is required. As an example of the structure in the robot control device 20 that performs calibration for knowing the size of the dot interval of the dot pattern displayed on the teach pendant 30, in this embodiment, the size information is calculated in the robot control device 20 based on information related to the display 32 of the teach pendant 30 and stored in association with the image data of the dot pattern.

[0054] The size information calculation unit 122 of the robot control device 20 can, for example, calculate the size of the dot spacing when the dot pattern is displayed on the display 32, based on the number of pixels in the grid spacing in the dot pattern image data and the resolution and size information of the display 32 of the teaching operation disk 30. For example, the size information calculation unit 122 may also be configured to perform the following steps.

[0055] (Step b1) Receive input of information (resolution and size) related to the display 32 via the teaching operation panel 30.

[0056] (Step b2) Calculate the above-mentioned size information based on the image data of the dot pattern and information related to the display 32.

[0057] (Step b3) The calculated size information is stored in the storage unit 22 in association with the image data of the dot pattern.

[0058] The image data storage control unit 123 has the function of storing image data and size information of dot patterns in association in the storage unit 22. Furthermore, when image data of multiple dot patterns with different dot intervals are prepared in advance, the size of the dot interval displayed on the display 32 can be calculated for each dot pattern (image data) as shown in Table 1 below, and saved in association with the image data. Thus, the teach pendant 30 can be used as a variety of calibration fixtures with different dot intervals.

[0059] [Table 1]

[0060]

[0061] Figure 3A This describes the overall process related to teaching a predetermined pattern in calibration or position detection. First, the operator performs processing to acquire and display image data of the predetermined pattern for calibration or position detection on the teaching operation panel 30 (second device) (step 1).

[0062] Figure 3B It means Figure 3A The flowchart describes the processing steps in step S1. According to the operator's instructions, the image data acquisition unit 131 of the teach pendant 30 (second device) acquires image data of a predetermined pattern for calibration or position detection from the robot control device 20 (first device) (step S11). At this time, if the robot control device 20 holds image data of multiple patterns as described above, the image data acquisition unit 131 can also be configured to display a list of multiple patterns on the display 32 and accept user operations to select the desired image data from the list.

[0063] Next, the image data display control unit 132 of the teach pendant 30 causes the display 32 to display an image of a predetermined pattern acquired from the robot control device 20 (step S12). Thus, the teach pendant 30 can be used as a fixture for calibration or position detection.

[0064] like Figure 3A As shown, next, the operator positions the teach pendant 30 (second device) displaying a predetermined pattern at a predetermined position that can be captured by the vision sensor 70 for calibration and position correction (step S2). Then, the operator causes the vision sensor 70 to capture an image of the teach pendant 30 displaying the predetermined pattern, causing the robot control device 20 to perform calibration or position detection (step S3). In this case, the robot control device 20 (first device) retains the dimensional information when the teach pendant 30 displays the dot pattern, and therefore can use this dimensional information when performing calibration.

[0065] Figure 4 This indicates that the display 32 of the teaching operation panel 30 displays an image of the calibration dot pattern 301 through the above step S1. Figure 4 The dot pattern 301 shown satisfies the above-mentioned requirements (a1) to (a3) ​​for a dot pattern used for calibration. For example... Figure 4 As shown, the dot interval d on the display 32 in the state of displaying dot pattern 301 is known in the robot control device 20 responsible for performing calibration. Therefore, the robot control device 20 can perform calibration appropriately using the value of the dot interval d held by the robot control device 20. In addition, such dot pattern 301 can also be used to detect the position of an object or to set a coordinate system.

[0066] The operator can perform calibration or position detection simply by operating the teaching pendant 30 to download and display image data of a predetermined pattern from the robot control device 20. Therefore, the operator no longer needs to perform the time-consuming task of removing and setting up the fixtures used for calibration and position detection from their storage location, as was done previously.

[0067] In addition, as described above, by storing image data of dot patterns of various sizes in the robot control device 20, the operator can obtain image data of dot patterns of sizes that match the execution environment from the robot control device 20 at any time and display them on the teaching operation panel 30 for teaching.

[0068] According to this embodiment, the operator can display an image of a predetermined pattern on the teach pendant 30 at any time for calibration or position detection when needed. For example... Figure 5 As shown, the operator can also, while the operation screen 401 for teaching the robot 10 is displayed on the teaching operation panel 30, activate the image data acquisition unit 131 at any time when necessary to download image data of a predetermined pattern from the robot control device 20 and overlay it on the operation screen 401 for teaching. Figure 5 In the example, a dot pattern 301 is shown overlapping with the operation screen 401 used for teaching.

[0069] The brightness adjustment unit 133 of the teach pendant 30 has the following function: receiving information from the robot control device 20 (calibration program or position detection program) regarding the brightness of the image obtained from the teach pendant 30 by the vision sensor 70, which displays an image of a predetermined pattern, and adjusting the brightness of the image of the predetermined pattern displayed on the display 32. For example, if the information from the robot control device 20 indicates that the captured image is too dark, the brightness adjustment unit 133 increases the brightness of the image of the predetermined pattern on the display 32. Alternatively, if the information from the robot control device 20 indicates that the captured image is too bright, the brightness adjustment unit 133 decreases the brightness of the image of the predetermined pattern on the display 32. With this function, the brightness of the predetermined pattern can be automatically adjusted to an appropriate state regardless of the lighting environment in the workspace. The operator does not need to perform time-consuming tasks such as adjusting the lighting equipment in the workspace to adjust the brightness of the image obtained by the vision sensor 70 that displays the predetermined pattern.

[0070] Furthermore, the brightness adjustment unit 133 can also be equipped with the function of manually adjusting the display brightness of the image of the predetermined pattern. Even in this case, the operator can observe the state of the image captured by the vision sensor 70 and adjust the brightness of the image of the predetermined pattern displayed on the teaching operation panel 30 to an appropriate state. The operator does not need to perform time-consuming operations such as adjusting the lighting device in the work space in order to adjust the brightness of the image obtained by the vision sensor 70 capturing the predetermined pattern.

[0071] The teaching operation panel 30 (second device), which is responsible for displaying an image of a predetermined pattern, has the function of appropriately adjusting the brightness of the image display, so that the operator does not need to adjust the brightness of the lighting in the work space. Therefore, it can reduce the burden on the operator when performing calibration and position detection, and enable more efficient operation.

[0072] The image data registration unit 134 provides the robot control device 20 (first device) with the function of registering new image data of predetermined patterns for calibration or position detection. For example, the image data registration unit 134 provides the function of registering any image data from the image data stored in the teach pendant 30 (or an external storage device connected to the teach pendant 30) as new image data for calibration or position detection to the robot control device 20.

[0073] Figure 6This example illustrates a registration screen 402 for image data displayed on the display 32 of the teach pendant 30 via the function of the image data registration unit 134. The registration screen 402 displays a list of image data stored in the teach pendant 30 (or an external storage device connected to the teach pendant 30). The operator can register the selected image data as a new pattern in the storage unit 22 of the robot control device 20 by selecting the desired image data from the displayed list and pressing the OK button 411. The image data storage control unit 123 of the robot control device 20 provides the function of accepting registration requests from the image data registration unit 134 and saving new image data to the storage unit 22. Furthermore, as a newly registered pattern, any image that can be used for calibration or position detection can be created using various marks, graphics, symbols, characters, etc. Additionally, the image data registration unit 134 can also be configured to accept the registration of information related to the size of the pattern when registering image data for a new pattern. Size-related information includes, for example, the number of pixels in the dot pitch of a dot pattern, or the predetermined size (millimeters) of the dot pitch on the display, as described above. Alternatively, the size-related information could also be information related to the size of the mark (e.g., the size of circle C (number of dots), the size of circle C (square millimeters) when displayed on a specific monitor, etc.). Thus, the robot control device 20 can also retain information that maps image data to size information, as illustrated in Table 1 above, regarding new image data.

[0074] The above-described embodiment is a structural example in which image data of a predetermined pattern stored in the robot control device 20 (first device) is downloaded to and displayed on the teaching pendant 30 (second device). However, the second device for downloading and displaying image data of the predetermined pattern is not limited to the teaching pendant 30. For example, it may be configured to download and display image data of the predetermined pattern using a portable terminal device different from the teaching pendant 30. In this case, the terminal device can be positioned as a teaching device for performing teaching related to calibration or position detection.

[0075] Figure 7 This illustrates an example of the device structure of a teaching system 501A. In the teaching system 501A, image data of a predetermined pattern is downloaded and displayed via a tablet terminal 80, which is different from the teaching operation panel 30. Figure 8 This is a functional block diagram of the tablet terminal 80, illustrating the system architecture. (Example:) Figure 8As shown, the tablet terminal 80 includes an image data acquisition unit 181, an image data display control unit 182, and an image data registration unit 184. The functions of these image data acquisition units 181, image data display control units 182, and image data registration units 184 are the same as those of the image data acquisition units 131, image data display control units 132, and image data registration units 134 of the aforementioned teaching operation panel 30. Furthermore, in Figure 8 The diagram shows the display 82 and the operation unit 83, which are hardware components of the tablet terminal 80.

[0076] Furthermore, the tablet terminal 80 may also have the hardware structure of a general computer, including a processor 81, memory (ROM, RAM, non-volatile memory, etc.), storage device, display 82, operation unit 83, input / output interface, network interface, etc. The operation unit may also be configured as a touch operation panel integrated with the display.

[0077] The tablet terminal 80 may or may not be communicatively connected to the robot control device 20. In cases where the tablet terminal 80 cannot communicate with the robot control device 20, for example, the operator may download and save image data from the robot control device 20 to a USB memory. Furthermore, the operator can connect the USB memory to the tablet terminal 80 to acquire and display image data of desired patterns on the tablet terminal 80.

[0078] In this case, the robot control device 20 can retain information from both the display 32 of the teaching pendant 30 and the display 82 of the tablet terminal 80. Therefore, the robot control device 20 can calculate and retain the size of the dot spacing on the display 82 based on the number of pixels in the dot pattern image data and the resolution and size of the display 82 of the tablet terminal 80. Thus, as shown in Table 2 below, the robot control device 20 can retain, in association with the image data, the dot spacing size information when displaying the dot pattern on the teaching pendant 30 and the dot spacing size information when displaying the dot pattern on the tablet terminal 80.

[0079] [Table 2]

[0080]

[0081] Furthermore, the information shown in Table 2 can be expanded to accommodate three or more terminal devices. In this case, based on the number of pixels in the dot pattern's dot interval and the display information (resolution, size) of the three or more terminal devices, the dot interval when displaying the dot pattern on the three or more terminal devices can be calculated respectively.

[0082] Thus, in the case where the robot control device 20 has a structure that holds size information about multiple displays, the robot control device 20 (processor 21) can also be configured to, for example, be able to obtain information about which display to use in the execution of calibration or position detection via user input through the teach pendant 30.

[0083] exist Figure 7 and Figure 8 In the case of the structural example shown, the size of the dot interval when displaying the dot pattern on the tablet terminal 80 is known in the robot control device 20, so it is possible to obtain the same size as the reference. Figures 1 to 6 The advantages of the above-described embodiments are the same. Furthermore, in Figure 7 as well as Figure 8 In the case of the structural example shown, it is advantageous to simultaneously perform the operation of the robot 10 using the teaching operation panel 30 and the calibration or position detection using the tablet terminal 80.

[0084] Figure 9 This indicates that the image of the marker 302 acquired from the robot control device 20 is displayed on the display 82 of the tablet terminal 80. As an example, the marker 302 is a marker that includes mutually orthogonal lines a and b and a circle C of known size. As an example, the marker 302 is used as follows: In the case of using the marker 302 for object position detection, the operator pre-calibrates the vision sensor 70. Then, the operator operates the tablet terminal 80 to download the image data of the marker 302 from the robot control device 20 and display it on the display 82 (step S1). The operator sets the tablet terminal 80 with the marker 302 at a predetermined position on the object being measured (workbench, machine tool, etc.) (step S2), causing the robot control device 20 to execute the position detection program (step S3). The position detection program determines the three-dimensional position of the marker 302 based on the detected position of the marker 302 in the captured image, thus obtaining the three-dimensional position of the object being measured. Alternatively, a stereoscopic measurement method using the vision sensor 70 can also be used to measure the three-dimensional position of the marker 302. Therefore, even when performing such position detection, the operator does not need to perform operations such as taking out the marking fixtures or printed markings stored in other locations and setting them on the object.

[0085] Furthermore, when using a marker of the type 302 for object position detection, if teaching size information (such as the size of circle C) is required for the marker, the robot control device 20 (first device) may also retain the size information in association with the image data of the marker when the marker is displayed on the display 82 of the tablet terminal 80 (second device), and use the size information in object position detection.

[0086] The second device for displaying an image of a predetermined pattern can also be a machine tool used in conjunction with robot 10. Figure 10 This is a schematic diagram illustrating the equipment structure in this scenario. For example, this scenario corresponds to an application where the robot 10 is mounted on a trolley or AGV and positioned relative to the machine tool 90 at a predetermined location to move workpieces in and out of the machine tool 90. A control device (numerical control device) 91 integrated with a display 92 is assembled in the machine tool 90. The control device 91, for example, can download and display image data of a predetermined pattern from the robot control device 20 via a network. Alternatively, the image data of the predetermined pattern can be pre-registered in the storage unit of the control device 91.

[0087] Figure 10 The diagram illustrates a scenario where the display 92 of the control device 91 displays the mark 302. The display 92 of the control device 91 is mounted in a fixed position on the machine tool 90; therefore, by displaying the mark 302 at a predetermined position on the display 92, the mark 302 can be used as an indicator of the machine tool 90's position. In this case, for example, when the operator performs position detection, they operate the control device 91 to display the mark 302 on the display 92. The robot control device 20 captures the mark 302 displayed on the display 92 using the vision sensor 70, thereby measuring the positional relationship between the robot 10 and the machine tool 90. Furthermore, in this case, by possessing information related to the specifications of the display 92 of the control device 91 (resolution, screen size, etc.), the robot control device 20 can calculate and use the size information of the mark 302 displayed on the display 92 based on the image data of the mark 302 for detection. Additionally, in this configuration, the processor of the control device 91 can execute commands related to the tablet terminal 80. Figure 8 The functions described above are those of the image data acquisition unit 181, the image data display control unit 182, and the image data registration unit 184.

[0088] Second Implementation Method

[0089] Figure 11 This is a diagram showing the device structure of the teaching system 502 according to the second embodiment. The teaching system 503 includes a robot system 100A. Figure 11 In the teaching system 502 shown, the second device displaying an image of a predetermined pattern for calibration or position detection is a teaching operation panel 30, and the first device providing image data to the second device (teaching operation panel 30) is an external device different from the robot control device 20A. In this embodiment, as an example, the external device is another control device (numerical control device, robot control device, etc.) 220 configured within the factory where the robot system 100A is located. It is envisioned that the teaching operation panel 30 can communicate with the control device 220 via a wired or wireless connection through a network within the factory.

[0090] Furthermore, the control device 220 may also have a hardware structure similar to that of a general computer, which includes a processor 224, memory (ROM, RAM, non-volatile memory, etc.), storage unit 225, display unit, operation unit, input / output interface, network interface, etc. (see reference) Figure 12 ).

[0091] In this embodiment, the control device 220 (first device) stores image data of a predetermined pattern for calibration or position detection in the same form as the robot control device 20 in the first embodiment. Therefore, regarding the dot pattern, the control device 220 stores, in association with the image data of the dot pattern, size information of the dot interval when displaying the dot pattern on the display 32 of the teach pendant 30. In this embodiment, in order to provide the dot interval size information to the robot control device 20A responsible for calibration or position detection, an image representing the size information is displayed together with an image of the dot pattern on the display 32 of the teach pendant 30.

[0092] Figure 12 This is a functional block diagram illustrating the robot control device 20A, the teach pendant 30, and the control device 220 according to the second embodiment. The control device 220 includes a size information calculation unit 221 and an image data storage control unit 222. The size information calculation unit 221 and the image data storage control unit 222 have the same functions as the size information calculation unit 122 and the image data storage control unit 123 of the robot control device 20 in the first embodiment. That is, the size information calculation unit 221 and the image data storage control unit 222 can perform the same processing as described in steps (b1) to (b3) to store the image data and size information of the predetermined pattern in the storage unit 225. The storage unit 225 is a storage device composed of a non-volatile memory or a hard disk, which stores the image data and size information of the predetermined pattern, in addition to programs and various setting information related to mechanical control in the control device 220.

[0093] The image data acquisition unit 131 of the teaching operation panel 30 has the function of downloading image data and size information of a predetermined pattern from the storage unit 225 of the control device 220. The image data display control unit 132 and the brightness adjustment unit 133 have the same functions as those described in the first embodiment. The image data registration unit 134 provides the control device 220 with the function of registering new image data of a predetermined pattern for calibration or position detection.

[0094] The robot control device 20A includes a motion control unit 121 and a size recognition unit 124. The storage unit 22 stores robot programs, calibration programs, position detection programs, and various other setting information. The size recognition unit 124 provides the function of recognizing dimensions based on an image of size information displayed along with a predetermined pattern.

[0095] Figure 13A and Figure 13B This illustrates an example of displaying an image representing the size information of the dot spacing on the display 32 of the teach pendant 30 together with the dot pattern 301. Figure 13A The image G1 is shown as an example, displaying numerical values ​​representing point intervals as size information. Figure 13B The example shown is an image G2 that displays numerical codes for the point intervals as size information. The size information calculation unit 221 may also have the function of coding the calculated size of the point intervals as in image G2. The control device 220 may also store such images G1 or G2 as size information.

[0096] Figure 14A This describes the overall flow of the process related to teaching a predetermined pattern in calibration or position detection. Here, the flow of the process is described with an eye toward using a dot pattern as the predetermined pattern. First, the operator performs a process to acquire and display image data and size information of the predetermined pattern for calibration or position detection on the teaching operation panel 30 (second device) (step S1a).

[0097] Figure 14B It means Figure 14A The flowchart shows the processing content in step S1a. According to the operator's instructions, the image data acquisition unit 131 of the teaching operation panel 30 (second device) acquires image data and size information of a predetermined pattern for calibration or position detection from the control device 220 (first device) (step S11a). At this time, if the control device 220 holds image data of multiple patterns, the image data acquisition unit 131 can also be configured to display a list of multiple patterns on the display 32, allowing user operation to select the desired image data from the list.

[0098] Next, the image data display control unit 132 of the teach pendant 30 causes the display 32 to display the image and size information of the predetermined pattern obtained from the control device 220 (step S12a). Thus, the teach pendant 30 can be used as a fixture for calibration or position detection.

[0099] like Figure 14AAs shown, the operator then positions the teaching pendant 30, which displays an image of a dot pattern and an image representing size information, at a predetermined position for calibration and position correction that can be captured by the vision sensor 70 (step S2). Next, the operator causes the robot control device 20A to recognize the size information of the dot interval (step S2a). In this embodiment, the robot control device 20A can recognize the dot interval using any of the following methods (c1) to (c3).

[0100] (c1) The operator directly inputs the settings for the calibration procedure.

[0101] (c2) The robot control device reads and identifies the values ​​on the image.

[0102] (c3) The robot control device reads and recognizes the code information on the image.

[0103] like Figure 13A As shown, the above method (c1) is effective when an image G1 representing the numerical value of the dot interval is displayed together with an image of the dot pattern. In this case, the operator can observe the image of the size information displayed together with the image of the dot pattern, grasp the dot interval, and directly input it into the settings of the calibration procedure. Figure 18 Example of a calibration procedure execution screen 450. On execution screen 450, along with image G5 obtained by the vision sensor 70 capturing the teach pendant 30 in its predetermined position, an input field 451 for specifying point intervals is displayed. The operator can input the point intervals either by directly entering values ​​in input field 451 or by specifying them from a menu list of values. The operator can then perform the calibration by performing predetermined operations on execution screen 450 and confirm the results.

[0104] Furthermore, when the above method (c1) is used as a method for enabling the robot control device 20A to grasp the point interval, the robot control device 20A may not have the function of being a size recognition unit 124.

[0105] like Figure 13A As shown, the above method (c2) is effective when an image G1 representing the numerical value of the dot interval is displayed together with an image of the dot pattern. The dimension recognition unit 124 of the robot control device 20A has the function of recognizing the digits of the dimension information from an image captured by a teach pendant 30 showing the state of the dimension information image G1 displayed together with the dot pattern on the display 32 via a vision sensor 70, and providing this information to the calibration program. Various character recognition techniques known in the art can be used to recognize digits from an image.

[0106] like Figure 13BAs shown, the above method (c3) is effective when an image G2 representing a code indicating dot intervals is displayed together with an image of a dot pattern. The dimension recognition unit 124 of the robot control device 20A has the function of reading a code from an image obtained by capturing a teaching operation disk 30 showing the state of the image G2 (with dimension information displayed together with the dot pattern on the display 32) via a vision sensor 70, and providing it to the calibration program. The code can be a one-dimensional barcode or a two-dimensional barcode. Various code recognition techniques known in the art can be used in code recognition.

[0107] Next, the operator uses the vision sensor 70 to capture a point pattern, causing the robot control device 20A to perform calibration or position detection (step S3).

[0108] Furthermore, in the device structure of this embodiment, when using markers other than dot patterns as predetermined patterns displayed on the teaching operation panel 30 (second device), in application examples where the acquisition and display of size information in the teaching operation panel 30 (second device) is not required, the above-mentioned method can be substituted. Figures 14A-14B The processing flow is applied in the first embodiment. Figures 3A-3B The processing flow is shown. Furthermore, in the case of teaching marking during the position detection process, the control device 220 (first device) stores both the image data and structural information (size information, etc.) of the marking. Moreover, the teaching operation panel 30 (second device) can also acquire the image data and structural information of the marking from the control device 220 (first device) and provide the acquired structural information to the robot control device 20A for marking teaching. Alternatively, the teaching operation panel 30 can also display an image showing the size information of the marking on the display 32 along with the image of the marking. In this case, the robot control device 20A (size recognition unit 124) can recognize the size information from the captured image obtained by the vision sensor 70, which shows the image of the marking and its size information.

[0109] According to this embodiment, the operator can perform calibration or position detection at any time by displaying an image of a predetermined pattern on the teach pendant 30. The operator no longer needs to perform the time-consuming task of retrieving and setting up the fixtures used for calibration and position detection from their storage location, as was previously required. By displaying an image representing dimensional information together with an image of the predetermined pattern, the robot control device 20A can also recognize the dimensional information. Therefore, the operator can perform calibration simply by operating the teach pendant 30 to download and display image data of the predetermined pattern from the control device 220.

[0110] In addition, Figure 11In the illustrated device structure example, the second device for downloading and displaying image data of a predetermined pattern is the teaching operation panel 30. However, other portable terminal devices besides the teaching operation panel 30 can also be used as the second device. Furthermore, in this case, the terminal device is positioned as a teaching device for performing teaching related to calibration or position detection. In this case, the control device 220 stores the size information of each dot interval when displaying an image of a dot pattern on each terminal device, as described in Table 2 of the first embodiment, in association with the image data of the dot pattern. The control device 220 (processor 224) can also be configured to identify the terminal device requesting image data when there is a download request from the terminal device, and provide the terminal device with the size information corresponding to that terminal device along with the image data of the predetermined pattern. Furthermore, the second device for downloading and displaying image data of the predetermined pattern can also be a machine tool.

[0111] Based on this structure, various terminal devices other than the teaching operation panel 30 can display images of predetermined patterns for calibration and position detection.

[0112] In addition, Figure 11 The example shown is a machine control device 220, which is the first device for providing image data of a predetermined pattern. However, the first device for providing image data of a predetermined pattern can be various external devices. For example, the first device for providing image data of a predetermined pattern can also be a computer or cloud connected to a network of a teaching pendant 30 or other terminal device used by the operator.

[0113] Furthermore, in this embodiment, the structure in which the teach pendant 30 displays the size information received from the control device 220 together with an image of a predetermined pattern is described. However, the teach pendant 30 may also send the size information received from the control device 220 to the robot control device 20A, so that the robot control device 20A can utilize it in calibration and position detection.

[0114] Third Implementation Method

[0115] The first and second embodiments described above are structural examples of a first device for providing image data of a predetermined pattern, which has the following function: based on information related to the display of a second device displaying the image data of the predetermined pattern, it generates dot spacing size information when the dot pattern is displayed on the second device. In this embodiment, the second device displaying the image data of the predetermined pattern has the function of generating dot spacing size information.

[0116] Figure 15 This is a diagram showing the device structure of the teaching system 503 according to the third embodiment. The teaching system 503 includes a robot system 100B. Figure 15As shown, in the teaching system 503, the second device for displaying an image of a predetermined pattern used for calibration or position detection is a teaching operation panel 30B, and the first device providing image data to the second device (teaching operation panel 30B) is an external device 320 connected to the teaching operation panel 30B via a network. The network in this case may include a local area network (LAN) or a commercial network such as the Internet.

[0117] External device 320 may include various devices and computer systems such as control devices, computers, servers connected via commercial networks, and cloud computing, all configured in the same factory as robot system 100B. Furthermore, external device 320 may also have a hardware structure typical of a computer, including a processor 321, memory (ROM, RAM, non-volatile memory, etc.), storage unit 322, display unit, operation unit, input / output interface, network interface, etc. (see reference). Figure 16 ).

[0118] Figure 16 This is a functional block diagram showing the robot control device 20A, the teach pendant 30B, and the external device 320 in the teaching system 503 of the third embodiment. The external device 320 stores image data of predetermined patterns for calibration or position detection in the storage unit 322.

[0119] The teach pendant 30B functions as an image data acquisition unit 131, an image data display control unit 132, a brightness adjustment unit 133, and an image data registration unit 134. Detailed descriptions of these functions are omitted as described above. The image data acquisition unit 131 acquires image data of a predetermined pattern from an external device 320. The image data registration unit 134 registers new image data that can be used for calibration or position detection to the external device 320. The teach pendant 30B of this embodiment also includes a size information generation unit 135.

[0120] The size information generation unit 135 has the following function: when displaying image data of a dot pattern downloaded from the external device 320 on the display 32 based on the resolution and size information of the display 32, it calculates the size of the dot interval of the dot pattern. Specifically, the size information generation unit 135 parses the image of the dot pattern and calculates the number of pixels in the dot interval. Then, based on the resolution and size of the display 32, the size information generation unit 135 calculates the size of the dot interval on the display 32 and generates size information. The size information may be, for example, numerical values ​​or codes.

[0121] The image data display control unit 132 can display an image representing size information together with an image of a dot pattern on the display 32. As an example of the display method in this case, such as... Figure 13Aand Figure 13B As shown, there are examples of images that display numerical values ​​representing size information together with images of dot patterns, and examples of images that display size information coded together with images of dot patterns.

[0122] like Figure 16 As shown, the robot control device 20A may also have the same functional structure as the robot control device 20A in the second embodiment described above.

[0123] Figure 17A This describes the overall flow of processing related to teaching a predetermined pattern in calibration or position detection. Here, the process is described with an eye toward using a dot pattern as the predetermined pattern. First, the operator acquires image data of the predetermined pattern for calibration or position detection on the teaching operation panel 30B (second device) and performs processing for displaying it along with dimensional information (step S1b).

[0124] Figure 17B It means Figure 17A The flowchart shows the processing content in step S1b. According to the operator's instructions, the image data acquisition unit 131 of the teach pendant 30B (second device) acquires image data of a predetermined pattern for calibration or position detection from the external device 320 (first device) (step S11). The processor 321 of the external device 320 (first device) has the function of sending image data from the storage unit 322 to the teach pendant 30B according to a request from the teach pendant 30B. At this time, if the external device 320 holds image data of multiple patterns, the image data acquisition unit 131 can also be configured to display a list of multiple patterns on the display 32 and accept user operations to select the desired image data from the list.

[0125] Next, the size information generation unit 135 of the teaching operation panel 30B (second device) calculates the size information of the dot interval when displaying the dot pattern on the display 32 based on the number of pixels between the dots in the acquired dot pattern image data and the resolution and size information of the display 32 (step S11b). Then, the image data display control unit 132 displays the image representing the size information together with the image of the dot pattern on the display 32 (step S12a).

[0126] like Figure 17AAs shown, next, the operator positions the teaching pendant 30B, which displays an image of a dot pattern and an image representing size information, at a predetermined position for calibration and position correction, which can be captured by the vision sensor 70 (step S2). Then, the operator causes the robot control device 20A to identify the size information of the dot interval (step S2a). Alternatively, any of the methods (c1) to (c3) described in the second embodiment can be used to cause the robot control device 20A to identify the dot interval.

[0127] Then, the operator uses the vision sensor 70 to capture a point pattern to perform calibration or position detection (step S3).

[0128] Furthermore, in the device structure of this embodiment, when using markers other than dot patterns as predetermined patterns displayed on the teach pendant 30B (second device), in application examples where the generation and display of size information in the teach pendant 30B (second device) is not required, the above-described method can be substituted. Figures 17A-17B The processing flow is applied to the first embodiment. Figures 3A-3B The processing flow is shown. Furthermore, in the case of teaching where marking is required during the position detection process, the external device 320 (first device) stores the structural information of the marking along with the image data of the marking. Moreover, the teaching operation panel 30B (second device) can also acquire the image data and structural information of the marking from the external device 320 (first device) and provide the acquired structural information to the robot control device 20A for marking teaching.

[0129] Thus, in this embodiment, the dot interval of the dot pattern is calculated and displayed on the second device side that displays the image of the predetermined pattern, so it is not necessary to pre-generate and save size information in the first device that provides the image data of the predetermined pattern.

[0130] According to this embodiment, the operator can display an image of a predetermined pattern on the teach pendant 30B at any time for calibration or position detection. With this embodiment, the operator no longer needs to perform the time-consuming task of retrieving and setting up the fixtures used for calibration and position detection from their storage location, as was done previously. By displaying an image representing dimensional information together with an image of the predetermined pattern, the robot control device 20A can also recognize the dimensional information. Therefore, the operator can perform calibration simply by operating the teach pendant 30B to download and display the image data of the predetermined pattern from the external device 320.

[0131] In addition, Figure 15In the illustrated structural example, the second device that downloads and displays an image of a predetermined pattern from the first device (external device 320) is a teaching pendant 30B. However, this is merely an example. The second device that downloads the image of the predetermined pattern from the first device (external device 320), generates dimensional information, and displays it along with the image of the predetermined pattern could also be a portable terminal device different from the teaching pendant 30B. In this case, the terminal device is positioned as a teaching device for performing teaching related to calibration or position detection.

[0132] Furthermore, in this embodiment, the structure in which the teach pendant 30B displays the calculated size information together with the image of the predetermined pattern is described. However, the teach pendant 30B can also send the calculated size information to the robot control device 20A, so that the robot control device 20A can use it in calibration and position detection.

[0133] As explained above, according to each embodiment, the burden on operators in cases of calibration and position detection using predetermined patterns can be reduced, and these operations can be performed efficiently.

[0134] The functional allocation shown in the functional block diagrams of the above embodiments is illustrative, and various modifications are possible regarding the functional allocation. For example, the teach pendant 30 shown in the first embodiment may not have the functions of a brightness adjustment unit 133 and an image data registration unit 134.

[0135] As a variation of the above embodiment, another possible structural example is as follows: the first device responsible for providing image data also possesses image data of a predetermined pattern and an application program for displaying the image data, and provides the image data and such an application to the second device responsible for displaying the predetermined pattern. The application program may also have the following functions: acquiring information about the display of the device executing the application (resolution, size, etc.), and calculating size information when the predetermined pattern is displayed. The second device executing the application may also... Figures 13A-13B As shown, an image (a numerical or code image) representing the calculated size information is displayed together with an image of a pattern. Alternatively, a second device executing the application can also provide the calculated size information to the first device.

[0136] The following is an example for reference. Figure 7The structure is illustrated using the device structure described above. The robot control unit 20, serving as the first device, stores image data of a predetermined pattern and an application program for displaying the image data. The robot control unit 20 sends the application program along with the image data to the tablet terminal 80, serving as the second device. The tablet terminal 80 executes the application program, displays the image of the predetermined pattern, and calculates dimensional information. The tablet terminal 80 may also display an image (numerical value or code) representing dimensional information along with the image of the predetermined pattern. Alternatively, the tablet terminal 80 may send the calculated dimensional information to the robot control unit 20. The robot control unit 20 can use the dimensional information identified from the image captured by the vision sensor 70 or provided by the tablet terminal 80 for calibration and position detection.

[0137] In the above embodiments, the functional blocks in the functional block diagrams of the robot control device, teaching pendant, tablet terminal, and external devices can be implemented by one or more processors of these devices executing various software stored in the storage device, or they can be implemented by a structure based on hardware such as ASIC (Application Specific Integrated Circuit).

[0138] The program for performing teaching processes such as calibration or position detection in the above embodiments can be recorded on various computer-readable recording media (e.g., semiconductor memory such as ROM, EEPROM, flash memory, magnetic recording media, CD-ROM, DVD-ROM, etc.).

[0139] This disclosure has been described in detail, but it is not limited to the various embodiments described above. Various additions, substitutions, modifications, and partial deletions can be made to these embodiments without departing from the spirit of this disclosure, or from the spirit of this disclosure derived from the content described in the claims and their equivalents. Furthermore, these embodiments can also be implemented in combination. For example, in the embodiments described above, the order of each action and the order of each process are shown as an example and are not limited thereto. Similarly, the use of numerical values ​​or mathematical formulas in the description of the embodiments described above also applies.

[0140] The following notes further describe the above-described embodiments and variations.

[0141] (Note 1)

[0142] A teaching system (501, 501A, 501A, 502, 503) for performing teaching related to calibration or position detection, wherein the teaching system comprises:

[0143] The first device (20, 220, 320) has a first processor and a storage unit for storing image data for the calibration or position detection; and

[0144] The second device (30, 80, 30B) has a second processor and a display.

[0145] The second processor of the second device acquires the image data from the first device.

[0146] During the calibration or position detection process, the image represented by the image data is displayed on the display in order to enable the vision sensor to take a picture.

[0147] (Note 2)

[0148] According to the teaching system (501, 501A) described in Appendix 1, wherein,

[0149] The first device is a robot control device (20) that performs the calibration or position detection.

[0150] The first processor of the first device performs the calibration or position detection based on the captured image obtained by the vision sensor and displayed on the display of the second device.

[0151] The second device is a teach pendant (30) that is wired or wirelessly connected to the robot control device (20), or a device (80) that is different from the teach pendant.

[0152] (Note 3)

[0153] According to the teaching system (501, 501A) described in Appendix 2, wherein,

[0154] The storage unit of the first device (20) stores information related to the size of the image on the display when the image represented by the image data is displayed on the display of the second device, in association with the image data.

[0155] The first processor of the first device uses information related to the size during the calibration or position detection.

[0156] (Note 4)

[0157] According to the teaching system described in Appendix 1, wherein,

[0158] The teaching system also includes a robot control device (20A) having a third processor for performing the calibration or the position detection.

[0159] The first device is a different device (220, 320) from the robot control device.

[0160] The second device is a teach pendant (30, 30B) that is wired or wirelessly connected to the robot control device (20A), or a device different from the teach pendant.

[0161] (Note 5)

[0162] According to the teaching system (502) described in Appendix 4, wherein,

[0163] The storage unit of the first device (220) stores information related to the size of the image when the image represented by the image data is displayed on the display of the second device (30) in association with the image data.

[0164] The second processor acquires the image data and size-related information together with the first device (220).

[0165] The second processor performs any of the following processes:

[0166] (1) Display the image representing information related to the size and the image represented by the image data together on the display;

[0167] (2) Send information related to the size to the robot control device.

[0168] (Note 6)

[0169] According to the teaching system (503) described in Appendix 4, wherein...

[0170] The second processor, based on the acquired image data and information related to the resolution and size of the display, calculates size-related information for displaying the image based on the image data on the display.

[0171] The second processor performs any of the following processes:

[0172] (1) Display the image representing information related to the size and the image represented by the image data together on the display;

[0173] (2) The calculated information related to the size is sent to the robot control device.

[0174] (Note 7)

[0175] According to the teaching system (502, 503) described in Appendix 5 or 6, wherein,

[0176] The other device is any one of the mechanical control device, external device, or cloud.

[0177] (Note 8)

[0178] According to any one of Appendices 5 to 7, the teaching system (502, 503), wherein,

[0179] The image representing the size information is either an image of the numerical value of the size or an image of the size coded.

[0180] (Note 9)

[0181] According to any one of Appendices 5 to 8, the teaching system (502, 503), wherein,

[0182] The second processor displays the image representing information related to the size and the image represented by the image data together on the display.

[0183] The third processor of the robot control device (20A) identifies size-related information based on the captured image obtained from the image data represented by the image data captured by the vision sensor and displayed on the display of the second device (30, 30B) and the image representing information related to the size.

[0184] The identified size-related information is used during the calibration or position detection process.

[0185] (Postscript 10)

[0186] According to the teaching system (501, 501A, 502) described in Appendix 3 or 5, the first processor of the first device (20, 220) generates information related to the size based on the image data of the dot pattern and information related to the resolution and size of the display of the second device.

[0187] (Postscript 11)

[0188] According to any one of the appendices 1 to 10, the teaching system (501, 501A, 502, 503), wherein,

[0189] The second processor of the second device (30, 80, 30B) registers new image data of a predetermined pattern that can be used for the calibration or position detection, as well as information related to the size of the new image data, in the storage unit of the first device based on user operation.

[0190] (Postscript 12)

[0191] According to the teaching system (501, 501A, 502, 503) described in Appendix 1, wherein,

[0192] The second device is a teach pendant (30, 30B) connected to the robot control device.

[0193] The second processor of the second device obtains information related to the brightness of the captured image from the robot control device, and adjusts the brightness of the display of the image based on the brightness-related information, wherein the captured image is obtained by the second device by the vision sensor capturing the image represented by the image data and displaying it on the display.

[0194] (Postscript 13)

[0195] According to the teaching systems (501, 501A, 502) described in Appendix 1, wherein,

[0196] The storage unit of the first device also stores an application program for displaying the image represented by the image data.

[0197] The second processor acquires the image data and the application together from the first device.

[0198] The second processor executes the application program to display the image represented by the image data on the display of the first device.

[0199] (Postscript 14)

[0200] According to the teaching systems (501, 501A, 502) described in Appendix 13, wherein,

[0201] The application also has the function of calculating size-related information when displaying the image represented by the image data on the display based on information related to the display of the device executing the application.

[0202] The second processor performs any of the following processes:

[0203] (1) Display together the image representing information related to the size obtained by executing the application and the image represented by the image data;

[0204] (2) Send information related to the size to the first device.

[0205] (Postscript 15)

[0206] A teaching device (30, 80, 30B) is used for teaching in relation to calibration or position detection, wherein,

[0207] The teaching devices (30, 80, 30B) include a display (32, 82) and a processor (31, 81).

[0208] The processors (31, 81) acquire image data from external devices for the calibration or position detection.

[0209] During the calibration or position detection process, the image represented by the image data is displayed on the display in order to enable the vision sensor to take a picture.

[0210] (Postscript 16)

[0211] According to the teaching apparatus (30) described in Appendix 15, the processor further obtains information related to the size of the image when displaying the image represented by the image data on the display from the external device.

[0212] The processor performs any of the following processes:

[0213] (1) Display the image representing information related to the size and the image represented by the image data together on the display;

[0214] (2) Send information related to the size to the robot control device responsible for the calibration or position detection.

[0215] (Postscript 17)

[0216] According to the teaching device (30B) described in Appendix 15, wherein...

[0217] Based on the acquired image data and information related to the resolution and size of the display, the processor calculates size-related information for displaying the image based on the image data on the display.

[0218] The processor performs any of the following processes:

[0219] (1) Display the image representing information related to the size and the image represented by the image data together on the display;

[0220] (2) The calculated information related to the size is sent to the robot control device responsible for the calibration or position detection.

[0221] (Postscript 18)

[0222] According to any one of Appendices 15 to 17, the teaching device (30, 80, 30B), wherein,

[0223] The processors (31, 81) also register new image data of a predetermined pattern that can be used for the calibration or position detection, along with information related to the size of the new image data, in the storage unit of the external device based on user operation.

[0224] (Postscript 19)

[0225] According to the teaching apparatus (30, 30B) described in Appendix 15, wherein...

[0226] The external device is a robot control unit (20, 20A) that performs the calibration or position detection.

[0227] The processor (31) obtains information related to the brightness of the captured image from the robot control device, and adjusts the brightness of the display of the image based on the brightness-related information, wherein the captured image is obtained by the teaching device that captures the image represented by the image data and displays it on the display by the vision sensor.

[0228] (Postscript 20)

[0229] A robot control device (20, 20A), comprising:

[0230] Processor (21); and

[0231] Storage unit (22), which stores image data used for calibration or position detection,

[0232] The processor (21) sends the image data to the external device according to a request from the external device, and performs the calibration or position detection based on the captured image obtained by the vision sensor and displayed on the display of the external device.

[0233] (Postscript 21)

[0234] According to Appendix 20, the robot control device (20) wherein,

[0235] The storage unit stores information related to the size of the image on the display when the image data is displayed on the display of the external device, in association with the image data.

[0236] The processor uses information related to the size during the calibration or position detection.

[0237] (Postscript 22)

[0238] According to the robot control device (20) described in Appendix 19, wherein,

[0239] The processor (21) calculates information related to the size based on the image data and information related to the resolution and size of the display of the external device.

[0240] (Postscript 23)

[0241] According to the robot control device (20A) described in Appendix 20, wherein,

[0242] On the display of the external device, information related to the size of the image when it is displayed on the display is shown along with the image represented by the image data.

[0243] The processor (21) identifies the size-related information based on the captured image obtained from the image data represented by the image data captured by the vision sensor and displayed on the display of the external device, and the image represented by the size-related information.

[0244] The identified size-related information is used during the calibration or position detection process.

[0245] (Postscript 24)

[0246] A program for causing a computer's processor to perform the following steps:

[0247] Image data used for calibration or location detection is stored in the storage unit;

[0248] The image data is sent to the external device upon request; and

[0249] The calibration or position detection is performed based on an image obtained from an image data represented by the image data captured by the vision sensor and displayed on the display of the external device.

[0250] Explanation of reference numerals in the attached figures

[0251] 10 robots

[0252] 11 robotic arms

[0253] 20, 20A Robot Control Device

[0254] 21 processor

[0255] 22 Storage Department

[0256] 30, 30B Teaching Operation Panel

[0257] 31 processor

[0258] 32 monitors

[0259] 33 Operations Department

[0260] 40 Image Processing Device

[0261] 70 vision sensors

[0262] 80 tablet terminal

[0263] 81 processor

[0264] 82 monitor

[0265] 83 Operations Department

[0266] 90 machine tool

[0267] 91 control device

[0268] 92 monitor

[0269] 100, 100A, 100B robot systems

[0270] 121 Motion Control Department

[0271] 122 Dimension Information Calculation Department

[0272] 123 Image Data Storage Control Department

[0273] 124 Size Identification Unit

[0274] 131 Image Data Acquisition Department

[0275] 132 Image Data Display Control Unit

[0276] 133 Brightness Adjustment Section

[0277] 134 Image Data Registration Department

[0278] 135 Dimension Information Generation Department

[0279] 181 Image Data Acquisition Department

[0280] 182 Image Data Display Control Unit

[0281] Image Data Registration Department 184

[0282] 220 control device

[0283] 221 Dimension Information Calculation Department

[0284] 222 Image Data Storage Control Unit

[0285] 224 processor

[0286] 225 Storage Unit

[0287] 301-dot pattern

[0288] 302 marking

[0289] 320 external devices

[0290] 321 processor

[0291] 322 Storage Unit

[0292] 402 registration screen

[0293] 501, 501A, 502, 503 teaching systems.

Claims

1. A teaching system for performing teaching related to calibration or position detection, characterized in that, The teaching system has the following features: A first device having a first processor and a storage unit for storing image data used for the calibration or position detection; and The second device has a second processor and a display. The second processor acquires the image data from the first device. During the calibration or position detection process, the image represented by the image data is displayed on the display in order to enable the vision sensor to take a picture.

2. The teaching system according to claim 1, characterized in that, The first device is a robot control device that performs the calibration or position detection. The first processor of the first device performs the calibration or position detection based on the captured image obtained by the vision sensor and displayed on the display of the second device. The second device is a teach pendant that is wired or wirelessly connected to the robot control device, or a device different from the teach pendant.

3. The teaching system according to claim 2, characterized in that, The storage unit of the first device stores information related to the size of the image on the display when the image data is displayed on the display of the second device, in association with the image data. The first processor of the first device uses information related to the size during the calibration or position detection.

4. The teaching system according to claim 1, characterized in that, The teaching system also includes a robot control unit with a third processor that performs the calibration or position detection. The first device is different from the robot control device. The second device is a teach pendant that is wired or wirelessly connected to the robot control device, or a device different from the teach pendant.

5. The teaching system according to claim 4, characterized in that, The storage unit of the first device stores information related to the size of the image when the image represented by the image data is displayed on the display of the second device, in association with the image data. The second processor acquires the image data and size-related information together with the first device. The second processor performs any of the following processes: (1) Display the image representing information related to the size and the image represented by the image data together on the display; (2) Send information related to the size to the robot control device.

6. The teaching system according to claim 4, characterized in that, The second processor, based on the acquired image data and information related to the resolution and size of the display, calculates size-related information for displaying the image based on the image data on the display. The second processor performs any of the following processes: (1) Display the image representing information related to the size and the image represented by the image data together on the display; (2) The calculated information related to the size is sent to the robot control device.

7. The teaching system according to claim 5 or 6, characterized in that, The different devices are any of the mechanical control devices, external devices, or cloud devices.

8. The teaching system according to any one of claims 5 to 7, characterized in that, The image representing information related to the size is either an image of the numerical value of the size or an image of the size coded.

9. The teaching system according to any one of claims 5 to 8, characterized in that, The second processor displays the image representing information related to the size and the image represented by the image data together on the display. The third processor of the robot control device identifies size-related information based on the captured image obtained from the image data represented by the image data captured by the vision sensor and displayed on the display of the second device, and the image representing information related to the size. The identified size-related information is used during the calibration or position detection process.

10. The teaching system according to claim 3 or 5, characterized in that, The first processor of the first device generates size-related information based on the image data and information related to the resolution and size of the display of the second device.

11. The teaching system according to any one of claims 1 to 10, characterized in that, The second processor of the second device, based on user operation, registers new image data that can be used for the calibration or position detection, as well as information related to the size of the new image data, in the storage unit of the first device.

12. The teaching system according to claim 1, characterized in that, The second device is a teach pendant connected to the robot control device. The second processor of the second device obtains information related to the brightness of the captured image from the robot control device, and adjusts the brightness of the display of the image based on the brightness-related information, wherein the captured image is obtained by the second device by the vision sensor capturing the image represented by the image data and displaying it on the display.

13. The teaching system according to claim 1, characterized in that, The storage unit of the first device also stores an application program for displaying the image represented by the image data. The second processor obtains the application and the image data together from the first device. The second processor executes the application program to display the image represented by the image data on the display of the first device.

14. The teaching system according to claim 13, characterized in that, The application also has the function of calculating size-related information when displaying the image represented by the image data on the display based on information related to the display of the device executing the application. The second processor performs any of the following processes: (1) Display together the image representing information related to the size obtained by executing the application and the image represented by the image data; (2) Send information related to the size to the first device.

15. A teaching device for teaching in relation to calibration or position detection, characterized in that, The teaching device includes a display and a processor. The processor acquires image data from an external device for the calibration or location detection. During the calibration or position detection process, the image represented by the image data is displayed on the display in order to enable the vision sensor to take a picture.

16. The teaching device according to claim 15, characterized in that, The processor also obtains information from the external device related to the size of the image when displaying the image represented by the image data on the display. The processor performs any of the following processes: (1) Display the image representing information related to the size and the image represented by the image data together on the display; (2) Send information related to the size to the robot control device responsible for the calibration or position detection.

17. The teaching device according to claim 15, characterized in that, Based on the acquired image data and information related to the resolution and size of the display, the processor calculates size-related information for displaying the image based on the image data on the display. The processor performs any of the following processes: (1) Display the image representing information related to the size and the image represented by the image data together on the display; (2) The calculated information related to the size is sent to the robot control device responsible for the calibration or position detection.

18. The teaching device according to any one of claims 15 to 17, characterized in that, The processor also registers new image data that can be used for the calibration or location detection, along with information related to the size of the new image data, into the storage of the external device based on user operations.

19. The teaching device according to claim 15, characterized in that, The external device is a robot control unit that performs the calibration or position detection. The processor obtains information related to the brightness of the captured image from the robot control device, and adjusts the brightness of the display showing the image based on the brightness-related information, wherein the captured image is obtained by the teaching device that captures the image represented by the image data and displays it on the display by the vision sensor.

20. A robot control device, characterized in that, have: Processor; and The storage unit stores image data used for calibration or position detection. The processor sends the image data to the external device according to a request from the external device, and performs the calibration or position detection based on the captured image obtained by the vision sensor and displayed on the display of the external device.

21. The robot control device according to claim 20, characterized in that, The storage unit stores information related to the size of the image on the display when the image data is displayed on the display of the external device, in association with the image data. The processor uses information related to the size during the calibration or position detection.

22. The robot control device according to claim 21, characterized in that, The processor calculates information related to the size based on the image data and information related to the resolution and size of the display of the external device.

23. The robot control device according to claim 20, characterized in that, The image represented by the image data and information related to the size of the image when it is displayed on the external device's display are shown together. The processor identifies the size-related information from the captured image obtained based on the image data represented by the image data captured by the vision sensor and displayed on the display of the external device, and the image represented by information related to the size. The identified size-related information is used during the calibration or position detection process.

24. A program, characterized in that, To cause the computer's processor to perform the following steps: Image data used for calibration or location detection is stored in the storage unit; The image data is sent to the external device upon request from the external device. as well as The calibration or position detection is performed based on an image obtained from an image data represented by the image data captured by the vision sensor and displayed on the display of the external device.

Images